XXV.1 January + February 2018
Page: 66
Digital Citation

Launching problem space research in the frenzy of software production

Indi Young, Kunyi Mangalam

back to top 

"First we need to understand the problem." You hear this phrase more often these days in the applied realm of software development. Because of design thinking and lean UX, there is more awareness of the value of understanding the problem space. However, outside of design teams, no matter if you're in tech, commercial, industrial, or government, there is stiff resistance to taking time to understand the problem [1]. This is due to many factors, one of which is speed-to-solution and efficiency-of-production, and another of which is unfamiliarity with how qualitative data represents a population. The power and value that deep research brings—awareness of the wider perspective of the user—is often completely unseen by many executives. It can and does open up opportunities to grow the business in the right way. Here is how you might be able to open some eyes within your organization.

back to top  Insights


First, a couple of definitions as they apply outside of academia.

The solution space refers to any work you do with regard to an idea, content, product, or service. It's about how your organization supports people, internal or external. It's designing with compassion, acting upon your understanding of what people face. It's fixing things for people and making their world a better place. The knowledge that you act upon to create solutions can come from various sources, for example, generative and evaluative user research. A user is a person with a current or potential relationship to your organization, so references to the word user fall within the solution space. Any other word used to refer to someone with a relationship to your organization, such as customer or member, also connotes the solution space. In terms of gathering knowledge, user experience research and journey maps, which mostly contain research about people experiencing an offering, are also part of the solution space.

Problem space knowledge comes from research about people, without connection to an offering or to the concept of a user. The problem space is about turning away from your solutions (products or services) and toward people, for a time, to soak up a deep understanding of the way people think their way toward a purpose. During this interval, you seek to keep your mind focused on people so you can aggregate a deep understanding of their patterns of reasoning, reactions, and guiding principles. If your mind strays into solutions, then your focus on the person becomes diluted. You also weaken problem space research if your investigation emphasizes surface-level concepts that people use to represent themselves [2]. Staying at this surface level often leads to inconclusive frameworks around the objects described by people's opinions, preferences, and explanations. Problem space research goes deeper, allowing an organization to create a reliable framework:

  • Surface: opinions, preferences, explanations, statements of fact
  • Depth: inner thinking, reactions, guiding principles.

You can go to this same level of depth in the solution space. There, researchers usually collect the data by asking prescribed questions about usage, tasks, or goals. The findings are directly connected to users and the organization's products and services. In the problem space, depth is reached using active listening, allowing participants to describe their thinking and actions as they accomplish an intent or purpose, higher or broader than a particular goal that is related to the solution. We conducted an example study in 2014 to show how the focus of a problem space study is undiluted by the current capabilities of typical insurance companies, summarized as follows:

  • Task: gather information to fill out an insurance form
  • Goal: file an accident claim to cover my loss
  • Purpose: recover from an accident.

In the example study, we synthesized affinity patterns from the transcripts. People recovering from an accident reported broader thinking that wasn't on the radar of a typical insurance company before our study was conceived. For example: "Try to prevent this from happening again/to others" (see Figure 1). This included statements such as:

  • "Convince someone in charge to do something to prevent this from happening again."
  • "Change my actions so this doesn't happen again."
  • "Prevent an accident by following safe habits."
ins02.gif Figure 1. A peek at a section of the data collected for a near-miss-accident study. Starting at the 5th (rightmost) column you see actual quotes from the transcripts. Each concept the quotes represent is summarized in column 3 for easier comparison to other concepts. These are pulled together into patterns by the intent of the speaker, at two levels of hierarchy in columns 2 and 1. Comparisons are conducted from the bottom up. Column 4 contains the ID of the participant.

Later, when you turn away from the people and the problem space and back toward your solutions, you begin to select problems to solve and then weave together ideas that might support people. The aggregate patterns from the problem space inform and inspire idea generation, strategy decisions, design direction, and algorithmic limits. The new concepts brought to light by the data allow the organization to consciously choose a direction. Using our example, an insurance company can offer support for the concept of trying to prevent the accident from happening again or to others, such as by partnering with public-safety officials to warn other drivers of a temporary hazard like ice on a particular section of steep road, and then retracting the alert when the danger has passed.

back to top  Mental Model Diagrams, Opportunity Maps, and Thinking Styles

Two approaches bring a frame of inquiry that keeps you in the problem space: mental model diagrams and thinking styles. Other ethnographic research can also be used to explore the problem space, but in the software world, ethnography is most often applied within the solution space.

A mental model diagram is a horizon diagram that collects participants' inner reasoning, reactions, and guiding principles as they accomplish a larger purpose into towers that resemble a city skyline [3] (Figure 2). These towers are further grouped into "mental spaces" representing different areas of thinking, such as "find out if anyone was hurt" and "feel upset with myself." A mental model diagram is a type of affinity diagram structured by these hierarchies of towers and mental spaces. Organizations can add to the diagram with studies over time. Mental model diagrams have longevity because people's reasoning around a higher purpose, and indeed the purpose itself, does not change much, even though supporting services and technology do.

ins03.gif Figure 2. The mental model diagram from the example study about recovering from a near-miss accident.

Below the horizon line, an organization's services and technology are aligned to the towers they support. Taken together, the diagram is called an opportunity map (Figure 3). It compares how people think to how the organization supports them, reveals gaps where towers receive weak or no support, and illuminates assumptions about what constitutes "the problem" for people in their larger context. Teams use the opportunity map over the long term to generate new ideas, redevelop metrics, and prioritize efforts based on their positive impact to a person's experience with the organization.

ins04.gif Figure 3. This example of an opportunity map shows capabilities of an organization aligned to the concepts expressed by people and collected into towers above. This particular example also shows a layer of capabilities created by a competing organization.

Mental model diagram data is collected through listening sessions, in which participants articulate their deep inner reasoning about why and how they achieve a purpose. Listening sessions are different from interviews, which are common in the solution space and usually follow a set of agreed-upon questions. The transcripts from the listening sessions also act as the source of data for creating thinking styles, which are behavioral audience segments similar to personas or archetypes, representing an approach to the purpose. The term thinking styles is meant to imply flexibility, because as a person moves from context to context or grows in experience, their approach may change [4]. For example, a person's thinking style about driving a company truck might be different from their thinking style about driving their private vehicle. Thinking styles are written without reference to demographics, unless the scope being explored connects to a demographic factor, such as discrimination [5]. Used in conjunction with specific contexts, thinking styles help teams explore edge cases in a more rigorous, conclusive way.

Because of what has already failed in technology (e.g., the news bubbles and hijacking of people's attention) and what is coming (e.g., machine learning and emergent experiences), reliable methods for undiluted problem space exploration are needed. In the case of mental model diagrams, there are a lot of examples to follow. Over the past two decades, mental model diagrams have enabled progressive teams from a range of organizations to more confidently innovate around commercial and social challenges.

back to top  Overcoming the Reluctance

Within the culture of many for-profit and even nonprofit organizations, there are misgivings about work that does not directly improve the offering. This feeling pairs with a conviction that enough is already known about the problem. Team members may have years of experience in their industry and prefer to make progress rather than investigate any assumptions at the core of their work. Over the years, we've helped leaders show their organizations the value of problem space research. There are a couple of techniques we use to overcome this reluctance.

First, there are useful mindsets. Solving problems is fulfilling. Ideas convey status upon the teams that conceive them. Being asked to abandon the solution space and its visible development activities for a time is like being asked to abandon the mindset that brought you to where you are in your career. Entering the problem space becomes an exercise in reframing beliefs about work. Additionally, organizations race headlong through development cycles, applying agile and lean practices to make progress with a minimum viable product every few weeks. A mindset focused on the speed of the cycles has taken hold of business and has virtually prevented any contemplative approaches, except within innovation centers or where leaders have the power to establish this practice.

To help organizations gain better clarity about the role of the problem space, the advice we have is to decouple it from the design/develop/improve cycles or any of the other solution space activities (Figure 4). It is not a step within the solution space cycle. It does not come first but rather is a separate and ongoing resource. You don't need to study the problem space with every cycle of development. It can be added to with successive studies, once a year.

ins05.gif Figure 4. It is helpful to think of the problem space–aggregating data about the inner voices of people pursuing a purpose–as disconnected from the solution space. To allow undiluted knowledge to form, it can't be a step within the solution space. Problem space knowledge, such as mental model diagrams and thinking styles, supports more focused and specialized work in the solution space.

Another bit of advice comes in the form of an analogy that may help you explain how to fit problem space research into the culture of software development. Problem space research is like the sun, giving energy to all the spinning solution projects orbiting it (Figure 5). Lots of research and production happens on those planets, evolving the function and presentation of the service or product. The energies from the sun—this knowledge of mental spaces that were not on the radar, or thinking styles that had been ignored—provide vision to the team and help them focus their efforts to support specific edge cases. Problem space research is where you develop empathy so that you have a solid understanding you can use when you apply empathy when you are designing solutions. It is the pathos in Aristotle's triad of ethos/pathos/logos [6]. Perhaps the team will write algorithms specific to the person driving a company truck, in the context of defusing the tension between the person and the other driver involved in the accident. In a world of machine learning, specialized algorithms will be prevalent, along with the ability for these algorithms to listen to their users and hand off to other algorithms that may be able to better support a particular thinking style and context.

ins06.gif Figure 5. In this analogy, which may help explain the problem space, think of the sun as problem space knowledge accumulated over time. In the beginning, this problem space knowledge is the native knowledge of the team. It may then mature to include secondary research and then mature again to include primary research. The sun gives energy to support growth and clarity on every circling project.

Our final tip is to use language that resonates with decision makers at your organization. Non-researchers are often more comfortable with language that conveys certainty that solid results will come out of the expense and effort of research. Words like test, validate, determine, measure, and identify convey confidence. Words like explore, investigate, study, observe, and uncover tend to resonate best within the research-practitioner community.

An unexpected benefit of problem space research is that it requires decision makers to choose an audience and context to explore. Narrowing down from the-sky-is-the-limit creates fear that better opportunities might be found in a branch different from the one they chose. Part of the work going into problem space research is to help stakeholders carefully consider all the avenues to explore and become comfortable with a focus that will provide the most input for high-priority business opportunities.

The benefits of problem space research are strategic and recur over time. When you are working on your solutions, problem space research gives your organization both direction and subtle advantages.

back to top  References

1. Within the academic discipline of design research, awareness and adoption is much greater. In those areas, studies exploring the nature of creativity have produced papers that define concepts such as "co-evolution of problem-solution" and "wicked or tame problems." See Dorst, K. and Cross, N. Creativity in the design process: Co-evolution of problem–solution. Design Studies 22, 5 (2001), 425—437. And Rittel, H.W.J. On the Planning Crisis: Systems Analysis of the "First and Second Generations." 1972.

2. See the description of the bubble people build around themselves in: Gray, D. Liminal Thinking – Create the Change You Want by Changing the Way You Think. Two Waves, 2016.

3. See Young, I. Practical Empathy – For Collaboration and Creativity in Your Work. Rosenfeld Media, 2015; and Young, I. Mental Models – Aligning Design Strategy with Human Behavior. Rosenfeld Media, 2008.

4. The idea of flexible representations of audience groups is not new. It is known in marketing via various terms, including component lifestyles. See the marketing text book: Lamb, C.W., Jr. MKTG. South-Western College Pub., 11th edition, 2017.

5. See Describing Personas on, by Indi Young, Mar. 14, 2016;

6. Aristotle was referenced by Richard Buchanan, who shaped the Carnegie Mellon (CMU) School of Design in the mid-1990s to early 2000s. See Gajendar, U. Notes on the future of interaction design. Interactions (Sept.-Oct. 2017), 46.

back to top  Authors

Indi Young is a freelance problem space researcher and coach. She helps organizations reframe offerings, innovate, and strengthen the way they support people. She was a founding partner of Adaptive Path (acquired by Capital One) and has authored two books: Mental Models and Practical Empathy.

Kunyi Mangalam has used qualitative research for 30 years to reveal how people think, feel, and live, helping organizations understand their audiences more deeply and thus make better decisions.

back to top 

Copyright held by authors. Publication rights licensed to ACM.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2018 ACM, Inc.

Post Comment

No Comments Found