Forums

XXV.1 January + February 2018
Page: 66
Digital Citation

Launching problem space research in the frenzy of software production


Authors:
Indi Young, Kunyi Mangalam

back to top 

“First we need to understand the problem.” You hear this phrase more often these days in the applied realm of software development. Because of design thinking and lean UX, there is more awareness of the value of understanding the problem space. However, outside of design teams, no matter if you’re in tech, commercial, industrial, or government, there is stiff resistance to taking time to understand the problem [1]. This is due to many factors, one of which is speed-to-solution and efficiency-of-production, and another of which is unfamiliarity with how qualitative data represents a population. The power and value that deep research brings—awareness of the wider perspective of the user—is often completely unseen by many executives. It can and does open up opportunities to grow the business in the right way. Here is how you might be able to open some eyes within your organization.

back to top  Insights

ins01.gif

First, a couple of definitions as they apply outside of academia.

The solution space refers to any work you do with regard to an idea, content, product, or service. It’s about how your organization supports people, internal or external. It’s designing with compassion, acting upon your understanding of what people face. It’s fixing things for people and making their world a better place. The knowledge that you act upon to create solutions can come from various sources, for example, generative and evaluative user research. A user is a person with a current or potential relationship to your organization, so references to the word user fall within the solution space. Any other word used to refer to someone with a relationship to your organization, such as customer or member, also connotes the solution space. In terms of gathering knowledge, user experience research and journey maps, which mostly contain research about people experiencing an offering, are also part of the solution space.

Problem space knowledge comes from research about people, without connection to an offering or to the concept of a user. The problem space is about turning away from your solutions (products or services) and toward people, for a time, to soak up a deep understanding of the way people think their way toward a purpose. During this interval, you seek to keep your mind focused on people so you can aggregate a deep understanding of their patterns of reasoning, reactions, and guiding principles. If your mind strays into solutions, then your focus on the person becomes diluted. You also weaken problem space research if your investigation emphasizes surface-level concepts that people use to represent themselves [2]. Staying at this surface level often leads to inconclusive frameworks around the objects described by people’s opinions, preferences, and explanations. Problem space research goes deeper, allowing an organization to create a reliable framework:

  • Surface: opinions, preferences, explanations, statements of fact
  • Depth: inner thinking, reactions, guiding principles.

You can go to this same level of depth in the solution space. There, researchers usually collect the data by asking prescribed questions about usage, tasks, or goals. The findings are directly connected to users and the organization’s products and services. In the problem space, depth is reached using active listening, allowing participants to describe their thinking and actions as they accomplish an intent or purpose, higher or broader than a particular goal that is related to the solution. We conducted an example study in 2014 to show how the focus of a problem space study is undiluted by the current capabilities of typical insurance companies, summarized as follows:

  • Task: gather information to fill out an insurance form
  • Goal: file an accident claim to cover my loss
  • Purpose: recover from an accident.

In the example study, we synthesized affinity patterns from the transcripts. People recovering from an accident reported broader thinking that wasn’t on the radar of a typical insurance company before our study was conceived. For example: “Try to prevent this from happening again/to others” (see Figure 1). This included statements such as:

  • “Convince someone in charge to do something to prevent this from happening again.”
  • “Change my actions so this doesn’t happen again.”
  • “Prevent an accident by following safe habits.”
ins02.gif Figure 1. A peek at a section of the data collected for a near-miss-accident study. Starting at the 5th (rightmost) column you see actual quotes from the transcripts. Each concept the quotes represent is summarized in column 3 for easier comparison to other concepts. These are pulled together into patterns by the intent of the speaker, at two levels of hierarchy in columns 2 and 1. Comparisons are conducted from the bottom up. Column 4 contains the ID of the participant.

Later, when you turn away from the people and the problem space and back toward your solutions, you begin to select problems to solve and then weave together ideas that might support people. The aggregate patterns from the problem space inform and inspire idea generation, strategy decisions, design direction, and algorithmic limits. The new concepts brought to light by the data allow the organization to consciously choose a direction. Using our example, an insurance company can offer support for the concept of trying to prevent the accident from happening again or to others, such as by partnering with public-safety officials to warn other drivers of a temporary hazard like ice on a particular section of steep road, and then retracting the alert when the danger has passed.

back to top  Mental Model Diagrams, Opportunity Maps, and Thinking Styles

Two approaches bring a frame of inquiry that keeps you in the problem space: mental model diagrams and thinking styles. Other ethnographic research can also be used to explore the problem space, but in the software world, ethnography is most often applied within the solution space.

A mental model diagram is a horizon diagram that collects participants’ inner reasoning, reactions, and guiding principles as they accomplish a larger purpose into towers that resemble a city skyline [3] (Figure 2). These towers are further grouped into “mental spaces” representing different areas of thinking, such as “find out if anyone was hurt” and “feel upset with myself.” A mental model diagram is a type of affinity diagram structured by these hierarchies of towers and mental spaces. Organizations can add to the diagram with studies over time. Mental model diagrams have longevity because people’s reasoning around a higher purpose, and indeed the purpose itself, does not change much, even though supporting services and technology do.

ins03.gif Figure 2. The mental model diagram from the example study about recovering from a near-miss accident.

Below the horizon line, an organization’s services and technology are aligned to the towers they support. Taken together, the diagram is called an opportunity map (Figure 3). It compares how people think to how the organization supports them, reveals gaps where towers receive weak or no support, and illuminates assumptions about what constitutes “the problem” for people in their larger context. Teams use the opportunity map over the long term to generate new ideas, redevelop metrics, and prioritize efforts based on their positive impact to a person’s experience with the organization.

ins04.gif Figure 3. This example of an opportunity map shows capabilities of an organization aligned to the concepts expressed by people and collected into towers above. This particular example also shows a layer of capabilities created by a competing organization.

Mental model diagram data is collected through listening sessions, in which participants articulate their deep inner reasoning about why and how they achieve a purpose. Listening sessions are different from interviews, which are common in the solution space and usually follow a set of agreed-upon questions. The transcripts from the listening sessions also act as the source of data for creating thinking styles, which are behavioral audience segments similar to personas or archetypes, representing an approach to the purpose. The term thinking styles is meant to imply flexibility, because as a person moves from context to context or grows in experience, their approach may change [4]. For example, a person’s thinking style about driving a company truck might be different from their thinking style about driving their private vehicle. Thinking styles are written without reference to demographics, unless the scope being explored connects to a demographic factor, such as discrimination [5]. Used in conjunction with specific contexts, thinking styles help teams explore edge cases in a more rigorous, conclusive way.

Because of what has already failed in technology (e.g., the news bubbles and hijacking of people’s attention) and what is coming (e.g., machine learning and emergent experiences), reliable methods for undiluted problem space exploration are needed. In the case of mental model diagrams, there are a lot of examples to follow. Over the past two decades, mental model diagrams have enabled progressive teams from a range of organizations to more confidently innovate around commercial and social challenges.

back to top  Overcoming the Reluctance

Within the culture of many for-profit and even nonprofit organizations, there are misgivings about work that does not directly improve the offering. This feeling pairs with a conviction that enough is already known about the problem. Team members may have years of experience in their industry and prefer to make progress rather than investigate any assumptions at the core of their work. Over the years, we’ve helped leaders show their organizations the value of problem space research. There are a couple of techniques we use to overcome this reluctance.

First, there are useful mindsets. Solving problems is fulfilling. Ideas convey status upon the teams that conceive them. Being asked to abandon the solution space and its visible development activities for a time is like being asked to abandon the mindset that brought you to where you are in your career. Entering the problem space becomes an exercise in reframing beliefs about work. Additionally, organizations race headlong through development cycles, applying agile and lean practices to make progress with a minimum viable product every few weeks. A mindset focused on the speed of the cycles has taken hold of business and has virtually prevented any contemplative approaches, except within innovation centers or where leaders have the power to establish this practice.

To help organizations gain better clarity about the role of the problem space, the advice we have is to decouple it from the design/develop/improve cycles or any of the other solution space activities (Figure 4). It is not a step within the solution space cycle. It does not come first but rather is a separate and ongoing resource. You don’t need to study the problem space with every cycle of development. It can be added to with successive studies, once a year.

ins05.gif Figure 4. It is helpful to think of the problem space–aggregating data about the inner voices of people pursuing a purpose–as disconnected from the solution space. To allow undiluted knowledge to form, it can’t be a step within the solution space. Problem space knowledge, such as mental model diagrams and thinking styles, supports more focused and specialized work in the solution space.

Another bit of advice comes in the form of an analogy that may help you explain how to fit problem space research into the culture of software development. Problem space research is like the sun, giving energy to all the spinning solution projects orbiting it (Figure 5). Lots of research and production happens on those planets, evolving the function and presentation of the service or product. The energies from the sun—this knowledge of mental spaces that were not on the radar, or thinking styles that had been ignored—provide vision to the team and help them focus their efforts to support specific edge cases. Problem space research is where you develop empathy so that you have a solid understanding you can use when you apply empathy when you are designing solutions. It is the pathos in Aristotle’s triad of ethos/pathos/logos [6]. Perhaps the team will write algorithms specific to the person driving a company truck, in the context of defusing the tension between the person and the other driver involved in the accident. In a world of machine learning, specialized algorithms will be prevalent, along with the ability for these algorithms to listen to their users and hand off to other algorithms that may be able to better support a particular thinking style and context.

ins06.gif Figure 5. In this analogy, which may help explain the problem space, think of the sun as problem space knowledge accumulated over time. In the beginning, this problem space knowledge is the native knowledge of the team. It may then mature to include secondary research and then mature again to include primary research. The sun gives energy to support growth and clarity on every circling project.

Our final tip is to use language that resonates with decision makers at your organization. Non-researchers are often more comfortable with language that conveys certainty that solid results will come out of the expense and effort of research. Words like test, validate, determine, measure, and identify convey confidence. Words like explore, investigate, study, observe, and uncover tend to resonate best within the research-practitioner community.

An unexpected benefit of problem space research is that it requires decision makers to choose an audience and context to explore. Narrowing down from the-sky-is-the-limit creates fear that better opportunities might be found in a branch different from the one they chose. Part of the work going into problem space research is to help stakeholders carefully consider all the avenues to explore and become comfortable with a focus that will provide the most input for high-priority business opportunities.

The benefits of problem space research are strategic and recur over time. When you are working on your solutions, problem space research gives your organization both direction and subtle advantages.

back to top  References

1. Within the academic discipline of design research, awareness and adoption is much greater. In those areas, studies exploring the nature of creativity have produced papers that define concepts such as “co-evolution of problem-solution” and “wicked or tame problems.” See Dorst, K. and Cross, N. Creativity in the design process: Co-evolution of problem–solution. Design Studies 22, 5 (2001), 425—437. And Rittel, H.W.J. On the Planning Crisis: Systems Analysis of the “First and Second Generations.” 1972.

2. See the description of the bubble people build around themselves in: Gray, D. Liminal Thinking – Create the Change You Want by Changing the Way You Think. Two Waves, 2016.

3. See Young, I. Practical Empathy – For Collaboration and Creativity in Your Work. Rosenfeld Media, 2015; and Young, I. Mental Models – Aligning Design Strategy with Human Behavior. Rosenfeld Media, 2008.

4. The idea of flexible representations of audience groups is not new. It is known in marketing via various terms, including component lifestyles. See the marketing text book: Lamb, C.W., Jr. MKTG. South-Western College Pub., 11th edition, 2017.

5. See Describing Personas on Medium.com, by Indi Young, Mar. 14, 2016; https://medium.com/@indiyoung/describing-personas-af992e3fc527

6. Aristotle was referenced by Richard Buchanan, who shaped the Carnegie Mellon (CMU) School of Design in the mid-1990s to early 2000s. See Gajendar, U. Notes on the future of interaction design. Interactions (Sept.-Oct. 2017), 46.

back to top  Authors

Indi Young is a freelance problem space researcher and coach. She helps organizations reframe offerings, innovate, and strengthen the way they support people. She was a founding partner of Adaptive Path (acquired by Capital One) and has authored two books: Mental Models and Practical Empathy. indi@acm.org

Kunyi Mangalam has used qualitative research for 30 years to reveal how people think, feel, and live, helping organizations understand their audiences more deeply and thus make better decisions. kunyi@square-rainbow.com

back to top 

Copyright held by authors. Publication rights licensed to ACM.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2018 ACM, Inc.

Post Comment


@Mark Albin (2012 06 30)

This is a very interesting article about social bots, thanks for sharing.

@Aman Anderson (2012 07 18)

This is great
“So what’s the center of a design? In one sense, it is the designer’s nuanced understanding of the problem or opportunity at hand. The focus of design is problem solving, not self-expression.” - Uday Gajendar, Interaction Designer

@Bill Killam (2012 07 31)

This is a long overdue article.  And I couldn’t agree with it more.  I’m current working on yet another Federal RFP that is asking for us to do work using short cut methods that are likely make it harder to get them quality results, and we can probably propose a cheaper and more data rich approach if they didn’t specify how we had to do the job.  Sad.

@Demosthenes Leonard Zelig (2012 08 12)

Great Article, it is funny to notice that such huge corporations do not even bother to do a market research before releasing products on a new market. However, I guess we are still learning from our mistakes.

@karla.arosemenea@gmail.com (2012 10 24)

Hi everyone, In the Technological University of Panama there is also a movement. There is a 2 years MS in IT with a specializtation in HCI. We are also trying to include HCI as part of our main curricula. This year we started a research with a company interested on incorporating usability in their development. We expect to receive a Fulbright Scholar next year in this area…

Regards,

Karla Arosemena
Professor

@John Michael Sheehan (2012 11 06)

There are thousands of blogs that requires comments on them. What is the intention of blog comments? Sent From Blackberry.

@Junia Anacleto (2012 11 07)

A very shallow and naive view of a much more rich and complex context.
I am still waiting for a fair position paper to be presented.

@Rick Norton (2012 11 17)

Excellent article raising significant issues that are largely overlooked.  The prospect that the collapse of sustainability for a growth/consumption related societal model is inevitable, is a topic I have often wondered about, given the nature of capitalism as we know it today.  Even the “Great Recession” of current times gives me pause to wonder just how long we can keep this economic engine going before we have to face the reality that we are all going to have to learn to “live with less”.  (A quantitative assessment, not necessarily qualitative.)

Keep up the good work.  Hopefully, you will raise awareness of these topics.

@Noah McNeely (2012 11 27)

Very nice article, that raises meaningful questions.  I actually think that the idea of sustainable products and sustainable product development is a bit of a myth.  All products consume energy and other resources in one form or another during their production, use, or re-use.  The key, ultimately is to balance resource consumption with resource production, but we will always need to be producing new resources.  See my blog post on the subject at ( http://productinnovationblog.blogspot.com/2012/11/are-there-sustainable-materials_7159.html )

@ed.h.chi@gmail.com (2012 11 30)

The quote in the article mis-contextualize James Landay ‘s essay. James actually is actively working to break down those stereotypes, but you can’t do that without understanding what the deep problems are.

James’ blog post on this is at
http://dubfuture.blogspot.com/2011/12/china-will-overtake-us-in.html

@Lee Crane (2012 12 03)

This is a topic that is thought provoking and important.  The message explores how humans can escape and survive the world they have jumbled.  So many of the theories and ideas are basic.  Our future may look a lot like the distant past.  And indeed we may be happier for it.

@ 4996484 (2012 12 19)

this is a great article David and Silvia!  I’‘m so excited that you guys wrote this up and are showing everyone the complexities in this space. I hope Interactions features more of this kind of research on China.  Although I agree w/ @landay’s assessment of China’s creativity problem - but he’s working with a very different population than you guys. I think you research is absolutely on point - creative folks are going to hacker spaces like Xinchejian, they aren’t ending up in institutions like Tsinghua!  I explain more here:  http://www.88-bar.com/2012/12/where-are-all-the-creative-chinese-people-hanging-out-in-hacker-spaces-apparently/

@Joe (2013 01 04)

I think that if you study the Elliot Wave Theory it can answer your questions.

@Rafeeque (2013 01 06)

good one

@zhai (2013 01 16)

Enjoyed reading this article. I finally got why Harold wants to call it “the Fitts law”. If enough people write it that way I would never have to correct another submission making the embarrassing mistake of ‘Fitt’s law”.

I did not completely get the following remark though:

        “The Accot and Zhai paper about the Fitts Law [3] has a clever title that illustrates
        the rules on letters, “More than dotting the i’s…”—a bad pun on eyes.”

I came up with the title, but the word “eyes” never came to my mind. We meant that the point-and-click style of UI is like dotting the i’s everywhere—- placing a click on constrained targets as the fundamental action in interaction. Why not using ” Crossing the t’s ”  as an alternative action?  Indeed, we presented models of a new style of UI, which systematically reveals when crossing is superior to clicking,  hence the subtitle of the paper “Foundations for crossing-based interfaces.”

Shumin Zhai

@Mohamadou M. Amar (2013 03 22)

I am a Doctoral student in I/O Psychology with Touro UW and need to access your articles.

@Mohamadou Amar (2013 03 22)

Need access for Doctoral Research

@William Hudson (2013 04 09)

Gilbert overlooks the important issue that the ‘big boys’ largely do not appreciate the need for design all and the problems that real people have with technology. I admit that we’ve had a hard time selling UCD but I am not persuaded by the arguments here to abandon it. Perhaps have a look at my article on a similar subject - User Requirements for the 21st Century - where I take a more pragmatic view of trying to address real users’ needs in the development process. http://bit.ly/agile-ucd

@ 0343665 (2013 04 29)

Fantastic text. I came here by searching for people that quote the Standford study on multitasking. The introduction is fantastic as it builds up an argument that attention has some features that do not change over time.

@Simon Taylor (2013 04 30)

not wanting to do anything so grandiose as building a (technology for) a world parliament, I have in essence been working on the same problems and facing the seven challenges with a project called ‘company.’ [https://gust.com/c/littleelephantltd]

In 2011, working with senior software developers - gratis - although neither the ethical undertaking nor the promise of sweat equity were enough to keep them involved - I established the technical feasibility of ‘company.’
h
In 2012, turning from the ‘voluntary’ ‘principled’ participation model - because the attractions of real paying jobs had lost me my team - I received financial support from the New Zealand government. This part-funded an Intellectual Property Position Review - which government considered a pre-requisite - as commercial due diligence - to investing in an initial build, or beta. The IPPR recommended I do proceed… However, government offers only part-funding and without a team - either technical or commercial - there has been little to no investor interest.

As things stand at present, I have the tools and schematics for a beta build of something which would fit the sort of use imagined here. If you have any interest in helping, please contact me.

Best,
Simon Taylor