In the documentary “Keep the River on Your Right,” anthropologist Tobias Schneebaum is the cultural expert aboard a cruise ship traveling the waters near the Asmat region of New Guinea, where Schneebaum once lived. He brings fellow passengers ashore to witness tribal dances and circumcision rituals. Schneebaum characterizes these passengers as tourists who are interested visitors. In contrast, he presents himself as an observer who stayed.
It’s an interesting continuum: from passive interest to active examination, from temporarily visiting to semi-permanently staying.
One of the criticisms (and there are several) that academic anthropology and its adherents have voiced over the contextual approach taken by user researchers concerns the brevity of field time; methods and theory that assume the researcher will live with subjects for months or years are applied to projects lasting only weeks. What depth, critics ask, can be learned in such a short time? Contextual researchers engaged in design and usability activities are more than visitors and less than cohabitants; we are more than interested but less than exhaustive documenters. The key (as any methodologist will tell you) is to match the time invested to the level of insight required.
In product development there’s enormous pressure to produce results in reduced time. This is why there are practitioners advocating for sexy-sounding approaches such as “extreme user research,” “guerilla ethnography,” or “rapid ethnographic assessment.”
Insights in 33 minutes, or your money back?
Although I’m concerned over the mistaken belief that the time invested can be squeezed and squeezed again while still producing the same value, there’s a more important attribute of time to consider. Rather than the calendar time in the field, let’s take time as a mind-set and consider the pace at which we work.
Last summer I sat in on a focus-group-like session. We were at the end of a long table of people whom we had met in various observations and interviews throughout the previous week. One of the clients who had commissioned the work was sitting at our end of the table and operating the video camerano small task, with about 12 people engaged in conversation. At one point she turned to me and asked: “We don’t need to get this stuff right now, do we? Nothing’s happening, so I can stop recording?” Surprised, I encouraged her to keep the video rolling. Editing in-camera may have worked for Hitchcock, but it’s absolutely not the way to go for any sort of user-research process. You don’t necessarily know the value of what’s happening in the moment that it’s happening.
As a corollary, although I rarely immediately discard something that happens in these settings as not valuable, I do sometimes notice things that really excite me in the moment, things that are clearly quite valuable. But that’s usually a moment of discovering a pattern across multiple interviews. I would encourage our client with the video camera to simply slow down and let things unfold. It’s not that we aren’t structured or planful in our work, but much of the structuring happens in the preparation: finding the right people to talk to and figuring out what you want to talk with them about. Successfully executing the plan requires that we trust our process, and in many cases that’s about slowing down and building a space for the work to happen. We need to slow ourselves down, and we need to slow down our inner critics. We must be prepared to be surprised when we encounter something we weren’t even looking for. Our video camera (both literally and figuratively) must be on “record.”
Our clients shouldn’t approach contextual research expecting insights to magically appear, nor should they expect them to appear within any individual session. In order to recognize a pattern, we must encounter an individual instance multiple times. I recently taught a design research class to undergraduate industrial designers. For their final project, one team presented a series of needs and solutions that came from their research, except that they found a unique need in each of their subjects and designed a series of individualized solutions. Naively (and understandably), they didn’t see that patterns across multiple individuals were the more realistic opportunities to address with their solutions.
But how do we know that something we’ve found is important? How can we see if it is actionable, relevant, or insightful? For market researchers the insight is the coin of the realm. The word “insight” refers specifically to the output of a research process, less so to the dictionary definition“the act or result of apprehending the inner nature of things or of seeing intuitively.” Designers might speak about needs or opportunities in the same way a market researcher refers to insights.
At a recent presentation by the agency Directions Research, we were reminded that “insight is in the eye of the beholder, and the beholder is your client” and that if the insight isn’t new or isn’t needed, then it isn’t an insight. I have a little trouble with this definition. If we come to our clients with a framework, recommendation, opportunity, need, or (even) insight and the response is, “Well, we already knew that,” then we might want to work further with them to understand what has stopped them from acting on it. Alternatively, if the response is, “We don’t care about thatthis insight isn’t an insight because it’s not needed,” we haven’t done a good enough job of explaining the critical implications of what we are reporting. The insight is embedded in the context of the organization, their history, and their ability or willingness to take action.
What we need in order to get to these patterns across multiple data points is permission to be confused. Confusion is part of the process. Jakob Nielsen’s famous “Five Users Is Enough” graphic (http://www.useit.com/alertbox/20000319.html) shows that in usability testing there are more results for more users (up to a point of diminishing returns). But imagine graphing a more divergent process such as contextual research. Although the trend would be similar, if we zoomed in close enough, we wouldn’t see a smooth line, but a jagged progression as more data challenges previous conclusions and opens up new ones. If you remember your university physics course (and who among us doesn’t?) it’s similar to the relationship between Einsteinian and Newtonian physics: You wouldn’t see the Einsteinian effects unless you were able to look really close at that bowling ball dropped off the Leaning Tower of Pisa. But if you never took university physics, I should explain that Newton described a number of simple equations that captured the mathematical relationships between factors like force, mass, and acceleration. Much later Einstein theorized that (among other things) as objects increased in speed (heading toward the speed of light), their length increased along the direction of travel. When we look at incredibly high speeds, we presumably can see the effects Einstein described, but for most of what any of us can experience here on Earth, the model that Newton proved holds true (enough). So perhaps with the creative process of converging on an insight from a certain distance, it looks like the more people you look at, the more you know. But if you zoom into a different point along that process, you can see that we are probably zigging and zagging in a divergent fashion.
It’s worth being reminded from time to time that finding patterns and insights is an inherently creative process. You have to live with your data in order to get to a point where you can start to make sense of it anew. Closed-ended methodologies like focus groups and usability testing often involve a sequence of moving from one facility in a city to another facility in another city and finally delivering a report the day after returning home. And that may work for those approaches, but not every sort of insight can happen without time for reflection and gestation. Developing these insights is an act of creation, production, and generation. We are making a new thing: perhaps a new product, or service, or communication, or perhaps the new knowledge that will specifically inform the rest of the process of making a new product, service or communication. And we need to give ourselves permission to take timejust enough timeto nurture this necessary creative act.
Steve is the founder of Portigal Consulting, a boutique agency that helps companies discover and act on new insights about themselves and their customers. He is an accomplished instructor and public speaker, and an avid photographer who curates a Museum of Foreign Grocery Products in his home. Steve blogs regularly for All This ChittahChattah, at www.portigal.com/blog.
©2008 ACM 1072-5220/08/0700 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2008 ACM, Inc.