In 2005 I attended a BayCHI panel discussion entitled "User Research Strategies: What Works, What Does Not Work." The panel featured user research leaders from key design-y Silicon Valley tech firms: Adobe, Intuit, Yahoo!, eBay, and Google. After a number of (perhaps deservedly) self-congratulatory pitches from companies that had worked hard to deploy user research into every part of their design process, Maria Stonethe first person hired by Google to do user research full timestood up and kicked off her talk with the following:
"What I'd like to argue today is the goal should be good design, not more user data. It's always possible to collect very interesting user data. But if the goal is to create a great product, is collecting user data always the best way to go about it? Do great product ideas in fact come from interesting user data?... People who invented [Google websearch] had a great idea, and they focused on this idea and they improved it by...very targeted usability testing.... Should we have told those early search engineers, "You know what, you should go just study librarians for a couple of years. You would really learn a lot about how people do search and you really need to understand that space deeper"? My answer to that question is probably no. They were doing something great. They had lots of ideas on how to do how it well. They had good tools for improving their product the way it was. Doing the broader user research may have been interesting, but given their limited resources, it probably should not have been a priority."
Perhaps there was an element of rationalization, given the limited utilization of user research at Google at that time. Stone's rhetoricimplying that a study of searching behavior would be a two-year boondoggleappears somewhat defensive. While Stone acknowledged that the search marketplace was becoming increasingly complicated, and that 2005 might have been an appropriate time to consider broader user research, her thesis seemed almost contradictory: Maybe exploratory research isn't necessary at all?
Google produced an incredibly successful and easy-to-use killer app with Google search, without doing any exploratory user research. Now, of course, Google designs a wide range of software (including chat, calendar, word processing, email, mapping, spreadsheets, presentations, news reader, browsing, social networks, blogging, photo editing, and video sharing) and they have a much larger user research team, presumably doing as Stone recommended: a significant quantity of broad-based user research.
Stone's talk offered one view of how corporations navigate the relationship between design and research. While in Japan, I observed another corporate strategy tackling the same problem. I was working with clients to conduct in-home research, and we were invited to sit in on some (so-called) usability studies. The clients were gathering reactions to different designs for inkjet cartridges. They presented participants with two different examples: First, a familiar rectilinear plastic black version with a small white label indicating the color and part number, and second, an organic soft form, molded in the vibrant color of the ink. It's no big surprise which one people liked best. They flicked the black box away, but they squealed "Kawaii!" (cute, in Japanese) and grasped and stroked the soft, bright form: a very emotional reaction.
At the time I was pretty surprised that our client was using research for something like this. One solution was the undesigned status quo; the other was absolutely gorgeous. Why bother with the expense of research to "validate" some obviously excellent design work? Eventually, I realized that my client did this research not for insight but for persuasion. They needed to convince other people in the organization that going beyond the status quo was going to have a dramatic impact, and the best way to do that was through the supposed objectivity of sound bites gathered in a research setting. With that perspective, I had to admire my clients for understanding how to help great ideas succeed in their own culture.
The classic design-without-research approach is for designers to make it for themselves. This has been heralded as the best approach by software company 37signals: "Every product we build is a product we build for ourselves to solve our own problems." But there seem to be some cultural consequences for hewing to that attitude too dearly. Witness this 2006 blog post from 37signals :
"While we appreciate customers who take the time to write in and tell us what they want, the way people phrase things often leads to raised eyebrows. Every feature that's missing is essential, a must-have, and the fact that it's missing is killing someone. Yet the #1 thing that people like about our software is how simple it is. To give you an idea of what it's like to be on the receiving end, here are some excerpts from recent 37signals support emails and forum posts."
Unfortunately, the posted excerpts indicate a poorly concealed contempt for their own customers for being too intense, too clueless, or basically not cool. The mirror between designer and user cracks when you stop loving your user as you would love yourself.
I used to think there was a certain class of company for which "design for yourself" would work: Companies founded (and staffed) by enthusiasts for products like pro-audio gear, mountain bikes, or camping gear. Those companies tend to brand themselves as active participants who know what an extreme backpacker or serious dirt rider or gigging bass player would need. By extension, they hope customers will perceive their products as authentic and high quality. But I had my eyes opened a few months ago in a conversation with Steve Brown, head of design and user experience at Nortel, and formerly a partner at Fiori Product Development. Steve suggested that this approach may be fine for an entrepreneur who is starting a company, but he has seen many larger companies who believed they were the customer and were thus unable to innovate because they couldn't see the market differently.
While user-research-eschewing Apple is everyone's poster child for "design for yourself," I find Harley-Davidson to be a more compelling example (although I may be comparing Apple(s) and oranges). At Harley, Willie G. Davidson is the grandson of the original Davidson. Senior vice president and chief styling officer, he is known as Willie G. And he looks exactly like a guy who rides a Harley: big, bearded, and leather-clad. If we judge a bike by its fairing, the designer is the customer. That's part of the Harley brand: In a recent Harley-Davidson annual report, executives appear next to their bikes, and we know that they all ride. A crucial part of Willie G.'s role is to preserve the legacy of the brand; the company communicates that it is (and always has been) part of the culture for which it's designing. People at Harley, we believe, use the products and live the lifestyle. But underneath it all is a sense that Harley-Davidson, through its history, has created the brand (i.e., the products and their meaning) in partnership with its customers. For all the tribal connectedness Apple has facilitated, the company itself is not a participant. It is a benefactor.
Meanwhile, some companies announce proudly that they don't use market research because they are creating the vision, and people can't ask for the future until it's created for them. I once sat in the lobby of a major Asian consumer electronics firm and read a similar statement in one of those coffee-table-design-porn-slash-legacy books. But then I went into my meeting and kicked off a really interesting and impactful customer research activity, and no one once mentioned the chest-thumping statement in the book. Often the rejection of research comes out of a failure to understand how to conduct it effectively. For example, in Brunner and Emery's book Do You Matter, they champion the need to conduct upfront research in the design process to "understand what people are doing, how they're doing it, what's going on in their lives, what their issues are, and what problems they face." But a few paragraphs later, they appear to be very cautionary about evaluative research, throwing out the baby with badly executed bathwater:
[T]hey put a design in front of customers and say, "What do you think?" And the customers say, "Well I don't know; I don't know if I like this; it's new; it's scaring me; it's too big; it's too round; it's too square." That's the kind of response you get. People who use this kind of research come back and say to the designers "People think this is too squareyou've got to make it more round." Most customers have a hard time articulating their design preferences. You can do far better by watching, listening, and observing.
I'm a big fan of "what do you think?" questions because they let the participant respond on their own terms first. But to be effective, there's much more to consider: What do people tell you first; how do they tell you; what reasons do they give; how can you triangulate that response against other things you've learned about them; and how can you help them get to a point where they're engaged enough in this new idea to give a meaningful response? And of course, we don't have to take these answers literally and make our design more square or more round; we can see that those responses are trailheads to follow for a deeper understanding of how this new thing is or isn't making sense to them.
Like everything else in design and research (often overlapping terms that I've avoided specifying here), the answer to "design without research?" is, it depends. Among other factors, it depends on how much we already know about our customers (perhaps through our own experience). It depends on what we hope to learn and how we want to use that learning to create action. It depends on where we are in the development timeline of a product or service, and whether the product or service is new, metoo, innovative, or a redesign. It depends on business constraints like time to market, the maturity of the category, and the cost to evolve the design. No doubt it depends on other things as well. What do you think it depends on?
The company communicates that it is (and always has been) part of the culture for which it's designing. People at Harley, we believe, use the products and live the lifestyle.
Next time, I'll look at part two: "Research without design?" And even though we'll probably end up at "it depends" again, I expect the trip will be thought provoking.
1. "Useless, absurd, must, need, appalled, just, infuriating, essential, etc." Signal vs. Noise. 12 April 2006. <http://37signals.com/svn/archives2/useless_absurd_must_need_appalled_just_infuriating_essential_etc.php>
Steve Portigal is the founder of Portigal Consulting, a boutique agency that helps companies discover and act on new insights about themselves and their customers. He is an accomplished instructor and public speaker, and an avid photographer who curates a Museum of Foreign Grocery Products in his home. Steve blogs regularly for All This ChittahChattah, at www.portigal.com/blog.
©2009 ACM 1072-5220/09/0500 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2009 ACM, Inc.