Challenges to Design Research

XVII.2 March + April 2010
Page: 43
Digital Citation

An introduction to casual data, and how it’s changing everything

Lauren Serota, Dan Rockwell

back to top 

People today are empowered by online transparency: They see their actions take shape online. A plethora of outlets for opinions allow us to comment, share, and collaborate in spaces like Amazon or Facebook; we can be heard, and the machine of the Web records everything. These outlets are also excellent data-capturing tools and can be mediums for companies to engage with consumers—provided they're paying attention.

The connectedness of the Web and its ability to empower take traditional word-of-mouth to a hyper level. The more hyper this communication, the more useful (and simultaneously complicated) it becomes. Suddenly, we have a bounty of free and trackable data. With a shift in consumer behavior toward vocalization/co-creation and a newfound abundance in rich data, one must ask, what is the role of design research? Well, researchers' roles don't change as much as they become vital to properly understanding and utilizing this new wealth of information.

back to top  Lots of Data, What to Do?

People are swimming in data like never before. Services like Nike +, Daytum, and Mint allow anyone access to endless aggregations of personal information, thereby increasing self-awareness and shifting perception. Companies also utilize these tools to harvest commentary on Twitter, Facebook, and via customer-service interactions, connecting to the minds of some of their customers more than ever. However, having access to this information (which we're going to refer to as "casual data") doesn't mean that companies/brands, or consumers, for that matter, understand it or have the means to translate it into meaningful direction for business strategy, product development, or design. In most cases companies run the risk of acting too fast and have a shortsighted glimpse of online responses; there is a ton of swimming going on, but not enough surveying of the waters. David Armano summed up this conundrum well in a recent blog entry, stating: "Technology can help us listen to everything that's ever been said about ourselves or our company, but finding that one key, game-changing insight and then actually doing something about it is another story" [1].

Here, we use the customer experience with online shoe retailer Zappos as an example. It presents, by far, one of the best stories of a niche offer that engages its customer and improves user experience using available online tools. While focused on the customer in more ways than this article can detail, Zappos spent a lot of time developing practices to utilize the power of online consumer sentiment, hence their success story unfolded.

back to top  How Companies are Farming Data

First, let's talk about three ways that companies are currently using new media to harvest casual data. These approaches have varying levels of investment, outcome, and potential to influence a business overall.

1. Listening; query your brand and spot a pattern. Aggregating what people are saying about their experiences with a product or service, via an existing online medium (Twitter, Facebook, Ning, message boards, etc.), helps companies listen in on what consumers are talking about in regards to their brand or domain.

The current, most prevalent tool for mining these digital conversations is sentiment analysis, which allows anyone to search for keywords and sentiment, i.e., "____ sucks," "____ rocks," "I love ____ brand," "I hate ____ brand." At this basic level, you don't get much other than the whiff of a good or bad vibe related to your brand. A recent New York Times article makes mention of light-weight tools [2]; we think of them as experiments in data mining, a less useful but more interesting way to get a sense of general customer sentiment. More powerful tools trawl the Web from thousands of sources—as many as they can tap into. Some companies (like Zappos) incorporate casual data from listening by consolidating it and allowing it to be explored by their UX team.

What it's good for:

  • Monitoring the effect of a new product launch or ad campaign
  • Gaining competitive intelligence
  • Listening in on pain points
  • Developing ideas for future development (brain fodder)
  • Gaining inspiration from consumer suggestions (design ideas)


  • Utilize existing tools and services
  • Leverage unique key words
  • Analyze for sentiment and frequency
  • Map to related timeline (against events, trends, etc.)

Try this out: Go to, type in a brand name—Nokia, maybe Wendy's, anything you want—and see what you can find. Listen in. Add some sentiment queries on top, get creative—"____ sucks" or "_____awesome."

The Challenges: Searches are key word-specific, and most data comes without strong demographic or user information to help in synthesis. Getting rich meaning from this data is difficult.

Who's doing it: Everyone who cares to listen in.

2. Farming; create a specific place (branded or not) to listen more, push your agenda, or get more specific input. Launching a product/service and actively seeking insight by asking consumers to give directional input to iterate rough edges.

What it's good for:

  • Promotion
  • Establishing a presence
  • Projecting a sense of connection/care
  • Getting customers to give directed opinions, which can be used as design direction
  • Allowing users to experience and gain equity in your brand by giving feedback


  • Develop a portal/place for people to comment on your product or service
  • Ask for help
  • Reward people for engaging, keep them updated, and make them partners
  • Gain brand sustainability (people have equity), free ideas, and cheap advertising

The Challenges: It involves a bit of front-end planning and capitalizes mostly on an existing user base. Though input is thoughtful, these are not personal conversations happening between users and companies.

Who's doing it: Boeing, Dell, Starbucks, P&G, and Target

3. Engaging; reach out to the customer directly and have a conversation. Establishing a method of collecting and acting on consumer issues faster, and nurturing relationships with users to help build and evaluate products.

We see two methods of engaging: Acting on listening quickly in order to limit the vocalization of a bad experience and deliver resolution to the customer. At Zappos; we bought shoes from them; they came in the wrong size; we complained on Twitter; Zappos responded to us instantaneously, asking how to make it right. It utilizes the Web to pick off bad-sentiment scenarios like this and deliver good customer service in real time.

Revealing the bigger plan. Think of this like co-creation techniques in research: You bring the customer into the design process, make them part of your team, and tell them how they will matter. Starbucks's MyStarBucksIdea is the best example of this kind of engagement as it exists online. The company crowdsources its customers for ideas, lets them vote on the ideas in a Digg-like fashion, then pushes high-rated ideas through the innovation process and shares the project timeline so that contributors (customers) get a sense that their idea is being acted upon and can track it. Starbucks took farming to the next level and communicated the outcomes of user input back to customers.

What it's good for:

  • Immediate customer service (nip something in the bud)
  • Creating listening posts for consumers to engage
  • Employing farming methods to get feedback
  • Getting input for iterative product development
  • Sharing your road map to innovation (be transparent about changes and your org)
  • Treating customers as your staff
  • Gaining brand sustainability by engaging people both online and offline and letting them see their contribution


  • One-on-one interaction to intercept user feedback from listening
  • Develop a portal/place online, paired with real-life incorporation of customers
  • Reward people for engaging, keep them updated, and make them partners

Who's using it: Get Satisfaction, Zappos, Starbucks, and P&G

back to top  The Problem with Too Much Data

While there are a number of firms analyzing the surface value of casual data, there is a need to dig deeper to understand context and higher-level implications. The more connected we become, the more connected our data becomes, and the more we need a structured approach for making sense of it.

Companies accessing loads of customer data is not news. However, this casual data is not quantitative in nature. The emotional meaning behind casual data should not be analyzed statistically, and the methods used to gain this data are as important to understand as the data itself. If customer voice is harvested only through an existing medium (e.g., submitting a query for iPhone-related tweets), the results you get will be brief and will either be of intense glee—"new iPhone copy/paste function rocks!"—or intense distaste—"Apple sucks!"—leaving little room for understanding context of use, while still providing good touch points for product improvement. There is the potential for casual data to be more dangerous than helpful if it's not properly understood.

back to top  Ok, So What's Our Role?

The need to find long-term meaning via any quick, casual data-farming medium creates a niche opportunity for research firms to use their proven techniques to analyze and understand this abundance of user input. Professional researchers will be able to understand how casual data is useful, where it is applicable, and where there are still unanswered (and often unasked) questions. This will allow research companies to reinforce the practice of doing more in-depth research as a result of findings from this data, rather than allowing clients to consider and use this data (which is often incomplete) as if it were conclusive.

With a shift in consumer behavior toward vocalization/co-creation and a newfound abundance in rich data, one must ask, what is the role of design research?

Even tools with built-in analysis capabilities cannot play down the importance of involving a comprehensive research process. Design researchers look at data not only to understand design opportunities but also to come up with high-level emotional themes. If 10 people say that they want a certain feature from, what does that mean in terms of their needs and how will they benefit from that feature? Extrapolating concepts, ideas, and feedback into themes can help the design team understand trends and potential meta-themes, and consequently how to design new products and services that weren't necessarily articulated by their customers. Researchers also have the opportunity to help companies understand how to manage all of this data—does it need to lend itself to searching by future company stakeholders, or will it be regenerated? Having a plan for where the data goes can increase the value attained from it and help to track trends over time.

Design firms with research and strategy offerings may have a difficult time segueing into this type of an offer, as it is capitalizing on two parts of their business they aren't very good at billing for—analysis and synthesis. The ability to transform any kind of information into insights and solutions is one of the most valuable assets of designers' process, and it is currently the most ethereal and difficult to quantify. Articulating methods of analysis and synthesis and the importance of these stages both internally and to clients is one of the largest challenges that firms face in approaching this new opportunity [3].

As much as this is a golden opportunity for design-minded researchers, it is also a wake-up call for tried research titans. Brands have more options than ever when it comes to listening to their customers, whether they're "kosher" and in accordance with best practices or not. Market research and advertising have been the first to leverage emergent data-mining technologies, while old-school qualitative research has been slow to the scene. We see the bias issues and shudder at the lack of control with some of these harvest methods. Instead of negating the value of these tools based on their lack of compliance with our ideologies, we should acknowledge the opportunity to take what we can (and mind our own disclaimers) from available data and use it to our advantage when developing more comprehensive research plans; these tools can provide us a leg-up in our own game. Consider how ethnographic techniques, long-existing pillars of anthropology, have been employed and repurposed by the design industry. This movement online provides a new set of tools that are just beginning to be understood and will soon be adapted to suit design-specific needs.

Let's revisit Zappos. We've discussed how it is utilizing existing online tools (in this example, Twitter) as a means of engaging with its customers and resolving customer-service issues. We had a chat with Rob Siefker, senior manager of customer loyalty, who informed us of the following: Those tweets are logged and sent to the Zappos User Experience team, which uses this customer input in developing new features and improving existing ones. These new features and improvements are tested for acceptance and usability, and further iterated upon. Zappos is a good example of practically involving Web-generated user input as a springboard for customer service and new design opportunities, while not underestimating the importance of further validation and follow-up research. We can't say this approach has done wrong by Zappos—indeed, Amazon acquired it for $928 million last July.

back to top  What's Next?

The death of context? The day may soon come when online tools become so robust as to allow companies to conduct generative research efforts completely online. They will be able to harvest participants (reducing recruiting challenges based around geography), crowdsource data, and generate insights as rich as in-person interviews (with an added bonus: no data-entry time). Furthermore, these tools can be used to enrich or augment research efforts in other industries (for example, business-to-business) by providing a mechanism for gathering expert data or tracking workflow before or after fielding. The more that human-online interaction increases, the more potential patterns will be available for observation. And we haven't factored in emergent tools that pick up where verbal articulation leaves off—behavioral data capture (e.g., eye tracking) or online emotional recognition. The more the user is interacting with an "aware" product, sharing sentiment in a transparent way, the more we can gather it and use the resulting data.

All research, all the time. Advertising and insight portals like P&G moms, beinggirl, MyStarBucksIdea, and Dell ideastorm are research, customer service, and mind share wrapped into one. The idea of companies conducting research is interesting in itself, enough so that customers will benefit socially through contributing and also build equity in that particular brand. Lab-type culture is spreading as well. Recently Facebook announced its own "prototype" space, following Google's Labs model, whereby it can test out new unrefined ideas with the public, allowing the consumers to tweak them, experience them, and iterate with them.

Accuracy metrics. There are too few players in the social-media monitoring space openly policing themselves (or each other) regarding the accuracy or well-roundedness of a harvested body of casual data. Many tools are attempting to leverage community self-policing and measurement tools, but this is weak at best unless embraced by all members. Right now we trust the source for what it gives us, good or not. As with any new market, there are opportunities for tools to be cleaned up and diversified for specific use, as well as utilized in the same way that researchers now utilize varying methodologies per project scope.

Understanding demographics. Presently (especially with listening), it is very difficult to understand customer demographics. This goes to say that establishing a well-rounded demographic profile for data collection is near impossible, unless you are extending past farming into the depths of reaching out and engaging both users and non-users who may not be vocal online. As we climb up the bell curve and a majority of the general population uses online tools expressively, there is a chance to understand consumer information surrounding casual data points. Google currently does something similar with its ad algorithms.

The inevitable result of this new (and growing) abundance of casual data is a plethora of opportunities for researchers to diversify in technique and then specialize in process. Understanding how to best utilize this data will allow us a more meaningful connection with the people using products and services, and a more sophisticated means of translating these connections to beautiful and resonant products and services.

back to top  References

1. Wright, A. "Mining the Web for Feelings, Not Facts." New York Times, August 23, 2009;

2. Armano, D. "How Filter Failure Contributes to Business Failure";

3. Kolko J. "Abductive Thinking and Sensemaking: The Drivers of Design Synthesis." Design Issues 26, 1 (2010).

back to top  Authors

Lauren Serota is a design researcher and user experience designer with Lextant in Columbus, Ohio. Prior to earning a bachelor's degree in industrial design from the Savannah College of Art and Design, Serota spent her formative years in public relations and promotions for the electronic music industry. She developed her own cultural anthropology curriculum while at SCAD, and continues to seek out information and perspectives that challenge and reinterpret norms. Serota has spent time as an industrial design recruiter, interaction designer, and trend researcher. Her work incorporates her ever-present passion for cultural diversity and objectivity in the acquisition and analysis of consumer insights for product and service development. She is actively involved in the Industrial Designers Society of America and currently serves as chair of the IDSA Young Professionals interest section.

Dan Rockwell is the Tools Czar at Lextant, where he works with user experience, design research, and insight translation teams in the formation of new tools and applications to capture, analyze, and synthesize data in more efficient and useful ways. Along with his passion for research, Rockwell is a futurist of the highest degree and a self-professed trend hound. He actively monitors, shapes, and makes sense of trends in technology, fringe science, social media, data mining, collective intelligence, crowd sourcing and startup culture. In 2009 he formed Big Kitty Labs, an idea lab focusing on rapid concept development for Web and iPhone application platforms.

back to top  Footnotes


back to top  Figures


back to top 

©2010 ACM  1072-5220/10/0300  $10.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2010 ACM, Inc.

Post Comment

No Comments Found