XXI.3 May-June 2014
Page: 62
Digital Citation

Fast, cheap, and powerful user research

Jennifer McGinn, Christopher LaRoche

back to top 

As UX professionals, we often hear that there is no time or budget to conduct user research—sometimes from our peers [1]. By contrast, we want to assure you that in addition to creating an inexpensive mobile lab [2], you can conduct user research on any budget and any timeline, while still getting high-value user feedback for your teams and stakeholders.


Aside from the usability lab, the common costs associated with user research are recruiting, participant incentives, and time to conduct the research.

back to top  Participant Recruiting

Using a professional recruiter is sometimes necessary when you need to locate and schedule participants who are hard to find. In those cases, the recruiting costs will generally run you $150 to $200 a participant, plus an administrative fee. But if you don't have a budget for recruitment, here are several ways to recruit those participants with very little investment on your part:

  • Talk to sales and marketing. If you need participants who already have experience with your product, they will know where to find your customers and will usually welcome a new reason to reach out to them. They can give you access to customer lists and can be great partners, as they understand the need to gather data from users and customers.
  • Mine the licensee database. At one former company, we were allowed to download records for every copy of every software product, thanks to a marketing contact. While we had to ensure that we didn't contact anyone who had opted out of emails, we still had a huge database of potential participants to draw from.
  • Attend the local user group meeting. Your users may be hanging out at monthly meetings within commuting distance of your office. These meetings may be industry specific, such as program management groups, or they may be product specific, such as Java user groups. Talk to the people running the meetings and ask to say a few words during the introductions. Bring chocolates or swag (e.g., branded company pens) to exchange for participant contact information.
  • Use employees. In many cases, we have had products that are used internally before they are released to the public. In these cases, our only existing users work for our companies. Even when they aren't your current users, you may have people at your company who match your product's user profile (e.g., sales people, social media users, shoppers, or parents). Leverage your internal mailing lists, including all the information in email such as what research you'll be conducting (e.g., interviews, focus groups), what user profiles you're looking for, the dates of the research if you know them, the amount of time you'll need from the participants, where the sessions will be held, and the incentive that will be offered.
  • Get your networks working. Even if you can't use employees as participants, you can recruit their friends and family. Send an email to everyone in your location, building, or company, that includes all the information you would provide when recruiting employees. Use social media, professional networks, and LinkedIn groups as well.
  • Link on the website or home page. At one smaller company, one author was lucky enough to get a link on the home page, which allowed users to opt into future user research activities. While you could link it to a form, we just linked it to a survey that collected all their screening criteria as well as their contact information.
  • Use social media. Don't have access to the website? You can recruit users via LinkedIn groups, organization Facebook pages, and Twitter feeds. This is a slightly different take on a friends-and-family recruit, because rather than using your network, you're using the company's social media resources.

Sometimes customers participate because they are flattered that you value their opinions. But most often they participate because they know that they are the ones who will benefit from product improvements.

back to top  Participant Incentives

Again, when you need to use an external recruiter, you need to. And when you do, you will likely be paying your participants $100 to $200 for a one- to two-hour session. In general, the incentive is really a way to compensate the participant for the inconvenience of the session. So how can you drive down the costs of incentives?

  • Use customers. In addition to reducing your recruiting costs, using customers can reduce your incentive costs for several reasons. Sometimes they aren't allowed to accept incentives because of other agreements your companies have in place with each other. Sometimes they participate because they are flattered that you value their opinions. But most often they participate because they know that they are the ones who will benefit from product improvements. When we use customers, we offer either no incentive or inexpensive company-branded items such as hats, pens, or coffee mugs.
  • Use employees. Again, you often don't have to offer incentives to employees because they are motivated to make the company's products better. But we have found that $5 to $10 gift cards to trendy coffee shops, Amazon, or ice cream shops work wonders at getting employees to participate in sessions lasting up to 30 minutes.
  • Use food. Don't underestimate the power of a free lunch. For employees and friends-and-family recruits, we have offered lunch as the only incentive for focus groups, including $10 lunch vouchers for the company cafeteria to employees for sessions lasting up to an hour, and sandwiches during interviews.
  • Reduce the inconvenience of the session. You can go to the participant, rather than asking them to come to you. We used $25 incentives to visit people in their homes to learn about their social networking use. Likewise, you can visit your customers, but let's say your customers are in another state or country. No problem—you can use phone interviews or remote desktop sharing to bring the session to them. More on this later.

back to top  Quick User Feedback Methods

Now that you know how to drive down the cost of recruiting and incentives, how can you reduce the time it takes to gather user feedback?

  • The lobby evaluation. Camp out in the lobby of your building or the cafeteria (with permission, of course) and ask passersby to participate in your five- to 10-minute feedback activity. Offer a small incentive (say, a $5 card to a coffee shop), screen them on the spot, and run your session. We've been able to get 20–30 participants in short order.
  • Crowdsourcing. Many people will tell you how quickly and cheaply they have been able to get feedback using crowdsourcing systems such as Amazon's Mechanical Turk. We've had friends hold a study in a day for as little as $2 a participant. Likewise, if you're using your organization's social media channels, you can post links to get rapid user feedback for surveys, online card sorts, or other forms of unmoderated user research.
  • Go small. Sometimes the secret to getting quick user feedback is to scale back the size and duration of the activity to accommodate the needs of the design team and other stakeholders. If you're not running a baseline study of your product's usability, maybe you don't need 10 participants and two weeks to write the report. The RITE+Krug method [3] combines a small number of sessions and a quick debrief, so that Agile design and development teams can get feedback within a day and then move on.
  • The regular schedule. Another strategy taken from the Agile UX and Lean UX is to schedule a small number of user-feedback sessions one day every week or every other week. For example, schedule three participants every other Friday for 30-minute sessions. Show them what you have. Get feedback. Gather requirements. Have them draw. Make it a habit.
  • Remote user feedback. There is nothing that says you have to meet your participants in person. That might be nice to have, but remote user feedback allows you to reach users in other geographies whom you don't have the budget to visit. Remote user feedback most often consists of phone interviews or Web conferences, where stakeholders can listen in or participate as needed. If you don't already have Web-conferencing software, just search for "screen sharing" and you'll find several free options.

back to top  Other Supplementary Methods

Now that we have spent some time talking about how to reduce the costs of users in user research, how can you get data and feedback by sidestepping live users all together? These techniques may take a few hours or a few days, but the time spent will be more than worth the effort you put into them:

  • Competitive analysis. Don't pay users to tell you something you can learn by looking at your competitors. If all your competitors are putting "home" in the top left of the screen and "search" in the top right, you shouldn't be asking research participants where they expect to find these items. Not only can this analysis tell you what your competition is doing well, but you will also likely find things that don't work, so you can avoid those problems in your designs. Competitive analysis can be done at any point during the product lifecycle but is most helpful in the early stages, before and during the UI design.
  • Surveys. Surveys are fantastic for getting answers to questions that the team can't answer on their own. For example: How satisfied are users with our documentation? Would users want to take this class if it required an investment in hardware? What features of our website do they value most? Which of the three icons best says "save" to our users? Surveys can be sent out to existing users, posted on a Web page, or completed at the beginning of another activity, and they can be used at any point during the product lifecycle.
  • Data mining and analysis. Log files are your friends. While they may not tell you why something is happening, they will tell you what is happening. Databases you may find particularly useful would include customer support logs, bug databases, purchase histories, and website analytics. This type of data mining is helpful after your product or website is live and has been in use for a while.

We know you want to conduct user research, because it challenges our assumptions, surprises us all the time, and gives meaning to the recommendations we make. We want you to feel empowered, not by opinion, but by data and by your users. Not in a month, when it's too late, and not for tens of thousands of dollars that you can't get funded. We want you to get that user feedback, quickly and inexpensively. Hopefully, we've given you at least a couple of new ways to do that.

back to top  References

1. Treder, M. Beyond wireframing: The real-life UX design process. Smashing Magazine (Aug. 29, 2012); http://uxdesign.smashingmagazine.com/2012/08/29/beyond-wireframing-real-life-ux-design-process/

2. Collado, J.A., Mora, P.S., and Parham, E. A guerrilla usability lab with free software. Interactions 20, 3 (2013), 62–67; http://doi.acm.org/10.1145/2451856.2451871

3. McGinn, J. and Chang, A.R. RITE+Krug: A combination of usability test methods for Agile design. Journal of Usability Studies 8, 3 (2013), 61–68; http://www.upassoc.org/upa_publications/jus/2013may/JUS_McGinn_May_2013.pdf

back to top  Authors

Jen McGinn is a senior manager of user research for Oracle, where she leads the user research team for Oracle Cloud, Java, developer tools, WebCenter, and other middleware products. Prior to Oracle, she worked at Nokia, SolidWorks, and Sun Microsystems. She has an M.S. in human factors from Bentley University. jen.mcginn@oracle.com

Chris LaRoche is a usability consultant at MIT and a senior lecturer at Northeastern University, where he teaches courses in usability, user research, and content strategy. Prior to MIT, he worked at Autodesk, Sun Microsystems, and Genuity. He has an M.A. in Irish history and an M.S. in human factors. silvaire@gmail.com

back to top 

©2014 ACM  1072-5220/14/05  $15.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2014 ACM, Inc.

Post Comment

No Comments Found