Laura Dove, Arathi Sethumadhavan
An unfortunate polarization of research and design is becoming the norm in the practice of user experience (UX). Not too long ago the "us and them" mentality was reserved for interactions among usability, sales, marketing, and software-development colleagues. Somehow this schismatic attitude has also started to appear between UX researchers and designers. This forum is a plea to both UX researchers and UX designers to build and maintain a more collaborative approach for the benefit of our customers and users.
We decided to write this article while we were working at the same medical device company, to share our experience of benefiting from collaborationand of providing more value to our project as a result. Specifically, the stories shared here are about the design process benefiting from the mutual input of seasoned researchers and thoughtful designers.
In our experience, most UX professionals have stronger skills in either research or design. We come to the field from two somewhat different backgrounds: Laura entered the UX domain 15 years ago with a degree in studio art, and Arathi has a background in experimental psychology. Although we bring different talents, we both appreciate learning from each other and passing the benefits of this collaboration along to our project teams.
Recently, we worked on a couple of larger-scale project teams that required UX researchers and UX designers to work together to execute safe and innovative solutions. Comparing notes about our experiences, we realized good results happen when people collaborate effectively as a result of positive team dynamics and open communication. Poor collaboration, on the other hand, can take many shapes and have less predictable results. In some situations we were surprised to find our UX counterparts (researchers and designers) struggling to work with each other. In the end, we were able to resolve many of these collaboration struggles. However, we have heard of similar difficulties from other UX colleagues working on large-scale projects at other companies.
To get some perspective on what's currently happening in our field, we started thinking about how much has changed over the past 10 years. Many readers might recall the climate in the late 1990s, when businesses often seemed to react to UX professionals like an overactive human immune system. They tended to see UX professionals as outsiders who were trying to intrude into the product-development process. More recently, we have enjoyed greater acceptance; our field has grown as UX has become recognized as a business-success differentiator.
This has been fortunate for UX researchers and UX designers, and there has been an influx of new people into the field. But of course there have been trade-offs. Business leadership does not always understand the best way to deploy the specialized and varied UX skills needed for an organization's diverse needs. This applies to the mix of research and design skills required for a given business, product, and/or service. To add to this issue, some practitioners responding to new job/role opportunities may have unintentionally overestimated their capabilities and oversold their skills, instead of aligning themselves with an appropriate role and managing organizational leadership's expectations.
We don't think it is common for UX practitioners to have equally strong skills in both research and design. Some businesses have seen the variation in these skills and responded by creating separate jobs for researchers and designers. However, separating UX into these two separate research and design roles can also have a negative side, namely, the unintended consequence of pitting research and design against each other and encouraging disjointed product development. Based on our experience and the anecdotes we have heard, we believe this conflict is fueled by some reciprocal stereotypes:
- Designers express a desire to innovate and some perceive researchers as too slow, not forward-thinking, not accounting for the "wow" factor, and as using evaluation methods that are inherently biased against innovation.
- Researchers want to apply rigorous evidence and knowledge of underlying principles of human-machine or human-computer interaction to design decisions. Some perceive designers as ungrounded, and as making important decisions based on biased intuition and/or weak analysis.
As with many such characterizations, there is some myth and some truth in these sentiments. But in the end, these stereotypes can obscure the fact that we have shared goals. We are all in this field to make the world more user friendly. To the extent there is truth in these stereotypes, we need to respond not by building walls, but by finding ways to bridge the gaps. Both sides share the responsibility for:
- recognizing when we have an opportunity to design new interaction paradigms and envision what they might look like; and
- honoring the evidence to help evaluate the opportunities and create viable solutions.
Here are two examples of collaboration between researchers and designers on large projects that resulted in a more effective end product and would not have had the same benefits for users had one of the disciplines been excluded from the development process.
Example 1. In 2008, a new project undertook development of a next-generation medical device programmer, a computer system that communicates with patients' implanted devices, allowing clinicians to manage these devices and improve the status of the patients' cardiac function. Before UX practitioners were assigned to the project, upstream marketing had already defined an early design concept based on some ethnographic research.
The UX team subsequently assigned to this project consisted of four people with primarily creative backgrounds, including Laura. While the group was very enthusiastic about digging into the design work, it was also concerned that there would be only one trip to visit the proposed end-user geographies, limiting the opportunity to iterate and retest designs. The proposed geographies included India and China, which were unfamiliar environments to the business and the UX team. Given the challenging nature of the project, this creative-minded team struggled initially to define the study goals and protocol to make the most of the limited research opportunity.
Good progress began after a few consultation meetings with a colleague who had a strong experimental background. The research guru was able to quickly frame the project issues and focus the team's attention on key methods that, in the end, provided evidence to help evaluate trade-offs among different concept options. The important point was to not jump into design without answering any unresolved questions from the marketing-research activities. The researcher encouraged the team to do a systematic gap analysis on the existing data, which identified key user-profile and workflow unknowns. Despite hundreds of pages of site-visit data and reports from the upstream marketing research, the team required additional ethnographic data in these UX areas. Once these unknowns were identified, the team was then able to address these questions in their protocol by planning the site visits and selecting appropriate data-gathering methods.
The research expert's input was critical to the success of this project in a number of ways:
- Input from the research expert influenced the UX team to carry out contextual inquiry on specific areas of the users' workflow in new geographies and unfamiliar hospital types. Pointing the team in this direction allowed them to see some significant differences that were not captured in the earlier ethnographic work because it had overlooked these types of institutions. The additional research had significant implications for the final product. For example, the team learned about a general lack of structure in these environments that resulted in users having to manage many distractions as they conducted sessions with their patients. This inspired many aspects of the product design, from the industrial design to alerting systems.
- Business stakeholders had a vested interest in the concept that was initially proposed by marketing. The research expert encouraged the UX team to design alternative concepts for evaluation and helped them build a protocol to understand which of their potential solutions would not work and why. As it turned out, although it was a great place to start the project, the initial concept proved to be a poor fit when compared with alternative concepts that supported distracted users. Even though these results were not what the business stakeholders expected to hear, they appreciated all the innovation, which was inspired to a great extent by the research expert who helped frame research questions early and influenced the team to test their initial assumptions.
In short, engaging a research guru early in the project helped the creative types not only in the planning of the research effort, but also throughout the concept-development phase. It also helped produce alignment across the lifecycle of the project. In this case, designers were able to achieve much more on a limited budget and short timeline thanks to the strong collaboration with research experts.
Meanwhile, the design experts' input was also critical to the success of this project:
- Designers felt it was important to be sensitive to cultural differences in discussing trade-offs for different potential solution ideas. After consultation from the research expert on the gap-analysis exercise, one of the designers on the team was inspired to model a concept-comparison exercise on a car-shopping experience, because this method allowed users to react to concepts without being negative. Themes were selected for each concept based on the gap analysis. This ended up being a very efficient and effective way to capture detailed user needs.
- Once the entire extended team aligned around one concept based on user research results and business needs, designers were able to produce a slick new look and feel for the new applications and industrial design without inadvertently compromising usability. This more polished version of the design helped to engage project-management stakeholders who had previously had difficulty envisioning the value in the earlier concept's wireframe-style mock-ups.
Example 2. In the design of a next-generation cardiac catheter (a thin tube inserted into the heart from an incision in the groin), Arathi collaborated with an industrial designer to produce the final design. In this project, the critical piece of work that had the greatest influence was her selection of appropriate use-error analysis methods. Her method selection was tailored to the specific UX challenges she was encountering on this project.
Separating UX into two separate research and design roles can have a negative side, namely, the unintended consequence of pitting research and design against each other and encouraging disjointed product development.
First, a use-error analysis of catheter usage during the implant procedure was conducted. Arathi chose to employ a failure mode and effect analysis. This approach started with the task analysis of the implant procedure and involved identifying failure modes (e.g., What happens if a user actuates a control in the handle incorrectly? What happens if a user mistakes a control that is used to perform one function for another?). For each failure mode, potential user errors and the consequence of user errors were identified. Probabilities of the occurrence of each failure mode and the associated user errors and hazards were then computed. These probabilities were then used to compute the overall hazard estimate for each failure mode. The hazard estimate helped to systematically determine the safety consequences associated with various design concepts.
She also carried out observations, one-on-one interviews, and contextual inquiry with physicians who had experience performing myriad electrophysiology and interventional cardiology procedures. She ensured that the sample included representative users, including both interventional cardiologists and electrophysiologists, as well as representatives from across various geographies. This approach made the findings more robust.
This thorough research revealed important usability considerations for the design of a safe and effective product, which allowed Arathi to influence the direction of design concepts. For example, observations of the usage of the catheter in the implant environment and contextual inquiry revealed that physicians spent most of the time fixated on the fluoroscopic imagery to guide the catheter and rarely looked down at the handle. Thus, merely differentiating the controls in the handle visually (e.g., by color) would be insufficient and had the potential to create confusion and user errors. Instead, the controls needed to be distinguishable by touch, using redundant tactile cues: size, shape, texture, and location. The research also showed that the left hand of the physician needed to always stay at the point of entry of the catheter through the patient's skin. Thus, it identified the need to enable single-handed operation and minimal hand movement. These findings helped focus the designer on envisioning alternative solutions to these key user requirements, which were then prototyped and evaluated using quantitative and qualitative data from iterative usability testing.
Since designing for safety is a priority for a cardiac catheter tool, having a rigorous research approach was key to the success of this product. Creative industrial design, too, was a necessity. Arathi worked closely with a designer throughout the project to communicate important considerations from her findings, and the designer was able to translate these usability requirements into concepts for controls. This designer knew what type of controls would work best during a push versus a pull operation. Plus, he was able to provide workable options for angling, size, shape, and texture, which helped differentiate between the handle controls.
The challenges we will face as the UX discipline continues to evolve are not entirely unique. In a 1959 essay, "The Two Cultures," English novelist/scientist C.P. Snow pointed out that our society was increasingly adopting a reductionist approach to solving problems. Snow was concerned about a trend toward specialization in western societiesespecially as it negatively impacts interdependencies between creative and scientific work. Snow writes: "[T]hose in the two cultures [creative and scientific] can't talk to each other... when those two senses [creative and scientific] have grown apart, no society is going to be able to think with wisdom... this polarization is loss to us all, to us as people and to our society."
Ideal teams have a mix of researchers and designers who freely leverage each other's skill sets to achieve innovative, usable solutions to complex problems. Everyone benefits when UX research is planned and executed so that it produces results relevant to our design colleagues. In the same vein, UX design can be more powerful when it is open to the impact of good research. Practitioners should know their strengths and limitations, be able to communicate these, and be open to engaging other necessary resources. Understanding and valuing what others add to the mix will help us bridge any gaps between research and design. Integrating our skills and contributions will help us produce more effective, efficient, safe, and emotionally satisfying products for our users. And isn't that what user experience engineering is all about?
The authors would like to thank Nelson Soken, Beth Bullemer, and Steve Nelson for their assistance with these projects and for their outstanding research and design talents. Special thanks to Chaya Garg and Paul Blowers for their comments on an earlier version of this manuscript.
Laura Dove has a master's degree in HCI from DePaul University. For more than 10 years, she has applied design, user experience, and human-factors processes and principles in academic, industrial, and medical domains. She has served as secretary in the MilwauCHI local chapter.
Arathi Sethumadhavan has a Ph.D. in experimental psychology from Texas Tech University. Her research has applied human factors principles to domains including air-traffic control, the military, and, most recently, medicine. She currently serves as the department editor of the Research Digest for Ergonomics in Design.
©2012 ACM 1072-5220/12/11 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2012 ACM, Inc.