It started with an innocent query to the IxDA listserv . Someone was sure they had read an article in interactions magazine once but could not find it again: "Wasn't there something written sometime by someone about something like sample size in usability research?" asked an expectant interaction designer. Woe is the hapless interaction designer who is unprepared for the firestorm that follows the dreaded "sample size" question. Some 106 replies later, not only was the question clearly left unanswered, but worse, it also left many scratching their heads in genuine confusion. What is the essence of interaction design research? Is it data-driven and "scientific"? Is it exploratory and qualitative? No consensus was reached. Again.
This schizophrenia is both a blessing and a curse. On the one hand, an interaction designer has the freedom to assemble their research program like an artist assembling an installation: Whatever inspires them can indeed find a place in the final result. Yet such a lack of standards leads to a distinct lack of consistency and expertise. If interaction design research is whatever you want it to be, what is to stop other occupations from "colonizing" what ought to be the purview of interaction research? See, for example, Dan Formosa's article in the November + December 2009 issue of interactions, lamenting the intrusion of market research into the design field. When there are no standards, there is freedom. As Sartre said, we are "condemned to be free," meaning, when there are no predefined codes of conduct, then we must tragically, wonderfully, horribly create ourselves. The confusion over the essence of interaction design research is us, thrashing about as we desperately create ourselves.
This lack of standardization affects the practice of interaction design research. In particular, the dreaded sample-size debate is actually indicative of a larger issue of theoretical training. Interaction designers should embrace standardizationnot blindly, but with eyes wide openfor the benefit of interaction design research and for the profession itself.
Most people stumble into interaction design. Unlike a profession such as medicine, interaction design has a distinctively ill-defined apprenticeship. The proliferation of interaction-design job titles demonstrates this vagueness. A lack of standardization is liberating for many but has the unintended consequence of undermining the interaction designer's autonomy. To become an accountant, professor, or engineer, individuals must meet compulsory standards, pass examinations, and prove their mastery of the profession's "canon" of knowledge in order to practice it. Jobs that involve a canon are typically called a "profession" instead of a mere "occupation."
Indeed, a profession is not simply a job that requires skill. A profession differs from an occupation in that its members exercise exclusive control over a specific body of knowledge [2, 3, 4]. A profession must therefore have a clearly defined certification process, which in turn allows its members to exercise a sort of monopoly over the work itself. If a doctor is fired from a hospital, he or she continues to be a doctor. No hospital administrator can remove the ability to write prescriptions, for example; only a doctor's peers can remove or grant this ability. Those peers have decided the minimum acceptable standards to write prescriptions and practice medicine has been met; the hospital administrator's opinion is irrelevant. The power of the professional, then, is inextricably bound up with knowledge and training.
The "community of practice" is no substitute for a profession. It is merely the poor man's version of a profession, referring to the informal knowledge-sharing sessions of Xerox technicians, who bully each other instead of fighting for higher wages or more autonomy . As with copier repair, there is no body of knowledge that is collectively recognized as composing "interaction design," much less "interaction design research." In its 2006 survey, the Information Architecture Institute found that 48 percent of self-identified information architects had no formal training, and almost 3 percent of those surveyed "weren't sure" (!) if they had formal education . It is for this reason that there is much confusion about what interaction design research should really look like. No accountant questions how to gather data for creating a cash-flow statement. Certainly, there may be debate about the "right" method, and perhaps there are several schools of thought to which individual accountants tend to subscribe. But in the end, there is no debate that a cash flow statement has X, Y, and Z, and if it has A, B, and C, then it is not a cash flow statement, but a balance sheet.
Interaction designers have no such luxury. What exactly constitutes an "interaction"? Where does interaction design end and aesthetic design begin? These questions may seem overly theoretical to some, and indeed they are, but it is this very line of questioning that defines the professionalization process. What constitutes a dentist over a dental hygienist? Dentists and dental professors themselves defined that difference for their own benefit . Practitioners of a discipline must delineate the theoretical confines of a discipline (and the requisite knowledge that must be mastered to claim expertise in that discipline) in order to claim occupational autonomy. Simply forming a community of practice and gathering for shop talk is not sufficient. Xerox technicians have not successfully created a monopoly of knowledge over photocopiers; neither have they created a strong lobby for occupational control.
This is not to say that interaction design is completely bereft of an intellectual tradition. The Information Architecture Institute's industry survey did find that of the information architects that were formally trained, 40 percent of them had training in library science and another 12 percent in HCI. This suggests there is, at least, a significant number of practitioners (well, those identifying as "information architects") with similar training. The HCI and library science disciplines inculcate their students with a distinctively quantitative approach to research. The November 2009 annual meeting for the American Society for Information Science and Technology (ASIS&T) included a full-day pre-conference workshop, "infometrics" and "scientometrics," that trained participants on a multitude of quantitative methods. ASIS&T also maintains several "special interest groups" or SIGs that are specifically targeted around metrics, measurement, and quantitative methods. Not one SIG specializes in design or qualitative methods. The Computer Human Interaction (CHI) SIG in ASIS&T professes interest in "online users and their behavior," and not the symbolic, interpretive, or otherwise cultural aspects of the online experience.
Underneath this focus on metrics and behavior is a set of implicit assumptions within the HCI/information science tradition. This assumption cuts to the heart of the sample-size debate: What is the nature of the world and what is the best way to research it? Most researchers subscribe, at least in part, to two established schools of methodological thoughtquantitative and qualitative. While they may never be purely quantitative or qualitative in their research approaches, researchers tend to subscribe to the overall tenets of their school. The archetypical or "ideal type" quantitative researcher may not actually exist, but describing a methodological approach elucidates unspoken assumptions that many researchers may have.
Practitioners of a discipline must delineate the theoretical confines of a discipline (and the requisite knowledge that must be mastered to claim expertise in that discipline) in order to claim occupational autonomy.
The archetypical quantitative researcher first starts with the assumption that the world is a "real" place that exists independently of human beings . In other words, quantitative research has an objectivist ontology, one that assumes reality is an objective thing that can be researched. Accordingly, the ideal-type quantitative researcher also assumes that the scientific method is the best way to discover this reality, and that a researcher does not affect or shape the outcomes of the research if appropriate steps to avoid bias are taken. On the whole, this approach means looking for the most "typical" occurrence, one that has a necessarily statistical description .
By contrast, the archetypical qualitative researcher assumes the world is not an objective reality, but something that is constructed by us humans every moment of every day . Such a researcher considers how humans make sense of the world as having primary importance, so our methods are typically aimed at uncovering or "unriddling" this sense-making process . Numerical representations of the "typical" occurrence are irrelevant in this view because there is no typical occurrence.
One can see how scientific approaches to interaction design research evolved, therefore, from the objectivist, positivist research paradigm. In this paradigm, it makes sense to count and to find the average. And of course in order to do so, one must count sufficient numbers to make it statistically valid. But if one adopts the assumption that there is no such thing as "typical"the mental model is a conceptual representation, not a statistical representationthen it is a logical choice to reject "sample size" as important. The process of sense making is more important to the constructivist, interpretivist researcher.
It is unclear how many self-identified interaction designers would reject, wholesale, the title or description of information architect. Herein lies the problem. To reject information architecture in favor of interaction design is actually to reject the positivist tradition of information science in favor of design. This is a significant turn.
Design spans both art and science, making its ontological and epistemological position unclear. Design requires both the "logical character of the scientific approach and the intuitive and artistic dimensions of the creative effort." It spans both deductive and adductive logic; it is the "process of creation and decision making" . Interaction designers draw on both the science of decision making but also the art of creativity.
Is it any wonder then, where our collective schizophrenia comes from? We are fraught with existential angst by the very label of the occupation. We are not entirely sure if we are information scientists or if we are artists. We create our own professional identities as a bricolage, choosing pieces that suit us and rejecting those that don't. If there are no standards, there is freedom.
Interaction designers may feel blessed to draw on the scientific tradition for one research project, and on the interpretivist tradition for another. This may feel liberating. But it has its cost.
Professionals command higher pay, status, and autonomy precisely because they have agreed to subscribe to a canon of collected knowledge. They accept that they must prove their familiarity with, say, contracts law even though they do not plan to use it and could easily do without it, thank you very much. Professionals do endure such irrelevant learning because they recognize the benefits of having their occupation controlled, even somewhat, by their peers. They enjoy greater freedom at work . They have higher salaries . They can even withstand the slings and arrows of globalization and maintain their professional autonomy . Professionals know that by sacrificing a little, they get a lot.
So a call to action. Interaction designers, now is the time to define the theoretical boundaries of your knowledge. What exactly constitutes an interaction and how exactly might one design it? What is the difference between an interaction designer and an information architect? What, by extension, constitutes interaction design research? And finally, for once and for all, does an interaction designer need to care about sample size?
These questions must be answered. And we must answer them. I'm not suggesting that interaction designers drop everything and begin furiously debating in the pages of academic journals. Rather, I suggest that design educators begin instilling clear and defined canons of knowledge in their students, that practitioners begin adopting (gasp!) standards when hiring, and that, collectively, we pursue a consensus.
I present two illustrative examples of professionalization: engineering and medicine. Engineers structured their occupation and thereby collect some benefit, but physicians gained exclusive rights over key aspects of their practice, making their professionalization process much more successful. David Noble traces the professionalization of the engineer in his fascinating history America By Design . It was businessmen, not university-based researchers, who led the drive to professionalized engineering, resulting in engineers becoming "company men" instead of independent practitioners. Engineers successfully control entry into the profession but oftentimes rely on engineering employers for a professional identity. By contrast, physicians professionalized their occupation as a group of independent practitioners. Indeed, it is physicians that all other professions look to emulate . While there have been many recent changes that limit physician autonomy , physicians continue to maintain a near monopoly over the legal ability to prescribe drugs (in the United States, nurse practitioners can prescribe some drugs).
The lesson from these two professions is first to ensure that practitioners, not companies, drive professionalization. The Interaction Design Association and the Information Architecture Institute are great starts in this direction. But second, interaction designers must gain exclusive control over a certain body of knowledge. For example, interaction designers may seek to "own" accessibility-compliant website design. Interaction designers may end up with several schools of thoughtwhich is perfectly acceptable (there are, after all, Jungian psychiatrists as well as Freudians)but at the very least we will never waste another single pixel on the dreaded sample-size question!
1. http://www.ixda.org/discuss.php?post=46278 IXDA Discussion Forum/
2. Friedman, M. "Autonomy, Social Disruption and Women." In Relational Autonomy: Feminist Perspectives on Autonomy, Agency, and the Social Self, eds. MacKenzie, C. and Stoljar, N. Oxford: Oxford University Press, 2000.
Sam Ladner is a sociologist with an interest in the design of technology and its effect on organizations. She mixes private-sector consulting work with academic research and teaching. Using a range of methods including interviewing, observation, and ethnography, she consults on digital product design, organizational change, and the social aspects of technological innovation. Ladner holds a Ph.D. in sociology from York University. Ladner currently works for her own firm as consultant and principal with Copernicus Consulting Group and frequently partners with design firms.
©2010 ACM 1072-5220/10/0300 $10.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2010 ACM, Inc.