Tom Chi, Kevin Cheng
Usability as craft, art, intuition, inspiration, yes; but science? Well, no. And why would you want to lead folks down that path anyway?
Let’s debunk this notion of science in the name of a) practicality and b) preserving our right to change our (subjective) minds. Presenting usability testing as science ignores that a typical human-computer interaction is crafted from multiple perspectives: business, design, human factors, individual emotions. "Science" sounds like a rather lofty term for the quick-hit studies in vogue. With some development cycles as short as two weeks, and as few as five users per test advocated by increasing numbers of usability specialists, we find it both awkward and difficult to promote our usability services as scientific. The basis of usability testing has never been 100 percent coverage of the entire potential design space; we’re simply trying to weed out the worst from the lot.
We can think of a few reasons you might want to think of your usability efforts as science:
Usability work is convincing because it’s "scientific."
There is science and there is science. You surely should be able to convince others if your analyses are thorough and correct, and based on evidence you can support. There is truly a level of science in being able to interpret data. But choosing usability activities and selecting how to conduct them is much more craft than science. And moreover, coming up with a usable solution is design rather than science.
I’ll get more funding if I call usability "scientific."
Then you’ll need to know what others think "scientific" is, and live up to the moniker. Are you prepared to make your tests repeatable? Is that even possible? And for most software, is it desirable? Calling usability testing "scientific" in the hopes of getting more and better funding is absurd, given how most software development organizations are prepared to use usability professionals.
I’ll get more respect if I’m a usability scientist.
Nomenclature isn’t going to repair your credibility deficit. We’d like to suggest that respect comes when your product does well in the marketplace because you were able to convince the engineers to "do the right thing." They probably have a few choice names for you, anyway.
We should be holding usability testing to a high standard.
If you’re developing life-critical systems you need to have high standards and follow thempeoples’ lives depend on your system. But a lot of software can simply be tossed into the marketplace with the democratic notion that it will "stick" if it’s good, and fail if it’s not. Logic suggests that careful usability research conducted against well-considered standards is required for a complex system with serious consequences for interface failures. On the other hand, we don’t want to be the folks responsible for justifying extensive usability testing on, say, your media-player interface. There’s a big difference between being statistically relevant and being "good enough." The point of usability is to improve the product and for it to perform well in the market. Abstract numbers are just that unless they are attached meaningfully to a plan of action. If you don’t or can’t act on your usability results, who do you expect to care that you conducted great tests?
Software companies are going to follow the approach that gives them the best results.
We’ll get better product if we act more scientifically.
Software companies are going to follow the approach that gives them the best results. To our dismay, the "best" product isn’t the one with the best interface or even the one that best serves the users’ purpose. "Best" can be first to market, coolest, prettiest, fixes the worst previous problems, ships with the least effort, etc.defining "best" is infinitely variable.
If we’re not scientists, they think we’re artists…. And you know where that leads.
There’s worse things that could happen. Should we be declaring usability a science? We think calling usability "science" is not only overblown and unwarranted, it’s also dangerous: As a general rule, interface design decisions rely far more often on subjectively assessed qualitative data and anecdotal evidence than on rigorously defined and executed quantitative studies. Are we artists? Well yes, in a waybut there’s an awful lot of baggage with that term, too. We suggest that usability professionals are highly skilled craftspersons.
Don’t get us wrong, some of our best friends are scientists, and indeed some of our most important collaborators are scientists. Usability professionals are, happily or not, not among them. So let’s stop the science envy! We don’t need lab coats; we need to be able to work with the ambiguous, the provisional, the ephemeral. We need to be able to embrace the quirks of users and decide where the cut-off line is for edge cases. And most of all, we need the artistic freedom to improvise. Watertight standards and scientific values aren’t going to get us where we want to go.
©2005 ACM 1072-5220/05/0300 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2005 ACM, Inc.