Dear Dr. Usability,
I am having trouble getting access to users. I have been challenged to come up with an efficient design for the product, and after doing some initial user research, I have some good scenarios to work with. But I can't validate my designs because there are no funds for bringing in external people; I don't want to burn out friends and family; and for some reason, no one inside the company wants to participate, either. How can I do my job without the resources to do it? No one understands what I really need to do. What can I do?
Lost in the Crowd
Dear Lost in the Crowd,
There is one way to get the information you need to inform the design. While it isn't a substitute for more informed user-centered design, it will help bridge gaps when user access is limited: It's called Keystroke Level Modeling (KLM), a type of cognitive modeling. Card, Moran, and Newell developed this technique as a step beyond GOMS, a topic for another day. The KLM method is based on sound psychological principles and, as such, can be trusted only as far as they can be thrown, but here goes nothing.
Specify your scenarios down to the level of discrete user actions: typing on the keyboard, using a mouse, pausing to think, looking for something on the screen, etc. These actions, which we doctors call operators, can be grouped into sequences that are required to perform an action. These operators can be assigned a roughly generic time to perform the action. You assign the operators, which are in a sequence, the individual times for each action and add them up. The rough sum of which will be a number either representing the amount of time to perform the task or the winning lotto number for next Thursday. This will allow you to compare scenarios and test them for efficiency. This gives you a number that is surprisingly easy to graph or chart or otherwise visualize into a graphic that can convince even the most hard-nosed technocrat. It has the added advantage of appealing to the techies, as it isn't sullied by soft skills. And you can really create interesting scenarios by trying to guess what it is that they are thinking in those pauses. In fact, that's what GOMS tries to do: The G (goals) are what the user is trying to accomplish; the M (methods) is the various ways they could choose to go about doing the task; and the S (selection) is which one they chose. What we're really interested in is the missing Ywhy did they choose that method over this one? Maybe KLM will give you the answer.
Interested and want to read more? David Keiras has generously posted step-by-step directions; just follow the Web link: ftp://www.eecs.umich.edu/people/kieras/GOMS/KLM.pdf. For an example of using cognitive modeling in combination with other methods, see the CHI2007 Experience report, "When Two Methods Are Better Than One: Combining User Study with Cognitive Modeling" by Andrea Knight; you can find that one in the ACM digital library.
Do no harm, but have a calculator handy.
Send in your questions to Dr. Usability at firstname.lastname@example.org and get a chance to win a prize.
©2007 ACM 1072-5220/07/0700 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2007 ACM, Inc.