User-centered and participatory approaches to computer system design, development, and deployment stress the value and importance of looking closely at the work practices of computer-systems users. This often includes participant observation and impromptu, loosely structured interviews. Sometimes it includes active participation in the implementation and deployment of new technology. It always focuses on the users, their work practices and experiences with computer technology. Regardless of the scale or timeframe of engagements, many of us believe that this kind of effort directly contributes to successful projects. However, it may not be obvious to see how the various details of specific work activities, artifacts, and settings might actually impact high-level work process designs and decisions. It is also difficult to know, in advance, what unforeseen and important system requirements observations of use will uncover. This article is a personal reflection of how looking at the details of work activity can help guide the design of work and systems on a larger scale.
The Case (circa 1994)
A document management company was working with a university library to develop a computer-based system to support reserve readings for college courses, which are required in many classes. Reserve readings provide a convenient way for teachers to create custom texts and flexible curricula. On a relatively large, mostly commuter campus, providing access to reserve readings can be difficult for a couple of reasons. First, since the majority of students are commuters who attend part-time, usually at night and on some weekends, their time on campus and at the library, is limited. Thus, a large number of readings need to be available to many students who do not have much time to spend on campus. This results in a bottleneck at the reserve reading desk, pressure on the library staff, and a significant inconvenience to the students. The university had a sophisticated technology infrastructure and was eager and willing to install the prototype reserve reading solution and to experiment with providing online readings for a small set of courses.
The development team, of which I was part, had been part of a participatory co-development project on a digital library solution at another university prior to this project and was predisposed to looking at work practice and collaborating with library staff and students to understand their experiences in using online reserve readings. Engineers and designers visited the university, gathered high-level system and infrastructure requirements, and subsequently developed and delivered the hardware and software system. Reserve readings were scanned, indexed with a small set of identifiers (author, title, etc.), and stored as TIFF image files.
The semester started and a few system problems were uncovered. Most were fixed by remote access, but one problem required an on-site correction. We also wanted to see the system in use. A small team including a graphic and interface designer, a software developer, and a manager traveled to the university for a two-day visit. We spent time observing the use of the scanner, and once again looked at the activities at the reserve reading desk. The next day we applied the required printing update, and met with library administrators to review their operation and management experiences with the delivered system, and to talk about expanding its use on campus.
The graphic designer spent time collecting video of library staff at work and students using the library. In the evening we attended one of the classes using online readings and solicited the students’ experiences using the system. During the discussion some students asked why they couldn’t cut and paste parts of the reserve readings into Microsoft Word, or another text editor. They needed to write papers based on the readings, and the ability to incorporate quotations into their papers was very useful. At first, the development team was confused by the question, since it was obvious to them that one cannot cut-and-paste TIFF images of the reserve readings into Microsoft Word. Many students, however, did not know this. They were working with MS Word documents and the reserve readings constantlyboth of which are electronic. And since the cut-and-paste function could be used for several different kinds of electronic documents, why wouldn’t it work with the reserve readings?
Two things happened at the discussion between the students and us. First, we got first-hand experience of the behavior of the system from the student’s perspective, based on their experience with the technology and the nature of their work (reading and writing papers). Second, the discussion exposed a hitherto unrecognized requirement for a particular kind of functional coherence and interoperability among digital document formats and applications that are problematic for many computer users, then and now. For instance, people still use electronic documents that do not support cut-and-paste; (e.g. Adobe PDF, JPEG, and other document and Web page formats). So even though it’s frustrating that interoperation isn’t easy, repeated experience taught us not to be surprised when things didn’t work as we predicted. In hindsight, it seems that we should have been able to know what the students’ experience would have been, but in 1994 Web technology supporting reserve readings was something new. In addition, working with users was also new.
Another change between then and now is that differences in situated knowledge, experience, and practice are beginning to be well recognized. The agile approaches to software development recognize that interactions among developers and users are crucial to the construction of high quality, useful software1. As a result of this interest in engagements with users there are many opportunities for collaboration between agile and participatory design communities in the development and deployment of computer systems.
Implications for Design
I would like to draw out four implications for system design from the case described above. First, the experience illustrates how readily unexpected requirements can be discovered through use. Cut-and-paste interoperation among document formats might have been imagined to be important, but as designers and developers of the computer system we had no first hand experience of what it was like to be a student commuting to school and using reserve readings. And though imagining use can be helpful and generative for design, empirical data on use is what helps ground implementation and deployment. The good news is that this was discovered early in the deployment.
Second, making the trip and sitting with the students reinforced for us the value of discovering and observing the user’s experience. Of course, there is no way to predict what you might learn when you ask the users directly, and for many of us this is why we like interacting with users in the first place. On the other hand, it can be a bit frightening to ask users what they think of your latest system release. New technology always begets trial-and-error in the contexts of use, and I’ve found that it’s very helpful to interact frequently with users to discover the trials, the errors, and the improvisations developed to accommodate problems in system capabilities, usability, and performance.
Third, this case shows how quickly important requirements can be uncovered, and how easy it is to learn from observing activity in the contexts and settings of use. It helps to have designers and engineers skilled in methods of observation. In this case, our ability to make the most of a short trip depended on our prior experience in participatory, collaborative, user-centered approaches to system development. But the skills of observation and participatory interaction with users can be learned and there is no better teacher than experience.
A fourth, and more speculative reflection is that grounded, detailed observations can help avoid a kind of "us-them" perspective that gets in the way of successful deployments. The end-users’ experience of "cut-and-paste" in document editors led them to expect the same behavior in an image viewer. One way to look at this problem is to say that it’s only a matter of end-user expertise and experience, and more training is the answer. It is true that users can be trained to manage the seams between applications. Humans are good at this; we’re smart, resourceful, flexible, and adaptive. And many computer applications and systems are deployed successfully because the users are adaptable and can be trained. But a better way to look at this example is to see that an expectation for similar behavior among all digitized documents is a natural one, based on the work activities of the users. Because the expectation is so natural, satisfying it is a very reasonable system requirement. Why should people who are reading and writing papers need to know anything about the digitized document format?
It’s been my experience that real gains can be made in developing useful systems and deploying them successfully by adopting the view of the end users, working to understand and appreciate their work systems, their tasks, organizations and settings, and participating with them in the development of technology and the practices that make it useful. This example illustrates that you don’t have to look far or wide to make these gains.
William L. Anderson
Rye, NY 10580
Design Column Editors
One Wayside Rd
Burlington, MA 01803
Director, Systems Laboratory
Advanced Concepts & Design
35 Waterview Dr. MS 26-21
Shelton, CT 06484
1See www.agilealliance.org for information on agile approaches to software development. www.cpsr.org/program/workplace/workplace.html is a good starting point for the participatory design perspective.
©2003 ACM 1072-5220/03/0100 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2003 ACM, Inc.