Sara Bly, Elizabeth Churchill
Every designer knows the value of studying users to determine requirements for technology development. But how can you incorporate user domain knowledge "after the fact" into early design when a technology prototype already exists? We suggest a four-step matchmaking process to move from a design centered on technology to one centered on users. Our matchmaking process involves four steps:
- Describing the capabilities of the technology,
- Mapping those capabilities to associated work activities,
- Identifying work domains and specific example sites on the basis of the work activities, and
- Characterizing the work of the example sites to verify whether they match technology's capabilities.
Involving users in all aspects of design and evaluation is a growing trend [e.g., 5, 6]. Most approaches underline the importance of identifying existing situational requirements first and then working with users to co-design new technologies iteratively to fulfill those requirements. However, these methods say little about how to identify potential user domains when a technology already exists.
We consider here a design process for situations when a class of technologies already exists, but when user domains for codevelopment are not clearly established. Our approach is design through matchmaking. This process does not follow traditional idealized design models: moving from the capture of user requirements to design specification to iterative prototyping of tools and systems. Nor does it begin with a completed artifact of technology ready for testing. Rather, the process is a matchmaking process between the capabilities of a technology and the user domains wherein those capabilities may have the most impact. Through this initial matchmaking we ensure that a basic synergy exists; the basis for further user-centered co-design of the technology and work practice.
In this paper, we offer an example of our experience in design through matchmaking. We detail how our research has moved from a focus on general properties of a technology, virtual environments, to a focus on work domains and user-centered design. Following the four-step process illustrated in Figure 1, we first describe the capabilities of the technology that interest us. Then, using those capabilities, we describe characteristics of work activity that might benefit from such technologies. We give four examples of user domains that we expect to match those characteristics. We then examine one of these domains in greater depth to validate our characterization of the work. Finally, we discuss implications of the work activity for future innovations on the underlying technology.
We are involved in a project to explore the potential for virtual environments at work. Collaborative virtual environments are computer-based places or spaces wherein people can converse with each other through text, audio, or video channels [e.g., 3]. These multiuser virtual environments are based on a computational clientserver architecture to distribute a 2- or 3-dimensional digital presentation of a shared space. Actions, artifacts, and embodiments are all encapsulated within virtual locales. Such environments provide some view of the interaction that is shared by all participants, which, at the most basic level, provides more ground for establishing a shared understanding of the current situation.
Our work is currently focused on simple 2-D graphical and textual virtual environments. We want to concentrate on tools that are lightweightthat is, easily usable and configurable by end usersbut that do not require huge computing resources. Our project goal is to explore ways in which virtual environment technologies might be designed and developed to support a range of tasks, activities, and communication in work practice. For this, it is necessary that we find a match between the capabilities of virtual environment technologies and appropriate user domains that might benefit from such technology support.
We believe that six aspects of this class of technologies are particularly important and not always present within or readily integrated with other groupware systems that are available for the workplace. The set of capabilities that we use to define these environments include
- Continuously accessiblethe environments can be always on and always available.
- Computationally lightweightan accessible window can live on the desktop with minimal computing requirements.
- Capable of synchronous communicationsimmediate exchanges can occur regardless of participants' locations.
- Capable of asynchronous communicationshistory logs, messages. and automated message objects can be saved.
- Extensibleeasily programmable objects, extensible places, and representations can support multiple places, multiple people, and multiple shared objects.
- Visually and spatially metaphoric locales for activity and for virtual artifact storage can be easily understood and navigated.
These capabilities of virtual environments offer users a chance to explore the support of collaborative work in ways beyond more traditional desktop videoconferencing and shared application tools. For example, many existing groupware tools for the workplace are best suited for small groups of collaborators (e.g., desktop video conferencing), either synchronous or asynchronous interactions (but not both), or a shared view that is not easily extensible to multiple discussions and workspaces. Such systems concentrate on activity awareness and application sharing. Collaborative virtual environments offer lightweight but constant access and support for an intermingling of synchronous and asynchronous communications. In addition, virtual environments offer representations of extensible places and people and enable the integration of shared, task-relevant objects (e.g., text documents, data files) that can be moved from one virtual workspace to another.
When virtual environments have been used within workplace settings [e.g., 4], a number of features of the technology have been found to be useful. Our research indicates that when these environments are used, people like to leave open a window to the environments on their screen . This constant window extends a user's work domain, providing a portal to another space wherein friends and colleagues may be going about their business. Having continuous access supports ongoing work activity.
Lightweight entry and attention allow communication that plays a peripheral or supporting role to the ongoing task. Participants can move among windows on their desktops without having to launch new applications or initiate connections for conversation. Frequent and short interactions are easily possible.
Further, the incorporation of objects into spaces enables users to participate in tightly coupled, object-centered, problem solving activities as well as high-level discussions and/or loosely coupled coordination activities. Synchronous lightweight conversational and object support can be ongoing as part of task work regardless of where participants are physically located.
Lightweight virtual environments are persistent not only in that they can be constantly available but also in that they can maintain a log of what has occurred over time. These multiple aspects of persistence establish a different set of protocols for interaction than the "on-demand" setup of audio- and videoconferencing. Maintaining a log means that conversations and interactions can be also asynchronous; participants can interact in real-time or later review the log and add further comments.
A central feature of all these environments is that they allow multiple people to communicate with each other in the same virtual place at the same time. These environments suggest support for multiple people who may be in multiple physical places; they are able to meet and converse in the same virtual places.
Given that representations of objects and people are encapsulatedall objects and people inhabit roomsvirtual environments provide locales for activities. The extensible environment with a visual and spatial metaphor supports multiple people in changing interactions with a task focus (and thus locales for activity separation).
Our technology capabilities suggest characteristics of a user domain. The work is ongoing and benefits from frequent interactions. Distance (and even time) may separate the participants. The communication is oriented toward task accomplishment. Multiple people may be involved in interpersonal interactions that are varied and changing. The complexity of the work may require activity separation while maintaining a joint task focus. Table 1 summarizes the move from technology capabilities to work activity.
According to our characterizations of the work activity that appears most appropriate to virtual environment support, we considered a variety of work domains and the interactions and tasks in those domains. We made no attempt to identify every domain that might fit our work activity criteria but rather to find examples of such domains. A good match can provide a user-centered perspective to our work and inform the design of our technologies. In addition, a domain suitable for informing our design must be one to which we have access and that meets our definition of "business customer." In our case, we brainstormed to identify a number of different workplace environments. We then did cursory explorations (through interviews and observations) to characterize the work at each of these environments. Four potential domains are outlined here.
- Allied: A 40- to 50-member mathematics and computer science division conducting and supporting scientific research. Allied is of interest to our project because day-to-day work involves multiple people, ongoing collaborative projects, and constantly varying interactions across project groups. In addition, the Allied group is already using a LambdaMOO1 environment to support their communications .
- Bergerac: A widely distributed sales and consultancy company offering service support at client sites. We have focused on two consultants because their work involves geographic separation with a need for frequent, lightweight interactions. The consultants use both phone and e-mail for communication.
- Fusion: A scientific laboratory for conducting magnetic fusion energy experiments involving 50 to 60 scientists and technicians. Fusion involves a large number of collaborators with a tight coupling of task and communication, many of whom are geographically separate from the main control team. Face-to-face, phone, walkie-talkies, and paging all support communication but no single way exists yet to easily reach all participants.
- Zytec: A middle-level management team in a large international software company. The colleagues need to coordinate regularly and frequently use a combination of e-mail, phone, and fax to keep in touch.
For each site, Table 2 outlines the work corresponding to the characteristics of particular concern to us.
We identified work domains that we felt might prove appropriate for codevelopment of virtual environments and work practice. However, Table 2 indicates that of the activities we found most matched our technology capabilities, the work practices and communication requirements of the Allied and Fusion domains are strong potential matches to the affordances of virtual environment technologies. By contrast, although Bergerac and Zytec are interesting domains to investigate in terms of their communication requirements, in our view the work practice and technology matches are not as compelling.
The work activity of the fusion site was selected for closer analysis for several reasons. Initial characterizations of this site suggested a good match between the user communication needs and the affordances of lightweight virtual environments. The large number of people involved in time-critical, task-focused information exchange and decision making makes communication an ongoing and critical component of the work activity. The dispersal of the team throughout many locations makes information sharing, particularly discussions and decisions, difficult to manage. Given this initial indication that we might have a good match, we took a closer look at a specific work activityfusion experiments.
Fusion is a science of laboratory nuclear reactions for energy production. Experimentation takes place only at a few large central facilities worldwide. Although the preparation for a single experiment often takes months, we currently are concentrating only on the on-site activities during the actual day of an experimental run. This work demands real-time synchronization and exchange of data among multiple computer networks in experiments that involve as many as 40 to 50 scientists and technicians at a time .
A typical day for an experimental run has a general structure and organization. Following update and goal-setting face-to-face meetings, the day proceeds through a series of experimental "shots," each of which involves a number of steps. These are the setup of particular parameters, a check that all systems are ready, creation of a plasma, the actual firing of the neutral beams and data collection, and the subsequent initial data analysis for experiment success and failure diagnosis.
The 40 to 50 people involved in an experimental run day include scientists, engineers, operators, and technicians. The lead scientific team is responsible for the overall experiment, monitoring events and making decisions from one experimental run to the next to modify parameters or tune machinery to achieve their goals. The operators are responsible for carrying out the experiment, setting up the experimental parameters, and ensuring the synchronization of the systems. The diagnostics teams are responsible for collecting data, monitoring the sensoring equipment, ensuring data credibility, and measuring system outputs. Diagnostics teams may have experiments of their own that piggyback onto the primary experiment. The technicians are responsible for monitoring and fixing hardware. As one session leader remarked: So it's kind of like a NASA mission where everybody has their specialty and make sure that that part's working and then somebodyin this case meoversees putting all these together and making sure that everything is going to work together.
A large number of the people are in the main control room shown in part in Figure 2. The size and layout of this room dictate that face-to-face conversations and data sharing often require moving across the room to reach someone. In addition, many of the operators and technicians may be elsewhere in the building, controlling and monitoring specific aspects of the equipment. Other scientists, especially those who are running diagnostics or interested in the overall experiment progress, may be in different buildings on site or at laboratories far from the experimental facility. Each of the individuals involved in the experiment has immediate access to the data generated by a shot. Unlike the experimental shot data, however, communication does not lend itself to instant distribution throughout the network.
Broadcast messages, walkie-talkie and phone communications, line-of-sight interactions, and nearby colleague face-to-face conversations are all used to keep team members informed of the experiment status and progress as well as to discuss data analysis, equipment faults, and decisions. A few remote scientists recently began communicating with participants in the control room through an IRC chat session. Cameras and monitors are beginning to be used to broadcast the pre-ops meeting over the Internet and to give remote participants a view of the ongoing control room activity.
Despite all these various methods of communication, it is often difficult to know exactly what is happening, because decision making and information gathering are not available publicly. For example, it is currently not possible for a group of operators located in one part of the building to know that a shot is being delayed or why it is delayed, unless the problems are with the technologies for which they are responsible. Similarly, for scientists running diagnostics in another location altogether, there is no regular flow of information about what is happening, and sometimes there is not even a clear path for reporting back information obtained from the particular diagnostic. Although the constantly changing flow of information is the underpinning of the experimental process and structure of the day, much of the actual communication depends on individual channels.
Fusion experiments seem well matched to our virtual environment technology.
If we are truly to make the transition from a technology-centered to a user-centered project, we must be certain to step back from our technology and allow an understanding of users and their work practice to drive our technology design. The need for technologies to support multiple people, varied and changing interactions, ongoing conversations focused on task, and the separation of participants suggests that fusion experiments seem well matched to our virtual environment technology. Having identified a possible match, we must now examine more closely the work practice and the implications of that practice for the design of virtual environment technologies.
The fusion experiments are characterized by the complexity of the experimental apparatus and by the corresponding complexity of the experimental parameters. Although the experiment has been carefully designed and planned well in advance, these complexities mean that the physics may have to be tuned for each shot. Such decisions may depend on input from the main hardware components, from up to 30 instruments gathering data, and from any number of mini-experiments being run simultaneously. These decisions require accurate input, involve intense negotiations, and are time-critical.
Knowing what is happening and having timely input to and from any number of colleagues are important. One scientist points out the value of having colleagues immediately available:
- Well, sometimes it's good to have immediate access; you just swivel your neck around and find who you want to talk to.
They know that communication is not often a simple matter of changing experimental parameters but rather of joint problem solving:
- There are ways you have to interact with the beam operators... it's more than setting timing and if you lose 3 minutes getting that communication right, whether it's through CU-SeeMe or a telephone call or whatever it is, and then you do that 4 times the day, you've lost 1 or 2 shots and there's a real cost for that....
All the discussion around an event provides meaningful context for the results.
Setting up the parameters for the next shot is only one reason to be as close to the center of the communication as possible. All the discussion around an event provides meaningful context for the results:
- ...there's a lot of communication about how the shot went, how the run's going, that is not just the words someone types in a log page, or even sometimes just the words they're using, so I recognized that it's much better to be physically present in the control room if you can be.
- ...there's really no way to beat being physically in the control room for the whole run to really know what you have and what you don't have when the run's over. And it's nearly impossible if you're just trying to mine slog and session leader summary files and operator summary files after the fact when you weren't even present. Then it's nearly impossible to figure out.
Yet, despite the desire to have shared and instantaneous communication, it's clearly impossible for everyone to be near everyone else, and there are communication breakdowns.
- ...the communication in our own control room to some of the outlying people is not very good...I've been in that situation where they really don't know what's going on and aren't happy about it.
- If we were needed or if we wanted to do something, we either have to get on the phone or radio or on our little squawk box to communicate with someone.
We believe virtual environments can address some of these issues. A virtual environment can offer a central place for information exchange as well as a means of both separating and sharing various conversations and participants. A known locus of information and discussion can help with the general communication needs for remote participants and also for participants who are in the same room.
Within a virtual environment, conversations and information can be accessible to everyone, accessible only to those in a specific room, or private, as required. In addition, the logs provide a potentially rich source of ambient or background information that can be searched for clarifications and for activity awareness even when remote or otherwise engrossed. Because most of the data representations are already shown on computer windows, another workstation application is most likely not a significant change to the working practices of the group.
Nevertheless, a virtual environment will clearly not solve all the existing communication problems. It would not be possible or even desirable to create a communication situation that allows everyone to know what everyone else is doing, what information is needed where and when, and with whom to talk. The particulars of what can and cannot be accomplished will be addressed as a central part of the co-development phase, which should follow our initial matchmaking.
Having established a basic match, we are now prepared to proceed with more traditional user-centered design. We intend to observe and understand the work practice of the fusion experiments in order to inform the design and development of systems designed to support the particular communication and information sharing needs of this domain. Our observations and analysis will undoubtedly lead us to changes in technology and work practice. The process will be iterative. Over time, both our specific technology implementation and the work practices we affect will be in some way tailored to the match.
We have proposed a design matchmaking process that charts the compatibility of genres of technologies and work domains. The approach does not focus exclusively on the development of the technology or the existent work practices of a community, but rather draws on both. Through critical analyses of the technological affordances and of selected work domains, we evaluate potential matches. When the match appears positive, we iteratively matchmake between the technology and the work practices of the group.
Although we believe that a user-centered approach is preferred for technology design, it is nevertheless often the case that new and innovative technologies exist without being grounded in any particular user work activity. We have offered an example of such a technology focus and virtual environments for the workplace, and have described our process in finding a match with a user domain.
We established a four-step process for this critical analysis: describing the capabilities of the technology, mapping those capabilities to associated work activities, identifying work domains and specific example sites, and characterizing the work of those example sites with a perspective based on understanding the technology's capabilities. This process leads to an evaluation of the match and represents a precursor to specific work practice and technology codevelopment.
We particularly thank Tom Caspar and Bill Meyer at Lawrence Livermore National Lab, as well as the experiment teams at General Atomics, for all their help in building an initial understanding of the fusion experiment process. Much of the fusion experiment description was made possible by U.S. Department of Energy Contract DE-AC03-76SF00098. We thank FXPAL for support of our project work and design through matchmaking. Many thanks to Joe Sullivan, Scott Minneman, and Cathy Marshall for input on this article and on the matchmaking process.
1. Caspar, T.A., Meyer, W.M., Moller, J.M., Henline, P., Kieth, K., McHarg, B., Davis, S., and Greenwood, D. "Collaboratory Operations in Magnetic Fusion Scientific Research." interactions 5, 3 (1998), ACM Press.
Sara Bly Consulting
24511 NW Moreland Road
Hillsboro,OR 97124, USA
Elizabeth F. Churchill
FX Palo Alto Laboratory Inc.
3400 Hillview Avenue, Bldg 4
Palo Alto, CA 94304, USA
©1999 ACM 1072-5220/99/0300 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 1999 ACM, Inc.