Creating Creativity: User Interfaces for Supporting Innovation
Department of Computer Science
Human-Computer Interaction Laboratory of the Institute for Advanced Computer Studies, and the Institute for Systems Research
University of Maryland
College Park, MD 20742
The following abstracts are from recent issues and the forthcoming issue of ACM’s Transactions of Computer Human Interaction (ToCHI). They are included here to alert interactions’ readers to what research is being done in the field of Computer Human Interaction. The complete papers, when published, can be found in ACM’s Digital Library at http://www.acm.org/pubs/ contents/journals/tochi/1999-6/#3 and http://www.acm.org/pubs/citations/journals/tochi/1999-6-2/p133-dourish/.
One challenge for humancomputer interaction researchers and user interface designers is to construct information technologies that support creativity. This ambitious goal can be attained by building on an adequate understanding of creative processes. This paper offers a four-phase framework for creativity, called genex, that might assist designers in providing effective tools for users:
- Collect: learn from previous works stored in libraries, the Web, and other sources
- Relate: consult with peers and mentors at early, middle, and late stages
- Create: explore, compose, evaluate possible solutions
- Donate: disseminate the results and contribute to the libraries
Information technologies that allow more people to be more creative more of the time are likely to profoundly affect every institution. Education could expand from acquiring facts, studying existing knowledge, and developing critical thinking to better emphasize creating novel artifacts, insights, or performances. Medicine’s shift from applying standard treatments to tailoring treatments to each patient reflects the trend to personalization that is already increasing in marketing and media. Expectations of teachers, lawyers, and designers are likely to rise as creativity is expected on more occasions from more people. These changes will be welcomed by some and resisted by others. The challenge to leaders and participants will be to preserve appropriate elements of existing knowledge work while shaping new technologies and then incorporating them into the workplace.
The large body of literature on creativity offers diverse perspectives. Some writersI’ll call them inspirationalistsemphasize the remarkable "Aha!" moments in which a dramatic breakthrough magically appears. Stories of Archimedes (3rd century B.C.) jumping from his bath screaming "Eureka!" as he discovered hydrostatics, or Freidrich August Kekule’s (18291869) dream-given insight about the ring structure of benzene, emphasize the intuitive aspects of creativity.
A second group of writers on creativity, the structuralists, emphasizes more orderly approaches. They stress the importance of studying previous work and using methodical techniques to explore the possible solutions exhaustively. When a promising solution is found, the innovator evaluates its strengths and weaknesses, compares it to existing solutions, and refines the promising solution to make it implementable.
A third group, the situationalists, emphasizes the social and intellectual context as a key part of the creative process. They see creativity as embedded in a community of practice with changing standards, requiring a social process for approval from scientific journal editors, museum curators, or literary prize juries.
Each of the three perspectivesinspirationalism, structuralism, and situationalismleads to useful suggestions for eight activities during the four genex phases:
- Searching and browsing digital libraries
- Consulting with peers and mentors
- Visualizing data and processes
- Thinking by free associations
- Exploring solutions (what-if tools)
- Composing artifacts and performances
- Reviewing and replaying session histories
- Disseminating results
The eight activities and their integration form a research agenda for human-computer interaction and user interface design.
The goal of supporting more creativity by more people more of the time is attractive, but a danger exists that the genex framework might be ineffective or even limit creativity. Making easy access to previous work and current workers incurs a risk that more exotic ideas will be suppressed. Similarly, using creativity supports such as simulations and composition tools may restrict imagination to only what is possible with these tools. Consultations are time consuming, and discouraging advice for novel ideas is a possible outcome. Fear that others will plagiarize compositions or steal inventions is another legitimate concern. An understanding of the dangers is important in pursuing the positive possibilities.
My expectations are largely positive, but there are many problems, costs, and dangers in anything as ambitious as a tool and framework to support creativity. An obvious concern is that many people may not want to be more creative. Many cultures encourage respect for the past and discourage disruptive innovations. Promoting widespread creativity raises expectations that may change employment patterns, educational systems, and community norms. Introducing computer supports for creativity may produce greater social inequality as it raises the costs for those who wish to participate. Finally, these tools may be used equally by those who have positive and noble goals as well as by dictators or criminals who seek to dominate, destroy, or plunder.
Between the lofty ambitions and troubling fears lies the practical path of careful research and detailed design for the eight activities described in this paper. They need development, testing, and refinement to make them successful, reveal their flaws, and pursue alternatives. At every stage, widespread participation in design reviews increases the possibility that the resulting technologies will serve human needs in constructive and positive ways.
Systems Interactions and Macrotheory
MRC Cognition and Brain Sciences Unit
University of Sheffield
University of Bath
Oxford Brookes University
Over the last 25 years several research groups have sought to design and test theories for human-computer interaction (HCI) as one means of supporting design processes. Although interface technologies and heuristic methods for behavioral evaluations have advanced, progress toward developing useful theory has been modest. For design teams, the costs of applying theory are likely to outweigh the benefits. Further investment in "deep" theory is understandably regarded with skepticism. As the 21st century begins, the HCI community could simply acknowledge the argument that the world of interaction is just too complex for its properties to be meaningfully captured in abstract theoretical models and abandon their further development.
In Systems Interactions and Macrotheory, we argue in favor of continuing research on theoretical models. Our specific proposals open by defining the concept of an "interactor." An interactor is a basic unit that interacts with other basic units in a system. The concept can be applied at different levels of system decomposition, from organizational systems to specific models of psychological and computer systems. At the heart of our theoretical agenda is the claim that the trajectory of behavior for any system can be captured in a generic modeling framework (see figure below). This framework involves specifying four sets of system attributes and using them to draw inferences about that system’s behavior. These specifications define the configuration of interactors, their capabilities, the requirements that must be met to realize those capabilities, and the regime of dynamic control and coordination that applies at that system level.
We provide concrete examples of what we mean by a behavior trajectory. We go on to show how such models can be formally specified for the "behavior" of a user’s psychological architecture and for the conjoint behavior of a system composed of a user and computer software. Our preferred route for the specification of theory, which we also illustrate, uses the kinds of advanced mathematics currently being developed in computer science to model highly concurrent systems. We then discuss how the self-same ideas could be extended to an analysis of the behavior of higher order systems such as groups and organizations. The paper concludes by examining how the framework might help express some simple parallels between interactions at different levels of systems analysis.
Figure. A four-component framework for modeling behavior trajectories for any system of interactors. A trajectory is divided into phases and segments that structure the very-short-term, short-term, and long-term (VST, ST, LT) dynamics of interaction.
The agenda envisaged is ambitious and clearly requires many fundamental technical challenges to be addressed and solved over a long period. We argue that continued investment along these lines is still worthwhile. Abandoning theory has hidden costs. Without technically sound theory, practitioners will no doubt invent their own informal, or folk, theories to help them think about the problems and issues that are important to them in their context. The practice of HCI could then become like that of psychoanalysis, with one school of thought communicating internally about a given set of issues in very different terms from those adhering to another school of thought.
Distributed Cognition: Toward a New Foundation for HumanComputer Interaction Research
James Hollan, Edwin Hutchins, and David
University of California, San Diego
We are quickly passing through the historical moment when people work in front of a single computer, dominated by a small screen and focused on tasks involving only local information. For human-computer interaction to advance we need to better understand the emerging dynamic of interaction in which the focus task is no longer confined to the desktop but reaches into a complex networked world of information and computer-mediated interactions.
We think the theory of distributed cognition has a special role to play in understanding interactions between people and technologies, because its focus has always been whole environments: what we really do in them and how we coordinate our activity in them. Distributed cognition extends the reach of what is considered cognitive beyond the individual to encompass interactions between people and with resources and materials in the environment. It can be distinguished from other approaches by its commitment to two related theoretical principles.
The first of these principles concerns the boundaries of the unit of analysis for cognition. In every area of science, the choices made about the boundaries of the unit of analysis have important implications. In traditional views of cognition the boundaries are those of individuals. Sometimes the traditionally assumed boundaries are exactly right. For other phenomena, however, these boundaries either span too much or too little. Distributed cognition looks for cognitive processes, wherever they may occur, on the basis of the functional relationships of elements that participate together in the process. A process is not cognitive simply because it happens in a brain, nor is a process noncognitive simply because it happens in the interactions among many brains. In distributed cognition, one expects to find a system that can dynamically configure itself to bring subsystems into coordination to accomplish various functions. A cognitive process is delimited by the functional relationships among the elements that participate in it, rather than by the spatial collocation of the elements.
The second principle that distinguishes distributed cognition concerns the range of mechanisms that may be assumed to participate in cognitive processes. Whereas traditional views look for cognitive events in the manipulation of symbols inside individual actors, distributed cognition looks for a broader class of cognitive events and does not expect all such events to be encompassed by the skin or skull of an individual. In distributed cognition, one expects to find a system that can dynamically configure itself to bring subsystems into coordination to accomplish various functions.
Distributed cognition provides a radical reorientation of how to think about designing and supporting human-computer interaction. As a theory it is specifically tailored to understanding interactions among people and technologies. In this paper we propose distributed cognition as a new foundation for human-computer interaction, sketch an integrated research framework, and use selections from our earlier work to suggest how this framework can provide new opportunities in the design of digital work materials.
The Effective Use and Reuse of HCI Knowledge
The paper argues that new approaches for delivering knowledge in human-computer interaction (HCI) from theory to designers will be necessary in the 21st century. The challenge is one that has been with HCI since its beginnings, namely, how to create sound, theory-based knowledge that can be delivered in a digestible form for designers. The proposed solution is to build on Carroll’s task artifact theory to develop claims as the lingua franca between researchers and designers. First, the role of theory in HCI design to date is reviewed, including the progress made in bridging models that link cognitive theory and models of interaction; however, it is argued that direct application of cognitive theory to design is limited by problems with scalability. Claims are proposed as an alternative bridging representation that may enable theories to frame appropriate recommendations for designers and, conversely, enable designers to ask appropriate questions for theoretical research. However, claims provide design advice grounded in specific scenarios and examples, which limits their generality. The challenge is twofold, first to generalize claims so they become applicable to a wider range of problems, and second, to strengthen their link to theory.
Reuse of design knowledge and well-designed artifacts will be vital to the future success of HCI. However, adopting a patterns approach in isolation will not provide the answer. The key is to develop reusable knowledge that is based in theory and unfolds levels of increasing complexity to designers following the concept of minimalist theories of instruction. An extended schema of claims-related knowledge is described that links psychologically motivated design rationalewith scenarios, artifacts, design assumptions, background theory or derivationand generalized models of applications, tasks, and interaction patterns that situate a claim’s future context of reuse.
The bridge to theory uses claims and their scenarios as rich descriptions of design problems that can be resolved by theoretical models such as ICS in one direction, whereas insights generated by theoretical studies can be recorded in a contextual setting by claims in the other direction. Methods of factoring the contribution of claims to different aspects of the user interface, such as task support, interaction, and presentation, are given, leading to techniques for generalizing claims and their usage scenarios. The theory of domain knowledge is summarized to introduce generic models that situate claims in the context of future reuse. The prospects for reuse becoming an important mode of development, and techniques for generalizing claims for reuse are discussed, including generalizing claims beyond their original context. It is argued that the convergence of claims and the domain-theory generic models provide a way forward for developing reusable libraries of interactive components, thus facilitating systematic development of HCI patterns and reusable design knowledge more generally.
The approach is illustrated by a case study that involves extracting claims from one information retrieval application, generalizing claims for future reuse in information searching tasks, and reapplying claims in the Web-based Multimedia Broker application. The paper concludes by proposing that HCI knowledge should be grounded in theory and development of reusable "designer-digestible" packets will be an important contribution in the future.
Social Translucence: An Approach to Designing Systems that Support Social Processes
Social Computing Group
IBM T.J. Watson Research Center
P.O. Box 704
Yorktown Heights, NY 10598
Wendy A. Kellogg
Social Computing Group
IBM T.J. Watson Research Center
P.O. Box 704
Yorktown Heights, NY 10598
Our goal is to design systems that support coherent and productive communication among small to medium-sized groups. We begin by asking what properties of the physical world support graceful communication between humans in face-to-face situations and argue that the visibility of other people and their activities is critical. Whether it’s wrapping up a talk when the audience starts fidgeting, or deciding to forgo grocery shopping because the parking lot is jammed, social information provides the basis for inferences, planning, and coordination of activity.
First we describe the notion of social translucence, an approach to designing digital systems that emphasizes making social information visible within the system. Socially translucent systems support three processesvisibility, awareness, and accountabilitythat enable people to draw on their social experience to structure their mutual interactions. Note that making social information visible does not mean making everything visible; we use the word "translucence" to emphasize the vital tension between visibility and privacy. For example, the validity of an election depends on some aspects of it being public and other aspects being private.
What might it mean to have social translucence in a digital system? How might making social information more visible change the way digital systems are used? To address these questions we develop the idea of knowledge communities: conversationally based systems that support the creation, management, and reuse of knowledge in a social context. Next we discuss how social translucence might be incorporated into a digital system. Although many possible approaches exist, we have been exploring the use of social proxies: minimalist visual representations of people and their activities.
We discuss our two years of experience with social proxies in the context of Babble, a chat-like communication tool that allows its users to have synchronous or asynchronous textual conversations (See Figure 1). Babble’s social proxy uses a large circle to represent the conversation and colored dots (or marbles) to represent individuals. A marble inside the circle represents a user who is in the displayed conversation; a marble outside the circle is in some other conversation. When a user "speaks" (types) or "listens" (scrolls), his marble rapidly moves toward the center of the circle; during inactive periods the marble slowly drifts out to the edge of the circle. Thus, a tight cluster of marbles indicates that a focused interaction is happening, which tends to attract other users. Over time, the movements of marbles and their locations in the conversation space come to convey a lot of information to group members.
We conclude by discussing research issues raised by a socially translucent approach to design. The discussion ranges from the design of more complex social proxies (e.g., asynchronous proxies, or proxies that represent entire conversation spaces, as in Figure 2) to the creation of conversation visualizations (See Figure 3). Ultimately, we believe that making social activity visible will allow digital systems to become environments in which new social forms can be invented, imitated, adopted, adapted, and propagatedeventually supporting the same social innovation and diversity that can be observed in physically based cultures.
Past, Present, and Future of User Interface Software Tools
Brad Myers, Scott E. Hudson, and Randy
Carnegie Mellon University
A user interface software tool helps developers design and implement the user interface. Research on past tools has greatly influenced today’s developers; virtually all current applications were built using some form of user interface tool. In this paper we consider cases of both success and failure in past user interface tools. From these cases we extract a set of themes that can serve as lessons for future work. Using these themes, we characterize past tools by what aspects of the user interface they addressed, their threshold and ceiling, what path of least resistance they offer, how predictable they are to use, and whether they addressed a target that became irrelevant. We believe the lessons of these past themes are particularly important now, because increasingly rapid technological changes are likely to significantly change user interfaces. We are at the dawn of an era in which user interfaces are about to break out of the desktop box where they have been stuck for the past 15 years. This century opens with an increasing diversity of user interfaces on an increasing diversity of computerized devices. These devices include handheld personal digital assistants, cell phones, pagers, computerized pens, computerized notepads, and various kinds of desk- and wall-sized computers, as well as devices incorporated into everyday objects (such as those mounted on refrigerators or even embedded in truck tires). The increased connectivity of computers, initially evidenced by the World Wide Web, but spreading also with technologies such as personal-area networks, will also profoundly affect the user interface to computers. Another important force will be recognition-based user interfaces, especially speech, and camera-based vision systems. Other changes we see are an increasing need for 3-D and end-user customization, programming, and scripting. All of these changes will require significant support from the underlying user interface software tools.
Supporting Cognitive Models as Users
Frank E. Ritter
Pennsylvania State University
Gordon D. Baxter
University of York
University of Derby
University of Hertfordshire
Cognitive models are computer programs that simulate human performance. They have been useful in human-computer interaction by predicting task times, assisting users, and acting as surrogate users. If cognitive models could interact with the same interfaces that users interact with, the models would be easier to develop and would be easier to apply.
The figure below shows an approach to support cognitive models as users: a cognitive model interface management system (CMIMS). It is analogous to and based on user interface management systems (UIMS). It provides mechanisms designed to allow models to interact with all interfaces generated within a UIMS.
The feasibility of the CMIMS approach is demonstrated using a number of case studies and a review of related systems. Simulated hands and eyes were used (and reused) by a variety of models interacting with application interfaces that were developed using a range of tools (e.g., Tcl/Tk. Common Lisp, SLGMS).
These interaction mechanisms offer a way to support and constrain the performance of the model in the same ways that human performance is supported and constrained by interaction. The resulting models can be used to evaluate designs, test psychological theories, and provide a way to put assistive agents into an interface. Most existing UIMSs can and should be extended to create CMIMSs, and models can and should use CMIMSs to look at larger and more complex tasks. CMIMSs will help to exploit the synergy between the disciplines of cognitive modeling and HCI by supporting cognitive models as users.
Figure. A cognitive model tied to a user interface of a task simulation, where the model and the simulation may be running in different environments (programming languages, processes, computers). The simulated hand and eye are executed in the same user interface management system as the task simulation seen by the user.
HCI in the Global Knowledge-Based Economy: Designing to Support Worker Adaptation
Kim J. Vicente
Professor, P. Eng. Director,
Cognitive Engineering Laboratory Department of Mechanical and Industrial Engineering
University of Toronto 5 King’s College Road
Toronto, Ontario M5S 3G8, Canada
Increasingly, people are being required to perform open-ended intellectual tasks that require discretionary decision making. These demands require a unique approach to the design of computer-based support tools. A review of the characteristics associated with the global knowledge-based economy strongly suggests that the need for workers, managers, and organizations to adapt to change and novelty will increase. This is equivalent to a call for designing computer tools that foster continuous learning. There are reasons to believe that the need to support adaptation and continuous learning will only increase. Thus, in the 21st century, human-computer interaction should be concerned with explicitly designing for worker adaptation. The cognitive work analysis framework is briefly described as a potential programmatic approach to this practical design challenge.
Figure. A four-component framework for
modeling behavior trajectories for any system of interactors. A
trajectory is divided into phases and segments that structure the
very-short-term, short-term, and long-term (VST, ST, LT) dynamics
Figure. A cognitive model tied to a user
interface of a task simulation, where the model and the
simulation may be running in different environments (programming
languages, processes, computers). The simulated hand and eye are
executed in the same user interface management system as the task
simulation seen by the user.
©2000 ACM 1072-5220/00/1100 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2000 ACM, Inc.