Let's begin our journey in the 21st century by thinking about metaphors in user interfaces. I have defined them as the essential concepts in computer-mediated communication that substitute for the underlying code and terminology of operating systems, applications, and data. Instead, concepts are communicated through words, images, sounds, and even tactile form . An example familiar to most computer users is the desktop metaphor that substitutes a screen depicting something like a desktop covered with documents and folders for the underlying realities of data, functions, and how users manipulate them. In the disciplines of semiotics and rhetoric, this communication technique of metaphor is an important figure of spoken and visual communication. Although others (see, for example, ) and I consider metaphor to be a fundamental component of all user interfaces, not all professionals in the user-interface design community agree.
In the late 1980s, during informal conversation with Jaron Lanier, one of the prime innovators of virtual reality's new paradigms, he made a characteristically mysterious, oracular, and challenging pronouncement. He said that he considered most current user interfaces inadequate and envisioned a future time in which there would be virtual reality displays (e.g., advanced versions of headsets he helped invent) working with input devices (e.g., advance versions of DataGloves he helped invent) that would not require metaphors. He envisioned something like a musical instrument, such as a piano's keyboard on a device, which, when "played," "directly conveyed input" to displays that we, in turn, could directly experience.
I challenged Jaron about the notion of a user interface, a medium of communication, existing without metaphors, because I have been influenced by the likes of George Lakoff and Mark Johnson  to understand that all communication requires agreement on metaphorical underpinnings; otherwise, people talk in a skewed fashion, misunderstanding basic concepts or references or being baffled by them. If you are talking about football, and I think you're talking about soccer, or the latest marketing skirmish of our company, eventually one of us will become quite puzzled by the other, and we shall have to reconnect, to agree upon fundamental metaphors. The only kind of "communication" that can take place without metaphors is of a direct signaling kind. If I am foolish enough to place a lighted match below my hand, stimulus-response mechanisms are invoked that directly signal to my skin that it is experiencing a dangerously high temperature. It requires some internal computation in my "wetware," but not metaphorical intervention (substitution, the key to metaphors), to persuade me to move my hand.
Some future devices will certainly feature sophisticated virtual-reality displays or augmented-reality displays. Leaping forward a few generations and imagining direct neural input of signals, as envisioned by early cyberpunk authors like Vernor Vinge in True Names  or James Gibson in Neuromancer  one might ask where or what is the user interface under these extreme conditions? In this situation, user interfaces would be deprived potentially of any physical input devices and physical visual or acoustic display devices. What is left is a world of "pure" mental communication with signs. In my vocabulary, communication includes interaction. In fact, semiotics is usually predicated as a behavioristic science, asking how people behave or interact with signs in order to determine their meaning.
What you are left with is a kind of mental theater, or mental ceremonies. For me, this is the essence of user-interface design: envisioning facts, concepts, and emotions within dynamic, interactive symbolic and iconic artifacts. One thing seems certain: metaphors won't disappear; they are essential to having any communication at all. In fact, Lakoff and Johnson argue that most of these metaphorical references are spatial in nature; for example, in the English-language expressions "things are looking up" and "I've been getting into this new topic of semiotics."
OK, so where are computer metaphors going? Many analysts, prognosticators, and pontificators, such as David Gelerntner, Don Norman, and George Robertson  are calling for the end of the desktop metaphor. Many seem to sense that something is clumsy about fundamental notions of files and folders, of applications and data, embodied in the visual artifacts of government-surplus 1950s metal desks with manila folders stuffed in the drawers and lying in piles on top, with scattered arrays of papers, mostly full of text and an occasional chart or table.
Some have argued that this entire scheme is notoriously culture-biased. Chavan , for example, has argued that most people in India do not own desks or folders and do not have much experience with them. They do have bookshelves, however, with books that have chapters and pages. Perhaps if some Indian researchers at the National Institute of Design in Ahmedabad originally had invented an equivalent of the Xerox Star® at Xerox PARC or later the Apple Macintosh®, we might have had a completely different history of envisioning operating systems and windowing environmentsoops, if there had been any "windows" at all. If Chinese researchers at the Academy of Sciences in Beijing had invented modern computing, as the Chinese did for printing, perhaps a Chinese computer, and most others around the world, would have been displaying vertically unrolling scrolls, not Microsoft Windows®.
Baby faces, devices with small user interfaces, invariably shrink the visual real estate and emphasize acoustic and haptic (tactile) multimedia communication as well as visual. The desktop metaphor simply does not work in a size of 3 x 2 cm (1.18 x 0.78 in.). We need something else, conceptually and perceptually, to help us represent our key structures and processes. What might that be?
Speech interfaces (vocal user interfaces, or VUIs) offer new opportunities but still have hierarchies of objects and navigation, even when one cannot see them. For example, you can "move" within Sports to locate the place where you can purchase baseball tickets online. Even menus are a persistent navigation concept or metaphorical construct of all user interfaces. Other key pervasive concepts include options, tasks, preferences, decisions, and contexts (as in contextual awareness).
The early personal communicators, like the Apple Newton®, emphasized communication over computation as an essential computer-mediated assistant. Today, nouns and verbs of messaging associated with e-mail functions have become essential paradigms for many people's interactions with computers and a source of new, fundamental metaphorical concepts for almost all computer-mediated communication.
Some have argued that having to store files in separate groups related to applications that are also stored in separate groups might do as much damage to mental health, and productive time, as the harm that using the BASIC programming language was supposed to cause in the 1980s. Having to store e-mail messages and references about metaphors in a place separate from text documents about the same content seems clumsy, and a number of solutions have been proposed, such as Apple's OpenDoc  or some offshoots of the Be Operating System (, now part of Palm), which enable users to focus on contents rather than tools.
In the future, it seems likely that visualized metaphors will focus on agents that will help us in all of our regular and even irregular tasks. Some major challenges, it seems to me, are to visualize how agents gather information from us, how we can review what they know about us, and how they report the results of their autonomous activities to us. We have significant developments ahead in inventing the metaphorical apparatus of future butlers that take care of many of our needs, freeing us significantly to take care of other, "more important" (harumph) tasks. We may even need metaphor management software, as the Friend21 project from Japan pioneered in the late 1980s and early 1990s . This software would allow computer systems to swap metaphorical references when reconfiguring entire contexts of data and the best way to present that information to the user in his/her current context.
I do not foresee user interfaces of the future devoid of metaphors.
In any case, I do not foresee user interfaces of the future devoid of metaphors. In fact, the metaphor invention business seems likely to be busier than ever as commercial products seek to make themselves indispensable to our daily lives. One tool just published that might help us sort out these new concepts as fast as they are invented is Faith Popcorn and Adam Hanft's Dictionary of the Future . In this book you can find the just-in-time neologisms of the wordsmiths who shape our perceptions and our conceptions. Are you geared up for the future (to use a metaphor)?
10. Nonogaki, H. and Ueda, H. FRIEND21 Project: Two-tiered architecture for 21st-century human interfaces. In CHI Proceedings of Conference on Human Factors and Computing Systems (Denver, Colorado, May 711, 1995), pp. 160-161.
President, Aaron Marcus and Associates, Inc.
Aaron Marcus (Aaron@AmandA.com) is president of Aaron Marcus and Associates, Inc., Emeryville, CA, and New York, NY (www.AmandA.com). Mr. Marcus frequently lectures about user interface design and has authored or co-authored four books on this subject.
©2002 ACM 1072-5220/02/0300 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2002 ACM, Inc.