Movies and videos over the past century used human-computer interaction (HCI) to tell science-fiction (sci-fi) stories that envision alternate worlds. Sci-fi may have begun about 1260 AD/CE, when the philosopher Roger Bacon predicted “wagons may be built which will move with incredible speed and without the aid of beasts; flying machines can be constructed in which a man ... may beat the air with wings like a bird…” . Leonardo da Vinci some 250 years later visualized such flying machines and other sci-fi-like wonders. After World War II, science-fiction writers and filmmakers envisioned an optimistic world in which people flew everywhere and robots entertained us. These media depict the motion of people and things against a background.
In sci-fi movies and television, action is primary, followed by the story, and people/personalities. Sci-fi movie makers from the beginning were compelled to show HCI, because these visual media required compelling scenes to be filled with appropriate images. Sci-fi media makers were rapid prototypers from the start, among the first user-centered, user-experience designers of HCI. One difference between written literature and these visual media: Sci-fi authors would have to create names for new technologies, products, and objects; sci-fi media could simply show them without naming them, leaving it to the viewer to find the appropriate terms.
Sci-fi movies and videos added to sci-fi stories and illustrations. They visualized new paradigms for HCI metaphors, mental models, navigation, interaction, and appearance. Sometimes they visualized them decades before commercial product or service introduction. They not only gave developers “practice” at envisioning personas and use scenarios, but also “softened up” customers. That is, people became accustomed to what they might expect in future years.
One might even say that sci-fi media offer “test beds,” virtual focus groups, contextual-analysis environments, or ethnographic-study sites for products and services with advanced HCI capabilities. If R&D groups, as well as marketing groups, were to examine the films and videos closely, along with audience reactions to what they depict (which is not typically done in Hollywood screenings for market response), they might be able to detect trends of expressed wants and needs, levels of acceptance and rejection, and tolerance for fit in the lives of customers today and tomorrow. One might even be able to track cultural distinctions among worldwide sci-fi media distributions that would benefit R&D as well as global deployment of advanced products and services.
In an interesting, bizarre, and useful way, sci-fi media are an encapsulation of the entire world of HCI development transplanted into another context. Much can be learned if one takes the time to look. You might ask:
- What seems futuristic? What doesn’t?
- What masculine-versus-feminine issues emerge? Are there differences? Should there be?
- What people-versus-machines issues emerge?
- What did people get right about envisioning a future? What did they miss?
Let’s review some films to track these themes.
Metropolis, 1927, Directed by Fritz Lang
Metropolis is considered a classic of sci-fi, one of the first to show HCI extensively. The movie depicts the use of mechanical controls in future factories, (female) robots as a factory solution to rowdy workers, the use of videophone (television-like) displays about 20 to 40 years before their commercial introduction (introduced in the film without fanfare), and scenes of executive desks with “executive information systems” in the form of complex telephone control switchboards.
Flash Gordon, 1940
Early examples of video-like displays also appear in a Flash Gordon serial episode predating by decades the flat video screen in Kubrick’s 2001 from 1968. Movie makers seemed intent on absorbing the latest technology (including HCI) from sci-fi literature into the backgrounds of their storytelling.
Captain Video and Other Television Serials, 1950s
After the introduction of broadcast television in the 1950s, sci-fi serials such as Captain Video, Tom Corbett and His Space Cadets, and Rocketman featured heroes rescuing heroines or humanity. Most of these shows featured typical cluttered airplane-cockpit controls with sets of blinking lights added for visual impact. Human-factors analysis of these sets would probably have reported many errors of design.
Voyage to the Planet of Prehistoric Women, 1968, Directed by Peter Bogdanovich
Voyage to the Planet of Prehistoric Women, a low-budget film with 1950s technology throughout, shows the typical wall of blinking lights and twirling tapes, complex rocket-ship control panels with hundreds of hard-button displays, and a remarkable contrast between male astronauts, heavy with technology, and the female denizens of Venus, who communicate via telepathy, seemingly devoid of metal and plastic technology, to say nothing of much of their clothing. This men-women contrast is a recurring theme of sci-fi movies, continuing with Avatar (2010).
2001: A Space Odyssey, 1968, Produced and Directed by Stanley Kubrick
The iconic film 2001 features many scenes of HCI displays, most of them conventional enhancements to existing contemporary technologies. Notable among its off-target guesses at the future are the use of video payphones, meeting-room conference tables without desktop computers, and no mobile phones. The film does show touch-sensitive desktop displays and also one of the first comic HCI scenes: an astronaut contemplates instructions for using a toilet, which are so long it is doubtful he ever gets to use the facilities. One innovative, memorable scene shows Hal, the onboard computer system, lip-reading, eavesdropping on a secret conversation between two astronauts.
Star Trek, Television Series, 196669, and Film Series, from 1979, Created by Gene Roddenberry
In marked contrast with conservative HCI content in many movies of the 1960s and 1970s, Star Trek showed a wealth of innovative HCI devices and techniques, including medical diagnostic tools with intriguing, cryptic displays, and the ever-present, often-used voice communicators. Another notable aspect of Star Trek was its multiracial, multi-gender, and even multi-species crew, which implied much user-centered design to accommodate different kinds of alien personas and use scenarios.
Star Wars Series, 19772008, Directed by George Lucas
Star Wars impressed viewers with mythic visual storytelling, charming robots, dirty rocket ships (not the clean vehicles of the 1950s), but not much innovation in the realm of HCI. Besides an occasional projected virtual image and the mind control of objects, the most memorable interaction object may be the light saber.
Tron, 1982, Directed by Steven Listberger, and Tron Legacy, 2010, Directed by Joseph Kosinski
Tron 1982 presented an innovation of sci-fi movies by trying to represent the world of video games and code. Oddly, the movie shows almost no advanced HCI interaction sequences. The sequel in 2010, Tron Legacy, updates the action movie and continues the lack of significant HCI content. Unsurprisingly, the only major HCI scenes show nostalgic video-game displays and Unix-like textual code displays with traditional keyboards.
Terminator Series, 19842009, Directed by James Cameron
Much more inventive HCI appears in the Terminator series, starring a superior android (Arnold Schwarzenegger) with a Teutonic sense of humor. Here, HCI aficionados can find augmented-reality displays given a prominent role.
Brazil, 1985, Directed by Terry Gilliam
The sci-fi fantasy Brazil shows a notable combination of 19th- and 21st-century equipment from a dystopian future of high-tech tubes and terrorism. Many of the devices depicted show ingenious, neurotic attention to dysfunctional details, for example, a typist’s fingers-exoskeleton assisting her typing as she transcribes the screams of subjects being tortured, or the magnifying lenses placed over too-small (black-and-white!) displays in order to make them appear larger, but distorted. Ubiquitous HCI technology appears in a bizarre asylum for the deranged inhabitants of a world both appealing and appalling.
Total Recall, 1990, Directed by Paul Verhoeven
Total Recall presents a great deal of innovative, memorable HCI content. One of the more unforgettable but disgusting scenes features actor Arnold Schwarzenegger removing a tracking device from his nose. Other scenes show video-wall displays, wall-size X-ray displays, 3-D virtual avatars, body suits, and animatronic taxi drivers. One inadvertently comical, anachronistic HCI scene shows a hotel employee looking at a conventional CRT display of a hotel service computer from the 1970s, although the movie is set in 2084.
Matrix Trilogy, 1999 and 2003, Directed by the Wachowski Brothers
The Matrix Trilogy depicts a mass-fantasy future, a simulated reality created by sentient machines to pacify the human population, which is used as an energy source. Memorable HCI moments include the cascading waterfall of cryptic alphanumeric symbols, “plugging in” directly to computer systems (and the consequent virtual worlds as the “display”), the use of pills (chemicals) to aid in connecting to computer systems, and the array of CRT display screens that are supposed to convey a world of high technology, but which completely overlook the immense complexity of these displays and their likely uselessness. Nevertheless, the imagery is compelling, which is often the point of it all.
eXistenz, 1999, Directed by David Cronenberg
eXistenz, like Tron, treats the world of video games themselves, but without the code. The innovative, memorable HCI scenes feature biomorphic, bloody technology, in which parts of devices are built from animal tissues and bones. One memorable scene features a hero assembling a gun from leftover bones and tissue on the plate of food served to him, then using teeth as bullets to kill the waiter.
Minority Report, 2002, Directed by Steven Spielberg
Minority Report raised the general public’s awareness of hand gestures when Tom Cruise famously whisked photos around on a large transparent display. The world of 2054 shows many examples of advanced HCI, including personalized ads that recognize customers, extensive government surveillance, iris identification, extensive virtual reality, and projected 3-D images.
Aeon Flux, 2005, Directed by Karyn Kusama
Aeon Flux shows many inventive HCI technologies in its story of a post-apocalyptic era: cameras in the eyes, a human being with two extra hands transplanted to replace feet, pills to enable users to access information spaces, skin displays that show maps on the forearms, and contact lenses to transmit images to other viewers. The notable HCI technology plays a significant role in the story line.
Idiocracy, 2006, Co-written and Directed by Mike Judge
While most sci-fi films feature extremely intelligent people using sophisticated technology, the sci-fi comedy Idiocracy depicts a below-average hero who sleeps through 500 years to awaken as the smartest man on Earth among a dumbed-down population that needs childlike displays and constant reminders of sex to keep them entertained. Comical HCI scenes include a medical-center receptionist searching among simple symbols of ailments to diagnose a new patient and home-video displays that are primarily porn ads. This film raises questions about the personas assumed for the use of “advanced” HCI devices.
Avatar, 2009, Written and Directed by James Cameron
Avatar depicts earthmen of about 2150 marauding Pandora, which seems to have a strong feminine leadership. Once again, isolated, destructive men with metal are pitted against sensitive, wise, connected women (as well as animals and plants), a dialectic featured in other sci-fi films. The HCI technology favors three-dimensional and transparent screen displays, which have become the iconic representation of “advanced technology,” much as rotating magnetic tape drives were in past decades. Again, UX professional analysis and human-factors analysis might take a dim view of their usability and usefulness.
District 9, 2009, Directed by Neill Blomkamp
The South-African sci-fi thriller District 9 features inventive, detailed HCI depictions and new twists on user-centered design. The human hero who is morphing into an alien attempts to pilot an alien spaceship. Can one develop HCI systems so intelligent they can accommodate unexpected life forms? In another scene, the alien creature pilots its own ship using three-fingered interactions with virtual controls. Its elegant, graceful, fast-paced finger dance makes Tom Cruise’s character in Minority Report look like a toddler trying to walk.
Sci-fi movies and videos can serve as interesting, valuable material on which to run heuristic evaluations of the designs, to study future personas and use scenarios, and to inform designers of possible future technological, social, or cultural contexts. Among authors urging such study is Stuart Reeves in the pages of this publication recently , but in the past few years there are many authors focusing on the value of science-fiction media (see, for example, ). There might even emerge a new class of heuristics: What principles should we follow to make sure that alien creatures can use our products and services? Conversely, how should we evaluate alien displays and equipment shown in terms of their use by human beings? In an ironic twist, “normal” human beings might be classed as disabled or less-abled creatures, much as our world now classifies some individuals.
Sci-fi filmmakers might benefit from more skilled and extensive use of HCI professionals in making their movies (almost none are ever credited). As a consequence, viewers might be exposed to some of the truly latest technologies of R&D (not pop versions), including spherical sound, the Internet of objects, sophisticated personalization and location technologies, and virtual interactions. Perhaps then we would move beyond the transparent displays of backward type shown in Avatar and other imagery on which movie-production centers have currently fixated.
Likewise, advanced technology R&D centers could benefit from the inclusion of more visiting or “resident” science-fiction authors and futurists as consultants in dreaming up innovative approaches that stretch the imagination of engineers, marketers, business managers, and user-experience designers involved in developing new approaches to HCI. Bruce Sterling, a science-fiction writer, has served in that capacity. Brian David Johnson, a futurist at Intel, is another. There should be more.
Each world could learn from the “aliens” of the other. In 1992 and 1999, I introduced cyberpunk sci-fi authors to SIGCHI conferences, inviting them to predict the future of HCI. This kind of alien visit should occur on a more regular basis. Now it’s time to look more deeply at sci-fi’s latest trends, patterns, and predictions. The evidence of the past 100 years of imagining and visualizing the future is now more readily available to researchers, analysts, and designers. The challenges and opportunities of looking, listening, and learning from the past visions of the future and across professional boundaries remain to be explored for the benefit of all.
3. Marcus, A. The Past 100 Years of the Future: HCI in Science-Fiction Movie and Television. 2012; http://www.amanda.com/wp-content/uploads/2012/10/AM+A.SciFI+HCI.eBook_.LM10Oct12.pdf
Aaron Marcus is president of Aaron Marcus and Associates, Inc. and editor in chief emeritus of User Experience magazine. He is a pioneer of user-experience design and the first user-interface designer to be elected as both a member of the ACM CHI Academy and a Fellow of the American Institute of Graphic arts.
Sidebar: Suggested Further Reading
Emchowicz, A. and Nunneley, P. Future Toys: Robots, Astronauts, Spaceships, Ray Guns. New Cavendish Books, London, 1999.
Greenmeier, L. Professional seer. Scientific American 306, 5 (May 2012), 8083.
Hartwell, D.G., ed. The World Treasury of Science Fiction. Little Brown and Co., New York, 1989.
Heads-up and GUIs of the Future; http://hudsandguis.com/tag/guis/
Hogan, R. Towards a taxonomy of science fiction book trailers. 2008; http://www.mediabistro.com/galleycat/towards-a-taxonomy-of-science-fiction-book-trailers_b7440
Korschak, S.D., ed. Frank R. Paul: Father of Science Fiction Art. Castle Books, New York, 2010.
Marcus, A. Sci-Fi at CHI. Proc. of CH1 1999 (Pittsburgh, PA). ACM, New York, 1999.
Marcus, A. Sci-Fi and HCI: Past, present, future. Opening plenary address of Mensch und Computer Technical University, Chemnitz (June 7, 2011); http://uebermedien.org/retrospektive/video-keynote-aaron-marcus/
Marcus, A. and Koster, C., eds. Special Issue on UX d Science Fiction. User Experience 13, 2 (2013).
Robertson, F.M. Art of Imagination: 20th Century Visions of Science Fiction, Horror, and Fantasy. Collectors Press, Portland, Oregon, 2002.
©2013 ACM 1072-5220/13/07 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2013 ACM, Inc.