Can you remember the first science fiction movie you ever saw? What about the first science fiction program you ever saw on television? Can you remember what you thought about the user-interface design or user experience of any computer-based telecommunication system presented in these shows? And what about today? Do you cast a critical eye on the technology and its use whenever you watch the latest movie or video presentation about a world of the future?
I thought about these questions as I contemplated my own earliest memories. I am not the most dedicated sci-fi junkie, but I have been interested in visions of future worlds since I was small kid growing up in Omaha, Nebraska, more than half a century ago. Back then, I feasted on Marvel’s Weird Science Fantasy comic books, If and Galaxy science fiction pulp magazines, and Ace double-novel science fiction softcover books (which had a front cover on each end of the book and half of the book printed upside down). It was also during this time that I discovered H.G. Wells’s War of the Worlds and 1984. Television was in its first decade and featured Captain Video and Tom Corbett, Space Cadet. Even then Buck Rogers had been featured in movies and appeared on television. Heroes and villains occasionally spoke to and from wall-mounted flat-video displays, although most of the communication took place with desktop microphones and the occasional loud speaker. Control panels consisted of elaborate dials and gauges. I remember during the days of live television when one of Captain Video’s copilots accidentally knocked over the entire control panel (which probably consisted mostly of cardboard coated in metallic spray paint) and the entire desk-size apparatus fell over from the wall of their spaceship. Captain Video deftly picked it up and commented, "Luckily no wires were broken, Steve" (or whatever his copilot was named), then continued on with the show, as if nothing had happened.
These early experiences with images of future technology must have influenced my own early interest in user-interface design because when I was about ten years old, I built a "rocket-ship control panel" in the basement of my house, complete with blinking lights, plus dials, gauges, and clicking knobs salvaged from old radios, my father’s odds and ends, or local trashcan treasures. I don’t recall thinking about usability in my design; I focused more on the emotional impact of blinking lights. With this "sophisticated apparatus," my brother and I could make fantasy trips to distant planets and communicate with extra-terrestrials, at least in our imaginary use scenarios.
Movies and Television, a 50-Year Run
The earliest science fiction movie was George Méliès’s 1902 Le Voyage Dans la Lune. This magician-turned-filmmaker introduced innovative special effects: disappearances, double exposures, other photographic tricks, and elaborate sets. However, his user-interface innovation was modest. Most of us working now probably first encountered the images of computer-human interaction and communication as it was imagined in the science fiction movies of the ‘40s, ‘50s, ‘60s, ‘70s, and ‘80s, depending on when we got started, as people and professionals. The experience growing up with television and computer games has been quite different for more recent generations.
For many past decades, Hollywood, which generated a majority of the science fiction fantasies of CHI, HCI, UI, UX, or whatever you might call it, often seemed somewhat timid about showing advances in communication/interaction. I do recall everyone marveling in 1968 that AT&T Picturephones had become a reality by 2001 (that was only one of the overly optimistic estimates of future progress). Later, we would look in awe at Star Trek’s communicators while marveling at the lapel-pin-based verbal communication with the main computer systems.
Throughout the decades, the background technology would change and slowly upgrade. In the 1950s and ‘60s, twirling double magnetic tape reels and meaningless arrays of blinking lights were the metaphor for computer power. The lights became more and more sophisticated over the decades, but the communication media remained essentially the same. The Star Wars series beginning in 1976 innovated by making everything gritty and somewhat broken down, unlike the clean machines of 2001: A Space Odyssey. The control panels harkened back to earlier times by making some of the rocket-fighter stations reminiscent of World War II aircraft cockpits, much as the action itself was modeled on movie versions of WWII aircraft battles. Likewise for the Matrix series, which relied upon somewhat clunky old battle-station controls, the kind that were satirized effectively in Brazil (one of my favorite romantic but frightening views of the future). Even Ridley Scott’s eerie view of Los Angeles in Blade Runner showed essentially the same old urban displays, albeit larger, crustier, and more depressing than ever. One of the major visualization innovations was Tron, which didn’t innovate so much in handheld communication devices as it did in trying to envision software environments themselves.
Only in the past decade have some notably different visions emerged, perhaps because some set designers have hung around SIGGRAPH and SIGCHI conferences long enough, or because SIGCHI consultants are actually working with the movie and video production teams to provide new concepts.
In terms of new approaches, I am reminded of Tom Cruise dancing with data in Minority Report, of advertisements following his movements and directing targeted commercial announcements directly to him as he flees through a future cityscape. Here represented are interaction and environmental graphics that seemed thoughtful responses to what technology could deliver in the 21st century, not the 20th. These seem related to the transparent user interfaces of the recent television series Farscape, which also emphasizes more virtual than concrete physical equipment.
One exotic, memorable innovation in equipment was eXistenZ, a 1999 sci-fi crime thriller set inside a virtual-reality game world. Besides the innovative game-within-a-game-within-a-game premise of the movie, one of the more striking creations of this film seemed to be the organic devices that looked like mutant protoplasm combined with some metallic components. For example, a gun might look partly like a dead chicken leg, partly like fungus, and maybe a little like a traditional lethal device.
In the recent AeonFlux, we finally see ubiquitous computing products appear, like computation and communication built into clothing. One of the key figures actually talks into his sleeve, which lights up gently to let us know what is happening. At least he doesn’t talk into his shoe like Don Adams as Maxwell Smart in the Get Smart series from 1968-70. In another AeonFlux scene, characters ingest tiny bits of chemical or mechanical substances (nano-pills?) that enable them to communicate with other beings once the substances are absorbed. In another scene, a character’s ear lights up and we are thereby informed that the communication devices have been built into the ear itself. These are considerably more exotic visions of hardware and software than many contemporary films and videos.
Today, there are vast resources on the Internet and in print concerning the history of movies and video, specifically for the subject of science fiction. Amazon lists 1400 entries for sci-fi movies and 2200 entries for sci-fi television. However, most of these deal with the actors and actresses, the story lines, the directors and producers, even the locations. Almost none of these resources focuses on the human-computer and the computer-mediated human-human communication and interaction of these works. Extreme details abound, like those of the Internet Movie Database and the data collections of the American Film Institute. However, the history of user-interface design in movies and television, specifically in the science fiction genre, remains to be collected, researched, analyzed, and made available. This subject will, I hope, be a subject for future books, Internet offerings, and university course offerings, to say nothing of CHI and sci-fi conferences.
As some of you may recall, I organized two CHI panels, called "Sci-Fi at CHI," to which I invited leading science fiction writers to comment on what the future held in store for human-computer interaction and communication. I remember Bruce Sterling’s typically outrageous and funny remarks about the future, in which he imagined that the ubiquitous computing and communication device of the future would look like a handkerchief that could copy anything it was placed over, could present whatever images or sounds one wished, and could connect to the Internet to provide all possible communication and computation support.
Hollywood is typically several generations behind the latest actual achievements of technology. Movies and television have the challenge of making hard-to-imagine scenarios make sense to people who often cannot imagine the full power of user-interface innovation and creativity harnessed to human-computer and human-human communication and interaction. Perhaps the CHI community could help educate the film and television community more effectively.
One recent collaboration is that of Alexander Singer, a Los Angeles film director who worked on Star Trek, with scientists and engineers to produce a short film sponsored by DARPA about augmented cognition. The film features elaborate scenarios using special headsets that can scan the user’s brain and detect overload of certain regions, then shift the information media mix, say, from verbal to visual or from visual to acoustic, to enable the human being to think better. This represents a significant new direction for user-interface flexibility, somewhat reminiscent of the Japan Friend21 project in the late ‘80s and early `90s, which was reported on at CHI 1994. In that project, the researchers experimented with metaphor-management software that would change the entire paradigm of information display, in that case per user preference, not because some meta-system had determined that the users were having cognitive or emotional overload.
By studying the film and video representations of past decades, we may not only gain a better understanding of past illusions and delusions, but we might gain a better understanding of how to present new concepts to the general public and bring them up to date on the future of human-computer interaction and communication.
The following is merely a starter kit of Web sites devoted to the individual offerings or analyses of them:
6. The Future of Augmented Cognition, an Alexander Singer Film showcasing techniques soon to be improving performance on an exponential level everywherehttp://interactive.usc.edu/members/sfisher/archives/2005/09/im_forum_speake_2.html
About the Author:
Aaron Marcus is the founder and president of Aaron Marcus and Associates, Inc. (AM+A). He has degrees from both Princeton University and Yale University, in physics and graphic design, respectively. Mr. Marcus has published, lectured, tutored, and consulted internationally for more than 30 years.
©2006 ACM 1072-5220/06/0500 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2006 ACM, Inc.