The mess we've gotten ourselves into

XV.2 March + April 2008
Page: 22
Digital Citation

TIMELINESThemes in the early history of HCI—-some unanswered questions


Authors:
Ronald Baecker


Ron Baecker’s initial chapter in the 1987 volume of readings that he wrote and edited with Bill Buxton was a very influential reflection on HCI history. It was widely read, reprinted, and served as a model or starting point for subsequent histories. In this review of major themes, Ron poses questions and encourages us to seek out answers while we can. In the 1970s and 1980s I worked for two leading technology companies that eventually went out of business. They left surprisingly few traces. It would be ironic to lose knowledge of the origins of an industry that may preserve almost everything that transpires in the future. Ron’s questions demand continued attention; finding the answers may require years of effort.—Jonathan Grudin

 


We are grateful to the editors for this series, as it encourages us to reflect on the past and to understand that technological miracles do not spring fully formed from the minds of researchers. More typically, they arise from the thoughts and inspirations and long nights of effort of many who have worked in HCI in the past. As this series shows, the interplay among researchers can often be represented by timelines portraying strands of development or thought.

Because our work (variously called “human-computer interaction,” “interaction design,” and “knowledge media design”) has transformed the way human beings create knowledge, learn, think, communicate, and collaborate, we must record and understand our history. By gathering original sources, we can produce accurate, rich, and nuanced accounts of the intellectual history of our field. This is urgent because our opportunity to talk to and record the experiences of many who made seminal contributions is limited. Several have already passed away—Vannevar Bush on June 30, 1974, J.C.R. Licklider on June 26, 1990, Allen Newell on July 19, 1992, Herb Simon on February 9, 2001, Kristen Nygaard on August 10, 2002, Jeff Raskin on February 26, 2005, and, most recently, Brian Shackel on May 9, 2007.

Some valuable contributions have already been made. Several short histories provide useful overviews of HCI. There are books about Licklider and Doug Englebart, and excellent accounts of the early history of personal computing (see page 26).

From these and other sources we can sketch a timeline of significant early threads of our discipline (Figure 1), namely, hypertext (HT), interactive graphics and the concept of direct manipulation (DM), GUI and WIMP interfaces (GUIs), the role of graphic and industrial design (design), usability testing (testing), and understanding workplace context (context).

These six phenomena have been chosen from an even longer list of topics that could be considered highly significant in the development of HCI [1]. To me, they seem the most significant. Do we understand how these ideas and developments arose? What important unanswered questions come to mind?

Hypertext

The origin of hypertext is generally credited to Vannevar Bush’s 1945 article introducing the “Memex,” a mechanism he envisioned for storing documents and linkages among them, and for enabling paths of exploration through the document space. Technological visionaries Doug Engelbart and Ted Nelson creatively elaborated Bush’s vision in the 1960s. They envisioned using computers to build and manipulate richly structured complexes of interconnected, interlinked bodies of text. They realized, as Bush had not, that most information would be stored digitally rather than on microfilm. Yet their approaches differed substantively. Engelbart focused on hierarchic structures for ordinary documents to enable computers to support their preparation; Nelson was more interested in lateral links and interconnections to create novel text “spaces.” Engelbart looked to support creativity and problem-solving in teams; Nelson was excited by individual exploration and combining contributions from people with no formal ties.

Yet interesting questions still come to mind. Why, besides the need to wait for Moore’s Law to make hardware sufficiently inexpensive, did almost 20 years pass before we started to realize Bush’s vision? To what extent and how did Bush’s writings influence Engelbart and Nelson? Did Nelson and Engelbart interact and influence one another? What triggered the explosion of research on hypertext in the late ‘60s and ‘70s that led to the first annual conference in 1987 and the first commercial products? What were the key milestones in the path from there to the Web, regarded by many as the “killer app” of hypertext?

Interactive Computer Graphics and Direct Manipulation

Recent publications document the pioneering interactive computer graphics research at MIT Lincoln Laboratory, including Ivan Sutherland’s influential Sketchpad system in the early ‘60s. Sketchpad demonstrated the potential for effective computer-aided sketching and design through innovative concepts including hierarchic internal structure of computer-represented pictures; recursively defined operations on these pictures; master copies and instances; constraints on picture geometry; iconic representations of constraints; and elegant input techniques using a light pen. Yet questions remain. How and to what extent was Sketchpad influenced by early computer graphics projects such as that of Stephen Coons and Douglas Ross at the MIT Electronic Systems Lab? How did these developments inspire and launch the vigorous field of interactive computer graphics?

Sketchpad and other systems developed at Lincoln Lab were direct manipulation systems, satisfying the four criteria posited by Shneiderman in his important 1983 paper: “1. continuous representation of the object of interest; 2. physical actions ... instead of complex [typed] syntax; 3. rapid, incremental, reversible operations whose impact ... is immediately visible; and 4. layered or spiral approach to learning.” Yet this paper cites no work earlier than the late ‘70s, so the intellectual history of direct manipulation has yet to be written. The concepts were also present in early videogames such as Spacewar—developed at MIT in 1961-1962—in early computer-aided design programs, and in the pioneering computer-aided molecular-chemistry work of Cyrus Levinthal at MIT. Why did it take two decades to abstract this interaction style as a new paradigm?

GUIs and WIMP Interfaces

Related concepts are that of the Graphical User Interface (GUI) and the Windows Icons Menus Pointers (WIMP) style of interaction. In introducing a CHI 2005 panel on early work at Lincoln, Bill Buxton stated “it is hard to imagine the innovation that happened at Xerox PARC in the ‘70s having been possible without the foundation that Lincoln Labs provided.” I believe this is true, but the case needs to be made.

Did the work at Lincoln Lab inspire the development of what arguably was the first personal computer, the Alto, at the Xerox Palo Alto Research Center (PARC)? If so, how? Are there direct links between interactive graphics at Lincoln on calligraphic displays and the Alto bit-mapped display? The same question can be asked of the development, also at PARC, of Dick Shoup’s Superpaint color frame buffer. How did these lead to Xerox’s late and unsuccessful attempt to commercialize personal computing in the Star system, which influenced the design of the Apple Lisa, the predecessor of the Macintosh? Where are the earliest manifestations of each key component—bit-mapped displays, the desktop metaphor, What You See Is What You Get (WYSIWYG) document editing, overlapping windows, icons, and menus? How did the invention and refinement of these concepts interrelate?

A Broadening of Focus

We now reach a turning point in HCI history. The invention and widespread success of the PC enabled hundreds of millions of humans to interact with computers, in contrast to a few million organizational mainframes and tens of millions of minicomputers. A wider set of issues became relevant.

Graphic Design and Industrial in Interaction

For computers to become mainstream consumer products, they had to look as good as sports cars and hair dryers. Thus industrial design would play a key role. Furthermore, one reason the GUI became the dominant interface paradigm of the ‘80s was that graphic designers and visual artists began to exploit bit-mapped displays to make interfaces more attractive and communicative.

What were the roles of design pioneers such as Aaron Marcus and Edward Tufte in inspiring such developments? How did this movement influence hardware and interface software in seminal products such as the Star, the Apple II, and the Macintosh? How did the graceful design sense of these products manifest itself and become an essential element in other mainstream software products? How did industrial research groups, product-development teams, and scholars and practitioners from universities and design schools contribute and interact?

Usability Testing

By the ‘80s it became apparent that there was another implication of the fact that the computer had become a mass-market product for nonspecialists. To design software successfully required usability testing, a set of techniques that draw their inspiration from human factors.

What are the earliest known examples of user testing? How did the concept of “usability testing” evolve into more comprehensive “usability engineering?” Who built the first usability lab? What were the important innovations by hardware companies such as IBM and DEC, and by systems and software-development organizations? How did an understanding of the importance of usability testing spread from organization to organization? What was the role of CHI and other new conferences? To what extent were such innovations guided by systematic principles of user-centered, iterative design, such as those articulated by John Gould and his collaborators at IBM?

Understanding Workplace Context, and Designing for Humanization and Democratization

Another phenomenon that started in Europe and spread to North America was the commitment to ground system design in a deep understanding of workplace context. The British sociotechnical design movement and the Scandinavian collective-resource approach both aimed at humanizing the technology’s impact in the workplace. The latter philosophy later became the participatory design movement as it spread worldwide. North American recognition of the importance of work-place context and the role of methods rooted in anthropology and sociology was spurred by the influential work of Lucy Suchman at Xerox PARC in the mid-‘80s. Since then many social scientists have been hired by corporations such as IBM, Microsoft, and Intel. Yet we lack a comprehensive scholarly history of the roles in these developments of various individuals, corporations, and academic institutions.

Toward a Richer Understanding of the History of HCI

I have reviewed the early history of HCI—hypertext, direct manipulation, and the development of the GUI, then suggested that what happened next was a broadening of the field’s focus to incorporate the skills of graphic and industrial designers, applied psychologists, and social scientists. This brief article is not intended as the final word on any of these topics. Each short treatment could be a sketch of a future Timelines article, or, better yet, a Ph.D. thesis in the history of science and technology. This is my challenge to the readers.

We generally know the names of important contributors, but how they built on one another’s work is, for the most part, yet to be written. We don’t know what was happening in different places and the way ideas spread from country to country, especially in the days before email, the Internet, and the Web. We also have little insight into the interplay between academic research and industrial R&D, between publications and patents. Finally, except for the line from direct manipulation to the GUI, we have little understanding of how lines of development influenced each other.

Can we do better? Consider the history of medicine. The Wellcome Trust for the History of Medicine at University College London created the “Wellcome Witnesses to Twentieth Century Medicine"in 1993. There are currently 31 volumes, available in hard copy and online, that comprise important papers, records, photographs, and transcripts of daylong seminars in which significant figures in 20th-century medicine discuss specific discoveries or events in recent medical history.

There are some hopeful signs of similar activities in our field. Goldberg’s 1988 volume, A History of Personal Workstations, contains transcriptions of talks by major contributors to the development of personal workstations. More recently, the DigiBarn Computer Museum has held and recorded events with pioneers in the development of direct manipulation, the Alto, the Apple, the IBM PC, and the Macintosh.

Nonetheless, let us hope that some visionary corporation will step up and create the <your company’s name> Witnesses to Twentieth Century Human-Computer Interaction. It is urgent that this happen soon.

Acknowledgements

I am grateful to Delia Couto, whose research skills and diligent efforts assisted greatly in the preparation of this article, and to Jonathan Grudin for his encouragement and helpful suggestions. Thanks to Jennifer Keelan for acquainting me with the Wellcome Witnesses series, and to Eric Martin and William Newman for helpful suggestions.

References

1. Other worthy concepts and paradigms include (in no particular order) user-interface toolkits, evaluation methods, mental models, formal methods for describing human-computer interactions, interaction techniques, input devices, mobile devices, virtual reality, computer games, information visualization, speech input and output, and multimodal interfaces.

Author

Ronald M. Baecker
University of Toronto
rmb@kmdi.toronto.edu

About the Author

Ronald Baecker is professor of computer science, the Bell University Laboratories chair in human computer interaction, and founder and chief scientist of the Knowledge Media Design Institute at the University of Toronto. He was named one of the 60 “Computer Graphics Pioneers” by ACM SIGGRAPH, elected to the CHI Academy by ACM SIGCHI, and awarded the Canadian Human Computer Communications Society Achievement Award. He has been working in “HCI” since 1966.

EDITOR

Jonathan Grudin
jgrudin@microsoft.com

Footnotes

DOI: http://doi.acm.org/10.1145/1340961.1340968

Figures

F1Figure 1. Timeline of Early HCI History

Sidebar: The Early History of Personal Computing: A Bibliography

PART ONE: GENERAL SOURCES ON THE HISTORY OF HCI

One useful historical overview is Chapter 1 of Baecker, R.M. and Buxton, W. (1987), Readings in Human Computer Interaction: A Multidisciplinary Approach, Morgan Kaufmann. (A slightly improved version appears in Baecker, R.M., Grudin, J., Buxton, W., and Greenberg, S. (1995), Readings in Human Computer Interaction: Toward the Year 2000, Morgan Kaufmann.) Three others are Shackel, B. (1997), “Human-Computer Interaction—Whence and Whither,” Journal of the American Society for Information Science 48(11); Myers, B. (1998), “A Brief History of Human-Computer Interaction Technology,” interactions, March-April; and Grudin, J. (2007), “A Moving Target: The Evolution of Human-Computer Interaction,” in A. Sears and J. Jacko (Eds.), Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, Erlbaum.

Two excellent edited volumes on the early history of personal computers are Gupta, A. and Toong, Hoo-min D. (Eds.) (1985), Insights into Personal Computers, IEEE Press; and Goldberg, A. (Ed.) (1988), A History of Personal Workstations, ACM Press. A good journalistic account is Levy, S. (1984), Hackers, Anchor Press/Doubleday.

Licklider is discussed in depth in Waldrop, M.M. (2001), The Dream Machine: J.C.R. Licklider and the Revolution that Made Computing Personal, Penguin Books.

More than 40 recent interviews with important interaction designers are reported in Moggridge, B. (2007), Designing Interactions, MIT Press. Erickson, T. and McDonald, D. (Eds.) (in press), HCI Remixed, MIT Press, presents personal accounts of the impacts of seminal papers. A useful website is maintained by the Georgia Tech Program in Human-Centred Computing, see http://hcc.cc.gatech.edu/taxonomy/cat.php?cat=2.

PART TWO: HYPERTEXT

The original article is Bush, V. (1945), “As We May Think,” Atlantic Monthly 176(1). But see also Rayward W. B. (1994), “Visions of Xanadu: Paul Otlet (1868-1944) and Hypertext,” Journal of the American Society for Information Science 25(4), May, describing an early Belgian “information scientist” who anticipated some key aspects of hypertext.

Much has been written about Engelbart and Nelson. Most useful for learning about Engelbart is Bardini, T. (2000), Bootstrapping: Douglas Engelbart, Coevolution, and the Origins of Personal Computing, Stanford University Press, and Oinas-Kukkonen, H. (2007), “From Bush to Engelbart: ‘Slowly, some little bells were ringing,’” IEEE Annals of the History of Computing 29(2), April-June, 31-39, which relies on interviews, includes a comprehensive bibliography, and details Bush’s influence. A monumental early book is Nelson, T. (1974), Computer Lib: You Can and Must Understand Computers Now, and, on the flip side, Dream Machines: New Freedoms Through Computer Screens—a Minority Report, self-published, out of print.

A useful set of resources, http://www.cs.brown.edu/memex/, includes Andy Van Dam’s keynote address at the first Hypertext Conference in 1987.

PART THREE: INTERACTIVE COMPUTER GRAPHICS AND DIRECT MANIPULATION

For accounts of the early history of interactive graphics, see Hurst, J., Mahoney, M.S., Taylor, N.H., Ross, D.T. & Fano, R.M. (1989), “Retrospectives: The Early Years in Computer Graphics at MIT, Lincoln Lab, and Harvard,” ACM SIGGRAPH’89 Panel Proceedings, Part I and Part II; Machover, C. (1978), “A Brief, Personal History of Computer Graphics,” IEEE Computer 11(11), November; Wayne Carlson’s “Critical History of Computer Graphics and Animation,” (http://design.osu.edu/carlson/history/ID797.html); and also chapters by Gordon Bell, Doug Ross, and Wesley Clark in Goldberg (1988).

An important historical panel is Buxton, W. (2005), “Interaction at Lincoln Laboratory in the 1960s: Looking Forward — Looking Back.” Panel Introduction, Proc. CHI 2005, 1163-1167, also see http://www.billbuxton.com/Lincoln.html, and the ePresence video archive of the panel, http://epresence.tv/Presentation/3.

The Sketchpad thesis has been reprinted as Sutherland, I.E. (1963), “Sketchpad: A Man-Machine Graphical Communication System,” MIT Ph.D. Dissertation (http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.html).

Direct manipulation was defined in Shneiderman, B. (1983), “Direct Manipulation: A Step Beyond Programming Languages,” IEEE Computer, August.

The development of Spacewar is recounted in Levy, S. (1984), Hackers: Heroes of the Computer Revolution, Anchor Press, Chapter 3. A series of computer-aided design history timelines is at http://mbinfo.mbdesign.net/CAD-History.htm. An archive devoted to the history of using computer graphics to visualize biological macromolecules, starting with the work of Cyrus Levinthal and colleagues at MIT in 1964-67, is http://www.umass.edu/molvis/francoeur/.

PART FOUR: GUI AND WIMP INTERFACES

The best account of the development of the Xerox PARC Alto personal computer, the Superpaint color frame buffer, and the earliest implementations of the graphical user interface is Hiltzik, M. (1999), Dealers of Lightning: Xerox PARC and the Dawn of the Computer Age, Harper Business. An earlier journalistic account focusing more on the business context is Smith, D.K. and Alexander, R.C. (1988), Fumbling the Future: How Xerox Invented, Then Ignored, the First Personal Computer, William Morrow.

An excellent scholarly account of the desktop metaphor is in Blackwell, A. (2006), “The Reification of Metaphor as a Design Tool,” ACM Transactions on Computer-Human Interaction 13(4). The earliest WYSIWYG word processors were Bravo and Gypsy developed at Xerox PARC; for information about Gypsy development see the interviews with its developers, Tim Mott and Larry Tesler, in Moggeridge (2007).

Overlapping windows, a key feature of most GUIs, emerged in the pioneering Smalltalk environment developed by Alan Kay’s group at PARC (Kay, A., and Goldberg, A., 1976, Personal Dynamic Media, Xerox PARC Technical Report SSL-76-1). Early thoughts that led to the concept of personal dynamic media are found in Kay, Alan, “The Reactive Engine,” Ph.D. dissertation, University of Utah, 1969. A comprehensive first-person account of the development of Smalltalk is Kay, A. (1993), “The Early History of Smalltalk,” ACM Sigplan Notices 28(3). See also two recent publications: Barnes, S. (2007), “Alan Kay: Transforming the Computer into a Communications Medium,” IEEE Annals of the History of Computing 29(2), April-June; and Maxwell, J. (2007), “Tracing the Dynabook: A Study of Technocultural Transformations,” Ph.D. dissertation, Simon Fraser University, and http://thinkubator.ccsp.sfu.ca/Dynabook/, which provides many links to relevant sources.

A good single source on the Xerox Star is Johnson, J., Roberts, T.L., Verplank, W., Smith, D.C., Irby, C.H., Beard, M., and Mackey, K. (1989), IEEE Computer 22(9). Case Study D in Baecker and Buxton (1987) lists almost 40 other sources.

A good journalistic account of the development of the Apple Macintosh is Levy, S. (1994), Insanely Great: The Life and Times of Macintosh, the Computer that Changed Everything, Penguin Books. 118 stories about the development of the Macintosh and the people who created it are at http://www.folklore.org/index.py.

PART FIVE: GRAPHIC DESIGN AND INDUSTRIAL DESIGN IN INTERACTION DESIGN

To my knowledge, Aaron Marcus is the first graphic designer to commit himself to a career in interaction design. A pioneering early article applying graphic design expertise to the design of a page layout system is Marcus, A. (1971), “A Prototype Computerized Page-Design System,” Visible Language V(3), Summer 1971. Aaron began teaching tutorials on the subject in 1980 and established the design firm Aaron Marcus and Associates in 1982. Good interviews with Aaron are found at http://www.informationdesign.org/special/marcus_interview.php and http://www.amanda.com/resources/webword/webword_marcus.html.

An excellent history of Apple covering the development of the Apple II is Malone, M.S. (1999), Infinite Loop: How Apple, the World’s Most Insanely Great Computer Company, Went Insane, Currency Doubleday. Pages 122-123 discuss the roles of industrial designer Jerry Manock in developing the case for the Apple II and art director Rob Janov in developing a new Apple logo. See also http://apple2history.org/.

Levy (1984), Chapter 6, discusses the roles of Manock and graphic designer Susan Hare in developing the Macintosh. Interesting debates involving Steve Jobs and key designers and developers about whether the Mac should be more like a Beetle, a Ferrari, a Porsche, or a Cuisinart are documented in http://www.folklore.org/StoryView.py?project=Macintosh&story=More_Like_A_Porsche.txt.

Beginning with work on statistical graphics in the mid-‘70s, Edward Tufte has emerged as the preeminent information designer, setting standards for elegant design tailored to cognitive tasks such as understanding causality, comparison, and the effects of multiple variables on complex phenomena. A thoughtful and comprehensive interview with Tufte is Zachary, M. and Thrall, C. (2004), “An Interview with Edward Tufte,” Technical Communication 13(4). See http://www.edwardtufte.com/tufte/ for information about his four beautiful books, including the particularly influential first book, The Visual Display of Quantitative Information, 1983, 2001, Graphics Press.

PART SIX: USABILITY TESTING

The extensive usability testing in Star development is described in Bewley, W., Roberts, T., Schroit, D., and Verplank, W. (1983), “Human Factors Testing in the Design of Xerox’s 8010 ‘Star’ Office Workstation,” Proc. CHI ‘83, 72-77. User testing of the Lisa conducted by Larry Tesler is described in Levy (1994) Chapter 4, and also in http://www.folklore.org/StoryView.py?project=Macintosh&story=Do_It.txt&sortOrder=Sort%20by%20Date&detail=medium&search=user%20testing.

Arguably the most influential industrial research group to develop principles of user-centered, iterative design was IBM Yorktown Heights. Lessons learned were summarized in Gould, J. and Lewis, C. (1985), “Designing for Usability: Design Principles and What Designers Think,” Communications of the ACM 28(3). See also Gould, J. (1988), “How to Design Usable Systems,” Chapter 35 of Helander, M. (Ed.), Handbook of Human-Computer Interaction, North-Holland.

Another important group was at DEC, see for example Whiteside, J., Bennett, J., and Holtzblatt, K. (1988), “Usability Engineering: Our Experience and Evolution,” Chapter 36 of Helander. An excellent overview of the history and practice of usability engineering is Butler, K.A. (1996), “Usability Engineering Turns 10,” interactions, Jan. 1996.

A seminal vision of an applied information-processing psychology of human-computer interfaces that could reduce the need for usability testing is Card., S.K., Moran, T.P., and Newell, A. (1983), The Psychology of Human-Computer Interaction, Erlbaum.

PART SEVEN: UNDERSTANDING WORKPLACE CONTEXT

An excellent review of sociotechnical design, including its origins at the Tavistock Institute founded in London in 1946 and its interactions with developments in Scandinavia, is Mumford, E. (2006), “The Story of Socio-technical Design: Reflections on its Successes, Failures, and Potential,” Information Systems Journal 16. The Scandinavian approach to the design of computer-based systems is treated in depth in Floyd, C., Mehl, W.-M., Reisin, F.-M., Schmidt, G., and Wolf, G. (1989). “Out of Scandinavia: Alternative Approaches to Software Design and System Development,” Human-Computer Interaction 4(4). See also Ehn, P. (1988), Work-oriented Design of Computer Artifacts, Lawrence Erlbaum, esp. Chapter 11.

Hiltzig (1999), Chapter 14, describes how designers of the Gypsy word processor grounded their work in interviews with editors at the Ginn publishing subsidiary of Xerox. Chapter 21 discusses how the Xerox Systems Science Lab based new office system designs on an understanding of how people do their work. A landmark achievement was the Ph.D. dissertation Suchman, L. (1987), Plans and Situated Actions: The Problem of Human-Machine Communication, Cambridge University Press, which applied ethnomethodological methods to the analysis of an expert help system.

PART EIGHT: TOWARDS A RICHER UNDERSTANDING OF THE HISTORY OF HCI

For the Welcome Trust, see http://www.ucl.ac.uk/histmed/. The Computer History Museum’s website may be found at http://www.digibarn.com/. Most interesting is the Digibarn Computer Museum, with a website at http://www.digibarn.com/, that describes its “nonmuseum approach” to creating “a kind of ‘memory palace’ for the nerd-inclined [to] help ... piece together the amazing story of the invention of personal computing and Cyberspace.”

©2008 ACM  1072-5220/08/0300  $5.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2008 ACM, Inc.

 

Post Comment


No Comments Found