This article introduces the notion of visual thinking in HCI and interaction design. It also serves as a call for contributors to a new regular interactions section, the Visual Thinking Backpage Gallery (see sidebar).
Visual thinking is the use of imagery and other visual forms to make sense of the world and to create meaningful content. Digital imagery is a special form of visual thinking, one that is particularly salient for HCI and interaction design. Digital photographs qualify as digital imagery only when they are also visual thinkingthat is, when they are instrumental in making sense or creating meaning. The distinction between digital and analog imagery is neither strikingly germane nor material. Nonetheless, I use the term digital imagery since most image making these days involves digital sensors and/or digital post-processing and distribution, and this use of digital technologies, in turn, enables a wider proliferation of imagery than ever before. The terms digital imagery and imagery are nearly interchangeable nowadays.
In HCI and interaction design, we often focus on creating meaning through textual forms, even though we use imagery in all kinds of interactivity design. Long regarded as a foundational skill in traditional design disciplines, still-image making should also be regarded as a foundational skill in HCI and interaction design.
Visual Thinking: Meaning and Form
One perspectivenot uncommon, but not as widely held as one would likeis that design entails acts of making sense of form and acts of endowing form with meaning. While there are certainly other more and less elevated perspectives about design’s concerns and agency, this perspective of design as sense-making and meaning-making is alone enough to justify two key roles for imagery: as a mechanism of understanding and constructing meaning and form, and as a foundational skill implicated in interaction design and HCI. The most important concern of still-image making, in a professional sense, is to make sense of the world and reveal what is extraordinary and meaningful about everyday scenes.
Novice designers often erroneously think that elements of visual form are a matter of aesthetic preference. In fact, stylistic and layout and device elements of interactivity should, at best, scaffold meanings, or at least not distract from them. Meanings come from skillful, rigorous, and evidential use of text and imagery, rather than from layout and stylistic and device elements. Meaningful content in the form of meaningful text and imagery contributes far more to the impact of interactivity design than decisions about decorative elements of form. In the vernacular, a common notion is that the design is associated with the aesthetics of the elements of form, abstracted away from the construction of meaningful content. In the contrasting perspective of visual thinking, it is the meaningfulness of the visual and textual content that most affects the quality of a design.
Visual Thinking: Ontological Design
A more sophisticated view is the perspective of ontological designthat is, the notion that our actions are determineddesigned in a senseby our use of what we design, intentionally and unintentionally. In this view, designs are acts of choice about future ways of being. Lewis Hine’s photographs of child laborers in the early 20th century (see above) that led to the enactment of child labor laws in the U.S. illustrate this notion. Hine’s photographs are ontologically designed. Photographyimageryin this sense of social activism is not only a form of design but also a fundamental one that is critical to any notion of interactivity. Ontological design entails a values-rich perspective on sense-making and meaning-making.
Visual Thinking: HCI and Interaction Design
We are fortunate in HCI and design to have many great contemporary thinkers (see “Scholarship” section for an example of but a few) who have greatly elevated the discourse about HCI and interaction design’s agency in the world by providing text-based resources focusing on design theory, ethnography, critical theory, feminism, philosophy, cultural studies, ethics, and associated domains. The goal of this article and the Visual Thinking Backpage Gallery is to prompt our collective mastery of design by focusing on our construction and use of imagery as a core and fundamental competency in addition to our many other competenciesthat is, to elevate the visual discourse in much the same manner that these contemporary thinkers have elevated the textual one. In adding to our competencies, we can further develop our transdisciplinary mindset (see ), which is the designerly and values-oriented notion that the choices we make about what to do as goals motivated by values and ethics precede the choices we make about which domains of expertise and collections of methods to apply to achieve these goals.
The Role of Digital Imagery in Visual Thinking
By digital imagery, I mean primarily still photographic images that are instrumental in acts of visual thinking and that make use of digital technologies at any point in their production, processing, and/or distribution. Imagerydigital or notis a medium and form of visual thinking. With respect to visual thinking, imagery may play many different roles:
- as a material of interaction design;
- as documentary observation and photo-ethnography;
- as a form of information;
- as a media and associated technology;
- as a contrast and synthesis of analogue and digital worlds;
- as a technical and compositional skill;
- as shared and externalized memory and cognition;
- as social mechanism of awareness and agency of social change;
- as method and material of appearance and behavioral prototyping;
- as a special and distinct form of the digital commons;
- as mechanism of identity; and
- as a key component of professional presence and portfolio construction.
These roles are, in fact, the modular content of a course I teach called Digital Imagery as Meaning and Form in HCI and Designthe title of this article. This list is only the beginning of an inventory of the realized and potential roles of digital imagery in visual thinking.
As a matter of scholarship, this discussion is limited not only by space, but also by personal experience and exposure. There are many writers, designers, and image makers who engage in values-rich sense-making and meaning-making. My account is necessarily incomplete and should be taken only as indicative of a vast array of scholarship and creative activity.
The contemporary thinkers whose work enriches HCI by integrating design theory, ethnography, critical theory, feminism, philosophy, cultural studies, ethics, and associated domains include S. Bardzell and J. Bardzell, DiSalvo, Dourish, Forlizzi, Gaver, Nelson, Odom and Pierce, Sengers, Stolterman, Wakkary, and Zimmerman, to name far too few. The scope of this article as a short interactions piece does not permit complete referencing hereplease refer to the ACM Digital Library to find the work of these authors and related work. Also, kindly note that I am not discounting HCI’s rich origins in computing and psychology, but rather suggesting that these writers have augmented our discourse about HCI and interaction design.
The notion of ontological design owes to Heidegger and appears early in HCI in work by Winograd and Flores . In design literature, Willis has provided an account of ontological design . The notion of transdisciplinaritywhich is a call for transcending, rather than combining, disciplinesis described in Max-Neef , and by Blevis and Stolterman for the context of HCI . HCI has a tradition of authors who engage values-rich notions of design, including Friedman , Nardi , and Thomas .
In the tradition of Lewis Hine’s social activism through imagery, several contemporary image makers are notable, including Burtynsky , Jordan , and Menzel . Kingwell  provides rich interpretations of Burtynsky’s work in particular.
The notion of meaning and values in design is taken up by Alexander , Cross , Schön , as well as Margolin , Papanek , Tufte, and many others, including those listed here as contemporary thinkers within HCI and interaction design. G.K. VanPatter in particular has built a consultancy based on principles of sense- and meaning-making in the professional design-planning sphere. Most other reputable design-planning consultancies also focus on sense-and meaning-making over style and form.
Closing notes and Curatorial Directions
Appended to this article are three examples of contributions to the Visual Thinking Backpage Gallery. Each reflects a genre of the use of digital imagery, which we may consider inclusively within the theory, research, and practice of HCI and interaction design.
In A Matter of Diversity: Faces Imitating Electrical Plugs Imitating Faces, image contributors Hongyuan Jiang and Xiuchai Xu use digital imagery to create a humorous conceptual link between facial and hand expressions and the form of electrical plugs in various countries. To create this composite image, Jiang and Xu directed the models to adopt a particular pose, without telling any individual model how the image would be juxtaposed with a particular electrical plug or with other pairings of models and electrical plugs. The resulting image not only provides a conceptual link between human expressions and electrical devices, but it also in its methodology situates participantsthe models, in this casein a relationship between distinctive individual affective expression and otherwise neutral forms. The models’ individual expressions distinguish them as unique individuals independent of country-specific origins symbolized by the electrical plugs, signifying the issue of diversity.
In Clockwork Moths, Shad Gross illustrates not only his considerable pyrotechnic Photoshop skills, but also provides an image that deeply pushes the limits of the modern digital commons. The work is composed mainly of materials provided by Gross, himself, but is also derived from parts of images licensed under creative commons and other forms of relaxed copyright. Some of the images used in the composite image may only be used in derivative work, some of the images may be used for derivative work with attribution, and some of the images may be used for derivative work only if others are free to use the resulting imageas is the case here with this Backpage Gallery image.
In A Matter of Accessibility: Caution, I contribute my own image, which is simply a whimsical juxtaposition of simple elements captured opportunistically at a science center in Ontario. The sign in the lower right advises caution in both English and French. This caution sign contrasts with the vinyl appliqué sign denoting gender, which in its badly damaged state unintentionally advertises the potential consequences of not being cautious enough.
Expect to see a Visual Thinking Backpage Gallery image in forthcoming issues of interactions on the inside back page, and please see the sidebar for how to contribute your own images.
5. Friedman, B. and Kahn, Jr., P.H. Human values, ethics, and design. In The Human-Computer Interaction Handbook, J. A. Jacko and A. Sears, eds. L. Erlbaum Associates Inc., Hillsdale, NJ, 2002, 1177-1201.
7. Truthtable; http://www.truthtable.com/
Eli Blevis is an associate professor of informatics in the Human-Computer Interaction Design program of the School of Informatics and Computing at Indiana University, Bloomington. His primary area of research, and the one for which he is best known, is sustainable interaction design. His research also engages design theory, digital photography, and studio-based learning.
The images used here are: BronzeCopper0027 (Texture: #11578), Insects0001 (Texture: #18081). Insects0007 (Texture: #19473) from www.cgtextures.com; Christina Rutz www.flickr.com/photos/paparutzi/252725115/; Steve Jurvetson www.flickr.com/photos/jurvetson/17945646/; User “history_aficianado” www.flickr.com/photos/history_aficionado/532658724/; User “timlewisnm” www.flickr.com/photos/gozalewis/3256814461/.
Figure. The photographs of Lewis Wickes Hine are ontologically designed. His photos, like this 1908 image showing a young spinner working a cotton mill in North Carolina, inspired the enactment of child labor laws in the U.S.
Sidebar: The Visual Thinking Backpage Gallery: Call for Participation and Contributions
The purpose of this sidebar is to invite broad participation in a new interactions section we call the Visual Thinking Backpage Gallery, which will be presented on the last inside page of each (or every other) issue. Each backpage gallery installment will consist of a single page. The idea of the backpage gallery is to highlight and promote visual thinking, using the medium of still, possibly post-processed digital imagery in keeping with the role of interaction design as a design discipline.
You are invited to submit your best visual-thinking images to the editors-in-chief or to the curator:
Form of Contributions
We will select the best images for inclusion in future Visual Thinking Backpage Galleries. Please provide:
- a high-resolution image (preferably 300dpi)
- a title for your image
- your name and the name of any co-contributors (affiliations are unnecessary)
- optionally, a short description of the genre relating the image to design process
Your image may be constructed or modified in image-processing applications. It must be of professional quality and you must be the author and sole copyright holder of all materials used in your final image. Appended to this article are three examples of images that serve as models for the possible range of contributions.
©2011 ACM 1072-5220/11/0900 $10.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2011 ACM, Inc.