Tools and Technologies to Facilitate and Improve Design

XVII.2 March + April 2010
Page: 32
Digital Citation

Interaction criticism

Jeffrey Bardzell, Jay Bolter, Jonas Löwgren

back to top 

Criticism is an integral part of the ongoing knowledge construction that is embraced in the more mature design disciplines—architecture, industrial design—and in the arts. Critics interpret, contextualize, interrelate, abstract, and question the artifacts of design to clarify opportunities for designs to improve everyday life and to explore the ways in which designs deliver on this promise. In doing so, they feed an ongoing dialogue between design and criticism, through which knowledge grows for the benefit of practitioners, scholars, and the general public.

Interaction design, in general, does not really accommodate criticism and the role of the critic, with some exceptions in new media [1, 2], HCI [3, 4], and video-game studies. As HCI's interdisciplinary expansion continues to incorporate design, criticism's day is coming. As our work becomes increasingly culturally and socially complex, we will need both the "expert readings" of erudite critics and everyday design "crits" from practitioners to provide the knowledge we need to design.

We expect interaction criticism to emerge as a skilled practice, closely tied to interaction design. Our intention here is to fuel this development by providing an example of what interaction criticism could offer members of the interaction design community.

back to top  Design Case: mæve

Mæve was a presentation of student entries to the architectural competition Everyville held at the 2008 International Architectural Exhibition at the Venice Biennale [5, 6]. Mæve presented the student entries in the larger context of the MACE project, in which existing European repositories of architectural knowledge are connected and semantically integrated [7]. Mæve was developed by the interface design team of the University of Applied Sciences Potsdam, under the supervision of Boris Müller and Till Nagel (see Figure 1).

Mæve is built on an interactive tabletop display and several sets of paper cards, which can be detected when they are placed on the tabletop. When a card is placed on the tabletop display, an information structure appears around the card displaying media files, keywords, and related projects. If other cards are already on the tabletop display, the new card is connected to the existing ones to highlight similarities. Additionally, multimedia files selected from the tabletop display are projected onto a nearby wall. Up to 10 people can stand around the table, browse information, and manipulate the cards on the tabletop.

back to top  Criticism 1: Content Lives in the Interaction

The idea of using metadata to support faceted browsing of a database of exhibits and their connections is certainly nothing new. However, the mæve interface transforms the "contents" in terms of the user's experience. It is easy to imagine a traditional browser-based presentation of the contents of the mæve database, i.e., the student architectural competition entries with pictures, links, search boxes, and so on.

But when you approach the same information through mæve, something happens. The input cards in their material form are closely coupled to the dynamic response of links, images, and text on the table surface. The user's hands are a part of the interaction, as if one were shaping an organic material that is responsive and yet also somewhat unpredictable. In terms of interaction aesthetics, mæve strikes a delicate balance between pliability and autonomy that makes for an enchanting user experience.

This experience is not in some way separated from the "contents"—database records, image files, and text strings—as it would be in a model-view-controller scheme. From the user's point of view, the interaction is the contents. The mæve interaction design shapes the appreciation and appropriation of the architectural competition entries. It offers the user an experience different from that of a traditional screen-based interface to the same information.

A classic example in the same vein is the Visual Thesaurus that premiered on the Web in 1999 as a Java applet; the contents of a conventional thesaurus were presented as an animated network of words and their interrelations. The user would start by typing a word to look up synonyms. Words would drift into view, and if you clicked on one of them, it would become the new center and its synonyms would be introduced. Exploring the network of words and concepts was a fluid journey that would sometimes take you far from the word where you started. In a famous analysis, Khaslavsky and Shedroff characterized the interaction experience of the Visual Thesaurus as "seductive" and analyzed it in terms that also apply to mæve [8].

In the Visual Thesaurus, language itself started to appear dynamic, somewhat volatile, quite unlike the authoritative, canonical demeanor of a traditional paper thesaurus or a conventional Web thesaurus. Again, the interaction experience influenced the perception of the contents.

This is an observation that would seem obvious to any graphic designer, since those versed in visual communication have known for a long time that form and content are inseparable. Interaction design, however, has a different intellectual tradition. There is a strong tendency to think about information as something to be structured and architected in its pure form, then provided to the user through an interface that makes access as efficient and comprehensive as possible. The popular practice of wire-framing—to structure the contents of an information system while deliberately disregarding its appearance and fine-grained interaction behavior—is a useful example here.

We expect interaction criticism to emerge as a skilled practice, closely tied to interaction design. Our intention here is to fuel this development by providing an example of what interaction criticism could offer members of the interaction design community.

Mæve, Visual Thesaurus, and similar examples call into question the assumption that there is such a thing as pure information or content. A careful look at the user experience suggests the premise that form and content, interface and information are inseparable. The combining of form and content is true of every digital application, albeit in varying ways. Some applications are "content heavy," particularly those designed for certain kinds of expert use. The interface may be minimal, and the emphasis is on the expedited delivery of textual or numerical content. Other applications are designed more to create an experience for their users than to convey information. As digital devices are used by more and more people for viewing video, playing games, and participating in social media, the distinction between content and interface is becoming increasingly untenable and unproductive.

back to top  Criticism 2: Interaction Design and the Interface as Performance

For many writers concerned with digital media today, the key quality of the computer is its procedurality. In this view, an application is a program that runs as a set of looping procedures. The user interacts with an application by inputting (typing, moving the mouse, and so on) into this procedure. The user actually becomes a part of the procedure, and the best interfaces are ones in which the user is seamlessly incorporated into the loop. This procedural view is not wrong, but it does little to account for the meaning and appeal of many popular digital forms, including social media.

An alternative to the procedural view is what we may call the performative view. This approach draws on the rich tradition of performance studies in and for other media [9]. In this case the user is not only working in a looped relationship with the program, but is also working through the application to communicate with an audience. Many popular digital forms today are highly performative in this sense: The users project themselves or perform before an audience of friends, acquaintances, or indeed strangers in applications such as Facebook, Twitter, "World of Warcraft," and so on. These applications are successful precisely because they make it easier for users to reinvent their identities in the act of performance.

If there is a spectrum with such single-user applications as word processors and spreadsheets on one end and multiuser social media on the other, then mæve falls somewhere in the middle. It is an interface to a program for database search; users insert themselves into the event loop by specifying searches and following links. This (tight) coupling of user and system is what traditional HCI envisions. For this reason, the Web version of MACE lends itself to procedural analysis. The performative perspective, however, becomes useful in analyzing the mæve tabletop version.

To begin with, mæve operates as a performance space. Mæve was, after all, displayed at the Venice Biennale, a festival in which installation and performance art is featured. The table itself defines a space that accommodates multiple performers and onlookers. Furthermore, projection screens duplicate the table's visual fields and vastly expand the space for a potential audience.

Instead of working alone at a screen, where the user may be the only one who can see the results, mæve invites the user to perform publicly. Here, we compare the configuration of the table and screen to the apparatus of film and to the genre of science fiction film, specifically "Minority Report." In addition, the use of mæve (as shown in the video on the website) suggests other genres or types of performance. Even if only one user is doing a search, the actions are widely visible. The user is cast in the role of performer, in some ways like a performer of a musical instrument, whose search patterns will be visible to others in the installation. In the video we see several users at once; their efforts seem not to be a collaboration, but more of a collective performance in which the cards they throw down form an aesthetically interesting pattern of links. They seem to be performing the role of players of some sort of collectible card game. Their performance is casual and playful, in comparison with the seemingly serious task of searching an architectural database.

Our performance of mæve constitutes an act of self-expression. Whether we try to generate beautiful patterns on it or seek to explore similarities in the use of materials across multiple designers says something about us. It also establishes a certain relationship to the audience. Perhaps others nearby are impressed, intimidated, or excited by the public performance of mæve. This relationship, in turn, will lead to behaviors—avoidance, conversation starting, attempting to take control of the table, and so on.

To reiterate, the key to a performance studies approach is the acknowledgement that interaction occurs between or among people (performer and audience), not just between user and application. The goal of the analysis of interaction as performance is to complement the other approaches to yield techniques that may be useful in interaction design in an era of social and tangible digital media.

back to top  Criticism 3: Constructing the User of Mæve

The word "user" can be understood in two very different ways. We can think of the user either as "the kind of person that uses this application" or as "an actual person who uses this application." The second notion of user involves real people, who can be studied through social science, from surveys and contextual inquiry to cultural probes.

But when we design, we inevitably design with the other notion in mind, a certain kind of user. This hypothetical user, around whom we build personas and scenarios, does not exist in empirical reality and must be constructed out of a combination of available data and a designer's experience and imagination. This hypothetical user is, in a way, built into the resulting design. As a critical strategy, we can explore how designers build the hypothetical user into their designs and how that hypothetical user is "made visible" in the design itself.

Even a passing acquaintance with mæve helps us understand the way it constructs its users. Obviously, mæve assumes that users are interested in architecture. This assumption, in turn, presumes a certain level of knowledge of the field of architecture and its attendant technical vocabulary, theories, important examples, and history. It also assumes that users have a certain literacy with computer-mediated information seeking (e.g., navigation, selection, and evaluation skills). Its aesthetic styling, akin to interfaces in "Minority Report" and other sci-fi films, assumes a user who recognizes and understands science fiction's visual languages.

So far, we have considered mæve from the points of view of its contents, genre, and predominant style. A brief foray into film theory can take us even further. Film theorists such as Baudry have suggested that cinema thrusts spectators into a certain kind of subjectivity [10]. They point out that the movie theater situates viewers in dark rooms, immobilizes them in their chairs, projects large moving images before their eyes, establishes an unchangeable and uninterruptable pace (i.e., viewers can't pause and discuss or challenge what they see), and blurs the lines between photographic representation and viewers' own perception. In doing so, the cinema puts viewers in a dreamlike and even infantile state, and it predisposes them to certain cognitive processes and making them more susceptible to harmful ideologies.

So, what would it mean to ask how mæve imposes subjectivity on users? Mæve shares many characteristics with a film at the cinema: It is experienced only in the short term; it is not a mundane interface of the home or office. Mæve is the sole object of interest in a large public room that is dark and illuminated by screens. Like its cinematic forebear, mæve dominates the room, reducing its users to silhouettes and shadows. Participants are most visible and present when they are interacting with mæve.

Mæve allows for no learning curve; users cannot take it home and get to know it at their own pace. New users are facing a technology with the aesthetic stylings of the future; users are positioned as if they were in mæve's past and needed to catch up. As we have suggested, to use mæve is to be onstage: One's interactions on the table are projected on an even larger screen placed on a wall; they become visible to strangers from an even greater distance. Mæve thus amplifies the perceptual domination of the movie theater with a cognitive one; this advanced and futuristic software system turns a private practice into a public performance.

Though mæve was at least officially designed to support the rational investigation of contemporary architecture, it seems to diminish rather than empower its subjects, both perceptually and cognitively. Its users are thrust into a situation that is in a way prior to and bigger than they are. Mæve's perceptual and cognitive power over users brings to mind the concept of the sublime developed in aesthetics, which refers to phenomena that are on such a different scale from us that they both terrify and thrill us—common examples are lightning and vast mountain ranges. Instead of sublimity on a physical scale, mæve impresses on the technological scale; even its debugging mode is a work of display art (see Figure 2).

back to top  Discussion

What we have done so far is to apply three different perspectives to a common point of departure, and it is clear that mæve can be understood in quite different ways depending on who is doing the criticism. Does that mean that everything is up in the air, that it is all a matter of taste? What would looking at a demo and writing down what you thought of it have to do with the aim of nurturing interaction design knowledge?

The point is that our perspectives are relevant for pushing the state of the art in the field of interaction design. Criticism amounts to expert readings of design artifacts, communicating insights of value for the design field. Part of the expertise in critical practice lies in knowing the body of knowledge in the design field and knowing where the important contributions can be made, as follows:

  • The distinction between content and form is a diehard meme in HCI and interaction design, perpetuated by concepts such as function versus interface and wireframes versus graphics. However, mæve is a particularly good illustration of why the distinction is problematic and why the interaction is the content from the user's point of view.
  • As digital artifacts move out of solitary, task-oriented use situations and into the public and semi-public spaces of everyday life, it is becoming increasingly important to acknowledge that interaction occurs between and among people, not merely between user and application. Interaction design is design for social structures and performative practices, and mæve makes that point in a most emphatic way.
  • Finally, mæve helps us see how interaction design not only serves the empirically established needs of actual users, but also constructs its users by situating them in positions that are defined and bound by the choices they can make in performing their interactions.

The relevance of these points transcends mæve; the arguments are applicable in many situations of contemporary interaction design. The choice of mæve as the topic of our analyses is merely based on our sense that it is a good vehicle for understanding the points made and their significance.

As in traditional design, criticism has its place in a user-centered interaction design process. For instance, we have found it fruitful in the prolonged exploration of a specific design domain to interweave phases of creative production with phases of analytical reflection and criticism, aimed at creating concepts and insights to inform a subsequent creative phase. We have also found such concepts and insights to be useful in teaching settings, where they can provide direction to studio assignments. An example is the concept of pliability, which has grown out of several phases of design and reflection in the field of interactive visualization and which has been used successfully as an abstract goal for student explorations in interactive visualization within the framework of master's level studio classes [11]. Other pedagogical activities aiming at synthesizing design practice and critical practice include teaching criticism to interaction design students by asking them to analyze canonical work in interaction design, to analyze the finished work of their peers, and to analyze their peers' work in progress.

In conclusion, we argue that interaction criticism can offer valuable contributions to the discipline of interaction design. We have indicated and illustrated some of the steps toward bringing those contributions to fruition.

back to top  Acknowledgments

We are grateful to Boris Müller and the mœvedesign team for their permission to publish the images in this article.

back to top  References

1. Johnson, S. Interface Culture: How New Technology Transforms the Way We Create and Communicate. NY: Basic Books, 1997.

2. Bolter, J. and Gromala, D. Windows and Mirrors: Interaction Design, Digital Art and the Myth of Transparency. Cambridge, MA.: MIT Press, 2003.

3. Bertelsen, O. and Pold, S. "Criticism as an Approach to Interface Aesthetics." Proc. NordiCHI, NY: ACM. 2004.

4. Bardzell, J. "Interaction Criticism and Aesthetics." Proc. CHI, NY ACM, 2009.

5. mæve;

6. Nagel, T., Pschetz, L., Stefaner, M., Halkia, M., Müller, B. "Mæve: An Interactive Tabletop Installation for Exploring Background Information in Exhibitions." In Proc. 13th Conf. HCI International, Berlin: Springer, 2009.

7. MACE; "Metadata for architectural contents in Europe."

8. Khaslavsky, J., Shedroff, N. "Understanding the Seductive Experience." Communications of the ACM 42, 5 (1999): 45–49.

9. Schechner, R. Performance Studies: An Introduction. New York: Routledge, 2002.

10. Baudry, J.L. "Ideological Effects of the Basic Cinematographic Apparatus." In Narrative, Apparatus, Ideology, ed. P. Rosen, 286–298. New York: Columbia University Press, 1986 [1970].

11. Löwgren, J. "Toward an Articulation of Interaction Esthetics." New Review of Hypermedia and Multimedia 15, 2 (2009).

back to top  Authors

Jeffrey Bardzell is an assistant professor of HCI/ Design and new media at the School of Informatics in Indiana University -Bloomington. His research foci include aesthetic interaction, user experience design, amateur multimedia design theory and practice, and digital creativity. Currently, he is using theories from film, fashion, science fiction, and philosophical aesthetics to assess dominant constructions of "the user," especially in user experience design, through a comparative examination of humanistic constructions of "the subject." In more applied work, he is developing user experience evaluation methods.

Jay Bolter holds the Wesley Chair in New Media at the Georgia Institute of Technology and is the co-director of the Wesley Center for New Media Research and Education. He also directs the writing program in the School of Literature, Communication, and Culture. He is co-author of Windows and Mirrors (MIT Press, 2003), in which he and Diane Gromala explore the significance of digital art for the digital design community at large. He is also creating a design environment called DART to foster the creation of informal education, artistic expression and entertainment along with Blair MacIntyre and Maribeth Gandy.

Jonas Löwgren is professor of interaction design and co-founder at the School of Arts and Communication (K3), Malmö University, Sweden. He specializes in cross-media products, interactive visualization, and the design theory of digital materials. Löwgren has taught interaction design courses at universities and in companies since the early 1990s; and initiated the influential two-year master's program in interaction design at Malmö University in 1998. He has published some 50 peer-reviewed scientific papers and three books, including Thoughtful Interaction Design (with Erik Stolterman, published by MIT Press).

back to top  Footnotes


back to top  Figures

F1Figure 1. Mæve at the Venice Biennale.

F2Figure 2. The debugging mode of mæve.

back to top 

©2010 ACM  1072-5220/10/0300  $10.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2010 ACM, Inc.

Post Comment

No Comments Found