ACM Interactions

Authors:
Mikael Wiberg, Hiroshi Ishii, P. Dourish, Anna Vallgårda, Tobie Kerridge, Petra Sundström, Daniela Rosner, Mark Rolston

---

 

The theme for last year's ACM CHI 2012 conference—It's the experience—underscores an important shift in HCI research: a move away from a perspective that treats people and computers as two separate and distinct entities toward a perspective that acknowledges how people, computational materials, and even traditionally non-computational materials are coming together as a whole, forming our experiences in and of the world. This shift might enable us to rethink what computing can be about, and, accordingly, what experiences of interacting with computers might include. It also simultaneously prompts us to conceptualize computers not as black boxes, but as yet another design material operating in concert with other physical materials—again, with a focus on what these material assemblages can enable in terms of new user experiences and new practices.

At CHI 2012 we organized a panel titled "Material Interactions—From Atoms and Bits to Entangled Practices" [1]. The panel was specifically arranged to discuss assemblages of digital and physical materials and how these compositions might form and enable new experiences, as well as questions related to how we might conceptualize the inseparability of digital materials, user experiences, and the social context.

Aiming to address the current theoretical discourse in HCI focused on conceptualizing material integrations under the notion of materiality, we realized our panel session was quite timely. In browsing the technical program of last year's CHI conference, we noticed that it contained at least three full papers on materials and materiality, one best paper, one alt.chi talk, two Doctoral Consortium papers, two posters, one interactivity presentation, one video presentation, and one complete paper session explicitly focused on materiality as a way to conceptualize these issues. We sought to contribute to this vibrant stream of research in our field.

  The Need for a Shared Vocabulary and a Multifaceted Understanding

In the panel we acknowledged that although the topic is highly specific, we have developed a wide range of concepts in our field to speak about material interactions, from tangible UIs and physical-virtual systems to the broader concepts of entangled material practices and sociomateriality. We also acknowledged the need to develop a shared vocabulary to speak more specifically about material interactions and materiality, and from the panel discussion it was clear that the framing calls for a multifaceted understanding of this profound challenge to our field. Shifting perspective from how computers can be applied to work as good tools in the hands of their users toward acknowledging the computational as a material is indeed a radical shift in perspective in HCI—a shift from how IT is applied toward a focus on the material character of IT. In exploring this question, our panel both identified and exemplified the diverse theoretical and practical traditions from which conceptual models are drawn—from, on the one hand, a designerly engagement with materials, to, on the other, a more theoretical account of materiality derived from social and cultural studies.

---

Shifting perspective from how computers can be applied to work as good tools in the hands of their users toward acknowledging the computational as a material is indeed a radical shift in perspective in HCI.

---

Consequently, while the panelists agreed that a material perspective is necessary and that it brings with it some interesting opportunities for HCI, they also presented distinct and particular views on how this perspective should be framed, and appealed to different traditions as touchstones for an account of material interaction. Interesting to notice was that the panelists represented the whole spectrum, from a view on material interactions focused on total material integration to perspectives on how the digital can be further integrated in our physical and social world. The concluding statements from each panelist reflect these various perspectives.

For instance, Hiroshi Ishii said, "I see the future in the dynamic physical (and computational) material that can conform, transform, and inform. I call it radical atoms." This statement, anchored in recent publications [2], acknowledges this material lens in which the focus is on the potential of new materials that completely blend the physical and the digital. Following this focus on the potential of these new materials, Tobie Kerridge explained that "novel materials provide opportunities for identifying and sharing issues." While both Ishii and Kerridge set forth this potential in functional terms, Anna Vallgårda added an important aesthetic lens to the discussion in saying, "A material perspective offers richer aesthetics and new forms of interactions for technological designs," pointing out how such a material perspective eliminates the differences between designing with computational technology and other materials such as wood, polymers, or shape memory alloys [3]. More important, however, is that such elimination becomes an invitation to explore the qualities of other material expressions for computational things and thus form a path for more varied and richer material expressions for computational things. Furthermore, a material perspective invites material practices such as those familiar to disciplines including furniture design, fashion, industrial design, and art—practices of giving form to things as a means to achieve an expression and a function [4], thus making interaction design a matter of form-giving, where the computer itself slides into the background. Again, the focus here was heavily on the potential of the material lens but also on how this can enable new experiences of the digital and new experiences of our physical world when computational materials are part of the otherwise physical environment. Vallgårda's position, then, began to bridge from the designerly engagement that lies at the heart of Ishii and Kerridge's work to the beginning of a conceptual framework in which not just materials but also their social and cultural contexts become an object of analysis.

Looking further into digital integration through the term of materiality also presents us with a new theoretical lens, valuable both for design and for understanding computers in social contexts. As explained by Daniela Rosner, "In HCI, thinking with materiality shifts our locus of attention for understanding breakdowns and delays, tracing activity and growth, and revealing possibilities (and limitations) for design." Taking this theoretical perspective even further, Paul Dourish stated: "Materials are irrelevant. Materiality matters." In other words, it is not the "fact" of materials—or the material nature of interaction—that is analytically interesting. Indeed, he argued it would be a mistake to confuse the material with the simply tangible. Rather, it is the forms of materiality at work—the range of material properties at play, questions of durability, fragility, visibility, malleability, deformability, density, heft, and so on—that contribute to the sociocultural considerations, and that further, these perspectives could be brought to bear beyond the domain of "tangible" experience [5].

While the panel did cover this range from new digital materials to important implications for HCI, in applying this theoretical lens, we also had good examples of how design practices increasingly merge the digital and the physical. As described by the industry keynote panelist Mark Rolston from Frog Design, "Since most of the modern computing experience is about the world we live in (people, places, and things), it makes sense to move the user interface out into the material world." In saying so, Rolston pinpointed the importance of a design perspective governed by ideas of literally moving the digital material out there into the physical world. Finally, and in an attempt to slightly challenge this focus on the materials and to add a philosophical twist as to what really constitutes a material, Petra Sundström concluded that "it might not all be materials, but the perspective can help us be more innovative." In claming this, she pointed out an argument further elaborated in Sundström et al. [6] in which the digital material really is to be understood and explored as a material, and a material we need to consider in design like any other. The challenge that these positions express is precisely one of novelty; while some denied Rolston's implication that interaction had ever not been in the material world, Sundström's argument suggests that materiality is nonetheless seen as a source of innovation—the question, of course, being what sort of innovation!

  Experience Materials!

To summarize the discussion from the panel, a material lens offers a number of very different implications for HCI research and practice. For some, this lens repositions the material as the locus of inquiry and, in doing so, returns HCI research to its roots in computational exploration. Focusing on the basic elements of design practice places HCI alongside other materials-centric design disciplines, including industrial design, architecture, the fine arts, or glassblowing—areas that have always highlighted the importance of a close understanding of the materials in play. For others, a material lens introduces new opportunities for designers to think more broadly about what computation can comprise. It positions traditional or mundane materials (e.g., wood and metal) alongside new or novel materials (e.g., shape-changing polymers), expanding possibilities for design. Last, this lens offers new theoretical framings that acknowledge the material conditions and material consequences of design environments. It enables researchers to conceptualize the computational based on its properties and how, through their arrangement, they entangle with social practices in different ways, revealing particularities, surfaces, and temporal flows [7].

A good material understanding thus brings with it a potential for extending the reach of HCI. In applying this lens, we may more effectively explore the digital as a design material and how it might work in composition with other (physical) materials, on the one hand, and examine people's experiences with, through, and of digital materials, on the other. In short, a material lens presents us with a new way of seeing [8], and it brings with it a huge potential for advancing HCI—as a design discipline, as an academic field, and as a potential for new user experiences. The question around which many of our discussions revolved is also whether materiality provides a new ground for consideration of the relationship between HCI research and practice. While such a brief discussion could do little more than provoke the question in the first place, some agreement could nonetheless be struck: It's the experience, and experience matters!

  References

1. Wiberg, M., Ishii, H., Rosner, D., Vallgårda, A., Dourish, P., Sundström, P., Kerridge, T., and Rolston, M. Material interactions—from atoms and bits to entangled practices (Panel at CHI'12). Proc. of CHI'12 Extended Abstracts. 2012.

2. Ishii, H., Lakatos, D., Bonanni, L., and Labrune, J-B. Radical atoms: Beyond tangible bits, toward transformable materials. interactions 19, 1 (2012), 38–51.

3. Vallgårda, A. and Redstrom, J. Computational composites. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. 2007, 513–522.

4. Vallgårda, A. and Sokoler, T. A material strategy: Exploring the material properties of computers. International Journal of Design 4, 3 (2010), 1–14.

5. Dourish, P. and Mazmanian, M. Media as material: Information representations as material foundations for organizational practice. Proc. Third Intl. Symposium on Process Organization Studies (Corfu, Greece). 2011.

6. Sundström, P., Taylor, A., Grufberg, K., Wirström, N., Solsona, J., and Lundén, M. Inspirational bits—towards a shared understanding of the digital material. Proc. of CHI'11 (Vancouver, BC). 2011, 1561–1570.

7. Rosner, D.K. The material practices of collaboration. Proc. of the ACM Conference on Computer Supported Cooperative Work. 2012, 1155–1164.

8. Wiberg, M. and Robles, E. Computational compositions: Aesthetics, materials, and interaction design. International Journal of Design 4, 2 (2010), 65–76.

  Authors

The authors all participated in a panel held at CHI 2012 (Austin, Texas, May 5-10) titled "Material Interactions—From Atoms and Bits to Entangled Practices."

  Figures

Figure. Droplet is a tangible interface that explores the movement of information between digital and physical. Through light-based communication, data can be extracted and presented in a physical form, altering our perception and understanding.

 

©2013 ACM  1072-5220/13/03  $15.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2013 ACM, Inc.