ACM Interactions

In his book Designing Interactions (2006), Bill Moggridge focuses on how to design interactions with digital technologies. That makes sense if you think about interaction design as the practice of designing interactions. However, interactions cannot be fully designed, determined, restrained to a particular form, or fully predicted in the same way that a service can never be fully designed. At best we can design enabling preconditions that might enable or ease a particular form of interaction. In other words, we can design the material preconditions for a particular form of interaction—but we can never completely predict and design the interaction that unfolds. 

As we now move into the era of more physical forms of computing—including the development of the Internet of Things, smart objects, and embedded systems—it is quite easy to see how interaction design is increasingly about arranging material preconditions for interaction. However, that is actually true for any interaction design project. As pointed out by Dourish (2017), computing and information is always a material concern. No matter how abstract we think computing, information, and representations are, they all rely on material infrastructures, ranging from the server halls, to the fiber networks, to the electronics that enable computing in the first place.

From that perspective, interaction design becomes a design practice of imagining new forms of interactions, and then designing as good preconditions as possible to enable those particular forms of interactions to unfold. In my recent book The Materiality of Interaction (2018), I discuss these imagined forms of interactions and how to manifest them across physical and digital material. I talk about this under the notion of a “material-centered approach to interaction design.” Here, one might wonder if this is actually a turn away from our user-centered approach? My answer to this question is a clear no. In the book I propose that a material-centered approach to interaction design follow something I call “the interaction 1st principle,” which is the imagined form of interaction in focus for the interaction design project. I then suggest that in order to manifest that imagined form of interaction in computational materials it is necessary to have a good understanding of what materials are available (ranging from electronics, sensors, and analog materials, to hardware and software) and to know about material properties and how different materials can be reimagined and reactivated in a computational moment. Further, I suggest that a design challenge is how to bring those different materials into composition so as to enable a particular form of interaction. Accordingly, I suggest that a third component here is to have compositional skills to work across a whole range of materials in interaction design projects.

As we now move forward with AI as our next design material, we also need to think about what should be a matter of interaction, and what interactive systems can do for us, autonomously or semi-autonomously. I like to think about this in terms of the design of “scripted materialities”—again with a focus on how to manifest interaction design through material configuration, but here with a focus on how the materiality might change and reconfigure itself over time, in relation to different needs and usage. While a traditional understanding of materials might lead our thinking towards things that are stable and inflexible—just think about the expression “set in stone”—a huge challenge for moving forward will be to not abandon a material-centered approach to interaction design, but rather to reimagine what materials can do—to move from thinking about materials from the viewpoint of enabling structures to thinking about how dynamic material compositions can provide the necessary means for new forms of dynamics and new forms of interactions!

Mikael Wiberg