Nick Dulake, Eva Hornecker, Daniela Petrelli
To design a product for tangible interaction involves far more than the creation of a form. It requires us to fully understand how that form implements the concept, how it complements the technology, what it makes people do or feel, and how it augments the interactive experience. Over four years, we experimented in meSch with many forms (literal and scaled replicas, abstract concepts, framed devices) and different interactions (holding, wearing, manipulating, moving). This is a conversation about the design of the artifacts that mediated those interactions within different heritage settings. The participants are Eva Hornecker, an expert on tangible interaction, and Nick Dulake, the senior product designer behind several meSch installations. My role was merely to start the conversation, whose aim was to shed some light on the knowledge that feeds the process of designing interactive tangible artifacts. This conversation has been edited.
— Daniela Petrelli
Daniela Petrelli: In summer 2014 we compared two very different forms for exactly the same installation at an open-air archaeological site, the trenches of WWI in the Italian Alps . The soundscape was activated by the presence of a visitor either holding a book or wearing a belt (Figure 1). The interaction dynamics and content were the same; only the form changed. The user study showed that the book triggered the expectation of it being more of a guide to control the soundscape than a device for choosing stories.
|Figure 1. The interactive book and belt, along with Bluetooth speakers, evaluated in the trenches of WWI installation.|
Nick Dulake: The design of a form can be driven by a lot of different factors: What should it look like? Where is it going to be used? What is it trying to get the person to do? The book was designed for a cemetery , not for the trenches. Those two environments are really interesting. The wearable or handheld matches the scale of the environment. A handheld object [the book] would give more of a close-quarters experience: Because the graveyard is more compact, it is more acceptable to hold something, and the book is asking you to look down, at ground level, at headstones that are one, two meters away. With something that is worn as a way of carrying it [the belt], you are dipping in and out of it as a way of using it. In a sense you are separated from it; you are still wearing it but you are not physically holding it. Now, that might mean the person may walk a little bit longer. Being more mobile, the belt expands the range of what that person may do in a much wider and open environment. The impression I've got is that it's a different sense of urgency: If I ask you to hold the book to trigger sound while walking along, then you will be waiting for something to happen—you will expect it. While if I wear it, I am more fluid in how I move in the space because I'll be less anxious about whether it is going to do something or when.
Eva Hornecker: Then there is the effect of something we are familiar with. Form offers you physical affordances in terms of what to do, but there is also the level where you recognize the object and you know what kind of interactions or activity are connected with it. Mike Horn  calls it cultural form. It is not affordance that makes you use a skipping rope to jump; it's because you have grown up in a culture where children do that. So that may be why people associated different things with the book versus with the belt. The book—you read alone or maybe to a child, but it is not something you would share in a larger group or an open space. You expect the story to be in the book and not to come from the environment. So the book may have provided too many associations—it was overloaded.
Form offers you physical affordances in terms of what to do, but there is also the level where you recognize the object and you know what kind of interactions or activity are connected with it.
DP: So, in a way, it is better to design something that is not too familiar.
EH: Indeed, it worked quite well with the Loupe [a mobile phone running an AR app concealed in a wooden frame resembling a magnifying glass; see Demo Hour], maybe because what you are doing is still connected with that notion of seeing through it, to see more detail.
ND: The Loupe is closer to a metaphor, because the interaction starts in a familiar way: You look through it to look closely at something. But this is where it stops. Then you have to learn. Tilting and shaking to move on was not familiar—you wouldn't do that with a magnifying glass [the app follows a trail; tilting displays further content; shaking shows the next exhibit to find; see Demo Hour]. But with the book, everything was familiar: Turning the pages was familiar; the bookmark was familiar. We gave the visitors embedded technology that they didn't even have to know about; it was discrete. So there are two very different things going on here: 1) borrowing a metaphor for the design but then enhancing it with technology in a way that means the user has to learn new mechanics to use it and 2) designing something that is fully familiar.
EH: We can try to anticipate how people will react, but that is hard. This calls for early user testing to figure out if the metaphor is too strong. I think, in some respects, the book was very nice because it and the bookmark really worked well together. Maybe it just did not work in the context of walking around in nature and then having sound coming from somewhere else.
DP: It seems that to use a metaphor is a better way to implement tangible interaction...
ND: But then, why is the other way a lesser way to do it? I'm on the fence here. I don't think there is a good or a bad way. The problem we had with the book is that it was familiar as a book, but it was not a book. By changing the manufacturing of the book, people were looking for something else the book could do. By knowing it had technology inside, they thought it must do something else, although they did not know what. So they tried other things [e.g., to use the bookmark as a remote control], whereas the Loupe clearly has technology in it, even if they may never master it.
EH: This is why the Loupe was quite strong . The form invites the right, intended interaction. Holding the Loupe is very different from using a mobile phone because it has a handle, and probably is much more comfortable to hold over an extended period of time. A mobile phone is quite awkward in your hand; you have to grasp it tightly with your hand wide open. The Loupe has more affordance in terms of pointing it toward other things.
DP: The form changes the behavior of people, then.
EH: Yes, it does not look like a mobile phone, so you do not treat it as one. I've seen it in other tangible interaction projects where a screen is integrated in the design. If it is truly integrated, we don't treat it as a screen anymore—it is just part of that object. Also, if you give people a mobile phone or tablet, people quickly figure out how to get to the Web browser and whatever else. By inserting it in something that looks much more specific, you kind of channel their thinking in that specific direction, so they don't think anymore how to make it a generic device and do something completely different, because you create a mental frame. I think that is one of the strongest things about materiality and tangibles, which is similar to a cultural frame.
ND: The design of the casing of the Loupe meant that it was tamper proof, so you couldn't do anything you were not supposed to do with it. The phone was deeply embedded in the tangible interaction; it was used for its processing power, for its sensors as input, and for its screen as output. As you touched it, you realized you couldn't do anything with it unless you cracked it open.
DP: What do you think is the role of aesthetics in tangible interaction?
ND: There are several aspects to consider when designing for heritage. Take the Atlantic Wall [an exhibition where visitors used smart replicas to activate multimedia content; see Demo Hour]. The replicas were different shapes and sizes, but all were designed to be carried around (Figure 2). Some of the originals were much bigger; the beer stein mug was a liter, and we produced a 70mm high mug, so we played around with the scale to still be hand-size.
|Figure 2. Smart replicas from the Atlantic Wall exhibition.|
Also important was the material quality itself—the textural feel. It had an honesty to it. The replicas were made of similar material to the originals. They were not laminated to extend their lives throughout the six months of the exhibition. They were preserved by producing more replicas. The fragility of the original—its essence—is not lost in the reproduction or redesign. Observing visitors using the replicas, we noticed that they seemed to marry well with the exhibition. The replicas were very tactile—visitors were looking at them and they were part of the experience.
EH: The replicas at the Atlantic Wall were information retrieval objects, objects you carry around and use at the station to get the stories. There was no other digital action associated with it. Actually, I found some other interactions interesting because they put you in a situation and let you experience something from a first-person standpoint going through the motions. At the Feint exhibition on Greek Art, for example, being able to hold the kylix [a decorated wine vessel]—even if it was a replica—enabled you to touch the form and to hold it (see Not and Petrelli's article in this section).
DP: Museums can do more: The Italian Museum of War allows all visitors to handle original objects from WWI in an interactive station (see Demo Hour). A voice tells you to feel the weight and the texture, to observe the manufacturing, to imagine how the soldiers used it. It made the visit special and memorable, forging a connection with those who used the objects. But the media must not interfere with the attention on the artifact, so use audio to induce a sense of discovery and video only later if there is keen interest to know more .
EH: Yes, immersion helps you to relate to it. Something that looks interesting makes you want to interact with it. The TU Eindhoven industrial design group, in particular Caroline Hummels and Kees Overbeeke , talk about the aesthetic of interaction: If you are holding the device, interacting with it, how does the movement feel? Does it make you move in a jerky way or does it make you move in a smooth way? Does it feel nice?
DP: But in meSch we had evidence of strong feelings generated by very simple, plain objects. Take the smart pebble in the Fort Pozzacchio installation (Figure 3). It did not look interesting, but visitors talked about "being drawn into the stories" because they had it in their hand. How is that? How do you design an artifact that people want to interact with?
|Figure 3. The smart pebble from the Fort Pozzacchio installation.|
ND: I designed an object that is interesting to move in your hand. That was the starting point for me because while you can pick up a ball, it's just a ball. You can't do very much with it as you walk around. Whereas the pebble has a lovely arc on one side, a quick radius around the edge, and an arc on the other side so it has some momentum. It has an oval shape; it is like a coin through your fingers, something you can play around with in your hand. The other thing is the material quality itself: It was warm. It was made of polyurethane, so it had some weight—it felt heavier than it looked—and it didn't cool like the rest of the environment so it wasn't something alien to you when you picked it up. It felt friendly. Actually, from my memory, it retained residual body heat in the sense that if you held it in your hand for a period of time, put it down, and picked it up again, it would still have that warmth. It became part of you as you went through that cold space [a bomb shelter dug into the mountain]. For the period of time they used it, it was theirs. It was part of them.
So how do you design attachment? We know that a ball is wrong for this. We know that an arced form is comfortable. We know it allows people to fiddle. They can play with it, and if they play with it they attach to it, and if they attach to it they take it around, so we can start to articulate that thought process. But what we actually want is for people to just go "Oh, that's nice!" because the rest is sort of implicit; we are tapping into their feelings to accept that thing.
What we actually want is for people to just go "Oh, that's nice!" because the rest is sort of implicit; we are tapping into their feelings to accept that thing.
DP: You told me about the making of the Atlantic Wall and Fort Pozzacchio. Tell me about the lamp for Hadrian's Wall (Figure 4; see Demo Hour).
|Figure 4. The votive lamp from the Hadrian's Wall project, My Roman Pantheon.|
ND: You have to look at this in a different way. The lamp is another interaction arc. We are again creating something that fits your hand and has that quality of closeness, but that is heightened by the idea that it contains something precious, a gift for the gods. You have been given three lights from Juno for an offering to the gods, and for the Romans, offering to the gods was massive—it was the core of their society. So I tried to instill that through the design of a tangible device that had this quality of preciousness. The three clear windows on the top provide feedback, so the lamp goes a little bit further than the pebble, which does not give any feedback. You are looking down at it [to know how many offerings you still have], so this top-down view is very important. Going through my head was: Yes, it needs to have a good tactile experience; the form needs to be comfortable to hold. But the notion of cupping it with two hands and looking down at it—what would you see? So those three lights are recessed, protected, but you can still see them. They are accessible; they are part of the thing. The surface is a continuous flowing organic form. I think that is really important, because in an earlier design where there were just holes for the light to come through, it gave feedback, but it did not instill this sense of preciousness, that you have been given something with responsibility.
Maybe I am overrating it, but when people go "This is a very nice object! Oh, the lights go down there..." they are using those words because we have created it that way and we understand it. The lamp is induction charged, so when you take it, it's an immediately endearing device. It has a warmth to it; it has a glow in it; it has a heartbeat. The glow, the flickering, the heat—everything, actually, produces this magical object to hold and to move into the museum and use. So the intention behind the design was to protect the integrity of the story around the Roman religion that we were trying to tell.
EH: Such design really aims at eliciting emotions. Touch is the only reciprocal sense. Maybe it activates a different part of your brain, maybe a hand-intelligence. Touch is the sense where you have to be really close to things, so you make yourself vulnerable. There is at least anecdotal evidence that holding an object, touching it, creates an emotional connection. It's an intriguing thought, but I don't think there is a lot of research done on it.
Nick's description of how he designed those artifacts to fit the outcome of co-designed concepts (see Avram et al. in this section) shows the invaluable contribution that product design brings to tangible interaction. Eva's reflections point to the power that well-crafted objects and well-thought-out interactions have on the people experiencing them. Together, they offer a response to the call for more material, physical experiences as a way to engage with our material culture preserved in museum and heritage sites . — DP
1. Petrelli, D., Dulake, N., Marshall, M.T., Pisetti, A., and Not, E. Voices from the war: Design as a means of understanding the experience of visiting heritage. Proc. of ACM CHI'16 International Conference on Human Factors in Computing Systems. ACM, New York, 2016, 1033–1044.
2. The Sheffield General Cemetery, a Victorian cemetery now an historical city park https://www.gencem.org/
4. Damala, A., Hornecker, E., van der Vaart, M., van Dijk, D., and Ruthven, I. The Loupe: Tangible augmented reality for learning to look at Ancient Greek art. Mediterranean Archaeology and Archaeometry 16, 5 (2016), 73–85.
5. Not, E., Cavada, D., Maule, S., Pisetti, A., and Venturini, A. Digital augmentation of historical objects through tangible interaction. ACM Journal on Computing and Cultural Heritage 12, 3 (May 2019), Article 18.
Nick Dulake is a senior industrial designer at Sheffield Hallam University. He has more than 20 years of experiences in applying a user-centred methodology approach to create innovative design solutions. He uses technical and creative design methods to develop interventions with a focus on digital tangible interactions. firstname.lastname@example.org
Eva Hornecker is a professor in human-computer interaction at Bauhaus-Universtität Weimar. Tangible interaction and museum interactions are among her key research interests. She was a PI on the meSch project. email@example.com
Daniela Petrelli is a professor in interaction design at Sheffield Hallam University. She started researching technology for heritage in 1996. Her other research interests include personal memories, multilingual and multimedia information access, and visual analytics. Throughout her career so far she has received 12 international awards. She was the meSch project director. firstname.lastname@example.org
Copyright held by authors. Publication rights licensed to ACM.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2019 ACM, Inc.