XXIX.1 January - February 2022
Page: 58
Digital Citation

Material meets the city: A materials experience perspective on urban interaction design

Linda Hirsch, Eleni Economidou, Irina Paraschivoiu, Tanja Döring

back to top 

There are many aspects that shape our urban built environment, including human activity, spatial compositions, and materiality. Materials and their affordances and experiences play a particular role in this context, influencing what we perceive and how we interact [1]. In this article, we reflect on the place-material relationship by presenting natural materials and how they may influence people's spatial experience in the urban realm. Reacting to the increasing need to design for whole ecosystems, we present design opportunities to increase awareness of one's surroundings by applying aspects of materials-experience design. Our reflection shows that urban interaction design (UIxD) can benefit from emphasizing asynchronous spatial behavior patterns and activities. We advise integrating materials-experience design into UIxD approaches when designing for shared urban places by considering the limitations of each design direction.

ins01.gif Worn, smooth surfaces of wood benches reveal traces of use.

Material constructs influence and co-shape spatial concepts. In turn, places and their spatial setup determine the meaning, interpretation, and role of materials and what experiences they convey to users. In the past few decades, the main developments in UIxD looked at media facades as public displays or the augmentation of smart buildings toward more-sustainable energy consumption. These applications, though, have little connection to general daily activities and their social dynamics. In comparison, the urban outdoors is still dominated by physical artifacts and their material qualities with limited embedded digital technologies. Questions arise, such as what influence do material qualities and choices have on urban-space design? And what design opportunities and limitations derive from this influence?

back to top  Insights


In UIxD, current design trends deal with community engagement, psychological and physical well-being, and sustainable ecosystems. For each of these topics, interfaces are meant to support, connect, or enable, requiring them to be relatively calm, peripheral, adapted, and embedded into the context. These requirements limit the material choices and challenge designers to rethink and reexplore all kinds of materials and develop concepts toward beyond-human designs.

ins03.gif A shortcut developed by repeated interaction, which reflects the social habits of how this park is used through the natural material change.

Here, we introduce a selection of material types and places that contribute to the natural conditions of different urban ecosystems. We introduce design opportunities for novel spatial interaction concepts and user experiences and give examples of how the different material experiences change the spatial affordance and the perceived activity space per place. Our examples are not exhaustive but rather illustrate the bidirectional place-material relationship, its meaning, and design opportunities and limitations for UIxD and material-experience designers.

back to top  Materials Experience: Presenting Material Qualities for Interaction

When describing material qualities and affordances, one soon realizes that descriptive words, such as durable, rough, soft, or fragile, lead to semantic ambiguity and do not fully or precisely encapsulate said qualities and parameters. Besides embodying perceived qualities at certain times, materials age as a reaction to external forces and usage. They start telling a story of their own, shaping our interactions and experiences [2]. We took note of prevalent material themes based on a prior material collection, which we clustered into the categories ephemeral, reversible, and resilient; seasonal; and traditional and advanced.

Ephemeral, reversible, and resilient materials. In this category are material qualities regarding a material's state, whether permanent, temporary, or insignificant. One established interface material type within the HCI community is ephemerals. These types of UIs refer to interfaces applying materials with intrinsic, limited-life-span properties that are directly applied to the interaction, such as snow, fog, or soap bubbles [3]. Other material examples incorporating the ephemeral aspect include fluorescent, thermo-, or hydrochromic inks and shape-memory materials. These materials, however, are perceived as reversible, as their state change is nonpermanent. For example, thermochromic ink reacts to touch by color change due to the temperature difference. The change fades with leveled temperature, reverting to the original state. By comparison, resilient materials resist change persistently and irrevocably over time. Their weathering adds to the aesthetic experience of their original shape and affordance. A popular resilient material example is wood, one of the most used construction materials due to its versatility and durability.

Environments could make transparent how others used the space, what activities were conducted, and how humans and other types of life moved through it.

Seasonal materials. Materials perceived as natural or organic, such as vegetation, leaves, and green walls, are increasingly popular in interactive architecture. This trend aligns with the growing appeal of HCI research in plant-computer interaction, where research employs plants as both input and output devices, interfaces of nurture, and as inspiration for biomimetic systems [4]. Other natural materials that change state seasonally include elements such as rain, snow, or sand. Footprints, animal tracks, or other remnants of activity imprint on these seasonal materials and are then "erased" by weather. Such changes are typical shape changes revealing other living beings' presence within an ecosystem. Seasonal materials change quickly in open spaces due to weather and human activity. For example, a place freshly covered in snow might show traces of who and what was up early in the morning. But by lunchtime those signs may have dissolved under the movement patterns of the masses. When designing with seasonal, natural materials, decomposition is simultaneously a challenge and an opportunity.

Traditional versus advanced hybrid materials. More-traditional materials, such as metal, roof tiles, and stone, already dominate the urban built environment. We call them traditional because they are established materials in the built sector, appreciated for their durability and stability. However, they also remain mostly noninteractive and inflexible. In contrast, the relatively recent developments in HCI materials urge us to think of computation embedded into existing physical materials, resulting in novel, advanced hybrid materials. With the revolution of DIY materials and collaboration with material scientists and engineers, traditional, prosaic materials are reimagined and enhanced with computational capabilities. One such example is capacitive concrete [5], a new form of interactivity whose redesigned shape invites interaction in various urban environments. Advanced hybrid materials open up exciting possibilities for truly embedded UIxD interfaces for peripheral or calm interactions. The domain allows for citizen engagement and participation on a larger scale by fusing quick-living, digital, and persistent physical materials.

back to top  Spatial Interaction Concepts: Fostering Awareness in UIxD

As indicated above, material types and properties introduce, guide, and determine interaction concepts in the built environment. Nonetheless, the concepts are first driven by the user, stakeholder, and ecosystem needs. Particularly in UIxD, concepts focus on shared, multiuser spaces with little consideration of materials-experience design. Instead, they include environmental, social, and cultural considerations to connect the varying actors and users of an ecosystem and thus enable a sustainable cohabitation beyond the human. Below we present identified needs and design potentials focusing on the development of shared, urban green spaces. These needs and possibilities surfaced during our TEI '21 studio [5], which we then formulated into themes through follow-up reflections by and discussions among the authors.

Location- and time-based traces. Designing for an ecosystem requires higher transparency about the different actors, living beings, and behaviors included in the design considerations. This opens up the design space for location-based, real-time information that reveals the complex, versatile processes and relationships within a shared space. Increased spatial awareness and understanding might lead to a more responsible and considerate cohabitation and thus support the trend toward sustainable spatial design. Environments could make transparent how others used the space, what activities were conducted, and how humans and other types of life moved through it. By revealing traces of the ecosystem, the environment fosters socio-spatial awareness beyond the human perspective. Material traces or patinas of use indicate behavioral patterns, such as cutting corners by creating a footpath through a green area or the worn, smooth surfaces of a bench where people frequently sit. The same applies to animal traces, such as tracks or gnawed wood. The latter in particular represents ephemeral traces that disappear or dissolve quickly.

ins04.gif A hybrid, shape-changing door surface based on flexible textile and thin plywood sheets.

Increasing awareness of one's surroundings. Fostering awareness for places' vividness enables individuals to understand the overall atmosphere of a place, promoting a more intense in-the-moment experience that enriches people's well-being [6]. We see the potential in combining an environment-centered design approach with user-centered design to create more-engaging urban places. This includes following up on questions such as to what extent we should consider and reveal the surrounding ecosystem in UIxD, and what is natural and sustainable for individual users and the environment's fauna and flora. There is a codependency between taking advantage of a more transparent, communicative environment and the designers' increased responsibility for holistically accommodating an ecosystem when it comes to the outdoors.

Individual mindfulness and community building in shared green places. Green, natural areas within the urban realm include parks, community gardens, and water bodies. They serve a recreational and relaxing purpose that requires maintenance and caretaking. These shared public places already allow for increased spatial awareness and a more mindful interaction with oneself and the environment. Accordingly, their spatial concepts often include secluded spots for the individual, positioned near gathering places. Such spatial concepts arrive at giving the illusion of privacy in a shared, public context. In turn, urban green places can also foster a sense of community through shared activities that pursue a common goal, such as a community garden. A place's affordance of individual and social activities contributes to the feeling of being in the moment and increases the awareness of one's surroundings. Accordingly, the types of activities offered at a shared green place contribute essentially to its meaning, perception, and level of interaction [1].

ins05.gif Powder-coated metal tiles used as pixels on a physical display.

back to top  Material-Place Relationship

The examples mentioned above indicate the influence of the bidirectional relationships between materials, activities, people, and places on spatial concepts and designs.

One of these concepts is placemaking, which describes the process of turning a meaningless space into a meaningful place. The place's affordances and activity potential highly influence the process. In turn, these aspects depend on the material choices and properties and how artifacts are positioned relative to one another in space. Combinations of material types allow for different activities and aesthetics. They influence a place's atmosphere, which contributes to its overall meaning and understanding. People build relationships with places based on the experiences they make with and at them, which further reflects on the place's identity and dependencies.

Material-place relationship example: urban water bodies. Water can take any shape and size. Its embodiment defines whether the current runs fast, slow, or stands still. The basin of a fountain, for example, is often built at a comfortable height to sit and relax. It invites us to certain activities because of its spatial position relative to the ground and other objects. In comparison, a change in the water's state can turn a canal into a hockey or ice-skating rink in the winter. This shows the effect that material properties can have on their embodiment and socio-spatial experiences.

Place-material relationship example: public gardens. The spatial plant arrangement in public gardens essentially influences how individual plants grow, shape, and bloom. Some plants, for example, compete with one another or share nutrition, sunlight, and other resources. Accordingly, the spatial conditions of the place can either foster the longevity of vegetation or reduce it. It has a similar effect on other materials and their traces of use. The place's affordances can guide people toward certain artifacts or locations, which, at the same time, would boost their usage and accelerate their aging process.

Picture a shovel stored near a flower bed compared with a shovel stored in a shed. The one near the flower bed is easier to reach and thus might be used often. The shovel in the shed might stay unnoticed and hence untouched for a long time.

back to top  Connecting Material, People, Practices, and Places

Independent of the type of material (intangible, physical, digital, etc.), any interaction or encounter leaves traces of use and changes our surroundings. In the case of physical interaction, these traces have the form of footprints, wear marks, weathering, or indents made by human or other living beings. In contrast, these traces come in passive or active digital footprint form in the digital realm, such as cookies or other traceable activities or contributions. As mentioned earlier, revealing such traces from other-than-human actors in our ecosystem incorporates the potential for a more sustainable and considerate cohabitation within the shared places. However, this also questions the balance of revealing others' behavior and movement patterns and avoiding information overload. How and to what extent should we emphasize such relationships and codependencies? What is the relevant, spatially embedded evidence that fosters understanding and asynchronous communication? And which information should be kept hidden and inaccessible?

In turn, the type of trace and the associated root cause influence the perception and meaning-making of the place-material relationship. Accidentally or unconsciously caused signs of breakage or wear trigger negative reactions, whereas a patina effect can elevate a place as a sign of long-term social interaction. Any material dissolves or decomposes over time, representing a material evolution, and hence changes in spatial setups and affordances. The idea of evolving interfaces is not new. However, they show the need for dynamic, adapting interfaces in the urban realm that shape-shift and evolve. In relation to the materials experience framework [2], UIxD designs are placed on a time continuum considering the relationships between material, people, practice, and places.

back to top  Reflecting on the Materials Experience Perspective in UIxD

Bridging the gap between material-centered design and UIxD enables new perspectives and design opportunities for both fields, particularly as the concept of space is socially and materially constructed [1]. It also increases complexity in the already complex multiplayer, multidimensional construct of the urban realm. It introduces questions, such as to what extent can we integrate a materials experience perspective and what are the interferences between the two design perspectives (i.e., material-centered design and UIxD) that might also exclude each other? Real-world contexts incorporate many limitations and restrictions, including political, cultural, and technological ones. We have not looked at how these might change the material-place relationship and the respective design space. Yet we still see the potential of combining a materials experience perspective with urban interaction design approaches when designing for shared urban places.

back to top  Meaning for Future Developments and Trends

UIxD has been shifting away for some time from media facades, urban screens, and projection displays toward an embodied view of humans and other living beings in urban environments. We believe exploring the relationship between place and material can add meaningful perspectives to current developments in UIxD that point to situated interactions, multisensory experience, privacy, playfulness, and ecological perspectives.

One conundrum of UIxD is balancing visibility with unobtrusiveness. The intrinsic relationship between material and place can help achieve the vision of blended interaction due to the countless possibilities materials open for any specific context. For example, fluorescent or hydrochromic ink can be integrated within environments seamlessly while remaining visible to users. The material-place dyad can also drive new design approaches to multisensory experiences using material properties, texture, and potential haptic interactions.

Data privacy is becoming central to the design of urban interactions. A material-place perspective can be an alternative to the challenges of location-based applications. Embedded systems can collect anonymous data on uses and traces in certain public places rather than the demographic details of who those users are. This can ensure anonymity while also gathering relevant data about how a place is used.

The playful and social nature of urban life is the backbone of UIxD: Urban experiences are often movement-based, spontaneous, and fleeting. How we nurture these important qualities of public space in the context of physical distancing, public anxiety, and safety will continue to be a critical issue for UIxD for months and possibly years to come. Materials can provide forms of safe, place-based interactions, engaging citizens through traces or ephemeral interfaces.

UIxD is increasingly concerned with the undeniable role of urbanization in ecosystems and the well-being of human and nonhuman life. That implies questioning the role of UIxD itself and the part it plays in the degrading of the environment by generating waste through digital interfaces. What better way to set an example than by acknowledging the relationship between places and the natural materials that make them unique?

back to top  Acknowledgments

We thank all those who took part and contributed their expertise during our online TEI 2021 studio "Material Meets the City": Sepideh Ghodrat, Katja Knecht, Franzisca Maas, Vikram Mehta, Sara Nabil, Ville Paananen, and Wayne Wu.

back to top  References

1. Moradi, F. and Wiberg, M. Getting it going: Explorations at the intersection of moving bodies, information technology and architecture. In Architecture and Interaction: Human Computer Interaction in Space and Place (1st. ed.). N.S. Dalton, H. Schnädelbach, M. Wiberg, and T. Varoudis, eds. Springer, 2016.

2. Giaccardi, E. and Karana, E. Foundations of materials experience: An approach for HCI. Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, 2015, 2447–2456; https://doi-org.emedien.ub.uni-muenchen.de/10.1145/2702123.2702337

3. Döring, T., Sylvester, A., and Schmidt, A. A design space for ephemeral user interfaces. Proc. of the 7th International Conference on Tangible, Embedded, and Embodied Interaction. ACM, New York, 2013, 75–82; https://doi.org/10.1145/2460625.2460637

4. Aspling, F., Wang, J., and Juhlin, O. Plant-computer interaction, beauty and dissemination. Proc. of the Third International Conference on Animal-Computer Interaction. ACM, New York, 2016, Article 5, 1–10; https://doi.org/10.1145/2995257.2995393

5. Hirsch, L., Economidou, E., Paraschivoiu, I., Döring, T., and Butz, A. Material meets the city: Exploring novel and advanced materials for the smart urban built environment. Proc. of the 15th International Conference on Tangible, Embedded, and Embodied Interaction. ACM, New York, 2021, Article 66, 1–5; https://doi-org.emedien.ub.uni-muenchen.de/10.1145/3430524.3442701

6. Weijs-Perrée, M., Dane, G., and van den Berg, P. Analyzing the relationships between citizens' emotions and their momentary satisfaction in urban public spaces. Sustainability 12, 19 (2020), 7921; https://doi.org/10.3390/su12197921

back to top  Authors

Linda Hirsch is a third-year Ph.D. student at LMU Munich. In her research, she focuses on urban interaction design, including the exploration of urban materials and the development of design strategies to enrich and revive city history and art. linda.hirsch@ifi.lmu.de

Eleni Economidou is a Ph.D. candidate at the Center for Human-Computer Interaction at the University of Salzburg. As an architect and interaction designer, her research interests lie at the intersection of architecture and human-computer interaction. She is currently pursuing research on architectural interactivity in under-construction educational establishments. eleni.economidou@sbg.ac.at

Irina Paraschivoiu is a research fellow at the Center for Human-Computer Interaction at the University of Salzburg and a senior strategist at AR/VR studio Polycular. Her research focuses on enhancing the lived urban experience through augmentation, playfulness, and embodied interaction. As a practitioner, she has worked in urban planning, participation, and placemaking. irina.paraschivoiu@sbg.ac.at

Tanja Döring is a post-doc researcher in the Digital Media Lab at the University of Bremen. Her research focuses on HCI and materiality, AR/VR, and novel interaction techniques, including tangible and embodied interaction. She applies human-centered design in a variety of domains such as everyday activities, learning, health, and creative tasks. tanja.doering@uni-bremen.de

back to top 

©2022 ACM  1072-5520/22/01  $15.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2022 ACM, Inc.

Post Comment

No Comments Found