Sheep Dalton, Holger Schnädelbach, Tasos Varoudis, Mikael Wiberg
Recently there have been a number of commercial announcements about virtual and augmented reality technologies. These announcements present a familiar vision of HCI's future being one of spatially immersed information. While this might seem compelling, it is neither the first nor the largest vision of computing that sees space and spatiality at the core of making information habitable. We would like to argue that at the heart of this and related visions is an implicit acceptance that architecture and the built environment provide a compelling way to make computing comprehensible.
During the past few years, our field has demonstrated a growing interest in the area of "architecture and interaction," not least reflected in this forum . We can also see how this interest in thinking and practicing design across architecture and interaction design has established itself beyond this forum, not only in future-oriented research projects aimed at aligning, challenging, or crossing these areas, but also in terms of how these two areas have coevolved and been increasingly entangled in practice.
Within interaction design, we notice this relationship highlighted in the arrangement of conferences and workshops thematically oriented around smart buildings, the Internet of Things, interactive architecture, spatial interaction design, and ambient environments. Clearly there is a need at the moment for a vocabulary to address this growing area.
While it is interesting to speculate about where this area is going, it is also interesting to observe where we are right now, to summarize just how far we have gone, and to reflect on the origin of this movement as it relates to the history of HCI and interaction design. This article is an attempt to do that—to trace the background and current state of the architecture-interaction connection and to stimulate reflection and debate.
This reflection is timely in relation to our forthcoming book Architecture and Interaction , which fuses contributions from two successful CHI workshops and other work marking the state of the art in this area, creating a reference point for moving forward.
Tracing the origin of the architecture-interaction connection. The notion of space has always been part of HCI/interaction design research. Since the day computing abandoned the command-line interface, HCI has dealt, rather implicitly, with space. From the earliest 2D graphical user interfaces to 3D representations, gaming, and virtual reality, emerging into mobile computing, context-aware computing, urban computing, public displays, ambient computing, tangible computing, and ubiquitous computing, our awareness of space and its role in the interaction process is becoming more distinct. And as computing becomes embedded in our homes, our streets, and our buildings, the need to understand the role of space and architecture in HCI is becoming critical.
At the same time, we can notice how architecture is increasingly engaged with the digital experience. Architects are already introducing digital components into buildings. For example, architects at ART+COM have designed a museum for BMW using complex projections and ambient displays, while Kas Oosterhuis and Ilona Lénárd have used digital projectors to create complex adaptive spaces. Yet these practitioners have had little access to the techniques and methodologies of HCI, potentially compromising the user's or occupant's experience.
With the rise of the graphical user interface and later with the Web-based Internet, HCI evolved by extending its collaboration with those with a graphic design background. Historical precedent suggests that computing will, by necessity, begin to engage with architecture much in the way it did with the graphics community. Yet this can only happen successfully if both sides are aware of their own expert knowledge, have some understanding of the other's expertise, and, finally, have some awareness of their own ignorance regarding the other discipline.
Architecture and interaction—an example of HCI's multidisciplinary history. No matter which HCI textbook we open, we find statements about its multidisciplinary nature. This is also true for the area of architecture and interaction, which has evolved and been explored in the same ways as many other sub-areas of HCI. For instance, when we arranged the workshops at the ACM CHI conferences we noticed how the participants included computer scientists, architects, architectural robotics designers, cognitive psychologists, HCI researchers, and architectural researchers, just to give an idea of the scale of the community that came together. This demonstrates how architecture and interaction needs multiple perspectives in order to align spatial and interactional models of reality.
Moving beyond the merging of two distinct areas—architecture as a structure of HCI. If we scratch the surface of HCI we see architecture as a metaphor buried beneath it. HCI is awash with spatial and architectural metaphors: We have the home button; we navigate to a page; we surf the Web or the information superhighway; we click the back button; we mine information; the website is under construction; we get lost in cyberspace; we follow "trails of bread crumbs" to navigate up to the top level; and software is built by software architects who perform "cognitive walkthroughs." Even Donald Norman's seminal work  is littered with architectural details such as door push plates and shower systems as examples of affordances and cognitive models. It is of little surprise then that cognitively, computing, like architecture, is one of those areas that cannot be wholly appreciated from one perspective. Like entering a new physical space, operating complex software requires learning and exploration, which then form a cognitive model. For complex software and websites, new users behave like new residents of a building or neighborhood; they move beyond initial fixed memorized paths and memorized routes to combine different paths through the software flexibly. Eventually routes and commands become like words in a sentence, almost infinitely interchangeable in pursuit of a goal. Like a pedestrian or a driver, an expert user can navigate through a digital habitat with very little consideration or apparent mental effort. Even the realms of previously two-dimensional interaction spaces do not escape from the potential influence that research into space might bring. For example, there is some evidence to suggest that users who have difficulty navigating space have difficulty navigating websites . This raises questions for HCI. When we talk about being lost on a website or interface, are we talking metaphorically or literally in a cognitive sense? If the answer is literally, what can one of the most established professions dealing with navigation design tell us about the construction of software for navigability? Navigation is an ongoing concern in HCI, as it is in architecture and environmental psychology. As such, it is likely these two fields will begin to overlap to ever-stronger degrees.
For complex software and websites, new users behave like new residents of a building or neighborhood.
Architecture and interaction in relation to ubiquitous computing. Moving beyond the graphical user interface, Mark Weiser's original vision of ubiquitous computing  saw computing receding into the background. By background he meant the fabric of the world around us—including literal clothing but specifically the built environment. In Weiser's article, terms from architecture fill most of the world. His Sal character distinctly moves between rooms to create new contexts. Sal has an urban context of a neighborhood and a home; she navigates traffic to go to work; she shops, buys coffee. She looks through windows, uses offices, reads signs, enters meeting rooms, and leaves items near doors. In short, the very thing that computers were receding into was the architectural structure around us.
Direct descendants of Weiser's vision, smart homes started being built around the turn of the millennium. In a review of smart homes, Chan et al.  reported on 54 papers discussing smart home installations, mostly with a tele-care bias. Significant by its absence in this work is any information about the homes as buildings. There seems to be an implicit assumption of neutrality to the level of naturalness. The built environment that computing was meant to recede into did not come around accidentally; it is also the product of much investigation and reflection, the extent of which is currently unclear in computer science. In contrast, work around proxemics  suggests a dynamic role for space in the interaction process. We see this more complex use of space in architecture as a richer model for HCI to follow.
Moving interaction and architecture forward. Every subject functions by focusing on essential elements while blurring the nonessential. It is no surprise then that computer science and the wider HCI community have essentially disregarded greater comprehension of the elements of architectural design when pursuing the design of screen-based interfaces. When computers occupied rooms and communicated via terminals, this was a very rational simplification. We are now living in a world where computers are beginning to become rooms—thus simultaneously challenging the ideas of the computer as a "box" and the room as only a physically defined space. The future of computing seems very much to be one in which we will inhabit interactive systems more so than use them. The world of HCI began with an error, when the notion of a disembodied consciousness seemed to be natural division. This has been overtaken by work in embodied cognition, suggesting that our minds, bodies, and the environment are all integral to the way we perceive the world. From an interaction and architectural perspective, it is therefore necessary to consider the whole environment as part of our cognitive processes and our everyday lives. Just as the graphical user interfaces of the past required a greater dialogue with the field of graphic design, it seems equally natural that HCI will have to engage more fully with the world of spatial design. We are fortunate that while this realization has evolved slowly in the interaction community, it is being mirrored in the architectural world by a growing interest in the affordances that digital technologies present to architectural designers.
The future of computing seems very much to be one in which we will inhabit interactive systems more so than use them.
We live in a world where information is part of our everyday lives, where we don't dedicate time to "doing computing," where information is ever present and leaves a digital footprint, and where notions of online versus offline have become almost meaningless. As mentioned, we are heading toward a state where we will stop using computers and instead inhabit interactive and information-rich architecture. Many exciting challenges lie ahead. For example: How will we design the architectural interfaces to information and interaction to create relevant inhabitant experiences? How will we give inhabitants access to how their data is being captured, manipulated, used, and stored? What role can architecture play in protecting people's privacy and security? What does an interactively augmented environment mean for how we perceive our environment when we already know that mind, body, and environment co-shape this? Given the large number of signposts in both HCI and architecture, we suggest that the only natural expectation is stronger ties between the worlds of information and the design of physical spaces to address the challenges we now face.
7. Ballendat, T., Marquardt, N., and Greenberg, S. Proxemic interaction: Designing for a proximity and orientation-aware environment. Proc. of the ACM International Conference on Interactive Tabletops and Surfaces. ACM, 2010, 121–130.
Sheep Dalton is a senior lecturer in computing at Northumbria University, Newcastle, United Kingdom. email@example.com
Holger Schnädelbach is Nottingham Research Fellow at the University of Nottingham, United Kingdom. firstname.lastname@example.org
Tasos Varoudis is a senior fellow at the University College London (UCL), United Kingdom. email@example.com
Mikael Wiberg is a full professor in informatics at Umeå university, Sweden. firstname.lastname@example.org
Copyright held by authors. Publication rights licensed to ACM
The Digital Library is published by the Association for Computing Machinery. Copyright © 2016 ACM, Inc.