XXVI.5 September-October 2019
Page: 64
Digital Citation

Bridging the gap between material science and human-computer interaction

Isabel Qamar, Rainer Groh, David Holman, Anne Roudaut

back to top 

Many interactive devices such as laptops and mobile phones currently have static, planar shapes that are arguably not particularly adapted to the user. Recent developments in display and material technologies have enabled explorations into morphing devices [1,2] that can provide improved affordances for human interaction. From interactive spherical displays, to mobile phones that bend to notify a user of an incoming call, to pneumatic interfaces that expand to become exoskeletons or couches, there are many recent examples of shaped interface design. This transition from flat, planar shapes to morphing interfaces requires human-computer interaction (HCI) practitioners to learn about and adapt the advances made in material science and quickly apply them to shaped devices in an accessible manner.


The implementation of shape-adaptive interactive systems within HCI, however, is still far from having been achieved. The tools and methods that could have a significant impact on developing such systems tend to be confined to their industries (e.g., automotive or aerospace), are expensive, and are designed to support large-scale systems, making them inaccessible to researchers who have little to no expertise in material engineering. These industries' approach to product design also tends to differ in some obvious ways from that of HCI. For example, manufacturability is of significant importance to material engineers, with products often going through many design and development cycles with stringent requirements before entering production. This can often span years, or even decades. In contrast, HCI researchers have much shorter time frames to develop low-fidelity proof-of-concept prototypes, where human interactions, both physical and those involving cognition and perception, require particular consideration. This difference in approach and requirements is a significant factor in the disparity between these fields. A key question remains: How can the methods and processes in material science be harnessed by HCI researchers?

back to top  Insights


Here we discuss how the evolving relationship between HCI and material science can be framed and why synergies between the two fields are critical for the design of shape-changing devices. Our goal is to create a road map for designers who want to learn more about advances in material science and use them for the design of shape-changing interfaces. To achieve this, we began with a comprehensive and highly cited paper by Thill et al. [3] that presented a review of shape-changing concepts for aircraft morphing skins. Many of the ideas and concepts covered in this article are relevant to the HCI community. For example, shape-memory alloys (SMAs) have been used to actuate both morphing wings [4] and interactive surfaces for the design and visualization of physical forms [5] (Figure 1); elastomers have been employed in wing skins and in wearable devices and flexible electronics [6] (Figure 2); and origami and kirigami structures have been implemented in both space applications [7] and foldable devices, clothing, art, and furniture (Figure 3). We propose using the same categories of shape-changing mechanisms, namely stretchable structures, deployable structures, variable-stiffness materials, and shape-memory materials. By doing so, we not only highlight the existing links between fields but also introduce researchers to new forms and fabrication methods to enable shape change that could inspire the next generation of interactive shape-changing interfaces [8].

ins03.gif Figure 1. Shape memory alloys, which have been used to alter the shape of airfoils [4], have also been employed to create interactive surfaces that can be reprogrammed for real-time computer modeling of objects and spaces, and the construction of adaptable interfaces [5].
ins04.gif Figure 2. Elastomers, which have historically found their way into balloons, have more recently been used in the development of stretchable electronics [6] due to their low elastic modulus, which enables them to easily deform (i.e., strain) up to 1,000% of their original length.
ins05.gif Figure 3. The ancient Japanese art of cutting and folding (kirigami) found in pop up cards, for example, has also been applied to the design and fabrication of shape-morphing kirigami honeycomb structures for engineering applications [7].

back to top  Shape-Changing Technologies

Before discussing potential synergies between HCI and material science, we should first summarize the various types of shape-changing technologies:

HCI researchers have much shorter time frames to develop low-fidelity proof-of-concept prototypes, where human interactions require particular consideration.

  • Elastomers: Highly stretchable polymers (e.g., silicone) that can be found in seals and adhesives, gloves, tires, toy balloons, rubber bands, shock absorbers, and molded, flexible parts. More recent applications include soft robotics and stretchable electronics.
  • Auxetics: Materials with a negative Poisson's ratio (v); that is, they become wider when stretched and narrower when compressed as a result of the spatial arrangement of their internal architecture rather than their material properties. They are excellent shock absorbers and have also found their way into shoes, sculptures, and clothing due to the ability of these structures to physically realize complex curvatures.
  • Rollable: Structures that can be rolled away for storage and transportation and deployed when required. Everyday examples include roller blinds, garage shutters, and fabrics, while more advanced applications include the packing and deployment of lightweight solar sails and rollable carbon-fiber satellite booms.
  • Foldable: Structures (e.g., origami) that are easily deployed into 3D and flattened into 2D for storage and transport. Applications include packaging and containers, biomedical devices, and sandwich-panel cores.
  • Inflatable: Low-weight structures that can be inflated with gas and can conform to almost any shape. As a result of their low production cost, high strength, and ability to pack into small volumes, they have been popular choices for airbags, vehicle wheels, furniture, inflatable boats and buoyancy systems, soft robotics, and medical treatments.
  • Anisotropic: Refers to different material properties (e.g., stiffness) and therefore different morphing capabilities along different axes. Natural examples include wood and bone; man-made examples include corrugated cardboard and fiber-reinforced composites.
  • Multi-stable: Structures that undergo large, rapid deformation between multiple stable mechanical shapes. The Venus flytrap is a well-known natural example. Significant research has been conducted into bi-stable composite laminates that can snap from one shape into another for morphing wing applications.
  • Shape memory: Shape-memory alloys (SMAs) and shape-memory polymers (SMPs) exhibit a shape-memory effect due to their ability to change stiffness as a result of externally applied stimuli such as heat, UV, or moisture. SMAs have commonly been used as actuating mechanisms for HCI prototypes; SMPs have found applications in self-tightening sutures, morphing wings, deployable structures, and self-healing materials.

back to top  Bridging the Gap Between Fields

Our paper presented at CHI 2018 [8] highlighted that some shape-changing technologies have been widely adopted in HCI research (such as SMAs, foldable and inflatable structures, and elastomers). There are, however, several shape-changing mechanisms that have not found their way into HCI research in any significant way, including multi-stability, anisotropy, and auxetics. We believe this can be attributed to two key factors: 1) a lack of awareness and understanding of shape-changing technologies and their material characteristics by HCI researchers, and 2) a lack of availability of equipment, materials, and lab space for HCI researchers to support their work. SMAs, for example, are readily available online and require limited expertise. In contrast, the complex mathematics behind auxetic materials can make their fabrication challenging. Multi-stability and anisotropy have not been widely adopted within HCI, potentially due to an absence of appreciation for these mechanisms and the methods required to implement them. These are also relatively new fields in material science and work is still required by the community to mature the technology.

While the applications in material science and HCI may differ, the advantages and limitations of each of the technologies are the same, highlighting how these fields can benefit from working together.

Another consideration is that while both HCI and material science are multidisciplinary fields, their approaches to research are markedly different. For example, as mentioned earlier, manufacturability is an important factor in the design and development phases in material science, and projects often go through many development cycles before they are physically realized. HCI researchers often develop proof-of-concept prototypes in much shorter time frames due to the need to iterate on the design of these prototypes and to put them into the users' hands in order to gather precious empirical data. This difference in requirements and design approach to research is perhaps a key factor in the gap between these two fields.

The requirements of the shape change also play a large role in the choice of mechanism adopted for a particular application, including performance (e.g., strength and robustness), power consumption, actuation capabilities (e.g., maximum displacement and speed), and shape-change resolution [2] (e.g., granularity, curvature, strength, and speed). Deployable and stretchable structures, for example, can achieve large changes in area and/or volume; however, they may lack strength and robustness. In contrast, anisotropic and multi-stable structures tend to be more robust yet are more limited in the degree of shape change. SMAs are easy to use and can achieve multiple shape configurations; however, they are typically slow to respond to input, are capable of providing only a low actuation force, and require high power consumption. In contrast, multi-stable materials can achieve rapid deformation under little force but are limited in the number of shape configurations they can achieve. While the applications in material science and HCI may differ, the advantages and limitations of each of the technologies are the same, highlighting how these fields can benefit from working together.

back to top  Next Steps For Better Synergies

To accelerate the capabilities of HCI researchers in developing new shape-changing devices and to improve the interactions between different fields, we believe the following steps are vital:

Common language/syntax. Defining a common language between different research fields is critical in enabling universal understanding of terminologies and scientific methods. For example, Young's Modulus, strain, and fracture toughness are all well-known terms in material science that perhaps may not be familiar to researchers in HCI. Likewise, affordance, expressivity, and user evaluations are terms that material scientists may not be accustomed to in the context of product design. Our first step in this direction was to propose a literature review of morphing materials for the design of shape-changing interfaces, which was presented at CHI 2018. This paper provided a starting point, introducing HCI researchers to new syntaxes and terminologies within the field of shape-changing/morphing materials. The main contribution was a review of developments in shape-changing material science to establish a baseline literacy and to make recent work from material science available to the HCI community. This concept could be expanded to include scientific methodologies such as materials testing, computational analysis (e.g., finite element analysis, FEA), microscopy techniques, material processing, and manufacturing processes, all of which could accelerate scientific outputs from HCI.

Online platforms. By creating online platforms, researchers within HCI can access recent developments in other fields relevant to their work, to share their own outputs and to express what is needed to accelerate their research, for example, in terms of hardware manufacturing or material requirements. Furthermore, these platforms could provide a source of inspiration to the maker culture in terms of material design and manufacturing, enabling them to replicate material science outputs in a more affordable and accessible manner, which can feed directly into HCI research. As a first step in this direction, we uploaded the content of our CHI 2018 paper to www.morphui.com to improve awareness of outputs from each of these fields.

Software-design tools. To implement these materials into their own work, HCI researchers need a platform that provides them with an awareness of the behavior and characteristics of these materials/mechanisms (e.g., through simulation). An important step would be the creation of software-design tools to increase the accessibility of material outputs. These software-design tools would not require an in-depth understanding of the science behind such materials and would be developed by engaging both material engineers and HCI researchers in order to understand the needs and requirements of all participants.

Hardware-design tools. Creating hardware-design tools to enable the use of devices such as printers can facilitate the reproduction of material science outputs in a more accessible manner, therefore enabling researchers to quickly create and explore new interaction techniques. This could, for example, involve an assessment of current manufacturing techniques within the automotive industry and the adaptation and simplification of such processes (where possible) for use by any individual with limited resources.

back to top  Conclusion

While much of the development within HCI to date has been software based, an increasing number of projects are embedding off-the-shelf electronics for hardware prototyping. In order to move away from prototypes and toward end-use products, there is a need to go deeper in our understanding of the material itself. This requires a close relationship with current material engineering developments in order to access breakthroughs in new materials and combine them with interactive components. As the borders between fields of science are starting to fade, we need to rethink how we collaborate toward new research goals. Perhaps the greatest means of fostering synergies between these two fields is taking the leap and interacting with colleagues across disciplinary boundaries just for curiosity's sake.

back to top  References

1. Alexander, J., Roudaut, A., Steimle, J., Hornbæk, K., Alonso, M.B., Follmer, S., and Merritt, T. Grand challenges in shape-changing interface research. Proc. of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2018, Paper 299. DOI: https://doi.org/10.1145/3173574.3173873

2. Roudaut, A., Karnik, A., Löchtefeld, M., and Subramanian, S. Morphees: Toward high "shape resolution" in self-actuated flexible mobile devices. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2013, 593–602. DOI: https://doi.org/10.1145/2470654.2470738

3. Thill, C., Etches, J., Bond, I., Potter, K., and Weaver, P. Morphing skins. The Aeronautical Journal 112, 1129 (2008), 117–139. DOI: http://dx.doi.org/10.1017/S0001924000002062

4. Strelec, J.K., Lagoudas, D.C., Khan, M.A., and Yen, J. Design and implementation of a shape memory alloy actuated reconfigurable airfoil. Journal of Intelligent Material Systems and Structures 14, 4–5 (2003), 257–273. DOI: https://doi.org/10.1177/1045389X03034687

5. Coelho, M., Ishii, H., and Maes, P. Surflex: A programmable surface for the design of tangible interfaces. CHI'08 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, 2008, 3429–3434. DOI: https://doi.org/10.1145/1358628.1358869

6. Wessely, M., Tsandilas, T., and Mackay, W. Stretchis: Fabricating highly stretchable user interfaces. Proc. of the 29th Annual Symposium on User Interface Software and Technology. ACM, New York, 2016, 697–704. DOI: https://doi.org/10.1145/2984511.2984521

7. Neville, R., Scarpa, F. Design of shape morphing cellular structures and their actuation methods. Proc. of the 26th International Conference on Adaptive Structures And Technologies. 2015. DOI: http://dx.doi.org/10.13140/RG.2.1.4530.1527

8. Qamar, I.P.S., Groh, R., Holman, D., and Roudaut, A. HCI meets material science: A literature review of morphing materials for the design of shape-changing interfaces. Proc. of the 2018 Conference on Human Factors in Computing Systems. ACM, New York, NY, USA. DOI: https://doi.org/10.1145/3173574.3173948

back to top  Authors

Isabel Qamar is an EPSRC Research Fellow in Human-Computer Interaction at the University of Bristol, U.K., and at CSAIL, Massachusetts Institute of Technology. Her Ph.D. within the Bristol Composites Institute developed 3D-printed vascular networks for self-healing materials. She now researches shape-changing materials for human interaction. Isabel.Qamar@bristol.ac.uk

Rainer Groh is a Royal Academy of Engineering Research Fellow based at the Bristol Composites Institute of the University of Bristol. He is currently researching the application of fiber-steered carbon composites for lightweighting of space structures. He is also interested in using instabilities to imbed shape-changing functionality in structures. Rainer.Groh@bristol.ac.uk

David Holman is a research scientist at Tactual Labs (www.tactuallabs.com). He holds a Ph.D. from Queen's University and was a member of the Human Media Lab, one of Canada's premier multidisciplinary media laboratories. He designed and prototyped next-generation product concepts at Intel Corporation. David.Holman@tactuallabs.com

Anne Roudaut (http://anneroudaut.fr/) is a senior lecturer in computer science and currently leads the Bristol Interaction Group (http://www.biglab.co.uk), which specializes in creating innovative digital technologies. She has contributed seminal work on shape-changing interfaces and her research aims at shaping the software and hardware of these future devices. Roudauta@gmail.com

back to top 

Copyright held by authors. Publication rights licensed to ACM.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2019 ACM, Inc.

Post Comment

No Comments Found