Authors:
Felicia Fang-Yi Tan, Chitralekha Gupta, Dixon Prem Daniel Rajendran, Pattie Maes, Suranga Nanayakkara
The rapid development of smart devices and emerging AI capabilities holds immense potential to transform not only our daily interactions but also our pursuit of life goals. Existing user interfaces, however, often fall short of realizing this potential. Individuals with disabilities, who rely heavily on assistive technologies for daily functioning and independence, face significant challenges due to the limitations of current approaches. Consider a person with visual impairment navigating a supermarket. While traditional accessibility features, such as text-to-speech, offer some assistance with shopping, they often require a cumbersome series of interactions, hindering the user's ability to efficiently gather information and make choices. We argue that the perceived separateness and disconnect between accessibility devices and users hinder the experience of true empowerment.
Why is that? We believe that conversations have been overly focused on user-system coexistence and compatibility (e.g., "I will use this device" and "This is a user-friendly device") rather than integration and augmentation of ability (e.g., "This device is a part of me" and "I have evolved with technology to unlock my full potential"). A paradigm shift is needed from viewing technology as a separate tool to considering it as an integrated extension of the human body, mind, and identity. This new perspective of human augmentation is called assistive augmentation (Figure 1).
 | Figure 1. Transitioning from the traditional user-system compatibility perspective to the assistive augmentation perspective of true integration. |
Insights
→ Assistive augmentation is a subtle yet significant shift, moving from seeing technology as a separate tool to considering it an integrated extension of the human body, mind, and identity.
→ Two dimensions—ability (physical, cognitive, perceptual) and integration (bodily, temporal, identity, sociocultural)— provide a structured framework to fundamentally expand human potential.
Assistive augmentation aims to fundamentally expand human abilities—physical, cognitive, and perceptual—while ensuring that the technology becomes a natural extension of the person. For example, Adrianne Haslet-Davis, a ballroom dancer who lost her left leg in the 2013 Boston Marathon bombing, regained her ability to dance with a dance prosthesis. These types of technologies exemplify the transformative power of assistive augmentations. In this article, we dive into more examples that highlight the idea of technology seamlessly integrating with the user's mind, body, and identity. We also discuss the structured framework of assistive augmentation and propose a set of guiding principles underpinning the design and implementation of these technologies.
Assistive augmentation is defined by two core dimensions: ability and integration. We categorize ability into the perceptual, physical, and cognitive dimensions, and explore various methods for augmenting these capabilities—amplification, substitution, and extension to new modalities. The other core dimension of integration pertains to how these augmentations fit into our lives. Assistive augmentation approaches this holistically, through the bodily, temporal, identity, and social lenses.
Ability
Our concept of ability in assistive augmentation moves beyond the traditional view of one's level of physical limitation. We build upon the positive framing of ability established in ability-based design [1], recognizing that ability is not solely defined by perceptual, physical, and cognitive limitations but can also be situational. Someone with perfect vision might struggle to read a blurry document in low light. In other words, everyone experiences "disability" based on the context they find themselves in. This shift means that assistive augmentation is not just about compensating for what's missing. It can be applied to anyone, at any ability level, with the goal of contributing to a person's overall potential and well-being.
We employ a structured framework to categorize different ability dimensions. Perceptual dimension encompasses how we gather information from the world through our senses (sight, hearing, touch, etc.). It deals with input, with what we gather from the external environment. Physical dimension refers to our ability to perform actions and interact with the world through movement, coordination, and strength. It deals with output, with how we act through our bodies. Cognitive dimension focuses on our mental processes, such as thinking, learning, problem-solving, and memory. These processes allow us to perceive and perform physical actions.
Additionally, we identified three distinct augmentation methods: amplification, substitution, and extension. Figure 2 shows how these methods could augment the three categories of ability dimensions.
 | Figure 2. Categorization of augmentation methods, ability dimensions, and their examples. |
Amplify. Technologies such as eyeglasses and hearing aids exemplify this approach. They improve existing sensory capabilities by increasing their range or effectiveness. Similarly, exoskeletons allow users to lift heavy objects or run faster. Direct stimulation of the brain and nervous system can improve attention with haptic and multimodal rhythmic stimulation, as well as improve memory by targeting memory reactivation during deep sleep [2].
Substitute. This approach explores how sensory input can be redirected to different modalities. Imagine a person with visual impairments using a device that translates colors into sounds, such as artist Neil Harbisson's "eyeborg" that converts different wavelengths of light into audible vibrations. This remapping opens up new possibilities for perceptive channels that are available. On a physical/motor level, bionic limbs provide a replacement solution for people with leg amputations.
Extend. This method explores extension to new modalities, new ways of perceiving the world. Imagine being able to "visualize" electromagnetic fields or experience reality through the senses of an animal using virtual reality simulations. On a physical/motor level, we also can explore new possibilities, including wearable anthropomorphic robotic arms, such as MetaArms [3], which enable the user to interact with the world through additional limbs.
While these approaches to assistive augmentation offer an implementable toolbox for designing future assistive technologies, it is only through the integration with the user's life that we can truly begin to embrace technology as a seamless extension of the self.
Integration
The dimension of integration in assistive augmentation moves beyond bodily aspects, such as body ownership and body agency [4]. While bodily aspects are critical, we consider temporal (how regularly and consistently the augmentation is used), identity (alignment with the user's sense of self), and social (perception within the society and in cultural contexts) aspects as necessary for something to really become a "part of me."
Body integration is one of the key aspects of assistive augmentation. Bionic limbs, when well integrated with the body, can enable movements seamlessly with the user's intentions. Hugh Herr, a professor at MIT who lost his legs in a mountain-climbing accident, can climb, run, and dance even better with his bionic limbs than before. Body integration ensures that the augmentation feels like a natural extension of the user's body or cognitive system, without seeming like an external tool or a device. An assistive augmentation can be considered fully integrated with a user's life not just when it is physically close to their body, but when it has the potential to become an intrinsic part of their daily life and persona. Case in point: Wearing and using glasses becomes embedded in the user's habits and routines to the point where they feel incomplete without them. Glasses are also widely accepted socially, and often seen as a fashion accessory, further integrating them into the user's life. Therefore, in addition to body integration, we identify three other aspects of integration in assistive augmentation.
Temporal integration refers to how regularly and consistently the augmentation is used in daily life. This encompasses both daily usage and occasional usage for longer periods. Prosthetic limbs, for example, would be used in daily activities such as walking and running. High temporal integration is also seen in devices such as AiSee [5], which would assist in regular chores like shopping and become an indispensable part of the user's routine.
Identity integration considers how well the augmentation aligns with the user's sense of self that would be influenced by both intrinsic factors—for instance, the user's personal preferences and background—and extrinsic factors (e.g., the looks and features of the augmentation), and its impact on their daily life. An example is Neil Harbisson's antenna implant, which allows him to perceive colors as sounds. This augmentation integrates deeply with his identity as a cyborg artist, so much so that he insisted on and successfully managed to have the augmentation included in his passport photo.
Sociocultural integration addresses how well the augmentation is perceived within societal and cultural contexts. This includes considerations of societal norms, values, and potential stigmatization. Devices such as hearing aids are increasingly being designed to be discreet and stylish in order to improve social acceptability. The societal conversation around assistive augmentation has evolved from overcoming deficiencies to exploring potential and empowerment. As athlete Aimee Mullins points out, her prosthetic limbs give her the power to change her height and shape her identity—an ability that is not only accepted but also desired by her friends.
Figure 3 shows how these aspects are indicated by users of assistive augmentations. It is important to note that these aspects are interconnected and can influence one another. Strong sociocultural integration can enhance identity integration by providing social validation and acceptance, which in turn can encourage more consistent and regular use, thereby increasing temporal integration. Conversely, a lack of sociocultural acceptance might hinder the adoption and regular use of an augmentation, even if it integrates well with the body and aligns with the user's identity. Understanding and optimizing these interconnected aspects is crucial when designing assistive augmentations for successful integration into users' lives.
 | Figure 3. The integration aspects associated with different assistive augmentations, as indicated by their users. |
Principles of Assistive Augmentation
Stepping back to examine the bigger picture, what are the guiding principles that underpin the design and implementation of assistive augmentations? We've organized our principles into three layers: core, essential, and general design (Figure 4).
 | Figure 4. Three-layer structure of guiding principles for assistive augmentation. |
Core principles of assistive augmentation. Defining ability and improving integration are our core principles. They are essential to addressing our primary objective of fundamentally augmenting human capabilities, thereby distinguishing the assistive augmentation approach from other assistive perspectives.
Define the ability dimension and augmentation method. As detailed in the ability section, we frame augmentation methods as either amplifying, substituting, or extending. We also distinguish between the perceptual, physical, and cognitive dimensions of ability. In doing so, we hope to guide designers in developing focused solutions and empowering users to select technologies that best meet their individual needs and aspirations. When considering assistive augmentation, one should ask: Which augmentation method(s) do I want to use? Which ability dimension(s) do I wish to target? This approach ensures that assistive augmentation is tailored to fundamentally transform the user's unique capabilities, fostering greater independence and empowerment.
Assistive augmentation recognizes that technology should serve as a catalyst for human potential, not merely as a crutch.
Improve integration. Central to this principle is the belief that technology should feel intrinsic rather than extrinsic. The field of embodied cognition suggests that tools and technologies become extensions of ourselves when seamlessly incorporated into our bodily and cognitive functions. Assistive augmentation builds on this concept by considering the regularity of usage, as well as the identity and social dimensions of technology use. A well-integrated assistive augmentation not only minimizes disruption but also enhances the user's ability to perform tasks effortlessly, making assistive augmentation a natural part of their daily life.
Essential principles of assistive augmentation. We now turn to a subset of general principles that are essential to the assistive augmentation perspective. By focusing on the aspects of agency, interference, adaptation, and ethical considerations, we ensure that users are respected and empowered in meaningful ways.
Enhance user agency. Users should feel empowered to make decisions and have autonomy in how they interact with assistive tools, ensuring that they remain active participants rather than passive recipients of assistance.
This can be achieved by effectively utilizing contextual information and interpreting implicit and explicit user inputs. Contextual information is derived from the user's environment or context. Implicit user inputs may result from indirect interactions— this could include gestures, eye movements, and physiological signals that the user does not consciously direct toward the system. Explicit user inputs, on the other hand, respond to direct commands or actions taken by the user (e.g., voice commands, touch interactions, keyboard inputs).
Consider the earlier example of an exoskeleton that enhances the user's ability to lift heavy objects or run faster. Utilizing contextual information, the system could increase assistance when it senses that the user is navigating uneven terrain. It could interpret implicit inputs, such as subtle shifts in weight or posture, to provide seamless support and allow users to directly control (explicit input) the level of assistance. A smart prosthetic hand might automatically modify its grip based on the type of object being handled or on explicit commands from the user's muscle movements.
In line with Ben Shneiderman's human-centered AI approach, there is a need for a high level of human control to build trustworthy systems. This balance ensures that users have both the benefits of advanced automation and the necessary control to feel secure and confident in their interactions with these technologies. By maintaining this equilibrium, assistive augmentations can become more reliable, fostering greater trust and acceptance.
Minimize interference. A holistic design approach ensures that assistive augmentations do not disrupt aspects of a user's life in pursuit of optimizing others. To achieve minimal interference, assistive augmentation should focus on three aspects. First, the user's existing perceptual, physical, and cognitive abilities should not be impaired. For example, a text-to-speech device for a person with visual impairments should provide auditory information without overwhelming the user's ability to process other environmental sounds, ensuring that they remain aware of their surroundings. Second, assistive augmentations should accommodate and align with the user's natural body movements and behavior rather than forcing adaptation to unnatural or uncomfortable patterns. For instance, the exoskeleton should support and enhance natural walking patterns rather than imposing rigid or awkward movements. Finally, assistive augmentations should support rather than inhibit natural human-to-human interactions, preserving the quality and ease of social engagements. For example, would a device worn on the face possibly create barriers to facial expressions?
Ensure adaptability. We recommend that assistive augmentations learn and adjust to the user's habits, preferences, and evolving needs to secure a personalized and efficient experience, ultimately enhancing the user's sense of integration. This also involves harnessing contextual awareness, such as utilizing sensors and algorithms to automatically adjust to changes in the environment.
Uphold relevant ethical and privacy standards. Given the sensitive nature of the data often handled by assistive tools, it is essential to prioritize user privacy and data security. Addressing these considerations not only protects users but will also ultimately enhance the credibility and acceptance of assistive augmentation technologies. We recommend "Ethics of Artificial Intelligence and Robotics," by Vincent Mäller [6], and UNESCO's Ethics of Artificial Intelligence, among other resources, for further insights and expert perspectives on how to effectively integrate ethical considerations and privacy protections.
General design principles. In addition to the core and essential principles of assistive augmentation, we urge designers to familiarize themselves with the general design norms and principles that apply to most technologies. This may include UX principles, such as usability, accessibility, consistency, simplicity, and feedback.
While you might refer to classics like The Design of Everyday Things, by Don Norman, and Don't Make Me Think, by Steve Krug, we recommend reading widely, even beyond the fields of interaction design and HCI, to gather diverse perspectives and ideas. Ultimately, our vision for assistive augmentation hinges on the power of multidisciplinary thinking, which is essential for sparking innovation across fields and developing more holistic and comprehensive assistive technologies.
Conclusion
The hugely powerful field of assistive technology has long focused on helping individuals overcome their limitations. We know that we need to aim beyond that to empower people to thrive in a world that can sometimes be challenging, and even more so for some than others. We have delved into the principles, possibilities, and potential of a paradigm shift. At its heart, assistive augmentation recognizes that technology should serve as a catalyst for human potential, not merely as a crutch. Our task here is to harness this potential thoughtfully, ensuring that technology becomes a natural and empowering extension of ourselves, fundamentally transforming human ability. We hope you can join us in cheering on this vision: We can do more. We can be more.
References
1. Wobbrock, J.O., Kane, S.K., Gajos, K.Z., Harada, S., and Froehlich, J. Ability-based design: Concept, principles and examples. ACM Trans. on Accessible Computing 3, 3 (2011), Article 9, 1–27.
2. Whitmore, N.W., Chan, S., Zhang, J., Chwalek, P., Chin, S., and Maes, P. Improving attention using wearables via haptic and multimodal rhythmic stimuli. Proc. of the 2024 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2024, Article 80, 1–14.
3. Saraiji, MHD Y., Sasaki, T., Kunze, K., Minamizawa, K., and Inami, M. MetaArms: Body remapping using feet-controlled artificial arms. Proc. of the 31st Annual ACM Symposium on User Interface Software and Technology. ACM, New York, 2018, 65–74.
4. Mueller, F. et al. Human-computer integration: Towards integrating the human body with the computational machine. Foundations and Trends in Human-Computer Interaction 16, 1 (2022), 1–64.
5. Boldu, R., Matthies, D.J.C., Zhang, H., and Nanayakkara, S. AiSee: An assistive wearable device to support visually impaired grocery shoppers. Proc. of the A CM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 4 (2020), Article 119, 1–25.
6. Mäller, V.C. Ethics of artificial intelligence and robotics. Stanford Encyclopedia of Philosophy, E.N. Zalta and U. Nodelman, eds. Metaphysics Research Lab, Stanford University, 2023.
Authors
Felicia Fang-Yi Tan conducts HCI research at the National University of Singapore and NYU Tandon School of Engineering. Her research revolves around designing "quietly supportive" technologies that integrate seamlessly into daily life. She leverages experimental research methods to create human-centered systems that foster lifelong learning, health, and well-being. [email protected]
Chitralekha Gupta is a senior research fellow at the Augmented Human Lab in the School of Computing at the National University of Singapore. Her research interests lie at the intersection of audio, AI, and HCI, which includes designing audio generative models for creative and assistive applications. [email protected]
Dixon Prem Daniel Rajendran is a research fellow at the Augmented Human Lab in the School of Computing at the National University of Singapore. His research interests span information systems and HCI, with a focus on design science, game-based learning, gamification, and recommender systems. [email protected]
Pattie Maes is the Germeshausen Professor of Media Arts and Sciences at the MIT Media Lab. She runs the Fluid Interfaces research group. She is particularly interested in the topic of cognitive enhancement, or how wearable, immersive, and brain-computer interface systems can actively assist people with issues such as memory, attention, learning, decision making, communication, well-being, and sleep. [email protected]
Suranga Nanayakkara is an associate professor in the School of Computing at the National University of Singapore, as well as a senior member and distinguished speaker of ACM. In 2011, he founded the Augmented Human Lab to explore ways of designing intelligent human-computer interfaces that fundamentally augment our human capabilities. [email protected]
Copyright is held by the owners/authors. Publication rights licensed to ACM.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2025 ACM, Inc.
Post Comment
No Comments Found