Authors:
Stefan Schneegass, Albrecht Schmidt, Max Pfeiffer
Muscle movement is central to virtually everything we do, be it walking, writing, drawing, smiling, or singing. Even while we're standing still, our muscles are active, ensuring that we keep our balance. In a recent forum [1] we showed how electrical signals on the skin that reflect muscle activity can be measured. Here, we look at the reverse direction. We explain how muscles can be activated and how movements can be controlled with electrical signals.
Insights
The basic approach of electrical muscle stimulation (EMS) is to first attach electrodes to the skin above a muscle, then deliver an electrical signal via an electrode through the skin to the muscle. This signal imitates muscle activation similar to a nerve signal, which in turn results in an observable body movement. The muscle or muscle group to be activated is selected by the placement of the electrode. The signal (strength, duration, pattern) determines the activation level. In this article we explain the basics, show some examples of recent work, and provide links to hardware and software to experiment with this technology for new interfaces. This article is only a starting point—before you start experimenting, educate yourself properly on human physiology (see the Warning sidebar).
EMS is a technology that is well studied in medicine, biology, biomechanics, psychology, and art. A first encounter for many in the ubicomp and HCI research community was the dinner talk by the artist known as STELARC (http://stelarc.org/) at HUC2k, the second ubicomp conference in 2000. Ping Body was an installation in which ping commands could be sent to his muscles [2]. In the past decade, the human-computer interaction (HCI) community has discovered EMS, with more and more researchers considering its potential for novel user interfaces.
Activating Muscles
EMS stimulates muscles by using weak electrical signals that mimic the body's own electrical signals. These signals are applied via two electrodes attached to the user's body. The signal is transmitted transcutaneously (through the skin) via the electrodes to the nerve and muscle fibers. When placing electrodes on top of a muscle, users perceive a tactile sensation (i.e., a slight tickling) at the electrode's position. Since the motor nerves are stimulated, the targeted muscle contracts. The type of nerves stimulated by the signal depend on the signal's current and impulse width. In general, a current of 20–40 mA with a pulse width from 50–300 ms and a pulse frequency in the range of 1–150 Hz is capable of stimulating the motor nerve fibers. The threshold at which the muscle starts contracting also depends on several factors, such as the exact location of the electrode, the resistance of the skin, and the condition of the muscle. Slightly moving the electrodes, for example, can result in substantially changing the way the signal affects the body, thus changing the effect from tickling to contracting.
Several properties of EMS are unique compared with other types of haptic feedback. First and foremost, the user perceives EMS feedback via various senses. The user perceives the tickling phenomenon via their tactile sense and the actuated movement via their somatic sense. Furthermore, the results of the movement (e.g., lifting a finger or changing the direction of a walk) can also be perceived by various other senses. Second, the amount of energy consumed by EMS is much lower than in other types of haptic feedback, since the actual movement is generated by the user's body—the electric signal only initiates the movement. EMS devices also do not contain moving parts; thus, less attrition occurs than with, for example, vibrational feedback.
History and Current Applications
Luigi Galvani first explored the relation between electricity and muscle movement in the 1790s (see https://stimrx.com/history-of-ems/ for more on the history). Since then, different experiments have been conducted in which electricity was used in therapies. A prominent example is the artificial pacemaker, a device implanted in the chest that stimulates the human heart muscle. Even early on, EMS was used as a therapy to support the rehabilitation of stroke patients. These studies have proven the feasibility of EMS as a tool for therapy. For an overview of applications see [3].
EMS stimulates muscles by using weak electrical signals that mimic the body's own electrical signals.
EMS devices are increasingly being used for health and fitness applications. EMS can be used as a supplement to conventional muscle training. Here, the external electrical signal is used to contract the muscles, thus exercising them. Systems such as the miha bodytec (http://www.miha-bodytec.com/) support a full-body workout in which all muscles can be trained at once. In these systems, a signal is applied to the muscles while the user provides resistance.
Technically very similar are transcutaneous electrical nerve stimulation (TENS) devices used in pain treatment, which also apply an electrical signal. TENS devices deliver a signal that can be felt but that usually does not trigger muscle movement. Applying an electrical signal to painful spots (e.g., the back or the neck) can reduce discomfort.
Examples from the HCI Community
EMS was first introduced to the CHI community by Tamaki et al. in 2011 [4]. They presented the concept of actuating muscles in the arm to control the movement of the user's hand. This output was used to make users play musical instruments. Following this seminal work, several other application scenarios were realized in which the actuation of the arm provided additional benefit beyond simple tactile feedback. For example, Lopes et al. used movements of the arm to communicate how different everyday objects should be used [5]. A spray can makes the user shake it and a cup of hot water prevents the user from touching it to avoid burns. The user's ability to draw complex mathematical functions can be enhanced by applying EMS feedback while holding a pen [6]. The user's arm is actuated in such a way that the pen follows the exact graph of the function.
Combining muscle actuation with contextual information, different types of notifications can be realized implicitly. Twisting the arm so the user looks at their watch reminds them of upcoming appointments [7]. Besides actuating the user's arms, actuating the legs has been shown to be well suited for navigation. By actuating the sartorius muscle, the leg turns outward, which in return results in a change in walking direction [8]. The user is then able to find a specific destination just by walking straight.
There are several examples in which EMS provides sensations to the user other than moving parts of the body. By stimulating the equilibrium, the user's walking direction can be controlled on a coarse level [9,10]. Integrating the feedback into touch screens, for example, allows the augmentation of touch with texture. The user can then perceive properties of a digital object through the tactile sense [11].
A Toolkit for Prototyping EMS Feedback
Prototyping EMS feedback requires dedicated hardware to generate the EMS signal and to control the application of the signal to the user. The Let Your Body Move toolkit (open source hardware and software) supports developers in quickly generating EMS feedback (Figure 1) [12]. The toolkit consists of a hardware control module based on Arduino Nano with a Bluetooth low-energy chip. The control module has two galvanically separated circuits ensuring that only the EMS signal is applied to the user (Figure 2). The control module controls the EMS signal (i.e., reduces the intensity) of an off-the-shelf EMS device serving as a signal generator. Via a simple communication protocol, mobile devices can control the feedback in terms of both intensity and duration. More details, the software, and the hardware schematics can be found at the toolkit's webpage: https://bitbucket.org/MaxPfeiffer/letyourbodymove. Please read the warnings on the webpage carefully before starting your own experiments.
Vision, Potential, and Applications
These applications show how EMS allows us to exploit the human body itself as an output device. EMS thereby enables a more implicit form of output by directly controlling the user's body movements without requiring a high level of attention and perception.
The vision: interfaces that do not require the user's attention and that do not add to a task's cognitive load. Automating human actions, such as walking through a city center, finding the place where you meet friends, or reaching for the umbrella before leaving the apartment, are feasible in principle. In traditional user interfaces, cues are provided as visual, auditory, or tactile stimuli. Users then perceive these stimuli, interpret them, and take action. In contrast, it is assumed that EMS-based interfaces could cut out the cognitive involvement in such everyday tasks.
The ability to control the motion of different body parts creates novel application scenarios. Current questions in research look at how EMS can support motor learning, for example playing a piano or a guitar. The idea is that by providing in-situ actuation, small adjustments to movements can be made, allowing for improved movement patterns.
Fundamental Concerns
From a philosophical perspective, such interfaces raise the fundamental question of who is responsible for an action, as such interfaces interfere with a person's free will.
On a more practical level, one of the core design rules for user interfaces states that the user needs to stay in control of the action. While the loss of control of a digital interface might lead to poor usability, losing control of the body might result in more serious consequences. It is crucial to allow the user to override the signal when necessary and to create user trust in the system. The capability to override the EMS output is essential to avoid harm. Additionally, EMS actuation cannot be applied in all situations; for example, actuating certain muscles while driving a car might result in peril. As a design recommendation, we suggest applying only those signals that normal body movements can override.
Looking at the current state of the art, the attachment of the electrodes and the calibration procedure are fundamental technical challenges that need to be tackled. Attaching the self-adhesive electrodes, gradually increasing the signal strength, and relocating the electrodes makes up an iterative process that takes time. Integrating the electrodes into clothing and enabling an automated calibration procedure are two steps that still need to occur to enable real-world applications outside the lab.
Acknowledgments
This project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement no. 683008). We thank Pedro Lopes for the pictures of the UIST 2016 student competition.
References
1. Schmidt, A. Biosignals in human-computer interaction. Interactions 23, 1 (Dec. 2015), 76–79. DOI: http://dx.doi.org/10.1145/2851072
2. Stelarc. Ping Body. Performance at the Dutch Electronic Art Festival. 1996; http://v2.nl/events/ping-body; video at https://www.youtube.com/watch?v=smdBFx1EYGU
3. Lloyd, T., Domenico, G.D., Strauss, G.R., and Singer, K. A review of the use of electro-motor stimulation in human muscles. Australian Journal of Physiotherapy 32, 1 (1986), 18–30, ISSN 0004-9514; http://dx.doi.org/10.1016/S0004-9514(14)60640-1
4. Tamaki, E., Miyaki, T., and Rekimoto, J. PossessedHand: Techniques for controlling human hands using electrical muscles stimuli. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2011, 543–552. DOI: http://dx.doi.org/10.1145/1978942.1979018
5. Lopes, P., Jonell, P., and Baudisch, P. Affordance++: Allowing objects to communicate dynamic use. Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, 2015, 2515–2524. DOI: http://dx.doi.org/10.1145/2702123.2702128
6. Pfeiffer, M., Dünte, T., Schneegass, S., Alt, F., and Rohs, M. Cruise control for pedestrians: Controlling walking direction using electrical muscle stimulation. Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, 2015, 2505–2514. DOI: http://dx.doi.org/10.1145/2702123.2702190
7. Schneegass, S. and Rzayev, R. Embodied notifications: Implicit notifications through electrical muscle stimulation. Proc. of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. ACM, New York, 2016, 954–959. DOI: http://dx.doi.org/10.1145/2957265.2962663
8. Lopes, P., Yüksel, D., Guimbretière, F., and Baudisch, P. Muscle-plotter: An interactive system based on electrical muscle stimulation that produces spatial output. Proc. of the 29th Annual Symposium on User Interface Software and Technology. ACM, New York, 2016, 207–217. DOI: http://dx.doi.org/10.1145/2984511.2984530
9. Fitzpatrick, R.C., Wardman, D.L., and Taylor, J.L. Effects of galvanic vestibular stimulation during human walking. The Journal of Physiology 517, 3 (June 1999), 931–939.
10. Maeda, T., Ando, H., Amemiya, T., Nagaya, N., Sugimoto, M., and Inami, M. Shaking the world: Galvanic vestibular stimulation as a novel sensation interface. Proc. of ACM SIGGRAPH 2005 Emerging Technologies. ACM, New York, 2005.
11. Bau, O., Poupyrev, I., Israr, A., and Harrison, C. TeslaTouch: Electrovibration for touch surfaces. Proc. of the 23nd Annual ACM Symposium on User Interface Software and Technology. ACM, New York, 2010, 283–292. DOI: http://dx.doi.org/10.1145/1866029.1866074
12. Pfeiffer, M., Duente, T., and Rohs, M. Let your body move: A prototyping toolkit for wearable force feedback with electrical muscle stimulation. Proc. of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, New York, 2016, 418–427. DOI: http://dx.doi.org/10.1145/2935334.2935348
Authors
Stefan Schneegass is a postdoctoral researcher in the Human-Computer Interaction group at the University of Stuttgart. His research focuses on novel interaction techniques for mobile and ubiquitous computing devices, including wearable computing, usable security, and implicit interaction. He has a Ph.D. from the University of Stuttgart. stefan.schneegass@vis.uni-stuttgart.de
Albrecht Schmidt is a professor of human-computer interaction and cognitive systems at the University of Stuttgart. His research interests are at the intersection of ubiquitous computing and human-computer interaction, including large-display systems, mobile and embedded interaction, and tools to amplify human perception and the mind. He has a Ph.D. from Lancaster University. albrecht.schmidt@vis.uni-stuttgart.de
Max Pfeiffer is an associated researcher of the Situated Computing and Interaction Lab at the University of Münster. His research is in human-computer interaction, including situated, ubiquitous, and wearable computing with a focus on haptic feedback and spatial interaction. He did his Ph.D. at the University of Hanover. max.pfeiffer@uni-muenster.de
Figures
Figure 1. Concept of the Let Your Body Move toolkit including the mobile application, EMS device for signal generation, the control board, and electrodes attached to various locations on the user's body.
Figure 2. Circuit of the EMS hardware control module. The signal is generated by a commercial and certified EMS or TENS device, and the output signal is controlled by the circuit shown here.
Sidebar: UIST 2016 Student Competition
The UIST 2016 student competition was centered on EMS as a novel output technology. During the conference, students realized and demonstrated 17 different applications.
More information can be found at https://uist.acm.org/uist2016/contest
Sidebar: Warning!
- Educate yourself about physiology and EMS, and get proper training in the application of electrodes.
- Use only medically approved devices for generating the EMS signal.
- Read the devices manuals in detail before applying the signal.
- Always start with minimal signal strength and slowly increase the current until the user feels a sensation. Repeat this for each muscle and after each relocation of the electrode.
- Ensure that the user is comfortable at all times.
Copyright held by authors. Publication rights licensed to ACM.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2017 ACM, Inc.
Post Comment
No Comments Found