In this article I talk about how to interface—literally—with the human body. Of particular focus here is electrical interfaces that have gained popularity in the HCI community over the past several years. Neural and muscular activities in the human body generate measurable, discriminable electrical signals. In medical diagnostics, measuring and analyzing these signals is common practice, as is inducing muscular activity through electrical signals. With improved signal-acquisition technologies and advances in signal processing, the electrical interface for HCI is getting more attention.
In 1786, Luigi Galvani stumbled across the link between muscle activity and electricity in his experiments on dead frogs (Figure 1). He investigated the phenomenon further in an article published in 1791 about electricity as a vital force of life. These results were sensational at the time but have continued to excite people ever since. Mary Shelley's Frankenstein and movies based on the book hark back to this idea.
Science has moved on since then. Electrical signals measured from the body are important for diagnoses, and electro stimulation is used in various ways for rehabilitation and pain management. In the past 50 years, medicine, sports science, and psychology have made impressive advancements in these areas. When researchers from these fields glance at publications in HCI, they might feel our experiments are closer to Galvani's frog than to the state of the art, and that our visions are more fiction than reality.
Nonetheless, research in HCI that seeks to interface with the human body has its unique place and is vital to the relevance of electrophysiological research in turn. Understanding how to use electrical connections as new interaction modalities, creating interaction techniques and devices based on biosignals, and using physiological information as a means of evaluation is all clearly within our discipline. However, it is essential we understand the broader interdisciplinary state of the art; much of the prior work has been published outside the CHI community. The contributions we publish in our community should have a clear, novel contribution to HCI and should go beyond replicating experiments well known in the other disciplines.
Biosignals are signals from living organisms that provide information about the biological and physiological structures and their dynamics (see  for a more detailed explanation). Here are some examples that are useful in the context of HCI:
- Bioelectrical signals: signals that originate in nerves and muscles.
- Electrical conductance: Electrodermal activity refers to the variation of the electrical conductivity of the skin, in particular electrodermal resistance and electrodermal potential. Galvanic skin response (GSR) measures their combined values by measuring the resistance on the skin.
- Bioimpedance signals: the resistance measured when applying a small alternating current to tissue (typically μA and frequencies above 50kHz).
- Bioacoustic signals: Sounds created by changes in the body, such as blood flow, heart function, ventilation in the lungs, digestion, and movement, can be detected with microphones.
- Bio-optical signals: signals that capture the change in optical properties (even if not visible to the human eye) of an organism or a body part, for example, blood-oxygen saturation based on reflection or pulse rate by change in skin color.
There has been significant research in the medical field on each of these and many more signal types. The acquisition of such signals, their processing, and their interpretation are widely taught in undergraduate and graduate lectures in medicine, medical device technology programs, and sports science. There are also many textbooks on the general subject as well as on individual signals. The signals listed here have in common that they are captured as or converted into a time series of electrical signals that can be further analyzed for their known relationship with physical or psychological states, such as fatigue or anxiety. Here I will focus on passively measured electrical biosignals from muscular and neural activity, as they are a good starting point for experiments in this area.
Electrical signals measured from the human body typically originate from neural or muscular activity. Signals differ in their amplitude (microvolt to millivolt) and in their frequency (for details see ). Here are some examples of signals that can be captured using just surface electrodes:
- EMG (electromyogram): electrical signals generated by muscles. The amplitude is about 50 μV to 5 mV and the frequency ranges from 2 Hz to 500 Hz.
- ECG (electrocardiogram): the electrical signals that originate from the activity of the human heart. The amplitude is about 1 mV to 10 mV and the frequency ranges from 0.05 Hz to 100 Hz. Based on these signals, further measures such as pulse or pulse-rate variability can be calculated.
- EOG (electrooculogram): the electrical signals from the change of the corneo-retinal potential due to eye movement (see ). The amplitude is about 0.5 μV to 5 mV and the frequency ranges from dc up to 100 Hz.
- EEG (electroencephalogram): electrical signals from the brain measured on the scalp with a multichannel data-acquisition device. The amplitude is about 2 μV to 100 μV and the frequency ranges from 0.5 Hz up to 100 Hz. Specific frequency ranges are associated with different stages (e.g., the Theta range is 4 Hz to 8 Hz and changes are observed according to metal alertness; sleep spindles and k-complexes are bursts that occur during sleep phases).
- EP/RP (evoked-potential/event-related potential): electrical signals as responses of the brain to external (e.g., visual, auditory) stimuli. The range in frequency is from 1 Hz to 3 kHz and signals have an amplitude from 0.1 μV to 20 μV.
In medical applications, needle electrodes or implanted electrodes are used. However, for applications in HCI, typically only surface electrodes are appropriate. With surface electrodes the measured signal is always a combination (superposition) of many individual signals (e.g., from many neurons or motor units); hence, further signal processing is required for most use cases .
A good starting point is EMG or EOG, as it is easy to see if the measured data relates well to the actions performed.
The precise position where the electrodes are placed (for EEG on the head, for EOG in the facial area, and for EMG on skeletal muscles) defines which signal or superimposition of signals is acquired. The EMG and EOG signals measured are directly related to the local activity of the muscle or eye. The underlying assumption in EEG is that different brain regions are recruited for performing different tasks or actions (e.g., motor functions, sensory functions, emotional engagement, visual processing, etc.). Hence, the measured activity on the skull reflects to some extent these spatially distributed activities of the brain but is always a combined signal. The number of electrodes (e.g., 32, 64, or 128) is related to the ability to do source separation of cortical activity in the brain. More obvious are the implications of the locations for electrodes on muscles; typically the measured signal relates to the muscle in the area below.
Acquiring information with sensors and interfacing electrically with the human body has become much easier. Fifty years ago, biosignal-acquisition devices were costly and bulky. The operation of these devices was complicated, and their use was targeted at medical diagnostics and research in sports science.
Moving from large analog technologies to digital technologies led to a first wave of consumer devices in the 1980s, mainly with applications in biofeedback. One example of an envisioned commercial input device that was never released was the Atari Mindlink in 1984 (http://www.atarimuseum.com/videogames/consoles/2600/mindlink.html). This headband aimed at acquiring a simple electrical signal from the user's head. It was apparently not a breakthrough, though, sensing mainly muscle activity (EMG) in the forehead (e.g., frowning), and it did not really link to the mind because it did not read EEG signals.
Over the past 20 years, signal acquisition—in particular, analog-digital converters and operation amplifiers—has improved massively and become cheap. Signal-processing algorithms can be efficiently implemented on FPGAs and on cheap processors. Additionally, wireless transmission technologies (e.g., Bluetooth low energy) are widely available and can be easily integrated with the acquisition hardware. Figure 2 depicts the basic architecture of a physiological signal-acquisition system. Creating such a system has become much easier and is even feasible for hobbyists. Many DIY tutorials for biosignal acquisition are available; several open source projects are thriving in this domain (see below for some links).
The lower cost of hardware for acquiring biosignals has led to more people exploring these signals for projects. A good starting point for initial experiments and exploring the possibilities of this technology is a simple and inexpensive Arduino shield (https://backyardbrains.com/products/muscleSpikerShield). Accessible technologies like this have enabled many exciting (and also many boring) projects, ranging from emotion recognition to controllers for impaired users and biosignal-responsive artistic installations.
EMG, ECG, and EOG commonly use stick-on electrodes, basically an adhesive plaster with conductive material in the center. For EEG there are special caps with built-in electrodes. In general these electrodes require a gel (wet electrodes) to make a good electrical connection to the skin. In the past few years, a number of dry electrodes have become available, and there is active research and new products centered around incorporating dry electrodes into tight-fitting garments (e.g., sports bras).
As Galvani's frog experiment showed, applying electrical signals to a muscle will lead to contractions and movement. With advanced control electronics and well-placed electrodes, fine-grained control of muscle movement can be realized. The exciting fact here is that the applied electrical signal acts only as a control signal for the muscle; the signal does not need to provide the energy for the actual movement. Large movements can be initiated by the user with a small battery-driven device.
Sports medicine researches and uses electrostimulation . There are two main applications: a low-frequency modality (<15 Hz, < 10 percent voluntary muscle contraction force, to improve recovery in endurance sports) and the high-frequency modality (> 40 Hz, > 50 percent voluntary muscle contraction force, to improve strength and power). Electrical stimulation is also used in pain management; transcutaneous electrical nerve stimulation (TENS) devices are widely available for therapeutic use. These devices are connected via electrodes to the human body and apply a low-voltage electrical current.
There are also communities that explore the application of signals to the brain. Various DIY tutorials are out there for brain stimulation (electrical and magnetic) as well as for biohacking. This is extremely risky without advanced medical training. Although these technologies have become accessible, there are ethical questions that must be answered before such technologies are applied in HCI.
To start exploring the electrical interface to the human body, it is useful to get started with passively measuring one of the signals. A good starting point is EMG or EOG, as it is easy to see if the measured data relates well to the actions performed. Before experimenting, it is essential to get ethics approval and read up on the physiological concepts, the signal-acquisition principles, and most important the safety precautions. Without appropriate training, experimenting can be dangerous!
Depending on budget, one can acquire a medical-grade 128-channel data-acquisition unit, which offers a lot of possibilities, even for EEG. But to start, an Arduino shield (e.g., Muscle SpikerShield, https://backyardbrains.com/products/muscleSpikerShield; Muscle Sensor v3, https://www.sparkfun.com/products/13027) and simple stand-alone data-acquisition hardware (e.g., https://backyardbrains.com/products/) are providing great opportunities for building functional prototypes and experimenting with EMG control. Bhaskar et al.  present a very simple schematic (Figure 3) that implements an EMG recording circuit that can be connected to the sound card of a personal computer. OpenBCI (http://docs.openbci.com/tutorials/01-GettingStarted) and OpenEEG (http://openeeg.sourceforge.net/doc/index.html) allow exploring EEG and EP using open and well-documented platforms.
The term physiological computing (http://www.physiologicalcomputing.net/) and affective computing (http://affect.media.mit.edu/) are strongly related to using biosignals for human-computer interaction. A recent issue of the ACM ToCHI journal on physiological computing  provides an extensive overview of the state of the art in this field.
In upcoming issues we will look at some of these physiological signals, in particular EEG and electro stimulation, in more detail.
This work was supported by the projects SimpleSkin (European Commission, FP7 FET Open, #323849) and Quantitative Methods for Visual Computing (DFG, SFB-TRR161).
2. Malmivuo, J. Chapter 28: The Electric Signals Originating in the Eye. In Bioelectromagnetism: Principles and Applications of Bioelectric and Biomagnetic Fields. Oxford University Press, 1995. DOI: 10.1093/acprof:oso/9780195058239.003.0028
3. Konrad, P. The ABC of EMG: A practical introduction to kinesiological electromyography. Version 1.0. Noraxon Inc., 2005; http://www.noraxon.com/wp-content/uploads/2014/12/ABC-EMG-ISBN.pdf
6. Da Silva, H.P., Fairclough, S. Holzinger, A., Jacob, R., and Tan, D. Introduction to the special issue on physiological computing for human-computer interaction. ACM Trans. Comput.-Hum. Interact. 21, 6 (Jan. 2015), Article 29. DOI: http://dx.doi.org/10.1145/2688203
Albrecht Schmidt is a professor of human-computer interaction and cognitive systems at the University of Stuttgart. His research interests are at the intersection of ubiquitous computing and human-computer interaction, including large-display systems, mobile and embedded interaction, and tools to augment the human mind. He has a Ph.D. from Lancaster University. firstname.lastname@example.org
Figure 2. The major components in acquiring signals: electrodes, amplifiers and analog signal conditioning (e.g., filtering), analog-digital conversion, digital signal processing, and wireless connection.
Figure 3. A simple circuit to record EMG using the sound card of a personal computer (reproduced from ). ELECTRODE_1 and ELECTRODE_2 are attached to the surface on the human body above a muscle about an inch apart. ELECTRODE_GND is also attached to the human body in a non-active area (e.g., the elbow). The output (X1) is connected to the microphone-in of a computer. For more detailed information, please visit http://albrecht-schmidt.blogspot.de/2015/12/experimenting-with-emg.html
©2016 ACM 1072-5220/16/01 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2016 ACM, Inc.