Valentin Schwind, Katrin Wolf, Niels Henze
Computer graphics have made dramatic progress in recent years. They deliver almost perfect renderings that can barely be distinguished from reality. But while improving environmental realism significantly increases user immersion, increasing the realism of virtual characters seems to hinder their acceptance. Highly realistic computer-generated characters in movies, computer games, and immersive applications often evoke negative feelings—even disgust. Consequently, such characters can cause immense financial losses, such as Disney's Mars Needs Moms, which lost about $150 million. Similarly, computer games such as L.A. Noire or Mass Effect: Andromeda were highly criticized for their human characters that don't look quite right. While the film industry aims to create characters for acceptance or empathy, the domains of games and virtual reality (VR) aim to design characters to foster immersion in the virtual world. Observing unpleasant virtual characters is likely to disturb immersion, as is the off-putting appearance of one's own virtual body in VR.
Negative reactions to highly realistic characters had already been observed in the 1970s by Masahiro Mori, a roboticist who noticed humans' negative response to robots or prosthetics that are not quite human-like . Mori developed a graph that describes the non-linear relationship between human likeness and affinity (Figure 1). It suggests that with increasing human likeness, an entity becomes more and more accepted by humans, up to a point where it looks almost real. When the entity looks almost real, it falls into what Mori called the uncanny valley; only when human likeness increases further does the entity become accepted.
|Figure 1. Mori's graph of the uncanny valley , translated and simplified by MacDorman . The graph shows the uncanny valley and the non-linear relationship between human likeness and perceived affinity to an entity. Mori also assumes that movement, as indicated by the dashed line, increases the effect.|
Current research explains the uncanny valley as resulting from conflicting cues in a character's appearance, causing a perceptual mismatch. Humans readily accept unrealistic characters when they are consistently unrealistic, as frequently seen in cartoons. Conflicting cues arise when a character displays multiple levels of realism at the same time, such as unnaturally large eyes in a realistic face. Debaleena Chattopadhyay and Karl F. MacDorman noted that computer-generated models of virtual characters are inherently inconsistent in their realism. This inconsistency is caused by the creation process of a virtual character, as some features are more difficult to sculpt, texture, and render than others. The outcome is unequal levels of realism, which make it difficult to assign a category to the entity, increasing feelings of uncertainty in the observer. This uncertainty elicits unpleasant feelings such as eeriness or disgust . Research indicates that the uncanny valley triggers these feelings in humans to mark a potential threat or the risk of being infected with a transmissible disease. Interestingly, it has been shown that the uncanny valley occurs only when looking at humans and animals [3,4], not inanimate objects .
The uncanny valley poses serious challenges for human-computer interaction (HCI) and humanrobot interaction (HRI). Improved technologies per se are not able to address these challenges when artists have an unlimited number of design choices but no design guidelines to help them avoid the uncanny valley. On the contrary, technologies that can produce highly realistic characters further aggravate the problem, as even subtle designs by artists can significantly increase the uncanny valley's effect. Therefore, we need to think about how it can be avoided. What are some ways out of the valley? And what about the acceptance of avatars, when someone's own body in virtual reality is perceived as uncanny? Recent work aims to provide guidelines to avoid the eerie effects of virtual characters and make them acceptable as real humans. In the following, we provide an overview of some design guidelines developed to prevent the uncanny valley.
Steer clear of atypicalities at high levels of realism. One of the main findings of our work is that atypicalities (strong deviations from the human norm) for high levels of realism cause negative sensations in humans and animals . The negative effects of atypical features, such as unnaturally large eyes or human emotions in realistic animals, are larger than for characters with reduced realism. Consistently rendered realism reduces the negative effects of atypicality and increases affinity, as indicated by the first peak in the uncanny valley. This means that realistic renderings and detailed textures of skin or eyes should not be combined with features that are not human-like. Atypical features at high levels of realism can cause the uncanny valley. You can see this effect in the trailer to the upcoming film Alita: Battle Angel, where the character Alita has very realistic features but unnaturally large eyes (Figure 2).
|Figure 2. Atypical features at high levels of realism can cause the uncanny valley, as shown here in a still from the upcoming film Alita: Battle Angel.|
Avoid "dead eyes." A virtual character's eyes are especially important. Using eye tracking, we found that users fixate on the eyes before they consider other features in assessing a character as real or not real. This supports findings showing that people rate the realism and inconsistencies of human characters mainly based on the realism of the eyes. This also explains why skin makeup does not detract from animacy, in contrast with atypical eyes or the eyes of a dead person. This symptom of "dead eyes" is responsible for making artificial characters feel eerie and strange. Subtle features in the eyes communicate intentions, behavior, and well-being. These factors are essential in the assessment of and affinity for the depiction.
Use stylization and childish features for stylization. Research in evolutionary aesthetics finds that humans respond positively to any aesthetic stimuli experienced as being conducive to survival and reproduction. That is why many designs, even if they are not human-like, can be aesthetically pleasing, as they may include features that suggest fitness. Participants in our studies and trained artists also often use childish features to increase affinity in their character designs. Children and infants elicit protective instincts and feelings of caretaking, which are the origin of social communities and mutual understanding. Humans have these feelings even toward beings not of their kind, which is potentially one reason why young animals, toys, and teddy bears are not in the uncanny valley. However, childish features should not be applied to a realistic adult, as they would be atypical. Features that evoke childhood in a positive way, such as snub noses and round head shapes, can also be deliberately used for stylization and to prevent the uncanny valley.
Humans readily accept unrealistic characters when they are consistently unrealistic, as frequently seen in cartoons.
Use aesthetics and appealing features. Designers can increase a character's physical attractiveness to avoid uncanny effects. While creating faces using our faceMaker avatar generator (Figure 3), research participants chose appealing features for human faces: Symmetry and smooth skin have been found to be the most important factors in physical attractiveness. Additional desirable features for virtual faces include healthy and natural skin tones, natural and realistic proportions, clear gender cues, and volumetric and healthy-colored lips. Interestingly, participants did not seek perfection. Most of the participants reduced details on the skin, but not completely. This finding is in line with previous work showing that slight imperfections are still more appealing than perfectly smooth skin or facial symmetries. Regarding skin realism in face design, there is a narrow window in modulating the desired affinity; on the one hand, perfection is not as appealing as slight imperfection; on the other hand, too many imperfections cause uncanny effects. Balancing the character details can be a difficult challenge for designers aiming to increase the likeability of their characters. However, we can recommend an average face as a foundation for creating faces. It is the ideal base from which to make faces appealing and has the least distance to all human faces, which significantly simplifies customization and individualization for the user or artist.
|Figure 3. The avatar generator faceMaker.|
Lean on user creativity. For our research, the avatar generator faceMaker helps us to determine the preferred characteristics of virtual faces and avoid the unpleasant effects of the uncanny valley. For interactive applications in HCI such as games, an avatar creator allows individual customizations for every user. Our system offers 37 parameters, including the most important features such as gender, skin details, hair color, lip volume, and skin color. Interestingly, if they have a choice, users are more likely to create human-like rather than cartoon-like faces and to bridge the valley using attractive and individual features. Furthermore, users put a lot of effort into expressing themselves through the customization of an avatar. Thus, we learned that avatar customization itself can help users create the avatar they like and identify with, which decreases the uncanny valley. A positive side effect of avatar customization is that it can enhance social presence in online environments and increase the replayability of role-playing games.
Use the uncanny valley effect. Designers of virtual characters can use the uncanny valley to deliberately elicit eerie effects for villains and ambiguous characters. One example of such a villain is Gollum from Lord of the Rings, whose transformation from his former human appearance was even part of the movie's plot. His atypical appearance is accepted, as it underlines the nature of the character. Villains such as Peter Cushing's CGI double of Tarkin in Rogue One or Jeff Bridges' young double in Tron 2 are more controversial, as their virtual clones try to resemble the real actor. However, Rogue One was mainly criticized not for showing the CGI double of a villain but rather for the short appearance of Carrie Fisher's young double of Princess Leia at the very end of the film.
In VR, the effects of the uncanny valley are much greater than in other modalities. The fusion of vision and tactile sensations dramatically increases the experience of being someone or somewhere else. If the user is to feel immersed with their own avatar in VR, multiple aspects should be considered to avoid the negative effects of the uncanny valley:
Consider the user's diversity. Users are very familiar with the characteristics of their own body. If the user's virtual body is rendered with a high level of realism, even small deviations can cause strong discomfort, for example, when women use a male body and see a lot of hair on their arms and hands  (Figure 4). To enable immersion, not only the gender but also skin color and unique features should be considered. Alternatively, less realistic, androgynous VR bodies without specific gender cues or virtual clothes can help to avoid unpleasant effects. A camera mounted on a VR headset can also help to render the user's real body in VR. However, this can reduce immersion due to inconsistent styles and lighting between the real and virtual worlds. From our experience with avatar generators, customizations of the virtual self also help to prevent unpleasant effects in VR. The ability of the user to customize their appearance according to their preferences could help them to feel more immersed.
|Figure 4. Female participants are not immersed using male hands due to a perceived mismatch between VR hair and musculature and their appearance in real life.|
Avoid altered body structures at high levels of realism. Designers of stylized characters often change the physical structure of their characters. Cartoonists, for example, simplify their drawings and reduce the number of fingers. This kind of stylization was adopted and preserved by many video games. We found that a reduced number of fingers does not affect the feeling of presence with an abstract avatar . Users consider their avatar to be a useful tool or body extension; thus, they accept altered body structures using abstract styles. However, designers should not reduce the number of fingers at high levels of realism in VR, as it causes unpleasant feelings and has a negative effect on immersion. Some of our participants felt phantom pain or a strong disconnection to their own body and were confused by the incongruence of the vision and haptics. Thus, high levels of human likeness imply a matching body structure for immersive applications using virtual avatars.
Users get used to their avatar. Users can quickly get used to their avatar and the VR experience. In our studies, we observed that participants stopped complaining about their appearance after only a few minutes. Some users mentioned that after they got used to a virtual appearance, they felt disturbed when their body changed again. Thus, we assume that habituation effects may occur; users learn to interact with their avatar even if deviations and inconsistencies between the avatar and the real appearance exist.
If the user's virtual body is rendered with a high level of realism, even small deviations can cause strong discomfort.
Consider depth cues. The task performance of users can be affected by their avatar. Investigating typing in VR, we found that users with different typing proficiencies benefit from more realistic avatars. Inexperienced typists look at their avatar hands to coordinate their movements above the keyboard. We assume that inexperienced typists need more depth cues than experienced typists, who do not refer to depth cues to localize their hands. Additional surfaces, structures, shadings, shadows, and lights support the binocular disparity and provide more control for localizing their own body movement. Thus, designers and developers should consider whether users can smoothly perform such tasks using the virtual hand within their application. Depth cues can affect the task performance and must be considered when, for example, designers choose a flat or cartoon style.
Increased realism and the uncanny valley pose serious challenges for designers. Mori originally suggested that designers should avoid high levels of human likeness to achieve high levels of affinity. Our guidelines specify additional methods that tackle the problems with the uncanny valley. However, more research is needed in quantifying perception and interaction with virtual characters and avatars. In future research, we also should go beyond the uncanny valley: Are there design principles for virtual characters and avatars that not only avoid the uncanny valley but even increase the effectiveness of working or collaborating with other avatars in the virtual world? Can avatars and virtual characters help us to decrease mental workload and increase social presence? And we should ask more generic questions about how to integrate virtual characters as virtual assistants into our daily life. More research is needed to enable smooth interactions between humans and artificial characters.
1. Mori, M., MacDorman, K.F., and Kageki, N. The uncanny valley. IEEE Robotics and Automation Magazine 19, 2 (2012), 98–100; https://doi.org/10.1109/MRA.2012.2192811
2. Moore, R.K. A Bayesian explanation of the "uncanny valley" effect and related psychological phenomena. Scientfic Reports 2 (2012), 864; https://doi.org/10.1038/srep00864
3. Chattopadhyay, D. and MacDorman, K.F. Familiar faces rendered strange: Why inconsistent realism drives characters into the uncanny valley. Journal of Vision 16, 11 (2016), 7; https://doi.org/10.1167/16.11.7
4. Schwind, V., Leicht, K., Jäger, S., Wolf, K., and Henze, N. Is there an uncanny valley of virtual animals? A Quantitative and qualitative investigation. International Journal of Human-Computer Studies 111C (2017), 49–61; https://doi.org/10.1016/j.ijhcs.2017.11.003
5. Schwind, V., Knierim, P., Tasci, C., Franczak, P., Haas, N., and Henze, N. "These are not my hands!": Effect of gender on the perception of avatar hands in virtual reality. Proc. of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2017; https://doi.org/10.1145/3025453.3025602
6. Schwind, V., Knierim, P., Chuang, L., and Henze, N. "Where's Pinky?": The effects of a reduced number of fingers in virtual reality. Proc. of the 2017 CHI Conference on Computer-Human Interaction in Play; https://doi.org/10.1145/3116595.3116596
7. MacDorman, K.F. Androids as an experimental apparatus: Why is there an uncanny valley and can we exploit it. Proc. of CogSci-2005 Workshop: Toward Social Mechanisms of Android Science. 2005, 106–118; https://www.lri.fr/~sebag/Slides/uncanny.pdf
Valentin Schwind has a background in computer science with a focus on interactive media. Before starting his Ph.D., he worked as an art and technical director for CGI/VFX animations and video games. In July 2018 he finalized his Ph.D. thesis, which focuses on virtual characters, the uncanny valley, virtual applications, and the exploration of new design spaces in virtual reality. email@example.com
Katrin Wolf received her Ph.D. from TU Berlin, where she focused on understanding novel interaction techniques for mobile devices. She is currently professor of media informatics at the Hamburg University of Applied Sciences. Besides tangible and mobile interaction, she is also interested in improving the user experience of immersive environments. firstname.lastname@example.org
Niels Henze did his Ph.D. on mobile interaction with physical objects. He is currently professor of human-computer interaction at the University of Regensburg. His research focuses on understanding users, novel interactive systems, and using models to improve interaction with computing systems. email@example.com
©2018 ACM 1072-5520/18/09 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2018 ACM, Inc.