Raine Kajastila, Perttu Hämäläinen
The past decade has brought about interesting developments in combining video games, sports, and exercise. Motion games, also known as exergames or active video games, have become mainstream thanks to technologies like Microsoft Kinect, PlayStation Move, and Nintendo Wii. Unfortunately, commercial games are often optimized for an average customer facing a television in an average living room, which limits the variety and intensity of movements. This is a constraint for effective exercise and hinders the learning of real sports skills while playing motion games.
We argue that a way to support vigorous physical exercise and learning sports-related skills is to move motion games away from the living room to places where sports are actually practiced. This gives the player more space, allows increased movement diversity, and reduces the risk of colliding with furniture. A pleasurable exercise experience also requires appropriate ventilation, which indoor sports centers typically have. However, motion games are not always more entertaining than real practice, and a skill learned in a game might not transfer to a real sport. More research is needed on how to design and implement motion games that are both fun and effective as exercise and motor-learning environments.
We have investigated how practicing real skills can be made more motivating in real sports settings. Here, we introduce two approaches: a game played by jumping on a trampoline and an augmented climbing wall that enables getting feedback and playing games on a real climbing wall.
Trampolines are popular in recreational use and in practicing jumps for sports like snowboarding. However, directionless jumping soon becomes boring, and there are risks in trying more complicated tricks without proper skills. With our trampoline games (Figure 1), we aim to provide goals and feedback to keep players interested longer in preliminary training before attempting higher-risk maneuvers.
One of our studies investigated how playing a simple body-controlled game while jumping on a trampoline affects the exercise experience, and whether the game enables the learning of basic trampolining skills . We also wanted to see if traditional game mechanics such as jump-height exaggeration affect skill learning and player motivation. To study this, we developed a platform jumping game implemented using computer vision and a screen placed near a large trampoline. Players could see themselves jumping in a virtual world, and their jump height was either greatly exaggerated or matched to the real-world jump height on the trampoline.
According to the participants, trampoline training was regarded as fun by itself, but the game made it more engaging. Furthermore, jump-height exaggeration made the game even more empowering, as shown in the following reactions from participants:
- "More rewarding, like driving a race car instead of a scooter"
- "The exaggerated game feels somehow like superpowers in a game. Suddenly you get a boost."
Our study found no differences in learning high and precise jumps between the groups playing a game and the group without a game. Focusing on the game did not disturb the participants' jump training, and many participants considered the real-time feedback beneficial:
- "I don't pay so much attention to the jumping; it comes automatically when focusing on the game."
- "I forgot how I was jumping and concentrated on the result."
Extra empowerment in the game, in the form of jump-height exaggeration, did not affect the performance adversely, and most of the participants preferred the exaggerated version of the game. The participants felt the jump-height exaggeration was natural. But what really surprised us was that half of the participants did not even notice the exaggeration, even though they could jump many times over their own height in the game world. This suggests that extra empowerment in the human-computer feedback loop may be used to make the training more engaging without negatively affecting learning. In light of these results, we suggest that extra empowerment should be studied more in the design of human-computer interfaces for exercise and sports.
Games and technology can also bring climbing walls to life. We have developed an augmented climbing wall (Figures 2 and 3) that consists of a projector, a depth camera, and a computer for analyzing the climber's movements, providing feedback about the climber's performance, and creating meaningful tasks for the climber . The system can be used to project interactive graphics on the wall, such as chainsaws, which a climber needs to avoid.
Climbing can be thought of as a spatial puzzle solved using the body. The climbing routes (called problems in bouldering) are analogous to game levels. Interactive graphics can add new kinds of problem solving to the wall, which would not be possible with climbing holds only. Interactive graphics can create a new temporal dimension to climbing, for example, waiting for a moving chainsaw to pass. Events can also be triggered depending on what a climber does on the wall. These elements can make training more fun and motivate spending more time on the wall. Space can be saved, as a small wall can host long (or even infinite) routes with holds automatically revealed while the climber progresses. As games are played using real climbing movements, they benefit non-augmented climbing as well.
In our user experiment, a game of "chasing chainsaws" was seen to make the climbing practice fun—"Game feeling makes it fun"—however, it changed the climbing experience. Veteran climbers commented, "Not climbing anymore. Rushing from one big hold to another and keeping eye on the saw." Playing a chainsaw game is likely to increase endurance and strength but less likely to develop perfect technique.
In addition to getting one's daily dose of fun exercise, the augmented climbing wall could also be utilized for improving motor skills. This is still a work in progress. The challenge is in designing an effective human-computer interface for providing meaningful feedback about human movement while climbing. When implemented correctly, computer-generated augmented feedback can both motivate and guide motor-skill learning [3,4]. Without feedback, the learner may not fully understand what their body is doing and what to improve when practicing new skills.
Currently our augmented climbing wall provides the possibility of an automatic video replay of a previous climb with a side-by-side comparison to a model performance, which has been shown to be beneficial in other sports . Our future directions include visualizing relevant movement parameters in the video or directly on the wall. This should allow more detailed feedback, for example, pinpointing and visualizing differences between expert and beginner performances.
One important source of motivation in exercising is social contact. For example, the augmented wall can be used for sharing user-created routes and high scores. It is also possible to provide performance statistics for competing with others and for following individual improvement over time. We further believe an augmented exercise environment should be interesting for an audience. It could provide information about performance, enable the audience to interact with the climber, and keep other climbers entertained while waiting.
We expect the combination of real-time video analysis and video projections will be utilized in a variety of sports in the future. For example, many sports, such as gymnastics, dance, and football could benefit from interactive floor projections. A related and recent commercial application is Nike's House of Mamba, a full-size basketball court covered with LED displays for interactive graphics.
What seems to be lacking is a closer connection between human-computer interaction (HCI) and sports science, especially sports and exercise psychology. Although the motor-off learning and performance research done in the context of sports provides sound theories and practices for things like optimizing feedback and considering individual differences , relatively few HCI publications on bodily interaction cite sports science papers.
Motor-skill learning is an important part of all action games in which players move their bodies or manipulate game controllers in real time to reach goals. The role of feedback in motor learning and performance has been studied extensively (e.g., [3,6]), and computer-generated feedback is a field that naturally bridges HCI and sport sciences. While it has been shown that properly designed augmented feedback can both motivate and guide motor learning, motor learning in the context of non-realistic player-avatar mappings (e.g., exaggerated jump height on a trampoline) is still a new field.
Many consider it ideal to be able to learn a real sport by playing a motion game. Technology limits what can be practiced realistically, in terms of what movement qualities can be sensed. However, we argue that what needs to be practiced realistically is ultimately a design choice: Why focus on teaching an existing sport when one can create whole new augmented sports by designing both space and technology for both players and spectators? One example of this is the interactive LED basketball court mentioned earlier that may enable new types of games and captivating visualizations for the audience. Another example is Kick Ass Kung-Fu , a full-body martial arts game and performance shown in Figure 4. Such experiences can employ the principle of selective realism and combine realism and fantasy in appropriate proportions. In Kick Ass Kung-Fu, one performs real kicks, punches, and acrobatics on a realistic martial arts floor, while the fantasy elements include on-screen exaggeration of jumping height and running speed, and an energy supercharge triggered by shouting. Additionally, the lack of real contact with virtual opponents enables the exploration of aesthetic and creative movement that would be too risky in real sparring.
In the future, it will be interesting to observe the possible fusion of eSports (competitive tournament gaming) with the kinds of exertion games discussed here. Digital games are gradually becoming socially accepted spectator sports, and there are already professional gamers living off sponsorships and tournaments. In the future there might exist exergame professionals, such as athletes competing in martial arts or climbing games optimized as both user and spectator experiences. An open question for HCI and game research is how one can create motion games with eSports potential. Presently, most motion games do not have competitive communities, with Dance Dance Revolution as the notable exception.
3. Magill, R.A. and Anderson, D.I. The roles and uses of augmented feedback in motor skill acquisition. In Skill Acquisition in Sport: Research, Theory and Practice. N. Hodges and A.M. Williams, eds. Routledge, 2012.
7. Hämäläinen, P., Ilmonen, T., Höysniemi, J., Lindholm, M., and Nykänen, A. Martial arts in artificial reality. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2005, 781–790.
Raine Kajastila is a post-doctoral researcher at Aalto University in Helsinki, Finland. Raine did his Ph.D. on eyes-free and gestural user interfaces. His current research focuses on aurally/visually augmented exercise and sports-training environments. firstname.lastname@example.org
Perttu Hämäläinen is a professor of computer games at Aalto University. His primary research goal is to advance the understanding of computer science of human movement, encompassing analysis (e.g., computer vision), synthesis (e.g., procedural animation), as well as games and augmented feedback for motor learning. email@example.com
Figure 1. A real training environment can be combined with a fantasy world, where a player's skills can be exaggerated while still enabling motor learning and vigorous exercise. In our trampoline game setup, an image of a player on a trampoline is captured with a depth camera, allowing the player to see himself in the game world.
©2015 ACM 1072-5520/15/0300 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2015 ACM, Inc.