Javier Marco, Sandra Baldassarri, Eva Cerezo, Diana Xu, Janet Read
In the past 15 years, Tangible User Interfaces (TUIs) have emerged as an ideal technology for delivering child-computer interaction that is adapted to children's psychomotor and cognitive skills development. The rapid evolution of these tangible technologies has meant that there has been little or no time to build a foundation for the design of games and learning applications that could offer pleasant and useful experiences to children. Our research group specializes in multimodal and natural human-computer interaction and conducts child-focused research that highlights children's real needs and wants. This approach can be highly rewarding when designing new interfaces and interactions for children . Rather than designing and implementing finished applications by ourselves and then testing them with children, we work with them throughout the process.
Following this design philosophy, we have built a tangible tabletop prototype suitable for ages three to four. To do this, we have been inviting children into our lab to work with us in designing a tangible storytelling farm game.
The NIKVision tabletop system consists of several components. The first is a 70×70×45cm table (see Figure 1). Physically inert, tangible objects are used on the table; these are conventional rubber toys that children can grasp with their hands . Two visual-output information channels are supported: active projection on the surface of the table and a frontal computer monitor standing on the table. The projected 2-D graphic image provides input/output space coincidence, while the monitor shows a 3-D virtual environment. Recognition software tracks the toys movement and provides information to the system.
When we started this project, we did not have a clear idea for a tangible game suitable for young children, but we knew that farm toys were very popular with three- to four-year-olds. Therefore, we bought some rubber farm animals and modeled a 3-D virtual farm with animal avatars as inputs and outputs. This initial game had no structure: When an animal toy was placed on the table, it would be detected. The 3-D avatar of this same animal appeared on the monitor. Moving and rotating the toy on the table produced the same movements in the animal avatar (see Figure 2). We did not implement more interactive elements to the game because we were interested in the spontaneity and improvisation of the first group of children who visited our lab.
As we understood children's experiences playing with NIKVision, the farm game changed and evolved, while the role of the children during the design process changed from that of informant to that of user [3, 4].
At the first stage, children got involved in our lab test as "informants." Their role was to give clear design ideas for the structure of the game. As noted by others, we recognized that with children ages three to four, significant effort would be needed to establish "equal" communication channels between children and adults. In our first informant sessions, we let children play NIKVision together with their parents, which made the children comfortable with our lab and the tabletop. We could observe them playing and take notes of the interactions between the children and the toy animals to determine which of these interactions was especially fun for children. In order to observe improvisation, we decided to introduce new toys to the game that did not have 3-D avatars in the virtual farm; e.g., a little bucket toy was spontaneously used to feed the animals. These first experiences directed us in how to implement a more structured game.
After the first group of children played the initial version of the farm game, a pair of children (a four-year-old boy and a three-year-old girl) became more involved in our project. Their parents were able to bring them to our lab once a week, so the children became familiar with us and our environment. In this way, children took on the role of testers for each new implementation we made to the game  (see Figure 3).
At this stage, the farm game had a structure and goals. Children had to locate on the farm where each animal ate. We implemented a 3-D virtual farmer character for the 3-D farm. This character could promote children to carry out activities, detect if they were having problems or staying inactive for a long time, and encourage them to play. Here we received the first feedback from children in relation to the tone of voice and the way the farmer spoke to them. In the first version, the children did not like his voice, and they interpreted some of his expressions as yelling.
As the game was interactive, we had a new design element to worry about: Could children easily do the actions we were asking of them? Most of the actions required that children place a particular toy near a virtual farm object; of interest to the developer was learning how precise the children's movements would have to be. We implemented keyboard shortcuts to change these restrictions while they were playing: If we detected they were having problems placing the toys on a particular spot, we could modify the size of the hot spot until children could perform the action without difficulty.
During play, children helped us create new ways of interacting with the toys. A boy was lifting the hen toy on and off the table, playfully jumping the object. When we asked him what he was trying to do, he answered that he wanted the hen to lay eggs. Later, we implemented new code to detect jumps and added a nest for the hen. We tested this at the next test session. After refining the timing of the action, we observed that laying eggs became a favorite activity of the children who played. Nevertheless, a new kind of design problem emerged, as the children did not identify the 3-D nest we modeled as the place where the hens laid the eggs. The problem was not simply solved by remodeling the object; instead, we remodeled and also used the virtual farmer to reinforce that the nest was where the hen lays eggs.
We spent some months working to implement new activities until we felt comfortable with the interactions. It was during this time that we realized the game needed a story to connect all these activities to a more general goal. So, we wrote a story that included our animals and the farmer. Children use the toys to find ingredients to make a birthday cake for the farmer's son. Then, the animals play traditional games like hide-and-seek with the farmer's son.
The design resulted in a more complete and structured game, playable from beginning to end with two or three children. To get a sense of how the tabletop could be played with by many different children, we carried our prototype to several nurseries and schools where children could play the farm game in pairs. At this stage, their role in the design process was that of user. We did not interrupt the class routine; the tabletop was like any other classroom activity . Children went over to the table, played with the game, and then returned to their normal activities.
We used this new situation to evaluate different versions of the game. For example, we implemented three different behaviors of the farmer, varying the level of guidance he gave to the child to complete the game: providing the goal of each minigame "what" (e.g., "I need eggs"); suggesting "what" and "which" is involved (e.g., "put eggs with the hen"); and indicating "what," "which," and "how" children have to play (e.g., "do little jumps with the hen on the nest").
We used new ways to retrieve useful information from children. Sessions were recorded with video cameras. In addition, the game recorded the movements and actions of the toys on the table surface, so later we could redraw the paths made with a particular toy during a particular minigame.
Having big groups of children can often be chaotic. However, it was very useful in evaluating different unplayed variations of the game with children. We found that giving more guidance ("what, which, how") helped children complete the minigames quickly and directly to accomplish the goals. With a less talkative farmer, children had to "explore and discover" and appeared to have more fun.
Today when we see children playing with NIKVision and the newest farm game, we reflect on the days when we first started the project. Our primary concern was how to design tabletop games for very young children. From our experience, the solution seems obvious: "Let the experts talk." Taking our tabletop game to children gave us inspiration and ideas for providing solutions that we could never have imagined on our own.
This work has been partly financed by the Spanish DGICYT N°TIN2007-63025 and by the Government of Aragon: IAF N°2008/0574 and CyT N°2008/0486 projects.
1. Marco, J., Cerezo, E., Baldassarri, S., Mazzonne, E., Read, J. "Bringing Tabletop Technologies to Kindergarten Children." 23rd BCS Conference on Human Computer Interaction. Cambridge, United Kingdom: 2009.
4. Scaife, M., Rogers, Y. "Kids as Informants: Telling Us What We Didn't Know or Confirming What We Knew Already?" In The Design of Children's Technology, ed. Druin, A. 2750. San Francisco: Morgan Kaufmann Publishers, 1998.
5. Markopoulos, P., Read, J., MacFarlane, S., Hoysniemi, J. Evaluating Children's Interactive Products: Principles and Practices for Interaction Designers (Interactive Technologies). San Francisco: Morgan Kaufmann Publishers, 2008.
Javier Marco obtained a B.Sc. in computer science engineering in 1998. He is currently a Ph.D. candidate in computer science engineering at the University of Zaragoza, Spain. His thesis researches the benefits of tangible interfaces for children and their involvement during design and evaluation stages. Learn more at http://webdiis.unizar.es/~jmarco.
Sandra Baldassarri obtained a B.Sc. in computer science in 1992 and a Ph.D. in computer science engineering in 2004. She is an associate professor in the computer science department of the University of Zaragoza, Spain, and is member of the Advanced Computer Graphics Group (GIGA).
Eva Cerezo obtained a B.Sc. in physics in 1990 and the M.Sc. in nuclear physics in 1992. She received a Ph.D. in computer science in 2002 and is currently an associate professor in the Computer Sciences and Systems Engineering Department at the University of Zaragoza, Spain, as well as a member of the Advanced Computer Graphics Group (GIGA).
Diana Yifan Xu is a member of the Child Computer Interaction (ChiCI) Group at the University of Central Lancashire in the U.K. Her research involves tangible user interface and tangible interaction, evaluating novel technologies for and with children.
Janet Read has a first degree in mathematics from the University of Manchester and a Ph.D. in child-computer interaction from the University of Central Lancashire (UCLan). She is director of the Child Computer Interaction Group at UCLan in the U.K.
©2010 ACM 1072-5220/10/0100 $10.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2010 ACM, Inc.