Authors:
Andrés Lucero, Matt Jones, Tero Jokela, Simon Robinson
According to Cisco, in 2012 we reached the point where there were more mobile phones than people on Earth (there are now also more people with access to a mobile device than a toothbrush). What used to be a device mostly for professional use now pervades every aspect of our lives. Today's mobile devices allow us to stay in touch, document our lives, buy our groceries, find our way to our next appointment, and even read a good ole book. People often say they feel naked, or that they are missing something, when they leave their mobile phones at home or at the office. Moreover, we are getting our first mobile phones at an ever younger age. In Finland, where part of this work took place, children get their first mobile phone at age 7 (together with the keys to their family's home). As we can see from these examples, mobile phones have become our companions, witnesses to our lives. However, this has all come at a costa social cost, perhaps.
Sherry Turkle discusses some of the antisocial consequences that mobile phone use has had among teenagers in the U.S., critiquing networked life, a life where we become inseparable from our smartphones, where our social lives are happening online, where devices become substitutes for connecting with each other face-to-face, and where we would rather text than talk [1]. Indeed, we see some of these symptoms popping up in other places, too, such as Finland, Wales, and Chile. The truth is, we are spending an awful amount of time emailing, texting, facebooking, and tweeting in our Western world [2]. Our relationships with one another are mediated through screens, as opposed to happening directly in parks, cafes, pubs, or living rooms.
But there is hope. The very devices that have partly removed the human touch from interpersonal communications could be the key to reestablishing and enriching face-to-face physical interactions.
Toward Shared and Collocated Use of Mobile Devices
Mobile phones were originally conceived and have traditionally been utilized for individual use. In Western culture, we are not supposed to go into another person's mobile phone and peruse his or her list of contacts, photos, or visited websites. On occasion, people may pass their phones around and let others look at photos, for instance. In other cultures, sharing comes naturally, either due to economic pragmatics or because community rather than individual outlooks are prioritized. By and large, though, in Western contexts, it will take a major change in current mobile phone usage to open up the opportunities offered by shared use of mobile phones.
For the past three years, we have been looking at precisely this issue. Are people willing to share their mobile devices and engage in collaborative interactions? Using new display, sensor, and short-range communication technologies, we envision situations in which collocated users engage in collaborative activities using their devices, thus going from personal-individual toward shared-multiuser experiences and interactions. We have explored mobile collocated interactions, encouraging people to share their devices to create a collective experience or reach a common goal. Various physical and social contexts of use have been taken into account, such as teamwork at the office, sharing media content at home, and public expression in a pub (Figure 1), and for sharing educational stories in rural, developing-world contexts. Here, we will reflect on some of the issues and challenges people will face once mobile collocated interactions become commonplace.
Taking an Offline Break
Consider the amount of time you spend managing your life on your social networks. All of this online activity must come at the expense of face-to-face communication. We are spending less time physically with each other and more time with our mobile devices. And when we actually do get together, people seem to be glued to their screens, often ignoring those around them, avoiding eye contact [1]. Inspired by Turkle's arguments, we think it is time to lift our heads away from the screen and start noticing those around us.
Mobile collocated interactions offer a way to disconnect from the network for a while and take a break. This handheld-supported downtime could provide the necessary space to nurture our human relationships and make room for reflection [1]. To achieve this, we have been looking into human activities currently supported by digital technology, and have decided to provide a humanistic alternative [3], especially for those interactions that benefit from face-to-face communication.
One such example is photo sharing. Online photo-sharing services such as Flickr or Photobucket offer many benefits; however, they lack the richness of social interaction when compared with sharing paper prints between collocated users [4]. In Figure 2 we see four people discussing photos face-to-face using Pass-Them-Around. This app uses the metaphor of passing paper prints around, whereby each mobile phone becomes a physical container of individual photos.
What we see here is the type of mobile collocated interactions where attention naturally shifts between the artifact (the photo) and the presenter. In this case, while the presenter is telling the story behind this particular picture, the two people on the left are looking directly at him, and the presenter and the person on the right are looking at the photo. This type of interaction, mediated by mobile phones, fully supports the richness of real-world social interactions, at the same time taking full advantage of digital technology. For example, the app also contained a way to easily duplicate images, an automatic slideshow function, and ways to tile devices together to create a larger joint display.
Creating Joint Attention
When using their mobile phones, people have a tendency to hold their devices with one or two hands, with the screen facing toward them. People will usually adopt a particular device position, combined even with a second hand to cover the screen, either to browse private content, such as a confidential email, or to avoid glare. We are accustomed to manipulating our mobile devices this way, and as we all know, habits are hard to break.
For people to fully benefit from mobile collocated interactions, they must open up and start seeing their personal devices as shared, public devices. In mobile collocated interactions, phones are at the intersection of fully personal and fully shared use. This basic principle is exploited by board games that combine the individual use of tokens, money, and cards with the shared use of the dice and board. The mobile phone will sometimes behave as a token and sometimes be part of a larger board. In mobile collocated interactions, people should ideally be able to both make individual moves, such as browsing for private content, and play together. The shared uses will usually entail keeping the device flat on a table or projecting information on a wall so that all people, or at least more than one person, can comfortably see the shared content and get information about the current state of the interaction. People should be able to perceive the interactive space created by these shared devices as one entity. Siftables [5] are a good example of small mobile devices that show this type of behavior.
In Figure 3, we see how four devices create a common workspace. In this example, people are passing photos around a table, one by one, in sequence. When the owner of the photo collection moves on to the next photo on his device, the remaining three devices respond by automatically moving on to the next photo. What this creates in practice is that each person is free to decide whether to look at the photo on his device, across the table, or right next to him. Any change in the photo sequence will be automatically reflected on all devices simultaneously so everybody will be aware of how the state of the interaction has changed. If some participants were to hold devices in their hands, the illusion of creating a shared space would be broken. In Figure 4, we see how the participant on the bottom left is pointing to an interesting photo (or part of it) shown on the device to his right, thus drawing the attention of the group and the discussion to that particular device. The app also allows people to create group "huddles" and discuss photos by tiling devices to create larger displays (Figure 5).
What we have observed in our evaluations is that people do see the benefits in creating such shared interactive spaces and are genuinely positive about the possibilities these device ecosystems can open up. For instance, when tiling devices to display a composite larger version of a photo, we were surprised to see that people did not complain much about how bezels (the screen borders) partition the photos. As mentioned earlier, however, people have the tendency to see the mobile phones as personal devices, and so it will take some time for people to see the benefits of such interactions and adopt them en masse.
To effectively create joint attention between groups of collocated people, the number of participants involved in the interaction should be between two and 10. Beyond that, not all users will be able to comfortably perceive the displays of the mobile devices as part of a common interaction space. Larger public displays, such as LCD screens and projections, can then be added to create broader ecosystems of interaction [6,7,8].
Large-Scale Sharing
In the PicoTales prototype, we explored the feasibility and value of creating a much larger shared storytelling surface by combining projected output from interlinked mobiles [9]. Each user sketches on the touchscreen display of their device, and this image is projected via an attached small projector.
The images can be animated by the user, whose arm movements are recorded by the device's onboard sensors, with no need for additional infrastructure, such as a depth camera, to be deployed in the environment. Figure 6 illustrates the collaboration between two users who are creating a story together. When they have finished, the system uses the sensor data and sketch content to create a video representing the users' joint efforts; this video can be replayed later on each person's device, emailed, or shared via a social network. The current prototype allows up to four people to use their mobiles together, but in principle the approach is limited only by the physical projection space available.
Initially we imagined PicoTales being used in a range of developed-world scenariosin classrooms as part of a lesson or at home, perhaps as part of a "draw something"-style game. In our work, though, we have increasingly been involved in considering rural, developing-world contexts, which present interesting possibilities for shared interactions. One storytelling project found that technology that forced individual use and restricted collaboration was ill received by the community [10]. In contrast, PicoTales could lead to more appropriate, effective group sharing.
Getting Started
People have been thinking about connecting several devices into one ecosystem for some time. One recurring issue with multidevice ecosystems is how to get the mobile devices to "talk" to one anotherin other words, how to set up the connections in an ad hoc way so the devices can start exchanging information. Setting up connections between different devices is a widely documented problem for these types of collaborative interactions [6,7].
Unless people are able to spontaneously join a group and start interacting with others in a fast and easy way, they might lose interest in participating in mobile collocated interactions in the first place. The technical obstacles that must be overcome are: detecting devices, identifying the correct nearby devices that will participate as part of the group, and connecting those devices. Wireless connections do not provide the physical cues that allow us to determine which devices are connected. We build upon that lack of physical cues by using device proximity and touch interactions (together with audio-tactile feedback) to ensure that the right devices are connected. Figure 7 shows how the person holding the black device can pick which other phone to connect to by bringing their device within close range. In this example, the person has already connected the cyan phone and is about to add the magenta device. We also combine the process of opening the app with the device-connectivity steps mentioned earlier, so they become a single seamless process, as opposed to several fragmented steps.
Unless people are able to spontaneously join a group and start interacting with others in a fast and easy way, they might lose interest in participating in mobile collocated interactions in the first place.
Once people are participating in mobile collocated interactions, they should be able to just as easily leave the groupfor example, to go to the washroomand then rejoin it once they are back. Participants in our studies have also requested to be able to break up a group and join it once again a day or a week later. These comments point toward the fact that people don't want to think about the hassles of setting up a group, but rather about the potential joys of repeatedly sharing and interacting with others in mobile collocated interactions.
Sharing Personal Phones
Going back to our initial question of whether people are willing to share their mobile devices and engage in collaborative interactions, in our studies we have found that people are concerned about letting other people handle their personal phones. Often-mentioned reasons for those concerns were that they may spill drinks on the phone when using it in the context of a bar or cafe, or unintentionally damage it (e.g., scratch the back when moving the phone on a surface), beyond the normal wear and tear that happens from daily use. People seemed more comfortable with sharing their devices with people they know and trust, such as close friends and family members, as opposed to complete strangers. In such cases, the owners of the phones can more accurately judge whether or not someone will be careful with their phone. On the positive side, most participants felt that the benefit of engaging in ad hoc collocated social interactions using the phones outweighed the potential risk of damaging the devices. But first, we must be able to deliver useful and sensible mobile collocated experiences to people. This is the first step toward better balancing our networked life with our intimate, personal life.
References
1. Turkle, S. Alone Together: Why We Expect More from Technology and Less from Each Other. Basic Books, NY, 2011
2. Harper, R.H.R. Texture: Human Expression in the Age of Communications Overload. The MIT Press, Cambridge, MA, 2010.
3. Lanier, J. You Are Not a Gadget. Penguin Books, London, 2010.
4. Lucero, A., Holopainen, J. and Jokela, T. Pass-them-around: Collaborative use of mobile phones for photo sharing. Proc. of the 2011 Annual Conf. on Human Factors in Computing Systems. ACM, 2011, 17871796.
5. Merrill, D., Kalanithi, J. and Maes, P. Siftables: Towards sensor network user interfaces. Proc. of the 1st Inter. Conf. on Tangible and Embedded Interaction. ACM, 2007, 7578.
6. Terrenghi, L., Quigley, A. and Dix, A. A taxonomy for and analysis of multi-person-display ecosystems. Personal Ubiquitous Comput. 13, 8 (November 2009), 583598.
7. Greenberg, S., Marquardt, N., Ballendat, T., Diaz-Marino, R. and Wang, M. Proxemic interactions: The new ubicomp? interactions 18, 1 (January 2011), 4250.
8. Lucero, A., Holopainen, J. and Jokela, T. MobiComics: Collaborative use of mobile phones and large displays for public expression. Proc. of the 14th Inter. Conf. on Human Computer Interaction with Mobile Devices and Services. ACM, 2012, 383392.
9. Robinson, S., Vartiainen, E., Jones, M. and Marsden, G. PicoTales: Collaborative authoring of animated stories using handheld projectors. Proc. of the ACM 2012 Conf. on Computer Supported Cooperative Work. ACM, 2012, 671680.
10. Reitmaier, T., Bidwell, N.J., and Marsden, G. Field testing mobile digital storytelling software in rural Kenya. Proc. of the 12th Inter. Conf. on Human Computer Interaction with Mobile Devices and Services. ACM, 2010, 283286.
Authors
Andrés Lucero is a senior researcher at Nokia in Tampere, Finland, where he leads research projects. His interests lie in human-computer interaction (HCI), user-centered design, and design research. For more about Nokia's research work, see research.nokia.com.
Matt Jones is a professor of computer science at the FIT Lab, Swansea University, and the co-author of Mobile Interaction Design. For more about the lab, see www.fitlab.eu.
Tero Jokela is a principal researcher at Nokia Research Center in Tampere, Finland. His primary research interests include human-computer interaction and user interface software, as well as mobile multimedia applications.
Simon Robinson is a researcher at the FIT Lab, Swansea University. He is interested in interactions that allow people to maintain their immersion in their surroundings, rather than a device.
Figures
Figure 1. Participants collaboratively creating comic-strip panels during the MobiComics evaluation. Mobile collocated interactions provide a way to disconnect from the network for a while and engage in face-to-face social interactions.
Figure 2. Participants sharing photos using their mobile devices with Pass-Them-Around. The two people on the left look at the presenter as he tells the story behind his pictures.
Figure 3. Participants keeping their mobile devices on the table to create a common workspace. This configuration allows those involved in the interaction to extract information about the interaction's current state.
Figure 4. One participant points to the photo shown on the device to his right, thus refocusing the group's attention to that particular device during the Pass-Them-Around evaluation.
Figure 5. Different group huddles are created by combining two devices; three devices; four devices into two separate tiles and into one large tile that includes all four devices.
Figure 6. Sketching and projecting to create a shared story. Two users collaborating to animate the picoTale.
Figure 7. Using device proximity and touch interactions to connect devices.
©2013 ACM 1072-5220/13/03 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2013 ACM, Inc.
Post Comment
No Comments Found