XXII.4 July - August 2015
Page: 39
Digital Citation

Connecting devices for collaborative interactions

Tero Jokela, Ming Chong, Andrés Lucero, Hans Gellersen

back to top 

In our article in the March–April 2013 issue of Interactions, we discussed mobile collocated interactions and how groups of users can spontaneously combine their mobile devices to engage in rich shared activities and experiences [1]. Examples of such situations include sharing photographs and videos within a group of friends in a cafe, presenting and collaboratively editing documents in a business meeting, and playing multiplayer games with other family members in the living room. But before collocated users can engage in such interactions, they must first connect their devices into a group.

While forming a group may sound simple, it is actually a rather complex technical procedure. The devices must first discover the other available devices in proximity, and the users need to indicate which of these devices are intended to join the group. An ad hoc wireless network is then established to enable communication between the devices. As the users cannot see the wireless connections, the process of connecting devices should provide sufficient cues and security to ensure the right devices are connected. Since the intention is to enable spontaneous interactions, it should be possible to connect devices that have no prior knowledge of each other in a fast and easy manner. If the process for connecting devices is complicated or tedious, the users may lose interest in using multi-device applications entirely.

back to top  Insights


back to top  Existing Device-Binding Methods

The general problem of spontaneous device binding (also known as device association, pairing, or coupling) has been studied extensively in the fields of ubiquitous computing and network security. Researchers have proposed a wide range of different binding techniques. Examining the available methods from a user's perspective, we can sum up the techniques in four general categories [2]:

  • Input. Users generate a piece of information and enter it on their devices' user interface. For example, a method of Bluetooth pairing requires users to enter a passkey into the devices.
  • Matching. Users compare the output of devices to confirm or reject a connection. For example, many wireless sensors employ the technique of having users compare a numeric code displayed on the connecting devices and confirming whether the numbers are identical.
  • Guidance. Users perform a physical action on devices (such as touching, pointing, proximity) for steering them to find one another. For example, Android Beam requires users to bring devices together to establish a connection.
  • Enrollment. Users first pre-set a secret (e.g., a password) on their devices and then share the secret with devices that they want to connect. Using a Wi-Fi hotspot, for example, an admin first sets up a code and shares the code only with intended devices.

The majority of the existing work on device binding has been technology driven, investigating aspects such as establishing connections in an efficient manner, defining new security protocols, and exploring novel sensing methods. But binding methods are not just technical means of connecting devices: They also have strong social and emotional aspects. Many factors influence the users' preferences of binding methods, including the place, the social setting, and the other people present [3]. The earlier work on device binding also focused on scenarios of a single user pairing two devices—for example, a user connecting a wireless headset with a phone. However, scenarios involving multiple users differ in many respects from single-user scenarios, making the single-user binding methods not necessarily applicable in multi-user scenarios. On the other hand, a larger group of users enables many new approaches and strategies beyond single-user methods.

back to top  Natural Methods for Binding Groups

In our work, we have been looking at the binding process in medium-size groups of four to six users from a user's perspective. We have aimed to understand how people would naturally connect wireless devices, regardless of the technical limitations posed by current hardware. In an ideal setting, users should be able to connect their devices quickly, extemporaneously, and without explicit instructions. Therefore, we conducted a focus-group study with non-technical participants to find out how people connect devices as a group [4].

Binding methods are not just technical means of connecting devices: They also have strong social and emotional aspects.

To avoid biases caused by existing device interfaces, we used low-fidelity plastic props (with different dimensions, mobility, and rigidity) as surrogates (Figure 1a). We adopted the guessability study methodology [5] and asked participants to come up with their own techniques for connecting various groups of devices. As a group of four, the participants used a think-aloud protocol and took turns suggesting and explaining actions they would perform to connect a set of given devices (Figure 1b). Common suggestions included techniques that require pushing dedicated buttons, pointing or performing a gesture, and bringing devices into a physical touch.

Results showed that physicality (e.g., the shape and size of devices) influences how people perceive the interaction for group binding. Devices with a small surface area (with a diagonal of an inch) are difficult to enter commands into. People exploited device mobility and suggested techniques that require fast maneuvering actions, such as performing a gesture. In contrast, people prefer less movement for larger devices such as tablet computers, as the bulkiness makes maneuvering cumbersome. Another influential factor is affordance. The form factor of devices affords people to relate interaction techniques from other device types. For instance, our participants used the metaphor of pointing mobile phones as a remote control. But for bendable devices, their form factor no longer supports a pointing direction, in which case people would deform the shape, such as rolling up the device into a longitudinal shape for pointing.

Of the user-defined techniques we observed, one that our participants often suggested was the method of bringing devices into physical contact, such as touching devices at their corners or stacking devices to form a pile. Touch-based methods are fast and expressive. They enable better awareness of the group formation, as every member can easily perceive the touching actions. In addition, touch-based interactive technology, such as short-range communication like NFC, already pervasively exists in mobile devices. This makes the interaction particularly attractive for immediate adoption.

back to top  Binding with Device Touch

Based on these observations, we designed EasyGroups [6], a group-binding method based on device touch (Figure 2a). In EasyGroups, one person (called the leader) initiates a new group by starting the application. The leader adds another person into the group by bringing their devices to touch (Figure 3a). The leader continues to add new members by repeating the touch action with the new devices until everybody has been added. We also developed an alternative variant of EasyGroups, called Ring (Figure 2b), where all members contribute as equal peers. Similarly, one person first starts the application and creates a new group. This person then adds a second person by bringing devices to touch. The second person can add a third person, who can add a fourth person, and so on until all members have been added into the group. This way the group membership proceeds like a relay around the group.

In our user evaluation of EasyGroups with groups of four users, we noticed several interesting differences between the two approaches. In the original EasyGroups, the leader has strong control over who can join the group, which may be appropriate, for example, if the leader is sharing personal content. It also requires only one person to know how to set up a group, but selecting the leader may add democratic complexity to the group-creation process. On the other hand, the peer-based Ring method better brings people together and helps to create a greater sense of community, because everybody is involved in creating the group and interacting with the others. Ring also scales better to larger groups and longer distances between persons.

Overall, we found that when designing binding methods for groups of users, it is important to consider robustness in real-life conditions. While many methods can work well in theory or with mock-ups, in reality, applications involving multiple users and devices are complex distributed systems. As multiple devices are involved, there is a high risk that the devices may fail to detect each other, network connections between devices may be broken, or the software may crash in any device. Also, some users may not be aware of the steps they should follow, or be unable to do so, for example, arriving late or being occupied by other tasks such as phone calls. Therefore, the binding methods should be flexible and robust, allowing the users to adapt them to the changing needs of the situation and to recover from failures.

back to top  Flexibility to Dynamic Situations

To explore these aspects, we designed another, more flexible variant of EasyGroups called FlexiGroups [7]. Like EasyGroups, one person first creates a new group by starting the application. The person can add new people by touching their devices. However, any of the other members can also freely add any number of new people to the group. Note that this approach allows both the original EasyGroups and the Ring patterns, but also enables many other approaches.

We arranged a user evaluation where we asked groups of six users to connect their devices using the FlexiGroups method. While all groups encountered some problems and made mistakes, the robustness of the method allowed them to recover and continue to successfully complete the group-creation task. Most commonly, the group members opportunistically selected nearby devices to touch next. This way the group membership virally spread across the group, from one person to another (Figures 2c, 2d, and 3b). The viral patterns are efficient, require no advance planning, and keep everybody involved in the group-creation process.

The viral patterns are efficient, require no advance planning, and keep everybody involved in the group-creation process.

The participants also suggested patterns where several devices were touched simultaneously to make the group creation more efficient. In one variation, the person who creates the group puts their device at the center of the table, and the others who want to join the group put their devices next to it. In another variation, all devices are collected, and the person who creates the group then touches them all in one action. Although many techniques to pair two devices exist, group binding is not a simple extension of numerous pairwise connections with multiple users. Some people consider group binding as a single-step procedure, rather than divided into multiple pairings [4].

back to top  Setting Up the Group Together

As observed both in our studies and studies conducted by others [8], device association in groups is a highly collaborative activity. Users are eager to help each other when problems arise, and together they can solve and overcome most usability and technical problems encountered. Especially in larger groups, the main challenges are related to group work and social interactions within the group: making decisions and agreeing on a common strategy, coordinating and synchronizing actions, and keeping track of the others and the overall task status. An important consideration is also keeping everybody engaged in the process, as people easily get bored or distracted when they cannot do anything but wait for others to complete the group formation. Further, techniques that require users to surrender possession of their personal devices to another user are often inappropriate in a group setting, as users prefer to control their own devices due to privacy concerns.

In many groups, there are various roles within the group. For example, in a meeting, there could be a chairperson and a secretary, or in a game, there could be two competing teams and each team could have a captain. Further, the order of the devices can be important, for example, if the people are sitting around a table playing a game and taking turns. In such situations, the device order could also be used to estimate the relative positions of the people to enable simple spatial interactions, such as throwing virtual objects between device screens. We evaluated different methods for defining the order of the members during the group-formation phase [6,7]. Ideally, the order should be defined automatically, but in reality, this may be difficult to achieve, as it may require special tracking hardware. If members are added to the group in a specific order, for example, proceeding in counterclockwise direction around the group, the device order can be automatically determined based on the touching order. However, in our evaluations, most users found such an approach too restricting and unforgiving to errors. Instead, the users preferred to freely add all members to the group first and then define the order manually.

back to top  Future Challenges

All experiments with group-binding methods that we are aware of have been made in a usability laboratory under ideal conditions. While a lot has been learned from these experiments, in real life, various contextual and situational factors influence the group-creation process. Therefore, to deepen the understanding of group binding in different situations and tasks, we believe it will be important to study group-binding methods in more realistic settings and over extended periods of time with longitudinal field trials.

Looking into the future, we see an increasing diversity of devices with the emergence of new wearable form factors, including wrist-worn and head-worn devices. As more and more people wear such devices, situations where there are multiple people present with wearable devices will become commonplace. In those situations, wearable devices could support collaborative tasks and experiences through multi-user applications. However, existing binding methods that have been designed for conventional devices such as computers, smartphones, and tablets are not necessarily applicable to wearable devices, which are far more personal and intimate. For example, while touching can be a natural way of selecting another user's phone or tablet, it might be inappropriate when the device is head-mounted.

The existing methods also do not take advantage of unique features of wearable devices that could enable more natural and innovative ways to form groups. One way to conceptualize this is to consider wearable devices as already attached to their owners, so the binding of multi-user wearable devices can form through people's social interactions. A handshake, for example, could indicate a level of acquaintance, so devices can form a connection for people to share business contacts. A hug, on the other hand, is more intimate, so more personal information could be sent. This calls for new research from various domains on group-binding methods for connecting wearable devices.

back to top  References

1. Lucero, A., Jones, M., Jokela, T., and Robinson, S. Mobile collocated interactions: Taking an offline break together. Interactions 20, 2 (2013), 26–32.

2. Chong, M., Mayrhofer, R., and Gellersen, H. A survey of user interaction for spontaneous device association. ACM Computing Surveys 47, 1 (2014), Article 8

3. Ion, I., Langheinrich, M., Kumaraguru, P., and Čapkun, S. Influence of user perception, security needs, and social factors on device pairing method choices. Proc. SOUPS'10, Article 6.

4. Chong, M. and Gellersen, H. How groups of users associate wireless devices. Proc. CHI'13, 1559–1568.

5. Wobbrock, J.O., Morris, M.R., and Wilson, A.D. User-defined gestures for surface computing. Proc. CHI'09, 1083–1092.

6. Jokela, T. and Lucero, A. A comparative evaluation of touch-based methods to bind mobile devices for collaborative interactions. Proc. CHI'13, 3355–3364.

7. Jokela, T. and Lucero, A. FlexiGroups: Binding mobile devices for collaborative interactions in medium-sized groups with device touch. Proc. MobileHCI'14, 369–378.

8. Kainda, R., Flechais, I., and Roscoe, A.W. Two heads are better than one: Security and usability of device associations in group scenarios. Proc. SOUPS'10, Article 5.

back to top  Authors

Tero Jokela is a principal researcher at Nokia Technologies in Tampere, Finland. His research interests include human-computer interaction and user interface software, as well as mobile multimedia applications. tero.jokela@nokia.com

Ming Ki Chong is a postdoctoral researcher at Lancaster University. His research interests include human-computer interaction, ubiquitous computing, mobile interaction techniques, and personal informatics. mingki@acm.org

Andrés Lucero is an associate professor of interaction design at the University of Southern Denmark's Mads Clausen Institute. His research interests lie in human-computer interaction, co-design, and design research. lucero@acm.org

Hans Gellersen is a professor of interactive systems at Lancaster University's School of Computing and Communications. His research interests include ubiquitous computing, physical-digital interactive systems, and user interface technology. h.gellersen@lancaster.ac.uk

back to top  Figures

F1Figure 1. Guessability study. a) Plastic props used to simulate different devices. b) Participants suggesting a group-binding gesture during the experiment.

F2Figure 2. Different group creation patterns: a) EasyGroups. b) EasyGroups Ring variant. c-d) Examples of viral FlexiGroups patterns employed by evaluation participants. P1–P6 represent the participants, arrows show the touch actions between the participants, and the numbered white circles indicate the order of the touch actions.

F3Figure 3. a) Touching another device to add it to the group. b) Three people touching and adding devices in parallel.

back to top 

Copyright held by authors. Publication rights licensed to ACM.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2015 ACM, Inc.

Post Comment

No Comments Found