ACM Interactions

I use the term stream switching to refer to people simultaneously processing multiple streams of input information, each of which has its own context and background knowledge. This definition sounds similar to multitasking, but multitasking research usually focuses on a single individual, divided attention, and working memory capacity. Stream switching focuses on multiple people’s interactions and their mental models of each other. Below I offer examples and argue that stream switching merits further research. 

Stream switching

Stream switching draws on 1) the economic concept of switching costs and 2) the psychological concepts of perspective-taking and theory of mind. Economic switching costs are the financial, mental, and time-based costs to switch between products. The costs of switching from an Android to an Apple phone, for example, go far beyond the price tag and include learning how to use the phone’s software and much personal information transfer. Analogously, stream switching includes not only the basic attentional cost of focusing on a different input stream, but also the additional cognitive load of updating one’s mental model of the stream source. Is it accurate? Is it trustworthy? These questions relate to the psychological concepts of perspective-taking (Can you imagine others’ perspectives?) and theory of mind (Can you understand how different others’ knowledge and beliefs might be from yours?). 

Stream switching includes the more specific practice of code-switching. Code-switching was originally the linguistic practice of switching between languages depending on your context, and now refers more generally to the switching of identities depending on who’s around you. Someone might behave one way at home with family and another way in the outside world. Minorities often become quite skilled at code-switching since they have daily practice working within a different majority culture. I would hypothesize they are also highly skilled at stream switching. 

The idea that people vary in their stream switching ability is one of the reasons stream switching deserves more research. Can this critical skill be practiced or trained? We already know that some people are better than others at empathizing, understanding what other people are thinking and feeling. Research has correlated these individual differences with factors including reading more fiction, role-playing and reflection, the general practice of thinking more about one’s thinking. Perhaps this research could be extended to develop methods to measure one’s stream switching ability and methods of improving it. 

User experience (UX) implications

In user-centered design, we try to empathize with our users. We create personas to reduce our cognitive load of doing so. Here is Maria, the young professional with two children; what are her jobs to be done when she opens our app? A product designer with better stream-switching skills will truly be able to step into Maria’s world and build an accurate mental model of her goals and expectations in order to design a product that fits perfectly. That product then allows Maria to accomplish her goals more quickly with fewer errors. 

On the other hand, low-usability software requires the user to switch streams mid-use to imagine the intentions of the software designer or to model the inner workings of the software itself. You may have experienced an accounting system that was well-designed for accountants but not for you. While the accountants can easily track expenses, you can’t figure out how much money you have. Or consider a social media system. If you post certain content, is it clear who will see it? Will it lead your friends to see related ads? Or in a corporate calendaring system, if you invite people to a meeting, is it clear who will see the invite list? Being able to answer these questions requires highly usable software. Norman warned about gulfs in evaluation; today’s sociotechnical context requires evaluation of not only the system, but also of other users’ experiences with it. 

Frustrating usability situations increase stream switching, which burdens our cognitive capacity. When you have to switch streams to figure out where to click next, you have less attention to devote to the streams you were already juggling, for example, home life versus work life, or your supervisor’s mindset versus your teammates’. Bad interaction designs effectively steal our attention. 

Asymmetry, monitoring, and conflicting versions of truth

Many systems present asymmetric information to users, i.e., collaborators receive different information or levels of access, as in the calendaring example. It’s not always a problem; a person presenting slides should be able to see their presenter notes while the audience should not. But when you’re Zooming and ask, “Can you hear me?” or “Can everyone see my screen?”— that’s problematic asymmetry. You don’t have good cues about what other people are experiencing, so you have to ask explicitly. Having this information enables you to update your mental model of your colleagues’ experience in the meeting and enables you to stream switch more smoothly between conveying your points and monitoring your colleagues’ understanding. 

Asymmetric stream switching also appears if you use a virtual reality headset. When you enter the virtual world, you partially blind yourself to cues in the real world. To avoid tripping over a nearby chair, you need to switch streams consistently to monitor both the virtual world and the real world. If you carefully arrange the furniture in your room beforehand, creating a safe space for virtual exploration, then you can reduce the mental workload required to monitor that stream and focus more on the virtual stream with less overhead. 

Analogously, as supported by research on stereotype threat and Goffman’s idea of roles that people play in life, when people have a metaphorical “safe space” for collaboration with others despite diverse backgrounds, they can focus less on monitoring the stream of how they’re being perceived (“Will I say something offensive?”) and more on the stream of the collaborative task at hand. 

Finally, consider the number of people who have difficulty speaking with each other because of their dramatically different beliefs about what is true. This is a stream-switching problem with high asymmetries. It has become more difficult than ever before to imagine what it’s like to be on the other side, because the affordances to do so barely exist. Only by investing significant time in creating alternative social media accounts and filling them with clicks could someone start to experience the other’s perspective. The cognitive load of stepping into the shoes of the other person has become so high that it’s easier to discount them as foolish or deceived. It’s easier to stream switch and model the perspective of someone similar to yourself. In part, that’s why similar people are drawn to one another. But there is a very high likelihood that many people we work with, as well as our customers, are not similar to us. Research on the “contact hypothesis” shows that talking with people who are different than ourselves enables us to understand their perspectives with more empathy, even if it’s difficult. More than ever before, we need to increase our stream-switching abilities, which will enable us to understand others. 

Thanks to Joanne Marshall and Kaitlyn Ouverson for thoughtful feedback on these ideas. 

Stephen Gilbert