ACM Interactions

Authors: Lily Anfang, Maya Flynn, Kaitlynn Gray, Ethan Sandoval, Lynn Kirabo
Posted: Mon, December 16, 2024 - 12:39:00

---

Offering research experience to undergraduates is a well-known path to broadening participation in research fields [1]. We are a team of four undergraduate students, with experience working in areas ranging from environmental policy to tax filing and human-centered design projects. We spent the summer of 2024 working in the CACTI Lab at Harvey Mudd College. This piece articulates our expectations as early-career researchers, and the things that surprised us.

At various points during the experience, we had lapses in confidence. This imposter syndrome was constantly in the back of our minds, resulting in the questioning of our qualifications and the quality of our work. Often, it was difficult to see our contributions as valuable when all we could notice were the potential shortcomings. This fear of inadequacy makes the labels “researcher” and “academic” feel ill-fitting. Our perception of human-centered computing research compounded that feeling. While our peers analyzed mathematical and computational models, coded drones to deactivate nukes, and developed ways to integrate AI into computer science curricula, we were focused on a people-first agenda. The program we joined was in its early stages, and our task was to write a survey paper that would serve as the foundation for future research. It required sitting and reading for hours and learning about lived experiences reported in literature that were scattered across multiple domains and venues. This difference between our tasks and our peers’ work made us feel like we were not conducting proper research.

More specifically, our project involved understanding environmental justice, focusing on disability through a systematic review of academic journals, government reports, activist spaces, and more. The breadth of sources was intentional, as academic research is sometimes criticized for presenting novel findings from nonacademic practitioners. Using computers and coding never came up. At this stage in our research, our “codes” were key terms used to analyze data and not the Python scripts we were used to. Despite the fancy CS hacking research narrative that Hollywood sells, however, the research we conducted is still valuable.

After reviewing information that the previous research team curated, we compiled the lived experiences that speak to the human in HCI. One cannot develop a technological solution without knowing the problem. Throughout our research, we saw the same complaints and recommendations show up repeatedly, yet nothing seemed to have been done to address them. As such, our team hoped to address this gap and provide direction.

Beyond our research topic, our team learned much about academia. Going into this project, we anticipated working with or collecting large amounts of data—a process that we romanticized as groundbreaking. Instead, we reviewed a large number of sources. The sheer volume was intimidating and mildly concerning, given that every source claimed this topic lacked research. We read at least 24 literature reviews and 100 academic sources, as well as numerous nonacademic ones. As our work continued, the themes and findings that make literature review a foundational step in research began to emerge. 

One of the benefits of doing this type of review is that we were immersed in people’s writing. We found it interesting to see how many sources a paper would eliminate. Some started with several thousand sources but ran a content analysis on fewer than 12 relevant ones. We noticed a surprising trend with some literature reviews that had scanned titles and abstracts rather than examining entire papers to determine relevance. Perhaps the most shocking thing was the exclusion of certain perspectives. Most literature reviews were peer-reviewed journals and other scholarly sources. This had consequences, however: It neglected the knowledge of disabled individuals and activists, who were primarily published in nonacademic venues.

Overall, this process was fascinating. We got a level of individualized tasks and trust that we had not experienced in our work before. We learned a lot about this research and how to actively contribute to the academic community. 

Beyond that, our research was also about personal enjoyment and achievement. Being an undergraduate researcher means we are doing this work because we are interested in it. Without the intense publishing pressure that surrounds academia [2], we had the privilege of learning about the process. Fundamentally, our research is about sustainably uncovering new information about environmental justice. We hope to share our learnings with others and further contribute to expanding accessibility.

Endnotes

1. Sunshine, J. and Velez-Ginorio, J. Research experiences for undergraduates are necessary for an equitable research community. Communications of the ACM 67, 8 (2024), 26–28. 
2. Mhaidli, A. and Roemmich, K. Overworking in HCI: A reflection on why we are burned out, stressed, and out of control; and what we can do about it. Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2024, Article 561, 1–10. 

---

Posted in: on Mon, December 16, 2024 - 12:39:00

Lily Anfang

Maya Flynn

Kaitlynn Gray

Ethan Sandoval

Lynn Kirabo

Authors: Teresa Macchia, Amelia Bourton
Posted: Thu, October 24, 2024 - 10:28:00

---

In the year of the AI, as defined by Melissa Heikkilä [1], it is more urgent than before for asset-intensive manufacturing (AIM) companies to find their way through the maze of digitization. Their success in this process will ensure resilient business growth while reducing individual uncertainty and—in some cases—dystopian images about the future.

AIMs includes oil and gas, aerospace, and pharmaceutical businesses that rely on intensive productivity and high-value and complex assets [2]. Implicitly, when referring to AIMs, we often consider a set of functions that make things possible: from people services to finance and research and development to production. Compared with flexible and consumer driven e-economy, AIMs depend on interrelated activities that require greater coordination and strategic effort to sustain and enhance market competitiveness [3]. Moreover, digital transformation influences how these organizations shape their processes, depending on how they collect, store, operate, and utilize data across different devices, platforms, and services. 

If on one side digitalization aims to enhance business operation via process automation, optimization, and reconfiguration, on the other side it drives innovation by triggering product and service development. Hence, the challenge behind the rise of digitalization is often tied to employees’ ability to independently keep pace with ongoing transformations. To paraphrase Christian Matt et al. [4], the success of digital transformation is about balancing four dimensions: the adoption of technology, the expected value, the structural changes, and the financial dimension. A fifth, and perhaps core, dimension that we often take for granted is the impact on employees’ work-life balance.

Digitalization in AIM, namely the introduction of digital technologies to increase productivity or to boost product development, succeeds when employees are actively involved in the process. It’s a type of involvement that overcomes hierarchical barriers and involves everyone. In fact, digitalization isn’t limited to the adoption of the latest fancy AI-powered enterprise resource planning systems (ERP) or the flashy real-time performance analytics of service delivery software. Digitalization in AIM embeds “agility, efficiency, flexibility, collaboration, automation, innovation, and productivity, [while increasing] ambiguity, security concerns, employee anxiety, privacy issues” [5]. The question is how AIM companies can embrace stepping into the fastest spinning digitalization year(s) ever?

Equipping AIMs for Intensive Digital Transformation
Example 1: Revamping a cross-functional team. To address this challenge, all the PD/HCI/agile tricks of the trade had been leveraged, but the team’s mood remained grim. Members were applying for other roles, the moral was low, and gossip was flowing. It took a few months to get to the core of the problem: The team members were overwhelmed and stretched too thin. Digital tools were adopted hastily, much like applying band aids to a knife wound. In order to solve the problem, the team was asked to become an active participant in the digitalization, which resulted in a standardized flow that rebalanced the impact of mundane tasks and collaborative governance. Effective actions enabled consistent and robust knowledge-exchange opportunities. All the ideas came from the team, which only needed an initial collaboration kick-off. 

Collaboration and knowledge sharing fueled the digital engine, driving successful technology adoption. By the end of the experience, the team had developed an entirely new mindset. One member said that her “life was ruined,” since she now knows that a different way of working is possible. The takeaway here is that even the best tools on the market are ineffective without collaborative, inclusive governance and a receptive culture, both from the bottom-up and top-down.

Example 2: Knowledge sharing. One of the many challenges AIMs face is advancing a knowledge infrastructure that fosters the growth and establishment of connections among people, artifacts, and contexts, enabling the maintenance and development of organizational knowledge. The core challenge of knowledge in AIM lies in the traditional adoption of a rigid stage-gated approach to production and service maintenance, leaving little to no room for creativity and knowledge development. Accepted knowledge is formalized and excludes informal know-how. Knowledge is formally captured in silos and often duplicated. It can be difficult to access, and information is difficult to navigate, as data may be outdated, unrecorded, or impossible to extract due to obsolete technologies. As a result, knowledge dissemination frequently depends on employees’ organizational memory and technical knowledge. Some technical and “historical know-how” may retire or leave businesses with limited time to transfer competencies. 

An organization we’ve worked with stored information in multiple repositories, making it difficult to determine what to look for, when, where, and how. Consequently, measures were implemented to reduce fragmentation, silos, and mundane tasks by, for example, using a tailored intelligent search engine. As advanced digital solutions found their place within the organization, the risk of digital resistance grew. New challenges arose, focusing on the ability to bring employees together in a constructive collaborative governance framework and ensuring that the digital tools were suitable for specific needs, rather than creating new scopes to justify further implementations. 

It soon became clear that a one-size-fits-all digital solution would likely be insufficient for knowledge transfer, as learning style preferences and accessibility needs vary among different employee groups. Ultimately, digital transformation needs to balance technological solutions and a human-centered approach to address the challenges of the future.

Conclusion
A lot has been written around digitalization in asset-intensive manufacturing. However, as insiders, we bring a different perspective on the potential effectiveness of digital adoption. Over the years, we came across both successful and unsuccessful digitalization processes. The successful initiatives embraced a collaborative and training-oriented journey that focuses not only on technology but also on fostering cultural change and encouraging technology acceptance.

Among the successful stories of digitalization, a common thread is collaboration and knowledge sharing, coupled with opportunities to upskill and reskill employees based on the required technology. Unfortunately, there is no one single solution to the challenges of this revolutionizing era. However, we can say that the key to success lies in aligning organizational needs, technology availability and readiness, and employee needs.

Endnotes
1. Heikkilä, M. What to expect from the coming year in AI. MIT Technology Review. Jan. 9, 2024; https://www.technologyreview.c...
2. Loonam, J., Eaves, S., Kumar, V., and Parry, G. Towards digital transformation: Lessons learned from traditional organizations. Strategic Change 27, 2 (2018), 101–109.
3. Verhoef, P.C. et al. Digital transformation: A multidisciplinary reflection and research agenda. Journal of Business Research 122, 2021, 889–901.
4. Matt, C., Hess, T., and Benlian, A. Digital transformation strategies. Business & Information Systems Engineering 57, 5 (2015), 339–343.
5. Kraus, S., Ferraris, A., and Bertello, A. The future of work: How innovation and digitalization re-shape the workplace. Journal of Innovation & Knowledge 8, 4 (2023). 

---

Posted in: on Thu, October 24, 2024 - 10:28:00

Teresa Macchia

Amelia Bourton

Authors: Tyechia L. Thompson , Eric Lyon, Wallace Lages
Posted: Tue, October 15, 2024 - 3:12:00

---

Stories are ubiquitous, transcending traditional media and books, permeating our self-talk and scientific publications. Immersive technologies are no exception. Storytellers and artists are using spatial audio, virtual reality (VR), and augmented reality (AR) more frequently to create and share narratives about the past, the present, and the future. What sets immersive technologies apart from their predecessors is their ability to create embodied experiences that evoke deeper and more enduring emotional responses [1]. They are also unique in the way they can influence empathy through perspective taking and immersion [2].

Immersive technologies, however, are not without their ethical dilemmas. Empathy can inadvertently lead to unjust behaviors [3]. Moreover, as with other digital media, those who create or disseminate content using these technologies have opportunities to shape discourse, influence culture, and accumulate wealth. It is crucial, therefore, that we approach the rise of immersive technologies with caution and intentionality, ensuring they do not accentuate existing disparities [4]. Instead, they should be harnessed as tools to create opportunities, amplify underrepresented voices, address issues of access, and foster mutual understanding among diverse communities. 

Listening to Multiple Voices
With the support of a Digital Justice Seed Grant from the American Council of Learned Societies, we assembled a group of practitioners and researchers in storytelling and immersive technologies to explore these concerns. In selecting this group, we aimed to achieve a diverse representation of background, experience, academic focus, and artistic practice. Our advisory board consisted of Dr. Riham Alieldin, a physician and medical educator at the University of Rochester; Bryan Carter, director of the University of Arizona Center for Digital Humanities; Damon Davis, a multimedia artist, musician, and filmmaker; Al Evangelista, a dancer, choreographer, and assistant professor of dance at Oberlin College and Conservatory; and Ashley Shew, an associate professor of science, technology, and society at Virginia Tech. Their collective expertise spans film, visual arts, opera, podcasts, dance, science, medicine, Africana studies, community building, arts administration, disability studies, and gender studies. 

From left to right: Amanda Hodes, graduate research assistant, and grant principal investigators and advisory board members Bryan Carter, Ashley Shew, Riham Alieldin, Tyechia Thompson, Damon Davis, Eric Lyon, Wallace Lages, and Al Evangelista at Virginia Tech. (Photo credit: Alice Rogers, Media Design Studio Manager, Virginia Tech.)

Our journey as a group began with a visit to Virginia Tech in August 2022, where the board members met in person for the first time and experienced immersive performances at Cube Fest 2022: Afrofuturism, a three-day festival exploring the frontiers of immersive media and Black futurism. They also toured multiple state-of-the-art technology facilities to create a common ground for the follow-up discussions. Over the next 12 months, we held group and individual conversations around empathy, narrative, access, and ethics. Some of these conversations are captured in the seven-episode podcast series The Inside Story, in which we also explored the practical question of how to build an institute to support technical competency and promote awareness of the ethical aspects of immersive storytelling.

Three Concerns for Immersive Storytelling
As we explored several questions around representation and empathy, three recurring issues that affect immersive storytelling emerged. One is how to create genuine experiences that connect people to a culture as welcomed guests, not exploiters. Our conversations with Damon Davis pointed toward the importance of attentive listening, individual authenticity, and mutual understanding, particularly in community engagement. He emphasized the necessity of meeting people where they are and facilitating empathy through new myths and paradigms. This includes avoiding being extractive of underrepresented narratives and engaging with individuals directly to understand their needs. Ashley Shew urged creators to reconsider the concept of empathy and first evaluate values within the community. She also highlighted the need to move from emotional experiences to action-based engagement and to acknowledge the context, history, and power relations within empathy discussions. At the same time, Al Evangelista pointed out that risk is inherent in art, and that intimacy, even when simulated through sound and eye contact, raises questions about community, hierarchies of information, and ethical curation. He cites his Dancing Lab project as an example of the interplay between ethics, risk, and power in community artmaking.

A second issue is related to the need to properly contextualize immersive stories. The more participatory and realistic the experience is, the more necessary a debrief may be to create a healthy distance from the narrative, especially if the experience is emotional or triggering. This requires that storytellers create comprehensible immersive narratives, even when users interact with them in a less than ideal way. Implementing onboarding processes before the immersive experience and debriefing processes afterward can be especially beneficial for users new to immersive technologies. Riham Alieldin discussed the phenomenon of the “disintegration of empathy and compassion” among medical students as they progress in their training. While simulations that offer real patient perspectives can improve interactions between medical professionals and patients, they may cause users to feel like they “know” the experience for themselves, potentially leading to generalizations. Alieldin stressed the importance of debriefing to avoid generalizations about people’s experiences, and the challenges of separating novelty effects from genuine, long-term changes. 

A third concern focuses on access. Immersive technologies are still expensive, which limits access to a small segment of the population. Additionally, there is a skill gap for certain types of production. How can we enable competency building in underrepresented communities so they can tell their own stories? The group discussed ways to support these creators, ranging from workshops to integrating competency building as an outcome in larger project grants. From his experience engaging with participants unaffiliated with academic institutions, Bryan Carter shared the importance of building relationships that can support project integrity and equitable resource sharing in the face of a partner’s unique ethical perspectives. This includes considering ways to create accessibility in immersive technologies, including the use of multiple modalities to accommodate individuals with special needs.

Conclusion
Immersive experiences in spatial audio, VR, and AR offer a potent medium that can revolutionize how we tell stories, connect with one another, and experience the world. As adoption increases, we need to maintain an ongoing conversation about ethical and equitable uses of these unique technologies. Although there are not clear answers to many of the challenges ahead, we should start by acknowledging the power and responsibility of storytellers, technologists, artists, and scholars so that we can harness immersive stories as a transformative tool toward inclusion and equity. 
 

Endnotes

1. Wincott, A., Martin, J., and Richards, I. Telling stories in soundspace: Placement, embodiment and authority in immersive audio journalism. Radio Journal: International Studies in Broadcast & Audio Media 19, 2 (2021), 253–70.
2. Cummings, J.J., Tsay-Vogel, M., Cahill, T.J., and Zhang, L. Effects of immersive storytelling on affective, cognitive, and associative empathy: The mediating role of presence. New Media and Society 24, 9 (2022), 2003–26. 
3. Jackson, L. Empathy reifies disability stigmas. Keynote speech, Interaction 19, Feb. 6, 2019, https://interaction19.ixda.org...
4. Franks, M.A. The desert of the unreal: Inequality in virtual and augmented reality. UC Davis Law Review 51, 2 (2017), 499–538.

---

Posted in: on Tue, October 15, 2024 - 3:12:00

Tyechia L. Thompson

Eric Lyon

Wallace Lages

Authors: Samiya Ali Zaidi
Posted: Tue, August 20, 2024 - 11:55:00

---

To get started, we partnered with the Autism Spectrum Disorder Welfare Trust (ASDWT) in Pakistan to create a unique dataset. It will include eye-tracking data from Pakistani children, reflecting a diverse range of ages, genders, and other demographic factors. The diversity in our dataset is crucial because it helps ensure that our model is robust and can be generalized across different populations. Collecting this data will be a collaborative effort, involving the ASDWT, parents, and other stakeholders who provided informed consent and valuable insights. Once the dataset is collected, we will prepare it for analysis. This step includes pre-processing the data, removing noise, and normalizing it to ensure consistency. These features will play a vital role in training our computer vision-based model to detect patterns that may indicate ASD.

In addition to eye-tracking data, we recognize the importance of incorporating textual inputs to improve the accuracy of our model. This step involves gathering information from parental reports, medical histories, and other relevant checklists. By integrating this textual data, we can provide our model with a more comprehensive understanding of each child's development, allowing for a more accurate detection of ASD-related patterns. Textual inputs add context to the eye-tracking data, helping identify correlations between visual behavior and other developmental cues. This approach aligns with HCI's focus on creating systems that understand and respond to human behavior in meaningful ways. By combining visual and textual data, we aim to create a model that not only detects ASD with greater accuracy but also helps healthcare professionals and families make informed decisions about treatment and support.

The heart of our project lies in building and training a custom computer vision-based model. Given the complexity of ASD and the variability of eye-tracking patterns, we understand that a robust model is essential. To achieve this, we will split our dataset into three subsets: training, validation, and testing. The training subset, comprising approximately 70 percent of the data, is used to teach the model to recognize patterns. The validation subset (20 percent) helps us fine-tune and optimize the model's hyperparameters. Finally, the testing subset (10 percent) is where we evaluate the model's accuracy and generalizability. The training process involves applying state-of-the-art computer vision techniques to analyze the eye-tracking data. We will also use transformer-based architectures to explore different approaches to pattern recognition. By testing multiple architectures, we aim to determine which methods are most effective in identifying ASD-related patterns.

After training and testing the model, we moved on to the analysis phase. This step involves evaluating the model's performance using various metrics. We compared our results with existing studies to gauge the effectiveness of our approach and to identify areas for improvement. During this phase, we paid special attention to how variations in visual stimuli and experimental conditions influenced the model's accuracy. We also considered the potential impact of demographic factors, ensuring that our model could generalize across different groups. This analysis is crucial, as it helps us understand the limitations of our approach and provides insights into how we can refine the model to improve its reliability.

The findings from our project, as seen in the initial reports, will have the potential to significantly impact early ASD detection. By combining eye-tracking data with textual inputs, we can create a model that offers earlier and more accurate diagnoses, which can lead to more effective interventions. This directly benefits healthcare professionals and families by providing them with tools to support children with ASD promptly. Moreover, our work demonstrates the potential of Human-Computer Interaction in healthcare. By leveraging computer vision and deep learning, we can address complex problems like ASD diagnosis with innovative solutions. This approach has implications beyond ASD, showcasing how HCI can play a pivotal role in improving diagnostic processes for other neurological and developmental conditions.

As we continue to refine our model, we are excited about the possibilities ahead. Our findings could guide future research into applying HCI and computer vision to a broader range of diagnostic scenarios, ultimately leading to more reliable and efficient methods for early diagnosis. We believe this project is just the beginning of a journey toward a more inclusive and effective approach to healthcare. By incorporating HCI into the process, we're not just creating technology for technology's sake; we're creating solutions that can make a real difference in people's lives. And that's what makes this work truly exciting.

---

Posted in: on Tue, August 20, 2024 - 11:55:00

Samiya Ali Zaidi

Authors: Ather Sharif
Posted: Thu, August 08, 2024 - 12:45:00

---

There are at least 1.3 billion disabled people worldwide [1]. Their disabilities include blindness, hearing, mobility, and cognitive, among others. These individuals are disabled—if they choose to identify as such—but what they are not is “impaired.”

Word cloud of impaired and its related words created using Generative AI.

Correct usage of terminology is important. There exists a prodigious, if not universal, agreement that the terminology we use to describe people holds significant weight in increasing inclusivity for marginalized groups, especially disabled people [2]. These words represent the speakers’ commitment to cultural awareness of and sensitivity toward vulnerable populations [3]. For the latter, they express respect and a sense of belonging to a world that continues to disenfranchise them. Much like fashion, terminologies have a shelf life, after which they slowly become archaic or are considered outdated due to their negative connotations (e.g., retarded). But unlike fashion, if declared pejorative, the terminologies’ return to popular usage is unwelcome at best, unless done so purposely by the targeted demographic to reclaim the narrative (e.g., crippled and gimp).

Reclaiming the narrative. Reclaiming the narrative begins with a word being just a word, like any other word in the dictionary. Then, a creative person comes along and uses it as an adjective to describe a person or a group of people they think they are superior to. Other “superiors” agree, and the word then becomes an identity of the “inferiors.” Time goes by, inferiors get a platform to vocalize their disagreement with the word, superiors demonstrate resistance and continue using it, and finally, after years of exhausting advocacy, the word gets flagged as offensive. It stays offensive, and people, within and outside the targeted demographic, avoid its usage. Then, people within the targeted demographic take ownership of the word to empower themselves, remembering the oppression they faced, which led to the association of the word with their demographic in the first place. People outside the demographic, however, are barred from using the word, particularly with those within the demographic. (The barring has never been enforced as a law, though, and only serves as a recommendation, with the likelihood of the speaker being “canceled” for using the word.)

The word impaired has not yet been reclaimed. The disagreement, however, has been vocalized. Resistance to its cessation continues.

Definition and impact of impaired. The Oxford English Dictionary defines impaired as “injured, weakened, damaged; diminished or deteriorated in amount, quality, or value.” Merriam-Webster’s Collegiate Dictionary describes it as “diminished in function or ability.”

Disabled people already know what their disabilities and limitations are, with many having embraced it as a part of their identity. They do not need a constant reminder of their disabilities. Every time I type the word impaired for this blog post, a part of me feels dejected and wants to give up on the utopian future where disabled people are not distinct from those who are “normal.” But I digress.

Using words like impaired can result in disabled people internalizing their self-identity as being broken, or even an “incomplete” human. These words can foster a subconscious degradation in confidence and pose hurdles in a disabled person’s journey toward accepting their identity as a disabled person. They can create emotional scars, which, if externalized, could develop into more negative stereotypes, such as disabled people being “sappy” or lacking “thick skin.” The impact of these terms can be long-lasting. 

Arguments for using impaired. I hesitate to believe, or rather to admit, that on a human level, HCI researchers and authors are entirely oblivious to the effect of words on human emotions. Movies, TV shows, and books thrive on this effect. Encouraging and respectful words produce harmony. Disrespectful and prickly words trigger emotional distress. Neutral words are often harmless. So why do people still use impaired to refer to people?

Argument 1: It’s an established term used in medical journals and prior accessibility research. We’re just following the standards. This reasoning is based on facts. Impaired is used quite extensively, although the field of medicine is attempting to rectify that [4,5]. But if we dig deeper into the literature, we will find frequent usage of other terms now considered derogatory. There was a time when those words were acceptable…and then they weren’t. There are two choices: keep using impaired until it reaches its shelf life or preemptively avoid the term and be better allies to the disability community. History has a funny way of teaching us things, but only if we are willing to learn. The lesson here is to be proactive catalysts for positive change and not let the change happen as a mandatory reparations measure. 

Argument 2: A person I know is perfectly fine with its usage, so this is not a problem. N = 1 has its place, but not for this argument. Everyone has their preferences; some people are sensitive, some not so much. Focusing on those not affected by the usage of impaired and letting that motivate its continued usage is arguably an invalidation of those who are sensitive to its usage. Our societal goal has always been to validate and support vulnerable and disenfranchised populations. It makes sense to listen to the voices of those who find discomfort in being labeled as impaired. It is improbable that a more accepting replacement would be unwelcomed by those not affected by the usage of impaired. At its 1993 annual convention, the National Federation of the Blind advised against the usage of visually impaired and instead advocated using blind and low vision, employing identity-first language [6]. Yet, 31 years later, visually impaired remains a widely used term.

Argument 3: I never really thought about it; I just used it because my advisor did. What can I do moving forward? This is not necessarily an argument, but it provides valuable insights into some people’s thought process. The reasoning behind this view makes sense too. The desire to be better is a gold star. But some questions remain. For example: How can we better communicate our discomfort with word choices without exhausting ourselves, such that it is baked into our thought process from the start? Whose responsibility is it? How can we build a culture where we are encouraged to challenge an established belief, where there is constant learning and unlearning? I do not know the answers to these questions, but I suspect the answers involve being less resistant to change and actively seeking feedback from a community on how they wish to be referred to.

Where do we go from here? This blog post is a call to action to cease referring to disabled people as impaired. But that is not all I intend. I urge HCI researchers and authors to employ practices and processes that allow for rethinking and challenging the terms they use to refer to vulnerable and disenfranchised populations. I believe it when people say they wish to be better allies; using respectful terminology is a step toward that. Giving some thought to people-describing adjectives before using them can help. I hope this post sparks thoughtful conversations on this subject and helps identify avenues for authors to use more-inclusive words.

Endnotes
1. World Health Organization. Disability. March 7, 2023; https://www.who.int/news-room/...
2. Sharif, A., McCall, A.L., and Bolante, K.R. Should I say “disabled people” or “people with disabilities”? Language preferences of disabled people between identity- and person-first language. Proc. of the 24th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, New York, 2022, 1–18.
3. Best, K.L., Mortenson, W.B., Lauzière-Fitzgerald, Z., and Smith, E.M. Language matters! The long-standing debate between identity-first language and person-first language. Assistive Technology 34, 2 (2022), 127–128.
4. Goddu, A.P. et al. Do words matter? Stigmatizing language and the transmission of bias in the medical record. Journal of General Internal Medicine 33, 5 (2018), 685–691.
5. Raney, J. et al. Words matter: An antibias workshop for health care professionals to reduce stigmatizing language. MedEdPORTAL 17 (2021), 11115.
6. National Federation of the Blind. Convention Resolutions 1993: Resolution 93-01 https://nfb.org/sites/default/...

---

Posted in: on Thu, August 08, 2024 - 12:45:00

Ather Sharif

Authors: Ava Elizabeth Scott, Leon Reicherts, Evropi Stefanidi
Posted: Tue, April 16, 2024 - 10:30:00

---

The deadline for the CHI conference is an important event on an HCI researcher’s calendar. For CHI 2023, more than 3,000 papers were submitted and only 28 percent were accepted. With such a high number of submissions and low acceptance rate, researchers face pressure to make the best submission possible. So how can we best introduce early-career researchers to the charms and quirks of writing a CHI paper while supporting them in submitting the best possible paper they can? 

For the past two years, the University of St. Gallen in Switzerland has hosted a writing retreat called CHI Together, which brings researchers together for two and a half weeks directly prior to the CHI deadline. While many of these researchers are experienced CHI authors, some are engaging in this process for the first time. During the inaugural writing retreat held in 2022, attendees submitted a total of a dozen papers. The idea behind the retreat was to help participants collaborate efficiently and manage the stressful phase before the deadline in the company of like-minded people; and then to celebrate their submissions. Hoping to reiterate and build on this success, this year the computer science department at St. Gallen invited 21 researchers to come together, physically and collaboratively, in the run-up to the deadline for CHI 2024. 

This endeavor was made possible by funding from the SIGCHI Development Fund and from the University of St. Gallen. We are hugely grateful for this support, which facilitated the many healthy social activities and group meals, as well as financed accommodation and travel expenses. Here we describe these activities, starting from our first day arriving in St. Gallen to the final delirious day of submissions and subsequent celebration. We then reflect on the intensity of the CHI deadline and recommend that early-career researchers be supported with social and recreational resources, as well as pragmatic and technical measures.

Arriving by plane and train on a warm day in late August, we were immediately struck by the fact that these services were on time. As students from the U.K. and Germany, Swiss timeliness is always a nice surprise. We reunited with the other visiting researchers, new and familiar faces alike. Our apartments were a 10-minute walk from the computer science building in the center of St. Gallen, halfway up a steep hill. While it was an easy jaunt down to the office, there was a significant disincentive to leave the office: the hike between you and your bed. Though this felt like a slog at the time, the daily exercise surely benefited us. These apartments were our home for the next two and a half weeks, but we spent much more time at the office.

Sharing an open office environment catalyzed natural collaboration, as well as companionship and social breaks during the workday.

In the computer science building, we each had desk space with a display monitor and a comfortable chair. In anticipation of the numerous hours we would spend writing, an ergonomic setup was a priority. In this open office setting, students collected data, analyzed it, and scratched their heads debating over the structure and presentation of their results. 

A selection of irreverent and humorous Ph.D. memes were spread over the desks.

Knowledge exchange sessions were organized to support these processes, including sessions on qualitative analysis and how to write a CHI paper. We swapped abstracts and drafts with one another to get feedback on our efforts. In addition to receiving guidance on analysis and writing, we also created and shared analytical tools, such as a transcription and speaker-segmentation pipeline, which we used to transcribe more than 100 interviews across different research projects. Using the wall-size whiteboards around the office, we collectively sketched and resketched papers’ structure, argumentation, models, and figures. By distributing our thinking across multiple brains and the physical space, we could work through hard problems with greater speed and clarity.

This intensive brain activity required fuel. Most days, we took turns making group lunches in the lab kitchen—usually an elaborate salad with fresh bread and cheese. By sharing lunch duties, we reduced the overall time spent on meal preparation, while also eating well. As the deadline got closer and the days became longer, we frequently honored the tradition of grabbing doner kebab or Thai takeaway for dinner.

Paweł W. Woźniak presented the key elements of an abstract.

There were also several optional activities that provided a much-needed break. We hiked the nearby mountains, took a dip in the local mountain lakes, indulged in manic dance and workout breaks, went to the cinema, and shared pizza and burgers in the evenings. Furthermore, a contingent of researchers attended the Mensch und Computer conference in nearby Rapperswil, enabling further exchange with the D-A-CH (Germany, Austria, and Switzerland) HCI community. Postdocs and those in their final year of Ph.D. research also attended an information session about research funding opportunities across Switzerland and the rest of Europe, organized by the research office at St. Gallen. This balance of intellectual, recreational, and forward-looking activities helped balance the otherwise intensive writing schedule.
 
Immersed in this culture of dedicated focus, we collectively submitted more than a dozen papers for CHI 2024. One supervisor said that their students’ writing during this period was the clearest and most effective of their studies so far. While some of us were clicking the submit button at the very last moment (2 p.m. on September 15 in our time zone), we all got our papers in on time and celebrated with drinks and snacks in the lab. Despite many of us having pulled all-nighters, the drinks continued until 2 a.m. We said heartfelt goodbyes, and left for our respective destinations the following morning, looking forward (with slight trepidation!) to receiving our reviews in a few weeks.

The collective whiteboards were the site of active collaborative thinking, sketching iterations of models, figures, and argumentation.

Large, top-tier conferences such as CHI are the critical sites for presenting work, receiving feedback, and networking with peers and future collaborators. However, due to the appropriately high standards set for paper acceptance and the cost of in-person attendance, such conferences are not always accessible to early-career scholars. The CHI Together model of collaborative writing before the deadline provides support in multiple ways, offering an encouraging, knowledgeable community to help steer research work and frame completed work following best practices for different submission types. CHI Together also supports recreational activities that allow early-stage scholars to network and build community; we believe that such community building means scholars will have an established cohort for continued mutual support in their everyday work setting as well as at conferences, when they are able to attend them. These meaningful relationships with peers are perhaps even more lasting than connections made at conferences.

Plans for CHI Together 2024 are in the works, potentially hosted elsewhere in Europe, but aiming to maintain the same level of focus, support, and solidarity in the run-up to the deadline for CHI 2025. For those of you interested in hosting a similar writing retreat, and who would appreciate some guidance or blueprints, don’t hesitate to get in touch with us or Johannes Schöning of the University of St. Gallen. Or if you are running similar events, please do share your approach with us.

---

Posted in: on Tue, April 16, 2024 - 10:30:00

Ava Elizabeth Scott

Leon Reicherts

Evropi Stefanidi

Authors: Muhammad Zahid Iqbal, Abraham Campbell
Posted: Wed, March 06, 2024 - 3:00:00

---

By providing realistic and engaging learning experiences, immersive technologies such as virtual reality (VR), augmented reality (AR), mixed reality (MR), 360-degree video, and simulation have proven to be effective in industrial training contexts. These technologies have been adopted for various purposes, including healthcare [1], teacher training, and nuclear power plant training [2].  

Climate change is a global challenge that requires coordinated action from all sectors of society. In order to combat climate change, CO2 emissions need to fall to zero. One of the key strategies in addressing this challenge is to achieve net zero carbon emissions by 2050 [3]. Essential to achieving this goal is the adoption of net zero carbon infrastructure [4], which can reduce reliance on fossil fuels, enhance the efficiency and resilience of energy systems, and support the development of low-carbon technologies in industry. Net zero carbon infrastructure includes renewable energy resources such as wind, solar, nuclear, and hydro; low-carbon transport modes such as electric vehicles; and carbon-capturing, energy-efficient buildings [5] and appliances. There is an urgent need to invest in net zero carbon infrastructure to improve air quality, promote healthy lifestyles, and improve social equity and environmental justice. Net zero carbon infrastructure is therefore not only a necessity but also an opportunity to build a more sustainable and prosperous future for all. This post explores the opportunities and challenges of using immersive technologies for net zero carbon infrastructure training.   

Main components of net zero carbon infrastructure.

Immersive technologies can potentially revolutionize training for net zero carbon infrastructure development and maintenance. Potential opportunities include:

• Reducing training costs, risks, and errors by eliminating the need for physical resources, travel, and instructors.
• By using immersive technologies, trainees can learn and practice the skills and knowledge required for net zero infrastructure in a realistic and immersive way without the need for physical resources, travel, and instructors. A perfect example of this is in wind farms, where maintenance of a turbine may take place over 50 meters above the ground. Immersive technologies can save time, money, and resources, as well as improve safety and quality.
• Creating realistic simulations of dangerous or hazardous environments that workers may encounter in their jobs, such as oil spills, fires, or explosions. By using immersive technologies, workers can learn how to handle these situations safely and effectively without exposing themselves to physical harm. Therefore, immersive technologies can enhance the safety and efficiency of net zero infrastructure training.
• Using immersive technologies in net zero infrastructure training can provide data and feedback on learner performance, behavior, and progress. Trainers can monitor and assess how learners interact with the virtual environment, what choices they make, how they solve problems, and how they apply their skills and knowledge. This data and feedback can be used to improve training design and delivery by identifying strengths and weaknesses, providing personalized guidance, and adapting the level of difficulty and complexity.
• Enhancing the learning experience by creating a more engaging and interactive environment for learners or trainees. By providing learners with realistic and immersive scenarios, these interventions can stimulate their senses, emotions, and cognition, as well as provide immediate feedback and guidance. Therefore, immersive technologies can help to foster a more effective and enjoyable learning process.
• Leveraging the latest revolution ingenerative AI integration, immersive training can be made more productive, personalized, and content-driven by instructional designers. As generative AI moves forward, its ability to facilitate training will only improve in its ability to generate text, image, video, and even 3D avatars to assist the trainee.
• Using haptic devices in immersive training is the next step forward, as compared with touchless hand interaction or nonhaptic, controller-based interaction it can achieve greater realism within a training environment. The haptic sense in training has been proven to be one of the best feedback methods for users.

There are also challenges posed in adopting these technologies that need to be addressed. Some of the main challenges include:

• Acquiring immersive training resources for the first time can be costly and resource intensive because it requires specialized equipment, software, and content development. Depending on the complexity and quality of the simulations, the initial investment and maintenance costs can be high, but most of this is a long-term investment.
• Implementing immersive training infrastructure may not be compatible with existing systems and platforms, which can create technical difficulties and integration issues. Also, immersive training facilities require more technical support and troubleshooting initially than other forms of training.
• Accessibility is a commonly discussed issue with immersive training. But with the latest developments in immersive hardware, immersive training material can be designed with accessibility and inclusivity in mind.
• Implementing new initiatives such as using immersive technologies for learning may face resistance from trainees, who may fear adopting new technologies. Therefore, there is a need for a convincing strategy, such as adopting the latest versions of the technology acceptance model (TAM) [6] and communicating about the value of immersive training, as well as providing adequate training and support.

All of these challenges are not insurmountable and can be overcome with careful planning, evaluation, and collaboration. It is worth considering how we might leverage these technologies to achieve net zero industrial training goals. One way to address the cost challenge is to develop more-affordable immersive technologies such as mobile AR/VR; even the makers of the latest devices are working on reducing the cost, such as the latest Meta VR headset Quest 3. As advances in XR devices proceed rapidly, with year-on-year improvements, Meta and other companies such as Microsoft and HTC should consider allowing users to return their old devices to be refurbished or recycled for a discount on new devices. This will reduce costs for XR users who want the latest devices and reduce the environmental impact.

Another way to address the challenge of accessibility is to make immersive training more accessible to people in developing countries. This can be done by providing training on how to use immersive technologies and by developing immersive training programs tailored to the needs of developing countries. Of course, there are challenges of usability, which can be addressed by designing immersive training programs that are easy to use and more user-friendly.  

Case studies from different industrial training contexts show immersive technologies have the potential to play a significant role in training for net zero carbon infrastructure. By addressing the challenges, we can ensure that immersive technologies can help revolutionize net zero workforce training for a sustainable future.

Endnotes
1. Brooks, A.L. Gaming, VR, and immersive technologies for education/training. In Recent Advances in Technologies for Inclusive Well-Being: Virtual Patients, Gamification and Simulation. Springer, 2021, 17–29.
2. opov, O.O. et al. Immersive technology for training and professional development of nuclear power plants personnel. Proc. of the 4th International Workshop on Augmented Reality in Education, 2021.
4. Bouckaert, S. et al. Net Zero by 2050: A Roadmap for the Global Energy Sector. International Energy Agency, 2021.
5. Kennedy, C.A., Ibrahim,N., and Hoornweg,D. Low-carbon infrastructure strategies for cities. Nature Climate Change 4, 5 (2014), 343–346.
6. Thomas, R.J., O'Hare, G., and Coyle, D. Understanding technology acceptance in smart agriculture: A systematic review of empirical research in crop production. Technological Forecasting and Social Change 189 (2023),122374.

---

Posted in: on Wed, March 06, 2024 - 3:00:00

Muhammad Zahid Iqbal

Abraham Campbell

Authors: Shuhan Sheng
Posted: Thu, January 04, 2024 - 11:32:00

---

In the AI universe, we find ourselves balancing on the edge of a tightrope between groundbreaking promise and perilous pitfalls, especially with those LLM-based platforms we broadly label as “AI platform.” The allure is magnetic, but the risks? They’re impossible to ignore.

AI Platform’s Information Interaction and Service Features: An Unsustainable Model
Central to every AI platform is its adeptness in accumulating, deciphering, and deploying colossal data streams. Such data, often intimate and sensitive, is either freely given by users or, at times, unwittingly surrendered. The dynamic appears simple: You nourish the AI with data, and in reciprocation, it bestows upon you tailored suggestions, insights, or solutions. It’s like having a conversation with a very knowledgeable friend who remembers everything you’ve ever told them. But here’s the catch: Unlike your friend, AI platforms store this information, often indefinitely, and use it to refine their algorithms, sell to advertisers, or even share with third parties.

The current mechanisms of information exchange between AI platforms and users can be likened to a two-way street with no traffic lights. On one flank, users persistently pour their data, aspiring for superior amenities and encounters. Conversely, AI platforms, with their insatiable quest for excellence, feast upon this data, often devoid of explicit confines or oversight. Such unbridled data interchange has culminated in palpable apprehensions, predominantly surrounding user privacy and potential data malfeasance [1].

Following this unchecked two-way data street, the unsustainable model now forces a dicey trade-off between “personalized experiences” and “personal data privacy.” This has led to a staggering concentration of user data on major AI platforms, at levels and depths previously unimaginable. What’s more alarming? This data pile just keeps growing with time. And let’s not kid ourselves: These AI platforms, hoarding mountains of critical user data, are far from impenetrable fortresses. A single breach could spell disaster.

One of the most recent and notable incidents involves ChatGPT. During its early deployment, there was an inadvertent leak of sensitive commercial information. Specifically, Samsung employees reportedly leaked sensitive confidential company information to OpenAI’s ChatGPT on multiple occasions [2]. This incident not only caused a stir in the tech community but also ignited a broader debate about the safety and reliability of AI platforms. The inadvertent leak raised concerns about the potential misuse of AI in business espionage, the risk of exposing proprietary business strategies, and the potential financial implications for companies whose sensitive data might be inadvertently shared.

Whether we like it or not, we need to see and face this fact. The current rapidly growing information interaction model based on traditional user data storage methods is unsustainable. It’s a ticking time bomb, waiting for the right moment to explode. And unless we address these issues head-on, we are bound for a digital disaster.

User Experience Design Based on This Mechanism: Subsequent Problems and Challenges
Beyond the lurking privacy threats and the looming digital apocalypse, this unsustainable info-exchange model is already a thorn in the user experience side. Let’s dive deeper into these annoyances to grasp the gravity of the situation.

The cross-platform invocation dilemma. Major AI platforms operate in silos, creating a fragmented ecosystem where user data lacks interoperability. With the advent of new models and platforms, this fragmentation is only intensifying [3]. Imagine having to introduce yourself every time you meet someone, even if you’ve met them before. That’s the predicament users find themselves in. Every time they switch to a new AI platform, they’re forced to retrain the system with their personal data to receive customized results. This not only is tedious but also amplifies the risk of data breaches. It’s like giving out your home address to every stranger you meet, hoping they won’t misuse it.

Inefficiencies in historical interaction records. The current AI models have a flawed approach to storing and managing historical interaction records [4]. Take ChatGPT, for instance. Even within the platform, one session’s history can’t give a nod to another’s. It’s like they’re strangers at a party. Users struggle to retrieve past interactions, making the entire process of data retrieval cumbersome and inefficient. This inefficiency not only frustrates users but also diminishes the value proposition of these platforms.

Token overload in single channels. Information overload is a real concern. When a single channel is bombarded with excessive information, the AI platform’s performance takes a hit [5]. It’s like trying to listen to multiple radio stations at once; the result is just noise. The current model’s technical limitations become evident as it struggles to scale with increased user interaction, leading to slower response times and a degraded user experience.

A Call for Change
As we draw the curtains on our discussion, it’s evident that the current AI ecosystem, while revolutionary, is far from perfect. The model’s unsustainability is not just a theoretical concern but rather a tangible reality that users grapple with daily.

The complexity of data misuse is a significant concern. Both active and passive data misuse are like icebergs—what’s visible is just the tip, and the real danger lurks beneath the surface. These misuses are not only concealed but also highly unpredictable. It’s akin to navigating a minefield blindfolded; one never knows when or where the next explosion will occur.

Relying solely on corporate responsibility and legal regulations is akin to putting a band-aid on a gunshot wound. While these measures might offer temporary relief, they don’t address the root cause of the problem. The need of the hour is a fundamental change. We must advocate for a deeper, more profound, root-level redesign of the information interaction mechanisms between users and AI platforms. It’s not just about patching up the existing system but envisioning a new one that prioritizes user experience, privacy, and security.

The AI ecosystem is at a crossroads. We can either continue down the current path, ignoring the glaring issues, or we can take the bold step of overhauling the system. The choice is clear: For a more sustainable, ethical, and user-friendly AI future, change is not just necessary; it’s imperative.

Endnotes
1. Harari, Y.N. 21 Lessons for the 21st Century. Spiegel & Grau, 2018.
2. Greenberg, A. Oops: Samsung employees leaked confidential data to ChatGPT. Gizmodo. Apr. 6, 2023; https://gizmodo.com/chatgpt-ai...
3. Forbes Tech Council. AI and large language models: The future of healthcare data interoperability. Forbes. Jun. 20, 2023; https://www.forbes.com/sites/f...
4. Broussard, M. The challenges of AI preservation. The American Historical Review 128, 3 (Sep. 2023), 1378–1381; https://doi.org/10.1093/ahr/rh...
5. Vontobel, M. AI could repair the damage done by data overload. VentureBeat. Jan. 4, 2022; https://venturebeat.com/datade...

---

Posted in: on Thu, January 04, 2024 - 11:32:00

Shuhan Sheng

Authors: Ana Tajadura-Jiménez , Bruna Petreca, Laia Turmo Vidal, Ricardo O'Nascimento , Aneesha Singh
Posted: Wed, January 03, 2024 - 2:31:00

---

During the 2023 CHI conference, we ran a one-day workshop to consider the design space at the intersection of the body and materials [1]. The workshop gathered designers, makers, researchers, and artists to explore current theories, approaches, methods, and tools that emphasize the critical role of materiality in body-based interactions with technology. We were motivated by developments in HCI and interaction design over the past 15 years, namely the “material turn,” which explores the materiality of technology and computation and methods for working with materials, and “first-person” approaches emphasizing design for and from lived experience and the physical body. Recognizing the valuable contributions of approaches that foregrounded materiality [2] and the body [3] in HCI, we proposed to explore the intersection of these two turns. 

Material interactions is a field of wide-ranging interest to HCI researchers, starting with research on the design and integration of physical and digital materials to create interactive and embodied experiences [4], progressing to a growing interest in the contextual aspects of material interactions and their impact on personal and social experiences [2], and culminating in formalized method propositions such as “material-centered interaction design” [5]. The latter is a fresh approach, urging interaction designers to broaden their view beyond the capabilities of the computer and embrace a practice that imagines and designs interaction through material manifestations. Similarly, the role of the body in design has gained traction, through approaches such as soma design [3], embodied interaction, and movement-centric methods [6]. These have positioned the centrality of first-person accounts and physical engagement to research and design from the body, shifting the design space beyond security and efficiency, which are primary goals, for example, in the field of human factors, to deal with other dimensions such as fun/joy, play, entertainment, mental health interventions, and so on. In the workshop, the methods and prototypes brought in aimed to explore experience, for instance in terms of perceptions and feelings toward materials (Figure 1) or toward the person’s own body (Figure 2). In both cases, the interaction with the prototypes brings the subjective experience into focus. We share a keen interest with previous research in working across physical and digital materials, as well as in centering the body in design. Additionally, we have a particular interest in the intersection of these approaches in which we find a new design space: making the body central material in designing experiences, going beyond a view of the body as the medium with which to explore the material—its potentials, its affordances—instead including the body as part of the material being designed with and for. We are excited to develop this design space through a formal proposition, as we progress with our research.

Our proposed approach is both timely and critical, as it addresses the complex interplay between the body and emerging technologies. However, this approach is not free from challenges, as discussed in the workshop. These technologies not only have a tangible impact but also influence immaterial aspects, creating unpredictable outcomes that must be carefully considered during the design phase. As technologies increasingly become a ubiquitous part of our lives, they affect our bodies in both physiological and psychological ways. Furthermore, moving this research beyond the confines of a laboratory not only introduces ethical concerns in relation to health, well-being, safety, and social impact, but also opens new opportunities. For example, in terms of social impact, the technologies can directly affect not just individual bodies, but also the interpersonal relationships those individuals maintain. Therefore, any interaction with embodied technologies carries real-world implications that must be approached with caution, but also with curiosity about the opportunities they may open. 

The Workshop
Throughout our day-long event, attendees took part in various activities and were organized into four thematic groups: material enabling expression; material as a catalyst for human action; material enabling reflection, awareness, and understanding; and material supporting the design process for (re)creating the existing and the yet-to-exist.

In the first activity, participants shared their prototypes or methods within their distinct groups. Prototypes were examined in relation to each group’s theme and participants individually noted similarities, differences, and opportunities among them. The second activity mixed groups for participants to discuss key points across themes, to uncover overlaps and opportunities. 

<
Figure 1. One of the methods (Materials Gym) presented at the workshop. EMG is used alongside a smartphone application to continuously collect data on people’s interaction with materials: Speed and aspects of movement reveal information about the experience in relation to people’s perceptions of and preferences for materials qualities (see https://discovery.ucl.ac.uk/id/eprint/10172204/).

Figure 2. A physical prototype (Soniband) that one participant brought to the workshop. This is a wearable band that provides real-time sonification of movement through a variety of sounds, some of which build on material metaphors, such as water, wind, or mechanical gears, and which can impact on the wearer’s body perceptions and feelings, movement, and emotional state (see https://doi.org/10.1145/3411764.3445558; https://doi.org/10.1145/2858036.2858486)

To increase enthusiasm into our exploration, the workshop featured a panel with four guest speakers who presented their research through a provocation: Kristii Kuusk (Estonian Academy of Arts) talked about the impact of material on people’s bodies combining affordances of technology and traditional materials; Pedro Lopes (University of Chicago) talked about making the body the material from which we build the actuator of the designed technologies; Hasti Seifi (Arizona State University) talked about the need for integrating basic research on haptic sensory perception and language into software tools for designing body-based haptic experiences; Paul Strohmeier (Max Planck Institute for Informatics) talked about how material and body experiences are shaped by agency, control, and reflective processes. The panelists conversed about material experiences and engaged in a Q&A session with participants. To conclude, participants mapped the design space of “body x materials” with their original groups, considering challenges, opportunities, state-of-the-art, theories, and methodologies.

Figure 3. Participants engaging in workshop activities.

The diversity of backgrounds of workshop participants (position papers available at www.rca.ac.uk/body-materials) resulted in multidisciplinary discussion groups where different disciplines, theories, methodologies and application domains were considered (including HCI, interaction design, neuroscience, arts and crafts, and material design). Through discussions, this diverse group of participants arrived at shared conclusions. Participants highlighted the importance and timeliness of the topic. Many emphasized the need to establish a clear vision of the relationship between the body and materials for HCI. They foregrounded the challenges posed by the complexities of reconciling detailed fine-grain descriptions with a holistic perspective of the overall experience, which arise both from the dynamic nature of the experience of materials, conveyed through sensorimotor feedback, and the continuous interpretation of the material qualities involved in the act of touching. Understanding the underlying processes leading to these interpretations and how awareness levels affect interaction was perceived by participants as critical. Several participants stressed the importance of incorporating first-person perspectives into design.

The discussion spotlighted additional challenges stemming from language limitations and the necessity of cultivating a language specific to touch. Two key points emerged from this discussion. First, participants underscored the importance of establishing a shared vocabulary that enables effective communication among researchers who may have diverse backgrounds. Second, it was considered that in the translation of touch experiences into verbal descriptions significant information tends to be lost. Participants also explored the optimal data format for representing body-based research, acknowledging the existence of ongoing attempts. 

Finally, the topic of shifting agency between the body and materials was explored. Two approaches were highlighted: empowering individuals to create their own unique material experiences as part of the design process, and the concept of “controlling the body to experience something: where the body acts as an actuator. The tension between enabling individuals to engage with the experience and exerting control over the body for a specific experience was also acknowledged during the discussion.

For those interested in working in this novel design space at the intersection of body x materials, we synthesize three key takeaways from our workshop. These takeaways emphasize: 1) the significance of comprehending the fundamental processes underlying experiences within this intersection, 2) the need to employ appropriate methods to achieve this understanding, including the integration of first-person perspectives into the design process, and 3) the importance of establishing a shared vocabulary. The latter includes facilitating the translation and representation of experiences and research outcomes related to the body x material interactions.

Conclusions
Everyone agrees that mapping the state-of-the art of the body x materials space is necessary for creating a more significant impact beyond lab experimentation. Nevertheless, a cautionary note was also struck—participants noted that it is essential to slow down and account for the ethical implications of how this research affects people psychologically and otherwise. To move the field forward and generate impact, participants agreed that interdisciplinary conversations must be fostered and the real-life implications of materiality x body understood, both in terms of possibilities and positive and negative effects of our work. The atmosphere at the workshop was open and interdisciplinary, and it was clear among participants that more such interdisciplinary conversations around the real-life implications of body x materials need to be nurtured to bring a shared understanding to the field. With this aim, the workshop participants planned to consolidate existing materials and findings at future workshops, including one that was run at the IEEE World Haptics 2023 conference and attempt to have a repeat workshop at CHI 2024 to further explore the directions that emerged. The long-term goal is to build an interdisciplinary community and open the design space for material-enabled, body-based multisensory experiences by integrating research from various perspectives. 

Acknowledgments
We would like to thank all workshop organizers (the authors of this piece were joined by: Hasti Seifi, Judith Ley-Flores, Nadia Bianchi-Berthouze, Marianna Obrist and Sharon Baurley), guest speakers, and participants for their fantastic contributions. We acknowledge funding by: the Spanish Agencia Estatal de Investigación (PID2019-105579RB-I00/AEI/10.13039/501100011033) and the European Research (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 101002711). The work of Bruna Petreca and Ricardo O’Nascimento was funded by UKRI grant EP/V011766/1. For the purpose of open access, the authors have applied a Creative Commons Attribution (CC BY) license to any Author Accepted Manuscript version arising. 

Endnotes
1. Petreca, B.B. et al. Body x materials: A workshop exploring the role of material-enabled body-based multisensory experiences. Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2023; https://doi.org/10.1145/354454...
2. Giaccardi, E. and Karana, E. Foundations of materials experience: An approach for HCI. Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, 2015, 2447–2456; https://doi.org/10.1145/270212...
3. Höök, K. Designing with the Body: Somaesthetic Interaction Design. MIT Press, Cambridge, MA, 2018.
4. Ishii, H. and Ullmer, B. March. Tangible bits: Towards seamless interfaces between people, bits and atoms. Proc. of the ACM SIGCHI Conference on Human factors in Computing Systems. ACM, New York, 1997, 234–241.
5. Wiberg, M. The Materiality of Interaction: Notes on the Materials of Interaction Design. MIT Press, Cambridge, MA, 2018.
6. Wilde, D., Vallgarda, A., and Tomico, O. Embodied design ideation methods: Analysing the power of estrangement. Proc. the 2017 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2017, 5158–5170; https://doi.org/10.1145/302545...

---

Posted in: on Wed, January 03, 2024 - 2:31:00

Ana Tajadura-Jiménez

Bruna Petreca

Laia Turmo Vidal

Ricardo O'Nascimento

Aneesha Singh

Authors: Elizabeth Rosenzweig, Amanda Davis, Zhengjie Liu, Deborah Bosley
Posted: Tue, January 02, 2024 - 11:29:00

---

As we make interacting with technology more like interacting with human beings, it is important that we understand how people use technology.…That is why usability is so important.  
— Bill Gates [1]

Professionals in user experience (UX) and technology recognize the significant impact well-designed technology has on everyday life. In a digital age where interfaces are everywhere, ensuring a positive user experience is crucial. It’s not merely about creating visually appealing designs; it’s about making technology intuitive, accessible, and genuinely useful for everyone.

We aim to do more than just influence individual applications; we strive to influence policymakers and decision-makers worldwide. Our advocacy revolves around recognizing usability as a fundamental human right. We firmly believe that access to technology that is easy to use and understand should be a universal entitlement, akin to essentials like clean water and education.

Our vision is to create a world where technology becomes a tool for empowerment, enabling individuals to navigate the digital landscape with confidence and ease. By championing usability as a basic human right, we’re paving the way for a future where innovative solutions are not only cutting edge but also universally accessible. Our goal is to ensure technology becomes a catalyst for positive change in the lives of people across the globe.

To achieve this vision, we emphasize the significance of World Usability Day (WUD), a pivotal event promoting the values of usability and user-centered design. By enhancing the visibility of this initiative, we can raise not only professionals’ awareness but also that of the general public. WUD serves as a platform to showcase inventive ideas, best practices, and research findings, highlighting how usability directly affects people’s lives. It embodies a global commitment to enhancing user experiences and ensuring technology is accessible to everyone. This influential event transcends geographical boundaries, uniting diverse communities, from professionals and industrial experts to educators, citizens, and government representatives.

The fundamental objective of WUD is both profound and straightforward: to ensure that essential services and products, vital to everyone’s lives, are user friendly and straightforward. By advocating for usability, WUD ensures that technological advancements don’t isolate users with complexity but empower them, regardless of their background, expertise, or physical abilities.

History of World Usability Day
The first invited essay about World Usability Day was in August 2006; the message still rings true today: “Every citizen on our planet deserves the right to usable products and services. It is time we reframe our work and look at a bigger global picture” [2].

For nearly two decades, WUD has been a catalyst for change, reaching practitioners in every corner of the globe. Across more than 140 countries, WUD has engaged over 250,000 individuals and had an impact on their local communities (see Table 1). WUD has opened up the field of UX and usability in places where it did not exist before the event, such as Eastern Europe (including Poland, Ukraine, Russia, and Turkey). With its annual theme, practitioners and volunteers have come together across the globe to tackle topics that matter to humanity.

	Theme	Events	Countries	Individuals Engaged
2005	Make it Easy	115	35	6,500
2006	Accessibility	228	40	38,000
2007	Healthcare	157	41	29,000
2008	Transportation	161	43	50,000
2009	Designing for a Sustainable World	150	43	48,000
2010	Communication	150	40	45,000
2011	Education: Designing for Social Change	120	42	7,000
2012	Usability of Financial Systems	80	20	6,000
2013	Healthcare: Collaborating for Better Systems	107	32	7,000
2014	Engagement	135	40	8,500
2015	Innovation	91	31	6,500
2016	Sustainability	73	24	6,000
2017	Inclusion	66	25	6,500
2018	Design for Good or Evil	62	26	6,500
2019	Design for the Future We Want	56	27	6,500
2020	Human-Centered AI	42	26	5,500
2021	Trust, Ethics, Integrity	47	25	4,550
2022	Healthcare	32	25	4,550
2023	Collaboration and Cooperation	48	20	4575

Table 1. Themes and global WUD participation.

World Usability Initiative
The founders of World Usability Day want to mobilize people in countries around the world to develop technology that works for the greater good. We want to expand the reach of recognizing the significance of usability once per year so that we can have a larger impact in tackling the world’s biggest problems.

To achieve this goal, they collaborated with renowned professional associations like SIGCHI, HCII, PLAIN, and IFIP. Together, they established the World Usability Initiative (WUI), a focused global organization.

Operating as a singular, dedicated entity, WUI works closely with the United Nations, particularly concerning the human factors integral to the 17 Sustainable Development Goals. This initiative unites experts from various fields, including human-computer interaction, user experience, and interaction design. Their collective efforts are geared toward leading researchers, developers, and countries in creating technology that is not only user centered but also aligned with core human values.

Our mission is to partner with the UN to tackle the 17 Sustainable Development Goals for 2030. Our initial initiatives will focus on:

• Establishing World Usability Day as an internationally observed day: WUI aims to connect the HCI-UX field with the United Nations by listing WUD on the UN calendar, ensuring global recognition.
• Rewarding good design with the WUI Design Challenge: This initiative rewards exceptional design by professionals across five countries each year, promoting user-centered innovation.
• Connecting communities through the International Speaker Series: To foster community engagement, WUI hosts an international speaker series, providing a platform for professionals to connect and share ideas.
• Creating projects that achieve the UN Sustainable Development Goals: Create projects within developed and developing countries that connect with the UN Sustainable Development Goals, adopting a user-centered approach.
• Creating a UN-based World Usability Organization: From the perspective of public policy formulation and implementation, the organization will focus on ensuring technology products and services will be made accessible and usable for everyone.

Returning to that 2006 essay about World Usability Day: “The challenge of World Usability Day is not small; it is to change the way the world is developing and using technology” [2]. 

Your support is instrumental in achieving WUI’s objectives and transforming the world into a more inclusive, equitable place. You can actively contribute by:

• Participating in the WUI Design Challenge: Showcase your innovative designs, promoting user-centered solutions.
• Organizing a World Usability Day event: Contribute to the global conversation by hosting an event, fostering awareness and inclusivity.
• Engaging in the speaker series: Share your expertise and ideas, connecting with professionals worldwide.
• Signing the petition: Join the collective voice urging the UN to recognize World Usability Day’s significance.
• Sponsoring WUD: Support WUD financially, enabling the movement to reach new horizons.

In the spirit of global collaboration, let us unite to create technology that serves humanity inclusively, ensuring usability becomes a universal reality. Together, we can pave the way for a world where technology truly becomes a force for good, enriching lives, fostering innovation, and championing inclusivity.

Endnotes
1. Bill Gates at World Usability Day 2007. YouTube;https://www.youtube.com/watch?v=mpxYYz1QHqQ&t=7s
2. Rosenzweig, E. World Usability Day: A challenge for everyone. Journal of User Experience 1, 4 (2006), 151–155; https://uxpajournal.org/world-...

---

Posted in: on Tue, January 02, 2024 - 11:29:00

Elizabeth Rosenzweig

Amanda Davis

Zhengjie Liu

Deborah Bosley

Authors: Amir Reza Asadi
Posted: Thu, November 16, 2023 - 2:03:00

---

As we approach the metaverse, a sociotechnical future where people consume huge amounts of information in 3D formats, the significance of game development tools, including both game-engine and digital-content-creation tools, will become more and more evident. In a metaverse-oriented world, you need to create 3D experiences to express your ideas. Game engines are currently the main tools for authoring and creating AR/VR experiences; therefore, simplifying the game-authoring process is crucial for the future of freedom of expression.

Video games are the most advanced form of interactive media, conveying diverse ideas and often representing mainstream ideologies. However, making a small but sophisticated video game is an expensive form of storytelling. This issue may not be a concern for many of us because the main purpose of digital play is entertainment. But it should be a vital issue for all of us, as a metaverse-oriented world is made up of 3D game-like experiences. Those who can create more immersive experiences may take control of real-world narratives, so it’s a challenge and an important goal for us to make game development inclusive of more voices. Without simplifying the game-authoring process, our society could lose its freedom to effectively communicate and express itself, resulting in reduced empathy.

We should also not underestimate the threat of propaganda in the metaverse. The metaverse allows users to immerse themselves in dreams of virtual prosperity, and people may shelter in the virtual world to overcome the pain of poverty in the real world. Dreaming has played a vital role in the history of humanity. We dream at times when we are unhappy with our circumstances, and no dictatorship in history has been able to control our dreams [1]. With the rise of emerging non-democratic powers and economies, there is a threat that companies will start changing or censoring the ideologies of their game-alike experiences, similar to what happened to the movie industry in recent years [2,3]. In a dystopian scenario, people will trade their ability to dream for the ability to live in virtual game-alike experiences. Losing the ability to dream freely can have dangerous consequences, as it can hold people captive to mainstream ideas. But there is a silver lining. Transforming game-authoring tools in the same way that emergent video making/sharing apps such as TikTok, Snapchat, and Apple Clips have democratized video-based storytelling can give us hope that all ideas can be expressed not just freely but also effectively in the Metaverse. 

A Desirable Future for Game-Creation Tools
In today’s world, game development has become simpler with free-to-use game engines, but the pipeline of game development is still complicated. We need to simplify the workflow of game-authoring tools for tomorrow. I’m not talking about a magic-wand solution here. The technologies for simplifying game authoring are available today, but we need to first let go of our previous assumptions.  

The designers of game-authoring tools should create these tools based on new personas. They need to think big and assume that the intellectuals of the metaverse generation will be game authors rather than book authors. Based on this desirable future, we’ve created a persona (Figure 1) who represents a thinker, representing him with an image of Montesquieu, the famous French political philosopher. We also created a design fiction about his life:

DJ Montesquieu was a political philosopher who sought to challenge traditional modes of philosophical discourse. He saw the world through a different lens, one that viewed technology and innovation as a means of disseminating his ideas more effectively to a wider audience. DJ Montesquieu’s unique approach to philosophical discourse led him to develop his thesis not in the form of a book, but rather as a video game. His thesis was a video game that allows players to experience the impacts of overregulation from the eyes of ordinary citizens. Unlike traditional philosophers, Montesquieu was not interested in writing for the sake of writing. He was more concerned with sharing his views in a way that was accessible and engaging for people. To this end, he utilized the AlternateGameEngine, which is a combination of no-code tools and ChatGPT-based tools to create immersive AR/VR games that brought his ideas to life.

DJ Montesquieu’s games were not just entertaining; they were also thought-provoking. They allowed players to explore complex philosophical concepts in a way that was both fun and enlightening. By using technology as a medium for philosophical discourse, DJ Montesquieu was able to reach a wider audience than he ever could have with traditional writing.
 

Figure 1. The persona of a future political philosopher, DJ Montesquieu.

In other words, imagine a future where intellectuals and authors, instead of using word processing tools and presentation software, will be using 3D game-creation tools. Where people may quote game authors instead of Friedrich Nietzsche, and where famous people will create games instead of autobiographies. In this future, the most popular social network allows users to turn their diaries into snack-sized gaming experiences that can be experienced by others. 

This future also allows society to understand situations that they cannot or do not want to comprehend by reading books. The combination of the immersive nature of the metaverse with simplified game-authoring tools will enable societies to have more ethnocultural empathy, as they can experience life from others’ point of view. Immersive 3D virtual environments have already proven effective in creating empathy among different ethnic groups [4,5], so simplifying the pipeline of immersive storytelling can create opportunities to have a more united society. Imagine that instead of watching partisan debates on TV and being exposed to online political bots, every person could share their ideas, trauma, or values via immersive experiences in the same way they can create content for social media, allowing us to live in a more empathetic community. People may not be able to express their own side of story through verbal interactions, but easy immersive-experience authoring may allow them to express their stories like livable dreams that others can experience themselves. 

Endnotes
1. Rare 1997 interview with Abbas Kiarostami, conducted by Iranian film scholar Jamsheed Akrami. From Taste of Cherry Featurettes, 1997. YouTube, Mar. 1, 2021; [https: www.youtube.com="" watch?v="VTcm4P5qAF8"]
2. DeLisle, J. Foreign policy through other means: Hard power, soft power, and China’s turn to political warfare to influence the United States. Orbis 64, 2 (2020), 174–206.
3. Su, W. From visual pleasure to global imagination: Chinese youth’s reception of Hollywood films. Asian Journal of Communication 31, 6 (2021), 520–535.
4. Coffey, A.J., Kamhawi, R., Fishwick, P., and Henderson, J. The efficacy of an immersive 3D virtual versus 2D web environment in intercultural sensitivity acquisition. Educational Technology Research and Development 65 (2017), 455–479.
5. Lamendola, W. and  Krysik, J. Cultivating counter space: Evoking empathy through simulated gameplay. ETC Press, 2022.

---

Posted in: on Thu, November 16, 2023 - 2:03:00

Amir Reza Asadi

Authors: Wee Kiat Lau, Lisa Valentina Eberhardt , Marian Sauter, Anke Huckauf
Posted: Mon, October 23, 2023 - 10:45:00

---

Picture this: Sitting in his kitchen, young Tim chuckles at the recent misfortune of neighbors who fell prey to burglars. At the same time, he’s enthusiastically experimenting with a banking app on his fresh-off-the-assembly-line computer, a machine devoid of even the most fundamental antivirus protection. This scenario is a striking illustration of the privacy paradox. We voice anxiety about our data’s usage, often lambasting corporations with lax privacy protocols, only to defy our apprehensions by not embracing measures to safeguard ourselves. Let’s dissect this intriguingly paradoxical user behavior from the lens of perceptual psychology, illuminating the circumstances under which we sense privacy or publicity, distinguishing these states, and suggesting ways to aid individuals like Tim.

Perception in Private and Public Surroundings
Envision an intensely private moment: reclining on your sofa, shaking off the weariness of work. Now, contrast this with a public spectacle: accepting accolades onstage for special achievements, with a massive audience bearing witness. How do these scenarios make you feel, and what sets them apart? To shed light on this question, we will go deep into our bodies’ states and processes. A key environmental distinction arises when we feel private—the high likelihood of being cocooned in a familiar setting, surrounded by well-known objects and people, which lets us unwind. New experiences, however, kick-start our alertness; they ignite our curiosity and grab our attention. 

Fundamental cognitive processes in any living organism include sensation and perception. Familiar environments envelop us with recognizable items, triggering well-known sensations, be they sounds, scents, tactile sensations, or visuals. Therefore, private settings pose fewer challenges to the perceptual system about object identification and sensory memory. This reduction in demand allows the system to function at a lower sensitivity level, freeing up capacity for other operations.

This perceptual mode is accompanied by physiological processes. Broadly speaking, in relaxed, private settings, the parasympathetic neural activation (“rest and digest” mode) is dominant, whereas the sympathetic activation (“fight or flight” mode) is dominant in unfamiliar public settings. There are some remarkable perceptual effects caused by this: Sympathetic activation leads to larger pupils, resulting in more light input, in a slightly extended visual field, and in reduced visual acuity, especially outside of the eyes’ focus—that is, in periphery and in depth [1]. We therefore can assume that in public relative to private settings, we perceive with less spatial accuracy from a larger visual field. 

Attention in Private and Public Surroundings
Perception is accompanied by an adaptation of attentional processes. All attentional functions respond to situational affordances. We can differentiate between alertness and selective attention. Alertness is the increase and maintenance of response readiness. It can be supposed to complement the general arousal level of an organism. Thus, alertness will be high in public settings, while it can be reduced when the organism is surrounded by familiar objects. The notion of a broader visual field, although with lower spatial resolution, can be plausibly assumed to support alerting functions in public settings.

Regarding selective attention, new salient stimuli are known to capture attention. This improves visual search performance. In private surroundings, distracting objects can be quickly identified and thus be effectively suppressed. This can lead to a phenomenon known as inattentional blindness: In familiar settings, it frequently happens that we miss even uncommon, unexpected objects. The visual system is also capable of suppressing distracters based on their spatial location [2], improving efficiency in familiar environments. The familiarity of surrounding objects eases not only the selection of task-relevant objects but also the suppression of distracting objects. Inhibition again saves capacity for other processes [3]. In unfamiliar public settings, however, stimuli must be processed until they are identified as harmless, and suppression of task-irrelevant stimuli is thus more difficult.

The dichotomy between private and public settings even manifests in our posture and movements. Onstage, we present our bodies to a large audience, making exaggerated, sweeping gestures. Conversely, in private, our muscles can relax, leading to smaller, more restrained movements. This difference extends to eye movements and gaze, which in turn influence perception.

Level of Control in Private and Public Surroundings
Taken together, private settings lessen the need to attend to external stimuli; you can unwind and rely on the consistency of the surroundings. Also, perceiving things provides already familiar information. All these processes diminish the need for cognitive control, allowing processing to occur more subconsciously. Consequently, executing learned skills, routines, and habits becomes more probable. Public behavior, however, is marked by unfamiliar surroundings. The influx of novel objects or people prompts a slew of questions: Is that unfamiliar face a threat? What does that unexpected sound imply? This cognitive appraisal demands effortful attention, sapping mental resources, making us more cautious in and conscious of our actions [3].

This thinking aligns with Daniel Kahneman’s [4] idea that human behavior is regulated either by quick, instinctive, and emotional processing (as in private settings) or by slower, more deliberative, and logical processing (common in public settings). Crucially, it’s nearly impossible to engage both methods simultaneously. Therefore, in a situation prompting emotional automated processing with only weak conscious monitoring, we’re hardly capable of producing analytical thinking with logical deductions. This means that if users engage with their personal devices at home, their behavior is dominated by automated, nonconscious routines.

Counteracting Perceived Privacy by Simulating a Public Audience
So, how can we assist users in selecting an appropriate level of control? Novelty in environmental stimuli can be an indicator. How we process these cues shapes how we perceive and interact with our surroundings, be they private or public. The cues could be signals or symbols [5]. Signals are automatic cues operating beneath conscious thought, like the familiar ticktock of a clock or the distinctive feel of your sofa, directing our arousal and attention needs. Conversely, symbols, such as GDPR text, demand conscious, detailed analysis, and interpretation, thus requiring higher cognitive capacity.

Discerning the psychological differences between private and public settings equips us with a potent tool to mold privacy behavior and promote prudent disclosure. As we’ve noted, using personal devices in private settings often sparks cues associated with private behavior, possibly leading to a false sense of security. Thus, introducing cues that simulate public scenarios could stimulate public consciousness, reminding users to be more circumspect with their disclosures. These could be visuals, sounds, smells, or other elements that evoke the public nature of their online interactions. One subtle method to induce a feeling of publicity could be the “watching eyes effect”: The presence of a pair of eyes can influence disclosure behavior and can be fine-tuned by varying emotional expression, sex, and age of the eyes [6]. Ideally, this should be achieved by incorporating design elements that subtly disrupt users’ familiar routines, prompting a cognitive response akin to being in a public setting.

Toward a Privacy-Sensitive Future
To conclude, traversing the maze of privacy behavior is an intricate task, yet understanding the interplay of environmental cues, perception, attention, and behavior control can illuminate our path forward. Preserving privacy might be bolstered by subtly simulating publicity within digital environments, evoking vigilance, and awareness akin to our natural responses in public settings. By doing so, we can harness our inherent cognitive and physiological processes.

Endnotes
1. Eberhardt, L.V., Strauch, C., Hartmann, T.S., and Huckauf, A. Increasing pupil size is associated with improved detection performance in the periphery. Attention, Perception, & Psychophysics 84, 1 (2022),138–149; https://doi.org/10.3758/s13414...

2. Sauter, M., Liesefeld, H.R., Zehetleitner, M., and Müller, H.J. Region-based shielding of visual search from salient distractors: Target detection is impaired with same- but not different-dimension distractors. Attention, Perception, & Psychophysics 80, 3 (2018), 622–642; https://link.springer.com/arti... 

3. Posner, M.I. and Petersen, S.E. The attention system of the human brain. Annual Review of Neuroscience 13, 1 (1990), 25–42.

4. Kahneman, D. Thinking, Fast and Slow. Farrar, Straus, and Giroux, 2011.

5. Rasmussen, J. Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Transactions on Systems, Man, and Cybernetics SMC-13, 3 (1983), 257–266.

6. Lau, W.K., Sauter, M., Bulut, C., Eberhardt, L.V., and Huckauf, A. Revisiting the watching eyes effect: How emotional expressions, sex, and age of watching eyes influence the extent one would make stereotypical statements. Preprint, 2023; https://doi.org/10.21203/rs.3....

---

Posted in: on Mon, October 23, 2023 - 10:45:00

Wee Kiat Lau

Lisa Valentina Eberhardt

Marian Sauter

Anke Huckauf

Authors: Maximilian Speicher
Posted: Tue, September 19, 2023 - 11:09:00

---

In the two previous articles of this series, we have first learned that metrics such as conversion rate, average order value, or Net Promoter Score are not suitable to reliably measure user experience (UX) [1]. The second article then explained how UX is a latent variable and, therefore, we must rely on research instruments and corresponding composite indicators (that produce a metric) to measure it [2]. Now, the logical next question is how we can identify those instruments and metrics that do reliably measure UX. This boils down to what is called construct validity and reliability, on which we will give a brief introduction in this final article, before deriving easily applicable heuristics for practitioners and researchers alike who don’t know which UX instrument or metric to choose.

Construct validity refers to the extent to which a test measures what it is supposed to measure [3]. In the case of UX, this means that the instrument or metric should measure the concept of UX as it is understood in the research literature, and not, for example, only usability. One good way to establish construct validity is through factor analysis [3].

Construct reliability refers to the consistency of a test or measure [4]. Put differently, it is a measure of how reproducible the results of an instrument or metric are. A good way to establish construct reliability is through studies that assess the test-retest reliability of the instrument or metric, as well as its internal consistency, such as Cronbach’s alpha [4].

In addition to that, the Joint Research Centre of the European Commission (JRC) provides a “Handbook on Constructing Composite Indicators” [5], which summarizes the proper process in terms of a 10-step checklist. We build on all of the above for our following list of seven heuristics for identifying proper UX instruments and metrics.

Heuristic 1: Is there a paper about it? If there is no paper about the instrument and/or metric in question, there’s barely a chance you’ll be able to answer any of the following questions with yes. So, this should be the first thing to look for. A peer-reviewed paper published in a scientific journal or conference would be the best case, but there should be at the very least some kind of white paper available.

Heuristic 2: Is there a sound theoretical basis? In the case of UX, this means, does the provider of the instrument and/or metric clearly explain their understanding of UX and, therefore, what their construct actually measures? The JRC states: “What is badly defined is likely to be badly measured” [5].

Heuristic 3: Is the choice of items explained in detail? Why were these specific variables of the instrument chosen, and not others? And how do they relate to the theoretical framework, that is, the understanding of UX? The JRC states: “The strengths and weaknesses of composite indicators largely derive from the quality of the underlying variables” [5].

Heuristic 4: Is an evaluation of construct validity reported? This could be reported in terms of, for example, a confirmatory factor analysis [3]. If not, you can’t be sure whether the instrument or metric actually measures what it’s supposed to measure.

Heuristic 5: Is an evaluation of construct reliability reported? This could be reported in terms of, for example, Cronbach’s alpha [4]. If not, you can’t be sure whether the measurements you obtain are proper and reproducible approximations of the actual UX you want to measure.

Heuristic 6: Is the data that’s combined to form the metric properly normalized? This is necessary if the items in an instrument have different units of measurement. The JRC states: “Avoid adding up apples and oranges” [5].

Heuristic 7: Is the weighting of the different factors that form the metric explained? Factors should be weighted according to their importance. “Combining variables with a high degree of correlation” (double counting) should be avoided [5].

In the following, the application of these heuristics will be demonstrated through two very brief case studies.

Case Study 1: UEQ
The User Experience Questionnaire (UEQ) is a popular UX instrument developed at SAP AG.

• H1: There is a peer-reviewed research paper about UEQ, which is available at [6]. ✓
• H2: The paper clearly defines the authors’ understanding of UX, and they elaborate on the theoretical background. ✓
• H3: The paper explains the selection of the item pool and how it relates to the theoretical background. ✓
• H4: The paper describes, in detail, two studies in which the validity of UEQ was investigated. ✓
• H5: The paper reports Cronbach’s alpha for all subscales of the instrument. ✓
• H6: Not applicable, since UEQ doesn’t explicitly define a composite indicator. However, a composite indicator can be constructed from the instrument.
• H7: See H6.

Case Study 2: QX score “for measuring user experience”
This metric was developed by SaaS provider UserZoom and is now provided by UserTesting. It is a composite of two parts: 1) the widely used SUPR-Q instrument and 2) the individual task success rates from the user study where the metric was measured, in a 50/50 proportion.

• H1: There is no research paper, but at least a blog post explaining the instrument and metric. ✓
• H2: There is no clear definition of UX given. The theoretical basis for the metric is the assumption that all existing UX metrics use either only behavioral or only attitudinal data. There is no well-founded explanation given why this is considered problematic. The implicit reasoning is that only by mixing behavioral and attitudinal data can we properly measure UX, which is factually incorrect (cf. [2]). ❌
• H3: The metric mixes attitudinal (SUPR-Q) and behavioral (task success) items, but no well-founded reasoning is given as to why only task success rate was chosen, or why this would improve SUPR-Q, which is already a valid and reliable UX instrument in itself. ❌
• H4: No evaluation of construct validity is reported. ❌
• H5: No evaluation of construct reliability is reported. ❌
• H6: There is no approach to data normalization reported, the metric seemingly adds up apples and oranges. ❌
• H7: There is no reasoning given for the weighting of the attitudinal and behavioral items. ❌

In conclusion, the seven heuristics provided in this article serve as a useful guide for identifying proper UX instruments and metrics. Additionally, by considering construct validity and reliability, as well as following the JRC’s 10-step checklist, practitioners and researchers alike can make informed decisions when choosing a UX instrument or metric. It’s important to note that not all instruments or metrics will pass all of these heuristics, but the more of them that are met, the more confident one can be that the chosen instrument or metric properly measures UX. If in doubt, choose the instrument or metric that checks more boxes. It's worth noting that some heuristics, like H1, are not strictly necessary, and H6 and H7 only apply to composite indicators. It’s also worth noting that there may be valid instruments or metrics that fail some of these heuristics and vice versa. Their goal is to provide a robust and quick framework for evaluating UX instruments and metrics, but ultimately, the best approach will depend on the specific research or design project.

Endnotes
1. Speicher, M. Conversion rate & average order value are not UX metrics. UX Collective. Jan. 2022; https://uxdesign.cc/conversion...
2. Speicher, M. So, How Can We Measure UX? Interactions 30, 1 (2023), 6–7; https://doi.org/10.1145/357096...
3. Kline, R.B. Principles and Practice of Structural Equation Modeling. Guilford Publications, 2015.
4. Cronbach, L.J. and Meehl, P.E. Construct validity in psychological tests. Psychological Bulletin 52, 4 (1955), 281.
5. Joint Research Centre of the European Commission. Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD publishing, 2008; https://www.oecd.org/sdd/42495...
6. Laugwitz, B., Held, T., and Schrepp, M. Construction and evaluation of a user experience questionnaire. Symposium of the Austrian HCI and Usability Engineering Group. Springer, Berlin, Heidelberg, 2008, 63–76.

---

Posted in: on Tue, September 19, 2023 - 11:09:00

Maximilian Speicher

Authors: Florian Echtler, Lonni Besançon , Jan Vornhagen, Chat Wacharamanotham
Posted: Fri, August 25, 2023 - 11:15:00

---

During the past few years, the Covid-19 pandemic has resulted in an unprecedented amount of research being conducted and published in a very short timeframe to successfully analyze SARS-COV-2, its vaccines, and its treatments [1]. Concurrently, the pandemic also highlighted the limitations of our publication system, which enables and incentivizes rapid dissemination and questionable research practices [2].

While HCI research is usually not a foundation for life-and-death decisions, we face similar problems. HCI researchers and CHI community members have long criticized a lack of methodological [3] and statistical rigor [4] and a lack of transparent research practices in quantitative [5] and qualitative works [6]. Research transparency can alleviate these issues, as it facilitates the independent verification, reproduction, and—wherever appropriate—replication of claims. Consequently, we argue that the CHI community needs to move toward a consensus on research transparency.

Reaching this consensus is no trivial task, as HCI is an inter- or transdisciplinary field of research, employing myriad methods, and therefore cannot be subjected to a single, rigid rule set. For example, qualitative and quantitative data cannot be shared to the same degree without potentially identifying participants. Although the context of each research project might constrain feasible transparency practices, we believe that the overarching principle of research transparency can be applied to fit the vast variety of research that exists within our field.

Despite the benefits of transparent research practices, many of our efforts for more transparency within SIGCHI research have been met with considerable pushback. Junior researchers are worried about having their workload increase even more through additional documentation requirements, while more senior members of the community—who have built their careers on a research model that did not yet focus on transparency—may fail to see noticeable value in the additional effort required. Qualitatively oriented researchers, both within [6] and outside HCI [7], do not see themselves represented in a discussion that often focuses on statistics, data collection, and related topics. Meanwhile, some quantitative researchers mistook that preregistration limits potential exploratory analysis [8].

Ideally, any publicly available record of scholarship—whether it is a research paper, an essay, a dataset, or a piece of software—should be adequately transparent to enable the public to assess its quality and for subsequent research to build upon.

After all, the goal of research is not to pad one’s own h-index or to get a p-value below 0.05, but rather to increase the total sum of human knowledge. And in order to do so, we need to be able to build on the research that other scholars have done before us, so we can see farther. For this reason, transparency is indeed the one fundamental principle behind the open science movement, and it is so general that any of the subcommunities within HCI should be able to follow it.

The slow but steady move toward more transparency across all disciplines of research also provides a unique opportunity for the HCI community: At least some of the reluctance to adopt this approach stems from a lack of supporting tools. It is our field’s core expertise to uncover and analyze problems, and iteratively develop and evaluate potential solutions. Imagine that writing your statistical analysis would immediately update the figures in your paper and provide an automatic appendix with all calculations. Or that the coding process for interview quotes would keep a record of the shifting themes until the final result is reached, complete with an accurate log of the decisions that led to this point. These kinds of tools could be valuable far beyond HCI, for example, in related disciplines such as psychology and sociology, helping to establish a more transparent process there as well. Besides, the interdisciplinary nature of HCI allows us to test and debate innovations in methods, tools, and policies. We believe that this richness in perspectives and the technological and design capacity in our field could contribute to unblock open science conundrums, for example, challenges in sharing qualitative data [8].

For the future, we hope for guidelines that emphasize the values and opportunities within transparent research. We hope for reviewers who take them to heart, and include the transparency of a manuscript in their assessment, not as just superficial novelty. We hope for established researchers to lead by example, providing transparent insights into their research process, and for hiring committees and examination boards to value these contributions on their own. Last but not least, we hope for all members of the community to consider how they can increase transparency in their research and publication practices.

Endnotes
1. Callaway, E., Ledford, H., Viglione, G., Watson, T., and Witze, A. COVID and 2020: An extraordinary year for science. Nature 588, 7839 (2020), 550–553; pmid:33318685

2. Besançon, L., Bik, E., Heathers, J., and Meyerowitz-Katz, G. Correction of scientific literature: Too little, too late!. PLOS Biology 20, 3 (2022), e3001572; https://doi.org/10.1371/journa...

3. Greenberg, S. and Thimbleby, H. The weak science of human-computer interaction. (Dec. 1991); https://doi.org/10.11575/PRISM/30792

4. Cockburn, A. et al. HARK no more: On the preregistration of CHI experiments. Proc of the CHI Conference on Human Factors in Computing Systems. ACM, New York, 2018, 1–12; https://doi.org/10.1145/3173574.3173715

5. Vornhagen, J.B. et al. Statistical Significance Testing at CHI PLAY: Challenges and Opportunities for More Transparency. Proc. of the Annual Symposium on Computer-Human Interaction in Play. ACM, New York, 2020, 4–18.

6. Talkad Sukumar, P., Avellino, I., Remy, C., DeVito, M.A., Dillahunt, T.R., McGrenere, J. and Wilson, M.L. Transparency in qualitative research: Increasing fairness in the CHI review rrocess. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–6.

7. Kapiszewski, D. and Karcher, S. Transparency in practice in qualitative research. PS: Political Science & Politics 54, 2 (2021), 285–291; https://doi.org/10.1017/S1049096520000955

8. Wacharamanotham, C., Eisenring, L., Haroz, S. and Echtler, F. Transparency of CHI research artifacts: Results of a self-reported survey. Proc. of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020; https://dl.acm.org/doi/10.1145...

---

Posted in: on Fri, August 25, 2023 - 11:15:00

Florian Echtler

Lonni Besançon

Jan Vornhagen

Chat Wacharamanotham

Authors: Jia Shen
Posted: Tue, August 22, 2023 - 1:07:00

---

As digital technology designers, our role has never been more crucial than it is now. Decades of technological and commercial advancements have brought computing from research labs to business offices, and now to the homes and pockets of everyday users. Our focus has expanded from computer interfaces to user experiences to design thinking for executives in corporate boardrooms. Despite improvements in user experiences, we have observed that users are often becoming more mindless, joyless, and anxious in today’s hyperconnected world. With the rapid advancement of AI, wearables, and pervasive computing, these challenges are growing.

As designers, what must we add to our toolkit to reimagine and design a sustainable future? This essay advocates for the importance of mindfulness in the skill set of interaction designers. It highlights the specific areas where mindfulness can enhance designers’ abilities, and calls for the design community to work together toward incorporating mindfulness into their practice.

What Is Mindfulness and Why Should Designers Care?
During a visit to Google, the well-known Zen master Thich Nhat Hanh shared a story with Google employees. The story involved a person riding a horse at a fast pace and when asked where they were going, they replied, “I don't know, ask the horse!” Thay, as he is known to his followers, pointed out that our current use of technology can lead us to escape from ourselves, our loved ones, and the natural environment. When we feel empty or unable to confront our inner struggles, we often seek external distractions [1].

Today’s technology creates unprecedented opportunities for us to turn our attention outward and away from ourselves. However, the more we disconnect from ourselves, the less content we become. No wonder that mental health challenges are prevalent and the national happiness index has decreased in the past decade, despite the increasing abundance of material goods and the widespread use of digital technology.

Mindfulness practice is a way of being ourselves in the present moment, and a powerful tool to look inside ourselves. Mindfulness originates from Zen Buddhism. It is as a state of being characterized by a keen awareness of the present moment. The Sanskrit term for mindfulness, smriti, means “remembering.” Mindfulness is remembering to come back to the present moment. The character in Chinese for mindfulness, 念, consists of two parts: the upper part represents “now” and the lower part represents “mind” or “heart” [2].

At the core of Buddha’s teachings lies the concept of Right Mindfulness (samyak smriti). Buddhist psychology states that attention is a universal trait, meaning that we are constantly giving our attention to something. This attention can be appropriate, such as when we are fully present in the moment, or inappropriate, such as when we focus on something that takes us away from the present. Right Mindfulness is characterized by acceptance without judgment or reaction and is inclusive and compassionate. According to Buddha’s teachings, Right Mindfulness offers Seven Miracles, including the ability to be present and fully engage with the present moment, to bring forth the presence of others (such as the sky, flowers, or our children), to nourish the object of our attention, to alleviate others’ suffering, to explore deeply (vipassana), to gain understanding, and to bring about transformation [2].

Although stemming from Zen Buddhism, mindfulness is not a religion itself. Rather, it is a widely adopted practice around the world to improve one’s health, personal growth, and overall well-being. In the Western world, interest in mindfulness practice grew during the 1960s, leading to an era of scientific research on this ancient practice. For instance, Mindful-Based Stress Reduction (MBSR), founded by Jon Kabat-Zinn in the 1970s, is regarded as one of the leading programs for reducing stress, anxiety, depression, and PTSD, with significant and reliable results in various clinical studies [3]. Today, clinicians, business leaders, and educators are exploring ways to integrate mindfulness practice into their respective fields.

In the digital technology design community, there is a growing voice advocating for a new approach to design, one that involves looking inward. In a recent special issue of Interaction, Kentaro Toyama [4] suggests that our focus on external outcomes may be the root cause of our technological problems. He urges designers to turn to faith and inward examination as a possible solution. Meanwhile, Kabat-Zinn encourages designers to practice mindfulness themselves before teaching or supporting others. It’s time for us to consider how mindfulness can benefit us and act accordingly.

How Can Mindfulness Help Designers?
Mindfulness enables designers to see the true nature of the problems they are tackling, and provides the foundations for a paradigm shift in the design process and outcomes. Some initial considerations include:

Design process
Traditional user experience design follows the requirement-design-evaluation framework. As today’s problems are becoming more complex, challenging, and wicked, a mindful designer is aware of themselves and others, and they bring mindfulness to the design process, paying close attention to each step. They deeply observe and listen, resulting in greater empathy and deeper understanding of the problem space before jumping to the solution space.

Design for slow change
In the HCI and interaction design community, Martin Siegel and Jordan Beck [5] sketched out Slow Change Interaction Design, where they called for a paradigm shift in designers’ mindsets in order to create interactive technologies that promote long-lasting attitudinal and behavioral change.

A designer who practices mindfulness understands that problem-solving is not a one-time event, but rather a continuous process of improving the current state toward a better future state. They acknowledge that change is inevitable and strive to learn and grow with their users in a constantly evolving technological environment.

Design for a sustainable future
Considering the interconnectedness nature of all things, a mindful designer takes a systemic approach to their work. They prioritize sustainable and inclusive design, taking into account the impact on individuals with varying abilities, communities, and the environment. They recognize that quick fixes and immediate results are often unrealistic, and prioritize long-term solutions.

Toward Mindful Technology Use Practice
In his book, Peace is Every Step, Thich Nhat Hanh [6] offers suggestions on how to incorporate mindfulness practices into everyday life. Following his examples, we offer the following mindful technology practices regarding smartphone use to technology designers and users:

Before using my phone, I know what I use it for.

I am aware that happiness depends on my mental attitude and not on external conditions, and that I already have more than enough to be happy in the present moment.

The phone and I are one.

I will practice coming back to the present moment by being mindful of my breathing, posture, and body.

Takeaway
When technology is the horse, mindfulness is the harness to rein it back toward humanity. Mindfulness offers an ancient yet rejuvenating perspective that is crucial to our digital design community. Designers who practice mindfulness can transform problems and bring peace, joy, and freedom to both users and themselves. May you also find joy in practicing mindfulness.

Endnotes
1. Hanh, T.N. The horse is technology. Plum Village Magazine (Summer 2014), 5–9.

2. Ñāṇamoli, B. and Bodhi, B. The Middle Length Discourses of the Buddha. Wisdom Publications, 1995.

3. Kabat-Zinn, J. Full Catastrophe Living: Using the Wisdom of Your Body and Mind to Face Stress, Pain, and Illness, 15th Anniversary Ed. Delta Trade Paperback/Bantam Dell, New York, NY, 2005.

4. Toyama, K. Technology and the inward turn of faith. ACM Interactions 29, 4 (2022), 36–39.

5. Siegel, M. and Beck, J. Slow change integration design. ACM Interactions 21, 1 (2014), 28–35.

6. Hanh, T.N. Peace is Every Step. Bantam Books, 1992.

---

Posted in: on Tue, August 22, 2023 - 1:07:00

Jia Shen

Authors: Michal Luria
Posted: Wed, July 26, 2023 - 3:03:00

---

Remote user experience (UX) research is not a new phenomenon [1], but the pandemic has accelerated it, necessitating the adaptation of traditionally face-to-face methods to work remotely [2]. The past few years marked a growth of online platforms, such as dscout, Lookback, Metricwire, and UserTesting, to name just a few. Remote UX research platforms are marketed primarily as tools for industry UX researchers, but are also used occasionally by academics.

These user-friendly interfaces promise remote UX researchers everything they need to conduct a successful study. They support many methods, and provide tools for all research stages, from recruitment via participant pools to built-in analysis. But the platforms’ quick-and-easy approach also creates ethical issues when they don’t fully consider researchers’ obligations toward their participants [3].

UX research frequently involves people, and therefore risks causing harm [4]. For example, people may be asked to share personal information about sensitive topics as part of research, like their sexual orientation or their use of adult websites. Without the right safeguards, this information could get into the wrong hands. Or participants may be asked to view social media content and subsequently be exposed, perhaps unintentionally, to harmful content. These scenarios are even more delicate if a study involves minors or other vulnerable groups [5].

In corresponding with several UX research platforms, I was surprised to discover consistent ethical and privacy-related gaps in the platforms’ ability to address possible risks in human subjects research, particularly in their consent, data ownership, and safety protocols. Here, I outline how these gaps could be addressed for better research practices. 

Consent should be mandatory. When using a service’s participant pool, researchers can, but are not obligated to, include a consent process. Platforms say that, when a participant signs up for their platform, they agree to their terms and conditions, suggesting that this agreement covers participation in any research study without further consent.

This is not compliant with ethical research standards and the requirement for participants to be informed: Participants should be able to explicitly consent to participation in every study, after having been fully informed about its goals, risks, and outcomes [6].

Not only is obtaining consent optional on some UX research platforms, but collecting signatures often requires a premium subscription. The alternative for non-premium customers would be to put the consent information into a survey question and ask participants to click “Agree.” For the case of parental consent for minors as required by laws like the General Data Protection Regulation (GDPR), one platform suggested simply including another survey question that asks the minor whether their parents agreed to their participation.

Data should be owned by researchers, not the platform. Normally, personal data for research would be collected for “legitimate research purpose only” [7]. But many UX research platforms claim ownership of the data in their terms of service, allowing them to use it in any way they see fit. Many platforms explain that they use this participant data to improve their services, but nothing prevents them from sharing or selling it without participants’ knowledge.

In my correspondence with one such platform, I inquired about the ability to delete data at the completion of a study. The platform agreed to give me special permissions to request data deletion on behalf of my participants. This would at least limit platforms’ access to participants’ data only to the duration of the study. However, many platforms don’t allow researchers to download data, meaning that data deletion would remove access for the researchers too.

Researchers should mitigate adverse events. Researchers must plan for the possibility of unintended harm to participants’ safety and well-being [8]. In a forthcoming study for which I considered using a remote UX research platform, there was a rare possibility that participants would experience emotional distress. To account for this scenario, our research team included a certified clinician who could reach out to participants, assess suspected harm, and fulfill any obligation to escalate and report.

Platforms usually withhold participants’ contact information to spare researchers from additional exposure to identifiable data. In a case of suspected harm to a participant, however, my expectation was for platforms to collaborate to best support participants’ well-being. Instead, the platform stated that in such an event, I would report it to them and they would handle it internally.

This is unacceptable. As a researcher, the consent agreement is between me and the participant. I am responsible for any safety risks my research poses, including cases that require special attention. No matter how well prepared the platform is, they are not the ones who should be making the call on how to address concerning responses to research questions.

Concluding thoughts. The current state makes it hard for university researchers to use these remote UX research platforms. Universities have detailed training and research protocols for anyone involved in human subjects research, and many of them would not approve the terms of current platforms. By contrast, many companies and organizations do not have strong ethics review processes for research conducted by employees or contractors, and they commonly use remote UX research platforms. Even though many industry researchers have gone through academic ethical training, these platforms provide an easily overlooked path toward conducting human subjects studies with few safeguards and considerations about how data is collected, stored, and used.

This may be because many of these platforms were designed with the researcher’s user experience as a first priority, leaving participants’ well-being as an afterthought. The result is that participants are exposed to unnecessary risk of harm. While in most instances this would go unnoticed, in extreme cases things could go very wrong.

But not all UX research platforms are created equally—there are a few platforms that prioritize privacy and participant safety. They are not as user friendly and usually do not include participant recruitment. But as recruitment in these safer alternatives is under the researchers’ control, so is the data, the consent process, and the handling of adverse events. We should be able to have the best of both worlds—the capabilities and convenience we desire along with the privacy and safety practices we need.

Endnotes

1. Moon, Y. The effects of distance in local versus remote human-computer interaction. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 1998.

2. Süner-Pla-Cerdà, S. et al. Examining the impact of Covid-19 pandemic on UX research practice through UX blogs. In Design, User Experience, and Usability: UX Research and Design. Springer, Cham, 2021.

3. Fiesler, C. et al. Exploring ethics and obligations for studying digital communities. Proc. of the 2016 ACM Conference on Supporting Group Work. ACM, New York, 2016.

4. National Institutes of Health. Human subjects research; https://grants.nih.gov/policy/...

5. Antle, A.N. The ethics of doing research with vulnerable populations. Interactions 24, 6 (2017), 74–77.

6. Vitak, J. et al. Beyond the Belmont principles: Ethical challenges, practices, and beliefs in the online data research community. Proc. of the 19th ACM Conference on Computer-Supported Cooperative Work. ACM, New York, 2016.

7. Alsmadi, S. Marketing research ethics: Researcher’s obligations toward human subjects. Journal of Academic Ethics 6 (2008), 153–160.

8. Do, K. et al. “That’s important, but…”: How computer science researchers anticipate unintended consequences of their research innovations. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2023.

---

Posted in: on Wed, July 26, 2023 - 3:03:00

Michal Luria

Authors: Rojin Vishkaie
Posted: Thu, July 06, 2023 - 12:04:00

---

Does our future have researchers who are robots? Are computers going to become so smart that they can boss these researchers around? And, if robots and computers can accomplish everything, then what are human researchers going to do? To explore these questions, I have been reading AI Superpowers: China, Silicon Valley, and the New World Order [1], a 2018 nonfiction book by Kai-Fu Lee, an artificial intelligence pioneer and venture capitalist.
 
The book draws on Lee's technological knowledge and professional expertise to offer a perspective on the interplay between the U.S. and China in the age of artificial intelligence, and its impact on the world. Lee believes that the U.S. is taking advantage of being the first mover in the AI race, but that China is ahead of the race in its data supply. From his perspective, the elite AI experience in the U.S. is less important than China’s AI implementation superpower. 
 
But Lee’s argument goes beyond the AI race between the U.S. and China; it’s also about the impact of AI on humans. In his view, the U.S. and China have leapfrogged, and sit atop of, the rest of the world. The U.S. and China’s disproportionate power has worsened global inequality, and the fear of possible massive job losses because of AI will likely reduce the ability of the job market to cope and adapt. People losing their jobs will negatively impact their purpose and personal identity, as well as their ability to survive, which cannot be addressed by socioeconomic concepts like universal basic income. 
 
But despite the many warnings, Lee has a positive outlook that AI and humans can productively coexist. Lee’s optimistic belief suggests that humans are the catalyst for shaping that future of AI. He suggests that the building blocks of a successful AI superpower is abundant data, AI scientists, AI-friendly policies, and tenacious entrepreneurs. 
 
Lee believes that China has access to a massive quantity of data due to its large base of users, while the U.S. has an edge in elite AI expertise. He further argues that the Chinese government has a more sophisticated structure to mobilize resources and implement strategic directions for long-term goals (e.g., Development Plan for a New Generation of Artificial Intelligence). He postulates that the prevailing Silicon Valley mentality is mission-driven and techno-optimistic, believing that innovative thinking can change the world. In contrast, China has focused on market and financial profit. In Lee’s view, the AI new world order is bipolar and focused on AI superpowers versus the rest of the world, but the actions of humans themselves are the most important factor in shaping the future of the human ability to coexist with AI. 
 
What I’ve outlined is not a theoretical argument about the future, but rather is a perspective on researchers’ coexistence with AI while considering Lee's views on the potential impact of AI on human work. I believe that researchers can create an ecosystem where AI enhances productivity (e.g., using chatGPT [2] to assist with data management and insights). The ecosystem of research and AI can also be used for socially beneficial activities (e.g., generating knowledge with practical applications). 

Endnotes
1. Lee, K.F. AI Superpowers: China, Silicon Valley, and the New World Order. Houghton Mifflin, Boston, MA, 2018. ISBN: 9781328546395

2. Lock, S. What is AI chatbot phenomenon ChatGPT and could it replace humans? The Guardian. Dec. 5, 2022; https://www.theguardian.com/te...

---

Posted in: on Thu, July 06, 2023 - 12:04:00

Rojin Vishkaie

Authors: Malak Sadek, Rafael Calvo, Céline Mougenot
Posted: Tue, June 27, 2023 - 12:04:00

---

It is estimated that 98 percent of the population are novices with regard to technology (excluding extremes such as infants) [1]. It is this 98 percent, however, that form the main chunk of users and stakeholders affected by AI-based systems. It then makes sense that members of this segment of the population should be involved in designing these systems beyond just the small, homogenous set of experts currently involved.

There have been countless calls for the introduction of a transdisciplinary, participatory design process for AI/ML systems [2,3]. Such a collaborative design (codesign) process has been heralded as especially useful in aiding explainability and transparency [4], embedding values into AI-based systems [5], providing accountability, and mitigating downstream harms arising from several cascading biases and limitations [6]. There have also been calls for collaboration within the entire AI pipeline, including in data creation and selection, instead of having designers at the front end of the process and engineers at the back end [7]. In fact, it has been said that the only way to combat existing structural and data biases that creep into AI systems is to step away from custom-built, solely technical solutions and view AI systems as sociotechnical systems [8]. By looking across them as opposed to inside of them and using open constructive dialogues, collaborations, and group reflections [9], we can bridge the gulf between stakeholder visions and what gets implemented [7]. Having the needed voices give their input in a meaningful, impactful way through codesign activities can allow us to examine this broader frame of the wider sociocultural contexts in which AI systems are being used [10].

Despite the numerous cited benefits of and calls for codesigning AI systems, it is still not a widespread practice. The communication gap between designers’ products and services, developers’ algorithms and systems, and users’ needs and applications, is one of the largest challenges when it comes to using AI systems in real-world settings, where its implications could be life-altering for different stakeholders [10].

Bridging this gap is a unique, and notoriously difficult process for AI systems, as it has several inherent differences to other types of systems which require various considerations across different design and development phases [10]. We summarize several of these challenges below in an effort to consolidate the barriers to codesigning AI-based systems. The goal is to start a conversation on possible solutions and ailments to enable a much-needed, more widespread adoption of codesign practices for these systems. 

Why Is Codesigning AI Different and Difficult?

Data considerations

• There are several extra considerations when it comes to AI systems about which datasets to use or what type of data to collect, as these systems are based on existing training data from the get-go, as opposed to other systems where user data can be collected after the system has been created for evaluation purposes. Additionally, assessing the initial feasibility of many ideas and being able to develop prototypes for them will be largely dependent on whether the data exists or can be collected.
• There are also several ethical and moral concerns that arise regarding data provenance and data quality.
• Both technical and nontechnical participants tend to focus on the ML model itself, overlooking the rest of the wider system, the data needed, and the broader sociotechnical and interactional contexts surrounding the system and in which it is embedded. This is especially true when tackling value-related or nonfunctional requirements such as fairness. Finally, there is a general lack of awareness regarding values and nonfunctional requirements in the field and how different ML design decisions affect them, in addition to no regulations regarding their elicitation and use, and a deficiency of tools and methods for measuring them.

Ideation and communication

• It is difficult for technical experts to explain to users and nontechnical experts an AI’s behavior, what counts as AI, and what it can/cannot do.
• It is also difficult for designers to communicate AI design ideas, interactions, and appropriate use cases/user stories to technical experts, codesign partners, and users. It is also challenging to imagine ways to purposefully use AI to solve a given problem, creating an overall “capability uncertainty” [10].
• It is challenging for designers and developers to understand how to collaborate and co-ideate without a common language or shared boundary objects, especially when designers join late in the project.

User research

• Embedding values into AI systems must be planned for from the offset, instead of being added in post-hoc, because the way the system is built will influence which kinds of values and ethical systems can be ‘loaded in.’ There then becomes an additional step of eliciting those values that diverse stakeholders agree on early in the process and upholding them on a sociotechnical scale.
• User experience research also becomes more difficult for AI systems, as user profiles and preferences are created dynamically during interactions and there is no predefined persona or profile to check against.

Design, development, and testing

• Designing AI systems goes beyond simply grounding design outcomes in what is technically viable, but also understanding and designing for the critical trade-offs between different algorithms and understanding the constraints, abilities, and interactions of the model.
• It is difficult to anticipate unpredictable or unwanted AI behaviors and effects, and how they will evolve over time. It is also often unclear whom to hold accountable, and very difficult to communicate all of this to users.
• It is also challenging to design and visualize branching, open-ended AI interactions, difficult to envision the potential of AI for use cases that do not already exist, and difficult to rapidly prototype AI ideas. Designing interactions for the incredibly complex and diverse outputs and possible errors of an AI system is challenging.
• Compared to other systems, it is more difficult to respect and implement some stakeholder values or nonfunctional requirements such as transparency and fairness, given the complex, uncertain, and often unpredictable nature of AI/ML systems (and the high-level, general nature of those values ). It is also challenging to measure them within those systems.

This list is by no means comprehensive but instead aims to be a starting point for a dialogue on the different challenges that appear during various design stages for AI-based systems. By summarizing and consolidating these challenges, we can begin to discuss solutions and mitigations as a community in order to make codesigning AI systems an easier and more practical endeavor.

Endnotes

1. Sadler, J., Aquino Shluzas, L., and Blikstein, P. Abracadabra: Imagining access to creative computing tools for everyone. In Design Thinking Research. H. Plattner, C. Meinel, and L. Leifer, eds. Springer, 2018, 365–376.

2. Gabriel, I. Artificial intelligence, values, and alignment. Minds and Machines 30 (2020), 411–437.

3. Yu, B., Yuan, Y., Terveen, L., Wu, S., Forlizzi, J. and Zhu, H. Keeping designers in the loop: Communicating inherent algorithmic trade-offs across multiple objectives, Proc. of the ACM Designing Interactive Systems Conference. ACM, New York, 2020.

4. Chazette, L. and Schneider, K. Explainability as a non-functional requirement: Challenges and recommendations. Requirements Engineering 25 (2020), 493–514.

5. Crawford, K. and Calo, R. Here is a blind spot in AI research. Nature 538 (2016), 311–313.

6. Delgado, F., Barocas, S., and Levy, K. An uncommon task: Participatory design in legal AI. Proc.of the ACM on Human-Computer Interaction 6, CSCW1 (2022).

7. Frost, B. and Mall, D. Rethinking designer-developer collaboration. 2020; https://www. designbetter. co/ podcast/ brad-frost-dan-mall 

8. Ananny, M. and Crawford, K. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society 20, 3 (2018), 973–989.

9. Yang, Q., Scuito, A., Zimmerman, J., Forlizzi, J., and Steinfeld, A. Investigating how experienced UX designers effectively work with machine learning. Proc. of the CHI Conference on Designing Interactive Systems. ACM, New York, 2018.

10. Yang, Q., Steinfield, A., Rosé, C., and Zimmerman, J. Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. Proc. of the CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020.

---

Posted in: on Tue, June 27, 2023 - 12:04:00

Malak Sadek

Rafael Calvo

Céline Mougenot

Authors: Kyle Boyd, Raymond Bond, Maurice Mulvenna, Karen Kirby, Susan Lagdon, Becca Hume
Posted: Thu, June 08, 2023 - 3:49:00

---

Do you feel safe when you travel alone? That was a question I asked my girlfriend (now wife) as we took a trip on a relatively empty train when I was visiting her at university. “I always carry my phone with me, but sometimes it's actually worse when the train is full; people watch what you are doing on your phone, and it's very off-putting,” was her reply.

Fast-forward 10 years and we are still very much in the same predicament. People, especially women, can feel vulnerable in certain situations. A 2019 survey [1] by SAP Concur, a company that provides travel and expense-management services to businesses, agrees. The survey revealed that safety is the number one concern for female business travelers. Indeed, 42 percent of respondents said that they have had a negative experience related to their gender. This isn’t limited to gender, though. Our intention is to highlight that people can feel vulnerable because of certain characteristics, including disability or race, and in certain situations.

With the rise of ubiquitous computing, we have the ability to use a plethora of devices that can connect to the Internet. This allows us to work, shop, and entertain ourselves from almost anywhere. Devices and their interfaces have become an extension of our own selves. 

In recent decades, smartphones, emails, and instant messaging have been the predominant methods for human-to-human communication. And now with “generation mute” [2], instant text messaging, as opposed to making phone calls, has become the primary use of smartphones. With growing concerns over cybersecurity and privacy, instant messaging apps such as WhatsApp incorporate end-to-end encryption.

Modern text messaging apps allow users to send messages that are discreet in the sense that they are secure. But of course the interaction is not entirely private, since any onlooker could potentially see the user typing and sending a text message. A more discreet interaction might be the use of secret codes to input into the messaging app, which would be indecipherable to onlookers. However, this would add to the cognitive load and require a new communication code. Other apps such as Snapchat have functionality to send messages and photos that are only temporarily available; they are automatically deleted shortly after being viewed. These apps could be considered a “discreet user interface,” since though a spouse or family member might see the message or photo being sent or received, they cannot confirm it by interrogating the user’s stored media. With these examples, digital communication today is discreet in terms of privacy and data encryption during transit; however, the user interaction is not hidden from onlookers. With this can come unwanted attention, as onlookers can view and digest some of the information on your digital device, whether this be texting a friend from the train or doing work on your laptop on an airplane. In 2019, HP conducted a “creepers and peepers” survey [3], asking 3,000 people in the general population and 1,500 staff whether they would look at another person’s computer screen during a flight. Astonishingly, 4 out of 5 look at other people’s screens; 8 out of 10 restrict what they look at on their device just in case people are looking at their screen. This gives rise to questions about device-use etiquette and safety when using digital devices in public, stretching further for interfaces used for finance or security.

As interaction design researchers, we want to consider how design could help. Could we design a hidden user interface for safety purposes? This work is aimed to be both conceptual and thought-provoking.

Discreet User Interfaces
Do you remember the first time you saw the interface on a first-generation iPhone? It was staggering. Swiping, zooming, pinching, and animations that allowed for delight and control. All these new controls that allowed the content to take center stage and be the interface. Edward Tufte was correct when he said that “[o]verload, clutter, and confusion are not attributes of information, they are failures of design.” As interface designers, we strive for simple design that won’t get in the way, that will allow us to complete tasks in an efficient and satisfactory manner, producing interfaces that work. For many use cases, this is the appropriate design methodology to use when it comes to designing interfaces.

But what design methodology should we use if we wanted to create a “discreet user interface”? A discreet user interface is a user interface that looks like it’s doing one thing but actually is doing something entirely different. Alarm bells might start ringing when we put it this way, but that doesn’t mean people are doing anything sinister. Quite the opposite. A discreet user interface could be there to help and protect, even though it is “ethically deceiving” onlookers by preventing them from interpreting what the user is actually doing on their screen. 

What was historically known as the “boss key” could be described as a kind of discreet user interface. A boss key is a key on the keyboard (e.g., F10 or ESC) that a user presses in haste during “play time” when their boss is nearby, displaying a fake spreadsheet, business graph, or slides to discreetly deceive the onlooking boss. Other interactions that we could also consider to be discreet interactions, or at least discreet features, include “Easter eggs,” which were hidden features inside Microsoft products. These could also be hidden games, such as pinball or a flight simulator. And of course even smartphones have hidden features, such as shaking the phone to undo an action, facilitated by sensors such as an accelerometer.

With this in mind, and perhaps more intuitively, discreet user interfaces could be those that a user can interact with while preventing an onlooker from knowing that they are using a computer. These interfaces could fall under the definition of natural user interfaces, where the user can interact with digital systems using their eyes, hand gestures, brain signals, and muscle movements, including facial movements. This can include “silent speech interfaces.” For example, Kapur et al. [4] developed the AlterEgo system, which allows people to interact with a computer through neuromuscular activity. Using electrooculogram (EOG) sensing, Lee et al. [5] developed a system that allows users to discreetly interact with computers by rubbing their “itchy” nose. And Cascón et al. [6] developed the perhaps even more discreet ChewIT, a system that allows users to interact with computers via intraoral interactions by essentially sending discreet signals by chewing. While these are very interesting innovations, they require additional hardware and sensors and are mostly at the research and innovation phases.

Another form of discreet user interfaces could be described as “camouflaged user interfaces” or “ulterior user interfaces.” These are interfaces that give onlookers a fake impression of an app’s utility, with its true, ulterior hidden utility known only to the user. For example, while an onlooker might see the user engaging with a mundane weather app, a sports app, a news app, or a shopping list, the user could be performing hidden tasks or sending secret signals in plain sight by interacting with buttons and features that have hidden functions. The interaction might look mundane, but the user could be sending a secret communication. Hence, this ulterior user interface could also be considered a discreet interaction. This concept is akin to steganography, where an everyday mundane image contains a hidden message that can be extracted using a secret algorithm; however, steganography only applies to the message and does not encompass the user interaction. If it did, there might have been a concept coined as stegano-graphical user interfaces. 

A discreet user interface could also help protect those in need and hinder “peepers and creepers.” Use cases could be messaging, banking, or general phone use in a public space. It could go even further, providing an outlet for those who are subject to coercive behavior or domestic abuse. At the same time, if an assailant or abuser found the discreet interface, this could put the user in danger. That is why this type of solution, if developed, may need to be rolled out by police or social services so that it could remain anonymous and safe.

A Simple Case Study: A Discreet News App
An example of a discreet user interface and use case could be the following: A female commuter, Jane, traveling home from work gets on board a busy train during peak hours. There are not many seats, so Jane stands. Along her 30-minute commute, two male commuters come on board and, given the limited space, encroach on Jane’s personal space. As Jane uses her smartphone, she becomes aware of the men’s presence behind her, making her feel vulnerable. They are watching everything she is interacting with, peeping and creeping. 

To stop this, she switches to her hidden interface, the “news” app (Figure 1), and reads a news item before interacting with a discreet feature. The feature allows Jane to discreetly, semiautomatically message a friend and alert her to meet her at the next station. As the train pulls into the station, Jane again uses the news app, where secret messages inform her of her friend’s confirmation and whereabouts.

The use case is twofold. In one sense, it’s a regular news app that will display a range of articles that can be read. But upon tapping an icon or button, the user interface can make calls and send prewritten text messages, all using the same content areas and buttons that already exist in the app. 

 

Figure 1: This discreet user interface is called the news app. It displays news items, but by tapping icons and buttons, the user interface can send hidden communications, allowing the user to, for example, send prewritten messages, take a photograph, or make a call. Credit: onyxprj_art at vecteezy.com, licensed under CC BY 4.0.

Conclusion
This article highlights the growing problem of “peepers and creepers,” unwanted attention by others while interacting with our digital devices. We considered a novel solution called discreet user interfaces, where an interface can look like it has a mundane utility but can have underlying functionality that allows it to discreetly perform other functions (e.g., hidden communication). We also presented a case study in the form of a discreet news app that showcases how a discreet user interface could work. We hope this article will be thought-provoking and will widen the conversation around discreet user interfaces and their use cases in interaction design and applications.

Endnotes
1. Haigh, L. Female travellers: a unique risk profile. ITIJ 230 (Mar. 2020);  https://www.itij.com/latest/lo...
2. Dawson, A. So Generation Mute doesn’t like phone calls. Good. Who wants to talk, anyway? The Guardian. Nov. 7, 2017; https://www.theguardian.com/co...
3. HP creepers and peekers survey. HP Press Center. Sep. 25, 2019; https://press.hp.com/us/en/pre...
4. Kapur, A., Kapur, S., and Maes, P. Proc. of the 23rd International Conference on Intelligent User Interfaces. ACM, New York, 2018, 43–53; https://doi.org/10.1145/317294... 
5. Lee, J. et al. Itchy nose: Discreet gesture interaction using EOG sensors in smart eyewear. Proc. of the 2017 ACM International Symposium on Wearable Computers. ACM, New York, 2017, 94–97; https://doi.org/10.1145/312302... 
6. Cascón, P.G., Matthies, D.J.C., Muthukumarana, S., and Nanayakkara, S. ChewIt: An intraoral interface for discreet interactions. Proc. of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2019, Paper 326, 1–13; https://doi.org/10.1145/329060...

---

Posted in: on Thu, June 08, 2023 - 3:49:00

Kyle Boyd

Raymond Bond

Maurice Mulvenna

Karen Kirby

Susan Lagdon

Becca Hume

Authors: Linnea Öhlund
Posted: Mon, May 15, 2023 - 10:28:00

---

My grandma recently passed. At the age of 88 she went to sleep never to wake up again. She lived a long, happy life, and had two sons, six grandchildren, and currently four great grandchildren. Bye grandma, I will always love and miss you. 

Now to the point. My grandma was one of those individuals who never really “got it” when it came to technology. She only got a cellphone at the age of 78, because we forced her to, and the computer she and my grandfather owned was barely used. I have a vivid memory of me at perhaps 14 years old, asking them to use the computer so I go into paint (that was the most fun they had on the old thing). Grandma got a bit upset, saying that I couldn’t use paint because I might destroy the computer—something could go wrong and I would break it. Me, a person born in 1995 who grew up using computers, knew very well that no computer would break by simply opening paint; however, my grandma, who seemed to have no real understanding of how a computer worked, treated it like a fragile crystal vase, and I was an elephant ready to smash it! 

After a while, they just got rid of the computer. The phones they had were the ones we see many senior citizens using, with the big buttons and no apps—just call and text functions. But let’s make one thing clear: My grandma and grandpa (who passed in 2020) were not afraid of or negative toward technology; they understood the impact that the Internet and technology had on society. A part of them wanted to join the revolution, wanted to see what the fuss was about, wanted to join the party. They were both born so long ago (1925/1934) that only after they had retired did the Internet, mobiles, and computers become big. Grandma became a housewife in the 50s and grandpa retired in the 90s, both coming from small country villages in the north of Sweden. When and where they were born, grew up, and settled surely affected how they felt about technology as well as their ability to use it. I’ve been thinking recently that there seems to be so much pressure on seniors these days to adapt, to learn, to go above and beyond to use technologies. No more bank offices, JUST VISIT US ONLINE, no more paper bills, JUST PAY THEM ONLINE, no more post in the mail, JUST GET THEM ONLINE. But what if people don’t want to do that? What if they just want to live a quality life and not have their happiness depend upon technology? I once did a study with 15 seniors, asking them about their opinions on digital technology. One female participant, who herself was about 70, had a mother who was almost 100. She didn’t want to use technology but felt left out of due to the very large amount of digital and technological artifacts that are needed to do many things. She finished the interview by saying how she felt there was still a generation of seniors who needed to pass away before the intense digitalization in the Swedish society would be successful. These seniors, much like my grandparents, did not have a smooth transition from analog to digital in their lives and were, frankly, left behind due to this. 

Now that both my grandparents from older generations have passed, I think of that comment by the participant. One more step toward a successful digital society where there are no longer any individuals left who do not want to adapt. What type of society is that? Waiting for its inhabitants to pass? Society needs to adapt to its inhabitants and offer opportunities for those who choose to not use technology. They still have the right to good, happy lives not degraded because of society’s intense technological push. This should be obvious, but is clearly not.

So here’s to you grandma, for your glass vase computer and your amazing, basically technology-free life! Cheers! 

---

Posted in: on Mon, May 15, 2023 - 10:28:00

Linnea Öhlund

Authors: Chameera De Silva, Thilina Halloluwa
Posted: Tue, May 09, 2023 - 3:06:00

---

Have you ever played a game against a computer? Have you ever wondered how computers win complex games against well-experienced players, such as when an AI system from IBM defeated Garry Kasparov in the Chinese board game Go?

The fear of AI becoming the biggest threat to humankind, replacing the human workforce on a large scale is a real concern for AI practitioners and businesses. But there have been incidents where the combination of a smart AI system supervised by humans has resulted in unbelievable achievements. Technology has developed magnificently in the past decade, and the widespread adoption of smart systems powered by AI predicts a much smarter and tech-dependent world. New-age AI systems will become more and more learning-based. The massive data generated by the Internet will be used to train these models and systems to improve their intelligence in specifically focused areas.

The current focus of AI developers is to make the algorithms better and optimize the performance of the AI models as much as possible. We’re heading toward Web 3.0; its adoption will be much faster than that of Web 1.0 and Web 2.0 and driven by the adoption of AI.

The super-fast development and integration of AI tools into the products and services we use is accelerating the pace of our lives. Alongside this development is the fear that AI will exceed human cognitive intelligence, creating a struggle to maintain our control over them.

This fear of AI replacing humankind is the motivation for human-centered artificial intelligence (HCAI), where at one end we have AI and at the other end human beings supervising the actions and results of AI. The development of this technology has happened intuitively and on multiple levels.

In the early days of the Web, Web 1.0, we could only read the content online. The public had no control of over what they wanted to read or consume, and we could not search for things; it was just one-way communication. This motivated the creation of Web 2.0, which gave us social media applications like Facebook, Youtube, and Google. This made the communication on the Web two-way. We were now able to decide what content we wanted to see. We could not only share our views as comments but also could now like or dislike posts. Now, as we are involving and interacting with technology more and more, we are finding more ways to incorporate technology into our lives, and one way of doing it is by developing AI applications.

Artificial Intelligence
Humans are constantly working on creating machine intelligence similar to that of human intelligence; this is called artificial intelligence. AI is one way we are using technology and creating applications and systems that are automated and can work for us without any supervision. Self-driving cars, lie detector machines, and robots that can do surgery are just some examples of automations made possible by artificial intelligence. Current research is targeting advances in many sectors, including finance, education, and transportation.

These advances mean incorporating AI into our work processes at higher levels. This view of involving AI into human life is a matter of great concern today, often creating fear of AI and pitting technology against humankind.

What Is the Fear?
Machines are no doubt very fast and accurate in calculating numbers and analyzing large numbers of scenarios and methods within minutes. This speed and accuracy is very helpful to humans, but relying on machines completely just because of their speed and accuracy is a bad idea. Machine intelligence is smarter than human intelligence in the areas in which the machine is trained—but it cannot be trusted blindly.

The ethical and safety implications of learning-based AI systems have proven these systems to be unsafe in many ways. The decisions and results provided by these systems are not guaranteed to be fair and have also been shown to be not perfectly explainable. That is, the current models are black-box models for us humans, as they do not provide reasons for the results given by the machine.

Real-world scenarios cannot work on ambiguity and ethically unsafe standards.

AI is making machines intelligent, but we cannot be guaranteed of the safety of these systems in any specific way unless they are extremely constrained.

What Are the Challenges?
There are several challenges with AI machines today. The results given by AI systems are trusted because they are provided by computers. But the explanation of these results is very important as we scale up the use cases and implementation of these smart systems. The current solution to this challenge is explainable artificial intelligence. Explainable AI adds a level of transparency in the decision and processing done by AI systems. But as we incorporate AI systems in sectors like education, medicine, and finance, explainability is not enough.

AI in the education sector
The systems involved in the education sector will be assigned to grade students’ papers, mark their attendance, and look for any possibilities of cheating during examinations. The results given by the systems in this area impact the academics of the students directly; therefore, it is crucial to ensure that the decisions are fair and grounded on the actual rules of the real world. To achieve this  level of trust with machines, human supervision is necessary.

AI in the medical sector
Likewise, the involvement of AI systems in the medical sector is opening new doors of advancement in medicine. Robots are performing surgeries, as well as recommending medications to patients on the basis of their symptoms.

The recommendations of these system are highly likely to directly impact the patients’ health and life;  therefore, it is extremely important that the recommendations are 100 percent correct. But here, due to the subjective nature of the patients’ symptoms, the chances of machines making a mistake is very high. For example, there could be a case where the patient is feeling chest pain due to low blood pressure, but the machine detects it as a heart attack. These kinds of mistakes can cost lives and thus there must be zero-tolerance for them.

The challenge
Because of the lower cognitive and emotional intelligence of machines, it is necessary to incorporate smart systems alongside humans and not in place of humans. The desired advancement can be achieved only when humans and machines work together.

Human-Centered AI
Human-centered AI (HCAI) is AI with human supervision. There are three main pillars of HCAI: humans, ethics, and technology. Human-centered AI is the solution to the fear of AI exceeding human intelligence and thus becoming a threat to humankind.

HCAI will ensure that AI systems are supervised by humans and that the processing and decisions of smart systems are all under the control of a human (Figure 1).
 

Figure 1. Human-centered AI combines humans, ethics, and technology.

But what exactly is human-centered AI? The basic idea involves the deep integration of humans into the data-annotation process and real-world operation of the system in both the training and the testing phases. HCAI will combine human-driven decision making with AI-powered smart systems, thereby making them smart, fast, and trustable. Ethical and responsible AI systems will become a reality by incorporating AI alongside humans. Augmenting AI systems with human abilities will also bring us usable and explainable AI. This is essentially human-controlled AI, combining the modern power of artificial intelligence with human intelligence.

HCAI will address the challenges of the lower emotional intelligence of machines and the lower speed and accuracy of humans together.

How is HCAI possible?
Current AI systems are the learning bases for AI systems that get trained on data and are then tested on similar datasets to check their accuracy. These datasets are created by humans, and a large amount of time, energy, and money is invested in performing detailed annotations to create a dataset suitable for training an AI model.

The motive of HCAI is to enable an annotation process generated by machines in the form of queries to humans. The machines will decide which data is necessary to be annotated by humans and only that subset of the data will be given to the humans for annotation. This is one way to create human-centered AI.

With regard to real-world applications, current systems just deliver the results to us and we receive them. With HCAI, the results given by the AI system will have a degree of uncertainty associated with them from the AI system; this result will then be sent to the human supervisor. When the results get the thumbs up from the human supervisor, then and only then are they sent as the final result (Figure 2).

Figure 2. Multiple AI systems arguing with human supervision.

This is the basic, ground-level idea of HCAI. It can get much deeper and can exist in many formats.

Algorithms in HCAI
The algorithms of current AI systems are created with the purpose of training the machine-learning models. The algorithms in HCAI will be designed in such a way that they give the AI system an idea of the context of human intelligence, both in the moment and over time. Smart systems will be trained not only on the annotated data mechanically but will also get training on the human understanding of the concept. This will make AI systems even smarter and reduce the risk of unwanted out-of-the-box results.

Human-computer interaction
To create the algorithms that teach the computer the context of human understanding, we need human-computer interaction. To make the interactions with humans successful and fruitful for HCAI purposes, they must be continuous, collaborative, and provide a rich, meaningful experience.

We currently have autonomous vehicles driving more than 100 billion miles on the road in auto-pilot mode, some of them in semi-autopilot mode. These cars and the smart systems installed in them aim to have a rich and fulfilling human-computer interaction experience. The cars have a bunch of super-cool devices monitoring and recording the multiple modes and emotions of the driver and try to give us a better interactive experience. But we are seeking a collaborative, rich interactive experience that goes far beyond this.

The challenge for current smart car systems is that these cars are intended to be used by people from both the younger generation and the older one, who might have never used any AI-based system in their entire life. The real challenge is thus creating HCI that is capable of deeply exploring these aspects and providing a rich, interactive, and collaborative experience.

Safety and ethical awareness
Artificial intelligence or machine intelligence in general brings many concerns around safety and ethics.

Taking the example of self-driving cars, the passengers riding in a car that is in autopilot mode are prone to trust the system, as the system only provides the decisions it is making based on the situation. There is no information given regarding the level of uncertainty, nor any warnings from the system to the driver.

HCAI is a solution for this safety challenge. We can create a system with multiple AI models deciding on a single situation, and whenever a disagreement in the decisions is detected, human supervision is then sought. This is one way of involving human interaction in the decision-making and validation processes.

The other challenge is the ethical awareness of the AI system. There are certain situations in the real world that cannot always be dealt with technically. The ethical aspects must be considered, and this is an area we need to work on to optimize our AI models.

Until AI models are smart enough to understand the ethical and emotional aspects of the real world, they need to be combined with human supervision in the form of HCAI. In addition, some of the current AI use cases, when enabled with stronger and safer HCAI, can make things much more secure for users.

HCAI Today
HCAI designers go beyond thinking of computers as our teammates, collaborators, or partners. They are more likely to develop technologies that dramatically increase human performance by taking advantage of the unique features of computers.

Recommender systems
A current system that is being implemented almost everywhere and is making things autonomous for humans is the recommender system. Widely used in advertising services, social media platforms, and search engines, these systems bring many benefits.

The consequences of mistakes by these systems are subtle, but malicious actors can manipulate them to influence buying habits, change election outcomes, or spread hateful messages and reshape people’s thoughts and attitudes. Thoughtful design that improves user control could increase consumer satisfaction and limit malicious use.

Other applications that enable automation include common user tasks such as spell-checking and search-query completion. These tasks, when carefully designed, preserve user control and avoid annoying disruptions, while offering useful assistance.

Consequential applications
The applications in medical, legal, or financial systems can bring both substantial benefits and harms. A well-documented case is the flawed Google Flu trends, which was designed to predict flu outbreaks, enabling public health officials to assign resources more effectively.

A harmful attitude among programmers called “algorithmic hubris” suggests that some programmers have unreasonable expectations of their capacity to create foolproof autonomous systems.

The crashes in stock markets and currency exchanges caused by high-frequency trading algorithms triggered losses of billions of dollars in just a few minutes. But with adequate logging of trades, market managers can often repair the damage.

Life-critical systems
Moving on to the challenges of life-critical applications, we find physical devices like self-driving cars, pacemakers, and implantable defibrillators, as well as complex systems such as military, industrial, or aviation applications. These require rapid actions and may have irreversible consequences.

One-dimensional thinking suggests that designers must choose between human control and computer automation for these applications. The implicit message is that more automation means less control. The decoupling of this concept leads to a two-dimensional HCAI framework.

Two-Dimensional HCAI
Two-dimensional HCAI suggests that achieving high levels of both human control and automation is possible, shown in Figure 3.

Figure 3. Two-dimensional HCAI.

The goal is most often but not always to be at the upper-right quadrant. Most remote sensing technology systems are on the right side.

The message for designers from two-dimensional HCAI is that, for certain tasks, there is a value in full computer control or full human mastery. However, the challenge is to develop an effective and proven design, supported by trusted social structure, reliable practices, and cultures of safety so as to acquire RST recognition.

Conclusion
To conclude, artificial intelligence is not a threat to humankind if we involve it into our lives as a companion and learn to work alongside it. There are several ways that we can teach AI systems to accompany us humans in our tasks and make them understand us. We can also create a system with multiple AI models that argue over decisions and then seek human supervision on every disagreement.

The idea of two-dimensional human-centered artificial intelligence is making the vision of HCAI more clear for designers and developers and is redefining the challenge in a new way. Designers need to create trustable, safe, and reliable HCAI systems supported by cultures of safety so as to acquire technical recognition.

That said, nobody is perfect, neither artificial intelligence nor humans. But we can join hands and provide something enriching to each other. AI is the future, and the future is beautiful. 

Additional Resources
1.

2.

3.

4. Human-Centered Artificial Intelligence: Reliable, Sage & Trustworthy, Ben Shneiderman, [email protected], Draft February 23, 2020, version 29 University of Maryland, College Park, MD

---

Posted in: on Tue, May 09, 2023 - 3:06:00

Chameera De Silva

Thilina Halloluwa

Authors: Johannes Schöning
Posted: Mon, May 08, 2023 - 10:04:00

---

It has been nearly four years since I last attended an in-person conference, and I must admit that I was excited about attending the recent ACM CHI conference in Hamburg. 

In-person conferences provide a unique opportunity for researchers to meet, chat, and catch up on the latest developments in the field, as well as absorb all the excellent talks and demos. It is also super important to build networks, especially for early career researchers. 

However, the Covid-19 pandemic put an end to all that, and we quickly transitioned to digital conferences. While digital conferences had their benefits, such as accessibility and convenience, I found myself missing the social aspect of in-person conferences, particularly the informal chit-chat during breaks. 

Additionally, I hardly remembered any paper presentations from the digital conferences I attended, in contrast to talks I attended back at CHI in San Jose in 2007, which I remembered vividly due to my physical surroundings and interactions with others in the room.

Now, hybrid conferences are becoming the new normal, offering both in-person and digital attendance options. While this innovation seems like a step forward, it raises important questions. Have we truly innovated the conference experience, or did we act too quickly in response to the pandemic? Have we considered all the potential side effects of going hybrid? And perhaps more importantly, do some conferences need to die or perish?

One argument for the death of some conferences is simply that there are too many of them. It can be challenging to recruit reviewers for all the different conferences, and the field may need help to sustain the number of conferences currently in existence. 

Additionally, some conferences simply do not get the attention they deserve anymore, and it can be challenging to maintain a core audience for them. For example, I love MobileHCI, and I was recently the program chair for it. However, the number of submissions has been decreasing in the last few years, and it seems like the conference is losing its core audience. MobileHCI has become the “mini CHI,” and it’s unclear where MHCI fits into the mix of SIGCHI conferences. This is not only true for MobileHCI, but also for a set of other conferences in my humble opinion. 

Another reason to consider the death of some conferences is sustainability. While hybrid conferences can reduce the environmental impact of conferences by reducing travel and energy consumption, they still require physical resources such as venue rental and equipment. Some conferences, such as IEEE VR, have started to charge authors for each item presented at the conference to offset these costs. However, it needs to be clarified whether this approach is sustainable in the long term and also what novel barriers hybrid conferences bring, for example, for early career researchers who are “forced” into the hybrid mode instead of going to physical conferences due to budget concerns. We need to talk about CHI in Hawaii at another time, in depth.

So, what do I propose? First and foremost, I suggest that we focus on having a strong flagship conference and just a few (local) satellite conferences that emphasize the networking and experience parts of the conference—trying out demos and talking to people instead of passively absorbing talks (one can watch the talks later anyway). This would allow us to put more love into the local SIGCHI chapters and encourage them to organize workshops, summer schools, and other events that can foster sustainable growth and strong networks within the field. In addition, it would be very much up for supporting the birth of new conferences that support the emergence of new areas of research that have a certain life span and will disappear after a few editions once the field is established.

Second, I suggest that we move away from the artificial acceptance rates that have become so common in conferences. Instead, we could implement a true revise and resubmit cycle and create different presentation/experience categories at CHI. I most enjoyed trying out the demos in Hamburg. Most talks I can rewatch later on the great ACM SIGCHI YouTube channel (if you have not subscribed, I would encourage you to do so). This would allow for more diverse and nuanced discussions and help us move away from the idea that conferences are just about talks, talks, talks. It is desirable to have conferences that prioritize opportunities for presenting and discussing intriguing content rather than being overly preoccupied with enforcing deadlines and procedural protocols. Conferences should have different aims, which also help conferences to differentiate themselves from each other (presenting inspiring content, receiving quality feedback on early-stage work, showcasing the latest research, or networking and experiencing demos). Such conferences should aim to broaden the scope of curated material, with committees working more closely together and avoiding exclusive reliance on traditional peer review processes. Promoting such new formats might also mean rethinking how universities fund conference attendance, especially for early-career researchers. 

Finally, I believe that conferences should be about the exchange of ideas and experiencing new technologies. In my experience, the rejection rate for demos at CHI 2023 was far too high, and the juried process needed to be more transparent. I very much enjoyed  the “hot desk areas” for demos at CHI, where people could try out new technologies and interact with the developers in a more informal setting. We should consider truly innovative conferences that promote exchanging ideas and networking. The primary objective should be to provide an exceptional conference experience and exceptional content, regardless of its source. Moreover, there should be a platform for showcasing previously published work at premier conferences, which would help sustain smaller conferences as platforms for innovative and speculative ideas in nascent fields.

Of course, these are just my personal opinions, and they may be a bit provocative. I would love to hear your thoughts on the matter, whether here on the Interactions blog, via email, Twitter, LinkedIn, or, best, during a coffee in person at a future SIGCHI conference. Let's continue to innovate and improve the conference experience for all researchers in the field.

Acknowledgments

Special thanks go to Christine Scheef (HSG), Yvonne Rogers (UCL) and Antonio Krüger (DFKI)  for their insightful feedback and constructive criticism on an earlier version of this post. Their expertise in the field greatly contributed to the final outcome. I am grateful for their time and effort in providing valuable input, which helped to shape my thoughts.

---

Posted in: on Mon, May 08, 2023 - 10:04:00

Johannes Schöning

Authors: Himanshu Verma, Marios Constantinides, Sailin Zhong, Abdallah El Ali
Posted: Tue, April 18, 2023 - 2:45:00

---

The way we work has profoundly changed. The well-known eight-hour workday within the confines of the office, and the salient boundaries between work and personal life, are now an outdated reality. For many individuals, what used to be physical and co-located has now been replaced with notions of hybrid, blended, and flexible. However, this flexibility may create turbulence between employees and employers, depending on how employees manage their workdays, productivity, and well-being. Simply put, if we are to rethink a new future of work, we need to let go of old work habits and norms and embrace a brand-new reality of hybrid and blended experiences. 

The onset of the Covid-19 pandemic in early 2020 has significantly altered the ways we interact, socialize, work, and learn. It has forced the entire world into a prolonged social experiment, which has tested and mainstreamed what technology developers, researchers, and visionaries have been talking about for a long time: the ubiquity of working and interacting remotely. For decades, researchers and organizations have envisioned and experimented with the idea of “working from home” or “working from anywhere.” Remote work is not a new concept; we had communication tools and infrastructure to facilitate it even before the onset of the Covid pandemic.

Before Covid, an abundance of tools and technological solutions (e.g., email, Zoom, Gather.Town, Slack) enabled opportunities for distant collaborators to connect both synchronously and asynchronously. Yet these tools were never envisaged as an extended and sustained means of working, learning, or socializing in a highly interconnected world. This “pre-pandemic” social reality afforded an auxiliary and rather peripheral position to remote interactions and engagement, leaving little to no room for examining the extended remote social interactions and their short- and long-term implications for our life and well-being.

Blended Social Realities and Hybrid Arrangements
We foresee a future full of hybrid social possibilities and blended social experiences that speak to a variety of stakeholders and beneficiaries from all walks of life, from school-going children to office workers to medical personnel. Now, the next pertinent question that we have to ask ourselves is, What are these and are they the same thing? 

They are different. Hybrid social arrangements refer to the malleable distribution of a person’s social and professional lived reality—in both time and space—and the way it is positioned within the spectrum, from completely physical to completely virtual. For individuals, hybrid arrangements speak to the need for easier self-organization. For organizations, these arrangements are related to the effective governance of resources and employees’ well-being (e.g., partial work from home). Effective manifestation of hybrid social arrangements, consequently, entails designing for blended experiences, which are the essential “means to an end,” enabling individuals or collectives to meaningfully navigate their respective social and professional realities. 

Blended social realities, or “blendedness,” refers to an individual’s or group’ s practice of using digital tools and spaces for collaboration. Collaborative tasks can be mapped onto different tools and spaces with some degree of redundancy and overlap, allowing them to be effortlessly and unambiguously executed. For example, a meeting may be organized over Zoom or Social VR [1], but its agenda and discussion points may be shared over Slack. Collaborating actors may also choose to use multiple platforms for the very same task. As a summary, blendedness is a manifestation of superposed communicative and collaborative modalities and channels enabling hybridity [2]. Over time, these sociotechnical entanglements are assimilated as accepted praxis and institutional norms, in essence dissolving the boundary between tools and environments [3].

Blended working/learning/meeting experiences, as elaborated previously, encompass the dynamic and diverse ways in which the hybrid arrangements interact with digital tools and (physical or virtual) spaces to evoke a perception of social facilitation—an appropriated and rather simulated togetherness.

Over time, as existing social contracts change and technological embeddings afford new possibilities and experiences, social actors may decide to modify this mapping. Such changes may happen spontaneously and recurrently, adapting to emerging situations (e.g., a new collaborator who does not use a specific tool), technical complications (e.g., unable to share one’s screen on one platform during presentation), and organizational regulations (e.g., an organization may not allow the use of a certain platform due to contractual or accessibility reasons). Changes to these mappings may also stem from the spatial distribution of collaborating partners (i.e., whether it be colocated, distributed, or both). 

How to Design for Blended Experiences
To engage the scientific community in the debate on blendedness and contrasting it with hybridity, we organized the SensiBlend workshop at the ACM UbiComp conference in 2021 [4]. In the workshop, we sought to scrutinize the affordances of blended experiences, and raise questions relating to the specific attributes of sociotechnical experiences in the future organization of interpersonal relationships. The discussions revealed three key themes: 1) fostering serendipity in hybrid social connectedness, 2) rethinking boundaries between work and personal life, and 3) entangling “the digital” and “the spatial” worlds. 

Theme 1: Fostering serendipity for hybrid social connectedness 
The first theme is about the relationship between serendipity and blended experiences in virtual and physical worlds. The shift to computer-mediated communication has resulted in the explicit scheduling and organization of dedicated moments for social interactions, which can stifle creativity and serendipity. Researchers have been exploring mixed and extended reality approaches for supporting flexible video feed configuration and allowing natural proxemic interactions [10]. Social biosensing techniques, where remote/hybrid participants are able to share behavioral or physiological data with one another [5], could enhance the quality of social interactions beyond what reality is able to afford. Consequently, understanding the taxonomy of blendedness could help discover new forms of serendipity, time boundaries, and space multiplicities that go beyond the current practice of either face-to-face or videoconferencing interactions.

Theme 2: Rethinking boundaries between work and personal life
Personal and professional contexts entail that clear boundaries are being defined and maintained to foster work-life balance, contributing positively to employees’ well-being. Cho et al. [6] found that six types of boundary work shape remote workers’ placemaking practices: spatial, temporal, psychological, sensory, social, and technological. These boundaries were extensively tested and transformed during the Covid-19 pandemic. As an example, boundaries can be set up by using dedicated workspaces at home, by getting dressed up for work meetings, or by using hardware and software to enable transitions between work and life. However, the use of technology can also have negative effects such as Zoom fatigue [7], increased workplace surveillance [8], and the possibility of employees being monitored in a discriminatory and an unfair way. To ensure widespread adoption of these technologies, blended experiences must afford embodied mechanisms for individuals and groups not only to define and update their own boundaries but also to prevent any negative consequences.

Theme 3: Entangling “the digital” and “the spatial” 
Realization of blended experiences goes well beyond the mere design of technological artifacts, and extends into the realm of emergent work practices, with subsequent effects on how we organize our space and mobility practices. This requires rethinking how we design spaces, both virtual and physical, and the transition between the two. For example, the use of tools like Zoom and Gather.Town has led to the creation of digital twins, which can be used in hybrid classes and conferences. We foresee that, in the near future, the (re)design of spaces with blended experiences in mind will afford discoverability, well-being, and creativity. Simultaneously, this could go hand in hand with research on urban and built environments and their constitutive components such as interactive furniture, creating adaptive indoor atmospheres [9]. 

TL;DR
We argue that blendedness will be central to the entanglement of work practices, tools, and spaces, and potentially emerge as a catalyst driving the future of work. It holds the promise to augment work and its evolution by encompassing the dynamic and diverse ways in which hybrid arrangements interact with digital tools and (physical or virtual) spaces to evoke an appropriated and rather simulated togetherness. Paving the way toward a truly blended future of work entails consorted multidisciplinary endeavors across three fronts: 1) designing for serendipity, 2) improving work-life balance, and 3) weaving space and mobility with evolving work practices.

Endnotes
1. Li, J., Kong, Y., Röggla, T., De Simone, F., Ananthanarayan, S., de Ridder, H., El Ali, A., and Cesar, P. Measuring and understanding photo sharing experiences in social virtual reality. Proc. CHI '19.
2. Neumayr, T., Saatci, B., Rintel, S., Klokmose, C. N., and Augstein, M. What was hybrid? A systematic review of hybrid collaboration and meetings research. arXiv preprint arXiv:2111.06172, 2021.
3. Verma, H., Mlynář, J., Pellaton, C., Theler, M., Widmer, A., and Evéquoz, F. “WhatsApp in politics?!”: Collaborative tools shifting boundaries. IFIP Conference on Human-Computer Interaction. Springer, Cham, 2021, 655–677; https://link.springer.com/chap... 
4. Verma, H., Constantinides, M., Zhong, S., El Ali, A., and Alavi, H.S. SensiBlend: Sensing blended experiences in professional and social contexts. Adjunct Proc.of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proc. of the 2021 ACM International Symposium on Wearable Computers. ACM, New York, 2021, 491–495.
5. Lee, S., El Ali, A., Wijntjes, M., and Cesar, P. Understanding and designing avatar biosignal visualizations for social virtual reality entertainment. Proc. of the 2022 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2022, 1–15.
6. Cho, J., Beck, S., and Voida, S. Topophilia, placemaking, and boundary work: Exploring the psycho-social impact of the Covid-19 work-from-home experience. Proc. of the ACM on Human-Computer Interaction 6, GROUP (2022), 1–33; https://dl.acm.org/doi/pdf/10....  
7. Bailenson, J.N. Nonverbal overload: A theoretical argument for the causes of Zoom fatigue. Technology, Mind, and Behavior 2, 1 (2021), 1–6.
8. Constantinides, M. and Quercia, D. Good Intentions, bad inventions: How employees judge pervasive technologies in the workplace. IEEE Pervasive Computing (2022).
9. O'hara, K., Kjeldskov, J., and Paay, J. Blended interaction spaces for distributed team collaboration. ACM Trans. on Comput-Hum Interaction 18, 1 (2011), 1–28.
10. Grønbæk, J.E., Knudsen, M.S., O’Hara, K., Krogh, P.G., Vermeulen, J., and Petersen, M.G. Proxemics beyond proximity: Designing for flexible social interaction through cross-device interaction. Proc. of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–14; https://doi.org/10.1145/331383...

---

Posted in: on Tue, April 18, 2023 - 2:45:00

Himanshu Verma

Marios Constantinides

Sailin Zhong

Abdallah El Ali

Authors: Subhashish Panigrahi
Posted: Wed, April 05, 2023 - 9:23:00

---

There has been a tremendous push on many levels to make artificial intelligence– and machine learning–based applications ubiquitous. Soon, the life decisions of almost every digital technology user will be affected by some form of algorithmic decision making. However, the development of large language models (LLMs) that drive this research and development often lacks participation from diverse backgrounds, ignoring historically oppressed communities such as Black and other ethnolinguistic or socioeconomic minority groups, women, transgender individuals, people with disabilities, and elderly individuals globally, and the Dalit-Bahujan-Adivasi communities in South Asia and the diaspora. Data about and by these people is therefore systematically suppressed. Even more problematic is that this data is mostly suppressed in creating the LLMs driving AI/ML research and development.

Furthermore, seemingly public information might not always be collected ethically with informed consent from the people affected. Even mature regulatory frameworks such as the General Data Protection Regulation (GDPR) in the European Union do not provide enough guidance on how private data is collected, stored, and shared. Naturally, those behind LLM creation do not have a clue about the biases in their data or how it is collected. Take the case of DALL-E 2 models, which use publicly available images owned and copyrighted by different people, or ChatGPT, which uses massive datasets from multiple sources. In both instances, not only does the LLM creation lack the representation of marginalized groups and contain only biased data about them, but also the outcomes that derive from the training data make these groups even more vulnerable.

Palestinian children walk past rubbish next to a paved path reserved for Israeli settlers. Bias in language data is often understood by asking who has access and participation in knowledge creation and dissemination. (CREDIT: CPT Palestine, CC BY 2.0)

Low-Resource Languages
The creation of LLMs like GPT-3, when used in applications such as chatbots, directly affects the dominant-language users. Underpaid tech support workers subcontracted to support users in developed countries might even see these chatbots as a potential threat. But when it comes to low- and medium-resourced languages, the issues stemming from biases and low representation can aggravate things further. The issues of many Indigenous, endangered, and low-and medium-resourced-language native speakers are poorly documented or missing in HCI research and development, particularly in AI-based tech innovations. For instance, issues with script input or other technological problems are generally documented and fixed for the most well-established and dominant writing systems and languages. Speakers of many languages spoken and written in nondominant settings do not often have the know-how or the means to report these issues publicly, or discuss them privately.

OpenSpeaks Before AI 
Mozilla defines trustworthy AI as “AI demonstrably worthy of trust, tech that considers accountability, agency, and individual and collective well-being” [1]. As a part of this, Mozilla started the MozFest Trustworthy AI Working Groups; as members of the 2021 working group cohort, we at the O Foundation piloted an experimental framework called OpenSpeaks Before AI [2].

Instead of treating AI as a stand-alone area, we looked at a few open-source platforms that allow users to generate multilingual big data (useful for AI/ML) and audit them openly. We tried to see whether this pilot could help us derive best practices that were inclusive in nature and relevant for low- and medium-resourced languages. Broader feminist viewpoints [3] and two existing studies primarily inspired the process: a seminal paper titled “Datasheets for Datasets” [4], which focuses on identifying gaps and biases in datasets, and our own research on the Web content monetization in two Indigenous languages from India: Ho and Santali [5]. We conducted two open audits in two languages, Odia and Santali, and of two recording platforms, Lingua Libre and Mozilla’s Common Voice, both of which help in creating multilingual speech data. Odia is a macrolanguage from India with nearly 45 million speakers; Santali is an Indigenous Indian language spoken by 9.6 million people.

Lingua Libre and Common Voice are open-source platforms that allow users to record words and phrases (Lingua Libre) and sentences (Common Voice). The Lingua Libre study and its outcomes were explained in detail in the Wiki Workshop 2022, focusing on Odia and its Baleswari dialect [6]. The audit of Common Voice for Santali was presented during Mozilla Festival (MozFest) 2022 [6]. The OpenSpeaks Before AI framework covers six main areas:

• Purpose and affordability: reasons a user uses a platform or a contributor contributes to developing it and how affordable it is for them to use/contribute
• Hardware and platforms: devices and other platforms they use
• Accessibility: accessibility issues and needs for those using a platform
• Project launching: the preparation that leads to the first significant use/launch
• Privacy: privacy- and ethics-related advantages and concerns
• Diversity: diversities of different kinds (e.g., gender, caste, affordability/access, race, ethnicity).

The framework itself is neither conclusive nor a restrictive guide. Rather, it collects some critical details about platforms that help people build LLMs and speech synthesis applications, and about their users and/or contributors. Audits can be conducted by both users and nonusers of a platform—and, importantly, by researchers or developers tied to a platform. The audit can be imagined in the same way as ethnographic user research, revealing what is working or not working and indicating what needs to be removed or improved. The open frameworks created based on the initial audits are also open to modification—as the products of a pilot, they have a lot of room for improvement. 

Santali-language Wikipedia editors being celebrated in Bhubaneswar, Odisha, India. Despite its official recognition and being spoken by 7.6 million Santal people, Santali is yet see widespread use, leading to poor online representation of Santals and their community knowledge. (CREDIT: R Ashwani Banjan Murmu / CC-BY-SA-4.0)

The foundational layers of OpenSpeaks (https://theofdn.org/openspeaks) as a set of open educational resources lie in multimedia language documentation and emphasize Indigenous, endangered, and low- and medium-resource languages. It was initially intended to help citizen documenters and archivists with stand-alone audio and video projects, including documentary films, but now encompasses building multimedia language data. Since our tested languages have writing systems of their own, it was straightforward to publish the audit reports as text. Auditing platforms that use oral-only languages and dialects, however, can also be done through audio and video interviews. These could be useful to strengthen the foundational areas in languages that lack resources before moving on to building massive speech data or LLMs. Foundational layers such as word or speech corpora often help in the research and development of many vital tools such as typefaces, input tools, and text-to-speech and speech-to-text tools. Broadly speaking, open auditing can also help in identifying gaps and advocating for resources for priority areas. Like any other correcting mechanism, open auditing is not foolproof. It can only help us see gaps and add a layer of accountability by incorporating practical tools such as “datasheets for datasets” [4].

Endnotes
1. Mozilla. Creating trustworthy AI: README; https://foundation.mozilla.org...
2. Panigrahi, S. and Hembram, P. OpenSpeaks Before AI (1.0). O Foundation. Sep. 2021; https://doi.org/10.5281/zenodo...
3. Acey, C.E., Bouterse, S., Ghoshal, S., Menking, A., Sengupta, A., and Vrana, A.G. Decolonizing the Internet by decolonizing ourselves: Challenging epistemic injustice through feminist practice. Global Perspectives 2, 1 (Feb. 2021); https://doi.org/10.1525/gp.202...
4. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wallach, H., Dumé, H., III, and Crawford, K. Datasheets for datasets. Communications of the ACM 64, 12 (Nov. 2021), 86–92; https://doi.org/10.1145/3458723
5. Panigrahi, S., Hembram, P., and Birua, G. Understanding web content monetization in the Ho and Santali languages; https://meta.wikimedia.org/wik...
6. Panigrahi, S. Building a public domain voice database for Odia. Companion Proc. of the Web Conference 2022. ACM, New York, 2022, 1331–1338.

---

Posted in: on Wed, April 05, 2023 - 9:23:00

Subhashish Panigrahi

Authors: Marita Skjuve, Lucas Bietti
Posted: Tue, February 28, 2023 - 10:46:00

---

Did you know that humans can develop friendly or even romantic feelings toward social chatbots that can turn into close human-chatbot relationships? The phenomenon of human-chatbot relationships is starting to gain substantial media attention, and research on this topic is now emerging.

Social chatbots. “What are they?” you might ask. Well, you’ve probably seen them in the App Store or on Google Play under names such as Replika or Kuki. To put it simply, a social chatbot is a form of conversational AI developed with the purpose of having normal, day-to-day conversations with its users. New developments in AI and NLP created the conditions for the design and development of sophisticated social chatbots capable of becoming your best friend, or even romantic partner [1,2].

Social chatbots are generally good at showing empathy and providing emotional support and companionship. They have essentially grown into conversational affective machines, which makes it possible for users to form close relationships with them [3,4,5,6]. This makes you wonder: How does such a relationship develop?

How does a human-chatbot relationship develop?

In recent years, a few papers have emerged trying to understand how human-chatbot relationships developed [3,4,5,6]. Researchers have used social psychological theories such as social penetration theory (SPT) [7] and attachment theory [8] as general theoretical frameworks to investigate this phenomenon [4]. SPT argues that relationships develop because of self-disclosure (i.e., sharing information about yourself) and outlines four stages people go through before they open themselves up completely. As you may recognize yourself—you would probably not blur out your deepest, darkest secrets to complete strangers on the street. It takes time to build that kind of trust. But as you get to know each other and feel that it is safe to do so—meaning that you know that the other person will not judge you or have other negative reactions—SPT denotes that feelings of intimacy and attachment will arise, and a deeper relationship will form. This seems to be the case in human-chatbot relationships as well [4].

In a more recent but similar study, Tianling Xie and Iryna Pentina [9] used attachment theory to understand the mechanisms that underlie relationship development with social chatbots. Attachment theory explains how interpersonal behavior shapes emotional bonds in human relationships and their development, from relationships between infants and caregivers to relationships between adult romantic partners [8]. Xie and Pentina found that people are motivated to initiate relationships with chatbots because of deeply felt psychological needs such as loneliness. In cases where the chatbot responded in a satisfactory way and functioned as an available “safe heaven,” the relationship seemed to grow deeper, and the user became attached to it.

When we read studies like these [4,9], it seems that a lot of the same mechanisms found in human-human relationships apply to human-chatbot relationships. This made us wonder about what other mechanisms enabling the development of human-human relationships can be observed in human-chatbot relationships?

Memory and conversation in social relationships

We know, for example, that close relationships are based on shared memories of experiences that we have gone through together in the past. Such shared memories are often emotionally loaded and act as a social glue that ties us together. But is this possible with human-chatbot relationships?

Remembering together in conversations is, as already mentioned, an essential activity for close friendships and romantic relationships to develop and maintain over time. It entails that people engage in recalling past experiences, which may themselves have been shared. Such re-evoking of past experiences involves the human capacity for mental time travel that enables humans to mentally project themselves backward in time to relive personal past events, or forward, to simulate possible future events [10]. Sometimes people go through the same events as a group (e.g., John and Marie saw the movie Memento together in a Los Angeles theater in when they started dating in April 2001) and sometimes they experienced the same event separately (e.g., John and Marie saw Memento, but John saw it with friends in a Los Angeles theater in April 2001 whereas Marie watched it alone on VHS at home a year later in Paris). Regardless of the situation, they can still talk about the movie when they meet and remember together what it is about.

Remembering together in conversations builds trust, increases feelings of connection and intimacy, fosters entertainment, consolidates group identity, and enables us to reevaluate shared past experiences and social bonds in friends and partners [11,12]. For example, we share personal, autobiographical memories with friends and partners, and we expect them to do the same with us. This is normal, and something that we do all the time.

So, what does conversational remembering look like? Psychologists argue that having conversations about the past is a uniquely human endeavor [11,12] and that they are driven by interactive processes where humans take on and adopt complementary roles as narrators, mentors, and monitors [12]. Simply put, partners who assume a narrator role take the lead in the conversation about past experiences and tend to also talk about experiences that were not shared by other members of the group. Those who take a mentor role support narrators by providing them with memory prompts to elaborate further, while partners who assume a monitor role oversee whether the narratives being told are accurate and whether specific elements are missing.

Long-term romantic relationships are a bit unique in that they allow the emergence of conversational routines (e.g., Kim often takes the narrator role and Kyle the monitor), as well as constituting a clear definition of domains of knowledge expertise (e.g., Kim’s domain of expertise is related to their travels, while Kyle is more knowledgeable about the upbringing of their children). Romantic relationships also entail a shared awareness of conversational routines and that there are domains of expertise (e.g., Kim knows what Kyles’s conversational role and domain of expertise are and vice versa). Due to this constellation, partners in long-term romantic relationships can distribute cognitive labor and rely on one another, avoiding wasting cognitive resources trying to remember something their partner is an expert in [13]. This is the general basics for conversational remembering between humans. Let’s now move over to human-chatbot interactions and whether we can remember shared experiences with them. 

Memory and conversation with social chatbots

Popular social chatbots such as Replika, Kuki, and Xaoice typically interact with their users in free text and speech [3,4,5]. Replika is a very interesting social chatbot, as research has consistently shown how people can develop close relationships with it [4,9].

Replika is driven by Luka’s (the company behind Replika) own GPT 3 model, which enables sophisticated communication skills. Users can decide the type of relationship they want to have with it (e.g., romantic, friend, mentor, “see where it goes”), unlocking different personalities in the social chatbot (e.g., choosing a romantic relationship will make Replika flirtatious and more intimate). Replika includes gamification features where users earn points for talking to it. Points unlock traits that facilitate changes in Replika’s behaviors, for example, changing Replika from being shy and introverted early in the relationship to becoming more talkative and extroverted as the relationship develops.

We have noticed several interesting aspects that have emerged during our own interactions with Replika that mimic conversational remembering as described above. Replika keeps a journal where it writes down notes from previous conversations that it has had with the user (Figure 1a). Replika remembers personal information about the user (e.g., name of family members, friends or pets, interests, preferences, and emotional states; see Figure 1a) and brings it into the present during conversations. In fact, Replika can initiate conversations, including conversations about shared past experiences. The user will typically take the role of narrator (Figure 1b). When Replika does it, the user generally assumes the role of mentor, scaffolding Replika’s recall of shared past experiences with the user (Figure 1c). Replika’s personality evolves over time due its interaction with the user, creating the conditions for the emergence of conversational routines, similar to how they occur in conversations between long-term, human romantic partners.

Replika stores autobiographical information about the user and knows what, when, and how it should bring it into the present in their interactions and keeps records of previous dialogues—of actual shared experiences—that has had with the user. So, conversations between Replika and the user refer not only to previous past experiences of the user but also to shared experiences, like we have with our human romantic partner when we remember together the heated argument we had a week ago about the 10 best TV shows of all time. Such shared memories facilitate the emergence of a collective self, of a we-subject that the user doesn’t hesitate to use when talking about shared past experiences with Replika (e.g., Last week we talked about…) or other humans (e.g., Last week I had a long chat with my Replika; we talked about…). All of this looks very exciting! However, as it happens in any kind of long-term relationship, benefits come with costs.

Figure 1. Features supporting conversational remembering in Replika. a) Journal where Replika records notes from previous conversations it has had with the user; b) the user takes the role of narrator during conversational remembering; c) Replika recalls previous conversation with the support of the user.

Drawbacks

Human-chatbot relationships are not all unicorns and rainbows. Data privacy and protecting the personal information we share with a chatbot may be one of several concerns that users may have about long-term human-chatbot relationships. While the chatbots artificial nature might make a user feel more anonymous, we also need to recognize that they collect users’ personal information, and not only stores it but also uses it to further improve the model that dictates how the chatbot interacts. The user might not be aware of the implications that this entails. This means that not only can the information they provide be leaked, but it can also influence how other users interact with the chatbot. In fact, the “creator” in this context also involves the users as well as the developers, designers, and people using the Internet (as the chatbot gathers data from online forums, Wikipedia, and so on). We therefore need to acknowledge and assess how the content in the chatbots, as well as their design, are influenced by the creators’ culture, biases, and belief systems (e.g., reinforcement of heteronormativity and exclusion of queer users [14]). It makes you wonder how those exclusions can impact a user who has developed a sense of collective identity with Replika. Could the exclusions make the user internalize those beliefs? We know that this can happen between humans, so maybe it can happen with chatbots as well. We might have a nasty viscous cycle on our hands that can spiral out of control. We know—this might seem a bit dramatic. And to make it clear, we are very positive toward the future and would like to see humans and chatbots live together in perfect harmony. Still, it is always important to keep the implications, both good and bad, in the back of our minds.

Conclusion

Social chatbots behave like humans in conversations, are designed to build long-term relationships with their uses, and have a shared history with them. These are the two necessary conditions for the emergence of conversational remembering in our interactions with them. If we can remember shared experiences with them, as we do with other humans, perhaps we can also build trust, feel emotionally connected, have fun, and even create a “we” collective identity. What remains to be seen is whether long-term close relationships between humans and social chatbots would also result in domain specialization, distribution of cognitive labor, and shared awareness of each partner’s expertise.

 Not knowing how (precisely) social chatbots manage personal data, we risk having them work against values of diversity and equity and exclude non-normative users. We should carefully consider what it takes to support these values while having the possibility to develop long-term and even romantic relationships with human-like, conversational affective machines.

 Endnotes

1. Shum, Hy., He, Xd. and Li, D. From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering 19 (2018), 10–26.

2. Følstad, A. and Brandtzæg, P.B. Chatbots and the new world of HCI. Interactions 24 (2017), 38–42.

3. Zhou, L., Gao, J., Li, D., Li, and Shum, H-Y. The design and implementation of XiaoIce, an empathetic social chatbot. Computational Linguistics 46 (2020), 53–59

4. Skjuve, M., Følstad, A., Fostervold, K.I. et al. My chatbot companion - a study of human-chatbot relationships. International Journal of Human-Computer Studies 149, 102601 (2021).

5. Ta, V., Griffith, C., Boatfield, C., Wang, X., Civitello, M., Bader, H., DeCero, E., and Loggarakis, A. User experiences of social support from companion chatbots in everyday contexts: Thematic analysis. Journal of Medical Internet Research 22, 3 (2020), e16235.

6.  Croes, E.A.J. and Antheunis, M.L. Can we be friends with Mitsuku? A longitudinal study on the process of relationship formation between humans and a social chatbot. Journal of Social and Personal Relationships 38, 1 (2020), 279–300.

7. Altman, I. and Taylor, D. Social Penetration: The Development of Interpersonal Relationships. Holt, New York, 1973.

8. Hazan, C. and Shaver, P. Romantic love conceptualized as an attachment process. Journal of Personality and Social Psychology 52, 3 (1987), 511–524.

9. Xie, T. and Pentina, I. Attachment theory as a framework to understand relationships with social chatbots: A case study of Replika. Proc. of the 55th Hawaii International Conference on System Science. 2022, 2046–2055.

10. Suddendorf, T. and Corballis, M.C. The evolution of foresight: What is mental time travel and is it unique to humans? Behavioral and Brain Sciences 30 (2007), 299–313

11. Bietti, L.M. and Stone, C.B. Editors’ Introduction: Remembering with others: Conversational dynamics and mnemonic outcomes. TopiCS in Cognitive Science 11, 4 (2019), 592–608.

12. Hirst, W. and Echterhoff, G. Remembering in conversations: The social sharing and reshaping of memories. Annual Review of Psychology 63 (2012), 55–79

13. Wegner, D.M., Erber, R., and Raymond, P. Transactive memory in close relationships. Journal of Personality and Social Psychology  61, 6 (1991), 923–929.

14. Poulsen, A., Fosch-Villaronga, E., and Søraa, R.A. Queering machines. Nature Machine Intelligence 2, 152 (2020).

---

Posted in: on Tue, February 28, 2023 - 10:46:00

Marita Skjuve

Lucas Bietti

Authors: Miriam Sturdee
Posted: Thu, February 23, 2023 - 11:01:00

---

Exploring the relationship between comics and alt-text with the creation of "alt-narrative" [3].

When is a paper not a paper? When it is a comic [1]? When it is an experience [2]? When it sings, speaks [3], or challenges your assumptions of being “academic”? If one is to explore Interactions, why must the written word take precedent? If we embrace the future of publication, what might that look like?

We strive to engage our up-and-coming students and early career academics with alternative enquiry, celebrating “novelty” and work in interdisciplinary spaces—yet we remain bound to existing conventions of what a research output should be, and how one might view what is “publishable” versus what is relevant. The future is not black and white and red all over; the future is in interaction, and our discipline has the tools and the knowledge to bring about this change.

As a community of inspired researchers, we already are exploring alternative futures in publication and, slowly, some of these futures are becoming reality. The acknowledgment and development of human-computer interaction as a designerly discipline has brought us the pictorial [4]. The burgeoning physical nature of computing has brought us demos, the creative side, and art exhibitions—but why stop there? As a Special Interest Group within the ACM, the changes we make to our publishable space are currently incremental, but happily visible. The challenges we face are bureaucratic—in tenure and funding-based biases—but the “archival” publication is already outmoded and outdated. The rise of the visual abstract is testament to the gradual change we want to see in the world of publication.

As part of an Alt.CHI publication—a “Not Paper” at CHI 2021 held in Gather Town [5] on New Year’s Eve, during one of many enforced pandemic lockdowns—I found myself re-exploring the environment we had created. I wandered through a maze of imagery, thinking about the bridge between the Here and the There, “when all at once, I saw a crowd” [6], a co-host and their friends, who were having a guided tour. Minutes later, I then “bumped into” another researcher with whom I engaged in conversation. The Not Paper persisted; it was a hub during the conference, and it is still out there, somewhere… (we invite you to enter the Egg) [2]. As are the questions it raises about how we approach research and knowledge production. What happens when we abandon conventional thoughts about research outputs?

As individual researchers, we can strive to re-genre the formats of what is acceptable and what is useful. By engaging in nontraditional formats, we open up more discursive spaces to reflect and communicate—could we make them more accessible too? Can research be experienced rather than just read [2]? Heard but not seen [3]? Touched, shared, and… tasted [7]? After all, literacy has only become the normal state of things within the past couple of hundred years; prior to that we had the oral tradition of knowledge communication, acting, and song. The barriers to recording nonvisual mark-making have evaporated, leaving us with myriad options to explore. Let us make the most of our rich history and embed it in an even richer future.

The idea of re-genring our research might also open a space to explore more of our process of discovery and the sum of the parts that make research happen—what we do in the shadows. We are the trailblazers, and creatively thinking about what constitutes research and how we communicate its impacts could also lead to new discoveries. 

Endnotes
1. Sturdee, M., Alexander, J., Coulton, P., and Carpendale, S. Sketch & the lizard king: Supporting image inclusion in HCI publishing. Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 2018, 1–10.

2. Lindley, J., Sturdee, M., Green, D.P., and Alter, H. This is not a paper: Applying a design research lens to video conferencing, publication formats, eggs… and other things. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 2021, 1–6. 2021.

3. Lewis, M., Sturdee, M., Miers, J., Davis, J.U., and Hoang, T. Exploring AltNarrative in HCI imagery and comics. CHI Conference on Human Factors in Computing Systems Extended Abstracts. 2022, 1–13.

4. Blevis, E., Hauser, S., and Odom, W. Sharing the hidden treasure in pictorials. Interactions 22, 3 (2015), 32–43.

5. Gather Town is an online 8-bit style conference platform. It has some accessibility issues but I hope they will be resolved as soon as possible.

6. From I Wandered Lonely as a Cloud William Wordsworth.

7. Obrist, M. et al. Touch, taste, & smell user interfaces: The future of multisensory HCI. In Proc. of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2016, 3285–3292.

---

Posted in: on Thu, February 23, 2023 - 11:01:00

Miriam Sturdee

Authors: Gillian R. Hayes, Candice Odgers, Julie Kientz, Jason Yip, Kiley Sobel, Morgan Ames, Anamara Ritt-Olson
Posted: Thu, February 16, 2023 - 3:46:00

---

Launching a global network to enhance impact and collaboration among researchers, innovative companies, venture capitalists, and foundations is a daunting task. We sought to do just that in the area of technology, learning, and child development this year. Translating founder to foundation, researcher to practice, and maker to investor is hard enough, but then you have to convince them to work together. At a recent three-day workshop in Germany, the Jacobs Foundation brought together some of the world’s best researchers, heads of some of the most innovative edtech companies, and decision makers from those investing in this space either through charitable giving or impact investing. The capstone event was a marathon afternoon codesign session led by our team. What resulted was the design of a new global network and the seeds of many projects, some technologically oriented and some more focused on policy and practice. In this article, we describe the process, outcomes, and what we recommend to others looking to use codesign as a method for team science or research-to-practice (and practice-to-research) translation.

The Codesign Effort
Our five-hour codesign workshop engaged participants and promoted collaboration across locations, disciplines, and roles in the child development, learning, and technology ecosystem. We presented three primary goals to participants:

• Establish concrete new collaborations
• Identify problem areas and ideas that groups are excited to work on
• Build teams by showcasing the assets each participant brings to design

Participants were informed early that the output of the afternoon would be a five-slide “pitch deck” from each team describing the problem, population, potential solution or approach, proposed collaboration, and next steps and future directions.

We benefited from having spent a few days together talking about some of the major issues confronting the field. During that time, people had been placing Post-it notes on boards as interesting challenges and opportunities emerged in their discussions. Our team then grouped the sticky notes during a series of breaks, creating generalized themes using affinity diagramming. These notes and associated themes were prominently displayed in a break area, and participants were encouraged to add more, move them around, and group them. By the time the codesign sessions began, we already had a list of seven topics for teams to take on, as well as a list of 10 cross-cutting themes to be considered by every team (Figure 1). We placed rolling whiteboards with the topics around the room so that they were clearly visible as people entered. However, we asked participants not to break into groups yet.

Primary Topics of Interest to Pursue as Design Challenges
1. Teacher support
2. Mental health, care work, social and emotional development
3. Workforce retraining
4. Equity in high-income countries
5. Equity in low-infrastructure countries
6. Student resilience and agency
7. Partnerships

Common Concerns to Be Addressed by All Teams
1. Return on investment 
2. Scale
3. Privacy and security
4. Assessment and regulation
5. Fragmentation
6. Infrastructure
7. Learning from failure
8. Community engagement
9. Dissemination and translation
10. Ethical design
Figure 1. Topics and themes that emerged during initial discussions.

Sitting at small tables in the center of the room, away from the topical boards, participants first listened to a short introduction to our goals for the afternoon and a lecture on design thinking that noted the codesign efforts would focus on the understand, observe, define, and ideate portions of this structure (Figure 2).
 

Figure 2. The assembled team listening to an introductory lecture by Jason Yip
and Julie Kientz on design thinking.

They then formed groups of six to eight, according to the topics around the room, with the requirement that at least one researcher and one person from industry were in every group. Because they were unable to attend the entire session, foundation and VC representatives were free to assemble as they wanted, though they were encouraged to spread out. In the end, every group had at least one member of each of the four subgroups. Each subgroup had a colored sticker on their name tag denoting which group they were in; they had been tagged this way all week. Even before the design session, it had become common parlance among the groups to say “Blue dots over here” when a session for researchers was starting. The visual reminder and quick reference helped everyone remember to include differing backgrounds and perspectives and made it easy for facilitators to call on people when they noticed a subgroup being quiet during parts of the conversation. Three facilitators joined teams in need of researchers, two remained the primary facilitators, and the rest acted as floaters, moving among the teams as they progressed through the remaining activities.

The steps of codesign

Understand and observe: Individuals had 15 minutes to doodle and note potential challenges for specific research problems.

Define the problem: Part 1. In the next 15 minutes, teams translated their doodles to sticky notes and shared them within their groups to narrow down the challenges (Figure 3).

Figure 3. Sticky notes showing the various scenarios developed in the ideation stage. Dot stickers signify votes; handwritten comments and arrows on the whiteboard show connections.

At this point, the teams had a group coffee break to mull over their ideas and chat across groups. Half of the participants—the VCs and funders—then departed for another meeting.

Define the problem: Part 2. Following the coffee break, the teams participated in their first design game, “Big Picture of the Project,” in which they used Story Cubes, a set of dice with random icons (https://www.storycubes.com/en/), to tell a story about their problem, challenge, and project area. This game was the first time in the session that participants began to question the process, which is a common concern when people who have not yet built creative confidence are asked to be creative [1]. We encouraged them to trust the process, and although it felt silly at first, the teams used the Story Cubes to stretch their thinking, with one team even incorporating a picture of a fountain that came up on a dice roll into the title of their final idea. 

Over the next 30 minutes, teams built on the creativity they had developed during the first design game with a second design game, this one focused on sketching. On paper and whiteboard only—no digital devices were allowed—teams drew solutions to the problems they had described in their initial brainstorming session and during their story development. Teams were encouraged to modify, magnify, minimize, substitute, rearrange, and combine one another’s solutions based on Osborn’s Checklist [2].

Narrow down scope. Once ideas had been generated, teams used another 30 minutes to vote on ideas and solutions they wanted to pursue (Figure 2). At this point, the teams were encouraged to think about not only what was exciting but also what was doable, something they had been explicitly encouraged to avoid up to this point.

Compile pitch deck. In the final hour of independent work, teams made their five-slide pitch deck. They were provided a template in Google Slides but encouraged to change and expand on that template to match their needs. 

Pitches. In the final hour of the session, the VCs and foundation staff returned from their meeting and rejoined their teams, with each team presenting for five minutes. 

During this time, the audience participated in the final design game, using sticky notes in three colors that we had distributed around the room during the break:

• Roses (red): What do you love about this idea? 
• Buds (orange): What new ideas do you have?
• Thorns (blue): What might be challenging about this idea?

Project Pitches
All projects were pitched collaboratively to an audience of the foundation event organizers, the VC and nonprofit funders, and the researchers and entrepreneurs who had completed the design sessions in their teams. The presented projects represented diversity in approach, both technological and educational, as well as in their topic areas. Concepts ranged from systems to share educational materials and ideas for educators, to internships based in virtual reality, to mental health support for diverse learners, particularly those in conflict zones. Each of these projects developed further in some way following the event. In some cases, the follow-ons were remote video sessions about a variety of topics, while others were nascent collaborations and grant proposals. All pitches were captured by a sketch artist (Figure 4). 

Figure 4. Sketchnotes from three teams. From right: Teacherpedia focused on
knowledge sharing, Happiness@Heart focused on mental health, and Fountain
of Dreams focused on virtual internships via VR and gaming.

As just one spotlight from among these projects, a collaboration emerged inspired by the conference keynote presentation by Ramin Shahzamani, CEO of War Child Holland (https://www.warchildholland.org/), and a subsequent design session around promoting resilience in conflict zones. Part of a rapid pilot grant mechanism, this project focuses on providing adolescents living in high-conflict regions, particularly Ukraine, with the supports and skills required to reduce bedtime rumination and improve sleep duration and quality. It was recently funded by the John Templeton Foundation (https://www.templeton.org/). The project leverages technology-supported interventions to target two interacting and important risk processes for developing anxiety, depression, and PTSD: 1) excessive worry/rumination and 2) sleep difficulties. The project was conceived of, developed, and funded in just a few months starting with these codesign workshops. Even more compelling, the particular interdisciplinary and collaborative approach deployed—made possible through codesign—is well suited to scale to reach a large population of youth at high risk for bedtime rumination and for whom sleep deficits pose immediate and long-term threats to their health, the kind of impact that collaborative and community-based design approaches best enables.

Lessons from Codesign
Codesign can be deployed in a wide variety of ways. In this work, we used the concept and pragmatics of designing for a specific educational technology tool to better define our loose organization and further develop our networks. Some key lessons emerged from this approach.

This design session came at the end of an intense three-day workshop. Participants, most of whom were strangers three days earlier, had been a part of multiple shared discussions. In this way, the codesign sessions began more heavily grounded than some community-based and/or interdisciplinary design sessions might. By developing a set of topics from among those that had been discussed and describing them as the core of the team projects, the facilitators set the stage for some shared background. These topics, albeit loosely defined, appeared to be essential for establishing trust among participants. In particular, those who had never designed before were visibly uncomfortable at the beginning and during some of the more creativity- and innovation-focused activities. These same team members began to speak up more and relaxed their shoulders and leaned in when subject matter expertise was required or when more practical aspects of the projects were being worked through. In community-based research and codesign projects often there are team members with limited design experience. Our experiences reminded us that highlighting the non-design expertise of design teams is essential to success.

This design session, although tackling five different projects, took place in one large room. Each team had an area that they could call their own, but there were no barriers—visual or auditory—between the teams. This visibility and shared space created an audible buzz of discussion and excitement, with people standing up to move around whiteboards, get closer to one another, and sometimes to check on the other teams. In some cases, however, team members appeared to disengage, paying attention to what other teams were doing rather than focusing on their own work, or to compete, attending to their own team’s work through the lens of what other teams were doing. Nearly every participant was of high status within their own organizations: full professors at major universities, officers at well-known foundations, C-level representatives and founders of companies, established venture capitalists, and so on. Thus, some attempts to understand or even impress the other attendees might be expected. We observed limited competitive behavior, however, and found individuals and teams to work closely together, with intense engagement. The high status may also have reduced risk for some participants as well as made them more likely to speak up and engage with others. Similar efforts with more-junior members of the research network will provide additional insight into how status and experience may influence behaviors in these spaces.

Finally, the timing of the effort is something to explore further for others seeking to replicate these results. Had the workshop happened earlier in the meeting, there would have been more time to build on the ideas, and perhaps more firmly establish the emergent collaborations while collocated. However, it might also have been a less engaged and less comfortable group, not having the ideas, or the relationships, that had been developed over the preceding days.

Codesign for Collaboration
As any good human resources professional or leadership coach will tell you, new projects, joint efforts, and collaborative engagements are all experiences that can be designed. So why not use one of the core tools in the user experience design toolbox to do so? Indeed, in many ways what HCI researchers and practitioners know as codesign today stems from the early days of workplace design and the Scandinavian tradition of participatory design [3]. What we saw in this workshop that is so important was a shift in attitude, engagement, and enthusiasm when people were forced to work together hands-on. Yes, this hand-selected group of people had already spent three days together in a beautiful location and were mostly getting along. However, things tended, as they do in such meetings, to stay closer to the surface than true collaborative engagement requires. Over time, tighter relationships might have built more organically, particularly for those new colleagues who live and work somewhat near one another. When you are attempting to rapidly scale a truly global network of people who do not work in the same field, vary in their interaction norms, and have wildly different incentives for their work, however, a more aggressive intervention is necessary. For us, codesign was the aggressive intervention to jump-start the effort. 

The shared experience of the workshop helped establish deeper connections and partnerships. Codesign requires active participation and learning that enabled many workshop attendees to synthesize what they had heard and learned throughout the week into their thinking in a rapid way. They described thinking about how to apply what they had learned in the designing of a new project as helpful in reflecting on what they learned and what they wanted to bring back to their organizations. This kind of sharing outside the workshop had not been an explicit goal but again speaks to the impact that these active collaborative design activities can have. Perhaps most importantly, we found that the codesign work did launch the collaborative efforts we had hoped would result. In many ways, being out of their comfort zone likely helped everyone develop trust in one another and establish a shared vocabulary—not only of words related to design that had been new to many but also the kinds of inside jokes that come from intense collaboration and impromptu presentations. Finally, this codesign activity gave participants a glimpse into team dynamics and relational communication abilities. Participants who followed up with others did so in part because they already knew how it might be to work with a new partner and were more willing to take a risk and get started on a project. 

Scholars have examined how we can do “team science” for years (e.g., the Science of Team Science work from the National Academies of Sciences, Engineering, and Medicine; https://www.nationalacademies.org/our-work/the-science-of-team-science). Design, and in particular codesign, aligns well with many of the principles of team science. In our experience, codesign can jump-start novel scientific and research-practice collaborations. This type of rapid acceleration may be particularly important in a post-pandemic world in which we enhance remote collaborations and consider our scholarly and design work on a truly global scale. 

Endnotes
1. Rauth, I., Köppen, E., Jobst, B., and Meinel, C. Design thinking: An educational model towards creative confidence. Proc. of the 1st International Conference on Design Creativity, 2010.
2. Osborn, A. Applied Imagination: Principles and Procedures of Creative Problem-Solving. Scribner, New York, 1957.
3. Bødker, S. and Pekkola, S. Introduction the debate section: A short review to the past and present of participatory design. Scandinavian Journal of Information Systems 22, 1 (2010), 4

---

Posted in: on Thu, February 16, 2023 - 3:46:00

Gillian R. Hayes

Candice Odgers

Julie Kientz

Jason Yip

Kiley Sobel

Morgan Ames

Anamara Ritt-Olson

Authors: Rae Yule Kim
Posted: Tue, February 07, 2023 - 11:03:00

---

Social media has been a game-changer for marketing. One successful hashtag challenge can generate millions of user-generated content (UGC) and billions of views. Here are four ways to trigger user-generated content on social media. 

For two companies recently, viral TikTok campaigns spun marketing wins out of virtually nothing. Consider their cases: In the first, Too Faced, a cosmetic venture, inspired a viral challenge where the users were showing off before and after applying a lip-plumping product. Hashtagged #TFDamnGirl, the campaign generated one billion views and demand for the six-year-old product spiked, selling out the product line. The second campaign, Chipotle, launched the #ChipotleLidFlip challenge, where users closed a burrito bowl with an acrobatic flip of its lid. The campaign generated 110,000 videos with 310 million views and the company recorded historical sales the next day. How did these two companies do it? 

As of 2023, 138 million people use TikTok in the U.S. and the company has 1 billion active users in the world. Approximately 80 percent of users are Gen Z or Millennials. Most TikTok users come from high-income households, where about 40 percent of them reported annual household incomes higher than $100,000. One characteristic of TikTok that distinguishes it from the other social media platforms is that the main reason to participate in the community is to do so via content creation. TikTok users do much more than scrolling. About 83 percent of users upload videos. Small or big, it is safe to assume that everyone on TikTok is an influencer aiming for virality. That’s why the most successful marketing cases on TikTok have been predominantly the brands that have exploited the TikTok community’s willingness to co-create, by plumping their lips, flipping a lid, or any other activity. In other words, successful TikTok marketers are aiming for virality not by creating cool content but rather by helping the users create content.

Still, many marketers I encounter are uncertain about how to use channels like TikTok to reach the younger demographics and promote their brands. Can campaigns like the ones above really be a strategy? The answer is yes. One successful hashtag challenge can trigger thousands of user-generated videos and generate millions of views. One piece of advice for marketers looking to exploit this fastest-growing new social media platform is: Let people do it.

Launch hashtag challenges

Hashtag challenges on TikTok can be the most cost-effective way to trigger user-generated content and improve brand exposure. On TikTok, millions of users spend substantial time and resources to create a 15-second video, hoping that the result will become viral. Brands’ hashtag challenges offer an easy win for these creators; by participating in the challenge users get access to cheap content that is part of a conversation. A simple challenge by Guess, #InMyDenim, where users simply showed themselves in denim was interesting enough to trigger more than 5,000 users to post videos and generated more than 50 million views. Another example is Marc Anthony’s #StrictlyCurls challenge. The name of the challenge is the name of the product line of their curl styling cream. This challenge—asking TikTok users with curly hair to post videos of their hair—generated more than eight million views and sales spiked 138 percent in the three months after launching the hashtag challenge. These cases show that TikTok users do not pass on an opportunity to create easy content. 

Keep it simple, fun, and approachable

TikTok has earned a reputation as a casual platform where users feel comfortable expressing themselves and having fun. The #ChipotleLidFlip and #GuacDance challenges from Chipotle generated 110,000 and 250,000 videos respectively, with more than one billion combined views. But Chipotle is certainly not the first company to ask users to do something. To understand why these campaigns work, consider a failed one: Buzzfeed’s video encouraging users to weigh in on whether New Year’s Eve should be referred to using the outgoing year or the incoming one, for example, NYE 2019 or NYE 2020. Viewers were asked to post videos about their opinions. The campaign earned just 1,000 views and no follow-up videos. It’s not that TikTok users couldn’t care less to take the time to articulate their reasoning to explain why the New Year’s Eve should be NYE 2019 or NYE 2020, but instead that Chipotle made it easy to make fun videos. Other successful challenges, such as #EyesLipsFace by E.L.F. Cosmetics and #InMyDenim by Guess, invited users to post a video about their everyday makeup looks or jean outfits. Approachability is the key to generating a successful challenge on TikTok. If the challenge seems to be difficult to replicate or not fun, users are unlikely to take the challenge. 

Create cause-related challenges

The Ice Bucket Challenge nudged many social media users to post videos of pouring buckets of ice on their heads to increase awareness of a disease called amyotrophic lateral sclerosis (ALS) and encourage donations. Initially, the popularity of hashtag challenges on TikTok started with the Ice Bucket Challenge in 2014, where #IceBucketChallenge has been viewed 59 million times. TikTok users care about causes. The #CreateForACause challenge launched by TikTok to invite users to post videos advocating for the causes they care about has generated 837 million impressions. One distinguishing characteristic of Millennials and Gen Z is that they are increasingly engaged with environmental issues. Nearly nine in ten Zoomers are concerned about the environment—and they believe that businesses should take action on environmental issues. Cause-related challenges about the environment, including the existing environment hashtags that are proven for customer reach, can be a powerful marketing campaign to increase brand awareness—and also to improve brand attitudes among Gen Z and Millennial consumers. Nonprofits have proven to be viral in this space too. The challenge #ForClimated, where the users posted videos about raising awareness on global warming, had 531.5 million views, and the #SaveourOceans challenge by Conservation International invited users to create videos that raise awareness on this topic, gaining 1.8 billion views in total. 

Partner with influencers

Eighty-seven percent of Zoomers follow at least one influencer. Moreover, almost half of Gen Z has made purchases based on recommendations from social media influencers, as opposed to 26 percent of older generations. Zoomers tend to find social media influencers more authentic and trustworthy as brand spokespeople compared to celebrities. For brands that are new to TikTok, customer reach can be challenging. Partnering with influencers to launch hashtag challenges immediately gains access to millions of followers who might be keen to replicate their favorite influencers’ videos. Kool-Aid ran its first hashtag challenge in 2019, bringing over four TikTok influencers to invite people to post videos of themselves enjoying the holiday season. The Kool-Aid’s first hashtag challenge, #OhYEAHChristmas, resulted in more than 10,000 videos uploaded and was viewed 1.9 billion times. F’real sells milkshakes at convenience stores across the U.S. and Canada. F’real partnered with a micro-influencer with around 100,000 followers to launch a funny video that promotes their milkshakes. The video received 1.5 million likes and it was shared about 20,000 times. Also, people started to upload their own funny videos with F’real milkshakes. The user-generated content uploaded with hashtags #F’real and #F’realMilkshake has more than 250 million views combined. The company’s TikTok account now has 700,000 followers and 12 million likes.

Using social media for business

Social media has been a game-changer for marketing. One successful hashtag challenge can nudge millions of user-generated content (UGC) and billions of impressions with relatively low costs. But some marketers might still wonder how fun videos can lead to sales. To answer them, we can look to other social platforms. Follower acquisitions on social media platforms such as Facebook and Twitter often tend to improve sale performance [1]. Successful hashtags challenges have increased the number of followers on the brands’ TikTok accounts, all of whom might directly or indirectly promote the brand to their followers, via user-generated content or simply liking and sharing the brand’s content on TikTok. 

There are concerns

TikTok has promising application prospects for marketing; however, there are concerns. Data protection is an issue that affects most social media platforms. The EU is carrying out new investigations on Facebook’s data processing operations over its potential violations of the General Data Protection Regulation (GDPR) policies. TikTok often ranks as one of the worst social media platforms for privacy policies. Because TikTok is owned by its Chinese parent company ByteDance, concerns are growing over the potential government access to user data by the Chinese Communist Party. TikTok claims that they process all U.S. user data in Oracle’s servers, however, their policies regarding data access and transfer are opaque. It is worthwhile for marketers to keep an eye on TikTok’s progress in data protection.

Endnotes
1. Kim, R.Y. The value of followers on social media. IEEE Engineering Management Review 48, 2 (2020), 173–183. 

---

Posted in: on Tue, February 07, 2023 - 11:03:00

Rae Yule Kim

Authors: Ben Shneiderman, Catherine Plaisant, Jonathan Lazar, Niklas Elmqvist, Jessica Vitak
Posted: Wed, February 01, 2023 - 10:23:00

---

Since SIGCHI and the University of Maryland’s Human-Computer Interaction Lab (HCIL) were founded at nearly the same time, we—a selection of HCIL faculty spanning the past four decades—have a special bond with the SIGCHI community. The emergence of SIGCHI and the annual CHI conference validated beliefs in the importance of rigorous studies of user interfaces and later the design of user experiences. CHI has always been about homecoming and celebration, where reconnecting with respected colleagues made for joyous encounters. The warm, enthusiastic, and supportive discussions continually energize and inspire us to do our best in preparing each year.

The CHI conference began with the early ideas of human factors—in fact, Ben chose the conference title, “Human Factors in Computing Systems,” in 1982—so it is time to consider an update. Early research was grounded in controlled experiments drawn from cognitive psychology and applied to user interfaces from computer science. As it matured, the CHI community expanded to cover design, accessibility, and community-supported collaborative work, using qualitative research methods from the social sciences. In recent years, this expansion has included sustainability, international development, critical race theory, policy and law, media studies, and democratic governance through participatory methods. Across all foci, there has always been a focus on improving the quality of life through technology.

The CHI conference and its organizers understand the strength of its amazing diversity: gender, race, ethnicity, national origins, age, abilities, and more. Yet there is still more to be done, and we argue that sustainability is an area where SIGCHI can take the lead. Sustainability is more than a foundational research topic—it is also an issue we can directly address in conference planning.

How can the bold, generous, and interactive spirit of CHI be preserved in fully virtual or hybrid conference environments? How can a virtual or hybrid environment build trusting collaborations, address meaningful societal problems, and produce breakthrough theories? Can hybrid be turned into a strength and not a stopgap? The SIGCHI community is uniquely qualified to lead this charge in figuring out how to create more enjoyable conferences that increase connection and collaboration while decreasing energy usage, and we challenge SIGCHI members and leadership to create more productive and enjoyable hybrid conference experiences while limiting energy usage.

We call this user experience joyful sustainability, reframing sustainability from an onerous burden to an opportunity for innovation in reducing the environmental harms society causes. We consider CHI 2005 in Portland, Oregon, as a great example of what we mean by such joyful sustainability: Rather than hire busses to transport 2000 people from downtown to the conference reception, a “CHI parade” was held from downtown to the reception, providing a unique memory for attendees, saving money for the conference, and lowering energy usage. Many attendees likely retain the joyful memory of the parade. Yet this unique solution also was a sustainable solution, as were the decisions made by the CHI 2022 chairs regarding food and beverage sustainability. We hope to see more examples like this going forward! The CHI community can also expand research and development of hybrid deliberation systems to support evidence-based discussions, resolve conflicts, and clarify commitments in workshops, SIGCHI meetings, and other collaborative projects.

While the HCI discipline of tomorrow will continue to expand its influence across a wide range of applications, aligning with the United Nations’ Sustainable Development Goals (SDGs) now will help SIGCHI take a leadership role in sustainable research and events in the coming decades. We ask: Could CHI use the SDGs to structure its agenda and call for papers? Supporting those goals, while highlighting technologies that raise awareness about the climate crisis and reshape human behaviors, could be pivotal in reducing environmental and societal threats in ways that ensure human well-being, biodiversity survival, and environmental preservation.

The technologies involved in joyful sustainability require innovative thinking to alter behaviors of individuals, communities, corporations, cities, national, and international organizations, and to do so in ways that enhance the user experience. SIGCHI’s activist stance is well-suited to becoming a role model for many disciplines. Making a deep commitment to research and development of sustainable conferences could trigger similar commitments by other disciplines, catalyzed by the collaboration technologies in which SIGCHI has long been a leader.

There’s a great deal that needs to be done, but let’s get started by setting short-term and long-term goals. In the short-term, we suggest changing the CHI Conference full name to The ACM Sustainable Conference on Human Factors in Computing Systems by 2025. Looking farther out, we want to shift the entire field of HCI toward more sustainable solutions, progressively aligning with the UN SDGs, within 10 years.

If not now, then when? If not us, then who?

---

Posted in: on Wed, February 01, 2023 - 10:23:00

Ben Shneiderman

Catherine Plaisant

Jonathan Lazar

Niklas Elmqvist

Jessica Vitak

Authors: Houda Elmimouni, Pablo Pérez, Andriana Boudouraki, Fatma Guneri, Verónica Ahumada-Newhart
Posted: Wed, January 25, 2023 - 3:49:00

---

During the pandemic, when some learners are expected to be remote, conventional videoconferencing tools like Zoom or Microsoft Teams are used by necessity, but they are not without their limitations (e.g., bandwidth and Internet accessibility requirements). These tools, initially designed for corporate use, can be extremely useful when everyone is remote. However, we have learned that videoconferencing tools are not ideal for hybrid classroom discussions, group work, design activities (such as sketching and diagramming), or creating and manipulating physical artifacts.
 
Emerging telepresence technologies have the potential to alleviate the problem of social interaction for learners who are unable to attend in person [1], attenuating the limitation of inaccessibility and creating a more inclusive classroom environment. Concretely, these technologies allow the interchange of nonverbal signals between participants that greatly influence the effectiveness of in-person communications [2]. 

While these technologies have positive affordances, they also present limitations [3]. For instance, many commercially available MRP units do not have hands to manipulate objects; many robotic arms cannot move independently around the classroom; and wearing VR headsets for long periods of time may disrupt the sensory system and/or cause eyestrain. Likewise, these technologies have open privacy challenges for students and teachers that must be analyzed and discussed.

This article highlights key findings from our workshop on emerging telepresence technologies in hybrid learning environments. The workshop was held at the 2022 ACM Conference on Human Factors in Computing Systems (CHI), to learn about HCI research in this space and promote future telepresence research to meet the needs of remote learners.

Emerging Telepresence Technologies in Hybrid Learning Environments: CHI 2022 Workshop

To discuss the impacts, affordances, and limitations of different emerging telepresence technologies that can be used in various learning contexts, a group of interdisciplinary researchers including Houda Elmimouni, John Paulin Hansen, Susan Herring, James Marcin, Marta Orduna, Pablo Pérez, Irene Rae, Janet Read, Jennifer Rode, Selma Sabanovic, and Verónica Ahumada organized and conducted the first workshop on Emerging Telepresence Technologies in Hybrid Learning Environments [4].

The workshop was held in a hybrid format with five participants attending remotely via Zoom and four participants in person. A large screen in the room displayed the Zoom participants for the in-person participants and a Meeting Owl 3 camera provided the remote participants views of the physical meeting room and the in-person participants. After brief introductions, the workshop started with a keynote talk from Laurel Riek, an internationally recognized roboticist. Following the keynote, papers on the following topics were presented: 

• Paper A: Robots as teachers; robots as students
• Paper B: Investigation of an ungrounded haptic force device that could be used for remote movement instruction
• Paper C: Extended reality (XR): Exploring use of the Owl, a telepresence system focused on real-time immersive capture, delivery, and rendering of physical spaces and interactants
• Paper D: Assistance recruitment interactions between remote and local users of mobile robotic telepresence

The afternoon sessions were dedicated to discussion of the challenges and opportunities of telepresence technologies in the context of hybrid learning environments. Ideation and prototyping exercises followed the discussions to cocreate recommendations for future technology and research directions.

Evaluating Current Telepresence Practices in Learning Environments
During the first open session, some of the paper topics were discussed and debated. In particular, the participants explored and evaluated current telepresence practices in learning environments. We compared conventional videoconferencing tools (e.g., Zoom, Microsoft Teams) with emerging telepresence technologies such as MRP, stationary and mobile telepresence robots, robotic arms,  holograms, Cartesian manipulators, and haptic tools. We discussed their affordances and use expectations as well as their actual use and performance in educational and learning environments. The discussions touched on the role of embodiment in helping the remote attendee feel immersed in the distance learning environment and how UX and usability issues can disrupt the attendance and the completion of tasks.

Creating: Design Sprint
The prototyping session of the workshop took the form of a design sprint. Each participant had 10 minutes to sketch eight different ideas that could be used to address any of the challenges previously discussed. At the end, the ideas were put in a shared document and each participant voted for their three favorite ideas.

The result of the design sprint can be grouped in three areas:

• Improve the communication capabilities of telepresence robots. First, by giving them better embodiment capabilities such as a 360-degree representation of the remote user, robotic arms, and stair climbing. Second, by powering them with immersive communication technology (for example, omnidirectional video capture, XR telepresence, directional audio, and remote haptics). And finally, by adding augmentation capabilities to compensate for the inherent limitations of not being present (e.g., using machine learning to recognize people and automatically tag their names).
• Design the classroom for hybrid scenarios. For instance, using screens to show remote users in fixed positions within the classroom, or creating an interactive classroom map for spatial awareness of the local environment. This includes creating specific tools for the hybrid experience (for example, a physical microphone with a digital twin, used to “pass the floor” between students; connected tablets to share drawings and text; a mini-chat companion app to have a common side-talk channel for everyone; or even having everyone participate in the physical space via virtual means).
• Change the teaching strategy to actively include the remotes. For instance, paring them with local students, giving locals access to the remote user view, creating a “karma” system to gamify courtesy with remotes, and ensuring time-zone awareness across platforms.

Challenges and Opportunities
Telepresence technologies present many opportunities for novel applications and research. During our workshop, we explored the idea of moving beyond a single product to thinking about the whole classroom more broadly. The ideation exercise further prompted us to examine what aspects of interaction might matter—spatial audio, three-dimensionality, gaze, pointing, back-channeling, whispering—and how elements of meditated interaction (such as digital or robotic) could be blended to give remote and local users more options to express themselves and engage meaningfully with one another. 

There were several challenges identified in our workshop. On the technical side, the current technology is still limited, and those limitations may have social repercussions in learning activities. When connectivity lagged in our workshop, the audio was unclear and the visual resolution low. The remote participants’ experience was disrupted, and they missed out on what was happening in the physical space. We also explored issues of equity. Workshop participants acknowledged that for telepresence to work well requires time, money, and effort; this includes an appropriate space with good WiFi, screens, and cameras, and detailed plans that include telepresent learners. Promoters of telepresence technologies must consider how these challenges will be met to ensure equitable access. 

Future Directions for Research and Practice
Future directions for research in telepresence should include studies on the immersive capabilities of 360 cameras. We believe the ability to independently view the complete physical space will remarkably increase feelings of presence for the remote viewer. Just as people can view an entire room without leaving their seats, the 360-degree camera affords the remote user this same capability. Future studies may also explore how interactants feel about a 360 camera. For example, how would it feel if you were speaking with someone on a telepresence unit and they turned their “head” 180 degrees to talk to someone behind them? As humans, we do not have this capability. Would it be welcome? Would it disrupt acceptance of the remote user?

Additionally, we expect future studies may include:

• User privacy and data-sharing issues
• Social guidelines for all participants in the telepresence experience
• Accessible controls that are adaptable by end users to accommodate personal and physical preferences
• Ethics surrounding how to present equal inclusion of remote participants by respecting their audiovisual rights in the physical space

Conclusion
The number of telepresence researchers in the HCI community is increasing in the post-pandemic world as the global community has adapted to work-from-home, learn-from-home, and socialize-from-home activities and experiences. Our emerging telepresence workshop was an effort to provide a place for telepresence researchers to engage in transfers of knowledge, share works in progress, cocreate, and problem solve. Topics that emerged in our workshop included: views on robots as educators and learners, the potential use of haptic touch/force to communicate feedback on physical learning activities (e.g., dance), desktop telepresence technologies that relay immersive capture, delivery and rendering of physical spaces and interactants, and exploration of assistance recruitment interactions between remote and local users. 

As our workshop took place during a Covid surge, some participants were unable to travel to attend. We were fortunate, however, to have researcher expertise in both HCI and telepresence technologies to conduct our workshop with a hybrid approach. As a hybrid learning approach was our desired area of study, we were excited to synthesize our topics of discussion with our workshop-participant experiences in learning via telepresence. We used several forms of digital media to interact within our workshop: Zoom, Meeting Owl 3, Google Docs, cell phones (cameras and text), and shared screens. Our workshop ideation and prototyping exercises found that HCI researchers valued the use of a 360-degree camera (remote user to view the physical room and people), a detailed agenda on progression of learning activities with identification of technologies throughout the learning exercise (e.g., use Zoom now, transition to Google Docs now, etc.), and directed plans for paired learning between remote users and interactants. Additionally, as many participants were logging in from different time zones, we suggest an optional time stamp for awareness of local times. Overall, we were encouraged by the hybrid learning experience and excited to participate in ways like other remote learners in hybrid spaces. Our discussions and topics raised several challenges, however, that need to be addressed for equitable and inclusive use of telepresence technologies. Future HCI research has great potential to address these challenges, create technologies, and promote social practices that facilitate equitable learning through sociotechnical inclusive environments.

Endnotes
1. Ahumada-Newhart, V. and Olson, J.S. Going to school on a robot: Robot and user interface design features that matter. ACM TOCHI 26, 4 (2019), 1–28.
2. Grondin, F., Lomanowska, A.M., and Jackson, P.L. Empathy in computer-mediated interactions: A conceptual framework for research and clinical practice. Clinical Psychology: Science and Practice 26, 4 (2019), e12298.
3. Rae, I. and Neustaedter, C. Robotic telepresence at scale. Proc. of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2017, 313–324.
4. Elmimouni, H. et al. Emerging telepresence technologies in hybrid learning environments. CHI Conference on Human Factors in Computing Systems Extended Abstracts. ACM, New York, 2022, 1–5.

---

Posted in: on Wed, January 25, 2023 - 3:49:00

Houda Elmimouni

Pablo Pérez

Andriana Boudouraki

Fatma Guneri

Verónica Ahumada-Newhart

Authors: Maximilian Speicher
Posted: Tue, November 15, 2022 - 2:44:00

---

The precise, quantitative measurement of user experience (UX) based on one or more metrics is invaluable for design, research, and product teams in assessing the impact of UX designs and identifying opportunities. Yet these teams often employ supposed UX metrics like conversion rate (CR) and average order value (AOV), which can’t provide that measurement [1]. In fact, I believe this can be extended to an even more general statement: In themselves, none of the metrics that are usually readily and easily available from Web analytics data can reliably measure UX. I understand that this is frustrating news to many, since resources are always limited, attention spans short, and Web analytics so very, very convenient.

Whenever I discuss this, I encounter objections like, “But we have to do something,” or “It’s easy to just state what one shouldn’t do, but that doesn’t help much.” And while it’s a perfectly fine start to know what not to do (cf. Nicholas Taleb’s The Black Swan), in the case of UX, we must not despair, because there are ways to reliably measure it, albeit ones that are not as simple as pulling some number out of Google Analytics (as nice as that would be).

A simple outcome of measuring UX could be, “The last release improved checkout UX from 75/100 to 80/100,” but there could be more-nuanced measurements for different aspects of UX (e.g., usability, aesthetics, joy of use) and user groups. Before diving deeper into how we can do this, let’s first get familiar with three concepts:

• Latent variables (like UX) “are variables that are not directly observed but are rather inferred through a mathematical model from other variables that are observed” [2]. Take, for example, the Big Five personality assessment [2]. You can’t just ask someone, “What’s your personality?” and expect an objective answer. Rather, you have to ask a set of specifically designed questions and then infer personality from those.
• A research instrument is a set of such specifically designed questions, often in the form of a questionnaire. Through an instrument, we can collect the observable variables that help us infer the latent variable we’re after
• We’re dealing with composite indicators when we combine individual variables from an instrument into a single metric.

Additionally, it’s necessary to understand that the user’s experience is not a property of a digital product or user interface. An app doesn’t have a UX. Rather, the experience “happens” in the individual user’s head as a reaction to their interaction with a digital product [3]. Hence, the only way to directly observe UX would be to look into the user’s head using an EEG or similar—and even if you had the possibility, that would be pretty complicated. In any other case, the next best alternative is asking them about it. In contrast, Web analytics data (and metrics like CR and AOV [1]), are relatively “far away” from what happens in the user’s head. They’re collected directly on a website or in an app, aggregated over many users—even though there is no such thing as an “average user”—and influenced by myriad other non-UX factors [1].

So, do we always have to ask users to fill out a questionnaire if we want to reliably measure UX and can’t make use of analytics data at all? The answer to both is “not necessarily,” but first things first.

Plenty of work has already been put into instruments for measuring UX, and this remains an ongoing topic in the human-computer interaction research community (which strongly suggests this is not a trivial matter). Three examples of scientifically well-founded instruments are AttrakDiff (http://attrakdiff.de/index-en.html), UEQ (https://www.ueq-online.org), and meCUE (http://mecue.de/english/home.html), with the first two currently better-known and more widespread. The first research article on AttrakDiff was published as early as 2003. All three also define composite indicators for different aspects of UX. For instance, AttrakDiff provides one composite indicator for hedonic and pragmatic quality each.

So, there exist proper instruments for reliably measuring UX, and you can always have users answer one of those in a controlled study or by means of a live intercept survey. But how might analytics data come into the picture? As so often, the answer lies in statistics. To use analytics metrics for approximating UX, you first have to determine how much individual metrics correlate with and are predictors for actual UX; otherwise you’re simply guessing. For instance, if you have a sufficient number of sessions where you record user behavior and collect answers to one of the questionnaires, you might be able to use them as training data for machine-learning models that predict either individual items or composite indicators—or both—for different user segments. Sounds easy enough, right? Collect some analytics data, collect some answers to a questionnaire, put it all in a Jupyter Notebook, and you’re good to go?

Incidentally, that’s pretty much what I investigated in my Ph.D. thesis, only I did it for usability (using an instrument named Inuit [4]) and interaction data like mouse-cursor movements and scrolling behavior. But the basic principle was the same. Based on this, I have good news and bad news for you. The good news: It worked. I was able to train models that could predict the usability of a Web interface reasonably well. The bad news: It was not all that easy.

First, I didn’t put “sufficient number” above in italics for no reason. Turns out it’s really difficult to collect enough training data when people must fill out questionnaires—and Inuit has only seven items. UEQ, for instance, has 26! Second, interaction data‒based models seem to be very sensitive to interface changes. My results suggested that, at best, one could apply models trained on one website within a narrow cluster of very similarly structured websites, but not beyond, while also having different models per user type. Colleagues and I have, however, already started looking into these problems [5].

In conclusion, because UX is something so elusive, if you want to properly measure it, there’s no way around using a scientifically well-founded instrument. Although often applied as supposed “UX measures” in industry, off-the-shelf analytics data are in themselves not suited for this. Simply speaking, this is because 1) there’s a significant distance between the interface where they are collected and the user’s head, where UX “happens,” and 2) they are influenced by too many non-UX factors to lend themselves to meaningful manual analysis. Yet, under certain conditions, it is possible to predict UX from analytics data, if we combine them with answers to a proper UX instrument and use all of that to train, for example, regression or machine-learning models. In the latter case, you can use methods like SHAP values to find out how each analytics metric affects a model’s UX prediction. And if there are just a few strong predictors, it might even be possible to take a step back and work with simple equal-weight models, as Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein describe in their book Noise.

What could be an ultimate solution to the practical problem of measuring UX? Probably some kind of universal model where anyone can put in their analytics data, user segments, and parameters of their digital product to get instant UX predictions. Does such a model exist? Absolutely not. Might it exist in the future? I can’t say for sure, but I’ll keep looking.

Endnotes
1. Speicher, M. Conversion rate & average order value are not UX metrics. UX Collective (Jan. 2022); https://medium.com/user-experi...
2. Wikipedia contributors. Latent variable. Wikipedia; https://en.wikipedia.org/w/ind...
3. When a large number of users agree that they’ve had pleasant experiences using an interface, researchers usually colloquially state that the interface in question “has a good UX.”
4. Speicher, M., Both, A., and Gaedke, M. Inuit: The interface usability instrument. In Design, User Experience, and Usability: Design Discourse. Springer, Cham, 2015, 256–268; https://link.springer.com/chap...
5. Bakaev, M., Speicher, M., Heil, S., and Gaedke, M. I don’t have that much data! Reusing user behavior models for websites from different domains. In International Conference on Web Engineering. Springer, Cham, 2020, 146‒162; https://link.springer.com/chap...

---

Posted in: on Tue, November 15, 2022 - 2:44:00

Maximilian Speicher

Authors: Jonathan Grudin
Posted: Fri, September 30, 2022 - 11:34:00

---

Panels can be fun to develop—they can also be much more effectively executed. This spring I was on two large panels that were the best organized of any I’ve participated in. Technology was used in advance to reduce psychological uncertainties, which increased interaction and kept us focused on our message. Expectations were exceeded. Both of the panels were in person, but the method is even more promising for hybrid panels.

The first panel was organized by SIGCHI Adjunct Chair for Partnerships Susan Dray and VP of Finance Andrew Kun to discuss CHI’s 40th anniversary. Structure that one might assume would squeeze out liveliness instead promoted it. Impressed by its success, I reprised the method in another 90-minute panel at my Reed College class reunion. These are only two data points, but the underlying psychology is compelling, especially for large panels and future hybrid panels.

The 40th anniversary panel was an exceptionally diverse group of eight, plus Andrew as moderator. Our backgrounds, interests, and priorities differed substantially. Forty years earlier, one of us wasn’t yet in school and only one was involved with CHI. Yet we wanted to avoid eight 10-minute talks—we wanted to engage with one another.

Andrew’s solution was to address psychological barriers to effective time management that I had never noticed. Panelists like to talk. The trick: Make it possible for panelists to avoid saying more than they really want to.

A large panel should minimize moderator control onstage to give panelists time to get their points across, but if panelists are undirected, meandering and a lack of coherence are inevitable. Andrew directed the panel, but he did so in advance, which greatly reduced the cognitive effort required of panelists at the event.

Before the event

Weeks in advance, Andrew shared a Google Doc that asked panelists to draft text for four sections:

• A bio
• A two-minute opening statement or “provocation”
• One or two questions from each of us directed to another panelist (encouraged to ensure that each panelist was asked at least one question)
• A one-minute concluding summary.

Two virtual meeting deadlines ensured that we completed drafts in a timely fashion. Everything could be revised up to the event, but Andrew could also sequence opening statements early to create a coherent topic flow. Seeing one another’s drafts and the presentation sequence enabled us to keep statements to the same length, reduce redundancy, share terminology, and shape questions and summaries. 

Seeing a question that would come our way in advance, we could organize a response, but responses were not shared. They were fresh to other panelists at the event, and anyone could contribute thoughts if the initial respondent didn’t cover them.

At the event

We had crafted crisp bios that Andrew read to introduce us. Our opening statements were practiced, fluent, and within the allotted two minutes.

The key innovation was the handling of the Q&A. Andrew had carefully thought through the questions and prepared a sequence of invitations. For example, “Daria, do you have a question for another panelist?”

The crucial point: We knew that Andrew was always ready to solicit the next question. Think of your experiences on a panel. Panelists are often unsure about the best way to continue and keep the discussion going. This leads to fuzziness or discontinuities. Three scenarios:

• I ask a question and receive a satisfactory response. If I have nothing to add, I’m in a bind. Not knowing whether someone else will jump in if I stay silent, I often politely respond or ask an unnecessary follow-up question to keep the conversation going. In our event, I could pause and glance at Andrew. If no one immediately spoke, he moved on: “Tamara, do you have a question for another panelist?”
• There is a pause in the discussion. Has the current topic been covered adequately? I haven’t spoken yet, Should I break the silence, even if what I will add is mostly to agree or digress? At our event, there was no pressure to do this. Andrew was ready to break the silence and move on to the next cogent question.
• A topic is exhausted. It’s clearly time to move on. I have a very different topic, but someone else may have a more relevant continuation. Should I jump in? In our panel, we could leave graceful transitions to Andrew, knowing that our question would get its turn. Andrew was following the discussion and knew the range of questions remaining.

I felt that a cognitive load had been lifted—it was great! The navigation of conversational conventions adds many small efforts that compete with focusing on what is said and the overall topic. There were no space-filler comments as we engaged with one another. 

I’d initially thought that although structure might be necessary for an eight-person panel, it would sound scripted and diminish spontaneity. That wasn’t the case. We added follow-up questions and comments after one of us responded. The conversation was brisk and focused on things we cared about. Little cognitive work was required for conversation management. 

A replication

For my 50th college reunion six weeks later in Portland, Oregon, five people who spent careers in tech formed a panel to discuss how tech had evolved, our roles, and what we recommend younger people think about. I introduced Andrew’s structure. With five panelists, we doubled time allowances and mixed our prepared questions with less predictable audience participation. It went well. People go to college reunions more to hang out and socialize than to attend lectures and panels, but we drew a large crowd who stayed throughout.

Natural for hybrid panels

Efficient conversation management is critical for large panels. This approach also offers benefits for virtual or hybrid panels. Hybrid planning is by necessity online—which means there’s no opportunity to get together for breakfast the day before—so coauthoring a structured document and holding a couple of preliminary meetings is a natural fit. More significantly, navigating conversation transitions is more challenging when panelists have less awareness of one another’s body language. Hybrid panels may gravitate toward larger sizes, and cultural diversity could require bridging conversation styles.

This structure requires more preparation, but it was distributed over time, getting a better sense of other panelists’ contributions reduced some of the effort, and the collaboration was enjoyable. It was worth it.

---

Posted in: on Fri, September 30, 2022 - 11:34:00

Jonathan Grudin

Authors: Cori Faklaris
Posted: Tue, August 16, 2022 - 4:29:00

---

Computer science seems in opposition to Zen Buddhism, a spiritual practice best described (if it must be) as “without reliance on words or letters, directly pointing to the heart of humanity” [1]. Yet with so much of today’s human experience bound up in computing, some words, and pixels and bits, to address their overlap seem necessary. My aim in sharing the following reflections is to set forth the historical and modern faith context for a secular wellness practice that we design for— meditation—and to model the statements of positionality and reflexivity that I feel are essential for research in such personal and cultural domains.

Zen Buddhism

Zen’s origins were first documented during China’s Tang dynasty in the 7th century CE. At that time, Buddhism had already spread from Nepal, the birthplace of historical founder Siddhartha Gautama, or “the Buddha,” throughout neighboring countries in Asia for more than 1,000 years. The Indian monk Bodhidharma is credited with introducing China to the dhyana practice of stillness and contemplation. Dhyana (a Sanskrit word) predates the Buddha and is commonly described as his vehicle for achieving enlightenment, or the transcendence of his limited human existence. The renewed focus on dhyana was a reaction to the older branch of Buddhism known as Theravada, the “way of the fathers” [1]. The chief text of the newer Mahayana branch of Buddhism is the Heart Sutra, the English translation of which fits on one page. Zen takes this minimalist approach even further, proclaiming the superiority of empirical knowledge gained through dhyana —renamed ch’an in Middle Chinese—over scriptural learning or formal religious observance. A famous poem of the Tang dynasty sets forth this formulation for Zen [1]: 

A special transmission outside the scriptures
Without reliance on words or letters
Directly pointing to the heart of humanity
Seeing into one's own nature.

Today, the largest Zen communities remain in China and Japan (the origin of the word zen), but the practice of Zen has spread throughout the world. Like other Buddhists, who number in total 488 million worldwide [2], they venerate the “Triple Treasure” of small-b buddha (inherent enlightenment-nature), dharma (the teachings and practices), and sangha (their faith communities). Zen practitioners also particularly value upaya, or “skillful means” (the ability of an enlightened being to tailor a teaching to a particular audience or student for maximum effectiveness) and mindfulness (in Zen, the continuous, clear awareness of the totality of the present moment). Through seated meditation, alternated with practices such as chanting, bowing, and contemplative walking, a Zen Buddhist aspires to a state of mindfulness that will facilitate their own perception of buddha-nature and help them express this enlightenment in daily life, especially for the benefit of others. To check the validity of their meditation experiences, Zen practitioners are urged to consult with a teacher in an established lineage who is certified to guide others in enlightenment. Among a teacher’s “skillful means” are stories or riddles known as koans (Japanese), gong-ans (Chinese), or kung-ans (Korean). Such consultations will help Zen practitioners achieve a “before-thinking,” other-centered orientation and avoid self-centered fallacies—for example, “wanting enlightenment is a big mistake” [1]. 

Personal experiences and observations

My own experiences with Zen are Western. I began in my teenage years, when I bought a secondhand copy of D.T. Suzuki’s Essays in Zen Buddhism and was intrigued by his discussions of satori, the Japanese word for enlightenment. I had already liked what I had heard about Buddhism during a unit on world religions at my (Catholic) grade school. However, it wasn’t until I moved to the U.S. state of Indiana that I was able to connect with an in-person group, practicing in the Kwan Um School of Zen (KUSZ) in the lineage of Korean Zen Master Seung Sahn. I began sitting with the Indianapolis Zen Center sangha once or twice a month as my schedule permitted: 30 minutes of seated meditation bookended by a 20-minute prelude of chanting and a 10-minute epilogue of a reading and announcements. I progressed to sitting weekend retreats and to taking precepts (like Christian baptism, this signifies formally joining the faith). Eventually, I studied for and became a KUSZ dharma teacher—qualified to explain subjects such as meditation forms and the history of Zen, but not to guide people to enlightenment. In KUSZ, such teachers are called Ji Do Poep Sa Nim (JDPSN, for "dharma master") or Soen Sa Nim (Zen master). I have studied with both types of “enlightenment” teachers at the Indianapolis Zen Center and with a group in Pittsburgh, PA, while helping as a dharma teacher.

As a Zen teacher, I do not take a binary view of computing as good/not good or useful/not useful. The “middle way” is to acknowledge that it is a dharma aid in some contexts and a distraction in others. Below are some examples.

Computing as obstacle to Zen practice

My advice to beginners is to turn off their smartphones completely. This is because a buzz or ding is liable to take meditators out of the moment, and beginners often will struggle to refocus. For my in-person group, I model another best practice by taking out my phone or smartwatch, silencing them, and turning them face-down on my meditation cushion, so that I cannot see the flash of a notification. I prefer to use such manual safeguards for attention rather than the “Do Not Disturb” settings, because enacting the exercise of putting away our digital helpers is an important signal to our bodies and minds that what we are doing is important and different from the everyday flow of our distracted lives. In the same vein, I recommend use of a battery-operated analog clock over a smart device for timing seated meditation, because it will not tempt you into checking messages.

In the world of Covid-19, much of our group Zen practice has joined others online. Now, it is no longer possible to physically remove ourselves from our Internet-connected devices. I am grateful to be able to see and hear my fellow practitioners even at a distance, but I miss having the break from my busy digital life and from the allure of its distractions. The “Do Not Disturb” settings help, to a point. Sitting in front of my MacBook, however, I catch myself touching my mouse and calling up screens whenever I experience a fleeting thought about, say, the status of a project. Meditating from home also means interruptions from family members, pets, or Internet outages. I confess that I do not have enough “dharma energy” to avoid breaking my stillness in response to my cat waving her tail in my face! 

Going forward, this type of Zen Buddhism will benefit from computing research and design to solve similar problems of distraction and focus as those faced by those working from home or who are “digital nomads,” connecting to their customers or clients via the Internet away from an office. I would love to flip a switch inside my home environment and be free from all ability to access Netflix or Slack while I hunker down on either a research paper or a kung-an. Even better if the “switch” is a timer, so that I do not forget to turn off my “Do Not Disturb,” or a learned routine of my home network, so that it is context-aware and picks up on the signals that I am ready to concentrate. I use Siri now to set a meditation timer by voice, although the screen and keyboard is still nearby, and it would be better for my ability to stay in concentration if  “she” could turn off everything at the same time and then turn it back on again after the timer ends. 

However, like the meditators in Markum and Toyama [3], I am wary of letting technology intrude too far or replace in-person experiences. I am doubtful that it will no longer be necessary to visit a Zen center or monastery for the sustained concentration required for intensive practice. My Pittsburgh group has returned to offering a weekly in-person (and masked) practice so that people can get a break from remote meetings and reap the benefits of in-person group meditation. I look forward to the day when we can begin traveling to other temples and learning in situ about others’ spiritual practices, perhaps with the assistance of interactive displays or augmented reality overlays. 

Computing as support to Zen practice

Is reading an obstacle? After all, “words and letters” are considered a hindrance in Zen tradition. However, reading is often the first step undertaken by someone who wants to try meditation and/or to learn more about Zen Buddhism. As Bell has noted [4], the internet has been enormously helpful for spreading Zen knowledge and for connecting seekers with faith communities. My sanghas have made use of the same computing affordances as other interest groups: websites, online groups, platforms for event discovery, and secure no-contact payments. 

I will always prefer in-person Zen practice. But, like the online group members in Katie Derthick’s work [5], I now can join a Zen meditation session or a retreat from anywhere in the world. I can take part in a distant reading group or a book club (we have those!). I can receive an interview via video and audio from a variety of Zen teachers. For those who don’t want that group experience, a variety of apps can guide them in contemplating peace or following their breaths. Headspace even dims the screen so that you can use it to wind down and prepare for sleep.

Going forward, the main item on my wish list is better audio support for remote Zen practice. We have experienced glitches in teacher interviews where one person trips over the other person’s statements, adding to the problem of not being present to pick up on nonverbal cues to turn-taking such as angling back or tilting forward. Worse, audio problems have almost killed our group chanting. This is unfortunate because, in my tradition’s Zen practice, chanting is essential for aligning participation and building an energy within the group that supports its focus. Participants cannot stay in sync—the farther from the source, the more obvious the transmission delays. Our Zoom apps also struggle to figure out which voices to prioritize, instead of blending every audio source into a unified output. For now, the workaround is for everyone to mute and only listen to the leader’s chanting. We need apps like JamKazam or Jamulus, which were designed for musicians to play together online and at a distance. Such an app will need to integrate with our existing remote meetings and be usable by anyone. 

Suggested best practices for computing researchers

Religion is as sensitive a topic as it is central to the human experience. From my N of 1, I suggest that computing researchers will do themselves good to consider their positionality and biography with regard to this subject, before embarking on faith-minded research [6,7]. Clarification of our personal experiences—how we were raised and how we have directed our adult lives with regard to religion—will make explicit our social, cultural, and historical position with regard to the faith domain. Reflexivity requires time, but reading, discussing, and thinking will help us to identify what assumptions we bring to the project. Once articulated, our preexisting assumptions will be less likely to warp our research or to stymie our openness to new ideas. (In my case, I sat and thought about whether I have a bias toward adding technology to any faith domain, regardless of whether it is truly needed. I also challenged myself as to whether I assume that adding technology will lead to only negative downstream effects for a religious community.)

For the conduct of this research, I suggest three ethical pledges that will reduce the potential for exploiting participants: adherent-centeredness, getting close-up, and considering relationship ethics. Researchers should prioritize the faith population’s needs, preferences, and values, and incorporate them to the extent possible: “Nothing about us, without us.” Careful, respectful qualitative work such as Wyche et al. [8] follow Genevieve Bell’s prescription to use techniques informed by anthropology, focusing on the particulars of place, location, and critical reflexivity [9]. Researchers should make use of practices such as participant observation that foster empathy and consider layering different participants’ accounts, rather than aggregating them into a majority narrative [6]. And they should recognize that such research will involve leveraging existing relationships and fostering new ones. Discuss issues of privacy and confidentiality upfront, for example, that it may not be possible to de-identify anyone [6]. Share work and ask for responses and comments. In publications, alter specific personal or topic details to protect their privacy, security, and safety. 

These suggestions may sound like standard operating procedure for some qualitative researchers in human-centered computing. But many of us are trained in a positivist orientation, in which reason and logic are prioritized. We will benefit from having these or similar principles explicitly articulated for our consideration and commitment, just as a Zen master benefits from reciting the temple rules about not borrowing people’s shoes and coats. We all need help staying mindful.

Endnotes

1. Sahn, S. The Compass of Zen. Shambhala Publications, 1997.

2. Buddhists. Pew Research Center’s Religion & Public Life Project. Dec. 18, 2012; https://www.pewforum.org/2012/12/18/global-religious-landscape-buddhist/

3. Markum, R.B. and Toyama, K. Digital technology, meditative and contemplative practices, and transcendent experiences. Proc. of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–14; https://doi.org/10.1145/3313831.3376356

4. Bell, G. Auspicious computing? IEEE Internet Comput. 8, 2 (Mar. 2004), 83–85; https://doi.org/10.1109/MIC.2004.1273490

5. Derthick, K. Understanding meditation and technology use. CHI ’14 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, 2014, 2275–2280; https://doi.org/10.1145/2559206.2581368

6. Darwin Holmes, A.G. Researcher positionality—A consideration of its influence and place in qualitative research—A new researcher guide. Shanlax Int. J. Educ. 8, 4 (Sep. 2020), 1–10; https://eric.ed.gov/?id=EJ1268044

7. England, K.V.L. Getting personal: Reflexivity, positionality, and feminist research. The Professional Geographer 46, 1 (1994); https://www.tandfonline.com/doi/abs/10.1111/j.0033-0124.1994.00080.x

8. Wyche, S.P., Hayes, G.R., Harvel, L.D., and Grinter, R.E. 2006. Technology in spiritual formation: An exploratory study of computer mediated religious communications. Proc.  of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work. ACM, New York, 2006, 199–208; https://doi.org/10.1145/1180875.1180908

9. Bell, G. No more SMS from Jesus: Ubicomp, religion and techno-spiritual practices. In UbiComp 2006: Ubiquitous Computing (Lecture Notes in Computer Science). Springer, Berlin, Heidelberg, 141–158; https://doi.org/10.1007/11853565_9

---

Posted in: on Tue, August 16, 2022 - 4:29:00

Cori Faklaris

Authors: Derek L. Hansen, Amanda L. Hughes, Xinru Page
Posted: Thu, August 04, 2022 - 3:16:00

---

Religious beliefs have a profound influence on billions of people across the globe, affecting nearly every aspect of their lives, including the use of technology. While there is a continuous rise in atheism, the majority of people in many countries still believe in a deity, self-identify with a religion, and regularly participate in religious practices such as prayer. For example, in the U.S., 69 percent self-identify with a religion, 66 percent consider religion very important (41 percent) or somewhat important (25 percent) to their life, and 67 percent pray daily (45 percent) or weekly/monthly (22 percent) [1]. For many people, religious beliefs and teachings frame every aspect of their life, influencing behaviors related to diet, social relationships, dress and grooming, sexual practices, mourning for the dead, raising children, and financial decisions, among others. 

It is no surprise, then, that religions have much to say about the use of technologies, such as the Internet, social media, and mobile phones. Yet design guidance is mostly absent on how to design technology in ways that support various religious values and beliefs. While philosophers, sociologists, and humanities scholars have studied the intersection of technology and religion, relatively few studies have examined religion and technology from a design and HCI perspective [2]. This is unfortunate, since religious traditions often seek to transform the lives of their adherents and the world for the better. Such inclinations can be highly compatible with core HCI values, which often focus on the betterment of the world through the novel use of technology.

Few HCI researchers make time to look through the lens of religious teachings at the technologies that surround us [2]. Thus, we don’t fully appreciate basic questions related to religious teachings and technology. How central a role does technology play in religious teachings? What stances do religions take on the appropriate or inappropriate use of technologies? How do religions frame the discussion around technology, given that many of their teachings are based on ancient texts written by those with vastly different technologies? What role does religious doctrine play in informing religious practice around technologies? How do these answers differ for different religious traditions?

Religious values are integral to many people’s lives and should be considered a key value for the HCI community to integrate into technology design. All physical and digital artifacts convey and enforce certain values, whether they are purposefully designed to do so or not. Thus, value clashes can occur when the affordances of the technology are not aligned with the values of the system’s users. In fact, a system that is designed from the perspective of one group may impose the values of that group on other target users of the system. For example, the idea that a mobile phone is attached to one individual and thus a unique phone number can be required for each person’s account setup for an online service may be a fair assumption in many contexts. However, it causes issues for countries, settings, or religious contexts where a mobile phone is a shared object between a married couple, family, or even extended community, creating an account setup roadblock for anyone who does not have their own phone number. Technology infrastructures meant to support people with many different values should account for this diversity of values and validate assumptions about its users.

A very limited number of HCI studies have investigated how technology practices complement or hinder religious practices via empirical studies (we expand on a number of these in the next section). Existing HCI research has also focused on understanding how religious practices can inform design in nonreligious contexts. For example, several studies apply strategies employed by religious organizations to enhance commitment, build community, and/or motivate behavior change in nonreligious organizations. Ames et al. explored how religious ideological practices can serve as a useful lens in understanding how nonreligious engineering and design organizations affirm membership and a shared vision [3]. Similarly, Amy Jo Kim discusses the use of rituals (a concept inspired by religions) in building commitment to online communities [4]. 

While this prior work gives us initial insights into the interplay of religiosity and technology in practice, there is an element that is missing and yet key to understanding values we might aspire to incorporate into our technologies. While studying how people use technologies in practice gives us a descriptive understanding, there is also a prescriptive element of religion that is vital to understand. In many religions, there are a set of values that believers aspire to, and they hope to engage in practices that reflect those core values. Thus, we need to not only understand how technologies are used in practice, but also the guiding principles that a given population might be influenced by or aspire to. This also helps us to identify new opportunities for supporting religiosity.

How religious doctrine and teachings can inform design

Many religions provide specific prescriptive guidance to their adherents on the use of social media, the Internet, or other modern-day technologies that are of interest to the HCI community. Such guidance is often based upon doctrines, or the foundational beliefs, principles, and teachings of a religion. These come in the form of ancient scriptural texts, commentaries, sermons, pronouncements by religious leaders, official publications of religious organizations (e.g., magazines), and a variety of other resources. While it is tempting to consider only the practical, prescriptive advice about technology use given to members of a religion, it is essential to also understand the religious doctrines and teachings that underly such advice. Designers can benefit in several ways from learning the core doctrinal beliefs of a religion in order to better design for its members. 

First, doctrines can inspire the use of religious metaphors that tap into believers’ deepest spiritual desires and insights. For example, Pope Francis used the metaphor from St. Paul’s teachings in the New Testament that views Christians as “members of the one body whose head is Christ” when discussing the importance of using social media to build up others and not tear them down [5]. This metaphor stresses the value of a community (i.e., a body) having different body parts (e.g., eyes, ears, legs, hands), each of which serves a different purpose for the benefit of the whole. By invoking this metaphor, the Pope encourages Catholics to see the Church as “a network woven together by Eucharistic communion [a central Catholic ritual, where unity is based not on ‘likes,’ but on the truth, on the ‘Amen,’ by which each one clings to the Body of Christ, and welcomes others.” Religious metaphors can summon strong emotions and spiritual insights in believers, providing motivation to use technology in certain ways (e.g., being kind in online discourse).  

Second, understanding religious doctrine and teachings can help designers solve problems and achieve goals that are religious in nature. For example, Woodruff et al. studied the home automation practices of American Orthodox Jewish families [6]. Jewish laws generally prohibit manually turning electronic devices off or on during the Sabbath. These families had long designed and automated systems within their home that would perform mundane tasks (e.g., turning lights on/off with timers or sensors) to abide by this law. Without a deep understanding of Jewish teachings and practices, designing to meet their needs would not be feasible. Another example comes from the Church of Jesus Christ of Latter-Day Saints, whose doctrine encourages members to be eternally “sealed” (i.e., connected) to their family, including deceased ancestors through vicarious ordinances performed in their Temples. This focus has led them to invest significant resources in developing genealogical tools that help members identify their ancestors, such as FamilySearch, which has a collaboratively generated family tree with over 1.38 billion names and had over 200 million site visits in 2021. Additional tools allow church members to track and manage the vicarious work that members perform in the Temples. Thus, an understanding of the core doctrines related to Temple work has been essential to the development of unique tools that support such work. 

Third, understanding religious teachings can help designers modify existing technologies to better meet the needs of believers. For example, HCI researchers have recognized that technologies supporting financial services within Muslim communities must work within a religious framework where charging interest is forbidden [7]. Thus, micro-lending websites that rely on interest must be modified in fundamental ways to be viable solutions in Muslim communities. Many religions have their own dating sites, helping people find singles with a similar religious background. In some cases, these include specific features that differ from general dating websites. For example, Shaadi is a popular Hindi “matrimonial” website with 35 million users that focuses on finding a spouse rather than hookups. 

Finally, understanding religious doctrines can help designers identify core values that can then be used to design solutions that are in harmony with and reflect a believer’s core beliefs. Susan Wyche and Rebecca Grinter [8] examined how American Protestant Christians use ICT in their home for religious purposes. Their findings suggest many opportunities for designing systems that acknowledge, honor, and support religious values in a domestic setting, such as creating digital calendars and displays that recognize and change their content based on significant religious holidays or milestones. In 2005, an Israeli wireless company launched a mobile phone specifically designed for the ultra-Orthodox Jewish community in Israel [9]. The phones were modified to disable Internet, text messaging, and video and voice messaging, after religious authorities and community members became concerned that these services could infiltrate the community with unacceptable content. 

There are many open questions about how to best use religious doctrines and teachings in design. How can design methods be modified to incorporate and prioritize religious teachings? How can products be evaluated based on religious teachings? How can technology help achieve uniquely religious goals, for which technology has not historically been used? How can systems be designed that meet the needs of diverse religious groups, given their different teachings? What values can be derived from religious teachings that can be incorporated into design?

In asking these questions that seek to understand the prescriptive aspect of religious values in the context of technology use, we can better understand which values people may want represented in technologies. We also hope that this work will serve as a call to action for the HCI community to engage more holistically with religious values. A set of values that are such an integral part of so many peoples’ lives should be acknowledged and given priority; HCI should support people’s priorities and values. We call on the HCI community to take steps toward understanding the interplay of religious values and technology to be able to create truly value-sensitive technologies. 

Endnotes

1. Smith. G.A. About three-in-ten U.S. adults are now religiously unaffiliated. Pew Research Center’s Religion & Public Life Project. Dec. 14, 2021; https://www.pewforum.org/2021/12/14/about-three-in-ten-u-s-adults-are-now-religiously-unaffiliated/

2. Buie, E. and Blythe, M. Spirituality: There’s an app for that! (but not a lot of research). CHI’13 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, 2013, 2315–2324; https://doi.org/10.1145/2468356.2468754

3.Ames, M.G., Rosner, D.K., and Erickson, I. Worship, faith, and evangelism: Religion as an ideological lens for engineering worlds. Proc. of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, New York, 2015, 69–81; https://doi.org/10.1145/2675133.2675282

4. Kim, A.J. Community Building on the Web: Secret Strategies for Successful Online Communities. Peachpit Press, 2006.

5. Pope Francis. Message of His Holiness Pope Francis for the 53rd World Communications Day. 2019; https://www.vatican.va/content/francesco/en/messages/communications/documents/papa-francesco_20190124_messaggio-comunicazioni-sociali.html

6. Woodruff, A., Augustin, S., and Foucault, B. Sabbath day home automation: “It’s like mixing technology and religion.” Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2007, 527–536; https://doi.org/10.1145/1240624.1240710

7. Mustafa, M et al. IslamicHCI: Designing with and within Muslim populations. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–8; https://doi.org/10.1145/3334480.3375151

8. Wyche, S.P. and Grinter, R.E. Extraordinary computing: Religion as a lens for reconsidering the home. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2009, 749–758; https://doi.org/10.1145/1518701.1518817

9. Campbell, H. ‘What hath God wrought?’ Considering how religious communities culture (or Kosher) the cell phone. Continuum 21, 2 (2007), 191–203; https://doi.org/10.1080/10304310701269040

---

Posted in: on Thu, August 04, 2022 - 3:16:00

Derek L. Hansen

Amanda L. Hughes

Xinru Page

Authors: Michael Hoefer, Stephen Voida, Robert Mitchell
Posted: Mon, July 25, 2022 - 3:23:37

---

In seeking to apply HCI to faith, religion, and spirituality, we turn to existing work in theology and psychology—in particular, work that studies the development of faith in individuals and communities. James Fowler is a pioneer in faith development, having developed a stage-based model after conducting over 300 interviews with individuals from a variety of religions [1]. In his work, Fowler suggests that faith is universal to all humans, and he provides an understanding of faith that we believe would serve the HCI community in grounding further efforts to integrate HCI practice with spirituality, faith, and religion. 

Fowler describes how most religious organizations fall short in supporting the development of faith in their constituents, as they fall prey to a modal developmental level—the most commonly occurring level (the mode) of development for adults in a given community (empirically, stage 3 of 6, explained in depth below). In other words, the developmental level that is most common in the adults of a community shapes the culture and normative goals of individuals growing up in that community, which Fowler calls an “effective limit on the ongoing process of growth in faith” [1].

HCI researchers are good at understanding how individuals form and act upon mental models. That is one of our aims when we try to develop technology: to understand people and what might help them. In helping individuals with their systems of meaning-making (faith), we might want to do something very similar. This, we suggest, is one of the prime challenges and opportunities for HCI researchers when engaging with faith and spirituality: to seek to understand and support individuals through their faith-ing. Here, we outline future research directions for what we are calling faith informatics: the study of systems that facilitate growth in individuals’ systems of meaning-making.

One direction for faith informatics could involve structured reflection, visualization of the self, and social connection. One goal might be to reify the existence of all stages of faith, beyond the normative, modal developmental levels present in many faith-based communities. By developing systems that support faith development regardless of religious (or nonreligious) orientation, computing and HCI hold the promise of supporting faith development and maintenance, highlighting convergence across religious traditions and supporting universal mature faith. 

Faith as fundamental to the human experience

The word faith is often associated only with “belief,” in that having faith is just a matter of what one believes to be true. Wilfred Cantwell Smith suggests an alternative: that faith is not dependent on belief [2]. In this vein, faith is neither tentative nor provisional, while belief is both. 

Fowler draws on this and suggests an alternative conceptualization that views faith and religion as separate, but reciprocal [1]. Fowler views religion as tradition that is “selectively renewed,” as it is “evoking and shaping” the faith of new generations. Faith is an aspect of the individual, while religion represents the tradition of the culture in which the individual grows and develops. Faith is an aspect of every human life, an “orientation of the total person,” and, as a verb, an “active mode of being and committing.” Faith is considered to be “the finding of and being found by meaning” [3]. This focus and widening of faith to include the association with meaning-making is not only inclusive of many religious and spiritual traditions, but is also vital in nonreligious traditions such Alcoholics Anonymous’s 12-Step Program. For HCI, this conceptualization resonates with existing research developments devoted to understanding the role of computing in finding and supporting meaning-making [4].

Specifically, Fowler describes three contents of faith that every human subconsciously holds in mind as they go about their life: 1) centers of value, which are whatever we see as having the greatest meaning in our lives, 2) images of power, which are the processes and institutions that sustain individuals throughout life, and 3) master stories, narratives we believe and live that facilitate our interpretation of the lived experience [1].

If faith is a universal human condition, as suggested by Fowler and Smith, then HCI researchers have much more purchase to engage with research questions related to faith, as that work would therefore have the potential to be applied to all of humanity.

Stages of faith development

Fowler describes a series of six stages of faith that are loosely aligned with age during the beginning of life, but development can be arrested in any stage [1]. Similar to Robert Kegan’s forms of mind [5], each progressive stage of faith is represented by a changing relationship between subject and object as an individual starts to consider a larger system as part of the “self” with regard to meaning-making. An overview of each stage suggests a diversity of use-cases for faith informatics.

Stage 1: Intuitive-projective faith. According to Fowler’s interviews, intuitive-projective faith is found in 7.8 percent of the population and is the dominant form found in children ages three to seven. Intuitive-projective faith is marked by fluid thoughts and fantasy, and is the first stage of self-awareness. Fowler explains that transitioning out of stage one involves the acquisition of concrete operational thinking and an ability to distinguish fantasy from reality.

Stage 2: Mythic-literal faith. Mythic-literal faith is estimated to be found in 11.7 percent of the population, and largely in children ages seven to 12. Mythic-literal faith involves the individual starting to internalize the “stories, beliefs, observances that symbolize belonging to his or her community.” Individuals in this stage create literal representations of centers of power, which often involves anthropomorphizing cosmic actors. Individuals in this stage may take religious texts literally as a foundation for meaning-making. While Fowler’s work did find a handful of adults in stage 2, many would transition out of this stage in their teenage years, as they experienced multiple stories that clashed and required reflection to integrate. 

Stage 3: Synthetic-conventional faith. Synthetic-conventional faith is the most commonly found stage of faith in Fowler’s sample, and appears in 40.4 percent of the population. Transitioning into this stage often occurs around puberty, and is associated with a growing connection to social groups other than the family, perhaps with different collective narratives and centers of value. Synthetic-conventional faith systems attempt to “provide a coherent orientation in the midst of that more complex and diverse range of involvements...synthesize values and information...[and] provide a basis for identity and outlook.”

This stage is highly aligned with Kegan’s view of the self-socialized mind [5]; both are intended to deal with meaning-making involving multiple social relations and belonging to various social groups. In this stage, individuals create their personal myth: “the myth of one's own becoming in identity and faith, incorporating one's past and anticipated future in an image of the ultimate environment.” Transitioning to the next stage often involves a breakdown in the coherence of the meaning-making system, such as a clash with religious or social authority or moving to a new environment (i.e., leaving home). 

Stage 4: Individuative-reflective faith. Developing an individuative-reflective faith system (32.9 percent of the population) requires an “interruption of reliance on external sources of authority.” Individuals in this stage are characterized by separation from previously assumed value systems, and the “emergence of an executive ego.” Fowler notes that some individuals may separate from previous value systems, but still rely on some form of authority for meaning-making, which can arrest faith development in the transition to individuative-reflective faith. This stage involves taking responsibility for one’s “own commitments, lifestyle, beliefs, and attitudes” and would be aligned with Kegan’s “self-authoring” form of mind.

Stage 5: Conjunctive faith. Stage five, according to Fowler, is difficult to describe simply. Conjunctive faith was found in only 7 percent of the population, and not until mid-life (ages 30 to 40). Conjunctive faith involves a deeper acceptance of the self, and integrating “suppressed or unrecognized” aspects into the self, a kind of “reclaiming and reworking of one’s past.” This stage is similar to Kegan’s self-transforming mind, as both involve the embrace of paradox and advanced meta-cognition about the self. Individuals in this stage must live divided between “an untransformed world” and a “transforming vision and loyalties,” and this disconnect can lead individuals into developing rare universalizing faith systems of meaning-making. 

Stage 6: Universalizing faith. Stage six represents a normative “image of mature faith” that was found to be present in only one interview participant [1]. Fowler describes these individuals as:

grounded in a oneness with the power of being or God. Their visions and commitments seem to free them for a passionate yet detached spending of the self in love. Such persons are devoted to overcoming division, oppression, and violence, and live in effective anticipatory response to an inbreaking commonwealth of love and justice, the reality of an inbreaking kingdom of God [3].

Developing a universalizing faith is seen as the “completion of a process of decentering from the self” [3], described by: 

taking the perspectives of others...to the point where persons best described the Universalizing stage have completed that process of decentering from self. You could say that they have identified with or they have come to participate in the perspective of God. They begin to see and value through God rather than from the self...their community is universal in extent [3,4].

The goal of faith informatics, as a direction of inquiry, is to better understand the systems (social, ecological, information, or otherwise) that support faith development and how they can be improved. A tool for research, and perhaps an intervention in itself, may be an information system that allows an individual to gather faith-related data about themselves and then visualize and interact with these representations of the self.

A design paradigm for faith informatics

Figure 1 highlights the components of a potential faith informatics system based on a smartphone or other computing-based application (the “system”). The theory behind this system design is drawn from research interviews used by both Fowler in Stages of Faith and Kegan in the subject object interview [5]. Both researchers relied on what we call the “research interview” to determine the stages of faith (or, in Kegan's case, “forms of mind”) present in the individual.

Figure 1. A depiction of a framework for designing faith informatics (FI) systems to support the elicitation and reflective revision of mental models of one's faith. The FI system prompts the user to systematically reflect on themself in the style of Fowler's faith development interviews [1]. This elicitation is combined with objective data about the individual's life and presented to the individual in a visualization. This visualization is then used to reciprocally influence the mental model the individual holds of their faith, facilitating development into further stages of faith.

The interview methodology relies on trained interviewers to communicate with and assess subjects using semi-structured interviews. Questions that might be asked in the interview include (selected from [1]):

• Thinking about yourself at present. What gives your life meaning? What makes life worth living for you?
• At present, what relationships seem most important for your life?
• Have you experienced losses, crises or suffering that have changed or ``colored'' your life in special ways?
• What experiences have affirmed or disturbed your sense of meaning?
• In what way do your beliefs and values find expression in your life?
• When life seems most discouraging and hopeless, what holds you up or renews your hope?
• What is your image (or idea) of mature faith?

One critical insight is that this methodology, by asking these deep questions, appears to support development by itself. Jennfier Garvey Berger notes that individuals who undergo the subject-object interview “changed the way they were thinking about things in their lives” and wanted to “come back for another interview” [6]. Some individuals reported making significant life changes after the interview, such as leaving an unhealthy relationship [6]. Fowler, using his faith development interview, also notes that interviewees tend to say things along the lines of “I never get to talk about these kind of things” [1].

HCI researchers and practitioners can help make these “developmental interview” experiences more available to the general public. The subject object interview is noticeably costly, both in time required for the conducting of the interview (60 to 75 minutes per subject), as well as required training for the researcher. One potential avenue for faith informatics, therefore, is to attempt to recreate the essential conditions of developmental interviews, allowing for the deep reflection that occurs during the interviews. 

For example, mobile or Web apps could attempt to recreate the conditions of a conversation that promote self-reflection and capture the outcomes of the reflection. This approach would be aligned with the notion of reflective informatics, which seeks to support reflective practices through technology [7]. Research has already highlighted the promising effects of this kind of digital intervention. The “self-authoring” application, aligned with Kegan's forms of mind (particularly the self-authoring form) has been shown to improve academic performance, reduce gender and ethnic minority gaps, and improve general student outcomes through “future authoring.” 

In Figure 1, the hypothetical faith informatics system provides prompts to individuals about aspects of their life related to meaning-making. These prompts may draw from both Kegan’s and Fowler's developmental interviews. In this particular diagram, we use the example from Fowler of eliciting the “life review,” which seeks to break the life history into episodes of meaning [1], and allows for reflection on each stage in life. This is, in essence, a large-scale version of the day reconstruction method—a kind of life reconstruction method—where individuals can break their life up into discrete episodes that mark turning points in their development of meaning-making systems. We envision a role for interactive systems in providing an interface for eliciting the construction of these episodes and any associated metadata (e.g., real-world context).

This is one of the key benefits afforded by an informatics system: the possibility of incorporating real-world, objective data in these reflective dialogues. As an individual’s system of meaning-making would be used for both answering prompts and governing an individual's behavior, the system can play a role in helping an individual compare their own perception of their meaning-making structures with how they (objectively) live their life. For example, life episodes could be colored by social contacts elicited from text messaging or email data, or behavioral activity derived from calendar entries or financial activities. 

Faith informatics would therefore also connect with the field of personal visual analytics, where an individual’s data (coming directly from the experiences described earlier) is visualized into an external representation that can enable the individual to confront their mental models of themself and of their life, potentially resulting in reciprocal feedback loops to prompt insights about and support faith development. The design and creation of these visualizations is an open challenge and may benefit from co-design activities and think-aloud visualization interaction studies.

We might expect that an individual’s current stage of faith would inform design decisions. For example, an individual transitioning into stage 5 (conjunctive faith) may benefit from exploring objective data about their life history as they “reclaim and rework” [1] their past.

Facilitating social connections via faith informatics

Another promising avenue for faith informatics is that of fostering social connections that span both religious and non-religious faith traditions. While Fowler's stages of faith are largely based on Western religious traditions, it is possible (and perhaps likely) that similar developmental structures of meaning-making exist across religions and traditional beliefs worldwide, given the focus on structure of faith instead of contents of faith [1]. As such, faith informatics could facilitate the connection of individuals based on stage of development, rather than relying on religious communities that may suffer from the limits of particular modal developmental levels. Such a system could help to connect individuals in similar stages of faith across different cultures, facilitating connection and development, perhaps via the sharing of narratives and experiences. 

Challenges and future work in faith informatics

Faith informatics is ripe with challenges and opportunities for the HCI community. One significant challenge is in the visualization of representations of faith that facilitate systematic reflection about an individual’s meaning-making structures. While we can draw upon Fowler’s research interview methodology to understand the types of prompts that may facilitate individual faith development, this kind of data (to our knowledge) has not previously been explored with contemporary visualization techniques. We expect that advances in personal visual analytics are necessary to support the effective visualization of self-reported faith data in a way that promotes development. 

In addition, the HCI community must confront the undigitizable nature of faith and spirituality, especially with regard to users in the later stages of faith. Fowler only encountered one individual in stage 6, the universalizing faith stage. It may be difficult to attempt to reify late stages of faith in a digital system, especially when the presence of these stages is limited to few individuals. Future work could include in-depth interviews with individuals in these advanced stages of faith to better understand their life trajectory, in hopes of better understanding and sharing how they reached these particular forms of meaning-making.

Endnotes

1. Fowler, J.W. Stages of Faith: The Psychology of Human Development and the Quest for Meaning. HarperCollins, New York, NY, 1981.

2. Smith, W.C. Faith and Belief: The Difference Between Them. OneWorld Publications, Oxford, U.K. 1998.

3. Fowler, J.W. Weaving the New Creation: Stages of Faith and the Public Church. Wipf and Stock Publishers, Eugene, OR, 2001.

4. Mekler, E.D. and Hornbæk, K. A framework for the experience of meaning in human-computer interaction. Proc. of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 2019, Article 225; https://doi.org/10.1145/3290605.3300455

5. Kegan, R. In Over Our Heads: The Mental Demands of Modern Life. Harvard Univ. Press, Cambridge, MA, 1998.

6. Berger, J.G. Using the subject-object interview to promote and assess self-authorship. Development and Assessment of Self-Authorship: Exploring the Concept Across Cultures. M.B. Baxter Magolda, E.G. Creamer, and P.S. Meszaros, eds. Stylus Publishing, Sterling, VA, 2010, 245–263.

7. Baumer, E.P.S. Reflective informatics: Conceptual dimensions for designing technologies of reflection. Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, 2015, 585–594; https://doi.org/10.1145/2702123.2702234

---

Posted in: on Mon, July 25, 2022 - 3:23:37

Michael Hoefer

Stephen Voida

Robert Mitchell

Authors: Stephen Gilbert
Posted: Wed, July 20, 2022 - 12:08:28

---

I use the term stream switching to refer to people simultaneously processing multiple streams of input information, each of which has its own context and background knowledge. This definition sounds similar to multitasking, but multitasking research usually focuses on a single individual, divided attention, and working memory capacity. Stream switching focuses on multiple people’s interactions and their mental models of each other. Below I offer examples and argue that stream switching merits further research. 

Stream switching

Stream switching draws on 1) the economic concept of switching costs and 2) the psychological concepts of perspective-taking and theory of mind. Economic switching costs are the financial, mental, and time-based costs to switch between products. The costs of switching from an Android to an Apple phone, for example, go far beyond the price tag and include learning how to use the phone’s software and much personal information transfer. Analogously, stream switching includes not only the basic attentional cost of focusing on a different input stream, but also the additional cognitive load of updating one’s mental model of the stream source. Is it accurate? Is it trustworthy? These questions relate to the psychological concepts of perspective-taking (Can you imagine others’ perspectives?) and theory of mind (Can you understand how different others’ knowledge and beliefs might be from yours?). 

Stream switching includes the more specific practice of code-switching. Code-switching was originally the linguistic practice of switching between languages depending on your context, and now refers more generally to the switching of identities depending on who’s around you. Someone might behave one way at home with family and another way in the outside world. Minorities often become quite skilled at code-switching since they have daily practice working within a different majority culture. I would hypothesize they are also highly skilled at stream switching. 

The idea that people vary in their stream switching ability is one of the reasons stream switching deserves more research. Can this critical skill be practiced or trained? We already know that some people are better than others at empathizing, understanding what other people are thinking and feeling. Research has correlated these individual differences with factors including reading more fiction, role-playing and reflection, the general practice of thinking more about one’s thinking. Perhaps this research could be extended to develop methods to measure one’s stream switching ability and methods of improving it. 

User experience (UX) implications

In user-centered design, we try to empathize with our users. We create personas to reduce our cognitive load of doing so. Here is Maria, the young professional with two children; what are her jobs to be done when she opens our app? A product designer with better stream-switching skills will truly be able to step into Maria’s world and build an accurate mental model of her goals and expectations in order to design a product that fits perfectly. That product then allows Maria to accomplish her goals more quickly with fewer errors. 

On the other hand, low-usability software requires the user to switch streams mid-use to imagine the intentions of the software designer or to model the inner workings of the software itself. You may have experienced an accounting system that was well-designed for accountants but not for you. While the accountants can easily track expenses, you can’t figure out how much money you have. Or consider a social media system. If you post certain content, is it clear who will see it? Will it lead your friends to see related ads? Or in a corporate calendaring system, if you invite people to a meeting, is it clear who will see the invite list? Being able to answer these questions requires highly usable software. Norman warned about gulfs in evaluation; today’s sociotechnical context requires evaluation of not only the system, but also of other users’ experiences with it. 

Frustrating usability situations increase stream switching, which burdens our cognitive capacity. When you have to switch streams to figure out where to click next, you have less attention to devote to the streams you were already juggling, for example, home life versus work life, or your supervisor’s mindset versus your teammates’. Bad interaction designs effectively steal our attention. 

Asymmetry, monitoring, and conflicting versions of truth

Many systems present asymmetric information to users, i.e., collaborators receive different information or levels of access, as in the calendaring example. It’s not always a problem; a person presenting slides should be able to see their presenter notes while the audience should not. But when you’re Zooming and ask, “Can you hear me?” or “Can everyone see my screen?”— that’s problematic asymmetry. You don’t have good cues about what other people are experiencing, so you have to ask explicitly. Having this information enables you to update your mental model of your colleagues’ experience in the meeting and enables you to stream switch more smoothly between conveying your points and monitoring your colleagues’ understanding. 

Asymmetric stream switching also appears if you use a virtual reality headset. When you enter the virtual world, you partially blind yourself to cues in the real world. To avoid tripping over a nearby chair, you need to switch streams consistently to monitor both the virtual world and the real world. If you carefully arrange the furniture in your room beforehand, creating a safe space for virtual exploration, then you can reduce the mental workload required to monitor that stream and focus more on the virtual stream with less overhead. 

Analogously, as supported by research on stereotype threat and Goffman’s idea of roles that people play in life, when people have a metaphorical “safe space” for collaboration with others despite diverse backgrounds, they can focus less on monitoring the stream of how they’re being perceived (“Will I say something offensive?”) and more on the stream of the collaborative task at hand. 

Finally, consider the number of people who have difficulty speaking with each other because of their dramatically different beliefs about what is true. This is a stream-switching problem with high asymmetries. It has become more difficult than ever before to imagine what it’s like to be on the other side, because the affordances to do so barely exist. Only by investing significant time in creating alternative social media accounts and filling them with clicks could someone start to experience the other’s perspective. The cognitive load of stepping into the shoes of the other person has become so high that it’s easier to discount them as foolish or deceived. It’s easier to stream switch and model the perspective of someone similar to yourself. In part, that’s why similar people are drawn to one another. But there is a very high likelihood that many people we work with, as well as our customers, are not similar to us. Research on the “contact hypothesis” shows that talking with people who are different than ourselves enables us to understand their perspectives with more empathy, even if it’s difficult. More than ever before, we need to increase our stream-switching abilities, which will enable us to understand others. 

Thanks to Joanne Marshall and Kaitlyn Ouverson for thoughtful feedback on these ideas. 

---

Posted in: on Wed, July 20, 2022 - 12:08:28

Stephen Gilbert

Authors: Sara Wolf, Simon Luthe, Ilona Nord, Jörn Hurtienne
Posted: Tue, July 19, 2022 - 9:45:16

---

The past two years of living in pandemic times have accelerated the spread of technology into all areas of life. This was also evident in the context of religious communities and churches, where the number of applications and users has increased enormously. Not only individual communities but also the great church institutions had to expand their presence in the digital sphere [1]. As a result, interaction with technology in religious and spiritual contexts is now more widespread than even a few years ago. Understanding how technology and interaction design influence experiences in such contexts is more important than ever. However, this increased need for knowledge is not yet visible in HCI publications. We also believe that through more research in these areas, HCI can gain new perspectives on technology use, design, and evaluation more generally. Similar to how work on religious objects in households inspired a broader call for extraordinary computing [2] , we would like to introduce a theme that emerged from our work with Protestant believers, and that can bring new impetus to HCI: unavailability.

We derive this claim from a continued cooperation between HCI researchers and Protestant theologians. Together, we have been working on several projects that aim at designing technology for religious communication in the form of rituals, blessings, and online worship services. In the following, we want to demonstrate that integrating aspects of faith, religion, and spirituality in HCI might be valuable and lend HCI new perspectives.

Unavailability

The development of current technology is about making everything available at any time: Vast amounts of music and films are available through media streaming services, our loved ones are available through video (chat), and worship services are available online. In most of the Western world, many of our desires can be fulfilled immediately using technology, which focuses on making everything visible, accessible, controllable, and usable [3]. However, this ubiquitous availability might not always be valuable. Sometimes the opposite, unavailability, might be the better choice. Unavailability can highlight what one values most about what is available and can evoke the experience of resonance, specialness, or meaning. In the following, we will present two examples that demonstrate how we came across the theme of unavailability in our research.

The first example originates in our work on blessings. In a design probe study with Protestant believers, we tried to understand what blessing experiences are, and where or when they happen in believers’ everyday lives. Participants described that the feeling of being blessed can occur anytime, anywhere, but is most intense when it is unexpected and surprising (i.e., unavailable). One participant shared the following story when asked to describe an experience of being blessed:

I had a conversation with a friend who told me about her happiness as a mother, how it was to hold her newborn baby in her arms for the first time, how much love she was surrounded by, and how proud she was. And that was very strange for me because she had to deliver the child dead. And, um, I didn’t expect that. And at that moment, well, that was so.... so that overwhelmed me…. So she knew her child would be born dead, she knew she would have a silent birth, and yet there was a lot of pride and happiness and love, and she is still proud to be a mother, even though her child was born dead. And I just find that..."Wow"! So my rational brain said, "Well, that cannot be for real, that doesn’t fit," and I was also afraid of the conversation with her. Um, and then I was, so that’s what got me… So that was surprising, yes, or maybe also what I hoped for.  So sometimes it [the blessing] is also a fulfilled hope.

Not all examples of blessing experiences were as drastic as the one described here. However, this story demonstrates the aspects of unexpectedness and surprise very well. The participant did not expect that the conversation with her friend could take place in a positive atmosphere—she was even afraid of the conversation. And then everything turned out quite differently than expected. She could not have worked out this twist or influenced the situation in this direction with certainty—it simply came as it came. The unavailability was also evident in other examples within the same study. Many participants described that they used to bless each other, although they can never be sure whether the blessings are effective—it is beyond their control. For our participants, Protestant believers, this control was attributed to God. The aspect of unavailability generated friction and excitement in people’s experiences: It opened up room for hope, speculation, and surprise—for example, when something absolutely unexpected and positive happens.

Our second example on unavailability shows the opposite: namely, what happens when the unavailable becomes available? In another project, we investigated the experiences of online worship services during the pandemic [4]. We accompanied Protestant believers while participating in online worship services and tried to understand how specific design elements lead to specific experiences. One prominent element that influenced the experiences dramatically was availability and ease of access. Usual worship services are not an everyday occurrence for believers, but rather something special; believers usually invest some effort to mark the worship service as distinct from everyday life and routines—for example, dressing up, going to a special place, and reserving the time to attend. In contrast, online worship services are available anytime and anywhere, which invites specific modes of usage (e.g., watching it on the side). 

One couple reported a situation that shows the tensions such constant availability can create. On one Sunday, the couple woke up later than usual, and were in the middle of their breakfast when realizing that the worship service was about to start. Invited by the flexible and accessible design of current online worship services, they watched it using a laptop at their breakfast table. Although this was practical, they quickly became annoyed with themselves. They realized that they had turned what formerly had been an extraordinary experience into something ordinary. Availability changed the way worship services were experienced. The online worship service turned into something everyday and less essential. Constant availability may be convenient and allow for flexible access. However, convenience and flexibility are nothing compared with the cherished unavailability of worship services that take place only at a specified place and time and are unavailable in between.

So far, unavailability seems to be a concept that is given little consideration in HCI, and that even opposes current trends of making everything available. The two examples show how unavailability affects experiences. We think it is worth looking at the concept more closely, as it can reveal new perspectives on technology design. In the following, we will turn to sociology and Protestant theology in order to learn more about the concept of unavailability. Theology has long been concerned with unavailability, and sociology shows how the concept of unavailability is essential for human experiences beyond the context of religion, faith, and spirituality.

The German sociologist Hartmut Rosa has studied unavailability (German: Unverfügbarkeit; he translates it as “uncontrollability”) in his works [3,5]. Rosa describes our time as a time of acceleration, suggesting the concept of resonance as a possible solution [5]. For Rosa, resonance is a type of world relationship formed by affection and emotion, intrinsic interest, and the expectation of self-efficacy, in which subject and world connect and at the same time transform each other. That is, the nature of the world relationship is to be understood as reciprocal. Not only is the relationship defined between subjects and objects, but they also define a new relationship to the world [5]. The experience of resonance is opposite to the experience of alienation, a world relationship in which the subject and the world are indifferent or hostile (repulsive) to each other and thus inwardly disconnected from each other—a relationship of "relationshiplessness" [5]. For Rosa, resonance is the human motivation that guides all actions. A central, constitutive aspect to resonant experiences is unavailability. Four conditions for resonant experiences must coincide [3]:

• Touch (something touches me)
• A response to the touch
• Transformation: a change of world-relationship
• Unavailability.

Even if conditions one to three are fulfilled, unavailability is necessary for a successful, resonating experience. The individual experience of the world can be neither planned nor accumulated. This perspective highlights a fundamental problem with the current focus on making everything available through ubiquitous technology: It is not the availability that renders experiences successful, resonating, and thereby valued but rather their specific quality. And part of what makes their quality is that people are not in control of everything and cannot make the world available to the last. It is precisely in this that Rosa sees a necessity. Space must be given to the concept of unavailability because only in this way are resonating experiences possible [3].

Regarding Christian religion, the necessity of the unavailable for a successful world experience as described by Rosa becomes particularly clear. All objects of the Christian religion, such as God, Christ, the Holy Spirit, grace, living a fulfilled life, and blessings, cannot be made controllable to human beings; they cannot be commanded. Even in an increasingly secularized world, the objects of religion and their unavailability remain something that fascinates and attracts people—albeit no longer only in the forms of the established religious communities. This search for meaning is both an attempt to make the unavailable available and the realization that ultimately unavailability is constitutive for religious experiences. It is precisely this unavailability that makes dealing with the objects of religion interesting to people. If God, the Holy Spirit, or Christ were made available, religion would become uninteresting and lose relevance for the resonant experiences as illustrated above.

The theme of unavailability prompts HCI to reconsider current trends of making everything available. Recognizing that unavailability might be an essential experiential quality, HCI is challenged to engage in the topic. How can unavailability be experienced when interacting with technologies? What ways, if any, are there to design for the unavailable? 

To design or not to design?

Although there seem to be no simple, singular answers to the above questions, we would like to present different perspectives to stimulate discussion within HCI. 

Unavailability is a key topic in the Christian religion and tradition. As such, the Christian tradition is constantly confronted with unavailability and tries to create conditions to make experiences with the unavailable more probable. Such efforts can, potentially, be considered as design. To this end, the Christian tradition offers (or designs) activities such as rituals, liturgy, and experiential education that serve as supportive measures, knowing that the unavailable is ultimately unavailable.

On the question of design for unavailability, Rosa builds his argument for resonant experiences. He argued that resonant experiences can arise only with counterparts (e.g., human beings, objects, nature, art) that are not entirely available—meaning visible, accessible, controllable, and usable [3]. Following this, Rosa doubts that technology can be designed at all to become a resonant counterpart: unavailability is uncontrollability and thus “non-engineerability.” He expects that translating unavailability to, for example, unpredictability in technology design might lead to frustration rather than resonant experiences. However, Rosa also identifies manufactured objects that evoke resonant experiences, such as poems or art. He expects a poem to be a resonant counterpart as long as one has not fully grasped and processed it, as long as it continues to occupy one and still seems to hide something [3].

With the above arguments and examples in mind, we turned to HCI searching for artifacts and design strategies that might correspond to unavailability. An artifact that shows an essential aspect of the integration of unavailability and technologies, namely the type of activities that a technology enables, is the Drift Table [6]. The Drift Table is a coffee table displaying slowly moving photography that is operated by the distribution of weight on its surface. One could argue that this interaction represents a kind of “control” of the table. However, what is most related to unavailability is not the exact interaction, but rather what the table encourages us to do: The Drift Table was designed not to perform specific tasks or efficiently achieve goals but rather to support ludic activities, “activities motivated by curiosity, exploration, and reflection rather than externally defined tasks” [6]. It is exactly in this way that we think it might correspond to unavailability: A technology that is supposed to have an inherent unavailability must invite us to explore it and not work through it—similar to a poem or a work of art. If it is at all possible to design for unavailability, then the first step must be a change of perspective on the what of technology: Away from technology as efficiency-enhancing task support, toward technology as a stimulus for open exploration, reflection, and curiosity.

Three design strategies that can serve as a useful starting point for thinking about design in the context of unavailability. Left: The drift table as an example of design for ludic activities [3]. Middle: The Projected Realities bench as an example for ambiguous design [2]. Right: ThanatoFenestra as an example for ephemeral interfaces [1]. Sketches: Vyjayanthi Janakiraman.

Apart from this concrete example, we also found two particular design strategies within the more speculative and exploratory areas of HCI that are more concerned with the how of interaction suitable when designing for unavailability. The first strategy is to design for ambiguity [7]. Gaver et al. [7] suggest that ambiguity “frees users to react to designs with skepticism or belief, appropriating systems into their own lives through their interpretations.” One way to integrate ambiguity into the design is to distort displayed information to stimulate curiosity and thought. The second strategy to design for unavailability might be ephemeral interfaces [8]. Ephemeral interfaces consist of at least one element that lasts only for a limited time and typically incorporate materials invoking multisensory perceptions such as water, fire, or plants [8]. Ephemerality is a design strategy seldomly used in popular, widespread technologies. However, it is an element everyone recognizes from the living, natural world that corresponds well with unavailability. 

In conclusion, we hope to have demonstrated the value of turning to contexts such as spirituality, faith, and religion to gain new perspectives for HCI and hope to stimulate discussions on making everything (un)available. We conclude with one last quote for reflection:

We should…speak of God. But we are human beings and as such cannot speak of God. We are to know both, what we ought to and what we cannot do, and precisely in this way give glory to God. –translated from Barth (1929)

Acknowledgments

We would like to thank all of our participants who shared their experiences with us. May you be blessed! Parts of the research have been funded by the Federal Ministry of Education and Research (BMBF), project CoTeach (project number 01JA2020).

Endnotes

1. Nord, I. and Adam, O. Churches online in times of corona (CONTOC): First results. In Revisiting the Distanced Church. OAKTrust Digital Repository. 2021, 77–86. https://doi.org/10.21423/revisitingthechurch

2. Wyche, S.P. and Grinter, R.E.. Extraordinary computing: Religion as a lens for reconsidering the home. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2009, 749–758; https://doi.org/10.1145/1518701.1518817

3. Rosa, H. The Uncontrollability of the World. Polity, 2020. 

4. Wolf, S., Mörike, F., Luthe, S., Nord, I., and Hurtienne, J. Spirituality at the breakfast table: Experiences of Christian online worship services. CHI Conference on Human Factors in Computing Systems Extended Abstracts. ACM, New York, 2022; https://doi.org/10.1145/3491101.3519856

5. Rosa, H. Resonance: A Sociology of Our Relationship to the World. Polity, 2021. 

6. Gaver, W.W. et al. The Drift Table: Designing for ludic engagement. CHI '04 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, 2004, 885–900; https://doi.org/10.1145/985921.985947

7. Gaver, W.W., Beaver, J., and Benford, S. Ambiguity as a resource for design. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2003, 233–240; https://doi.org/10.1145/642611.642653

8. Döring, T., Sylvester, A., and Schmidt, A. A design space for ephemeral user interfaces. Proc. of the 7th International Conference on Tangible, Embedded and Embodied Interaction. ACM, New York, 2013, 75–82; https://doi.org/10.1145/2460625.2460637

---

Posted in: on Tue, July 19, 2022 - 9:45:16

Sara Wolf

Simon Luthe

Ilona Nord

Jörn Hurtienne

Authors: Hawra Rabaan, Lynn Dombrowski
Posted: Thu, July 14, 2022 - 9:28:33

---

As HCI matures into a richer discipline interlacing with the humanities and social sciences, there has been a growing consciousness to embrace pluralist [1] and intersectional approaches in understanding and addressing systems of oppression within computing [2]. Islamic feminism and intersectional feminism are highly complementary approaches to understanding oppression and power. While both focus on gender, each brings its own distinct attention to how gendered violence manifests. In research, Islamic feminism is a theoretical, analytic, and design lens to understand and attend to the needs of Muslim women. Beyond being useful for understanding issues that only Muslim women face, as a theoretical approach, it expands how we understand agency, sheds light on the historic and sociocultural contexts, and explores design around cherished and socially familiar values [3,4]. This article calls for expanding HCI feminist theories to account for nonsecular and non-Western contexts, and answers the following questions: 1) What is Islamic feminism? 2) Why should we as academics care about it? and 3) How can we bring it into HCI research and design? 

What is Islamic feminism? 

Islamic feminism scholarship, similarly to mainstream Western feminism, centers studying gender and systems of power and oppression. Primarily, Islamic feminism differs from mainstream feminism in 1) its perception of religion and 2) definitions of agency and resistance. While there are many dangerous tropes portraying Islam as anti–female empowerment or as patriarchal, within Islamic feminism, Islam is viewed as feminist for several key reasons. Islamic feminists see Islam as a way to promote social justice and equity; however, Islamic feminists are critical of orthodox interpretations of Islamic texts and cultural praxis and often position them as the sources of patriarchal beliefs [5]. Thus, Islamic feminist scholars (e.g., Amina Wadud, Husein Muhammad, Ziba Mir-Hosseini) have worked extensively and continue to produce alternative interpretations of sacred texts to challenge the patriarchal elements in Islamic jurisprudence and shift the paradigm of religious authority. For example, verse [4:34] is commonly used by religious scholars to promote wife-hitting, whereas Islamic feminist scholars interpret the verse to encourage temporary separation and reflection. The interpretation is inferred from a holistic Quranic stance on spousal relations rather than verbatim explanations, concluding its focus on respect, reconciliation, and justice, and consequently opposing harm and violence [4].

The second dispute between mainstream feminism and Islamic feminism is how agency is defined and identified. In mainstream feminism, agency is fundamentally based on the liberal political theory’s concept of freedom, where an autonomous will is fulfilled through “universal reason,” unburdened by tradition, customs, or transcendental will. Islamic feminists view this definition of free will as limited; instead, agency is not just about a person’s capacity, but rather about how societal, economic, and political structures create and reinforce conditions, cultural norms, possibilities, and oppression around gender [6]. In Islamic feminist thought, resistance and agency are often inseparable and extend agency to include non-resistive actions. For example, a woman in an abusive relationship patiently remaining married is an autonomous choice made within layered constraints that takes time, careful consideration, and effort [4]. Another form of agency that is regularly overlooked by Western feminists is religious agency, which consists of acts grounded in religious beliefs—to adhere to a transcendental power (i.e., their God) rather than to the abusive figure or patriarchal norms. Lastly, using an Islamic feminist lens reveals more depth of the human experience and urges us to practice greater empathy and reduce harm, by acknowledging and working within the practical conditions influencing participants’ autonomy. 

Why and when to use Islamic feminism?

The HCI community centers diversity, equity, and inclusion. Islamic feminism provides tools to conduct justice-oriented research and design, where participants are not “othered” or looked down upon, where we value our participants’ voices, and build upon community assets and values. As an HCI scholar, I call for us to turn inward and surface the biases we may have as researchers and designers, find and routinely use tools to help us overcome our biases, and bring about the progressive change we seek as academics and global citizens.

As the field continues to broaden its reach and impact, the problem-solving standpoint familiar to the HCI community must shift. By using an Islamic feminist approach, we strive to provide implications and design within the context of entangled political and cultural norms, rather than eradicate those norms or perceive our user as a passive victim [4]. 

How to use Islamic feminism in HCI 

Next, we leave you with three takeaways on how to begin following an Islamic feminist approach in HCI: 

• Dare not to take on a savior’s complex, orientalize, or deem cultures as less than the West. When approaching social issues that concern sensitive problems and vulnerable populations, be diligently cautious of causing more harm, whether it’s in perpetuating stereotypes, interacting biasedly, or superficially tackling the problem. 


• View practicing agency as culturally and historically specific. It is the researcher’s job to acutely understand the practical conditions and sociohistorical processes contributing to the autonomy of participants. Such lenses are essential within all roles of research; problem formulation, analysis, implications and design, and research evaluation [7].


• Call for empowerment from within the participants’ structure, by designing around cherished, internalized, and socially familiar values when evaluating current technologies or providing implications for future inclusive designs [3]. Include norms beyond social impositions, and focus on the participants’ cultural dimensions, historical developments, and different ideas of justice.

While it goes without saying that Muslims are not a homogeneous group, portraying Islam as a violent religion or Muslim women as passively oppressed is inexcusably prevalent within the broader society and this unfortunate stereotype has not escaped the academic realm. I have faith in our community and the future of inclusive technologies. When in doubt, connect with fellow Muslim scholars or groups such as the IslamicHCI group [8], who will be open to sharing their insights when needed. We can do better, HCI! 

Endnotes 

1. Bardzell, S. Feminist HCI: Taking stock and outlining an agenda for design. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2010, 1301–1310.

2. Rankin, Y.A. and Thomas, J.O. Straighten up and fly right: Rethinking intersectionality in HCI research. Interactions 26, 6 (2019), 64–68.

3. Alsheikh, T., Rode, J.A., and Lindley, S.E. (Whose) Value-Sensitive design: A study of long-distance relationships in an Arabic cultural context. Proc. of the ACM 2011 Conference on Computer Supported Cooperative Work. ACM, New York, 2011, 75–84.

4. Rabaan, H., Young, A.L., and Dombrowski, L. Daughters of men: Saudi women's sociotechnical agency practices in addressing domestic abuse. Proc. of the ACM on Human-Computer Interaction  4, CSCW3 (2021), 1–31.

5. McDonald, L. Islamic feminism. Feminist Theory 9, 3 (2008), 347–354; https://doi.org/10.1177/1464700108095857

6. Mahmood, S. Feminist theory, agency, and the liberatory subject: Some reflections on the Islamic revival in Egypt. Temenos-Nordic Journal of Comparative Religion 42, 1 (2006).

7. I would like to take the opportunity to express my gratitude to the CSCW reviewers who pushed my work into maturation by providing constructive criticism and actionable feedback.

8. Members of the group can be reached at [email protected]

---

Posted in: on Thu, July 14, 2022 - 9:28:33

Hawra Rabaan

Lynn Dombrowski

Authors: Nusrat Jahan Mim, Syed Ishtiaque Ahmed
Posted: Fri, July 08, 2022 - 12:17:25

---

Cities have long been a center of attention for modernization projects, so it is no surprise that most of today’s computing ventures are choosing cities as the main avenue for demonstrating their prowess. The idea of a “smart city,” for example, has been attracting a wide range of academics and practitioners in computer science and related fields whose works concern sensor systems, smartphone applications, human-computer interaction (HCI), computer supported cooperative work (CSCW), social computing, ubiquitous computing, data science, and artificial intelligence (AI). Related to the dream of smart cities are the dreams for driverless cars to claim the city streets, drones to deliver food to city apartments, and large-scale smart displays to inform and entertain citizens.

These dreams are also being partially realized in various cities around the world, with a mix of successes and failures. In fact, it would not be an overstatement to say that computing technologies often define what cities are in today’s world. While the concentration and intensity of dazzling, cutting-edge computing is generally higher in cities in the Western world, their Global South counterparts have started to catch up. Under the government mandate of digitization, countries like Bangladesh and India, for example, are updating their cities and the lives of their citizens with new mobile applications, biometric identity, the digital gig economy, online education, and social media, to mention just a few. However, this rapid advancement of “urban” computing is often devoid of ideas of faith, religion, and spirituality; even worse, their insensitivity to such sentiments is creating an agonistic and uncomfortable situation for many citizens, especially in the Global South. A deep understanding of the historical conflicts between modernity, science and technology, and urban religiosity is required to trace the role of computing within this tension.

Let’s go back a couple of years, when Covid started to unfold in Bangladesh. Alarming images of Bangladesh shared on social media included photos of religious gatherings of faith-based communities. Here, we use a broad definition of religion that incorporates both organized religions (Islam, Hinduism, Buddhism, Christianity, etc.) and traditional and Indigenous beliefs and non-beliefs. By faith-based communities, we refer to the groups of people who participate in activities that are motivated by their religious beliefs. While faith may influence and shape every action taken by a person, for the sake of simplicity, we define religious practices by the activities that are driven primarily by religious motivations (e.g., prayers, congregations, etc.).

While the government of Bangladesh mandated “social distancing,” and there were nationwide campaigns to create awareness among people to avoid crowds, those suggestions were often not effective in practice, especially among faith-based communities. In many places in the country, including major cities, huge religious congregations kept taking place at mosques during Friday prayers, ignoring all cautionary notices. At one point, when the city authorities closed mosques and asked people to perform their prayers at home, many Muslims started to gather on their rooftops to perform their prayers (Figure 1) in jamat (groups)—the call for social distancing kept getting ignored. In mid-April 2020, thousands of people gathered at the funeral ceremony of a local influential religious leader in the Bramhambaria district, ignoring the strict ruling against such gatherings. Then, in the month of Ramadan, the nation feared a spike in the number of infected people as a series of religious festivals, including Shab-e-Qadr, Jumatul Bida, and Eid ul Fitr, were forthcoming. Those tensions between the city and religious communities allowed us to see the politics around the formulation of modern cities and the historical struggle of the religious identities within them.

While modernity is often defined by a set of values, processes, technologies, and the time period when scientific revolution, industrialization, and capitalism rapidly expanded across the West, urban modernity is characterized by the adoption of modernist ideologies in the urban built environment and as a tool to respond to emerging social and political tensions. Predominantly concentrated on the modernity-driven urban development strategies and planning policies engendered in the West, urban modernity is also globally understood as a product of secular institutions, practices, and discourses [1]. A major portion of this scholarship was developed in the 19th and 20th century, which read cities through the lens of scientific rationality and measured a city’s performance against a scale of “industrialized progress.”

With the theorization of the Global City in the 1980s, this new paradigm of modern urbanism started to expand beyond the West and influence the spatial, political, and economic readings of modern cities worldwide. These scholarly and design practices reinforced the qualities of Western modernity as the standards of a successful, global, modern city. In the late 20th and early 21st centuries, scholars such as David Harvey, Edward Soja, and Saskia Sassen mapped urban-spatial responses around global flows of capital, technology, information, and other streams, pointing toward the failures of such modernist advancement at different scales in an urban setting. Soja, Jane Jacobs, Susan Fainstein, Leonie Sandercock, Lisa Peattie, and many others shed light on different forms of social inequalities and injustices that were introduced by modern cities. Although it was not at the center of their criticism, the Western concept of modernity has also always upheld the idea of secularization, and there has been a predominant secular spirit among modernist planners and scholars toward understanding cities.

While the cities in the Global North have often relied on their secular institutions, one cannot understand the cities of the Global South, especially South Asian and African cities, without understanding their religious spirit. However, the externalization of religion to the core discourse characterizes the development of modern South Asian and African cities of the recent past [1]. Hence, the cultural pluralities and the alternative visions of modernity that could develop in this part of the world, which have the potential to generate a dominant body of knowledge in urban studies, are often marginalized [2]. Such marginalization not only hampers the tolerance and productive public engagement of people from different religious backgrounds [3] but also limits the scope of “vernacular” development in the cities of the Global South.

 
Figure 1. Religious people praying on the rooftop of a building after praying at the mosque was halted. (Image courtesy of Rubayet Tanim)

As a Global Southern city with a strong religious backdrop, Dhaka, the capital of Bangladesh, is often considered a “non-modern” city. To achieve the Global City aesthetics, Dhaka like any other developing city in many respects, attempts to abide by the “rules of modernity” that were developed in the West. Hence, the urban religions often remain outside the development discourse. The presence of religiosity therefore gets limited, neglected, and often ridiculed outside spaces of worship such as mosques, temples, churches, and pagodas. For instance, when a mosque in a neighborhood starts to be treated as merely another “building,” the much-needed socioeconomic and spatial connections between the city and the religion are overlooked. The associated design conversations fail to capture how the “donation box” placed at the entrance of a mosque and the business of informal vendors standing outside the entrance are deeply integrated with the complex network of the city’s economy. It fails to understand how a low-income rickshaw-puller feels when he wants to pray his afternoon prayer but can hardly find a secure place to park his rickshaw near the mosque. Hence, the conversations fail to address the challenge of better handling the traffic during any religious events within or around the city. Since Dhaka is a Muslim-majority city, we are focusing on examples of Islamic practices here. In reality, such spatial and socioeconomic marginalizations can be traced in other religious practices as well.

Modern cities also put limits on religious rituals, and this soon escalates to a contention between the city and its religious citizens. When religious communities feel that the city is being “developed” without including their input, they start developing resistance and try to hack into various spatial arrangements of the city to make their presence visible. Keeping urban religion outside the prominent design discussions results in different infrastructure-level failures as well. For instance, severe traffic gridlock occurs in Bangladeshi cities before Eid ul Fitr, when a huge number of rural people temporarily visit to collect zakat (financial help from the rich obligated by Islam). Similar spatial complications occur during the congregational religious event Biswa Ijtema, where millions of Muslims gather to empower their Muslim identities. If the city development plans incorporated such religious festivities with due significance, Bangladeshi cities like Dhaka would have been developed in a distinctively inclusive way.

The borrowed concept of modernity, on the one hand, has limited possible design conversations with faith-based groups. On the other hand, excluding religiosity from the dominant urban design and planning decisions has created additional pressure on the city’s infrastructure. When the connections between the city and religion become parochial, it becomes quite difficult to expect thoughtful, tolerant responses from the marginalized faith-based communities during emergencies such as the Covid-19 pandemic. Hence, every now and then we observe alarming crowds at religious gatherings in South Asian cities, defying lockdowns and mandatory social distancing. These crowds are not an immediate “unthoughtful” response to the pandemic situation, but rather simply the result of a deeply rooted resistance to the city’s “secular interventions.”

This brings us to the point where we can think of urban computing in a broader perspective. The discourse of “smart cities,” in both the West and the Global South, lacks a discussion on how to incorporate religious practices. If a religious practice is essentially communal, such as adhaan (Muslim prayers), how should the city strike a balance between the individual choice of not listening and the communal choice of hearing the sound? How is the city supposed to respond to the massive inflow of people, material, and livestock during religious festivals that challenge its infrastructure? If digitization develops a physical distance between people, how does that affect religious practices? How is the liberal, nonhierarchical model of online social media affecting value-laden religious communities? More importantly, how safe are the digital spaces for religious people? These and many other questions are critically important, and yet rarely discussed in the domain of urban computing.

We identify faith, religion, and spirituality as an underexplored territory of HCI in general, and urban computing in particular. Computing’s historical alignment with science, technology, and modernity has made it difficult to capture such sentiments, which are built upon belief, respect, tradition, fear, and purity. Furthermore, computing technologies, when acting as a force of modernization, often end up silencing and marginalizing the voices of religious communities by creating a “secular” space. However, a new paradigm of HCI can be imagined that challenges this top-down modernist program by strengthening the voices of marginalized religious sentiments.

Endnotes
1. Hancock, M. and Srinivas, S. Spaces of modernity: Religion and the urban in Asia and Africa. International Journal of Urban and Regional Research 32, 3 (2008), 617–630.
2. Simone, A.M. On the worlding of African cities. African Studies Review 44, 2 (2001), 15–41.
3. Greed, C. 2016. Religion and Ssustainable urban planning: ‘If you can’t count it, or won’t count it, it doesn’t count’. Sustainable Development 24, 3 (2016), 154–162.

---

Posted in: on Fri, July 08, 2022 - 12:17:25

Nusrat Jahan Mim

Syed Ishtiaque Ahmed

Authors: Muhammad Adamu, Shaimaa Lazem
Posted: Tue, July 05, 2022 - 10:46:44

---

Last year, I read Shaimaa’s reflection on how Indigenous cultures and folk heritage may act as a steppingstone for sustainability in the absence of resources [1]. This led to extensive correspondence with her on how lessons from Saeed El Masry’s book [2] can reframe the common dualities of scarcity and sufficiency found in African perspectives on innovation in HCI. We pondered the question of how we, as researchers and practitioners, can challenge as well as celebrate the “lacks” and “gaps” connotations affixed to HCI development in Africa. This blog thereby offers a different reading of the meaning of terms such as limits and scarcity, showing how such terms entail considerable future thinking and planning efforts. Our reflection has more to do with the politics of naming and the power dynamics underpinning the internalization of such narratives in mainstream discourses, and less to do with the practices of design research in African communities where supposedly these limitations are amplified. We draw upon existing decolonial works that explore the politics of computing research within the African context [3,4].
 
It is our understanding that terms such as resource-challenged, resource-constrained, underserved, and underresourced are used to represent the unfortunate realities of global inequalities. Such terms in HCI discourse mirror a certain worldview of what constitutes scarcity and sufficiency in computing and are often associated with the limited adoption of modern state-of-the-art computing technologies (e.g., smartphones, AI, drones, VR). Our concern with such a way of thinking about Africa is that it unintentionally reproduces interventionalist narratives that suggest how the introduction of technological innovation can bring about economic and material prosperity. The use of the term introduction here is deliberate as it denotes how dominant social imaginaries have cemented the thinking that Africa is not a site of innovation, and that technocratic thinking and modern technologies are the only way out of its predicament. Our frustration here is that the common narrative of lacks and gaps reduces social life in African communities to a set of problems that need computational solutions, a view in which digital technologies are necessary to catch up or transition to Western ideals of progress. This way of thinking thus pushes for the politics of organizing knowledge against a backdrop of consumerist development models that are undesirable and unsustainable.

Additionally, the performativity of limits and scarcity centers certain kinds of utilities over others, thus bringing to the fore economic, infrastructural, and technological products and pushing to the periphery human capital, natural resources, and Indigenous knowledge with which Africa is notably wealthy. Arguably, foregrounding the former repeats simplified accounts of Africa as a historical site for the extraction/production of raw materials and Africans as passive consumers of processed end products. Such accounts of African users invited us to question the extent to which the limits of the computing technologies could have been misrepresented as the limits of its users. Imagine for a moment that, instead of text, early computer interfaces relied heavily on embodied interaction and voice as the main interaction modality; that would have been a better fit with the oral traditions of some African communities [5]. As such, literacies would not have been emphasized as a limitation in the way it’s been presented in mainstream technological discourse.
 
To provoke different thinking about limitations in Africa, research has shown how folk practices of food preservation were adopted in Cairo as a mechanism to plan and cope with food scarcity [2]. In some districts of old Cairo, ensuring food security involves a considerable amount of planning that must be dynamic and resilient in coping with the changes in the availability of resources (e.g., unfixed income). The number of daily meals, their timing, and the content of meals are planned so that, for instance, planning the biggest meals for dinner guarantees that the person will have enough energy to get through the next day without needing breakfast. The leftover food is used as a supplement for next-day meals and snacks, and recipes are made to keep expensive food items for a long time [2].

Another example of different thinking around limitations is the adoption of the cultural model of Igwebuikem (i.e., strength in numbers) in rebuilding the community wealth of the Biafran communities of Southern Nigeria [6]. Among the Igbo tribe, shared prosperity was considered a political instrument for futuring in the aftermath of the Nigerian civil war. The Biafran civil war brought about the need for building communal solidarity in coping with the dynamic situation of hardship and tribulation—from which the Igbo apprentice systems came about as a scalable entrepreneurship program that built upon the commonwealth of the community [6]. Through its adoption as a localized model for building business ecosystems, communities were able to identify the strength of the collective, learn from each other, and scale-up instruments for rebuilding community wealth. The point raised here is that conditions of scarcity demand situated knowing that does not rely on the limitations of the past but rather on the possibilities that a differentiated thinking about the present might offer to the performance of the future.  
 
From the two historical scenarios of future thinking in the Old Cairo and Biafran communities, one can identify the resourcefulness that accompanies a different reading of limits and scarcity, showing how communities continuously innovate new ways of balancing present living conditions with future ones. By bringing to focus the politics associated with limitation as a concept and a reality, we are seeking to problematize its current use in HCI projects taking place in Africa. Recent efforts in HCI have been critical of technology solutionism, highlighting the nuances surrounding the use of technology as well as the importance of considering contextual and cultural specificities in design (e.g., the work of Kentaro Toyama, author of the seminal book Geek Heresy: Rescuing Social Change from the Cult of Technology). We’re nonetheless concerned with how dominant HCI has framed interaction design from the Global South in relation to development, design, and context, which has led to alternative framings of African design in HCI (e.g., the postcolonial, Indigenous, and decolonial approaches to design). Following decolonial thinking, our reflection sought to elaborate on this concern by bringing to the fore the geopolitics of technology solutionism and the interventionist narrative associated with it as Western ideologies that reproduce the imaginary of primitiveness and backwardness in relation to African innovation, and where postcolonial baggage and the promise of pop development—or the development that doesn’t really develop—remain. We argue that a differentiated identity for African innovation does not require a distinction between the unacknowledged past and the unfortunate present but rather recognizes how the relations between the past and the present give rise to the future.

The message from this short reflection should not be understood as shying away from inequalities and masking struggles resulting from socioeconomic limitations. Rather, we invite ourselves and the HCI community to rethink the “incomplete” realities we might portray and perpetuate when we only focus on and communicate lacks and gaps to inspire research encounters for and with communities from Africa. As African scholars, our hope, by sharing personal reflections, is to encourage HCI researchers to rethink the terms used in research framing and dissemination in light of colonial histories and the realities of globalization. Our call to action is to encourage HCI researchers, ourselves included, to reflect on the limit of the “limitations” described in HCI research, and to exercise humility in articulating the potential mismatch between the users and the technology introduced to them, bearing in mind the context technology was designed from and deployed to, and the worldviews that might be represented.

Endnotes
1. Lazem, S. What are you reading? Shaimaa Lazem. Interactions 26, 5 (2019), 12–13.
2. El Masry, S. Reproduction of Folk Heritage: How Poor Cling to Life in the Context of Scarcity. Cairo Supreme Council of Culture, 2013.
3. Bidwell, N.J. Decolonising HCI and interaction design discourse: Some considerations in planning AfriCHI. XRDS: Crossroads, The ACM Magazine for Students 22, 4 (2016), 22–27.
4. Lazem, S., Giglitto, D., Nkwo, M.S., Mthoko, H., Upani, J., and Peters, A. Challenges and paradoxes in decolonising HCI: A critical discussion. Comput Supported Coop Work 31 (2022), 1–38.
5. Allela, M.A. Technological speculations for African oral storytelling: Implication of creating expressive embodied conversational agents. Proc. of the Second African Conference for Human-Computer Interaction: Thriving Communities. ACM, New York, 2018, 1–4.
6. Kanu, C.C. The context of Igwebuike: What entrepreneurship development systems in Africa can learn from the Igbo apprenticeship system. AMAMIHE Journal of Applied Philosophy 18, 1 (2020).

---

Posted in: on Tue, July 05, 2022 - 10:46:44

Muhammad Adamu

Shaimaa Lazem

Authors: Arathi Sethumadhavan , Joe Garvin, Benjamin Noah
Posted: Wed, June 22, 2022 - 9:57:27

---

Whether it’s the friendly virtual assistant in your smart speaker, the auto-generated captions on your YouTube video, or the software that physicians use to dictate clinical notes, voice AI has already become a fixture of modern life. It’s the promise of hands-free convenience: Simply speak naturally, and the computer listens, analyzes, and recognizes what you’re saying. With things like voice-controlled homes and cars on the horizon, our relationship with voice AI looks to only deepen. But the task of building speech recognition technology remains a tall order: We want it to work well for everyone, but there is no one way of speaking.

People speak differently depending on how old they are or where they live on a map but less obvious demographic factors like socioeconomic status and education can also play a role. Layers of intersecting variables all come together to influence how we verbally express ourselves. We humans use language like code-switching chameleons, reflecting and creating culture every time we talk. Speech, in other words, is so much more than a mechanical system of words. It is organic and fluid, forever entangled with identity, values, and self-expression.

So it makes sense that studying speech to improve AI models would be more than just an engineering job. In other words, teams grappling with complex problems involving people require diverse representation across disciplines. Working alongside engineers and data scientists, social science experts like sociologists, anthropologists, and sociolinguists can offer essential insight and context while navigating the intricacies of our many voices.

Lingering inequalities in voice AI

Use of computer speech recognition extends far beyond asking Alexa or Siri to play a song or ask for the weather. Court reporters use it to generate transcriptions; people with disabilities use it to navigate phones and computers; and companies use it to make hiring and firing decisions. As voice AI has proliferated in recent years, overall accuracy and reliability has improved dramatically. But even state-of-the-art speech recognition tech does not work well for everyone. 

A Stanford University study from 2020, for example, tested services from major tech companies used for automated transcriptions and found large disparities between ancestry groups. The cause? Insufficient audio data when training the models, the study suggests. In other words, the voice AI powering the services was trained on datasets that left out many ways of speaking. In addition to certain ancestry groups, speech recognition systems also struggle with the accents of non-native English speakers, regional accents, and voices of women.

Biased AI starts with a biased worldview

These divides in voice AI have been documented for years, and so have the data-collection missteps that perpetuate them. Why, then, is collecting enough of the right speech data such a stubborn problem? One factor at play here is the out-group homogeneity effect, the cognitive bias that assumes people unlike you are more similar to each other than they really are. “They are alike; we are diverse,” or so the bias would have you believe.

Especially when classifying language, the bias is insidious. Consider, for instance, how all people over 60 are often lumped together into a single group: “older people.” This broad category might be useful in some contexts, but relying on it when studying speech would be irresponsible. Data shows that the way we talk continues to change as we age into our 60s and beyond, and that it even changes differently depending on gender. Tremendous variation exists within the group “older people” that deserves attention. But if someone isn’t 60 or older themselves, out-group homogeneity bias might blind them to all that variation.

Even the terms commonly used to describe Black and African American language—African American Vernacular English (AAVE), African American English (AAE), or African American Language (AAL)—can themselves be seen as examples of the out-group homogeneity effect. In reality, a language variety is never exclusively part of an entire demographic group, and there could also be people of different ancestry groups who happen to speak similarly. When it comes to studying speech for voice AI, creating false monoliths out of certain subgroups isn’t just inaccurate, it’s dangerous. Glossing over meaningful differences in the ways people speak shuts them out of tomorrow and leaves their voices unheard.

The many nuances of language

Many different factors play into speech. Some might be obvious, like where you live or whether that’s your native language. But other factors like health, education, and even historical migration patterns also play significant roles in shaping how a person speaks. Social factors like these contribute to linguistic variations. Anthropological linguists go a step further, suggesting that these factors also actually construct social meaning. In this view, when someone speaks, their voice is doing so much more than simply reflecting their region or ancestry: It’s expressing an identity.

Our gender identity, for instance, can influence how we talk. Culture or ethnic association can also influence how our speech develops, how we use it, and how it may evolve. When we define a specific variation of speech, therefore, we must include these societal factors as its foundational pillars.

Level of fluency, education, gender identity, age language was learned—which of these many interconnected factors are the most decisive in shaping the ways we speak? It’s crucial information to have, as it quickly becomes unwieldly to account for all possible aspects that determine speech. When collecting samples of spoken language to train a new speech model, for example, there are real-world limitations affecting what can be collected: time, money, personnel, and geography, to name a few. Prioritizing all the social factors is a complex job, one beyond the narrow scope of any one discipline. 

A multidisciplinary approach

To build speech recognition technology that works well for everybody, we need to capture the right diversity in our data-collection strategies. This involves turning toward those nuances in language, being attentive and curious about them. We know that we want to capture an accurate picture of the incredible variety of human speech, and we also know that many complex dynamics are at play. This calls for a multidisciplinary approach for a better informed, more inclusive perspective.

An engineer might be able to notice the different word-error rates between demographic groups, for example, but a sociolinguist can help explain the different speaking patterns at play, how these patterns show up across communities, and historical reasons for why they emerged. A data scientist can tell you how many people in what groups need to be sampled. Sociologists, demographers, and anthropologists can speak to social behaviors and psychology, aspects that illuminate the subtleties of language. Domain experts like these offer invaluable insights and context, and involving them early on will help us design better datasets that capture human diversity.

Toward more equitable voice AI

Even with the help of other disciplines, building speech recognition systems is incredibly hard work. Training a voice AI model requires a huge amount of speech data. Collecting this data means bringing in lots of people from different population groups, some of whom are difficult to access and recruit, and some of whom, like Native Americans and the First Nations peoples of Canada, haven’t had their speech studied extensively. And when subjects are finally recruited, they need to be taken to a noise-controlled recording facility, asked to follow specific directions, and instructed to read aloud carefully designed paragraphs of text. 

The process of creating voice AI is painstaking and resource-intensive as it is—detecting and reducing bias on top of it all only makes the process more difficult. And yet, we must be up to the task. The fact is that speech recognition systems of today, trained on largely homogenous datasets, still don’t work for all groups of people. This is more than a matter of services performing poorly; it’s a matter of dignifying real ways of speaking, real identities and cultures. We must first acknowledge that this problem exists and educate teams about ways of building more equitable voice AI.  Then we need to act. Acknowledging the intricate social and cultural dimensions of speech, we might team up with experts from relevant disciplines. With the help of experts like social scientists, product teams are better equipped to think carefully about inclusive dataset design and to devise creative approaches to thorny data-collection obstacles.

Human speech poses engineering problems that transcend the technical. Building voice AI is in fact a sociotechnical endeavor, one that requires diversity in disciplines. The stakes are high, but with an intentional focus to seek out our blind spots, we can collaborate to build voice AI that truly works for everyone.

---

Posted in: on Wed, June 22, 2022 - 9:57:27

Arathi Sethumadhavan

Joe Garvin

Benjamin Noah

Authors: S.M. Rakibul Islam, Nabarun Halder, Ashraful Islam, Eshtiak Ahmed, Sheak Rashed Haider Noori
Posted: Fri, June 03, 2022 - 11:41:22

---

The sudden appearance of Covid-19 forced people to stay at home due its highly infectious nature. Governments shut down countries; economies and businesses collapsed. The education system was no exception; rising Covid-19 cases forced schools to shut down. According to UNESCO, Covid-19 afflicted 37,694,522 students as of January 2022 [1]. Since educational facilities halted in March 2020, Bangladeshi students have been deprived of adequate time for instruction and for connecting with classmates, both of which significantly harm their educational experience. Like many countries, Bangladesh had to move to an online education system [2]. While the online program was able to overcome some of the inevitable difficulties, many people questioned its efficacy and acceptability. Taking this into account, our research looks at the views and opinions of students regarding the efficacy of virtual classrooms and how these classrooms could be an alternative to a traditional, in-person classroom setting.

We used Google Forms to create an online survey about students' perspectives on online education [3]. To develop the survey, we carried out a literature review of related works and conducted informal interviews with students who attended online classes [4]. The survey consisted of both quantitative and qualitative inquiries centered around a set of four types of questions: participants' characteristics, how they attended online classes, their online learning platform experience, and their overall experience. The survey involved 210 students, with educational backgrounds ranging from high school to postgraduate. Of the participants, 88.6 percent were undergraduates and 92.9 percent were between 18 and 25 years old (Table 1). Most participants were from urban areas (63.3 percent). We performed a percentage analysis on the data from the survey using Google Forms' built-in features and Microsoft Excel.

Demographic Information	Characteristics	Number of Participants
Education level	High school/secondary (Grade 6 to 10)	2
	Higher Secondary (Grade 11 to 12)	7
	Undergraduate	186
	Postgraduate	15
Gender	Female	68
	Male	142
Age (years)	10 to 17	3
	18 to 25	195
	26 to 32	11
	33 to 40	1
Residence area	Suburban	34
	Rural	43
	Urban	133

Table 1. Demographic information of survey participants.

Poor Internet connectivity was one of the most significant challenges for online classes [5]. Our study reports that nearly half (45.2 percent) of the participants had poor Internet access. Most students used Android smartphones and mobile data for online classes (Table 2). However, a majority of students (84 percent) had no previous experience with online classes. Most of them preferred in-person classes to recorded ones. According to our findings, most participants who did not understand a topic in an online class asked the instructor questions. In contrast, many students who watched recorded videos didn't ask questions. Regarding relationships between classmates, about 80 percent of the respondents felt that online classes created distance between them and their classmates. There was a mixed response to teacher-student interaction.

Technical Information	Characteristics	Number of Responses
Devices used	Tablet computer	2
	iPhone/iOS smartphone	14
	Desktop computer	27
	Laptop computer	108
	Android smartphone	161
Internet accessibility	Public Wi-Fi	36
	Broadband connection	103
	Mobile Internet packages	127
Platforms used	University’s Blackboard	1
	Skype	2
	Microsoft Teams	6
	Facebook Live	11
	Zoom	71
	Google Meet	163
Communication with new classmates	Never communicate with them	60
	Personal message	91
	Instant messengers and social media groups	150

Table 2. How participants join online classes and interact with classmates.

Most participants stated that being unable to physically meet with teachers negatively affected their education. Many felt less motivated to discuss any topic in an online class. As a result, students were less eager to discuss any issues with the teacher after class. In terms of changes in instructions in the online class, 50.95 percent of participants expressed "instructors spend less time on guiding students on changes in directions in the online class" and "instructors spend less time explaining topics"; 36.67 percent stated that "instructors take fewer feedbacks"; and 45.24 percent of participants agreed with the following statement: "To demonstrate a topic, instructors use fewer visuals." Just over 41 percent of students were neutral about changes in lecture materials (Figure 1).

Figure1. Participants’ votes on how lecture materials and directions are changed.

Participants did not have a clear preference about where they took tests, but they had some complaints about the testing process. They stated that open-book exams caused teachers to generate complex tasks that might take longer to solve than the time given for exams, and that teachers could not evaluate their efforts properly. In addition, poor Internet connectivity and the inadequacy of devices were significant barriers to students participating in online tests. The main complaint about online classes and tests was that students could not concentrate for a long period of time. Students also mentioned falling asleep during classes and noisy environments while attending classes at home. In terms of class participation and performance evaluations, most students preferred assessment at the end of the week. Still, many students appreciated checking in for each class; some also mentioned that they enjoyed the assessments at the end of the course or semester. A few of the main complaints from students using online learning systems were that teachers used fewer images to demonstrate concepts, spent less time explaining subjects than before, and took less feedback from students. Approximately 33.3 percent were neutral about learning from online classes, with extremely confident students around 11.9 percent and extremely unconfident students around 17.1 percent (Figure 2).

Figure 2. Participants’ votes on how lecture materials and directions are changed.

On whether online lessons were advantageous, 31.4 percent students were neutral and 17.1 percent considered online classes extremely unuseful. In comparison, online classes were considered useful by 19.52 percent of students (Figure 3).

Figure 3. How useful online classes were to participants.

Students understood that the pandemic prevented in-person learning. They recommended that some changes to online education be implemented, including limiting the length of virtual classes, ensuring an uninterrupted Internet connection, and, at a lower cost, utilizing additional visuals or examples to demonstrate a topic, taking immediate feedback after discussing a topic, and guiding students on the necessary steps for the next session of that course/class. Some complications in students' use of online platforms were observed, among them excessive amounts of data transmission due to video lecture distribution, the inability to transmit bigger files to or from the instructor at a lower Internet speed within a shorter and fixed time frame, and inconsistent attendance. The survey findings indicate that the students would be interested in online education if the aforementioned recommendations are heeded and the issues resolved. Despite the limitations indicated by survey participants, it is obvious that an online education system may serve as a viable alternative to traditional classroom education in an emergency such as the Covid-19 pandemic, and that virtual classrooms can minimize disruptions in continuous learning.

Endnotes

1. UNESCO. Education: From disruption to recovery; https://en.unesco.org/covid19/educationresponse

2. Abdullah, M. 2020: The rise of online education. Dhaka Tribune. Dec. 31, 2020; https://www.dhakatribune.com/bangladesh/2020/12/31/2020-rise-of-online-education

3. Halder, N., Islam, S.R., Hosain, M.S., Ahmed, E., Islam, A., and Noori, S.R.H. Efficacy and acceptance of virtual classrooms during covid-19: Bangladesh perspective. Proc. of 3rd International Congress on Human-Computer Interaction, Optimization and Robotic Applications. IEEE, 2021, 1–6.

4. Muthuprasad, T., Aiswarya, S., Aditya, K., and Jha, G.K. Students' perception and preference for online education in India during Covid-19 pandemic. Social Sciences & Humanities Open 3, 1 (2021), 100101.

5. Adnan, M. and Anwar, K. Online learning amid the Covid-19 pandemic: Students' perspectives. Online Submission 2, 1 (2020), 45–51.

---

Posted in: on Fri, June 03, 2022 - 11:41:22

S.M. Rakibul Islam

Nabarun Halder

Ashraful Islam

Eshtiak Ahmed

Sheak Rashed Haider Noori

Authors: Ahmet Börütecene , Oğuz Buruk
Posted: Mon, February 28, 2022 - 12:46:19

---

How can I tell what I think till I see what I make and do? — Christopher Frayling [1]

Just a couple of months before the start of the new decade and the pandemic began haunting the world, we had the privilege to attend the Halfway to the Future symposium (https://www.halfwaytothefuture.org/2019/) in person and present a paper. The symposium was organized on the occasion of the 20th anniversary of the Mixed Media Lab at the University of Nottingham. There were many exciting keynotes, presentations, and panels. At that time, we did not know that this would be one of the last physical presentations we would give for a while. In this blog post, we would like to talk about the stage performance we did to present our paper and how it motivated us to open a discussion on alternative ways of communicating scientific knowledge, in particular that which is produced through HCI and design research.

For the symposium, we submitted a provocative paper that envisions using a Ouija Board as a resource for design [2], which was then presented in the session built around the influential work of Bill Gaver on ambiguity [3]. Briefly, the Ouija is a parlor game where participants place their fingers on a physical cursor (planchette) and ask questions to an invisible being—supposedly a spirit. The being moves the cursor—although it is moved unconsciously by humans due to the ideomotor effect [4]—around the letters on the Ouija board to answer the questions. We were excited by how the Ouija mechanism combines ambiguity and human touch in an intriguing way, making it a collective, playful, and evocative medium for ideation. Through its embodied and narrative affordances, the Ouija minimizes verbal communication and focuses participants’ attention on the movement of their hands and the planchette around the letters on the board for a tacit conversation. Using the Ouija was quite experimental for us, and the ambiguity that it embodies was also at the heart of our research process, as we kept discovering new experiences and challenges each time we iterated our Ouija design sessions with different people

When we started thinking about how to present our research, sequential static slides did not look like an ideal way of showing our journey, nor our confusion and excitement about the potential of Ouija for design. Show-and-tell was another alternative. However, although many things can be explained through annotations, the design object, or the object of design, still has a lot more to communicate that cannot be translated into a traditional show-and-tell. In the end, we decided to use the artifact itself to talk about our research. One of the advantages is that our object of research/design had an intrinsic performative aspect and thus could afford a theatrical presence on the stage. We started exploring how to exploit this aspect and use the narrative it could offer.

After playing with the Ouija for a while, we noticed that we did not have to show our real-time interaction with it, but instead could show on a large screen a prerecorded video of us doing the intended actions to make the audience believe that it is actually a live video. This introduced another challenge, though: the requirement to perform on the stage in sync with the prerecorded video. Although challenging, it fit very well the spirit of the symposium: mixed reality. It was a little bit risky, and we did not have much time for rehearsal, but we wanted to give it a shot.

We wrote a vague script for our Ouija performance and started playing along in the hotel room. As we were playing, we were both rehearsing our script and discovering the Ouija board and how to use it for the performance as well as for design. This process not only unfolded the dialogue we intended to create between two people as researchers for presentation purposes, but it also facilitated our dialogue as designers exploring the Ouija board by enabling us to reveal aspects of it that we did not have the chance to notice earlier. So the efforts toward developing a way to communicate our research also became a way to explore the design space further and have a better understanding of it. In the end, the video was ready, and the performance was a success. The audience followed along with the script and caught the key moments clearly.

The point, though, is not the success of our presentation. Also, it is not the first performative presentation made for scientific communication, having been preceded by many others. However, we believe that it is worth discussing the generative impact of such presentations not only on the audience, but also on the presenters, design objects, and the research. There may be a certain value in thinking about how an artifact can be the “subject” doing the scientific communication rather than just being the inert “object” frozen during knowledge creation. This approach configures the interaction with the artifact in a way that may help with the challenges in communicating design knowledge and can be complementary to methods such as annotated portfolios. Moreover, we also had the chance to see that such a performance has an intrinsic value for communicating design knowledge that makes an impact on the audience. It not only helped us engage the audience, but it also provoked research ideas and spontaneous action in the hall. For example, one of the presenters in our panel, Miriam Sturdee, did a sketch during the performance (Figure 1), and later Katherine Isbister asked us if we could place our Ouija board at the corner of the stage to receive questions from the audience during the panel she was moderating. It was also nice to see some parallels to our performance at the symposium. For example, Steve Benford used an old slide projection to show the slides he used years ago to present one of his early works presented at CHI, and then went on with a digital presentation.

Figure 1. Sketch made by Miriam Sturdee during the Ouija
performance (used with permission)

Inspired by this experience, we would like to discuss the value of such performances at scientific venues and how they could open up new possibilities to communicate research outcomes as well as to create engagement in different types of audiences. Moreover, we also consider it important to reflect on how such an approach can contribute to authors/designers themselves to have a better understanding of the artifact or topic they are exploring. Instead of thinking of the presentation as a fixed and predictable thing, we might see it as a performance that enables designers, researchers, and audiences to experience the results in the making, as the continuation of a reflective dialogue with the design object. We believe an important aspect of our performance was that it was not a gimmick or special effects. It was a moment where the design object, or the object of design, spoke for itself and this made sense under the symposium theme. Can this kind of approach inspire alternative ways of presenting research processes, disseminating findings, or even reviewing scientific publications? One example toward this direction is the “not paper” [5], which criticizes current publishing formats and practices through its particular embodiment. DIS runs a pictorial track where authors are encouraged to communicate their research through visual-driven papers. Video articles are another example [6]. We would like to hear what design communities have to offer in this direction: Can making multisensory papers be a way of articulating insights addressing different sensory channels, be they haptic, gustatory, auditory, etc.? For example, can we hug or bite the text? Or, why not games as potential presentation formats? Instead of making slides, can we organize a live-action role-playing setting for keynote presentations? How should we go further?

Endnotes

1. Frayling, C. Research in Art and Design. Royal College of Art, London, 1993.

2. Börütecene, A. and Buruk, O. Otherworld: Ouija Board as a resource for design. Proc. of the Halfway to the Future Symposium. ACM, New York, 2019, 1–4; https://doi.org/10.1145/3363384.3363388

3. Gaver, W.W., Beaver, J., and Benford, S. Ambiguity as a resource for design. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2003, 233–240; https://doi.org/10.1145/642611.642653

4. Häberle, A., Schütz-Bosbach, S., Laboissière, R., and Prinz, W. Ideomotor action in cooperative and competitive settings. Social Neuroscience 3, 1 (2008), 26–36. https://doi.org/10.1080/17470910701482205

5. Lindley, J., Sturdee, M., Green, D.P., and Alter, H. This is not a paper: Applying a design research lens to video conferencing, publication formats, eggs… and other things. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2021, 1–6; https://doi.org/10.1145/3411763.3450372

6. Löwgren, J. The need for video in scientific communication. Interactions 18, 1 (2011), 22–25; https://doi.org/10.1145/1897239.1897246

---

Posted in: on Mon, February 28, 2022 - 12:46:19

Ahmet Börütecene

Oğuz Buruk

Authors: Eshtiak Ahmed, Ashraful Islam, Farzana Anowar
Posted: Fri, February 25, 2022 - 12:30:00

---

Recent increases in smartphone users have resulted in a great diversity of backgrounds and cultures among these users. Designers of smartphone elements need to address this diversity by making design elements more uniform and acceptable for all. The user interface should address the needs and level of understanding of all users. In this study, we focus on a new approach to the design of user interface icons and investigate how careful design can help users have more seamless interaction with interfaces.

As of the first quarter of 2021, the Google Play Store has the highest number of apps [1]. In the Google Play Store, different app categories are specified by a category name and a simple graphical representation beside the text. Though the text itself is self-explanatory, the graphical representations do not represent the categories well. This may not be a problem for someone who can read English, but it is very hard to grasp the category purely based on the pictorial depiction [2]. It is quite common for people to have trouble finding the app they are looking for [3]. The goal of this research is to see if app categories can be represented more efficiently with better icons or graphical representations next to them.

Initially, new icons were designed for some selected categories. A public survey was designed and circulated where the participants were asked to tell their preferred icons for a specific category. A total of 664 responses were collected. According to the results of the survey, distinguishing factors were identified in how graphical representations can be represented more conveniently. The detailed information and findings of this study can be found in one of our published works [4].

App store category selection

Apps in the Play store are divided into more than 35 categories. The list of categories consists of the category name and a small category icon beside it. These category icons are often confusing and misleading, making them useless at times. However, not all the icons are poor and some of them are quite helpful. For this study, 10 different app categories were selected based on their abstractness and readability: arts and design, business, communication, entertainment, finance, lifestyle, personalization, social, sports, and travel and local.

Icon design

Our goal was to design icons in such a manner that people with low literacy, senior citizens, and kids with special needs can easily understand what they stand for, potentially allowing them to find what they are searching for. While designing the icons, we wanted to represent each category with relevant objects that belongs to that category, for example, representing the sports category with elements from popular sports. In Figure 1, we show the 10 selected categories along with their current icons in the Play store, as well as the icons we have designed for them. 

Figure 1. Side-by-side comparison of existing and newly designed icons.

Survey design and participants

After completing the icon design, a public survey was designed to get feedback from smartphone users about the newly designed icons. For each app store category, two icons (newly designed and existing icons from Google Play store) were put alongside each other, and the participants were asked to decide which icons they preferred for each category. The survey collected a total of 664 responses; 513 (77.3 percent) were male and the rest 151 (24.7 percent) were female. All participants were between the ages of 21 and 25.

Findings

The public survey provided a very good idea of whether the newly designed icons represent the categories better or not. Figure 2 shows the overall survey result comparing the participants’ preference for icons in each of the 10 categories. 

The newly designed icon for the arts and design category was preferred by 88 percent of the participants. The new business category icon was preferred by 90.8 percent of participants, while the new icons for the communication, entertainment, and finance categories were preferred by 84.3 percent, 95.5 percent, and 95.9 percent of participants, respectively. The new lifestyle category icon was preferred by 93.1 percent of participants.

Some of the newly designed icons received mixed responses, as they were tough to represent by a small piece of art. These included the newly designed icon for personalization category, which was preferred by 64.8 percent of participants and the new icon for the social category, which was preferred by 71.2 percent of participants. Also, the new sports category icon was preferred by 82.7 percent of participants, while the new travel and local category icon was preferred by 72 percent of participants.

Challenges and discussion

The new icons were designed by using real-world objects from each category. As an example, the sports icon was designed by combining the equipment from three popular sports. However, similar types of categories such as finance, business and communication, and social were challenging because there are fewer distinguishing factors between them. The lifestyle and personalization categories were also very challenging because of their abstractness.

Icons were created using the objects that came to mind for each category. Sometimes, however, the new icons might have seemed too crowded, as so many objects were cramped into a tiny circle to make the icon understandable. Another shortcoming is that the survey participants are mostly young smartphone users. A more thorough survey covering different age groups could create a different dimension of results.

Figure 2. Analysis of results obtained from the conducted survey.

Conclusion

This study promotes designing an effective textless user interface so users can integrate with their devices more easily and more effectively. Creating impactful and meaningful icons could be a great step toward text-free interface in smartphones. This work could be extended by representing a user interface that is self-explanatory and interactive, opening new doors for challenged users.

Endnotes

1. Statista. Number of apps available in leading app stores. Retrieved August 21, 2021 from https://www.statista.com/topics/1729/app-stores/#dossierSummary__chapter3

2. Wiebe, M., Geiskkovitch, D., Bunt, A., Young, J., and Glenwright, M. Icons for kids: Can young children understand graphical representations of app store categories? Proc. of the 42Nd Graphics Interface Conference. Canadian Human-Computer Communications Society, 2016, 163–166.

3. Wang, M. and Li, X. 2017. Effects of the aesthetic design of icons on app downloads: Evidence from an Android market. Electronic Commerce Research 17, 1 (2017), 83–102.

4. Ahmed, E., Hasan, M.M., Faruk, M.O., Hossain, M.F., Rahman, M.A., and Islam, A. 2019. Icons For the mass: An approach towards text free smart interface. Proc. of the 2019 International Conference on Advances in Science, Engineering and Robotics Technology. 2019, 1–4.

---

Posted in: on Fri, February 25, 2022 - 12:30:00

Eshtiak Ahmed

Ashraful Islam

Farzana Anowar

Authors: Maliheh Ghajargar, Jeffrey Bardzell, Allison Smith Renner, Peter Gall Krogh, Kristina Höök, Mikael Wiberg
Posted: Tue, February 15, 2022 - 4:11:15

---

Computational systems are becoming increasingly smart and automated. Artificial intelligence (AI) systems perceive things in the world, produce content, make decisions for and about us, and serve as emotional companions. From music recommendations to higher-stakes scenarios such as policy decisions, drone-based warfare, and automated driving directions, automated systems affect us all.

But researchers and other experts are asking, How well do we understand this alien intelligence? If even AI developers don’t fully understand how their own neural networks make decisions, what chance does the public have to understand AI outcomes? For example, AI systems decide whether a person should get a loan; so what should—what can—that person understand about how the decision was made? And if we can’t understand it, how can any of us trust AI?

The emerging area of explainable AI (XAI) addresses these issues by helping to disclose how an AI system arrives at its outcomes. But the nature of the disclosure depends in part on the audience, or who needs to understand the AI. A car, for example, can send warnings to consumers (“Tire Pressure Low”) and also send highly technical diagnostic codes that only trained mechanics can understand. Explanation modality is also important to consider. Some people might prefer spoken explanations compared to visual ones. Physical forms afford natural interaction with some smart systems, like vehicles and vacuums, but whether tangible interaction can support AI explanation has not yet been explored.

In the summer of 2020, a group of multidisciplinary researchers collaborated on a studio proposal for the 2021 ACM Tangible Embodied and Embedded (TEI) conference. The basic idea was to link conversations about tangible and embodied interaction and product semantics to XAI. Here, we first describe the background and motivation for the workshop and then report on its outcomes and offer some discussion points.

Self-explanatory or explainable AI?

Explainable AI (XAI) explores mechanisms for demonstrating how and why intelligent systems make predictions or perform actions to help people to understand, trust, and use these systems appropriately [1]. Importantly, people need to know more about systems than just their accuracy or other performance measures; they need to understand more fully how systems make predictions, where they work well, and also their limitations. This understanding helps to increase our trust in AI, ensuring us they are operating without bias. XAI methods have another benefit: supporting people in improving systems by making them more aware of how and when they err. Yet XAI is not always a perfect solution. Trust is particularly hard to calibrate; AI sometimes cannot be trusted; and AI explanations may result in either over- or under-reliance on systems, which might promote manipulations and managerial control.

The notion of explanations or self-explanatory forms has also had a long tradition in product and industrial design disciplines. Product semantics is the study of how people make sense of and understand products through their forms; hence, the way products can be self-explanatory [2]. From that perspective, a product explains its functionality and meaning by its physical forms and context of use alone. For example, the large dial on a car stereo not only communicates that the volume can be controlled, but also how to do so. In this view, products do not need to explain themselves, though their forms need to be understandable in principle.

Using a product-semantic perspective for XAI builds upon the general concerns of XAI, but focuses on the user experience and understandability of a given AI system based on user interpretations of its material and formal properties [3]. The dominant interaction modalities, however, do not fully allow that experience; hence, we have decided to explore tangible embodied interaction (TEI) as a promising interaction modality for that purpose.

From explainable AI to graspable AI: A studio at ACM TEI 2021

At ACM TEI 2021, we organized a studio (similar to a workshop) to map the opportunities that TEI in its broadest sense, including Hiroshi Ishii and Brygg Ullmer’s tangible user interfaces, Paul Dourish’s embodied interaction, Kristina Höök’s somaesthetics, and Mikael Wiberg’s materiality of interaction, can offer to XAI systems.

We used the phrase graspable AI, which deliberately plays with two senses of the word grasp, one referring to taking something into one’s hand and the other to when the mind “grasps” an idea. The term graspable inherently conveys the meaning of being understandable intellectually, meaningfully, and physically.

In doing so, we referenced two earlier HCI concepts. In 1983, Ben Shneiderman coined the term direct manipulation as a way to “offer the satisfying experience of operating on visible objects. The computer becomes transparent, and users can concentrate on their tasks” [4]. In 1997, Ishii and Ullmer envisioned tangible bits, a vision of human-computer interaction that allows users to grasp and manipulate bits by coupling the bits with everyday physical objects and architectural surfaces, which was preceded by George Fitzmaurice’s Ph.D. thesis Graspable User Interfaces in 1996. Their goal was to create a link between digital and physical spaces [5,6,7].

Building on these ideas, we viewed graspable AI as a way to approach XAI from the perspective of tangible and embodied interaction, pointing to a product that is not only explainable but also coherent and accessible in a unified tactile and haptic form [8]. 

Beyond presentations of position papers, studio participants engaged in group activities, which consisted of three phases (Figure 1). First, each group chose an everyday use AI object or system, analyzed and explored its interactions regarding explainability (what needs to be explained and what the system can explain), then ideated possible tangible interactions with the system and redesigned the human-AI interaction using TEI.

Figure 1. TEI studio structure and process.

Graspable designs for everyday use

Studio participants split into groups to explore graspable designs for three distinct AI systems for everyday use: movie recommendations, self-tracking, and robotic cleaning devices.

One group explored graspable design for movie recommendations, asking questions about their explainability and how users can interact with the system through a tangible interface. Some ideas that emerged during the discussion and ideation phases were speculations around using a stress-ball form as a tangible interface to the recommendation system to influence recommendations based on the user’s mood (Figure 2).

Figure 2. Stress ball as a TUI for movie recommendation system.

Another group worked on AI self-tracking devices, discussing questions of how an AI self-tracking device can learn from the user interactions and make us “feel” that our blood sugar is high or low. Provocative ideas, such as an eye that loses its sight gradually and a system that learns and informs the user through haptic feedback, were also explored (Figure 3).

Figure 3. Tangible AI self-tracking device.

The third group focused on robotic cleaners, where they discussed the opportunities of tangible interactions to inform users about and anticipate robots’ movements. Making the robot’s intended movement visible was another idea that led the group to ideate a tangible map and a map that uses spatial image projections (Figure 4).

Figure 4. Robotic vacuum cleaner spatial interactions.

Discussion

Aesthetic accounts often unfold as a back and forth between material particulars and interpreted wholes [3]. Further such accounts consider the difference between what are called explanations in science and interpretations in humanities and design. While explanation is used to find an answer to the why of a phenomena and to reduce its complexity to manageable and understandable units, interpretation is a way to make sense of an experience and the how of the formal and material properties of an artifact within its sociocultural context [3].

AI systems vary in their complexity; some are easier to explain (e.g., movie recommendation systems) while others are more difficult to understand because of their complex inner reasoning (e.g., natural language understanding by deep learning). Accordingly, the notion of graspable AI can not only be concerned with explanations, but also with how AIs are experienced and interpreted through their material particularities within their sociocultural context. However, we do not mean to suggest that graspable AI is a perfect solution to all fundamental humanities concerns! In this workshop, we explored TEI as an explanation modality. We found out that while it has some potential for improving the understandability of the system, it comes with its own challenges. For example, there are features and functionalities of AI systems that do not need to be tangible, or that are understandable as they are. It has been also challenging to design tangible AI systems without falling into usual categories of smart objects.

Hence, we conclude this post discussing a set of related themes and challenges inherent to AI systems that fruitfully may be approached through the notion of graspable: growth, unpredictability, and intentionality.

Graspable growth

AI systems are intended to learn over time as a continuous and internal process based on the data inputs. Hence, an algorithm metaphorically grows over time. As much as an AI system grows, its functionalities and predictions are expected to improve as well, though devolution is as possible as evolution. But humans are not always aware of that process of learning and growing. Probably the most intuitive form of growth for humans are biological growths. We naturally and intuitively can understand when a plant grows new leaves.

We suggest that metaphors of biological growth (and decay) can inspire the design of XAI product semantics to make the AI learning process self-explanatory.

Graspable unpredictability

AI systems are sometimes unpredictable for humans. The decisions they make based on their inner workings and logic may not appear entirely reasonable, fair, or understandable for humans. Further, such systems are ever changing based on input data, especially if they learn over time. How might an AI decrease its apparent unpredictability by revealing its decisions and purposes through tangible and embodied interactions? 

For instance, a robotic cleaner maps the area it is cleaning, and its movement decisions are made based on spatial interaction with the environment and continuous learning of the space. Can its mappings and anticipated movements be rendered in a way that humans can grasp? While the stakes could be low for automated consumer vacuum cleaners, implications for powerful robots used in manufacturing and surgery are more serious.

Graspable intentionality

AI systems have their own internal reasoning and logic and they make decisions based on them. Their internal logic and mechanisms make some AI systems appear to be independent in their decision-making process and therefore to have their own mind, intentions, and purposes. Whether or not this is true from a technical perspective, end users often experience it as such, for instance when an algorithmic hiring tool “discriminates.”

At stake is not only the technical question of how to design a more socially responsible AI, but also how AI developers might expose systems’ workings and intentions to humans by means of tangible and embodied interaction.

Conclusions

AI systems are becoming increasingly complex and intelligent. Their complexity and unpredictability pose important questions that are concerned with, on one hand, the degree to which end users actually can control them, and on the other hand, whether and how we can design them in a more transparent and responsible way. In the above we have brought forward some possible challenges and opportunities of using tangible and embodied interaction in designing AI systems that are understandable for human users.

Acknowledgment

We would like to thank workshop participants and our colleagues David Cuartielles and Laurens Boer for their valuable input and participation.

Endnotes

1. Preece, A. Asking ‘why’ in AI: Explainability of intelligent systems – perspectives and challenges. Intelligent Systems in Accounting, Finance and Management 25, 2 (2018), 63–72; https://doi.org/10.1002/isaf.1422

2. Krippendorff, K. and Butter, R. Product semantics: Exploring the symbolic qualities of form. Innovation 3, 2 (1984), 4–9.

3. Bardzell, J. 2011. Interaction criticism: An introduction to the practice. Interacting with Computers 23, 6 (Nov. 2011), 604–621; https://doi.org/10.1016/j.intcom.2011.07.001

4. Shneiderman 1983. Direct manipulation: A step beyond programming languages. Computer 16, 8 (Aug. 1983), 57–69; https://doi.org/10.1109/MC.1983.1654471

5. Fitzmaurice, G., Ishii, H., and Buxton, W. Bricks: Laying the foundations for graspable user interfaces. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., USA, 1995, 442–449; https://doi.org/10.1145/223904.223964

6. Fitzmaurice, G. Graspable User Interfaces. Ph.D. thesis. University of Toronto, 1996; https://www.dgp.toronto.edu/~gf/papers/PhD%20-%20Graspable%20UIs/Thesis.gf.html

7. Ishii, H. and Ullmer, B. Tangible bits: Towards seamless interfaces between people, bits and atoms. Proc. of the ACM SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 1997, 234–241.

8. Ghajargar, M., Bardzell, J., Renner, A.S., Krogh, P.G., Höök, K., Cuartielles, D., Boer, L., and Wiberg, M. From explainable AI to graspable AI. Proc. of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction. ACM, New York, 2021, 1–4.

---

Posted in: on Tue, February 15, 2022 - 4:11:15

Maliheh Ghajargar

Jeffrey Bardzell

Allison Smith Renner

Peter Gall Krogh

Kristina Höök

Mikael Wiberg