Early detection of autism
Authors:
Samiya Ali Zaidi
Posted: Tue, August 20, 2024 - 11:55:00
To get started, we partnered with the Autism Spectrum Disorder Welfare Trust (ASDWT) in Pakistan to create a unique dataset. It will include eye-tracking data from Pakistani children, reflecting a diverse range of ages, genders, and other demographic factors. The diversity in our dataset is crucial because it helps ensure that our model is robust and can be generalized across different populations. Collecting this data will be a collaborative effort, involving the ASDWT, parents, and other stakeholders who provided informed consent and valuable insights. Once the dataset is collected, we will prepare it for analysis. This step includes pre-processing the data, removing noise, and normalizing it to ensure consistency. These features will play a vital role in training our computer vision-based model to detect patterns that may indicate ASD.
In addition to eye-tracking data, we recognize the importance of incorporating textual inputs to improve the accuracy of our model. This step involves gathering information from parental reports, medical histories, and other relevant checklists. By integrating this textual data, we can provide our model with a more comprehensive understanding of each child's development, allowing for a more accurate detection of ASD-related patterns. Textual inputs add context to the eye-tracking data, helping identify correlations between visual behavior and other developmental cues. This approach aligns with HCI's focus on creating systems that understand and respond to human behavior in meaningful ways. By combining visual and textual data, we aim to create a model that not only detects ASD with greater accuracy but also helps healthcare professionals and families make informed decisions about treatment and support.
The heart of our project lies in building and training a custom computer vision-based model. Given the complexity of ASD and the variability of eye-tracking patterns, we understand that a robust model is essential. To achieve this, we will split our dataset into three subsets: training, validation, and testing. The training subset, comprising approximately 70 percent of the data, is used to teach the model to recognize patterns. The validation subset (20 percent) helps us fine-tune and optimize the model's hyperparameters. Finally, the testing subset (10 percent) is where we evaluate the model's accuracy and generalizability. The training process involves applying state-of-the-art computer vision techniques to analyze the eye-tracking data. We will also use transformer-based architectures to explore different approaches to pattern recognition. By testing multiple architectures, we aim to determine which methods are most effective in identifying ASD-related patterns.
After training and testing the model, we moved on to the analysis phase. This step involves evaluating the model's performance using various metrics. We compared our results with existing studies to gauge the effectiveness of our approach and to identify areas for improvement. During this phase, we paid special attention to how variations in visual stimuli and experimental conditions influenced the model's accuracy. We also considered the potential impact of demographic factors, ensuring that our model could generalize across different groups. This analysis is crucial, as it helps us understand the limitations of our approach and provides insights into how we can refine the model to improve its reliability.
The findings from our project, as seen in the initial reports, will have the potential to significantly impact early ASD detection. By combining eye-tracking data with textual inputs, we can create a model that offers earlier and more accurate diagnoses, which can lead to more effective interventions. This directly benefits healthcare professionals and families by providing them with tools to support children with ASD promptly. Moreover, our work demonstrates the potential of Human-Computer Interaction in healthcare. By leveraging computer vision and deep learning, we can address complex problems like ASD diagnosis with innovative solutions. This approach has implications beyond ASD, showcasing how HCI can play a pivotal role in improving diagnostic processes for other neurological and developmental conditions.
As we continue to refine our model, we are excited about the possibilities ahead. Our findings could guide future research into applying HCI and computer vision to a broader range of diagnostic scenarios, ultimately leading to more reliable and efficient methods for early diagnosis. We believe this project is just the beginning of a journey toward a more inclusive and effective approach to healthcare. By incorporating HCI into the process, we're not just creating technology for technology's sake; we're creating solutions that can make a real difference in people's lives. And that's what makes this work truly exciting.
Posted in:
on Tue, August 20, 2024 - 11:55:00
Samiya Ali Zaidi
I am disabled but not ‘impaired’
Authors:
Ather Sharif
Posted: Thu, August 08, 2024 - 12:45:00
There are at least 1.3 billion disabled people worldwide [1]. Their disabilities include blindness, hearing, mobility, and cognitive, among others. These individuals are disabled—if they choose to identify as such—but what they are not is “impaired.”
![](/images/uploads/blogs/wordcloud_web_1.jpg)
Word cloud of impaired and its related words created using Generative AI.
Correct usage of terminology is important. There exists a prodigious, if not universal, agreement that the terminology we use to describe people holds significant weight in increasing inclusivity for marginalized groups, especially disabled people [2]. These words represent the speakers’ commitment to cultural awareness of and sensitivity toward vulnerable populations [3]. For the latter, they express respect and a sense of belonging to a world that continues to disenfranchise them. Much like fashion, terminologies have a shelf life, after which they slowly become archaic or are considered outdated due to their negative connotations (e.g., retarded). But unlike fashion, if declared pejorative, the terminologies’ return to popular usage is unwelcome at best, unless done so purposely by the targeted demographic to reclaim the narrative (e.g., crippled and gimp).
Reclaiming the narrative. Reclaiming the narrative begins with a word being just a word, like any other word in the dictionary. Then, a creative person comes along and uses it as an adjective to describe a person or a group of people they think they are superior to. Other “superiors” agree, and the word then becomes an identity of the “inferiors.” Time goes by, inferiors get a platform to vocalize their disagreement with the word, superiors demonstrate resistance and continue using it, and finally, after years of exhausting advocacy, the word gets flagged as offensive. It stays offensive, and people, within and outside the targeted demographic, avoid its usage. Then, people within the targeted demographic take ownership of the word to empower themselves, remembering the oppression they faced, which led to the association of the word with their demographic in the first place. People outside the demographic, however, are barred from using the word, particularly with those within the demographic. (The barring has never been enforced as a law, though, and only serves as a recommendation, with the likelihood of the speaker being “canceled” for using the word.)
The word impaired has not yet been reclaimed. The disagreement, however, has been vocalized. Resistance to its cessation continues.
Definition and impact of impaired. The Oxford English Dictionary defines impaired as “injured, weakened, damaged; diminished or deteriorated in amount, quality, or value.” Merriam-Webster’s Collegiate Dictionary describes it as “diminished in function or ability.”
Disabled people already know what their disabilities and limitations are, with many having embraced it as a part of their identity. They do not need a constant reminder of their disabilities. Every time I type the word impaired for this blog post, a part of me feels dejected and wants to give up on the utopian future where disabled people are not distinct from those who are “normal.” But I digress.
Using words like impaired can result in disabled people internalizing their self-identity as being broken, or even an “incomplete” human. These words can foster a subconscious degradation in confidence and pose hurdles in a disabled person’s journey toward accepting their identity as a disabled person. They can create emotional scars, which, if externalized, could develop into more negative stereotypes, such as disabled people being “sappy” or lacking “thick skin.” The impact of these terms can be long-lasting.
Arguments for using impaired. I hesitate to believe, or rather to admit, that on a human level, HCI researchers and authors are entirely oblivious to the effect of words on human emotions. Movies, TV shows, and books thrive on this effect. Encouraging and respectful words produce harmony. Disrespectful and prickly words trigger emotional distress. Neutral words are often harmless. So why do people still use impaired to refer to people?
Argument 1: It’s an established term used in medical journals and prior accessibility research. We’re just following the standards. This reasoning is based on facts. Impaired is used quite extensively, although the field of medicine is attempting to rectify that [4,5]. But if we dig deeper into the literature, we will find frequent usage of other terms now considered derogatory. There was a time when those words were acceptable…and then they weren’t. There are two choices: keep using impaired until it reaches its shelf life or preemptively avoid the term and be better allies to the disability community. History has a funny way of teaching us things, but only if we are willing to learn. The lesson here is to be proactive catalysts for positive change and not let the change happen as a mandatory reparations measure.
Argument 2: A person I know is perfectly fine with its usage, so this is not a problem. N = 1 has its place, but not for this argument. Everyone has their preferences; some people are sensitive, some not so much. Focusing on those not affected by the usage of impaired and letting that motivate its continued usage is arguably an invalidation of those who are sensitive to its usage. Our societal goal has always been to validate and support vulnerable and disenfranchised populations. It makes sense to listen to the voices of those who find discomfort in being labeled as impaired. It is improbable that a more accepting replacement would be unwelcomed by those not affected by the usage of impaired. At its 1993 annual convention, the National Federation of the Blind advised against the usage of visually impaired and instead advocated using blind and low vision, employing identity-first language [6]. Yet, 31 years later, visually impaired remains a widely used term.
Argument 3: I never really thought about it; I just used it because my advisor did. What can I do moving forward? This is not necessarily an argument, but it provides valuable insights into some people’s thought process. The reasoning behind this view makes sense too. The desire to be better is a gold star. But some questions remain. For example: How can we better communicate our discomfort with word choices without exhausting ourselves, such that it is baked into our thought process from the start? Whose responsibility is it? How can we build a culture where we are encouraged to challenge an established belief, where there is constant learning and unlearning? I do not know the answers to these questions, but I suspect the answers involve being less resistant to change and actively seeking feedback from a community on how they wish to be referred to.
Where do we go from here? This blog post is a call to action to cease referring to disabled people as impaired. But that is not all I intend. I urge HCI researchers and authors to employ practices and processes that allow for rethinking and challenging the terms they use to refer to vulnerable and disenfranchised populations. I believe it when people say they wish to be better allies; using respectful terminology is a step toward that. Giving some thought to people-describing adjectives before using them can help. I hope this post sparks thoughtful conversations on this subject and helps identify avenues for authors to use more-inclusive words.
Endnotes
1. World Health Organization. Disability. March 7, 2023; https://www.who.int/news-room/...
2. Sharif, A., McCall, A.L., and Bolante, K.R. Should I say “disabled people” or “people with disabilities”? Language preferences of disabled people between identity- and person-first language. Proc. of the 24th International ACM SIGACCESS Conference on Computers and Accessibility. ACM, New York, 2022, 1–18.
3. Best, K.L., Mortenson, W.B., Lauzière-Fitzgerald, Z., and Smith, E.M. Language matters! The long-standing debate between identity-first language and person-first language. Assistive Technology 34, 2 (2022), 127–128.
4. Goddu, A.P. et al. Do words matter? Stigmatizing language and the transmission of bias in the medical record. Journal of General Internal Medicine 33, 5 (2018), 685–691.
5. Raney, J. et al. Words matter: An antibias workshop for health care professionals to reduce stigmatizing language. MedEdPORTAL 17 (2021), 11115.
6. National Federation of the Blind. Convention Resolutions 1993: Resolution 93-01 https://nfb.org/sites/default/...
Posted in:
on Thu, August 08, 2024 - 12:45:00
Ather Sharif
Ather Sharif has a Ph.D. in computer science from the University of Washington. His research focuses broadly on human-computer interaction and specifically on the intersection of accessibility, visualization, and personalization. His work involves making online data visualizations accessible to screen-reader users. He is the chairman of the Executive Board of the Disability Empowerment Center.
[email protected]
View All Ather Sharif's Posts
Doing CHI together: The benefits of a writing retreat for early-career HCI researchers
Authors:
Ava Elizabeth Scott,
Leon Reicherts,
Evropi Stefanidi
Posted: Tue, April 16, 2024 - 10:30:00
The deadline for the CHI conference is an important event on an HCI researcher’s calendar. For CHI 2023, more than 3,000 papers were submitted and only 28 percent were accepted. With such a high number of submissions and low acceptance rate, researchers face pressure to make the best submission possible. So how can we best introduce early-career researchers to the charms and quirks of writing a CHI paper while supporting them in submitting the best possible paper they can?
For the past two years, the University of St. Gallen in Switzerland has hosted a writing retreat called CHI Together, which brings researchers together for two and a half weeks directly prior to the CHI deadline. While many of these researchers are experienced CHI authors, some are engaging in this process for the first time. During the inaugural writing retreat held in 2022, attendees submitted a total of a dozen papers. The idea behind the retreat was to help participants collaborate efficiently and manage the stressful phase before the deadline in the company of like-minded people; and then to celebrate their submissions. Hoping to reiterate and build on this success, this year the computer science department at St. Gallen invited 21 researchers to come together, physically and collaboratively, in the run-up to the deadline for CHI 2024.
This endeavor was made possible by funding from the SIGCHI Development Fund and from the University of St. Gallen. We are hugely grateful for this support, which facilitated the many healthy social activities and group meals, as well as financed accommodation and travel expenses. Here we describe these activities, starting from our first day arriving in St. Gallen to the final delirious day of submissions and subsequent celebration. We then reflect on the intensity of the CHI deadline and recommend that early-career researchers be supported with social and recreational resources, as well as pragmatic and technical measures.
Arriving by plane and train on a warm day in late August, we were immediately struck by the fact that these services were on time. As students from the U.K. and Germany, Swiss timeliness is always a nice surprise. We reunited with the other visiting researchers, new and familiar faces alike. Our apartments were a 10-minute walk from the computer science building in the center of St. Gallen, halfway up a steep hill. While it was an easy jaunt down to the office, there was a significant disincentive to leave the office: the hike between you and your bed. Though this felt like a slog at the time, the daily exercise surely benefited us. These apartments were our home for the next two and a half weeks, but we spent much more time at the office.
![](/images/uploads/blogs/Image1_web.jpg)
Sharing an open office environment catalyzed natural collaboration, as well as companionship and social breaks during the workday.
In the computer science building, we each had desk space with a display monitor and a comfortable chair. In anticipation of the numerous hours we would spend writing, an ergonomic setup was a priority. In this open office setting, students collected data, analyzed it, and scratched their heads debating over the structure and presentation of their results.
![](/images/uploads/blogs/Image2_web.png)
A selection of irreverent and humorous Ph.D. memes were spread over the desks.
Knowledge exchange sessions were organized to support these processes, including sessions on qualitative analysis and how to write a CHI paper. We swapped abstracts and drafts with one another to get feedback on our efforts. In addition to receiving guidance on analysis and writing, we also created and shared analytical tools, such as a transcription and speaker-segmentation pipeline, which we used to transcribe more than 100 interviews across different research projects. Using the wall-size whiteboards around the office, we collectively sketched and resketched papers’ structure, argumentation, models, and figures. By distributing our thinking across multiple brains and the physical space, we could work through hard problems with greater speed and clarity.
This intensive brain activity required fuel. Most days, we took turns making group lunches in the lab kitchen—usually an elaborate salad with fresh bread and cheese. By sharing lunch duties, we reduced the overall time spent on meal preparation, while also eating well. As the deadline got closer and the days became longer, we frequently honored the tradition of grabbing doner kebab or Thai takeaway for dinner.
![](/images/uploads/blogs/Image3_web.jpg)
Paweł W. Woźniak presented the key elements of an abstract.
There were also several optional activities that provided a much-needed break. We hiked the nearby mountains, took a dip in the local mountain lakes, indulged in manic dance and workout breaks, went to the cinema, and shared pizza and burgers in the evenings. Furthermore, a contingent of researchers attended the Mensch und Computer conference in nearby Rapperswil, enabling further exchange with the D-A-CH (Germany, Austria, and Switzerland) HCI community. Postdocs and those in their final year of Ph.D. research also attended an information session about research funding opportunities across Switzerland and the rest of Europe, organized by the research office at St. Gallen. This balance of intellectual, recreational, and forward-looking activities helped balance the otherwise intensive writing schedule.
Immersed in this culture of dedicated focus, we collectively submitted more than a dozen papers for CHI 2024. One supervisor said that their students’ writing during this period was the clearest and most effective of their studies so far. While some of us were clicking the submit button at the very last moment (2 p.m. on September 15 in our time zone), we all got our papers in on time and celebrated with drinks and snacks in the lab. Despite many of us having pulled all-nighters, the drinks continued until 2 a.m. We said heartfelt goodbyes, and left for our respective destinations the following morning, looking forward (with slight trepidation!) to receiving our reviews in a few weeks.
![](/images/uploads/blogs/image4_web.png)
The collective whiteboards were the site of active collaborative thinking, sketching iterations of models, figures, and argumentation.
Large, top-tier conferences such as CHI are the critical sites for presenting work, receiving feedback, and networking with peers and future collaborators. However, due to the appropriately high standards set for paper acceptance and the cost of in-person attendance, such conferences are not always accessible to early-career scholars. The CHI Together model of collaborative writing before the deadline provides support in multiple ways, offering an encouraging, knowledgeable community to help steer research work and frame completed work following best practices for different submission types. CHI Together also supports recreational activities that allow early-stage scholars to network and build community; we believe that such community building means scholars will have an established cohort for continued mutual support in their everyday work setting as well as at conferences, when they are able to attend them. These meaningful relationships with peers are perhaps even more lasting than connections made at conferences.
Plans for CHI Together 2024 are in the works, potentially hosted elsewhere in Europe, but aiming to maintain the same level of focus, support, and solidarity in the run-up to the deadline for CHI 2025. For those of you interested in hosting a similar writing retreat, and who would appreciate some guidance or blueprints, don’t hesitate to get in touch with us or Johannes Schöning of the University of St. Gallen. Or if you are running similar events, please do share your approach with us.
Posted in:
on Tue, April 16, 2024 - 10:30:00
Ava Elizabeth Scott
Ava Elizabeth Scott is a Ph.D. candidate at University College London. As part of the Ecological Study of the Brain DTP, her current research prioritizes ecological validity by using interdisciplinary approaches to investigate how metacognition and intentionality can be supported by technological interfaces .
[email protected]
View All Ava Elizabeth Scott's Posts
Leon Reicherts
Evropi Stefanidi
Training for net zero carbon infrastructure through interactive immersive technologies
Authors:
Muhammad Zahid Iqbal,
Abraham Campbell
Posted: Wed, March 06, 2024 - 3:00:00
By providing realistic and engaging learning experiences, immersive technologies such as virtual reality (VR), augmented reality (AR), mixed reality (MR), 360-degree video, and simulation have proven to be effective in industrial training contexts. These technologies have been adopted for various purposes, including healthcare [1], teacher training, and nuclear power plant training [2].
Climate change is a global challenge that requires coordinated action from all sectors of society. In order to combat climate change, CO2 emissions need to fall to zero. One of the key strategies in addressing this challenge is to achieve net zero carbon emissions by 2050 [3]. Essential to achieving this goal is the adoption of net zero carbon infrastructure [4], which can reduce reliance on fossil fuels, enhance the efficiency and resilience of energy systems, and support the development of low-carbon technologies in industry. Net zero carbon infrastructure includes renewable energy resources such as wind, solar, nuclear, and hydro; low-carbon transport modes such as electric vehicles; and carbon-capturing, energy-efficient buildings [5] and appliances. There is an urgent need to invest in net zero carbon infrastructure to improve air quality, promote healthy lifestyles, and improve social equity and environmental justice. Net zero carbon infrastructure is therefore not only a necessity but also an opportunity to build a more sustainable and prosperous future for all. This post explores the opportunities and challenges of using immersive technologies for net zero carbon infrastructure training.
![](/images/uploads/blogs/Figure1_web_2.png)
Main components of net zero carbon infrastructure.
Immersive technologies can potentially revolutionize training for net zero carbon infrastructure development and maintenance. Potential opportunities include:
- Reducing training costs, risks, and errors by eliminating the need for physical resources, travel, and instructors.
- By using immersive technologies, trainees can learn and practice the skills and knowledge required for net zero infrastructure in a realistic and immersive way without the need for physical resources, travel, and instructors. A perfect example of this is in wind farms, where maintenance of a turbine may take place over 50 meters above the ground. Immersive technologies can save time, money, and resources, as well as improve safety and quality.
- Creating realistic simulations of dangerous or hazardous environments that workers may encounter in their jobs, such as oil spills, fires, or explosions. By using immersive technologies, workers can learn how to handle these situations safely and effectively without exposing themselves to physical harm. Therefore, immersive technologies can enhance the safety and efficiency of net zero infrastructure training.
- Using immersive technologies in net zero infrastructure training can provide data and feedback on learner performance, behavior, and progress. Trainers can monitor and assess how learners interact with the virtual environment, what choices they make, how they solve problems, and how they apply their skills and knowledge. This data and feedback can be used to improve training design and delivery by identifying strengths and weaknesses, providing personalized guidance, and adapting the level of difficulty and complexity.
- Enhancing the learning experience by creating a more engaging and interactive environment for learners or trainees. By providing learners with realistic and immersive scenarios, these interventions can stimulate their senses, emotions, and cognition, as well as provide immediate feedback and guidance. Therefore, immersive technologies can help to foster a more effective and enjoyable learning process.
- Leveraging the latest revolution ingenerative AI integration, immersive training can be made more productive, personalized, and content-driven by instructional designers. As generative AI moves forward, its ability to facilitate training will only improve in its ability to generate text, image, video, and even 3D avatars to assist the trainee.
- Using haptic devices in immersive training is the next step forward, as compared with touchless hand interaction or nonhaptic, controller-based interaction it can achieve greater realism within a training environment. The haptic sense in training has been proven to be one of the best feedback methods for users.
There are also challenges posed in adopting these technologies that need to be addressed. Some of the main challenges include:
- Acquiring immersive training resources for the first time can be costly and resource intensive because it requires specialized equipment, software, and content development. Depending on the complexity and quality of the simulations, the initial investment and maintenance costs can be high, but most of this is a long-term investment.
- Implementing immersive training infrastructure may not be compatible with existing systems and platforms, which can create technical difficulties and integration issues. Also, immersive training facilities require more technical support and troubleshooting initially than other forms of training.
- Accessibility is a commonly discussed issue with immersive training. But with the latest developments in immersive hardware, immersive training material can be designed with accessibility and inclusivity in mind.
- Implementing new initiatives such as using immersive technologies for learning may face resistance from trainees, who may fear adopting new technologies. Therefore, there is a need for a convincing strategy, such as adopting the latest versions of the technology acceptance model (TAM) [6] and communicating about the value of immersive training, as well as providing adequate training and support.
All of these challenges are not insurmountable and can be overcome with careful planning, evaluation, and collaboration. It is worth considering how we might leverage these technologies to achieve net zero industrial training goals. One way to address the cost challenge is to develop more-affordable immersive technologies such as mobile AR/VR; even the makers of the latest devices are working on reducing the cost, such as the latest Meta VR headset Quest 3. As advances in XR devices proceed rapidly, with year-on-year improvements, Meta and other companies such as Microsoft and HTC should consider allowing users to return their old devices to be refurbished or recycled for a discount on new devices. This will reduce costs for XR users who want the latest devices and reduce the environmental impact.
Another way to address the challenge of accessibility is to make immersive training more accessible to people in developing countries. This can be done by providing training on how to use immersive technologies and by developing immersive training programs tailored to the needs of developing countries. Of course, there are challenges of usability, which can be addressed by designing immersive training programs that are easy to use and more user-friendly.
Case studies from different industrial training contexts show immersive technologies have the potential to play a significant role in training for net zero carbon infrastructure. By addressing the challenges, we can ensure that immersive technologies can help revolutionize net zero workforce training for a sustainable future.
Endnotes
1. Brooks, A.L. Gaming, VR, and immersive technologies for education/training. In Recent Advances in Technologies for Inclusive Well-Being: Virtual Patients, Gamification and Simulation. Springer, 2021, 17–29.
2. opov, O.O. et al. Immersive technology for training and professional development of nuclear power plants personnel. Proc. of the 4th International Workshop on Augmented Reality in Education, 2021.
4. Bouckaert, S. et al. Net Zero by 2050: A Roadmap for the Global Energy Sector. International Energy Agency, 2021.
5. Kennedy, C.A., Ibrahim,N., and Hoornweg,D. Low-carbon infrastructure strategies for cities. Nature Climate Change 4, 5 (2014), 343–346.
6. Thomas, R.J., O'Hare, G., and Coyle, D. Understanding technology acceptance in smart agriculture: A systematic review of empirical research in crop production. Technological Forecasting and Social Change 189 (2023),122374.
Posted in:
on Wed, March 06, 2024 - 3:00:00
Muhammad Zahid Iqbal
Muhammad Zahid Iqbal is an assistant professor in immersive technologies at Teesside University. He also works as an associate faculty member at the University of Glasgow. He completed Ph.D. in computer science with a specialization in immersive technologies from University College Dublin. His research vision is to explore the convergence of immersive technology, digital twins, the metaverse, and digital transformations.
[email protected]
View All Muhammad Zahid Iqbal's Posts
Abraham Campbell
Abraham Campbell is an assistant professor at University College Dublin, Ireland. He also served as faculty member of Beijing-Dublin International College, a joint initiative between UCD and BJUT. He is a Funded Investigator for the CONSUS SFI Center and was a Collaborator on the EU-Funded AHA—AdHd Augmented Project.
[email protected]
View All Abraham Campbell's Posts
The unsustainable model: The tug of war between LLM personalization and user privacy
Authors:
Shuhan Sheng
Posted: Thu, January 04, 2024 - 11:32:00
In the AI universe, we find ourselves balancing on the edge of a tightrope between groundbreaking promise and perilous pitfalls, especially with those LLM-based platforms we broadly label as “AI platform.” The allure is magnetic, but the risks? They’re impossible to ignore.
AI Platform’s Information Interaction and Service Features: An Unsustainable Model
Central to every AI platform is its adeptness in accumulating, deciphering, and deploying colossal data streams. Such data, often intimate and sensitive, is either freely given by users or, at times, unwittingly surrendered. The dynamic appears simple: You nourish the AI with data, and in reciprocation, it bestows upon you tailored suggestions, insights, or solutions. It’s like having a conversation with a very knowledgeable friend who remembers everything you’ve ever told them. But here’s the catch: Unlike your friend, AI platforms store this information, often indefinitely, and use it to refine their algorithms, sell to advertisers, or even share with third parties.
The current mechanisms of information exchange between AI platforms and users can be likened to a two-way street with no traffic lights. On one flank, users persistently pour their data, aspiring for superior amenities and encounters. Conversely, AI platforms, with their insatiable quest for excellence, feast upon this data, often devoid of explicit confines or oversight. Such unbridled data interchange has culminated in palpable apprehensions, predominantly surrounding user privacy and potential data malfeasance [1].
Following this unchecked two-way data street, the unsustainable model now forces a dicey trade-off between “personalized experiences” and “personal data privacy.” This has led to a staggering concentration of user data on major AI platforms, at levels and depths previously unimaginable. What’s more alarming? This data pile just keeps growing with time. And let’s not kid ourselves: These AI platforms, hoarding mountains of critical user data, are far from impenetrable fortresses. A single breach could spell disaster.
One of the most recent and notable incidents involves ChatGPT. During its early deployment, there was an inadvertent leak of sensitive commercial information. Specifically, Samsung employees reportedly leaked sensitive confidential company information to OpenAI’s ChatGPT on multiple occasions [2]. This incident not only caused a stir in the tech community but also ignited a broader debate about the safety and reliability of AI platforms. The inadvertent leak raised concerns about the potential misuse of AI in business espionage, the risk of exposing proprietary business strategies, and the potential financial implications for companies whose sensitive data might be inadvertently shared.
Whether we like it or not, we need to see and face this fact. The current rapidly growing information interaction model based on traditional user data storage methods is unsustainable. It’s a ticking time bomb, waiting for the right moment to explode. And unless we address these issues head-on, we are bound for a digital disaster.
User Experience Design Based on This Mechanism: Subsequent Problems and Challenges
Beyond the lurking privacy threats and the looming digital apocalypse, this unsustainable info-exchange model is already a thorn in the user experience side. Let’s dive deeper into these annoyances to grasp the gravity of the situation.
The cross-platform invocation dilemma. Major AI platforms operate in silos, creating a fragmented ecosystem where user data lacks interoperability. With the advent of new models and platforms, this fragmentation is only intensifying [3]. Imagine having to introduce yourself every time you meet someone, even if you’ve met them before. That’s the predicament users find themselves in. Every time they switch to a new AI platform, they’re forced to retrain the system with their personal data to receive customized results. This not only is tedious but also amplifies the risk of data breaches. It’s like giving out your home address to every stranger you meet, hoping they won’t misuse it.
Inefficiencies in historical interaction records. The current AI models have a flawed approach to storing and managing historical interaction records [4]. Take ChatGPT, for instance. Even within the platform, one session’s history can’t give a nod to another’s. It’s like they’re strangers at a party. Users struggle to retrieve past interactions, making the entire process of data retrieval cumbersome and inefficient. This inefficiency not only frustrates users but also diminishes the value proposition of these platforms.
Token overload in single channels. Information overload is a real concern. When a single channel is bombarded with excessive information, the AI platform’s performance takes a hit [5]. It’s like trying to listen to multiple radio stations at once; the result is just noise. The current model’s technical limitations become evident as it struggles to scale with increased user interaction, leading to slower response times and a degraded user experience.
A Call for Change
As we draw the curtains on our discussion, it’s evident that the current AI ecosystem, while revolutionary, is far from perfect. The model’s unsustainability is not just a theoretical concern but rather a tangible reality that users grapple with daily.
The complexity of data misuse is a significant concern. Both active and passive data misuse are like icebergs—what’s visible is just the tip, and the real danger lurks beneath the surface. These misuses are not only concealed but also highly unpredictable. It’s akin to navigating a minefield blindfolded; one never knows when or where the next explosion will occur.
Relying solely on corporate responsibility and legal regulations is akin to putting a band-aid on a gunshot wound. While these measures might offer temporary relief, they don’t address the root cause of the problem. The need of the hour is a fundamental change. We must advocate for a deeper, more profound, root-level redesign of the information interaction mechanisms between users and AI platforms. It’s not just about patching up the existing system but envisioning a new one that prioritizes user experience, privacy, and security.
The AI ecosystem is at a crossroads. We can either continue down the current path, ignoring the glaring issues, or we can take the bold step of overhauling the system. The choice is clear: For a more sustainable, ethical, and user-friendly AI future, change is not just necessary; it’s imperative.
Endnotes
1. Harari, Y.N. 21 Lessons for the 21st Century. Spiegel & Grau, 2018.
2. Greenberg, A. Oops: Samsung employees leaked confidential data to ChatGPT. Gizmodo. Apr. 6, 2023; https://gizmodo.com/chatgpt-ai...
3. Forbes Tech Council. AI and large language models: The future of healthcare data interoperability. Forbes. Jun. 20, 2023; https://www.forbes.com/sites/f...
4. Broussard, M. The challenges of AI preservation. The American Historical Review 128, 3 (Sep. 2023), 1378–1381; https://doi.org/10.1093/ahr/rh...
5. Vontobel, M. AI could repair the damage done by data overload. VentureBeat. Jan. 4, 2022; https://venturebeat.com/datade...
Posted in:
on Thu, January 04, 2024 - 11:32:00
Shuhan Sheng
An entrepreneurial spirit and design visionary, Shuhan Sheng cofounded an industry-first educational-corporate platform, amassing significant investment. After honing his craft in interaction design at ArtCenter College of Design, he now leads as chief designer and product director for two cutting-edge AI teams in North America. His work has garnered multiple international accolades, including FDA and MUSE Awards.
[email protected]
View All Shuhan Sheng's Posts
Body x Materials @ CHI: Exploring the intersections of body and materiality in a full-day workshop
Authors:
Ana Tajadura-Jiménez ,
Bruna Petreca,
Laia Turmo Vidal,
Ricardo O'Nascimento ,
Aneesha Singh
Posted: Wed, January 03, 2024 - 2:31:00
During the 2023 CHI conference, we ran a one-day workshop to consider the design space at the intersection of the body and materials [1]. The workshop gathered designers, makers, researchers, and artists to explore current theories, approaches, methods, and tools that emphasize the critical role of materiality in body-based interactions with technology. We were motivated by developments in HCI and interaction design over the past 15 years, namely the “material turn,” which explores the materiality of technology and computation and methods for working with materials, and “first-person” approaches emphasizing design for and from lived experience and the physical body. Recognizing the valuable contributions of approaches that foregrounded materiality [2] and the body [3] in HCI, we proposed to explore the intersection of these two turns.
Material interactions is a field of wide-ranging interest to HCI researchers, starting with research on the design and integration of physical and digital materials to create interactive and embodied experiences [4], progressing to a growing interest in the contextual aspects of material interactions and their impact on personal and social experiences [2], and culminating in formalized method propositions such as “material-centered interaction design” [5]. The latter is a fresh approach, urging interaction designers to broaden their view beyond the capabilities of the computer and embrace a practice that imagines and designs interaction through material manifestations. Similarly, the role of the body in design has gained traction, through approaches such as soma design [3], embodied interaction, and movement-centric methods [6]. These have positioned the centrality of first-person accounts and physical engagement to research and design from the body, shifting the design space beyond security and efficiency, which are primary goals, for example, in the field of human factors, to deal with other dimensions such as fun/joy, play, entertainment, mental health interventions, and so on. In the workshop, the methods and prototypes brought in aimed to explore experience, for instance in terms of perceptions and feelings toward materials (Figure 1) or toward the person’s own body (Figure 2). In both cases, the interaction with the prototypes brings the subjective experience into focus. We share a keen interest with previous research in working across physical and digital materials, as well as in centering the body in design. Additionally, we have a particular interest in the intersection of these approaches in which we find a new design space: making the body central material in designing experiences, going beyond a view of the body as the medium with which to explore the material—its potentials, its affordances—instead including the body as part of the material being designed with and for. We are excited to develop this design space through a formal proposition, as we progress with our research.
Our proposed approach is both timely and critical, as it addresses the complex interplay between the body and emerging technologies. However, this approach is not free from challenges, as discussed in the workshop. These technologies not only have a tangible impact but also influence immaterial aspects, creating unpredictable outcomes that must be carefully considered during the design phase. As technologies increasingly become a ubiquitous part of our lives, they affect our bodies in both physiological and psychological ways. Furthermore, moving this research beyond the confines of a laboratory not only introduces ethical concerns in relation to health, well-being, safety, and social impact, but also opens new opportunities. For example, in terms of social impact, the technologies can directly affect not just individual bodies, but also the interpersonal relationships those individuals maintain. Therefore, any interaction with embodied technologies carries real-world implications that must be approached with caution, but also with curiosity about the opportunities they may open.
The Workshop
Throughout our day-long event, attendees took part in various activities and were organized into four thematic groups: material enabling expression; material as a catalyst for human action; material enabling reflection, awareness, and understanding; and material supporting the design process for (re)creating the existing and the yet-to-exist.
In the first activity, participants shared their prototypes or methods within their distinct groups. Prototypes were examined in relation to each group’s theme and participants individually noted similarities, differences, and opportunities among them. The second activity mixed groups for participants to discuss key points across themes, to uncover overlaps and opportunities.
<
Figure 1. One of the methods (Materials Gym) presented at the workshop. EMG is used alongside a smartphone application to continuously collect data on people’s interaction with materials: Speed and aspects of movement reveal information about the experience in relation to people’s perceptions of and preferences for materials qualities (see https://discovery.ucl.ac.uk/id/eprint/10172204/).
![](/images/uploads/blogs/Figure2_web_2.jpg)
Figure 2. A physical prototype (Soniband) that one participant brought to the workshop. This is a wearable band that provides real-time sonification of movement through a variety of sounds, some of which build on material metaphors, such as water, wind, or mechanical gears, and which can impact on the wearer’s body perceptions and feelings, movement, and emotional state (see https://doi.org/10.1145/3411764.3445558; https://doi.org/10.1145/2858036.2858486)
To increase enthusiasm into our exploration, the workshop featured a panel with four guest speakers who presented their research through a provocation: Kristii Kuusk (Estonian Academy of Arts) talked about the impact of material on people’s bodies combining affordances of technology and traditional materials; Pedro Lopes (University of Chicago) talked about making the body the material from which we build the actuator of the designed technologies; Hasti Seifi (Arizona State University) talked about the need for integrating basic research on haptic sensory perception and language into software tools for designing body-based haptic experiences; Paul Strohmeier (Max Planck Institute for Informatics) talked about how material and body experiences are shaped by agency, control, and reflective processes. The panelists conversed about material experiences and engaged in a Q&A session with participants. To conclude, participants mapped the design space of “body x materials” with their original groups, considering challenges, opportunities, state-of-the-art, theories, and methodologies.
![](/images/uploads/blogs/Figure3_web_3.jpg)
Figure 3. Participants engaging in workshop activities.
The diversity of backgrounds of workshop participants (position papers available at www.rca.ac.uk/body-materials) resulted in multidisciplinary discussion groups where different disciplines, theories, methodologies and application domains were considered (including HCI, interaction design, neuroscience, arts and crafts, and material design). Through discussions, this diverse group of participants arrived at shared conclusions. Participants highlighted the importance and timeliness of the topic. Many emphasized the need to establish a clear vision of the relationship between the body and materials for HCI. They foregrounded the challenges posed by the complexities of reconciling detailed fine-grain descriptions with a holistic perspective of the overall experience, which arise both from the dynamic nature of the experience of materials, conveyed through sensorimotor feedback, and the continuous interpretation of the material qualities involved in the act of touching. Understanding the underlying processes leading to these interpretations and how awareness levels affect interaction was perceived by participants as critical. Several participants stressed the importance of incorporating first-person perspectives into design.
The discussion spotlighted additional challenges stemming from language limitations and the necessity of cultivating a language specific to touch. Two key points emerged from this discussion. First, participants underscored the importance of establishing a shared vocabulary that enables effective communication among researchers who may have diverse backgrounds. Second, it was considered that in the translation of touch experiences into verbal descriptions significant information tends to be lost. Participants also explored the optimal data format for representing body-based research, acknowledging the existence of ongoing attempts.
Finally, the topic of shifting agency between the body and materials was explored. Two approaches were highlighted: empowering individuals to create their own unique material experiences as part of the design process, and the concept of “controlling the body to experience something: where the body acts as an actuator. The tension between enabling individuals to engage with the experience and exerting control over the body for a specific experience was also acknowledged during the discussion.
For those interested in working in this novel design space at the intersection of body x materials, we synthesize three key takeaways from our workshop. These takeaways emphasize: 1) the significance of comprehending the fundamental processes underlying experiences within this intersection, 2) the need to employ appropriate methods to achieve this understanding, including the integration of first-person perspectives into the design process, and 3) the importance of establishing a shared vocabulary. The latter includes facilitating the translation and representation of experiences and research outcomes related to the body x material interactions.
Conclusions
Everyone agrees that mapping the state-of-the art of the body x materials space is necessary for creating a more significant impact beyond lab experimentation. Nevertheless, a cautionary note was also struck—participants noted that it is essential to slow down and account for the ethical implications of how this research affects people psychologically and otherwise. To move the field forward and generate impact, participants agreed that interdisciplinary conversations must be fostered and the real-life implications of materiality x body understood, both in terms of possibilities and positive and negative effects of our work. The atmosphere at the workshop was open and interdisciplinary, and it was clear among participants that more such interdisciplinary conversations around the real-life implications of body x materials need to be nurtured to bring a shared understanding to the field. With this aim, the workshop participants planned to consolidate existing materials and findings at future workshops, including one that was run at the IEEE World Haptics 2023 conference and attempt to have a repeat workshop at CHI 2024 to further explore the directions that emerged. The long-term goal is to build an interdisciplinary community and open the design space for material-enabled, body-based multisensory experiences by integrating research from various perspectives.
Acknowledgments
We would like to thank all workshop organizers (the authors of this piece were joined by: Hasti Seifi, Judith Ley-Flores, Nadia Bianchi-Berthouze, Marianna Obrist and Sharon Baurley), guest speakers, and participants for their fantastic contributions. We acknowledge funding by: the Spanish Agencia Estatal de Investigación (PID2019-105579RB-I00/AEI/10.13039/501100011033) and the European Research (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 101002711). The work of Bruna Petreca and Ricardo O’Nascimento was funded by UKRI grant EP/V011766/1. For the purpose of open access, the authors have applied a Creative Commons Attribution (CC BY) license to any Author Accepted Manuscript version arising.
Endnotes
1. Petreca, B.B. et al. Body x materials: A workshop exploring the role of material-enabled body-based multisensory experiences. Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2023; https://doi.org/10.1145/354454...
2. Giaccardi, E. and Karana, E. Foundations of materials experience: An approach for HCI. Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, 2015, 2447–2456; https://doi.org/10.1145/270212...
3. Höök, K. Designing with the Body: Somaesthetic Interaction Design. MIT Press, Cambridge, MA, 2018.
4. Ishii, H. and Ullmer, B. March. Tangible bits: Towards seamless interfaces between people, bits and atoms. Proc. of the ACM SIGCHI Conference on Human factors in Computing Systems. ACM, New York, 1997, 234–241.
5. Wiberg, M. The Materiality of Interaction: Notes on the Materials of Interaction Design. MIT Press, Cambridge, MA, 2018.
6. Wilde, D., Vallgarda, A., and Tomico, O. Embodied design ideation methods: Analysing the power of estrangement. Proc. the 2017 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2017, 5158–5170; https://doi.org/10.1145/302545...
Posted in:
on Wed, January 03, 2024 - 2:31:00
Ana Tajadura-Jiménez
Ana Tajadura-Jiménez is an associate professor at the DEI Interactive Systems Group, Universidad Carlos III de Madrid. She leads the i_mBODY lab (
https://www.imbodylab.com) focused on interactive, multisensory, body-centered experiences, at the intersection between the fields of HCI and neuroscience. She is principal investigator of the MagicOutFit and the BODYinTRANSIT projects.
[email protected]
View All Ana Tajadura-Jiménez 's Posts
Bruna Petreca
Bruna Petreca is a senior research fellow in human experience and materials at the Materials Science Research Centre of the Royal College of Art. She co-leads the Consumer Experience Research Strand of the UKRI Interdisciplinary Textile Circularity Centre and is a co-investigator on the project EPSRC Consumer Experience Digital Tools for Dematerialisation.
[email protected]
View All Bruna Petreca's Posts
Laia Turmo Vidal
Laia Turmo Vidal is a Ph.D. candidate in interaction design and HCI at Uppsala University, Sweden. In her research, she investigates how to support movement teaching and learning through interactive technology. Her research interests include embodied design, cooperative social computing, and play.
[email protected]
View All Laia Turmo Vidal's Posts
Ricardo O'Nascimento
Ricardo O'Nascimento is a postdoctoral researcher in human experience and materials at the Material Science Research Centre of the Royal College of Art. His research explores how new technologies challenge and enhance human perception with focus on on-body interfaces and hybrid environments.
ricardo.o'[email protected]
View All Ricardo O'Nascimento 's Posts
Aneesha Singh
Aneesha Singh is an associate professor in human-computer interaction at UCL Interaction Centre. She is interested in the design, adoption, and use of personal health and well-being technologies in everyday contexts, focusing on sensitive and stigmatized conditions. Her research areas include digital health, ubiquitous computing, multisensory feedback, and wearable technology.
[email protected]
View All Aneesha Singh's Posts
Usability: The essential ingredient for sustainable world development
Authors:
Elizabeth Rosenzweig,
Amanda Davis,
Zhengjie Liu,
Deborah Bosley
Posted: Tue, January 02, 2024 - 11:29:00
As we make interacting with technology more like interacting with human beings, it is important that we understand how people use technology.…That is why usability is so important.
— Bill Gates [1]
Professionals in user experience (UX) and technology recognize the significant impact well-designed technology has on everyday life. In a digital age where interfaces are everywhere, ensuring a positive user experience is crucial. It’s not merely about creating visually appealing designs; it’s about making technology intuitive, accessible, and genuinely useful for everyone.
We aim to do more than just influence individual applications; we strive to influence policymakers and decision-makers worldwide. Our advocacy revolves around recognizing usability as a fundamental human right. We firmly believe that access to technology that is easy to use and understand should be a universal entitlement, akin to essentials like clean water and education.
Our vision is to create a world where technology becomes a tool for empowerment, enabling individuals to navigate the digital landscape with confidence and ease. By championing usability as a basic human right, we’re paving the way for a future where innovative solutions are not only cutting edge but also universally accessible. Our goal is to ensure technology becomes a catalyst for positive change in the lives of people across the globe.
To achieve this vision, we emphasize the significance of World Usability Day (WUD), a pivotal event promoting the values of usability and user-centered design. By enhancing the visibility of this initiative, we can raise not only professionals’ awareness but also that of the general public. WUD serves as a platform to showcase inventive ideas, best practices, and research findings, highlighting how usability directly affects people’s lives. It embodies a global commitment to enhancing user experiences and ensuring technology is accessible to everyone. This influential event transcends geographical boundaries, uniting diverse communities, from professionals and industrial experts to educators, citizens, and government representatives.
The fundamental objective of WUD is both profound and straightforward: to ensure that essential services and products, vital to everyone’s lives, are user friendly and straightforward. By advocating for usability, WUD ensures that technological advancements don’t isolate users with complexity but empower them, regardless of their background, expertise, or physical abilities.
History of World Usability Day
The first invited essay about World Usability Day was in August 2006; the message still rings true today: “Every citizen on our planet deserves the right to usable products and services. It is time we reframe our work and look at a bigger global picture” [2].
For nearly two decades, WUD has been a catalyst for change, reaching practitioners in every corner of the globe. Across more than 140 countries, WUD has engaged over 250,000 individuals and had an impact on their local communities (see Table 1). WUD has opened up the field of UX and usability in places where it did not exist before the event, such as Eastern Europe (including Poland, Ukraine, Russia, and Turkey). With its annual theme, practitioners and volunteers have come together across the globe to tackle topics that matter to humanity.
| Theme
| Events
| Countries
| Individuals Engaged
|
2005
| Make it Easy
| 115
| 35
| 6,500
|
2006
| Accessibility
| 228
| 40
| 38,000
|
2007
| Healthcare
| 157
| 41
| 29,000
|
2008
| Transportation
| 161
| 43
| 50,000
|
2009
| Designing for a Sustainable World
| 150
| 43
| 48,000
|
2010
| Communication
| 150
| 40
| 45,000
|
2011
| Education: Designing for Social Change
| 120
| 42
| 7,000
|
2012
| Usability of Financial Systems
| 80
| 20
| 6,000
|
2013
| Healthcare: Collaborating for Better Systems
| 107
| 32
| 7,000
|
2014
| Engagement
| 135
| 40
| 8,500
|
2015
| Innovation
| 91
| 31
| 6,500
|
2016
| Sustainability
| 73
| 24
| 6,000
|
2017
| Inclusion
| 66
| 25
| 6,500
|
2018
| Design for Good or Evil
| 62
| 26
| 6,500
|
2019
| Design for the Future We Want
| 56
| 27
| 6,500
|
2020
| Human-Centered AI
| 42
| 26
| 5,500
|
2021
| Trust, Ethics, Integrity
| 47
| 25
| 4,550
|
2022
| Healthcare
| 32
| 25
| 4,550
|
2023
| Collaboration and Cooperation
| 48
| 20
| 4575
|
Table 1. Themes and global WUD participation.
World Usability Initiative
The founders of World Usability Day want to mobilize people in countries around the world to develop technology that works for the greater good. We want to expand the reach of recognizing the significance of usability once per year so that we can have a larger impact in tackling the world’s biggest problems.
To achieve this goal, they collaborated with renowned professional associations like SIGCHI, HCII, PLAIN, and IFIP. Together, they established the World Usability Initiative (WUI), a focused global organization.
Operating as a singular, dedicated entity, WUI works closely with the United Nations, particularly concerning the human factors integral to the 17 Sustainable Development Goals. This initiative unites experts from various fields, including human-computer interaction, user experience, and interaction design. Their collective efforts are geared toward leading researchers, developers, and countries in creating technology that is not only user centered but also aligned with core human values.
Our mission is to partner with the UN to tackle the 17 Sustainable Development Goals for 2030. Our initial initiatives will focus on:
- Establishing World Usability Day as an internationally observed day: WUI aims to connect the HCI-UX field with the United Nations by listing WUD on the UN calendar, ensuring global recognition.
- Rewarding good design with the WUI Design Challenge: This initiative rewards exceptional design by professionals across five countries each year, promoting user-centered innovation.
- Connecting communities through the International Speaker Series: To foster community engagement, WUI hosts an international speaker series, providing a platform for professionals to connect and share ideas.
- Creating projects that achieve the UN Sustainable Development Goals: Create projects within developed and developing countries that connect with the UN Sustainable Development Goals, adopting a user-centered approach.
- Creating a UN-based World Usability Organization: From the perspective of public policy formulation and implementation, the organization will focus on ensuring technology products and services will be made accessible and usable for everyone.
Returning to that 2006 essay about World Usability Day: “The challenge of World Usability Day is not small; it is to change the way the world is developing and using technology” [2].
Your support is instrumental in achieving WUI’s objectives and transforming the world into a more inclusive, equitable place. You can actively contribute by:
- Participating in the WUI Design Challenge: Showcase your innovative designs, promoting user-centered solutions.
- Organizing a World Usability Day event: Contribute to the global conversation by hosting an event, fostering awareness and inclusivity.
- Engaging in the speaker series: Share your expertise and ideas, connecting with professionals worldwide.
- Signing the petition: Join the collective voice urging the UN to recognize World Usability Day’s significance.
- Sponsoring WUD: Support WUD financially, enabling the movement to reach new horizons.
In the spirit of global collaboration, let us unite to create technology that serves humanity inclusively, ensuring usability becomes a universal reality. Together, we can pave the way for a world where technology truly becomes a force for good, enriching lives, fostering innovation, and championing inclusivity.
Endnotes
1. Bill Gates at World Usability Day 2007. YouTube;https://www.youtube.com/watch?v=mpxYYz1QHqQ&t=7s
2. Rosenzweig, E. World Usability Day: A challenge for everyone. Journal of User Experience 1, 4 (2006), 151–155; https://uxpajournal.org/world-...
Posted in:
on Tue, January 02, 2024 - 11:29:00
Elizabeth Rosenzweig
Elizabeth Rosenzweig is a design researcher who uses technology to make the world a better place. She believes that the best design comes from good research through user-centered design, and as the founder of World Usability Day has been able to push the boundaries of the status quo. She holds four patents on intelligent design for image management and is the author of Successful User Experience: Strategies and Roadmaps.
[email protected]
Website:
https://designresearchforgood.org/
View All Elizabeth Rosenzweig's Posts
Amanda Davis
Amanda Davis, founder of and lead consultant at Experiment Zone, excels in guiding companies on usability and conversion rate optimization. As an active board member of the World Usability Initiative, she champions initiatives for global user-friendly experiences, embodying expertise at the forefront of user-centric design and advocacy.
[email protected]
View All Amanda Davis's Posts
Zhengjie Liu
Zhengjie Liu is professor emeritus of HCI/UXD at Dalian Maritime University in China. He is an HCI pioneer who has been working in the field since 1989, and especially has helped the development of UXD practice in industry in China. He has served international communities in various committees, including for ACM SIGCHI, IFIP TC.13 Committee on HCI and ISO WGs for HCI standards, focusing on promoting HCI in developing worlds. He is awardee of ACM SIGCHI Lifetime Service Award in 2017 and IFIP TC.13 Pioneers Award in 2013.
[email protected]
View All Zhengjie Liu's Posts
Deborah Bosley
Deborah S. Bosley is the founder and principal of The Plain Language Group, LLC. For the past 20 years, she has worked with Fortune 500 companies, government agencies, and non-profits to make complex content easy to understand. She provides training, plain language revisions, usability testing, and expert witness testimony.
[email protected]
View All Deborah Bosley's Posts
Democratizing game authoring: A key to inclusive freedom of expression in the metaverse
Authors:
Amir Reza Asadi
Posted: Thu, November 16, 2023 - 2:03:00
As we approach the metaverse, a sociotechnical future where people consume huge amounts of information in 3D formats, the significance of game development tools, including both game-engine and digital-content-creation tools, will become more and more evident. In a metaverse-oriented world, you need to create 3D experiences to express your ideas. Game engines are currently the main tools for authoring and creating AR/VR experiences; therefore, simplifying the game-authoring process is crucial for the future of freedom of expression.
Video games are the most advanced form of interactive media, conveying diverse ideas and often representing mainstream ideologies. However, making a small but sophisticated video game is an expensive form of storytelling. This issue may not be a concern for many of us because the main purpose of digital play is entertainment. But it should be a vital issue for all of us, as a metaverse-oriented world is made up of 3D game-like experiences. Those who can create more immersive experiences may take control of real-world narratives, so it’s a challenge and an important goal for us to make game development inclusive of more voices. Without simplifying the game-authoring process, our society could lose its freedom to effectively communicate and express itself, resulting in reduced empathy.
We should also not underestimate the threat of propaganda in the metaverse. The metaverse allows users to immerse themselves in dreams of virtual prosperity, and people may shelter in the virtual world to overcome the pain of poverty in the real world. Dreaming has played a vital role in the history of humanity. We dream at times when we are unhappy with our circumstances, and no dictatorship in history has been able to control our dreams [1]. With the rise of emerging non-democratic powers and economies, there is a threat that companies will start changing or censoring the ideologies of their game-alike experiences, similar to what happened to the movie industry in recent years [2,3]. In a dystopian scenario, people will trade their ability to dream for the ability to live in virtual game-alike experiences. Losing the ability to dream freely can have dangerous consequences, as it can hold people captive to mainstream ideas. But there is a silver lining. Transforming game-authoring tools in the same way that emergent video making/sharing apps such as TikTok, Snapchat, and Apple Clips have democratized video-based storytelling can give us hope that all ideas can be expressed not just freely but also effectively in the Metaverse.
A Desirable Future for Game-Creation Tools
In today’s world, game development has become simpler with free-to-use game engines, but the pipeline of game development is still complicated. We need to simplify the workflow of game-authoring tools for tomorrow. I’m not talking about a magic-wand solution here. The technologies for simplifying game authoring are available today, but we need to first let go of our previous assumptions.
The designers of game-authoring tools should create these tools based on new personas. They need to think big and assume that the intellectuals of the metaverse generation will be game authors rather than book authors. Based on this desirable future, we’ve created a persona (Figure 1) who represents a thinker, representing him with an image of Montesquieu, the famous French political philosopher. We also created a design fiction about his life:
DJ Montesquieu was a political philosopher who sought to challenge traditional modes of philosophical discourse. He saw the world through a different lens, one that viewed technology and innovation as a means of disseminating his ideas more effectively to a wider audience. DJ Montesquieu’s unique approach to philosophical discourse led him to develop his thesis not in the form of a book, but rather as a video game. His thesis was a video game that allows players to experience the impacts of overregulation from the eyes of ordinary citizens. Unlike traditional philosophers, Montesquieu was not interested in writing for the sake of writing. He was more concerned with sharing his views in a way that was accessible and engaging for people. To this end, he utilized the AlternateGameEngine, which is a combination of no-code tools and ChatGPT-based tools to create immersive AR/VR games that brought his ideas to life.
DJ Montesquieu’s games were not just entertaining; they were also thought-provoking. They allowed players to explore complex philosophical concepts in a way that was both fun and enlightening. By using technology as a medium for philosophical discourse, DJ Montesquieu was able to reach a wider audience than he ever could have with traditional writing.
![](/images/uploads/blogs/Persona_web.jpg)
Figure 1. The persona of a future political philosopher, DJ Montesquieu.
In other words, imagine a future where intellectuals and authors, instead of using word processing tools and presentation software, will be using 3D game-creation tools. Where people may quote game authors instead of Friedrich Nietzsche, and where famous people will create games instead of autobiographies. In this future, the most popular social network allows users to turn their diaries into snack-sized gaming experiences that can be experienced by others.
This future also allows society to understand situations that they cannot or do not want to comprehend by reading books. The combination of the immersive nature of the metaverse with simplified game-authoring tools will enable societies to have more ethnocultural empathy, as they can experience life from others’ point of view. Immersive 3D virtual environments have already proven effective in creating empathy among different ethnic groups [4,5], so simplifying the pipeline of immersive storytelling can create opportunities to have a more united society. Imagine that instead of watching partisan debates on TV and being exposed to online political bots, every person could share their ideas, trauma, or values via immersive experiences in the same way they can create content for social media, allowing us to live in a more empathetic community. People may not be able to express their own side of story through verbal interactions, but easy immersive-experience authoring may allow them to express their stories like livable dreams that others can experience themselves.
Endnotes
1. Rare 1997 interview with Abbas Kiarostami, conducted by Iranian film scholar Jamsheed Akrami. From Taste of Cherry Featurettes, 1997. YouTube, Mar. 1, 2021; [https: www.youtube.com="" watch?v="VTcm4P5qAF8"]
2. DeLisle, J. Foreign policy through other means: Hard power, soft power, and China’s turn to political warfare to influence the United States. Orbis 64, 2 (2020), 174–206.
3. Su, W. From visual pleasure to global imagination: Chinese youth’s reception of Hollywood films. Asian Journal of Communication 31, 6 (2021), 520–535.
4. Coffey, A.J., Kamhawi, R., Fishwick, P., and Henderson, J. The efficacy of an immersive 3D virtual versus 2D web environment in intercultural sensitivity acquisition. Educational Technology Research and Development 65 (2017), 455–479.
5. Lamendola, W. and Krysik, J. Cultivating counter space: Evoking empathy through simulated gameplay. ETC Press, 2022.
Posted in:
on Thu, November 16, 2023 - 2:03:00
Amir Reza Asadi
Psychological privacy: How perceptual publicity can support perceived publicity
Authors:
Wee Kiat Lau,
Lisa Valentina Eberhardt ,
Marian Sauter,
Anke Huckauf
Posted: Mon, October 23, 2023 - 10:45:00
Picture this: Sitting in his kitchen, young Tim chuckles at the recent misfortune of neighbors who fell prey to burglars. At the same time, he’s enthusiastically experimenting with a banking app on his fresh-off-the-assembly-line computer, a machine devoid of even the most fundamental antivirus protection. This scenario is a striking illustration of the privacy paradox. We voice anxiety about our data’s usage, often lambasting corporations with lax privacy protocols, only to defy our apprehensions by not embracing measures to safeguard ourselves. Let’s dissect this intriguingly paradoxical user behavior from the lens of perceptual psychology, illuminating the circumstances under which we sense privacy or publicity, distinguishing these states, and suggesting ways to aid individuals like Tim.
Perception in Private and Public Surroundings
Envision an intensely private moment: reclining on your sofa, shaking off the weariness of work. Now, contrast this with a public spectacle: accepting accolades onstage for special achievements, with a massive audience bearing witness. How do these scenarios make you feel, and what sets them apart? To shed light on this question, we will go deep into our bodies’ states and processes. A key environmental distinction arises when we feel private—the high likelihood of being cocooned in a familiar setting, surrounded by well-known objects and people, which lets us unwind. New experiences, however, kick-start our alertness; they ignite our curiosity and grab our attention.
![](/images/uploads/blogs/PrivacyPublicity_web.png)
Fundamental cognitive processes in any living organism include sensation and perception. Familiar environments envelop us with recognizable items, triggering well-known sensations, be they sounds, scents, tactile sensations, or visuals. Therefore, private settings pose fewer challenges to the perceptual system about object identification and sensory memory. This reduction in demand allows the system to function at a lower sensitivity level, freeing up capacity for other operations.
This perceptual mode is accompanied by physiological processes. Broadly speaking, in relaxed, private settings, the parasympathetic neural activation (“rest and digest” mode) is dominant, whereas the sympathetic activation (“fight or flight” mode) is dominant in unfamiliar public settings. There are some remarkable perceptual effects caused by this: Sympathetic activation leads to larger pupils, resulting in more light input, in a slightly extended visual field, and in reduced visual acuity, especially outside of the eyes’ focus—that is, in periphery and in depth [1]. We therefore can assume that in public relative to private settings, we perceive with less spatial accuracy from a larger visual field.
Attention in Private and Public Surroundings
Perception is accompanied by an adaptation of attentional processes. All attentional functions respond to situational affordances. We can differentiate between alertness and selective attention. Alertness is the increase and maintenance of response readiness. It can be supposed to complement the general arousal level of an organism. Thus, alertness will be high in public settings, while it can be reduced when the organism is surrounded by familiar objects. The notion of a broader visual field, although with lower spatial resolution, can be plausibly assumed to support alerting functions in public settings.
Regarding selective attention, new salient stimuli are known to capture attention. This improves visual search performance. In private surroundings, distracting objects can be quickly identified and thus be effectively suppressed. This can lead to a phenomenon known as inattentional blindness: In familiar settings, it frequently happens that we miss even uncommon, unexpected objects. The visual system is also capable of suppressing distracters based on their spatial location [2], improving efficiency in familiar environments. The familiarity of surrounding objects eases not only the selection of task-relevant objects but also the suppression of distracting objects. Inhibition again saves capacity for other processes [3]. In unfamiliar public settings, however, stimuli must be processed until they are identified as harmless, and suppression of task-irrelevant stimuli is thus more difficult.
The dichotomy between private and public settings even manifests in our posture and movements. Onstage, we present our bodies to a large audience, making exaggerated, sweeping gestures. Conversely, in private, our muscles can relax, leading to smaller, more restrained movements. This difference extends to eye movements and gaze, which in turn influence perception.
Level of Control in Private and Public Surroundings
Taken together, private settings lessen the need to attend to external stimuli; you can unwind and rely on the consistency of the surroundings. Also, perceiving things provides already familiar information. All these processes diminish the need for cognitive control, allowing processing to occur more subconsciously. Consequently, executing learned skills, routines, and habits becomes more probable. Public behavior, however, is marked by unfamiliar surroundings. The influx of novel objects or people prompts a slew of questions: Is that unfamiliar face a threat? What does that unexpected sound imply? This cognitive appraisal demands effortful attention, sapping mental resources, making us more cautious in and conscious of our actions [3].
This thinking aligns with Daniel Kahneman’s [4] idea that human behavior is regulated either by quick, instinctive, and emotional processing (as in private settings) or by slower, more deliberative, and logical processing (common in public settings). Crucially, it’s nearly impossible to engage both methods simultaneously. Therefore, in a situation prompting emotional automated processing with only weak conscious monitoring, we’re hardly capable of producing analytical thinking with logical deductions. This means that if users engage with their personal devices at home, their behavior is dominated by automated, nonconscious routines.
Counteracting Perceived Privacy by Simulating a Public Audience
So, how can we assist users in selecting an appropriate level of control? Novelty in environmental stimuli can be an indicator. How we process these cues shapes how we perceive and interact with our surroundings, be they private or public. The cues could be signals or symbols [5]. Signals are automatic cues operating beneath conscious thought, like the familiar ticktock of a clock or the distinctive feel of your sofa, directing our arousal and attention needs. Conversely, symbols, such as GDPR text, demand conscious, detailed analysis, and interpretation, thus requiring higher cognitive capacity.
Discerning the psychological differences between private and public settings equips us with a potent tool to mold privacy behavior and promote prudent disclosure. As we’ve noted, using personal devices in private settings often sparks cues associated with private behavior, possibly leading to a false sense of security. Thus, introducing cues that simulate public scenarios could stimulate public consciousness, reminding users to be more circumspect with their disclosures. These could be visuals, sounds, smells, or other elements that evoke the public nature of their online interactions. One subtle method to induce a feeling of publicity could be the “watching eyes effect”: The presence of a pair of eyes can influence disclosure behavior and can be fine-tuned by varying emotional expression, sex, and age of the eyes [6]. Ideally, this should be achieved by incorporating design elements that subtly disrupt users’ familiar routines, prompting a cognitive response akin to being in a public setting.
Toward a Privacy-Sensitive Future
To conclude, traversing the maze of privacy behavior is an intricate task, yet understanding the interplay of environmental cues, perception, attention, and behavior control can illuminate our path forward. Preserving privacy might be bolstered by subtly simulating publicity within digital environments, evoking vigilance, and awareness akin to our natural responses in public settings. By doing so, we can harness our inherent cognitive and physiological processes.
Endnotes
1. Eberhardt, L.V., Strauch, C., Hartmann, T.S., and Huckauf, A. Increasing pupil size is associated with improved detection performance in the periphery. Attention, Perception, & Psychophysics 84, 1 (2022),138–149; https://doi.org/10.3758/s13414...
2. Sauter, M., Liesefeld, H.R., Zehetleitner, M., and Müller, H.J. Region-based shielding of visual search from salient distractors: Target detection is impaired with same- but not different-dimension distractors. Attention, Perception, & Psychophysics 80, 3 (2018), 622–642; https://link.springer.com/arti...
3. Posner, M.I. and Petersen, S.E. The attention system of the human brain. Annual Review of Neuroscience 13, 1 (1990), 25–42.
4. Kahneman, D. Thinking, Fast and Slow. Farrar, Straus, and Giroux, 2011.
5. Rasmussen, J. Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Transactions on Systems, Man, and Cybernetics SMC-13, 3 (1983), 257–266.
6. Lau, W.K., Sauter, M., Bulut, C., Eberhardt, L.V., and Huckauf, A. Revisiting the watching eyes effect: How emotional expressions, sex, and age of watching eyes influence the extent one would make stereotypical statements. Preprint, 2023; https://doi.org/10.21203/rs.3....
Posted in:
on Mon, October 23, 2023 - 10:45:00
Wee Kiat Lau
Lisa Valentina Eberhardt
Marian Sauter
Marian Sauter is a principal investigator in the General Psychology group at Ulm University. He is interested in selective attention and exploratory interactive search. He also works on applied topics such using gaze to predict quiz performance in online learning environments.
[email protected]
View All Marian Sauter's Posts
Anke Huckauf
Anke Huckauf is the chair of General Psychology at Ulm University, specialized in perceptual psychology and human-computer interaction. Currently, she serves as dean of the Faculty of Engineering, Informatics, and Psychology at Ulm University.
[email protected]
View All Anke Huckauf's Posts
Seven heuristics for identifying proper UX instruments and metrics
Authors:
Maximilian Speicher
Posted: Tue, September 19, 2023 - 11:09:00
In the two previous articles of this series, we have first learned that metrics such as conversion rate, average order value, or Net Promoter Score are not suitable to reliably measure user experience (UX) [1]. The second article then explained how UX is a latent variable and, therefore, we must rely on research instruments and corresponding composite indicators (that produce a metric) to measure it [2]. Now, the logical next question is how we can identify those instruments and metrics that do reliably measure UX. This boils down to what is called construct validity and reliability, on which we will give a brief introduction in this final article, before deriving easily applicable heuristics for practitioners and researchers alike who don’t know which UX instrument or metric to choose.
Construct validity refers to the extent to which a test measures what it is supposed to measure [3]. In the case of UX, this means that the instrument or metric should measure the concept of UX as it is understood in the research literature, and not, for example, only usability. One good way to establish construct validity is through factor analysis [3].
Construct reliability refers to the consistency of a test or measure [4]. Put differently, it is a measure of how reproducible the results of an instrument or metric are. A good way to establish construct reliability is through studies that assess the test-retest reliability of the instrument or metric, as well as its internal consistency, such as Cronbach’s alpha [4].
In addition to that, the Joint Research Centre of the European Commission (JRC) provides a “Handbook on Constructing Composite Indicators” [5], which summarizes the proper process in terms of a 10-step checklist. We build on all of the above for our following list of seven heuristics for identifying proper UX instruments and metrics.
Heuristic 1: Is there a paper about it? If there is no paper about the instrument and/or metric in question, there’s barely a chance you’ll be able to answer any of the following questions with yes. So, this should be the first thing to look for. A peer-reviewed paper published in a scientific journal or conference would be the best case, but there should be at the very least some kind of white paper available.
Heuristic 2: Is there a sound theoretical basis? In the case of UX, this means, does the provider of the instrument and/or metric clearly explain their understanding of UX and, therefore, what their construct actually measures? The JRC states: “What is badly defined is likely to be badly measured” [5].
Heuristic 3: Is the choice of items explained in detail? Why were these specific variables of the instrument chosen, and not others? And how do they relate to the theoretical framework, that is, the understanding of UX? The JRC states: “The strengths and weaknesses of composite indicators largely derive from the quality of the underlying variables” [5].
Heuristic 4: Is an evaluation of construct validity reported? This could be reported in terms of, for example, a confirmatory factor analysis [3]. If not, you can’t be sure whether the instrument or metric actually measures what it’s supposed to measure.
Heuristic 5: Is an evaluation of construct reliability reported? This could be reported in terms of, for example, Cronbach’s alpha [4]. If not, you can’t be sure whether the measurements you obtain are proper and reproducible approximations of the actual UX you want to measure.
Heuristic 6: Is the data that’s combined to form the metric properly normalized? This is necessary if the items in an instrument have different units of measurement. The JRC states: “Avoid adding up apples and oranges” [5].
Heuristic 7: Is the weighting of the different factors that form the metric explained? Factors should be weighted according to their importance. “Combining variables with a high degree of correlation” (double counting) should be avoided [5].
In the following, the application of these heuristics will be demonstrated through two very brief case studies.
Case Study 1: UEQ
The User Experience Questionnaire (UEQ) is a popular UX instrument developed at SAP AG.
- H1: There is a peer-reviewed research paper about UEQ, which is available at [6]. ✓
- H2: The paper clearly defines the authors’ understanding of UX, and they elaborate on the theoretical background. ✓
- H3: The paper explains the selection of the item pool and how it relates to the theoretical background. ✓
- H4: The paper describes, in detail, two studies in which the validity of UEQ was investigated. ✓
- H5: The paper reports Cronbach’s alpha for all subscales of the instrument. ✓
- H6: Not applicable, since UEQ doesn’t explicitly define a composite indicator. However, a composite indicator can be constructed from the instrument.
- H7: See H6.
Case Study 2: QX score “for measuring user experience”
This metric was developed by SaaS provider UserZoom and is now provided by UserTesting. It is a composite of two parts: 1) the widely used SUPR-Q instrument and 2) the individual task success rates from the user study where the metric was measured, in a 50/50 proportion.
- H1: There is no research paper, but at least a blog post explaining the instrument and metric. ✓
- H2: There is no clear definition of UX given. The theoretical basis for the metric is the assumption that all existing UX metrics use either only behavioral or only attitudinal data. There is no well-founded explanation given why this is considered problematic. The implicit reasoning is that only by mixing behavioral and attitudinal data can we properly measure UX, which is factually incorrect (cf. [2]). ❌
- H3: The metric mixes attitudinal (SUPR-Q) and behavioral (task success) items, but no well-founded reasoning is given as to why only task success rate was chosen, or why this would improve SUPR-Q, which is already a valid and reliable UX instrument in itself. ❌
- H4: No evaluation of construct validity is reported. ❌
- H5: No evaluation of construct reliability is reported. ❌
- H6: There is no approach to data normalization reported, the metric seemingly adds up apples and oranges. ❌
- H7: There is no reasoning given for the weighting of the attitudinal and behavioral items. ❌
In conclusion, the seven heuristics provided in this article serve as a useful guide for identifying proper UX instruments and metrics. Additionally, by considering construct validity and reliability, as well as following the JRC’s 10-step checklist, practitioners and researchers alike can make informed decisions when choosing a UX instrument or metric. It’s important to note that not all instruments or metrics will pass all of these heuristics, but the more of them that are met, the more confident one can be that the chosen instrument or metric properly measures UX. If in doubt, choose the instrument or metric that checks more boxes. It's worth noting that some heuristics, like H1, are not strictly necessary, and H6 and H7 only apply to composite indicators. It’s also worth noting that there may be valid instruments or metrics that fail some of these heuristics and vice versa. Their goal is to provide a robust and quick framework for evaluating UX instruments and metrics, but ultimately, the best approach will depend on the specific research or design project.
Endnotes
1. Speicher, M. Conversion rate & average order value are not UX metrics. UX Collective. Jan. 2022; https://uxdesign.cc/conversion...
2. Speicher, M. So, How Can We Measure UX? Interactions 30, 1 (2023), 6–7; https://doi.org/10.1145/357096...
3. Kline, R.B. Principles and Practice of Structural Equation Modeling. Guilford Publications, 2015.
4. Cronbach, L.J. and Meehl, P.E. Construct validity in psychological tests. Psychological Bulletin 52, 4 (1955), 281.
5. Joint Research Centre of the European Commission. Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD publishing, 2008; https://www.oecd.org/sdd/42495...
6. Laugwitz, B., Held, T., and Schrepp, M. Construction and evaluation of a user experience questionnaire. Symposium of the Austrian HCI and Usability Engineering Group. Springer, Berlin, Heidelberg, 2008, 63–76.
Posted in:
on Tue, September 19, 2023 - 11:09:00
Maximilian Speicher
Maximilian Speicher is a computer scientist, designer, researcher, and ringtennis player. Currently, he is director of product design at BestSecret and cofounder of UX consulting firm Jagow Speicher. His research interests lie primarily with novel ways to do digital design, usability evaluation, augmented and virtual reality, and sustainable design.
[email protected]
View All Maximilian Speicher's Posts
Toward a consensus on research transparency for HCI
Authors:
Florian Echtler,
Lonni Besançon ,
Jan Vornhagen,
Chat Wacharamanotham
Posted: Fri, August 25, 2023 - 11:15:00
During the past few years, the Covid-19 pandemic has resulted in an unprecedented amount of research being conducted and published in a very short timeframe to successfully analyze SARS-COV-2, its vaccines, and its treatments [1]. Concurrently, the pandemic also highlighted the limitations of our publication system, which enables and incentivizes rapid dissemination and questionable research practices [2].
While HCI research is usually not a foundation for life-and-death decisions, we face similar problems. HCI researchers and CHI community members have long criticized a lack of methodological [3] and statistical rigor [4] and a lack of transparent research practices in quantitative [5] and qualitative works [6]. Research transparency can alleviate these issues, as it facilitates the independent verification, reproduction, and—wherever appropriate—replication of claims. Consequently, we argue that the CHI community needs to move toward a consensus on research transparency.
Reaching this consensus is no trivial task, as HCI is an inter- or transdisciplinary field of research, employing myriad methods, and therefore cannot be subjected to a single, rigid rule set. For example, qualitative and quantitative data cannot be shared to the same degree without potentially identifying participants. Although the context of each research project might constrain feasible transparency practices, we believe that the overarching principle of research transparency can be applied to fit the vast variety of research that exists within our field.
Despite the benefits of transparent research practices, many of our efforts for more transparency within SIGCHI research have been met with considerable pushback. Junior researchers are worried about having their workload increase even more through additional documentation requirements, while more senior members of the community—who have built their careers on a research model that did not yet focus on transparency—may fail to see noticeable value in the additional effort required. Qualitatively oriented researchers, both within [6] and outside HCI [7], do not see themselves represented in a discussion that often focuses on statistics, data collection, and related topics. Meanwhile, some quantitative researchers mistook that preregistration limits potential exploratory analysis [8].
Ideally, any publicly available record of scholarship—whether it is a research paper, an essay, a dataset, or a piece of software—should be adequately transparent to enable the public to assess its quality and for subsequent research to build upon.
After all, the goal of research is not to pad one’s own h-index or to get a p-value below 0.05, but rather to increase the total sum of human knowledge. And in order to do so, we need to be able to build on the research that other scholars have done before us, so we can see farther. For this reason, transparency is indeed the one fundamental principle behind the open science movement, and it is so general that any of the subcommunities within HCI should be able to follow it.
The slow but steady move toward more transparency across all disciplines of research also provides a unique opportunity for the HCI community: At least some of the reluctance to adopt this approach stems from a lack of supporting tools. It is our field’s core expertise to uncover and analyze problems, and iteratively develop and evaluate potential solutions. Imagine that writing your statistical analysis would immediately update the figures in your paper and provide an automatic appendix with all calculations. Or that the coding process for interview quotes would keep a record of the shifting themes until the final result is reached, complete with an accurate log of the decisions that led to this point. These kinds of tools could be valuable far beyond HCI, for example, in related disciplines such as psychology and sociology, helping to establish a more transparent process there as well. Besides, the interdisciplinary nature of HCI allows us to test and debate innovations in methods, tools, and policies. We believe that this richness in perspectives and the technological and design capacity in our field could contribute to unblock open science conundrums, for example, challenges in sharing qualitative data [8].
For the future, we hope for guidelines that emphasize the values and opportunities within transparent research. We hope for reviewers who take them to heart, and include the transparency of a manuscript in their assessment, not as just superficial novelty. We hope for established researchers to lead by example, providing transparent insights into their research process, and for hiring committees and examination boards to value these contributions on their own. Last but not least, we hope for all members of the community to consider how they can increase transparency in their research and publication practices.
Endnotes
1. Callaway, E., Ledford, H., Viglione, G., Watson, T., and Witze, A. COVID and 2020: An extraordinary year for science. Nature 588, 7839 (2020), 550–553; pmid:33318685
2. Besançon, L., Bik, E., Heathers, J., and Meyerowitz-Katz, G. Correction of scientific literature: Too little, too late!. PLOS Biology 20, 3 (2022), e3001572; https://doi.org/10.1371/journa...
3. Greenberg, S. and Thimbleby, H. The weak science of human-computer interaction. (Dec. 1991); https://doi.org/10.11575/PRISM/30792
4. Cockburn, A. et al. HARK no more: On the preregistration of CHI experiments. Proc of the CHI Conference on Human Factors in Computing Systems. ACM, New York, 2018, 1–12; https://doi.org/10.1145/3173574.3173715
5. Vornhagen, J.B. et al. Statistical Significance Testing at CHI PLAY: Challenges and Opportunities for More Transparency. Proc. of the Annual Symposium on Computer-Human Interaction in Play. ACM, New York, 2020, 4–18.
6. Talkad Sukumar, P., Avellino, I., Remy, C., DeVito, M.A., Dillahunt, T.R., McGrenere, J. and Wilson, M.L. Transparency in qualitative research: Increasing fairness in the CHI review rrocess. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–6.
7. Kapiszewski, D. and Karcher, S. Transparency in practice in qualitative research. PS: Political Science & Politics 54, 2 (2021), 285–291; https://doi.org/10.1017/S1049096520000955
8. Wacharamanotham, C., Eisenring, L., Haroz, S. and Echtler, F. Transparency of CHI research artifacts: Results of a self-reported survey. Proc. of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020; https://dl.acm.org/doi/10.1145...
Posted in:
on Fri, August 25, 2023 - 11:15:00
Florian Echtler
Florian Echtler is an associate professor at Aalborg University in Denmark and a researcher in human-computer interaction. His core topics are centered around the intersection of ubiquitous computing, peer-to-peer networking, and security/privacy. He is a founding editor of the
Journal of Visualization and Interaction (JoVI;
https://www.journalovi.org/).
[email protected]
View All Florian Echtler's Posts
Lonni Besançon
Lonni Besançon is an assistant professor at Linköping University working with the Division of Media and Information Technology. His research focus is on 3D interaction for visualization research, on the one hand, and the role of visual representation for the understanding and communication of statistical uncertainty, on the other. He is a founding editor of the
Journal of Visualization and Interaction (JoVI;
https://www.journalovi.org/).
[email protected]
View All Lonni Besançon 's Posts
Jan Vornhagen
Jan Vornhagen is a Ph.D. fellow at the IT University of Copenhagen. He studies research practices and theory use in HCI games research, as well as the cognitive and emotional experience of videogames.
[email protected]
View All Jan Vornhagen's Posts
Chat Wacharamanotham
Chat Wacharamanotham is an independent researcher. His research focuses on understanding and developing tools for planning, reporting, reading, and sharing quantitative research. He co-organizes the Transparent Statistics in Human–Computer Interaction group (https://transparentstatistics.org) and is a founding editor of the
Journal of Visualization and Interaction (JoVI;
https://www.journalovi.org/).
[email protected]
View All Chat Wacharamanotham's Posts
Toward a mindful designer: Mindfulness as an essential skill for interaction designers
Authors:
Jia Shen
Posted: Tue, August 22, 2023 - 1:07:00
As digital technology designers, our role has never been more crucial than it is now. Decades of technological and commercial advancements have brought computing from research labs to business offices, and now to the homes and pockets of everyday users. Our focus has expanded from computer interfaces to user experiences to design thinking for executives in corporate boardrooms. Despite improvements in user experiences, we have observed that users are often becoming more mindless, joyless, and anxious in today’s hyperconnected world. With the rapid advancement of AI, wearables, and pervasive computing, these challenges are growing.
As designers, what must we add to our toolkit to reimagine and design a sustainable future? This essay advocates for the importance of mindfulness in the skill set of interaction designers. It highlights the specific areas where mindfulness can enhance designers’ abilities, and calls for the design community to work together toward incorporating mindfulness into their practice.
What Is Mindfulness and Why Should Designers Care?
During a visit to Google, the well-known Zen master Thich Nhat Hanh shared a story with Google employees. The story involved a person riding a horse at a fast pace and when asked where they were going, they replied, “I don't know, ask the horse!” Thay, as he is known to his followers, pointed out that our current use of technology can lead us to escape from ourselves, our loved ones, and the natural environment. When we feel empty or unable to confront our inner struggles, we often seek external distractions [1].
Today’s technology creates unprecedented opportunities for us to turn our attention outward and away from ourselves. However, the more we disconnect from ourselves, the less content we become. No wonder that mental health challenges are prevalent and the national happiness index has decreased in the past decade, despite the increasing abundance of material goods and the widespread use of digital technology.
Mindfulness practice is a way of being ourselves in the present moment, and a powerful tool to look inside ourselves. Mindfulness originates from Zen Buddhism. It is as a state of being characterized by a keen awareness of the present moment. The Sanskrit term for mindfulness, smriti, means “remembering.” Mindfulness is remembering to come back to the present moment. The character in Chinese for mindfulness, 念, consists of two parts: the upper part represents “now” and the lower part represents “mind” or “heart” [2].
At the core of Buddha’s teachings lies the concept of Right Mindfulness (samyak smriti). Buddhist psychology states that attention is a universal trait, meaning that we are constantly giving our attention to something. This attention can be appropriate, such as when we are fully present in the moment, or inappropriate, such as when we focus on something that takes us away from the present. Right Mindfulness is characterized by acceptance without judgment or reaction and is inclusive and compassionate. According to Buddha’s teachings, Right Mindfulness offers Seven Miracles, including the ability to be present and fully engage with the present moment, to bring forth the presence of others (such as the sky, flowers, or our children), to nourish the object of our attention, to alleviate others’ suffering, to explore deeply (vipassana), to gain understanding, and to bring about transformation [2].
Although stemming from Zen Buddhism, mindfulness is not a religion itself. Rather, it is a widely adopted practice around the world to improve one’s health, personal growth, and overall well-being. In the Western world, interest in mindfulness practice grew during the 1960s, leading to an era of scientific research on this ancient practice. For instance, Mindful-Based Stress Reduction (MBSR), founded by Jon Kabat-Zinn in the 1970s, is regarded as one of the leading programs for reducing stress, anxiety, depression, and PTSD, with significant and reliable results in various clinical studies [3]. Today, clinicians, business leaders, and educators are exploring ways to integrate mindfulness practice into their respective fields.
In the digital technology design community, there is a growing voice advocating for a new approach to design, one that involves looking inward. In a recent special issue of Interaction, Kentaro Toyama [4] suggests that our focus on external outcomes may be the root cause of our technological problems. He urges designers to turn to faith and inward examination as a possible solution. Meanwhile, Kabat-Zinn encourages designers to practice mindfulness themselves before teaching or supporting others. It’s time for us to consider how mindfulness can benefit us and act accordingly.
How Can Mindfulness Help Designers?
Mindfulness enables designers to see the true nature of the problems they are tackling, and provides the foundations for a paradigm shift in the design process and outcomes. Some initial considerations include:
Design process
Traditional user experience design follows the requirement-design-evaluation framework. As today’s problems are becoming more complex, challenging, and wicked, a mindful designer is aware of themselves and others, and they bring mindfulness to the design process, paying close attention to each step. They deeply observe and listen, resulting in greater empathy and deeper understanding of the problem space before jumping to the solution space.
Design for slow change
In the HCI and interaction design community, Martin Siegel and Jordan Beck [5] sketched out Slow Change Interaction Design, where they called for a paradigm shift in designers’ mindsets in order to create interactive technologies that promote long-lasting attitudinal and behavioral change.
A designer who practices mindfulness understands that problem-solving is not a one-time event, but rather a continuous process of improving the current state toward a better future state. They acknowledge that change is inevitable and strive to learn and grow with their users in a constantly evolving technological environment.
Design for a sustainable future
Considering the interconnectedness nature of all things, a mindful designer takes a systemic approach to their work. They prioritize sustainable and inclusive design, taking into account the impact on individuals with varying abilities, communities, and the environment. They recognize that quick fixes and immediate results are often unrealistic, and prioritize long-term solutions.
Toward Mindful Technology Use Practice
In his book, Peace is Every Step, Thich Nhat Hanh [6] offers suggestions on how to incorporate mindfulness practices into everyday life. Following his examples, we offer the following mindful technology practices regarding smartphone use to technology designers and users:
Before using my phone, I know what I use it for.
I am aware that happiness depends on my mental attitude and not on external conditions, and that I already have more than enough to be happy in the present moment.
The phone and I are one.
I will practice coming back to the present moment by being mindful of my breathing, posture, and body.
Takeaway
When technology is the horse, mindfulness is the harness to rein it back toward humanity. Mindfulness offers an ancient yet rejuvenating perspective that is crucial to our digital design community. Designers who practice mindfulness can transform problems and bring peace, joy, and freedom to both users and themselves. May you also find joy in practicing mindfulness.
Endnotes
1. Hanh, T.N. The horse is technology. Plum Village Magazine (Summer 2014), 5–9.
2. Ñāṇamoli, B. and Bodhi, B. The Middle Length Discourses of the Buddha. Wisdom Publications, 1995.
3. Kabat-Zinn, J. Full Catastrophe Living: Using the Wisdom of Your Body and Mind to Face Stress, Pain, and Illness, 15th Anniversary Ed. Delta Trade Paperback/Bantam Dell, New York, NY, 2005.
4. Toyama, K. Technology and the inward turn of faith. ACM Interactions 29, 4 (2022), 36–39.
5. Siegel, M. and Beck, J. Slow change integration design. ACM Interactions 21, 1 (2014), 28–35.
6. Hanh, T.N. Peace is Every Step. Bantam Books, 1992.
Posted in:
on Tue, August 22, 2023 - 1:07:00
Jia Shen
Jia Shen is a professor of information systems in the Norm Brodsky College of Business at Rider University. Her research is at the intersections of experience design, cognition, digital innovation, and the latest on digital well-being. Previously she researched social commerce, virtual world systems, and online learning.
[email protected]
View All Jia Shen's Posts
Privacy and ethics concerns using UX research platforms
Authors:
Michal Luria
Posted: Wed, July 26, 2023 - 3:03:00
Remote user experience (UX) research is not a new phenomenon [1], but the pandemic has accelerated it, necessitating the adaptation of traditionally face-to-face methods to work remotely [2]. The past few years marked a growth of online platforms, such as dscout, Lookback, Metricwire, and UserTesting, to name just a few. Remote UX research platforms are marketed primarily as tools for industry UX researchers, but are also used occasionally by academics.
These user-friendly interfaces promise remote UX researchers everything they need to conduct a successful study. They support many methods, and provide tools for all research stages, from recruitment via participant pools to built-in analysis. But the platforms’ quick-and-easy approach also creates ethical issues when they don’t fully consider researchers’ obligations toward their participants [3].
UX research frequently involves people, and therefore risks causing harm [4]. For example, people may be asked to share personal information about sensitive topics as part of research, like their sexual orientation or their use of adult websites. Without the right safeguards, this information could get into the wrong hands. Or participants may be asked to view social media content and subsequently be exposed, perhaps unintentionally, to harmful content. These scenarios are even more delicate if a study involves minors or other vulnerable groups [5].
In corresponding with several UX research platforms, I was surprised to discover consistent ethical and privacy-related gaps in the platforms’ ability to address possible risks in human subjects research, particularly in their consent, data ownership, and safety protocols. Here, I outline how these gaps could be addressed for better research practices.
Consent should be mandatory. When using a service’s participant pool, researchers can, but are not obligated to, include a consent process. Platforms say that, when a participant signs up for their platform, they agree to their terms and conditions, suggesting that this agreement covers participation in any research study without further consent.
This is not compliant with ethical research standards and the requirement for participants to be informed: Participants should be able to explicitly consent to participation in every study, after having been fully informed about its goals, risks, and outcomes [6].
Not only is obtaining consent optional on some UX research platforms, but collecting signatures often requires a premium subscription. The alternative for non-premium customers would be to put the consent information into a survey question and ask participants to click “Agree.” For the case of parental consent for minors as required by laws like the General Data Protection Regulation (GDPR), one platform suggested simply including another survey question that asks the minor whether their parents agreed to their participation.
Data should be owned by researchers, not the platform. Normally, personal data for research would be collected for “legitimate research purpose only” [7]. But many UX research platforms claim ownership of the data in their terms of service, allowing them to use it in any way they see fit. Many platforms explain that they use this participant data to improve their services, but nothing prevents them from sharing or selling it without participants’ knowledge.
In my correspondence with one such platform, I inquired about the ability to delete data at the completion of a study. The platform agreed to give me special permissions to request data deletion on behalf of my participants. This would at least limit platforms’ access to participants’ data only to the duration of the study. However, many platforms don’t allow researchers to download data, meaning that data deletion would remove access for the researchers too.
Researchers should mitigate adverse events. Researchers must plan for the possibility of unintended harm to participants’ safety and well-being [8]. In a forthcoming study for which I considered using a remote UX research platform, there was a rare possibility that participants would experience emotional distress. To account for this scenario, our research team included a certified clinician who could reach out to participants, assess suspected harm, and fulfill any obligation to escalate and report.
Platforms usually withhold participants’ contact information to spare researchers from additional exposure to identifiable data. In a case of suspected harm to a participant, however, my expectation was for platforms to collaborate to best support participants’ well-being. Instead, the platform stated that in such an event, I would report it to them and they would handle it internally.
This is unacceptable. As a researcher, the consent agreement is between me and the participant. I am responsible for any safety risks my research poses, including cases that require special attention. No matter how well prepared the platform is, they are not the ones who should be making the call on how to address concerning responses to research questions.
Concluding thoughts. The current state makes it hard for university researchers to use these remote UX research platforms. Universities have detailed training and research protocols for anyone involved in human subjects research, and many of them would not approve the terms of current platforms. By contrast, many companies and organizations do not have strong ethics review processes for research conducted by employees or contractors, and they commonly use remote UX research platforms. Even though many industry researchers have gone through academic ethical training, these platforms provide an easily overlooked path toward conducting human subjects studies with few safeguards and considerations about how data is collected, stored, and used.
This may be because many of these platforms were designed with the researcher’s user experience as a first priority, leaving participants’ well-being as an afterthought. The result is that participants are exposed to unnecessary risk of harm. While in most instances this would go unnoticed, in extreme cases things could go very wrong.
But not all UX research platforms are created equally—there are a few platforms that prioritize privacy and participant safety. They are not as user friendly and usually do not include participant recruitment. But as recruitment in these safer alternatives is under the researchers’ control, so is the data, the consent process, and the handling of adverse events. We should be able to have the best of both worlds—the capabilities and convenience we desire along with the privacy and safety practices we need.
Endnotes
1. Moon, Y. The effects of distance in local versus remote human-computer interaction. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 1998.
2. Süner-Pla-Cerdà, S. et al. Examining the impact of Covid-19 pandemic on UX research practice through UX blogs. In Design, User Experience, and Usability: UX Research and Design. Springer, Cham, 2021.
3. Fiesler, C. et al. Exploring ethics and obligations for studying digital communities. Proc. of the 2016 ACM Conference on Supporting Group Work. ACM, New York, 2016.
4. National Institutes of Health. Human subjects research; https://grants.nih.gov/policy/...
5. Antle, A.N. The ethics of doing research with vulnerable populations. Interactions 24, 6 (2017), 74–77.
6. Vitak, J. et al. Beyond the Belmont principles: Ethical challenges, practices, and beliefs in the online data research community. Proc. of the 19th ACM Conference on Computer-Supported Cooperative Work. ACM, New York, 2016.
7. Alsmadi, S. Marketing research ethics: Researcher’s obligations toward human subjects. Journal of Academic Ethics 6 (2008), 153–160.
8. Do, K. et al. “That’s important, but…”: How computer science researchers anticipate unintended consequences of their research innovations. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2023.
Posted in:
on Wed, July 26, 2023 - 3:03:00
Michal Luria
Michal Luria is a researcher at the Center for Democracy & Technology. Her work makes use of immersive and human-centered design research methods to envision and critique interactions with technology. Using these methods, she translates research insights into thought-provoking interactions and necessary discussions of technology ethics and policy.
[email protected]
View All Michal Luria's Posts
AI superpowers and the human element: Implications for researchers
Authors:
Rojin Vishkaie
Posted: Thu, July 06, 2023 - 12:04:00
Does our future have researchers who are robots? Are computers going to become so smart that they can boss these researchers around? And, if robots and computers can accomplish everything, then what are human researchers going to do? To explore these questions, I have been reading AI Superpowers: China, Silicon Valley, and the New World Order [1], a 2018 nonfiction book by Kai-Fu Lee, an artificial intelligence pioneer and venture capitalist.
The book draws on Lee's technological knowledge and professional expertise to offer a perspective on the interplay between the U.S. and China in the age of artificial intelligence, and its impact on the world. Lee believes that the U.S. is taking advantage of being the first mover in the AI race, but that China is ahead of the race in its data supply. From his perspective, the elite AI experience in the U.S. is less important than China’s AI implementation superpower.
But Lee’s argument goes beyond the AI race between the U.S. and China; it’s also about the impact of AI on humans. In his view, the U.S. and China have leapfrogged, and sit atop of, the rest of the world. The U.S. and China’s disproportionate power has worsened global inequality, and the fear of possible massive job losses because of AI will likely reduce the ability of the job market to cope and adapt. People losing their jobs will negatively impact their purpose and personal identity, as well as their ability to survive, which cannot be addressed by socioeconomic concepts like universal basic income.
But despite the many warnings, Lee has a positive outlook that AI and humans can productively coexist. Lee’s optimistic belief suggests that humans are the catalyst for shaping that future of AI. He suggests that the building blocks of a successful AI superpower is abundant data, AI scientists, AI-friendly policies, and tenacious entrepreneurs.
Lee believes that China has access to a massive quantity of data due to its large base of users, while the U.S. has an edge in elite AI expertise. He further argues that the Chinese government has a more sophisticated structure to mobilize resources and implement strategic directions for long-term goals (e.g., Development Plan for a New Generation of Artificial Intelligence). He postulates that the prevailing Silicon Valley mentality is mission-driven and techno-optimistic, believing that innovative thinking can change the world. In contrast, China has focused on market and financial profit. In Lee’s view, the AI new world order is bipolar and focused on AI superpowers versus the rest of the world, but the actions of humans themselves are the most important factor in shaping the future of the human ability to coexist with AI.
What I’ve outlined is not a theoretical argument about the future, but rather is a perspective on researchers’ coexistence with AI while considering Lee's views on the potential impact of AI on human work. I believe that researchers can create an ecosystem where AI enhances productivity (e.g., using chatGPT [2] to assist with data management and insights). The ecosystem of research and AI can also be used for socially beneficial activities (e.g., generating knowledge with practical applications).
Endnotes
1. Lee, K.F. AI Superpowers: China, Silicon Valley, and the New World Order. Houghton Mifflin, Boston, MA, 2018. ISBN: 9781328546395
2. Lock, S. What is AI chatbot phenomenon ChatGPT and could it replace humans? The Guardian. Dec. 5, 2022; https://www.theguardian.com/te...
Posted in:
on Thu, July 06, 2023 - 12:04:00
Rojin Vishkaie
Rojin Vishkaie is a senior user experience Researcher in Amazon. Her work focuses on human-centered design and evaluation of technology, with recent use cases in devices, mixed-reality education, museums, and children’s gaming and toys. Opinions expressed are solely her own and do not express the views or opinions of her employer.
http://rojinvishkaie.com [email protected]
View All Rojin Vishkaie's Posts
Why codesigning AI is different and difficult
Authors:
Malak Sadek,
Rafael Calvo,
Céline Mougenot
Posted: Tue, June 27, 2023 - 12:04:00
It is estimated that 98 percent of the population are novices with regard to technology (excluding extremes such as infants) [1]. It is this 98 percent, however, that form the main chunk of users and stakeholders affected by AI-based systems. It then makes sense that members of this segment of the population should be involved in designing these systems beyond just the small, homogenous set of experts currently involved.
There have been countless calls for the introduction of a transdisciplinary, participatory design process for AI/ML systems [2,3]. Such a collaborative design (codesign) process has been heralded as especially useful in aiding explainability and transparency [4], embedding values into AI-based systems [5], providing accountability, and mitigating downstream harms arising from several cascading biases and limitations [6]. There have also been calls for collaboration within the entire AI pipeline, including in data creation and selection, instead of having designers at the front end of the process and engineers at the back end [7]. In fact, it has been said that the only way to combat existing structural and data biases that creep into AI systems is to step away from custom-built, solely technical solutions and view AI systems as sociotechnical systems [8]. By looking across them as opposed to inside of them and using open constructive dialogues, collaborations, and group reflections [9], we can bridge the gulf between stakeholder visions and what gets implemented [7]. Having the needed voices give their input in a meaningful, impactful way through codesign activities can allow us to examine this broader frame of the wider sociocultural contexts in which AI systems are being used [10].
Despite the numerous cited benefits of and calls for codesigning AI systems, it is still not a widespread practice. The communication gap between designers’ products and services, developers’ algorithms and systems, and users’ needs and applications, is one of the largest challenges when it comes to using AI systems in real-world settings, where its implications could be life-altering for different stakeholders [10].
Bridging this gap is a unique, and notoriously difficult process for AI systems, as it has several inherent differences to other types of systems which require various considerations across different design and development phases [10]. We summarize several of these challenges below in an effort to consolidate the barriers to codesigning AI-based systems. The goal is to start a conversation on possible solutions and ailments to enable a much-needed, more widespread adoption of codesign practices for these systems.
Why Is Codesigning AI Different and Difficult?
Data considerations
- There are several extra considerations when it comes to AI systems about which datasets to use or what type of data to collect, as these systems are based on existing training data from the get-go, as opposed to other systems where user data can be collected after the system has been created for evaluation purposes. Additionally, assessing the initial feasibility of many ideas and being able to develop prototypes for them will be largely dependent on whether the data exists or can be collected.
- There are also several ethical and moral concerns that arise regarding data provenance and data quality.
- Both technical and nontechnical participants tend to focus on the ML model itself, overlooking the rest of the wider system, the data needed, and the broader sociotechnical and interactional contexts surrounding the system and in which it is embedded. This is especially true when tackling value-related or nonfunctional requirements such as fairness. Finally, there is a general lack of awareness regarding values and nonfunctional requirements in the field and how different ML design decisions affect them, in addition to no regulations regarding their elicitation and use, and a deficiency of tools and methods for measuring them.
Ideation and communication
- It is difficult for technical experts to explain to users and nontechnical experts an AI’s behavior, what counts as AI, and what it can/cannot do.
- It is also difficult for designers to communicate AI design ideas, interactions, and appropriate use cases/user stories to technical experts, codesign partners, and users. It is also challenging to imagine ways to purposefully use AI to solve a given problem, creating an overall “capability uncertainty” [10].
- It is challenging for designers and developers to understand how to collaborate and co-ideate without a common language or shared boundary objects, especially when designers join late in the project.
User research
- Embedding values into AI systems must be planned for from the offset, instead of being added in post-hoc, because the way the system is built will influence which kinds of values and ethical systems can be ‘loaded in.’ There then becomes an additional step of eliciting those values that diverse stakeholders agree on early in the process and upholding them on a sociotechnical scale.
- User experience research also becomes more difficult for AI systems, as user profiles and preferences are created dynamically during interactions and there is no predefined persona or profile to check against.
Design, development, and testing
- Designing AI systems goes beyond simply grounding design outcomes in what is technically viable, but also understanding and designing for the critical trade-offs between different algorithms and understanding the constraints, abilities, and interactions of the model.
- It is difficult to anticipate unpredictable or unwanted AI behaviors and effects, and how they will evolve over time. It is also often unclear whom to hold accountable, and very difficult to communicate all of this to users.
- It is also challenging to design and visualize branching, open-ended AI interactions, difficult to envision the potential of AI for use cases that do not already exist, and difficult to rapidly prototype AI ideas. Designing interactions for the incredibly complex and diverse outputs and possible errors of an AI system is challenging.
- Compared to other systems, it is more difficult to respect and implement some stakeholder values or nonfunctional requirements such as transparency and fairness, given the complex, uncertain, and often unpredictable nature of AI/ML systems (and the high-level, general nature of those values ). It is also challenging to measure them within those systems.
This list is by no means comprehensive but instead aims to be a starting point for a dialogue on the different challenges that appear during various design stages for AI-based systems. By summarizing and consolidating these challenges, we can begin to discuss solutions and mitigations as a community in order to make codesigning AI systems an easier and more practical endeavor.
Endnotes
1. Sadler, J., Aquino Shluzas, L., and Blikstein, P. Abracadabra: Imagining access to creative computing tools for everyone. In Design Thinking Research. H. Plattner, C. Meinel, and L. Leifer, eds. Springer, 2018, 365–376.
2. Gabriel, I. Artificial intelligence, values, and alignment. Minds and Machines
30 (2020), 411–437.
3. Yu, B., Yuan, Y., Terveen, L., Wu, S., Forlizzi, J. and Zhu, H. Keeping designers in the
loop: Communicating inherent algorithmic trade-offs across multiple objectives, Proc. of the ACM Designing Interactive Systems Conference. ACM, New York, 2020.
4. Chazette, L. and Schneider, K. Explainability as a non-functional requirement: Challenges and recommendations. Requirements Engineering 25 (2020), 493–514.
5. Crawford, K. and Calo, R. Here is a blind spot in AI research. Nature 538 (2016), 311–313.
6. Delgado, F., Barocas, S., and Levy, K. An uncommon task: Participatory design in
legal AI. Proc.of the ACM on Human-Computer Interaction 6, CSCW1 (2022).
7. Frost, B. and Mall, D. Rethinking designer-developer collaboration. 2020; https://www. designbetter. co/ podcast/ brad-frost-dan-mall
8. Ananny, M. and Crawford, K. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society 20, 3 (2018), 973–989.
9. Yang, Q., Scuito, A., Zimmerman, J., Forlizzi, J., and Steinfeld, A. Investigating how
experienced UX designers effectively work with machine learning. Proc. of the CHI
Conference on Designing Interactive Systems. ACM, New York, 2018.
10. Yang, Q., Steinfield, A., Rosé, C., and Zimmerman, J. Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. Proc. of the CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020.
Posted in:
on Tue, June 27, 2023 - 12:04:00
Malak Sadek
Malak Sadek is a design engineering Ph.D. candidate working in the space of conversational agents and value-sensitive design. Her background is in computer engineering and human-computer interaction, and she uses this mixed background to constantly look for ways to bridge between technology and design.
[email protected]
View All Malak Sadek's Posts
Rafael Calvo
Rafael A. Calvo, is a professor at Imperial College London focusing on the design of systems that support well-being in the areas of mental health, medicine, and education, and on the ethical challenges raised by new technologies.
[email protected]
View All Rafael Calvo's Posts
Céline Mougenot
Hidden in plain sight: Discreet user interfaces
Authors:
Kyle Boyd,
Raymond Bond,
Maurice Mulvenna,
Karen Kirby,
Susan Lagdon,
Becca Hume
Posted: Thu, June 08, 2023 - 3:49:00
Do you feel safe when you travel alone? That was a question I asked my girlfriend (now wife) as we took a trip on a relatively empty train when I was visiting her at university. “I always carry my phone with me, but sometimes it's actually worse when the train is full; people watch what you are doing on your phone, and it's very off-putting,” was her reply.
Fast-forward 10 years and we are still very much in the same predicament. People, especially women, can feel vulnerable in certain situations. A 2019 survey [1] by SAP Concur, a company that provides travel and expense-management services to businesses, agrees. The survey revealed that safety is the number one concern for female business travelers. Indeed, 42 percent of respondents said that they have had a negative experience related to their gender. This isn’t limited to gender, though. Our intention is to highlight that people can feel vulnerable because of certain characteristics, including disability or race, and in certain situations.
With the rise of ubiquitous computing, we have the ability to use a plethora of devices that can connect to the Internet. This allows us to work, shop, and entertain ourselves from almost anywhere. Devices and their interfaces have become an extension of our own selves.
In recent decades, smartphones, emails, and instant messaging have been the predominant methods for human-to-human communication. And now with “generation mute” [2], instant text messaging, as opposed to making phone calls, has become the primary use of smartphones. With growing concerns over cybersecurity and privacy, instant messaging apps such as WhatsApp incorporate end-to-end encryption.
Modern text messaging apps allow users to send messages that are discreet in the sense that they are secure. But of course the interaction is not entirely private, since any onlooker could potentially see the user typing and sending a text message. A more discreet interaction might be the use of secret codes to input into the messaging app, which would be indecipherable to onlookers. However, this would add to the cognitive load and require a new communication code. Other apps such as Snapchat have functionality to send messages and photos that are only temporarily available; they are automatically deleted shortly after being viewed. These apps could be considered a “discreet user interface,” since though a spouse or family member might see the message or photo being sent or received, they cannot confirm it by interrogating the user’s stored media. With these examples, digital communication today is discreet in terms of privacy and data encryption during transit; however, the user interaction is not hidden from onlookers. With this can come unwanted attention, as onlookers can view and digest some of the information on your digital device, whether this be texting a friend from the train or doing work on your laptop on an airplane. In 2019, HP conducted a “creepers and peepers” survey [3], asking 3,000 people in the general population and 1,500 staff whether they would look at another person’s computer screen during a flight. Astonishingly, 4 out of 5 look at other people’s screens; 8 out of 10 restrict what they look at on their device just in case people are looking at their screen. This gives rise to questions about device-use etiquette and safety when using digital devices in public, stretching further for interfaces used for finance or security.
As interaction design researchers, we want to consider how design could help. Could we design a hidden user interface for safety purposes? This work is aimed to be both conceptual and thought-provoking.
Discreet User Interfaces
Do you remember the first time you saw the interface on a first-generation iPhone? It was staggering. Swiping, zooming, pinching, and animations that allowed for delight and control. All these new controls that allowed the content to take center stage and be the interface. Edward Tufte was correct when he said that “[o]verload, clutter, and confusion are not attributes of information, they are failures of design.” As interface designers, we strive for simple design that won’t get in the way, that will allow us to complete tasks in an efficient and satisfactory manner, producing interfaces that work. For many use cases, this is the appropriate design methodology to use when it comes to designing interfaces.
But what design methodology should we use if we wanted to create a “discreet user interface”? A discreet user interface is a user interface that looks like it’s doing one thing but actually is doing something entirely different. Alarm bells might start ringing when we put it this way, but that doesn’t mean people are doing anything sinister. Quite the opposite. A discreet user interface could be there to help and protect, even though it is “ethically deceiving” onlookers by preventing them from interpreting what the user is actually doing on their screen.
What was historically known as the “boss key” could be described as a kind of discreet user interface. A boss key is a key on the keyboard (e.g., F10 or ESC) that a user presses in haste during “play time” when their boss is nearby, displaying a fake spreadsheet, business graph, or slides to discreetly deceive the onlooking boss. Other interactions that we could also consider to be discreet interactions, or at least discreet features, include “Easter eggs,” which were hidden features inside Microsoft products. These could also be hidden games, such as pinball or a flight simulator. And of course even smartphones have hidden features, such as shaking the phone to undo an action, facilitated by sensors such as an accelerometer.
With this in mind, and perhaps more intuitively, discreet user interfaces could be those that a user can interact with while preventing an onlooker from knowing that they are using a computer. These interfaces could fall under the definition of natural user interfaces, where the user can interact with digital systems using their eyes, hand gestures, brain signals, and muscle movements, including facial movements. This can include “silent speech interfaces.” For example, Kapur et al. [4] developed the AlterEgo system, which allows people to interact with a computer through neuromuscular activity. Using electrooculogram (EOG) sensing, Lee et al. [5] developed a system that allows users to discreetly interact with computers by rubbing their “itchy” nose. And Cascón et al. [6] developed the perhaps even more discreet ChewIT, a system that allows users to interact with computers via intraoral interactions by essentially sending discreet signals by chewing. While these are very interesting innovations, they require additional hardware and sensors and are mostly at the research and innovation phases.
Another form of discreet user interfaces could be described as “camouflaged user interfaces” or “ulterior user interfaces.” These are interfaces that give onlookers a fake impression of an app’s utility, with its true, ulterior hidden utility known only to the user. For example, while an onlooker might see the user engaging with a mundane weather app, a sports app, a news app, or a shopping list, the user could be performing hidden tasks or sending secret signals in plain sight by interacting with buttons and features that have hidden functions. The interaction might look mundane, but the user could be sending a secret communication. Hence, this ulterior user interface could also be considered a discreet interaction. This concept is akin to steganography, where an everyday mundane image contains a hidden message that can be extracted using a secret algorithm; however, steganography only applies to the message and does not encompass the user interaction. If it did, there might have been a concept coined as stegano-graphical user interfaces.
A discreet user interface could also help protect those in need and hinder “peepers and creepers.” Use cases could be messaging, banking, or general phone use in a public space. It could go even further, providing an outlet for those who are subject to coercive behavior or domestic abuse. At the same time, if an assailant or abuser found the discreet interface, this could put the user in danger. That is why this type of solution, if developed, may need to be rolled out by police or social services so that it could remain anonymous and safe.
A Simple Case Study: A Discreet News App
An example of a discreet user interface and use case could be the following: A female commuter, Jane, traveling home from work gets on board a busy train during peak hours. There are not many seats, so Jane stands. Along her 30-minute commute, two male commuters come on board and, given the limited space, encroach on Jane’s personal space. As Jane uses her smartphone, she becomes aware of the men’s presence behind her, making her feel vulnerable. They are watching everything she is interacting with, peeping and creeping.
To stop this, she switches to her hidden interface, the “news” app (Figure 1), and reads a news item before interacting with a discreet feature. The feature allows Jane to discreetly, semiautomatically message a friend and alert her to meet her at the next station. As the train pulls into the station, Jane again uses the news app, where secret messages inform her of her friend’s confirmation and whereabouts.
The use case is twofold. In one sense, it’s a regular news app that will display a range of articles that can be read. But upon tapping an icon or button, the user interface can make calls and send prewritten text messages, all using the same content areas and buttons that already exist in the app.
![](/images/uploads/blogs/hidden-plain-site-image-01_sm.jpg)
Figure 1: This discreet user interface is called the news app. It displays news items, but by tapping icons and buttons, the user interface can send hidden communications, allowing the user to, for example, send prewritten messages, take a photograph, or make a call. Credit: onyxprj_art at vecteezy.com, licensed under CC BY 4.0.
Conclusion
This article highlights the growing problem of “peepers and creepers,” unwanted attention by others while interacting with our digital devices. We considered a novel solution called discreet user interfaces, where an interface can look like it has a mundane utility but can have underlying functionality that allows it to discreetly perform other functions (e.g., hidden communication). We also presented a case study in the form of a discreet news app that showcases how a discreet user interface could work. We hope this article will be thought-provoking and will widen the conversation around discreet user interfaces and their use cases in interaction design and applications.
Endnotes
1. Haigh, L. Female travellers: a unique risk profile. ITIJ 230 (Mar. 2020); https://www.itij.com/latest/lo...
2. Dawson, A. So Generation Mute doesn’t like phone calls. Good. Who wants to talk, anyway? The Guardian. Nov. 7, 2017; https://www.theguardian.com/co...
3. HP creepers and peekers survey. HP Press Center. Sep. 25, 2019; https://press.hp.com/us/en/pre...
4. Kapur, A., Kapur, S., and Maes, P. Proc. of the 23rd International Conference on Intelligent User Interfaces. ACM, New York, 2018, 43–53; https://doi.org/10.1145/317294...
5. Lee, J. et al. Itchy nose: Discreet gesture interaction using EOG sensors in smart eyewear. Proc. of the 2017 ACM International Symposium on Wearable Computers. ACM, New York, 2017, 94–97; https://doi.org/10.1145/312302...
6. Cascón, P.G., Matthies, D.J.C., Muthukumarana, S., and Nanayakkara, S. ChewIt: An intraoral interface for discreet interactions. Proc. of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2019, Paper 326, 1–13; https://doi.org/10.1145/329060...
Posted in:
on Thu, June 08, 2023 - 3:49:00
Kyle Boyd
Kyle Boyd is a lecturer of interaction design at the Belfast School of Art at Ulster University. His research interests and publications include interaction design, user experience, digital healthcare, and usability. He recently cochaired the 15th Irish Human Computer Interaction Symposium at Ulster University.
[email protected]
View All Kyle Boyd's Posts
Raymond Bond
Raymond Bond is a professor of human computer systems and a distinguished research fellow at Ulster University. His work includes applications of data science and HCI within biomedical and healthcare informatics (digital health). He has more than 400 research outputs and has chaired or cochaired a number of conferences, including the 32nd International BCS Human Computer Interaction Conference and the 45th and 46th Annual Conferences of the International Society for Computerized Electrocardiology.
[email protected]
View All Raymond Bond's Posts
Maurice Mulvenna
Maurice Mulvenna is a researcher in computer science and artificial intelligence at Ulster University known for his contribution to interdisciplinary research in digital mental health sciences with colleagues in psychology, nursing, and healthcare. He has published over 400 peer-reviewed publications and has been a principal investigator or investigator on more than 120 international research projects.
[email protected]
View All Maurice Mulvenna's Posts
Karen Kirby
Karen Kirby is a senior lecturer and registered practitioner psychologist in the School of Psychology at Ulster University. Her current areas of research interest are understanding child and adolescent mental health and developmental trauma, the impact of trauma histories on various mental health issues, and preventative mental health and novel programs and technologies.
[email protected]
View All Karen Kirby's Posts
Susan Lagdon
Susan Lagdon is a lecturer in psychology at Ulster University. Her research interests include domestic and sexual violence and abuse, particularly the mental health implications of interpersonal trauma and the availability and types of support for victims.
[email protected]
View All Susan Lagdon's Posts
Becca Hume
Becca Hume is an Ulster University MFA graduate and founder of the software company TapSOS. Her primary research and development is in the area of public safety, critical communications, and vulnerable users. She leads research in enhanced nonverbal technologies and is a British Sign Language user (BSL Level 6).
[email protected]
View All Becca Hume's Posts
Happily technology free
Authors:
Linnea Öhlund
Posted: Mon, May 15, 2023 - 10:28:00
My grandma recently passed. At the age of 88 she went to sleep never to wake up again. She lived a long, happy life, and had two sons, six grandchildren, and currently four great grandchildren. Bye grandma, I will always love and miss you.
Now to the point. My grandma was one of those individuals who never really “got it” when it came to technology. She only got a cellphone at the age of 78, because we forced her to, and the computer she and my grandfather owned was barely used. I have a vivid memory of me at perhaps 14 years old, asking them to use the computer so I go into paint (that was the most fun they had on the old thing). Grandma got a bit upset, saying that I couldn’t use paint because I might destroy the computer—something could go wrong and I would break it. Me, a person born in 1995 who grew up using computers, knew very well that no computer would break by simply opening paint; however, my grandma, who seemed to have no real understanding of how a computer worked, treated it like a fragile crystal vase, and I was an elephant ready to smash it!
After a while, they just got rid of the computer. The phones they had were the ones we see many senior citizens using, with the big buttons and no apps—just call and text functions. But let’s make one thing clear: My grandma and grandpa (who passed in 2020) were not afraid of or negative toward technology; they understood the impact that the Internet and technology had on society. A part of them wanted to join the revolution, wanted to see what the fuss was about, wanted to join the party. They were both born so long ago (1925/1934) that only after they had retired did the Internet, mobiles, and computers become big. Grandma became a housewife in the 50s and grandpa retired in the 90s, both coming from small country villages in the north of Sweden. When and where they were born, grew up, and settled surely affected how they felt about technology as well as their ability to use it. I’ve been thinking recently that there seems to be so much pressure on seniors these days to adapt, to learn, to go above and beyond to use technologies. No more bank offices, JUST VISIT US ONLINE, no more paper bills, JUST PAY THEM ONLINE, no more post in the mail, JUST GET THEM ONLINE. But what if people don’t want to do that? What if they just want to live a quality life and not have their happiness depend upon technology? I once did a study with 15 seniors, asking them about their opinions on digital technology. One female participant, who herself was about 70, had a mother who was almost 100. She didn’t want to use technology but felt left out of due to the very large amount of digital and technological artifacts that are needed to do many things. She finished the interview by saying how she felt there was still a generation of seniors who needed to pass away before the intense digitalization in the Swedish society would be successful. These seniors, much like my grandparents, did not have a smooth transition from analog to digital in their lives and were, frankly, left behind due to this.
Now that both my grandparents from older generations have passed, I think of that comment by the participant. One more step toward a successful digital society where there are no longer any individuals left who do not want to adapt. What type of society is that? Waiting for its inhabitants to pass? Society needs to adapt to its inhabitants and offer opportunities for those who choose to not use technology. They still have the right to good, happy lives not degraded because of society’s intense technological push. This should be obvious, but is clearly not.
So here’s to you grandma, for your glass vase computer and your amazing, basically technology-free life! Cheers!
Posted in:
on Mon, May 15, 2023 - 10:28:00
Linnea Öhlund
Human-centered artificial intelligence: The solution to fear of AI
Authors:
Chameera De Silva,
Thilina Halloluwa
Posted: Tue, May 09, 2023 - 3:06:00
Have you ever played a game against a computer? Have you ever wondered how computers win complex games against well-experienced players, such as when an AI system from IBM defeated Garry Kasparov in the Chinese board game Go?
The fear of AI becoming the biggest threat to humankind, replacing the human workforce on a large scale is a real concern for AI practitioners and businesses. But there have been incidents where the combination of a smart AI system supervised by humans has resulted in unbelievable achievements. Technology has developed magnificently in the past decade, and the widespread adoption of smart systems powered by AI predicts a much smarter and tech-dependent world. New-age AI systems will become more and more learning-based. The massive data generated by the Internet will be used to train these models and systems to improve their intelligence in specifically focused areas.
The current focus of AI developers is to make the algorithms better and optimize the performance of the AI models as much as possible. We’re heading toward Web 3.0; its adoption will be much faster than that of Web 1.0 and Web 2.0 and driven by the adoption of AI.
The super-fast development and integration of AI tools into the products and services we use is accelerating the pace of our lives. Alongside this development is the fear that AI will exceed human cognitive intelligence, creating a struggle to maintain our control over them.
This fear of AI replacing humankind is the motivation for human-centered artificial intelligence (HCAI), where at one end we have AI and at the other end human beings supervising the actions and results of AI. The development of this technology has happened intuitively and on multiple levels.
In the early days of the Web, Web 1.0, we could only read the content online. The public had no control of over what they wanted to read or consume, and we could not search for things; it was just one-way communication. This motivated the creation of Web 2.0, which gave us social media applications like Facebook, Youtube, and Google. This made the communication on the Web two-way. We were now able to decide what content we wanted to see. We could not only share our views as comments but also could now like or dislike posts. Now, as we are involving and interacting with technology more and more, we are finding more ways to incorporate technology into our lives, and one way of doing it is by developing AI applications.
Artificial Intelligence
Humans are constantly working on creating machine intelligence similar to that of human intelligence; this is called artificial intelligence. AI is one way we are using technology and creating applications and systems that are automated and can work for us without any supervision. Self-driving cars, lie detector machines, and robots that can do surgery are just some examples of automations made possible by artificial intelligence. Current research is targeting advances in many sectors, including finance, education, and transportation.
These advances mean incorporating AI into our work processes at higher levels. This view of involving AI into human life is a matter of great concern today, often creating fear of AI and pitting technology against humankind.
What Is the Fear?
Machines are no doubt very fast and accurate in calculating numbers and analyzing large numbers of scenarios and methods within minutes. This speed and accuracy is very helpful to humans, but relying on machines completely just because of their speed and accuracy is a bad idea. Machine intelligence is smarter than human intelligence in the areas in which the machine is trained—but it cannot be trusted blindly.
The ethical and safety implications of learning-based AI systems have proven these systems to be unsafe in many ways. The decisions and results provided by these systems are not guaranteed to be fair and have also been shown to be not perfectly explainable. That is, the current models are black-box models for us humans, as they do not provide reasons for the results given by the machine.
Real-world scenarios cannot work on ambiguity and ethically unsafe standards.
AI is making machines intelligent, but we cannot be guaranteed of the safety of these systems in any specific way unless they are extremely constrained.
What Are the Challenges?
There are several challenges with AI machines today. The results given by AI systems are trusted because they are provided by computers. But the explanation of these results is very important as we scale up the use cases and implementation of these smart systems. The current solution to this challenge is explainable artificial intelligence. Explainable AI adds a level of transparency in the decision and processing done by AI systems. But as we incorporate AI systems in sectors like education, medicine, and finance, explainability is not enough.
AI in the education sector
The systems involved in the education sector will be assigned to grade students’ papers, mark their attendance, and look for any possibilities of cheating during examinations. The results given by the systems in this area impact the academics of the students directly; therefore, it is crucial to ensure that the decisions are fair and grounded on the actual rules of the real world. To achieve this level of trust with machines, human supervision is necessary.
AI in the medical sector
Likewise, the involvement of AI systems in the medical sector is opening new doors of advancement in medicine. Robots are performing surgeries, as well as recommending medications to patients on the basis of their symptoms.
The recommendations of these system are highly likely to directly impact the patients’ health and life; therefore, it is extremely important that the recommendations are 100 percent correct. But here, due to the subjective nature of the patients’ symptoms, the chances of machines making a mistake is very high. For example, there could be a case where the patient is feeling chest pain due to low blood pressure, but the machine detects it as a heart attack. These kinds of mistakes can cost lives and thus there must be zero-tolerance for them.
The challenge
Because of the lower cognitive and emotional intelligence of machines, it is necessary to incorporate smart systems alongside humans and not in place of humans. The desired advancement can be achieved only when humans and machines work together.
Human-Centered AI
Human-centered AI (HCAI) is AI with human supervision. There are three main pillars of HCAI: humans, ethics, and technology. Human-centered AI is the solution to the fear of AI exceeding human intelligence and thus becoming a threat to humankind.
HCAI will ensure that AI systems are supervised by humans and that the processing and decisions of smart systems are all under the control of a human (Figure 1).
![](/images/uploads/blogs/Figure1_web2.png)
Figure 1. Human-centered AI combines humans, ethics, and technology.
But what exactly is human-centered AI? The basic idea involves the deep integration of humans into the data-annotation process and real-world operation of the system in both the training and the testing phases. HCAI will combine human-driven decision making with AI-powered smart systems, thereby making them smart, fast, and trustable. Ethical and responsible AI systems will become a reality by incorporating AI alongside humans. Augmenting AI systems with human abilities will also bring us usable and explainable AI. This is essentially human-controlled AI, combining the modern power of artificial intelligence with human intelligence.
HCAI will address the challenges of the lower emotional intelligence of machines and the lower speed and accuracy of humans together.
How is HCAI possible?
Current AI systems are the learning bases for AI systems that get trained on data and are then tested on similar datasets to check their accuracy. These datasets are created by humans, and a large amount of time, energy, and money is invested in performing detailed annotations to create a dataset suitable for training an AI model.
The motive of HCAI is to enable an annotation process generated by machines in the form of queries to humans. The machines will decide which data is necessary to be annotated by humans and only that subset of the data will be given to the humans for annotation. This is one way to create human-centered AI.
With regard to real-world applications, current systems just deliver the results to us and we receive them. With HCAI, the results given by the AI system will have a degree of uncertainty associated with them from the AI system; this result will then be sent to the human supervisor. When the results get the thumbs up from the human supervisor, then and only then are they sent as the final result (Figure 2).
Figure 2. Multiple AI systems arguing with human supervision.
This is the basic, ground-level idea of HCAI. It can get much deeper and can exist in many formats.
Algorithms in HCAI
The algorithms of current AI systems are created with the purpose of training the machine-learning models. The algorithms in HCAI will be designed in such a way that they give the AI system an idea of the context of human intelligence, both in the moment and over time. Smart systems will be trained not only on the annotated data mechanically but will also get training on the human understanding of the concept. This will make AI systems even smarter and reduce the risk of unwanted out-of-the-box results.
Human-computer interaction
To create the algorithms that teach the computer the context of human understanding, we need human-computer interaction. To make the interactions with humans successful and fruitful for HCAI purposes, they must be continuous, collaborative, and provide a rich, meaningful experience.
We currently have autonomous vehicles driving more than 100 billion miles on the road in auto-pilot mode, some of them in semi-autopilot mode. These cars and the smart systems installed in them aim to have a rich and fulfilling human-computer interaction experience. The cars have a bunch of super-cool devices monitoring and recording the multiple modes and emotions of the driver and try to give us a better interactive experience. But we are seeking a collaborative, rich interactive experience that goes far beyond this.
The challenge for current smart car systems is that these cars are intended to be used by people from both the younger generation and the older one, who might have never used any AI-based system in their entire life. The real challenge is thus creating HCI that is capable of deeply exploring these aspects and providing a rich, interactive, and collaborative experience.
Safety and ethical awareness
Artificial intelligence or machine intelligence in general brings many concerns around safety and ethics.
Taking the example of self-driving cars, the passengers riding in a car that is in autopilot mode are prone to trust the system, as the system only provides the decisions it is making based on the situation. There is no information given regarding the level of uncertainty, nor any warnings from the system to the driver.
HCAI is a solution for this safety challenge. We can create a system with multiple AI models deciding on a single situation, and whenever a disagreement in the decisions is detected, human supervision is then sought. This is one way of involving human interaction in the decision-making and validation processes.
The other challenge is the ethical awareness of the AI system. There are certain situations in the real world that cannot always be dealt with technically. The ethical aspects must be considered, and this is an area we need to work on to optimize our AI models.
Until AI models are smart enough to understand the ethical and emotional aspects of the real world, they need to be combined with human supervision in the form of HCAI. In addition, some of the current AI use cases, when enabled with stronger and safer HCAI, can make things much more secure for users.
HCAI Today
HCAI designers go beyond thinking of computers as our teammates, collaborators, or partners. They are more likely to develop technologies that dramatically increase human performance by taking advantage of the unique features of computers.
Recommender systems
A current system that is being implemented almost everywhere and is making things autonomous for humans is the recommender system. Widely used in advertising services, social media platforms, and search engines, these systems bring many benefits.
The consequences of mistakes by these systems are subtle, but malicious actors can manipulate them to influence buying habits, change election outcomes, or spread hateful messages and reshape people’s thoughts and attitudes. Thoughtful design that improves user control could increase consumer satisfaction and limit malicious use.
Other applications that enable automation include common user tasks such as spell-checking and search-query completion. These tasks, when carefully designed, preserve user control and avoid annoying disruptions, while offering useful assistance.
Consequential applications
The applications in medical, legal, or financial systems can bring both substantial benefits and harms. A well-documented case is the flawed Google Flu trends, which was designed to predict flu outbreaks, enabling public health officials to assign resources more effectively.
A harmful attitude among programmers called “algorithmic hubris” suggests that some programmers have unreasonable expectations of their capacity to create foolproof autonomous systems.
The crashes in stock markets and currency exchanges caused by high-frequency trading algorithms triggered losses of billions of dollars in just a few minutes. But with adequate logging of trades, market managers can often repair the damage.
Life-critical systems
Moving on to the challenges of life-critical applications, we find physical devices like self-driving cars, pacemakers, and implantable defibrillators, as well as complex systems such as military, industrial, or aviation applications. These require rapid actions and may have irreversible consequences.
One-dimensional thinking suggests that designers must choose between human control and computer automation for these applications. The implicit message is that more automation means less control. The decoupling of this concept leads to a two-dimensional HCAI framework.
Two-Dimensional HCAI
Two-dimensional HCAI suggests that achieving high levels of both human control and automation is possible, shown in Figure 3.
![](/images/uploads/blogs/Figure03_web.png)
Figure 3. Two-dimensional HCAI.
The goal is most often but not always to be at the upper-right quadrant. Most remote sensing technology systems are on the right side.
The message for designers from two-dimensional HCAI is that, for certain tasks, there is a value in full computer control or full human mastery. However, the challenge is to develop an effective and proven design, supported by trusted social structure, reliable practices, and cultures of safety so as to acquire RST recognition.
Conclusion
To conclude, artificial intelligence is not a threat to humankind if we involve it into our lives as a companion and learn to work alongside it. There are several ways that we can teach AI systems to accompany us humans in our tasks and make them understand us. We can also create a system with multiple AI models that argue over decisions and then seek human supervision on every disagreement.
The idea of two-dimensional human-centered artificial intelligence is making the vision of HCAI more clear for designers and developers and is redefining the challenge in a new way. Designers need to create trustable, safe, and reliable HCAI systems supported by cultures of safety so as to acquire technical recognition.
That said, nobody is perfect, neither artificial intelligence nor humans. But we can join hands and provide something enriching to each other. AI is the future, and the future is beautiful.
Additional Resources
1.
2.
3.
4. Human-Centered Artificial Intelligence: Reliable, Sage & Trustworthy, Ben Shneiderman, [email protected], Draft February 23, 2020, version 29 University of Maryland, College Park, MD
Posted in:
on Tue, May 09, 2023 - 3:06:00
Chameera De Silva
Chameera De Silva is an artificial intelligence/data engineer with more than three years of experience in application design, development, testing, and deployment. He is highly experienced in writing code and algorithms as well as building complex neural networks through various programming languages. He possesses an unbridled passion for AI and has comprehensive knowledge of machine-learning concepts and other related technologies.
[email protected]
View All Chameera De Silva's Posts
Thilina Halloluwa
Rethinking the SIGCHI conference landscape: Why some must evolve or perish
Authors:
Johannes Schöning
Posted: Mon, May 08, 2023 - 10:04:00
It has been nearly four years since I last attended an in-person conference, and I must admit that I was excited about attending the recent ACM CHI conference in Hamburg.
In-person conferences provide a unique opportunity for researchers to meet, chat, and catch up on the latest developments in the field, as well as absorb all the excellent talks and demos. It is also super important to build networks, especially for early career researchers.
However, the Covid-19 pandemic put an end to all that, and we quickly transitioned to digital conferences. While digital conferences had their benefits, such as accessibility and convenience, I found myself missing the social aspect of in-person conferences, particularly the informal chit-chat during breaks.
Additionally, I hardly remembered any paper presentations from the digital conferences I attended, in contrast to talks I attended back at CHI in San Jose in 2007, which I remembered vividly due to my physical surroundings and interactions with others in the room.
Now, hybrid conferences are becoming the new normal, offering both in-person and digital attendance options. While this innovation seems like a step forward, it raises important questions. Have we truly innovated the conference experience, or did we act too quickly in response to the pandemic? Have we considered all the potential side effects of going hybrid? And perhaps more importantly, do some conferences need to die or perish?
One argument for the death of some conferences is simply that there are too many of them. It can be challenging to recruit reviewers for all the different conferences, and the field may need help to sustain the number of conferences currently in existence.
Additionally, some conferences simply do not get the attention they deserve anymore, and it can be challenging to maintain a core audience for them. For example, I love MobileHCI, and I was recently the program chair for it. However, the number of submissions has been decreasing in the last few years, and it seems like the conference is losing its core audience. MobileHCI has become the “mini CHI,” and it’s unclear where MHCI fits into the mix of SIGCHI conferences. This is not only true for MobileHCI, but also for a set of other conferences in my humble opinion.
Another reason to consider the death of some conferences is sustainability. While hybrid conferences can reduce the environmental impact of conferences by reducing travel and energy consumption, they still require physical resources such as venue rental and equipment. Some conferences, such as IEEE VR, have started to charge authors for each item presented at the conference to offset these costs. However, it needs to be clarified whether this approach is sustainable in the long term and also what novel barriers hybrid conferences bring, for example, for early career researchers who are “forced” into the hybrid mode instead of going to physical conferences due to budget concerns. We need to talk about CHI in Hawaii at another time, in depth.
So, what do I propose? First and foremost, I suggest that we focus on having a strong flagship conference and just a few (local) satellite conferences that emphasize the networking and experience parts of the conference—trying out demos and talking to people instead of passively absorbing talks (one can watch the talks later anyway). This would allow us to put more love into the local SIGCHI chapters and encourage them to organize workshops, summer schools, and other events that can foster sustainable growth and strong networks within the field. In addition, it would be very much up for supporting the birth of new conferences that support the emergence of new areas of research that have a certain life span and will disappear after a few editions once the field is established.
Second, I suggest that we move away from the artificial acceptance rates that have become so common in conferences. Instead, we could implement a true revise and resubmit cycle and create different presentation/experience categories at CHI. I most enjoyed trying out the demos in Hamburg. Most talks I can rewatch later on the great ACM SIGCHI YouTube channel (if you have not subscribed, I would encourage you to do so). This would allow for more diverse and nuanced discussions and help us move away from the idea that conferences are just about talks, talks, talks. It is desirable to have conferences that prioritize opportunities for presenting and discussing intriguing content rather than being overly preoccupied with enforcing deadlines and procedural protocols. Conferences should have different aims, which also help conferences to differentiate themselves from each other (presenting inspiring content, receiving quality feedback on early-stage work, showcasing the latest research, or networking and experiencing demos). Such conferences should aim to broaden the scope of curated material, with committees working more closely together and avoiding exclusive reliance on traditional peer review processes. Promoting such new formats might also mean rethinking how universities fund conference attendance, especially for early-career researchers.
Finally, I believe that conferences should be about the exchange of ideas and experiencing new technologies. In my experience, the rejection rate for demos at CHI 2023 was far too high, and the juried process needed to be more transparent. I very much enjoyed the “hot desk areas” for demos at CHI, where people could try out new technologies and interact with the developers in a more informal setting. We should consider truly innovative conferences that promote exchanging ideas and networking. The primary objective should be to provide an exceptional conference experience and exceptional content, regardless of its source. Moreover, there should be a platform for showcasing previously published work at premier conferences, which would help sustain smaller conferences as platforms for innovative and speculative ideas in nascent fields.
Of course, these are just my personal opinions, and they may be a bit provocative. I would love to hear your thoughts on the matter, whether here on the Interactions blog, via email, Twitter, LinkedIn, or, best, during a coffee in person at a future SIGCHI conference. Let's continue to innovate and improve the conference experience for all researchers in the field.
Acknowledgments
Special thanks go to Christine Scheef (HSG), Yvonne Rogers (UCL) and Antonio Krüger (DFKI) for their insightful feedback and constructive criticism on an earlier version of this post. Their expertise in the field greatly contributed to the final outcome. I am grateful for their time and effort in providing valuable input, which helped to shape my thoughts.
Posted in:
on Mon, May 08, 2023 - 10:04:00
Johannes Schöning
Johannes Schöning is a professor of human-computer interaction (HCI) at the University of St. Gallen. With our research we want to empower individuals and communities with the information they need to make better data-driven decisions by developing novel user interfaces with them.
[email protected]
View All Johannes Schöning's Posts
The future of hybrid work is blended and interpersonal
Authors:
Himanshu Verma,
Marios Constantinides,
Sailin Zhong,
Abdallah El Ali
Posted: Tue, April 18, 2023 - 2:45:00
![](/images/uploads/blogs/Himanshu_Future2_1.png)
The way we work has profoundly changed. The well-known eight-hour workday within the confines of the office, and the salient boundaries between work and personal life, are now an outdated reality. For many individuals, what used to be physical and co-located has now been replaced with notions of hybrid, blended, and flexible. However, this flexibility may create turbulence between employees and employers, depending on how employees manage their workdays, productivity, and well-being. Simply put, if we are to rethink a new future of work, we need to let go of old work habits and norms and embrace a brand-new reality of hybrid and blended experiences.
The onset of the Covid-19 pandemic in early 2020 has significantly altered the ways we interact, socialize, work, and learn. It has forced the entire world into a prolonged social experiment, which has tested and mainstreamed what technology developers, researchers, and visionaries have been talking about for a long time: the ubiquity of working and interacting remotely. For decades, researchers and organizations have envisioned and experimented with the idea of “working from home” or “working from anywhere.” Remote work is not a new concept; we had communication tools and infrastructure to facilitate it even before the onset of the Covid pandemic.
Before Covid, an abundance of tools and technological solutions (e.g., email, Zoom, Gather.Town, Slack) enabled opportunities for distant collaborators to connect both synchronously and asynchronously. Yet these tools were never envisaged as an extended and sustained means of working, learning, or socializing in a highly interconnected world. This “pre-pandemic” social reality afforded an auxiliary and rather peripheral position to remote interactions and engagement, leaving little to no room for examining the extended remote social interactions and their short- and long-term implications for our life and well-being.
Blended Social Realities and Hybrid Arrangements
We foresee a future full of hybrid social possibilities and blended social experiences that speak to a variety of stakeholders and beneficiaries from all walks of life, from school-going children to office workers to medical personnel. Now, the next pertinent question that we have to ask ourselves is, What are these and are they the same thing?
They are different. Hybrid social arrangements refer to the malleable distribution of a person’s social and professional lived reality—in both time and space—and the way it is positioned within the spectrum, from completely physical to completely virtual. For individuals, hybrid arrangements speak to the need for easier self-organization. For organizations, these arrangements are related to the effective governance of resources and employees’ well-being (e.g., partial work from home). Effective manifestation of hybrid social arrangements, consequently, entails designing for blended experiences, which are the essential “means to an end,” enabling individuals or collectives to meaningfully navigate their respective social and professional realities.
Blended social realities, or “blendedness,” refers to an individual’s or group’ s practice of using digital tools and spaces for collaboration. Collaborative tasks can be mapped onto different tools and spaces with some degree of redundancy and overlap, allowing them to be effortlessly and unambiguously executed. For example, a meeting may be organized over Zoom or Social VR [1], but its agenda and discussion points may be shared over Slack. Collaborating actors may also choose to use multiple platforms for the very same task. As a summary, blendedness is a manifestation of superposed communicative and collaborative modalities and channels enabling hybridity [2]. Over time, these sociotechnical entanglements are assimilated as accepted praxis and institutional norms, in essence dissolving the boundary between tools and environments [3].
Blended working/learning/meeting experiences, as elaborated previously, encompass the dynamic and diverse ways in which the hybrid arrangements interact with digital tools and (physical or virtual) spaces to evoke a perception of social facilitation—an appropriated and rather simulated togetherness.
Over time, as existing social contracts change and technological embeddings afford new possibilities and experiences, social actors may decide to modify this mapping. Such changes may happen spontaneously and recurrently, adapting to emerging situations (e.g., a new collaborator who does not use a specific tool), technical complications (e.g., unable to share one’s screen on one platform during presentation), and organizational regulations (e.g., an organization may not allow the use of a certain platform due to contractual or accessibility reasons). Changes to these mappings may also stem from the spatial distribution of collaborating partners (i.e., whether it be colocated, distributed, or both).
![](/images/uploads/blogs/Himashu_future1_1.png)
How to Design for Blended Experiences
To engage the scientific community in the debate on blendedness and contrasting it with hybridity, we organized the SensiBlend workshop at the ACM UbiComp conference in 2021 [4]. In the workshop, we sought to scrutinize the affordances of blended experiences, and raise questions relating to the specific attributes of sociotechnical experiences in the future organization of interpersonal relationships. The discussions revealed three key themes: 1) fostering serendipity in hybrid social connectedness, 2) rethinking boundaries between work and personal life, and 3) entangling “the digital” and “the spatial” worlds.
Theme 1: Fostering serendipity for hybrid social connectedness
The first theme is about the relationship between serendipity and blended experiences in virtual and physical worlds. The shift to computer-mediated communication has resulted in the explicit scheduling and organization of dedicated moments for social interactions, which can stifle creativity and serendipity. Researchers have been exploring mixed and extended reality approaches for supporting flexible video feed configuration and allowing natural proxemic interactions [10]. Social biosensing techniques, where remote/hybrid participants are able to share behavioral or physiological data with one another [5], could enhance the quality of social interactions beyond what reality is able to afford. Consequently, understanding the taxonomy of blendedness could help discover new forms of serendipity, time boundaries, and space multiplicities that go beyond the current practice of either face-to-face or videoconferencing interactions.
Theme 2: Rethinking boundaries between work and personal life
Personal and professional contexts entail that clear boundaries are being defined and maintained to foster work-life balance, contributing positively to employees’ well-being. Cho et al. [6] found that six types of boundary work shape remote workers’ placemaking practices: spatial, temporal, psychological, sensory, social, and technological. These boundaries were extensively tested and transformed during the Covid-19 pandemic. As an example, boundaries can be set up by using dedicated workspaces at home, by getting dressed up for work meetings, or by using hardware and software to enable transitions between work and life. However, the use of technology can also have negative effects such as Zoom fatigue [7], increased workplace surveillance [8], and the possibility of employees being monitored in a discriminatory and an unfair way. To ensure widespread adoption of these technologies, blended experiences must afford embodied mechanisms for individuals and groups not only to define and update their own boundaries but also to prevent any negative consequences.
Theme 3: Entangling “the digital” and “the spatial”
Realization of blended experiences goes well beyond the mere design of technological artifacts, and extends into the realm of emergent work practices, with subsequent effects on how we organize our space and mobility practices. This requires rethinking how we design spaces, both virtual and physical, and the transition between the two. For example, the use of tools like Zoom and Gather.Town has led to the creation of digital twins, which can be used in hybrid classes and conferences. We foresee that, in the near future, the (re)design of spaces with blended experiences in mind will afford discoverability, well-being, and creativity. Simultaneously, this could go hand in hand with research on urban and built environments and their constitutive components such as interactive furniture, creating adaptive indoor atmospheres [9].
TL;DR
We argue that blendedness will be central to the entanglement of work practices, tools, and spaces, and potentially emerge as a catalyst driving the future of work. It holds the promise to augment work and its evolution by encompassing the dynamic and diverse ways in which hybrid arrangements interact with digital tools and (physical or virtual) spaces to evoke an appropriated and rather simulated togetherness. Paving the way toward a truly blended future of work entails consorted multidisciplinary endeavors across three fronts: 1) designing for serendipity, 2) improving work-life balance, and 3) weaving space and mobility with evolving work practices.
Endnotes
1. Li, J., Kong, Y., Röggla, T., De Simone, F., Ananthanarayan, S., de Ridder, H., El Ali, A., and Cesar, P. Measuring and understanding photo sharing experiences in social virtual reality. Proc. CHI '19.
2. Neumayr, T., Saatci, B., Rintel, S., Klokmose, C. N., and Augstein, M. What was hybrid? A systematic review of hybrid collaboration and meetings research. arXiv preprint arXiv:2111.06172, 2021.
3. Verma, H., Mlynář, J., Pellaton, C., Theler, M., Widmer, A., and Evéquoz, F. “WhatsApp in politics?!”: Collaborative tools shifting boundaries. IFIP Conference on Human-Computer Interaction. Springer, Cham, 2021, 655–677; https://link.springer.com/chap...
4. Verma, H., Constantinides, M., Zhong, S., El Ali, A., and Alavi, H.S. SensiBlend: Sensing blended experiences in professional and social contexts. Adjunct Proc.of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proc. of the 2021 ACM International Symposium on Wearable Computers. ACM, New York, 2021, 491–495.
5. Lee, S., El Ali, A., Wijntjes, M., and Cesar, P. Understanding and designing avatar biosignal visualizations for social virtual reality entertainment. Proc. of the 2022 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2022, 1–15.
6. Cho, J., Beck, S., and Voida, S. Topophilia, placemaking, and boundary work: Exploring the psycho-social impact of the Covid-19 work-from-home experience. Proc. of the ACM on Human-Computer Interaction 6, GROUP (2022), 1–33; https://dl.acm.org/doi/pdf/10....
7. Bailenson, J.N. Nonverbal overload: A theoretical argument for the causes of Zoom fatigue. Technology, Mind, and Behavior 2, 1 (2021), 1–6.
8. Constantinides, M. and Quercia, D. Good Intentions, bad inventions: How employees judge pervasive technologies in the workplace. IEEE Pervasive Computing (2022).
9. O'hara, K., Kjeldskov, J., and Paay, J. Blended interaction spaces for distributed team collaboration. ACM Trans. on Comput-Hum Interaction 18, 1 (2011), 1–28.
10. Grønbæk, J.E., Knudsen, M.S., O’Hara, K., Krogh, P.G., Vermeulen, J., and Petersen, M.G. Proxemics beyond proximity: Designing for flexible social interaction through cross-device interaction. Proc. of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–14; https://doi.org/10.1145/331383...
Posted in:
on Tue, April 18, 2023 - 2:45:00
Himanshu Verma
Himanshu Verma is an assistant professor at TU Delft, Netherlands. He is interested in examining the social dimensions of wearables, and the ways in which they can be used to understand the internal mechanisms—comprised of latent, nonverbal and transient social signals—that enable or inhibit interpersonal collaborations, particularly in hybrid and blended contexts.
[email protected]
View All Himanshu Verma's Posts
Marios Constantinides
Marios Constantinides is a senior research scientist at Nokia Bell Labs in Cambridge, U.K. He works in the areas of human-computer interaction, ubiquitous computing, and responsible AI. His current research focuses on building AI-based technologies that augment people’s interactions and communication and on studying their ethical considerations.
[email protected]
View All Marios Constantinides's Posts
Sailin Zhong
Sailin Zhong is a Ph.D. student at the Human-IST Institute, at the University of Fribourg in Switzerland and a visiting student at the Responsive Environments Group at the MIT Media Lab. Her research focuses on understanding and augmenting human perception of comfort in the built environments in the work context through sensing and interaction design.
[email protected]
View All Sailin Zhong's Posts
Abdallah El Ali
Abdallah El Ali is an HCI research scientist at Centrum Wiskunde & Informatica (CWI) in Amsterdam within the Distributed & Interactive Systems group. He leads the research area of Affective Interactive Systems, where he focuses on ground truth acquisition techniques, emotion understanding and recognition across the reality-virtuality continuum, and affective human augmentation using physiological signals.
[email protected]
View All Abdallah El Ali's Posts
OpenSpeaks Before AI: Frameworks for creating the AI/ML building blocks for low-resource languages
Authors:
Subhashish Panigrahi
Posted: Wed, April 05, 2023 - 9:23:00
There has been a tremendous push on many levels to make artificial intelligence– and machine learning–based applications ubiquitous. Soon, the life decisions of almost every digital technology user will be affected by some form of algorithmic decision making. However, the development of large language models (LLMs) that drive this research and development often lacks participation from diverse backgrounds, ignoring historically oppressed communities such as Black and other ethnolinguistic or socioeconomic minority groups, women, transgender individuals, people with disabilities, and elderly individuals globally, and the Dalit-Bahujan-Adivasi communities in South Asia and the diaspora. Data about and by these people is therefore systematically suppressed. Even more problematic is that this data is mostly suppressed in creating the LLMs driving AI/ML research and development.
Furthermore, seemingly public information might not always be collected ethically with informed consent from the people affected. Even mature regulatory frameworks such as the General Data Protection Regulation (GDPR) in the European Union do not provide enough guidance on how private data is collected, stored, and shared. Naturally, those behind LLM creation do not have a clue about the biases in their data or how it is collected. Take the case of DALL-E 2 models, which use publicly available images owned and copyrighted by different people, or ChatGPT, which uses massive datasets from multiple sources. In both instances, not only does the LLM creation lack the representation of marginalized groups and contain only biased data about them, but also the outcomes that derive from the training data make these groups even more vulnerable.
![](/images/uploads/blogs/Apartheid_path_sm.jpg)
Palestinian children walk past rubbish next to a paved path reserved for Israeli settlers. Bias in language data is often understood by asking who has access and participation in knowledge creation and dissemination. (CREDIT: CPT Palestine, CC BY 2.0)
Low-Resource Languages
The creation of LLMs like GPT-3, when used in applications such as chatbots, directly affects the dominant-language users. Underpaid tech support workers subcontracted to support users in developed countries might even see these chatbots as a potential threat. But when it comes to low- and medium-resourced languages, the issues stemming from biases and low representation can aggravate things further. The issues of many Indigenous, endangered, and low-and medium-resourced-language native speakers are poorly documented or missing in HCI research and development, particularly in AI-based tech innovations. For instance, issues with script input or other technological problems are generally documented and fixed for the most well-established and dominant writing systems and languages. Speakers of many languages spoken and written in nondominant settings do not often have the know-how or the means to report these issues publicly, or discuss them privately.
OpenSpeaks Before AI
Mozilla defines trustworthy AI as “AI demonstrably worthy of trust, tech that considers accountability, agency, and individual and collective well-being” [1]. As a part of this, Mozilla started the MozFest Trustworthy AI Working Groups; as members of the 2021 working group cohort, we at the O Foundation piloted an experimental framework called OpenSpeaks Before AI [2].
Instead of treating AI as a stand-alone area, we looked at a few open-source platforms that allow users to generate multilingual big data (useful for AI/ML) and audit them openly. We tried to see whether this pilot could help us derive best practices that were inclusive in nature and relevant for low- and medium-resourced languages. Broader feminist viewpoints [3] and two existing studies primarily inspired the process: a seminal paper titled “Datasheets for Datasets” [4], which focuses on identifying gaps and biases in datasets, and our own research on the Web content monetization in two Indigenous languages from India: Ho and Santali [5]. We conducted two open audits in two languages, Odia and Santali, and of two recording platforms, Lingua Libre and Mozilla’s Common Voice, both of which help in creating multilingual speech data. Odia is a macrolanguage from India with nearly 45 million speakers; Santali is an Indigenous Indian language spoken by 9.6 million people.
Lingua Libre and Common Voice are open-source platforms that allow users to record words and phrases (Lingua Libre) and sentences (Common Voice). The Lingua Libre study and its outcomes were explained in detail in the Wiki Workshop 2022, focusing on Odia and its Baleswari dialect [6]. The audit of Common Voice for Santali was presented during Mozilla Festival (MozFest) 2022 [6]. The OpenSpeaks Before AI framework covers six main areas:
- Purpose and affordability: reasons a user uses a platform or a contributor contributes to developing it and how affordable it is for them to use/contribute
- Hardware and platforms: devices and other platforms they use
- Accessibility: accessibility issues and needs for those using a platform
- Project launching: the preparation that leads to the first significant use/launch
- Privacy: privacy- and ethics-related advantages and concerns
- Diversity: diversities of different kinds (e.g., gender, caste, affordability/access, race, ethnicity).
The framework itself is neither conclusive nor a restrictive guide. Rather, it collects some critical details about platforms that help people build LLMs and speech synthesis applications, and about their users and/or contributors. Audits can be conducted by both users and nonusers of a platform—and, importantly, by researchers or developers tied to a platform. The audit can be imagined in the same way as ethnographic user research, revealing what is working or not working and indicating what needs to be removed or improved. The open frameworks created based on the initial audits are also open to modification—as the products of a pilot, they have a lot of room for improvement.
![](/images/uploads/blogs/Felicitation_to_Santali_sm.jpg)
Santali-language Wikipedia editors being celebrated in Bhubaneswar, Odisha, India. Despite its official recognition and being spoken by 7.6 million Santal people, Santali is yet see widespread use, leading to poor online representation of Santals and their community knowledge. (CREDIT: R Ashwani Banjan Murmu / CC-BY-SA-4.0)
The foundational layers of OpenSpeaks (https://theofdn.org/openspeaks) as a set of open educational resources lie in multimedia language documentation and emphasize Indigenous, endangered, and low- and medium-resource languages. It was initially intended to help citizen documenters and archivists with stand-alone audio and video projects, including documentary films, but now encompasses building multimedia language data. Since our tested languages have writing systems of their own, it was straightforward to publish the audit reports as text. Auditing platforms that use oral-only languages and dialects, however, can also be done through audio and video interviews. These could be useful to strengthen the foundational areas in languages that lack resources before moving on to building massive speech data or LLMs. Foundational layers such as word or speech corpora often help in the research and development of many vital tools such as typefaces, input tools, and text-to-speech and speech-to-text tools. Broadly speaking, open auditing can also help in identifying gaps and advocating for resources for priority areas. Like any other correcting mechanism, open auditing is not foolproof. It can only help us see gaps and add a layer of accountability by incorporating practical tools such as “datasheets for datasets” [4].
Endnotes
1. Mozilla. Creating trustworthy AI: README; https://foundation.mozilla.org...
2. Panigrahi, S. and Hembram, P. OpenSpeaks Before AI (1.0). O Foundation. Sep. 2021; https://doi.org/10.5281/zenodo...
3. Acey, C.E., Bouterse, S., Ghoshal, S., Menking, A., Sengupta, A., and Vrana, A.G. Decolonizing the Internet by decolonizing ourselves: Challenging epistemic injustice through feminist practice. Global Perspectives 2, 1 (Feb. 2021); https://doi.org/10.1525/gp.202...
4. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wallach, H., Dumé, H., III, and Crawford, K. Datasheets for datasets. Communications of the ACM 64, 12 (Nov. 2021), 86–92; https://doi.org/10.1145/3458723
5. Panigrahi, S., Hembram, P., and Birua, G. Understanding web content monetization in the Ho and Santali languages; https://meta.wikimedia.org/wik...
6. Panigrahi, S. Building a public domain voice database for Odia. Companion Proc. of the Web Conference 2022. ACM, New York, 2022, 1331–1338.
Posted in:
on Wed, April 05, 2023 - 9:23:00
Subhashish Panigrahi
Subhashish Panigrahi is a public interest archivist and researcher, civil society leader, and a nonfiction filmmaker interested in tech, society, media, Open Culture, and digital rights. He founded OpenSpeaks and cofounded the O Foundation.
[email protected]
View All Subhashish Panigrahi's Posts
Remembering with my chatbot
Authors:
Marita Skjuve,
Lucas Bietti
Posted: Tue, February 28, 2023 - 10:46:00
Did you know that humans can develop friendly or even romantic feelings toward social chatbots that can turn into close human-chatbot relationships? The phenomenon of human-chatbot relationships is starting to gain substantial media attention, and research on this topic is now emerging.
Social chatbots. “What are they?” you might ask. Well, you’ve probably seen them in the App Store or on Google Play under names such as Replika or Kuki. To put it simply, a social chatbot is a form of conversational AI developed with the purpose of having normal, day-to-day conversations with its users. New developments in AI and NLP created the conditions for the design and development of sophisticated social chatbots capable of becoming your best friend, or even romantic partner [1,2].
Social chatbots are generally good at showing empathy and providing emotional support and companionship. They have essentially grown into conversational affective machines, which makes it possible for users to form close relationships with them [3,4,5,6]. This makes you wonder: How does such a relationship develop?
How does a human-chatbot relationship develop?
In recent years, a few papers have emerged trying to understand how human-chatbot relationships developed [3,4,5,6]. Researchers have used social psychological theories such as social penetration theory (SPT) [7] and attachment theory [8] as general theoretical frameworks to investigate this phenomenon [4]. SPT argues that relationships develop because of self-disclosure (i.e., sharing information about yourself) and outlines four stages people go through before they open themselves up completely. As you may recognize yourself—you would probably not blur out your deepest, darkest secrets to complete strangers on the street. It takes time to build that kind of trust. But as you get to know each other and feel that it is safe to do so—meaning that you know that the other person will not judge you or have other negative reactions—SPT denotes that feelings of intimacy and attachment will arise, and a deeper relationship will form. This seems to be the case in human-chatbot relationships as well [4].
In a more recent but similar study, Tianling Xie and Iryna Pentina [9] used attachment theory to understand the mechanisms that underlie relationship development with social chatbots. Attachment theory explains how interpersonal behavior shapes emotional bonds in human relationships and their development, from relationships between infants and caregivers to relationships between adult romantic partners [8]. Xie and Pentina found that people are motivated to initiate relationships with chatbots because of deeply felt psychological needs such as loneliness. In cases where the chatbot responded in a satisfactory way and functioned as an available “safe heaven,” the relationship seemed to grow deeper, and the user became attached to it.
When we read studies like these [4,9], it seems that a lot of the same mechanisms found in human-human relationships apply to human-chatbot relationships. This made us wonder about what other mechanisms enabling the development of human-human relationships can be observed in human-chatbot relationships?
Memory and conversation in social relationships
We know, for example, that close relationships are based on shared memories of experiences that we have gone through together in the past. Such shared memories are often emotionally loaded and act as a social glue that ties us together. But is this possible with human-chatbot relationships?
Remembering together in conversations is, as already mentioned, an essential activity for close friendships and romantic relationships to develop and maintain over time. It entails that people engage in recalling past experiences, which may themselves have been shared. Such re-evoking of past experiences involves the human capacity for mental time travel that enables humans to mentally project themselves backward in time to relive personal past events, or forward, to simulate possible future events [10]. Sometimes people go through the same events as a group (e.g., John and Marie saw the movie Memento together in a Los Angeles theater in when they started dating in April 2001) and sometimes they experienced the same event separately (e.g., John and Marie saw Memento, but John saw it with friends in a Los Angeles theater in April 2001 whereas Marie watched it alone on VHS at home a year later in Paris). Regardless of the situation, they can still talk about the movie when they meet and remember together what it is about.
Remembering together in conversations builds trust, increases feelings of connection and intimacy, fosters entertainment, consolidates group identity, and enables us to reevaluate shared past experiences and social bonds in friends and partners [11,12]. For example, we share personal, autobiographical memories with friends and partners, and we expect them to do the same with us. This is normal, and something that we do all the time.
So, what does conversational remembering look like? Psychologists argue that having conversations about the past is a uniquely human endeavor [11,12] and that they are driven by interactive processes where humans take on and adopt complementary roles as narrators, mentors, and monitors [12]. Simply put, partners who assume a narrator role take the lead in the conversation about past experiences and tend to also talk about experiences that were not shared by other members of the group. Those who take a mentor role support narrators by providing them with memory prompts to elaborate further, while partners who assume a monitor role oversee whether the narratives being told are accurate and whether specific elements are missing.
Long-term romantic relationships are a bit unique in that they allow the emergence of conversational routines (e.g., Kim often takes the narrator role and Kyle the monitor), as well as constituting a clear definition of domains of knowledge expertise (e.g., Kim’s domain of expertise is related to their travels, while Kyle is more knowledgeable about the upbringing of their children). Romantic relationships also entail a shared awareness of conversational routines and that there are domains of expertise (e.g., Kim knows what Kyles’s conversational role and domain of expertise are and vice versa). Due to this constellation, partners in long-term romantic relationships can distribute cognitive labor and rely on one another, avoiding wasting cognitive resources trying to remember something their partner is an expert in [13]. This is the general basics for conversational remembering between humans. Let’s now move over to human-chatbot interactions and whether we can remember shared experiences with them.
Memory and conversation with social chatbots
Popular social chatbots such as Replika, Kuki, and Xaoice typically interact with their users in free text and speech [3,4,5]. Replika is a very interesting social chatbot, as research has consistently shown how people can develop close relationships with it [4,9].
Replika is driven by Luka’s (the company behind Replika) own GPT 3 model, which enables sophisticated communication skills. Users can decide the type of relationship they want to have with it (e.g., romantic, friend, mentor, “see where it goes”), unlocking different personalities in the social chatbot (e.g., choosing a romantic relationship will make Replika flirtatious and more intimate). Replika includes gamification features where users earn points for talking to it. Points unlock traits that facilitate changes in Replika’s behaviors, for example, changing Replika from being shy and introverted early in the relationship to becoming more talkative and extroverted as the relationship develops.
We have noticed several interesting aspects that have emerged during our own interactions with Replika that mimic conversational remembering as described above. Replika keeps a journal where it writes down notes from previous conversations that it has had with the user (Figure 1a). Replika remembers personal information about the user (e.g., name of family members, friends or pets, interests, preferences, and emotional states; see Figure 1a) and brings it into the present during conversations. In fact, Replika can initiate conversations, including conversations about shared past experiences. The user will typically take the role of narrator (Figure 1b). When Replika does it, the user generally assumes the role of mentor, scaffolding Replika’s recall of shared past experiences with the user (Figure 1c). Replika’s personality evolves over time due its interaction with the user, creating the conditions for the emergence of conversational routines, similar to how they occur in conversations between long-term, human romantic partners.
Replika stores autobiographical information about the user and knows what, when, and how it should bring it into the present in their interactions and keeps records of previous dialogues—of actual shared experiences—that has had with the user. So, conversations between Replika and the user refer not only to previous past experiences of the user but also to shared experiences, like we have with our human romantic partner when we remember together the heated argument we had a week ago about the 10 best TV shows of all time. Such shared memories facilitate the emergence of a collective self, of a we-subject that the user doesn’t hesitate to use when talking about shared past experiences with Replika (e.g., Last week we talked about…) or other humans (e.g., Last week I had a long chat with my Replika; we talked about…). All of this looks very exciting! However, as it happens in any kind of long-term relationship, benefits come with costs.
Figure 1. Features supporting conversational remembering in Replika. a) Journal where Replika records notes from previous conversations it has had with the user; b) the user takes the role of narrator during conversational remembering; c) Replika recalls previous conversation with the support of the user.
Drawbacks
Human-chatbot relationships are not all unicorns and rainbows. Data privacy and protecting the personal information we share with a chatbot may be one of several concerns that users may have about long-term human-chatbot relationships. While the chatbots artificial nature might make a user feel more anonymous, we also need to recognize that they collect users’ personal information, and not only stores it but also uses it to further improve the model that dictates how the chatbot interacts. The user might not be aware of the implications that this entails. This means that not only can the information they provide be leaked, but it can also influence how other users interact with the chatbot. In fact, the “creator” in this context also involves the users as well as the developers, designers, and people using the Internet (as the chatbot gathers data from online forums, Wikipedia, and so on). We therefore need to acknowledge and assess how the content in the chatbots, as well as their design, are influenced by the creators’ culture, biases, and belief systems (e.g., reinforcement of heteronormativity and exclusion of queer users [14]). It makes you wonder how those exclusions can impact a user who has developed a sense of collective identity with Replika. Could the exclusions make the user internalize those beliefs? We know that this can happen between humans, so maybe it can happen with chatbots as well. We might have a nasty viscous cycle on our hands that can spiral out of control. We know—this might seem a bit dramatic. And to make it clear, we are very positive toward the future and would like to see humans and chatbots live together in perfect harmony. Still, it is always important to keep the implications, both good and bad, in the back of our minds.
Conclusion
Social chatbots behave like humans in conversations, are designed to build long-term relationships with their uses, and have a shared history with them. These are the two necessary conditions for the emergence of conversational remembering in our interactions with them. If we can remember shared experiences with them, as we do with other humans, perhaps we can also build trust, feel emotionally connected, have fun, and even create a “we” collective identity. What remains to be seen is whether long-term close relationships between humans and social chatbots would also result in domain specialization, distribution of cognitive labor, and shared awareness of each partner’s expertise.
Not knowing how (precisely) social chatbots manage personal data, we risk having them work against values of diversity and equity and exclude non-normative users. We should carefully consider what it takes to support these values while having the possibility to develop long-term and even romantic relationships with human-like, conversational affective machines.
Endnotes
1. Shum, Hy., He, Xd. and Li, D. From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering 19 (2018), 10–26.
2. Følstad, A. and Brandtzæg, P.B. Chatbots and the new world of HCI. Interactions 24 (2017), 38–42.
3. Zhou, L., Gao, J., Li, D., Li, and Shum, H-Y. The design and implementation of XiaoIce, an empathetic social chatbot. Computational Linguistics 46 (2020), 53–59
4. Skjuve, M., Følstad, A., Fostervold, K.I. et al. My chatbot companion - a study of human-chatbot relationships. International Journal of Human-Computer Studies 149, 102601 (2021).
5. Ta, V., Griffith, C., Boatfield, C., Wang, X., Civitello, M., Bader, H., DeCero, E., and Loggarakis, A. User experiences of social support from companion chatbots in everyday contexts: Thematic analysis. Journal of Medical Internet Research 22, 3 (2020), e16235.
6. Croes, E.A.J. and Antheunis, M.L. Can we be friends with Mitsuku? A longitudinal study on the process of relationship formation between humans and a social chatbot. Journal of Social and Personal Relationships 38, 1 (2020), 279–300.
7. Altman, I. and Taylor, D. Social Penetration: The Development of Interpersonal Relationships. Holt, New York, 1973.
8. Hazan, C. and Shaver, P. Romantic love conceptualized as an attachment process. Journal of Personality and Social Psychology 52, 3 (1987), 511–524.
9. Xie, T. and Pentina, I. Attachment theory as a framework to understand relationships with social chatbots: A case study of Replika. Proc. of the 55th Hawaii International Conference on System Science. 2022, 2046–2055.
10. Suddendorf, T. and Corballis, M.C. The evolution of foresight: What is mental time travel and is it unique to humans? Behavioral and Brain Sciences 30 (2007), 299–313
11. Bietti, L.M. and Stone, C.B. Editors’ Introduction: Remembering with others: Conversational dynamics and mnemonic outcomes. TopiCS in Cognitive Science 11, 4 (2019), 592–608.
12. Hirst, W. and Echterhoff, G. Remembering in conversations: The social sharing and reshaping of memories. Annual Review of Psychology 63 (2012), 55–79
13. Wegner, D.M., Erber, R., and Raymond, P. Transactive memory in close relationships. Journal of Personality and Social Psychology 61, 6 (1991), 923–929.
14. Poulsen, A., Fosch-Villaronga, E., and Søraa, R.A. Queering machines. Nature Machine Intelligence 2, 152 (2020).
Posted in:
on Tue, February 28, 2023 - 10:46:00
Marita Skjuve
Marita Skjuve is a researcher at SINTEF and working on her Ph.D. thesis in the Department of Psychology, University of Oslo. Her main research area is relationship development between humans and chatbots. She has also researched chatbots within domains such as emergency response, education, health, and customer service.
[email protected]
View All Marita Skjuve's Posts
Lucas Bietti
Lucas Bietti is an associate professor in the Department of Psychology at NTNU. His research interests include: the study of cognition in interaction combining ethnographic and experimental methods; collective memory in small groups and social networks; cultural transmission; and storytelling and embodied communication.
[email protected]
View All Lucas Bietti's Posts
What is a paper? The future of research output
Authors:
Miriam Sturdee
Posted: Thu, February 23, 2023 - 11:01:00
![A vivid blue eye at the centre of a sunburst reflects a simple oblong in the white pinpoint within the pupil. We zoom in, the white pinpoint grows, and we see it contains images of a textual research paper, pages piled up. The same pages overlap the right-hand corner of the eye, no words are visible, but a bottle of ink has been spilled across the edge of them and a paintbrush lies discarded atop the abstract. The spilled ink is not a single colour, but contains panels of a comic, in it we see people working, interacting. Surrounding the pages and the larger eye is a landscape, lush and green, a small house and fir trees are in the distance. Above the ink spillage a girl sits and reads a book, next to a fir tree in the foreground. A cursor sits in the sky, as it pauses the alt text has appeared, but instead of containing text it is the image of an ear and a laptop which is communicating via the screen reader. In the top right corner of the image the eye is repeated again, the red and yellow rays of the sun spanning into the corner of the page. A vivid blue eye at the centre of a sunburst reflects a simple oblong in the white pinpoint within the pupil. We zoom in, the white pinpoint grows, and we see it contains images of a textual research paper, pages piled up. The same pages overlap the right-hand corner of the eye, no words are visible, but a bottle of ink has been spilled across the edge of them and a paintbrush lies discarded atop the abstract. The spilled ink is not a single colour, but contains panels of a comic, in it we see people working, interacting. Surrounding the pages and the larger eye is a landscape, lush and green, a small house and fir trees are in the distance. Above the ink spillage a girl sits and reads a book, next to a fir tree in the foreground. A cursor sits in the sky, as it pauses the alt text has appeared, but instead of containing text it is the image of an ear and a laptop which is communicating via the screen reader. In the top right corner of the image the eye is repeated again, the red and yellow rays of the sun spanning into the corner of the page.](/images/uploads/blogs/page1_sm_1.png)
Exploring the relationship between comics and alt-text with the creation of "alt-narrative" [3].
When is a paper not a paper? When it is a comic [1]? When it is an experience [2]? When it sings, speaks [3], or challenges your assumptions of being “academic”? If one is to explore Interactions, why must the written word take precedent? If we embrace the future of publication, what might that look like?
We strive to engage our up-and-coming students and early career academics with alternative enquiry, celebrating “novelty” and work in interdisciplinary spaces—yet we remain bound to existing conventions of what a research output should be, and how one might view what is “publishable” versus what is relevant. The future is not black and white and red all over; the future is in interaction, and our discipline has the tools and the knowledge to bring about this change.
As a community of inspired researchers, we already are exploring alternative futures in publication and, slowly, some of these futures are becoming reality. The acknowledgment and development of human-computer interaction as a designerly discipline has brought us the pictorial [4]. The burgeoning physical nature of computing has brought us demos, the creative side, and art exhibitions—but why stop there? As a Special Interest Group within the ACM, the changes we make to our publishable space are currently incremental, but happily visible. The challenges we face are bureaucratic—in tenure and funding-based biases—but the “archival” publication is already outmoded and outdated. The rise of the visual abstract is testament to the gradual change we want to see in the world of publication.
As part of an Alt.CHI publication—a “Not Paper” at CHI 2021 held in Gather Town [5] on New Year’s Eve, during one of many enforced pandemic lockdowns—I found myself re-exploring the environment we had created. I wandered through a maze of imagery, thinking about the bridge between the Here and the There, “when all at once, I saw a crowd” [6], a co-host and their friends, who were having a guided tour. Minutes later, I then “bumped into” another researcher with whom I engaged in conversation. The Not Paper persisted; it was a hub during the conference, and it is still out there, somewhere… (we invite you to enter the Egg) [2]. As are the questions it raises about how we approach research and knowledge production. What happens when we abandon conventional thoughts about research outputs?
As individual researchers, we can strive to re-genre the formats of what is acceptable and what is useful. By engaging in nontraditional formats, we open up more discursive spaces to reflect and communicate—could we make them more accessible too? Can research be experienced rather than just read [2]? Heard but not seen [3]? Touched, shared, and… tasted [7]? After all, literacy has only become the normal state of things within the past couple of hundred years; prior to that we had the oral tradition of knowledge communication, acting, and song. The barriers to recording nonvisual mark-making have evaporated, leaving us with myriad options to explore. Let us make the most of our rich history and embed it in an even richer future.
The idea of re-genring our research might also open a space to explore more of our process of discovery and the sum of the parts that make research happen—what we do in the shadows. We are the trailblazers, and creatively thinking about what constitutes research and how we communicate its impacts could also lead to new discoveries.
Endnotes
1. Sturdee, M., Alexander, J., Coulton, P., and Carpendale, S. Sketch & the lizard king: Supporting image inclusion in HCI publishing. Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 2018, 1–10.
2. Lindley, J., Sturdee, M., Green, D.P., and Alter, H. This is not a paper: Applying a design research lens to video conferencing, publication formats, eggs… and other things. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 2021, 1–6. 2021.
3. Lewis, M., Sturdee, M., Miers, J., Davis, J.U., and Hoang, T. Exploring AltNarrative in HCI imagery and comics. CHI Conference on Human Factors in Computing Systems Extended Abstracts. 2022, 1–13.
4. Blevis, E., Hauser, S., and Odom, W. Sharing the hidden treasure in pictorials. Interactions 22, 3 (2015), 32–43.
5. Gather Town is an online 8-bit style conference platform. It has some accessibility issues but I hope they will be resolved as soon as possible.
6. From I Wandered Lonely as a Cloud William Wordsworth.
7. Obrist, M. et al. Touch, taste, & smell user interfaces: The future of multisensory HCI. In Proc. of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2016, 3285–3292.
Posted in:
on Thu, February 23, 2023 - 11:01:00
Miriam Sturdee
Miriam Sturdee is a lecturer at Lancaster University, working on intersections of art, design, and computer science. She is a practicing artist and designer and has an MFA in visual communication. Her publications explore areas of futuring, sketching, and drawing, alternative research outputs, and psychology.
[email protected]
View All Miriam Sturdee's Posts
Codesign and the art of creating a global research-practice network built for impact
Authors:
Gillian R. Hayes,
Candice Odgers,
Julie Kientz,
Jason Yip,
Kiley Sobel,
Morgan Ames,
Anamara Ritt-Olson
Posted: Thu, February 16, 2023 - 3:46:00
Launching a global network to enhance impact and collaboration among researchers, innovative companies, venture capitalists, and foundations is a daunting task. We sought to do just that in the area of technology, learning, and child development this year. Translating founder to foundation, researcher to practice, and maker to investor is hard enough, but then you have to convince them to work together. At a recent three-day workshop in Germany, the Jacobs Foundation brought together some of the world’s best researchers, heads of some of the most innovative edtech companies, and decision makers from those investing in this space either through charitable giving or impact investing. The capstone event was a marathon afternoon codesign session led by our team. What resulted was the design of a new global network and the seeds of many projects, some technologically oriented and some more focused on policy and practice. In this article, we describe the process, outcomes, and what we recommend to others looking to use codesign as a method for team science or research-to-practice (and practice-to-research) translation.
The Codesign Effort
Our five-hour codesign workshop engaged participants and promoted collaboration across locations, disciplines, and roles in the child development, learning, and technology ecosystem. We presented three primary goals to participants:
• Establish concrete new collaborations
• Identify problem areas and ideas that groups are excited to work on
• Build teams by showcasing the assets each participant brings to design
Participants were informed early that the output of the afternoon would be a five-slide “pitch deck” from each team describing the problem, population, potential solution or approach, proposed collaboration, and next steps and future directions.
We benefited from having spent a few days together talking about some of the major issues confronting the field. During that time, people had been placing Post-it notes on boards as interesting challenges and opportunities emerged in their discussions. Our team then grouped the sticky notes during a series of breaks, creating generalized themes using affinity diagramming. These notes and associated themes were prominently displayed in a break area, and participants were encouraged to add more, move them around, and group them. By the time the codesign sessions began, we already had a list of seven topics for teams to take on, as well as a list of 10 cross-cutting themes to be considered by every team (Figure 1). We placed rolling whiteboards with the topics around the room so that they were clearly visible as people entered. However, we asked participants not to break into groups yet.
Primary Topics of Interest to Pursue as Design Challenges
1. Teacher support
2. Mental health, care work, social and emotional development
3. Workforce retraining
4. Equity in high-income countries
5. Equity in low-infrastructure countries
6. Student resilience and agency
7. Partnerships
Common Concerns to Be Addressed by All Teams
1. Return on investment
2. Scale
3. Privacy and security
4. Assessment and regulation
5. Fragmentation
6. Infrastructure
7. Learning from failure
8. Community engagement
9. Dissemination and translation
10. Ethical design
Figure 1. Topics and themes that emerged during initial discussions.
Sitting at small tables in the center of the room, away from the topical boards, participants first listened to a short introduction to our goals for the afternoon and a lecture on design thinking that noted the codesign efforts would focus on the understand, observe, define, and ideate portions of this structure (Figure 2).
![](/images/uploads/blogs/Figure2_web_1.jpg)
Figure 2. The assembled team listening to an introductory lecture by Jason Yip
and Julie Kientz on design thinking.
They then formed groups of six to eight, according to the topics around the room, with the requirement that at least one researcher and one person from industry were in every group. Because they were unable to attend the entire session, foundation and VC representatives were free to assemble as they wanted, though they were encouraged to spread out. In the end, every group had at least one member of each of the four subgroups. Each subgroup had a colored sticker on their name tag denoting which group they were in; they had been tagged this way all week. Even before the design session, it had become common parlance among the groups to say “Blue dots over here” when a session for researchers was starting. The visual reminder and quick reference helped everyone remember to include differing backgrounds and perspectives and made it easy for facilitators to call on people when they noticed a subgroup being quiet during parts of the conversation. Three facilitators joined teams in need of researchers, two remained the primary facilitators, and the rest acted as floaters, moving among the teams as they progressed through the remaining activities.
The steps of codesign
Understand and observe: Individuals had 15 minutes to doodle and note potential challenges for specific research problems.
Define the problem: Part 1. In the next 15 minutes, teams translated their doodles to sticky notes and shared them within their groups to narrow down the challenges (Figure 3).
![](/images/uploads/blogs/Figure3_web_2.jpg)
Figure 3. Sticky notes showing the various scenarios developed in the ideation stage. Dot stickers signify votes; handwritten comments and arrows on the whiteboard show connections.
At this point, the teams had a group coffee break to mull over their ideas and chat across groups. Half of the participants—the VCs and funders—then departed for another meeting.
Define the problem: Part 2. Following the coffee break, the teams participated in their first design game, “Big Picture of the Project,” in which they used Story Cubes, a set of dice with random icons (https://www.storycubes.com/en/), to tell a story about their problem, challenge, and project area. This game was the first time in the session that participants began to question the process, which is a common concern when people who have not yet built creative confidence are asked to be creative [1]. We encouraged them to trust the process, and although it felt silly at first, the teams used the Story Cubes to stretch their thinking, with one team even incorporating a picture of a fountain that came up on a dice roll into the title of their final idea.
Over the next 30 minutes, teams built on the creativity they had developed during the first design game with a second design game, this one focused on sketching. On paper and whiteboard only—no digital devices were allowed—teams drew solutions to the problems they had described in their initial brainstorming session and during their story development. Teams were encouraged to modify, magnify, minimize, substitute, rearrange, and combine one another’s solutions based on Osborn’s Checklist [2].
Narrow down scope. Once ideas had been generated, teams used another 30 minutes to vote on ideas and solutions they wanted to pursue (Figure 2). At this point, the teams were encouraged to think about not only what was exciting but also what was doable, something they had been explicitly encouraged to avoid up to this point.
Compile pitch deck. In the final hour of independent work, teams made their five-slide pitch deck. They were provided a template in Google Slides but encouraged to change and expand on that template to match their needs.
Pitches. In the final hour of the session, the VCs and foundation staff returned from their meeting and rejoined their teams, with each team presenting for five minutes.
During this time, the audience participated in the final design game, using sticky notes in three colors that we had distributed around the room during the break:
• Roses (red): What do you love about this idea?
• Buds (orange): What new ideas do you have?
• Thorns (blue): What might be challenging about this idea?
Project Pitches
All projects were pitched collaboratively to an audience of the foundation event organizers, the VC and nonprofit funders, and the researchers and entrepreneurs who had completed the design sessions in their teams. The presented projects represented diversity in approach, both technological and educational, as well as in their topic areas. Concepts ranged from systems to share educational materials and ideas for educators, to internships based in virtual reality, to mental health support for diverse learners, particularly those in conflict zones. Each of these projects developed further in some way following the event. In some cases, the follow-ons were remote video sessions about a variety of topics, while others were nascent collaborations and grant proposals. All pitches were captured by a sketch artist (Figure 4).
![](/images/uploads/blogs/Hayes_Fig4_web.png)
Figure 4. Sketchnotes from three teams. From right: Teacherpedia focused on
knowledge sharing, Happiness@Heart focused on mental health, and Fountain
of Dreams focused on virtual internships via VR and gaming.
As just one spotlight from among these projects, a collaboration emerged inspired by the conference keynote presentation by Ramin Shahzamani, CEO of War Child Holland (https://www.warchildholland.org/), and a subsequent design session around promoting resilience in conflict zones. Part of a rapid pilot grant mechanism, this project focuses on providing adolescents living in high-conflict regions, particularly Ukraine, with the supports and skills required to reduce bedtime rumination and improve sleep duration and quality. It was recently funded by the John Templeton Foundation (https://www.templeton.org/). The project leverages technology-supported interventions to target two interacting and important risk processes for developing anxiety, depression, and PTSD: 1) excessive worry/rumination and 2) sleep difficulties. The project was conceived of, developed, and funded in just a few months starting with these codesign workshops. Even more compelling, the particular interdisciplinary and collaborative approach deployed—made possible through codesign—is well suited to scale to reach a large population of youth at high risk for bedtime rumination and for whom sleep deficits pose immediate and long-term threats to their health, the kind of impact that collaborative and community-based design approaches best enables.
Lessons from Codesign
Codesign can be deployed in a wide variety of ways. In this work, we used the concept and pragmatics of designing for a specific educational technology tool to better define our loose organization and further develop our networks. Some key lessons emerged from this approach.
This design session came at the end of an intense three-day workshop. Participants, most of whom were strangers three days earlier, had been a part of multiple shared discussions. In this way, the codesign sessions began more heavily grounded than some community-based and/or interdisciplinary design sessions might. By developing a set of topics from among those that had been discussed and describing them as the core of the team projects, the facilitators set the stage for some shared background. These topics, albeit loosely defined, appeared to be essential for establishing trust among participants. In particular, those who had never designed before were visibly uncomfortable at the beginning and during some of the more creativity- and innovation-focused activities. These same team members began to speak up more and relaxed their shoulders and leaned in when subject matter expertise was required or when more practical aspects of the projects were being worked through. In community-based research and codesign projects often there are team members with limited design experience. Our experiences reminded us that highlighting the non-design expertise of design teams is essential to success.
This design session, although tackling five different projects, took place in one large room. Each team had an area that they could call their own, but there were no barriers—visual or auditory—between the teams. This visibility and shared space created an audible buzz of discussion and excitement, with people standing up to move around whiteboards, get closer to one another, and sometimes to check on the other teams. In some cases, however, team members appeared to disengage, paying attention to what other teams were doing rather than focusing on their own work, or to compete, attending to their own team’s work through the lens of what other teams were doing. Nearly every participant was of high status within their own organizations: full professors at major universities, officers at well-known foundations, C-level representatives and founders of companies, established venture capitalists, and so on. Thus, some attempts to understand or even impress the other attendees might be expected. We observed limited competitive behavior, however, and found individuals and teams to work closely together, with intense engagement. The high status may also have reduced risk for some participants as well as made them more likely to speak up and engage with others. Similar efforts with more-junior members of the research network will provide additional insight into how status and experience may influence behaviors in these spaces.
Finally, the timing of the effort is something to explore further for others seeking to replicate these results. Had the workshop happened earlier in the meeting, there would have been more time to build on the ideas, and perhaps more firmly establish the emergent collaborations while collocated. However, it might also have been a less engaged and less comfortable group, not having the ideas, or the relationships, that had been developed over the preceding days.
Codesign for Collaboration
As any good human resources professional or leadership coach will tell you, new projects, joint efforts, and collaborative engagements are all experiences that can be designed. So why not use one of the core tools in the user experience design toolbox to do so? Indeed, in many ways what HCI researchers and practitioners know as codesign today stems from the early days of workplace design and the Scandinavian tradition of participatory design [3]. What we saw in this workshop that is so important was a shift in attitude, engagement, and enthusiasm when people were forced to work together hands-on. Yes, this hand-selected group of people had already spent three days together in a beautiful location and were mostly getting along. However, things tended, as they do in such meetings, to stay closer to the surface than true collaborative engagement requires. Over time, tighter relationships might have built more organically, particularly for those new colleagues who live and work somewhat near one another. When you are attempting to rapidly scale a truly global network of people who do not work in the same field, vary in their interaction norms, and have wildly different incentives for their work, however, a more aggressive intervention is necessary. For us, codesign was the aggressive intervention to jump-start the effort.
The shared experience of the workshop helped establish deeper connections and partnerships. Codesign requires active participation and learning that enabled many workshop attendees to synthesize what they had heard and learned throughout the week into their thinking in a rapid way. They described thinking about how to apply what they had learned in the designing of a new project as helpful in reflecting on what they learned and what they wanted to bring back to their organizations. This kind of sharing outside the workshop had not been an explicit goal but again speaks to the impact that these active collaborative design activities can have. Perhaps most importantly, we found that the codesign work did launch the collaborative efforts we had hoped would result. In many ways, being out of their comfort zone likely helped everyone develop trust in one another and establish a shared vocabulary—not only of words related to design that had been new to many but also the kinds of inside jokes that come from intense collaboration and impromptu presentations. Finally, this codesign activity gave participants a glimpse into team dynamics and relational communication abilities. Participants who followed up with others did so in part because they already knew how it might be to work with a new partner and were more willing to take a risk and get started on a project.
Scholars have examined how we can do “team science” for years (e.g., the Science of Team Science work from the National Academies of Sciences, Engineering, and Medicine; https://www.nationalacademies.org/our-work/the-science-of-team-science). Design, and in particular codesign, aligns well with many of the principles of team science. In our experience, codesign can jump-start novel scientific and research-practice collaborations. This type of rapid acceleration may be particularly important in a post-pandemic world in which we enhance remote collaborations and consider our scholarly and design work on a truly global scale.
Endnotes
1. Rauth, I., Köppen, E., Jobst, B., and Meinel, C. Design thinking: An educational model towards creative confidence. Proc. of the 1st International Conference on Design Creativity, 2010.
2. Osborn, A. Applied Imagination: Principles and Procedures of Creative Problem-Solving. Scribner, New York, 1957.
3. Bødker, S. and Pekkola, S. Introduction the debate section: A short review to the past and present of participatory design. Scandinavian Journal of Information Systems 22, 1 (2010), 4
Posted in:
on Thu, February 16, 2023 - 3:46:00
Gillian R. Hayes
Gillian R. Hayes is the Kleist Professor of Informatics, vice provost for graduate education, and dean of the Graduate Division at the University of California, Irvine. Her research interests are in human-computer interaction, ubiquitous computing, assistive and educational technologies, and health informatics.
[email protected]
View All Gillian R. Hayes's Posts
Candice Odgers
Candice Odgers is a professor in the Department of Psychological Science and associate dean for research in the School of Social Ecology at the University of California, Irvine. She is a quantitative and developmental psychologist with expertise in adolescent mental health, intensive longitudinal data analysis, and experience sampling methodologies. Her research focuses on how early, daily, and online experiences influence children’s health and development.
[email protected]
View All Candice Odgers's Posts
Julie Kientz
Julie A. Kientz is a professor in and chair of the Department of Human Centered Design and Engineering at the University of Washington. Her research focuses on understanding and reducing the user burdens of interactive technologies for health, education, and families through the design of future applications.
[email protected]
View All Julie Kientz's Posts
Jason Yip
Jason C. Yip is an associate professor of digital youth at the Information School and an adjunct assistant professor in the Department of Human Centered Design and Engineering at the University of Washington. He builds innovative technologies for new collaborations and examines how current technological trends already influence family collaborations around learning.
[email protected]
View All Jason Yip's Posts
Kiley Sobel
Kiley Sobel is a senior user experience researcher at the language-learning edtech company Duolingo, where she does research with kids and parents on Duolingo ABC, the company’s free English literacy learning app for kids.
[email protected]
View All Kiley Sobel's Posts
Morgan Ames
Morgan Ames is an assistant adjunct professor in the School of Information and interim associate director of research for the Center for Science, Technology, Medicine and Society at the University of California, Berkeley, where she teaches data science and administers the Designated Emphasis in Science and Technology Studies.
[email protected]
View All Morgan Ames's Posts
Anamara Ritt-Olson
Anamara Ritt-Olson is an associate professor in public health at the University of California, Irvine. Her research focuses on adolescent well-being. She creates, teaches, and evaluates health promoting programs to that end.
[email protected]
View All Anamara Ritt-Olson 's Posts
How to use social media (properly) for business
Authors:
Rae Yule Kim
Posted: Tue, February 07, 2023 - 11:03:00
Social media has been a game-changer for marketing. One successful hashtag challenge can generate millions of user-generated content (UGC) and billions of views. Here are four ways to trigger user-generated content on social media.
For two companies recently, viral TikTok campaigns spun marketing wins out of virtually nothing. Consider their cases: In the first, Too Faced, a cosmetic venture, inspired a viral challenge where the users were showing off before and after applying a lip-plumping product. Hashtagged #TFDamnGirl, the campaign generated one billion views and demand for the six-year-old product spiked, selling out the product line. The second campaign, Chipotle, launched the #ChipotleLidFlip challenge, where users closed a burrito bowl with an acrobatic flip of its lid. The campaign generated 110,000 videos with 310 million views and the company recorded historical sales the next day. How did these two companies do it?
As of 2023, 138 million people use TikTok in the U.S. and the company has 1 billion active users in the world. Approximately 80 percent of users are Gen Z or Millennials. Most TikTok users come from high-income households, where about 40 percent of them reported annual household incomes higher than $100,000. One characteristic of TikTok that distinguishes it from the other social media platforms is that the main reason to participate in the community is to do so via content creation. TikTok users do much more than scrolling. About 83 percent of users upload videos. Small or big, it is safe to assume that everyone on TikTok is an influencer aiming for virality. That’s why the most successful marketing cases on TikTok have been predominantly the brands that have exploited the TikTok community’s willingness to co-create, by plumping their lips, flipping a lid, or any other activity. In other words, successful TikTok marketers are aiming for virality not by creating cool content but rather by helping the users create content.
Still, many marketers I encounter are uncertain about how to use channels like TikTok to reach the younger demographics and promote their brands. Can campaigns like the ones above really be a strategy? The answer is yes. One successful hashtag challenge can trigger thousands of user-generated videos and generate millions of views. One piece of advice for marketers looking to exploit this fastest-growing new social media platform is: Let people do it.
Launch hashtag challenges
Hashtag challenges on TikTok can be the most cost-effective way to trigger user-generated content and improve brand exposure. On TikTok, millions of users spend substantial time and resources to create a 15-second video, hoping that the result will become viral. Brands’ hashtag challenges offer an easy win for these creators; by participating in the challenge users get access to cheap content that is part of a conversation. A simple challenge by Guess, #InMyDenim, where users simply showed themselves in denim was interesting enough to trigger more than 5,000 users to post videos and generated more than 50 million views. Another example is Marc Anthony’s #StrictlyCurls challenge. The name of the challenge is the name of the product line of their curl styling cream. This challenge—asking TikTok users with curly hair to post videos of their hair—generated more than eight million views and sales spiked 138 percent in the three months after launching the hashtag challenge. These cases show that TikTok users do not pass on an opportunity to create easy content.
Keep it simple, fun, and approachable
TikTok has earned a reputation as a casual platform where users feel comfortable expressing themselves and having fun. The #ChipotleLidFlip and #GuacDance challenges from Chipotle generated 110,000 and 250,000 videos respectively, with more than one billion combined views. But Chipotle is certainly not the first company to ask users to do something. To understand why these campaigns work, consider a failed one: Buzzfeed’s video encouraging users to weigh in on whether New Year’s Eve should be referred to using the outgoing year or the incoming one, for example, NYE 2019 or NYE 2020. Viewers were asked to post videos about their opinions. The campaign earned just 1,000 views and no follow-up videos. It’s not that TikTok users couldn’t care less to take the time to articulate their reasoning to explain why the New Year’s Eve should be NYE 2019 or NYE 2020, but instead that Chipotle made it easy to make fun videos. Other successful challenges, such as #EyesLipsFace by E.L.F. Cosmetics and #InMyDenim by Guess, invited users to post a video about their everyday makeup looks or jean outfits. Approachability is the key to generating a successful challenge on TikTok. If the challenge seems to be difficult to replicate or not fun, users are unlikely to take the challenge.
Create cause-related challenges
The Ice Bucket Challenge nudged many social media users to post videos of pouring buckets of ice on their heads to increase awareness of a disease called amyotrophic lateral sclerosis (ALS) and encourage donations. Initially, the popularity of hashtag challenges on TikTok started with the Ice Bucket Challenge in 2014, where #IceBucketChallenge has been viewed 59 million times. TikTok users care about causes. The #CreateForACause challenge launched by TikTok to invite users to post videos advocating for the causes they care about has generated 837 million impressions. One distinguishing characteristic of Millennials and Gen Z is that they are increasingly engaged with environmental issues. Nearly nine in ten Zoomers are concerned about the environment—and they believe that businesses should take action on environmental issues. Cause-related challenges about the environment, including the existing environment hashtags that are proven for customer reach, can be a powerful marketing campaign to increase brand awareness—and also to improve brand attitudes among Gen Z and Millennial consumers. Nonprofits have proven to be viral in this space too. The challenge #ForClimated, where the users posted videos about raising awareness on global warming, had 531.5 million views, and the #SaveourOceans challenge by Conservation International invited users to create videos that raise awareness on this topic, gaining 1.8 billion views in total.
Partner with influencers
Eighty-seven percent of Zoomers follow at least one influencer. Moreover, almost half of Gen Z has made purchases based on recommendations from social media influencers, as opposed to 26 percent of older generations. Zoomers tend to find social media influencers more authentic and trustworthy as brand spokespeople compared to celebrities. For brands that are new to TikTok, customer reach can be challenging. Partnering with influencers to launch hashtag challenges immediately gains access to millions of followers who might be keen to replicate their favorite influencers’ videos. Kool-Aid ran its first hashtag challenge in 2019, bringing over four TikTok influencers to invite people to post videos of themselves enjoying the holiday season. The Kool-Aid’s first hashtag challenge, #OhYEAHChristmas, resulted in more than 10,000 videos uploaded and was viewed 1.9 billion times. F’real sells milkshakes at convenience stores across the U.S. and Canada. F’real partnered with a micro-influencer with around 100,000 followers to launch a funny video that promotes their milkshakes. The video received 1.5 million likes and it was shared about 20,000 times. Also, people started to upload their own funny videos with F’real milkshakes. The user-generated content uploaded with hashtags #F’real and #F’realMilkshake has more than 250 million views combined. The company’s TikTok account now has 700,000 followers and 12 million likes.
Using social media for business
Social media has been a game-changer for marketing. One successful hashtag challenge can nudge millions of user-generated content (UGC) and billions of impressions with relatively low costs. But some marketers might still wonder how fun videos can lead to sales. To answer them, we can look to other social platforms. Follower acquisitions on social media platforms such as Facebook and Twitter often tend to improve sale performance [1]. Successful hashtags challenges have increased the number of followers on the brands’ TikTok accounts, all of whom might directly or indirectly promote the brand to their followers, via user-generated content or simply liking and sharing the brand’s content on TikTok.
There are concerns
TikTok has promising application prospects for marketing; however, there are concerns. Data protection is an issue that affects most social media platforms. The EU is carrying out new investigations on Facebook’s data processing operations over its potential violations of the General Data Protection Regulation (GDPR) policies. TikTok often ranks as one of the worst social media platforms for privacy policies. Because TikTok is owned by its Chinese parent company ByteDance, concerns are growing over the potential government access to user data by the Chinese Communist Party. TikTok claims that they process all U.S. user data in Oracle’s servers, however, their policies regarding data access and transfer are opaque. It is worthwhile for marketers to keep an eye on TikTok’s progress in data protection.
Endnotes
1. Kim, R.Y. The value of followers on social media. IEEE Engineering Management Review 48, 2 (2020), 173–183.
Posted in:
on Tue, February 07, 2023 - 11:03:00
Rae Yule Kim
Joyful sustainability: Now is the time
Authors:
Ben Shneiderman,
Catherine Plaisant,
Jonathan Lazar,
Niklas Elmqvist,
Jessica Vitak
Posted: Wed, February 01, 2023 - 10:23:00
Since SIGCHI and the University of Maryland’s Human-Computer Interaction Lab (HCIL) were founded at nearly the same time, we—a selection of HCIL faculty spanning the past four decades—have a special bond with the SIGCHI community. The emergence of SIGCHI and the annual CHI conference validated beliefs in the importance of rigorous studies of user interfaces and later the design of user experiences. CHI has always been about homecoming and celebration, where reconnecting with respected colleagues made for joyous encounters. The warm, enthusiastic, and supportive discussions continually energize and inspire us to do our best in preparing each year.
The CHI conference began with the early ideas of human factors—in fact, Ben chose the conference title, “Human Factors in Computing Systems,” in 1982—so it is time to consider an update. Early research was grounded in controlled experiments drawn from cognitive psychology and applied to user interfaces from computer science. As it matured, the CHI community expanded to cover design, accessibility, and community-supported collaborative work, using qualitative research methods from the social sciences. In recent years, this expansion has included sustainability, international development, critical race theory, policy and law, media studies, and democratic governance through participatory methods. Across all foci, there has always been a focus on improving the quality of life through technology.
The CHI conference and its organizers understand the strength of its amazing diversity: gender, race, ethnicity, national origins, age, abilities, and more. Yet there is still more to be done, and we argue that sustainability is an area where SIGCHI can take the lead. Sustainability is more than a foundational research topic—it is also an issue we can directly address in conference planning.
How can the bold, generous, and interactive spirit of CHI be preserved in fully virtual or hybrid conference environments? How can a virtual or hybrid environment build trusting collaborations, address meaningful societal problems, and produce breakthrough theories? Can hybrid be turned into a strength and not a stopgap? The SIGCHI community is uniquely qualified to lead this charge in figuring out how to create more enjoyable conferences that increase connection and collaboration while decreasing energy usage, and we challenge SIGCHI members and leadership to create more productive and enjoyable hybrid conference experiences while limiting energy usage.
We call this user experience joyful sustainability, reframing sustainability from an onerous burden to an opportunity for innovation in reducing the environmental harms society causes. We consider CHI 2005 in Portland, Oregon, as a great example of what we mean by such joyful sustainability: Rather than hire busses to transport 2000 people from downtown to the conference reception, a “CHI parade” was held from downtown to the reception, providing a unique memory for attendees, saving money for the conference, and lowering energy usage. Many attendees likely retain the joyful memory of the parade. Yet this unique solution also was a sustainable solution, as were the decisions made by the CHI 2022 chairs regarding food and beverage sustainability. We hope to see more examples like this going forward! The CHI community can also expand research and development of hybrid deliberation systems to support evidence-based discussions, resolve conflicts, and clarify commitments in workshops, SIGCHI meetings, and other collaborative projects.
While the HCI discipline of tomorrow will continue to expand its influence across a wide range of applications, aligning with the United Nations’ Sustainable Development Goals (SDGs) now will help SIGCHI take a leadership role in sustainable research and events in the coming decades. We ask: Could CHI use the SDGs to structure its agenda and call for papers? Supporting those goals, while highlighting technologies that raise awareness about the climate crisis and reshape human behaviors, could be pivotal in reducing environmental and societal threats in ways that ensure human well-being, biodiversity survival, and environmental preservation.
The technologies involved in joyful sustainability require innovative thinking to alter behaviors of individuals, communities, corporations, cities, national, and international organizations, and to do so in ways that enhance the user experience. SIGCHI’s activist stance is well-suited to becoming a role model for many disciplines. Making a deep commitment to research and development of sustainable conferences could trigger similar commitments by other disciplines, catalyzed by the collaboration technologies in which SIGCHI has long been a leader.
There’s a great deal that needs to be done, but let’s get started by setting short-term and long-term goals. In the short-term, we suggest changing the CHI Conference full name to The ACM Sustainable Conference on Human Factors in Computing Systems by 2025. Looking farther out, we want to shift the entire field of HCI toward more sustainable solutions, progressively aligning with the UN SDGs, within 10 years.
If not now, then when? If not us, then who?
Posted in:
on Wed, February 01, 2023 - 10:23:00
Ben Shneiderman
Catherine Plaisant
Catherine Plaisant is a senior research scientist at the Human-Computer Interaction Laboratory of the University of Maryland, and INRIA International Chair. She has written more than 200 technical publications (and many CHI videos), and coauthored with Ben Shneiderman the last three editions of Designing the User Interface. She is member of the CHI Academy since 2015 and received the SIGCHI Lifetime Service Award in 2020.
[email protected]
View All Catherine Plaisant's Posts
Jonathan Lazar
Jonathan Lazar is a professor in the College of Information Studies (iSchool) at the University of Maryland. He is the author/editor of 15 books, including
Research Methods in Human-Computer Interaction (with Feng and Hochheiser), director of the Trace Center, and a faculty member in the Human-Computer Interaction Lab (HCIL).
[email protected]
View All Jonathan Lazar's Posts
Niklas Elmqvist
Niklas Elmqvist is a full professor in the iSchool (College of Information Studies) at the University of Maryland, College Park. He received his Ph.D. in computer science in 2006 from Chalmers University in Gothenburg, Sweden. From 2016 to 2021, he served as the director of the Human-Computer Interaction Laboratory (HCIL) at University of Maryland. His research area is information visualization, human-computer interaction, and visual analytics.
[email protected]
View All Niklas Elmqvist's Posts
Jessica Vitak
Jessica Vitak is an associate professor in the College of Information Studies and director of the Human-Computer Interaction Lab (HCIL) at the University of Maryland. Her research evaluates the privacy and ethical implications of big data, the Internet of Things, and other “smart” technologies.
[email protected]
View All Jessica Vitak's Posts
Emerging telepresence technologies in hybrid learning environments
Authors:
Houda Elmimouni,
Pablo Pérez,
Andriana Boudouraki,
Fatma Guneri,
Verónica Ahumada-Newhart
Posted: Wed, January 25, 2023 - 3:49:00
During the pandemic, when some learners are expected to be remote, conventional videoconferencing tools like Zoom or Microsoft Teams are used by necessity, but they are not without their limitations (e.g., bandwidth and Internet accessibility requirements). These tools, initially designed for corporate use, can be extremely useful when everyone is remote. However, we have learned that videoconferencing tools are not ideal for hybrid classroom discussions, group work, design activities (such as sketching and diagramming), or creating and manipulating physical artifacts.
Emerging telepresence technologies have the potential to alleviate the problem of social interaction for learners who are unable to attend in person [1], attenuating the limitation of inaccessibility and creating a more inclusive classroom environment. Concretely, these technologies allow the interchange of nonverbal signals between participants that greatly influence the effectiveness of in-person communications [2].
While these technologies have positive affordances, they also present limitations [3]. For instance, many commercially available MRP units do not have hands to manipulate objects; many robotic arms cannot move independently around the classroom; and wearing VR headsets for long periods of time may disrupt the sensory system and/or cause eyestrain. Likewise, these technologies have open privacy challenges for students and teachers that must be analyzed and discussed.
This article highlights key findings from our workshop on emerging telepresence technologies in hybrid learning environments. The workshop was held at the 2022 ACM Conference on Human Factors in Computing Systems (CHI), to learn about HCI research in this space and promote future telepresence research to meet the needs of remote learners.
Emerging Telepresence Technologies in Hybrid Learning Environments: CHI 2022 Workshop
To discuss the impacts, affordances, and limitations of different emerging telepresence technologies that can be used in various learning contexts, a group of interdisciplinary researchers including Houda Elmimouni, John Paulin Hansen, Susan Herring, James Marcin, Marta Orduna, Pablo Pérez, Irene Rae, Janet Read, Jennifer Rode, Selma Sabanovic, and Verónica Ahumada organized and conducted the first workshop on Emerging Telepresence Technologies in Hybrid Learning Environments [4].
The workshop was held in a hybrid format with five participants attending remotely via Zoom and four participants in person. A large screen in the room displayed the Zoom participants for the in-person participants and a Meeting Owl 3 camera provided the remote participants views of the physical meeting room and the in-person participants. After brief introductions, the workshop started with a keynote talk from Laurel Riek, an internationally recognized roboticist. Following the keynote, papers on the following topics were presented:
- Paper A: Robots as teachers; robots as students
- Paper B: Investigation of an ungrounded haptic force device that could be used for remote movement instruction
- Paper C: Extended reality (XR): Exploring use of the Owl, a telepresence system focused on real-time immersive capture, delivery, and rendering of physical spaces and interactants
- Paper D: Assistance recruitment interactions between remote and local users of mobile robotic telepresence
The afternoon sessions were dedicated to discussion of the challenges and opportunities of telepresence technologies in the context of hybrid learning environments. Ideation and prototyping exercises followed the discussions to cocreate recommendations for future technology and research directions.
Evaluating Current Telepresence Practices in Learning Environments
During the first open session, some of the paper topics were discussed and debated. In particular, the participants explored and evaluated current telepresence practices in learning environments. We compared conventional videoconferencing tools (e.g., Zoom, Microsoft Teams) with emerging telepresence technologies such as MRP, stationary and mobile telepresence robots, robotic arms, holograms, Cartesian manipulators, and haptic tools. We discussed their affordances and use expectations as well as their actual use and performance in educational and learning environments. The discussions touched on the role of embodiment in helping the remote attendee feel immersed in the distance learning environment and how UX and usability issues can disrupt the attendance and the completion of tasks.
Creating: Design Sprint
The prototyping session of the workshop took the form of a design sprint. Each participant had 10 minutes to sketch eight different ideas that could be used to address any of the challenges previously discussed. At the end, the ideas were put in a shared document and each participant voted for their three favorite ideas.
![](/images/uploads/blogs/design-sprint_sm_1.jpg)
The result of the design sprint can be grouped in three areas:
- Improve the communication capabilities of telepresence robots. First, by giving them better embodiment capabilities such as a 360-degree representation of the remote user, robotic arms, and stair climbing. Second, by powering them with immersive communication technology (for example, omnidirectional video capture, XR telepresence, directional audio, and remote haptics). And finally, by adding augmentation capabilities to compensate for the inherent limitations of not being present (e.g., using machine learning to recognize people and automatically tag their names).
- Design the classroom for hybrid scenarios. For instance, using screens to show remote users in fixed positions within the classroom, or creating an interactive classroom map for spatial awareness of the local environment. This includes creating specific tools for the hybrid experience (for example, a physical microphone with a digital twin, used to “pass the floor” between students; connected tablets to share drawings and text; a mini-chat companion app to have a common side-talk channel for everyone; or even having everyone participate in the physical space via virtual means).
- Change the teaching strategy to actively include the remotes. For instance, paring them with local students, giving locals access to the remote user view, creating a “karma” system to gamify courtesy with remotes, and ensuring time-zone awareness across platforms.
Challenges and Opportunities
Telepresence technologies present many opportunities for novel applications and research. During our workshop, we explored the idea of moving beyond a single product to thinking about the whole classroom more broadly. The ideation exercise further prompted us to examine what aspects of interaction might matter—spatial audio, three-dimensionality, gaze, pointing, back-channeling, whispering—and how elements of meditated interaction (such as digital or robotic) could be blended to give remote and local users more options to express themselves and engage meaningfully with one another.
There were several challenges identified in our workshop. On the technical side, the current technology is still limited, and those limitations may have social repercussions in learning activities. When connectivity lagged in our workshop, the audio was unclear and the visual resolution low. The remote participants’ experience was disrupted, and they missed out on what was happening in the physical space. We also explored issues of equity. Workshop participants acknowledged that for telepresence to work well requires time, money, and effort; this includes an appropriate space with good WiFi, screens, and cameras, and detailed plans that include telepresent learners. Promoters of telepresence technologies must consider how these challenges will be met to ensure equitable access.
Future Directions for Research and Practice
Future directions for research in telepresence should include studies on the immersive capabilities of 360 cameras. We believe the ability to independently view the complete physical space will remarkably increase feelings of presence for the remote viewer. Just as people can view an entire room without leaving their seats, the 360-degree camera affords the remote user this same capability. Future studies may also explore how interactants feel about a 360 camera. For example, how would it feel if you were speaking with someone on a telepresence unit and they turned their “head” 180 degrees to talk to someone behind them? As humans, we do not have this capability. Would it be welcome? Would it disrupt acceptance of the remote user?
Additionally, we expect future studies may include:
- User privacy and data-sharing issues
- Social guidelines for all participants in the telepresence experience
- Accessible controls that are adaptable by end users to accommodate personal and physical preferences
- Ethics surrounding how to present equal inclusion of remote participants by respecting their audiovisual rights in the physical space
Conclusion
The number of telepresence researchers in the HCI community is increasing in the post-pandemic world as the global community has adapted to work-from-home, learn-from-home, and socialize-from-home activities and experiences. Our emerging telepresence workshop was an effort to provide a place for telepresence researchers to engage in transfers of knowledge, share works in progress, cocreate, and problem solve. Topics that emerged in our workshop included: views on robots as educators and learners, the potential use of haptic touch/force to communicate feedback on physical learning activities (e.g., dance), desktop telepresence technologies that relay immersive capture, delivery and rendering of physical spaces and interactants, and exploration of assistance recruitment interactions between remote and local users.
As our workshop took place during a Covid surge, some participants were unable to travel to attend. We were fortunate, however, to have researcher expertise in both HCI and telepresence technologies to conduct our workshop with a hybrid approach. As a hybrid learning approach was our desired area of study, we were excited to synthesize our topics of discussion with our workshop-participant experiences in learning via telepresence. We used several forms of digital media to interact within our workshop: Zoom, Meeting Owl 3, Google Docs, cell phones (cameras and text), and shared screens. Our workshop ideation and prototyping exercises found that HCI researchers valued the use of a 360-degree camera (remote user to view the physical room and people), a detailed agenda on progression of learning activities with identification of technologies throughout the learning exercise (e.g., use Zoom now, transition to Google Docs now, etc.), and directed plans for paired learning between remote users and interactants. Additionally, as many participants were logging in from different time zones, we suggest an optional time stamp for awareness of local times. Overall, we were encouraged by the hybrid learning experience and excited to participate in ways like other remote learners in hybrid spaces. Our discussions and topics raised several challenges, however, that need to be addressed for equitable and inclusive use of telepresence technologies. Future HCI research has great potential to address these challenges, create technologies, and promote social practices that facilitate equitable learning through sociotechnical inclusive environments.
Endnotes
1. Ahumada-Newhart, V. and Olson, J.S. Going to school on a robot: Robot and user interface design features that matter. ACM TOCHI 26, 4 (2019), 1–28.
2. Grondin, F., Lomanowska, A.M., and Jackson, P.L. Empathy in computer-mediated interactions: A conceptual framework for research and clinical practice. Clinical Psychology: Science and Practice 26, 4 (2019), e12298.
3. Rae, I. and Neustaedter, C. Robotic telepresence at scale. Proc. of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2017, 313–324.
4. Elmimouni, H. et al. Emerging telepresence technologies in hybrid learning environments. CHI Conference on Human Factors in Computing Systems Extended Abstracts. ACM, New York, 2022, 1–5.
Posted in:
on Wed, January 25, 2023 - 3:49:00
Houda Elmimouni
Houda Elmimouni is a Computing Innovation Fellow and a postdoc in the Department of Informatics at Indiana University Bloomington. She conducts empirical research at the intersection of human-computer interaction and computer-mediated communication. Her research interests include robotic telepresence, privacy, and diversity and values.
[email protected]
View All Houda Elmimouni's Posts
Pablo Pérez
Pablo Pérez is a lead scientist at Nokia Extended Reality Lab in Madrid, Spain. His research interests cover the whole area of real-time immersive communications and telepresence, from the compression and transmission problems to the user quality of experience.
[email protected]
View All Pablo Pérez's Posts
Andriana Boudouraki
Andriana Boudouraki is a Ph.D. student at the Mixed Reality Lab, University of Nottingham. Her thesis examines interactions via mobile robotic telepresence and the value of this technology in workplaces. Her research interests include computer-mediated communication, HRI, and how systems, technology, and spaces can support effective hybrid interactions.
[email protected]
View All Andriana Boudouraki's Posts
Fatma Guneri
Fatma Guneri is a research engineer at HEMiSF4iRE, Lille Catholic University. Her research topics concern alternative workplaces and remote work. Her studies about telework also cover students’ online learning experiences during the pandemic. Recently, she has been active in research groups interested in well-being in workplaces, particularly academic places.
[email protected]
View All Fatma Guneri's Posts
Verónica Ahumada-Newhart
Verónica Ahumada-Newhart is an assistant professor of health informatics and human-robot interaction in the School of Medicine, Department of Pediatrics, at the University of California, Davis. She is director of the Technology and Social Connectedness (TASC) Lab housed in UC Davis Health’s Center for Health and Technology.
[email protected]
View All Verónica Ahumada-Newhart's Posts
So, how can we measure UX?
Authors:
Maximilian Speicher
Posted: Tue, November 15, 2022 - 2:44:00
The precise, quantitative measurement of user experience (UX) based on one or more metrics is invaluable for design, research, and product teams in assessing the impact of UX designs and identifying opportunities. Yet these teams often employ supposed UX metrics like conversion rate (CR) and average order value (AOV), which can’t provide that measurement [1]. In fact, I believe this can be extended to an even more general statement: In themselves, none of the metrics that are usually readily and easily available from Web analytics data can reliably measure UX. I understand that this is frustrating news to many, since resources are always limited, attention spans short, and Web analytics so very, very convenient.
Whenever I discuss this, I encounter objections like, “But we have to do something,” or “It’s easy to just state what one shouldn’t do, but that doesn’t help much.” And while it’s a perfectly fine start to know what not to do (cf. Nicholas Taleb’s The Black Swan), in the case of UX, we must not despair, because there are ways to reliably measure it, albeit ones that are not as simple as pulling some number out of Google Analytics (as nice as that would be).
A simple outcome of measuring UX could be, “The last release improved checkout UX from 75/100 to 80/100,” but there could be more-nuanced measurements for different aspects of UX (e.g., usability, aesthetics, joy of use) and user groups. Before diving deeper into how we can do this, let’s first get familiar with three concepts:
- Latent variables (like UX) “are variables that are not directly observed but are rather inferred through a mathematical model from other variables that are observed” [2]. Take, for example, the Big Five personality assessment [2]. You can’t just ask someone, “What’s your personality?” and expect an objective answer. Rather, you have to ask a set of specifically designed questions and then infer personality from those.
- A research instrument is a set of such specifically designed questions, often in the form of a questionnaire. Through an instrument, we can collect the observable variables that help us infer the latent variable we’re after
- We’re dealing with composite indicators when we combine individual variables from an instrument into a single metric.
Additionally, it’s necessary to understand that the user’s experience is not a property of a digital product or user interface. An app doesn’t have a UX. Rather, the experience “happens” in the individual user’s head as a reaction to their interaction with a digital product [3]. Hence, the only way to directly observe UX would be to look into the user’s head using an EEG or similar—and even if you had the possibility, that would be pretty complicated. In any other case, the next best alternative is asking them about it. In contrast, Web analytics data (and metrics like CR and AOV [1]), are relatively “far away” from what happens in the user’s head. They’re collected directly on a website or in an app, aggregated over many users—even though there is no such thing as an “average user”—and influenced by myriad other non-UX factors [1].
So, do we always have to ask users to fill out a questionnaire if we want to reliably measure UX and can’t make use of analytics data at all? The answer to both is “not necessarily,” but first things first.
Plenty of work has already been put into instruments for measuring UX, and this remains an ongoing topic in the human-computer interaction research community (which strongly suggests this is not a trivial matter). Three examples of scientifically well-founded instruments are AttrakDiff (http://attrakdiff.de/index-en.html), UEQ (https://www.ueq-online.org), and meCUE (http://mecue.de/english/home.html), with the first two currently better-known and more widespread. The first research article on AttrakDiff was published as early as 2003. All three also define composite indicators for different aspects of UX. For instance, AttrakDiff provides one composite indicator for hedonic and pragmatic quality each.
So, there exist proper instruments for reliably measuring UX, and you can always have users answer one of those in a controlled study or by means of a live intercept survey. But how might analytics data come into the picture? As so often, the answer lies in statistics. To use analytics metrics for approximating UX, you first have to determine how much individual metrics correlate with and are predictors for actual UX; otherwise you’re simply guessing. For instance, if you have a sufficient number of sessions where you record user behavior and collect answers to one of the questionnaires, you might be able to use them as training data for machine-learning models that predict either individual items or composite indicators—or both—for different user segments. Sounds easy enough, right? Collect some analytics data, collect some answers to a questionnaire, put it all in a Jupyter Notebook, and you’re good to go?
Incidentally, that’s pretty much what I investigated in my Ph.D. thesis, only I did it for usability (using an instrument named Inuit [4]) and interaction data like mouse-cursor movements and scrolling behavior. But the basic principle was the same. Based on this, I have good news and bad news for you. The good news: It worked. I was able to train models that could predict the usability of a Web interface reasonably well. The bad news: It was not all that easy.
First, I didn’t put “sufficient number” above in italics for no reason. Turns out it’s really difficult to collect enough training data when people must fill out questionnaires—and Inuit has only seven items. UEQ, for instance, has 26! Second, interaction data‒based models seem to be very sensitive to interface changes. My results suggested that, at best, one could apply models trained on one website within a narrow cluster of very similarly structured websites, but not beyond, while also having different models per user type. Colleagues and I have, however, already started looking into these problems [5].
In conclusion, because UX is something so elusive, if you want to properly measure it, there’s no way around using a scientifically well-founded instrument. Although often applied as supposed “UX measures” in industry, off-the-shelf analytics data are in themselves not suited for this. Simply speaking, this is because 1) there’s a significant distance between the interface where they are collected and the user’s head, where UX “happens,” and 2) they are influenced by too many non-UX factors to lend themselves to meaningful manual analysis. Yet, under certain conditions, it is possible to predict UX from analytics data, if we combine them with answers to a proper UX instrument and use all of that to train, for example, regression or machine-learning models. In the latter case, you can use methods like SHAP values to find out how each analytics metric affects a model’s UX prediction. And if there are just a few strong predictors, it might even be possible to take a step back and work with simple equal-weight models, as Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein describe in their book Noise.
What could be an ultimate solution to the practical problem of measuring UX? Probably some kind of universal model where anyone can put in their analytics data, user segments, and parameters of their digital product to get instant UX predictions. Does such a model exist? Absolutely not. Might it exist in the future? I can’t say for sure, but I’ll keep looking.
Endnotes
1. Speicher, M. Conversion rate & average order value are not UX metrics. UX Collective (Jan. 2022); https://medium.com/user-experi...
2. Wikipedia contributors. Latent variable. Wikipedia; https://en.wikipedia.org/w/ind...
3. When a large number of users agree that they’ve had pleasant experiences using an interface, researchers usually colloquially state that the interface in question “has a good UX.”
4. Speicher, M., Both, A., and Gaedke, M. Inuit: The interface usability instrument. In Design, User Experience, and Usability: Design Discourse. Springer, Cham, 2015, 256–268; https://link.springer.com/chap...
5. Bakaev, M., Speicher, M., Heil, S., and Gaedke, M. I don’t have that much data! Reusing user behavior models for websites from different domains. In International Conference on Web Engineering. Springer, Cham, 2020, 146‒162; https://link.springer.com/chap...
Posted in:
on Tue, November 15, 2022 - 2:44:00
Maximilian Speicher
Maximilian Speicher is a computer scientist, designer, researcher, and ringtennis player. Currently, he is director of product design at BestSecret and cofounder of UX consulting firm Jagow Speicher. His research interests lie primarily with novel ways to do digital design, usability evaluation, augmented and virtual reality, and sustainable design.
[email protected]
View All Maximilian Speicher's Posts
Using technology to improve communication in panels
Authors:
Jonathan Grudin
Posted: Fri, September 30, 2022 - 11:34:00
Panels can be fun to develop—they can also be much more effectively executed. This spring I was on two large panels that were the best organized of any I’ve participated in. Technology was used in advance to reduce psychological uncertainties, which increased interaction and kept us focused on our message. Expectations were exceeded. Both of the panels were in person, but the method is even more promising for hybrid panels.
The first panel was organized by SIGCHI Adjunct Chair for Partnerships Susan Dray and VP of Finance Andrew Kun to discuss CHI’s 40th anniversary. Structure that one might assume would squeeze out liveliness instead promoted it. Impressed by its success, I reprised the method in another 90-minute panel at my Reed College class reunion. These are only two data points, but the underlying psychology is compelling, especially for large panels and future hybrid panels.
The 40th anniversary panel was an exceptionally diverse group of eight, plus Andrew as moderator. Our backgrounds, interests, and priorities differed substantially. Forty years earlier, one of us wasn’t yet in school and only one was involved with CHI. Yet we wanted to avoid eight 10-minute talks—we wanted to engage with one another.
Andrew’s solution was to address psychological barriers to effective time management that I had never noticed. Panelists like to talk. The trick: Make it possible for panelists to avoid saying more than they really want to.
A large panel should minimize moderator control onstage to give panelists time to get their points across, but if panelists are undirected, meandering and a lack of coherence are inevitable. Andrew directed the panel, but he did so in advance, which greatly reduced the cognitive effort required of panelists at the event.
Before the event
Weeks in advance, Andrew shared a Google Doc that asked panelists to draft text for four sections:
- A bio
- A two-minute opening statement or “provocation”
- One or two questions from each of us directed to another panelist (encouraged to ensure that each panelist was asked at least one question)
- A one-minute concluding summary.
Two virtual meeting deadlines ensured that we completed drafts in a timely fashion. Everything could be revised up to the event, but Andrew could also sequence opening statements early to create a coherent topic flow. Seeing one another’s drafts and the presentation sequence enabled us to keep statements to the same length, reduce redundancy, share terminology, and shape questions and summaries.
Seeing a question that would come our way in advance, we could organize a response, but responses were not shared. They were fresh to other panelists at the event, and anyone could contribute thoughts if the initial respondent didn’t cover them.
At the event
We had crafted crisp bios that Andrew read to introduce us. Our opening statements were practiced, fluent, and within the allotted two minutes.
The key innovation was the handling of the Q&A. Andrew had carefully thought through the questions and prepared a sequence of invitations. For example, “Daria, do you have a question for another panelist?”
The crucial point: We knew that Andrew was always ready to solicit the next question. Think of your experiences on a panel. Panelists are often unsure about the best way to continue and keep the discussion going. This leads to fuzziness or discontinuities. Three scenarios:
- I ask a question and receive a satisfactory response. If I have nothing to add, I’m in a bind. Not knowing whether someone else will jump in if I stay silent, I often politely respond or ask an unnecessary follow-up question to keep the conversation going. In our event, I could pause and glance at Andrew. If no one immediately spoke, he moved on: “Tamara, do you have a question for another panelist?”
- There is a pause in the discussion. Has the current topic been covered adequately? I haven’t spoken yet, Should I break the silence, even if what I will add is mostly to agree or digress? At our event, there was no pressure to do this. Andrew was ready to break the silence and move on to the next cogent question.
- A topic is exhausted. It’s clearly time to move on. I have a very different topic, but someone else may have a more relevant continuation. Should I jump in? In our panel, we could leave graceful transitions to Andrew, knowing that our question would get its turn. Andrew was following the discussion and knew the range of questions remaining.
I felt that a cognitive load had been lifted—it was great! The navigation of conversational conventions adds many small efforts that compete with focusing on what is said and the overall topic. There were no space-filler comments as we engaged with one another.
I’d initially thought that although structure might be necessary for an eight-person panel, it would sound scripted and diminish spontaneity. That wasn’t the case. We added follow-up questions and comments after one of us responded. The conversation was brisk and focused on things we cared about. Little cognitive work was required for conversation management.
A replication
For my 50th college reunion six weeks later in Portland, Oregon, five people who spent careers in tech formed a panel to discuss how tech had evolved, our roles, and what we recommend younger people think about. I introduced Andrew’s structure. With five panelists, we doubled time allowances and mixed our prepared questions with less predictable audience participation. It went well. People go to college reunions more to hang out and socialize than to attend lectures and panels, but we drew a large crowd who stayed throughout.
Natural for hybrid panels
Efficient conversation management is critical for large panels. This approach also offers benefits for virtual or hybrid panels. Hybrid planning is by necessity online—which means there’s no opportunity to get together for breakfast the day before—so coauthoring a structured document and holding a couple of preliminary meetings is a natural fit. More significantly, navigating conversation transitions is more challenging when panelists have less awareness of one another’s body language. Hybrid panels may gravitate toward larger sizes, and cultural diversity could require bridging conversation styles.
This structure requires more preparation, but it was distributed over time, getting a better sense of other panelists’ contributions reduced some of the effort, and the collaboration was enjoyable. It was worth it.
Posted in:
on Fri, September 30, 2022 - 11:34:00
Jonathan Grudin
Jonathan Grudin has been active in CHI and CSCW since each was founded. He has written about the history of HCI and challenges inherent in the field’s trajectory, the focus of a course given at CHI 2022. He is a member of the CHI Academy and an ACM Fellow.
[email protected]
View All Jonathan Grudin's Posts
An American Zen Buddhist’s reflections on HCI research and design for faith-based communities
Authors:
Cori Faklaris
Posted: Tue, August 16, 2022 - 4:29:00
Computer science seems in opposition to Zen Buddhism, a spiritual practice best described (if it must be) as “without reliance on words or letters, directly pointing to the heart of humanity” [1]. Yet with so much of today’s human experience bound up in computing, some words, and pixels and bits, to address their overlap seem necessary. My aim in sharing the following reflections is to set forth the historical and modern faith context for a secular wellness practice that we design for— meditation—and to model the statements of positionality and reflexivity that I feel are essential for research in such personal and cultural domains.
Zen Buddhism
Zen’s origins were first documented during China’s Tang dynasty in the 7th century CE. At that time, Buddhism had already spread from Nepal, the birthplace of historical founder Siddhartha Gautama, or “the Buddha,” throughout neighboring countries in Asia for more than 1,000 years. The Indian monk Bodhidharma is credited with introducing China to the dhyana practice of stillness and contemplation. Dhyana (a Sanskrit word) predates the Buddha and is commonly described as his vehicle for achieving enlightenment, or the transcendence of his limited human existence. The renewed focus on dhyana was a reaction to the older branch of Buddhism known as Theravada, the “way of the fathers” [1]. The chief text of the newer Mahayana branch of Buddhism is the Heart Sutra, the English translation of which fits on one page. Zen takes this minimalist approach even further, proclaiming the superiority of empirical knowledge gained through dhyana —renamed ch’an in Middle Chinese—over scriptural learning or formal religious observance. A famous poem of the Tang dynasty sets forth this formulation for Zen [1]:
A special transmission outside the scriptures
Without reliance on words or letters
Directly pointing to the heart of humanity
Seeing into one's own nature.
Today, the largest Zen communities remain in China and Japan (the origin of the word zen), but the practice of Zen has spread throughout the world. Like other Buddhists, who number in total 488 million worldwide [2], they venerate the “Triple Treasure” of small-b buddha (inherent enlightenment-nature), dharma (the teachings and practices), and sangha (their faith communities). Zen practitioners also particularly value upaya, or “skillful means” (the ability of an enlightened being to tailor a teaching to a particular audience or student for maximum effectiveness) and mindfulness (in Zen, the continuous, clear awareness of the totality of the present moment). Through seated meditation, alternated with practices such as chanting, bowing, and contemplative walking, a Zen Buddhist aspires to a state of mindfulness that will facilitate their own perception of buddha-nature and help them express this enlightenment in daily life, especially for the benefit of others. To check the validity of their meditation experiences, Zen practitioners are urged to consult with a teacher in an established lineage who is certified to guide others in enlightenment. Among a teacher’s “skillful means” are stories or riddles known as koans (Japanese), gong-ans (Chinese), or kung-ans (Korean). Such consultations will help Zen practitioners achieve a “before-thinking,” other-centered orientation and avoid self-centered fallacies—for example, “wanting enlightenment is a big mistake” [1].
Personal experiences and observations
My own experiences with Zen are Western. I began in my teenage years, when I bought a secondhand copy of D.T. Suzuki’s Essays in Zen Buddhism and was intrigued by his discussions of satori, the Japanese word for enlightenment. I had already liked what I had heard about Buddhism during a unit on world religions at my (Catholic) grade school. However, it wasn’t until I moved to the U.S. state of Indiana that I was able to connect with an in-person group, practicing in the Kwan Um School of Zen (KUSZ) in the lineage of Korean Zen Master Seung Sahn. I began sitting with the Indianapolis Zen Center sangha once or twice a month as my schedule permitted: 30 minutes of seated meditation bookended by a 20-minute prelude of chanting and a 10-minute epilogue of a reading and announcements. I progressed to sitting weekend retreats and to taking precepts (like Christian baptism, this signifies formally joining the faith). Eventually, I studied for and became a KUSZ dharma teacher—qualified to explain subjects such as meditation forms and the history of Zen, but not to guide people to enlightenment. In KUSZ, such teachers are called Ji Do Poep Sa Nim (JDPSN, for "dharma master") or Soen Sa Nim (Zen master). I have studied with both types of “enlightenment” teachers at the Indianapolis Zen Center and with a group in Pittsburgh, PA, while helping as a dharma teacher.
As a Zen teacher, I do not take a binary view of computing as good/not good or useful/not useful. The “middle way” is to acknowledge that it is a dharma aid in some contexts and a distraction in others. Below are some examples.
Computing as obstacle to Zen practice
My advice to beginners is to turn off their smartphones completely. This is because a buzz or ding is liable to take meditators out of the moment, and beginners often will struggle to refocus. For my in-person group, I model another best practice by taking out my phone or smartwatch, silencing them, and turning them face-down on my meditation cushion, so that I cannot see the flash of a notification. I prefer to use such manual safeguards for attention rather than the “Do Not Disturb” settings, because enacting the exercise of putting away our digital helpers is an important signal to our bodies and minds that what we are doing is important and different from the everyday flow of our distracted lives. In the same vein, I recommend use of a battery-operated analog clock over a smart device for timing seated meditation, because it will not tempt you into checking messages.
In the world of Covid-19, much of our group Zen practice has joined others online. Now, it is no longer possible to physically remove ourselves from our Internet-connected devices. I am grateful to be able to see and hear my fellow practitioners even at a distance, but I miss having the break from my busy digital life and from the allure of its distractions. The “Do Not Disturb” settings help, to a point. Sitting in front of my MacBook, however, I catch myself touching my mouse and calling up screens whenever I experience a fleeting thought about, say, the status of a project. Meditating from home also means interruptions from family members, pets, or Internet outages. I confess that I do not have enough “dharma energy” to avoid breaking my stillness in response to my cat waving her tail in my face!
Going forward, this type of Zen Buddhism will benefit from computing research and design to solve similar problems of distraction and focus as those faced by those working from home or who are “digital nomads,” connecting to their customers or clients via the Internet away from an office. I would love to flip a switch inside my home environment and be free from all ability to access Netflix or Slack while I hunker down on either a research paper or a kung-an. Even better if the “switch” is a timer, so that I do not forget to turn off my “Do Not Disturb,” or a learned routine of my home network, so that it is context-aware and picks up on the signals that I am ready to concentrate. I use Siri now to set a meditation timer by voice, although the screen and keyboard is still nearby, and it would be better for my ability to stay in concentration if “she” could turn off everything at the same time and then turn it back on again after the timer ends.
However, like the meditators in Markum and Toyama [3], I am wary of letting technology intrude too far or replace in-person experiences. I am doubtful that it will no longer be necessary to visit a Zen center or monastery for the sustained concentration required for intensive practice. My Pittsburgh group has returned to offering a weekly in-person (and masked) practice so that people can get a break from remote meetings and reap the benefits of in-person group meditation. I look forward to the day when we can begin traveling to other temples and learning in situ about others’ spiritual practices, perhaps with the assistance of interactive displays or augmented reality overlays.
Computing as support to Zen practice
Is reading an obstacle? After all, “words and letters” are considered a hindrance in Zen tradition. However, reading is often the first step undertaken by someone who wants to try meditation and/or to learn more about Zen Buddhism. As Bell has noted [4], the internet has been enormously helpful for spreading Zen knowledge and for connecting seekers with faith communities. My sanghas have made use of the same computing affordances as other interest groups: websites, online groups, platforms for event discovery, and secure no-contact payments.
I will always prefer in-person Zen practice. But, like the online group members in Katie Derthick’s work [5], I now can join a Zen meditation session or a retreat from anywhere in the world. I can take part in a distant reading group or a book club (we have those!). I can receive an interview via video and audio from a variety of Zen teachers. For those who don’t want that group experience, a variety of apps can guide them in contemplating peace or following their breaths. Headspace even dims the screen so that you can use it to wind down and prepare for sleep.
Going forward, the main item on my wish list is better audio support for remote Zen practice. We have experienced glitches in teacher interviews where one person trips over the other person’s statements, adding to the problem of not being present to pick up on nonverbal cues to turn-taking such as angling back or tilting forward. Worse, audio problems have almost killed our group chanting. This is unfortunate because, in my tradition’s Zen practice, chanting is essential for aligning participation and building an energy within the group that supports its focus. Participants cannot stay in sync—the farther from the source, the more obvious the transmission delays. Our Zoom apps also struggle to figure out which voices to prioritize, instead of blending every audio source into a unified output. For now, the workaround is for everyone to mute and only listen to the leader’s chanting. We need apps like JamKazam or Jamulus, which were designed for musicians to play together online and at a distance. Such an app will need to integrate with our existing remote meetings and be usable by anyone.
Suggested best practices for computing researchers
Religion is as sensitive a topic as it is central to the human experience. From my N of 1, I suggest that computing researchers will do themselves good to consider their positionality and biography with regard to this subject, before embarking on faith-minded research [6,7]. Clarification of our personal experiences—how we were raised and how we have directed our adult lives with regard to religion—will make explicit our social, cultural, and historical position with regard to the faith domain. Reflexivity requires time, but reading, discussing, and thinking will help us to identify what assumptions we bring to the project. Once articulated, our preexisting assumptions will be less likely to warp our research or to stymie our openness to new ideas. (In my case, I sat and thought about whether I have a bias toward adding technology to any faith domain, regardless of whether it is truly needed. I also challenged myself as to whether I assume that adding technology will lead to only negative downstream effects for a religious community.)
For the conduct of this research, I suggest three ethical pledges that will reduce the potential for exploiting participants: adherent-centeredness, getting close-up, and considering relationship ethics. Researchers should prioritize the faith population’s needs, preferences, and values, and incorporate them to the extent possible: “Nothing about us, without us.” Careful, respectful qualitative work such as Wyche et al. [8] follow Genevieve Bell’s prescription to use techniques informed by anthropology, focusing on the particulars of place, location, and critical reflexivity [9]. Researchers should make use of practices such as participant observation that foster empathy and consider layering different participants’ accounts, rather than aggregating them into a majority narrative [6]. And they should recognize that such research will involve leveraging existing relationships and fostering new ones. Discuss issues of privacy and confidentiality upfront, for example, that it may not be possible to de-identify anyone [6]. Share work and ask for responses and comments. In publications, alter specific personal or topic details to protect their privacy, security, and safety.
These suggestions may sound like standard operating procedure for some qualitative researchers in human-centered computing. But many of us are trained in a positivist orientation, in which reason and logic are prioritized. We will benefit from having these or similar principles explicitly articulated for our consideration and commitment, just as a Zen master benefits from reciting the temple rules about not borrowing people’s shoes and coats. We all need help staying mindful.
Endnotes
1. Sahn, S. The Compass of Zen. Shambhala Publications, 1997.
2. Buddhists. Pew Research Center’s Religion & Public Life Project. Dec. 18, 2012; https://www.pewforum.org/2012/12/18/global-religious-landscape-buddhist/
3. Markum, R.B. and Toyama, K. Digital technology, meditative and contemplative practices, and transcendent experiences. Proc. of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–14; https://doi.org/10.1145/3313831.3376356
4. Bell, G. Auspicious computing? IEEE Internet Comput. 8, 2 (Mar. 2004), 83–85; https://doi.org/10.1109/MIC.2004.1273490
5. Derthick, K. Understanding meditation and technology use. CHI ’14 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, 2014, 2275–2280; https://doi.org/10.1145/2559206.2581368
6. Darwin Holmes, A.G. Researcher positionality—A consideration of its influence and place in qualitative research—A new researcher guide. Shanlax Int. J. Educ. 8, 4 (Sep. 2020), 1–10; https://eric.ed.gov/?id=EJ1268044
7. England, K.V.L. Getting personal: Reflexivity, positionality, and feminist research. The Professional Geographer 46, 1 (1994); https://www.tandfonline.com/doi/abs/10.1111/j.0033-0124.1994.00080.x
8. Wyche, S.P., Hayes, G.R., Harvel, L.D., and Grinter, R.E. 2006. Technology in spiritual formation: An exploratory study of computer mediated religious communications. Proc. of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work. ACM, New York, 2006, 199–208; https://doi.org/10.1145/1180875.1180908
9. Bell, G. No more SMS from Jesus: Ubicomp, religion and techno-spiritual practices. In UbiComp 2006: Ubiquitous Computing (Lecture Notes in Computer Science). Springer, Berlin, Heidelberg, 141–158; https://doi.org/10.1007/11853565_9
Posted in:
on Tue, August 16, 2022 - 4:29:00
Cori Faklaris
Cori Faklaris is a doctoral candidate in human-computer interaction at Carnegie Mellon University in Pittsburgh, PA. She researches the social-psychological factors of cybersecurity and other protective behaviors. She also is a dharma teacher in the Kwan Um School of Zen.
[email protected]
View All Cori Faklaris's Posts
Designing for religiosity: Extracting technology design principles from religious teachings
Authors:
Derek L. Hansen,
Amanda L. Hughes,
Xinru Page
Posted: Thu, August 04, 2022 - 3:16:00
Religious beliefs have a profound influence on billions of people across the globe, affecting nearly every aspect of their lives, including the use of technology. While there is a continuous rise in atheism, the majority of people in many countries still believe in a deity, self-identify with a religion, and regularly participate in religious practices such as prayer. For example, in the U.S., 69 percent self-identify with a religion, 66 percent consider religion very important (41 percent) or somewhat important (25 percent) to their life, and 67 percent pray daily (45 percent) or weekly/monthly (22 percent) [1]. For many people, religious beliefs and teachings frame every aspect of their life, influencing behaviors related to diet, social relationships, dress and grooming, sexual practices, mourning for the dead, raising children, and financial decisions, among others.
It is no surprise, then, that religions have much to say about the use of technologies, such as the Internet, social media, and mobile phones. Yet design guidance is mostly absent on how to design technology in ways that support various religious values and beliefs. While philosophers, sociologists, and humanities scholars have studied the intersection of technology and religion, relatively few studies have examined religion and technology from a design and HCI perspective [2]. This is unfortunate, since religious traditions often seek to transform the lives of their adherents and the world for the better. Such inclinations can be highly compatible with core HCI values, which often focus on the betterment of the world through the novel use of technology.
Few HCI researchers make time to look through the lens of religious teachings at the technologies that surround us [2]. Thus, we don’t fully appreciate basic questions related to religious teachings and technology. How central a role does technology play in religious teachings? What stances do religions take on the appropriate or inappropriate use of technologies? How do religions frame the discussion around technology, given that many of their teachings are based on ancient texts written by those with vastly different technologies? What role does religious doctrine play in informing religious practice around technologies? How do these answers differ for different religious traditions?
Religious values are integral to many people’s lives and should be considered a key value for the HCI community to integrate into technology design. All physical and digital artifacts convey and enforce certain values, whether they are purposefully designed to do so or not. Thus, value clashes can occur when the affordances of the technology are not aligned with the values of the system’s users. In fact, a system that is designed from the perspective of one group may impose the values of that group on other target users of the system. For example, the idea that a mobile phone is attached to one individual and thus a unique phone number can be required for each person’s account setup for an online service may be a fair assumption in many contexts. However, it causes issues for countries, settings, or religious contexts where a mobile phone is a shared object between a married couple, family, or even extended community, creating an account setup roadblock for anyone who does not have their own phone number. Technology infrastructures meant to support people with many different values should account for this diversity of values and validate assumptions about its users.
A very limited number of HCI studies have investigated how technology practices complement or hinder religious practices via empirical studies (we expand on a number of these in the next section). Existing HCI research has also focused on understanding how religious practices can inform design in nonreligious contexts. For example, several studies apply strategies employed by religious organizations to enhance commitment, build community, and/or motivate behavior change in nonreligious organizations. Ames et al. explored how religious ideological practices can serve as a useful lens in understanding how nonreligious engineering and design organizations affirm membership and a shared vision [3]. Similarly, Amy Jo Kim discusses the use of rituals (a concept inspired by religions) in building commitment to online communities [4].
While this prior work gives us initial insights into the interplay of religiosity and technology in practice, there is an element that is missing and yet key to understanding values we might aspire to incorporate into our technologies. While studying how people use technologies in practice gives us a descriptive understanding, there is also a prescriptive element of religion that is vital to understand. In many religions, there are a set of values that believers aspire to, and they hope to engage in practices that reflect those core values. Thus, we need to not only understand how technologies are used in practice, but also the guiding principles that a given population might be influenced by or aspire to. This also helps us to identify new opportunities for supporting religiosity.
How religious doctrine and teachings can inform design
Many religions provide specific prescriptive guidance to their adherents on the use of social media, the Internet, or other modern-day technologies that are of interest to the HCI community. Such guidance is often based upon doctrines, or the foundational beliefs, principles, and teachings of a religion. These come in the form of ancient scriptural texts, commentaries, sermons, pronouncements by religious leaders, official publications of religious organizations (e.g., magazines), and a variety of other resources. While it is tempting to consider only the practical, prescriptive advice about technology use given to members of a religion, it is essential to also understand the religious doctrines and teachings that underly such advice. Designers can benefit in several ways from learning the core doctrinal beliefs of a religion in order to better design for its members.
First, doctrines can inspire the use of religious metaphors that tap into believers’ deepest spiritual desires and insights. For example, Pope Francis used the metaphor from St. Paul’s teachings in the New Testament that views Christians as “members of the one body whose head is Christ” when discussing the importance of using social media to build up others and not tear them down [5]. This metaphor stresses the value of a community (i.e., a body) having different body parts (e.g., eyes, ears, legs, hands), each of which serves a different purpose for the benefit of the whole. By invoking this metaphor, the Pope encourages Catholics to see the Church as “a network woven together by Eucharistic communion [a central Catholic ritual, where unity is based not on ‘likes,’ but on the truth, on the ‘Amen,’ by which each one clings to the Body of Christ, and welcomes others.” Religious metaphors can summon strong emotions and spiritual insights in believers, providing motivation to use technology in certain ways (e.g., being kind in online discourse).
Second, understanding religious doctrine and teachings can help designers solve problems and achieve goals that are religious in nature. For example, Woodruff et al. studied the home automation practices of American Orthodox Jewish families [6]. Jewish laws generally prohibit manually turning electronic devices off or on during the Sabbath. These families had long designed and automated systems within their home that would perform mundane tasks (e.g., turning lights on/off with timers or sensors) to abide by this law. Without a deep understanding of Jewish teachings and practices, designing to meet their needs would not be feasible. Another example comes from the Church of Jesus Christ of Latter-Day Saints, whose doctrine encourages members to be eternally “sealed” (i.e., connected) to their family, including deceased ancestors through vicarious ordinances performed in their Temples. This focus has led them to invest significant resources in developing genealogical tools that help members identify their ancestors, such as FamilySearch, which has a collaboratively generated family tree with over 1.38 billion names and had over 200 million site visits in 2021. Additional tools allow church members to track and manage the vicarious work that members perform in the Temples. Thus, an understanding of the core doctrines related to Temple work has been essential to the development of unique tools that support such work.
Third, understanding religious teachings can help designers modify existing technologies to better meet the needs of believers. For example, HCI researchers have recognized that technologies supporting financial services within Muslim communities must work within a religious framework where charging interest is forbidden [7]. Thus, micro-lending websites that rely on interest must be modified in fundamental ways to be viable solutions in Muslim communities. Many religions have their own dating sites, helping people find singles with a similar religious background. In some cases, these include specific features that differ from general dating websites. For example, Shaadi is a popular Hindi “matrimonial” website with 35 million users that focuses on finding a spouse rather than hookups.
Finally, understanding religious doctrines can help designers identify core values that can then be used to design solutions that are in harmony with and reflect a believer’s core beliefs. Susan Wyche and Rebecca Grinter [8] examined how American Protestant Christians use ICT in their home for religious purposes. Their findings suggest many opportunities for designing systems that acknowledge, honor, and support religious values in a domestic setting, such as creating digital calendars and displays that recognize and change their content based on significant religious holidays or milestones. In 2005, an Israeli wireless company launched a mobile phone specifically designed for the ultra-Orthodox Jewish community in Israel [9]. The phones were modified to disable Internet, text messaging, and video and voice messaging, after religious authorities and community members became concerned that these services could infiltrate the community with unacceptable content.
There are many open questions about how to best use religious doctrines and teachings in design. How can design methods be modified to incorporate and prioritize religious teachings? How can products be evaluated based on religious teachings? How can technology help achieve uniquely religious goals, for which technology has not historically been used? How can systems be designed that meet the needs of diverse religious groups, given their different teachings? What values can be derived from religious teachings that can be incorporated into design?
In asking these questions that seek to understand the prescriptive aspect of religious values in the context of technology use, we can better understand which values people may want represented in technologies. We also hope that this work will serve as a call to action for the HCI community to engage more holistically with religious values. A set of values that are such an integral part of so many peoples’ lives should be acknowledged and given priority; HCI should support people’s priorities and values. We call on the HCI community to take steps toward understanding the interplay of religious values and technology to be able to create truly value-sensitive technologies.
Endnotes
1. Smith. G.A. About three-in-ten U.S. adults are now religiously unaffiliated. Pew Research Center’s Religion & Public Life Project. Dec. 14, 2021; https://www.pewforum.org/2021/12/14/about-three-in-ten-u-s-adults-are-now-religiously-unaffiliated/
2. Buie, E. and Blythe, M. Spirituality: There’s an app for that! (but not a lot of research). CHI’13 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, 2013, 2315–2324; https://doi.org/10.1145/2468356.2468754
3.Ames, M.G., Rosner, D.K., and Erickson, I. Worship, faith, and evangelism: Religion as an ideological lens for engineering worlds. Proc. of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, New York, 2015, 69–81; https://doi.org/10.1145/2675133.2675282
4. Kim, A.J. Community Building on the Web: Secret Strategies for Successful Online Communities. Peachpit Press, 2006.
5. Pope Francis. Message of His Holiness Pope Francis for the 53rd World Communications Day. 2019; https://www.vatican.va/content/francesco/en/messages/communications/documents/papa-francesco_20190124_messaggio-comunicazioni-sociali.html
6. Woodruff, A., Augustin, S., and Foucault, B. Sabbath day home automation: “It’s like mixing technology and religion.” Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2007, 527–536; https://doi.org/10.1145/1240624.1240710
7. Mustafa, M et al. IslamicHCI: Designing with and within Muslim populations. Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–8; https://doi.org/10.1145/3334480.3375151
8. Wyche, S.P. and Grinter, R.E. Extraordinary computing: Religion as a lens for reconsidering the home. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2009, 749–758; https://doi.org/10.1145/1518701.1518817
9. Campbell, H. ‘What hath God wrought?’ Considering how religious communities culture (or Kosher) the cell phone. Continuum 21, 2 (2007), 191–203; https://doi.org/10.1080/10304310701269040
Posted in:
on Thu, August 04, 2022 - 3:16:00
Derek L. Hansen
Derek L. Hansen is a professor at Brigham Young University’s School of Technology’s information technology program. His research focuses on understanding and designing social technologies, tools, and games for the public good. He has received over $2 million in funding to develop and evaluate novel technical interventions, games, and simulations.
[email protected]
View All Derek L. Hansen's Posts
Amanda L. Hughes
Amanda L. Hughes is an associate professor of information technology in Brigham Young University’s School of Technology. Her current work investigates crisis informatics and the use of information and communication technology (ICT) during crises and mass emergencies, with particular attention to how social media affect emergency response organizations.
[email protected]
View All Amanda L. Hughes's Posts
Xinru Page
Xinru Page works in the field of human-computer interaction researching privacy, social media, technology adoption, and values in design. Her research has been funded by the NSF, Facebook, Disney Research, Samsung, and Yahoo! Labs. She has also worked in the information risk industry leading interaction design and as a product manager.
[email protected]
View All Xinru Page's Posts
Faith informatics: Supporting development of systems of meaning-making with technology
Authors:
Michael Hoefer,
Stephen Voida,
Robert Mitchell
Posted: Mon, July 25, 2022 - 3:23:37
In seeking to apply HCI to faith, religion, and spirituality, we turn to existing work in theology and psychology—in particular, work that studies the development of faith in individuals and communities. James Fowler is a pioneer in faith development, having developed a stage-based model after conducting over 300 interviews with individuals from a variety of religions [1]. In his work, Fowler suggests that faith is universal to all humans, and he provides an understanding of faith that we believe would serve the HCI community in grounding further efforts to integrate HCI practice with spirituality, faith, and religion.
Fowler describes how most religious organizations fall short in supporting the development of faith in their constituents, as they fall prey to a modal developmental level—the most commonly occurring level (the mode) of development for adults in a given community (empirically, stage 3 of 6, explained in depth below). In other words, the developmental level that is most common in the adults of a community shapes the culture and normative goals of individuals growing up in that community, which Fowler calls an “effective limit on the ongoing process of growth in faith” [1].
HCI researchers are good at understanding how individuals form and act upon mental models. That is one of our aims when we try to develop technology: to understand people and what might help them. In helping individuals with their systems of meaning-making (faith), we might want to do something very similar. This, we suggest, is one of the prime challenges and opportunities for HCI researchers when engaging with faith and spirituality: to seek to understand and support individuals through their faith-ing. Here, we outline future research directions for what we are calling faith informatics: the study of systems that facilitate growth in individuals’ systems of meaning-making.
One direction for faith informatics could involve structured reflection, visualization of the self, and social connection. One goal might be to reify the existence of all stages of faith, beyond the normative, modal developmental levels present in many faith-based communities. By developing systems that support faith development regardless of religious (or nonreligious) orientation, computing and HCI hold the promise of supporting faith development and maintenance, highlighting convergence across religious traditions and supporting universal mature faith.
Faith as fundamental to the human experience
The word faith is often associated only with “belief,” in that having faith is just a matter of what one believes to be true. Wilfred Cantwell Smith suggests an alternative: that faith is not dependent on belief [2]. In this vein, faith is neither tentative nor provisional, while belief is both.
Fowler draws on this and suggests an alternative conceptualization that views faith and religion as separate, but reciprocal [1]. Fowler views religion as tradition that is “selectively renewed,” as it is “evoking and shaping” the faith of new generations. Faith is an aspect of the individual, while religion represents the tradition of the culture in which the individual grows and develops. Faith is an aspect of every human life, an “orientation of the total person,” and, as a verb, an “active mode of being and committing.” Faith is considered to be “the finding of and being found by meaning” [3]. This focus and widening of faith to include the association with meaning-making is not only inclusive of many religious and spiritual traditions, but is also vital in nonreligious traditions such Alcoholics Anonymous’s 12-Step Program. For HCI, this conceptualization resonates with existing research developments devoted to understanding the role of computing in finding and supporting meaning-making [4].
Specifically, Fowler describes three contents of faith that every human subconsciously holds in mind as they go about their life: 1) centers of value, which are whatever we see as having the greatest meaning in our lives, 2) images of power, which are the processes and institutions that sustain individuals throughout life, and 3) master stories, narratives we believe and live that facilitate our interpretation of the lived experience [1].
If faith is a universal human condition, as suggested by Fowler and Smith, then HCI researchers have much more purchase to engage with research questions related to faith, as that work would therefore have the potential to be applied to all of humanity.
Stages of faith development
Fowler describes a series of six stages of faith that are loosely aligned with age during the beginning of life, but development can be arrested in any stage [1]. Similar to Robert Kegan’s forms of mind [5], each progressive stage of faith is represented by a changing relationship between subject and object as an individual starts to consider a larger system as part of the “self” with regard to meaning-making. An overview of each stage suggests a diversity of use-cases for faith informatics.
Stage 1: Intuitive-projective faith. According to Fowler’s interviews, intuitive-projective faith is found in 7.8 percent of the population and is the dominant form found in children ages three to seven. Intuitive-projective faith is marked by fluid thoughts and fantasy, and is the first stage of self-awareness. Fowler explains that transitioning out of stage one involves the acquisition of concrete operational thinking and an ability to distinguish fantasy from reality.
Stage 2: Mythic-literal faith. Mythic-literal faith is estimated to be found in 11.7 percent of the population, and largely in children ages seven to 12. Mythic-literal faith involves the individual starting to internalize the “stories, beliefs, observances that symbolize belonging to his or her community.” Individuals in this stage create literal representations of centers of power, which often involves anthropomorphizing cosmic actors. Individuals in this stage may take religious texts literally as a foundation for meaning-making. While Fowler’s work did find a handful of adults in stage 2, many would transition out of this stage in their teenage years, as they experienced multiple stories that clashed and required reflection to integrate.
Stage 3: Synthetic-conventional faith. Synthetic-conventional faith is the most commonly found stage of faith in Fowler’s sample, and appears in 40.4 percent of the population. Transitioning into this stage often occurs around puberty, and is associated with a growing connection to social groups other than the family, perhaps with different collective narratives and centers of value. Synthetic-conventional faith systems attempt to “provide a coherent orientation in the midst of that more complex and diverse range of involvements...synthesize values and information...[and] provide a basis for identity and outlook.”
This stage is highly aligned with Kegan’s view of the self-socialized mind [5]; both are intended to deal with meaning-making involving multiple social relations and belonging to various social groups. In this stage, individuals create their personal myth: “the myth of one's own becoming in identity and faith, incorporating one's past and anticipated future in an image of the ultimate environment.” Transitioning to the next stage often involves a breakdown in the coherence of the meaning-making system, such as a clash with religious or social authority or moving to a new environment (i.e., leaving home).
Stage 4: Individuative-reflective faith. Developing an individuative-reflective faith system (32.9 percent of the population) requires an “interruption of reliance on external sources of authority.” Individuals in this stage are characterized by separation from previously assumed value systems, and the “emergence of an executive ego.” Fowler notes that some individuals may separate from previous value systems, but still rely on some form of authority for meaning-making, which can arrest faith development in the transition to individuative-reflective faith. This stage involves taking responsibility for one’s “own commitments, lifestyle, beliefs, and attitudes” and would be aligned with Kegan’s “self-authoring” form of mind.
Stage 5: Conjunctive faith. Stage five, according to Fowler, is difficult to describe simply. Conjunctive faith was found in only 7 percent of the population, and not until mid-life (ages 30 to 40). Conjunctive faith involves a deeper acceptance of the self, and integrating “suppressed or unrecognized” aspects into the self, a kind of “reclaiming and reworking of one’s past.” This stage is similar to Kegan’s self-transforming mind, as both involve the embrace of paradox and advanced meta-cognition about the self. Individuals in this stage must live divided between “an untransformed world” and a “transforming vision and loyalties,” and this disconnect can lead individuals into developing rare universalizing faith systems of meaning-making.
Stage 6: Universalizing faith. Stage six represents a normative “image of mature faith” that was found to be present in only one interview participant [1]. Fowler describes these individuals as:
grounded in a oneness with the power of being or God. Their visions and commitments seem to free them for a passionate yet detached spending of the self in love. Such persons are devoted to overcoming division, oppression, and violence, and live in effective anticipatory response to an inbreaking commonwealth of love and justice, the reality of an inbreaking kingdom of God [3].
Developing a universalizing faith is seen as the “completion of a process of decentering from the self” [3], described by:
taking the perspectives of others...to the point where persons best described the Universalizing stage have completed that process of decentering from self. You could say that they have identified with or they have come to participate in the perspective of God. They begin to see and value through God rather than from the self...their community is universal in extent [3,4].
The goal of faith informatics, as a direction of inquiry, is to better understand the systems (social, ecological, information, or otherwise) that support faith development and how they can be improved. A tool for research, and perhaps an intervention in itself, may be an information system that allows an individual to gather faith-related data about themselves and then visualize and interact with these representations of the self.
A design paradigm for faith informatics
Figure 1 highlights the components of a potential faith informatics system based on a smartphone or other computing-based application (the “system”). The theory behind this system design is drawn from research interviews used by both Fowler in Stages of Faith and Kegan in the subject object interview [5]. Both researchers relied on what we call the “research interview” to determine the stages of faith (or, in Kegan's case, “forms of mind”) present in the individual.
![](/images/uploads/blogs/frameWorkV6_web.jpg)
Figure 1. A depiction of a framework for designing faith informatics (FI) systems to support the elicitation and reflective revision of mental models of one's faith. The FI system prompts the user to systematically reflect on themself in the style of Fowler's faith development interviews [1]. This elicitation is combined with objective data about the individual's life and presented to the individual in a visualization. This visualization is then used to reciprocally influence the mental model the individual holds of their faith, facilitating development into further stages of faith.
The interview methodology relies on trained interviewers to communicate with and assess subjects using semi-structured interviews. Questions that might be asked in the interview include (selected from [1]):
- Thinking about yourself at present. What gives your life meaning? What makes life worth living for you?
- At present, what relationships seem most important for your life?
- Have you experienced losses, crises or suffering that have changed or ``colored'' your life in special ways?
- What experiences have affirmed or disturbed your sense of meaning?
- In what way do your beliefs and values find expression in your life?
- When life seems most discouraging and hopeless, what holds you up or renews your hope?
- What is your image (or idea) of mature faith?
One critical insight is that this methodology, by asking these deep questions, appears to support development by itself. Jennfier Garvey Berger notes that individuals who undergo the subject-object interview “changed the way they were thinking about things in their lives” and wanted to “come back for another interview” [6]. Some individuals reported making significant life changes after the interview, such as leaving an unhealthy relationship [6]. Fowler, using his faith development interview, also notes that interviewees tend to say things along the lines of “I never get to talk about these kind of things” [1].
HCI researchers and practitioners can help make these “developmental interview” experiences more available to the general public. The subject object interview is noticeably costly, both in time required for the conducting of the interview (60 to 75 minutes per subject), as well as required training for the researcher. One potential avenue for faith informatics, therefore, is to attempt to recreate the essential conditions of developmental interviews, allowing for the deep reflection that occurs during the interviews.
For example, mobile or Web apps could attempt to recreate the conditions of a conversation that promote self-reflection and capture the outcomes of the reflection. This approach would be aligned with the notion of reflective informatics, which seeks to support reflective practices through technology [7]. Research has already highlighted the promising effects of this kind of digital intervention. The “self-authoring” application, aligned with Kegan's forms of mind (particularly the self-authoring form) has been shown to improve academic performance, reduce gender and ethnic minority gaps, and improve general student outcomes through “future authoring.”
In Figure 1, the hypothetical faith informatics system provides prompts to individuals about aspects of their life related to meaning-making. These prompts may draw from both Kegan’s and Fowler's developmental interviews. In this particular diagram, we use the example from Fowler of eliciting the “life review,” which seeks to break the life history into episodes of meaning [1], and allows for reflection on each stage in life. This is, in essence, a large-scale version of the day reconstruction method—a kind of life reconstruction method—where individuals can break their life up into discrete episodes that mark turning points in their development of meaning-making systems. We envision a role for interactive systems in providing an interface for eliciting the construction of these episodes and any associated metadata (e.g., real-world context).
This is one of the key benefits afforded by an informatics system: the possibility of incorporating real-world, objective data in these reflective dialogues. As an individual’s system of meaning-making would be used for both answering prompts and governing an individual's behavior, the system can play a role in helping an individual compare their own perception of their meaning-making structures with how they (objectively) live their life. For example, life episodes could be colored by social contacts elicited from text messaging or email data, or behavioral activity derived from calendar entries or financial activities.
Faith informatics would therefore also connect with the field of personal visual analytics, where an individual’s data (coming directly from the experiences described earlier) is visualized into an external representation that can enable the individual to confront their mental models of themself and of their life, potentially resulting in reciprocal feedback loops to prompt insights about and support faith development. The design and creation of these visualizations is an open challenge and may benefit from co-design activities and think-aloud visualization interaction studies.
We might expect that an individual’s current stage of faith would inform design decisions. For example, an individual transitioning into stage 5 (conjunctive faith) may benefit from exploring objective data about their life history as they “reclaim and rework” [1] their past.
Facilitating social connections via faith informatics
Another promising avenue for faith informatics is that of fostering social connections that span both religious and non-religious faith traditions. While Fowler's stages of faith are largely based on Western religious traditions, it is possible (and perhaps likely) that similar developmental structures of meaning-making exist across religions and traditional beliefs worldwide, given the focus on structure of faith instead of contents of faith [1]. As such, faith informatics could facilitate the connection of individuals based on stage of development, rather than relying on religious communities that may suffer from the limits of particular modal developmental levels. Such a system could help to connect individuals in similar stages of faith across different cultures, facilitating connection and development, perhaps via the sharing of narratives and experiences.
Challenges and future work in faith informatics
Faith informatics is ripe with challenges and opportunities for the HCI community. One significant challenge is in the visualization of representations of faith that facilitate systematic reflection about an individual’s meaning-making structures. While we can draw upon Fowler’s research interview methodology to understand the types of prompts that may facilitate individual faith development, this kind of data (to our knowledge) has not previously been explored with contemporary visualization techniques. We expect that advances in personal visual analytics are necessary to support the effective visualization of self-reported faith data in a way that promotes development.
In addition, the HCI community must confront the undigitizable nature of faith and spirituality, especially with regard to users in the later stages of faith. Fowler only encountered one individual in stage 6, the universalizing faith stage. It may be difficult to attempt to reify late stages of faith in a digital system, especially when the presence of these stages is limited to few individuals. Future work could include in-depth interviews with individuals in these advanced stages of faith to better understand their life trajectory, in hopes of better understanding and sharing how they reached these particular forms of meaning-making.
Endnotes
1. Fowler, J.W. Stages of Faith: The Psychology of Human Development and the Quest for Meaning. HarperCollins, New York, NY, 1981.
2. Smith, W.C. Faith and Belief: The Difference Between Them. OneWorld Publications, Oxford, U.K. 1998.
3. Fowler, J.W. Weaving the New Creation: Stages of Faith and the Public Church. Wipf and Stock Publishers, Eugene, OR, 2001.
4. Mekler, E.D. and Hornbæk, K. A framework for the experience of meaning in human-computer interaction. Proc. of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 2019, Article 225; https://doi.org/10.1145/3290605.3300455
5. Kegan, R. In Over Our Heads: The Mental Demands of Modern Life. Harvard Univ. Press, Cambridge, MA, 1998.
6. Berger, J.G. Using the subject-object interview to promote and assess self-authorship. Development and Assessment of Self-Authorship: Exploring the Concept Across Cultures. M.B. Baxter Magolda, E.G. Creamer, and P.S. Meszaros, eds. Stylus Publishing, Sterling, VA, 2010, 245–263.
7. Baumer, E.P.S. Reflective informatics: Conceptual dimensions for designing technologies of reflection. Proc. of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, 2015, 585–594; https://doi.org/10.1145/2702123.2702234
Posted in:
on Mon, July 25, 2022 - 3:23:37
Michael Hoefer
Michael Hoefer is a third-year Ph.D. student studying computer and cognitive science at the University of Colorado Boulder. He is generally interested in studying social systems at various scales, and developing informatics systems that serve as problem solving interventions at each level. His application areas include dreaming, sustainability, and systematic well-being.
[email protected]
View All Michael Hoefer's Posts
Stephen Voida
Stephen Voida is an assistant professor and founding faculty of the Department of Information Science at CU Boulder. He directs the Too Much Information (TMI) research group, where he and his students study personal information management, personal and group informatics systems, health informatics technologies, and ubiquitous computing.
[email protected]
View All Stephen Voida's Posts
Robert Mitchell
Robert D. Mitchell is a retired pastor in the Desert Southwest Conference of the United Methodist Church and an Oblate in Saint Brigid of Kildare Monastery. He holds a Ph.D. in education and formation from the Claremont School of Theology.
[email protected]
View All Robert Mitchell's Posts
Stream switching: What UX, Zoom, VR, and conflicting truths have in common
Authors:
Stephen Gilbert
Posted: Wed, July 20, 2022 - 12:08:28
I use the term stream switching to refer to people simultaneously processing multiple streams of input information, each of which has its own context and background knowledge. This definition sounds similar to multitasking, but multitasking research usually focuses on a single individual, divided attention, and working memory capacity. Stream switching focuses on multiple people’s interactions and their mental models of each other. Below I offer examples and argue that stream switching merits further research.
Stream switching
Stream switching draws on 1) the economic concept of switching costs and 2) the psychological concepts of perspective-taking and theory of mind. Economic switching costs are the financial, mental, and time-based costs to switch between products. The costs of switching from an Android to an Apple phone, for example, go far beyond the price tag and include learning how to use the phone’s software and much personal information transfer. Analogously, stream switching includes not only the basic attentional cost of focusing on a different input stream, but also the additional cognitive load of updating one’s mental model of the stream source. Is it accurate? Is it trustworthy? These questions relate to the psychological concepts of perspective-taking (Can you imagine others’ perspectives?) and theory of mind (Can you understand how different others’ knowledge and beliefs might be from yours?).
Stream switching includes the more specific practice of code-switching. Code-switching was originally the linguistic practice of switching between languages depending on your context, and now refers more generally to the switching of identities depending on who’s around you. Someone might behave one way at home with family and another way in the outside world. Minorities often become quite skilled at code-switching since they have daily practice working within a different majority culture. I would hypothesize they are also highly skilled at stream switching.
The idea that people vary in their stream switching ability is one of the reasons stream switching deserves more research. Can this critical skill be practiced or trained? We already know that some people are better than others at empathizing, understanding what other people are thinking and feeling. Research has correlated these individual differences with factors including reading more fiction, role-playing and reflection, the general practice of thinking more about one’s thinking. Perhaps this research could be extended to develop methods to measure one’s stream switching ability and methods of improving it.
User experience (UX) implications
In user-centered design, we try to empathize with our users. We create personas to reduce our cognitive load of doing so. Here is Maria, the young professional with two children; what are her jobs to be done when she opens our app? A product designer with better stream-switching skills will truly be able to step into Maria’s world and build an accurate mental model of her goals and expectations in order to design a product that fits perfectly. That product then allows Maria to accomplish her goals more quickly with fewer errors.
On the other hand, low-usability software requires the user to switch streams mid-use to imagine the intentions of the software designer or to model the inner workings of the software itself. You may have experienced an accounting system that was well-designed for accountants but not for you. While the accountants can easily track expenses, you can’t figure out how much money you have. Or consider a social media system. If you post certain content, is it clear who will see it? Will it lead your friends to see related ads? Or in a corporate calendaring system, if you invite people to a meeting, is it clear who will see the invite list? Being able to answer these questions requires highly usable software. Norman warned about gulfs in evaluation; today’s sociotechnical context requires evaluation of not only the system, but also of other users’ experiences with it.
Frustrating usability situations increase stream switching, which burdens our cognitive capacity. When you have to switch streams to figure out where to click next, you have less attention to devote to the streams you were already juggling, for example, home life versus work life, or your supervisor’s mindset versus your teammates’. Bad interaction designs effectively steal our attention.
Asymmetry, monitoring, and conflicting versions of truth
Many systems present asymmetric information to users, i.e., collaborators receive different information or levels of access, as in the calendaring example. It’s not always a problem; a person presenting slides should be able to see their presenter notes while the audience should not. But when you’re Zooming and ask, “Can you hear me?” or “Can everyone see my screen?”— that’s problematic asymmetry. You don’t have good cues about what other people are experiencing, so you have to ask explicitly. Having this information enables you to update your mental model of your colleagues’ experience in the meeting and enables you to stream switch more smoothly between conveying your points and monitoring your colleagues’ understanding.
Asymmetric stream switching also appears if you use a virtual reality headset. When you enter the virtual world, you partially blind yourself to cues in the real world. To avoid tripping over a nearby chair, you need to switch streams consistently to monitor both the virtual world and the real world. If you carefully arrange the furniture in your room beforehand, creating a safe space for virtual exploration, then you can reduce the mental workload required to monitor that stream and focus more on the virtual stream with less overhead.
Analogously, as supported by research on stereotype threat and Goffman’s idea of roles that people play in life, when people have a metaphorical “safe space” for collaboration with others despite diverse backgrounds, they can focus less on monitoring the stream of how they’re being perceived (“Will I say something offensive?”) and more on the stream of the collaborative task at hand.
Finally, consider the number of people who have difficulty speaking with each other because of their dramatically different beliefs about what is true. This is a stream-switching problem with high asymmetries. It has become more difficult than ever before to imagine what it’s like to be on the other side, because the affordances to do so barely exist. Only by investing significant time in creating alternative social media accounts and filling them with clicks could someone start to experience the other’s perspective. The cognitive load of stepping into the shoes of the other person has become so high that it’s easier to discount them as foolish or deceived. It’s easier to stream switch and model the perspective of someone similar to yourself. In part, that’s why similar people are drawn to one another. But there is a very high likelihood that many people we work with, as well as our customers, are not similar to us. Research on the “contact hypothesis” shows that talking with people who are different than ourselves enables us to understand their perspectives with more empathy, even if it’s difficult. More than ever before, we need to increase our stream-switching abilities, which will enable us to understand others.
Thanks to Joanne Marshall and Kaitlyn Ouverson for thoughtful feedback on these ideas.
Posted in:
on Wed, July 20, 2022 - 12:08:28
Stephen Gilbert
Stephen B. Gilbert is associate director of Iowa State University's Virtual Reality Application Center and director of its human-computer interaction graduate program, as well as an associate professor in industrial and manufacturing systems engineering. His research interests focus on technology to advance cognition, human-autonomy teaming, and XR usability.
[email protected]
View All Stephen Gilbert's Posts
Unavailability: Food for thought from Protestant theology
Authors:
Sara Wolf,
Simon Luthe,
Ilona Nord,
Jörn Hurtienne
Posted: Tue, July 19, 2022 - 9:45:16
The past two years of living in pandemic times have accelerated the spread of technology into all areas of life. This was also evident in the context of religious communities and churches, where the number of applications and users has increased enormously. Not only individual communities but also the great church institutions had to expand their presence in the digital sphere [1]. As a result, interaction with technology in religious and spiritual contexts is now more widespread than even a few years ago. Understanding how technology and interaction design influence experiences in such contexts is more important than ever. However, this increased need for knowledge is not yet visible in HCI publications. We also believe that through more research in these areas, HCI can gain new perspectives on technology use, design, and evaluation more generally. Similar to how work on religious objects in households inspired a broader call for extraordinary computing [2] , we would like to introduce a theme that emerged from our work with Protestant believers, and that can bring new impetus to HCI: unavailability.
We derive this claim from a continued cooperation between HCI researchers and Protestant theologians. Together, we have been working on several projects that aim at designing technology for religious communication in the form of rituals, blessings, and online worship services. In the following, we want to demonstrate that integrating aspects of faith, religion, and spirituality in HCI might be valuable and lend HCI new perspectives.
Unavailability
The development of current technology is about making everything available at any time: Vast amounts of music and films are available through media streaming services, our loved ones are available through video (chat), and worship services are available online. In most of the Western world, many of our desires can be fulfilled immediately using technology, which focuses on making everything visible, accessible, controllable, and usable [3]. However, this ubiquitous availability might not always be valuable. Sometimes the opposite, unavailability, might be the better choice. Unavailability can highlight what one values most about what is available and can evoke the experience of resonance, specialness, or meaning. In the following, we will present two examples that demonstrate how we came across the theme of unavailability in our research.
The first example originates in our work on blessings. In a design probe study with Protestant believers, we tried to understand what blessing experiences are, and where or when they happen in believers’ everyday lives. Participants described that the feeling of being blessed can occur anytime, anywhere, but is most intense when it is unexpected and surprising (i.e., unavailable). One participant shared the following story when asked to describe an experience of being blessed:
I had a conversation with a friend who told me about her happiness as a mother, how it was to hold her newborn baby in her arms for the first time, how much love she was surrounded by, and how proud she was. And that was very strange for me because she had to deliver the child dead. And, um, I didn’t expect that. And at that moment, well, that was so.... so that overwhelmed me…. So she knew her child would be born dead, she knew she would have a silent birth, and yet there was a lot of pride and happiness and love, and she is still proud to be a mother, even though her child was born dead. And I just find that..."Wow"! So my rational brain said, "Well, that cannot be for real, that doesn’t fit," and I was also afraid of the conversation with her. Um, and then I was, so that’s what got me… So that was surprising, yes, or maybe also what I hoped for. So sometimes it [the blessing] is also a fulfilled hope.
Not all examples of blessing experiences were as drastic as the one described here. However, this story demonstrates the aspects of unexpectedness and surprise very well. The participant did not expect that the conversation with her friend could take place in a positive atmosphere—she was even afraid of the conversation. And then everything turned out quite differently than expected. She could not have worked out this twist or influenced the situation in this direction with certainty—it simply came as it came. The unavailability was also evident in other examples within the same study. Many participants described that they used to bless each other, although they can never be sure whether the blessings are effective—it is beyond their control. For our participants, Protestant believers, this control was attributed to God. The aspect of unavailability generated friction and excitement in people’s experiences: It opened up room for hope, speculation, and surprise—for example, when something absolutely unexpected and positive happens.
Our second example on unavailability shows the opposite: namely, what happens when the unavailable becomes available? In another project, we investigated the experiences of online worship services during the pandemic [4]. We accompanied Protestant believers while participating in online worship services and tried to understand how specific design elements lead to specific experiences. One prominent element that influenced the experiences dramatically was availability and ease of access. Usual worship services are not an everyday occurrence for believers, but rather something special; believers usually invest some effort to mark the worship service as distinct from everyday life and routines—for example, dressing up, going to a special place, and reserving the time to attend. In contrast, online worship services are available anytime and anywhere, which invites specific modes of usage (e.g., watching it on the side).
One couple reported a situation that shows the tensions such constant availability can create. On one Sunday, the couple woke up later than usual, and were in the middle of their breakfast when realizing that the worship service was about to start. Invited by the flexible and accessible design of current online worship services, they watched it using a laptop at their breakfast table. Although this was practical, they quickly became annoyed with themselves. They realized that they had turned what formerly had been an extraordinary experience into something ordinary. Availability changed the way worship services were experienced. The online worship service turned into something everyday and less essential. Constant availability may be convenient and allow for flexible access. However, convenience and flexibility are nothing compared with the cherished unavailability of worship services that take place only at a specified place and time and are unavailable in between.
So far, unavailability seems to be a concept that is given little consideration in HCI, and that even opposes current trends of making everything available. The two examples show how unavailability affects experiences. We think it is worth looking at the concept more closely, as it can reveal new perspectives on technology design. In the following, we will turn to sociology and Protestant theology in order to learn more about the concept of unavailability. Theology has long been concerned with unavailability, and sociology shows how the concept of unavailability is essential for human experiences beyond the context of religion, faith, and spirituality.
The German sociologist Hartmut Rosa has studied unavailability (German: Unverfügbarkeit; he translates it as “uncontrollability”) in his works [3,5]. Rosa describes our time as a time of acceleration, suggesting the concept of resonance as a possible solution [5]. For Rosa, resonance is a type of world relationship formed by affection and emotion, intrinsic interest, and the expectation of self-efficacy, in which subject and world connect and at the same time transform each other. That is, the nature of the world relationship is to be understood as reciprocal. Not only is the relationship defined between subjects and objects, but they also define a new relationship to the world [5]. The experience of resonance is opposite to the experience of alienation, a world relationship in which the subject and the world are indifferent or hostile (repulsive) to each other and thus inwardly disconnected from each other—a relationship of "relationshiplessness" [5]. For Rosa, resonance is the human motivation that guides all actions. A central, constitutive aspect to resonant experiences is unavailability. Four conditions for resonant experiences must coincide [3]:
- Touch (something touches me)
- A response to the touch
- Transformation: a change of world-relationship
- Unavailability.
Even if conditions one to three are fulfilled, unavailability is necessary for a successful, resonating experience. The individual experience of the world can be neither planned nor accumulated. This perspective highlights a fundamental problem with the current focus on making everything available through ubiquitous technology: It is not the availability that renders experiences successful, resonating, and thereby valued but rather their specific quality. And part of what makes their quality is that people are not in control of everything and cannot make the world available to the last. It is precisely in this that Rosa sees a necessity. Space must be given to the concept of unavailability because only in this way are resonating experiences possible [3].
Regarding Christian religion, the necessity of the unavailable for a successful world experience as described by Rosa becomes particularly clear. All objects of the Christian religion, such as God, Christ, the Holy Spirit, grace, living a fulfilled life, and blessings, cannot be made controllable to human beings; they cannot be commanded. Even in an increasingly secularized world, the objects of religion and their unavailability remain something that fascinates and attracts people—albeit no longer only in the forms of the established religious communities. This search for meaning is both an attempt to make the unavailable available and the realization that ultimately unavailability is constitutive for religious experiences. It is precisely this unavailability that makes dealing with the objects of religion interesting to people. If God, the Holy Spirit, or Christ were made available, religion would become uninteresting and lose relevance for the resonant experiences as illustrated above.
The theme of unavailability prompts HCI to reconsider current trends of making everything available. Recognizing that unavailability might be an essential experiential quality, HCI is challenged to engage in the topic. How can unavailability be experienced when interacting with technologies? What ways, if any, are there to design for the unavailable?
To design or not to design?
Although there seem to be no simple, singular answers to the above questions, we would like to present different perspectives to stimulate discussion within HCI.
Unavailability is a key topic in the Christian religion and tradition. As such, the Christian tradition is constantly confronted with unavailability and tries to create conditions to make experiences with the unavailable more probable. Such efforts can, potentially, be considered as design. To this end, the Christian tradition offers (or designs) activities such as rituals, liturgy, and experiential education that serve as supportive measures, knowing that the unavailable is ultimately unavailable.
On the question of design for unavailability, Rosa builds his argument for resonant experiences. He argued that resonant experiences can arise only with counterparts (e.g., human beings, objects, nature, art) that are not entirely available—meaning visible, accessible, controllable, and usable [3]. Following this, Rosa doubts that technology can be designed at all to become a resonant counterpart: unavailability is uncontrollability and thus “non-engineerability.” He expects that translating unavailability to, for example, unpredictability in technology design might lead to frustration rather than resonant experiences. However, Rosa also identifies manufactured objects that evoke resonant experiences, such as poems or art. He expects a poem to be a resonant counterpart as long as one has not fully grasped and processed it, as long as it continues to occupy one and still seems to hide something [3].
With the above arguments and examples in mind, we turned to HCI searching for artifacts and design strategies that might correspond to unavailability. An artifact that shows an essential aspect of the integration of unavailability and technologies, namely the type of activities that a technology enables, is the Drift Table [6]. The Drift Table is a coffee table displaying slowly moving photography that is operated by the distribution of weight on its surface. One could argue that this interaction represents a kind of “control” of the table. However, what is most related to unavailability is not the exact interaction, but rather what the table encourages us to do: The Drift Table was designed not to perform specific tasks or efficiently achieve goals but rather to support ludic activities, “activities motivated by curiosity, exploration, and reflection rather than externally defined tasks” [6]. It is exactly in this way that we think it might correspond to unavailability: A technology that is supposed to have an inherent unavailability must invite us to explore it and not work through it—similar to a poem or a work of art. If it is at all possible to design for unavailability, then the first step must be a change of perspective on the what of technology: Away from technology as efficiency-enhancing task support, toward technology as a stimulus for open exploration, reflection, and curiosity.
![The figure contains three sketches, each representing a concept. The concepts are sketched in black and white and essential elements are highlighted in yellow. In addition, the essential elements are described in short text notes. The text notes are: Left, The Drift table: “Ludic activities. Promote curiosity, exploration and reflection. De-emphasise the pursuit of external goals. Maintain openness and ambiguity. Withhold a clear interpretation or narrative of use.” Middle, The Projected Realities bench: “Ambiguity. Present information with limited context. Create tension through the combination of contradictions (viewing vs sitting, familiar vs strange). Leave space for individual interpretation. Right, ThanatoFenestra: “Ephemerality. Include elements that last for a limited time. Use materials that evoke multisensory perceptions. Rely on mechanisms known from the living nature.](/images/uploads/blogs/temp_file_Wolf_image_web1.jpg)
Three design strategies that can serve as a useful starting point for thinking about design in the context of unavailability. Left: The drift table as an example of design for ludic activities [3]. Middle: The Projected Realities bench as an example for ambiguous design [2]. Right: ThanatoFenestra as an example for ephemeral interfaces [1]. Sketches: Vyjayanthi Janakiraman.
Apart from this concrete example, we also found two particular design strategies within the more speculative and exploratory areas of HCI that are more concerned with the how of interaction suitable when designing for unavailability. The first strategy is to design for ambiguity [7]. Gaver et al. [7] suggest that ambiguity “frees users to react to designs with skepticism or belief, appropriating systems into their own lives through their interpretations.” One way to integrate ambiguity into the design is to distort displayed information to stimulate curiosity and thought. The second strategy to design for unavailability might be ephemeral interfaces [8]. Ephemeral interfaces consist of at least one element that lasts only for a limited time and typically incorporate materials invoking multisensory perceptions such as water, fire, or plants [8]. Ephemerality is a design strategy seldomly used in popular, widespread technologies. However, it is an element everyone recognizes from the living, natural world that corresponds well with unavailability.
In conclusion, we hope to have demonstrated the value of turning to contexts such as spirituality, faith, and religion to gain new perspectives for HCI and hope to stimulate discussions on making everything (un)available. We conclude with one last quote for reflection:
We should…speak of God. But we are human beings and as such cannot speak of God. We are to know both, what we ought to and what we cannot do, and precisely in this way give glory to God. –translated from Barth (1929)
Acknowledgments
We would like to thank all of our participants who shared their experiences with us. May you be blessed! Parts of the research have been funded by the Federal Ministry of Education and Research (BMBF), project CoTeach (project number 01JA2020).
Endnotes
1. Nord, I. and Adam, O. Churches online in times of corona (CONTOC): First results. In Revisiting the Distanced Church. OAKTrust Digital Repository. 2021, 77–86. https://doi.org/10.21423/revisitingthechurch
2. Wyche, S.P. and Grinter, R.E.. Extraordinary computing: Religion as a lens for reconsidering the home. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2009, 749–758; https://doi.org/10.1145/1518701.1518817
3. Rosa, H. The Uncontrollability of the World. Polity, 2020.
4. Wolf, S., Mörike, F., Luthe, S., Nord, I., and Hurtienne, J. Spirituality at the breakfast table: Experiences of Christian online worship services. CHI Conference on Human Factors in Computing Systems Extended Abstracts. ACM, New York, 2022; https://doi.org/10.1145/3491101.3519856
5. Rosa, H. Resonance: A Sociology of Our Relationship to the World. Polity, 2021.
6. Gaver, W.W. et al. The Drift Table: Designing for ludic engagement. CHI '04 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, 2004, 885–900; https://doi.org/10.1145/985921.985947
7. Gaver, W.W., Beaver, J., and Benford, S. Ambiguity as a resource for design. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2003, 233–240; https://doi.org/10.1145/642611.642653
8. Döring, T., Sylvester, A., and Schmidt, A. A design space for ephemeral user interfaces. Proc. of the 7th International Conference on Tangible, Embedded and Embodied Interaction. ACM, New York, 2013, 75–82; https://doi.org/10.1145/2460625.2460637
Posted in:
on Tue, July 19, 2022 - 9:45:16
Sara Wolf
Sara Wolf is a passionate HCI researcher working at the intersection of faith, spirituality, religion, and technology. She is currently doing her Ph.D. with a focus on technology-mediated (religious) rituals and works in the CoTeach project, and as an HCI lecturer at the Institute of Human-Computer-Media at the University of Würzburg.
[email protected]
View All Sara Wolf's Posts
Simon Luthe
Simon Luthe is a practical theologian and religious educator working on pop culture issues and at the intersection of faith and technology. He is currently doing his Ph.D. in a research project on blessing spaces in VR/AR at the University of Würzburg. He is also a vicar in the parish of Heide, Schlewig Holstein.
[email protected]
View All Simon Luthe's Posts
Ilona Nord
Ilona Nord is professor of Protestant theology and religious education at Julius-Maximilians-Universität, Würzburg, Germany. Her research interests include religious teaching and learning, developing new methods for teacher education in a digitalized world and churches online in times of Corona.
[email protected]"
View All Ilona Nord's Posts
Jörn Hurtienne
Jörn Hurtienne is a professor of psychological ergonomics at Julius-Maximilians-Universität, Würzburg, Germany. His research interests include the theory and design for intuitive use, image schemas and primary metaphors as well as user experience design for fundamental values.
[email protected]
View All Jörn Hurtienne's Posts
Breaking stereotypes: Islamic feminism and HCI
Authors:
Hawra Rabaan,
Lynn Dombrowski
Posted: Thu, July 14, 2022 - 9:28:33
As HCI matures into a richer discipline interlacing with the humanities and social sciences, there has been a growing consciousness to embrace pluralist [1] and intersectional approaches in understanding and addressing systems of oppression within computing [2]. Islamic feminism and intersectional feminism are highly complementary approaches to understanding oppression and power. While both focus on gender, each brings its own distinct attention to how gendered violence manifests. In research, Islamic feminism is a theoretical, analytic, and design lens to understand and attend to the needs of Muslim women. Beyond being useful for understanding issues that only Muslim women face, as a theoretical approach, it expands how we understand agency, sheds light on the historic and sociocultural contexts, and explores design around cherished and socially familiar values [3,4]. This article calls for expanding HCI feminist theories to account for nonsecular and non-Western contexts, and answers the following questions: 1) What is Islamic feminism? 2) Why should we as academics care about it? and 3) How can we bring it into HCI research and design?
What is Islamic feminism?
Islamic feminism scholarship, similarly to mainstream Western feminism, centers studying gender and systems of power and oppression. Primarily, Islamic feminism differs from mainstream feminism in 1) its perception of religion and 2) definitions of agency and resistance. While there are many dangerous tropes portraying Islam as anti–female empowerment or as patriarchal, within Islamic feminism, Islam is viewed as feminist for several key reasons. Islamic feminists see Islam as a way to promote social justice and equity; however, Islamic feminists are critical of orthodox interpretations of Islamic texts and cultural praxis and often position them as the sources of patriarchal beliefs [5]. Thus, Islamic feminist scholars (e.g., Amina Wadud, Husein Muhammad, Ziba Mir-Hosseini) have worked extensively and continue to produce alternative interpretations of sacred texts to challenge the patriarchal elements in Islamic jurisprudence and shift the paradigm of religious authority. For example, verse [4:34] is commonly used by religious scholars to promote wife-hitting, whereas Islamic feminist scholars interpret the verse to encourage temporary separation and reflection. The interpretation is inferred from a holistic Quranic stance on spousal relations rather than verbatim explanations, concluding its focus on respect, reconciliation, and justice, and consequently opposing harm and violence [4].
The second dispute between mainstream feminism and Islamic feminism is how agency is defined and identified. In mainstream feminism, agency is fundamentally based on the liberal political theory’s concept of freedom, where an autonomous will is fulfilled through “universal reason,” unburdened by tradition, customs, or transcendental will. Islamic feminists view this definition of free will as limited; instead, agency is not just about a person’s capacity, but rather about how societal, economic, and political structures create and reinforce conditions, cultural norms, possibilities, and oppression around gender [6]. In Islamic feminist thought, resistance and agency are often inseparable and extend agency to include non-resistive actions. For example, a woman in an abusive relationship patiently remaining married is an autonomous choice made within layered constraints that takes time, careful consideration, and effort [4]. Another form of agency that is regularly overlooked by Western feminists is religious agency, which consists of acts grounded in religious beliefs—to adhere to a transcendental power (i.e., their God) rather than to the abusive figure or patriarchal norms. Lastly, using an Islamic feminist lens reveals more depth of the human experience and urges us to practice greater empathy and reduce harm, by acknowledging and working within the practical conditions influencing participants’ autonomy.
Why and when to use Islamic feminism?
The HCI community centers diversity, equity, and inclusion. Islamic feminism provides tools to conduct justice-oriented research and design, where participants are not “othered” or looked down upon, where we value our participants’ voices, and build upon community assets and values. As an HCI scholar, I call for us to turn inward and surface the biases we may have as researchers and designers, find and routinely use tools to help us overcome our biases, and bring about the progressive change we seek as academics and global citizens.
As the field continues to broaden its reach and impact, the problem-solving standpoint familiar to the HCI community must shift. By using an Islamic feminist approach, we strive to provide implications and design within the context of entangled political and cultural norms, rather than eradicate those norms or perceive our user as a passive victim [4].
How to use Islamic feminism in HCI
Next, we leave you with three takeaways on how to begin following an Islamic feminist approach in HCI:
- Dare not to take on a savior’s complex, orientalize, or deem cultures as less than the West. When approaching social issues that concern sensitive problems and vulnerable populations, be diligently cautious of causing more harm, whether it’s in perpetuating stereotypes, interacting biasedly, or superficially tackling the problem.
- View practicing agency as culturally and historically specific. It is the researcher’s job to acutely understand the practical conditions and sociohistorical processes contributing to the autonomy of participants. Such lenses are essential within all roles of research; problem formulation, analysis, implications and design, and research evaluation [7].
- Call for empowerment from within the participants’ structure, by designing around cherished, internalized, and socially familiar values when evaluating current technologies or providing implications for future inclusive designs [3]. Include norms beyond social impositions, and focus on the participants’ cultural dimensions, historical developments, and different ideas of justice.
While it goes without saying that Muslims are not a homogeneous group, portraying Islam as a violent religion or Muslim women as passively oppressed is inexcusably prevalent within the broader society and this unfortunate stereotype has not escaped the academic realm. I have faith in our community and the future of inclusive technologies. When in doubt, connect with fellow Muslim scholars or groups such as the IslamicHCI group [8], who will be open to sharing their insights when needed. We can do better, HCI!
Endnotes
1. Bardzell, S. Feminist HCI: Taking stock and outlining an agenda for design. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2010, 1301–1310.
2. Rankin, Y.A. and Thomas, J.O. Straighten up and fly right: Rethinking intersectionality in HCI research. Interactions 26, 6 (2019), 64–68.
3. Alsheikh, T., Rode, J.A., and Lindley, S.E. (Whose) Value-Sensitive design: A study of long-distance relationships in an Arabic cultural context. Proc. of the ACM 2011 Conference on Computer Supported Cooperative Work. ACM, New York, 2011, 75–84.
4. Rabaan, H., Young, A.L., and Dombrowski, L. Daughters of men: Saudi women's sociotechnical agency practices in addressing domestic abuse. Proc. of the ACM on Human-Computer Interaction 4, CSCW3 (2021), 1–31.
5. McDonald, L. Islamic feminism. Feminist Theory 9, 3 (2008), 347–354; https://doi.org/10.1177/1464700108095857
6. Mahmood, S. Feminist theory, agency, and the liberatory subject: Some reflections on the Islamic revival in Egypt. Temenos-Nordic Journal of Comparative Religion 42, 1 (2006).
7. I would like to take the opportunity to express my gratitude to the CSCW reviewers who pushed my work into maturation by providing constructive criticism and actionable feedback.
8. Members of the group can be reached at [email protected]
Posted in:
on Thu, July 14, 2022 - 9:28:33
Hawra Rabaan
Hawra Rabaan is a Ph.D. candidate in human-computer Interaction at IUPUI’s School of Informatics and Computing. Rabaan’s work combines social work and HCI by focusing on examining sociotechnical practices in response to domestic violence and designing for countering domestic violence within the Muslim community through a transformative justice lens.
[email protected]"
View All Hawra Rabaan's Posts
Lynn Dombrowski
Lynn Dombrowski is an associate professor in the human-centered computing department in the School of Informatics and Computing at Indiana University – Purdue University in Indianapolis. She runs the Sociotechnical Design Justice lab, where she studies, designs, and evaluates computational systems focused on social inequity issues with her students.
[email protected]
View All Lynn Dombrowski's Posts
Faith, modernity, and urban computing
Authors:
Nusrat Jahan Mim,
Syed Ishtiaque Ahmed
Posted: Fri, July 08, 2022 - 12:17:25
Cities have long been a center of attention for modernization projects, so it is no surprise that most of today’s computing ventures are choosing cities as the main avenue for demonstrating their prowess. The idea of a “smart city,” for example, has been attracting a wide range of academics and practitioners in computer science and related fields whose works concern sensor systems, smartphone applications, human-computer interaction (HCI), computer supported cooperative work (CSCW), social computing, ubiquitous computing, data science, and artificial intelligence (AI). Related to the dream of smart cities are the dreams for driverless cars to claim the city streets, drones to deliver food to city apartments, and large-scale smart displays to inform and entertain citizens.
These dreams are also being partially realized in various cities around the world, with a mix of successes and failures. In fact, it would not be an overstatement to say that computing technologies often define what cities are in today’s world. While the concentration and intensity of dazzling, cutting-edge computing is generally higher in cities in the Western world, their Global South counterparts have started to catch up. Under the government mandate of digitization, countries like Bangladesh and India, for example, are updating their cities and the lives of their citizens with new mobile applications, biometric identity, the digital gig economy, online education, and social media, to mention just a few. However, this rapid advancement of “urban” computing is often devoid of ideas of faith, religion, and spirituality; even worse, their insensitivity to such sentiments is creating an agonistic and uncomfortable situation for many citizens, especially in the Global South. A deep understanding of the historical conflicts between modernity, science and technology, and urban religiosity is required to trace the role of computing within this tension.
Let’s go back a couple of years, when Covid started to unfold in Bangladesh. Alarming images of Bangladesh shared on social media included photos of religious gatherings of faith-based communities. Here, we use a broad definition of religion that incorporates both organized religions (Islam, Hinduism, Buddhism, Christianity, etc.) and traditional and Indigenous beliefs and non-beliefs. By faith-based communities, we refer to the groups of people who participate in activities that are motivated by their religious beliefs. While faith may influence and shape every action taken by a person, for the sake of simplicity, we define religious practices by the activities that are driven primarily by religious motivations (e.g., prayers, congregations, etc.).
While the government of Bangladesh mandated “social distancing,” and there were nationwide campaigns to create awareness among people to avoid crowds, those suggestions were often not effective in practice, especially among faith-based communities. In many places in the country, including major cities, huge religious congregations kept taking place at mosques during Friday prayers, ignoring all cautionary notices. At one point, when the city authorities closed mosques and asked people to perform their prayers at home, many Muslims started to gather on their rooftops to perform their prayers (Figure 1) in jamat (groups)—the call for social distancing kept getting ignored. In mid-April 2020, thousands of people gathered at the funeral ceremony of a local influential religious leader in the Bramhambaria district, ignoring the strict ruling against such gatherings. Then, in the month of Ramadan, the nation feared a spike in the number of infected people as a series of religious festivals, including Shab-e-Qadr, Jumatul Bida, and Eid ul Fitr, were forthcoming. Those tensions between the city and religious communities allowed us to see the politics around the formulation of modern cities and the historical struggle of the religious identities within them.
While modernity is often defined by a set of values, processes, technologies, and the time period when scientific revolution, industrialization, and capitalism rapidly expanded across the West, urban modernity is characterized by the adoption of modernist ideologies in the urban built environment and as a tool to respond to emerging social and political tensions. Predominantly concentrated on the modernity-driven urban development strategies and planning policies engendered in the West, urban modernity is also globally understood as a product of secular institutions, practices, and discourses [1]. A major portion of this scholarship was developed in the 19th and 20th century, which read cities through the lens of scientific rationality and measured a city’s performance against a scale of “industrialized progress.”
With the theorization of the Global City in the 1980s, this new paradigm of modern urbanism started to expand beyond the West and influence the spatial, political, and economic readings of modern cities worldwide. These scholarly and design practices reinforced the qualities of Western modernity as the standards of a successful, global, modern city. In the late 20th and early 21st centuries, scholars such as David Harvey, Edward Soja, and Saskia Sassen mapped urban-spatial responses around global flows of capital, technology, information, and other streams, pointing toward the failures of such modernist advancement at different scales in an urban setting. Soja, Jane Jacobs, Susan Fainstein, Leonie Sandercock, Lisa Peattie, and many others shed light on different forms of social inequalities and injustices that were introduced by modern cities. Although it was not at the center of their criticism, the Western concept of modernity has also always upheld the idea of secularization, and there has been a predominant secular spirit among modernist planners and scholars toward understanding cities.
While the cities in the Global North have often relied on their secular institutions, one cannot understand the cities of the Global South, especially South Asian and African cities, without understanding their religious spirit. However, the externalization of religion to the core discourse characterizes the development of modern South Asian and African cities of the recent past [1]. Hence, the cultural pluralities and the alternative visions of modernity that could develop in this part of the world, which have the potential to generate a dominant body of knowledge in urban studies, are often marginalized [2]. Such marginalization not only hampers the tolerance and productive public engagement of people from different religious backgrounds [3] but also limits the scope of “vernacular” development in the cities of the Global South.
![](/images/uploads/blogs/Mim_figure1.jpg)
Figure 1. Religious people praying on the rooftop of a building after praying at the mosque was halted. (Image courtesy of Rubayet Tanim)
As a Global Southern city with a strong religious backdrop, Dhaka, the capital of Bangladesh, is often considered a “non-modern” city. To achieve the Global City aesthetics, Dhaka like any other developing city in many respects, attempts to abide by the “rules of modernity” that were developed in the West. Hence, the urban religions often remain outside the development discourse. The presence of religiosity therefore gets limited, neglected, and often ridiculed outside spaces of worship such as mosques, temples, churches, and pagodas. For instance, when a mosque in a neighborhood starts to be treated as merely another “building,” the much-needed socioeconomic and spatial connections between the city and the religion are overlooked. The associated design conversations fail to capture how the “donation box” placed at the entrance of a mosque and the business of informal vendors standing outside the entrance are deeply integrated with the complex network of the city’s economy. It fails to understand how a low-income rickshaw-puller feels when he wants to pray his afternoon prayer but can hardly find a secure place to park his rickshaw near the mosque. Hence, the conversations fail to address the challenge of better handling the traffic during any religious events within or around the city. Since Dhaka is a Muslim-majority city, we are focusing on examples of Islamic practices here. In reality, such spatial and socioeconomic marginalizations can be traced in other religious practices as well.
Modern cities also put limits on religious rituals, and this soon escalates to a contention between the city and its religious citizens. When religious communities feel that the city is being “developed” without including their input, they start developing resistance and try to hack into various spatial arrangements of the city to make their presence visible. Keeping urban religion outside the prominent design discussions results in different infrastructure-level failures as well. For instance, severe traffic gridlock occurs in Bangladeshi cities before Eid ul Fitr, when a huge number of rural people temporarily visit to collect
zakat (financial help from the rich obligated by Islam). Similar spatial complications occur during the congregational religious event
Biswa Ijtema, where millions of Muslims
gather to empower their Muslim identities. If the city development plans incorporated such religious festivities with due significance, Bangladeshi cities like Dhaka would have been developed in a distinctively inclusive way.
The borrowed concept of modernity, on the one hand, has limited possible design conversations with faith-based groups. On the other hand, excluding religiosity from the dominant urban design and planning decisions has created additional pressure on the city’s infrastructure. When the connections between the city and religion become parochial, it becomes quite difficult to expect thoughtful, tolerant responses from the marginalized faith-based communities during emergencies such as the Covid-19 pandemic. Hence, every now and then we observe alarming crowds at religious gatherings in South Asian cities, defying lockdowns and mandatory social distancing. These crowds are not an immediate “unthoughtful” response to the pandemic situation, but rather simply the result of a deeply rooted resistance to the city’s “secular interventions.”
This brings us to the point where we can think of urban computing in a broader perspective. The discourse of “smart cities,” in both the West and the Global South, lacks a discussion on how to incorporate religious practices. If a religious practice is essentially communal, such as
adhaan (Muslim prayers), how should the city strike a balance between the individual choice of not listening and the communal choice of hearing the sound? How is the city supposed to respond to the massive inflow of people, material, and livestock during religious festivals that challenge its infrastructure? If digitization develops a physical distance between people, how does that affect religious practices? How is the liberal, nonhierarchical model of online social media affecting value-laden religious communities? More importantly, how safe are the digital spaces for religious people? These and many other questions are critically important, and yet rarely discussed in the domain of urban computing.
We identify faith, religion, and spirituality as an underexplored territory of HCI in general, and urban computing in particular. Computing’s historical alignment with science, technology, and modernity has made it difficult to capture such sentiments, which are built upon belief, respect, tradition, fear, and purity. Furthermore, computing technologies, when acting as a force of modernization, often end up silencing and marginalizing the voices of religious communities by creating a “secular” space. However, a new paradigm of HCI can be imagined that challenges this top-down modernist program by strengthening the voices of marginalized religious sentiments.
Endnotes
1. Hancock, M. and Srinivas, S. Spaces of modernity: Religion and the urban in Asia and Africa.
International Journal of Urban and Regional Research 32, 3 (2008), 617–630.
2. Simone, A.M. On the worlding of African cities.
African Studies Review 44, 2 (2001), 15–41.
3. Greed, C. 2016. Religion and Ssustainable urban planning: ‘If you can’t count it, or won’t count it, it doesn’t count’.
Sustainable Development 24, 3 (2016), 154–162.
Posted in:
on Fri, July 08, 2022 - 12:17:25
Nusrat Jahan Mim
Syed Ishtiaque Ahmed
Syed Ishtiaque Ahmed is an assistant professor of computer science at the University of Toronto. For the past 12 years, he has been conducting ethnography and design research with various marginalized groups in the Global South.
[email protected]
View All Syed Ishtiaque Ahmed's Posts
Reflections on the politics of African ‘limitations’ in HCI research
Authors:
Muhammad Adamu,
Shaimaa Lazem
Posted: Tue, July 05, 2022 - 10:46:44
Last year, I read Shaimaa’s reflection on how Indigenous cultures and folk heritage may act as a steppingstone for sustainability in the absence of resources [1]. This led to extensive correspondence with her on how lessons from Saeed El Masry’s book [2] can reframe the common dualities of scarcity and sufficiency found in African perspectives on innovation in HCI. We pondered the question of how we, as researchers and practitioners, can challenge as well as celebrate the “lacks” and “gaps” connotations affixed to HCI development in Africa. This blog thereby offers a different reading of the meaning of terms such as limits and scarcity, showing how such terms entail considerable future thinking and planning efforts. Our reflection has more to do with the politics of naming and the power dynamics underpinning the internalization of such narratives in mainstream discourses, and less to do with the practices of design research in African communities where supposedly these limitations are amplified. We draw upon existing decolonial works that explore the politics of computing research within the African context [3,4].
It is our understanding that terms such as resource-challenged, resource-constrained, underserved, and underresourced are used to represent the unfortunate realities of global inequalities. Such terms in HCI discourse mirror a certain worldview of what constitutes scarcity and sufficiency in computing and are often associated with the limited adoption of modern state-of-the-art computing technologies (e.g., smartphones, AI, drones, VR). Our concern with such a way of thinking about Africa is that it unintentionally reproduces interventionalist narratives that suggest how the introduction of technological innovation can bring about economic and material prosperity. The use of the term introduction here is deliberate as it denotes how dominant social imaginaries have cemented the thinking that Africa is not a site of innovation, and that technocratic thinking and modern technologies are the only way out of its predicament. Our frustration here is that the common narrative of lacks and gaps reduces social life in African communities to a set of problems that need computational solutions, a view in which digital technologies are necessary to catch up or transition to Western ideals of progress. This way of thinking thus pushes for the politics of organizing knowledge against a backdrop of consumerist development models that are undesirable and unsustainable.
Additionally, the performativity of limits and scarcity centers certain kinds of utilities over others, thus bringing to the fore economic, infrastructural, and technological products and pushing to the periphery human capital, natural resources, and Indigenous knowledge with which Africa is notably wealthy. Arguably, foregrounding the former repeats simplified accounts of Africa as a historical site for the extraction/production of raw materials and Africans as passive consumers of processed end products. Such accounts of African users invited us to question the extent to which the limits of the computing technologies could have been misrepresented as the limits of its users. Imagine for a moment that, instead of text, early computer interfaces relied heavily on embodied interaction and voice as the main interaction modality; that would have been a better fit with the oral traditions of some African communities [5]. As such, literacies would not have been emphasized as a limitation in the way it’s been presented in mainstream technological discourse.
To provoke different thinking about limitations in Africa, research has shown how folk practices of food preservation were adopted in Cairo as a mechanism to plan and cope with food scarcity [2]. In some districts of old Cairo, ensuring food security involves a considerable amount of planning that must be dynamic and resilient in coping with the changes in the availability of resources (e.g., unfixed income). The number of daily meals, their timing, and the content of meals are planned so that, for instance, planning the biggest meals for dinner guarantees that the person will have enough energy to get through the next day without needing breakfast. The leftover food is used as a supplement for next-day meals and snacks, and recipes are made to keep expensive food items for a long time [2].
Another example of different thinking around limitations is the adoption of the cultural model of Igwebuikem (i.e., strength in numbers) in rebuilding the community wealth of the Biafran communities of Southern Nigeria [6]. Among the Igbo tribe, shared prosperity was considered a political instrument for futuring in the aftermath of the Nigerian civil war. The Biafran civil war brought about the need for building communal solidarity in coping with the dynamic situation of hardship and tribulation—from which the Igbo apprentice systems came about as a scalable entrepreneurship program that built upon the commonwealth of the community [6]. Through its adoption as a localized model for building business ecosystems, communities were able to identify the strength of the collective, learn from each other, and scale-up instruments for rebuilding community wealth. The point raised here is that conditions of scarcity demand situated knowing that does not rely on the limitations of the past but rather on the possibilities that a differentiated thinking about the present might offer to the performance of the future.
From the two historical scenarios of future thinking in the Old Cairo and Biafran communities, one can identify the resourcefulness that accompanies a different reading of limits and scarcity, showing how communities continuously innovate new ways of balancing present living conditions with future ones. By bringing to focus the politics associated with limitation as a concept and a reality, we are seeking to problematize its current use in HCI projects taking place in Africa. Recent efforts in HCI have been critical of technology solutionism, highlighting the nuances surrounding the use of technology as well as the importance of considering contextual and cultural specificities in design (e.g., the work of Kentaro Toyama, author of the seminal book Geek Heresy: Rescuing Social Change from the Cult of Technology). We’re nonetheless concerned with how dominant HCI has framed interaction design from the Global South in relation to development, design, and context, which has led to alternative framings of African design in HCI (e.g., the postcolonial, Indigenous, and decolonial approaches to design). Following decolonial thinking, our reflection sought to elaborate on this concern by bringing to the fore the geopolitics of technology solutionism and the interventionist narrative associated with it as Western ideologies that reproduce the imaginary of primitiveness and backwardness in relation to African innovation, and where postcolonial baggage and the promise of pop development—or the development that doesn’t really develop—remain. We argue that a differentiated identity for African innovation does not require a distinction between the unacknowledged past and the unfortunate present but rather recognizes how the relations between the past and the present give rise to the future.
The message from this short reflection should not be understood as shying away from inequalities and masking struggles resulting from socioeconomic limitations. Rather, we invite ourselves and the HCI community to rethink the “incomplete” realities we might portray and perpetuate when we only focus on and communicate lacks and gaps to inspire research encounters for and with communities from Africa. As African scholars, our hope, by sharing personal reflections, is to encourage HCI researchers to rethink the terms used in research framing and dissemination in light of colonial histories and the realities of globalization. Our call to action is to encourage HCI researchers, ourselves included, to reflect on the limit of the “limitations” described in HCI research, and to exercise humility in articulating the potential mismatch between the users and the technology introduced to them, bearing in mind the context technology was designed from and deployed to, and the worldviews that might be represented.
Endnotes
1. Lazem, S. What are you reading? Shaimaa Lazem. Interactions 26, 5 (2019), 12–13.
2. El Masry, S. Reproduction of Folk Heritage: How Poor Cling to Life in the Context of Scarcity. Cairo Supreme Council of Culture, 2013.
3. Bidwell, N.J. Decolonising HCI and interaction design discourse: Some considerations in planning AfriCHI. XRDS: Crossroads, The ACM Magazine for Students 22, 4 (2016), 22–27.
4. Lazem, S., Giglitto, D., Nkwo, M.S., Mthoko, H., Upani, J., and Peters, A. Challenges and paradoxes in decolonising HCI: A critical discussion. Comput Supported Coop Work 31 (2022), 1–38.
5. Allela, M.A. Technological speculations for African oral storytelling: Implication of creating expressive embodied conversational agents. Proc. of the Second African Conference for Human-Computer Interaction: Thriving Communities. ACM, New York, 2018, 1–4.
6. Kanu, C.C. The context of Igwebuike: What entrepreneurship development systems in Africa can learn from the Igbo apprenticeship system. AMAMIHE Journal of Applied Philosophy 18, 1 (2020).
Posted in:
on Tue, July 05, 2022 - 10:46:44
Muhammad Adamu
Muhammad Adamu is a postdoctoral researcher in ImaginationLancaster, a design-led research lab at Lancaster University. His research focuses on developing approaches to the design and deployment of Indigenous technologies with and for African communities.
[email protected]
View All Muhammad Adamu's Posts
Shaimaa Lazem
Shaimaa Lazem is an Egyptian associate research professor at the City of Scientific Research and Technology Applications (SRTA-City). She completed her Ph.D. in HCI from Virginia Tech. She is interested in HCI in non-Western cultural contexts, participatory design, and decolonizing HCI. She is the cofounder of the Arab-HCI community (
https://arabhci.org).
[email protected]
View All Shaimaa Lazem's Posts
Speech is human and multifaceted. Our approach to studying it should be the same.
Authors:
Arathi Sethumadhavan ,
Joe Garvin,
Benjamin Noah
Posted: Wed, June 22, 2022 - 9:57:27
Whether it’s the friendly virtual assistant in your smart speaker, the auto-generated captions on your YouTube video, or the software that physicians use to dictate clinical notes, voice AI has already become a fixture of modern life. It’s the promise of hands-free convenience: Simply speak naturally, and the computer listens, analyzes, and recognizes what you’re saying. With things like voice-controlled homes and cars on the horizon, our relationship with voice AI looks to only deepen. But the task of building speech recognition technology remains a tall order: We want it to work well for everyone, but there is no one way of speaking.
People speak differently depending on how old they are or where they live on a map but less obvious demographic factors like socioeconomic status and education can also play a role. Layers of intersecting variables all come together to influence how we verbally express ourselves. We humans use language like code-switching chameleons, reflecting and creating culture every time we talk. Speech, in other words, is so much more than a mechanical system of words. It is organic and fluid, forever entangled with identity, values, and self-expression.
So it makes sense that studying speech to improve AI models would be more than just an engineering job. In other words, teams grappling with complex problems involving people require diverse representation across disciplines. Working alongside engineers and data scientists, social science experts like sociologists, anthropologists, and sociolinguists can offer essential insight and context while navigating the intricacies of our many voices.
Lingering inequalities in voice AI
Use of computer speech recognition extends far beyond asking Alexa or Siri to play a song or ask for the weather. Court reporters use it to generate transcriptions; people with disabilities use it to navigate phones and computers; and companies use it to make hiring and firing decisions. As voice AI has proliferated in recent years, overall accuracy and reliability has improved dramatically. But even state-of-the-art speech recognition tech does not work well for everyone.
A Stanford University study from 2020, for example, tested services from major tech companies used for automated transcriptions and found large disparities between ancestry groups. The cause? Insufficient audio data when training the models, the study suggests. In other words, the voice AI powering the services was trained on datasets that left out many ways of speaking. In addition to certain ancestry groups, speech recognition systems also struggle with the accents of non-native English speakers, regional accents, and voices of women.
Biased AI starts with a biased worldview
These divides in voice AI have been documented for years, and so have the data-collection missteps that perpetuate them. Why, then, is collecting enough of the right speech data such a stubborn problem? One factor at play here is the out-group homogeneity effect, the cognitive bias that assumes people unlike you are more similar to each other than they really are. “They are alike; we are diverse,” or so the bias would have you believe.
Especially when classifying language, the bias is insidious. Consider, for instance, how all people over 60 are often lumped together into a single group: “older people.” This broad category might be useful in some contexts, but relying on it when studying speech would be irresponsible. Data shows that the way we talk continues to change as we age into our 60s and beyond, and that it even changes differently depending on gender. Tremendous variation exists within the group “older people” that deserves attention. But if someone isn’t 60 or older themselves, out-group homogeneity bias might blind them to all that variation.
Even the terms commonly used to describe Black and African American language—African American Vernacular English (AAVE), African American English (AAE), or African American Language (AAL)—can themselves be seen as examples of the out-group homogeneity effect. In reality, a language variety is never exclusively part of an entire demographic group, and there could also be people of different ancestry groups who happen to speak similarly. When it comes to studying speech for voice AI, creating false monoliths out of certain subgroups isn’t just inaccurate, it’s dangerous. Glossing over meaningful differences in the ways people speak shuts them out of tomorrow and leaves their voices unheard.
The many nuances of language
Many different factors play into speech. Some might be obvious, like where you live or whether that’s your native language. But other factors like health, education, and even historical migration patterns also play significant roles in shaping how a person speaks. Social factors like these contribute to linguistic variations. Anthropological linguists go a step further, suggesting that these factors also actually construct social meaning. In this view, when someone speaks, their voice is doing so much more than simply reflecting their region or ancestry: It’s expressing an identity.
Our gender identity, for instance, can influence how we talk. Culture or ethnic association can also influence how our speech develops, how we use it, and how it may evolve. When we define a specific variation of speech, therefore, we must include these societal factors as its foundational pillars.
Level of fluency, education, gender identity, age language was learned—which of these many interconnected factors are the most decisive in shaping the ways we speak? It’s crucial information to have, as it quickly becomes unwieldly to account for all possible aspects that determine speech. When collecting samples of spoken language to train a new speech model, for example, there are real-world limitations affecting what can be collected: time, money, personnel, and geography, to name a few. Prioritizing all the social factors is a complex job, one beyond the narrow scope of any one discipline.
A multidisciplinary approach
To build speech recognition technology that works well for everybody, we need to capture the right diversity in our data-collection strategies. This involves turning toward those nuances in language, being attentive and curious about them. We know that we want to capture an accurate picture of the incredible variety of human speech, and we also know that many complex dynamics are at play. This calls for a multidisciplinary approach for a better informed, more inclusive perspective.
An engineer might be able to notice the different word-error rates between demographic groups, for example, but a sociolinguist can help explain the different speaking patterns at play, how these patterns show up across communities, and historical reasons for why they emerged. A data scientist can tell you how many people in what groups need to be sampled. Sociologists, demographers, and anthropologists can speak to social behaviors and psychology, aspects that illuminate the subtleties of language. Domain experts like these offer invaluable insights and context, and involving them early on will help us design better datasets that capture human diversity.
Toward more equitable voice AI
Even with the help of other disciplines, building speech recognition systems is incredibly hard work. Training a voice AI model requires a huge amount of speech data. Collecting this data means bringing in lots of people from different population groups, some of whom are difficult to access and recruit, and some of whom, like Native Americans and the First Nations peoples of Canada, haven’t had their speech studied extensively. And when subjects are finally recruited, they need to be taken to a noise-controlled recording facility, asked to follow specific directions, and instructed to read aloud carefully designed paragraphs of text.
The process of creating voice AI is painstaking and resource-intensive as it is—detecting and reducing bias on top of it all only makes the process more difficult. And yet, we must be up to the task. The fact is that speech recognition systems of today, trained on largely homogenous datasets, still don’t work for all groups of people. This is more than a matter of services performing poorly; it’s a matter of dignifying real ways of speaking, real identities and cultures. We must first acknowledge that this problem exists and educate teams about ways of building more equitable voice AI. Then we need to act. Acknowledging the intricate social and cultural dimensions of speech, we might team up with experts from relevant disciplines. With the help of experts like social scientists, product teams are better equipped to think carefully about inclusive dataset design and to devise creative approaches to thorny data-collection obstacles.
Human speech poses engineering problems that transcend the technical. Building voice AI is in fact a sociotechnical endeavor, one that requires diversity in disciplines. The stakes are high, but with an intentional focus to seek out our blind spots, we can collaborate to build voice AI that truly works for everyone.
Posted in:
on Wed, June 22, 2022 - 9:57:27
Arathi Sethumadhavan
Arathi Sethumadhavan is the head of research for Ethics & Society at Microsoft, where she works at the intersection of research, ethics, and product innovation. She has brought in the perspectives of more than 13,000 people, including traditionally disempowered communities, to help shape ethical development of AI and emerging technologies such as computer vision, NLP, intelligent agents, and mixed reality. She was a recent fellow at the World Economic Forum, where she worked on unlocking opportunities for positive impact with AI, to address the needs of a globally aging population. Prior to joining Microsoft, she worked on creating human-machine systems that enable individuals to be effective in complex environments like aviation and healthcare. She has been cited by the Economist and the American Psychological Association and was included in LightHouse3’s 2022 100 Brilliant Women in AI Ethics list. She has a Ph.D. in experimental psychology with a specialization in human factors from Texas Tech University and an undergraduate degree in computer science.
[email protected]
View All Arathi Sethumadhavan 's Posts
Joe Garvin
Joe Garvin is a writer for Ethics & Society at Microsoft via Allovus, where he helps share research insights with a broader audience. He has previously written for the University of Washington and
City Arts magazine. He has a bachelor’s degree in English literature and a master’s degree in communication with a specialization in digital media.
[email protected]
View All Joe Garvin's Posts
Benjamin Noah
Ben Noah is a senior design researcher for the Ethics & Society group at Microsoft (Cloud & AI), where he supports strategy on responsible development of AI technologies, focusing on the collection of diverse datasets. His previous research experience included modeling cognitive workload using eye-tracking metrics and the design of modern operator control systems for the refinery industry. He has a Ph.D. in industrial engineering with a specialization in human factors from Penn State University, and a bachelor's degree in mechanical engineering from the University of Illinois.
[email protected]
View All Benjamin Noah's Posts
Toward Understanding the Efficacy of Virtual Classrooms Among Bangladeshi Students During Covid-19
Authors:
S.M. Rakibul Islam,
Nabarun Halder,
Ashraful Islam,
Eshtiak Ahmed,
Sheak Rashed Haider Noori
Posted: Fri, June 03, 2022 - 11:41:22
The sudden appearance of Covid-19 forced people to stay at home due its highly infectious nature. Governments shut down countries; economies and businesses collapsed. The education system was no exception; rising Covid-19 cases forced schools to shut down. According to UNESCO, Covid-19 afflicted 37,694,522 students as of January 2022 [1]. Since educational facilities halted in March 2020, Bangladeshi students have been deprived of adequate time for instruction and for connecting with classmates, both of which significantly harm their educational experience. Like many countries, Bangladesh had to move to an online education system [2]. While the online program was able to overcome some of the inevitable difficulties, many people questioned its efficacy and acceptability. Taking this into account, our research looks at the views and opinions of students regarding the efficacy of virtual classrooms and how these classrooms could be an alternative to a traditional, in-person classroom setting.
We used Google Forms to create an online survey about students' perspectives on online education [3]. To develop the survey, we carried out a literature review of related works and conducted informal interviews with students who attended online classes [4]. The survey consisted of both quantitative and qualitative inquiries centered around a set of four types of questions: participants' characteristics, how they attended online classes, their online learning platform experience, and their overall experience. The survey involved 210 students, with educational backgrounds ranging from high school to postgraduate. Of the participants, 88.6 percent were undergraduates and 92.9 percent were between 18 and 25 years old (Table 1). Most participants were from urban areas (63.3 percent). We performed a percentage analysis on the data from the survey using Google Forms' built-in features and Microsoft Excel.
Demographic Information
|
Characteristics
|
Number of
Participants
|
Education level
|
High school/secondary
(Grade 6 to 10)
|
2
|
Higher Secondary (Grade
11 to 12)
|
7
|
Undergraduate
|
186
|
Postgraduate
|
15
|
Gender
|
Female
|
68
|
Male
|
142
|
Age (years)
|
10 to 17
|
3
|
18 to 25
|
195
|
26 to 32
|
11
|
33 to 40
|
1
|
Residence area
|
Suburban
|
34
|
Rural
|
43
|
Urban
|
133
|
Table 1. Demographic information of survey participants.
Poor Internet connectivity was one of the most significant challenges for online classes [5]. Our study reports that nearly half (45.2 percent) of the participants had poor Internet access. Most students used Android smartphones and mobile data for online classes (Table 2). However, a majority of students (84 percent) had no previous experience with online classes. Most of them preferred in-person classes to recorded ones. According to our findings, most participants who did not understand a topic in an online class asked the instructor questions. In contrast, many students who watched recorded videos didn't ask questions. Regarding relationships between classmates, about 80 percent of the respondents felt that online classes created distance between them and their classmates. There was a mixed response to teacher-student interaction.
Technical Information
|
Characteristics
|
Number of
Responses
|
Devices used
|
Tablet computer
|
2
|
iPhone/iOS smartphone
|
14
|
Desktop computer
|
27
|
Laptop computer
|
108
|
Android smartphone
|
161
|
Internet accessibility
|
Public Wi-Fi
|
36
|
Broadband connection
|
103
|
Mobile Internet packages
|
127
|
Platforms used
|
University’s Blackboard
|
1
|
Skype
|
2
|
Microsoft Teams
|
6
|
Facebook Live
|
11
|
Zoom
|
71
|
Google Meet
|
163
|
Communication with new classmates
|
Never communicate with them
|
60
|
Personal message
|
91
|
Instant messengers and social
media groups
|
150
|
Table 2. How participants join online classes and interact with classmates.
Most participants stated that being unable to physically meet with teachers negatively affected their education. Many felt less motivated to discuss any topic in an online class. As a result, students were less eager to discuss any issues with the teacher after class. In terms of changes in instructions in the online class, 50.95 percent of participants expressed "instructors spend less time on guiding students on changes in directions in the online class" and "instructors spend less time explaining topics"; 36.67 percent stated that "instructors take fewer feedbacks"; and 45.24 percent of participants agreed with the following statement: "To demonstrate a topic, instructors use fewer visuals." Just over 41 percent of students were neutral about changes in lecture materials (Figure 1).
![](/images/uploads/blogs/temp_file_Figure1_web5.jpg)
Figure1. Participants’ votes on how lecture materials and directions are changed.![]()
Participants did not have a clear preference about where they took tests, but they had some complaints about the testing process. They stated that open-book exams caused teachers to generate complex tasks that might take longer to solve than the time given for exams, and that teachers could not evaluate their efforts properly. In addition, poor Internet connectivity and the inadequacy of devices were significant barriers to students participating in online tests. The main complaint about online classes and tests was that students could not concentrate for a long period of time. Students also mentioned falling asleep during classes and noisy environments while attending classes at home. In terms of class participation and performance evaluations, most students preferred assessment at the end of the week. Still, many students appreciated checking in for each class; some also mentioned that they enjoyed the assessments at the end of the course or semester. A few of the main complaints from students using online learning systems were that teachers used fewer images to demonstrate concepts, spent less time explaining subjects than before, and took less feedback from students. Approximately 33.3 percent were neutral about learning from online classes, with extremely confident students around 11.9 percent and extremely unconfident students around 17.1 percent (Figure 2).
![](/images/uploads/blogs/temp_file_Figure2_web3.jpg)
Figure 2. Participants’ votes on how lecture materials and directions are changed.
On whether online lessons were advantageous, 31.4 percent students were neutral and 17.1 percent considered online classes extremely unuseful. In comparison, online classes were considered useful by 19.52 percent of students (Figure 3).
![](/images/uploads/blogs/temp_file_Figure3_web2.jpg)
Figure 3. How useful online classes were to participants.
Students understood that the pandemic prevented in-person learning. They recommended that some changes to online education be implemented, including limiting the length of virtual classes, ensuring an uninterrupted Internet connection, and, at a lower cost, utilizing additional visuals or examples to demonstrate a topic, taking immediate feedback after discussing a topic, and guiding students on the necessary steps for the next session of that course/class. Some complications in students' use of online platforms were observed, among them excessive amounts of data transmission due to video lecture distribution, the inability to transmit bigger files to or from the instructor at a lower Internet speed within a shorter and fixed time frame, and inconsistent attendance. The survey findings indicate that the students would be interested in online education if the aforementioned recommendations are heeded and the issues resolved. Despite the limitations indicated by survey participants, it is obvious that an online education system may serve as a viable alternative to traditional classroom education in an emergency such as the Covid-19 pandemic, and that virtual classrooms can minimize disruptions in continuous learning.
Endnotes
1. UNESCO. Education: From disruption to recovery; https://en.unesco.org/covid19/educationresponse
2. Abdullah, M. 2020: The rise of online education. Dhaka Tribune. Dec. 31, 2020; https://www.dhakatribune.com/bangladesh/2020/12/31/2020-rise-of-online-education
3. Halder, N., Islam, S.R., Hosain, M.S., Ahmed, E., Islam, A., and Noori, S.R.H. Efficacy and acceptance of virtual classrooms during covid-19: Bangladesh perspective. Proc. of 3rd International Congress on Human-Computer Interaction, Optimization and Robotic Applications. IEEE, 2021, 1–6.
4. Muthuprasad, T., Aiswarya, S., Aditya, K., and Jha, G.K. Students' perception and preference for online education in India during Covid-19 pandemic. Social Sciences & Humanities Open 3, 1 (2021), 100101.
5. Adnan, M. and Anwar, K. Online learning amid the Covid-19 pandemic: Students' perspectives. Online Submission 2, 1 (2020), 45–51.
Posted in:
on Fri, June 03, 2022 - 11:41:22
S.M. Rakibul Islam
S.M. Rakibul Islam obtained a B.S. in computer science and engineering from Daffodil International University, Bangladesh. His research interests include HCI, user interaction, and applied machine learning.
[email protected]
View All S.M. Rakibul Islam's Posts
Nabarun Halder
Nabarun Halder obtained a B.S. in computer science and engineering from Daffodil International University, Bangladesh. His research interests include HCI, user interaction, and applied machine learning.
[email protected]
View All Nabarun Halder's Posts
Ashraful Islam
Ashraful Islam is a Ph.D. candidate in computer science at the University of Louisiana at Lafayette. His research interests include HCI, mHealth, and user experience. He earned an M.S. in computer science from the same institution.
[email protected]
View All Ashraful Islam's Posts
Eshtiak Ahmed
Eshtiak Ahmed is a Ph.D. student at Tampere University, Finland. His research interests include HCI, social robotics, and user experience. He earned an M.Sc. in human-technology interaction (HTI) from the same institution.
[email protected]
View All Eshtiak Ahmed's Posts
Sheak Rashed Haider Noori
Sheak Rashed Haider Noori is an associate professor of computer science who also serves as associate chair of the Department of Computer Science and Engineering at Daffodil International University, Bangladesh. His research covers HCI, gamification, and design science.
[email protected]
View All Sheak Rashed Haider Noori's Posts
The spirit of scientific communication
Authors:
Ahmet Börütecene ,
Oğuz Buruk
Posted: Mon, February 28, 2022 - 12:46:19
How can I tell what I think till I see what I make and do? — Christopher Frayling [1]
Just a couple of months before the start of the new decade and the pandemic began haunting the world, we had the privilege to attend the Halfway to the Future symposium (https://www.halfwaytothefuture.org/2019/) in person and present a paper. The symposium was organized on the occasion of the 20th anniversary of the Mixed Media Lab at the University of Nottingham. There were many exciting keynotes, presentations, and panels. At that time, we did not know that this would be one of the last physical presentations we would give for a while. In this blog post, we would like to talk about the stage performance we did to present our paper and how it motivated us to open a discussion on alternative ways of communicating scientific knowledge, in particular that which is produced through HCI and design research.
For the symposium, we submitted a provocative paper that envisions using a Ouija Board as a resource for design [2], which was then presented in the session built around the influential work of Bill Gaver on ambiguity [3]. Briefly, the Ouija is a parlor game where participants place their fingers on a physical cursor (planchette) and ask questions to an invisible being—supposedly a spirit. The being moves the cursor—although it is moved unconsciously by humans due to the ideomotor effect [4]—around the letters on the Ouija board to answer the questions. We were excited by how the Ouija mechanism combines ambiguity and human touch in an intriguing way, making it a collective, playful, and evocative medium for ideation. Through its embodied and narrative affordances, the Ouija minimizes verbal communication and focuses participants’ attention on the movement of their hands and the planchette around the letters on the board for a tacit conversation. Using the Ouija was quite experimental for us, and the ambiguity that it embodies was also at the heart of our research process, as we kept discovering new experiences and challenges each time we iterated our Ouija design sessions with different people
When we started thinking about how to present our research, sequential static slides did not look like an ideal way of showing our journey, nor our confusion and excitement about the potential of Ouija for design. Show-and-tell was another alternative. However, although many things can be explained through annotations, the design object, or the object of design, still has a lot more to communicate that cannot be translated into a traditional show-and-tell. In the end, we decided to use the artifact itself to talk about our research. One of the advantages is that our object of research/design had an intrinsic performative aspect and thus could afford a theatrical presence on the stage. We started exploring how to exploit this aspect and use the narrative it could offer.
After playing with the Ouija for a while, we noticed that we did not have to show our real-time interaction with it, but instead could show on a large screen a prerecorded video of us doing the intended actions to make the audience believe that it is actually a live video. This introduced another challenge, though: the requirement to perform on the stage in sync with the prerecorded video. Although challenging, it fit very well the spirit of the symposium: mixed reality. It was a little bit risky, and we did not have much time for rehearsal, but we wanted to give it a shot.
We wrote a vague script for our Ouija performance and started playing along in the hotel room. As we were playing, we were both rehearsing our script and discovering the Ouija board and how to use it for the performance as well as for design. This process not only unfolded the dialogue we intended to create between two people as researchers for presentation purposes, but it also facilitated our dialogue as designers exploring the Ouija board by enabling us to reveal aspects of it that we did not have the chance to notice earlier. So the efforts toward developing a way to communicate our research also became a way to explore the design space further and have a better understanding of it. In the end, the video was ready, and the performance was a success. The audience followed along with the script and caught the key moments clearly.
The point, though, is not the success of our presentation. Also, it is not the first performative presentation made for scientific communication, having been preceded by many others. However, we believe that it is worth discussing the generative impact of such presentations not only on the audience, but also on the presenters, design objects, and the research. There may be a certain value in thinking about how an artifact can be the “subject” doing the scientific communication rather than just being the inert “object” frozen during knowledge creation. This approach configures the interaction with the artifact in a way that may help with the challenges in communicating design knowledge and can be complementary to methods such as annotated portfolios. Moreover, we also had the chance to see that such a performance has an intrinsic value for communicating design knowledge that makes an impact on the audience. It not only helped us engage the audience, but it also provoked research ideas and spontaneous action in the hall. For example, one of the presenters in our panel, Miriam Sturdee, did a sketch during the performance (Figure 1), and later Katherine Isbister asked us if we could place our Ouija board at the corner of the stage to receive questions from the audience during the panel she was moderating. It was also nice to see some parallels to our performance at the symposium. For example, Steve Benford used an old slide projection to show the slides he used years ago to present one of his early works presented at CHI, and then went on with a digital presentation.
![](/images/uploads/blogs/temp_file_miriam_drawing_oguz_web1.jpg)
Figure 1. Sketch made by Miriam Sturdee during the Ouija
performance (used with permission)
Inspired by this experience, we would like to discuss the value of such performances at scientific venues and how they could open up new possibilities to communicate research outcomes as well as to create engagement in different types of audiences. Moreover, we also consider it important to reflect on how such an approach can contribute to authors/designers themselves to have a better understanding of the artifact or topic they are exploring. Instead of thinking of the presentation as a fixed and predictable thing, we might see it as a performance that enables designers, researchers, and audiences to experience the results in the making, as the continuation of a reflective dialogue with the design object. We believe an important aspect of our performance was that it was not a gimmick or special effects. It was a moment where the design object, or the object of design, spoke for itself and this made sense under the symposium theme. Can this kind of approach inspire alternative ways of presenting research processes, disseminating findings, or even reviewing scientific publications? One example toward this direction is the “not paper” [5], which criticizes current publishing formats and practices through its particular embodiment. DIS runs a pictorial track where authors are encouraged to communicate their research through visual-driven papers. Video articles are another example [6]. We would like to hear what design communities have to offer in this direction: Can making multisensory papers be a way of articulating insights addressing different sensory channels, be they haptic, gustatory, auditory, etc.? For example, can we hug or bite the text? Or, why not games as potential presentation formats? Instead of making slides, can we organize a live-action role-playing setting for keynote presentations? How should we go further?
Endnotes
1. Frayling, C. Research in Art and Design. Royal College of Art, London, 1993.
2. Börütecene, A. and Buruk, O. Otherworld: Ouija Board as a resource for design. Proc. of the Halfway to the Future Symposium. ACM, New York, 2019, 1–4; https://doi.org/10.1145/3363384.3363388
3. Gaver, W.W., Beaver, J., and Benford, S. Ambiguity as a resource for design. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2003, 233–240; https://doi.org/10.1145/642611.642653
4. Häberle, A., Schütz-Bosbach, S., Laboissière, R., and Prinz, W. Ideomotor action in cooperative and competitive settings. Social Neuroscience 3, 1 (2008), 26–36. https://doi.org/10.1080/17470910701482205
5. Lindley, J., Sturdee, M., Green, D.P., and Alter, H. This is not a paper: Applying a design research lens to video conferencing, publication formats, eggs… and other things. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2021, 1–6; https://doi.org/10.1145/3411763.3450372
6. Löwgren, J. The need for video in scientific communication. Interactions 18, 1 (2011), 22–25; https://doi.org/10.1145/1897239.1897246
Posted in:
on Mon, February 28, 2022 - 12:46:19
Ahmet Börütecene
Ahmet Börütecene is an assistant professor in the division of Media and Information Technology at Linköping University, Sweden where he is doing design research and teaching, mostly in the field of interaction design and visualization. His current research focus is haptic human-AI interactions, mixed reality experiences, and more-than-human smart cities.
[email protected]
View All Ahmet Börütecene 's Posts
Oğuz Buruk
Oğuz ‘Oz’ Buruk is an industrial product designer who completed his Ph.D. in interaction design at Koç University–Arçelik Research Center for Creative Industries (KUAR). He currently is a senior research fellow in the Gamification Group, Tampere University, Finland. His work focuses on playful bodily technologies, critical design, design fiction, extended reality environments, and human-nature-machine interactions.
[email protected]
View All Oğuz Buruk's Posts
Modifying category icons in smartphone app stores for better understanding: A user-centric approach
Authors:
Eshtiak Ahmed,
Ashraful Islam,
Farzana Anowar
Posted: Fri, February 25, 2022 - 12:30:00
Recent increases in smartphone users have resulted in a great diversity of backgrounds and cultures among these users. Designers of smartphone elements need to address this diversity by making design elements more uniform and acceptable for all. The user interface should address the needs and level of understanding of all users. In this study, we focus on a new approach to the design of user interface icons and investigate how careful design can help users have more seamless interaction with interfaces.
As of the first quarter of 2021, the Google Play Store has the highest number of apps [1]. In the Google Play Store, different app categories are specified by a category name and a simple graphical representation beside the text. Though the text itself is self-explanatory, the graphical representations do not represent the categories well. This may not be a problem for someone who can read English, but it is very hard to grasp the category purely based on the pictorial depiction [2]. It is quite common for people to have trouble finding the app they are looking for [3]. The goal of this research is to see if app categories can be represented more efficiently with better icons or graphical representations next to them.
Initially, new icons were designed for some selected categories. A public survey was designed and circulated where the participants were asked to tell their preferred icons for a specific category. A total of 664 responses were collected. According to the results of the survey, distinguishing factors were identified in how graphical representations can be represented more conveniently. The detailed information and findings of this study can be found in one of our published works [4].
App store category selection
Apps in the Play store are divided into more than 35 categories. The list of categories consists of the category name and a small category icon beside it. These category icons are often confusing and misleading, making them useless at times. However, not all the icons are poor and some of them are quite helpful. For this study, 10 different app categories were selected based on their abstractness and readability: arts and design, business, communication, entertainment, finance, lifestyle, personalization, social, sports, and travel and local.
Icon design
Our goal was to design icons in such a manner that people with low literacy, senior citizens, and kids with special needs can easily understand what they stand for, potentially allowing them to find what they are searching for. While designing the icons, we wanted to represent each category with relevant objects that belongs to that category, for example, representing the sports category with elements from popular sports. In Figure 1, we show the 10 selected categories along with their current icons in the Play store, as well as the icons we have designed for them.
![](/images/uploads/blogs/Fig1_web.jpg)
Figure 1. Side-by-side comparison of existing and newly designed icons.
Survey design and participants
After completing the icon design, a public survey was designed to get feedback from smartphone users about the newly designed icons. For each app store category, two icons (newly designed and existing icons from Google Play store) were put alongside each other, and the participants were asked to decide which icons they preferred for each category. The survey collected a total of 664 responses; 513 (77.3 percent) were male and the rest 151 (24.7 percent) were female. All participants were between the ages of 21 and 25.
Findings
The public survey provided a very good idea of whether the newly designed icons represent the categories better or not. Figure 2 shows the overall survey result comparing the participants’ preference for icons in each of the 10 categories.
The newly designed icon for the arts and design category was preferred by 88 percent of the participants. The new business category icon was preferred by 90.8 percent of participants, while the new icons for the communication, entertainment, and finance categories were preferred by 84.3 percent, 95.5 percent, and 95.9 percent of participants, respectively. The new lifestyle category icon was preferred by 93.1 percent of participants.
Some of the newly designed icons received mixed responses, as they were tough to represent by a small piece of art. These included the newly designed icon for personalization category, which was preferred by 64.8 percent of participants and the new icon for the social category, which was preferred by 71.2 percent of participants. Also, the new sports category icon was preferred by 82.7 percent of participants, while the new travel and local category icon was preferred by 72 percent of participants.
Challenges and discussion
The new icons were designed by using real-world objects from each category. As an example, the sports icon was designed by combining the equipment from three popular sports. However, similar types of categories such as finance, business and communication, and social were challenging because there are fewer distinguishing factors between them. The lifestyle and personalization categories were also very challenging because of their abstractness.
Icons were created using the objects that came to mind for each category. Sometimes, however, the new icons might have seemed too crowded, as so many objects were cramped into a tiny circle to make the icon understandable. Another shortcoming is that the survey participants are mostly young smartphone users. A more thorough survey covering different age groups could create a different dimension of results.
![](/images/uploads/blogs/temp_file_Fig2_web2.jpg)
Figure 2. Analysis of results obtained from the conducted survey.
Conclusion
This study promotes designing an effective textless user interface so users can integrate with their devices more easily and more effectively. Creating impactful and meaningful icons could be a great step toward text-free interface in smartphones. This work could be extended by representing a user interface that is self-explanatory and interactive, opening new doors for challenged users.
Endnotes
1. Statista. Number of apps available in leading app stores. Retrieved August 21, 2021 from https://www.statista.com/topics/1729/app-stores/#dossierSummary__chapter3
2. Wiebe, M., Geiskkovitch, D., Bunt, A., Young, J., and Glenwright, M. Icons for kids: Can young children understand graphical representations of app store categories? Proc. of the 42Nd Graphics Interface Conference. Canadian Human-Computer Communications Society, 2016, 163–166.
3. Wang, M. and Li, X. 2017. Effects of the aesthetic design of icons on app downloads: Evidence from an Android market. Electronic Commerce Research 17, 1 (2017), 83–102.
4. Ahmed, E., Hasan, M.M., Faruk, M.O., Hossain, M.F., Rahman, M.A., and Islam, A. 2019. Icons For the mass: An approach towards text free smart interface. Proc. of the 2019 International Conference on Advances in Science, Engineering and Robotics Technology. 2019, 1–4.
Posted in:
on Fri, February 25, 2022 - 12:30:00
Eshtiak Ahmed
Eshtiak Ahmed is a Ph.D. student at Tampere University, Finland. His research interests include HCI, social robotics, and user experience. He earned an M.Sc. in human-technology interaction (HTI) from the same institution.
[email protected]
View All Eshtiak Ahmed's Posts
Ashraful Islam
Ashraful Islam is a Ph.D. candidate in computer science at the University of Louisiana at Lafayette. His research interests include HCI, mHealth, and user experience. He earned an M.S. in computer science from the same institution.
[email protected]
View All Ashraful Islam's Posts
Farzana Anowar
Tangible XAI
Authors:
Maliheh Ghajargar,
Jeffrey Bardzell,
Allison Smith Renner,
Peter Gall Krogh,
Kristina Höök,
Mikael Wiberg
Posted: Tue, February 15, 2022 - 4:11:15
Computational systems are becoming increasingly smart and automated. Artificial intelligence (AI) systems perceive things in the world, produce content, make decisions for and about us, and serve as emotional companions. From music recommendations to higher-stakes scenarios such as policy decisions, drone-based warfare, and automated driving directions, automated systems affect us all.
But researchers and other experts are asking, How well do we understand this alien intelligence? If even AI developers don’t fully understand how their own neural networks make decisions, what chance does the public have to understand AI outcomes? For example, AI systems decide whether a person should get a loan; so what should—what can—that person understand about how the decision was made? And if we can’t understand it, how can any of us trust AI?
The emerging area of explainable AI (XAI) addresses these issues by helping to disclose how an AI system arrives at its outcomes. But the nature of the disclosure depends in part on the audience, or who needs to understand the AI. A car, for example, can send warnings to consumers (“Tire Pressure Low”) and also send highly technical diagnostic codes that only trained mechanics can understand. Explanation modality is also important to consider. Some people might prefer spoken explanations compared to visual ones. Physical forms afford natural interaction with some smart systems, like vehicles and vacuums, but whether tangible interaction can support AI explanation has not yet been explored.
In the summer of 2020, a group of multidisciplinary researchers collaborated on a studio proposal for the 2021 ACM Tangible Embodied and Embedded (TEI) conference. The basic idea was to link conversations about tangible and embodied interaction and product semantics to XAI. Here, we first describe the background and motivation for the workshop and then report on its outcomes and offer some discussion points.
Self-explanatory or explainable AI?
Explainable AI (XAI) explores mechanisms for demonstrating how and why intelligent systems make predictions or perform actions to help people to understand, trust, and use these systems appropriately [1]. Importantly, people need to know more about systems than just their accuracy or other performance measures; they need to understand more fully how systems make predictions, where they work well, and also their limitations. This understanding helps to increase our trust in AI, ensuring us they are operating without bias. XAI methods have another benefit: supporting people in improving systems by making them more aware of how and when they err. Yet XAI is not always a perfect solution. Trust is particularly hard to calibrate; AI sometimes cannot be trusted; and AI explanations may result in either over- or under-reliance on systems, which might promote manipulations and managerial control.
The notion of explanations or self-explanatory forms has also had a long tradition in product and industrial design disciplines. Product semantics is the study of how people make sense of and understand products through their forms; hence, the way products can be self-explanatory [2]. From that perspective, a product explains its functionality and meaning by its physical forms and context of use alone. For example, the large dial on a car stereo not only communicates that the volume can be controlled, but also how to do so. In this view, products do not need to explain themselves, though their forms need to be understandable in principle.
Using a product-semantic perspective for XAI builds upon the general concerns of XAI, but focuses on the user experience and understandability of a given AI system based on user interpretations of its material and formal properties [3]. The dominant interaction modalities, however, do not fully allow that experience; hence, we have decided to explore tangible embodied interaction (TEI) as a promising interaction modality for that purpose.
From explainable AI to graspable AI: A studio at ACM TEI 2021
At ACM TEI 2021, we organized a studio (similar to a workshop) to map the opportunities that TEI in its broadest sense, including Hiroshi Ishii and Brygg Ullmer’s tangible user interfaces, Paul Dourish’s embodied interaction, Kristina Höök’s somaesthetics, and Mikael Wiberg’s materiality of interaction, can offer to XAI systems.
We used the phrase graspable AI, which deliberately plays with two senses of the word grasp, one referring to taking something into one’s hand and the other to when the mind “grasps” an idea. The term graspable inherently conveys the meaning of being understandable intellectually, meaningfully, and physically.
In doing so, we referenced two earlier HCI concepts. In 1983, Ben Shneiderman coined the term direct manipulation as a way to “offer the satisfying experience of operating on visible objects. The computer becomes transparent, and users can concentrate on their tasks” [4]. In 1997, Ishii and Ullmer envisioned tangible bits, a vision of human-computer interaction that allows users to grasp and manipulate bits by coupling the bits with everyday physical objects and architectural surfaces, which was preceded by George Fitzmaurice’s Ph.D. thesis Graspable User Interfaces in 1996. Their goal was to create a link between digital and physical spaces [5,6,7].
Building on these ideas, we viewed graspable AI as a way to approach XAI from the perspective of tangible and embodied interaction, pointing to a product that is not only explainable but also coherent and accessible in a unified tactile and haptic form [8].
Beyond presentations of position papers, studio participants engaged in group activities, which consisted of three phases (Figure 1). First, each group chose an everyday use AI object or system, analyzed and explored its interactions regarding explainability (what needs to be explained and what the system can explain), then ideated possible tangible interactions with the system and redesigned the human-AI interaction using TEI.
![Figure 1 is showing a diagram describing three phases of the workshop activity, discussions, summary and ideations.](/images/uploads/blogs/temp_file_Figure1_web4.jpg)
Figure 1. TEI studio structure and process.
Graspable designs for everyday use
Studio participants split into groups to explore graspable designs for three distinct AI systems for everyday use: movie recommendations, self-tracking, and robotic cleaning devices.
One group explored graspable design for movie recommendations, asking questions about their explainability and how users can interact with the system through a tangible interface. Some ideas that emerged during the discussion and ideation phases were speculations around using a stress-ball form as a tangible interface to the recommendation system to influence recommendations based on the user’s mood (Figure 2).
![Figure 2 shows an orange stress ball in the hands of a person used as a controller for movie streaming application.](/images/uploads/blogs/temp_file_Figure2_web2.jpg)
Figure 2. Stress ball as a TUI for movie recommendation system.
Another group worked on AI self-tracking devices, discussing questions of how an AI self-tracking device can learn from the user interactions and make us “feel” that our blood sugar is high or low. Provocative ideas, such as an eye that loses its sight gradually and a system that learns and informs the user through haptic feedback, were also explored (Figure 3).
![Figure 3 includes a person who is wearing a self tracking device on their arm and feels the haptic feedback.](/images/uploads/blogs/temp_file_Figure3_web1.jpg)
Figure 3. Tangible AI self-tracking device.
The third group focused on robotic cleaners, where they discussed the opportunities of tangible interactions to inform users about and anticipate robots’ movements. Making the robot’s intended movement visible was another idea that led the group to ideate a tangible map and a map that uses spatial image projections (Figure 4).
![Figure 4 shows a robotic vacuum cleaner and how it interacts with the space for orientations and movements.](/images/uploads/blogs/Figure4_web.jpg)
Figure 4. Robotic vacuum cleaner spatial interactions.
Discussion
Aesthetic accounts often unfold as a back and forth between material particulars and interpreted wholes [3]. Further such accounts consider the difference between what are called explanations in science and interpretations in humanities and design. While explanation is used to find an answer to the why of a phenomena and to reduce its complexity to manageable and understandable units, interpretation is a way to make sense of an experience and the how of the formal and material properties of an artifact within its sociocultural context [3].
AI systems vary in their complexity; some are easier to explain (e.g., movie recommendation systems) while others are more difficult to understand because of their complex inner reasoning (e.g., natural language understanding by deep learning). Accordingly, the notion of graspable AI can not only be concerned with explanations, but also with how AIs are experienced and interpreted through their material particularities within their sociocultural context. However, we do not mean to suggest that graspable AI is a perfect solution to all fundamental humanities concerns! In this workshop, we explored TEI as an explanation modality. We found out that while it has some potential for improving the understandability of the system, it comes with its own challenges. For example, there are features and functionalities of AI systems that do not need to be tangible, or that are understandable as they are. It has been also challenging to design tangible AI systems without falling into usual categories of smart objects.
Hence, we conclude this post discussing a set of related themes and challenges inherent to AI systems that fruitfully may be approached through the notion of graspable: growth, unpredictability, and intentionality.
Graspable growth
AI systems are intended to learn over time as a continuous and internal process based on the data inputs. Hence, an algorithm metaphorically grows over time. As much as an AI system grows, its functionalities and predictions are expected to improve as well, though devolution is as possible as evolution. But humans are not always aware of that process of learning and growing. Probably the most intuitive form of growth for humans are biological growths. We naturally and intuitively can understand when a plant grows new leaves.
We suggest that metaphors of biological growth (and decay) can inspire the design of XAI product semantics to make the AI learning process self-explanatory.
Graspable unpredictability
AI systems are sometimes unpredictable for humans. The decisions they make based on their inner workings and logic may not appear entirely reasonable, fair, or understandable for humans. Further, such systems are ever changing based on input data, especially if they learn over time. How might an AI decrease its apparent unpredictability by revealing its decisions and purposes through tangible and embodied interactions?
For instance, a robotic cleaner maps the area it is cleaning, and its movement decisions are made based on spatial interaction with the environment and continuous learning of the space. Can its mappings and anticipated movements be rendered in a way that humans can grasp? While the stakes could be low for automated consumer vacuum cleaners, implications for powerful robots used in manufacturing and surgery are more serious.
Graspable intentionality
AI systems have their own internal reasoning and logic and they make decisions based on them. Their internal logic and mechanisms make some AI systems appear to be independent in their decision-making process and therefore to have their own mind, intentions, and purposes. Whether or not this is true from a technical perspective, end users often experience it as such, for instance when an algorithmic hiring tool “discriminates.”
At stake is not only the technical question of how to design a more socially responsible AI, but also how AI developers might expose systems’ workings and intentions to humans by means of tangible and embodied interaction.
Conclusions
AI systems are becoming increasingly complex and intelligent. Their complexity and unpredictability pose important questions that are concerned with, on one hand, the degree to which end users actually can control them, and on the other hand, whether and how we can design them in a more transparent and responsible way. In the above we have brought forward some possible challenges and opportunities of using tangible and embodied interaction in designing AI systems that are understandable for human users.
Acknowledgment
We would like to thank workshop participants and our colleagues David Cuartielles and Laurens Boer for their valuable input and participation.
Endnotes
1. Preece, A. Asking ‘why’ in AI: Explainability of intelligent systems – perspectives and challenges. Intelligent Systems in Accounting, Finance and Management 25, 2 (2018), 63–72; https://doi.org/10.1002/isaf.1422
2. Krippendorff, K. and Butter, R. Product semantics: Exploring the symbolic qualities of form. Innovation 3, 2 (1984), 4–9.
3. Bardzell, J. 2011. Interaction criticism: An introduction to the practice. Interacting with Computers 23, 6 (Nov. 2011), 604–621; https://doi.org/10.1016/j.intcom.2011.07.001
4. Shneiderman 1983. Direct manipulation: A step beyond programming languages. Computer 16, 8 (Aug. 1983), 57–69; https://doi.org/10.1109/MC.1983.1654471
5. Fitzmaurice, G., Ishii, H., and Buxton, W. Bricks: Laying the foundations for graspable user interfaces. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., USA, 1995, 442–449; https://doi.org/10.1145/223904.223964
6. Fitzmaurice, G. Graspable User Interfaces. Ph.D. thesis. University of Toronto, 1996; https://www.dgp.toronto.edu/~gf/papers/PhD%20-%20Graspable%20UIs/Thesis.gf.html
7. Ishii, H. and Ullmer, B. Tangible bits: Towards seamless interfaces between people, bits and atoms. Proc. of the ACM SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 1997, 234–241.
8. Ghajargar, M., Bardzell, J., Renner, A.S., Krogh, P.G., Höök, K., Cuartielles, D., Boer, L., and Wiberg, M. From explainable AI to graspable AI. Proc. of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction. ACM, New York, 2021, 1–4.
Posted in:
on Tue, February 15, 2022 - 4:11:15
Maliheh Ghajargar
Maliheh Ghajargar is a design researcher and associate senior lecturer in interaction technologies with a background in industrial design. Her research interests are within the areas of design research and human-AI interaction. Her latest research project, “Graspable AI,” concerns designing human-AI tangible and explainable interactions.
[email protected]
View All Maliheh Ghajargar's Posts
Jeffrey Bardzell
Jeffrey Bardzell is professor of informatics and director of HCI/design in the Luddy School of Informatics, Computing, and Engineering at Indiana University Bloomington. As a leading voice in critical computing and HCI/design research, he has helped to shape research agendas surrounding critical design, design theory and criticism, creativity and innovation, aesthetics, and user experience. He is co-author of
Humanistic HCI (Morgan Claypool, 2015) and co-editor of
Critical Theory and Interaction Design (MIT Press, 2018).
[email protected]
View All Jeffrey Bardzell's Posts
Allison Smith Renner
Alison Smith Renner is a research scientist and engineer with 12-plus years experience designing, building, and evaluating intelligent systems and interactive visualizations for data exploration, analysis, and augmented decision making. Her research lies at the intersection of AI/ML and human-computer interaction, building explainable and interactive AI/ML systems to engender trust, improve performance, and support human-machine collaboration.
[email protected]
View All Allison Smith Renner's Posts
Peter Gall Krogh
Peter Gall Krogh is trained as architect and product designer. He is professor in design and heads the Socio-Technical Design group at the Department of Engineering at Aarhus University.
He contributes to service and interaction design both in doing and theorizing based on co-design techniques with a particular interest in aesthetics, collective action, and proxemics.
[email protected]
View All Peter Gall Krogh's Posts
Kristina Höök
Kristina Höök is a professor in interaction design at the Royal Institute of Technology (KTH). She is a SIGCHI ACM CHI Academy member since 2020, an ACM Distinguished Scientist since 2014, and editor in chief of ToCHI, but most of all fascinated and devoted to developing ideas around interaction design, aesthetics and movement—or, as she frames it: soma design.
[email protected]
View All Kristina Höök's Posts
Mikael Wiberg
Mikael Wiberg is a full professor in informatics at Umeå University, Sweden. Wiberg's main work is within the areas of interactivity, mobility, materiality, and architecture. He is a co-editor in chief of
ACM Interactions, and his most recently published book is
The Materiality of Interaction: Notes on the Materials of Interaction Design (MIT Press, 2018).
[email protected]
View All Mikael Wiberg's Posts
We need to get rid of significance in A/B testing, seriously!
Authors:
Maximilian Speicher
Posted: Tue, February 08, 2022 - 11:20:00
In my previous job, I was responsible for all A/B testing in an e-commerce department. We worked with an external partner and took care of the whole package—from generating hypotheses to developing the tests to providing a report of the results. Those reports were presented to a variety of stakeholders, including product management, merchandise, and the director of e-commerce. To conduct and analyze the tests, we worked with a dedicated experimentation platform that calculated all the numbers per treatment in the traditional, frequentist way. We had the number and share of visitors, values for the different metrics that were measured, a single number for the uplift (or effect size) of the primary metric, and a percentage denoting “confidence.” Our reports displayed more or less the same numbers, but wrapped in a bit of story, with a recap of the test hypothesis and design and recommendations for action. This practice, however, unveiled a number of issues and misunderstandings.
First, it was problematic that we communicated a single number for effect size. This was misunderstood by many stakeholders (and I can’t blame them for that) as the uplift we would definitely make, given that the test was “significant” (see below). But that number denoted only the maximum likelihood estimate. The uplift expected by stakeholders deviated from the true expected value. Sometimes, after implementing a successful test, this manifested in questions such as, “Why don’t we see the 5 percent uplift that we measured during the test in our analytics numbers?”
Second, in tandem with the above estimate, confidence was also often misinterpreted. In our case, it was 1-p, with p being the probability of results at least as extreme as the observed one, if there were no difference between control and treatment. And while this cannot even be interpreted as the chance that there’s any uplift at all [1], many stakeholders mistook it for the actual chance to achieve the reported maximum likelihood estimate (“The test has shown a 96 percent chance of 5 percent uplift”). Truth be told, we didn’t do a very good job of explaining all the numbers—p-values in particular are just so far from being intuitively understandable [1]—so part of the blame was on us. Yet, reporting something along the lines of “We’re confident there’s a chance of uplift, but we can’t exactly tell how much” is also not an option in fast-paced business environments with short attention spans. It’s a catch-22.
Third, and most problematically, there was always the magical and impeccable threshold of “95 percent confidence” or “95 percent significance” (i.e., a significance level of α=.05) in stakeholders’ minds. If a treatment had a confidence of ≥95 percent, it was considered good; otherwise it was considered bad, no further questions asked.
All of the above led to the following decision model when a test was completed:
if (confidence ≥ .95 && uplift) {
implement treatment;
} else {
keep control;
}
And this is just plain wrong.
First of all, one can argue that a significance level of α=.05 is rather arbitrary. After all, economists usually work with α=.1 in their experiments, so why not use that? Still, “95 percent significance” was etched into many of our stakeholders’ minds simply because it’s the most widely used threshold. Now, working with such a relatively tight threshold might rob us of a lot of information, since for a result that falls short of the magical 95 percent confidence, the only thing we can really deduce is that we can’t reject the null hypothesis at the given significance level—and especially not that the control should be kept!—and if we’re above 95 percent, we can’t even reliably communicate what uplift one’s probably going to make. In the else part of the above decision model, we’ve probably discarded many a treatment that would’ve proven good if we hadn’t relied on p-values so much.
To make things worse, at a more fundamental level, a p-value from a single experiment is very much meaningless. Steven Goodman writes, “[T]he operational meaning of a P value less than .05 was merely that one should repeat the experiment. If subsequent studies also yielded significant P values, one could conclude that the observed effects were unlikely to be the result of chance alone. So ‘significance’ is merely that: worthy of attention in the form of meriting more experimentation, but not proof in itself” [2]. Additionally, p-values can vary greatly between repetitions of the same experiment [3,4].
In other words, to a certain degree it’s simply random if a single result lies on this or that side of the magical threshold.
That being said, as Jakob Nielsen explains, there are sometimes very good reasons to implement a treatment, even if it did not achieve a “significant result” [5]. After all, significant or not, there’s always a certain probability for a certain uplift, which has to be weighed against implementation and opportunity costs [5]. But if maximum likelihood estimates and p-values are not suited to communicate that and facilitate informed decisions, what could be a better way? One potential remedy that’s noted by both Steven Goodman [2] and Nassim Taleb [3] is to rely on a Bayesian approach, which has a number of advantages, including interpretability [2,4] and the possibility to report probabilities for minimum uplifts (based on a complementary cumulative distribution function).
So, what did we do? We did some good ole user research and talked to stakeholders about our reports and what they needed to make informed decisions. For all of the reasons stated above, we got rid of the notion of significance altogether. Instead, together with our partner, we started using Bayesian inference to calculate minimum uplifts for certain probabilities. Additionally, using a formula from our controlling department, we translated all the rather abstract conversion rates and average order values into expected added revenue per month. That is, at its core, our reports read, for example, “With a probability of 95 percent, we’ll make ≥10,000 euros a month; with a probability of 90 percent, we’ll make ≥15,000 euros a month,” and so on. Now, without some magical threshold to rely on, our stakeholders had to actively decide whether they deemed the probability of a certain minimum additional revenue high enough to justify implementation of the corresponding treatment. They could calculate an expected ROI and make an informed decision based on that.
I don’t mean to say that we invented something new. There are already plenty of A/B testing tools going the Bayesian way, including Google Optimize, VWO [6], iridion, and Dynamic Yield, to name just a few. And yet, there are some—and many experimenters—who still blindly rely on notions of significance, which just doesn’t make sense. There is no sudden change in the quality of a result solely because it passes an arbitrary threshold [3]; and it is perfectly fine to conclude that in a given situation, 80 percent is a high enough chance to make an additional 500,000 euros a month.
The more I think and read about this topic, the more I’m convinced that p-values and significance are utterly ill-suited tools for A/B testing. They’re difficult to understand, interpret, and communicate; unreliable; prevent perfectly good treatments from being implemented; and there’s a better alternative. Therefore, we should get rid of them.
Endnotes
1. Google. Optimize Resource Hub—for those used to frequentist approaches. 2021; https://support.google.com/optimize/answer/7404625#dont-significance-and-p-values-tell-me&zippy=%2Cin-this-article
2. Goodman, S. A dirty dozen: twelve p-value misconceptions. Seminars in Hematology 45, 3 (2008), 135‒140; http://www.ohri.ca/newsroom/seminars/SeminarUploads/1829%5CSuggested%20Reading%20-%20Nov%203,%202014.pdf
3. Taleb, N.N. A short note on p-value hacking. arXiv preprint arXiv:1603.07532, 2016; https://arxiv.org/pdf/1603.07532.pdf
4. Amrhein, V., Korner-Nievergelt, F., and Roth, T. The earth is flat (p > 0.05): Significance thresholds and the crisis of unreplicable research. PeerJ 5 (2017), e3544; https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5502092/
5. Nielsen, J. Handling insignificance in UX Data [Video]. YouTube, 2021; https://www.youtube.com/watch?v=QkY8bM5bAOA
6. Stucchio, C. Bayesian A/B testing at VWO. Whitepaper. Visual Website Optimizer, 2015; https://www.chrisstucchio.com/pubs/VWO_SmartStats_technical_whitepaper.pdf
Posted in:
on Tue, February 08, 2022 - 11:20:00
Maximilian Speicher
Maximilian Speicher is a computer scientist, designer, researcher, and ringtennis player. Currently, he is director of product design at BestSecret and cofounder of UX consulting firm Jagow Speicher. His research interests lie primarily with novel ways to do digital design, usability evaluation, augmented and virtual reality, and sustainable design.
[email protected]
View All Maximilian Speicher's Posts
Could tech upgrades increase Covid risk? Covid-19 risk in public washrooms
Authors:
Ilona Posner
Posted: Wed, February 02, 2022 - 10:40:43
![Dyson AirBlade Wash + Dry faucet instructions in English and French. Photo by Ilona Posner](/images/uploads/blogs/temp_file_Instructions1_web1.jpg)
Dyson AirBlade Wash + Dry faucet instructions in English and French. Photo by Ilona Posner
Many airports closed drinking fountains for fear of spreading Covid-19. Or perhaps they did it to increase the sale of bottled water. In either case, travelers need to search hard to find free drinking water to refill their bottles after passing through security, before boarding flights.
Zurich airport was one place with easy access to clean and trustworthy Swiss drinking water! On my trip through this airport in November 2021, I was surprised and saddened to see a change.
Previously, this airport had traditional washroom faucets that dispensed water at any temperature and were activated by sensors touch-free. The old faucets in the airport were replaced by the high-tech Dyson AirBlade Wash + Dry all-in-one faucet and hand-dryer device.
So, now instead of being able to get warm water for washing or cold water for drinking, these faucets deliver either warm water or warm air at high-pressure. The user must place their hands directly under the center of the faucet for water, and to the sides for air. These instructions are written in German on stickers attached to the faucets and on trilingual signs in English, German, and French on the mirror in front of the sink (see below). Travelers speaking other languages, or those who are visually impaired, need not bother with these instructions.
![Faucet instructions in English, French, & German on the mirror (left). German only instructions on the faucet handle (right). Photos by Ilona Posner](/images/uploads/blogs/temp_file_Instructions2_web1.jpg)
Faucet instructions in English, French, and German on the mirror (left). German only instructions on the faucet handle (right). Photos by Ilona Posner
Unfortunately, this important information is not at all obvious to the weary travelers passing through the airport! Instead of water, they are often surprised by a blast of air from the faucet. My first attempt at handwashing resulted in me being showered by foamy white soap. The same thing happened to my washroom neighbor. I almost got a picture of the annoyed lady cleaning the wet mess off the shiny black countertop, while swearing in German under her breath!
![Wet counters with Dyson AirBlade Wash & Dry faucets in two different places at the Zurich airport (left) and the Louvre Museum, Paris (right). Photos by Ilona Posner](/images/uploads/blogs/temp_file_wetcounters_web1.jpg)
Wet counters with Dyson AirBlade Wash + Dry faucets at the Zurich airport (left) and the Louvre Museum, Paris (right). Photos by Ilona Posner
In my day job as a user experience consultant, I conduct usability tests of technology to see how real users react. Given this informal two-user usability test, I am concerned about the decreased hygiene at the Zurich airport. Considering all the different types of dirt people might try to wash off in an airport, anytime, not just during a pandemic!
Imagine the horror of a sudden gust of forced-air sending filth flying around the room, to land on unsuspecting passengers. I have traveled with babies and had to clean them in airport washrooms. This difficult task has been made harder by the Dyson AirBlade Wash + Dry.
Ironically, just one week before the Zurich airport, a student shared an image of the Dyson AirBlade Wash + Dry during a discussion of good and bad designs in my UX Design class at the University of Toronto. In the next class, I shared my airport story with this same device.
One more detail worth mentioning is that this change of faucets happened during the pandemic. At this same time as Zurich airport upgraded the faucets to AirBlades Wash + Dry, other airports and public places turned off forced-air hand dryers due to concerns about spreading the coronaviruses with forced air propelled by hand dryers.
!['Gesspert Für Ihre Sicherheit' ('Closed for your safety') sign on a Dyson AirBlade Hand-dryer in Graz, Austria (left). 'Caution, hand dryer out of service. Please use the provided paper towels,' sign at the Lindt Home of Chocolate Museum, Kilchberg, Switzerland (right). Sept 2021. Photos by Ilona Posner](/images/uploads/blogs/temp_file_closed_caution_web1.jpg)
“Gesspert Für Ihre Sicherheit” (“Closed for your safety”) sign on a Dyson AirBlade hand dryer in Graz, Austria (left). "Caution, hand dryer out of service. Please use the provided paper towels" sign at the Lindt Home of Chocolate Museum, Kilchberg, Switzerland, September, 2021 (right). Photos by Ilona Posner
Before concluding, let’s estimate the cost of the rollout for replacing the traditional faucets at Zurich International Airport. Don’t forget that to quench the travelers’ thirst they will also need to add water fountains to replace the functionality lost with temperature controlling faucets. Plus, I don’t know the value of commission earned from the sale of all the hardware. But let’s just look at a rough estimate:
![TABLE: Comparison cost of traditional faucet set up with Dyson AirBlade Wash + Dry.](/images/uploads/blogs/temp_file_Table1.png)
The rough estimate shows about a tenfold difference between the original faucets and the Dyson upgrade.
Unfortunately, some technology upgrades do not improve users’ experience despite the cost.
Post Script: On my return trip home through ZRH Zurich Airport, I again encountered these Dyson faucets. I noticed that when the airflow activated the faucet moved at its base on the sink. Its forced air was so strong that it almost propelled the faucet by blowing it out of the sink. The movement of plumbing components is never a good thing; I can speak from experience having had many plumbing projects in my home. This suggests that the Dyson faucets will require additional maintenance during their lifetime, suggesting an increase to the above cost estimate.
Finally, after I passed security, I was thrilled to find a washroom with traditional faucets. These older designs enabled me to refill my water bottle with cool, clean Swiss tap water that I could safely bring with me on board the plane for the nine-hour trip home. Thanks, ZRH!
![Traditional sinks in an airport washroom.](/images/uploads/blogs/temp_file_trad_faucets_web1.jpg)
Airport washroom with traditional faucets.
Posted in:
on Wed, February 02, 2022 - 10:40:43
Ilona Posner
ILONA POSNER is a user experience consultant and educator. She enjoys encounters with good design and is enraged by bad design which jeopardizes users' experience, health, and safety. ILONA teaches UX design to computer science students at the University of Toronto and tries to help techies develop empathy for their users.
[email protected]
View All Ilona Posner's Posts
Rethinking the Participatory Design Conference experience
Authors:
Rachel Clarke
Posted: Mon, January 31, 2022 - 4:07:02
I read an article recently that suggested with the growth of online and hybrid conferences in response to Covid-19, there were more opportunities for everyone. The focus of the article was on celebrating the enabling potential and design of technology to make this happen. But after spending the past two years attending online conferences and now serving as general co-chair for the 2022 Participatory Design Conference (PDC 2022), I think it’s a mistake to focus on what technology can or cannot bring to the table in trying to make conferences more equitable.
Technology, in all shapes and sizes, offering all sorts of novel interactions for online conferencing and meetings, has been at the front of our brains for many working hours as we hastily integrated and abandoned new apps while juggling alternative ways of working and domestic life. Many conferences took on this challenge recognizing that our reliance on the face-to-face conference model, while benefiting many, also excludes many interesting scholars and practitioners. Extortionate prices for flights, accommodation, and attendance; being away from family; care responsibilities; language; access to existing networks; CO2 emissions—all are key issues that have always existed, but recently became more apparent. In organizing PDC 2022 with co-chairs Yoko Akama and John Vines, we recognized these challenges before the pandemic. We felt there were also other approaches to reconceptualizing what conferences could look like if they were more decentralized, encouraging a greater respect for a diversity of place-based responses to particular themes before working on the technology.
![](/images/uploads/blogs/temp_file_PDC-Places_forWeb1.jpg)
PDC 2022 branding by Ellen McConnell
PDC Places
PDC 2020 was hosted in Colombia, despite the fact this had to quickly become an online-only event. This showed what could be done with a more localized approach to engaging a global network of scholars and practitioners doing and discussing PD while drawing from a strong tradition of political Latin American activists and theorists.
PDC Places is a new initiative for PDC 2022 where distributed face-to-face and hybrid gatherings will take place across 11 different locations around the globe before, during, and after the conference program in August. As general chairs, we were keen to find ways of addressing and reaffirming our commitments to equity and sustainability and a more distributed place-based approach to PD as a diverse set of practices and concerns, beyond Europe and the U.S., despite technically hosting part of the conference in Newcastle, U.K.
We’ve been lucky to have the extraordinary expertise of Reem Talhouk and Andy Dearden leading the way in helping to take this vision forward through a lively network of scholars, practitioners, and industry professionals. We initially received 18 expressions of interest to run PD events around the globe, which created a lot discussion between potential hosts and Places chairs to define how we can practically make this vision happen. We’re now working on consolidating opportunities for meaningful connection across Places hosts including creating a flexible program and branding strategy that can be personalized and work with different languages.
Time zones, language, and facilitation
Some of the ways we are trying to build these connections is carefully planning for time-zone scheduling, multilingual translation, and facilitation. We’ve spent hours mapping time zones, creating “touchpoints” for when different countries can reasonably connect with each other online and without people feeling the need to wake up in the middle of the night. One of the challenges of global online conferences is people not sharing the same time zone. It sounds obvious, but most conferences take place in one time zone that best suits the conference hosts. This makes a lot of sense, but time zone scheduling has enabled us to rethink and plan for running hybrid and online sessions, being mindful of when different countries are likely more able to connect across Europe and Africa, Australasia, and the Americas to avoid the limits of time-zone silos.
We’re planning for a day of hybrid multilingual events to include keynotes and panels. It was our intention to spread these multilingual events throughout the conference, but working with Newcastle University’s School of Modern Languages, our plan is to have several different languages, and focus on one day, due to the complexities of translator scheduling, setup, and costs. We are, however, also hoping that more informal translation can take place. One of our workshop chairs, Angelika Strohmayer, mentioned a conference she had attended recently where people provided more intimate whispered translations encouraging more informal interpretation, which is in the spirit of generosity and the kind of conference atmosphere we hope to create.
It’s also important to recognize the importance of creating connection through careful facilitation between global communities. Nowhere is this more important than in facilitating hybrid online and face-to-face dialogue. Facilitation is definitely a craft not always appreciated, but we’ve learned a lot on how important facilitation and chairing sessions is to making any discussion work well and making the most of our time together, whether this be online, face-to-face, or in a hybrid environment. We’re hoping we can do this in a way that supports panel and paper chairs to embrace this approach providing guidance and sharing practice and experience from the past two years.
Online platform
And finally, yes, the technology. We have started to design a bespoke online platform that bolts together different videoconferencing, scheduling, and networking tools that run on low bandwidth, mindful of places where intermittent online access still presents lags and outages. We have already used these platforms in our lab during large-scale global gatherings including MozFest, IFRC Climate Red, and ServDes at RMIT. We’ll have all the usual things that the PD community have nurtured over the years—presentations for full papers, exploratory papers, situated actions and exhibition, workshops and doctoral colloquia, beyond academia, panels, and awards—but we are also encouraging people to be creative and experiment with hybrid formats. There will also be representation from our PDC Places communities and activity to further bring to life events happening across the globe. There has been so much discussion about how the online experience has diminished the conference experience as a whole. Many of us have felt this, but there are also people doing some really creative things with the technology too, not necessarily creating better experiences than being face-to-face, but aiming to create alternative experiences with what they have to hand.
The technology is important for helping us achieve these things, but we are also very excited to welcome people to Newcastle city itself as part of the conference experience with our unique urban, rural, coastal landscape. Some of our workshops will be hosted at the Urban Sciences Building at Open Lab, and our situated actions and exhibition program will have a combination of online content, short films, live performances and a visual art exhibition. Rather than have a significant number of presentations, though, we will focus on making the most of our time together for focused discussion too where, if people can make it, we will see you in person very soon.
PDC 2022 will take place online from August 19th, with events in Newcastle upon Tyne, U.K. from August 30 to September 1, 2022 (
https://pdc2022.org)
Posted in:
on Mon, January 31, 2022 - 4:07:02
Rachel Clarke
Rachel Clarke is a design researcher and senior lecturer in interaction design at Open Lab, Newcastle University. Her work focuses on the politics of participatory design practice, international development, and smart urbanization in more-than-human worlds.
[email protected]
View All Rachel Clarke's Posts
Black literary technoculture and Afrofuturist rupture
Authors:
Kristen Reynolds
Posted: Wed, October 20, 2021 - 10:31:41
As an epigraph to her incomplete book, Parable of the Trickster, Octavia Butler writes, “[T]here’s nothing new under the sun, but there are new suns.” I take this to be her call for opening ourselves up to those other worlds that are already here, yet we have been conditioned to ignore. To seek them out and challenge ourselves to explore and embody the different modes of being that emerge within them. Butler’s vision of the current world and the other worlds we might one day inhabit is far from utopian in nature. Even her Parables series reveals a possible future in which Butler’s version of a more just, sustainable, and philosophically mutable world thrives in a bubble alongside a white, colonialist world that endures just beyond the trees that encapsulate it. People in this space must constantly negotiate the axis of Afrofuturist possibility and colonialist modernity that structures the world as many of us have come to know it. In these negotiations, Butler grapples with what it means to access otherwise ways of being in a world where technologies of whiteness and coloniality maintain and expand European/American imperial projects globally. How, then, can we use Butler’s call to elucidate the theory and the practice of Black techno past/present/future? A partial answer to this question resides in Black speculative visioning in N. K. Jemisin’s creative works and in Simone Browne’s concept of dark sousveillance.
Jemisin’s work, for example, presents methods that challenge anti-Blackness, resisting the colonial disciplining that limits our vision of possible futures. Her Broken Earth Trilogy begins with a literal earth-shattering rupture—an earthquake with global implications that is brought on by the power of a Black man who desires to set the stage for the creation of a new world. Through this rupture, we are exposed not simply to something new but to a different way of being that has been there all along. Jemisin questions the limits of the world we have been conditioned to accept and explores what might be found when we break away from it to explore a landscape beyond those limits. Because Jemisin’s work also contends with Black world/love/meaning-making amid the backdrop of anti-Black violence, the trilogy can also be read as asking and attempting to answer two questions: What do we know about Blackness and what do we want to know about Blackness? Put another way, what do we know about Blackness beyond the violence and dispossession that white supremacist coloniality foregrounds? What of Blackness exists in excess of these limits? The work of Afrofuturism (and the Black speculative more broadly) offers an answer to Black being in excess of the limitations of the white colonial mind, Black being in and as rupture, Black being beyond white imagination. With this work in mind, we learn not only of Black being in resistance to anti-blackness but also of Black being as the foundation for different worlds and the technologies that might emerge within them. Afrofuturist practice as presented in Butler’s and Jemisin’s fictional worlds give us the tools to think technology otherwise and develop road maps for living alternate technological worlds. They are not promises, but possibilities.
This reading of the philosophies at work in Butler’s and Jemisin’s narratives facilitates my analysis of Black critiques of technology as Afrofuturist practice. Consider, for example, Simone Browne’s discussion of dark sousveillance in Dark Matters. In the introduction, she describes it as “an imaginative place from which to mobilize a critique of racializing surveillance…[one which] plots imaginaries that are oppositional and that are hopeful for another way of being.” She is clear in articulating that “acts that might fall under the rubric of dark sousveillance are not strictly enacted by those who fall under the category of blackness” (emphasis mine). Dark sousveillance as a rupturing Afrofuturist practice centers Blackness and in so doing situates Black folks as the architects of possible, otherwise techno-futures. In these futures, all people who are subjected to the dehumanizing gaze of white epistemologies and surveillance technologies find opportunities for realizing alternative ways of being. Put another way, Afrofuturism is a Black epistemic project generating worlds that counter and/or erode surveillance as we know it. In this worldmaking, we are privileged to encounter Blackness as both a foundation to and entry point into other relations of seeing and being seen.
The ideal project and practice of Afrofuturism resides in part in how it transforms innovation and traces its lineages. How does Harriet Jacobs’s decision to steal away to an attic—away from the sight of those who might harm her—anticipate Black practices of hiding away in the present moment? How are Black organizers’ and activists’ practices of thwarting today’s conventions of surveillance (phone, video, social media) both rooted in a long history of Black counter/antisurveillance and generative of otherwise ways of seeing and being seen (or not)? These issues are of course much more complex than presented here, but it remains that these Afrofuturist ruptures, these innovations, are happening around us daily. It is incumbent upon us all to see them. Doing so requires that Blackness be seen as the transformative worldmaking force that it is. Afrofuturism gives us that.
Posted in:
on Wed, October 20, 2021 - 10:31:41
Kristen Reynolds
Kristen Reynolds is a Ph.D. student in American Studies at the University of Minnesota. Her research explores the liberatory aspects of Black speculative fiction, particularly engaging with the ways in which it allows us to posit alternative techno-futures that explore realities beyond the limiting and oppressive systems that define our presents and pasts.
[email protected]
View All Kristen Reynolds's Posts
Where is the transparency of the new ACM Violations Database?
Authors:
Angelika Strohmayer
Posted: Thu, September 23, 2021 - 10:28:16
After a summer of discussions and action about racism in our community in 2020 (see, e.g., [1]), the spring of 2021 raised further concerns about harassment and oppression. Long before this eventful year, however, I and many others have had both private and somewhat public conversations about the need to reckon with the trauma in our community, whether it is due to sexual harassment, bullying, racism, ableism, sexism, or any other host of harms that are experienced and perpetrated on a daily basis (see, e.g., [2,3]).
As shown in some recent Interactions articles and publications, we appear to be starting to take these concerns more seriously as a community. I have seen articles that start to ask questions about what institutional and informal systems and practices we need to be able to handle the trauma, and how we can reduce experiences of harm. The recently announced ACM Violations Database (published in a blog post for our SIGCHI community on May 25, 2021: https://sigchi.org/2021/05/the-new-acm-violations-database/) is one way of grappling with these harms. But for it to function in our SIGCHI community, which is part of the ACM, we need clear, transparent communication channels and community engagement with the database, as well as conversations about how, why, and when it is used. I, and I’m sure many others, have questions about the system, its functions, and its uses.
With this blog post, I want to start a public conversation about the violations database and how, or even whether, it addresses harms in our community. I feel it is important to establish that I am writing this piece because I have not received adequate, or in some cases any, answers to questions I have raised about the database. I will start by sharing some of my initial thoughts on the transparency, or rather the lack thereof, of the development, use, and communication of and around this new system. I hope that through public discussion, the community that makes SIGCHI, and the ACM, the prestigious body that it is, will be listened to in the continued development of the database.
I welcome the violations database as a real effort to address and change conditions, but do not agree with how it is being shared, communicated, or developed. The SIGCHI blog post introducing it included the following: “If you have further questions regarding ACM policies, please contact the person indicated at the end of this piece. For questions about the ACM’s Violations Database, please contact [email protected]. Questions regarding the SIGCHI process can be sent to [email protected].”
As a researcher who has done work with sex-work peer-alerting systems in the U.K. [4] and Canada [5], especially having looked in detail at the architecture, trust, social, and political contexts in which these databases sit; as a SIGCHI member who has experienced abuse and harassment from members of our community; and as a person who works to change our academic cultures around power and abuse, of course I had questions.
I drafted an email outlining questions I had about the use of the system in relation to SIGCHI and sent it to the address mentioned in the blog post. In the email, I tried to find a balance between 1) being supportive of this initiative and how it puts into practice some of the things that were outlined years earlier in the ACM Harassment Policy and the ACM Code of Ethics and Professional Conduct; and 2) asking critical questions about the use of the database, based on my personal and research expertise. My questions included ones about the methodologies of the database’s creation, the role of the advocate mentioned in the post, the decision-making processes behind the database, and how its sustainable use was envisioned at the SIGCHI level.
Some of my questions were partially answered, but most of them were not. Instead, I was directed to the ACM policies mentioned in the blog post, without an indication of where in the policies I should look, and encouraged to email Vicki Hanson, CEO of the ACM. I mustered up my courage and fed my healthy disregard for authority and sent another email.
Perhaps the most disappointing response in this whole exchange did not relate to a question, but rather to my offer of support as an expert who has done research for multiple years with databases that can be called upon to see whether someone has previously been reported for having perpetrated harms. My expertise was not acknowledged, nor was there any understanding of my knowledge as someone who herself had previously experienced abuse. Instead, I received this response: “I should note that the ACM has significant resources including their own lawyers who inform all they do.” This is just one more example of how SIGCHI and the ACM do not appreciate community-driven initiatives [6].
In my opinion, a violations database is not something that should be developed solely by lawyers. It should predominately be driven by our community and the expertise that is within it; it needs to be embedded in our politics and our social structures, and, most importantly, it should center survivors and their experiences—not those who cause and perpetuate the harm. A violations database should not be developed by the same people in an organization who have repeatedly avoided difficult conversations about harms that are experienced by their members and “seem to repeatedly move in a more inclusive direction only to undermine such efforts” [6].
I find it inexplicable that the ACM should leave me and fellow members in the dark about why, how, and when the violations database was produced. But it is even more inexplicable that the (now former) SIGCHI president wasn’t informed either, as was shared with me in an email on May 30, 2021: “Honestly, as of a little over a month ago, I didn’t even know this database was coming.” If neither members nor the executive committees of the SIGs that make up the ACM were informed of this development, whom is the violations database meant to be for?
So what happens now?
Since my initial questions, I have had conversations with others, realizing there are of course many more questions that remain unanswered. But since I had not received an answer from the ACM CEO to any of my questions at the time of writing this piece—I emailed her on May 31 and followed up on June 11 and the 29th and on July 19—and since the named SIGCHI contact person for questions on the blog post was unable to provide adequate responses to many of my questions, I have also been angry about the lack of transparency—the disregard for open communication channels, or even basic information about the system.
It should not be up to me or any other SIGCHI member to contact the CEO of the ACM to get basic information about this new system that could greatly affect, both positively and negatively, how we exist in our community and at our events. I should not require the courage to email the most powerful people in “the world's largest educational and scientific computing society” [7] to learn the most basic information about this new system. Conversely, I would also argue that it should not be up to the CEO of the ACM to have to answer basic questions about such a system. This whole experience makes me wonder who is in charge of the ACM Violations Database and its uses if there is no point of contact for questions about the system.
To understand how the ACM Violations Database could function in our SIGCHI community, we need more information about it. SIGCHI members and others in the community need space to discuss its use and many potential misuses. We need more details, more context, more understanding. We need responsive infrastructures in place through which we can ask questions and have conversations about harms and violations. None of these currently exist.
This blog post is part of my work on understanding the new system and how we could make something like it work to improve safety at SIGCHI events. After talking to friends and colleagues (some of whom do research on violence and technologies; some of whom are on organizing committees of SIGCHI conferences), I decided it was important to start a public conversation about the violations database. I thank the Interactions blog editors for giving me space to air my concerns. I hope to see others join this public conversation by centering those who have experienced violence, who have gone through ACM, SIGCHI, or other institutional complaints processes, and those who are experts on related topics.
There is so much trauma in our community, some of which has been caused by others in our community as well as the hierarchies and infrastructures of power that govern SIGCHI and the ACM. Finding ways of addressing this harm through structural change such as the violations database are very welcome, at least by me. But for the database to work, we must have information on its intentions and uses; we must have space to ask questions and receive answers; we must center those who have experienced trauma in how we hold space for and reduce opportunities of harm in the future.
Out of professional courtesy, Interactions sent my article to Vicki Hanson prior to publication.
Professor Hanson informed Interactions that she had had technical difficulties with receiving my e-mails. Soon after, on August 24, she was able to answer my questions, and on September 2, we spoke about the violations database, its relation to the complaints process within ACM, the transparency of communication channels between ACM, SIGs, and members in relation to this development, as well as opportunities for change to current processes and systems. I’m thankful to have had this constructive conversation, and look forward to continuing to work with others on issues related to the complaints procedure and violations database.
Endnotes
1. Grady, S.D., Wisniewski, P., Metoyer, R.,Gibbs, P., Badillo-Urquiola, K., Elsayed-Ali, S., Yafi, E. Addressing institutional racism within initiatives for SIGCHI’s diversity and inclusion. Interactions blog. Jun. 11, 2020; https://interactions.acm.org/blog/view/addressing-institutional-racism-within-initiatives-for-sigchis-diversity-an
2. fempower.tech. An open letter to first-time attendees. fempower.tech. May 7, 2018; https://fempower.tech/2018/05/07/chi-2018-an-open-letter-to-first-time-attendants/
3. fempower.tech. An open letter to the CHI community. fempower.tech. May 7, 2018; https://fempower.tech/2018/05/07/chi-2018-an-open-letter-to-the-chi-community/
4. Strohmayer, A., Laing, M., and Comber, R. Technologies and social justice outcomes in sex work charities: Fighting stigma, saving lives. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2017.
5. Strohmayer, A., Clamen, J., and Laing, M. Technologies for social justice: Lessons from sex workers at the front lines. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2019.
6. fempower.tech. A call for respect, inclusion, fairness, and transparency in SIGCHI. Interactions blog. Dec. 3, 2020; https://interactions.acm.org/blog/view/a-call-for-respect-inclusion-fairness-and-transparency-in-sigchi
7. https://www.acm.org/about-acm/about-the-acm-organization
Posted in:
on Thu, September 23, 2021 - 10:28:16
Angelika Strohmayer
Angelika Strohmayer is an interdisciplinary technology researcher, working closely with third sector organizations and other stakeholders to creatively integrate digital technologies in service delivery and advocacy work. She aims to work collaboratively on in-the-world projects that engage people at all stages of the research process to engender change toward a more just world.
[email protected]
View All Angelika Strohmayer's Posts
Artificial intelligence is changing the surveillance landscape
Authors:
Arathi Sethumadhavan ,
Esra Bakkalbasioglu
Posted: Wed, June 30, 2021 - 12:46:38
The following is a review of recent publications on the issue of AI and surveillance and does not reflect Microsoft's opinion on the topic.
The term surveillance is derived from sur, which means from above, and veillance, which means to watch. Theoretical approaches to surveillance can be traced back to the 18th century with Bentham’s prison-panopticon. The panopticon premise involved a guard in a central tower watching over the inmates. The “omnipresence” of the guard was expected to deter the prisoners from transgression and encourage them to self-discipline. Today, the declining costs of surveillance hardware and software coupled with increased data storage capabilities with cloud computing, have lowered the financial and practical barriers to surveil a large population with ease. As of 2019, there were an estimated 770 million public surveillance cameras around the world, and the number is expected to reach 1 billion this year.
During the Covid-19 pandemic, we have been witnessing a massive global increase in surveillance as well. For example, technologies are used to allow people to check if they have come in contact with other Covid-19 patients, map people's movements, and track whether quarantined individuals have left their homes. While such surveillance technologies can help combat the pandemic, they also introduce the risk of normalizing surveillance.
China is home to the top surveilled cities in the world
Of the top 20 most surveilled cites in the world (based on the number of cameras per 1000 people), 16 of them are in China. Outside of China, London and the Indian cities of Indore, Hyderabad, and Delhi are estimated to be the most surveilled cities in the world.
![](/images/uploads/blogs/most_surveilled_Table_web.jpg)
The 20 most surveilled cities in the world (cameras per person). Source: Comparitech.
AI-based surveillance technologies are being deployed globally at an unfathomable rate
Surveillance technologies are being deployed at an extremely fast rate globally. In fact, as of 2019, at least 75 countries were using various AI technologies such as smart policing systems, facial recognition, and smart city platforms for surveillance purposes. Every nation has a unique approach to surveillance shaped by its technological landscape and economic power as well as its social, legal, and political systems. As of 2019, China has installed more than 200 million facial recognition-powered cameras for various purposes, ranging from identifying thefts and finding fugitives, spotting jaywalkers, designing targeted advertisements, and detecting inappropriate behaviors in classrooms. In India, the Internet Future Foundation has estimated 42 ongoing facial recognition projects, with at least 19 systems expected to be used by state-level police departments and the National Crime Records Bureau for surveilling Indian citizens. Earlier this year, the Delhi police used facial recognition technology to track down suspects allegedly involved in violent clashes during the farmers’ tractor march in the nation’s capital.
![](/images/uploads/blogs/temp_file_Map1.jpeg)
AI Surveillance Technology Origin. Source: Carnegie Endowment For International Peace
Growing deployment in various sectors criticized by civil liberties advocacy groups
The growing reach and affordability have led to increased use of AI-based surveillance technologies in law enforcement, retail stores, schools, and corporate workplaces. Several of these applications are fraught with criticism. For example, several police departments in the U.S are believed to have misused facial recognition systems by relying on erroneous inputs such as celebrity photos, computer-generated faces, or artist sketches. The stakes are too high to rely on unreliable or wrong inputs. Similarly, this year, news of Lucknow police in Northern India wanting to use AI-enabled cameras that can read expressions of distress on women's faces when subjected to harassment was met with backlash, with civil rights advocates noting how it could violate women’s privacy and exacerbate the situation. Further, the scientific basis behind the use of AI to read “distress” was deemed unsound. In the education sector, a few schools in the U.S. are relying on facial recognition systems to identify suspended students and staff members as well as other threats. Civil liberties groups argue that in these scenarios there is a lack of evidence that there is a positive correlation between the use of the technology and the desired outcome (e.g., increased safety, increased productivity). Further, critics also contend that the investigation of petty crimes does not justify the use of surveillance technologies, including the creation of a massive facial recognition database.
Surveillance technologies raise public concerns
Widespread use of new surveillance technologies has posed valid privacy concerns among the general public, in several nations. For example, a survey conducted by the Pew Research Center revealed that more than 80 percent of Americans believe that they have zero-to-very-little control over their personal data collected by companies (81 percent) and government agencies (84 percent). A 2019 survey conducted in Britain showed that the public is willing to accept facial recognition technology when there are clear benefits but want their government to impose restrictions on the use of the technology. Further, the desire to opt-out of the technology was higher for individuals belonging to ethnic minority groups, who were concerned about the unethical use of the technology. These findings demonstrate the need to involve the public early and often in the design of such technologies.
Need for legal frameworks and industry participation to address public concerns
Recently, the European Commission introduced a proposal for a legal framework to regulate the use of AI technologies. As part of this, they have proposed banning the use of real-time remote biometric identification systems by law enforcement, except for a limited number of uses like missing children, terrorist attacks, and serious crimes. While the U.S. does not have federal laws regulating AI surveillance, some cities have taken restrictive measures around the use of such technologies by law enforcement. For example, San Francisco became the first U.S. city to ban the use of facial recognition by local agencies, followed by Somerville, Oakland, Berkeley, and Brookline. A number of companies, including Microsoft, have called for the creation of new rules, including a federal law in the U.S. grounded in human rights, to ensure the responsible use of technologies like facial recognition.
Posted in:
on Wed, June 30, 2021 - 12:46:38
Arathi Sethumadhavan
Arathi Sethumadhavan is the head of research for Ethics & Society at Microsoft, where she works at the intersection of research, ethics, and product innovation. She has brought in the perspectives of more than 13,000 people, including traditionally disempowered communities, to help shape ethical development of AI and emerging technologies such as computer vision, NLP, intelligent agents, and mixed reality. She was a recent fellow at the World Economic Forum, where she worked on unlocking opportunities for positive impact with AI, to address the needs of a globally aging population. Prior to joining Microsoft, she worked on creating human-machine systems that enable individuals to be effective in complex environments like aviation and healthcare. She has been cited by the Economist and the American Psychological Association and was included in LightHouse3’s 2022 100 Brilliant Women in AI Ethics list. She has a Ph.D. in experimental psychology with a specialization in human factors from Texas Tech University and an undergraduate degree in computer science.
[email protected]
View All Arathi Sethumadhavan 's Posts
Esra Bakkalbasioglu
Esra Bakkalbasioglu is a design researcher at Microsoft, focused on AI ethics. Her recent research includes developing disclosure mechanisms in different domains. She has a Ph.D. in interdisciplinary social sciences from the University of Washington.
[email protected]
View All Esra Bakkalbasioglu's Posts
The labor behind the tools: Using design thinking methods to examine content moderation software
Authors:
Caroline Sinders,
Sana Ahmad
Posted: Thu, June 24, 2021 - 10:02:38
Content moderation is widely known to be hidden from the public view, often leaving the discourse bereft of operational knowledge about social media platforms. Media and scholarly articles have shed light on the asymmetrical processes of creating content policies for social media and their resulting impact on the rights of marginalized communities. Some of this work, including documentaries, investigative journalism, academic research, and the accounts of individual whistleblowers, has also uncovered the practices of moderating user-generated content on social media platforms such as YouTube, Facebook, and TikTok. In doing so, the complex and hidden outsourcing relationships between social media companies and third-party companies—often located in different geographical areas—have been made visible. Most importantly, the identification of these companies and their outsourcing practices has brought to public attention the secretive work processes and working conditions of content moderators.
In this article we illustrate the unique method we undertook to examine the software used in the content moderation process. In December 2020, we held two focus-group research workshops with 10 content moderators. Our participants included former and current employees of a third-party IT services company in Berlin that supplies content moderation services to a social media monopoly based in the U.S. Many participants in our study were immigrants, most of whom had lived in Germany for less than five years. For this group, a lack of German-language skills meant fewer employment opportunities, which led them to apply for content moderation jobs. (As this article focuses on the methodological part of our research workshops, we will not elaborate on the recruitment and work process of the participants.)
The content moderation process is highly confidential; employees are forbidden from describing the work they do and how they do it. When crafting these workshops, we were inspired by collective memory practices. The workshops allowed us to use design thinking exercises to uncover and gauge the infrastructural design, user interface, and user experience design of the systems the participants worked with. While design thinking has a broad meaning and definition, it can be used to problem solve and to create new software or designs; we reverse engineered the process and used exercises designed to help frame questions and translate ideas into tangible interfaces and architectural layouts. From her time as a UX designer in industry, Caroline observed that product designers often lean toward building a specific or concrete “thing,” be it a new product, product augmentation, or process when using design thinking exercises. Our goal was to ground the participants in the space of making. By asking people about their day, the hardships they face, the structure of their workday, and how they iteratively approach their work, a foundation is created to think through building something that would support that work. These exercises helped ground the moderators in the logic of how their software functions and spark memories of the software they use. By working with multiple content moderators in a workshop setting, we created a space of organic reflection and comparison of the tools and protocols they used, leading to discussions on how the Berlin-based employer and the social media client managed the work. In holding our workshops, we followed necessary research protocols, informed by the research ethics standards of the WZB Berlin Social Science Center.
The workshop was divided into five exercises, with each exercise iterating off the previous one. It began with participants writing out a workflow of their day, such as the first thing they see when they log in, what do they do after logging in, where their tasks are stored, and how they engage in tasks. Activities belonging to the first, second, and third exercises were aimed at deciphering the layouts, workflows, and design. Activities four and five had a group discussion format that invited participants to share their experiences of workplace surveillance and propose work-related changes and potential improvements.
Designing the Workshops
Ensuring the privacy of content moderators was integral to this project. Considering the company’s secretive work practices, we were aware of the potential threats to moderators’ job security; therefore, they were not requested to provide screenshots of their work. As an additional precautionary measure, the wireframes have been redrawn for this article. The workshops were held using the audio communication function of a Web-based, open-source software, with appropriate attention given to anonymizing the participants.
Apart from privacy considerations, there were other challenges. To examine the infrastructural design, UX, and UI of the software, we needed to guide participants through the process to draw the software they used for moderating content. Design is a specific medium with a language and vernacular of its own. Content moderators may not know the names of or how to describe the elements in the software they use. Therefore, we designed the flow of activities to accommodate this constraint and guide the moderators through an ideation process that could ensure their participation. Our approach was to create exercises that slowly, organically, and iteratively helped the participants sketch the software they used.
Workshop Structure
Both workshops began with presentations by the organizers. Sana introduced the existing studies on the labor process of content moderation and the importance of undertaking this research project, followed by Caroline, who explained basic design elements that we assumed the participants would see in their own software. The main question guiding the workshops was: How can we examine the design of the content moderation software through the experiential inputs of the workers while protecting them? With the collective knowledge of our organizing team—including Sana’s academic research on the labor process of content moderation in third-party IT BPO companies in India and Caroline’s design background and expertise in designing digital tools and software—we aimed to answer the research question in a multidisciplinary manner. The practical knowledge of two student assistants at the WZB Berlin Social Science Center further benefited the workshops.
As mentioned earlier, the indispensable aspect of the workshops was their iterative structure; knowledge about the content moderation software was developed gradually, using a step-by-step process wherein participants could build out, think, and iterate on the software design they were recalling. This gave them space to reflect and redraw for accuracy. Each activity started with general questions on the content moderation process, including the generic layout of the software and the moderators’ workflows, which became increasingly specified in the process of adding complexities in each subsequent workshop activity. Much like generating an outline, we could then go more granular, asking about smaller, more specific features and adding more-qualitative questions on the kinds of content they moderated, how their company organized their workday, and how their team used their software. Such a sequential technique allowed the participants to ease into the design thinking process.
The first activity was planned to obtain an undetailed view of the content moderators’ daily workflows and routines. Accordingly, the participants were asked to list the daily sequences of their work. Through this, we could first determine the type of software used, for example, whether or not it was Web-based. Other elements that could be assessed from the participants’ experiences included their observations while logging in to start the moderation work, the first task they undertook after logging in, and the software functions available, including those related to internal communication (e.g., the ability to send emails to management or messages to team members or other coworkers). This overview was important in getting a preliminary grasp on how workers accessed content moderation tasks and whether they were able to exercise flexibility between carrying out work-related tasks and other assignments.
The activity was useful in persuading the participants to revisit their daily work routine and recall the essential elements of their work process. It also gave them a basis for remembering the different kinds of screens and windows in their software.
Building on the first, the second activity allowed the participants to further iterate and lay out the different screens and states of their work software, pulling directly from the initial list they had made. The list helped the participants look back at what each “state” or “screen” held and the kind of actions each screen allowed for.
The third exercise was focused specifically on the software page layout and gaining insights into the granular elements. During this activity, the participants started drawing and building out a rough wireframe, with focus placed on elements such as menus and buttons and locating different information in distinct pages and positions in the software. Along with their illustrations, participants also shared with us the experiential narratives of the content moderation process, including the management control embedded in the software. Our questions related to these themes spurred responses from the participants.
![](/images/uploads/blogs/Image1.jpg)
![](/images/uploads/blogs/Image2.jpg)
Wireframes from activity 3 in the second workshop, where participants detailed the software page layout.
The workshops then proceeded to the final activities, which invited participants to engage in group discussions. The fourth activity picked up on the themes of workplace surveillance and monitoring strategies by management. This included examining participants’ views on the surveillance technologies possibly embedded in the content moderation software. The fifth and final activity drew on participants’ understanding of the content moderation process and the ways in which they imagined work could be made better for moderators. In doing so, they were also provoked to think about the probability of machine-learning tools being used in their work, and whether these could potentially affect their job security.
Conclusion We see the merits of this research method, especially in being able to draw out human-machine interaction through the collective memories of our participants. Our workshops yielded software design layouts, which are unique given the limited information available on the labor processes of social media content moderation. At the same time, conducting a workshop or focus group interviews can be more fruitful when combined with one-on-one interviews. Considering the precarious background of our participants and their limited possibilities for exercising collective struggles against management, we managed to create a space where current and former content moderators were able to share their work-related experiences and management interactions with us and with one another by focusing on the content moderation software. Future research on the use of technical control and novel ways of labor resistance using technology can enrich the existing research on content moderation.
Posted in:
on Thu, June 24, 2021 - 10:02:38
Caroline Sinders
Caroline Sinders is a machine learning designer/user researcher, artist, and digital anthropologist examining the intersections of natural language processing, artificial intelligence, abuse, online harassment, and politics in digital conversational spaces. She is the founder of Convocation Design + Research and has worked with organizations such as Amnesty International, Intel, IBM Watson, and the Wikimedia Foundation.
[email protected]
View All Caroline Sinders's Posts
Sana Ahmad
Sana Ahmad is a doctoral candidate at the Freie Universität Berlin writing her thesis on the outsourced practices of content moderation in India. Her project is aimed at understanding the outsourcing relationships between global social media companies and the supplier companies based in India. In doing so, she looks at the content moderation process and the working conditions of Indian content moderators.
[email protected]
View All Sana Ahmad's Posts
Covid-19, education, and challenges for learning-technology adoption in Pakistan
Authors:
Muhammad Zahid Iqbal,
Abraham Campbell
Posted: Fri, February 05, 2021 - 2:14:12
![](/images/uploads/blogs/Keyword-Cloud-map2_forWeb.jpg)
Creating educational disruption everywhere, the Covid-19 pandemic has hindered the lives of students and, sadly, will probably have a lasting impact on their future academic lives. What has gone relatively unnoticed is that it has created far more difficulties in developing countries. This is due to the fact that these countries were already lacking in internet accessibility, e-learning solutions providers, and government policies for developing localized education technology, as well as personal resources among students. In managing the Covid-19 crisis better than many countries, Pakistan avoided the need for a full lockdown and set an example for the world. Its savvy policies even kept the economy running. Despite its proximity to neighbors China (where the first Covid-19 infection was found) and India (the second-most-affected country), Pakistan is surprisingly safe when compared with Europe and the U.S., with around a 98 percent recovery rate.
Educational technology has been rapidly advancing—smartphones, tablets, augmented and virtual reality, and high-speed Internet, 4G, and 5G connectivity. All of this makes online learning more productive, adaptive, and accessible. In fact, the e-learning industry is currently valued at more than $200 billion and is expected to top $375 billion by 2026. Even so, Pakistan has some of the world’s worst educational outcomes. For example, it has the world’s second-highest number of children not in school: 22.8 million children ages 5 to16, which is 40 percent of Pakistan’s school-age children.
In an unfortunate twist, the onset of the pandemic coincided with Pakistan’s struggle to implement a uniform curriculum across all provinces. As coronavirus control measures spread throughout South Asia, departments of education and higher-level universities found themselves poorly or, in most cases, completely unprepared for online learning and delivering distance learning. In the past, Pakistan had closed educational institutes due to terrorist attacks and political threats, but there was still no official policy around online education.
Pakistan has an emerging mobile-phone-user market—currently 75 percent of the population uses mobile technology. But out of a population of 220 million—the fifth most-populous country in the world—there are only 76.38 million internet users. That’s only 35 percent of the population, with only 17 percent using social media. Facts on the ground show that accessibility to the Internet is the major hurdle to adopting an e-learning system. Resistance to adopting technology or new learning pedagogy and being used to the classroom environment also play a negative role in resistance to online learning policy.
E-learning initiatives in Pakistan and future prospects
When schools were forced to shut, Pakistan started seeking alternative solutions to the globally adopted “suspending classes without stopping learning” policy [1]. One idea was the establishment of a national TV channel for education that would provide equal educational opportunities for all students. The channel programmed content for kindergarten through high school and provided one lesson per day to each grade, so students would have to watch in shifts. Also, during the second wave of Covid, Radio Pakistan started transmitting “radio school” to promote virtual learning in the country for primary-level students, as a part of an effort toward overcoming the digital divide.
In the higher-education sector, Virtual University is at the forefront of virtual learning, providing full-time online learning courses, from bachelor's to Ph.D. level, in different subjects. As the pandemic disrupted education, Microsoft Teams were deployed in Pakistani universities to build connection between students and teachers. Previously, Microsoft and the Citizens Foundation (TCF) collaborated to provide technology-based education in underdeveloped areas. The eLearn Punjab program has generated educational content based on videos and illustrations for primary and secondary school classes. And in tackling the digital divide in gender, The Malala Fund has investigated Covid as an amplifying factor for the girls’ education crisis in Pakistan.
![](/images/uploads/blogs/temp_file_Figure1_forWeb1.jpg)
Figure 1. Challenges and possible solutions for the educational landscape of Pakistan.
Lessons learned from the pandemic can be used as an opportunity to redesign learning spaces and restructure the curriculum to facilitate student learning, as shown in Figure 1. This abrupt wake-up call should prompt all relevant stakeholders to reflect on the true purpose of schools and the future of learning in this country.
In Pakistani institutes, there is a lack of technically trained teachers to run online classes smoothly. To strengthen blended, distance, and online learning, there is a need to provide more awareness and accessibility to MOOCs, Coursera, and EdX. There is also a need to develop innovative, immersive learning technologies and modern education spaces using virtual, augmented, and mixed reality technology [2]. These technologies, along with the use of artificial intelligence (AI) and machine learning (ML), can change the future of learning by helping us build more interactive, personalized, and productive learning solutions. More specifically, when we talk about practical, hands-on learning in STEM, where there is an urgent need for learning material, augmented reality can provide virtual material to help teach with the kinesthetic learning approach [3].
Technologically developed countries have innovative and advanced systems for e-learning, allowing them to stay in the loop and keep the learning flow active during this academic year. But in Pakistan, online learning is at a nascent stage. Having started as emergency remote learning, it needs further investment to create more adoption and overcome limitations. Along with the establishing the Internet in remote areas, developing specialized authoring tools, and creating awareness for getting the most out of online learning, faculty need training to use online modalities and innovative pedagogies to reduce cognitive load and increase interactivity. In his book 21 Lessons for the 21st Century [4], Yuval Noah Harari highlighted a drawback of the current state of education:The focus is on traditional academic skills rather than critical thinking and adaptability, which are needed to create opportunities and success in the future [2]. This critical period, which is moving us rapidly toward the adoption of e-learning, can spark more focus on providing Internet facilities in remote areas, developing more innovative, low-cost learning solutions, and creating more adaptive and effective methods of learning in the near future.
Endnotes
1. Zhang, W. et al. Suspending classes without stopping learning: China’s education emergency management policy in the Covid-19 outbreak. J. Risk Financial Manag. 13, 3 (2020), 55; https://doi.org/10.3390/jrfm13030055
2. Campbell, A.G. et al. Future mixed reality educational spaces. Proc. of 2016 Future Technologies Conference. IEEE, 2016.
3. Köse, H. and Güner-Yildiz, N. Augmented reality (AR) as a learning material in special needs education. Education and Information Technologies (2020), 1-16.
4. Harari, Y.N. 21 Lessons for the 21st Century. Random House, 2018.
Posted in:
Covid-19 on Fri, February 05, 2021 - 2:14:12
Muhammad Zahid Iqbal
Muhammad Zahid Iqbal is a Ph.D. researcher in the School of Computer Science, University College Dublin, Ireland. His research interests are human-computer interaction, augmented reality in education, touchless interactions technologies, artificial intelligence, and e-learning. He is alumni of the Heidelberg Laureate Forum.
[email protected]
View All Muhammad Zahid Iqbal's Posts
Abraham Campbell
Abraham G. Campbell is an assistant professor at University College Dublin (UCD), Ireland, who is currently teaching as part of Beijing-Dublin International College (BJUT), a joint initiative between UCD and BJUT
[email protected]
View All Abraham Campbell's Posts
Technology use and vulnerability among seniors in Sweden during Covid
Authors:
Linnea Öhlund
Posted: Wed, January 20, 2021 - 2:20:00
From its place of origin, Wuhan, China, the coronavirus has spread worldwide and infected millions of people. Countries have adopted various strategies to curb the spread of the virus, social isolation being one. Sweden is one of few countries that has never closed down entirely but instead has relied on individuals' own responsibility in taking adequate precautions and following recommendations. A part of the strategy has been to recommend that individuals over the age of 70 physically isolate and not have any unnecessary social contact with others to protect themselves from the virus. Despite the many criticisms of the Swedish Covid-19 strategy, many seniors living at home have followed the recommendations of isolating themselves from any non-vital social and physical interaction.
A common way of thinking of this group is that they don't use, don’t want to use, or can't use technology, and that if they simply could use it, many aspects of their life would be improved [1]. This is nowadays somewhat of a preconception, but a study from 2020 found that 87 percent of Swedish people age 66 and up use the Internet. The current global situation of Covid-19 presents a golden opportunity to find out if seniors are using digital technology in ways that help them face negative consequences of isolation, and, more generally, if they have started using technology in a more socially connecting way. With these thoughts in mind, I set out to interview 15 people over the age of 70 living in Sweden. In this blog post, I present a summary of the results of those discussions and discuss how participant demographics, vulnerability, and not using or overusing technology can all be factors that play into the results.
Participant demographics
The ages of the participants were between 69 and 81. Half of them were married, living with a partner in a house or big apartment, and about half were single and lived in an apartment. Participants were contacted through email (due to Covid), which told me in advance that they most likely owned some digital artifact and have enough knowledge to navigate it. They were all young enough to have been working with computers and technology for a substantial part of their working life, which has provided them with knowledge and experience. Most of the participants had a university education. This information tells us that they come from a particular socio-economic background, which means that they have previous knowledge to understand digital technology and presumed capital to buy it. I present the results in the following three categories; the demographics of the participants will be a resurfacing pattern.
Feeling vulnerable in society
During the interviews, despite no question being directly asked about vulnerability, many felt that in Swedish society today, seniors are not treated nicely. Especially now during Covid, many thought they have been labeled vulnerable in a negative sense, and that other age groups, but predominately younger individuals (15 to 35), would be disrespectful and view them with contempt. They felt that the heavy restrictions for individuals 70 and over had not only protected them but also stigmatized them further.
Technology use
I was a little surprised to find that all of the seniors in the study used digital technology daily. They all had smartphones, computers, tablets, and multiple apps such as Facebook, digital banks, Wordfeud, and newspapers. Although not all of them felt particularly skilled or even interested in technology, all of them had a certain type of confidence regarding technology that little previous research mentions. After some discussions, the participants admitted that they played Wordfeud longer, scrolled Facebook more, and made more video calls, but many said they didn't do this more than normal.
Experiencing further negative feelings by using technology
All participants used technology, but some still felt socially isolated and lonely. In some cases, participants even felt that using technology made them feel sadder and more isolated because they would remember how life was before. Most of the participants also mentioned family or friends who did not use technology and felt left out of society. Many felt frustrated that digital change is happening at such a fast pace in Sweden. This quick change is casting a shadow over a specific part of society, mostly related to age. Some individuals do not have the experience from their work-life that allows them to use technology, nor do they have the means to buy technology. According to some of the participants, the individuals who cannot or will not use technology are therefore left out from many options that would have given them a better quality of life.
Discussion
Even though technology can serve as a tool to connect with friends and family, using it does not automatically give you a socially isolated, happy life, free of negative feelings. Many seniors already use technology to a large extent. Still, in periods of isolation, negative feelings seem to increase. Using technology to attempt to curb these negative feelings already seems to be done by many seniors. But overusing technology for longer periods also appears to render more negative emotions, because it is forced upon them and not chosen. Furthermore, in Swedish society, despite many seniors being tech-savvy, many are left out because they do not use digital technology, which means not having the same opportunities for a higher quality of life as others.
This study and the results from it can be further summarized in four takeaways related to seniors as a vulnerable group, their demographics, and their use of technology when trying to create a better quality of life for themselves. These four points may provide further insight into seniors as a vulnerable group and specifically this type of demographic:
- The Covid crisis has meant that seniors have faced further stigmatization in society. They feel like they are being pointed out as a problem and not respected because they may face many negative consequences from Covid-19.
- Many seniors under the age of 80 have extensive knowledge of digital technologies and systems and use them daily without critical challenges.
- The usage of technology has gone up slightly during this Covid period to make up for the lost contact with others, but feelings of loss and sadness remain.
- Overusing technology can create further negative feelings, but not using digital technology at all means being left out of society and missing out on opportunities that could generate a better quality of life. The reasons for not using technology today may be linked to not having enough experience to understand it or enough money to buy it.
Endnotes
1. Khosravi, P., Rezvani, A., and Wiewiora, A. The impact of technology on older adults’ social isolation. Computers in Human Behavior 63, (2016), 594–603; https://doi.org/10.1016/j.chb.2016.05.092
Posted in:
Covid-19 on Wed, January 20, 2021 - 2:20:00
Linnea Öhlund
Utopian futures for sexuality, aging, and design
Authors:
Britta Schulte,
Marie Søndergaard,
Rens Brankaert,
Kellie Morrissey
Posted: Tue, January 19, 2021 - 1:35:54
![](/images/uploads/blogs/AgeSexTech_Pic_Story_cr.jpg)
This excerpt is from a letter to a future self, a story written at a workshop held at DIS 2020 [1], where participants reframed and reimagined what intimacy might mean for the aging body and what role technology might play. Aging and the changes to the body it brings with it are often portrayed as something negative, a time of loss and fading away. Images of older bodies are rarely publicized or celebrated; in fact, old age is more often expressed through a (black and white) image of a hand placed on a shoulder. Although initiatives such as #nomorewrinklyhands try to make this lazy messaging visible as well as counter this stereotype, the prevailing societal fear of growing and appearing older means that— deliberate or not—we can tend to erase images of bodies that are engaged in processes of aging. However, when we ignore the aging body, we also erase experiences such as menopause and the changes—positive and negative—that this period brings for people undergoing it. When ignoring the body, we erase close intimate practices that are part of caregiving, including bathing, dressing, and close physical support. When ignoring the body, we erase experiences of intimacy and sexuality and the important part they play in our well-being.
Increasingly, HCI practitioners problematize and critique the way we, as a field, frame and address aging and the aging body [2,3]. This framing impacts the ways we configure and design new technologies that address (or ignore) the well-being of older people [4]. While the community is growing in this respect, many research and design projects in HCI still often steer away from intimacy and sexuality, choosing instead to focus on how older participants can gain satisfaction through family or civic life instead [5,6]. Although there is a healthy body of work in HCI on innovating surveillance technology or memory aids for older people, HCI has similarly balked in engaging with the “bodywork” that is also necessary in aged care (and which is often left to women carers, or to low-paid care workers). Elsewhere, however, intimacy is a reoccurring topic for HCI research (see, e.g., [7] for an overview). And beyond this, as a young, cross-disciplinary field, HCI often expands the border of “what can be talked about”: Here, sexuality is often a subject of design research, exploring technology through the lenses of sex toys [8,9], and kink [10], while others explore the role sexuality could play in HCI research through workshops at top-level conferences [11,12]. But it can be argued that there is a comparable pattern here: Through these publications and projects, sexuality is framed at least implicitly as a prerogative of the young and able-bodied.
Organized as part of the ACM Designing Interactive Systems conference in 2020, our workshop, titled “Don’t Blush: Ageing, Sexuality & Design,” explored the (dis)comforts surrounding the potential role of technology in sexuality and intimacy in later life [1]. Bringing together researchers and practitioners from many disciplines, we particularly wanted to imagine speculative, positive futures of aging to help us visualize a sex-positive society. The workshop relied heavily on storytelling as a means to approach, communicate, and ground this sensitive topic in a manner that all felt comfortable with. In a first step, we used three stories—two descriptive of current scenarios (drawn from current practice), one speculative— to kickstart the discussion and familiarize ourselves with the different themes this topic might encompass. The stories described 1) the experiences of a couple not able to find space together privately in a care home, 2) experiences of nursing staff with a “Mr. Carter” and his unsatisfied urges, and 3) the story of a couple who asked the salesman of a robobutler company if the robot could provide support in the bedroom. These initial discussions were useful to find common ground and as a tool to develop a shared vocabulary on intimacy and sexuality. Participants were able to raise the topics important to them, outline blindspots, and relate the stories to their own areas of expertise, as well as to the things they found interesting from the other participants’ workshop submissions. After extensive discussions, smaller groups collaborated on writing a new story. The only direction provided was that the stories were to be optimistic, maybe even utopian, so that, instead of lamenting the status quo, we could develop visions of what we want to see.
The four resulting stories exceeded our expectations of what could be possible in such a short amount of time, as they were subtle, sensitive, and sensual accounts that allowed us to collectively think the “unthinkable.” Drawing on the variety of experiences the participants brought to the table, the stories covered a range of experiences, including: 1) enriching bodily experiences through lingerie that is suitable for people using incontinence pads, 2) sensitive, tailored sexual care package subscriptions openly advertised in care homes, 3) love letters to the body and self-care rituals surrounding menarche and menopause, and 4) a sex-positive older blogger embraced by her family.
![](/images/uploads/blogs/AgeSexTech_Pic_ThreePines_cr.jpg)
This illustration was developed alongside story 2, illustrating the different sex packages available in the fictional future care home.
Even though these stories were planned out as utopian stories, they are inherently grounded in the everyday, the mundane, and the values of the writers. Drawing on the body as a focal point has enabled the authors of these stories to discuss societal changes not in the abstract, but rather on a personal, embodied level. Even though all stories describe deeply personal encounters, they all link to changes beyond these experiences, which are articulated through the artifacts that the stories’ protagonists use. The erotic lingerie that Jo—a nonbinary character—uses to change their body (image) is not only their own imagination: Jo also reflects how it has been shared widely through Instagram adverts, showing that society has made space for conversations around desire, changing bodies, and incontinence to happen. The care home presented in story 2 advertises the sexuality packages they developed openly, hinting at a whole history of conversations, changes, and decisions that took place beforehand. Through the family members, who are somewhat uncomfortable about the idea, we get a hint as to how far society has adapted to it.
As with every good workshop, “Don’t Blush” left us with more questions than answers. But we are convinced that these questions are useful for us as a field to move toward technologies that are truly supportive of the lived experience of older bodies. We tried to summarize some of them here to stimulate discussion within the field.
If we acknowledge that older adults might have or wish for an active sex life, what does that mean for the technologies we develop in this area? How can we ensure that the technologies we create support the joy, dignity, and privacy that we would allow everyone else? How do we develop research and design strategies that approach the question in suitable, sensitive, and satisfying ways?
If we acknowledge that good care in older age means caring for the body as well as ensuring people’s basic health and safety, how can we extend our understanding of intimacy, privacy, and dignity to improve and enrich often-ignored body work (bathing, dressing, and toileting)?
If we acknowledge the aging body and the changes it goes through, what directions does this open up for us to explore through our research and design work? How can we make space for the body in our research and keep an open dialogue about experiences, staying with the (dis)comfort of such conversations?
We hope to keep this conversation going and growing. All organizers, as well as most participants, came from a Western European perspective, with a strong focus on the U.K. and the Netherlands. Both the stories that inspired the work and those that came out of the workshop embody a certain understanding of sexuality, intimacy, as well as aging and caregiving. The majorities of the stories focus on a female perspective, which again mirrors the composition of the workshop participants. Even though not always explicitly, most stories further present a heterosexual outlook. While this is a limitation of the current work, it is also an explicit invitation to build on these stories, contradict them, and expand them. In addition, the experiences and wishes of people in their later lives themselves are missing here. We are planning to respond to this by using the stories developed in the workshop as conversation starters and other ways of qualitative and co-design research with aging people. We will further distil the insights and stories from the workshop into a zine to be shared within the academic and non-academic audience. You can get a copy by contacting [email protected]. If you want to be part of this conversation, please get in touch or join the conversation at #agesextech.
Endnotes
1. Schulte, B.F., Morrissey, K., Juul Søndergaard, M.L., and Brankaert, R. Don’t blush: Sexuality, aging & design. Companion Publication of the 2020 ACM Designing Interactive Systems Conference. ACM, New York, 2020, 405–408; https://doi.org/10.1145/3393914.3395915
2. Kannabiran, G., Hoggan, E., and Hansen, L.K. Somehow they are never horny! Companion Publication of the 2020 ACM Designing Interactive Systems Conference. ACM, New York, 2020, 131–137; https://doi.org/10.1145/3393914.3395877
3. Schulte, B. and Hornecker, E. Full frontal intimacy - on HCI, design, & intimacy. Companion Publication of the 2020 ACM Designing Interactive Systems Conference. ACM, New York, 2020, 123–129; https://doi.org/10.1145/3393914.3395889
4. Vines, J., Pritchard, G., Wright, P., Olivier, P., and Brittain, K. An age-old problem: Examining the discourses of ageing in HCI and strategies for future research. ACM Transactions on Computer-Human Interaction 22, 1 (2015), 2.
5. Reuter, A., Bartindale, T., Morrissey, K., Scharf, T., and Liddle, J. Older voices: Supporting community radio production for civic participation in later life. Proc. of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2019, 1–13; https://doi.org/10.1145/3290605.3300664
6. Welsh, D., Morrissey, K., Foley, S., McNaney, R., Salis, C., McCarthy, J., and Vines, J. Ticket to talk: Supporting conversation between young people and people with dementia through digital media. Proc. of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2018, 1–14; https://doi.org/10.1145/3173574.3173949
7. Hassenzahl, M., Heidecker, S., Eckoldt, K., Diefenbach, S., and Hillmann, U. All you need is love: Current strategies of mediating intimate relationships through technology. ACM Trans. Comput.-Hum. Interact. 19, 4 (Dec. 2012), 1–19; https://doi.org/10.1145/2395131.2395137
8. Bardzell, J. and Bardzell, S. Pleasure is your birthright: Digitally enabled designer sex toys as a case of third-wave HCI. Proc. of the 2011 Annual Conference on Human Factors in Computing Systems. ACM Press, New York, 2011, 257; https://doi.org/10.1145/1978942.1978979
9. Juul Søndergaard, M.L., and Hedegaard Schiølin, K. Bataille’s bicycle: Execution and/as eroticism. Executing Practices (2017), 179.
10. Buttrick, L., Linehan, C., Kirman, B., and O’Hara, D. Fifty shades of CHI: The perverse and humiliating human-computer relationship. Proc. of the Extended Abstracts of the 32nd annual ACM Conference on Human Factors in Computing Systems. ACM Press, New York, 2014, 825–834; https://doi.org/10.1145/2559206.2578874
11. Brewer, J., Kaye, J., Williams, A., and Wyche, S. Sexual interactions: Shy we should talk about sex in HCI. CHI ’06 Extended Abstracts on Human Factors in Computing Systems. ACM Press, New York, 2006, 1695; https://doi.org/10.1145/1125451.1125765
12. Kannabiran, G., Ahmed, A.A., Wood, M., Balaam, M., Tanenbaum, T.J., Bardzell, S., and Bardzell, J. Design for sexual wellbeing in HCI. Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2018, 1–7; https://doi.org/10.1145/3170427.3170639
Posted in:
on Tue, January 19, 2021 - 1:35:54
Britta Schulte
Britta F. Schulte is a postdoc at Bauhaus University Weimar. Her work explores our relationships toward technologies for elderly care and the ageing body, with a strong focus on intimacy and sexuality. In her works she often uses speculative and creative approaches such as storytelling and design fictions in many forms.
[email protected]
View All Britta Schulte's Posts
Marie Søndergaard
Marie Louise Juul Søndergaard is an interaction designer and design researcher. Her work explores critical-feminist design of digital technologies for intimate health, such as menarche, menopause, and sexual pleasure. She is currently a postdoc in interaction design and digital women’s health at KTH Institute of Technology, Stockholm, Sweden.
[email protected]
View All Marie Søndergaard's Posts
Rens Brankaert
Rens Brankaert is professor of health innovations and technology at Fontys University of Applied sciences and assistant professor at Eindhoven University of Technology (TU/e). His work focuses around the design of technology, systems, and services for and with people living with dementia by using design research and living lab approaches.
[email protected]
View All Rens Brankaert's Posts
Kellie Morrissey
Kellie Morrissey is a lecturer in design for health and wellbeing at the School of Design in the University of Limerick, Ireland. Her work focuses on experience-centered and phenomenological approaches to the co-design of digital technologies for and with marginalized participants.
[email protected]
View All Kellie Morrissey's Posts
Cultivating activism with speculative design
Authors:
Richmond Wong,
Nick Merrill
Posted: Fri, December 18, 2020 - 4:29:36
As design researchers, we love our speculative methods—methods for imagining possible futures—and opening them up to discussion and critique. But what good do they do? Designing speculative futures to discuss values, ethics, safety, and security can feel naive, as fellow researchers are being dismissed for doing the work of ethics.
We want to believe that to imagine possible futures is to be able to change them: to surface discussions of social values and ethics so that we may imagine worlds to work toward (or avoid). But, as prior work has observed, who gets to speculate matters a great deal [1]. Of course, scholarly production of speculative artifacts has its place. But can it make change—lasting change—on the ground?
Our past work has used speculative designs—creating fictional products, headlines, and scenarios for others to react to and play within—to surface discussion and consideration of values, ethics, and alternative notions of security or safety [2,3]. We envisioned these as techniques that could be adopted within existing product design practices.
In a (perhaps subtle) shift, we discuss the role speculative methods may have in fostering activism and dissent, particularly among so-called rank-and-file tech workers such as designers, UX professionals, and engineers.
This concept is not without precedent. Turkopticon, a platform for organizing Mechanical Turk workers, has created lasting infrastructures for workers mobilizing on the platform [4]. Prior work has also involved activists in speculative practices; Asad et al. had activists produce prototypes that expressed their particular needs [5]. In some of our own prior work, we describe infrastructural speculations: a call to use speculative design techniques to center systems of power and imagine alternative ones in our speculative work—questions that are often relegated to the background when speculating about technology and use [6].
We’re motivated by the desire to produce critically oriented practices that can become part of a lasting infrastructure among tech workers—a practice for critiquing technology as common as think-alouds or user personas are for building them. In the midst of widespread public skepticism of technology companies, and a fair share of tech-worker-led dissent and activism (via letter writing, walkouts, and other forms of organizing), there is an opportunity to identify, describe, and discuss points of dissent and refusal of “business as usual.” Speculative methods allow us to imagine, construct, and communicate alternative social relationships and configurations of power. So a critical, speculative method could, with groundwork, become an industry-wide practice for fostering such dissent.
Yet, as Timnit Gebru’s recent dismissal from Google, and broader dismissals of activist-workers (many of whom are PoC, women, trans, and nonbinary) across technology companies illustrate, fostering dissent among tech workers requires more than new speculative techniques. It requires social and organizational change; it requires solidarity among workers. Even if someone comes armed with worthy critique, without worker organizing, their analyses can be met with outright hostility.
We are excited to see the development of new tools and methods for surfacing questions related to values, ethics, bias, and more, often combining speculative methods with approaches such as design fiction, value sensitive design, or participatory design. But many of these interventions—including our own, at times—have abstracted away social issues crucial to the potential adoption and use of these tools: questions of workplace power, the precarity and risk involved in organizing or critiquing, and who carries the burden of that precarity. Our work, which centers structures of economic power and capital, has not engaged deeply enough on how these forces shape the adoption of our practices.
What can speculative practices do for activism? We approach this question humbly. Design, even with a critical orientation, cannot “solve” technology’s problems without touching the social and political structures within which these technologies, and their development, are entangled. Speculative design alone will not save us. Simply raising conversations will not necessarily lead to change. Without an underlying political commitment, we risk that speculative work gets re-appropriated by the systems we attempt to critique [7]. Worse, we risk ignoring the hard groundwork already done by activists, union organizers, and people working in local communities to advocate for more fair, just technical practices.
As we look toward our future work with these practices, we ask ourselves: What pragmatic and tactical work can speculative practices do today to help workers, activists, educators, and organizers already working on the ground to achieve their goals? And to help people who are beginning to ask critical questions become more inclined toward activism?
Our new challenge is to use speculative design to create methods difficult for corporations to co-opt, perhaps methods that take place outside of the corporate world. Even the dystopian visions of speculative methods are seen by some as the next disruptive product.
Toward these ends, and building on the work of colleagues and co-conspirators, we suggest:
- Changing when/where speculative design is done. Deploy speculative design outside of work contexts. While user-centered design methods take place in contexts of work, speculative methods for critique should take place in contexts of organizing and activism.
- Changing with/for whom speculative work is done. Create speculative designs with and for more targeted activist audiences, rather than defaulting to sharing them broadly for general public discussion. Activists are one audience. But speculative work can also make the comfortable, such as the C-suite, uncomfortable. These audiences should not be ignored either.
- Changing what speculations are about. Shift speculative designs away from easy-to-reappropriate imagined products toward depicting futures through other forms
This process is easier described than enacted. Making the methods, then mobilizing them, takes significant work, and academics will need to work with activists on the ground. Our goal in sharing these reflections is to inspire students, researchers, and practitioners to join in doing that work; to expand and (re)orient speculative methods to further justice and activism, joining existing critical perspectives on design methods [8]. Speculative methods have the capacity to inspire meaningful change, meaningful dissent. We hope our critical self-reflection will spark interest in building reusable, dare we say dangerous methods for fostering activism and dissent. We hope these questions will help our community build them.
* Both authors contributed equally to this piece.
Endnotes
1. O’Leary, J.T., Zewde, S., Mankoff, J., and Rosner, D.K. Who gets to future? Race, representation, and design methods in Africatown. Proc. of the 2019 CHI Conference on Human Factors in Computing Systems, 1–13; https://doi.org/10.1145/3290605.3300791
2. Merrill, N. 2020. Security fictions: Bridging speculative design and computer security. Proc. of the 2020 ACM Designing Interactive Systems Conference, 1727–1735; https://doi.org/10.1145/3357236.3395451
3. Wong, R.Y., Mulligan, D.K., Van Wyk, E., Pierce, J., and Chuang, J. Eliciting values reflections by engaging privacy futures using design workbooks. Proc. of the ACM on Human Computer Interaction 1, CSCW. 2017; https://doi.org/10.1145/3134746
4. Irani, L. and Silberman, M.S. 2014. From critical design to critical infrastructure. Interactions 21, 4 (2014), 32–35; https://doi.org/10.1145/2627392
5. Asad, M., Fox, S., and Le Dantec, C.A. Speculative activist technologies. Proc. of iConference 2014; https://doi.org/10.9776/14074
6. Wong, R.Y., Khovanskaya, V., Fox, S.E., Merrill, N., and Sengers, P. Infrastructural speculations: Tactics for designing and interrogating lifeworlds. Proc. of the 2020 CHI Conference on Human Factors in Computing Systems, 1–15; https://doi.org/10.1145/3313831.3376515
7. Wong, R.Y. and Khovanskaya, V. Speculative design in HCI: From corporate imaginations to critical orientations. In New Directions in 3rd Wave HCI. M. Filimowicz, ed. Springer, 2018, 175–202; https://doi.org/10.1007/978-3-319-73374-6_10
8. Schultz, T., Abdulla, D., Ansari, A., Canlı, E., Keshavarz, M., Kiem, M., Prado de O. Martins, L., and Vieira de Oliveira, P.J.S. What is at stake with decolonizing design? A roundtable. Design and Culture 10, 1 (2018), 81–101; https://doi.org/10.1080/17547075.2018.1434368
Posted in:
on Fri, December 18, 2020 - 4:29:36
Richmond Wong
Richmond Wong (
https://www.richmondywong.com) is a postdoctoral fellow at the University of California Berkeley Center for Long-Term Cybersecurity. His research focuses on how technology professionals address social values and ethical issues in their work, and on developing design-centered methods to surface discussion of ethical issues related to technology.
[email protected]
View All Richmond Wong's Posts
Nick Merrill
Nick Merrill is a researcher at the UC Berkeley Center for Long-Term Cybersecurity, where he directs the Daylight Lab. He is interested in the social process of threat identification: how and why people identify particular security threats, and who gets to do so.
[email protected]
View All Nick Merrill's Posts
A call for respect, inclusion, fairness, and transparency in SIGCHI
Authors:
Fempower.tech
Posted: Thu, December 03, 2020 - 3:59:36
We are writing this blog post as a response to the discussions about exclusion and oppression within SIGCHI that occurred on the Interactions blog in summer 2020, and a call for respect, inclusion, fairness, and transparency in SIGCHI. Our collective, fempower.tech, started the #CHIversity campaign at the 2017 Human Factors in Computing (CHI) conference because we didn’t feel welcome in previous years [1]. Through this, we created our own space within SIGCHI. We are one of many groups working to make SIGCHI more inclusive and welcoming to everyone who wants to be a part of this community. SIGCHI is a volunteer-led organization that is not only shaped by elected leaders but also by community members who care. Over time, communities create their own ways of working to make changes in their organizations. Sub-communities form when individuals or groups don’t see themselves represented or fitting in to the larger community.
Grassroots groups like AccessSIGCHI or fempower.tech and formalized groups like the Realizing that All Can be Equal (R.A.C.E.) team do this work because we hope SIGCHI can be better. However, when these groups take actions, in some cases encouraged by SIGCHI leaders, they can encounter opposition, disapproval, and accusations of wrongdoing. For example, the R.A.C.E. inclusion team recently explained how the SIGCHI Executive Committee (EC) halted their diversity and inclusion work [2], and Jen Mankoff discussed how the EC hampered her efforts to address accessibility issues in the community by suggesting she violated ACM policy [3]. Based on these descriptions, as well as the experiences that some fempower.tech members have had while doing inclusion-related work in the SIGCHI community, we as fempower.tech observe a pattern that suggests that those holding power to make decisions in SIGCHI do not value community-driven inclusion efforts.
When grassroots or formalized groups have worked to build a more supportive community for themselves, the SIGCHI EC has sometimes responded in hostile ways that undermine or proactively stop these volunteer efforts (as described in the preceding paragraph). We believe that a volunteer-led organization should be open to engage with community efforts to improve situations for those who experience marginalization. We are disappointed that SIGCHI has repeatedly failed to choose a more constructive and responsive approach when engaging with community efforts.
Jen Mankoff’s post reminds us that marginalization and oppression are not one-time, isolated experiences. They are systemic concerns that affect people’s everyday existence. Many are working to make changes, formally and informally, to dismantle the barriers of racism, ableism, sexism, and other forms of oppression. Groups like the Inclusion Teams, SIGCHI CARES, AccessSIGCHI, or the CHI2019 Allyship initiative offered hope that SIGCHI wanted to tackle problems related to marginalization. Yet, by stating that the R.A.C.E. inclusion team and Jen Mankoff, an AccessSIGCHI leader, had violated ACM policy, the EC appeared to undermine its own efforts. Members of fempower.tech who are part of equity-seeking groups have experienced similar (micro)aggressions and scapegoating. This has led us to feel used, unsupported, or unwelcome at meetings and/or conferences.
Creating or supporting initiatives led by people from marginalized groups—and then challenging their work—exploits the good intentions and beliefs of SIGCHI leaders, members, and volunteers who are trying to make positive change. This erodes trust and damages communities already experiencing marginalization. What is especially unsettling about the R.A.C.E. team’s experience is that the people who were doing the work that the institution requested were undermined when their efforts gained traction in the community [2]. Indeed, SIGCHI has repeatedly started inclusion-related projects without providing them a clear path to success. Through these actions, SIGCHI has let its members down, time and time again.
Scholar Reshmi Dutt-Ballerstadt notes that “failing to interrogate institutional decision-making processes while claiming to work towards social justice” is one way individuals (un)consciously sustain white supremacy [4]. When organizations apply their rules inconsistently, in ways that silence volunteers and activists without decision-making power, they not only perpetuate the status quo but also actively harm movements toward more just, caring, and inclusive communities.
How can we make changes in an organization that seems to repeatedly move in a more inclusive direction only to undermine such efforts? For example, the lack of year-to-year continuity between conference organization and new initiatives makes it seem that the work and energy that volunteers expend to make improvements in one year is not valued in the next. Instead of expecting people in leadership roles to make these decisions, community negotiations could decide which initiatives need to be carried forward.
We advocate for structural change that recognizes the interlocking nature of marginalizations. Such change requires a combination of: sustainable resourcing for initiatives such as the Inclusion Teams, SIGCHI CARES, and the Allyship program; individuals unlearning harmful behaviors; better communication with the community; and grassroots activism having pathways to hold governing bodies to account. We urge that these groups be given the necessary, equitable resources and support to ensure their efforts toward a more inclusive SIGCHI are sustainable and equipped to alter old structures. This means official inclusion-related groups must have the power to hold the EC and other bodies’ decision-making processes accountable. Finally, the EC and other governing groups must have the will to enact the changes these groups recommend.
Change is challenging, and the work of volunteering is not equitable, especially across a group as diverse as SIGCHI consisting of students, precariously employed researchers, professionals, and faculty. Given this, the best way forward for our community is to heed the various calls to action which liberate us all [5]. This requires work from the EC and other formalized SIGCHI bodies. It requires them to take our caring critique of their systems seriously and to behave in ways that support rather than harm those already experiencing marginalization. As we said at the beginning of this article, we critique and actively work to improve structures for those of us who experience marginalization precisely because we care about this community and hope it can help all of us thrive. As fempower.tech, we want to work with the SIGCHI community as a whole: grassroots activists, international members, formal advocacy groups, and the EC. We need collective action and concrete changes: no more relying on individuals and incrementalism. Indeed, “change happens slowly” is a narrative that centers those closest to power rather than those experiencing harm.
Many of the requests that others have made [2,3,6,7] and that we amplify are not impossible demands or blue-sky thinking. They are what should be the baseline in just systems. SIGCHI’s own mission and vision statements say as much:
SIGCHI MISSION
: ACM SIGCHI facilitates an environment where its members can invent and develop novel technologies and tools, explore how technology impacts people’s lives, inform public policy, and design new interaction techniques and interfaces. We are an interdisciplinary field comprising academics, practitioners, and educators, and we welcome a variety of approaches to solve these complex problems. The mission of ACM SIGCHI is to support the professional growth of its members who are interested in how people interact with technologies and how technology changes society.
SIGCHI VISION: We aim to enhance our members’ ability to innovate and understand technologies for the greater public good.
With this blog post, we join others who want to help SIGCHI achieve its vision of supporting the greater public good. We as fempower.tech ask that SIGCHI meet grassroots and formalized groups with respect and equality rather than opposition and aggression. In short, we ask that SIGCHI work toward realizing the mission and vision that it celebrates itself as already doing.
Endnotes
1. Strohmayer, A., Bellini, R., Meissner, J., Mitchell Finnigan, S., Alabdulqader, E., Toombs, A., and Balaam, M. #CHIversity: Implications for equality, diversity, and inclusion campaigns. Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, Paper alt03, 1–10; https://doi.org/10.1145/3170427.3188396
2. Grady, S.D., Wisniewski, P., Metoyer, R., Gibbs, P., Badillo-Urquiola, K., Elsayed-Ali, S., and Yafi, E. Addressing institutional racism within initiatives for SIGCHI’s diversity and inclusion. Interactions blog. Jun. 11, 2020;
https://interactions.acm.org/blog/view/addressing-institutional-racism-within-initiatives-for-sigchis-diversity-an
3. Mankoff, J. A challenging response. Interactions blog. Jun. 17, 2020; https://interactions.acm.org/blog/view/a-challenging-response
4. Dutt-Ballerstadt, R. A checklist to determine if you are supporting white supremacy. Inside Higher Ed. Jan. 12, 2018; https://www.insidehighered.com/advice/2018/01/12/checklist-determine-if-you-are-supporting-white-supremacy-opinion
5. Irani, L. “A call to action for the ACM” liberates all of us. Interactions blog. Jun. 29, 2020; https://interactions.acm.org/blog/view/a-call-to-action-for-the-acm-liberates-all-of-us
6. Harrington, C., Rankin, Y., Jones, J., Brewer, R., Erete, S., Dillahunt, T., and Brown, Q. A call to action for the ACM. Interactions blog. Jun. 22, 2020; https://interactions.acm.org/blog/view/a-call-to-action-for-the-acm
7. Rankin, Y.A., and Thomas, J. Straighten up and fly right: Rethinking intersectionality in HCI research. Interactions 26, 6 (2019), 64–68; https://doi.org/10.1145/3363033
Posted in:
on Thu, December 03, 2020 - 3:59:36
Fempower.tech
Fempower.tech are an international network and collective of feminist researchers, practitioners, and activists working with digital technologies. They aim to raise awareness of feminist issues in technology research by being overtly critical and political within the field, raising voices of underrepresented groups and topics, presenting tangible outcomes, and taking on an activist role for this. They create supportive and collaborative environments in their workplaces, within academia, industry, and at international conferences.
View All Fempower.tech 's Posts
How fragmentation can undermine the public health response to Covid-19
Authors:
Andrew Tzer-Yeu Chen
Posted: Fri, October 30, 2020 - 5:24:04
At this point, we are all familiar with Covid-19 and its impacts on ourselves, our communities, and our world. Responses to the disease have largely been led by local, national, and international public health agencies, who have activated their pandemic plans and opened the epidemiological toolkit of modeling, testing, isolation and movement restrictions, surveillance, and contact tracing. When it comes to contact tracing, it’s natural for people to see the tech-heavy world around them, then hear about the common manual process of human investigators and phone calls, and ask, “Why can’t technology help make this better?” But it’s not as simple as “add more technology”—the complex way in which users and societies interact with the technology has significant impacts on its effectiveness. When efforts are not well coordinated, leading to fragmentation in system design and user experience, the public health response can be negatively impacted. This article briefly covers the journey of how contact tracing registers and digital diaries evolved in New Zealand during the Covid-19 pandemic, how the lack of central coordination led to poor outcomes, and ultimately how this was improved.
Contact tracing registers and digital diaries?
Dr. Ayesha Verrall notes that “rapid case detection and contact tracing, combined with other basic public health measures, has over 90 percent efficacy against Covid-19 at the population level, making it as effective as many vaccines” [1]. Contact tracing involves identifying people who have been in contact with an infected person, and therefore who may have been unknowingly exposed to the infectious disease. By identifying the contacts and rapidly isolating and testing those individuals, the chains of transmission in the community are cut off, limiting the spread of the disease. Importantly, contact tracers need to find potential contacts who are unknown to the infected person, and can only do this by tracing the movement of the person to find others who may have overlapped in time and place.
With the pervasive nature of digital technologies, there has been a lot of discussion globally around digital contact tracing solutions, particularly around Bluetooth-enabled smartphone apps (including the Apple/Google protocol) and wearable devices [2,3,4]. Proponents of the technology offer the promise that these solutions can achieve better completeness (finding more contacts of cases, especially where the identities are not known to the case) and speed (finding contacts and testing/isolating them faster). However, in the absence of a validated, effective digital contact tracing solution initially, a number of governments opted for simpler, lower-tech methods of collecting data about people’s movements.
The contact tracing register (or visitor/customer check-in log) has been deployed around the world (Figure 1). Individuals are asked to provide their personal details at businesses and other places of interest, so that if a venue is identified as potential exposure site, then the register can be provided to contact tracers to quickly find people who were there at the relevant time. Digital diaries have also been introduced to help people keep track of their own movements to support their recollection if they get interviewed by a contact tracer—the distinction being that instead of the venue or the government holding the records, the individual themselves maintain and control their logs.
![](/images/uploads/blogs/Fig01_sm.jpg)
Figure 1. A pen-and-paper contact tracing book along with two QR codes for digital diaries in Wellington, New Zealand.
Too many solutions
In New Zealand, a strict Level 4 lockdown was implemented across the country on March 25. Most people (with the exception of essential workers) stayed at home, and the high level of compliance meant that within two weeks the number of active cases began to fall. After four weeks, on April 27, some restrictions (particularly around schools and takeaway food services) were eased at Level 3, and then most restrictions were lifted at Level 2 on May 13 (with the exception of physical distancing and gathering limits). As the country moved into Level 3, the government introduced a requirement under the Public Health Response Order for all businesses to maintain contact tracing registers. These registers required visitors and customers to provide their entry/exit times, name, address, and contact details to the business in case they are needed for contact tracing purposes. The government provided a template for businesses to print and use.
A number of criticisms were leveled at the pen-and-paper contact tracing registers that most businesses used initially. Customers had to provide their personal information on a piece of paper that was visible to all customers, creating a privacy risk. This led to real privacy breaches, such as a female customer being harassed by a male restaurant worker after he took her details from a contact tracing register. There were also some concerns about “dirty pen” risks (if everyone is using the same pen, could that become a vector for virus transmission?), usability (can people be bothered providing their details at every business they go to?), validity (could people provide false details?), and enforcement (can a business deny entry to someone who refuses to provide their details?).
Private software developers took the initiative to come up with better solutions. They reasoned that most people have smartphones (NZ has 80 to 85 percent smartphone penetration), and that using digital tools would mitigate or resolve some of the risks associated with pen-and-paper approaches. Within a week, there were over 30 tools available, almost all using QR codes, with a variety of system architectures and user flows. Some QR codes directed the user to a URL (thus requiring a mobile Internet connection); others required a specific standalone app to interpret the code. Some stored the data on a central server owned and controlled by the developer; others stored the data on the phone for the user’s reference only in a decentralized way (i.e., the “digital diary” approach). Some collected only a name and contact email address; others also asked for phone numbers and residential addresses —it was unclear what would be genuinely necessary for contact tracers to find people. Some tools were offered for free; others required businesses to pay a monthly fee, and two of the largest City Councils bulk purchased licenses of one product for businesses in their cities. Some providers had developed full privacy policies; others said that speed-to-deployment was more important. Unfortunately, there was duplication of effort, and many developers found themselves reinventing the wheel and then struggling under the burden of providing tech support for their products.
![](/images/uploads/blogs/temp_file_Fig02_sm1.jpg)
Figure 2. Photos of various QR codes from different providers in New Zealand, crowdsourced by the author from Twitter.
Almost every business soon adopted a QR code from one of these private providers, but with very little coordination or information around which systems were trustworthy or superior, chaos ensued. A lack of familiarity with how QR codes work among the public also led to significant confusion, with people getting frustrated when some QR codes worked and others didn’t. This was not helped by a number of the QR code posters using similar branding, such as the yellow diagonal stripes that were used in government messaging about Covid-19 (Figure 2)—some posters even using government logos to make their posters look more official. Most of the systems had centralized approaches, which also led to concerns about security and potential unauthorized reuse of data held by private corporations. Some businesses simply created Web forms and added clauses in their privacy policies that allowed them to reuse the collected data for marketing purposes, attracting a stern message from the privacy commissioner. However, an interesting counterargument was that since there were so many different tools, the data was fragmented between different providers and therefore no single company held too much data.
The undirected approach also meant that there was insufficient consideration for the needs of marginalized people. Posters were sometimes placed in positions that were inaccessible for disabled populations. Some businesses removed pen-and-paper registers entirely, making it impossible for participation by the digitally excluded—people without smartphones, or without the skills to effectively use the smartphone, or without expensive mobile Internet data. There was also some confusion about whether or not digital diary approaches (with the data staying on the device) complied with the regulatory requirement for businesses to maintain registers.
The government steps in
On May 20, a week into Level 2, the Ministry of Health launched the NZ COVID Tracer app. This was (and is) also a QR-code-based system with a “digital diary” approach. The app was accompanied by its own QR code standard, which contained a unique Global Location Number for each business, and therefore could be scanned without requiring an Internet connection. Data about check-ins (where people had been at what time) stayed on the device, and the user can choose to release that information to a human contact tracer if identified as a close contact of a known case. The app also allowed individuals to provide up-to-date contact details to the Ministry of Health, which would help contact tracers find them more quickly if necessary.
Unfortunately, QR codes were everywhere by this stage. Some businesses tried to provide multiple options (as shown in Figure 3) with clearer instructions. But this didn’t stop people from being confused about the proliferation of QR codes. The two loudest complaints about the government app were that 1) it wasn’t compatible with older devices (requiring at least Android 7.0 or iOS 12 at launch) and 2) the app didn’t recognize most of the QR codes that were available (Figure 4). The government app wasn’t designed to work with the other QR codes (which from a technical perspective might seem obvious, but for non-technical folks was bewildering). Displaying the government QR code was not mandatory, so many businesses didn’t even have it as an option.
![](/images/uploads/blogs/temp_file_Fig03_sm1.jpg)
Figure 3. Businesses attempting to provide clearer instructions on which QR codes to use, crowdsourced by the author from Twitter.
This fragmentation harmed the uptake of the government app because people felt that existing tools served the same purpose. Within a week of launch, about 380,000 users were registered, equivalent to approximately 10 percent of the adult population of four million people. Registrations plateaued, and a month later sat around 570,000. Meanwhile, the number of QR codes being scanned each day was counted through Web analytics events, slowly ramping up initially as businesses started printing and displaying government QR codes, settling around 50,000 scans per day in early June. Given the size of the population, this was clearly not enough activity to give us confidence that the data from the app would be useful in the event of a further outbreak. However, the government app was the only one that reported statistics about usage, so we don’t have data about how widely other tools might have been used.
![](/images/uploads/blogs/temp_file_Fig041.jpg)
Figure 4. Screenshots from users complaining that the NZ COVID Tracer app wasn’t recognizing the QR code, when they were in fact scanning QR codes from other providers, crowdsourced by the author from Twitter.
It turned out that the government app had actually been in development (with a private sector partner) for at least a month. The specific reasons for why the app was released late have not been made clear yet, although it should be noted that the government has a higher onus to “do things correctly” and needed to prepare a full privacy impact assessment, undergo an independent security audit, have the app checked by the government cybersecurity bureau, and complete other steps that weren’t required for private developers.
By July, it appeared that we had the pandemic under control. New Zealand experienced 102 days in a row without any community cases of Covid-19 detected. The Prime Minister moved the country to Level 1, lifting almost all restrictions except for border controls. Most people became complacent around the risks of Covid-19, with daily scan counts dropping to 10,000 in early July. There were even reports that some businesses were taking their posters down because they felt the QR codes were no longer necessary.
An improvement to the app in June introduced exposure notification functionality. Contact tracers could identify and securely broadcast a place and time where an active case had been, and then the app would check that against the check-in logs on the device and notify the user if an overlap was found. In late July, a further improvement was made to allow users to add manual entries, mostly to account for venues that did not have a government QR code. However, while there were minor upticks in registration and usage after these releases, the activity level still remained very low.
Consolidation is the solution
On August 11, the Prime Minister announced that four cases of community transmission had been found in Auckland (the largest city in New Zealand). There were no links to overseas travel, so it was highly likely that there were other undetected cases in the community. Given the recent experiences of places like Victoria, Australia, where second waves have grown quickly, the government implemented a second lockdown, with stricter movement restrictions in Auckland. In the next day, they also announced that displaying a NZ COVID Tracer QR code in a prominent place would become mandatory for all businesses by the following week. It would still be optional for individuals to scan, but at least the codes had to be available for people to scan if they wanted to. This decision wasn’t without precedent—Singapore required their SafeEntry QR codes to be displayed at all businesses in May.
This announcement caused three things to happen over the following week. First, the private developers with the most prevalent QR codes agreed that consolidation was necessary, and advised their customers to switch to the government QR code. Second, businesses largely complied, with the number of government QR codes increasing four times over the subsequent two weeks (from approximately 87,000 to 324,000). Third, the presence of the disease in the community in New Zealand and the accompanying lockdown elevated the seriousness of the situation, and more people began to scan the NZ COVID Tracer QR codes.
![](/images/uploads/blogs/Fig05_sm2.jpg)
Figure 5. Daily scan counts from the NZ COVID Tracer app, overlaid with significant events. Data sourced from the NZ Ministry of Health.
The number of daily scans shot up, from approximately 30,000 per day before the second wave to over two million per day (Figure 5). The number of registered users also increased, from 640,000 before the second wave to just over two million users (approximately 50 percent of the adult population, although duplicate registrations are not accounted for) as of September 4. While the change in context was a significant driver for shifting user behaviors, moving away from the fragmented system clearly helped increase participation in the system too.
Unfortunately, this increase in participation came too late to be of significant help for the second wave. In the event of an outbreak, contact tracers need at least 14 days of movement logs for infected cases in order to help find close contacts. While the government did use the app to publish six exposure notifications, and some close contacts were found faster because they had updated their contact details, it seems that ultimately the app is yet to find significant numbers of close contacts or new cases. As New Zealand now comes out of its second lockdown, we can only hope that the current level of participation continues to grow for NZ COVID Tracer, and is sustained long enough to help defend against a potential third wave of cases in New Zealand.
Leadership and communication
When a global pandemic catches the world by surprise, a strong response is needed to contain, mitigate, and recover from its impacts. The public expects government to play a leading role in coordinating this response, but many individuals also want to do something to contribute. While people are to be lauded for utilizing their skills to support the broader community, undirected efforts can lead to confusion, duplication of effort, and ultimately harm the overall response. This is not the fault of the individuals—the responsibility lies with the public health agencies to make good use of the available resources and to clearly communicate with people about what is and isn’t needed. Normally, competition between private entities in the open market might be desirable to drive innovation, but in a pandemic, we really need something that just works and supports progress on public health outcomes.
In a country that has had strong communication with the public overall, the confusion around contact tracing registers has been an unfortunate blemish for New Zealand. This case study shows that fragmentation can lead to disparate and negative user experiences, which can harm trust in the system and lead to low participation. In the context of a global pandemic, trust is one of the things we need the most for an effective response.
Acknowledgments
The author thanks the local Twitter community for contributing their images of QR codes to the dataset, and for engaging on digital contact tracing during the COVID-19 pandemic. The author also acknowledges members of Koi Tū: The Centre for Informed Futures for discussions around the use of technology for contact tracing.
Endnotes
1. Verrall, A. Rapid audit of contact tracing for Covid-19 in New Zealand. New Zealand Ministry of Health. Apr. 20, 2020; https://www.health.govt.nz/publication/rapid-audit-contact-tracing-covid-19-new-zealand
2. Wilson, A.M. et al. Quantifying SARS-CoV-2 infection risk within the Apple/Google exposure notification framework to inform quarantine recommendations. medRxiv. Jul. 19, 2020; https://www.medrxiv.org/content/10.1101/2020.07.17.20156539v1
3. Asher, S. Coronavirus: Why Singapore turned to wearable contact-tracing tech. .Jul. 4, 2020; https://www.bbc.com/news/technology-53146360
4. Alkhatib, A. We need to talk about digital contact tracing. Interactions 27, 4 (Jul.–Aug 2020), 84; http://interactions.acm.org/archive/view/july-august-2020/we-need-to-talk-about-digital-contact-tracing
Posted in:
Covid-19 on Fri, October 30, 2020 - 5:24:04
Andrew Tzer-Yeu Chen
Andrew Tzer-Yeu Chen is a research fellow with Koi Tū: The Centre for Informed Futures, a transdisciplinary think tank at the University of Auckland, New Zealand. His background is in computer engineering, investigating computer vision surveillance and privacy. His research interests now sit at the intersection of digital technologies and society.
[email protected]
View All Andrew Tzer-Yeu Chen's Posts
Future directions for situationally induced impairments and disabilities research
Authors:
Garreth Tigwell,
Zhanna Sarsenbayeva,
Benjamin Gorman,
David Flatla,
Jorge Goncalves,
Yeliz Yesilada,
Jacob Wobbrock
Posted: Tue, October 06, 2020 - 1:29:59
Mobile devices are our constant companions. We use them in varied contexts and situations, such as outside on a cold street, lying down in a dark bedroom, and commuting to work on a train. These situations challenge our ability to use mobile devices and can negatively influence our interactions with them. For example, we slow down or make more errors when typing, or select a wrong button. These adverse contextual factors have been referred to as situationally induced impairments and disabilities (SIIDs), or sometimes just situational impairments for short.
The experience of SIIDs during mobile interaction applies to users of all abilities, as SIIDs adversely affect both non-disabled and disabled user groups [1]. But SIIDs have been shown to exacerbate negative user experiences in mobile interaction for people with disabilities [2]. Thus, by accommodating SIIDs during mobile interaction, we can provide solutions to improve user experience for people with and without disabilities [1].
Research in the field of SIIDs is conducted in four main areas: Understanding (e.g., [3]), sensing (e.g., [4]), modeling (e.g., [5]), and adapting (e.g., [5]). Understanding provides knowledge about the effects of SIIDs on mobile interaction; sensing allows building mechanisms to detect the presence and extent of SIIDs; while modeling and adapting enable the creation of models and interfaces to accommodate SIIDs. The main criticism about research conducted on SIIDs is the lack of systematic knowledge of the effects of underexplored SIIDs and the combination of different SIIDs on mobile interaction. Furthermore, there is an absence of built-in sensing, modeling, and adapting mechanisms on conventional smartphones to detect and accommodate these SIIDs. Therefore, the research community should strive to push the SIIDs research agenda further in order to build a roadmap for future work in the field.
![](/images/uploads/blogs/temp_file_figure-1_sm2.jpg)
Figure 1. Workshop organizers and attendees enjoying a meal after an engaging full-day workshop.
Workshop purpose, structure, and goals
We organized a one-day workshop for CHI 2019 in Glasgow, Scotland, called Addressing the Challenges of Situationally-Induced Impairments and Disabilities in Mobile Interaction. The purpose of the workshop was to assemble researchers whose work is related to SIIDs so that we could identify current research gaps and define new directions for future research. The workshop included five main events throughout the day: “lightning” presentations, focus groups, a panel discussion, exploring scenarios, and a town hall meeting, followed by an evening meal (Figure 1). A detailed structure is found in our workshop proposal paper [2].
The four goals of our workshop were to: 1) provide a space for organizers and participants to share their expertise and insights on SIIDs, 2) engage attendees to discuss and identify gaps in the SIIDs research space, 3) ideate new solutions that could mitigate the effects of SIIDs, and 4) create and strengthen an international collaborative SIIDs research network.
The workshop included 17 participants (including organizers), collectively representing universities in eight countries across four continents. Our workshop call for submission was purposely made to be broad to allow participants to submit whatever they felt was relevant. For example, submissions could be in the form of position papers, case studies, empirical studies, or new interaction methods. Submissions were limited to eight pages (including references) and we used arXiv to keep a record of the accepted papers (the proceedings can be found at https://arxiv.org/html/1904.05382). Our participants were each given four minutes to discuss their work during the lightning presentations event (Figure 2) so that we could dedicate as much time as possible to identifying necessary future work.
![](/images/uploads/blogs/temp_file_figure-2_sm1.jpg)
Figure 2. A workshop attendee giving a short presentation about their work.
Activity 1: Focus group activity
Our attendees were divided into groups and given 1.5 hours for the first activity:
- Task: The groups reflected on the lightning presentations and deliberated on challenges they have faced when conducting SIIDs research.
- Outcome: The groups highlighted key parts of their discussion to share with the other workshop groups.
A list of example questions were provided to drive the discussion related to research methods and equipment, recruitment, ethical challenges for SIIDs studies, challenges utilizing sensor data and modeling SIIDs, and the limitations of adaptation. Figure 3 provides an example of one group’s record of challenges that they identified.
![](/images/uploads/blogs/figure-3_sm.jpg)
Figure 3. An example of challenges discussed by one of the groups.
Recruitment can be a challenge for SIIDs research, particularly when seeking to run longitudinal and ecologically valid studies—the costs incurred can be higher. Lab studies are beneficial for isolating factors to study, but more work needs to be conducted outside to increase the ecological validity of results. There should be particular consideration given to active observation approaches rather than only passive observation. Guided tours might be one way to mitigate some of the challenges of in-the-wild research and to address IRB concerns for participant safety. People with permanent disabilities should also be recruited, since they can also experience situational impairments and need very specific solutions to address SIIDs.
Modeling and adaptation are promising solutions to addressing SIIDs. Artificial intelligence and machine learning research could support the efforts of HCI researchers, but there is a lack of ground-truth data. It is important to parameterize the environment to help understand the relevant context; it may be possible to do this in an unobtrusive way such as by using smartwatches for sensing. However, although logging data can help address SIIDs, it may cause people to become uncomfortable, as devices become more aware of context, which highlights various legal, security, and privacy challenges. There was also concern that through addressing SIIDs, people’s own skill development might diminish, as devices work more independently and people may be encouraged to interact at times when they should be focusing on their environment (e.g., while driving).
Activity 2: Exploring scenarios
Our attendees were divided into groups and given 1.25 hours for the second activity:
- Task: The groups explored random scenarios and focused on the gaps in understanding, sensing, modeling, and adapting, in regards to the context, technology, and task.
- Outcome: The groups recorded the unique issues identified from the card game and shared the findings during the town hall meeting.
![](/images/uploads/blogs/figure-4_sm.jpg)
Figure 4. Example scenarios created by taking one card from three piles (a situation, a piece of technology, a task).
We gave each group a set of prompt cards to facilitate this activity. The cards covered three categories (a situation, a piece of technology, a task). We had at least 10 ideas for each category. The cards were laid out in the three categories, stacked in a random order, and the groups drew a card from the top of each pile so that the three cards made up a scenario (see Figure 4). The group could draw three cards that propose something common and relatable, such as “jogging,” “using phone,” and “reply to a message,” or something less familiar, such as “outside in the rain,” “wearing an AR headset,” and “call an Uber.” Sometimes a combination of cards was drawn that suggested an unlikely scenario, but the purpose was to quickly generate unique scenarios that the group could use to identify challenges and possibly where new research efforts need to be focused. We added wildcards to each pile so the group could determine their own entry (e.g., a technology wildcard would allow the group to invent some future mobile device to add to the scenario). The benefit of this approach was that the groups were less constrained to the ideas written on our cards.
Here are three examples generated by the participants and the issues that were considered:
- On a beach under an umbrella using a tablet and needing to unlock the device. The group considers that a person may have sand on their hands, and depending on how they unlock their device, this might be a challenge. For example, if using a fingerprint to unlock and it does not work due to sand, what other unlock methods are there? This highlights the need for various fallback methods to address SIIDs. But also the user may want control of what those fallback mechanisms look like—maybe the user does not want to compromise on biometric unlocking? Perhaps a user’s voice signature could be used for unlocking?
- Running in a park and receiving an email response on a smartwatch. Focusing on the email will distract the user from running and their environment. It could be difficult to read a long message and to respond to the email, which means more focus has to be given to the task. A user may have privacy concerns for reading in public depending on the content of the email. There may also be network inconsistencies while outside. The technology should aim to sense the level of danger and possibly delay notification if it is not an urgent task. Perhaps the smartwatch could detect that the user is running and defer its interruption?
- At the airport sending a message using a laptop. A person traveling can be stressed, tired, and likely distracted as they are listening for important flight information. The user is likely concerned about conserving their battery and dealing with limited network connectivity. Here, it is not only environmental factors that lead to SIIDs, but also internal factors. For example, the concept of emotions is purely situational; it is unclear if emotions directly influence the way we interact with mobile devices. In this scenario, the device could detect that the user is in an airport and make the changes necessary to reflect the current mood of the user. Perhaps future laptops could detect users’ stress levels and avoid contributing to information overload?
It is clear we need to be accurate and flexible with SIIDs solutions. There is little room for error when sensing the environment for potential dangers and determining the best method for interaction. Furthermore, an individual user will have their own needs that must be met and these are not likely to be static over time or for particular tasks. For example, some emails are urgent and others are not. There needs to be an easy way for the user to make these aspects known to the device to enable alternative modes of interaction.
Should we always adapt/accommodate?
Currently, most research in the area of SIIDs has been conducted within the laboratory environment. A laboratory environment limits our understanding of SIIDs, as it strictly controls and excludes the effects of accompanying factors that might be present in a real-world scenario. For this reason, we suggest that more research should be conducted in the wild. It is also necessary to study the effects of combined SIIDs, as it is common for a user to experience the effects of multiple SIIDs at once, for example being outside in a cold and noisy environment late at night. We argue that these future directions would increase our understanding of SIIDs and create new insights, potentially revealing new behaviors of people observed under realistic conditions with multiple SIIDs present.
We also suggest further understanding and investigation of the effects of SIIDs on mobile interaction according to the 2D space presented by Wobbrock et al. [1]: from-within (emotions, mood, mental well-being), from-without (difficult terrain, lack of connectivity/power), and mixed (combination of external and internal) factors. It is pivotal for the research agenda to understand the effects of SIIDs from this perspective in order to progress further by building sensing, modeling, and adapting mechanisms for these SIIDs. Furthermore, if the similarity between the effects of underexplored SIIDs and permanent impairments is established, it can further enable the creation of design solutions to accommodate users of all abilities (i.e., permanently, periodically, or situationally impaired).
Moreover, research has shown that the personal and individual characteristics of users are very important when building sensing, modeling, and adapting mechanisms to address SIIDs. The challenges of building individual models and adaptive interfaces [1] can be overcome by optimization algorithms and formulating cost functions [2]. These should be sensitive to any of the user’s privacy and security concerns. In addition, it is also important for these mechanisms to provide adequate judgements of SIIDs to decide if the adaptation should take place at all, especially in high-risk, high-cost situations, when visual and attentional needs should be focused on tasks of higher priority (e.g., crossing a busy road).
Finally, we suggest that the research should expand to include a wider range of devices [6], for example smart watches, fitness and activity trackers, and other less common wearable technology such as AR glasses, which we foresee becoming the new norm. Considering potential SIIDs when designing new technology is important in order to build in necessary solutions from the outset, rather than after the technology is adopted by the user. Our CHI 2019 workshop highlighted these (and many other) issues that the attendees and, we hope, many other researchers will undertake, creating more aware, responsive, accessible, and safe mobile technologies that are usable by everyone.
Endnotes
1. Wobbrock, J.O., Gajos, K.Z., Kane, S.K., and Vanderheiden, G.C. Ability-based design. Communications of the ACM 61, 6 (2018), 62–71.
2. Tigwell, G.W., Sarsenbayeva, Z., Gorman, B.M., Flatla, D.R., Goncalves, J., Yesilada, Y., and Wobbrock, J.O. Addressing the challenges of situationally-induced impairments and disabilities in mobile interaction. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2019, 1–8.
3. Sarsenbayeva, Z., van Berkel, N., Hettiachchi, D., Jiang, W., Dingler, T., Velloso, E., Kostakos, V., and Goncalves, J. Measuring the effects of stress on mobile interaction. Proc. of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 1 (2019), 1–18.
4. Goel, M., Findlater, L., and Wobbrock, J. WalkType: Using accelerometer data to accommodate situational impairments in mobile touch screen text entry. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2012, 2687–2696.
5. Gajos, K.Z., Weld, D.S., and Wobbrock, J.O., Automatically generating personalized user interfaces with Supple. Artificial Intelligence 174, 12-13 (2010), 910–950.
6. Akpinar, E., Yeşİlada, Y., and Temİzer, S. The effect of context on small screen and wearable device users’ performance - A systematic teview. ACM Comput. Surv. 53, 3 (Jun. 2020), Article 52; https://doi.org/10.1145/3386370
Posted in:
on Tue, October 06, 2020 - 1:29:59
Garreth Tigwell
Zhanna Sarsenbayeva
Zhanna Sarsenbayeva is a postdoctoral research fellow in the School of Computing and Information Systems at the University of Melbourne. Her research interests include accessibility, ubiquitous computing, human-computer interaction, and affective computing.
[email protected]
View All Zhanna Sarsenbayeva's Posts
Benjamin Gorman
David Flatla
Jorge Goncalves
Jorge Goncalves is a senior lecturer at the University of Melbourne, Australia. His research interests include crowdsourcing, social computing and ubiquitous computing. He received a Ph.D. in computer science from the University of Oulu.
[email protected]
View All Jorge Goncalves's Posts
Yeliz Yesilada
Jacob Wobbrock
Fighting Coronavirus with Faith: Religious and Parareligious Responses to Covid-19 in Bangladesh
Authors:
Sharifa Sultana,
A.K.M. Najmul Islam,
Syed Ishtiaque Ahmed
Posted: Wed, September 16, 2020 - 1:39:52
Many communities in Asia, Africa, and Latin America—often left out of human-computer interaction (HCI) discussions—defy the boundary of scientific rationality and seek healthcare and well-being from various traditional faith-based practices. For example, South African indigenous traditional healers serve 60 percent of their population. And thousands of Peruvians use the ancient tradition of mystical healing and shamanism for their well-being, as do many people in South Asia [1,2]. Here we broadly term such faith-based practices witchcraft. During crises like pandemics, witches often use local beliefs and alternative rationales to explain diseases, their spread, and their impacts, as well as to innovate protective measures. Recently, modern scientific innovations and computing technologies have started encroaching on these faith-based practices. Covid-19 has demonstrated how modern scientific rationality and local alternative rationality interact—something that has remained understudied in our mainstream HCI literature for a long time.
Today’s HCI is increasingly interested in alternative assumptions and decision making around well-being in the Global South. Our own work in this vein that involves ethnographic field studies in rural Bangladesh, reveals that the rural healers, witches, and their followers explain Covid-19 using religious, parareligious, and supernatural metaphors and narratives. In this ongoing fieldwork, so far we have engaged with more than 30 male and female participants. This post presents part of our findings and focuses on the rural conceptualization of the coronavirus, its relation to morality, and a few possible ways of stopping the virus from rural perspectives. We explain these through the three lenses presented below.
“Microbe is a curse that spreads through unhappiness”
The healers and witches (known as Kabiraz) told us that Covid-19 was a strange and new situation for them that they had never seen before. However, they also told us that they had experienced similar challenging situations before, and that they could use that knowledge to handle Covid-19. These witches defined the coronavirus as an “evil spirit and a curse.” Over the course of our conversations, a witch also mentioned that usually people with weak physical and mental health are more vulnerable to such curses. That is why she criticized the hard rules like quarantining, because that would impact negatively on the mind of the weak people (i.e., make them even more unhappy).
![](/images/uploads/blogs/figure_1.jpg)
|
![](/images/uploads/blogs/figure_2.jpg)
|
Figure 1. (Left) A set of Tantra that includes a Jantra on a mud pot and green and red chillies. This has to be put beside the main door of the house to keep the curse away. (Right) Tabiz seller claiming that it will prevent infection from Covid-19 (a collected image recently went viral among Bangladeshi Facebook users [3]).
Another witch told us how they were trying to keep their clients’ houses safe from Covid-19 during this crisis. She explained that Covid-19 patients were possessed by a bad spirit that is full of curse. The method of stopping the curse included a set of activities called Tantra. She also showed us an example of Tantra that used some Jantra (scripts for recital, usually placed within geometric shapes) on a mud pot, some scripts from witch books, and some spices (Figure 1, left). The pot with the Jantra on it had to be put in front of the main door of the house, and the spices had to be hanging from somewhere close to the pot. Thus, the pot would radiate a positive energy that makes cursed people feel uncomfortable upon coming near the house and discard the idea of visiting. The villagers were producing such Jantra on a mass scale and putting them into amulets called Tabiz, selling them in their local markets (Figure 1, right).
“Immorality is the main vulnerability”
The villagers mentioned a wide range of people who could be the victims of Covid-19, including people who maintain a life that does not conform to village norms, for example people who were engaged in illegal businesses or who consumed Haram (not permissible in Islam) foods and commodities. Our discussion with the rural women led us to dissect the story of a man in their village who had recently traveled back from Kuwait. The women told us that that person was no longer a “local person” to them since he broke many local moral rules. One of the women said,
He went there [Kuwait] by bribing for a duplicate (fake) passport and visa. He was living in Kuwait for many years. Most likely, he was engaged in an illegal business there as he did not have the real papers. So, he must be more vulnerable to this curse...
The women also told us that the local elders and leaders of the neighborhood urgently discussed this issue and found him to be “dangerous” for the other residents of the village—especially if he mixed with the locals and went to the tea stalls, mosques, and marketplaces. This was less because that person might spread the virus directly, and more because that person could influence others to indulge in immorality (which would eventually cause Covid-19). They decided not to be friendly with that person (because he was already cursed), and to not allow him to move freely in their village.
Design, rules, and faith
The villagers adapted to the quarantine in their own way. We found religious faith and local beliefs playing a major role in rural Bangladesh while the villagers adapted to the quarantine and hygiene measures. For example, a woman who had rented the extended part of her house explained how they updated the house hygiene rules for themselves and the renters by adapting to the Islamic way of washing body parts (Odhu):
We have a new rule in our house that now no one is allowed to enter the house without having “Odhu” and using soap for it.
She also referred to Islamic literature to support some of her rules:
Once Mohammad (PBUH) was cured by reciting Surah Al-Nas and Surah Al-Falaq when someone spelled him with black magic. We ask the residents to recite these two Surah loudly during that cleaning time.
This and many such examples show how the local responses to Covid-19 in rural Bangladesh are rooted in and shaped by the villagers’ spiritual beliefs and traditional practices. The modern scientific advice is neither rejected nor accepted by the villagers: They acted according to their own narrative, and developed a communal effort to combat Covid-19 in their own way.
Our ongoing and long-term engagement with these rural communities in Bangladesh demonstrates how millions of people in the Global South live outside the realm of modern scientific knowledge, and how they interact with modernity, science, and technology using their own faith-based narratives. Taking these local narratives and traditional faiths seriously essentially dismantles HCI’s current orientation toward modern science and the perception of computing as a vehicle for it. Through our work, we join the growing movement within HCI to ask whose health? whose harm? and whose knowledge? and challenge the hegemony of the secular Western science in computing practices that attempts to colonize over millions of lives in the Global South. We believe that we need to decenter Western scientific knowledge as the main source of HCI design to address the contextual needs of millions of people in the world, especially in the time of public health emergencies like Covid-19.
Endnotes
1. Sultana, S. and Ahmed, S.I. Witchcraft and HCI: Morality, modernity, and postcolonial computing in rural Bangladesh. Proc. of the 2019 SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2019.
2. Sultana, S., Ahmed, S.I. and Fussell S.R. Parar-daktar understands my problems better: Disentangling the challenges to designing better access to healthcare in rural Bangladesh. Proc. of the ACM on Human-Computer Interaction 3, CSCW (2019), 168.
3. https://www.facebook.com/groups/538670766844801/permalink/561537171224827/
Posted in:
Covid-19 on Wed, September 16, 2020 - 1:39:52
Sharifa Sultana
Sharifa Sultana is an HCI researcher. She is interested in ICTD, critical computing, well-being, and feminist HCI. She uses both quantitative and qualitative (ethnographic) techniques to study marginalized rural populations in Bangladesh and aims to design computational tools and systems to address the challenges for the rural low-education population while accessing information. Currently, she is pursuing a Ph.D. in information science at Cornell University.
[email protected]
View All Sharifa Sultana's Posts
A.K.M. Najmul Islam
Syed Ishtiaque Ahmed
Syed Ishtiaque Ahmed is an assistant professor of computer science at the University of Toronto. For the past 12 years, he has been conducting ethnography and design research with various marginalized groups in the Global South.
[email protected]
View All Syed Ishtiaque Ahmed's Posts
Learning remotely, making locally: Remote digital fabrication instruction during a pandemic
Authors:
Jennifer Jacobs,
Nadya Peek
Posted: Fri, September 04, 2020 - 11:29:41
The Covid-19 pandemic has fundamentally disrupted learning by requiring a society-wide shift to remote instruction. This shift has raised specific challenges for students and educators in classes that rely on physical making. We teach digital fabrication—a subject that combines computational design tools and computer-controlled fabrication machines. When our campuses in Santa Barbara and Seattle shut down during the start of the pandemic, we had to grapple with a question also facing many other instructors across art, design, science, and engineering: How can students engage in hands-on making without access to workshops, labs, and studios, and the physical equipment, materials, and tools within them?
The following is an account of how we approached this challenge of remote instruction for digital fabrication and our reflections on the experience. In the two weeks between the end of the winter quarter and the start of the spring quarter, we converted our courses from a centralized model that relied on university makerspaces to a distributed model that relied on mini 3D printer makerspaces in students' homes. Our observations from our classes and our own reflections as instructors reveal several trade-offs of structuring a course around an at-home 3D printer. The projects produced by our students hint at how hobbyist equipment can support powerful forms of learning that are less feasible in shared makerspaces. Simultaneously, we found that housing fabrication equipment at home creates new demands and new forms of labor for both students and instructors. By sharing our experience, we hope to provide some practical options for physical prototyping classes during the Covid-19 era. Furthermore, our students’ experience of living and working with 3D printers on a daily basis sheds additional light on the pitfalls, pleasures, and potential of personal fabrication.
What Is Digital Fabrication and Why Do We Teach It?
Digital fabrication encompasses a wide range of design and manufacturing practices, including laser cutting, 3D printing, printed circuit board production, automated knitting, and robotic milling. Digital fabrication technologies are used in an equally broad range of manufacturing domains ranging from architecture, textile production, consumer electronics, woodworking, and ceramics. Despite the diverse applications of digital fabrication technology, all digital fabrication workflows generally center on the same three stages: digitally specifying a design through computer-aided design software (CAD), converting designs to machine instructions via computer-aided manufacturing software (CAM), and producing physical parts through computer-numerically-controlled (CNC) machine operation. This workflow is powerful and distinct from standardized mass-manufacturing techniques such as injection molding because it enables low-volume, custom forms of production without sacrificing accuracy or repeatability [1].
As instructors, we apply CAD, CAM, and CNC tools and technologies toward many different learning opportunities. Learning digital fabrication enables students in art, design, and engineering to develop skills for rapid physical prototyping. Digital fabrication also allows educators to situate computer science, mechanical engineering, mathematics, and other STEM fields in the context of physical making [2,3]. Digital fabrication also supports new forms of design-oriented research and critical making. By creating their own digitally fabricated products, students directly experience the complexity of producing robust, beautiful, and functional artifacts and confront the tensions that emerge when moving between digital representations and physical materials. For this reason, we believe applied digital fabrication is necessary for any student engaged in making research or practice.
“How Are You Going to Teach Digital Fabrication Without a Makerspace?”
The classes we teach—HCDE 598: Digital Fabrication and MAT 549X: Computational Fabrication—make up part of a broader curriculum in interdisciplinary departments that combine engineering, art, design, theory, and practice. HCDE 598 was developed by Nadya Peek within the Human-Centered Design and Engineering (HCDE) department at the University of Washington. HCDE offers a master’s degree that trains students in user experience, human-computer interaction, product design, and interface design. The classes are predominantly scheduled in the evening and many of the students work full-time during the day. HCDE 598 is an elective engineering course designed to introduce students to CAD and prototyping tools for making physical artifacts. Concepts covered include tolerance/fit, flexural design, mold making, and parametric design. Twenty students enrolled in the course in Spring 2020, although typically the class accommodates 40. Two of Nadya’s Ph.D. students TA’ed HCDE 598, although typically the course is allocated one TA.
MAT 594X was developed by Jennifer Jacobs as a course in the the Media Arts and Technology (MAT) graduate department at the University of California, Santa Barbara. MAT 594X targets Ph.D. and master’s students within MAT, as well as graduate students in computer science, and similar to HCDE 598, aims to help students develop prototyping and manufacturing skills for research, art, and design applications. MAT 594X differs from HCDE 598 in its emphasis on computational fabrication; students use programming languages, including Python and Grasshopper, to design for and control digital fabrication machines. Weekly course topics blend computational design methods (e.g., affine transformations, parametric surface representations, topology optimization, and mesh repair) with digital fabrication concepts (e.g., G-Code syntax, CAM simulation, machine calibration, designing for existing objects). Twelve students enrolled in MAT 594X, as is standard in graduate classes in MAT.
Pivoting to remote instruction came at a time of great uncertainty. The start of the pandemic upended higher education and completely changed many aspects of daily life. These changes were all just beginning in late March and early April, near the start of the spring quarter. Restructuring our classes for remote instruction required on-the-fly planning. We managed through trial and error, by implementing the following changes.
What we bought
Pre-pandemic, we had the privilege to teach in university makerspaces, which generally have a large spatial footprint, an established community, and hundreds of thousands of dollars worth of equipment and software. HCDE 598 is normally conducted in the MILL, a UW makerspace that gives students access to laser cutters, 3D printers, CNC mills, water-jet cutters, metalworking tools, sewing/embroidery machines, and shared computers with CAD/CAM software. MAT 594X was scheduled to be conducted with equipment in the Elings Hall Innovation Workshop, a space with laser cutters, desktop CNC mills, and a variety of 3D printers.
Because machine workflows and material outcomes are critical to our course learning outcomes, we focused on pivoting to remote approaches that would preserve some form of hands-on physical fabrication. In this respect, we were somewhat fortunate. Many forms of digital fabrication equipment have become increasingly inexpensive and widespread. Hobbyist-grade 3D printers, in particular, are now available for $200–$300. In other words, it is now possible for students to acquire a no-frills CNC machine for around the same price as an engineering textbook. This opportunity enabled us to restructure our courses from a model that relied on large university digital fabrication workshops to individual (much simpler) digital fabrication workshops in each student’s home. We expanded the class bill-of-materials (BOM) to include equipment and tools that provided the basics of a home makerspace. In both classes, our BOM was centered around Creality Ender 3 Pro, a single extruder, fused-deposition modeling (FDM) 3D printer, which, at the time, cost about $250. In HCDE 598, Nadya’s BOM also included measurement tools such as calipers, hand tools such as box cutters, and materials including 3D-printing filament, casting silicone, plaster, and cardboard. The total BOM, including the printer, was approximately $350. Most items were purchased by students and either shipped directly to their home or hand delivered by Jennifer and Nadya. In MAT 594X, the BOM focused primarily on the printer and starter rolls of filament. Students were periodically shipped new filament as needed, and provided with a $100 materials budget for the final project, which they used to purchase specialty filament, casting materials, and electronic and lighting components. Jennifer used a combination of her own research funds and departmental resources to purchase printers and supplies. These funds would not have been sufficient if MAT 594X had included a larger number of students.
What we taught
The shift to remote instruction and the loss of university workshop access also required us to redesign our weekly project-based assignments. We had to alter or remove assignments that required unavailable equipment, such as laser cutters, CNC mills, or electronics fabrication. In the process, we focused on ways to adapt many of the concepts that we originally sought to teach. Because HCDE 598 was intended to teach students about critical stages in the fabrication pipeline, Nadya developed new assignments that focused on tolerance/fit, compliant mechanisms, system integration, and tooling geometries to forms that were possible with additive fabrication and at-home equipment. For example, students had to design and 3D print a series of clips that used flexures to hold together cardboard. To demonstrate their clips, they made animated GIFs of themselves shaking a cardboard/clip construction without it falling apart. In MAT594X, Jennifer similarly created new assignments that focused on integrating computer programming with the FDM 3D printing. For example, instead of using an existing slicer software (a program that converts geometry to G-Code for printing), students had to write programs that produced G-Code directly, in a format compatible with the Ender 3 Pro’s controller software.
Kevin Philbin’s clip testing.
While we tried to keep the focus on hands-on making, the lack of additional equipment led both of us to add additional material on software-based CAD, including more complex parametric and generative design, rendering, and simulation. Students in HCDE 598 used Rhino/Grasshopper, Adobe Creative Suite, and Cura (a popular slicer software) on their personal computers for CAD and CAM. MAT 594X was centered on Rhino/Grasshopper and Cura, and Jennifer added Fusion 360, Meshmixer, Meshlab, and mobile-based photogrammetry software for additional generative design and 3D scanning assignments.
We also adjusted the ways we evaluated assignments. We created project grading rubrics that explicitly specified alternative pathways for project completion aside from a completed physical product. For example, in MAT 594X, students could focus on documentation of different (potentially failed) steps, provide substantial peer feedback and support, or focus on a written reflection about how the course concepts aligned with their research objectives. These steps were designed to compensate for the limitations of hobbyist equipment, and the stressors of the pandemic.
![](/images/uploads/blogs/temp_file_sam_chain_sm1.jpg)
Interlocking chain created by Samuelle Bourgault using Python scripting to generate G-Code. Link
How we taught
Digital fabrication involves a pipeline with many different steps and stumbling blocks across the CAD, CAM, and CNC stages. Pre-pandemic teaching in a workshop had the important benefit of allowing us to highlight detailed or tacit components of the fabrication pipeline in the process of working with a machine. We could also actively respond to issues like material irregularities, tool failure, and machine calibration in person. Like many instructors, the shutdown led us to transition our classes into synchronous and asynchronous components across different platforms and formats.
We both held weekly synchronous class sessions over Zoom: a weekly four-hour session for HCDE, and biweekly one-hour sessions for MAT 594X. These sessions were recorded and made available online following each class. In MAT 594X, synchronous class sessions were shortened in comparison to in-person classes. This resulted in less time for sharing student work, discussing readings, and answering technical questions. Jennifer focused her synchronous sessions on providing high-level overviews of technical content, reviewing a small number of student project outcomes, and touching on highlights from the reading discussions.
The narrow communication bandwidth of online video conferencing technologies limited both the amount and the quality of in-person instruction we could engage in. In particular, we realized early on that we could not respond on the fly to CAD and machine operation issues, or demonstrate complex manufacturing workflows over Zoom. We therefore tried to provide detailed guidance and troubleshooting in advance by preparing higher volumes and more detailed forms of contextual material; This included written technical walkthroughs, prerecorded video tutorials, post-class reminders of important gotchas, and links to additional vetted material students could consult. For example, rather than just providing completed code examples, Jennifer embedded incremental code samples in written step-by-step instructions that added details on software use, and checks to perform prior to fabrication. Nadya regularly checked in on student engagement by conducting weekly anonymous surveys to that included questions on students’ well-being and learning goals.
The absence of a shared physical workshop also resulted in a loss of an important peer learning environment. To compensate, we set up dedicated Slack workspaces for communication outside of the Zoom sessions and actively cultivated the online community by responding to student questions, celebrating in-progress work, and linking to fabrication news and research. We also required students to document their work online. HCDE 598 students were required to document their progress on public Github webpages that were shared with the class. MAT594X students used Google Slides to post reading reflections, and Google Docs or Instructables to publish in-progress and final project outcomes. Peer feedback was a component of students' grades, and was evaluated through the quality and number of comments students gave on Slack, project pages, and online reading reflections.
![](/images/uploads/blogs/temp_file_solen_lamps_sm1.jpg)
Sölen Kiratli’s iterations on a lamp design that would work with the support structure of the printers. Link
The Effects of Remote Fabrication on Student Experience
Our observations during the course and feedback from our students suggest that remote instruction with distributed hobbyist 3D printers is a viable method for teaching graduate-level digital fabrication courses. In fact, this model may offer unique learning opportunities over courses in centralized makerspaces. However, there were also several undesirable effects that emerged when shifting the workshop from the university to the home.
Learning opportunities of hobbyist machines
The printers we shipped to students had reduced features and capabilities in comparison with industrial equipment in university workshops. They lacked soluble support material, temperature-controlled build chambers, automated calibration settings, and had significantly smaller print volumes, slower print speeds, and reduced accuracy. These factors restricted the size, form, and geometry of the objects the students could produce, which initially caused frustration among some students. Early in MAT59X, several students wondered if different software tools or more advanced machines existed that could ensure successful prints each time. From our prior teaching experience, these reactions are common for newcomers to digital fabrication. All digital fabrication machines, regardless of sophistication, impose constraints and limitations. Producing successful digitally fabricated products requires learning how to design for these constraints. While the hobbyist printers imposed more severe constraints than a $20,000 FDM printer, they still enabled students to learn how to develop design strategies for specific manufacturing processes.
The use of individual hobbyist printers also had advantages when compared with how students access machines in a workshop. Low-cost platforms have the benefit of appearing more approachable to newcomers, and they can significantly reduce the risks of experimentation. Furthermore, unlike staff-managed workshop equipment, individual printers also enabled students to have constant access to the machine, and required them to learn about machine maintenance. Nadya leveraged this opportunity by making the printer’s assembly and initial calibration one of the first assignments in HCDE 598. The printer assembly steps are comparable to those of flat-pack furniture, but the fine-tuning of the machine is key to its performance. To tune the printers, students relied on our assembly documentation, discussed tips and tricks on Slack, and referred to YouTube videos that demonstrated printer-specific tuning methods. By the end of the spring quarter, students in both courses had tuned and modified their machines to a degree that went significantly beyond the manufacturer documentation. This engagement enabled students to familiarize themselves with the machine's implementation details and performance possibilities in a form that would not have been feasible in a shared-use setting.
![](/images/uploads/blogs/temp_file_spider_sam_sm1.jpg)
Samuelle Bourgault’s press-fit, modular design. Link
We believe that constant access and freedom to maintain and experiment with a personal machine were factors that directly contributed to the high volume of successful products students created in both classes. We observed that students achieved higher quality 3D-printed parts and greater numbers of design iterations than students in previous classes and workshops who printed on shared machines of comparable quality in shared makerspaces. We also saw students experiment with printing their designs in different orientations and exploring different kinds of trade-offs. For example, Samuelle Bourgault, a student in MAT 594X, printed multiple variations of a design first as a solid part that required an elaborate support structure, and later as a series of modular pieces that required manual assembly, but reduced print time by removing the need for support material. Her strategy was shared with other students who later used it in their own designs. Repeated design iterations were common in both courses and went beyond simple optimizations. Victor Allen, a student in HCDE 598, made piles of different design iterations for a camera mount, experimenting with the material strength and incorporating off-the-shelf parts.
Victor Allen “Graveyard of Parts” is shown left, which he rejected while iterating on components (right).
We found that students were able to create different kinds of artifacts by developing custom fabrication processes for their machines. In some cases this involved close integration of manual manipulation and machine fabrication. Yanrong Chen in HCDE 598 created a complex sculpture of interlocking chains and birdhouses, which were printed as interlocking structures by pausing the printer at key moments and inserting previously completed parts. Completing the sculpture involved many tens of hours of print time that were interspersed with regular adjustments or actions made by Yanrong. This process meant that rather than creating a final CAD model of the entire sculpture, she was able to work iteratively, creating a CAD model and printing it only after completing portions of the sculpture and reflecting on their form. In other cases, students extended the functionality of their printers through modifications. Several students in HCDE 598 exchanged components (such as the fans or power supplies) or 3D-printed components to improve performance (such as clips for wire management, holders for work surface illumination, or filament guides). One student in MAT 594X, Mert Toka, modified the interaction of the printer by creating programs that could stream snippets of print commands in real time in response to user input. For his final project he developed a new interface for the printer that used sketching with a tablet to perform semi-real-time operation of the machine. His interface enabled designers to draw freeform curves on different layers, and then produce a series of interpolated curves on the interstitial layers to complete the geometry.
![](/images/uploads/blogs/temp_file_mert_system_sm1.jpg)
Mert Toka’s interactive drawing CAD-CAM system. Link
In considering these outcomes, it’s worth emphasizing that student learning opportunities were not limited by working with less sophisticated equipment and reduced access to professional facilities. To the contrary, students enacted powerful design workflows and successful project outcomes because they were working with cheap printers in their own homes.
![](/images/uploads/blogs/temp_file_YanrongChen-Chains_sm1.jpg)
![](/images/uploads/blogs/temp_file_YanrongChen-InProgress_sm1.jpg)
Yanrong Chen’s sculpture of interlocking chains and birdcages. Her approach to using the machine by pausing it during execution and inserting previously-printed parts enabled a novel workflow.
Personal fabrication during a pandemic
Personal fabrication—wherein individuals to fabricate their own devices through digital tools—is an idea that has been present in maker and technology discourse for at least the past 15 years [4]. Attitudes on the potential of personal fabrication are varied. There’s evidence that suggests many present forms of personal fabrication are primarily restricted to wealthy technology enthusiasts [5], which contrasts with projections that personal fabrication could fundamentally alter manufacturing and consumption trends on a societal level [6]. Personal fabrication also garnered new focus during the pandemic, when people with printers (in many cases the same printer we used in our classes) began to manufacture personal protective equipment for healthcare workers, with a mix of results.
Modules of Della Sigrest’s extensible lamp, designed in Grasshopper and printed with wood-fill filament.
The experiences from our classes offer an alternate case by which to examine personal fabrication. In addition to facilitating hands-on making in a remote instruction context, shipping printers to students’ homes created a situation where students lived with their printers, used them on a daily basis, and created objects for personal use with them. For example, Mengjia Zhu, a MAT 549X student, created a program that generated designs for a self-watering planter that could be adjusted based on the size of the plant and the desired water-immersion level, while incorporating an arbitrary existing outer geometry for the planter basket. Della Sigrest, in HCDE 598, created a modular lamp that integrated with internal lighting components to create different patterns of light diffusion.
![](/images/uploads/blogs/temp_file_mengjia_planter_sm1.jpg)
Mengjia Zhu’s parametric self-watering planter. Link
Della’s lamp and Mengjia’s planter are examples of using at-home, affordable digital fabrication machines to support the creation of custom, beautiful, functional artifacts. However, the machines themselves did not simplify or speed up the process of developing these artifacts. Nor did they support fundamentally new design and manufacturing workflows. To the contrary, such products required students to engage in workflows that reflect elements of real-world design, manufacturing, and craft. Mengjia used four different CAD software tools, along with custom code tools to produce her finished planter model, and she performed rigorous material testing and machine tuning to ensure that her final design would be watertight. Della’s final lamp was the result of 12 successive prototypes. At each stage she systematically explored variations of wall thicknesses and geometric patterning. These rich processes contradict product-focused visions of personal fabrication where consumers create custom objects with minimal effort and knowledge. Instead, they show how personal digital fabrication—similar to other forms of personal craft—is often dependent on deep engagement in learning, design, testing, iteration, and peer support. There’s also evidence that suggests this degree of engagement was not simply the result of students needing to complete assignments. Following the completion of the quarter, students in both courses continued using their printers to make personal objects for their homes, employing prototyping, revision, and experimentation in the process.
The presence of the printers in students’ homes also resulted in changes to the students’ routines and daily activities. Initially, cohabitating with a printer led to some playful responses. Many students in HCDE 598 named their printers, and a student in MAT 594X began referring to her printer as her new pet. Other shifts in routines were more serious. Because students often lived with roommates or occupied small studio apartments, they often kept their printers in their bedrooms. This, coupled with long print times and the fact that the printers generated heat, smells, and machine noises while active, resulted in students coordinating their schedules around their printers. Weidi Zhang, a student MAT 594X, kept her printer in her bedroom, and was (reasonably) concerned about letting the printer run while she was sleeping. This created a situation where she only had about 10 hours to print each day, which led to additional stress when prints failed or produced undesirable results. Such experiences were not uncommon. One student had to re-solder the wires on the stepper motors after her cat chewed through them. Another reported that his roommates also had to adjust to the printer, and commented on its constant noise.
It’s not difficult to envision future scenarios where the constraints of continuous operation of an at-home 3D printer would be infeasible for students. Adjusting assignments so they require less print time, or even choosing a different machine with faster fabrication speeds (e.g., a craft vinyl cutter) might be appropriate choices in the future. More broadly, our courses gave us additional insight into the practical reality of at-home “replicator” technology. While digital fabrication technologies will continue to improve, maintenance, safety, and space concerns will persist, and will manifest in specific ways for people living in small, shared spaces.
Pleasure in making at home
There were elements of at-home 3D printing that provided important forms of stress relief and pleasure. Students in both courses repeatedly expressed their delight at being able to make physical objects and seeing the products made by their classmates. They also talked about the enjoyment and satisfaction they experienced with 3D printing in contrast to the challenges of the quarantine and shutdown, or their experience in other classes. Students remained enthusiastic about the act of physical production in their final project reflections, and many continued to use their printers over the summer. These observations align with the well-established sense of pleasure many people experience from physical handicraft [7]. We suspect that the novelty of the at-home 3D-printer setup played some role in the students’ excitement. Yet it’s also possible that at-home fabrication offered rare opportunities for physical interaction when students were otherwise restricted to videoconferences, chat, and screen-based work in their classes and research.
Pleasure and delight are also fundamentally important to students’ overall well-being. The spring of 2020 contained many sources of stress and anxiety for students. Health concerns, social unrest in response to police brutality, isolation, concerns about graduation and career prospects—all of these issues required us to adjust our teaching approach to support students. This involved new forms of emotional and physical labor on our part, including driving supplies to students’ houses, holding more office hours at arbitrary times, and devoting course time for well-being check-ins and discussions of racial justice. The most extreme labor we undertook involved the restructuring of the courses themselves. Although we felt it was unreasonable to rework a lab-based course in less than two weeks, we made the attempt, at the cost of other projects and personal well-being, because we were concerned about our students. We were not alone in this regard: Many of our colleagues made similar sacrifices for student learning and well-being. If remote physical making courses are to be sustainable in the future, we must find ways to provide educators with the adequate resources and support to develop and execute such classes.
![](/images/uploads/blogs/temp_file_weidi-sculpture_sm1.jpg)
Procedurally created lamps by Weidi Zhang created using photogrammetry data. The sculptures were accepted to the 2020 IEEE VIS Art Program. Link
Planning for Future Remote Fabrication Instruction
Covid-19 will impact future education. At the time of this writing, we are both planning for the possibility of teaching digital fabrication remotely in the 2020–2021 academic year. It is also possible that universities and other educational institutions may preserve some elements of remote instruction after the pandemic subsides. Furthermore, the experience of teaching remotely has enabled us to reflect on our past approach to teaching digital fabrication in person. The following are practical lessons we took away from our experience, which will inform our approach to teaching digital fabrication in the future.
Equipment selection and costs
We had positive experiences with the Ender 3 Pro and would use the printer in future classes. Overall, the printer served as a robust platform for newcomers with a higher-than-expected ceiling for more advanced use cases. The Ender 3 Pro uses the widespread, open-source Marlin control software and its hardware design is partially based on the highly popular, iterated, and open source Prusa i3 printer. As a result, tuning tips, replacement parts, and firmware error messages are widely documented and available. We also had better experiences with these machines than other more expensive machines we currently have access to in shared makerspaces, including machines by Dremel, Monoprice, and Makerbot.
![](/images/uploads/blogs/temp_file_KhangLee-Messager_sm1.jpg)
A messaging user interface with movable parts cast in plaster by Khang Lee. Link
It’s essential to be transparent about the financial factors of our courses. Remote instruction has already exacerbated inequalities for low-income students and institutions [8]. While the printers we used were significantly cheaper than most printers, we recognize that a several hundred dollar equipment cost per student would present a serious burden for many educators and students. Both of our classes had fewer than 20 students, but cost several thousand dollars total. These costs were borne by a combination of students, faculty, and the university. If we were to teach remote digital fabrication classes in the future, we would consider a $350 per student material and machine budget a baseline requirement. Requiring students to cover that cost themselves would be detrimental for student equity and inclusion. We strongly believe the learning opportunities of remote fabrication courses merit institutional funding, but we also recognize that resource constraints will not make this feasible in many cases.
Building a curriculum with multiple pathways
We found that it was crucial to make the coursework accessible and adaptable to the changing situation. We offered flexible fabrication assignments that allowed for multiple levels of engagement. We designed assignments to prompt students to demonstrate course concepts, but left it open for students to choose the application and degree of complexity. For example, an early assignment in Nadya’s course was for students to design cookie cutters. A simple circle would have met the assignment criteria, but many students took the assignment as an opportunity to design and print a highly personalized and complex series of cookie cutters. This enabled them to be creative and invested while also tailoring their projects to the bandwidth they had available that week.
Helping students complete the coursework remotely also required a different approach. Working from home creates competing demands on time and attention (in different degrees for different students), and the challenge of learning physical and tacit forms of production in isolation with limited resources heightens these demands. We compensated for these constraints by providing flexible, asynchronous, and high-bandwidth forms of course engagement, including expanded office hours, peer technical support, expansive written documentation, and multi-tiered assignments. Based on student feedback and course outcomes, we believe these instructional factors were equally critical in achieving student learning outcomes as the printers themselves.
![](/images/uploads/blogs/temp_file_BenChickadel-Retainer_sm1.jpg)
|
![](/images/uploads/blogs/temp_file_CamilaProffitt-Printing-On-Fabric_sm1.jpg)
|
HCDE 598 students Ben Chickadel and Camila Proffitt used the class projects as an opportunity to make things for their family members. Ben created a mold and cast a series of example dental retainer parts (left) his wife could use in her dental practice. Camila Proffitt made a series of complex hats with 3D printed structures and embellishments printed directly on fabric (right) for her mother.
We recognize that written instruction materials, online documentation of student projects, asynchronous communication platforms, and peer support and feedback are often components of in-person courses (our past courses included). However, the pandemic required us to rely on these elements in a new way. Rather than serve as a means to reinforce content and interactions from in-person class sessions, we depended on asynchronous platforms to provide essential forms of support and instruction that would not have been possible in remote synchronous lectures or demonstrations. Furthermore, the quantity of support we provided was substantially greater than that of previous in-person classes. We strongly suspect that both the variety and quantity of learning support provided in our remote classes would also improve learning outcomes in in-person digital fabrication courses. However, these efforts required substantially more instructor labor, not all of which can be rolled over into future courses.
Integrating well-being and learning concerns
We are still in a crisis. Initial attempts to re-open universities to students have shown the significant health and safety risks of resuming regular campus operations. Like the courses of our colleagues, our classes were first and foremost a method for safely teaching physical making without any in-person elements. Setting up mini-makerspaces in student homes enabled that. But in addition to the critical safety benefit of avoiding in-person contact, we believe the success of our courses hinged on being attentive to other forms of student well-being. Regular check-ins and peer learning components were designed to cultivate a sense of community and engagement. Flexible expectations on assignments and course attendance were designed to accommodate students’ need for privacy and rest and to account for the competing demands on their time at home. These considerations for student well-being cannot be disentangled from the learning objectives and outcomes of our courses.
In our remote digital fabrication classes we were still able to teach the core elements of CAD, CAM, and CNC production. We are proud of our students’ work and the efforts they took to support one another. Although our approach is new, we suspect that the potential of remote digital fabrication instruction is broad. At-home mini-makerspaces could offer remote learning opportunities for students across art, engineering, science, and design departments. Furthermore, this model may offer new learning opportunities in machine operation and maintenance, experimentation, and design iteration that are less feasible in shared makerspaces. Yet if we wish to create future learning opportunities that are sustainable for instructors and effective for students, it is also necessary to be realistic about the substantial costs and labor that remote fabrication entails. As our collective experiment with remote learning continues, it is our sincere hope that physical making classes will not disappear, and the safety and well-being of students, staff, and faculty will be maintained.
Endnotes
1. Thompson, R. Manufacturing Processes for Design Professionals. Thames & Hudson, London, 2007.
2. Jacobs, J. and Buechley, L. Codeable objects: Computational design and digital fabrication for novice programmers. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 2013, 1589–1598; >https://doi.org/10.1145/2470654.2466211
3. Peek, N., Coleman, J., Moyer, I., and Gershenfeld, N. Cardboard machine kit: Modules for the rapid prototyping of rapid prototyping machines. Proc. of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2017, 3657–3668.
4. Gershenfeld, N. How to make almost anything: The digital fabrication revolution. Foreign Aff. 91, 43 (2012).
5. Buechley, L. Thinking about making. Presented at EYEO 2014; http://leahbuechley.com/?p=16
6. Anderson, C. In the next industrial revolution, atoms are the new bits. Wired, Jan. 25, 2010; >https://www.wired.com/2010/01/ff_newrevolution/
7. McCullough, M. Abstracting Craft: The Practiced Digital Hand. MIT Press, U.K., 1998.
8. Goldstein, D., Popescu, A., and Hannah-Jones, N. As school moves online, many students stay logged out. The New York Times. Apr. 6, 2020; https://www.nytimes.com/2020/04/06/us/coronavirus-schools-attendance-absent.html
Posted in:
Covid-19 on Fri, September 04, 2020 - 11:29:41
Jennifer Jacobs
Jennifer Jacobs is an assistant professor in Media Arts and Technology at UC Santa Barbara. She studies ways to support expressive computer-aided design and manufacturing by developing new computational tools that integrate computer programming with traditional materials and manual control. She received her Ph.D. from the MIT Media Lab and her research has been presented at international venues including SIGGRAPH, CACM, Ars Electronica and and CHI.
[email protected]
View All Jennifer Jacobs's Posts
Nadya Peek
Nadya Peek is an assistant professor in the Department of Human Centered Design & Engineering (HCDE), where she directs the Machine Agency. Her work focuses on unconventional digital fabrication tools, small scale automation, networked control systems, and advanced manufacturing. Spanning electronics, firmware, software, and mechanics, her research focuses on harnessing the precision of machines for the creativity of individuals.
[email protected]
View All Nadya Peek's Posts
Planning and designing for the inevitable
Authors:
Pallabi Roy Singh
Posted: Wed, August 26, 2020 - 12:08:58
If there is one big lesson from Covid-19, it is that we live in a world of risk. Disasters and epidemics and pandemics are part and parcel of the fabric of our existence—although that recognition makes them no less tragic when they occur. The Covid-19 catastrophe can be traced back to the actions of only a handful of people, yet it is something that attests to humankind’s vulnerability before powerful individuals. In fact, the chaos that Covid-19 has brought to the world has been more devastating than that of any other previously occurring catastrophe. However, it cannot be the end; there is a glimmer of light at the end of the tunnel. Humanity is still there and thriving. Viewed as such, Covid-19 will prove to be as much a testimony to human durability as it will be to human vulnerability.
Having said that, I strongly believe that disaster preparedness and emergency management are the need of the hour. And, as user experience professionals, it is probably the right and the best time for us to start designing and building products that can help the world avoid the pitfalls of a catastrophe—or at least manage our way out of one. So, let's gear up and get ourselves out.
The widespread ownership of smart mobile devices means that disaster communication has reached a new level in terms of ease, speed, and quantity of information sharing. Mobile applications have become ubiquitous, and people are well versed in using them. If we look at the various mobile applications for disaster management today, there are already quite a few. These applications broadly fall into different categories, such as alert generation, emergency preparedness, guide mapping to nearby help centers, simulation-based impact modeling and risk assessment, crisis mapping, geo-tagged situational information and crowdsourcing, volunteer tracking and management, and family tracing and reunification. And these are really good applications! So, before we begin to design and develop an application to combat disasters, we must see how we can adopt a best-of-breed approach and come up with a solution for general users who can easily find support in an emergency.
Best practices for designing a disaster-management application
Designing a disaster-management application can seem like a very promising proposal, but there are some challenges. For example, the user base of a disaster-management application is strong and diverse, so the design must suit the needs of different groups. However, if we adhere to the following best practices we can ensure that the design of the application is relevant and useful for all groups of users:
- The research strategy, as an overarching view of research needs and priorities, forms the basis of a research plan. Therefore, devising a research strategy should be the first step in the design process. Studies have shown that the development of a disaster-management application is often geared toward showcasing technological innovation rather than meeting user requirements. Therefore, before proceeding further, we should review our research objectives and questions with a few hazard and disaster researchers and emergency-management practitioners because they are better equipped to shed light on the diverse needs of users dealing with a disaster or emergency. In other words, understanding user psychology in response to disasters is a fundamental prerequisite for designing an application. It will help us in coming up with a better approach to our research and in prioritizing the features to be included in the application.
- While designing a disaster-management application, we must ensure that it successfully complies with the heuristics. If we remember to follow Jakob Nielsen's 10 general principles for interaction design, we are sure to come up with an aesthetically pleasing, minimalist design—one that provides a lot of flexibility to users and has the benefit of recognition over recall. We must keep in mind that the user base of a disaster-management application will be large and varied; it will be used by all groups of people in case of an emergency, from educated to uneducated, from young to old. So the simpler the design the better. Simple designs also reduce the workload on the server, which can improve load times even more.
- Ensure that the application is easy to use and easy to understand for everyone—it can save lives when it matters the most. It is an established fact that in order to provide a truly comfortable user experience, applications should offer some language support, which may involve some form of content localization. However, the prospect of having to manage and support a number of languages is too daunting, especially without the budgets or expertise to embrace localization. The key here is to invite altruism. Volunteers sometimes decide to contribute some of their time to localize content. So, volunteer-based collaborative translation or crowdsourcing can play a big role in simplifying the content for the native users of a disaster-management application.
- Color coding of UI elements plays a critical role in a disaster-management application because it helps the users see and easily interpret the alert levels. For example, if we are designing a pre-disaster warning application, we can use color-coded images on the interactive map of the locations affected by or under the threat of any kind of disaster. Similarly, we can use color codes in the application to generate alerts, alarms, or notifications according to different alert levels.
Color
|
Alert Level
|
Description
|
White
|
Information only
|
Disaster identified
Monitoring and watching
For information only; no impact expected
|
Yellow
|
Moderate
|
Disaster impending
Warning, monitoring, and watching
Preparedness phase
|
Orange
|
Severe
|
Disaster threat imminent
Preparedness phase
|
Red
|
Extreme
|
Response and action phase
|
Green
|
Normal
|
All clear
Cancellation of warning
|
- The use of intuitive icons can play a big role in improving the usability of a disaster-management application. While it might not be possible to design an icon entirely from scratch, it should still be possible to choose an icon that encompasses both sign-like and symbol-like properties. The wide spectrum of users of a disaster-management application require universally recognizable (symbol-like) icons that also represent what they purport to represent (sign-like). Therefore, we should use icons in our designs that are obvious and intuitive across a range of cultures and experiences, but at the same time so basic as to be innately recognized.
- Including FAQs and interactive learning materials in a disaster-management application could prove useful in preparing users for unforeseen eventualities. It will establish the resourcefulness of the application. For example, we can post a list of frequently asked questions selected based on our research data, a list of guidelines and an information glossary, visualizations and maps of real-time data, and video tutorials and illustrations to guide users in crisis situations.
- Use of advanced communication technology is important to ensure proactive and strategic communication in real time. For example, advanced SOS, emergency signaling capability, and live video capture will help to ensure a two-way communication mechanism to address the needs of users in a crisis. From the design perspective, the placement of the UI controls related to these features is critical; they should be placed in the application window such that they are easily findable.
Studies have proven that there is huge potential for online and mobile emergency solutions to be used by a broad range of users. However, they also highlight the need for advertising these apps and educating people about them, as well as the need for ensuring users of the privacy and security of these solutions. If we can ensure that all these principles are in place in our designs, I believe we will be well prepared to fight any crisis like Covid-19 in the future.
Posted in:
Covid-19 on Wed, August 26, 2020 - 12:08:58
Pallabi Roy Singh
Pallabi Roy Singh is a UX researcher, usability analyst, and content strategist at Cadence Design Systems, where she leads numerous design projects for desktop applications. Before joining Cadence, she had worked in TCS and Ericsson. Her articles and research papers have been published in many national and international journals.
[email protected]
View All Pallabi Roy Singh's Posts
Auto-UI: Global perspectives
Authors:
Christian Janssen,
Ronald Schroeter,
Nic Bidwell,
Yong Gu Ji,
Ignacio Alvarez,
Shan Bao,
Myounghoon Jeon,
Linda Boyle,
Stella Donker,
Lewis Chuang,
Wendy Ju,
Andrew Kun
Posted: Wed, August 12, 2020 - 10:40:49
ACM SIGCHI Auto-UI is a growing community, but one in which some continents were less involved than expected and hoped for. For the 2019 conference in Utrecht, the Netherlands, we made various targeted attempts to grow and diversify our international community, with support from the ACM SIGCHI Development Fund. Our efforts resulted in a growth in the number of Asian participants, which made up almost 20 percent of the attendees. In this blog, we briefly reflect on our initiatives and on a panel discussion focusing on research topics that matter more globally to the Auto-UI field.
How we reached out
Based on discussions with members from underrepresented groups, we:
- formed a diversity, inclusion, and international outreach team to contact key partners;
- provided welcome pages in various languages to give a quick overview of the conference (www.auto-ui.org/19). These helped with local promotion;
- posted on local social media such as Kakao (Korea) and WeChat (China);
- broadened our pool of associate chairs through an open call and by promoting reviewers from target countries that had done good reviewing work in the past; and
- awarded travel fellowships to graduate students from target countries.
Global perspectives panel: Discussion points
At the conference, a panel of established members from academia and industry discussed “global perspectives.” The panel had combined professional experience on five continents:
- Ronald Schroeter (Queensland University of Technology, Australia)
- Nicola Bidwell (International University of Management, Namibia)
- Yong Gu Ji (Yonsei University, Korea)
- Ignacio Alvarez (Intel Corporation, U.S.)
Next we briefly summarize the discussion topics.
What is a car?
There are differences in what is considered as a car and what a car means for people between countries and cultures. For example, in Namibia, people need to drive long distances between towns. As many people do not own cars and bus travel is not always affordable or convenient, a tradition of shared mini-bus taxis and car sharing has emerged, with technology used to arrange it. People have re-appropriated Facebook and WhatsApp to coordinate seats in cars (and also delivering things for people); this sharing ties into local customs and African philosophies of sharing.
By contrast, in Korea, the culture is strongly focused on people owning their own car. Ignacio Alvarez shared his industry experience from China and the U.S. Although in both cultures the car is often a means to an end, it also represents a form of freedom and individuality. More and more, it is also becoming a status symbol. In China, for example, a car might also just be parked on the sidewalk to demonstrate the wealth of its owner.
How is culture guiding automotive interactions?
Culture is not static, but alive. Cultural norms and expectations can also change within a person’s lifetime. This in turn can impact automotive interactions. For example, when one becomes a parent, one might have other views on what features of a car are important (e.g., safety instead of speed).
Sometimes, expectations are wrong. For example, in Korea, many interfaces are in English, not Korean. Although this is perceived to look cool among locals, it can hinder understandability and thereby hinder user experience and safety.
By contrast, African cultures, such as in Namibia, often promote collectivism, which contrasts with Korea’s stronger focus on the individual. In Korea, services for sharing cars are not that popular, whereas in Namibia they are essential. At the same time, despite the focus on the individual, within Korea there are also popular services in which one can order a driver to drive one’s car when one cannot drive it themselves. Yet cars are not always designed for use by someone else than the owner.
What can be learned from the Global South?
We gained several insights from the Global South. In particular, we learned about African cultures of repair (to make things last) and practically hacking solutions. We also saw a focus on designing for human values, with a strong social ethic to not exclude others. Many cars in Namibia and Africa are bought secondhand from Asia. All cars are a little different, and people who specialize in repairs of specific types or brands of cars often live far apart. However, they benefit from strong online communities that share information on fixing electronics. This sharing focus relates to the CSCW community’s focus on shared work, while the hacking movement aligns with the maker movement within CHI.
The resilient repair communities in the Global South might also inspire opportunities in an age where automated systems are being developed. Specifically, the resilience of the phone-repair culture in India, China, and Namibia shows that people with initially little domain knowledge can quickly learn from each other. Rather than specializing in one skill, through a network of collaborators, people can learn from each other and exchange knowledge and experiences.
Another consideration is that the design of a car is typically catered to its “first life” in the original country of purchase. However, car reuse in Africa is not considered sufficiently, even though such extended use of technology might be beneficial in times of climate change. The pictures below of a Japanese car in Namibia illustrate two examples: 1) Controls are partially in English and partially in Japanese, and 2) the satellite navigation system only has maps from Japan, and thinks the car is driving near the ocean in Japan instead of on a Namibian road. Both aspects harm the driver’s safety and comfort.
![](/images/uploads/blogs/temp_file_Fig_11.jpg)
Example of a car that seems to have been designed for its first life only. This car was originally released on the Japanese market but now drives in Namibia. The controls of the car are partially in symbols, partially in English, and partially in Japanese, thereby creating confusion for the driver in Namibia who cannot read Japanese.
![](/images/uploads/blogs/Fig_2.jpg)
Another example of a car that seems to have been designed for the first life only. This car was originally released on the Japanese market but now drives in Namibia. The satellite navigation system only has maps of Japan, and not of Namibia. Therefore, the in-car satellite navigation system thinks that the car is driving near the ocean in an area in Japan, instead of on a road in Namibia.
Reuse also comes with downsides, as current design does not consider this context sufficiently. In particular, some of the material used in cars and other electronic devices is toxic. In many African countries, there is not sufficient protection for the makers to handle these substances—and some of these makers are underage children.
What are the hot topics?
Within Korea, there is a culture of early adoption of new technology. Combined with the many smart infrastructures and high population density, it is an interesting country in which to test new interaction styles and advanced automated vehicle forms. This is in line with trends within the Auto-UI community, in which there is a focus on automated driving from many angles, including the human user, other traffic participants, and the larger ecology and infrastructure.
At the same time, within Korea there is also an interest in understanding the basic science and engineering of interaction techniques. For example, how can touch- and audio-based in-car interaction be improved? Progress in this area requires more fundamental science and engineering research.
More globally, a hot topic is automated driving safety. Yet, despite this growing interest, there remains a fundamental challenge: Safety is understood in different manners in different regions of the world and in different cultures. Some countries are more risk tolerant than others and for industry to satisfy both the utilitarian aspect of vehicles as well as a global notion of safety is sometimes challenging.
We welcome suggestions
Our outreach efforts are only a first step. We welcome suggestions on how to further improve the experience for conference attendees at Auto-UI 2020 (U.S.) and Auto-UI 2021 (South Korea). Suggestions can be emailed to the authors and to the Auto-UI steering committee: [email protected].
Posted in:
on Wed, August 12, 2020 - 10:40:49
Christian Janssen
Christian P. Janssen is an assistant professor of experimental psychology at Utrecht University. He received his Ph.D. in human-computer interaction from UCL (2012). His research is dedicated to understanding human attention, human-automation interaction, and adaptive behavior. He was one of the general chairs of the ACM SIGCHI’s Auto-UI 2019 conference.
[email protected]
View All Christian Janssen's Posts
Ronald Schroeter
Ronald Schroeter is a Senior Research Fellow at CARRS-Q, Queensland University of Technology, where he obtained his Ph.D. in 2012. He embraces multidisciplinary research across HCI/HMI, design, psychology, and road safety. He has served in various chairing roles of the Auto-UI conference since 2013, including Diversity and Inclusion Chair in 2019.
[email protected]
View All Ronald Schroeter's Posts
Nic Bidwell
Nic Bidwell is an adjunct professor in the Faculty of ICT at International University of Management, Namibia. Her research focuses on interaction design for rural communities, mostly in Africa and/or with indigenous people. She was an invited panelist at the Auto-UI 2019 conference.
[email protected]
View All Nic Bidwell's Posts
Yong Gu Ji
Yong Gu Ji is a professor and head in the Department of Industrial Engineering at Yonsei University. He is the director of the Human Factors and Interaction Design Lab. He received his Ph.D. in human-computer interaction from Purdue University (2001). His research is directed at understanding user experience and complexity in human-vehicle interaction. He is one of the general chairs of the Auto-UI 2021 conference.
[email protected]
View All Yong Gu Ji's Posts
Ignacio Alvarez
Ignacio Alvarez is a senior research scientist at the Autonomous Driving Research Lab in Intel Labs, where he develops software, system, and simulation Tools to accelerate the adoption of safe automated driving technologies. Previous to Intel, he worked for eight years at BMW, leading R&D and product development for advanced driver assistance systems, vehicle telematics services, and user interface solutions in Europe, America, and Asia. He received his international Ph.D. in computer science from the University of the Basque Country (Spain) and Clemson University (U.S.) in 2011. His research is focused on the development of intelligent connected automated vehicles that augment human mobility with safer and more enjoyable experiences.
[email protected]
View All Ignacio Alvarez's Posts
Shan Bao
Shan Bao is an associate professor in the Department of Industrial and Manufacturing Systems Engineering, University of Michigan-Dearborn, and an associate research scientist in the University of Michigan Transportation Research Institute. She received her Ph.D. in mechanical and industrial engineering from the University of Iowa in 2009. Her research interests focus on human factors issues related to connected and automated vehicle technologies, and big data analysis.
[email protected]
View All Shan Bao's Posts
Myounghoon Jeon
Myounghoon Jeon is an associate professor of industrial and systems engineering and computer science at Virginia Tech. He received his Ph.D. in engineering psychology and human-computer interaction from Georgia Tech. His research is dedicated to designing better interactions with robots and vehicles with focus on sound and emotion. He was the diversity and inclusion co-chair of the Auto-UI 2019 and 2020 conferences, and is the general co-chair of the Auto-UI 2021 conference.
[email protected]
View All Myounghoon Jeon's Posts
Linda Boyle
Linda Ng Boyle is a professor at the University of Washington. She is a co-organizer of the International Symposium on Human Factors in Driving Assessment and on the steering committee for Automotive User Interface. She is a co-author of the book Designing for People: An Introduction to Human Factors Engineering.
[email protected]
View All Linda Boyle's Posts
Stella Donker
Stella F. Donker is an associate professor of experimental psychology at Utrecht University. She received her Ph.D. in medical sciences from the University of Groningen (2002). She is interested in human movement, attention, and human-machine interaction. She was one of the general chairs of the Auto-UI 2019 conference.
[email protected]
View All Stella Donker's Posts
Lewis Chuang
Lewis L. Chuang is an Akademischer Rat at the Institute of Media Informatics at LMU Munich. He received his Ph.D. in behavioral neuroscience from Eberhard-Karls-Universität Tübingen. He investigates mechanisms of focused attention and reorienting, especially in the context of human-machine interactions. He relies extensively on gaze tracking, EEG, and psychophysics in his research. He is co-organizing the 2020 Neuroergonomics Conference.
[email protected]
View All Lewis Chuang's Posts
Wendy Ju
Wendy Ju is an assistant professor at the Jacobs Technion-Cornell Institute at Cornell Tech and in the Information Science field at Cornell University. She has innovated numerous methods for early-stage prototyping of automated systems to understand how people will respond to systems before the systems are built. She has a Ph.D. in mechanical engineering from Stanford, and a master’s in Media Arts and Sciences from MIT. Her monograph The Design of Implicit Interactions was published in 2015.
[email protected]
View All Wendy Ju's Posts
Andrew Kun
Andrew L. Kun is professor of electrical and computer engineering at the University of New Hampshire. His primary research interest is in-vehicle human-computer interaction. He serves as the ACM SIGCHI interim vice president for conferences, and as steering committee co-chair of the ACM AutomotiveUI conference series.
[email protected]
View All Andrew Kun's Posts
Learning and education in HCI: A reflection on the SIG at CHI 2019
Authors:
Viktoria Pammer-Schindler,
Erik Harpstead,
Benjamin Xie,
Betsy DiSalvo,
Ahmed Kharrufa,
Petr Slovak,
Amy Ogan,
Joseph Jay Williams,
Michael Lee
Posted: Tue, August 04, 2020 - 11:09:15
The field of human-computer interaction (HCI) has always been interested in aspects of learning. HCI researchers have spent decades investigating how people learn to use interfaces, with designing for learnability being a key HCI design principle. And with technology constantly changing, everyone from developers to end users must constantly (re)learn how to use, adapt, and improve the tools around them.
More recently, there has been an even greater interest in learning within HCI communities, perhaps best evidenced by the increase in learning-relevant papers submitted to CHI. This, in turn, led to the creation of the newly established Learning, Education, and Families subcommittee, with 191 papers submitted in its first year at CHI 2019, and 179 papers submitted for CHI 2020. Many factors contributed to this boom: more learning scientists engaging with the HCI community, a renewed university-level emphasis on online learning technologies (e.g., MOOCs, learning-management systems), and an increased interest and awareness of the necessity of lifelong learning and the possibilities of using computer technologies to support this need. This interest, in turn, has surfaced the realization that a) user-centered design is critical for educational technologies, which may be particularly research-intensive when involving more novel technologies such as mixed reality or AI-based systems, and b) designing for learning has specific design and interaction issues that go beyond more generally applicable design knowledge.
The SIG at CHI 2019
At CHI 2019 we organized a Special Interest Group (SIG) on Learning, Education, and HCI to bring together researchers across the CHI community and discuss the position of learning within the field [1]. Our goal was to gather a broad set of perspectives from across the field and better understand what community members mean when they talk about learning. Here are some of the key insights from that SIG:
- HCI researchers interested in learning and education design for a vastly heterogeneous set of learning and educational settings. Subsequently, in order to enable communication and cross-fertilization in this HCI subcommunity, a clear foundation in learning sciences should be presented with each research contribution. This addresses the interdisciplinary challenge of designing for learning, which means integrating knowledge from different disciplines, while at the same time allowing the HCI community as a whole to increase shared knowledge about learning.
- HCI researchers are aware of the tensions between joy and usefulness in learning, and of the complex issue of motivating and nudging learners in order to foster learning that is as engaging and effective as possible.
- HCI contributions on learning and education may need to be exploratory, qualitative, and longitudinal rather than quantitative and experimentally comparative, especially if the contributions involve novel interventions to promote learning. This exploratory nature is common in HCI research; in this inherently multidisciplinary domain, such an approach needs to be further integrated and aligned with additional appropriate methodologies.
- HCI can bring to learning and instructional sciences significant knowledge about the design of interventions, such as methodological knowledge about participatory design and similar design and research methods. HCI can also bring design knowledge, such as affordances of different types of technologies, as well as usability and user experience best practices.
The SIG was organized as three roundtable discussions, such that at every point in time each of the following three questions was being discussed at two or three tables:
- What does learning mean to us?
- What are valid ways of evaluating a learning contribution?
- How can we foster the relationship between learning and HCI?
Results of Discussions
Below we summarize the discussions had on these questions by over 50 discussion participants at CHI 2019 in Glasgow and highlight overarching issues for research in human-computer interaction on learning and education.
What does learning mean to us? This question turned out to be successful at highlighting the wide variety of different types of learning that HCI researchers are interested in, such as: learning for different age groups; formal and informal learning; understanding learning from the perspective of different learning theories, such as constructionism, behaviorism, or sociocultural learning theories; and designing for learning based on different paradigms for structuring learning, such as experiential or collaborative learning.
The major insight, at a meta level, that we draw from this face-to-face discussion is that interdisciplinary discussion in HCI on learning needs to be open to a wide variety of learning and instructional theoretical backgrounds; at the same time, we need to ensure that discussions of learning remain accessible to the wide variety of HCI researchers. The discussion indicated that to meet both goals—inclusivity and accessibility—every learning contribution in HCI would need to present a solid explanation of its learning and instructional sciences background. While this may feel redundant to us, some scaffolding of learning theory is needed to communicate the assumptions underlying the work for other HCI researchers interested in learning and education.
In parallel, in an even more heterogeneous fashion, this question led SIG participants to reflect on issues that are not only fact-based but that also relate to values with respect to learning, such as whether HCI designers and researchers should prioritize engagement over effectiveness, or how much guidance versus how much freedom and associated challenge is suitable for learners, acknowledging that every dichotomy can be resolved by answering “both.” Even acknowledging foundational insights from learning and instructional sciences, for example on the effectiveness of guidance, these questions and others remain issues of concrete complexity in every single instance of technology design. HCI researchers therefore emphasize that one way to move forward is to look at persuasive and motivational design, which aims to incite motivation in learners to achieve as much joy and usefulness as possible at the same time.
Finally, we identified a fundamental tension: learning versus education. Participants tended to share the view that the two are not the same; that while the former emphasizes the change within the learning entities (and despite acknowledgement that learning can happen at an informal team or formal organizational level, the focus of most participants was on human individuals as learning entities), the latter emphasizes the design of formal environments and structures that lead to learning. It would seem then, in principle, that HCI researchers who explore how to set up computational environments that are conducive to learning should feel closer to education; however, this was not the prevalent feeling. This is an interesting development that we can only ask the community of researchers interested in the interdisciplinary field to take up in argument and discussion.
What are valid ways of evaluating a learning contribution? This question was discussed in order to understand the extent to which there was a shared agreement on what kinds of contributions the subcommunity of HCI research on learning and education is uniquely positioned to make. We highlight here that this question was not intended to provoke general discussion on what a CHI contribution is, or to question other research communities’ understanding of the matter.
From a learning viewpoint, what constitutes a suitable evaluation hinges on the goal of the intervention, the perspective on learning, and the learning context. For instance, from a cognitive perspective, a learning intervention can be evaluated in terms of performance of the learner on pre- and posttests of knowledge. On the other hand, from a sociocultural perspective, an appropriate study would consider a learner’s broader context, including social norms, learning culture, and available artifacts in the learner’s environment. Inherent to theoretical framings are assumptions as to what factors are important to learning and how to evaluate them. However, beyond underlying theories, a point of discussion was that while HCI researchers focus on designing technologies for or with learners, learners seem to be excluded from the earlier, yet crucial, stage of defining the goals of an intervention. This has important consequences for evaluation, in that researchers should evaluate not only what they think is important, but also whether the learners themselves have met their own goals. What researchers consider measures of success or failure of an intervention may be very different from how learners value an intervention. Consequently, the validity of learners’ assessment of an intervention as part of a scientific evaluation of the intervention remained a contentious issue among the workshop participants. This is an example of how exploratory and participatory design methods need to be integrated and aligned with other research methodologies, such that interventions can be suitably evaluated from multiple perspectives.
Beyond this, the question remains of what particular research approaches are specific and inherent to research at the intersection of learning sciences, instructional sciences, and HCI. A major realization was that HCI research is ultimately interventionist, as it is design oriented. So, while deep understanding of given contexts is relevant to design, the ultimate goal of HCI is to construct design-relevant knowledge. Thus, it is critical to focus on the learning process in addition to outcomes. This was highlighted as especially important when the goal is developing skills rather than knowledge. Examining the process helps identify what was termed “struggle points” throughout the learning process and how these relate to, or can be mitigated by, the introduction of technology.
When talking about formal education in schools, the schools’ culture and disposition to learning, plus the level of emphasis on summative tests and ways of evidencing learning (e.g., high school versus primary) are found to have significant consequences on the design and evaluation of educational technology interventions.
How can we foster the relationship between learning and HCI? While learning has always been a subcurrent within HCI, the modern incarnation of the field of educational technology has largely grown around several subcommunities adjacent to but separate from HCI. In thinking about how we might build more connections between these communities and HCI more broadly, we asked what learning work might be able to contribute to HCI and what HCI work can contribute to learning sciences.
In some discussions, participants expressed how learning theories and practices might inform HCI and vice versa. For example, in the HCI traditions of participatory design, learning for the participants is one of the key elements, but few designers look to literature on learning to help structure their participatory design activities. Learning scientists see potential for their theories and knowledge to help shape this work [2]. In addition, participatory design practices have much to teach the learning sciences in terms of ways to invite teachers, administrators, parents, students, and stakeholders to be more involved in design or educational technology and information [3].
Another example is in learning analytics research, which has only recently started to substantially take up HCI methods to design and evaluate analytics and visualizations, such that in learning analytics literature, using HCI methods for design and analysis is considered a novel contribution (e.g., [4,5]). This illustrates that even between two deeply specialized fields such as analytics and HCI, which both could be considered to be highly related to computer science, a constant interdisciplinary communication and transfer of knowledge is necessary and challenging.
Another perspective gained from this question was the possibility of leveraging new insights from learning and education research to address a need for updated curricula around how to teach HCI concepts themselves. The landscape of HCI pedagogy has become broad, multi-institutional, and international. We see some initial work that highlights the potential for work on learning to inform how to judge which forms of HCI education are more or less effective for which uses, as well as inform best practices in the sharing of instructional materials [6]. As HCI education develops, the community needs to ask: What are core learning goals [7]? What is the balance between practice and research [8]? How do we identify the most successful pedagogy when teaching traditions in computer science, psychology, and design can be so divergent [9]?
An overarching issue in discussions was the relationship between research and (educational) practice that seems to be particularly relevant for this subcommunity. To understand learning and education, and to observe the expected impact, significant interaction with stakeholders outside of labs is required. While we recognize that laboratory studies have their place in HCI and other communities, we also recognize that the context of educational technology is in a broader practice of education and adoption and use. Though we can build technologies, test them in the lab, and prove that learning happens if they are used, we also need HCI research to design for teacher, administrator, and parent use so they are able and motivated to provide access to students from all walks of life.
We conclude with reiterating the goals of this SIG and the larger initiative to recognize learning and education within HCI research. We want to 1) discuss inclusive cross-disciplinary perspectives on learning, 2) define future directions and qualities for learning and education contributions in CHI, and 3) build a community across research/practice boundaries. By doing so, our hope is to not be stringent or imposing on what learning is or is not, while still basing HCI research on what is established knowledge in the learning sciences.
Moving Forward
The roundtable discussions were a highly interactive and rewarding format for discussions, such that at any time there were at least eight different discussions ongoing in the room. This was despite the fact that there were only three core questions asked, such that multiple tables, and multiple rounds of discussions, engaged with the same question. This highlights the high interest in learning within the HCI community, and the timeliness of the special interest group and the new subcommittee for learning, education, and families established at CHI 2019, and continued at CHI 2020.
Discussions at any time were also of a very heterogeneous level, showing that there is, as yet, no shared systematic way of answering the foundational questions of what learning means in an HCI community, how to foster learning through technology design, and specific characteristics of a high-quality contribution to the intersection between learning and HCI. We see this as a push, for ourselves as well as others researching and practicing at the intersection of learning and instructional sciences and HCI, to continue interdisciplinary discussion, and at the same time try to identify particular knowledge and perspectives that come out of the intersection of HCI and learning research.
Endnotes
1. Xie, B., Harpstead, E., DiSalvo, B., Slovak, P., Kharrufa, A., Lee, M.J., Pammer-Schindler, V., Ogan, A., and Williams, J.J. Learning, education, and HCI. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, Paper SIG09. ACM, New York, 2019.
2. DiSalvo, B. Participatory design through a learning science lens. Proc. of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 2016.
3. DiSalvo, B., Yip, J., Bonsignore, E., and DiSalvo, C. Participatory design for learning. In Participatory Design for Learning. Routledge, 2017, 3–6.
4. Ahn, J., Campos, F., Hays, M., and DiGiacomo, D. Designing in context: Reaching beyond usability in learning analytics dashboard design. Journal of Learning Analytics6, 2 (2019), 70–85.
5. Buckingham Shum, S., Ferguson, R., and Martinez-Maldonado, R. Human-centred learning analytics. Journal of Learning Analytics6, 2 (2019), 1–9.
6. Vorvoreanu, M., Gray, C.M., Parsons, P., and Rasche, N. Advancing UX education: A model for integrated studio pedagogy. Proc. of the 2017 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, 2017.
7. Churchill, E., Preece, J., and Bowser, A. Developing a living HCI curriculum to support a global community. Extended Abstracts on Human Factors in Computing Systems. ACM, 2014.
8. Gray, C.M., Stolterman, E., and Siegel, M.A. Reprioritizing the relationship between HCI research and practice: Bubble-up and trickle-down effects. Proc. of the 2014 Conference on Designing Interactive Systems. ACM, 2014.
9. Wilcox, L., DiSalvo, B., Hennemann, D., and Wang, Q. Design in the HCI classroom: Setting a research agenda. Proc. of the 2019 on Designing Interactive Systems Conference. ACM, 2019.
Posted in:
on Tue, August 04, 2020 - 11:09:15
Viktoria Pammer-Schindler
Viktoria Pammer-Schindler is an associate professor at Graz University of Technology. She researches, and develops sociotechnical interventions for, digital transformation, with a focus on workplace learning and knowledge construction in different workplace contexts, such as modern manufacturing, or in strategic development of data-centric business models.
[email protected]
View All Viktoria Pammer-Schindler's Posts
Erik Harpstead
Erik Harpstead is a systems scientist in the HCI Institute at Carnegie Mellon University. His work focuses on leveraging computational theories of human learning to develop smarter tools and processes for designers of educational technologies and games to interrogate their products and consider how well they manifest designers’ intentions.
[email protected]
View All Erik Harpstead's Posts
Benjamin Xie
Betsy DiSalvo
Betsy DiSalvo is an associate professor in the School of Interactive Computing at Georgia Institute of Technology. At Georgia Tech she leads the Culture and Technology (CAT) Lab, where researchers study cultural values and how those values impact technology use, learning, and production.
[email protected]
View All Betsy DiSalvo's Posts
Ahmed Kharrufa
Ahmed Kharrufa is a lecturer in interaction design at Newcastle University, where he leads educational technology research at Open Lab. His research focuses on the design, development, implementation, and evaluation of processes and technologies than can bridge the gap between schools and their communities as well as enhance learning and the learning experience.
[email protected]
View All Ahmed Kharrufa's Posts
Petr Slovak
Petr Slovak is an assistant professor in human-computer interaction at King’s College London. He also holds an Honorary Research Fellow position at Evidence-Based Practice Unit at UCL and a Visiting position at the Human-Centred Computing group at Oxford University. His research is focused on envisioning, designing, and evaluating new technology-enabled mental health interventions.
[email protected]
View All Petr Slovak's Posts
Amy Ogan
Amy Ogan is the Thomas and Lydia Moran Assistant Professor of Learning Science in the Human-Computer Interaction Institute at Carnegie Mellon University. She is an educational technologist focusing on ways to make learning experiences more engaging, effective, and enjoyable.
[email protected]
View All Amy Ogan's Posts
Joseph Jay Williams
Joseph Jay Williams is an assistant professor in HCI and computer science, designing intelligent adaptive educational and health technology by using randomized A/B comparisons to bridge statistical machine learning with crowdsourcing and psychology.
[email protected]
View All Joseph Jay Williams's Posts
Michael Lee
Michael J. Lee is the Dorman-Bloom Assistant Professor of Informatics at the New Jersey Institute of Technology (NJIT). There, he directs the Gidget Lab, which focuses on designing, creating, and testing technology-focused educational tools for all. His research is funded by the National Science Foundation and Oculus Research, and has received several best paper awards.
[email protected]
View All Michael Lee's Posts
Online embodied interaction: Learning physical interaction design online
Authors:
Annika Waern,
Andreas Bergqvist,
Shuang Feng,
Nikolay Georgiev,
Karan Manjunath,
Alessandra Semeraro,
Ruochen Wang,
Laia Turmo Vidal
Posted: Mon, August 03, 2020 - 10:22:03
On March 17, 2020, the rector at Uppsala University, Sweden, declared that all teaching would move online, effective immediately. The authors of this blog post were teachers and students in a full-time course on Embodied Interaction, scheduled to start on March 26. This is a design course in which students apply social, spatial, bodily, and material theories to a design project. Here we report on the experience of working with such a project, which requires a lot of engagement with physical perspectives, entirely online.
Experience
When the pandemic situation started to unfold, the teachers decided that the theme of the year would be “social closeness in times of physical distancing.” The theme was chosen both for its timely relevance and because it would make working online throughout the project easier. The design teams ended up working mostly online, developing a variety of ways to deal with their own physical distancing.
To investigate their domains, most teams ended up organizing semi-staged studies. For example, one team asked couples living apart to watch films together. The team members would visit an acquaintance who was in a distance relationship to observe them watching a video together with their distant partner. Another team carried out autoethnographic investigations of their own practices of connecting remotely to their parents in their home countries, a study that grew into also asking the parents to keep diaries.
In previous years, each team had worked together on one single design. This year, to make it possible for teams to meet online, the course asked for individual designs. Teams would still work together on selecting and investigating a domain for their designs, and near the end of the course on analyzing their designs toward a more generic design concept [1], of which their designs would be examples, together forming a portfolio [2]. Doing individual designs while working in a team worked exceedingly well. The combination allowed for the diversity and creativity within the team to come through, while still allowing them to work together with domain and design concepts. Team members were also motivated and able to give insightful and constructive critique to each other. It did, however, create a lot of work for the students, and also meant that the final designs could not be as polished as in previous years.
The course includes a bodystorming session as a very important part of the design ideation phase. The session introduces the students to embodied design ideation using resources such as their own bodies, the space, and various design materials [3]. In this remote version of the course, participants were connected only through their individual video streams. Since nobody, including the teachers, could say if this would work, the whole class did a small experiment a week in advance. Each team planned and executed a brief bodystorming exercise, with the sole requirement that participants would not be constrained by having to look at their screens, to increase the ideation possibilities for bodily and spatial interaction. It was noted that having a facilitator give verbal instructions during bodystorming was a good way of achieving this. The facilitator gave instructions to, for example, explore materials and move around, reminding participants of design possibilities beyond the screen. That bodystorming was performed in each person’s home environment was also an important, and for one team, crucial, resource. The home provided personally meaningful materials and familiar spaces. Challenges in online bodystorming related to difficulties in observing participants, as they could easily move away from the camera. Several teams therefore decided to collect written and hand-drawn input from the participants throughout the session. This also allowed participants to convey a private and first-person perspective on their ideas.
The most challenging aspect of the design project was testing with users. While students could use Arduinos, sensors, and actuators to develop prototypes, it became difficult or impossible to ship these to test participants. Testing designs within the team became a go-to method. But most teams also employed a kind of user-assisted testing: They recruited external participants, and these were instructed in both how to re-create aspects of the prototype and how to simulate its use. Some participants were recruited in pairs and took turns simulating the prototype for the other. Since most teams worked with the same external participants throughout, this allowed testers to become co-creators, as design intentions needed to be both communicated and shared. Testing was accompanied by “think-aloud” responses and semi-structured interviews carried out over videoconferencing, as ways of gathering feedback.
In previous years, the final demonstrations of the designs were done onsite, at relevant locations for the chosen domains, giving teachers and other students a chance to get a feel for the designs in action. This year, these demonstrations were done over videoconferencing. One method that worked well for demonstration was a kind of distributed use case scenarios setup. These were small, distributed theater performances wherein the team members enacted a usage situation by taking different roles in a distributed setting. The use of videoconferencing allowed students to choose their locations carefully, and even get help from friends and family in enacting their designs. In these demos, one person would typically act as narrator/facilitator to make the situation clear for the audience. On one occasion, a family member had been recruited for this role, holding the camera and instructing the other members of the family. Since prototypes were typically only in one place, demonstrators would sometimes ask the audience members to recruit objects in their close vicinity and use them as placeholders for the prototypes.
Discussion
Going online in the context of a pandemic is not an ideal situation. Many of the problems encountered had to be solved by the students, and the teachers were constantly amazed by their creativity and initiative. Giving students great flexibility was crucial in making this work. Teachers also continuously reminded students to look for opportunities in the situation, rather than get stuck in problems such as lack of access to users or technology. But the setup also increased the workload in an already very stressful time. Providing greater flexibility means that some of the responsibilities for planning that normally fall on teachers are pushed to students, such as in the teams needed to practice both their bodystorming and final demonstration in advance.
While it is important to not let the exceptional become the norm, our experiences show that it is not impossible to move even a physical design project online. We have highlighted some strategies and techniques that worked well for us. While physical prototyping is not easily moved online and much of the point of Arduino programming was lost, low prototype fidelity instead allowed for an increased focus on usage situations and material qualities. Finally, while the intimate nature of everyone participating from their home environments is not without its challenges, it turned into one of the most important design resources for the students of this course.
Endnotes
1. Löwgren, J. Annotated portfolios and other forms of intermediate-level knowledge. Interactions 20, 1 (2013), 30–34.
2. Gaver, B. and Bowers, J. Annotated portfolios. Interactions 19, 4 (2012), 40–49.
3. Segura, E.M., Vidal, L.T., and Rostami, A. Bodystorming for movement-based interaction design. Human Technology 12, 2 (2016).
Posted in:
Covid-19 on Mon, August 03, 2020 - 10:22:03
Annika Waern
Annika Waern is professor and chair of human-computer interaction at Uppsala University. Her research focuses on technology-supported play and playful interactions in physical space.
[email protected]
View All Annika Waern's Posts
Andreas Bergqvist
Andreas Bergqvist is a lecturer in HCI at Uppsala University, Sweden, with a background in software development and game design. His research interests relate to programming as an expressive medium, empowerment in technology enhanced play, and agency in games.
[email protected]
View All Andreas Bergqvist's Posts
Shuang Feng
Nikolay Georgiev
Karan Manjunath
Alessandra Semeraro
Ruochen Wang
Laia Turmo Vidal
Laia Turmo Vidal is a Ph.D. candidate in interaction design and HCI at Uppsala University, Sweden. In her research, she investigates how to support movement teaching and learning through interactive technology. Her research interests include embodied design, cooperative social computing, and play.
[email protected]
View All Laia Turmo Vidal's Posts
How we need to change for more inclusive and open publication practices
Authors:
Julie Williamson
Posted: Fri, July 10, 2020 - 11:18:44
Tldr;
While progress toward more inclusive and open publications can be made through top-down policy, power within SIGCHI is distributed across a wide range of volunteers beyond the Executive Committee, for example conference Steering Committees, conference leadership, and the author and reviewer communities. The real work of enacting change comes from advocacy at all these levels:
- We could reduce bias in peer review by implementing two-way anonymization, but members of the community must advocate to conference leadership and steering committees to enact change.
- While advances in accessibility have been put into policy by the Executive Committee, authors must also put in work and learn how to provide accessible content.
- Moving toward universal open access will require authors to take on work to support automated processes and be prepared to contribute to the fixed costs of publishing.
I have no doubts that there are structural inequalities that prevent all members of our community from participating in publication equally. I firmly believe that breaking through these barriers will require everyone to do additional work and learn new skills. Many of the challenges we face require widespread behavioral change and cannot be enacted through top-down policy alone. What power volunteer leaders have is quickly extinguished in the face of widespread resistance to change.
I speak from a position of relative privilege and power: I am a member of the ACM Publications Board with significant experience drafting and applying publication policies, including the Conflict of Interest Policy for ACM Publications and Policy on Author Name Changes. I am the SIGCHI Vice President for Publications with significant experience deploying publication production pipelines for the new TAPS process, advising conference leadership, and advocating for author rights on a variety of issues. I enjoy spending time building my knowledge on publication policies and writing on open access issues. I consider myself an expert on nonprofit publishing and have positions that give me an opportunity to express and further develop this expertise.
So here is my call to the community: If you value inclusion, accessibility, and openness then you must be willing to change, take on new work, and learn new skills to make this happen.
Two-way anonymized reviewing
In "A Call to Action for the ACM," Black scholars made the direct recommendation to implement either fully open or two-way anonymization for peer review. There are interesting experiments in open reviewing, like the many conferences using https://openreview.net/. I am more excited about two-way anonymization, given the significant evidence that one-way anonymization is biased in many ways.
Adopting two-way anonymization requires additional work for reviewers at all levels. It requires everyone to learn the details of the relevant conflict of interest policy, reflect on their conflicts, and declare these in advance. This can be a tedious task, requiring everyone to read long lists of potential conflicts and mark their selections thoughtfully. But putting in this work is crucial to reducing bias at all levels of reviewing. Especially in the case of conferences like ACM CHI, where program committees bid on papers, complete reviews, and hold positions of power over acceptance in the context of one-way anonymization, this upfront work to declare conflicts is vital.
Accessible format and content
I recognize the longstanding de-prioritization of accessibility in publications. Although there has been significant progress on accessibility at SIGCHI—for example, accessibility audits of peer review software, producing accessible HTML proceedings, and providing author instructions for accessible content—progress is often impeded by communities that are slow to change.
I have been actively involved in advocating for accessibility in SIGCHI publications, most notably through the transition to the new authoring templates, which enables the creation of accessible HTML documents. The greatest frustrations I experience are community responses that although accessibility is important, it was not “worth the extra work” to make it possible.
The reality is that everyone will be required to do additional work to create accessible publications, and it will require us to learn new skills like writing figure descriptions and providing numerical summaries of visualizations. Providing these details is also good research practice, advancing the transparency and reproducibility of our research and making visual elements of publications indexable. As SIGCHI VP for Publications, I can ensure this work doesn’t create an undue burden, but it is not possible to make this work nonexistent.
Open access
Authors, reviewers, and volunteers do a significant amount of work to prepare, review, and present publications. For ACM publications, this volunteer labor is also supported by professional staff, who turn a collection of documents into a structured proceedings that is of publishable quality, is searchable, and is available for download. This includes work such as defining and enforcing standardized metadata, supporting and training volunteers who oversee publications, preparing proceedings for upload into the content management system of the DL, and releasing proceedings. Much of this happens in the background, and relies on manual effort from a small number of people. As an indication of scale, in 2019 ACM added 34,000 full-text articles to the DL. ACM employs 20 staff in the Publications department, three of whom focus on production.
If we want to move toward universal open access, we need to optimize and automate our processes and develop new financial models. For example, the new ACM templates make it possible to automate metadata collection and validation, but authors must learn how to use these new processes. Authors may need to pay additional fees, such as submission fees, increased conference fees, or presentation fees to provide continued support to the fixed costs of publication.
What do we do now?
Top-down policy alone will be ineffective to make these changes. In practice, the SIGCHI Executive Committee has little or no “power” over the distributed network of volunteers that make up SIGCHI. Advocates for change will need to engage with a wide range of volunteer leaders at all levels—conference chairs, steering committees, individual authors and reviewers—to be effective
Implementing two-way anonymization through policy is only part of the process: Conference leadership must be willing to participate. In my experience, taking agency away from conference leadership through top-down policies like this creates toxic friction. When I was involved in drafting the Conflict of Interest Policy for ACM Publications, a general resistance to two-way anonymization came up throughout the process. As an Associate Chair and Subcommittee Chair for the User Experience and Usability CHI Subcommittee, I’ve also seen how these positions of power can depend on one-way anonymization; I expect there would be resistance to two-way anonymization. To enact changes like this, we need pressure from the top down supported by pressure from the bottom up.
Policies on accessibility, like work I have completed enforcing the use of templates that generate accessible documents, are only partially effective in creating accessible conference proceedings. For example, requiring authors to provide figure descriptions would be a simple policy, but ensuring that content is appropriate and useful is a significant challenge. When forced to provide figure descriptions, a large proportion of authors simply copy the captions, which adds nothing of value for accessibility. Making progress requires everyone to hold conference leaders and authors accountable.
ACM recently announced a commitment to go fully open access within five years “if this can be achieved in a sustainable way.” Authors must be prepared to take on additional work when manual, costly labor is replaced by systems that automate these tasks. For example, copyright processes that were supported by vendors may need to be run by volunteers and automated systems. This approach means authors must complete these forms correctly and within deadlines without the support of a vendor to ensure publication of their work. Additionally, authors, conference attendees, and libraries may be required to pay more to maintain financial stability currently supported through subscription fees.
In the positions I hold, I actively work for change through policy. However, I recognize that most of the hard work is in “socializing” important ideas so that policies are accepted and effective. If there is an appetite for change in the community, it will take action from the community at all levels. Where there are gaps in policy, we work to express our values in new policies, and where policies are unfair, we work to change them.
Posted in:
on Fri, July 10, 2020 - 11:18:44
Julie Williamson
Julie Williamson is a faculty member in the Glasgow Interactive Systems Section at the University of Glasgow. Her research focuses on non-planar displays and immersive experiences for public and social settings. Her service focuses on publication matters at SIGCHI and ACM, including open access, accessibility, and proceedings production processes.
[email protected]
View All Julie Williamson's Posts
Announcing a new CHI subcommittee: Critical and Sustainable Computing
Authors:
Rob Comber,
Shaowen Bardzell,
Jeffrey Bardzell,
Mike Hazas,
Michael Muller
Posted: Wed, July 08, 2020 - 10:30:03
Reflecting rising interest in sustainability, social justice, aesthetic experiences, and critical computing throughout the HCI community in the past decade, ACM CHI now features a subcommittee devoted to such concerns: the Subcommittee on Critical and Sustainable Computing. Pursuing meaningful alternatives to the status quo, the subcommittee will encourage papers that explore how computing and computing research may contribute to fair and just relations between individuals, social and cultural groups, and whole societies, locally and globally—all in the pursuit of fulfillment and flourishing. This new focus should not only contribute to the CHI research community but also offer new resources for practitioners.
The goal of this subcommittee, which is a unification of two separate subcommittee proposals, is to create a home for the research at CHI that deals with the place of technology and technology-oriented practices in creating a fairer, more sustainable, and flourishing society. We aim to do so in light of the long history of work, in HCI and beyond, that has put computing and technology design in a critical spotlight—whether in the emanicipatory forms of, for instance, postcolonial, intersectional, queer, cultural, participatory, and feminist computing, or through the application and interrogation of critical theory, philosophy, ethics, and aesthetics.
Writing this in April 2020, it seems like an odd time to celebrate. In the face of the current pandemic—when many people are suffering physically and mentally, losing their jobs (and in some countries, their healthcare), and the stability of life-as-normal is unwound for vast stretches of society—it is our hope that this subcommittee can be one place to see a way forward for HCI research. Although the subcommittee has been more than two years in the making, its timeliness is evident. Now, perhaps more than ever, we need high-quality research that questions our assumptions, challenges positions of power (including our own), and attends to the social, environmental, and economic inequities and injustices of society, and the roles that technology and technology-oriented practices have in perpetuating and ameliorating them.
This is a continuation of decades of work in HCI, CSCW, and critique. The roots of the values that underpin this subcommittee extend back to the mid-1970s, with the 1975 Aarhus Decennial Conference focusing on critical computing; the lively discussion of HCI’s impact on society has grown ever since. The critical stance of HCI research has been continually sharpened to consider people—our own individual and collective expressions of our humanity and solidarity—as more than just interchangeable cogs in the machine. Critical, cooperative, participatory, and justice-oriented design have taught us to question who gets to or is allowed to make the machine. And now we must also ask, who benefits from it? In the past 10 years alone, papers submitted to CHI dealing with topics of sustainability, social justice, development, cultural computing, Indigeneity, feminist HCI, emancipation, race, intersectionality, and the relationship of HCI to politics, activism, ethics, and the legal and societal impacts of computing, have more than doubled.
And our community is also changing, both in formal ways—ACM has updated its ethics policy, the SIGCHI Executive Committee has established SIGCHI CARES, conferences now include sustainability and diversity and inclusion chairs—and in informal ways, such as CHIversity and the Sustainable HCI SIG, which have organized and created a more inclusive space for researchers at conferences and other SIGCHI professional events. This subcommittee will hopefully be the best of both—a rigorous and rich space to collaboratively increase the quality and standing of the work that this subcommittee stands for, and a strong and supportive community of researchers working together. We expect our community and subcommittee to be one that grows, but also one that helps researchers within our community to grow. We also hope to attract new interest from researchers in science and technology studies, the humanities, the arts, and policy studies, among others, to enrich HCI scholarship. We are strong advocates, in our work and service, for diverse voices, perspectives, and approaches. We recognize the significant practical, institutional, disciplinary, ethical, and personal challenges of the work this subcommittee is designed for. We want to be sure that authors know that their work will be respected for what it is.
As research questions change, and new epistemologies and methods are integrated within HCI, the community also needs to accept—indeed, to foster—new formats of intellectual expression. For example, increasingly the community is using essays and pictorials as new forms with which to carry forward the fundamental goal of any research community: the collective construction of knowledge. The space that we can create can also be open to new ways of thinking. Authors in fields of sustainability are pushing against individualistic and consumerist notions in our designing and behavior; authors in critical computing have long moved with interpretivist epistemologies; and those working in social justice and on structural inclusion are questioning and mitigating imbalances of power.
In short, the Critical and Sustainable Computing subcommittee aspires to support expressions of research that foster deliberative self-awareness and care in the research, design, and development of interactive systems. It engages the broader HCI community’s own contributions—both positive and negative—to concerns such as criticality and ethics in computing, social justice, and the climate crisis. Yet it does so inspired by individual and social flourishing, by artful ways of being, and by the brave actions of those within computing and beyond who have challenged societies to be more just.
Acknowledgments
We have been overwhelmed by the support we have received. We want to thank Pernille Bjørn, Eli Blevis, Mark Blythe, Susanne Bødker, Nicola Dell, Tawanna Dillahunt, Paul Dourish, Lone Koefoed Hansen, Ann Light, Silvia Lindtner, and Phoebe Sengers for their solidarity, mentorship, and intellectual inspiration. These people and many more remind us of the point of it all: to create a conducive and nurturing scholarly environment, if we can, for those who are in it with us, and for those who come after.
Posted in:
Covid-19 on Wed, July 08, 2020 - 10:30:03
Rob Comber
Rob Comber is an HCI researcher and associate professor in communication at KTH Royal Institute of Technology. His research focuses on issues related to the democracy of technology, including social and environmental sustainability, social justice, and feminism, and to specific applications of computing technology, including civic society, food, and social media.
[email protected]
View All Rob Comber's Posts
Shaowen Bardzell
Shaowen Bardzell is professor of informatics at Indiana University Bloomington’s Luddy School of Informatics, Computing, and Engineering. Her research explores the contributions of design, feminism, and social science to support technology’s role in social change. She is the co-author of
Humanistic HCI (Morgan Claypool, 2015) and co-editor of
Critical Theory and Interaction Design (MIT Press, 2018).
[email protected]
View All Shaowen Bardzell's Posts
Jeffrey Bardzell
Jeffrey Bardzell is professor of informatics and director of HCI/design in the Luddy School of Informatics, Computing, and Engineering at Indiana University Bloomington. As a leading voice in critical computing and HCI/design research, he has helped to shape research agendas surrounding critical design, design theory and criticism, creativity and innovation, aesthetics, and user experience. He is co-author of
Humanistic HCI (Morgan Claypool, 2015) and co-editor of
Critical Theory and Interaction Design (MIT Press, 2018).
[email protected]
View All Jeffrey Bardzell's Posts
Mike Hazas
Mike Hazas is a professor of HCI at Uppsala University. His research interrogates how digital systems contribute to increasing standards for comfort, fidelity and service provision, and thus tend to ratchet energy demand. He was co-chair of the Specific Application Areas subcommittee for CHI 2015—2017.
[email protected]
View All Mike Hazas's Posts
Michael Muller
Michael Muller is a Ph.D. research staff member at IBM Research, working at the intersection of social science and AI, with a background in participatory design and diversity studies. He is an ACM Distinguished Scientist, a member of the SIGCHI Academy, and an IBM Master Inventor.
[email protected]
View All Michael Muller's Posts
For CS
Authors:
Loren Britton,
Helen Pritchard
Posted: Tue, July 07, 2020 - 11:27:18
In the text that follows, we share our conversations during the early days of the pandemic. In this moment when many things get reduced to usefulness, we propose stories that are positioned to teach us more connection, how to be in contact, and with less certainty.
In this conversation, we draw on Fred Moten’s concept of chance and scandal, and develop it within computer science as a path to justice and freedom. We share our discussion of how scandal might break through established practices. Working through what it might mean, we dream of a computer science otherwise, where diverse practices might flourish. Through a back and forth filled with vulnerabilities and uncertainties, we rethink how we might make affirmative interventions.
We touch on projects we love, how we met, what we are reading, the work of maintenance, rising oppressions, and the desire for less determinacy. In this spirit, we propose that we need more stories of: scandal and enjoyment, friendships, refusals, inaction, hums, modes of survival, modes of non/commitment, damage, namings, cruising, feelings, pockets, tooling up, reading, dreaming, writing, making, smelling, rhythms and flows, tunings, cusps, insensibilities, and wayward practice.
For us, this conversation asks questions of multiple CS practices: computer science, chance and scandal, committed survival, care and shelter, chocolate and strawberries, cushions and support, collective strategies, chancer scientist, cohabitation and sharing, conditions and structures, choice and scandal, careful slug, collective scandal, crip studies, composed silliness, compulsory sleep, canceled stories, crying sabotage, carceral states, cut and scale, considerable scaffolding, collapsing species, collective suffering, companion story.
In reaching toward scandals that are chance, and less closely tied to actions, we loop to calling for CS as a figuration on its own terms, now. CS is a figuration, a playful theoretical assemblage that helps us practice:
- CS (chance and scandal) is a demand.
- CS (chance and scandal) has everything to do with CS (computer science) and CS (committed survival).
- CS (chance and scandal) is not an event or critical break after which new normativities are established.
- CS (chance and scandal) can be trained for, with the commitment toward undoing another source of oppression like: extractivism, optimization, white supremacy, carceration, tokenization, othering, reproductive capitalism, the family, binaries, linearity, attention.
- CS (chance and scandal) can be found when inaction is present.
- CS (chance and scandal) can be a form of failure less closely tied to action.
- CS (chance and scandal) helps to set up an expanded gestural repertoire of “how” and “what” “to do”
- CS (chance and scandal) can be resourced.
Our aim is a commitment to survival with CS.
March 18–May 11, 2020
I put a crisis-call meeting event with you in my calendar. I remember I wanted us to speak about the translation of crisis, and everything that was quickly unfolding within it. I was being contacted and asked to respond to technology designs in response to Covid-19 and the new normal. I thought we needed to have a crisis call about the demarcation of this so-called new normal and how CS (computer science) was responding to it.
Yes! We were speaking about the proposals for monitoring movements during the lockdowns, quarantine, and shelter in place, such as contact tracing and using data from social media with image recognition and machine learning. We had been writing about contact tracing apps and the lack of public discussion happening around their development. Your partner reminded us that these technologies for/in crisis drew on designs that had been supposedly discarded due to ethical concerns. For example, many of the proposals for monitoring movement of Covid-19 in Europe were first developed (and discarded) for tracking refugees and migrants. She said, maybe that’s a good place to start in a way that connects to an HCI audience?
IKR [I know, right], so much of the framing (based on violent practices) that attempts to establish normality in a situation that just isn’t, reminds me how much rides on business as usual. It’s exciting (and also terrible) to think about this crisis as a way to resist all these little normalizing practices that come with new normals—practices that get pushed out when the process of normalization appears. I don’t think the normal that we’ve had is so great to start with—not for me, not for a lot of others.
I think it’s really important for us to resist renormalization during times of crisis as much as we do in other times (which were already in crisis tbh). In spaces of trans*feministtechnoscience (T*FTS) [1], we have worked hard to refuse to reinstate normative structures through our work. I really think we have done this collectively through resisting and responding to the scientific agendas in HCI and CS (computer science) more broadly that often reflect gender-normative, racist, heterosexist, classist, and ableist assumptions that are used to justify, create, and enforce social inequalities [2]. This resistance is scandalous. We have as a community dreamt and worked on design practices or engagements with technoscience that highlight how specific practices reinforce or hold in place normative ideas of what the body is or can be, what nature is, what computers are. As Alex Taylor asks in his blog post: “What worlds are we making possible?” These practices and engagements often do slow things down… or put a spanner in the works on solution-driven practices.
Yes! Or are less committed to stability and progression, but we shouldn’t give up on these approaches during crisis moments just because they might be less tied to action or to fast solutions.
This history reminds me of the precarity within which so many marginal practices inside and outside of CS (computer science) broadly and CS (chance and scandal) specifically operate. I wonder how we could show CS (care and shelter) toward the maintenance of non-normative practices of engagement like the ones we are talking about here. Thinking with Maria Puig de la Bellacasa’s speculative ethics of care [3], there’s a lot of care needed that is both very direct—I need someone to pay my rent or how do I develop technologies for resistance? and less so—how do I comfort loved ones? or how do I allow for practices of tunings in my work? These precarities are in need of some CS (chocolate and strawberries) as well as some CS (cushions and support). Sitting with not knowing and not jumping to solutions might be part of this care. For designers, engineers, lovers, companions, this “care for not knowing” could replace normalization as the way to deal with the new-normal answer.
How might we mobilize this care for not knowing?
I think this care for not knowing has been held by a community of practitioners who have worked on the reframing of questions, epistemologies, methods, results, and interpretations embedded within CS practices. What we are making space for with our CS figure is the destabilization of what CS is defined and known to be. CS has been debated historically and can be debated still, but who gets to determine what comes to matter is still very much dependent on moments of translation, moments in which that which might not be recognized as CS becomes so. Instead of asking who gets to have a voice, we ask: Which practices get to produce knowledge? There has been an important history of this at the intersections of HCI and T*FTS, but in the context of computer science, it often exists only on the margins or peripheries of recognized practice within the discipline [4]. Due to what the existing and stabilized version of what CS (computer science) is, it is difficult for these practices to be translated as recognizable to the discipline. Part of the work of science is in the effort it takes to arrive at a stable agreement on a problem, situation, or thing. When developing the CS (citizen sense) infrastructures, we worked together with citizens on a design process that could both make legible air pollution but also a set of practices that could upset the usual politics of expertise and evidence [5]. We interrogated technologies as both interdependent but also as sites of normative conditions in a way that continuously resisted reinstating new normals, while holding on to the particular politics that were at stake. And Jennifer (Gabrys) proposed that instead of care as a set of normative relations we might rethink care “as a speculative mode of encounter” formed through practices that engage with harm [6].
When you talk about this I am also reminded of the work that Aimi Hamraie and Kelly Fritsch are doing with the “Crip Technoscience Manifesto” [7] on interdependence. Hamraie and Fritsch focus on centering disabled people as doers and makers. They point to the etymology of the word access, which bifurcates into the meanings of a “kind of attack” as well as an “opportunity enabling contact.” They mention many projects that offer alternatives, such as Mapping Access a collective mapping project that brings people together to collect access information about the everyday built environment.
Resisting normalization as a form of care might also be way of “flipping the system,” as described by the research collective Our Data Bodies Project (ODB). During their research, they found that one of the reoccurring themes was that many people experience a feeling of paranoia, raw emotions, and memories when talking about data collection and data-driven systems. They have been working with communities to flip the scripts and strategies used in data collection, to move from paranoia to power, to “think about CS (collective strategies) for now and in the future.”
I think all these research projects focus on the importance of working with interdependence and rejecting the new normal—scandalous practices.
![](/images/uploads/blogs/MaintainingIndeterminacy_web.jpg)
Maintaining Indeterminacy by Loren Britton.
March 18–May 11, 2020
Of course, there are always historical underlying causes. I’m fearful that in multiple contexts we see the underlying realities of oppression in a newly brutal way because of this lockdown. We see this both in the ways in which big tech takes up contact tracing apps, which clearly show the issues with contact with health authorities and commercial partners, and further in the police state regulating in a heightened way by arresting the homeless, people of color, and sex workers for practicing survival. Both of these brutalities are posed as modes of care. And this is a care that is based on predetermining the subjects of care (or not).
In relation to our proposition of care for not knowing, I’ve been thinking about our maintaining CS project. It feels like I’m engaged with caring for indeterminacy and indeterminate care all the time right now. Has it crossed your mind too?
I feel the same way. When we wrote the proposal for the workshop series, it was more about proposing other ways of relating to each other, slowing down, making different or new formats. We are both having to do a lot of work to maintain spaces for indeterminate practices and thinking but also on a daily basis inhabit a deep embrace of indeterminacy toward modes of survival. All of which is not recognized by institutions.
So it’s interesting that we are thinking about our work on maintaining indeterminacy during this time, because it also calls us to consider what this indeterminacy is that we are committed to and how it becomes a way in which our practice is figured within T*FTS. Since our Oracle practice [8] workshop, I’ve been caught with Alexis Pauline Gumb’s proposal of unknowing.
I’ve been thinking about Gumb’s work too, and specifically working with the practice part, by practicing the indeterminate, i.e., not aiming to overly control the path in which the practice will develop but rather allowing the materiality of the environment and the conditions to emerge and to allow for the emergence of knowledge that exists outside of the previously nameable.
And this is a much bigger move than how less determinate paths are often imagined. For example, when indeterminacy is designed in HCI, there might be a discussion to loosen the outcome by changing the mode of touch interaction from an on-off switch to a galvanized moisture sensor.
Ha! Indeed I think we are demanding something else here...
Yes, I don’t think that all that might come up needs to be named either. I think that unknowing and indeterminacy are far apart. Indeterminacy to me seems to be about practicing the space of the possible, whereas unknowing seems to be about knowing from where you begin and then unlearning that thing.
Practicing indeterminacy is a kind of space for chance that is necessary in order to practice and to think the world differently. Chance, as a verb, is to do something despite its being dangerous or of an uncertain outcome. We see something very different in the ways in which Covid-19 is addressed by technoscience; secure answers are demanded and possibilities are collapsed. To be a practicing CS (chancer scientist) means that it becomes almost immoral to demand spaces of dreaming or indeterminacy as a response, a mode of responding in which facts might not be so secure.
We read a text a while ago by Kathryn Yusoff [9]. I like how she troubles response. Before action, representation, affirmation she wonders, What is response? What you’re saying about Covid-19, care, and response brings up a lot. I am thinking about how Jeff Bezos has gained millions of dollars through the continued extraction of his workers. I have read that many workers at Whole Foods got a raise during this time; however, they still do not have paid sick leave or hazard insurance. It’s this very fine line between being an essential worker while also being treated as dispensable.
What is considered essential in cohabitation is deeply entangled with questions of comfort. Yusoff proposes that another mode of cohabitation would be to be in a mode of sharing, CS (cohabitation and sharing), even when this sharing is uncomfortable and causes discomfort. To cohabit would be to share conditions and place, CS (conditions and sharing). Instead, today we exist in extreme separations of condition and place. This is aided by big tech, for example, those who are tied to forms of labor that mean that they are not able to shelter in place.
All sorts of people continue to survive and thrive through so much CS (committed survival). I’m reading Saidya Hartman’s Wayward Lives, Beautiful Experiments [10] right now. That book is full of so many vignettes of thriving and joy. Hartman talks about kinship as a resource and a practice of survival. It makes me realize again how many histories are ignored and the need for narratives outside of a dominant archive [11]. I don’t think the point is to romanticize, but rather to value lived experience that is not couched within oppressive norms.
May 11–May 18, 2020
How do we make space to maintain CS (choice and scandal)? and chance, oops? All of it is a figuration like Haraway’s SF signifier [12]: CS—(computer science), (chance and scandal), (chocolate and strawberries), (couches and snuggles), (committed survival), (co-habitation and sharing), (chancer scientist), (careful slug).
I guess when we first started discussing this, we were talking about indeterminacy and then we found chance and scandal.
Yes! I have been reading Black and Blur by Fred Moten [13]. In there he has this beautiful quote:
Indeterminacy doesn’t ground freedom or equality (by way of some magic operation whereby the absence of basis becomes a basis). Rather, they are part of a complex field of scandal and chance, wherein the very idea of ground remains to be retheorized [...].
I shared this with you and we started talking through the differences between indeterminacy and chance and scandal and what they might mean.
In the context of thinking about technoscience, we came to understand this proposal from Moten that practices of survival and freedom within computing will not be achieved through a design for the play of indeterminacy, which might fracture the existing moral grounds on which it is built. Instead, freedom and equality are made possible only through the disruptive moments of scandal (which may be tied to enjoyment) that reroute, destroy, turn over, or refuse attachments in a complex and uneven field.
This complex and uneven field might actually be understood as CS.
Yes! And we might recognize this field as attachments of the infra-ordinary, a term that Tina Campt uses to describe the “everyday practices we don’t always notice and whose seeming insignificance requires excessive attention. Attending to the infra-ordinary and the quotidian reveals why the trivial, the mundane, or the banal are in fact essential to the lives of the dispossessed and the possibility of black futurity” [14].
Instead of talking about the new normal that emerges from the crisis, or that is the crisis, we are thinking about really the shuffling of everyday attachments that Covid-19 brings about on CS, and the new openings for scandal and chance that it presents or limits.
Yes! And I think here is where making our desires for CS figurations of chance and scandal, or cushions and support, are important, because desire allows us an openness.
This also gets me thinking about Andrea Long Chu’s [15] work around whose responsibility it is to desire trans* people. I like how she talks about trans* and desire as being shaped co-constitutively and how it is our responsibility to develop desire for all sorts of variation, which is like a desire in a complex field. It’s a practice of attuning to how to collectively resist, with joy. It reminds me actually of when we met! Thinking with all sorts of conditions for collectivities, what grounds we can meet on, and what scandals can be—CS (collective scandal).
When we met at Collective Conditions, those experiments with the types of resistances and the laying of new possibilities for CS were very much dunked in questions of responsibility. The “we” at Collective Conditions did a lot of working toward how to desire differently, and to desire different infrastructures, including technical ones—and making desire into an analytic is scandal in itself.
Yes!
Endnotes
1. Trans*FeministTechnoscience (T*FTS) is defined as a branch of science studies and practices, that recognizes the inseperability of the merging of boundaries and the inseparability between science/technology/society (technoscience) and remakes the material semiotic boundaries of the body, nature, and technology. We use the formula of the star (*) which sharpens the points of the intersections of antiracist, queer, trans-disciplinary and intersectional sensibilities alongside broader STS.
2. Cipolla, C., Gupta, K., Rubin, D.A., and Willey, A., eds. Queer Feminist Science Studies: A Reader. Univ. of Washington Press, Seattle; London, 2017.
3. Bellacasa, M.P. De La. Matters of Care: Speculative Ethics in More Than Human Worlds. Univ. of Minnesota Press, 2017.
4. Some examples of these practices are: Oracle Practice (c.f Pritchard, Snodgrass, Britton, Morrison, Moll 2020) (https://facctconference.org/2020/acceptedcraftsessions.html#burn), Micha Cardenas - Unstoppable Project (https://michacardenas.sites.ucsc.edu/unstoppable/), Meltionary (http://meltionary.com/), Underground Division (https://hackersanddesigners.nl/p/The_Underground_Division)
5. Pritchard, H. and Gabrys, J. From citizen sensing to collective monitoring: Working through the perceptive and affective problematics of environmental pollution. GeoHumanities 2, 2 (2016), 354–371, 2016.
6. Gabrys, J. Citizen sensing, air pollution and fracking: From ‘caring about your air’ to speculative practices of evidencing harm. The Sociological Review 65, 2_suppl (2017), 172–192.
7. Hamraie, A. and Fritsch, K. Crip technoscience manifesto. Catalyst: Feminism, Theory, Technoscience 5, 1 (2019).
8. Gumbs, A.P. 17th Floor: A pedagogical oracle from/with Audre Lorde. Journal of Lesbian Studies 21, 4 (2017), 375–390; DOI: 10.1080/10894160.2016.1164519
9. Yusoff, K. Insensible worlds: Postrelational ethics, indeterminacy and the (k)nots of relating. Environment and Planning D: Society and Space 31, 2, (2013), 208–226, 2013.
10. Hartman, S.V. Wayward Lives, Beautiful Experiments. Norton & Company, 2019.
11. For a discussion on Sadiya Hartman’s work taken up within HCI see: Daniela Rosner’s book: Critical Fabulations: Reworking the Methods and Margins of Design. MIT Press, 2018.
12. Harraway, D. Staying with the Trouble: Making Kin in the Chthulucene. Duke Univ. Press, 2016.
13. Moten, Fred. Black and Blur. Duke Univ. Press, 2017.
14. Campt, T.M. Listening to Images. Duke Univ. Press, 2017.
15. Chu, A.L. Females. Verso Books, 2019.
Posted in:
Covid-19 on Tue, July 07, 2020 - 11:27:18
Loren Britton
Loren Britton is an interdisciplinary artist based in Berlin. Focusing on radical pedagogy, play, and unthinking oppression, they make objects that reposition and collaborations that unlearn. Britton is responsible to questions of techno-science, anti-racism, trans*feminism, and making accessibilities (considering class and dis/ability). Britton researches within Gender/Diversity in Informatics Systems (GeDIS) at the University of Kassel, Germany.
[email protected]
View All Loren Britton's Posts
Helen Pritchard
Helen Pritchard is the head of Digital Arts Computing and a lecturer in computational art at Goldsmiths, University of London. Her work brings together the fields of computational aesthetics, more-than-human geographies, and Trans*FeministTechnoScience to consider the impact of computational practices on environmental justice. She is the co-editor of
Data Browser 06: Executing Practices published by Open Humanities Press (2018).
[email protected]
View All Helen Pritchard's Posts
Heeding the call for action
Authors:
Sarah Fox
Posted: Thu, July 02, 2020 - 10:40:23
Those who profess to favor freedom and yet deprecate agitation are men who want crops without plowing the ground.
— Frederick Douglass [1]
It's time to implement the immensely clear action items Christina N. Harrington, Yolanda Rankin, Jasmine Jones, Robin Brewer, Sheena Erete, Tawanna Dillahunt, Quincy Brown, and others list in their “Call to Action for the ACM” and “Open Letter” [2,3]. Embedded in the authors’ urgent appeal is the “opportunity for our entire association to reflect on our practices and ways we can advance computing as a science, profession, and a catalyst for change in an ever-changing society” [4]. Heed this call, take these action items to your department heads and campus administrators. Gather your voice with others in the SIGCHI community pushing the Executive Committee to respond to repeated reports of institutional racism and ableism from colleagues and peers [4,5]. Contest the endemic problem of computing research rooted in racial hierarchies of difference, and actively oppose the tech to prison pipeline [6]. This is the dismantling work necessary to build the anti-racist institutional cultures we need.
It is not about being quick to write statements of allyship devoid of specific commitments, but rather doing the ongoing work to make substantive and long-lasting change. For predominantly white institutions, this involves ceding space and committing material resources. Placing value on the service labor disproportionately performed by Black and brown scholars, through adequate funding, tenure considerations, and by sharing its burdens. For white, cisgender faculty members like myself, this means calling-in and calling-out those within our immediate environments who reproduce racism through their policies and actions, and remedying harms caused by our own misaligned efforts. It involves examining how we attribute the knowledge work that forms the basis of much of our critical consciousness as a field, and correcting the citational practices that have erased Black women's intellectual leadership and ongoing contributions [7].
If there’s one thing that we learn in volunteering for the ACM (myself, as an associate chair, workshop organizer, etc.), it’s the skill of agitation. Agitating peers to accept review requests, finish them by the deadline, respond to rebuttals, submit position papers, and register for conferences in time for the early bird rate. It’s the work of bothering colleagues in order to grow and enrich the community. Agitation is neither gentle nor harsh, but necessarily unrelenting. Materially, it is a mixing up; it refuses for things to sediment back into the status quo.
It seems about time we collectively use this skill the ACM has taught us and put it to good use. Let’s agitate where we stand for the vital changes Harrington, Rankin, Jones, Brewer, Erete, Dillahunt, and Brown call for in their letter. As focus shifts under the compounding crises of this time, persistent work is crucial in order to ensure that such actions are realized. Let’s insist on the transformation required for these changes to last—in our labs, at our home institutions, within the ACM.
The thing about action items is they have to be done. Now’s the time. Agitate!
Endnotes
1. Douglass, F. Two Speeches, by Frederick Douglass: One on West India Emancipation, Delivered at Canandaigua, Aug. 4th, and the Other on the Dred Scott Decision, Delivered in New York, on the Occasion of the Anniversary of the American Abolition Society, May, 1857. C.P. Dewey, printer, American Office, 1857.
2. Harrington, C., Rankin, Y., Jones, J., Brewer, R., Erete, S., Dillahunt, T., and Brown, Q. A call to action for the ACM. ACM Interactions blog. Jun. 22, 2020; https://interactions.acm.org/blog/view/a-call-to-action-for-the-acm
3. Black in Computing Collective. An open letter & call to action to the computing community from Black computer scientists and our allies. Black in Computing. Jun. 8, 2020; https://blackincomputing.org/
4. Grady, S.D., Wisniewski, P., Metoyer, R., Gibbs, P., Badillo-Urquiola, K., Elsayed-Ali, S., and Yafi, E. Addressing institutional racism within initiatives for SIGCHI’s diversity and inclusion. ACM Interactions blog. Jun. 11, 2020; https://interactions.acm.org/blog/view/addressing-institutional-racism-within-initiatives-for-sigchis-diversity-an
5. Mankoff, J. 2020. A challenging response. ACM Interactions blog. Jun. 17, 2020; https://interactions.acm.org/blog/view/a-challenging-response
6. Coalition for Critical Technology. Abolish the #TechToPrisonPipeline. Medium. Jun. 22, 2020; https://medium.com/@CoalitionForCriticalTechnology/abolish-the-techtoprisonpipeline-9b5b14366b16
7. Rankin, Y. and Thomas, J. Straighten up and fly right: Rethinking intersectionality in HCI research. ACM Interactions 26, 6 (2019), 64.
Posted in:
on Thu, July 02, 2020 - 10:40:23
Sarah Fox
Sarah Fox is an assistant professor at Carnegie Mellon University in the Human Computer Interaction Institute, where she directs the Tech Solidarity Lab. Her research focuses on how technological artifacts challenge or propagate social exclusions by examining existing systems and building alternatives.
[email protected]
View All Sarah Fox's Posts
Reflections on planning and running a virtual Doctoral Consortium at CHI 2020
Authors:
Duncan Brumby,
Koji Yatani,
Leah Findlater
Posted: Tue, June 30, 2020 - 10:28:52
This article describes what was done to run a virtual Doctoral Consortium at CHI 2020. The event was originally planned as an in-person, two-day event to take place in Honolulu, Hawaii, on April 25–26, 2020. Because of the Covid-19 pandemic, the decision was taken in March that the CHI conference would not go ahead as a physical event. Most events were canceled; however, the Doctoral Consortium continued as a self-contained track and was swiftly reorganized to successfully run as a virtual event on April 28.
Here, we describe the key steps involved in organizing the Doctoral Consortium as a virtual event, from the technology involved to how to schedule activities. Given the expected long-term impact of the Covid-19 pandemic on conferences for the coming year, we hope that this article provides useful information for the organizers of similar events.
Why Doctoral Consortiums are Important and Must Continue
Doctoral consortiums are valuable and important community-forming events for Ph.D. students. CHI’s Doctoral Consortium format and goals are typical: Students apply by submitting application materials to help the chairs select a diverse group of students to be invited to attend. Care is taken to select students from different institutions, and during the conference, a two-day workshop takes place in which students present their work and give feedback to each other.
One of the most important features of a Doctoral Consortium is that students develop and establish their network of peers from the broader international research community beyond their immediate university lab. The connections made during a Doctoral Consortium often last, as students go on to become established members of their research community and continue to cross paths and collaborate over the years. This kind of community-building activity is very important and must continue, despite the challenges of running virtual events during the current pandemic.
Initial Planning for CHI 2020 Doctoral Consortium
The Doctoral Consortium track at CHI is extremely competitive. This year, 85 students applied and 20 were accepted. Students were required to submit a proposal describing their research along with contextual information about their Ph.D. program (full details on the call are available here: https://chi2020.acm.org/authors/doctoral-consortium/). The selection process considered both the quality of the research as well as additional factors to identify a set of students that were diverse in terms of backgrounds and topics.
By February, 20 students had been selected to participate in the Doctoral Consortium and external funds had been secured to help support travel expenses to attend the conference in person. In addition to the three co-chairs, three mentors had also been invited to join the consortium to give advice and feedback to students. These mentors were invited fairly late in the process, throughout January and February, and were selected from respected members of the CHI research community. As outlined below, mentors play a key role in supporting and facilitating a smoothly running Doctoral Consortium, even more so when it comes to running this as a virtual event.
Transitioning to a Virtual Doctoral Consortium
The decision to run the Doctoral Consortium as a virtual event was made only six weeks before the event. While we did have a list of students who had been accepted, there was still much to do.
The first priority was to identify whether the students who applied still had the capacity and interest to take part in a virtual event. At this time, many countries were beginning to issue mandatory lockdown and stay-at-home orders. The sudden and unexpected transition was challenging for many, and it was unclear whether the students would be able to participate. There were also secondary concerns, such as identifying the time zone that participants would be able to join and what kind of activities they were most interested in. To answer these initial questions, a brief survey was constructed and sent to participants at the end of March.
Working Across Time Zones
Participants were spread across a wide range of time zones (e.g., Oceania, Asia, Europe, and North America). One of the first major decisions was to decide how long the event should be and at what time it should run. The two main options considered were: 1) having one longer event with core overlapping hours, and 2) having two separate shorter events. Our thinking on this decision was informed by reflections from Simon Buckingham Shum on organizing the Doctoral Consortium at LAK ’20 [1]. We decided to split our Doctoral Consortium into two separate four-hour events:
- DC-A: Tuesday, April 28, 2020, 9:00–13:00 UTC
- DC-B: Tuesday, April 28, 2020, 16:00–20:00 UTC
Participants naturally clustered into two groups of 10 based on their preferred meeting time. It’s worth highlighting here that people’s preferences were not based simply on their local time zone. Instead, participants joined the group that suited them best: Some preferred to get up early, while others preferred to stay up late.
A further benefit of running the event as two separate short events meant that each meeting had fewer participants, potentially making communication in the virtual meeting more manageable. Of course, a downside of this decision is that the entire group of participants never came together as one as they would have during an in-person conference.
Scheduling Activities
The next priority was to establish a basic schedule for activities during the two four-hour sessions. Each session had an identical template structure:
- Welcome and introductions (10 min)
- Talk 1 (7 min)
- Talk 2 (7 min)
- Breakout group discussion: two rooms, one for each speaker (15 mins)
- Talk 3 (7 min)
- Talk 4 (7 min)
- Breakout group discussion: two rooms, one for each speaker (15 mins)
- Break (10 min)
- Talk 5 (7 min)
- Talk 6 (7 min)
- Breakout group discussion: two rooms, one for each speaker (15 mins)
- Talk 7 (7 min)
- Talk 8 (7 min)
- Breakout group discussion: two rooms, one for each speaker (15 mins)
- Break (10 min)
- Talk 9 (7 min)
- Talk 10 (7 min)
- Social time (1 hr)
A shared and editable document was sent to participants as soon as possible to help them plan for the meeting. Students were allocated a talk slot in the schedule; this was done in a random order without any special effort given to group students into thematic clusters based on topic. Inevitably there were some requests to switch sessions soon after the initial draft schedule was shared; these requests were all accommodated.
Along with the schedule, all of the students’ extended abstracts were also shared using Google Drive (which had to be done because the papers had not yet appeared in the ACM Digital Library). It was important to share these papers as early as possible, as it gave all of the participants an opportunity to dive deeper and learn about each other’s work prior to the meeting. Finally, speakers were asked to provide a link to slides ahead of the meeting; this worked as a backup should a presenter run into difficulties with screen sharing during their talk (more on that soon).
Welcome and Introductions
The meeting started with a welcome. Every person in the meeting gave a short introduction: name, affiliation, current location and time of day, and a brief overview of their research topic. Introductions were made in a random order based on the layout of participants on the chair’s screen. This meant that mentors and students were interleaved. This welcome event was scheduled to take 10 minutes; however, in both meetings it ran over considerably, taking 20 minutes. It was the only part of the meeting to run over, suggesting that more time should have been allocated. Taking the time for introductions is very important for a successful virtual meeting, as it allows each participant time to speak to the group.
Short Talks
Participants were asked to prepare a five-minute presentation covering three points:
- What’s your research question?
- What work have you done so far?
- What work still needs to be done to complete your Ph.D.?
We opted for short talks, so as to keep down the overall duration of each meeting and to enable longer breakout group discussions. From our personal experience, shorter talks seem to work better in this context, as it’s hard to hold people’s attention online. We considered asking participants to prepare a prerecorded talk, but in the end decided against it due to concerns that it would place too many extra demands on participants ahead of the meeting. In the end, all of the Doctoral Consortium participants were able to give strong and compelling talks about their work within the given time limit.
On the Importance of Mentors
Mentors play a critical role in the smooth running of a Doctoral Consortium; their role is even more vital in a virtual event. Initially three mentors were recruited. But with the event now split in two, more help was clearly needed. Within a couple of days we had successfully recruited a diverse group of 10 mentors who were experts in different areas of HCI research.
Several of the newly appointed mentors were outstanding early-career researchers, who themselves had been members of the CHI Doctoral Consortium over the past few years. There was enthusiasm from this group to give back and also to share advice on how to navigate the steps immediately after completing a Ph.D. Partnering early career researchers with more established senior researchers made for an exciting mix of perspectives on advising the Doctoral Consortium participants.
Technology for Running a Virtual Doctoral Consortium
We relied on four tools to run the event:
- Zoom for hosting the video call, sharing slides, and setting up breakout rooms (see Figure 1 for participants in the two sessions). A paid account was used to enable longer meetings. We used Zoom’s Meeting mode rather than the Webinar mode, as this allowed for more interaction between participants. Participants were encouraged to have their video on with their microphone muted unless they were speaking. Security for virtual meetings is an important concern, and this was reinforced to us by reports of Zoombombing taking place during another CHI-related event the day before our meeting (see Barry Brown’s reflections on how this was handled). We therefore took care to set a password for the meeting and share the link only with meeting participants directly (i.e., we did not put the link on a public website). Finally, it’s worth noting that the role of chairing a virtual meeting is extra demanding, as attention is needed for managing all the tools and ensuring everyone has access. For example, setting up and assigning people to breakout rooms takes time and requires attention; it is difficult to carefully listen to a talk while doing this simultaneously.
- Slack for text communication and sharing notes and other resources. All participants were invited to join a few weeks before the event. Slack was very useful for enabling conversations to occur both before and after the meeting. Several mentors reported that they shared written feedback with students and used this tool to backchannel with students on extended questions. While Zoom enables in-meeting chat, this feature operates only during the meeting and cannot easily be accessed by all participants once the meeting has ended.
- Google Docs for sharing the schedule and links to slides and papers.
- Email for essential communication and reminders. Despite having alternative forms of communication such as Slack, email is still a universal tool for reaching people. Emails were sent by the chair to both students and mentors to share important information (e.g., schedule and meeting link). Reminders were sent one week and 24 hours before the meeting.
![](/images/uploads/blogs/temp_file_Figure1_web3.jpg)
Figure 1. Participants in the two sessions of the virtual Doctoral Consortium at CHI 2020.
We’re All Working from Home Now
One of the requirements for taking part in a virtual meeting is that participants have access to a good, stable Internet connection and their own computers with a camera and a microphone. However, there are well-documented issues with speed inequities between different societal groups and locations. During the meeting, some participants did experience issues with poor Internet speeds and had to work around this by turning off their camera or by stopping screen sharing. We planned for such events by asking all speakers to share a copy of their slides ahead of time in case screen sharing became impossible.
While participants should ideally find a quiet space, free from interruption, many live in homes with others and have to work in shared living spaces, where background interruptions and distractions are likely to occur. Everyone appreciates a level of understanding on these issues.
Accessibility
There are well-established arrangements in place to provide accessibility services at physical conferences; these must be retained during the transition to virtual events. Our event included sign language interpretation; one participant coordinated with the interpreters and shared meeting details ahead of the event. Interpreters require frequent breaks, so they often work in small teams and switch throughout the meeting. Special attention is needed when assigning breakout rooms to make sure that the interpreters are assigned to the same virtual room as the people they are supporting during the meeting.
One thing that we regret not doing is enabling closed captioning throughout the meeting for all participants. Several participants reported that this would have been useful, since it would have created a script that could be consulted when something was missed. In Zoom, an assigned meeting participant can manually add closed captions, or a third-party service can be connected to stream captions. Other meeting platforms, such as Microsoft Teams and Google Meet, have integrated automatic closed-captioning services that may be sufficient for some participants but are not equivalent to having a human transcriptionist.
Don’t Record
Just because you can record a digital event does not mean that you should. Doctoral Consortiums are intended to be safe spaces in which students can give and receive feedback from their peers—private, invitation-only events. In keeping with the spirit of a traditional in-person Doctoral Consortium, the event was not recorded.
Breakout Rooms
The schedule was structured so that there was a pattern of two seven-minute talks followed by a 15-minute breakout room discussion. One breakout room was set (and named for) each of the speakers. Prior to the meeting, two mentors were assigned to each speaker; all other participants were randomly assigned to one of the two groups. Mentor-mentee assignments were shared with all participants several days before the event so that mentors could prepare by reading the extended abstracts of their mentees ahead of time. Mentors gave their preferences for who they would like to work with (usually based on a topicality fit), and these preferences were almost always met.
Mentors played a key role in chairing the breakout-room discussions. Pre-assigning them to students was beneficial for the smooth running of the virtual meeting. The chair of the meeting can set up the breakout rooms—however, once they’re launched, participants leave the main meeting space and go to the virtual breakout rooms. Mentors chaired these breakout meetings and often started the conversation by asking their mentee about the areas where they would most value feedback.
In Zoom, there are options to automatically set the duration of breakout rooms and to set how much warning participants get to wrap up their conversation before being returned to the main meeting space. Using these features kept everything running on time.
Ending on a Social Hour
The meetings ended with an open-ended discussion with all participants. One limitation of the virtual format is that these informal discussions are more difficult to manage than when run at an in-person event. There are challenges in turn-taking associated with virtual meeting tools. The option of breakout rooms was given but not taken up. Instead, an informal Q&A between students and mentors developed.
At conferences, social events usually follow the main meeting (e.g., an evening meal together at a restaurant). In planning the virtual event, several participants said that they would be interested in retaining this social element, for instance, by staying on the call to have a virtual beer together. However, on the day this did not happen. Maybe a four-hour virtual meeting is simply too exhausting?
Conclusion
Doctoral consortiums are important training and networking events for Ph.D. students. They must continue despite the challenges of conducting conferences as virtual events during the Covid-19 pandemic. In this article, we have reflected on our experience of rapidly reorganizing the CHI 2020 Doctoral Consortium to a virtual event over a six-week period. In the end, it was an enjoyable and successful event that retained many of the core features of an in-person Doctoral Consortium. Time will tell whether the longer-term objective of helping this cohort of students develop those deep and long-lasting connections was achieved.
List of Participants
Co-Chairs:
- Duncan Brumby, University College London
- Koji Yatani, University of Tokyo
- Leah Findlater, University of Washington
Mentors:
- Ed Cutrell, Microsoft Research
- Anna Feit, ETH Zurich
- Simone Kriglstein, University of Vienna
- Neha Kumar, Georgia Institute of Technology
- Kai Kunze, Keio University
- Shaun Lawson, Northumbria University
- Pedro Lopes, University of Chicago
- Bektur Ryskeldiev, University of Tsukuba
- Katta Spiel, Vienna University of Technology
- Maria Wolters, University of Edinburgh
Students:
- Rahul Arora, University of Toronto. Creative Expression with Immersive 3D Interactions; https://doi.org/10.1145/3334480.3375028
- Manshul Belani, IIIT-Delhi. Evaluating Virtual Reality as a Medium for Vocational Skill Training; https://doi.org/10.1145/3334480.3375027
- Fabio Catania. Politecnico di Milano. Conversational Technology and Natural Language Visualization for Children’s Learning; https://doi.org/10.1145/3334480.3375041
- Samantha W. T. Chan, The University of Auckland. Biosignal-Sensitive Memory Improvement and Support Systems; https://doi.org/10.1145/3334480.3375031
- Ana Paula Chaves, Northern Arizona University. Should my Chatbot be Register-Specific? Designing Appropriate Utterances for Tourism; https://doi.org/10.1145/3334480.3375033
- Amy Yo Sue Chen, Simon Fraser University. Giving Form to Temporality: Extending Design Practices and Methodologies for “Slow” Interaction; https://doi.org/10.1145/3334480.3375024
- Mayara Costa Figueiredo, University of California, Irvine. Self-Tracking for Fertility Care: A Holistic Approach; https://doi.org/10.1145/3334480.3375029
- Alexander Eiselmayer, University of Zurich. Supporting the Design and Analysis of HCI Experiments; https://doi.org/10.1145/3334480.3375038
- Yasmin Felberbaum, University of Haifa. Enhancing Physical Activity of Walking and Dual-Tasking among Older Adults Using Personalized Navigation Practice; https://doi.org/10.1145/3334480.3375034
- Rakibul Hasan, Indiana University. Reducing Privacy Risks in the Context of Sharing Photos Online; https://doi.org/10.1145/3334480.3375040
- Jialun Aaron Jiang, University of Colorado Boulder. Identifying and Addressing Design and Policy Challenges in Online Content Moderation; https://doi.org/10.1145/3334480.3375030
- Awais Hameed Khan, The University of Queensland. Participatory Design Tools: Exploring the Value of Design through Materiality; https://doi.org/10.1145/3334480.3375023
- Doris Jung-Lin Lee, University of California, Berkeley. Towards an Integrated Solution for Intelligent Visual Data Discovery; https://doi.org/10.1145/3334480.3375035
- Matthew Seita, Rochester Institute of Technology. Designing Automatic Speech Recognition Technologies to Improve Accessibility for Deaf and Hard-of-Hearing People in Small Group Meetings; https://doi.org/10.1145/3334480.3375039
- William Seymour, University of Oxford. A Design Philosophy for Agents in the Smart Home; https://doi.org/10.1145/3334480.3375032
- Carlos E. Tejada, University of Copenhagen. Print-and-Play: 3D-printed Interactive Objects Without Assembly or Calibration; https://doi.org/10.1145/3334480.3375025
- Milka Trajkova. Indiana University, Indianapolis. Designing AI-Based Feedback for Ballet Learning; https://doi.org/10.1145/3334480.3375036
- Yan Wang, RMIT University. Understanding the Design of Playful Gustosonic Experiences; https://doi.org/10.1145/3334480.3375022
- Daricia Wilkinson, Clemson University. All Experiences Matter: Designing Safety Tools for Vulnerable Groups in the Caribbean; https://doi.org/10.1145/3334480.3375037
- Mohan Zalake, University of Florida. Advisor: Agent-based Intervention Leveraging Individual Differences to Support Mental Wellbeing of College Students; https://doi.org/10.1145/3334480.3375026
Posted in:
Covid-19 on Tue, June 30, 2020 - 10:28:52
Duncan Brumby
Duncan Brumby is professor of human-computer interaction (HCI) at University College London (UCL), where he directs the HCI MSc program. His research focuses on understanding how people manage digital distractions. He is editor-in-chief of the
International Journal of Human-Computer Studies and has been on the CHI organizing committee for the past few years.
[email protected]
View All Duncan Brumby's Posts
Koji Yatani
Koji Yatani is an associate professor and 2017 UTokyo Excellent Young Researcher in the Department of Electrical Engineering and Information Systems, School of Engineering, University of Tokyo, where he leads the Interactive Intelligent Systems Laboratory (
http://iis-lab.org). His current research focuses on novel sensing technology for interaction, productivity/creativity support, and usable security. He is the steering committee chair for ACM UbiComp.
[email protected]
View All Koji Yatani's Posts
Leah Findlater
Leah Findlater is an associate professor in human centered design and engineering at the University of Washington, where she directs the Inclusive Design Lab. Her research focuses accessible computing and human-centered machine learning.
[email protected]
View All Leah Findlater's Posts
‘A call to action for the ACM’ liberates all of us
Authors:
Lilly Irani
Posted: Mon, June 29, 2020 - 10:53:23
Women of today are still being called upon to stretch across the gap of male ignorance and to educate men as to our existence and our needs. This is an old and primary tool of all oppressors to keep the oppressed occupied with the master's concerns.
— Audre Lorde [1]
“A Call to Action for the ACM” is a gift. The piece was authored collaboratively by computing scholars Christina N. Harrington, Yolanda Rankin, Jasmine Jones, Robin Brewer, Sheena Erete, Tawanna Dillahunt, and Quincy Brown. It is tremendous and exhausting work to stretch across the gap of ignorance to argue for the validity of black scholars’ needs. It is already unfair that those in power set the terms of legible arguments and legitimate discourse, whether implicitly or explicitly. That the ACM and cultures of computing more widely reproduce racism is plainly in evidence: “Black faculty in CS and Information Sciences make up only 1.8% [of the professoriate], while only 1.7% of new PhD earners are black,” Harrington and her collaborators remind us. This, in a country where, according to the census, 13% of people are black. The authors’ gift to us is that they and so many others have outlined specific processes that reproduce this racism. For those of us who do not fully understand their recommendations, the burden is on us to do our homework to do so. The burden is on us to do the difficult work of reorganizing our organizations, routines, and sensibilities to take the recommended actions. The problems ACM practices create for black scholars are specific, but black scholars’ struggle to name their “real conditions” amidst racism, as Lorde argues, have much in common with the struggles of those who battle homophobia, gender norms, and other forms of racism. By doing the work the authors call on us—and specifically ACM leadership—to do, we will be better scholars of computing and society. We will be better people, liberated as Frantz Fanon argues [2], from the ways that racist practices hold all of us back from fuller humanity.
It is not enough to add more black computing scholars to the pipeline without looking at the harms and exclusion perpetuated inside the pipeline’s practices of evaluation, support, and leadership. ACM—though not only ACM—has a problem of whose perspectives, values, and practices are valued in leadership, in ethical judgement, in evaluation processes, and even in the politics of friendship that glues so much of this intellectual community together. Each one of helps reproduce a racist system when we fail to challenge and transform how we do research, resources, and friendship.
I know it can feel deeply uncomfortable for some to hear the word racist attached to a set of practices undertaken with good intentions. If you feel this discomfort, name it, observe it, and learn to move past it. Use it as an entry point for gaining deeper self-knowledge that expands your capacity to do anti-racist work [3]. It feels far worse to be a marginalized scholar treated as if their life and intellect matters less than a community’s fear of confronting change. “A Call to Action” reaches out in good faith. The ACM must return the favor.
Endnotes
1. Lorde, A. The master’s tools will never dismantle the master’s house. In Sister Outsider: Essays and Speeches. Crossing Press, Berkeley, 1984.
2. Fanon, F. Black Skin, White Masks. Grove Press, New York, 1967.
3. Malnarich, G. Learning community classrooms and educating for critical hope. In Teaching the Whole Student: Engaged Learning with Heart, Mind, and Spirit. Stylus, Sterling, VA, 2017, 57–78.
Posted in:
on Mon, June 29, 2020 - 10:53:23
Lilly Irani
Lilly Irani is an associate professor of communication & science studies at University of California, San Diego. She also serves as faculty in the Design Lab, Institute for Practical Ethics, the program in critical gender studies, and sits on the academic advisory board of AI Now (NYU). She is author of
Chasing Innovation: Making Entrepreneurial Citizens in Modern India (Princeton Univ. Press, 2019). She has a Ph.D. in Informatics from University of California, Irvine.
[email protected]
View All Lilly Irani's Posts
The best tech for contact tracing? Systems designed for healthcare workers
Authors:
Margaret Bourdeaux,
Mary Gray,
Barbara Grosz
Posted: Fri, June 26, 2020 - 2:37:22
Given the ongoing sense of urgency, some, particularly in technology circles, wonder if waiting for a healthcare worker to track down the virus makes sense. In a fight that depends on quickly finding and containing the coronavirus, couldn’t we do more with our mobile phones to put our country in a better position? Could the digital technologies that already connect us—sensing and tracing our every move—stop infected people in their tracks and help skip the step of large armies of human contact tracers having to call people? Unfortunately, this line of thinking has dominated much of the debate around how best to apply technology to contact tracing in the U.S.
In many ways, the phrase contact tracing elides the harder and most important part of a contact tracer’s work. Identifying and tracing potentially infected people are only the first steps in the process. Frontline human contact tracers need to persuade people, particularly those at our community’s margins—whether undocumented workers or the elderly living alone—to disclose a Covid-19 status that could stigmatize or further isolate them. Much of the work in stopping a wildly infectious disease involves communities and healthcare workers quickly mustering the resources to monitor and manage patients’ health. Yet these healthcare heroes have never had technologies built to specifically make the challenges of their jobs less daunting. That is why we need the HCI research community, well versed in human-centered design, to take the lead on building the technological innovations to trace and treat Covid-19.
Contact tracing is a core element of all plans for safely easing the social distancing and shutdowns resulting from the pandemic. For more than a century, this epidemiologists’ tool has proved the key to saving lives threatened by infectious diseases. By systematically mapping those exposed to contagious infections, it enables the containment of disease transmission. The success of contact tracing depends not only on meticulous data collection but also on the effective counseling of the people exposed and at risk. These two critical components must roll out as a coordinated effort. The people cornered by the pandemic are vulnerable and fearful for themselves and their loved ones. That is why successful contact tracing requires the counseling skills of trained healthcare workers connected to the communities that they serve. Perhaps more surprisingly, the data collection does too.
The magnitude of the labor force needed for contact tracing is daunting, leading some to propose that technology should take over. On Friday, April 10, Apple and Google announced joining forces to reduce the need for human contact tracers and allow people to take contact tracing into their own hands. These two tech companies, and many others, aim to empower anyone with a cellphone to anonymously text their Covid-19 infection status and to receive text alerts if they have come close to someone infected with the virus. Other computer scientists [1] and technologists [2] are developing methods for ensuring privacy and anonymity. But there are growing concerns that states and consumers might end up with poorly built apps that either leak information [3] or are simply ineffective [4]. We—a computer scientist, social scientist, and a physician who is involved in standing up the contact tracing program in Massachusetts—have deep reservations about the effectiveness and equity of this particular approach. We do, however, believe that human contact tracer teams would benefit enormously from a different approach to deploying technology to meet the challenges of contact tracing. Medical communities and public health depend on bringing a human-centered design approach to the challenge of large-scale contact tracing. The HCI research community could develop the technological support systems that healthcare workers and their patients will need to weather the threats of Covid-19.
Let’s review why digital contact tracing apps are, at best, a partial solution.
Cellphone data will miss, for example, data from an infected person who leaves their cellphone behind when they go grocery shopping. It will also miss the millions of Americans who don’t have their own cellphones or live in rural parts of the country with limited cellular or Internet access. More crucially, cellphone tracking alone cannot accurately report the nature of contacts. High-risk contacts range from being within six feet of an infected person who is not wearing a mask for longer than 10 minutes to touching a contaminated surface. Irrelevant contact data collected by technology on autopilot will needlessly consume precious human contact tracers’ time. The data collection needed for successful contact tracing relies on trained healthcare workers who are able to assess the potential risk associated with each interaction. This in turn requires the ability to patiently encourage someone who is sick and anxious to remember who they interacted with under relevant conditions.
Most problematic, however, is assuming that the nuances of contract tracing can be reduced to simply tracing networks of contacts from location data. To succeed, contact tracing programs require that people trust the entity to whom they are reporting. Trust is built on empathy [5], patience, and the ability to help someone who has just been exposed to a life-threatening disease. The delivery of troubling or frightening health news takes more than the ping of a text message. Human contact tracers are able to guide a rattled parent to think through who their child might have played with at a neighborhood potluck two weeks ago or help an undocumented immigrant find support and care should they fall ill. They also convey understanding and help people marshal the resources they will need to sustain a 14-day quarantine after they have been exposed. Thus, contact tracing hinges on deeply human exchanges. There is no app for that.
Too narrow a focus on cellphone location technologies could distract policymakers from building out the ranks of workers critical to implementing a comprehensive contact-tracing strategy. And a focus on this use of technology alone could distract computer scientists and systems engineers from developing the technologies that could make contact tracing more efficient and effective. These monumental contact tracing efforts require technologies built to assist rather than replace the legions of healthcare workers necessary for the months that there is neither a treatment nor a vaccine for Covid-19. Technologies, properly designed to support healthcare personnel in the tasks their work comprises, will be essential.
Supporting the Pillars of Contact Tracing
The Massachusetts Covid-19 Community Tracing Collaborative (CTC) has deployed the first comprehensive contact tracing program in the U.S. The CTC combines the leadership and state resources of the Baker administration’s Covid-19 Response Command Center, the state Department of Public Health, and the Massachusetts Health Connector with the deep domain expertise of Partners in Health, a Boston-based healthcare nonprofit known for tackling outbreaks of Ebola, HIV, and other deadly infections around the world. Partners in Health is training thousands of healthcare workers to fan out, virtually, across the state to begin the hard work of contact tracing. To determine the ways in which technology can help requires first understanding the four pillars that make contact tracing effective. These four key elements are:
- Identifying who has tested positive for Covid-19
- Helping those who have tested positive identify people they may have exposed to Covid-19
- Contacting those exposed and at risk of contracting Covid-19, connecting them with testing facilities, and counseling them through effective self-quarantining measures
- Finding a safe place to shelter those who cannot quarantine or isolate at home, equipping them with the resources they need, and monitoring their health and well-being.
The fourth pillar is especially challenging, because of the diversity of people needing such help—including frontline medical personnel, people fleeing domestic violence, and the homeless. We will need technologies that connect people to ongoing community-based support and guidance that’s relevant to their needs and worthy of their trust.
Computing technologies have great potential to assist with the work entailed by each pillar. For them to be useful and effective, though, requires that technologists work with epidemiologists and expert contract tracers to understand their work and their workflows, and from this, to identify their information needs and determine the best ways to meet them. The development of these technologies must follow established methods for human-computer interaction design, including the critical steps of iterative design. Otherwise they may make contact tracers’ work more difficult and time consuming in ways that electronic health records have done for doctors and gig-work platforms have done for gig workers [6].
To be helpful, contact-tracing technologies must be designed to work well for the people using them—namely, both human healthcare tracers and ordinary citizens whose contacts must be traced. It involves more than dropping a virtual pin on the map that logs the location of an infected person. Technical specs, like Bluetooth ecosystems or applications of what’s called differential privacy prioritize anonymity. But those technologies cannot get at the nature and quality of contact with any accuracy.
Contact tracing involves talking with people, bringing empathy and care to a call and evaluating and assessing the type of contact someone might have had with another potentially infected person. Until the world’s population has herd immunity (hopefully through effective and equitable vaccination programs), we will need contact tracing. This contact tracing will involve building rapport and keeping in touch with people so that when an inevitable outbreak throws us into a panic, there’s a calm, well-trained professional to guide us through the best next steps to protect ourselves and each other. In these cases, there’s no technological substitute for the distinctly human capacity of a healthcare worker.
Recommendations for Future Covid-19 Tech Directions
So, what kinds of technologies could make a difference? They must be intentionally built to assist rather than attempt to replace the most vital piece of contact tracing: the caregiver in the healthcare loop. Tech designed to help healthcare workers coordinate care for families and friends in the midst and aftermath of the virus will win the day. We list a few suggestions below, drawn from computer science and engineering research, that point toward the kinds of computer-human collaborative work technologies from which a newly recruited army of human contact tracers (HCTs) would benefit as they attack Covid-19 head on:
- Dynamic electronic reference tools that provide well-indexed access to answers to likely challenging questions, thus helping contact tracers offer consistent responses when asked something they aren’t sure how to answer
- For long-term monitoring, shared HCT records could assist teams in collaborating as they muster resources for someone who cannot quarantine at home and in routing care where needed
- Secure, shared databases and local area networks to support networked HCT teams doing intakes. HCT teams will need to coordinate initial calls, particularly to vulnerable, hard-to-reach groups, so that more than one person with the necessary language or cultural background can reach out, multiple times, for data collection
- Last, health workers and their agencies, across private-public partnerships, will need secure, centralized data storage that they can, collectively, trust to share information at the local, state, and federal levels.
Glamorizing the promise of tech and assuming it can substitute for the healthcare worker at the core of contact tracing’s effectiveness could waste more than time—it could cost lives. Who among the millions of Americans with either poor Internet access or no smartphone at all will we miss along the way? And, more important, what is in place to help people comply with quarantine and isolation directives?
As is well documented in epidemiology, those who have the most to risk when asked to share information about themselves can be the hardest to reach in a pandemic. Think of the risks an undocumented delivery person carries right now. They are in the thick of the threat that this virus poses because they are bringing food and comfort to anyone who can afford to order home deliveries. These workers need more than an app to navigate the risks that they face if they fall ill with Covid-19 and must come forward. Data collection, even the most privacy attuned, is only the first step. Tech innovation could be a game changer in the hands of health-work professionals. Teams of healthcare workers, assisted by technologies built to support their workflows and teamwork, will be critical in the coming months. Ultimately, the best technological interventions will be the ones that ask epidemiologists how they do their best work. It is worth investing in the contact-tracing technologies we need not just for Covid-19 this year but also for the inevitable next pandemic.
It is not too late to make a coordinated offensive strike. We could combine the power of human- centered tech design with the irreplaceable kindness of frontline health workers to help us keep Covid-19 at bay until we have a vaccine. In fact, doing anything less misses the opportunity of our lifetimes to unleash the deeply social potential of technologies.
Endnotes
1. https://pact.mit.edu/
2. https://covidsafe.cs.washington.edu/
3. Coronavirus: Security flaws found in NHS contact-tracing app. BBC News Technology. May 19, 2020; https://www.bbc.com/news/technology-52725810
4. Landau, S. Looking beyond contact tracing to stop the spread. Lawfare. Apr. 10, 2020; https://www.lawfareblog.com/looking-beyond-contact-tracing-stop-spread
5. Barber, B. The Logic and Limits of Trust. Rutgers Univ. Press, New Brunswick, NJ, 1983.
6. Gray, M.L. and Suri, S. Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Houghton Mifflin Harcourt, 2019.
Posted in:
Covid-19 on Fri, June 26, 2020 - 2:37:22
Margaret Bourdeaux
Margaret Bourdeaux is the policy liaison for Partners in Health’s Covid-19 contact tracing program. She holds appointments at Harvard Medical School, Brigham and Women’s Hospital, and the Belfer Center for Science and International Affairs at the Harvard Kennedy School of Government. Bourdeaux conducts research and fieldwork focused on health systems and institutions in conflict-affected states.
[email protected]
View All Margaret Bourdeaux's Posts
Mary Gray
Mary L. Gray is a senior principal researcher at Microsoft Research, a Berkman Klein Faculty Affiliate at Harvard University, and a faculty member in the School of Informatics, Computing, and Engineering, with affiliations in anthropology and gender studies, at Indiana University. Her research focuses on how everyday uses of technologies transform people’s lives.
[email protected]
View All Mary Gray's Posts
Barbara Grosz
Barbara Grosz is Higgins Research Professor of Natural Sciences in the Paulson School of Engineering and Applied Sciences at Harvard University and external faculty at the Santa Fe Institute. Her pioneering research in artificial intelligence (AI) has developed foundational theories of multi-agent collaboration and applied them to the design of healthcare coordination systems.
[email protected] Photo by Rose Lincoln.
View All Barbara Grosz's Posts
Listen, learn, respond, act
Authors:
Lucy Suchman
Posted: Thu, June 25, 2020 - 11:32:41
For those of us who occupy the closed worlds of privilege, it is time to listen. I write as a cisgender woman who has lived my life as White, academically educated, and economically secure, thanks to the initiative of my atheist Jewish grandparents who escaped the pogroms of the late 19th century and settled in New York City’s Lower East Side. I myself am now a settler on unceded Indigenous land belonging to Coast Salish peoples, in that nexus of empire and colonialism, British Columbia. Living in North America today as unmarked/unracialized offers extraordinary advantage, at the same time that it encourages the profound and abiding ignorance on which privilege depends.
The appearance in Interactions of the “Call to Action for the ACM” from Christina Harrington and co-authors [1], accompanied by the call to action to the computing community by Black colleagues and supporters [2], can be read as an appeal—better a demand—that we begin to address the ignorance of privilege and redress the injustices that it enables. Within the context of the ACM, that must include a critical understanding of the role that professional associations play, not only in facilitating interactions/connections, but also in policing the credentials of membership and the boundaries of what counts as authoritative knowledge, not least through the politics of citation. The fact that there are members with the requisite knowledge and sensibilities to see and articulate how legacies of systemic racism and injustice manifest in the contemporary ACM and the fields that it represents should be embraced, as an invaluable resource for reparation and collective transformation.
As the authors of an earlier Interactions post titled “Addressing Institutional Racism Within Initiatives for SIGCHI’s Diversity and Inclusion” observe: “Institutional racism does not have to be intentional or malicious to disadvantage minority groups. It merely has to occur in a way that harms those who are in the minority who have less power” [3]. The definition of harm, crucially, must come from those who are affected and be responded to in ways that those who experience the harm identify. In this case, the call is to strengthen the role of the authors in SIGCHI initiatives aimed at researching questions of diversity and inclusion, to support their ongoing learning, and to expand their responsibilities. As with all good design initiatives, they emphasize, this would necessarily be an iterative process of mutual learning.
The history of computing, as we know, is inextricably entangled with histories of (particularly U.S.) militarism. U.S. militarism, in turn, is sustained by and perpetuates geopolitical legacies based on territorial and strategic control through violence and intimidation. The foundation of colonial power is and always has been imaginaries of racialized (primarily White) supremacy, the justification for the subordinations necessary for exploitative labor. As Black Lives Matter is teaching us today, that history runs through the veins of the U.S. from its founding in slavery, through to its contemporary investments in the business of Black and Brown incarceration, whether in (increasingly privatized) prisons or in immigrant detention centers. Until that history, including the multiple forms of dehumanization and exploitation on which it rests, is acknowledged, the computing professions comprising the ACM will remain closed to the pluriverse of knowledges that might otherwise inform the design and development of computer-based systems, and our possibilities for collective informing and communicating. An opening up to other knowledges requires, first and foremost, overcoming deep-seated institutional prejudices that mistake meritocracy for “those who look like me,” difference for deficiency, and provincialism (for example, of Silicon Valley tech) for worldliness.
As I approach emerita status and the enormous privilege of a research-activist retirement, I have never been more conscious of the limits of my knowledge. My growing awareness of the vast range of Black (particularly feminist) scholarship that has been largely erased from the canons of mainstream pedagogy is a humbling, at times overwhelming, experience [4]. The ACM has the tremendous benefit of members who are at once committed to the potential of computing and to the rebuilding of the Association beyond symbolic gestures toward “diversity and inclusion”; these are the authors of this Call. The next steps are for us to listen, to learn, to respond, and to act.
Lucy Suchman
Recipient, 2010 ACM SIGCHI Lifetime Research Award
June 24, 2020
Endnotes
1. Harrington, C.N., Rankin, Y., Jones, J., Brewer, R., Erete, S., Dillahunt, T., and Brown, Q. A call to action for the ACM Interactions blog; http://interactions.acm.org/blog/view/a-call-to-action-for-the-acm
2. Black in Computing and Our Allies for Equity and Fairness. An open letter & call to action to the computing community from Black in Computing and Our Allies. Jun. 8, 2020; https://blackincomputing.org/
3. Grady, S.D., Wisniewski, P., Metoyer, R., Gibbs, P., Badillo-Urquiola, K., Elsayed-Ali, S., and Yafi, E. Addressing institutional racism within initiatives for SIGCHI’s diversity and inclusion. Interactions blog; https://interactions.acm.org/blog/view/addressing-institutional-racism-within-initiatives-for-sigchis-diversity-an
4. As an example of powerful pedagogy in Black feminist theorizing see: Rankin, Y. and Thomas, J. Straighten up and fly right: Rethinking intersectionality in HCI research. Interactions 26, 6 (Nov.–Dec. 2019), 64; https://interactions.acm.org/archive/view/november-december-2019/straighten-up-and-fly-right#R13
Posted in:
on Thu, June 25, 2020 - 11:32:41
Lucy Suchman
Lucy Suchman is professor of the anthropology of science and technology at Lancaster University in the U.K. Before taking up her present post she was a principal scientist at Xerox’s Palo Alto Research Center (PARC), where she spent twenty years as a researcher. She is the author of
Human-Machine Reconfigurations (2007); in 2010 she received the ACM SIGCHI Lifetime Research Award.
[email protected]
View All Lucy Suchman's Posts
A call to action for the ACM
Authors:
Christina Harrington,
Yolanda Rankin,
Jasmine Jones,
Robin Brewer,
Sheena Erete,
Tawanna Dillahunt,
Quincy Brown
Posted: Mon, June 22, 2020 - 5:49:56
On June 8, 2020, several Black scholars, academic researchers, graduate students, practitioners, and other members and affiliates of the Association of Computing Machinery (ACM) gathered to write a letter to express concern for the ACM’s response to this current political moment. The letter, shown below, detailed the importance of solidarity needed from such a major association that is an academic home for many of us. We have chosen to amplify this letter on public platforms in hopes to engage the association in a conversation about issues of systemic racism and injustices that have arisen associated with special interest groups and an overall need to examine the ways in which solidarity is communicated across the organization. We hope to make more common the practice of having conversations that while uncomfortable, will push the computing world to being more equitable and just. Our call is not alone in this. We are joined by a larger community of Black individuals in computing who wish to create more welcoming experiences for all who engage across the world of computing.
While we acknowledge and appreciate the effort from ACM’s Diversity and Inclusion Council to engage our letter and revamp the public response on ACM’s behalf, we feel it is important to hear from the ACM Governance and Officers who are in leadership. Many of our original recommendations have been integrated into the newly posted response on acm.org, yet we have not heard directly from anyone in leadership. Our direct request is to engage in a conversation with the ACM Council Leadership. Although an improvement, a revised statement alone is not the solution. Nor should this be relegated as simply a minority issue, but an opportunity for our entire association to reflect on our practices and ways we can advance computing as a science, profession, and a catalyst for change in an ever-changing society.
As of the date of this publication, we have been contacted by ACM’s CEO and are scheduling to meet. We look forward to continuing to strive for progress.
June 8, 2020 (we acknowledge the receipt of our original letter by the Diversity and Inclusion Council and a revised statement on the ACM website since that time)
Dear ACM Officers and Governing Body,
We are writing as a cohort of Black academic researchers, scholars, practitioners, designers, and students that are affiliated with ACM. Many of us are involved across several special interest groups and have served and engaged with the association from being contributing authors of technical papers, chairs of committees, SIG officers, conference paper reviewers, and conference organizers. While we acknowledge the recent statement that was shared on the ACM website, we are concerned with the vagueness and brevity of the ACM’s stance on the systemic racism and other social injustices that are currently causing civil unrest and being brought to the forefront across the world. These injustices gravely impact Black ACM members and students across the various disciplines. These injustices not only impact our ability to focus and produce at this time but our everyday survival. As researchers, scientists, and ethnographers, we understand the importance of specificity and transparency in how we discuss and address injustices that disproportionately impact marginalized communities. The current statement presented on the ACM website, while generally reaffirming the ACM’s commitment to inclusiveness, is woefully inadequate and warrants a more insightful actionable position from the ACM leadership.
There is a collective feeling among many Black researchers within the larger ACM community that the current statement fails to acknowledge our trauma and distress, and ultimately does not promote feelings of inclusivity and solidarity for many of us. Instead, the statement suggests a vague intention to foster equality and respect for all individuals across the ACM, without any mention of the direct impact on Black lives, which matter, that are in fact threatened by recent events, and historically. According to the ACM’s Commitment to Diversity & Inclusion website, ‘The ACM community encompasses everyone who works in computing or applies computing in another domain…Because its community is so broad, diversity and inclusion are central to ACM’s mission.’ Despite this statement, many of us within the ACM community feel compelled to draw attention to the systemic racism and exclusionary practices evident within the ACM leadership and the larger community. Descendants of the African diaspora have little representation within the larger ACM community, which claims to support diversity and inclusion of all its constituents while also highlighting and awarding the action of social impact among its members (According to a recent Taulbee Survey, Black faculty in CS and Information Sciences make up only 1.8%, while only 1.7% of new Ph.D. earners are Black). Because we are invested in the health and social impact of this community on its fellow researchers as well as the world at large, we feel compelled to express our concerns and to suggest both immediate and long-term actions as well as resources that the association might benefit from. There is a graveness and urgency to the current situation our country is experiencing — the acknowledgment of racism that runs rampant throughout the United States; the civil unrest and cries for social justice; the protests in the streets, on social media, in the news; and, in this instance, written communications to organizations, institutions, etc. This presents an important and opportune time for the ACM to recommit to a truly inclusive environment, particularly at a time when researchers are working tirelessly to meet publication deadlines and cultivate research projects that strengthen and enhance the computing discipline. We have aggregated a set of recommendations that can help not only to communicate ACM’s support of computing scholars who are currently struggling to focus amidst global civil protests against injustices but to also address some of the racial disparities among the association.
ACM-Related Recommendations for Immediate and Long-term Action:
- Revise the ACM’s official statement to specifically condemn acts of violence against Black people, systemic racism perpetuated in our professional community, and indifference. **
Position (Representation and Leadership)
- Nominate and recruit Black scholars from within the ACM community for leadership among ACM Boards and Committees. **
- Include Black scholars and their scholarly contributions in computing or computing-related curriculum as an example of epistemic resistance — rejecting academic strategies that silence the scholarship and testimonial authority of Black scholars in computing.
- Engage scholars, students, and departments from Historically Black Colleges and Universities (HBCUs) and other Minority-Serving Institutions (MSIs) in the larger ecosystem of the ACM to acknowledge the contribution of research coming from these institutions. This support may look like long-term scholarships for students or programs that support conference attendance. **
People (Publication Review Boards, Conferences)
- Create a board of ethics to implement values and aims of ACM’s Code of Ethics, including a specific mandate to govern the way technological research is promoted among marginalized and vulnerable populations, especially those that have been proven to disproportionately impact Black and Brown communities.
- Ensure that all panels feature representation of Black and Brown scholars who are knowledgeable in the topic discussed. This should be especially ensured for panels and workshops which focus on topics of race and technology, such as intersectionality or critical race theory. Black and Brown scholars should not be drowned out from these conversations, particularly Black women scholars who have contributed to the canon of intersectionality.
- Further diversify conference organizing committees, persons serving in conference leadership roles, and scholars invited to serve as plenary/keynote speakers. **
- Include sessions and activities at conferences and workshops focused on combating implicit bias and other forms of bias, particularly as applicable to technological development and impacts on creating a more inclusive society.
Practices
- Publish annual reports from each Special Interest Group (SIG) about demographics of participation in their sponsored events and initiatives, to whom and where funds are being allocated.
- Ensure that there is a diverse body of participants to inform and evaluate technology research and development that stands to exacerbate inequalities and inequities.
- Set aside money directly to fund events that focus on amplifying the scholarship of Black+Brown scholars in computing, and funds to broaden the participation of Black+Brown aspiring scholars in computing — undergraduate and graduate students, and junior researchers and faculty. **
- Create an equal opportunity accreditation committee for U.S. institutions to evaluate colleges and universities with respect to their inclusion of historically disadvantaged groups. This committee will ensure that federal funding of computing is in alignment with existing statues for equal opportunity, namely Title VI and VII of the Civil Rights Act of 1964.
- Remove structural power structures in the ACM review process which disadvantage Black scholars and other marginalized populations (e.g., either full transparency of reviewers and authors or double-blind process in which reviewers and authors are unknown). Reviewers should be encouraged to reflect on how identities inherently denote biases through an optional statement of positionality.
In addition to these ACM-related recommendations, we have aggregated a list of resources with which the ACM could benefit from engaging. This support could look like financial support, engaging with organizational principles and foundations in our own ethics, creating collaborations and partnerships, and including their leadership in events such as conferences, workshops, or panels.
** each of these statements now appear in the new ACM statement on acm.org.
Academic-Related Resources:
Cite Black Women is a campaign that advocates for “people to engage in a radical praxis of citation that acknowledges and honors Black women’s transnational intellectual production.”
Scholars for Social Justice define themselves as “a new formation of progressive scholars committed to promoting and fighting for a political agenda that insists on justice for all, especially those most vulnerable.”
Center for Critical Race and Digital Studies is “a network of prominent, public scholars of color who produce research, distribute knowledge, and convene stakeholders at the intersections of race and technology.”
blackcomputeHer is dedicated to supporting computing+tech education and workforce development for black women and girls. Our aim is to create rich technical programming, lead empirical research, and disseminate information that addresses the lack of inclusive innovation in tech across education and industry.
Social Justice Organizations that Advocate for Racial Justice:
NAACP Legal Defense Fund: Through litigation, advocacy, and public education, LDF seeks structural changes to expand democracy, eliminate disparities, and achieve racial justice in a society that fulfills the promise of equality for all Americans.
American Civil Liberties Union: The ACLU fights government abuse vigorously defends individual freedoms including speech and religion, a woman’s right to choose, the right to due process, citizens’ rights to privacy, and much more.
Color of Change: “We design campaigns powerful enough to end practices that unfairly hold Black people back, and champion solutions that move us all forward. Until justice is real.”
The LGBTQ Racial Justice Fund: The LGBTQ Racial Justice Fund is a collaborative philanthropic initiative focused on catalyzing change.
We also would like to highlight a larger effort put forth by the Black computing community here and an associated database of public response statements on racial justice and diversity.
Signed by members and affiliates of ACM,
- Rediet Abebe, Ph.D.
- Monica Anderson-Herzog, Ph.D.
- Elodie Billionniere, Ph.D.
- Robin N. Brewer, Ph.D.
- Douglas A. Brooks, Ph.D.
- Quincy Brown, Ph.D.
- Vetria Byrd, Ph.D.
- Curtis C. Cain, Ph.D.
- Marietta Cameron, Ph.D.
- Loretta Cheeks, Ph.D.
- Shaundra B. Daily, Ph.D.
- Tawanna R. Dillahunt, Ph.D.
- Edward C. Dillon, Jr., Ph.D.
- Brandon Dominique, Ph.D. Student
- Alyssa Donawa, Ph.D. Student
- Samuel J. Eaves, II, Ph.D.
- Sheena Erete, Ph.D.
- Denae Ford, Ph.D.
- Christina Gardner-McCune, Ph.D.
- Pamela Gibbs, Ph.D. Student
- Shamika Goddard, Ph.D. Student
- Siobahn Day Grady, Ph.D.
- Xava Grooms, Ph.D. Student
- Karen Hare, Ph.D.
- Christina N. Harrington, Ph.D.
- Leshell Hatley, Ph.D.
- Raquell Holmes, Ph.D.
- Earl W. Huff, Jr., Ph.D. Student
- Corey Jackson, Ph.D.
- Andrea E Johnson, Ph.D.
- Brittany Johnson, Ph.D.
- Jasmine Jones, Ph.D.
- Russ Joseph, Ph.D.
- Michel A. Kinsy, Ph.D., ACM Member
- Krystal A. Maughan, Ph.D. Student
- Aqueasha Martin-Hammond, Ph.D.
- Marlon Mejias, Ph.D.
- Ihudiya Finda Ogbonnaya-Ogburu, Ph.D. Student
- Chinasa T. Okolo, Ph.D. Student
- Imani Palmer, Ph.D.
- Andrea G. Parker, Ph.D.
- Timothy M. Pinkston, Ph.D., ACM Fellow
- Yolanda A. Rankin, Ph.D.
- Sekou L. Remy, Ph.D.
- Gloire B. Rubambiza, Ph.D. Student
- Angela D. R. Smith, Ph.D. Candidate
- Amber Solomon, Ph.D. Student
- Perry Sweeper, Sc.D.
- Jakita O. Thomas, Ph.D.
- Nicki Washington, Ph.D.
- Bryant W. York, Ph.D., ACM Fellow
* While these concerns are expressed by many in the community, some did not feel comfortable signing their names transparently for fear of potential repercussions.
Posted in:
on Mon, June 22, 2020 - 5:49:56
Christina Harrington
Christina N. Harrington is an assistant professor in the College of Computing and Digital Media at DePaul University. Her research focuses on how collectivism in design can support social change in areas such as health equity and digital access. She is the director of the Equity and Health Innovations Research Lab at DePaul and an Encore Public Voices fellow with the OpEd Project.
[email protected]
View All Christina Harrington's Posts
Yolanda Rankin
Yolanda A. Rankin is an assistant professor in the School of Information at Florida State University and the director of the DEsigning TechnOlogies for the UndeRserved (DETOUR) Research Lab which explores designing technologies with and for underserved populations. She is the recipient of the 2016 Woodrow Wilson Early Career Enhancement Fellowship.
View All Yolanda Rankin's Posts
Jasmine Jones
Robin Brewer
Sheena Erete
Sheena Erete is an associate professor in the College of Computing and Digital Media at DePaul University, where she co-directs the Technology for Social Good | Research and Design Lab.
View All Sheena Erete's Posts
Tawanna Dillahunt
Tawanna Dillahunt is an associate professor at the University of Michigan’s School of Information. Working at the intersection of human-computer interaction; environmental, economic, and social sustainability; and equity, her research investigates and implements technologies to support the needs of marginalized people.
[email protected]
View All Tawanna Dillahunt's Posts
Quincy Brown
Quincy Brown is the co-founder of blackcomputeHER.org and Director of Engagement and Research at AnitaB.org. She was previously a Program Director at AAAS and Senior Policy Advisor in the White House Office of Science and Technology Policy. She has supported women and girls in computing for more than a decade.
[email protected]
View All Quincy Brown's Posts
A challenging response
Authors:
Jennifer Mankoff
Posted: Wed, June 17, 2020 - 2:32:05
Being an ally means being uncomfortable.
—R.A.C.E. team, addressing institutional racism within initiatives for SIGCHI’s diversity and inclusion
I write this to support the courageous R.A.C.E. Diversity and Inclusion team members who documented their experiences in their recent blog post addressing material impacts of institutional racism. Their crucial yet risky labor to hold leaders accountable cast my recent accessibility experiences within SIGCHI differently and helped me realize how my silence made me complicit in perpetuating behavior that is wrong and must change. Although I am a senior member of the SIGCHI community and long-term tenured faculty, as a disabled female scholar I still feel vulnerable in sharing my story in this post. Nonetheless, I choose to speak out to amplify the voices of those who are experiencing institutional prejudice.
As feminist writer Sara Ahmed points out in her book On Being Included, data is “an important resource for diversity workers” because it can be used to expose “the gap between official descriptions of diversity and what the organization is doing.” Knowing this, I have led AccessSIGCHI efforts to produce a biennial report that documents accessibility issues within SIGCHI and sets goals for organizational change. We released three such reports. However, problems surfaced with the SIGCHI Executive Committee (EC) when, upon reading the third report, published in fall 2019, the EC inquired about the origins of the data used for the 2019 report’s summary, which were taken from CHI conference post-survey accessibility questions [1]. At the time, I had direct access to this data as a CHI Steering Committee member. I explained to the EC that I had requested and received permission to use this data for AccessSIGCHI. To my shock and surprise, the EC then sent me a “formal warning” and informed me that I had violated an EC policy, potentially broken data privacy laws, and that ACM would be “advised this had occurred.”
I lost countless hours in discussions, investigating what had happened, marshalling support, crafting a response, and managing my feelings of anxiety and alarm. I had never been notified about the policy I was accused of violating despite having requested and received similar data for the two prior reports. Ultimately, the EC issued a retraction. What did I learn from this experience, and how does it apply to EC’s reaction to the R.A.C.E. team’s vital efforts to collect the very data that can close gaps between official and unofficial realities of inclusion and diversity?
- It is vital that we understand the cost, both in time and wellbeing, of such aggressive actions against those who are doing transformational advocacy work. Being extremely senior helped me to mobilize support, professional and emotional, and I remain grateful these accusations did not involve my more vulnerable and junior collaborators in AccessSIGCHI. Until one is on the receiving end of such demands, one cannot understand their toll and inhibitory effects. These effects are magnified exponentially when applied to those less powerful.
- We must recognize that systems of oppression go beyond individuals. Institutional and structural factors are usually at issue when there is a sustained pattern of misbehavior, even if each individual member of an institution (such as the EC) were acting in good faith and are actively working to increase inclusion. Institutions must guard against becoming defensive or non-compliant when members wish to collect and share data, even if doing so may lead to changes in their very structures and policies. “You must get used to being uncomfortable and get used to this not being about your feelings if you plan to help and not hinder people of color in their efforts for racial justice.” says Ijeoma Oluo in So You Want to Talk about Race. This is done by responding with openness, transparency, and engagement. Citing policy when volunteers collect and seek to publish such data comes across as threatening, silencing, racist and ableist to those on the receiving end. While the EC issued an apology to the R.A.C.E. team for their volunteering experience, they do not admit to wrongdoing, commit to self-education, or actively address structural or institutional problems and plans for eliminating them. It is time for the EC to make use of the expertise of SIGCHI members to engage in an unbiased study of institutional racism’s presence and impact. Without a commitment to public self-examination, it is difficult to believe much will change.
- A commitment to inclusion that does not comprise direct action will succeed only in helping the majority to feel better. Those being excluded still have to deal with the problems that remain. When diversity work is just “about generating the ‘right image’ and correcting the wrong one” [Ahmed, On Being Included], it should not be surprising if that work appears ineffective to those most directly affected. For example, incremental improvements in accessibility, combined with occasional backward sliding, gave rise to a protest at CHI 2019. Listening in these situations is not enough, particularly when the EC actively resists change, despite clear instructions about needed reforms [2]. For example, although the EC voted on and approved an accessibility chair after the 2019 protest [3], after over a year, the EC has not filled this role. In their blog post, the R.A.C.E. team asks them to “Make us leaders.” Will the EC comply?
I want to thank the R.A.C.E. team for helping me to find power to tell my story and affirm they are not alone and that our work must involve structural transformation. When work is done by marginalized groups, such as the R.A.C.E. team and AccessSIGCHI, it is especially important to nurture and cultivate their perspectives. The cost of the commitment of individual time to these efforts should not be underestimated, especially given the likelihood that many such individuals are continually being asked to put extra time into representing their community as well as advocating for themselves. Even small blows to these efforts have the potential to eliminate a gift that could otherwise help our community to better itself.
I benefit tremendously from the work that SIGCHI does to coalesce and advance our profession, including its positive actions to advance inclusion and accessibility, and I want this worthwhile organization to succeed. However, I am opposed to seeing our professional organization move forward at the expense of its most vulnerable members. It is our collective duty to use the information shared by the R.A.C.E. team and in this blog post to ensure the EC does not ignore, subvert, and marginalize the requests for action of community members who are racial minorities, disabled, or less enfranchised.
—Jen Mankoff
Endorsed by the members of AccessSIGCHI [4]:
- Cynthia Bennett, Carnegie Mellon University
- Megan Hofmann, Carnegie Mellon University
- Anne Spencer Ross, University of Washington
- Tiago Guerreiro, Universidade de Lisboa
- Rua Williams, University of Florida
- Richard Ladner, University of Washington
- Erin Brady, Indiana University, Purdue University Indianapolis
- Raja S. Kushalnagar, Gallaudet University
- Elaine Schaertl Short, Tufts University
- Karyn Moffatt, McGill University
- Jennifer Rode, University College London
- Anne Marie Piper, University of California Irvine
- Stacy Branham, University of California Irvine
- Kristen Shinohara, Rochester Institute of Technology
- Kyle Rector, University of Iowa
- Dhruv Jain, University of Washington
Endnotes
1. An anonymous copy of all of emails and the retraction are in the linked document.
2. See the July 2015 EC Meeting Minutes and the talk given to the EC the day after the protest in 2019.
3. See the May 2019 EC Meeting Minutes
4. I (Jen Mankoff) take full responsibility for this post and the ways in which I have handled these challenges. AccessSIGCHI group members very much share any credit for the positive actions that I have been part of and wanted to show their support of this blog message.
Posted in:
on Wed, June 17, 2020 - 2:32:05
Jennifer Mankoff
Jennifer Mankoff is a CHI Academy member and the Richard E. Ladner Professor at the University of Washington. Her research is focused on giving people the voice, tools, and agency to advocate for themselves. She strives to bring both structural and personal perspectives to her work. Mankoff has been recognized with an Alfred P. Sloan Fellowship, IBM Faculty Fellowship, and Best Paper awards from ASSETS, CHI, and Mobile HCI. She has chronic Lyme disease and identifies as disabled.
[email protected]
View All Jennifer Mankoff's Posts
We need to talk about digital contact tracing
Authors:
Ali Alkhatib
Posted: Mon, June 15, 2020 - 4:34:49
Recently, Apple and Google discussed developing and distributing a digital contact-tracing system that will inform people when they’ve been exposed to someone who’s contracted Covid-19, and communicate to people that they’ve been exposed to you if you later test positive yourself. Apple has since deployed a beta of iOS 13 with the first parts of this system exposed to developers and users. At the time of this writing in late April and early May, we’re desperate for information and weary from not knowing who’s caught Covid-19, who’s still vulnerable, who gets it worse and why, or even how to treat it. We’re desperate for any information we can get our hands on. This proposal by Apple and Google promises some information that we can finally dig into. Unfortunately, this system of digital tracing isn’t going to work, and we need to stop the plan before it gets off the ground.
I have written about “digital forests” and the harmful downstream effects of simplifying and reducing how we track and measure our messy world (and earlier this year gave a working talk about it). I wrote about how we develop models to describe the world—and our lives within it—that are necessarily but also dangerously reductive. In his book Seeing Like a State, James Scott writes about foresting in Europe, the Great Leap Forward, and other case studies of centralizing ideologies that ignore or trample on the delicate balances of our social and natural ecologies [1]. During the Industrial Revolution, it was difficult to reason about air pollution so we wrote it off entirely; decades later, that came to haunt us in the form of acid rain and widespread deteriorating public health.
Digital contact tracing has all the hallmark characteristics of these case studies. If we’re not careful, we’re going to adopt these systems, and the facile, naive models of the world that these systems create will give us a dangerously incomplete picture of the world. Worse, if these systems become the principal drivers of our policies, we’ll go from looking at an incomplete map of the world to navigating with one.
The schemes we’re reading about—some from joint partnerships such as Apple and Google; others unilaterally presented by surveillance startups such as Palantir and Clearview AI—all offer their own flavors of omnipresent surveillance and differ in bits and pieces. I’m going to focus on the only credible proposal—the one Apple and Google have floated—because I hardly even know where to start with the other proposals. Suffice it to say that we shouldn’t take Palantir’s offer to surveil us even more than they already do. As for Clearview, they scraped all of our images without our consent and subsequently endured numerous embarrassing data breaches.
Let’s talk about what digital contact tracing generally entails by using Apple and Google’s system as a model case. If you know what their proposal entails, feel free to skip ahead to the next section. But chances are that something has changed in the time since I started writing this article, so if for no other reason than a clear timeline, you should probably at least skim for a sense of where things stood when I was writing.
What This Scheme Will Do (More or Less, as of May 2020)
The simplest, most direct way to implement digital contact tracing would be to track where you are at any given moment. If you spend 20 minutes at a coffee shop, somewhere there’ll be a record that you were at Starbucks from 12:00 p.m to 12:20 p.m. Somewhere, theoretically, there would be records for everyone participating in this scheme, such that if someone wanted to ask the system “Who was at Starbucks at 12:10 p.m.?” a list would come back with your name on it. To be honest, Google probably already has the means to produce a list like this from Google Maps. You can even request a record of all the places Google Maps knows you’ve been over a certain timeline.
Instead of recording where you are the way Google Maps does or other location-tracking apps you’re familiar with, under this scheme your phone records whom you’re near. To illustrate what this means, let’s make up and walk through an example.
Under this program, your phone would send out a signal every 5 minutes with a unique word. So from 12 p.m. to 1 p.m., your phone would send out messages like this:
12:00: Apple
12:05: Banana
12:10: Orange
12:15: Kiwi
12:20: Potato
12:25: Cherry
12:30: Tomato
12:35: Pear
12:40: Coconut
12:45: Orange
12:50: Peach
12:55: Eggplant
13:00: Kumquat
In this system, no other phone is sending out these exact words; if anyone ever hears Kumquat, it’s because their phone was near your phone at 1 p.m. If my phone has Kumquat in its archive, it means that I was near you at 1 p.m.
Let’s say I was in line behind you at a coffee shop for 10 minutes from 12:12 to 12:22. Since I’d be close to you, I’d get the messages Kiwi (at 12:15) and Potato (at 12:20), and my phone would hold on to that. I don’t get your name, number, or any other information about you—just the two words. We’re each just sending out unique words every five minutes and keeping track of what we happen to hear from all the things around us. This app isn’t keeping track of who we are, or where we are, just what words it’s hearing.
Now let’s say that you feel sick at 1 p.m., so you go to the doctor and a PCR test comes back positive for Covid-19. You’ll go into your phone and say “I just tested positive for Covid-19” and your phone will send the last several days’ worth of words to some database. Something like {Apple, Banana, Orange, Kiwi, Potato, Cherry, Tomato, Pear, Coconut, Orange, Peach, Eggplant, Kumquat}, but spanning several days or weeks, so we’re talking about thousands of these words.
When my phone checks in with the database and downloads all the new words it has, it’ll check and find that I had heard “Apple, Banana,” which means I must have been near enough for our phones to chat for something like 10 minutes. Is 10 minutes a big deal? No, not really. But what if I had heard all of the words above except for Apple and Banana? That would mean that from 12:10 p.m. to 1 p.m. I was close enough to you that our phones were talking. The 10-minute example is something like standing behind someone in line at a coffee shop. The 50-minute example is more like having lunch at the table next to you at a restaurant.
Let’s change the scenario from coffee shops and restaurants to something more sensitive: Let’s say both of us were at an STD clinic at the same time for 45 minutes. What’s relevant is that we were someplace together at the same time, not where that place was. This approach promises to give us the someplace together answer without revealing the STD clinic detail. All anyone knows is that you sent out a bunch of words, and I heard the words that put us in the same place for about 45 minutes. Where exactly were you when you sent Tomato? Doesn’t matter. What was I doing when I received Eggplant? Don’t worry about it.
Designing a system that goes out of its way like this to avoid knowing the literal location you’re at wasn’t the simplest approach, and it’s at least a little praiseworthy that this system is being developed in a slightly complicated and confusing way with the right goal in mind. If you look at Strava or Snapchat or some other location-tracking app, you can always see yourself as a dot on a map, moving around perhaps with other dots on the same map; this throws out that entire approach. In the context of Covid-19, all anyone really needs to know is that we were near each other. This approach strives to give us that and nothing else.
I think it’s important that we understand what this system does so we can talk meaningfully about what it doesn’t achieve, and what it doesn’t even bother trying to do. Which is a lot.
Welcome Back
We’re gearing up to sacrifice substantial amounts of our privacy and anonymity in the world in exchange for a woefully dubious solution to our uncertainty. To quote Arundhati Roy, “If we were sleepwalking into a surveillance state, now we are running toward it because of the fear that is being cultivated” [2]. Let’s talk about why: First, this system isn’t going to give us a better sense of the world in the most crucial places; second, this system is going to undermine our privacy and dignity in ways that we can only begin to imagine right now.
Let’s start with why this system won’t work.
Digital contact tracing will exclude the poor, children, and myriad other uncounted groups
In the description I made earlier, I pointed out that Apple and Google’s plan calls for some sort of proximity detection, probably involving Bluetooth and some of the fancier, newer technology that your smartphone may have if it was introduced in the past few years. Some of the technical requirements of this system preclude older devices from working, meaning the people with older smartphones won’t be able to benefit from alerts to this system unless they upgrade to newer phones—something they almost certainly can’t afford to do, nor should they be obligated to in order to receive necessary information about risk exposure. Ars Technica recently published a report that several billion smartphones don’t have the technology necessary to participate in the proximal location-sensing scheme Apple and Google are talking about [3]:
The particular kind of Bluetooth “low energy” chips that are used to detect proximity between devices without running down the phone’s battery are absent from a quarter of smartphones in active use globally today, according to analysts at Counterpoint Research. A further 1.5 billion people still use basic or “feature” phones that do not run iOS or Android at all. “[M]ost of these users with the incompatible devices hail from the lower-income segment or from the senior segment which actually are more vulnerable to the virus” [3].
Every day we get more data showing that communities that have historically been excluded or deliberately untreated by our healthcare infrastructure are particularly vulnerable both in contracting Covid-19 and in mortality. As Gina Neff has written, the locations linked to new Covid-19 cases are prisons, food processing facilities (specifically meatpacking), and nursing homes [4]. These are settings in which people are under immense pressure to take whatever work they can find, including high-risk essential work like logistics for Amazon in their obscenely dangerous warehouses [5], or delivering food and groceries to us despite receiving basically or literally no functional personal protective equipment, putting them in direct contact with potentially hundreds of people every day.
The lines don’t divide just along class; digital contact tracing omits children, substantially problematizing the picture we get as soon as families are involved. These aren’t minor issues; they’re major gaps that will lead to systematically and unaccountably poor data. The spaces and the people we should be most concerned for, and paying the most attention to, won’t show up in a digital contact-tracing system like any of the ones proposed so far.
Digital contact tracing staggeringly misses the point of care that we should be most concerned with.
This system’s exclusions will decisively undermine its accuracy and endanger everyone
The risks of acting on bad data—the result of excluding the poor, children, and other high-risk groups—are unspeakably high. This is different from working with a small sample size or even from working with no data. Building a dataset that excludes entire categories of people—as we’re beginning to do with Native American Covid-19 patients [6]—skews our vision of the world in ways we won’t be able to account for, and with confidence that’s unearned, ultimately steamrolling the groups that we leave out. This is why the CDC employs “sentinel surveillance” to track the flu—because acting only on the data we get from people who are wealthy enough to go to a doctor for the flu would give them a dangerously misleading picture of its spread every year [7]. Everyone needs access to the same level of care, with no omissions, or we risk lulling ourselves into oblivious complacence while this virus and others like it sweep through our communities.
We should care that the gaps in our knowledge will be consequential and non-random, sabotaging our efforts to understand what’s actually going on in the world and undermining our response.
Proximity is a dangerously simplistic way to model Covid transmission
There are issues inherent to proximity-based contact tracing that have nothing to do with access to the technology, as well. Recent studies have shown that people don’t need to be in proximity to someone with Covid-19 to catch it—they can just be downwind of that person [8]. It’s not just that people’s access to technology is messy and more complicated than people living in Silicon Valley tend to think (as evidenced by the push for bandwidth and device-intensive education tech)—it's that the world doesn’t model particularly easily, and certainly doesn’t fit the bubble of what’s within range of a Bluetooth module on your smartphone.
I wrote about this in my paper on “street-level algorithms” at CHI in 2019. People are constantly finding new ways to exist and make their ways through the world, so algorithmic systems that try to model our behavior and interactions will always be a step behind [9]. We’ll have to constantly follow up on the errors these systems make, struggling to fix the damage they do in their false positives and false negatives, all in this hopeless chase to automate what essentially needs humans.
Digital contact tracing systems that render the world as normally distributed space with spheres of influence and contact characterized by radio waves will consistently leave us with dangerously wrong pictures of our exposure.
Proximal tracing doesn’t maintain privacy the way advocates seem to think it does
We also need to talk about what proximal contact tracing offers and what it doesn’t, especially since we’re adopting this contrived system to avoid “absolute” locations. I said earlier how absolute location tracking might say that we were both at some GPS coordinates. Or it might offer a street address. In either case, it would reveal that we were both at some semantically meaningful place from 8 p.m. to 10 p.m., which could potentially be embarrassing or even damaging. Proximal tracking promises instead to reveal only that we were together between 8 p.m. and 10 p.m.
The problem is that neither of these approaches is particularly helpful to me when I’m trying to maintain privacy about where I meaningfully am. While there are unique risks associated with my absolute location being revealed (for instance, revealing where I live by showing where I tend to be every night), a revelation that I was among many people who are under investigation or otherwise suspected of some socially improper behavior is potentially just as damning as if there’s a record that puts us all at the same mailing address for that time—like if all of us have Kumquat on our phones.
The details of whether this data is held in centralized or decentralized locations doesn’t matter much if the notification that I’ve been exposed to Covid-19 forces me to surface to seek care, revealing my association with that group.
Proximal location tracking has already recently been used to out dozens, perhaps hundreds, of gay men in Morocco [10]. It didn’t help that their exact locations were obscured; they were “nearby.” They were outed by their associations, not by their coordinates. And credible fears about whether law enforcement organizations will exploit these systems to harm vulnerable groups have proven difficult to squash, particularly with misleading and confusing statements by government officials [11].
This isn’t to say that absolute location tracking is the right approach, or that proximal tracking is a misstep, but that this distinction misses the point and fails to address the concern: We need laws that guarantee against the use of any data we collect for anything other than the confidential care of Covid-19. And we need to earn the trust of people giving us that data—something Clearview, Palantir, and arguably tech companies in general are inherently incapable of doing, given their business models of advertising and data mining that motivate their very existence.
We need real guarantees of protection against the use of this data for any purpose beyond the containment of Covid-19, and we need to commit to those principles.
These Are Just the Beginning of the Issues
The decisions we make now are going to dog us for generations. It’s important that we move quickly, but it’s as important that we don’t run headfirst into a surveillance state that offers nothing but oppression and more uncertainty.
I spent five years at Stanford studying computer science and noticed an insight among tech folks that I hadn’t observed as much elsewhere. It was that if you have a complex mathematical problem that you can restate as another problem, or as a collection of other problems, then you might find down the road you can solve one of those problems and dramatically simplify the original task. Suddenly something that takes hundreds or thousands of hours to compute can be done virtually instantly.
I saw this kind of thinking applied to social problems all the time. Social problems are sticky and entangled and don’t seem to reduce very well. But if you can perform a kind of problem-space arbitrage and convert the social problem into a technical one, you might find an insight in mathematics or computation later on that collapses the problem into something more manageable, or even trivial. It’s enticing.
It’s also wrongheaded. It’s unproductive because it doesn’t solve either the old social problems or the new technical ones, and it’s dangerous because it obscures the real problems that we need to address—the ones we all recognize and otherwise feel entitled to engage with as fellow humans.
There are technical problems that need to be solved in this pandemic, but this isn’t one of them. There are so many issues of uncertainty that digital contact tracing introduces, and so many gaps in whom it protects, that we need to reject it now before it gets off the ground, before we get lured into a false sense of certainty that everyone is covered and protected when in reality all we’re accomplishing is deepening an existing divide between the empowered, the wealthy, the influential… and the people holding all of them up.
We need certainties in our life. Systematizing contact tracing into a digital system won’t give us what it claims to offer, and nor will it give us certainty. If we need some certainties, here are a few:
- We need more personal protective equipment for everyone. If we want people working in warehouses, nursing homes, and meat-processing facilities, we need to give them the means to avoid infection.
- We need more tests. We need so many tests for so many people that the cost becomes zero and the hurdle disappears. Los Angeles recently announced that every resident will get free tests whether they get referred for a test or not. We need more of this.
- We need to train human contact tracers. Algorithms will never be able to adequately model the world we live in and construct, and will never be able to keep up with all the weird, quirky qualities of the spaces we build and occupy. This job calls for human intelligence.
- This is such a given that it’s almost not worth saying, but we need to change our relationship with labor. A year ago, if someone at a coffee shop felt sick, they might not have bothered to see a doctor. They almost certainly would have tried to work through it (at least at first). I think we all appreciate now how dangerous that is, but the threat of crushing medical debt and homelessness coerces people into work. That culture of coercive labor needs to end, or the rest of our lives will be punctuated by pandemics fueled and ultimately spread by workers who were compelled to continue working
There’s already so much we need to do, and so much that needs to change, for us to have anything resembling the lives we appreciated before this pandemic. We don’t also need to build and subject ourselves to omnipresent digital surveillance via personal devices. We must reject digital contact tracing. We need to care for one another.
Some Addenda
Here are some questions I’ve gotten while writing this article and some brief thoughts in response:
- What about DP3T? DP3T (Decentralized Privacy-Preserving Proximity Tracing) suffers a lot of the issues raised above. It still essentially requires tech and will leave out the sites of super-spreading (prisons, nursing homes, meat processing facilities); it still mostly avoids the public health policy changes that need to happen; it still goes for proximity versus absolute location tracking, which I outlined earlier isn’t addressing the problem. In the introductory paragraph of their README file, they describe the project as seeking to “provide a technological foundation to help slow the spread of the SARS-CoV-2 virus” [12]. Like I said before, we don’t need to center technology in coming up with solutions, and in fact this will only deepen divides in care that are already staggering.
There’s also a point to be made that technological foundation usually means something more like “the political decisions were baked into the code and you can’t do anything about them unless you have a Github account and file an issue or make a pull request.” But I’ll leave that to people who aren’t yet completely exhausted making this point.
- What about Taiwan and South Korea? They adopted digital contact tracing and it seems to have worked. There are a few issues to this: First, there were substantial issues with Taiwan’s and South Korea’s deployments of digital contact tracing, stemming from uncomfortable revelations about peoples’ private lives [13,14]. The second is that the way the US has cleaved a boundary between wealthy and poor—informed by slavery and white supremacy and urbanization and housing policies—really complicates any effort to translate tech-centric policies and programs from other cultures.
It’s also worth saying that in Taiwan and South Korea, people can go see a doctor without worrying about crushing medical debt. Paid sick leave is a guarantee. People use masks and when the government tells them what to do, they generally trust their political leadership not to be corrupt or incompetent. America enjoys almost none of these advantages. Digital contact tracing wasn’t a silver bullet (even in the context that more people in Taiwan and South Korea have smartphones); it was one of many more decisive characteristics of their economies, politics, and cultures that made it possible for people to get tested when they needed to get tested, to stay home when they needed to stay home, and to listen to the local and national leaders who were themselves advised by credible medical and epidemiological experts.
Endnotes
1. Scott, J.C. Seeing Like A State: How Certain Schemes to Improve the Human Condition Have Failed. Yale Univ. Press, 1998.
2. Roy, A. Author and activist Arundhati Roy on Covid-19 and the Indian response. France24. May 23, 2020; https://www.youtube.com/watch?v=CSZ-PtlY4HE
3. Bradshaw, T. 2 billion phones cannot use Google and Apple contact-tracing tech. Ars Technica. Apr. 20, 2020; https://arstechnica.com/tech-policy/2020/04/2-billion-phones-cannot-use-google-and-apple-contract-tracing-tech/
4. Neff, G. To fight this pandemic we must use stories. Twitter. Apr. 30, 2020; https://twitter.com/ginasue/status/1255847973797462016
5. Press, A. Amazon workers say warehouse health precautions are insufficient. JewishCurrents. Apr. 28, 2020; https://jewishcurrents.org/amazon-workers-say-warehouse-health-precautions-are-insufficient/
6. Nagle, R. Native Americans being left out of US coronavirus data and labelled as ‘other.’ The Guardian. Apr. 24, 2020; https://www.theguardian.com/us-news/2020/apr/24/us-native-americans-left-out-coronavirus-data
7. Roush, S. Chapter 19: Enhancing Surveillance. In CDC Manual for the Surveillance of Vaccine-Preventable Diseases. 2017; https://www.cdc.gov/vaccines/pubs/surv-manual/chpt19-enhancing-surv.html
8. Lu, J., Gu, J., Li, K., Xu, C., Su, W., Lai, Z., Zhou, D., Yu, C., Xu, B., and Yang, Z. COVID-19 outbreak associated with air conditioning in restaurant, Guangzhou, China, 2020. Emerging Infectious Diseases 26, 7 (2020).
9. Alkhatib, A. and Bernstein, M. Street-level algorithms: A theory at the gaps between policy and decisions. Proc. of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2019, Article 530; https://doi.org/10.1145/3290605.3300760
10. Alami, A. Dozens of gay men are outed in Morocco as photos are spread online. New York Times. Apr. 26, 2020; https://www.nytimes.com/2020/04/26/world/middleeast/gay-morocco-outing.html
11. Morrison, S. Minnesota law enforcement isn’t ‘contact tracing’ protesters, despite an official’s comment. Vox Recode. Jun. 1, 2020; https://www.vox.com/recode/2020/6/1/21277393/minnesota-protesters-contact-tracing-covid-19.
12. @veale et al. Decentralized Privacy-Preserving Proximity Tracing-readme. 2020; https://github.com/DP-3T/documents/blob/1b49aef42ec6fb59f17929fec3b54ec88b636405/README.md
13. Kim, N. ‘More scary than coronavirus’: South Korea’s health alerts expose private lives. The Guardian. Mar. 5, 2020; https://www.theguardian.com/world/2020/mar/06/more-scary-than-coronavirus-south-koreas-health-alerts-expose-private-lives
14. Singer, N. and Sang-Hun, C. 2020. As coronavirus surveillance escalates, personal privacy plummets. New York Times. Mar. 23, 2020; https://www.nytimes.com/2020/03/23/technology/coronavirus-surveillance-tracking-privacy.html
Posted in:
Covid-19 on Mon, June 15, 2020 - 4:34:49
Ali Alkhatib
Ali Alkhatib is a research fellow at the Center for Applied Data Ethics, an initiative of the Data Institute at the University of San Francisco. Originally trained in anthropology, he now mostly investigates how people relate to individual algorithmic systems and with algorithmically mediated social ecologies by adapting theoretical lenses and frameworks originating in the social sciences to understand these phenomena.
[email protected]
View All Ali Alkhatib's Posts
Comics as Covid-19 response: Visualizing the experience of videoconferencing with aging relatives
Authors:
Ernesto Priego,
Peter Wilkins
Posted: Fri, June 12, 2020 - 10:40:33
In the period between March and May 2020, we have been working on simultaneous projects employing qualitative methods to create comics in response to the Covid-19 pandemic. We have been motivated and inspired by disability and feminist scholarship, taking on board evidence-based warnings about the dangers of well-meaning but counterproductive “empathy building” exercises in design and health interventions [1].
Interested in further developing the participatory narrative research affordances of comics, we have rehearsed with autoethnographic methods, reflecting on our own experience during the pandemic. We have operated from the positioning that “the primary ethical standard against which any autoethnography should be evaluated is ‘an ethic of accountability'” [2]. This piece hopes to contribute to meeting that principle.
In its “Mental health and psychosocial considerations during the Covid-19 outbreak,” the World Health Organization (WHO) advised to “keep in regular contact with loved ones (e.g., via telephone, email, social media or video conference)” [3]. Similar advice regarding contact via videoconference has been issued by several international charities, including HelpAge International.
The emotional aspect of comics and comics reading goes beyond the remit of this contribution, but it has been an empirical consideration for us. The WHO has also advised paying “attention to your own needs and feelings” and to “engage in healthy activities that you enjoy and find relaxing” [3]. The autoethnographic work of making a comic served this purpose too, allowing for the integration of an enjoyable activity (drawing, making comics, sharing them with family and friends) during times that have been indeed stressful for all of us.
It is in this critical context that we brainstormed ways of responding creatively to the Covid-19 pandemic. The common experience of caring for and staying in touch with our own relatives remotely prompted the co-design of the one-page comic shown in Figure 1.
![](/images/uploads/blogs/remote-caring-comic_web.jpg)
Figure 1. Art by Peter Wilkins; Editing by Ernesto Priego (2020) CC-BY. [4]
Visualizing the family videoconference
The comic depicts two middle-aged brothers having a FaceTime call with their parents during the lockdown in Vancouver, Canada. One of those brothers is also one of the authors: Peter. His family members gave their express consent to be drawn (and, in one instance, named by first name) in a comic representing a family videoconference call that would feature Peter himself too. As autoethnographic work, the comic is the result of principles of artistic and narrative practice; for example, the representation and naming of living persons follow accepted practice in autobiographical comics [5].
The comic highlights an initial discomfort with the technology that gives way to accommodation and enjoyment. The parents are shown having some trouble engaging with the technology, but eventually they get to grips with it and enjoy the meeting with their sons, though the tech keeps reasserting itself. We think the comic can help, through humor, to visualize the idea that videoconferencing platforms can serve effectively as substitutes for physical presence—if not perfectly, at least closely enough to afford satisfaction to all participants.
The characters’ foibles assert themselves as they would in physically present encounters, while the technology affords new venues for these foibles: the mother’s anxiety over the buttons, the father’s attempts to get in the picture. Certain repetitions in the conversation (“stop touching your face”) and actions on screen (dad trying to get in the camera view) led to the choice of images and text.
The comic was created through a combination of manual and digital methods, employing traditional analog drawing tools and ProCreate. It was initially reviewed in rough sketch form, then in color, then in a semifinal version. Three iterations followed after formative and summative feedback from the participants represented, and a final version was agreed on by all parties.
It is tempting to flout stay-at-home guidelines to visit parents, but we know that the risks to people over 70 is significant. The comic aims to encourage the use of videoconferencing software to connect with aging parents and relatives, rather than take the risk of physically visiting them during the Covid-19 crisis.
As of April 14, 2020, fatality rates for those over 80 years of age was five times the global average. On May 1, 2020, the United Nations released the Secretary-General’s Policy Brief: “The Impact of Covid-19 on Older Persons.” It stated that “each of us—states, businesses, international organizations, companies, communities, friends and family—need to step up our effort to support older persons. We must do everything possible to preserve their rights and dignity at all times” [6]. At the time of writing this, the Covid-19 outbreak keeps having a devastating impact on the lives of older persons. It is everyone’s responsibility to contribute to their well-being with the means we have at our disposal, however humble they may be.
Limitations
The comic is idiosyncratic and subjective; it represents the interactions of one specific family. No formal data gathering or analysis went into creating the image beyond first-person observations of a single family by one of its members during a series of family calls. As a visual medium, comics present significant accessibility challenges for those with reading and/or visual disabilities or neurodiversity. Accessibility is an essential requirement for us in design work and it would be possible, if circumstances allow, to produce a detailed descriptive alt-text and/or a version of the comic in a PDF with machine-readable text so it could be read aloud.
Acknowledgments
Both authors contributed equally to this article. With many thanks to colleagues at the Centre for Human-Computer Interaction Design, City, University of London, and at Douglas College, Canada; to Simon Grennan, Francisco de la Mora, and to all the Wilkins and Priego-Morris extended families. For more on the authors’ previous work using comics, please visit https://blogs.city.ac.uk/parablesofcare/
Endnotes
1. Bennett, C.L. and Rosner, D.K. The promise of empathy: Design, disability, and knowing the “other.” Proc. of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2019, Paper 298, 1–13; https://doi.org/10.1145/3290605.3300528
2. Medford, K. Caught with a fake ID: Ethical questions about slippage in autoethnography. Qualitative Inquiry 12 (2006), 853–864; https://doi.org/10.1177/1077800406288618
3. World Health Organization. Mental health and psychosocial considerations during the COVID-19 outbreak. Mar. 18, 2020; https://www.who.int/docs/default-source/coronaviruse/mental-health-considerations.pdf?sfvrsn=6d3578af_2
4. Wilkins, P. and Priego, E. A Comic Visualising the Experience of Video-conferencing with Aging Parents During the COVID-19 Pandemic. 2020; https://doi.org/10.25383/city.12348959.v1
5. El Refaie, E. Visual modality versus authenticity: The example of autobiographical comics. Visual Studies 25, 2 (2010), 162–174, DOI: 10.1080/1472586X.2010.502674
6. United Nations. Secretary-General’s Policy Brief: The impact of COVID-19 on older persons. May 1, 2020; https://www.un.org/development/desa/ageing/news/2020/05/covid-19-older-persons/
Posted in:
Covid-19 on Fri, June 12, 2020 - 10:40:33
Ernesto Priego
Ernesto Priego (
https://epriego.blog) is a researcher and lecturer affiliated with the Centre for Human Computer-Interaction Design, City, University of London. He leads the Parables of Care: Creative Responses to Dementia Care project. He is also the editor in chief of The Comics Grid: Journal of Comics Scholarship, which he co-edits. Priego has a Ph.D. in information science from University College London.
[email protected]
View All Ernesto Priego's Posts
Peter Wilkins
Peter Wilkins leads the programs for at-risk and refugee youth at the Training Group at Douglas College, Canada. He is an editor of
The Comics Grid: Journal of Comics Scholarship. His comics work includes
1982: The Year I Saw the Jam (writer/artist);
Parables of Care (adapter, editor); and
I Know How This Ends (artist). He also co-edits and writes for
Graphixia, A Conversation about Comics. Wilkins has a Ph.D. in English and critical theory from the University of California, Irvine.
[email protected]
View All Peter Wilkins's Posts
Addressing institutional racism within initiatives for SIGCHI’s diversity and inclusion
Authors:
Siobahn Day Grady,
Pamela Wisniewski,
Ron Metoyer,
Pamela Gibbs,
Karla Badillo-Urquiola,
Salma Elsayed-Ali ,
Eiad Yafi
Posted: Thu, June 11, 2020 - 11:00:32
Realizing that All Can be Equal (or R.A.C.E) was the name that we gave ourselves when we volunteered in November 2018 to be ACM SIGCHI Innovators for Diversity and Inclusion (D&I). Our D&I team was composed of seven individuals, primarily junior scholars, who all identify as racial minorities and who had all personally experienced the ways in which academia could be exclusionary. We were eager to contribute to the efforts to make SIGCHI more inclusive of diverse perspectives, including our own. This blog post recounts the shared pain and experience of our team, which ultimately culminated in our resignation (see resignation letter).
Race and ethnicity are difficult topics to discuss. They evoke feelings of guilt and defensive reactions from those in the majority, as well as anger and frustration from those in the minority. We acknowledge that this blog post may evoke similar emotions, which made us hesitant to post it. Yet, the protests to bring justice for George Floyd and the #BlackLivesMatter movement made us realize that now is not the time for silence. It is more important than ever to shed light on the institutional racism that exists in our world.
The term institutional racism was first coined in 1967 by Black political activists Stokely Carmichael and Charles Hamilton. They described institutional racism as:
... less overt, far more subtle, less identifiable in terms of specific individuals committing the acts. But it is no less destructive of human life. …[it] originates in the operation of established and respected forces in the society, and thus receives far less public condemnation than [individual racism] [1].
Sir William Macpherson later defined institutional racism in 1999 as:
The collective failure of an organisation to provide an appropriate and professional service to people because of their colour, culture, or ethnic origin. It can be seen or detected in processes, attitudes and behaviour which amount to discrimination through unwitting prejudice, ignorance, thoughtlessness and racist stereotyping which disadvantage minority ethnic people [2].
Institutional racism does not have to be intentional or malicious to disadvantage minority groups. It merely has to occur in a way that harms those who are in the minority who have less power. This was what happened when we were told by ACM SIGCHI leadership to immediately take down our IRB-approved survey on diversity, inclusion, and marginalization within SIGCHI. We spent nearly a year working on this survey as part of our D&I service, but once it began to get traction (over 100 responses), it was blocked. To add insult to injury, we were accused of conflicting with ACM's interests because we wanted to disseminate our findings back to the SIGCHI community via a peer-reviewed publication. Our survey was then superseded with an “official” SIGCHI diversity and inclusion survey, which erased and appropriated our efforts.
As a group of racial minorities (three of whom are Black), we were not surprised by these actions but were deeply disappointed that this happened under the guise of promoting diversity and inclusion within SIGCHI. There were many alternative solutions that could have bolstered the efforts of our D&I team instead of stymieing them. For example, the SIGCHI D&I leaders could have asked us to lead the efforts for the official SIGCHI D&I survey, giving us credit for our work and the opportunity to take a larger, more visible service leadership role within SIGCHI. Yet these types of opportunities are rarely given to racial minorities, even though they are pivotal in helping us succeed in the academy.
Instead, ACM policies were (mis)interpreted and applied inconsistently and in a way that hurt a racial minority group. Other ACM volunteer groups have launched surveys with the intent to publish and did so with the support of ACM SIGCHI leadership. This makes us question whether the examination of marginalization and exclusionary practices within the SIGCHI community— led by a minority group—was just too uncomfortable for SIGCHI D&I leaders, who are predominantly part of the majority.
We requested reparations to address what had transpired (see response letter), but those requests were ignored. As a group of racial minorities, we constantly ask ourselves if the situation would have been different had we not been people of color? As minorities in academia, these forms of oppression are some of the most dangerous due to their subtlety and heedlessness of those in the majority, who often refuse to acknowledge that such forms of institutional racism still occur. Yet we in the minority know the truth. Therefore, we must ask ourselves not whether institutionalized racism exists, but rather how we might work to fix it?
Our intention in publishing this blog post is to initiate uncomfortable conversations about race and minority status that must occur within the SIGCHI community if we want to eradicate institutional racism and promote equity and inclusion. We recognize that this will be a continuous, iterative process for all of us. Therefore, we end this blog post by providing some actionable ways to change the status quo based on our own experiences and insights from the 112 SIGCHI members who responded to our survey:
- Admit that institutional racism exists: SIGCHI might be relatively better than other communities, but institutional racism is still present in our community. The only way to truly address it is to first acknowledge it exists.
- Promote the work of minorities: Even if you do not completely agree with it. Sometimes the ways in which minority groups address a topic will be different than those in the majority, and that is good. We know that our survey had an edgy undertone of addressing marginalization within SIGCHI, rather than merely improving existing diversity and inclusion initiatives. This was by design as we wanted to amplify the voices of those who felt angry and unheard.
- Refrain from performative allyship: Performative allyship is the act of supporting those in oppression either privately or publicly so that it makes the person feel or look good but does not facilitate real change. True allies should be willing to take risks on behalf of those who are being marginalized and share in some of the burden required for change. In other words, being an ally means being uncomfortable.
- Volunteerism is not an excuse: Often within SIGCHI, we are reminded that our organization is a volunteer-run community; thus, while there are mistakes made, everyone is doing their best. This is not an excuse to allow implicit biases, institutional racism, and other exclusionary practices to persist within our community. Many of our survey respondents described SIGCHI as a clique, where outsiders are unfairly treated and the “rich get richer.” In short, being a SIGCHI volunteer/leader is a position of power and should be treated as such.
- Make us leaders: The only way to truly change institutional racism within an organization is to have equal leadership, where there is no majority or minority. It is not enough to have majority leaders who are supportive of minorities within the lower ranks. The goal of diversity and inclusion is not simply to make sure that minorities are within the community, it is to ensure their success.
Thank you for taking the time to read our blog post. We hope that our blog post moves you to positive action. On a final note, we want to protect the vulnerable junior members of our group who have the most to lose from speaking publicly about this experience. We hope that you respect this goal and help us achieve it.
Endnotes
1. Carmichael, S., Hamilton, C.V., and Ture, K. Black Power: The Politics of Liberation in America. Vintage, 1992.
2. Macpherson, W., Cook, T., Sentamu, J., and Stone, R. The Stephen Lawrence Inquiry. 1999.
Posted in:
on Thu, June 11, 2020 - 11:00:32
Siobahn Day Grady
Siobahn Day Grady is an assistant professor of information systems at North Carolina Central University. She is a black woman, second-generation college graduate, and AAAS IF/THEN® Ambassador. She seeks to broaden participation in computing, especially for women and girls of color in STEM. Her research includes human-computer interaction and machine learning.
[email protected]
View All Siobahn Day Grady's Posts
Pamela Wisniewski
Pamela Wisniewski is a newly tenured associate professor in the Department of Computer Science at the University of Central Florida. She is a mixed-race, first-generation college student, a mom, and a scholar activist who conducts human-computer interaction research to promote adolescent online safety and benefit other vulnerable populations.
[email protected]
View All Pamela Wisniewski's Posts
Ron Metoyer
Ron Metoyer is an associate professor of computer science ande and associate dean in the College of Engineering at the University of Notre Dame. He is an African American first-generation college graduate who works at the intersection of human-computer interaction, information visualization, and broadening participation in computing.
[email protected]
View All Ron Metoyer's Posts
Pamela Gibbs
Pamela Gibbs is an information and interaction design Ph.D. candidate at the University of Baltimore. She is a black woman and second-generation college graduate. Her research focuses on the impact of utilizing gaming to encourage learning in math. She is passionate about creating equity for underestimated groups.
[email protected]
View All Pamela Gibbs's Posts
Karla Badillo-Urquiola
Karla Badillo-Urquiola is a Ph.D. candidate at the University of Central Florida. She is a Latina woman in STEM, member of the ACM SIGCHI Latin American HCI Community, and a mother of two. Her research in the field of HCI lies at the intersection of online safety and foster youth.
[email protected]
View All Karla Badillo-Urquiola's Posts
Salma Elsayed-Ali
Salma Elsayed-Ali is a Ph.D. student at the University of Maryland, College Park. Her research sits at the intersection of Human-Computer Interaction (HCI), design, and computer-supported cooperative work (CSCW).
[email protected]
View All Salma Elsayed-Ali 's Posts
Eiad Yafi
Eiad Yafi is an assistant professor at the Malaysian Institute of Information Technology, Universiti Kuala Lumpur. He is a Syrian researcher and activist promoting diversity, inclusion, and justice in the Arab world, which has been facing unrest and a long history of injustice.
[email protected]
View All Eiad Yafi's Posts
How can designers fight the coronavirus?
Authors:
Nikhil Welankar
Posted: Wed, June 10, 2020 - 9:41:53
Most of us had never imagined that countries would have to declare lockdowns to protect their citizens from the coronavirus outbreak. In its wake, government authorities, health departments, doctors, nurses, security services, police, scientists, food suppliers, medical suppliers, nonprofit organizations, and industry leaders across the world are uniting and working tirelessly. And the rest of humanity is wholeheartedly thankful. At the same time, some innovative minds are exploring cost-effective masks, robot helpers for isolation wards, and space-effective ventilators to solve contextual problems. This made me wonder: How can designers fight the coronavirus?
We designers are problem solvers. We apply design thinking to solve problems, to improve the daily lives of consumers, and to help businesses achieve their goals. How can we now apply our skills to deal with this pandemic?
As a part of the design thinking process, we use various user research methods to understand pain points, needs, and wish lists. Can we apply user research methods to understand misconceptions and perceptions about coronavirus? To understand symptoms? Or how about to understand patients’ experiences with medical services or the struggles of corona warriors? Such research can provide inputs to help improve the overall support ecosystem. Remote user research methods such as digital surveys, remote usability testing, virtual focus group sessions, digital diary studies, and telephone interviews can all be applied. Methods such as competitive analysis in the corporate world help us understand the weaknesses and strengths of the competitors; similar methods can be applied to study how other countries and organizations have dealt with the situation, which can help improve our own processes and frameworks moving forward. Considering the world’s socio-cultural-economic diversity, contextual understanding of the problem is also critical. Human-centered design is the key to providing relevant solutions.
Once our research is completed, we identify the exact problem and understand the gaps with regard to an existing ecosystem. We can clearly define problems, mental models, misconceptions, and needs through various analysis methods. This definition is important in order to ideate the solution.
Once the problem is identified, we create high-level concepts to explore possible solutions. The solution can be a product, a service, a process, or a framework. Designers can envision a lot of physical or digital solutions to help Covid-19 patients and the support staff. They can also play a great role in spreading health awareness, especially in the world of social media, where rumors can be more viral than authentic scientific information.
Once the ideation is completed, we create a quick and tangible model of the envisioned solution. This early-stage prototype helps in validating our direction before spending a lot of effort in building the solution. We test our prototype with the end users to get their feedback. Designers can also create prototypes to explore various cost-effective, space-effective, and contextual product/service ideas in order to deal with the Covid-19 situation. These products can be in the form of mobile apps, physical kiosks, health check-up kits, or even a service framework to improve patients’ overall medical experience.
Looking at the rapid spread of Covid-19, we certainly don’t have time in hand to apply the classical design-thinking process. We need to identify key challenges quickly and address them through creative solutions as soon as possible. In the remaining part of this article, I would like to highlight some of the challenges and possible solutions, as per my limited understanding.
Key challenges and possible solutions
Lack of awareness about Covid-19 is one of the most pressing challenges observed across the world. When the outbreak in China was first reported, people from other countries thought it might just be a local issue in China. That left room for Covid-19 to spread—until it was too late. By the time citizens and local authorities realized its severity, it was already taking a toll. Still, a large part of some populations are not fully aware of the coronavirus. How can designers solve this challenge?
There are many ways this can be addressed. Designers can use all possible mediums—print, audio, video, animation, mobile apps, games, and social media—to spread awareness in an engaging way. Designers can redesign authentic information channels such as those provided by the WHO and the UN, as well as government websites and mobile apps, in order to make them stand out. Social media strategists can run WhatsApp groups, Twitter feeds, Facebook pages, Instagram accounts, and text- and voice-message-based services to spread information. Recently a telecom service provider started providing coronavirus information instead of a ringtone when a caller makes a call. We can also design offline channels, such as a coronavirus awareness poster hung at a local grocery store or a coronavirus-awareness brochure printed on a grocery bag.
In the semi-urban and rural areas of India lacking a strong Internet presence, TV and regional language radio are the effective mediums. Designers, artists, writers, poets, singers, musicians, dancers, and storytellers can play an important role. Storytelling is the most effective approach, and the coronavirus story should be told in the form of a traditional folk art specific to the region. For example, for centuries in Maharashtra, traditional musical forms such as kirtan and powada have been effectively applied to spread social awareness. In Gujarat, katha and pravachan are the popular forms to raise such awareness.
MIT researchers have shown that fake news spreads faster than the truth. This becomes a major roadblock when it comes to false information about Covid-19. Rumors and false information can create panic, especially when there is an unprecedented situation like Covid-19. Social media companies have been working hard to remove fake information but it is not an easy task. Fortunately, anyone can help: Citizens can report fake information themselves. On the other hand, media houses have become more cautious to avoid spreading fake information due to this transparent ecosystem. But more focused and organized efforts are required. Government and private stakeholders need to collaboratively work on a twofold framework, first to remove the fake information and second to propagate authentic information. It would be worth exploring if an automated framework based on the artificial intelligence and data science could remove fake information at a massive scale. Researchers, design thinkers, product designers, AI experts, and data scientists can collaborate to build such product-service-based frameworks. Such frameworks will be effective, especially in large, socioculturally diverse countries such as India, China, and the U.S.
Personal and social hygiene are the important factors in keeping the coronavirus at bay. But this is challenging in densely populated countries. Hygiene protocols need to be propagated through advertising campaigns, celebrity endorsement, and demonstrations. Recently, celebrities from the Indian film and sports worlds published videos of them washing their hands. It created a deep impact, as people adore these figures. And there are many ways in which hygiene protocols can be propagated and rewarded. Incentives and discounts, for instance, can encourage people: The cleanest suburb or apartment/society can get a prize or discount on its water or electricity bill, creating healthy competition. A gamified mobile app can be designed for conducting a hygiene-based competition, with a score card for, say, apartment complexes or neighborhoods to keep the momentum. Educational documentaries, short films, advertisements, animated stories, newspaper cartoons, and games can help in spreading hygiene awareness. Recently, a few private companies in India have designed a Sanitizing Tunnel, through which people can pass to while they are out to buy groceries. Such contextual solutions need to be explored further.
In many cases, senior citizens live alone in apartments, as their children are far away in another city or country. With housekeeping staff unavailable due to the lockdown, senior citizens are not able to cook their meals and also at times unable to step out for groceries due to health issues. This challenge urgently needs to be tackled. Contextual solutions include senior citizen help groups formed by local societies and township bodies that deliver necessary food, medicine, and groceries. WhatsApp groups have formed to help senior citizens with medical emergencies. These seniors may not be that tech savvy or may not have smartphones to be able to respond digitally. A simple call- or text-based helpline service could be a user-friendly option. Text or voice instructions in a regional language can help users who don’t have English proficiency. A local registry of lonely senior citizens, along with frequent connections with them, can be a very helpful means of support. Such responsibility can be shouldered by the local residents, where one family can make calls to 10 senior citizens one in a week to check on their well-being. More such contextual solutions need to be explored.
When an entire city, state, or country is under lockdown due to the pandemic, mental health problems are bound to rise. Recently, a few people committed suicide out of fear of getting infected by the coronavirus. When you lock down or put restrictions on a large population, managing mental health becomes an imperative. Such mandatory loneliness or togetherness may result in an outbreak of frustration, depression, and anger. Designers can conceptualize digital channels to facilitate remote counseling sessions, helpline support, remote yoga and meditation sessions, remote group activities, and motivational games.
In short, as all professionals—especially medical and security staff—are working tirelessly, designers can play a vital role by exploring contextual, relevant, and effective design solutions. Designers can apply various research methods to understand needs, pain points, and gaps in the current system in order to improve it. Technologies such as mobile apps, social media, artificial intelligence, IoT, 3D printing, AR, and VR can empower designers in exploring the next generation of solutions to solve grassroots problems. Designers can explore contextual and localized solutions that consider the socio-cultural-lingual diversity of the world.
Finally, I would like to reiterate the philosophy of universal brotherhood and humanity. All the countries are one family. Let’s fight against Covid-19 together to restore peace, health, and happiness in the world.
Posted in:
Covid-19 on Wed, June 10, 2020 - 9:41:53
Nikhil Welankar
Nikhil Welankar is an Experience Director with over 18 years of global industry experience. He has conducted more than 250 user interviews and more than 150 usability tests for 70-plus UX design projects. He has also authored five research papers and conducted upwards of 50 UX awareness sessions.
[email protected]
View All Nikhil Welankar's Posts
What Is the future of data sharing for research?
Authors:
Giovanna Vilaza
Posted: Mon, June 08, 2020 - 9:30:55
Digital data collection for health research usually follows well-established methods. In many of the labs that work with mobile sensing, research subjects are provided with consent forms, task instructions, and sensor devices or apps. Once the research subjects agree to participate, the expectation is that they will comply with the procedures and allow their lives to be digitally tracked. After that, they are usually dismissed.
Even though participants are such a vital part of scientific discoveries, they are often considered objects to be observed—one more entry in a database. Such well-established ways of placing those being monitored into passive roles have gained even more prominence. With the surge of Covid-19, there has been a noticeable increase in initiatives for health surveillance. From contact tracing to apps that monitor daily symptoms, the pervasiveness of smartphones is being exploited to collect data from large segments of the population. It is a blossoming field for those who work in this area, as the urgency to understand this illness is pushing mobile sensing in ways never before seen.
Given this sudden demand for broader behavioral monitoring, the debates over population-level surveillance have gone mainstream. In the particular case of contact tracing, academics are now discussing issues of individual privacy, the consequences of false positives (and negatives), and the actual efficacy of such an approach [1]. On the other hand, the media, governments, and tech companies are claiming that transmission speed may be reduced only if a significant part of the population is monitored continuously. Contact tracing has been enforced in countries in Asia and framed as a way to “help authorities identify virus hotspots and better target health efforts” [2].
By providing arguments that surveillance is the right path for recovery, governments and the media are forging a positive-only view of the subject. A consequence of the support for contact tracing and other symptom-tracking approaches could be a radical change in how people perceive privacy threats and accept being monitored for the “public good.” It could be speculated that efforts from the public sector and big corporations to convey the benefits of surveillance could lead the masses to believe this one-sided version of the story, without weighing its risks. Decisions about disclosure are known to involve a trade-off. If the perception of social or individual benefits is stronger than the identified possible risks, people are willing to share sensitive information [3].
If a shift toward more public acceptance of health surveillance indeed prevails, national-level repositories of the mobile-sensing data could also become very attractive to governments and scientists [4]. Large-scale platforms containing information such as clinical diagnoses, mobile-sensing data, and behavioral tracking data could allow incredible epidemiological discoveries. Before Covid-19, the landscape of such platforms was dominated by genetic bio-banks and clinical-trials repositories. Mobile-generated data was still a novelty. Nowadays, massive centralized data centers containing information about thousands (or millions) of individuals are growing around the world, such as All of Us in the U.S. and iCarbonX in China; they include digital sensors as a data source.
However, if a shift toward more acceptance does not prevail, large-scale surveillance will be at risk of low cohort diversity. First, to derive significant and fair conclusions from a dataset, a diverse range of people with different characteristics is required. Unless most of the population is tracked, the knowledge acquired will not be representative and may benefit only those who were available and agreed to be monitored. As Daniela and Nicole Rosner discuss on this blog, “prioritizing the most likely to be reachable tends to benefit well-educated white people who have already long benefited from the healthcare system.” What can HCI, UX, and technology design practitioners and academics offer to facilitate more inclusive recruitment for data platforms? What knowledge, tools, and evidence have we produced (or can we provide) that can be useful in this context?
Besides inclusive recruitment, the search for public acceptance should not overwrite the need to consider possible impacts on all the segments of the population. Individuals from different backgrounds might have a different understanding of potential privacy risks, and people with stigmatized clinical diagnoses might suffer from the consequences of a data leak asymmetrically [5]. Broader acceptance should not result in less public diligence about privacy and how data can be abused. Such individual differences need to be taken into account because ill-intentioned initiatives may lure people with the promise of future advances in research, but come with a hidden agenda [6]. As Christopher Frauenberger states, “We might see the coronavirus serving as the scapegoat to implement modes of mass behavior manipulation by private companies.” How could HCI knowledge and approaches be used to support and protect citizens from these scenarios? Could HCI help overcome the uneven understanding of risks and help tackle vulnerabilities in case of privacy breaches?
As mentioned earlier, digital data collection for health research often follows well-established approaches. The pandemic has brought more attention to the subject of population surveillance, as seen in the reflections from Rosner and Frauenberger. However, the Covid-19 emergency has not changed the passive role attributed to those having their symptoms and contacts monitored. Most of the decisions about what data will be tracked, how it will be used, and who will have access to it are made from the top: by governments, health authorities, research institutions, and big corporations. When data repositories are built this way, power and knowledge are given to those who store the data, not to those who provide it [4]. This serves to strengthen the already existing inequalities between contributors and receivers.
The most significant change that the pandemic should bring is not that surveillance becomes more broadly accepted. A real change would be to see those proposing surveillance platforms finally placing citizens at the core of their decisions, by listening to their concerns and providing them with direct protection and benefits. If people are to be asked to open up their lives for health surveillance or research, they should be respected, and their preferences prioritized. It is about time we put more efforts into understanding the needs from the different segments of the population and design for more inclusive participation and agency in research. The well-established approaches for data collection do not suffice anymore, as behavioral monitoring is being considered at a national level. Aggregated data might mean better healthcare now and in the future, but it is also a tool for power and mass control [6]. The path to reach acceptance should involve respect, transparency, and an ethic of involvement by communities from all backgrounds [7].
More than ever, those who are in public, academic, and industry positions hold the responsibility of taking into account any potential for harm that novel ideas can bring to each individual. This pandemic, or any other alarming situation in the future, should not mean that moral principles and personal autonomy are put aside. Large-scale digital surveillance for public health may gain momentum with contact tracing now. Still, we need to keep reflecting, discussing, and pushing for an ethical development in the field, through the papers we write, the products we build, and the ideas we share with others.
The pandemic has been a challenging time in many aspects, but it can also mark a moment when meaningful changes began. It forced many to stop, and some to reconsider how things have been done until now—and how different they could be. From this process, hopefully, a brighter future can emerge for data sharing, health surveillance, and research platforms alike—a future in which acceptance does not mean renouncement of rights and values, but rather a conscious choice based on terms and conditions that are negotiated and never imposed. This should become the new normal. The next advances in data-collection practices depend on us, researchers and designers in the HCI and health tech field, as we choose how we conduct our own projects and support those of our community.
Endnotes
1. Gillmor, D.K. Principles for technology-assisted contact-tracing. ACLU white paper. April 16, 2020.
2. Phartiyal. S. India orders coronavirus tracing app for all workers. Reuters Technology News. May 2, 2020.
3. King, J. "Becoming part of something bigger" Direct to consumer genetic testing, privacy, and personal disclosure. Proc. of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-33.
4. Kostkova, P., Brewer, H., de Lusignan, S., Fottrell, E., Goldacre, B., Hart, G., Koczan, P., Knight, P., Marsolier, C., McKendry, R.A., Ross, E., Sasse. A., Sullivan, R., Chaytor, S., Stevenson, O., Velho, R., and Tooke, J. Who owns the data? Open data for healthcare. Frontiers in public health 4 (2016), 7.
5. Petelka, R., Van Kleunen, L., Albright, L., Murnane, E., Voida, S., and Snyder, J. Being (in) visible: Privacy, rransparency, and disclosure in the self-management of bipolar disorder. Proc. of the 2020 CHI Conference on Human Factors in Computing Systems. ACM, New York, 2020, 1–14.
6. Zuboff, S. Big other: Surveillance capitalism and the prospects of an information civilization. Journal of Information Technology 30 (2015), 1.
7. Costanza-Chock, S. Design Justice: Community-led Practices to Build the Worlds We Need. MIT Press, 2020.
Posted in:
Covid-19 on Mon, June 08, 2020 - 9:30:55
Giovanna Vilaza
Giovanna Vilaza is a TEAM early-stage researcher, halfway to her Ph.D. in the Department of Health Tech, Technical University of Denmark. Her current project is about a participant-centered future for behavioral monitoring in open-access data platforms. She is a University College London and KTH Royal Institute of Technology alumni.
[email protected]
View All Giovanna Vilaza's Posts
A solution without a problem? Seeking questions to ask and problems to solve within open, civic data
Authors:
Caroline Sinders
Posted: Mon, June 01, 2020 - 10:16:42
The city of Amsterdam is using artificial intelligence (AI) to help sort through and triage their version of 311 calls. Chicago is using AI to help analyze and decrease rat infestations and to prevent them in the future. There is value in applying AI to urban challenges, but that value must come with explicit protections to privacy and citizen safety. AI has known issues with bias, from a widely documented inability to recognize different genders and races to its use in predictive policing. Thus, when using AI as infrastructure or in technology that interacts with society, safeguards must be put into place to mitigate harm. When, how, and why we use AI must be analyzed and unpacked. This article will examine a potential use case for using AI in understanding city and civic data, and the potential benefits as well as harms that could arise.
What data is right for civic challenges?
The power of AI is that it can recognize patterns at scale and offer insights and predictions. For civic technology, this may be a great way to see patterns across a city to potentially prompt deep investigations into recurring mold, faulty pipes, and potholes across neighborhoods and larger districts. These problems should be viewed as data; for example, the reports that citizens make are data. Data is important—it's the backbone of artificial intelligence. Smart cities, IoT devices, and open government initiatives are all creating astounding amounts of data. Data presents stories about the people who live in the communities and cities where the data was gathered, and this data must be safeguarded against all different kinds of bad actors. Every data point from a software application, from the amount of usage to data from a social network, is made by humans. Meaning, data isn’t just something cold or quantitative; it’s inherently human. Thus, the privacy and transparency of datasets—who collects it and how it’s stored—is incredibly important.
The problem, and the lynchpin, with using AI will always be data. Data can be messy and unstructured, so even before thinking about using AI, one has to ask: Does the data we need even exist? If so, is our dataset big enough? Is it structured? And there is always the moral quandary: Do we really need AI here or does a city or community just need more human power behind this problem?
AI in cities...right now
The city of Amsterdam is using AI to help sort through their version of 311 calls (free service-request calls in cities ) in a product called Signal (Figure 1). Residents can file requests about issues in public spaces through calls, social media, or a Web forum. Tamas Erkelens, program manager of the Chief Technology Office innovation team that built Signal, told me in an interview about the process that routs service requests, saying that users had to pick their own categories. But, as he said, “Users think in problems and not in governmental categories. Instead, we started asking users to describe the problem or take a photo, and we use machine learning or computer vision to [help sort the problems].” Signal uses AI to help sort residents’ requests into different categories, following the logic of the user to put residents—and residents’ frustrations—first as a priority for complaint triage. Erkelens also told me that his team has successfully created a model capable of detecting more than 60 categories in the Dutch language but that the system is also audited by a person. This human auditing is important to ensure safety and understanding. Even when we leverage AI to handle an influx of complaints, humans are still needed to assure quality.
![](/images/uploads/blogs/Signal_web.jpg)
Figure 1. Screenshot from Amsterdam’s Signal app.
Data: It’s more than just numbers
When working with civic technology and data initiatives from local, state, and federal governments, a variety of problems can pop up. Generally, there may not be similar data standards across cities, or even across the same city’s various departments, and datasets can be of differing sizes and structures. Comparing or combining data from different municipalities can be difficult. As Georgia Bullen, executive director of Simply Secure, a nonprofit focusing on privacy and design, explained, "A lot of the big problems internally for cities are actually that different departments have different datasets, and it’s really hard to combine them...Cities are storing data in different ways, and they have different policies in place that affect what those values even mean.”
When integrating any form of technology into civic and civil services, the technology itself coupled with the service it’s intended to augment must be viewed holistically within the context of a service design problem. That is, if an app is trying to help solve the problem of how to reduce the number of residents contacting the city over potholes, the app combined with the city service must be viewed as an integrated and holistic problem under the service design for the user experience of contacting the city. What kind of biases or issues can the app add? What kind of unintentional hierarchies can it create? How much data is it creating, and how is that data stored? These are some questions that designers and engineers can ask in this situation.
We can look at the New York City’s 311 as an example of unintended design consequences. A 311-type of system can lead to the police being called, even unnecessarily, when the system is very easy to use, or because of patterns learned by machine learning. As Noel Hidalgo of Beta NYC, a civic technology organization, says, “It’s really easy to file a noise complaint on the 311 app.” The 311 app promotes noise complaints as something you can report using the service. In the app, when a user selects “apartment” to denote more information about the noise complaint, the police immediately become notified. So even if a user is just trying to report a noise complaint without contacting the police, they actually can’t do that because of how the app is designed.
The problem with this design for 311 is that it relies on a belief in government to respond to your needs. For a lot of citizens, government can be a thing to fear, such as neighborhoods of color who face over-policing or gentrification, which can lead to clashes among new and old residents. For some citizens, their local and federal governments, including 311, are not systems that they can necessarily trust. There are so many facets to the way in which a governmental system can be used, but do citizens understand that, and does the AI model reflect all of these facets? The model and AI system must be designed for how citizens actually interact with governmental systems, meaning that they may not completely understand or fully use 311 (e.g., calling reactively instead of preemptively). Yet the system must also be designed for how the city or government needs it to be used. This means 311 should be designed for both reactive and preemptive reports.
A real-world example
As an example, let’s unpack Million Trees NYC, a citywide, publicly and privately funded program to plant one million trees in New York City from 2007 to 2015. If a researcher wanted to use machine learning to understand why certain trees are planted in certain parts of New York City, that researcher would need to consider a number of factors, from different boroughs’ tree-planting budgets, to soil conditions, to tree prices and availability, to the resources available in different neighborhoods to water and care for the trees, to the specific cost-benefit analyses of planting in certain postal codes. And then other questions come up: How many trees existed before?
Million Trees NYC could hypothetically use AI to discover which tree types were successful, meaning which trees grew and flourished, and where they grew and flourished, to help determine optimal conditions for tree planting. Even posing this question raises more questions. Can we assume that wealthier neighborhoods get more trees? If not, do trees survive at the same rate across New York City? Across New York State? Exploring this data just to find a problem to solve requires asking many initial questions. A researcher would need the additional expertise of understanding all of the factors listed above and more to find the right dataset and build the right model. The deeper data story isn’t just that rent is high in the New York City borough of Manhattan versus Queens. It’s that even planting trees is complicated by policy, history, and other systems. Using any robust system like AI to unpack large datasets can help call out systemic inequity in cities, but the datasets themselves must also be analyzed. Intention is important.
Data is complex. What is needed to make the data more whole?
A dataset is so much more than the initial spreadsheet you have. What are the factors that contributed to that dataset? Those factors not captured in your initial dataset are the factors that make it more whole. A first step when working with data is to outline all of the related data pieces, similar to a recipe. Bread isn’t just flour and water; it’s flour that is sifted, to which a certain amount of water is then added. With my trees example, I picked the data apart: Where are trees in New York City currently? Historically? What kinds? More important, how do we know what makes a tree healthy?
Bullen stresses that we ask specific questions, such as what makes a tree survive? Can we extrapolate that from data? And what makes the money spent on that tree useful? That is, if planting a tree costs X amount of money, what about really growing and caring for that tree? Were the tree box filters big enough for the root systems? What were the weather conditions, and who is responsible for caring for the tree?
All of those factors—weather, tree box filter size, kind of tree, history of the neighborhood, environmental history—are related to the Million Trees dataset, even if those data points aren’t captured in that one dataset. All of these types of factors are not just questions, but also data points that need to be interrogated while collecting and analyzing data.
Don’t just identify, but question and audit patterns in the data
When a pattern emerges from a dataset, don’t take it at face value. Ask: Why is this occurring? Is it correct? Does it feel right? What would happen if this was wrong? What would the real-world outcomes be? Would it cause any person real-world harm? Whatever question you’re analyzing, try asking it from the opposite point of view. For example, when analyzing what makes a tree healthy and looking at neighborhoods that have healthy trees, look into what kinds of trees you’re looking at, and ask: What is the history of those neighborhoods? Now try analyzing unhealthy trees and look for whether this refers to the same tree types. What season were they planted in? Historically, what trees were in that neighborhood? You might find how a city has changed over many decades. For example, a highway that cuts across a neighborhood or borough can have downstream affects of radically changing that neighborhood. A highway adds noise and air pollution, which in turn can cause lower property values. This kind of historical analysis needs to be taken into account with city data, and in this case, tree data. Data has so many other kinds of factors that can affect outcomes.
What are the parallel and contextual data related to this dataset?
From the dataset, can you tell if trees that are dying are dying in neighborhoods that have been systematically underserved? And how do you put that into context? Think about parallel context data: What else has happened in these spaces in the past 20 years? 30 years? Bullen points out that a lot of the datasets for civic technology may be only 10 years old, and that there may not be a lot of available historical data. The patterns coming out of your dataset are made up of so many nuances, with historical roots in policies such as gentrification, segregation, and redlining, which may be reflected in technology. A city is complex, so a dataset, even about trees, will have all of the complexities and biases of a city (such as redlining or segregation) built into it.
Expanding the idea of bad data
Bad data isn’t necessarily harmful data, or data that seems adversarial at first glance. It’s data that is incomplete. Examples of this can appear when analyzing the dataset about trees and how well trees grew in different parts of New York. There are a lot of factors that affect tree growth that aren’t in that dataset. This kind of fuller reasoning makes the Million Trees dataset incomplete and it needs to be viewed as such.
Bringing intersectionality to data
Designing for cities, without AI, is already a nuanced, thorny, large, and, at times, difficult problem. It is a wicked design problem, bound by legislation, bureaucracy, architecture, and then technology, as cities themselves and those who work in civic technology update to keep up with changing technology. Technology affects cities across the board, from the architecture, hardware, and design of cities that are becoming “smarter” cities, to the software and processes of civil servants updating cities with new kinds of technology. Adding AI into the mix is not easy and shouldn’t be engaged with as a kind of pan-techno-solutionism, even when looking at something as seemingly benign as tree-planting data. As it’s been outlined in this article and in other published examples, AI and technology writ large amplifies bias and injustice, regardless of how complete or incomplete a dataset is. But AI could be used effectively if we unpack how deep technology solutions inside of cities work, and where AI would fit into the mix on top of preexisting technology. Technology alone doesn’t solve problems and can create unforeseen problems, as we see specifically with 311; those issues must also be scrutinized. What is the line between calling 311 to fix a problem and the solution to that problem being to deploy a police car? Are users aware of what the response will be when placing a 311 call? This kind of response needs to be unpacked, examined, and then fixed. This similar kind of interrogation must be applied to AI as well. Cities have a diversity of inhabitants, and that means a diversity of responses to problems. So how technology responds to, sorts, and understands those issues and then provides solutions must always be analyzed as potential design flaws, as well as policy flaws when the solutions result in unintended harm. Moving forward, all technology, including AI, must take an intersectional perspective, especially in cities, where historical racism and injustice were a part of the status quo, and the legacy of that injustice is still affecting data and design in cities around us today.
Posted in:
on Mon, June 01, 2020 - 10:16:42
Caroline Sinders
Caroline Sinders is a machine learning designer/user researcher, artist, and digital anthropologist examining the intersections of natural language processing, artificial intelligence, abuse, online harassment, and politics in digital conversational spaces. She is the founder of Convocation Design + Research and has worked with organizations such as Amnesty International, Intel, IBM Watson, and the Wikimedia Foundation.
[email protected]
View All Caroline Sinders's Posts
Is remote the new normal? Reflections on Covid-19, technology, and humankind
Authors:
Yvonne Rogers
Posted: Thu, May 28, 2020 - 1:46:39
![](/images/uploads/blogs/temp_file_sociald_web1.jpg)
Covid-19 forced governments to urge full or partial lockdown measures to slow the progression of the pandemic. By the end of March, more than 100 countries had “locked down” billions of people. During that time, Yvonne Rogers wrote a series of blog posts on the topic of “remote,” structured around the themes of living, working, numbers, and tracking (the full articles and more posts are available on her website: https://www.interactiveingredients.com). She asks: Is remote the new normal? As we contemplate when we will all meet again face-to-face, Rogers helps us reflect on what remote means now for living and working, while also considering fresh ideas on how we plan to slow the pandemic with technology and save lives.
Remote working: March 19, 2020
Since March 12, 2020, we have been working remotely, as the university instructed us to do because of the escalation of coronavirus. It feels like I have had more videoconferencing meetings than hot dinners! One moment it is Skype, the next Teams, then Zoom—many have been back-to-back. Even though it is great that we can keep in touch in this virtual way, it is frankly exhausting—but in a different way from a usual tiring day at work. While my twice-daily train commute takes a toll, being glued to a screen for hours on end, talking to virtual colleagues and students elevates fatigue to a new dimension. The exhaustion is less physical than it is enervating, like after a long Sunday of too much binge watching.
This phenomenon has since been dubbed Zoom fatigue. Part of the new tiredness stems from meetings that differ greatly from the usual—dealing with so many updates each day on what has been planned, decided, revealed, or mandated by government, university, or university department. Right now, an awful lot of “cascading” is interspersed with checking up on and reassuring each other. There seems to be much less actual work, but one hopes this will shift once routines begin to settle into place.
Then, out of the blue, you might get an email from one of your colleagues letting you know they are not feeling well and have begun self-isolating. It is quite anxiety-inducing, worrying if they have contracted Covid-19. I have heard now from quite a few people that they have developed flu-like symptoms and are self-isolating; some situations seem more serious than others. It is all a bit discombobulating, like Russian roulette. You can but hope they will be better the next day.
Today it was raining, so we put our umbrellas up, making it easier for us to social distance. At one point I entered a shop, and while waiting in line, other customers came in and stood two meters apart from us and each other, abiding by government guidance. It felt a little strange and silly as we carefully navigated the small place. But even though it seemed unnatural, it felt prudent.
Later in the morning, when I peered out of my study window that looks onto a primary school playground, I saw many 5- and 6-year-old children playing together during their break time without a care in the world (as it happens, it was probably the last time for a while; U.K. schools closed soon after). For them, the concept of social distancing must seem alien. As for taking part in social isolation, it must seem even stranger: Why can’t we go out to play? Why must we stay indoors without physical contact with our grandparents or playmates?
This new world order brings out the best and worst in people. There are so many acts of kindness being reported that it makes you feel warm and fuzzy, realizing that, as human beings, we like to look out for each other. However, the flipside is just how many people are looking out for themselves. Many can’t resist the temptation to stock up on tins, bread, toilet paper, and other staples. Panic buying maybe, but it gives them something to do and makes them feel safe. I found myself today, after failing to find any cereal left on the shelves, wondering if I should buy the last remaining cereal bar that I spotted. I would never normally buy such a thing, let alone eat it. But self-restraint is tough when irrational fears enter our psyche. As it turned out, one of my neighbors came to the rescue and brought around a tasty loaf of bread.
We have all started to sign off on emails with “Stay safe.”
![](/images/uploads/blogs/temp_file_bekind_web1.jpg)
Remote living: March 25, 2020
So the official lockdown kicked in on Monday, March 23, 2020, for those of us living in the U.K. The rules are a bit more lenient than in some countries, where they have draconian curfew measures in place. Here, we are allowed to leave home for one exercise session a day, and to go shopping to buy necessary food and medicine. We are also allowed to go to work if we absolutely can’t work from home. So for the time being, if you are a construction worker, you can carry on as normal—as long as you social distance. It seems like a good time to be a crane operator, high in the sky looking down on empty streets. I suspect not for long. It must be very difficult for a government to balance its country’s economic needs against how best to flatten the pandemic curve—all while determining how to change human behavior into something so very different from how people normally live their lives.
Yesterday I did my beach walk alone at 8 a.m. to start the day. There were many joggers and dog walkers. There was also plenty of space, so we managed to keep our distance. The sea looked calm and serene, and for that hour I could think of something other than coronavirus. Today I am saving up my permitted outside exercise for later in the day. By midafternoon yesterday, I was getting quite restless. At one point I looked at my watch thinking it was 4:15 p.m.—nearly time for a planned chat with a friend—only to discover it was actually 3:15. Normally, being surprised by finding I have an extra hour would be a joy. I can easily fill it in by catching up on work. This time, I can honestly say my heart sank a little, reminding me of when I was a teenager on a long Sunday when the clock stood still…
Meanwhile, all around us is a flurry of activity online. I see that lovely Amanda is streaming her yoga class to us at UCL on Friday at work. A couple just got married in Birmingham, before the country banned weddings; over 100 guests watched it being livestreamed on Facebook. There are also sweet videos of grandchildren now doing the rounds, waving at their grandparents through the windows or patio doors in their garden, some even squashing their little noses up to the glass. Lots of people have celebrated their birthdays with their friends and family by holding their birthday cakes up to the camera for others to see.
Eating alone together online has also started to become popular again among families and friends. I remember a few years ago, when Skype was becoming mainstream, some of us tried it as an experiment. For example, when I was in South Africa on sabbatical, I had a Skype dinner with a friend back in the U.K. It was nice to catch up with her while doing something, but the eating part actually felt quite odd. We had both served ourselves something simple—a pasta dish—and started eating at about the same time. But somehow picking up our knives and forks together did not synchronize, and the eating of the meal did not feel natural. The smells, tastes, and noises of eating together were lost in translation.
At the end of a tiring day of back-to-back remote work meetings, I now look forward to a FaceTime chat with a friend or two, glass of wine in hand. Of course, it is no substitute for the real thing, but it can be surprisingly relaxing and enjoyable. We make sure we have a good laugh, crack some jokes, and try to see the funny side of life. And then it’s dinner, Netflix, and the 10 o’clock news before going to bed.
Another Groundhog Day in these strange times.
Remote numbers: April 9, 2020
The day before the coronavirus lockdown started in the U.K., I had a smart meter fitted in my house. After the engineer finished, he walked me through all the various functions shown on the digital display. A dashboard of numbers provides all sorts of stats and data about how much electricity and gas you are using and how much they cost per hour, alongside an easy-to-read traffic-light barometer that moves into red bars if you are using a lot of energy (e.g., when boiling a kettle) while rewarding you with green bars when you are being energy efficient. The idea is that you use the various numbers and bars to change your behavior, and in doing so, reduce your energy usage and save money.
I looked at the display a few times but did nothing to change my own behavior. Quite the opposite, in fact. I started using more electricity and gas, making more cups of tea, cooking more meals, spending more hours in front of my laptop and TV, and doing more washing—all a result of being stuck at home 24/7. The best place for the display? Hidden in a drawer.
Meanwhile, like everyone else, I have been gripped by the numbers that come out each day about coronavirus—uncomfortably so. The tally of new cases and new deaths rises daily. At first, two or three people dying was considered shocking. Now we are up to nearly 1,000 a day in the U.K. It is no longer shocking but expected. We have all become engrossed by the graphs that the scientists generate to help the layperson understand what the numbers mean with respect to where we are in the quest to flatten the curve. They project how steep the curve is each day relative to day zero. The color-coded ones show where the U.K. is relative to other countries we might care about. I catch myself comparing how we are doing against the U.S. or Italy—thinking we are better off or not doing as bad. Why are we being shown this, as if it was a competition? To make us feel better? Comparative graphs are a mechanism commonly used in behavioral change, known as social norms. By seeing how well you are doing relative to others (e.g., peers, other families, neighboring cities or countries), you can relax if you are below the others or worry if you are above—there is a loud and clear indication of whether you are using more or spending more (if it is exercise, the reverse is true).
More and more of these visualizations are appearing, including Sky’s “Coronavirus: How many people have died in your area? Covid-19 deaths in England mapped.” Residents of remote areas like Suffolk can let out a big sigh of relief that there are no big blobs nearby. Those who live in London or other densely populated areas, on the other hand, will notice big blobs splatted over their home turf. No wonder so many Londoners flocked to the countryside when they could—that is, before those who live there full time told them where to go.
For the most part, there is little we can do other than worry when looking at these comparative coronavirus graphs. They are fodder, too, for the media and politicians. For example, this headline: “Singapore Wins Praise For Its COVID-19 Strategy. The U.S. Does Not.” A CNN headline was more in tune with the way science happens, through competing predictions and hypotheses: “New U.S. Model Predicts Much Higher Covid-19 Death Toll in UK. But British Scientists Are Skeptical.” The U.S. team predicts that nearly 70,000 will die in the U.K. The British scientists, on the other hand, predicted only 20,000 to 30,000 would die in the U.K., based on their brand of mathematical modeling. Who do we believe?
It goes without saying that mathematical models need lots of data in order to make accurate predictions. When predicting the weather, a tsunami, or an earthquake, millions of data points are used. The current pandemic, however, in comparison has relatively few data points that can be used. It would be hubris not to remember the failure of Google’s Flu Trends program a few years back, when its developers claimed, based on analyzing people’s search terms for flu, that they could produce accurate estimates of flu prevalence two weeks earlier than official data. Sadly, it failed to do this for the peak of the 2013 flu season. Then, they had access to big data—masses of it. The current modelers only have access to small data—very little of it. Let’s hope all the lockdown restrictions that have been put in place in nearly every country, based on current predictions and remote numbers, fares better.
We can but hope.
Remote tracking: April 13, 2020
It is great to see tech companies coming together to help curb the coronavirus. Apple and Google have been collaborating on a platform that could help governments worldwide monitor, track, and manage the pandemic more effectively. Their proposed system works by using Bluetooth and encryption keys, enabling data collection from phones that have been in close proximity with each other. From this data, it can be inferred who else phone owners have been close to for a set period of time (e.g., the quarantine period of 14 days). Users can also alert health authorities if they have been diagnosed with Covid-19; conversely, the system can text users if they detect that their phone, and indirectly themselves, have been in close contact with someone who has been diagnosed with the virus. The term coined for this new form of remote tracking is contact tracing, as illustrated by Apple and Google’s graphic (Figure 1).
![Contact tracing.jpg](/images/uploads/blogs/contact_tracing.jpg)
Figure 1. Apple and Google’s contact tracing system.
If everyone opted in to the system and carried their phone at all times, it could prove an efficient way of letting people know to self-isolate before they unwittingly spread the virus to others. Epidemiologists would also be able to analyze massively more data and be able to develop more accurate predictions. Governments could be better informed about the efficacy of introducing different policies and restrictions about human movement. It seems to be a win-win. However, it requires fairly universal buy-in to the philosophy and the practice as the best way to stop the global spread of the virus. There may be some resistance when it comes to privacy concerns. But such worries need to be weighed against the potential gains of having a pervasive tracking system in place whose sole objective is for the greater public good.
One way to address these concerns is to reassure the public. Much thought has gone into how to avoid unnecessary data collection; Google and Apple’s proposed method of contact tracing is limited in what it tracks and how the data it collects is stored. Compared with GPS that tracks people’s physical location, their proposed use of Bluetooth technology is to pick up signals of only those mobile phones that are nearby, sampled every five minutes. Hence, the data collected won’t know that you were on a bus or in the supermarket at a certain time. It will know only that you were close to a person who has just been diagnosed with Covid-19. This is an important point to be really clear about—as to how much of what someone is doing is actually being tracked. It also helps to address privacy concerns if the data being collected is encrypted.
To enable such a tracking system to have widespread uptake, governments can either be authoritarian and imposing (as is the case in several countries in Asia) or democratic and encouraging—through educating, persuading, incentivizing, and nudging people to opt in. However, this takes time, during which dissenting voices in the press and on social media, together with conspiracy theorists, may create a groundswell of worry. To overcome scaremongering and anxiety requires open debate about what is acceptable and what is not, and how this can change over time and in different cultures and circumstances. Consider CCTV: It is now widely accepted in many countries as a technological deterrent against crime, yet when it first became mainstream in some countries like the U.K. and Germany, many people were up in arms, not least the Snoopers Charter. Since then, however, public opinion has changed. Police authorities found the cameras very useful in helping in their investigations and through acting as a deterrent. Nowadays, cameras of every shape and size have become the order of the day, from webcams worn by frontline workers to massive multiplex CCTV security setups in shopping malls.
Part of my research agenda is to investigate public opinion and sentiment about “creepy data.” We carry out studies to see which technologies people find acceptable and which make them fell uncomfortable, compromised, or threatened. In the early days of mobile phones, I worked on a project called Primma (https://www.open.ac.uk/blogs/primma/) that investigated how to enable people to manage the privacy of their own mobile devices within a framework of acceptable policies. One of our user studies, called Contravision, explored public reactions to a fictitious future technology called DietMon. The proposed tech enabled people who seriously needed to lose weight to track their calorie consumption by providing them with information on their phones about the amount of calories in the food they were contemplating eating. A chip was also embedded in their arm that sent data about their physiological states to their GP. Participants were shown either negative or positive videos of how people managed their everyday lives when using such monitoring tech. Their reactions were mixed. Some people were grossed out; others saw the potential benefits of the system. Importantly, it resulted in an open debate where a diversity of different perspectives was explored—in sharp contrast with the scaremongering that the media often presents to the public. In the end, many different opinions and concerns were voiced.
In another study we conducted (see https://quantifiedtoilets.com/), which investigated concerns over the use of tracking in public, one person said, “Privacy is important. But I would like to know if I was sick and this is a good way to do it.” This sentiment is at the heart of the current contact-tracing dilemma.
Closing words
My next blog is called “Remote nurturing.” I extol the virtues of all the latest crazes that promote being social and feeling human—pub quizzes, making bread, street concerts, growing vegetables. Now that some of the lockdown restrictions are beginning to ease throughout the world, we can begin to establish a new normal, helping each other out while we gradually discover what it means to be together again—albeit at an indefinite social distance.
Acknowledgments
Many thanks to Johannes Schöning for editing the blog entries for this article.
Posted in:
Covid-19 on Thu, May 28, 2020 - 1:46:39
Yvonne Rogers
Yvonne Rogers is the director of the Interaction Centre at UCL (UCLIC) and a deputy head of the computer science department. She is interested in how technology transforms what it means to be human.
[email protected]
View All Yvonne Rogers's Posts
After the iron horse: Covid-19 responses in education
Authors:
Jonathan Grudin
Posted: Wed, May 27, 2020 - 10:25:09
Research from the distant past regains relevance! From 1998 to 2000 and 2017 to 2020, I focused on higher education. Suddenly, some of the early work is again relevant. I’ll describe it here, along with some thoughts arising from conversations with educators, administrators, and my daughters.
Forced by Covid-19 into functioning remotely, the education field quite sensibly seized upon familiar and reliable “digital iron horse” substitutions. Zoom was widely adopted for class meetings in primary and secondary schools that had access to technology, and more broadly for university lectures and final exams in TA-monitored breakout rooms. The limitations of the webcam wave, however, soon steered attention to asynchronous approaches. Mark Guzdial reviewed their uses for remote learning but said that he hopes not to need them again [1].
Twenty years ago, a group I was part of at Microsoft Research deployed streaming-media prototypes to explore real-time and asynchronous remote education. Students and faculty had little time to prepare. Our systems were used in lecture courses at the University of Washington and MIT, and in multisession internal Microsoft training courses, with the results published in CHI, CSCW, and beyond. This was pre-wireless, running on PCs less powerful than your phone. The human side, though, has not changed as much.
Remote live lectures and audience feedback
Studio audiences were devised for early radio comedies and dramas because performers feed off audience energy (studio audiences were also used to record laugh tracks). In March, late-night television hosts had to adjust to the disappearance of live studio audiences. Classroom instructors faced the same challenge.
In “Evolving Use of a System for Education at a Distance” [2], my colleagues and I describe Flatland, a system to explore numerous tools for student feedback. It included video of the presenter, slides including interactive forms, student questions that they could vote to prioritize, a chat panel, Too Fast/Too Slow and Clear/Confusing buttons, hand-raising functionality, and a list of attendees (Figure 1). We reported on feature use and non-use, but high-level observations are more significant today.
![](/images/uploads/blogs/temp_file_Figure1_web2.jpg)
Figure 1. Flatland, presenter view. A reminder of what interfaces looked like in 1999!
Here are some key findings:
- Teaching style matters. The most effective classroom lecturers can have difficulty online. One great instructor’s classroom approach was to race through material and monitor students closely to see where to slow down. Unfortunately, students did not use the Too Fast button. With no feedback telling him when to brake, he finished lectures in half the allotted time. He knew that few students had kept up but could not adjust. Less extroverted lecturers who prepare methodically can fare better.
- Video is great for connecting but not for feedback. Flatland did not have student webcams. Today, seeing everyone at the beginning of a class can be wonderful, although once class starts, video must be managed carefully to avoid distraction. Surprisingly, video is not good for passive feedback. In a separate controlled study, four-person groups carried out engaging tasks either face to face or connected by high-resolution video. Task performance was similar, but when connected by video, people’s faces were far less expressive. Deadpan images won’t help instructors gauge student reactions.
- Establish social conventions for student feedback and interaction. Feedback tools have a learning curve to reach consistent use. For example, an instructor who uses the chat channel to greet early class arrivals creates an often unmet expectation that chat will be watched during the lecture. Will a hand icon be used in voting initiated by the instructor, or initiated by students to signal a desire to comment or speak? Can students verbally jump in? Appropriate social conventions must be designed and communicated.
Recorded lectures and flipped classes
We also built asynchronous video systems to support a “flipped classroom” approach, in which students watch a lecture before the class meets and spend class time discussing or working with the material presented. We went further, enabling students to interact before the class (Figure 2). A student watching a prerecorded lecture and slide presentation could contribute to a time-indexed discussion: Questions and comments by previous viewers scroll by in sync with the lecture. Versions of the Microsoft Research Annotation System were used at MIT and the University of Washington, and in internal training courses [3].
![](/images/uploads/blogs/temp_file_Figure2_web1.jpg)
Figure 2. Viewers could access the table of contents, discussion, or private notes (lower left). The topic of the comment by a previous viewer that is closest to the current point in the video replay is highlighted, with the full comment text below the slide.
Some key findings from this project:
- On-demand viewing with pre-class discussion created work for faculty. Instructors had to record lectures without an audience (though today some instructors reuse last year’s lectures without raising eyebrows). When the annotation system succeeded in fostering extensive discussion, faculty had to read it and could struggle to use the subsequent class period effectively: The discussion was over. One professor complained that to use it again, he would have to design different uses for class time.
Students generally liked having discussion and personal notes indexed to the video. We also devised advanced approaches for varying playback speed (faster or slower) while maintaining comprehensibility. Faster playback could increase learning by focusing attention. Other advantages: Students can choose when to watch a lecture, and with remote classes they save commuting and walking time.
- Procrastination. Prerecorded lectures benefit students who watch the video but adversely impact those who do not and thus can’t follow the class discussion. This effect was amplified by our Discussion feature, which also penalized the many who watched the lecture at the last hour, leaving no time to participate in the online discussion. To address this, we modified the system to support the insertion of group tasks [4] and exercises/quizzes [5] during playback, which could be assigned for completion days before the class. Most students preferred questions that popped up unannounced during the lecture.
- One size does not fit all. A crucial discovery was that each instructor wanted custom features based on their course content and use. Our out-of-the-box system was not flexible enough. An MIT film professor needed two video windows. We were asked to create an annotation playlist capability to group comments together. Terminology was tripped over: After a programming language course had minimal Discussion activity, a student explained, “I had questions, but I didn’t want to discuss C.”
- Shorter lectures? Long lectures are motivated by the cost of converging an instructor and students in a room, but long lectures are usually less appealing and effective than short lectures. Remote education makes the breaking of long lectures into segments more practical.
Next steps
Here I described video-based systems that most students (but few faculty) said they would sign up for again. Having to respond to Covid-19 is terrible, but it is also an opportunity to reflect on pedagogy and technology. From chalkboard to overhead projector to projected slides—what’s next? A visionary lecture system [6] built by David Kellermann at the University of New South Wales creates a learning community in large courses that include remote students. This crisis is also forcing a close look at assessment. Can we assure integrity in high-stakes remote exams without intrusive video systems? Should we move more rapidly to other means of achieving and measuring learning outcomes?
Education is more than learning. Engagement and motivation, key concerns today, are socially constructed. Whether motivated by competition or collaboration with peers, by discussions between classes, or by a smile from a teacher, students lose such incentives and drivers when the school bubble is replaced by the bubble of home. Seeing my daughters’ athletic awards and graduation ceremonies via video rather than a banquet or assembly is a reminder of the creative challenge in designing effective technology to supplement the natural real-time and face-to-face interactions that people have relied on for millions of years.
Endnotes
1. Guzdial, M. How I’m lecturing during emergency remote teaching. Blog post, Apr. 6, 2020.
2. White, S.A., Gupta. A., Grudin, J., Chesley, H., Kimberly, G., and Sanocki, E. Evolving use of a system for education at a distance. Proc. of HICSS 2000. https://doi.org/10.1109/HICSS.2000.926729
3. Bargeron, D. and Grudin, J. As users grow more savvy: Experiences with a multimedia annotation tool. Proc. of HICSS 2004. https://doi.org/10.1109/HICSS.2004.1265260
4. LeeTiernan, S. and Grudin, J. Fostering engagement in asynchronous learning through collaborative multimedia annotation. Proc. INTERACT 2001, 472–479. http://www.jonathangrudin.com/wp-content/uploads/2017/03/MRASengagement.pdf
5. LeeTiernan, S. and Grudin, J. Supporting engagement in asynchronous education. CHI '03 Extended Abstracts on Human Factors in Computing Systems. 2003, 888–889. https://doi.org/10.1145/765891.766051
6. Kellermann, D. Transforming the learning experience (video). Campus Connection Summit 2020.
Posted in:
Covid-19 on Wed, May 27, 2020 - 10:25:09
Jonathan Grudin
Jonathan Grudin has been active in CHI and CSCW since each was founded. He has written about the history of HCI and challenges inherent in the field’s trajectory, the focus of a course given at CHI 2022. He is a member of the CHI Academy and an ACM Fellow.
[email protected]
View All Jonathan Grudin's Posts
A crash course in online learning
Authors:
David Youngmeyer
Posted: Fri, May 22, 2020 - 5:58:53
As in many countries, when New Zealand went into lockdown in response to the Covid-19 pandemic in late March, tertiary education providers had to suddenly move from face-to-face teaching to online learning. Although providers already offered some online courses and had considered the need to expand online learning, the requirement for an immediate about face while a trimester was in progress was a jolt for both teachers and students alike.
In New Zealand, universities and other tertiary providers responded by halting face-to-face classes for a short period before resuming teaching activities completely online.
The limited response time frame, along with differing levels of expertise in online teaching, varying expectations among learners, and the shared context of a pandemic lockdown created a unique learning environment. This has required adaptation and flexibility from both teachers and students. Participating in online learning during the pandemic has provided a window into our interaction with technology and each other in a unique social setting.
While not without issues [1], the online video platform Zoom has shown itself to be a helpful tool that can effectively re-create multiple educational activities, including lectures, tutorials, and lecturer office hours. While Zoom has been used particularly in the business world pre-pandemic, many of today’s users have been either introduced to it recently or required to use the tool in new ways and in a different context [2]. As such, new social behaviors are being produced.
A tutorial via Zoom, for example, mimics an in-person tutorial by allowing for real-time interaction, along with visual, auditory, and written communication. Physical proximity is missing, but users gain the ability to see all other participants on their screen (or at least their names when their video is turned off or unavailable).
Talking to a classmate while the tutor is addressing the class is mimicked via the private chat function. This is less disruptive when done online but it is a distraction. When added to other distractions, such as managing a mobile phone and dealing with people in the same physical space, it has the potential to be a kind of “phubbing” [3] on steroids. It may be that multitasking while participating in an online tutorial is as disruptive and unwelcome as researchers have found the use or mere presence of mobile phones to be during face-to-face interactions [4,5].
The ability of users to control their video input—when that functionality is available—adds a new dimension to the traditional tutorial, where a student is either definitively present or absent. Online, a student may be able to see and hear other users yet keep their own appearance private. Why would they do this? Possibly for security reasons, but also because of appearance-related issues (no haircuts under lockdown) or because they want to maintain the privacy of the home environment. Some users add a still photo instead of video, while others seek to control their physical background with a virtual background. Turning off the video creates a new social situation with communication issues related to a lack of visual cues. It is akin to a student joining an in-person tutorial by telephone, while being able to see the other participants.
The online tutorial can also support the sharing of work—such as visual arts projects— where feedback can be given by the tutor and other students. This is achieved by a student sharing their screen and the tutor pointing digitally to elements of the work on-screen. This scenario effectively replicates its counterpart in a physical classroom, with a good amount of real-time interactivity.
It is important to acknowledge that online learning may not be best suited for certain types of courses, such as those requiring hands-on activities and access to specialized equipment. Additionally, it may be problematic for those on the other side of the digital divide [6]. The sudden move online in response to the pandemic, however, has allowed a good deal of learning to continue, as opposed to the alternative of closing shop completely. At the same time, new user behaviors and online social situations have been created that are deserving of further study.
Endnotes
1. Doyle, P., Mortensen, J., and Clifford, D. The trouble with Zoom. Australian Financial Review. Mar. 24, 2020.
2. Neate, R. Zoom booms as demand for video-conferencing tech grows. The Guardian. Mar. 31, 2020.
3. Chotpitayasunondh, V. and Douglas, K.M. How “phubbing” becomes the norm: The antecedents and consequences of snubbing via smartphone. Computers in Human Behaviour 63 (2016), 9–18.
4. Kadylak, T., Makki, T.W., Francis, J., Cotton, S.R., Rikard, R.V., and Sah, Y.J. Disrupted copresence: Older adults’ views on mobile phone use during face-to-face interactions. Mobile Media & Communication 6, 3 (2018), 331–349.
5. Przybylski, A.K. and Weinstein, N. Can you connect with me now? How the presence of mobile communication technology influences face-to-face conversation quality. Journal of Social and Personal Relationships 30, 3 (2012), 237–246.
6. Vishkaie, R. Hit by the pandemic, war, and sanctions: Building resilience to face the digital divide in education. ACM Interactions ‘Covid-19 blog’, Apr. 10, 2020.
Posted in:
Covid-19 on Fri, May 22, 2020 - 5:58:53
David Youngmeyer
David Youngmeyer studies design at the University of Waikato. He plans to start Ph.D. research in 2021 in the field of human-computer interaction. He holds an M.A. in communication from the University of Maryland.
[email protected]
View All David Youngmeyer's Posts
Design-led innovation: Lessons from the Scientific Revolution
Authors:
Ron Gabay
Posted: Thu, May 21, 2020 - 2:58:42
In recent years, design has established itself as a strategic business asset that helps organizations innovate and grow. From first being associated with aesthetics, design has become a measure for user centricity and an engine for creating meaningful experiences and improved business performance. An increasing number of organizations recognize the value of design and seek to interweave design capabilities into their operations. With the ongoing market pressures to stay relevant and competitive through innovation, organizations face the multifaceted challenge of integrating business and design, while also experiencing an unprecedented rate of change.
By examining other periods of disruption and innovation in human history, we learn that this kind of challenge is not completely new. The Scientific Revolution, much like today’s market, was a period of disruption that brought with it an unprecedented rate of change. Different disciplines and groups, similar to business and design professionals today, started collaborating on topics that at that time were considered distinct and isolated. Throughout the years, research about the Scientific Revolution has produced insights about interdisciplinary work and its role in generating breakthroughs. This article examines these insights in order to provide perspectives on how to interweave business and design in the current disruptive era. In addition, to put these perspectives in context, a critical review of design sprints is offered.
The business value of design
As organizations become data driven and seek to quantify their decision-making processes, collecting data about the business value of design is key. Today, we not only feel that good design is good business, but we also have the data to prove it. This makes design compatible with the modern, data-driven business world. Consultancy firm McKinsey conducted one of the most extensive studies on the financial value of design in 2018, collecting more than 2 million pieces of financial data and recording more than 100,000 design actions [1]. By correlating design actions (e.g., appointing a chief design officer or establishing a user satisfaction matrix) and financial performance (e.g., revenue or shareholder returns), McKinsey found correlations between investments in design and improved financial performance (Figure 1).
![](/images/uploads/blogs/Fig1_web.png)
Figure 1. Source: McKinsey Design Index, The Business Value of Design Report, 2018 [1].
Another study conducted by the Design Management Institute (DMI) tracked the share value of design-centric companies. The DMI monitored investments in design and their impact on companies’ performance relative to the S&P 500 Index. Over a 10-year period, the Design Value Index (DVI) showed that design-centric companies have outperformed the S&P 500 Index by more than 200 percent [2] (Figure 2).
![](/images/uploads/blogs/Fig2_web.png)
Figure 2. Source: DMI Design Value Index, Design Management Institute [2].
The data-driven understanding of the business value of design amplifies the question of how best to integrate business and design. To answer this question, one must first gain a deeper understanding of business and design, in particular their differences.
How business and design are different
The first dimension in the multifaceted challenge of integrating business and design in today’s world is their unique and distinct nature. Table 1 highlights some of the key differences between business and design. By examining various criteria, it is clear that business and design differ in many aspects.
![](/images/uploads/blogs/Table1_v3.jpg)
Table1. Business and design comparison [3].
While the comparison may simplify the complexity and interconnectedness of business and design, its aim is to provide a viewpoint, a viewpoint that emphasizes that business and design are opposing yet complementary in nature. Moreover, the table reveals that business and design are also incommensurable (do not share a common measure). This incommensurability presents itself through different terms, methods, and processes. Thus, any attempt to lead business and design integration must address their unique characteristics and incommensurability. The failure to recognize and properly address the lack of common measures may lead to tension, misunderstanding, and even hostility between business and design professionals. To learn how to deal with incommensurability and prevent it from hindering innovation, it is worthwhile to investigate the Scientific Revolution.
The Scientific Revolution and incommensurability
The Scientific Revolution, which took place during the 16th and 17th centuries, was a disruptive period that generated many breakthroughs. During this period, new knowledge about physics, biology, mathematics, astronomy, and chemistry emerged, transforming how people view the world. In his influential book The Structure of the Scientific Revolutions [4], Thomas Kuhn reviews the history of science. Kuhn, an American philosopher of science, shares his observations about different scientific theories and their relationship to one another. He identifies the presence of incommensurability when he fails to compare and see developments from one scientific theory to another.
The inability to compare or see how one theory relates to the next is rooted in the fact that different scientific groups worked independently and used different terms and methods. The lack of common measures created a situation where groups were neither able to communicate nor exchange valuable information. Moreover, because of this incommensurability, relevant information, and even solutions that could have helped other groups create breakthroughs, were simply overlooked.
Kuhn’s Loss—a significant barrier for design-led innovation
According to Kuhn, incommensurability became a significant barrier for breakthroughs and led to scientific stagnation. The main reason for this stagnation was that different scientific groups were simply not able to communicate. Groups could not understand and benefit from previous knowledge or perspectives due to inconsistency and unfamiliarity with one another’s terms, methods, or contexts. This notion of not being able to funnel and leverage existing knowledge toward a new paradigm is referred to as Kuhn’s Loss.
Business and design professionals, much like different scientific groups during the Scientific Revolution, have their unique terminologies and methods. For example, as the table above shows, business perceives an individual as a customer who is defined by terms such as age, gender, or income. Design, on the other hand, views customers as humans and studies their personality, needs, and desires. Overall, as design-led innovation becomes a new paradigm founded on business and design integration, facing incommensurability is inevitable. Therefore, organizations that wish to maximize the powers of design-led innovation must address the concept of Kuhn’s Loss.
No-Overlap Principle
To avoid or minimize the negative effect of incommensurability, Kuhn developed the No-Overlap Principle. The principle suggests that terms must not have an overlapping meaning or reference, and that when one term is used, it must be explained with its relevant context. For instance, “to learn the term liquid, one must also master the terms solid and gas” [5]. In the world of innovation, a No-Overlap Principle could be valuable for various reasons.
First, today’s world presents complex challenges that require companies to adopt interdisciplinary approaches. For business and design teams to work effectively, they need to be able to communicate and understand each other. For example, exchange value (revenue, costs, and profits) needs to be explained alongside use value (experience, emotions, and desires). Alternatively, customer or market analysis ought to be accompanied by a qualitative evaluation that includes personality types, cultural backgrounds, and traditions. Second, in a world that constantly changes and in which technology is easily accessible, human capital becomes the most significant source of innovation. The No-Overlap Principle supports some of the basic human needs from Maslow’s hierarchy: recognition and respect [6]. In essence, the principle expresses to business and design professionals that they are both essential contributors to innovation and success.
A critical review: Design sprints
With the increasing attention that the business value of design receives, companies seek to interweave design into their operations. One way that design is being introduced to companies is through the design sprint, a five-day framework for solving challenges by applying design tools and methodologies [7]. A design sprint uses design activities such as research, user journey creation, sketching, and prototyping to efficiently generate meaningful concepts. Design sprints are becoming a popular multidisciplinary method for design-driven innovation. Since the goal of a design sprint is to leverage design methods and processes in the business environment, we must ask ourselves whether design sprints satisfy the No-Overlap Principle.
The good. Design sprints introduce design to organizations and non-designers in a bite-size, five-day process. A sprint is usually a controlled environment where design activities and methodologies can be introduced to various stakeholders. The fact that sprints take place in a neutral environment (usually not where daily business meetings take place) creates the opportunity to generate the right physical and psychological setting for learning, experimenting, and engaging with design for the first time. The physical and psychological distance from daily routines can help loosen some of the participants’ deeply ingrained business mindsets and defuse tensions or preexisting hostility.
Design sprints also provide organizations and non-designers a degree of certainty about the design processes, steps, and expected outcomes. The following is an example from the website of Google Ventures (creators of the design sprint):
On Monday, you’ll map out the problem and pick an important place to focus. On Tuesday, you’ll sketch competing solutions on paper. On Wednesday, you’ll make difficult decisions and turn your ideas into a testable hypothesis. On Thursday, you’ll hammer out a high-fidelity prototype. And on Friday, you’ll test it with real live humans [8].
The five-day, step-by-step framework corresponds with how businesses operate and assess processes. Overall, design sprints can be an effective way to introduce design to non-designers because they simplify design in a manner to which non-trained participants can relate. In other words, a design sprint does satisfy, to some extent, the No-Overlap Principle through familiarity with terms and methods.
The bad. Design sprints have a very clear set of rules and milestones that make design relatable and comprehensible to non-designers. However, design sprints also erode some core qualities of professional design, which, in turn, minimize its potential impact. First, design sprints portray design as a linear process. Much like a recipe, a sprint provides the notion that if only the predefined steps are followed, an innovative Michelin-star dish awaits at the end of day five. Design sprints provide participants the false impression that design is a clean, short process when in reality it is often iterative, messy, and long.
Moreover, the design sprint’s gate-like process pushes participants to explore and arrive at meaningful directions within a short, predefined period. This future-oriented exploration is complex and involves many unknown territories where “mistakes” are inevitable. Yet in the search for innovation, mistakes are actually experiments. This notion, coupled with the usual business pressure to deliver, could harm and limit the potential impact and quality of design. Essentially, during the sprint, non-designers are asked to produce quality design work by engaging in activities that are often new and counterintuitive to them. We must question whether participants will truly immerse in exploration and will be able to refrain from the old business habit of associating potential mistakes (experiments) with failure.
Overall, a design sprint may satisfy the No-Overlap Principle by introducing design terms and methodologies to non-designers. Yet it fails to provide the broader context and deep understanding for design. If an organization is not design-centric, it will not be able to develop or scale the outcomes of a design sprint, as exceptional as they might be, in a meaningful manner. Essentially, running design sprints without a border integration of design into business will limit an organization’s ability to innovate because it alters and reduces design to its lowest common denominator.
Conclusion: The future of business and design integration
Design-led innovation has established itself as a catalyst for growth and differentiation. Across the globe, organizations seek to build and scale design capabilities to tackle current problems and to shape the future. To unleash and interweave the powers of business and design as a source for innovation, it is important to recognize the extent to which they differ and that they do not share a common measure. Interdisciplinary frameworks such as design sprints do help in democratizing design by reducing design to a level that non-designers can relate to. Nevertheless, reducing design may actually limit its puzzle-solving powers and long-term integration within the business context. I believe that organizations that wish to become design-centric and reap the fruits of interdisciplinary work must understand that the lack of common measure between business and design does not imply shifting toward finding the lowest common denominator. Instead, organizations should hire and invest in specialized professionals who can continue to perfect their skills while addressing the concept of Kuhn’s Loss.
The key is finding T-shaped individuals—experts in their own domain who are able to collaborate across disciplines or functions. These individuals will help organizations avoid the trap of overspecialization, which leads to the loss of flexibility, openness, and curiosity—all cornerstones of innovation. Ultimately, within the organizational structure, organizations must find the right people and incentivize them to work together because, as C.P. Snow noted in his famous 1959 book The Two Cultures, “The clashing point of two subjects, two disciplines, two cultures—of two galaxies, so far as that goes—ought to produce creative chances. In the history of mental activity, that has been where some of the breakthroughs came” [9].
Endnotes
1. The Business Value of Design - McKinsey Report. 2018.
2. Rae, J. Design value index exemplars outperform the S&P 500 index (again) and a new crop of design leaders emerge. dmi: Review - Design Management and Innovation 27, 4 (2017), 4–11.
3. Gabay, R. Breaking the wall between business and design—Becoming a hedgefox. Design Management Journal 13, 1 (2018), 30–39.
4. Kuhn, T.S. The Structure of Scientific Revolutions (Second Edition, Enlarged ed.). The Univ. of Chicago Press, 1962.
5. Oberheim, E. and Hoyningen-Huene, P. The incommensurability of scientific theories. The Stanford Encyclopedia of Philosophy (Fall 2018 Edition), E.N. Zalta, ed.
6. Maslow, A.H. A theory of human motivation. Psychological Review 50, 4 (1943), 370–396.
7. Knapp, J., Zeratsky, J., and Kowitz, B. Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days. Bantam Press, London, 2016.
8. The Design Sprint - Google Ventures (GV) Official Website.
9. Snow, C.P. The Two Cultures and the Scientific Revolution. Cambridge Univ. Press, 1959.
Posted in:
on Thu, May 21, 2020 - 2:58:42
Ron Gabay
Ron Gabay is the head of innovation and venture design at JoyVentures. He builds and manages the foundation for venture creation by interweaving technology, science, and design. His passion is to enable cross-disciplinary collaborations that foster curiosity, human-centricity, and creativity while applying a commercial lens. He holds a BA in industrial design and an MBA.
[email protected]
View All Ron Gabay's Posts
Evaluating immersive experiences during Covid-19 and beyond
Authors:
Anthony Steed,
Francisco Ortega,
Adam Williams,
Ernst Kruijff,
Wolfgang Stuerzlinger,
Anil Ufuk Batmaz,
Andrea Won,
Evan Suma Rosenberg ,
Adalberto Simeone,
Aleshia Hayes
Posted: Tue, May 19, 2020 - 4:52:13
The Covid-19 pandemic has disrupted life as we once knew it. The safety and well-being of people are paramount, and there is no exception for the human-computer interaction (HCI) field. Most universities and research labs have closed non-critical research labs. With that closure and the student populations having left campus, in-person user studies have been suspended for the foreseeable future. Experiments that involve the usage of specialized technology, such as virtual and augmented reality headsets, create additional challenges. While some head-mounted displays (HMDs) have become more affordable for consumers (e.g., Oculus Quest), there are still multiple constraints for researchers, including the expense of high-end HMDs (e.g., Microsoft Hololens), high-end graphics hardware, and specialized sensors, as well as ethical concerns around reusing equipment that comes in close contact with each participant and may be difficult to sterilize. These difficulties have led the extended reality (XR) community (which includes the virtual reality (VR) and augmented reality (AR) research communities) to ask how we can continue to practically and ethically run experiments under these circumstances. Here, we summarize the status of a community discussion of short-term, medium-term, and long-term measures to deal with the current Covid-19 situation and its potential longer-term impacts. In particular, we outline steps we are taking toward community support of distributed experiments. There are a number of reasons to look at a more distributed model of participant recruitment, including the generalizability of the work and potential access to target-specific, hard-to-reach user groups. We hope that this article will inform the first steps toward addressing the practical and ethical concerns for such studies [1].
There are currently no strong ethical guidelines for designing and running experiments in VR and AR for HCI. Most of the VR and AR studies in HCI are conducted in research institutions, where researchers must follow local laws and the directions of the local institution’s ethics board. VR and AR systems allow researchers to control the virtual environment and collect detailed user data in ways that might not be familiar to participants, so careful consideration of participant privacy is especially important. Further, some experiments might require direct supervision through an experimenter while the user interacts with the virtual environment, for example, to watch for behaviors that circumvent the objectives of the experiment. The rules and laws for remote data collection and direct supervision of experiments, which can vary between different countries and regions, becomes an issue.
Short-term solution: Use lab personnel and infrastructure
The most immediate solution to performing remote experiments is to collaborate between labs to provide participants for each other’s experiments. The subjects are likely to be lab members, or people associated with the labs in some manner, who have the correct equipment at hand.
A well-known concern for most work with human subjects is the issue of working with populations of convenience. This problem can be particularly acute in this case. Groups of lab members may have too much knowledge about the field to react “naturally.” They may guess the experimenter’s aims and intentionally or unintentionally behave in accordance with or in opposition to them. They may also have strong existing opinions about interaction or visualization techniques, which can bias the outcomes. Finally, their experience with XR—either AR or VR—may make it difficult to generalize their data to the general population. Specifically, their expertise in the use of these platforms can be a confound to the outcomes of usability testing new tools and experiences.
However, there are also circumstances in which distributed studies across labs could be better than the usual population of convenience. Rather than a mix of participants who are familiar with XR and new to XR, a population of lab members would all be familiar with the equipment. This might be more generalizable to populations who might actually engage with the research. Assuming that lab members are in general less susceptible to simulator sickness (e.g., through self-selection), it could also reduce the risk of losing participants to this affliction. In short, it’s important to carefully consider which tasks can be best run with experienced participants.
Medium-term solution: Recruit external users who have the necessary hardware
In order to develop a more sustainable participant pool, a more organized effort is needed to start recruiting outside of the research labs. This phase is still limited to participants who have the equipment required for a given user study. However, given that six million people currently own a VR headset, there is clearly the potential to reach out to these individuals. Unfortunately there are no easy-to-use tools to run VR experiments online, and there are various technical issues with implementing and distributing experiments to consumer devices. A few early works, though, have demonstrated the possibility of controlling enough aspects of the design to produce usable results (e.g., [2,3]).
An initial step would be a website that allowed participants to register for recruitment and research labs to advertise their experiments. This system could use crowdsourcing websites to recruit participants, who would then be redirected to the site. This in itself brings many challenges. Simultaneous efforts by different regions (e.g., the EU, the U.K., the U.S., Canada, Japan, and Brazil) would be required to grow the effort by collaborating and seeking regional funds. For the site to be successful, different regional needs would need to be considered. For example, a study approved by an ethics board in the U.S. might not be acceptable to a panel in another country. This involves not only ethics but also local laws, such as the European GDPR.
Long-term solutions: Generate pools of users through funded hardware distribution
While the medium-term solution will improve the way we do remote distributed experiments, one suggestion for the long term is to provide equipment to a pool of subjects. Based on the experiences learned in the previous phase, this solution will continue to improve the tools and methods created, but it needs to identify ways of finding participants who can be lent equipment, in the hope that they then use it to participate in multiple experiments. This in itself is an expensive goal, but we believe it is possible because the equipment might become cheaper. Some governmental scientific funding bodies (e.g., the National Science Foundation in the U.S.) could provide funds to acquire the required infrastructure to expand the pool. This would offer an opportunity beyond Covid-19. First, it would allow us to have expert users test a VR application while also having access to naive users when needed. It would also enable us to validate research results with different subject pools from multiple regions, and would remove the need to bring participants into the lab—unless this were required due to specialized hardware needs or other restrictions imposed by an experiment’s design.
In other words, we could seed a community of participants with specialized technologies to allow for a more diverse subject pool. This would ideally look like the distribution of HMDs or similar technologies to volunteers around the world, who in exchange would agree to participate in experiments. These seeded participants would be registered through a citizen science crowdsourcing site. The benefit of having these seeded users would be a new level of diversity among participants. The distributed HMDs will be new technologies (at least for some time) and these participants would represent novice users. The research lab would still be able to provide incentives (e.g., additional compensation) to maintain interest among these users. This style of crowdsourcing has some precedents. Some people make a portion of their income from crowdsourcing sites such as Amazon Mechanical Turk and Prolific or by participating in paid medical trials. In some cases, these participants are not ideal test subjects, because they may be very experienced with common experimental tasks or situations (e.g., overfamiliarity with the “trolley problem” in psychological experiments). However, having an official pool of representative, well-compensated participants could also address issues of undercompensation and unrepresentative samples.
Ethical considerations
While ethical (and legal) considerations may vary depending on the country, review board, and institution, the following are some points to consider:
- Pooling students to run each other’s experiments. While this seems to be an attractive idea, there is the problem that faculty members might induce students to take part in the pool. Thus, it is essential to have strong requirements about there not being an inducement to take part; for example, it cannot affect grades, funding, or progress toward degrees. A possible solution is to add a “non-inducement” clause while checking that it is enforced.
- Desktop sharing. If desktop screen sharing is used (e.g., to make it easier for the experimenter to control the remote apparatus), this poses potential ethical risks for the participants. For example, the experimenter may then see personal notifications. One potential solution is to use only window sharing for the VR application, if this is viable. Still, it is vital that these risks are assessed and correctly managed by the experimenter and are mentioned in the ethics review process.
- Running studies on social VR platforms. This poses several data-processing problems, as the confidentiality and security of emerging platforms is not assured. The platforms may require personal information to sign up, and as users we can’t be sure what data is being collected. While these risks may be alleviated through careful design (e.g., recruiting participants within the platform), they pose new concerns compared to, say, collecting data on social media platforms.
- To that end, open platforms do exist and are gaining ground. Platforms that can be hosted on a server secured by the operator, such as Mozilla Hubs, or custom-made solutions solve many data-protection problems. We expect exemplar or template systems to emerge in the next couple of months.
- While using videoconferencing and screen sharing to assist with remotely operating equipment are attractive, they present new challenges. In particular, they may be hosted or relayed by servers in different countries and may not be secure. This is one area where institutions may have policies driven by contract agreements with existing providers. Screen sharing presents specific problems because of the potential risks of personal-data disclosure. Therefore, it is important to be aware of such limitations before designing an experiment.
Health considerations
Safety issues may constrain some types of experiments. For example, while labs are often wide open spaces, domestic environments used for VR might be small and/or cluttered. While applications can be coded to fall within the “guardian” space that the user configures for the system, this might change. Thus, while games that encourage exaggerated movements are commonplace, we suggest not involving dynamic expansive gestures. Further, we suggest making sure that the experiment operates within a modest amount of space, and if, say, locomotion is important, that this is a key filter in the recruitment of participants.
Another issue, especially if hardware is shared among participants, is “hardware quarantine.” If a headset is used by only a single person, hygiene may be less of an issue. However, once hardware is shared, obviously hygiene has to be taken into account. Hardware should always be cleaned thoroughly between participants, but extra precautions will need to be taken to limit the possibility of, for example, spreading an infection. Using disinfectant wipes and additional masks that the user can wear underneath an HMD can be valuable, but they only offer limited protection. Recently, decontamination systems have become available that make use of ultraviolet light and nano coatings that offer additional benefits, yet they will also not cover every nook and cranny. Current research seems to suggest that contamination on surfaces may cease to exist after 72 hours for Covid-19 [4]. As such, cycling through HMDs that have been put away for some time may be an additional precaution. Currently, a combination of hygienic measures seem most appropriate, and users should always be informed about potential risks (e.g., in informed consent forms). Not doing so seems unethical.
Validity of the results
Running remote experiments makes keeping a uniform apparatus difficult. In a lab study, typically there is only a single apparatus. Yet remote participants might have varying hardware and, if not explicitly requested by the experimenters, even different headsets. Clearly, controlling for the uniformity of setups helps in isolating other factors that could affect the results. If, in order to reach a sufficiently large number of participants, it becomes necessary to relax the conditions for exclusion (e.g., by allowing users with different headsets to participate), it remains an open question how to consider the validity of results obtained in such a manner.
It can be argued that one of the goals of this type of research is to devise novel methods that can then be applied to a wide range of setups, which could then be expected to continue providing comparable performance, as indicated by the empirical results. However, some types of experiments can be too dependent on the specific combination of headset and controllers. Thus, if remote experiments with heterogeneous hardware become an acceptable platform to run experiments, how far of a divergence in hardware can be accepted? For example, an interaction technique designed to be operated with the now-standard trigger button of a specific VR controller might still work the same way with a controller designed by another maker, even though the ergonomics of the device will differ. Promoting the culture of replicating studies might provide the solution to these challenges.
Remote experiment design guidelines
Participants can be recorded completing the experiments over Zoom or any videoconferencing platform. However, these platforms have associated security risks. Another approach is for researchers to observe participants through a videoconferencing platform, without recording. This would also provide more control of the consistency of the procedure between participants. Because recording a participant and their home adds additional concerns for privacy, researchers should weigh the benefits of recording versus observing the participant in real time as they participate in the research study. Labs can still use research assistants to run consistent studies remotely without recording a participant’s home, personal space, and/or unwilling family members in their environment.
Some general remote experiment designs recommendations can be found in [5]. Researchers should remove or minimize as many accessibility barriers as possible. This can be achieved by adding feedback systems such as text, voice, and interface prompts. Researchers should also make sure that the language used is accessible to their intended audience. With the lack of a live audio or video connection, it might not be possible to further explain instructions after an experiment has started. Reminders can help to ensure that participants complete remote experiments. We also suggest that experimenters take cultural and regional differences into consideration. For instance, for a VR/AR driving simulator designed for left-hand driving countries, user performance and experience might vary in right-side driving contexts. Similarly, for VR text-entry studies, authors might consider the many different keyboard layouts around the world.
The required mental workload of the participant should be reduced to a minimum. This can be achieved by removing set-up steps and automating parts of the experiment. Screen or input switching during the experiment should be limited. If possible, all surveys should be displayed within the HMD, or at least in an application that is automatically started on the desktop. Since current text-entry UI solutions in VR are not as efficient as keyboards, experimenters might want to choose drop-down menus, sliders, radio buttons, or even voice-recording options to collect survey data in virtual environments. In a perfect setup, the entire experiment would be run from one executable, with consent, instructions, task, and post surveys all completed while wearing the HMD.
Experimenters should collect data on the device used, the specs of that device, the computer used, and the frame rate at which the experiment ran (ideally not just the mean, but also the standard deviation or other meaningful statistical information). The instructions of the experiment could be recorded in advance as a video or an animation in VR and shown to the participants. This could also include any training or context necessary to complete the experiment.
Before launching the experiment, researchers should solicit feedback from their own (and potentially other) labs. Ideally, experiments should be piloted with participants from outside the research group. Such feedback will help solve any setup challenges or other potential sticking points during the experiment and highlight potential safety concerns. This will also allow for a more accurate prediction of experiment completion times, which can then be communicated to participants. During such pilot studies, any applicable screen-capturing protocol can also be tested.
If the experiment is to be run over a video call, the connection speed for both parties should be tested. This will inform whether it is possible to record the video on the participant’s computer or the experimenter’s computer, as needed.
Any data logged by the application should ideally then be automatically uploaded over the network to avoid using methods that could de-anonymize data, such as asking participants to send log files to the experimenter via email. For this, the system needs to be able to detect whether or not the upload of the data has happened successfully. Should it have failed, the application would indicate where these files are stored and how to upload them anonymously, for example, via a file-upload web form that does not collect any data besides the log file. This, however, might open the potential of “fake data” being uploaded maliciously. Thus, experimenters should consider solutions to verify if the data being uploaded is genuine. Experimenters should also consider different end-to-end encryption methods to protect the participants’ data.
With this new style of remote experimentation, it is also advisable to ask participants questions about their entire experience of the experiment after completion, to allow for continuous improvement. These questions could include inquiring about their overall satisfaction, levels of immersion, the ease of use of the system, and how intuitive or clear the process was. More suggestions for testing for remote experiments can be found in [6].
While VR and AR HMDs are more popular than ever, they are currently not as widely used as TVs, monitors, or smartphones. As a result, most research on such systems is being conducted in research laboratories within specially designed environments. For instance, tracking base stations may need to be positioned according to the purpose of the experiment to achieve the needed accuracy. Windowless laboratories may be necessary to avoid incoming sunlight and increase tracking reliability. Meeting such environmental constraints might not be possible in participants’ homes, so design decisions may be different in distributed experiments. Experimenters have to consider these differences and design their experiments accordingly.
Remote experiment open questions
- What is the best protocol for the transmission of projects and experimental data?
- How can payment be sent to participants?
- How much of a limiting factor is participants’ bandwidth for streaming video and results?
- What are the ethical considerations needed to ensure the privacy of participants’ data?
- What is the trade-off between acceptable quality of a recording versus its size?
- Should we expect that participants have space on their computers to record to?
- Is having experimental results streamed to computers outside of a lab a potential ethics issue?
- What is the best way to monitor for simulator sickness when the experimenter is not present?
Advice to reviewers
Regardless of how this new wave of human-subjects experiments is handled, reviewers must be aware of the changing nature of article submissions. In response to the concerns around user studies during Covid-19, many conferences in HCI and VR/AR have sent out calls for participation, highlighting the appropriateness of contributions from systems, design, methodology, literature review, or other contributions focused less on user studies. It falls upon the program chairs to communicate these new criteria down the line to program committee members and reviewers. This change in mindset will be a collective process across all of HCI and AR/VR. Authors must clearly describe in the submissions the exact way the experiment was administered and also discuss the pool from which the applicants were recruited. For example, if an experiment predominantly uses members of VR/AR labs or enthusiasts, this has the potential to distort the outcomes. This is due to existing biases or perceptions coming from frequent exposure to VR/AR systems, relative to participants who have rarely or never experienced VR/AR. Such a biased participant pool could thus represent a limitation for a study. Yet, given Covid-19, these limitations may not invalidate the work being presented, as long as the authors are clear in the description of the participant pool and the reviewers are encouraged to work with the understanding that this is currently one of the very few options for running VR/AR studies. In other words, transparency of the process is one of the best ways that we can usher in this new wave of publications.
Conclusion
In situations such as the current pandemic, the use of the short-, medium-, and long-term solutions discussed earlier enables the field of HCI and XR to continue to forge forward with experimental work. A secondary benefit of the use of members of other labs in the community is that it increases the amount of transparency in the field by making people more aware of the exact nature of each other’s experiments. It could potentially improve the external validity of experiments by increasing the diversity of platforms and participants used for a given task. At the vary least, Covid-19 has strengthened this community and inspired new collaborations between researchers. While there are both ethical and practical concerns for distributed user studies, solutions for XR will likely be useful for other areas of HCI, and, indeed, any field that relies on human experimentation. This article provides a starting point. We hope other articles will follow with more specific information, either expanding topics presented here or offering new ideas.
Endnotes
1. We invite people to join the discussion. The current community is formed of researchers from the IEEE VR community, specifically a discussion launched at the online conference in March 2020. Email [email protected] to be added to the discussion.
2. Steed, A., Friston, S., Lopez, M.M., Drummond, J., Pan, Y., and Swapp, D. An ‘in the wild’ experiment on presence and embodiment using consumer virtual reality equipment. IEEE Transactions on Visualization and Computer Traphics 22, 4 (2016), 1406–1414.
3. Ma, X., Cackett, M., Park, L., Chien, E., and Naaman, M. Web-based VR experiments powered by the crowd. Proc. of the 2018 World Wide Web Conference. ACM, 2018, 33–43.
4. van Doremalen, N., Bushmaker, T., Morris, D.H., Holbrook, M.G., Gamble, A., Williamson, B.N., Tamin, A., Harcourt, J.L., Thornburg, N.J., Gerber, S.I., Lloyd-Smith, J.O., de Wit, E., and Munster, V.J. Aerosol and surface stability of SARS-CoV-2 as compared with SARS-CoV-1. New England Journal of Medicine 382, 16 (2020), 1564–1567.
5. Cooper, M. and Ferreira, J.M. Remote laboratories extending access to science and engineering curricular. IEEE Transactions on Learning Technologies 2, 4 (2009), 342–353.
6. Nickerson, J.V., Corter, J.E., Esche, S.K., and Chassapis, C. A model for evaluating the effectiveness of remote engineering laboratories and simulations in education. Computers & Education 49, 3 (2007), 708–725.
Posted in:
Covid-19 on Tue, May 19, 2020 - 4:52:13
Anthony Steed
Anthony Steed is head of the Virtual Environments and Computer Graphics group in the Department of Computer Science at University College London. He has over 25 years’ experience in developing virtual reality and other forms of novel user interface. He received the IEEE VGTC’s 2016 Virtual Reality Technical Achievement Award.
[email protected]
View All Anthony Steed's Posts
Francisco Ortega
Francisco R. Ortega is an assistant professor at Colorado State University and director of the Natural User Interaction Lab (NUILAB). His main research area focuses on improving user interaction in 3D user interfaces by eliciting (hand and full-body) gesture and multimodal interactions, developing techniques for multimodal interaction, and developing interactive multimodal recognition systems. His secondary research aims to discover how to increase interest for CS in non-CS entry-level college students via virtual and augmented reality games.
[email protected]
View All Francisco Ortega's Posts
Adam Williams
Adam Williams is a Ph.D. student in computer science at Colorado State University. His research is on multimodal inputs for augmented reality, specifically, user-elicited gesture and speech interactions. His research goals are to create novice-friendly interactions for 3D learning environments.
[email protected]
View All Adam Williams's Posts
Ernst Kruijff
Ernst Kruijff is a professor of human-computer interaction at the Institute of Visual Computing at Bonn-Rhein-Sieg University of Applied Sciences and adjunct professor at SFU-SIAT in Canada. His research looks at the usage of audio-tactile feedback methods to enhance interaction and perception within the frame of AR view management, VR navigation, and hybrid 2D/3D mobile systems.
[email protected]
View All Ernst Kruijff's Posts
Wolfgang Stuerzlinger
Wolfgang Stuerzlinger is a full professor at the School of Interactive Arts + Technology at Simon Fraser University in Vancouver, Canada. His work aims to gain a deeper understanding of and to find innovative solutions for real-world problems. Current research projects include better 3D interaction techniques for virtual and augmented reality applications, new human-in-the-loop systems for big-data analysis, and the characterization of the effects of technology limitations on human performance.
[email protected]
View All Wolfgang Stuerzlinger's Posts
Anil Ufuk Batmaz
Anil Ufuk Batmaz has a Ph.D. in biomedical engineering from University of Strasbourg. He is currently affiliated with Simon Fraser University as post-doctoral fellow working on human-computer interaction and virtual and augmented reality.
[email protected]
View All Anil Ufuk Batmaz's Posts
Andrea Won
Andrea Stevenson Won is an assistant professor in the Department of Communication at Cornell University. She directs the Virtual Embodiment Lab, which focuses on how mediated experiences change people’s perceptions, especially in immersive media. Research areas include the therapeutic applications of virtual reality, and how nonverbal behavior as rendered in virtual environments affects collaboration and teamwork.
[email protected]
View All Andrea Won's Posts
Evan Suma Rosenberg
Evan Suma Rosenberg is an assistant professor in the Department of Computer Science and Engineering at the University of Minnesota. His research interests are situated at the intersection of virtual/augmented reality and HCI, encompassing immersive technologies, 3D user interfaces, and spatial interaction techniques.
[email protected]
View All Evan Suma Rosenberg 's Posts
Adalberto Simeone
Adalberto L. Simeone is an assistant professor in the Department of Computer Science at KU Leuven in Belgium. His research lies in the intersection of 3D interaction and virtual reality with human-computer interaction. He is motivated by a deep interest in making immersive experiences more accessible by everyone.
[email protected]
View All Adalberto Simeone's Posts
Aleshia Hayes
Aleshia Hayes is an assistant professor at the University of North Texas. She is passionate about developing, evaluating, and iterating on technology used for learning in formal and informal environments. She runs the SURGE XR Lab where she has led interdisciplinary research with partners from manufacturing, defense, psychology, and education.
[email protected]
View All Aleshia Hayes's Posts
On physical and social distancing: Reflections on moving just about everything online amid Covid-19
Authors:
Mikael Wiberg
Posted: Mon, May 18, 2020 - 10:55:04
On March 11, the World Health Organization (WHO) declared that the spread of Covid-19 constituted a pandemic. They stated that that the virus was not just a threat to public health, but also a crisis that would affect every sector of public life: “All countries must strike a fine balance between protecting health, minimizing economic and social disruption, and respecting human rights” [1].
In response to the pandemic, the WHO recommended physical distancing [2], maintaining a physical distance between people and reducing the number of times people come into close contact with each other. The WHO proposed the term physical distancing as opposed to social distancing, due to the fact that it is a physical distance that prevents transmission. With people practicing physical distancing, they proposed that people could remain socially connected via technology.
Despite this push for physical distancing, social distancing has become the more common term, and people are now trying to maintain social connectivity via digital technology. As a result of rapidly moving most of our social relations—work, education, family—online, videoconferencing systems are now being used more than ever.
But what does it mean to move so much of our social contact online? Is an online meeting the same as a face-to-face meeting? Probably not. Is videoconferencing with your friends and family the same as being together physically? Probably not. And what about the term social distancing when we are in fact only recommended to practice physical distancing— while maintaining our social connections over the Internet? What are we to make of physical and social distance here? Though the differences between the two may at first glance appear obvious, something more complicated seems to be unfolding during the spread of Covid-19 and the lockdowns that most of us are having to come to terms with.
HCI, I want to suggest, offers a way to think about the complexities that the two forms of distance provoke. We have more than three decades of research on face-to-face interaction, and have conducted research on the difference between face-to-face and online interaction. In addition, there is a whole strand of research in sociology, social psychology, and environmental psychology on the important role of physical closeness—from basic communication aspects and the role of physical places for being together, to more complex questions involving matters of being together, belonging to groups, having special bonds, and feeling closeness to others (i.e., connections, belonging, attachment, and coupling). In short, there are multiple reasons why physical closeness is fundamental to us as humans— and maybe also why physical distancing feels so hard to practice for lots of people, for a number of socially rooted reasons. On the other hand, there are also many people who already always experience life with physical distance, and those who have no option to social distance (e.g., due to incarceration). Here, the Covid-19 pandemic might serve as an eye opener to show what this means on a personal and social level.
Still—and amid the ongoing coronavirus pandemic we do not really have any alternatives—many of us need to practice physical distancing. So can we say something more specific about why videoconferencing cannot fully compensate for face-to-face interactions? Do we have any established theories that can help us understand the difference between these two forms of interaction? And more fundamentally, what happens when we remove face-to-face interaction as an available mode of interaction—when we need to abandon face-to-face interaction and move online? In this article, I will reflect on these questions and also pinpoint a few things that we as an HCI research community might need to reflect on as we move forward.
HCI on the role of face-to-face interactions
In my own research over the past 20 years, I have been interested in various aspects of “the local.” When I did my Ph.D. in the late 1990s, it was concerned with collocated groups of people and how to support mobile and collocated users with digital meeting technologies [3]. After that, I continued to focus on the role of physical places. I have looked at architecture, proxemics, and material interactions—all aspects of being together, with each other, and in close relation to materials, things, and places that represent a large part of our everyday lives—at least until recently.
“The local” has also served as a generative concept for lots of work in HCI. In fact, face-to-face interaction, or technology support for “same time, same place” interactions, was one of the four basic modes of interaction proposed by Ellis et al. back in 1991 [4] (Figure 1).
![](/images/uploads/blogs/Figure1_web.png)
Figure 1. Interaction time/location matrix by Ellis et al. [4].
In fact, in HCI face-to-face interaction has been assumed to always be there as an available mode of interaction. Still, most technologies have been designed to bridge distances (e.g., email, the telephone, and the Internet). But what happens when we no longer have face-to-face interaction as an available form of interaction? Just a few months ago, that would have been a far-fetched and hypothetical question, but all of a sudden this pandemic has drastically removed this face-to-face mode of interaction for many of us. From being the most natural and taken-for-granted form of interaction, we all of a sudden cannot get together physically and find ourselves having to follow the WHO’s advice and stay in touch through the use of digital technologies.
In recent years, HCI has been even more clear about the importance of being together and the critical questions that need to be addressed, as we are increasingly acknowledging this as a fundamental part of being human. Our field has also explored issues of being separated, isolated, and apart, and to what extent we can design technologies that support loneliness and togetherness. If being together is fundamental to us as humans, then we also need to examine questions concerning how we are coming together. We see this in recent HCI research on the role of our bodies, on gender, and on inequalities. Being together is a very complex matter, and these questions also illustrate the richness and complexity of our social relations. Now, if we cannot be physically together, then it is not merely a matter of lacking communication tools—it’s a fundamental dimension of our societies, from the small-scale context of individual relations to large-scale matters of humans coming together to form groups, communities, and cities.
So what does this imply as we are trying to move just about everything online? Could it be that physical distancing also prevents us from being together? Probably so. In the next section, I turn to media richness theory (MRT) to attempt to shed some light on why videoconferencing probably cannot fully compensate for face-to-face interactions.
Physical distancing also implies social distancing!
So why is it the case that people still refer to social distancing when it’s actually about physical distancing? And why do we have this boom in videoconferencing? Why haven’t emails, messages, and phone calls satisfied us?
Well, if we turn to media richness theory we might see a pattern. In short, media richness theory (MRT) states that all communication media vary in their ability to enable users to communicate, which, in turn, depends on a medium’s richness. Further, MRT places all communication media on a continuous scale based on their ability to support communication, from simple information exchange to more complex forms of communication (e.g., negotiations, body language, emotions) [5]. For example, a simple message could be communicated in a short email, whereas a more complex message would be better supported via face-to-face interaction.
If MRT is correct, it makes sense that people feel socially distant from each other even though they might still stay in contact via the use of videoconferencing systems. No matter which technology we use to mediate our interactions, it cannot compensate for the richest form of interaction: face-to-face interaction. Further, MRT might explain the current videoconferencing boom, and why we have shifted not only from face-to-face interactions to online meetings, but also toward an increased use of videoconferencing technologies—the second-best alternative to face-to-face meetings.
While we try to use video to compensate for the lack of face-to-face interactions, we are probably also experiencing the difference between these two modes of interaction. It is hard to feel close to others over video, as it’s harder to communicate body language, gestures, and emotions. Further, it makes our everyday interactions bounded to particular sessions, and with that comes the risk of breaking the continuous flow of informal, spontaneous, and everyday encounters—the glue that keeps us together. When we practice physical distancing, we can no longer just bump into each other or maintain a shared common ground as we spontaneously meet. Instead, we need to actively seek and establish interactions— typically through planning and invitations—for a meeting, or just to hang out for a while. Physical distancing means that interactions more than ever demand an active decision to seek contact with others. Many of us might struggle to maintain the everyday connections we have from seeing each other in workplaces and around the neighborhood, or from just seeing familiar strangers at the bus stop.
In short, physical distancing also implies social distancing, even if we try to compensate for some parts of it through the use of digital technologies.
So what can be done?
The classic saying is “you can run, but you can´t hide.” At the current moment, it is actually the other way around. We can practice physical distancing, we can go into self-quarantine, and some countries even practice complete lockdowns. That is, we are trying to keep ourselves separated, to hide from the virus, and to make it harder for the virus to spread. But we cannot run. In some countries we cannot even take a walk in the park, and in most countries there are traveling restrictions implemented in an attempt to slow the spread of the virus. In short, we need to stay put and hide—maybe for a long time.
But there are things we can do. In fact, people are already doing a lot—from their homes and over the Internet. We see lots of creative examples online of how people try to do meaningful, and even funny things, while staying at home.
In relation to the move from face-to-face to online interactions, there are also a number of additional things we can do. If we cannot meet face-to-face at the current moment, we can increase the frequency of our interactions. Not only do interactions depend on the richness of the media, but also on how well we know each other. If we increase the frequency of our interactions we can still share our everyday experiences with each other. While this might at first be seen as a strictly Covid-19-related recommendation, it is also crucial as we move beyond the pandemic. For people who are old, sick, or disabled, or for those that have friends and relatives who live far away, an increase in the frequency of interactions can mean the world.
There are also things technology designers can do. We can improve the technology. We can design better systems with better functions, and better video and audio quality. We can also make these tools lightweight, to make it easier to connect and have spontaneous interactions. In fact, HCI has a whole strand of research on how to design for lightweight interactions.
And we can explore modes of interactions beyond being there. Beyond any approach to mimic or compensate for face-to-face interactions, we can follow the suggestions made by Jim Hollan and Scott Stornetta [6] and explore the things we can do online that might actually be harder to do in a face-to-face setting. In short, to think beyond face-to-face interactions as the raw model for online interactions, and instead explore interactions “beyond being there.” A good example here is documented in Barry Brown’s article in this issue, where he describes how parts of this year’s CHI conference were held online, how it was arranged across different media channels, and how it was fundamentally organized as an online event rather than a poor copy of a face-to-face conference. In fact, it was a good example of how we can reimagine a conference as a socially well-established practice amid the pandemic we’re all facing.
And what can HCI learn from this?
So what can we learn from this? Well, probably that we cannot take anything for granted— not even face-to-face as a mode of interaction. Over the past 30 years, we have relied on face-to-face meetings both in our professions and as a raw model for the development of other modes of interaction. This might have prevented us from really exploring alternatives, and that put us in a vulnerable position when we suddenly had to move everything online.
Another lesson we can learn is that we should probably not use one mode of interaction as a raw model for another mode of interaction. This is an implication for designers as well as for HCI researchers. Instead of developing technologies that at best mimic face-to-face interactions, technologies should provide functionality “beyond being there.” And for HCI researchers, this means that it is less interesting to compare different modes of interaction and more interesting to explore what the whole palette of different modes of interaction means for us in terms of being together. This is also an important path for HCI to take after the pandemic—to learn more about what we can do for people who will continue to struggle with physical and social distance even when the recommendations for physical distancing are removed.
Further, and while we have lots of methods developed for how to carefully introduce new technologies in social settings, we also need more knowledge on rapid technology deployment processes (as the current situation has so drastically and brutally shown), and more knowledge on how this is changing practices and our everyday lives.
Finally, and maybe the most important takeaway from this reflection, is that interactions matter. Amid the ravaging pandemic we do not only need food and a place to hide away—we also need each other. Physical distancing might be what many of us need to practice at the current moment, but the fact that people refer to this as social distancing stresses how we cannot make a living if we lack ways of being together. As we move online, we need to make sure that we establish new social practices that not only compensate for what we are missing, but also add new forms of connectedness. As formulated by the WHO, “We’re in this together.” That should also be the case now as we’re now moving just about everything online.
Endnotes
1. https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020
2. Harris, M., Adhanom Ghebreyesus, T. Liu, T., Ryan, M.J., Vadia; Van Kerkhove, M.D., Diego, Foulkes, I., Ondelam, C., Gretler, C., Costas. COVID-19. World Health Organization. Mar. 25, 2020;
3. Wiberg, M. In between mobile meetings – Exploring seamless ongoing interaction support for mobile CSCW, PhD thesis, Umeå University, Sweden, 2001.
4. Ellis, C.A., Gibbs, S.J., and Rein, G.L. Groupware: Some issues and experiences. Communications of the ACM 34, 1 (1991), 39–58.
5. Daft, R.L. and Lengel, R.H. Information richness: a new approach to managerial behavior and organizational design. Research in Organizational Behavior 6 (1984), 191–233.
6. Hollan, J. and Stornetta, S. Beyond being there. Proc. of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 1992, 119–125; https://doi.org/10.1145/142750.142769
Posted in:
Covid-19 on Mon, May 18, 2020 - 10:55:04
Mikael Wiberg
Mikael Wiberg is a full professor in informatics at Umeå University, Sweden. Wiberg's main work is within the areas of interactivity, mobility, materiality, and architecture. He is a co-editor in chief of
ACM Interactions, and his most recently published book is
The Materiality of Interaction: Notes on the Materials of Interaction Design (MIT Press, 2018).
[email protected]
View All Mikael Wiberg's Posts
This is not the new normal: Studying during a pandemic
Authors:
Ana-Catalina Sabie,
Katharina Brunnmayr,
Kristina Weinberger,
Renée Sophie Singer,
Rafael Vrecar,
Katta Spiel
Posted: Wed, May 13, 2020 - 2:31:59
![Illustration of an overwhelmed student sitting in front of laptop, her face in her hands. A broken piggy bank, a large stack of papers and a long TODO-list are scattered around the laptop on the desk. In the background above the student, semi-abstract icons symbolize data influx related to COVID-19, financial problems, loss of casual interaction, cancellation of lectures, connectivity problems, loss of productivity, and subsequent demands through emails.](/images/uploads/blogs/singer_art_web.png)
Illustration by Renée Sophie Singer.
Austria began its lockdown on March 10, 2020. Just three days later, the government issued stay-at-home orders, quarantined entire villages, decreed non-emergency medical procedures to be moved to a later date, and closed all but essential shops, as well as hospitals, retirement homes, schools, and day care facilities. As university classes started the week before, lecturers and students had one session before everyone’s life was turned upside down. I teach Critical Theory of Media and Informatics and noticed that the students were struggling. Well-meaning advice for considering their situation was available online, but I had seen very few accounts from students themselves. So I asked.
Even though in the middle of April there were two weeks planned without lectures, the students asked to meet at our regular time. Not all of them came, but those who did appreciated just sitting together, if apart, and discussing whatever was on their mind. I suggested they might want to write down their experiences and share them to create community with other students and allow lecturers to better understand students’ perspectives. In the following paragraphs is an account of their experiences. They want you to know that as Austrian master’s students, they are aware of the many privileges they have: healthcare is generally available to most, if not all; Austria—despite being the source of several Covid-19 clusters in Europe—is itself not very affected; and study costs are generally low, though about 60 percent of students work alongside their studies [1]. Additionally, their perspective is limited to a computer science curriculum, including some design and humanities-related courses. Hence, they cannot report on issues like lab availability or the lack of technical aptitude that some students might experience. Nonetheless, they are finding this time difficult to navigate. Given their accounts, we invite readers to consider and potentially extrapolate how their own students might feel. Because the students are not alright… and we should not expect them to be.
— Katta Spiel
Material constraints
We are five students in a master’s degree program in media and human-centered computing at TU Wien in Austria. As students of computer science, many of us came to remote learning from a very privileged position. All of the authors are used to working on their laptops, and students as well as lecturers have above average technology skills and could be expected to move their courses online in a timely manner. Despite this advantageous starting position, we still encounter infrastructure-related problems regularly.
Student housing often cannot offer a stable Internet connection or a quiet and undisturbed working space to all occupants. Previously, this issue was mitigated due to the availability of learning spaces and Internet access at currently inaccessible university buildings. In addition to this lack of space, transferring all courses from attended lectures to remote teaching has not always been a smooth process. Lesson plans dependent on personal interaction or hardware availability had to be changed, scheduling restrictions regarding collisions were thrown to the wind, and everyone had to work with a fundamental lack of information. With the abrupt shutdown of universities, there was a lot of confusion around which courses would take place at all, and how as well as where students could get the necessary information. Our university was not prepared to shift to remote learning on a large scale, so while the necessary infrastructure to stream lectures was technically available, other courses had to individually identify ways of meeting up online.
One of the main effects of this lack of preparation is that students have to navigate a litany of different services across our courses. In Table 1, we list 16 different tools that we need to use for communication or collaboration in our learning, including at least one VoIP (voice over IP) service per course. All of these come with their own account requirements and notification ecology, as well as a frequent need for updates, little to no opportunity for personalization, and vastly different workflows. Next to the lack of choice or consent our educators gave us in the matter, privacy and data protection are questionable in some of these tools.
As an example: Critical Theory of Informatics decided to use Mumble, which is an open source tool hosted on external servers that requires the creation of an account. However, other lectures use the university-wide Zoom installation, leaving students with the choice of either installing and using the tool (accepting potential breaches of privacy) or dropping the lecture. In some cases, lecturers are free to look for alternatives together to find a tool that is suitable for everyone in the course. Finding a solution that fits all requirements is virtually impossible due to limited material and cognitive resources.
Platform/Technology |
Modality |
Availability |
Hosting |
Slack |
Text, Audio chat* |
Limited free plan, closed source |
External |
Email |
Text |
|
Internal |
Zoom |
Audio/Video chat* |
Limited free plan, closed source |
External |
GoToMeeting |
Audio/Video chat* |
Limited free plan, closed source |
External |
Jitsi |
Audio/Video chat* |
Open source |
External^ |
Mumble |
Audio chat |
Open source |
External^ |
Skype |
Audio/Video chat* |
Limited free plan, closed source |
External |
Etherpad§ |
Text |
Open source |
External |
YouTube |
Video and Stream |
Advertising, closed source |
External |
LectureTube |
Video and Stream |
In-house development |
Internal |
Moodle |
Learning Platform |
Open source |
Internal |
SAP |
Text |
B2B, closed source |
Internal |
Xodo |
PDF Annotation |
Free for teaching, closed source |
External |
GoogleDocs |
Text Editing |
Closed source |
External |
GitHub |
Programming |
Open source |
External^ |
GitLab |
Programming |
Open source |
Internal |
Table 1 (* participant or time limit in free plan, ^ could be self-hosted but is not, § two different services).
In addition to setting up a variety of different tools, we also need to keep up with information provided on various university websites, via third-party services, over email, or on the lecturers’ personal homepages. Of the courses that have sent out any updates, some have been canceled altogether while others have adapted their mode to provide as good a learning environment as possible; still others switched to purely independent study courses after providing all the necessary reading materials. However, most continued in some way, though often lacking any specific support for coping with the situation.
We have found that listening to a lecture or participating in discussion via video chat is entirely doable, despite the seemingly unavoidable technical problems. However, we have found that remote discussions require considerably more energy (due to the monomodality and issues like noise or lag) than their in-person equivalent, while leading to more shallow and unsatisfying conversations.
Mourning casual sociality
One of the biggest impacts on our learning has been the loss of personal contact with lecturers and peers. Conference calls and lecture streams can be a substitute for the lectures and in-class discussion, but fail to support informal social interaction that is not only part of each course but also fundamental to peer-driven learning. We miss the casual discussions through which we can clarify questions we might have had and exchange our thoughts on a given topic. It becomes additionally more difficult to identify and coordinate suitable groups for assignments, which then leads to difficulties in submitting on time.
This lack of casual interaction removes a layer of support between students and lecturers. Making new contacts and talking to other students provides reassurance, additional explanations, new ideas, and valuable future connections. While it is technically possible to do this in video calls, current technology comes with severe limitations for casual group conversations, as subtle social cues and body movements have to be largely inferred and there is no spatial separation between private spaces and learning spaces. Additionally, most of us are already fatigued from having several multi-hour calls to attend each week.
Too much to do
As mentioned in the introduction, most students finance their studies by working at least part time. Some of us are in essential jobs and have to cover more shifts to protect our more vulnerable coworkers, while others have lost their jobs, leading to financial distress. In addition, while by and large we are not in a risk group ourselves, many of those close to us are. In this situation, lecturers replacing their lectures with reading assignments poses an additional problem. As computer science students, we do not have the experience and skill to effectively work through papers on our own with little to no guidance. Instead of attending a 90-minute lecture, we have to spend hours reading several papers each week while missing out on the perspective and interpretation of our lecturers. We are still able to learn by just reading the material, but the presentation, interpretation, and framing matters to our understanding. With changing lecture times and information on assignments being dispersed and irregular, it’s easy to miss a class session or a task; whereas in regular personal meetings lecturers make sure to answer any questions we might have and explain tasks in more detail, instead of silently adding them somewhere while expecting us to adhere to strict deadlines.
There seems to be the general assumption that we have tons of free time available because the university is closed. However, we are struggling to keep up with our jobs and potential loss of work, reorienting our entire learning routines, and, well, dealing with the baseline anxiety of experiencing a pandemic.
Teaching versus learning
Dealing calmly with the current situation is certainly important. However, it would be reassuring for us if lecturers admitted that they are as overwhelmed as we are. While some acknowledge that this difficult situation leads to inefficiency, there is also a lack of guidance on their part. In addition, tasks like maintaining forums, communicating changes, and looking at course participants’ submissions are often handled by student teaching assistants, who have to negotiate their peers’ needs with their employers’ requests and expectations.
We see a fundamental problem in the differing views on what the university understands as teaching and what students actually need for our learning. It seems to us that there is a widespread misconception that uploaded videos are an appropriate substitute for in-person lessons in a classroom. However, they are only marginally sufficient to our learning experience. We are aware that, currently, concessions are made, and do not expect our teachers to go above and beyond in these difficult times. But we need to not hear that this is some kind of new normal, or about how this could be an opportunity for creating new digital teaching curricula. While this is all we have right now, we value what in-person lessons bring to us. For example, a student might have questions during a lecture that could lead to a more in-depth discussion on a certain topic. With the current situation, students might not even have the option of posing questions. If they do, the environments are much more permanent. Learning is a process that includes vulnerability, as learning means also learning through mistakes and reflecting on misconceptions. Posting questions in a forum or text context means we have to expose our vulnerability more fundamentally and might still receive either no or only short, unsatisfactory answers, where the labor of going in depth might not feel warranted.
Our intent here is not to complain and add to the pile of already overwhelmed educators. We reach out a hand in solidarity, one that asks you to check in with your students instead of focusing on the formal content. We are struggling as well. This is not normal, so let’s not pretend that it could be.
— Catalina Sabie, Katharina Brunnmayr, Kristina Weinberger, Renée Sophie Singer, and Rafael Vrecar
Conclusion
When I initially suggested writing this piece, I thought it was something to do, a somewhat productive way to engage students critically with the current situation; during the writing process, I realized how cathartic it was for them. What struck me even more though is that upon hearing their stories (some of which have not even made it in here), I was humbled and ashamed, but also in awe. Humbled because even though I thought I was aware of their perspective and tried accommodating them, I had not understood their experience nearly enough. Ashamed because I realized that in moving my lectures and seminars online, I only ever thought about my classes individually, never in the context of a curriculum. In awe because of their honesty and bravery to share their stories with us. I hope you can have similar conversations with your students about their experiences during this pandemic, though it should not require a shared experience to attend to those who struggle. And maybe we can take that with us once we return to in-person classes again.
— Katta Spiel
Endnotes
1. Hauschildt, K., Vögtle, E.M., and Gwosć, C. Social and Economic Conditions of Student Life in Europe: EUROSTUDENT VI 2016-2018: Synopsis of Indicators. wbv, 2018.
Posted in:
Covid-19 on Wed, May 13, 2020 - 2:31:59
Ana-Catalina Sabie
Katharina Brunnmayr
Kristina Weinberger
Renée Sophie Singer
Renée Sophie Singer is a user researcher, UX designer, web developer, and non-professional photographer, graphic designer, and illustrator. She is currently studying in the master’s program in media and human centered computing at TU Wien.
[email protected]
View All Renée Sophie Singer's Posts
Rafael Vrecar
Katta Spiel
Katta Spiel is a postdoctoral researcher in the HCI Group at TU Wien. They currently research exceptional norms by focusing on marginalized bodies in interaction design. Their research combines critical theories with transformative designs focusing largely on aspects of gender and disability.
[email protected]
View All Katta Spiel's Posts
Notes on running an online academic conference, or how we got zoombombed and lived to tell the tale
Authors:
Barry Brown
Posted: Tue, May 12, 2020 - 10:11:09
Due to the coronavirus, the biggest conference in our research field (the ACM CHI Conference on Human Factors in Computing Systems) was canceled in April, just a few weeks before it was scheduled to be held in Hawaii. As the Nordics have a strong research community in this area, it was logical to put together an online event for local paper authors. We asked all Nordic authors with accepted CHI 2020 papers if they would like to present; authors of 51 of the 69 papers accepted. To make a manageable event, we needed to set up parallel tracks, giving authors sufficient time to present their work but also enough time for questions and discussion. Online events are much more exhausting to attend than face-to-face meetings, so we made a number of decisions to create an event where participants could practically maintain their attention and energy:
- We ran with 10-minute talks and 10-minute discussion sections. This gave us enough time to deal with technical problems and kept content to a single bite, but also allowed authors to get feedback and engage in discussion. Having 10 minutes for questions turned out to be important because it appears to take an online audience a little longer to digest the talk and formulate questions. Moreover, given the constraints on informal interaction at an online conference like this, we wanted to emphasize and foster interaction among speakers and participants. Having good session chairs was important, as they could start the discussion by asking the first two questions before the audience jumped in. There are other ways of doing this that might be better, but we felt our model kept things closest to the feel of a “real” conference.
- We limited each session to one hour, with at least 15 minutes break between sessions.
- The event ran “from lunch to lunch” over two days. This maximized participants’ energy levels, and allowed us to run an afterwork event. It also improved the odds that community members from other continents across the world could have a chance to join at least part of the event at a sensible time in their time zones. We added a closing keynote, which also worked well to close the event by bringing everyone together into the same meeting space at the end of the conference.
What technology to use
If you are going to run a virtual conference, deciding on what applications to use isn’t easy—clearly tools for virtual meetings are evolving quickly and users will have differing preferences. We chose to use Zoom because we had seen it work well before in a few earlier events and because it supports screen sharing and audience participation. While technically the platform worked for us, clearly it is a tool lacking in user-centered design. In particular, the preferences are a minefield, laid with hundreds of poorly worded options across a collection of confusing webpages and dialogue boxes. Moreover, some of these options you really need to get right—or things can go wrong quickly. Feature-wise Zoom is excellent, but if you use it to run an event, you will come to passionately hate its interface—it’s full of idiocies. For example, whenever a presenter shares their screen, by default everyone’s Zoom client also goes to full screen—even if you are just watching the presentation in the background. What other app do you know of that goes full screen without asking you first? But, for better or worse, Zoom is a communication app that most people have installed, and for a public event where you want to encourage participation, choosing another less-familiar system would come at some cost. Zoom also has a “webinar” mode, but we really wanted to have the audience share their video to support participation and interaction as much as possible, so we ran the event as a meeting.
Zoom as a company has also made many deeply dubious decisions, so much so that a lot of people refuse to install the app. So as to provide a second way to access the conference, we also streamed the event to YouTube Live. Again, we chose this because most people are familiar with YouTube. The Zoom integration with YouTube mostly works, except it would occasionally drop the connection and generate a new live feed with a new URL, so we had to frantically change the links on our website to the new URLs. YouTube users could also not ask questions or participate in the meeting beyond spectating. From the audience side of things, though, this Zoom+YouTube combination seemed to work well; the audience could participate and ask questions, and if they had problems with Zoom—or were just shy—they could watch on YouTube while remaining anonymous. Some relied on the combination to enable smoother session-hopping. Needless to say, you need some sort of communication backchannel for real-time communication among organizers as you run an event like this (e.g, Slack). This helps maintain awareness across tracks, troubleshoot quickly if problems arise, and foster a sense of being in it together.
How we set up Zoom and YouTube
We had three parallel tracks, and we had to manage these three tracks while homebound and not actually meeting one another. So, we had three of the organizers each use a dedicated machine to run each of the three tracks. On each machine we created a public Zoom meeting, with no password and no waiting room, that allowed access to those who had registered accounts with Zoom—we wanted to make the event as open as possible. And while this was probably a mistake, it did mean that users didn’t have to register in advance; they could just click the link. As the event was free of charge for participants, we wanted to take the opportunity to welcome everyone—something not possible at our traditional face-to-face conferences. Adding a password wouldn’t have improved security, since we would have had to share the password openly too. As the chair, I also had another machine on the side so that I could move between tracks to monitor how things were going and monitor the Slack channel for organizers. We created new Zoom meetings for each track, keeping them running for the entire day. We didn’t use our own personal meeting room IDs; this meant that each day had a different meeting URL, which caused some problems with participants trying to visit old links. To lock down Zoom, we also made it so participants were muted upon entry to the room. We also restricted screen sharing to just the meeting hosts, and made all the speakers meeting hosts before each session. This was slightly clumsy, because it meant that speakers had to arrive before the session started (which sometimes didn’t happen) and that someone behind the scenes had to watch for when they arrived and quickly make them into co-hosts. After our troubles with zoombombing (see below), we also changed the mute setting so that only the hosts could unmute. We then monitored the event and unmuted people ourselves when they were asking questions. We used the chat feature to let people express that they’d want to ask a question, switching chat settings on and off so as to make sure there wouldn’t be any disturbance during the talks. We forgot to turn off the whiteboard and screen annotation features, which caused us all sorts of problems, so make sure you turn them off.
For each track, the meeting was streamed onto YouTube. YouTube asks for an individual account for each live stream, and also has a waiting period between creating a new account and being able to live stream. To deal with this, in the end we just used our personal YouTube accounts for each live stream. Zoom also seemed to randomly drop the connection to YouTube, which caused the stream to move to a new URL every now and then, leading to the problem of “URL roulette.”
URL roulette
Before the event, I diligently created redirection URLs (e.g., track1.chinordic2020.org) so that users could just go to that URL and be connected to the relevant Zoom or YouTube feed. Unfortunately, due to the problems with Zoom and YouTube the valid URLs changed quite a bit throughout the day. And with Web redirects you can never be sure if someone will load a cached copy and be sent to an old, invalid URL. We discovered this problem just a few hours before the event started, leading us to frantically change the website and program with the actual links rather than the redirects [1]. The main program for the event was held in a Google doc. This worked great because we could change it and it updated instantly, and as it is a Web app it doesn’t get cached. Unfortunately, in an attempt to be helpful I had also put the URLs onto the website. This was on Google sites, and while it was relatively easy to update, caches meant that we still got a stream of people complaining they were being sent to the wrong URL.
Zoombombing
After the introduction talk, we started the main proceedings in each of our three tracks. It was at this point that our third track was zoombombed. The URL for this track was posted to some sort of forum and we were flooded by kids trying to disrupt the event. They played music from the “Tiger King,” which was kind of funny; showed pornography on their camera—much less funny; and drew swastikas on our first speaker’s presentation—not funny at all. Warning: If this is the content you expect at an academic conference you are going to the wrong sort of academic conferences.
While we probably should have locked down our meeting more beforehand, we also wanted to keep the event open to as many people as possible. We could have set up pre-registration using something like Eventbrite, but sometimes people discover an event on the day it starts and want to drop in. We could have tried to restrict attendance to those from universities, but we also wanted to run an inclusive event. What we did in the end was ask each participant to put in their full name, and kick off those with fake names (the kid who called themself Oswald Mosley at least had some historical knowledge). Any noise or weird videos also got a user kicked off, and we had (thankfully) selected that a user could not rejoin the meeting if they were kicked out. When we were getting attacked, we also enabled the waiting room, and messaged users who were waiting individually to get them to enter a full name before we would let them into the meeting. While this might not be the best way to lock down your meeting (and certainly took a lot of work, with three extra people working behind the scenes when the attack was going on, in addition to the session chair and the organizer hosting the meeting on their machine), it was the best we could come up with quickly.
Zoom has a particularly stupid feature that allows anyone in a meeting to annotate the screen share — so our zoombombers started drawing onto presenters' slides as they were presenting. There is an option to turn this off somewhere on the huge list of Zoom preferences, but if you don’t turn it off in advance, each presenter needs to go in and turn it off individually. Therefore we had to ask our presenters to do this, before they shared the screen, until we could restart the meeting for the next day. This is one of the many points where it is obvious that Zoom is not really set up for the sort of large event we were running — the annotate feature is so basic it can really only be used for vandalizing others' slides (note to Zoom: a shared notepad feature would actually be useful. Drawing on slides , not so much). Congratulations to Zoom for building a feature that’s really only useful for its Zoombombing users!
In the end, after the amazing teamwork and quick thinking of my co-organizers and others who stepped up to help, the track was delayed by only 20 minutes, and all presenters managed to give their talks and get questions. Congratulations to the amazing team! Zoombombing is pretty horrible, though, particularly for the speakers and chairs who had to deal with this while running the event. Presenting your work is a tough enough experience as it is, so I can only imagine how terrible it is having offensive content thrown at you while trying to do it. It takes place as part of the long depressing history of online violence against women and marginalized minority groups. That those involved did such a good job of reconfiguring the event to deal with it does not take away from the bigger problem.
How to run online talks
Presenting online is challenging, and watching online talks can be, frankly, a little boring. Academic paper presentations are themselves also (surprise!) hit or miss. For this reason, we originally intended to get as many speakers as possible to prerecord their talks, thinking this would get us more polished presentations. Our original plan was to get copies (or links) of videos from our presenters, and then release these to audience members, who could then stream them locally. Thinking this through, though, we couldn’t make sure that everyone would be watching the presentations at the same time, not to mention the challenge of getting everything to work on their own. Attendees might also want to watch the event in the background, so asking them to click on links for each talk could disrupt this. We decided instead to ask presenters to play the videos over Zoom, using the shared screen feature. When we tested this, it worked rather poorly, particularly for videos within presentations. Interestingly, when we actually did this at the conference, the videos worked much more smoothly, suggesting either that something had gone wrong with our testing or that Zoom increases the bandwidth for large events.
Streaming the videos over Zoom, though, makes it impossible to also play the video locally, since there is no way to just turn the audio off on one app—the audio would come from Zoom, and from your video player too. So in the end we just had presenters stream their presentations through Zoom, or present live, and didn’t bother collecting anyone’s videos. This worked well, with some quirks. Presentations are better if you can see the presenter alongside the slides. Prerecording a talk using something like loom.com does a good job of recording your face and displaying it alongside the slides. But if you then play this through Zoom, Zoom also displays a live video of the presenter waiting to answer questions. So you see the presenter twice—once recorded, once live (as they are watching their own talk, something many speakers commented on not enjoying particularly much!). Recorded presentations, although they can be potentially much more professionally done, are also a bit more flat compared to live presentations. There is something energizing about having a presenter actually doing the presentation live, with the audience present, even if they can’t actually technically interact much with the presenter. If I were to run the event again I think I would ask presenters to just present live, as it just gave the event a much better feel.
Audience participation
We really tried to encourage audience members to stream their video. In Zoom this means that you can get a “gallery” view of those attending. There are lots of reasons for this—it makes the event more sociable, encourages audience members to pay more attention (because you’re being watched yourself), and gives you a chance to see who else is attending what sessions (and a chance to see friends from “across the room”). We kept this optional, since there are lots of good reasons why people cannot or might not want to keep their video on. But the value of having audience members with their cameras on was also a little limited with Zoon’s UI—at maximum you can see only 25 other audience members, and when someone is presenting you can see only five audience members. This takes away from the cache so you can get audience reactions as you are presenting.
We are used to having a round of applause at the end of each talk (and usually after the questions too). We had directed our chairs to be as explicit as possible and ask for the applause at the end of each talk. If the presenter is still presenting their last slide, taking up the screen, most audience members don’t see each other clapping; and because the microphones are muted you sometimes only hear the single applause of the chair—or no clapping at all. While you can see the audience clapping, this creates a slightly discouraging experience. Not having it work well makes you realize that applause is important. Having muted microphones also means that you don’t hear the audience laughing at jokes, heckles, or the like, which again undermines the experience somewhat (living in Glasgow for seven years, I learned what a good heckle is).
Discussions
Being a Nordic event, we wanted to value and encourage participation and discussion. Actually, without discussion what point is there in having an event at all? You could just put a link to pre-recorded videos on a website—clearly the sort of dead, lifeless land occupied by webinars. To avoid that feel, we increased the discussion time for each paper to 10 minutes, shortening the paper presentation time from 15 to 10 minutes. This was also partially because we thought we might lose some time to technical problems and delays (which we actually had very few of in the end), but also to make for a more interesting event compared to just sitting down and reading the papers.
It is not that paper discussions at conferences are to be held up as the gold standard. Questions are often terrible, and many academics go all “word salad” in their answers. Some great, high-impact papers get no questions when presented because they are just too far ahead of where the audience is thinking. Other papers get acclaim and active discussion because they spark interest among a passionate but confused section of the audience. But questions clearly do give conferences the live feel—there is always the chance of dispute and argument, new ideas emerging through interaction, or authors being called to account for their work.
Doing this online is obviously not going to be as smooth as when everyone is in the same physical room (at least not while we are all still new to online events like this), but it’s crucially important that you get some sort of interaction going—at least a bit of discussion. At our event we made it a rule that to ask a question you had to ask it in the text chat (like standing at the mic), and that the chair would then ask you to unmute and ask your question (or we would unmute the person asking the question and remute them at the end). Originally we had asked for people to type their question, and although I had hoped this might let the chair select the best questions, in the end many people just seemed to type “I have a question.” We had been warned also that getting questions in an online event can be difficult, so we had prepared the chairs to expect to ask one or two questions themselves. In all cases papers got questions from the audience, but for some it took three or four minutes for the audience to get the questions going. In one or two cases, the chair asked a question and was about to give up, but then four or five questions came in from the audience and a good discussion ensued. Conversation even got heated at times—one questioner got overexcited and muted the session chair so they could ask their question! So while the medium certainly has some limitations, I think we managed to run a participatory event where the audience was actually involved in what was going on. Key lessons here, though, are 1) be patient when waiting for audience questions or interaction, 2) do not worry about silence, and 3) expect session chairs to ask questions, rather than seeing them as the question asker of last resort.
Session chairing
Clearly session chairs were pretty essential to our event. One helpful piece of advice I had been given was the need for session chairs to be more explicit about everything that is going on in the event. Since not everyone sees everyone else (something the tools don’t help with), participants often have no clue what is going on and where we are in terms of the ceremonies. This means the chair has to be explicit and tell everyone when to clap, when to stop asking questions, when to ask a question, and so on. This can feel like the opposite of a well-run social event where you “don’t see the strings” and can make everything feel rather clumsy, but it is pretty essential for having the event work at all.
In fact, for our event, session chairs might have been better called discussants. The chair had to really know the work enough to throw in a few questions, but also be confident enough to allow the audience to think for a little bit (sometimes as long as a minute), sometimes providing a little filler conversation to give everyone time to catch up. It was also important that alongside the chair there was an organizer running the technical aspects of the meeting and helping coordinate the track across sessions, guiding participants regarding breaks and events in parallel tracks. In the case of our zoombombing, it became clear that it’s better to have two or three such people at the ready to handle and monitor different issues. Running the event is one thing; making the technology work so that the session chair and speakers can focus on their tasks is another. So you need to plan on having at least two people per track to make things run smoothly.
Afterwork
I remember being told at the first conference I went to that “it wasn’t important what talks you went to see but who you didn’t go and see the talks with.” Clearly, we can’t emulate that kind of experience online. Yet, we can at least create some “stuff to talk about” that people can refer to when they eventually do meet up after the event. We scheduled a one-hour afterwork event where we made use of Zoom’s breakout rooms to put people into smaller groups of seven or so people, who chatted for 10 minutes, then went back to the main room and talked a little bit about what had been said in the breakout rooms. To be honest, this was more a structured informal discussion than an “afterwork,” although we did manage to do some singing and dancing at the end (an impromptu performance of Abba, if you must ask). Hardly the full conference experience, but still a chance to catch up a little with others. I am sure there are much better ways of doing this and that we’ll get better at these interactions with more experience.
Why bother?
After reading all this you might wonder if it is worth such an effort in running an online event. Instead of having an online conference you could just put videos up on a website, and even attempt to have some sort of offline discussion around the papers. I am sure that would be a good way of communicating research results, and would be much less effort than having a specific event at a specific time. There are two important elements you would miss with that setup, however. Having an event at a particular time and place gives you the “liveness” of a real event. While you can go back later, there are advantages to actually watching it live, such as taking part in the discussion. Just like most sports fans choose to watch events live, this liveness is something to be valued. Moreover, there are lots of effort/encouragement calculations with having a specific event—it motivates participation (I can just go and watch this thing to get an overview), plus offers the sense of having a shared experience with others. With all the content that is available online, having an event also makes it more likely that people will actually make the time to attend the talks and discussions. Clearly the tools we have right now aren’t really designed to support a good online shared experience, but we hope they will get better over time.
Acknowledgments
An event like this needs a lot of different people to be involved. In the end we needed five chairs running things behind the scenes: Marianela Ciolfi Felice and Kristina Höök came up with the original idea and shaped the event, then Donald McMillan and Airi Lampinen came on board to help with getting the technology into shape. On the day itself Mareike Glöss, Ville Sundberg, and Asreen Rostami stepped in to deal with the zoombombing. Sara Eriksson, Riyaj Shaikh, and Kasper Karlgren helped with testing the event format and setting up the website. The event relied upon the smooth chairing skills of Mikael Wiberg, Hans Gellersen, Eva Eriksson, Sarah Homewood, Aske Mottelson, Antti Salovaara, Simo Hosio, Juho Pääkkönen, Jessica Cauchard, Marie Louise Juul Sondergaard, Susanne Bodker, Kashyap Todi, Harko Verhagen, Alexandra Weilenmann, and Mikael B. Skov.
Endnotes
1. I later found that my mistake was that I set the redirect up as "permanent 302" not "temporary 301"—if you set it as a temporary redirect this might support rapid changes.
Posted in:
Covid-19 on Tue, May 12, 2020 - 10:11:09
Barry Brown
Barry Brown is a research professor at the University of Stockholm, where he helps to runs the STIR research group. His two most recent books have been published by Sage and MIT Press, focusing on how to research the use of digital technology, and the study and design of leisure technologies. He previously worked as the research director of the Mobile Life research centre (2011–2017) and as an associate professor in the Department of Communication at UCSD (2007–2011).
[email protected]
View All Barry Brown's Posts
Designs on solidarity
Authors:
Daniela Rosner,
Nicole Rosner
Posted: Fri, May 08, 2020 - 10:47:48
If you have come here to help me you are wasting your time, but if you have come because your liberation is bound up with mine, then let us work together. — Lilla Watson
Over the past weeks we find ourselves pausing before signing off on email messages. We used to include a habitual "How are you?" or "I hope this email finds you well." But these days such sentiments carry new weight and urgency. As academics with the privilege to find this moment and its precarity novel, we feel those words of care break from their former function as social conventions: They might be the only words that matter. We feel compelled to acknowledge what we do not know about the situation of the person we contact, to reach out with concern. Might they have fallen ill? Lost a loved one? Been fired from a job? Sometimes the thought occurs to us that they might not be there at all. Independent of the image, we seek words of strength. After coming across a Twitter thread started on the subject [1], I (Daniela) sometimes use the phrase “in solidarity.”
Solidarity has a short but potent history in the fields of design and HCI. Some who use the term invoke its feminist and activist affiliations (e.g., [2]). Others speak to ideas of equity and allyship, a relationship forged across hierarchies of difference (e.g., [3]). Even given a pervasive sense of doom, solidarity suggests that we can work through it together. Rather than hide behind our own individual problems, we can reckon with ongoing social upheaval through expressions and acts of mutual support.
In our current moment, solidarity gains heightened currency as Covid attacks people along existing lines of inequality. As Ruha Benjamin stresses in a recent talk, “The virus is not simply a biological entity, but a biopolitical reality which travels along well-worn patterns of inequity...It may not set out [to] discriminate, but the structures in which it circulates certainly do” [4]. The same entrenched injustices that maintain institutionalized forms of racism such as incarceration, policing, and housing policy as well as disparities in income, education, and life expectancy, just to name a few, are exacerbated during the Covid-19 pandemic. Some of the highest rates of Covid-19 have emerged in prisons, where people—disproportionately people of color—are trapped in dehumanizing conditions that overwhelmingly conflict with nearly every health and safety guideline; for many, this amounts to a death sentence [5]. Across the world, poor, disabled, and racialized groups are more likely to suffer severe effects on their lives and livelihoods due to the virus. Yet such struggles are also increasingly hidden from sight as many people continue to shelter in place. Speaking of such violences as academic blindspots, Veena Das [6] surmises, “The only question is how we might learn to see what is happening before our eyes.” This pandemic has pushed us to “see” and understand what urban scholars have long critiqued: the tendency for spatial interventions into social life to perpetuate rather than ameliorate existing inequalities, creating new excuses for dispossession and forms of segregation. Today, those processes are increasingly taking place online, where designers, those of us who develop technology, imagine virtual spaces to solve evolving social problems. A sense of togetherness, a feeling that our lives are tied up in one another’s, may feel increasingly impossible. But this connectedness is also increasingly vital to our individual and collective survival.
Bringing these concerns to tech design, we see that technology, such as apps developed to track Covid-19, perpetuates the very same inequalities [7]. Take the example of developing new tools for contact tracing. To scope the challenge and inform a design process, we might choose to run a remote study with as many people as we can find who might participate. The more people reached the better, we might think. Yet, as we know from prior work [8], prioritizing the most likely to be reachable (as in, with flexible schedules and reliable internet, email, and videoconferencing access) tends to benefit well-educated white people who have already long benefited from the healthcare system. Correspondingly, ignoring people less reachable or treating reachability as a universal good will tend to deny basic rights, such as rights to privacy, and erase the lived experience of continually disadvantaged groups such as communities of color, those living in poverty, and those with prior health conditions. Unless those researchers and designers take seriously the conditions that produce systemic inequalities, such as the danger of surveillance among particular populations, this early work will effectively contribute to reinforcing disparities. The same could be said of design projects in a wide range of areas, whether aimed at virtual learning or religious life.
With solidarity in mind, perhaps we have been thinking in the wrong direction. When it comes to Covid-19, maybe it matters less what we in HCI have to offer those affected [9]. Instead, maybe it’s how the virus is affecting what we should have been doing already. The inequalities we see and experience are often socio-spatial—segregation, confinement, lack of services or resources. Today, those socio-spatial inequalities that urban interventions continually reinscribe are playing out in digital tools that we design. We need to avoid the pitfalls of urban designers. We need to learn from their mistakes as well as our own. We need to bring new habits of being to our worlds of design. Concerns for elegance or novelty cannot override basic needs, equal access, and participatory channels for users to take part in making these worlds work for them and their rights.
There’s also a problem with talking about being "in solidarity" as privileged individuals. While our designs may address inequality, many of us have not suffered its consequences. We work on promoting equity, yet we also understand that existing inequalities perversely benefit our careers. We’re experiencing upheaval as radical and uneven. We’re experimenting with ways of working, coping, and maintaining a sense of responsibility in partial response. We’re trying to grapple with the momentousness of the situation. We’re also trying to survive as individuals in order to aid the survival of our families, friends, colleagues, and communities. It takes much more than a word or two—it takes rethinking what we do from the start.
Returning to the email sign-offs that began our reflection, the novel coronavirus has taught that the need for solidarity has no bounds. Within our professional worlds, email is not usually the realm where we encounter suffering. But today it has increasingly become another space where we reach out with gestures of care. How can this new sense of uncertainty in realms we imagined as stable and secure push us to be more conscious designers, developers, academics, citizens, neighbors, friends, and family members? How can phrases such as “in solidarity” truly activate an ethics of care in all aspects of our lives?
We need to do better. To pay more attention. To engage more deeply. To grapple with our own entanglement in everyday inequalities in order to stymie their reproduction and actively promote equity. Inspired by ongoing calls for mutual aid [4,6,10], we need to ask with greater urgency: Where does responsibility lie? As Edna Bonhomme [10] warns: "This is a time for solidarity and to fight back—to figure out a cure for this and to avoid the scapegoating of migrants or ethnic minorities." Engaging a legacy of solidarity within UX and HCI does not ”solve” the range of challenges presented by and within our current moment. Instead it offers one of several sites for opening a conversation across dynamic and uneven geographies of difference. What it means to do ethical design now involves reassessing our collective accountabilities. It means rethinking the worlds we should have been building all along.
Acknowledgments
Both coauthors contributed equally to this work.
Endnotes
1. Hamraei, A. Twitter. Apr. 3, 2020;
2. Kumar, N., Karusala, N., Ismail, A., Wong-Villacres, A., and Vishwanath, A. Engaging feminist solidarity for comparative research, design, and practice. Proc. of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–24.
3. Dye, M., Kumar, N., Schlesinger, A., Wong-Villacres, M., Ames, M.G., Veeraraghavan, R., O'Neill, J., Pal, J., and Gray, M.L. Solidarity across borders: Navigating intersections towards equity and inclusion. Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing. 2018, 487–494.
4. Benjamin, R. Black skin, white Masks: Racism, vulnerability, and refuting black pathology. Talk given Apr. 15, 2020;
5. Taylor, K-Y. The black plague. The New Yorker. Apr. 16, 2020.
6. Das, V. Facing Covid-19: My land of neither hope nor despair. American Ethnologist. May 1, 2020; h
7. Alkhatib, A. We need to talk about digital contact tracing. Personal weblog. May 1, 2020;
8. Costanza-Chock, S. Design justice: Towards an intersectional feminist framework for design theory and practice. Proc. of the Design Research Society. 2018.
9. Bourdeaux, M., Gray, M.L., and Grosz, B. How human-centered tech can beat COVID-19 through contact tracing. The Hill. Apr. 21, 2020;
10. Bonhomme, E. Covid-19 denialism and xenophobia. Spectre Journal. Apr. 20, 2020.
Posted in:
Covid-19 on Fri, May 08, 2020 - 10:47:48
Daniela Rosner
Daniela Rosner is an associate professor in human centered design & engineering (HCDE) at the University of Washington and co-director of the Tactile and Tactical Design (TAT) Lab. Her research critically investigates the ethical and participatory dimensions of design methods, particularly within sites historically marginalized within engineering cultures such as electronics maintenance and needlecraft. She is the author of
Critical Fabulations: Reworking the Methods and Margins of Design (MIT Press). During the 2019–20 academic year she is working in Berlin as an artist-in-residence at MPIWG and a visiting scholar at Humboldt University.
[email protected]
View All Daniela Rosner's Posts
Nicole Rosner
Nicole Rosner is a postdoctoral fellow in the Mansueto Institute for Urban Innovation and a postdoctoral scholar affiliated with the UChicago Department of Anthropology. Her research concerns the everyday politics of city-making and the violent reproduction of social, spatial, and racial inequality. Her regional interests lie in Latin America, particularly Brazil. She received her Ph.D. in sociocultural anthropology from the University of California, Berkeley with a designated emphasis in global metropolitan studies, her M.Sc. in city design and social sciences from the London School of Economics, and her B.A. from Harvard University with honors. She is currently working on a book project tentatively titled:
Remaking the City, Unmaking Democracy: The Afterlives of Urban Renewal in Rio de Janeiro.
[email protected]
View All Nicole Rosner's Posts
The emerging need for touchless interaction technologies
Authors:
Muhammad Zahid Iqbal,
Abraham Campbell
Posted: Wed, May 06, 2020 - 1:47:42
With the spread of Covid-19, the world of interaction technology research has completely changed: The pandemic has created a higher demand for technologies that allow us to avoid touching devices. Before the pandemic, the world had a harder time understanding the importance of touchless technology, and even then it was not imagined in this context. The gesture-based technologies and hand interaction that have been adopted in research have thus far not been popular outside of research labs. There are several issues in the design, development, and adoption of such technologies that should be addressed in the near future.
This is a time when the average human being can understand why there is a need for touchless interaction, which was not so easy to explain in the past. This technology is not only important for healthcare workers interacting with medical equipment, but also in the use of ATMs, vending machines, and learning devices—all great examples of where we need touchless interaction.
Touchless interaction is possible with augmented reality technology, which uses gesture and interaction controller sensors to create a bridge between virtual and real environments. Touchless interaction technology has also been explored in the following research areas: touchless technology in surgery using gesture based technology [1], use of inertial sensors for gesture-based interaction with medical images [2], use of Kinect [3] and Leap Motion devices for touchless interaction in surgery [4]. It has also been explored in education as motion-based touchless games for learning using Kinect [5], and in medical education [6] and anatomy-learning applications using Leap Motion controllers [7]. Mainly in education, these technologies were developed to allow interaction with virtual objects, but they are also viable for avoiding hand interaction with digital devices.
When taking an elevator, you should not have to worry if the buttons have been pressed by a Covid-19 patient. Replacing this button-based interaction with a gesture or interactive hand controller can handle such cases and move the world forward. This particular case would be addressed using a gesture-based sensing system [8] that receives the gesture data to help the user to interact with the operating system of the elevator and avoid the hand touch.
The rapid adoption of biometric systems to monitor workplace attendance, as official identification, to control the security of digital devices, and now in the use of ATM machines has created a need for touchless fingerprint detection systems in these areas. Touchless ATM machines are the potential need of the time. A touchless fingerprint payment system has addressed this issue in a mobile device as touchless biometric payment.
Currently, tracking devices like Kinect, Leap Motion, and the recent development of MediaPipe by Google are some great resources to integrate the touchless interactions in digital devices. By considering the design challenges, issues about their stability and accuracy will be addressed, which can help the world move toward the development of better touchless interfaces.
Endnotes
1. O'Hara, K., Gonzalez, G., Sellen, A., Penney, G., Varnavas, A., Mentis, H., Criminisi, A., Corish, R., Rouncefield, M., Dastur, N., and Carrell, T. Touchless interaction in surgery. Communications of the ACM 57, 1 (2014), 70–77.
2. Jalaliniya, S., Smith, J., Sousa, M., Büthe, L., and Pederson, T. Touch-less interaction with medical images using hand & foot gestures. Proc. of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication. 2013, 1265–1274.
3. Campbell, M. Kinect imaging lets surgeons keep their focus. NewScientist. May 16, 2012;
4. Manolova, A. System for touchless interaction with medical images in surgery using Leap Motion. ICCC 2014.
5. Bartoli, L., Corradi, C., Garzotto, F., and Valoriani, M. Exploring motion-based touchless games for autistic children's learning. Proc. of the 12th International Conference on Interaction Design and Children. 2013, 102–111.
6. Nicola, S., Stoicu-Tivadar, L., Virag, I., and Crişan-Vida, M. Leap motion supporting medical education. Proc. of 2016 12th IEEE International Symposium on Electronics and Telecommunications. IEEE, 2016, 153–156.
7. Al-Razooq, A., Boreggah, B., Al-Qahtani, L., and Jafri, R. Usability evaluation of a leap motion-based educational application. Advances in Human Factors, Business Management, Training and Education. Springer, Cham, 2017, 171–185.
8. Scoville, B.A., Simcik, P.A., and Peterson, E.C. U.S. Patent No. 10,023,427. U.S. Patent and Trademark Office, Washington, DC, 2018.
Posted in:
Covid-19 on Wed, May 06, 2020 - 1:47:42
Muhammad Zahid Iqbal
Muhammad Zahid Iqbal is a Ph.D. researcher in the School of Computer Science, University College Dublin, Ireland. His research interests are human-computer interaction, augmented reality in education, touchless interactions technologies, artificial intelligence, and e-learning. He is alumni of the Heidelberg Laureate Forum.
[email protected]
View All Muhammad Zahid Iqbal's Posts
Abraham Campbell
Abraham G. Campbell is an assistant professor at University College Dublin (UCD), Ireland, who is currently teaching as part of Beijing-Dublin International College (BJUT), a joint initiative between UCD and BJUT
[email protected]
View All Abraham Campbell's Posts
‘Together, we dance alone’: Building a collective toolkit for creatives in a pandemic
Authors:
Kat Braybrooke
Posted: Tue, May 05, 2020 - 11:04:21
![](/images/uploads/blogs/art_braybrooke.jpg)
It seems to me that, if we can talk about such a thing as the tasks of resilience, then today these tasks will share that quality of taking responsibility: not an impossible, meaningless responsibility for the world in general, but one that is specific and practical, and may be different for each of us. — Dougald Hine, 2012
As the twin towers smoldered on September 11, 2001, the electronic music composer William Basinski produced The Disintegration Loops, a recording of decades-old tapes crumbling into decay alongside live footage of that fateful day’s sunset in ruins [1]. In doing so, the collective paralysis of a historic disaster became something timeless—and 20 years later Basinski’s piece continues to compel people to come together and share it in spaces of eulogy and renewal. Creative responses to crises like this, which ask us not only to consume but also to reflect and rebuild, remind us just how interconnected we all are, our lives made up of recursive loops of cause and effect. Such encounters ‘rewire our imaginations,” the science fiction author Kim Stanley Robinson argues [2]. As such, they light fires of possibility inside us—the kind of collective sonic booms that can enable, ever so briefly, alternative ways of living-with to emerge. The theorists Stuart Hall and Doreen Massey have described these as the “cracks” that dismantle and transform the systems of unequal power that structure our society [3]. How can design thinking respond to such moments of crisis and opportunity with sensitivity, in ways that transcend disciplinary and cultural divides? How can collective paralysis foster collective action?
These were the kinds of questions that consumed my attention as the Covid-19 pandemic descended upon us. I work on projects such as CreaTures, the Mozilla Festival, and Superrr that bring people together to imagine new socio-ecological futures across the arts and cultural industries. The more I spoke to the creative practitioners with whom I have collaborated on these projects, however—from artists, makers, and designers to hackers, curators, and educators—the more I started to realize just how hard-hit many of them would be by the persistent uncertainties of the virus and its impacts.
Those with precarious livelihoods are faced with two emergencies at once: The first, a health crisis; the second, economic instability. The data on this is already striking: Museums and archaeological sites from Egypt to Asia have had to close their doors and furlough staff for months, and a report by the Fairwork Foundation indicates that half of the world’s estimated 50 million gig workers have already lost their jobs. In the U.K. and Europe, a majority of arts, heritage, and culture charities are under significant threat, and over 60 percent of makers surveyed by the Crafts Council report a loss of income of over £5,000 in the next six months. In the U.S., a census of freelance art workers by Art Handler found that 90 percent do not have paid leave, and 80 percent are worried about rent. An open letter to museums and galleries, meanwhile, is making the rounds to express concern about increasing layoffs targeting precarious staff at cultural institutions like MoMA and LA MOCA. Most worryingly, we are seeing creatives across all sectors state that they lack access to the support and help they need.
The Covid Creatives Toolkit emerged from these uncertainties as a mutual-aid effort aimed at offering some of that much-needed support, by helping creative practitioners who found themselves needing to quickly migrate their practice onto digital places and spaces as a result of the virus. My collaborators and I noticed that many of the kits, guides, and other resources that were currently being populated with creatives in mind remained geographically skewed toward North American perspectives, or did not allow external contributions. For these reasons, we wanted to offer an open resource focused on free offerings with a global reach, that could be maintained by creatives themselves. Starting from an open call and a tweet asking for help, the kit’s contents were compiled by 30 curators and countless unnamed contributors worldwide, who came to it from across the arts, technology, community, academia, and gig work.
As such, the toolkit has become a living archive that articulates what co-creation as a form of care-making can look like in a crisis. Public contributions to the kit have varied widely, from mutual education and collaborative digital gatherings aimed at challenging social isolation, such as the Uroboros Festival, ANTIUNIVERSITY, and Disruptive Fridays, to film lists, meditative browser extensions for BAME communities, and digital dance parties to promote well-being. The eight featured chapters of the toolkit, from “Digital Gathering Spaces” to “Digital Tools for Creation and Support” to “Digital Well-being,” are wide in scope and offer ongoing documentation of the resources creatives are most in need of as the pandemic progresses. Chapter 8, for example, features much-needed data points on how Covid-19 is impacting creatives. Initially suggested by an anonymous contributor, it has evolved into one of the kit’s most valuable offerings. Another primary focus of the kit emerged from public requests for leads on organizing and bargaining for collective rights, with contributions from organizations that take action against exploitative practices such as the UVW Designers & Cultural Workers Union and FrenaLaCurva.
The Covid Creatives Toolkit has also benefited from the efforts of a decentralized curatorial team, made up of creatives around the world who have volunteered their time to help compile it. Here are some reflections on the process from five of those curators, in their own words.
From design fusionist Kasia Molga in Margate, U.K.: “It is such a strange time for everyone, and some people might be able to cope with this lockdown and anxiety better than others. Many digital creatives are fluent in using network tech to feel connected, but in many cases there is a need to have an anchor or a new routine to keep grounded (while the ground is shifting) and to have a strong base to continue be creative. I believe that this is what the Covid Creatives Toolkit is providing—a resource for everyone to feel anchored.”
From the writer, artist and film producer Tiffany Sia in Hong Kong: “Digital communication is insufficient in many ways, but to envision new forms of community, we must practice care and mutual aid—in its multitudinal forms—across long distance. Not just in these times, but in this next long century. Resistance against a virus is a global effort. Activism bends around these circuits, and at best, manifests as transnational efforts, sharing digital resources, methods, and tools. Creative practices must follow suit. And digital life expands not just as modes for where production happens; we have to trust in the new ways of togetherness. For the Covid Creatives Toolkit, my contributions were mostly focused on cinema. How can we watch together across timezones? How can we share work, experimental films, which were otherwise kept in the annals of email archives or shown at festivals? Through cinema upon these new channels, we must form a different kind of being together, of sharing dreams. How can we continue to forge a popular consciousness while being physically apart?”
From Jaz Hee-jeong Choi, director of the Care-full Design Lab at RMIT’s School of Design in Melbourne, Australia: “As a Korean on an extended journey back to Australia just as the borders were closing around the world, I was acutely aware of the rapid changes in people’s perceptions and behaviors in response to Covid-19. Many, including myself, find themselves in between the need to do more of everything, like talking about and making or designing things … and do less of everything we had been doing collectively to date, and perhaps try to listen and/or reflect instead. For this reason, I refused to participate in or share calls for immediate actions, especially those calling for “doing more” from creative practitioners, many of whom were facing imminent threat to their livelihoods … but when the invitation came to contribute to curating the toolkit, I immediately agreed, with thanks. This was because … it clearly embodied openness and humility; it was not asking people to do things for particular outcomes. Rather, it held the space to share, but with care. This means acknowledgment of the diverse … needs of many different creative practitioners in vastly different situations; calling out to those who may not be thinking about what a toolkit might offer to them as a reader or contributor; and ensuring that if they want to, their voices are explicitly heard.“
From Eirini Malliaraki, who works on new AI projects at the Alan Turing Institute in London, U.K.: “I really enjoyed the process of curating the Creatives Toolkit, and particularly the speed and ease of collaboration with the other curators. Information (and misinformation) about Covid has been spreading at an unprecedented scale, and communities need support to make sense of it. The curators of the various response kits and guides are fundamentally sensemakers: They scan, filter, and organize the informational landscape as an act of collective care … emphasiz[ing] elements that resonate with the lived experience and needs of a community.”
From Storytellers United initiator Philo van Kemenade in Hilversum, Netherlands: “I see a unique role for online forms of gathering that are structurally multidisciplinary. Like a bazaar, they are organized around the realization that people coming together from with different backgrounds have more to offer to and consequently more to gain from each other … Compared to their offline equivalents, online community efforts need to work extra hard to establish and maintain trust. What makes people dedicate time and attention to an online space? How does a communal space embody trust and where does it come from? Can it be earned or transferred? I feel that through the personal approach in curation and sharing, the Covid Creatives Toolkit did a great job at building on existing networks of trust … and is an excellent example of a framework for collective value creation via distributed contributions.”
Decentralized co-creation also has its limitations, however. The Covid Creatives Toolkit has required the dedicated attention of its volunteer curators to manage its contributions and to disseminate it widely enough to include diverse perspectives. It has been a challenge to gather content and suggestions outside of Europe and North America—and the limited translation capacities of the toolkit’s platform mean it is less replicable than I had hoped. Free resources like the Creatives Toolkit are also left with far too few options for hosting their content on easily accessible digital spaces. As a result, projects of this kind must use free tools provided by proprietary digital platforms, which gather revenue from the data traces of their own users. We also currently lack the social infrastructure to collaborate with others creating similar toolkits elsewhere—and curators like Eirini Malliaraki have rightly asked why we cannot foster resilience not only within the many different communities affected by Covid-19, but also between them.
These experiences illustrate how the process of taking care, as defined by Maria Puig Bellacasa as “those doings needed to create, hold together, and sustain life's essential heterogeneity by creating relation, in ways that recognize interdependence” [4], can emerge through co-creation in times of crisis in ways that build solidarity—and also how that process can be both messy and complicated. Like Basinski’s Disintegration Loops, the Covid Creatives Toolkit is a product of its time, a reminder of how Covid-19 has rewired our imaginations. It is a reflection of the mutual-aid networks built around it, and the challenges they face. In the words of the Zapatistas, it is a “world where many worlds fit.” By coming together in a time when so many of those involved are isolated and vulnerable to new forms of precarity, mutual-aid toolkits teach us that the claim of “knowing” something is inconceivable without acknowledging the multitude of interdependencies that have made that knowledge possible. As the anthropologist Arturo Escobar puts it, “All creation is collective, emergent, and relational; it involves historically and epistemically situated persons—[and] never autonomous individuals” [5]. I believe it is in these collective worlds upon worlds in all their messiness that the real work of design thinking as a viable form of future-making begins. For it is in such spaces of collective co-creation that we learn who we really are as a species and as a biosphere, and who we really want to become.
Acknowledgments
I would like to thank the following people in particular for volunteering their time to co-create the Covid Creatives Toolkit as its curators and allies: Marc Barto, Katy Beale, Andrea Botero, Tanya Boyarkina, Jaz Hee-jeong Choi, Hanna Cho, Sophie Dixon, Tracy Gagnon, Janet Gunter, Lara Houston, Sophie Huckfield, Philo van Kemenade, Jamilla Knight, Helen Leigh, Ann Light, Thor Magnusson, Eirini Malliaraki, Mauree Aki Matsusaka, Kasia Molga, Dina Ntziora, Mirena Papadimitriou, Annika Richterich, Anika Saigal, Anouska Samms,Tiffany Sia, Andrew Sleigh, Alex Taylor, and the CreaTures network of researchers and practitioners, who are developing creative practices for transformational futures across Europe, for their support and inspiration. I would also like to thank the many who continue to make suggestions, share, and maintain the toolkit. As Innervisions puts it: “Together, we dance alone.”
Endnotes
1. A recording of The Disintegration Loops is available here.
2. See: “The Coronavirus Is Rewriting Our Imaginations,” New Yorker, May 1, 2020.
3. Hall, S., Massey, D. and Rustin, M., eds. After Neoliberalism? The Killburn Manifesto. Vol. 53. Lawrence & Wishart, London, 2013. Free to read online here.
4. Bellacasa, M.P. de la. 2012. ‘Nothing comes without its world’: Thinking with care. The Sociological Review 60, 2 (2012),197–216.
5. Escobar, A. Designs for the Pluriverse. Duke Univ. Press, Durham, NC, 2018.
Posted in:
Covid-19 on Tue, May 05, 2020 - 11:04:21
Kat Braybrooke
Life less normal
Authors:
Alex Taylor
Posted: Thu, April 30, 2020 - 8:41:00
Is that how we lived then? But we lived as usual. Everyone does, most of the time. Whatever is going on is as usual. Even this is as usual, now. We lived, as usual, by ignoring. Ignoring isn't the same as ignorance, you have to work at it [1]. — Margaret Atwood, The Handmaid’s Tale
We’ve heard plenty of chatter about normal life in the last few weeks. Lots has been said about a departure from the normal, and questions are repeatedly being asked about what disruptions we must endure to normal life to reduce the spread of the novel coronavirus, and to eventually help find a way to return to normal.
Through critical thinking in feminist, race, and intersectional scholarship, we know though that this “normal”—ordinary life before Covid-19—is suffused with complications and surfaces acute problems for many across society. For people often assigned to the margins—people of color, the homeless, the colonized, the disabled, the low-waged, the unemployed, the displaced, and so on—normalcy relies on long histories of prejudice and continued exploitation. For many millions, globally, “the normal” is a life in precarity that demands continued endurance.
As we live through the Covid-19 pandemic, these inequalities are becoming increasingly apparent. Coverage in the popular press shows just how widespread and deeply rooted the effects of the imbalances are, and how lethal their consequences can be. From hardships felt by low-paid key workers and those on the front lines, to the disproportionate numbers of deaths among ethnic populations in ostensibly wealthy, modern enclaves (most strikingly among health workers in the Global North), the brutal inequities and injustices of late capitalism are being felt [2].
In HCI, and through parallel research in science and technology studies, we also know that technological systems and scientific programs [3] serve to sustain many of these injustices. Technoscientific systems and infrastructures that seek to monitor and optimize human behavior and productivity, or that manage the functioning and health of bodies, enforce an idea of normal that obscures the brutal realities and erases those at the margins, sometimes violently [4].
At this moment of worldwide disruption from “the normal,” then, it seems another question we could be asking is whether we want to reimagine what, exactly, we want to “return to.” And, for HCI, we might ask what versions of technology we might imagine to disrupt the troubling normalcy that marks our times. The question I want to think with here is: What worlds are we making possible?
Let’s start then with this idea that will be familiar to many readers—that is, how the status quo—what we think of as normal—masks and erases those at the margins of society. From our experiences with Covid-19, we know that crises can make visible those who are usually out of sight. Such disruptions to the normal also bring into sharp relief the technoscientific systems that the few profit from and how they are reliant on discrimination and exploitation [5]. So the exploitation of gig workers and Wetherspoons staff, but also cleaners, migrants, carers, and people involved in mass food production and supply chains are a necessary part of sustaining the normal. Crises, like the one we are in, surface the dependencies intrinsic in “ordinary” society and who is exploited to maintain normalcy.
For me, the critical point here is that the challenges we’re facing are deeply structural [6] and are deeply entangled with the sociotechnical systems we work on in HCI [7].
Think about this with respect to the spread of Covid-19. The efforts to limit its impact have, of course, been varied and uneven. There have been reports of the virus and its technologies of mitigation and containment being used to reassert the balance(s) of power and wealth in society, and to exert control over the already marginalized and exploited—a biopolitics of our time.
This impact is set alongside concentrated incidences of job losses, as well as fraud and crime. For us, I think, questions must be asked of how technologies and versions of technoscience are being mobilized. Everything from access to testing and ventilation equipment, to the machinery for “rebooting the economy,” to distributing state-backed welfare, need to be examined to understand how the sociotechnical, the sociopolitical, and healthcare are being entangled. And how these entanglements are amplifying already deeply set injustices and discrimination.
The point I want to make here is not just that the technologies we envision and work on play an active role in these conditions. Nor do I want to make any exaggerated claims about the impact HCI has had on the technology sector. Rather, my claim is that we (in our urge to design interactive systems that appeal to the many) are inexorably intertwined in worlds that furnish and sustain the conditions for exploitation and discrimination. We are not innocent bystanders serving up neutral technologies or indeed fixes [8]; we are integral and complicit in worlds that make many lives a lot less like the normal we are accustomed to and, to be frank, a lot less bearable.
I’ve struggled here to choose an example to illustrate this point, not because there are too few, but because the examples are everywhere when we choose to notice. Let me illustrate my argument, then, by first touching briefly on a realm of work that has been central to HCI pretty much from its inception, remote collaboration and videoconferencing. I then want to turn to what might seem an unrelated area, the technoscientific capacities that enable exploitative, global, animal farming and food supply chains. Placed together, spanning varied realms and scales, we’ll see that the ideas and logics in HCI intertwine with many of the inequities that are surfacing during the coronavirus crisis.
Videoconferencing, for many of us, has become a regular feature of work during the pandemic. With daily calls via Skype, Zoom, Microsoft Teams, etc., those in HCI will be reminiscing about its seminal research covering the interactional challenges of remote working via video, and the human work involved in coping with dropouts and partial views of interlocutors and the spaces they are working in. We will also remember that videoconferencing was seen as one way of creating a more accessible workplace for those with disabilities or who need to work flexibly. Who could have imagined videoconferencing and the troubles of remote talk would have come into their own in the time of a global pandemic?
Yet what many in HCI will have also overlooked, including myself, is just how divisive, societally, remote, computer-based work would be in 2020. Covid-19 has made it strikingly clear that a significant proportion of undervalued and low-wage work must by and large be performed in-person. Those most at risk in society—careworkers, cleaners, bus and delivery drivers, packaging and factory workers, and so on—are at risk because they simply have to be “in place” to work, and at the same time don’t have the privilege or choice not to work.
The turn to knowledge work in HCI was then a turn away from the less privileged and a corresponding investment in a very narrow and distinctive class in society, the wealthy and educated. And in turning its attention away from those who have to be at work, HCI also turned away from large swathes of ethnic populations and race groups. The shocking statistics of Covid-19’s disportionate impact on black, asian, and other minority ethic groups will take some time to fully explain. However, among other important determinants, I’m confident a need to be physically at work will be a critical factor.
Again, the point to take away here is not that HCI and its research into remote work and videoconferencing are the direct cause of the inequities that surround us today. Nor is it to suggest we’ve not contributed to programs that prioritize fair and equal access to ICT. It’s that we have played—arguably unwittingly—a part in furnishing a world in which the wealthy and privileged have the choice to work remotely, to isolate and socially distance, and to stay safe. HCI is part of a rationalizing of work and labor that makes a version of normal possible, perhaps even probable. In responding to the current crisis, I believe it is then incumbent on us first to notice how we are implicated in these worlds and then to think how we might use our design methods and outputs to create the conditions for many more potential worlds, and alternatives that might just offer better ways of living and dying together.
We turn now to the seemingly distant world of animal farming and food supply chains. Though understandable attention is being given to wet markets in China—those that sell live animals and often exotic species—the dangers we must acknowledge are a good deal closer. Consider the results of an article published in 2018 by epidemiologist Madhur Saharan Dhingra and her colleagues [9]. The authors use a survey of avian flu viruses to show that highly pathogenic cases are far more likely to emerge through commercial poultry farming and intensive production systems, and correspondingly their occurrence is more likely in high-income countries. It’s also conditions like these that accelerate the spread of zoonotic diseases, diseases that make the jump between species. Avian flu and coronaviruses are thus more likely to move between species and to humans in factory farming conditions, where animals are kept tightly packed and huge quantities of effluent have the opportunity to flow between systems of food production [10].
Of course, we know that the scale of this farming and scope for the spread of diseases relies on technologies that sense physiological functions, monitor activity, and track the mass transportation of bodies. Although we might argue the concerns of HCI are a long way from animal farming, a very particular logic of bodies is being applied that feels not unfamiliar. Bodies, here, are reduced to quantitative measures and optimal metrics for maximum productivity yields. Moreover, value is assigned and generated through the production and proliferation of data, and the transactional potential it affords. HCI might not be directly involved in designing and building technology for factory farming, but it is deeply entangled in a logic that enables it and allows it to perpetuate.
Consider this further down the supply chain. The human labor of food production, so often hidden from us when normalcy prevails, is, in this crisis, attracting attention [11]. The pandemic is revealing the precarity of low-wage immigrant populations who ordinarily work thanklessly to supply us with food. With these workers routinely classified as unskilled and easily replaceable, we see at one and the same time how undervalued people’s lives can be, but also how critical they are to normality. Again, a technoscientific logic operates here, one of extraction where systems of monitoring and surveillance are deployed to extract maximal labor from people working across global supply chains. Far more sophisticated than the Taylorism applied to the factory floor at the turn of the 20th century, algorithmic technologies manage and optimize globally distributed supply chains against demand, locating human labor among the flows of just-in-time production. The remarkable achievement is that maximum extraction and productivity operates across scales and locations, from the factory farm, to laborers along the supply chain, to the infrastructures of circulation. It’s hardly surprising that human bodies, and indeed other living bodies, appear marginal, if not expendable.
Of no coincidence are the parallels with the remarkable work from Lily Irani and Noopur Raval. They show how the piecemeal tasks of Turkers and the monitored activities of gig workers slot into interlocking technoscientific and capitalist logics. Our medical imaging software [12] and takeaway orders, for instance, so much a part of the everyday and in different ways recognized as critical in the pandemic, at once depend on a normally invisible labor that sustain flows of capital and wealth worldwide.
It should then be clear that the technologies we are preoccupied with in HCI—technologies that count, monitor, calculate, identify, etc., all across geographically dispersed networks of fiber and wireless communication channels—are implicated in a version of normal that is exploitative and injust. The intensive farming of animals and our food supply chains are just examples of where computing and computational technologies afford and sustain logics in which inequity and exploitation are prerequisites. Although this structural machinery undergirds our dependence on an injustice that feels removed from us, it aligns with the same axes of power and wealth, and amplifies the conditions in which nonhuman-born viruses can establish themselves and thrive in humans.
In HCI, I believe we need ways of understanding how technology and technoscientific infrastructures create very particular conditions for sociotechnical relations and indeed multispecies relations. For example, how technoscience is implicated in deforestation and the massive depletion of wildlife habitats; how it affords a machinic logic in the transportation and slaughter of animals; how it persists in reducing human labor to counts and metrics; and how it creates the conditions for microbes and what emerge as human pathogens to flourish literally in our backyards.
I also believe HCI and design must face the challenge of imagining how life might be otherwise, in and after the pandemic. Perhaps it is about more than what worlds are we making possible? The question to be asked might be better put: What technoscientific interventions might make other worlds possible?
Finding ways to mitigate the spread of Covid-19, supporting, for example, contact tracing, symptom tracking, and immunity certification are undoubtedly important goals. However, the longer-term challenge for those of us invested in design and technology’s proliferation is to look beyond these immediate fixes. We need to be asking what multiscalar modes and practices might be reimagined to be responsive to and responsible for the seemingly separate technoscientific realms of managing human pandemics and caring for our sociotechnical and multispecies relations. We need to be imagining worlds that resist singular or monolithic ways of valuing life, that question the logics of extraction and transaction, and that make possible a multiplicity of ways of living together. As Justin Smith writes in his article “It’s All Just Beginning”: “These are not end times...What this is, rather, is a critical shift in the way we [need to] think about the human, the natural, and the overlap between these.”
Endnotes
1. Thanks to Constantine Sandis for reminding me of the resonances of The Handmaid’s Tale to our contemporary moment.
2. See: Miami's flawed Covid-19 testing system exposes city's rich-poor divide
3. Yusoff, K. A Billion Black Anthropocenes or None. Univ. of Minnesota Press, 2018.
4. Star, S.L. and Strauss, A. Layers of silence, arenas of voice: The ecology of visible and invisible work. Computer Supported Cooperative Work (CSCW) 8, 1–2 (1999), 9–30.
5. Klein, N. and Peet, R. Book Review: The Shock Doctrine: The Rise of Disaster Capitalism. Human Geography 1, 2 (2008), 130–133.
6. Zoe WIlliams makes this point forcefully in her Guardian piece: "We say we value key workers, but their low pay is systematic, not accidental."
7. Katrin Fritsch makes a similar point by raising the specter of the hyperobject in her Medium article: “Back to Normal?! Data and Technology in Times of Crises"
8. Tamar Sharon writes about the complications of companies like Apple and Google building global contract tracing infrastructures in ”When Google and Apple get privacy right, is there still something wrong?"
9. Dhingra, M.S., Artois, J., Dellicour, S., Lemey, P., Dauphin, G., Von Dobschuetz, S., Van Boeckel, T.P., Castellan, D.M., Morzaria, S., and Gilbert, M. Geographical and historical patterns in the emergences of novel highly pathogenic avian influenza (HPAI) H5 and H7 viruses in poultry. Frontiers in Veterinary Science 5 (2018), 84.
10. Science journalist Sonia Shah details this in The Nation: “Think Exotic Animals Are to Blame for the Coronavirus? Think Again”
11. A member of Angry Workers who works at a Bakkavor factory, April 16: "'Don’t call us heroes’: Life on a Production Line”
12. Wang, S., Kang, B., Ma, J., Zeng, X., Xiao, M., Guo, J., Cai, M., Yang, J., Li, Y., Meng, X., and Xu, B. A deep learning algorithm using CT images to screen for Corona Virus Disease (COVID-19) medRxiv. Apr. 4, 2020; DOI: 10.1101/2020.02.14.20023028
Posted in:
Covid-19 on Thu, April 30, 2020 - 8:41:00
Alex Taylor
Alex Taylor is a sociologist at the Centre for Human Centred Interaction Design at City, University of London. With a fascination for the entanglements between social life and machines, his research ranges from empirical studies of technology in everyday life to speculative design interventions.
[email protected]
View All Alex Taylor's Posts
Design thinking around Covid-19: Focusing on the garment workers of Bangladesh
Authors:
Nova Ahmed,
Rahat Jahangir Rony ,
Kimia Tuz Zaman
Posted: Tue, April 28, 2020 - 9:57:47
What’s going to happen to these garment workers? It was a question from my young colleagues Rahat and Kimia. We were working with garment workers in Bangladesh, where the garment sector is one of the leading economic sectors, with around 4 million workers involved in over 5,800 factories [1]. But it was more than work. During our qualitative studies in January and February 2020, we spent weekends with them in their houses. We heard about their dreams, hopes, and aspirations; their mundane days and their frustrations. If you are a qualitative researcher, you will know what this is like; for others, I want to say it is like we have brought parts of them—their feelings—back here with us. When the first Covid-19 patient was acknowledged in Bangladesh in March 2020, all we could think about was their congested houses, dense workplaces, and lack of savings for healthcare and emergency support.
Before going into their current concerns and design-related possibilities, we wanted to take you to them and into their houses, as shown in Figures 1 and 2. We talked to 55 garment workers in the urban areas of Mirpur within Dhaka city and the suburbs of Ashulia and Gazipur during January and February 2020. These workers are not living in the slums, but their houses were in areas with congested multistoried buildings, one very close to the other. Many of these buildings are not fully complete, often lacking paint and railings on staircases. Each floor of the building holds three to four rooms—sometimes five to six rooms—with a family living in each room. All of the houses we visited, however, had a very complete and elaborate kitchen and washroom, shared across families. Families live together with individual dreams and concerns, but with shared support for each other.
![](/images/uploads/blogs/temp_file_Figure1_web1.jpg)
Figure 1. The surrounding area in Gazipur where many garment workers reside; our interview took place in rightmost corner space. February 2020.
![](/images/uploads/blogs/Figure2_web.jpg)
Figure 2. A discussion in Mirpur, Dhaka. February 2020.
Having had the critical experience of working with women before, we were expecting the struggles of female garment workers that are common in our region [2]. But the baseline employment scenario is different here—job security is higher for women in the garment industry (90 percent women [1]). The priority given to women shows up at home, where their stable jobs are accepted and their spouses take other responsibilities, many struggling to maintain a continuous flow of income. There were signs of blossoming equality starting in these homes.
It is a positive insight that women are more empowered in this sector and can play a significant role in families, but the picture is not so flowery when looking at the Covid-19 pandemic. This community does not save anything for their healthcare, investing more in children and at most in possibly buying a cow in the village, using their regular income to support their daily lives. Women work together in the shared kitchen, which is why the social distancing required for safety during Covid-19 does not make sense here under the current living conditions.
When the government-imposed lockdown started in Bangladesh in early April, all garment factories were closed immediately without providing wages. Though it was promised by the garment industry associations that all the workers would be paid their wages in a timely fashion, the reality is that very few factories paid their workers their full salary. Some of the workers we talked to previously reached out to us during the lockdown in Bangladesh, sharing their daily anxieties. They were staying in their residences, still waiting for the garment factories to reopen. We also have seen in the media that many garment workers gathered and protested for their wages during this lockdown period, without maintaining any kind of social distance.
Our ongoing support systems in Bangladesh are designed with a top-down approach—the solutions, helplines, and risk maps are generated by the authorities. There are also volunteers, foundations, and NGOs who work together or separately on support systems and fundraising, advertising heavily on web platforms and social media. However, the garment worker community are not exposed to the technology world, as they are busy with their laborious day jobs [2]. Listening to the workers, following up during the pandemic, it was clear that they wanted to speak, to share how they have been feeling. But there is no such platform to share their voices and feelings. If the garment workers need emergency support, it will be challenging for them because they do not know what support systems are available. There is support for people in extreme poverty as well as support for middle and upper-class people over technology platforms, but this community falls in between.
In Singapore, they tried to reduce infected patients by tracking the cellphones of infected residents and deploying quarantines by clustering the community around infected people. This technology is used by all citizens for healthcare purposes. That is how they can separate the infected community and provide better support. But the context in developing nations is different. Most of the people here are poor and require financial support and measures to ensure food security. Nationwide lockdowns cause scarcity in low-income communities. Additionally, these communities have less access to technology. The Ehsaas scheme (an SMS-based system) is a cash-collection system for the poor people of Pakistan, but now it is not feasible during this pandemic due to the lockdown. Thus, getting blessings from any deployed technology is a challenge.
But a problem can open up design opportunities—eventually leading us to the day when we have solutions that are inclusive, open, and supportive. We need a design that incorporates workers’ voices to generate a support system. There will be varying requirements, where one person just wants to share their feelings with someone while others are looking for a way to secure food for the next month. There will be requirements for emotional support as well as support in finding a healthcare provider. Though all the workers we spoke with owned mobile phones, they have a distant relationship with mobile technology. The support elements are present but the connectors are missing. The current connections are one-directional—flowing from authorities toward the community. Most of the decisions depend on the authorities, which is why all communities are not treated equally. We need an easy-to-use interface that doesn’t invade one’s privacy, and that requires minimal technology access. It can be a phone number to call and share how one is feeling, or it can be a virtual contact online.
We understand that the aspirations of this community have been deeply affected by the uncertainties stemming from the lack of proper support. There is a burning requirement to incorporate a communication link from the garment worker community to the supporting authorities. We believe that the post-Covid-19 days should be our days of hope.
Endnotes
1. Islam, M.S. Ready-made garments exports earning and its contribution to economic growth in Bangladesh. GeoJournal (2020), 1–9.
2. Sambasivan, N. and Holbrook, J. Toward responsible AI for the next billion users. Interactions 26, 1 (2018), 68–71.
Posted in:
Covid-19 on Tue, April 28, 2020 - 9:57:47
Nova Ahmed
Rahat Jahangir Rony
Kimia Tuz Zaman
Entanglements
Authors:
Christopher Frauenberger
Posted: Mon, April 27, 2020 - 5:27:42
As people around the world try to make sense of a new normal, many commentators are saying that this global pandemic will change us permanently. Indeed, the shifts in the ways that we run our societies are seismic and could hardly be imagined only a few months ago—social distancing, travel restrictions, and the shutdown of large parts of our economy, all implemented within a few weeks. Even if this may, and hopefully will, be only temporary, the experience of living through coronavirus will stay with us. We will have seen what is possible, both because it was necessary and because we chose to do so. Aspects of our existence that seemed to be set in stone will have turned out to be up for debate. The virus will have changed who we are.
Recently, I made an argument for a similar form of entanglement with the nonhuman world, that of technology. In [1], I argued that our intimate relationship with digital technology has become equally existential, in that the digital things we create fundamentally change who we are. Consequently, the key for guiding our creation of technological futures should be the political question of who we want to be as part of the world we happen to share with other things and beings—a holistic ethico-onto-epistemological perspective [2] that treats questions of being (ontology), knowledge creation (epistemology), and responsibility and purpose in the world (ethics) as inseparable from each other. While man-made technology is of course different from the virus in important ways, I find this relational, posthuman perspective also to be a very effective lens for making sense of our response to the pandemic, as it plays out in the context of technology as well as more generally.
As we find ourselves in a messy situation that is hard to assess, we debate what is the right or wrong response to this pandemic. In many ways, it is like a crash course on ethical dilemmas: Whose lives do we save—literally as well as in the sense of livelihoods—and at what cost? Who gets left behind? Who exerts that power and by what authority? And what will happen afterward? The main political arenas in this debate include public health, the economy, and technology, and we currently reconfigure these arenas by redefining some central relations between human and nonhuman actors [3]. This mattering [2], these discursive material practices, distribute agency and power in new ways. It makes certain things possible, while making other things very difficult, and it enacts new lines of differences and othering. While there are certainly many different ways in which this plays out, I want to pick out two examples that struck me as particularly relevant for the field of HCI.
Like in many other countries, schools were officially closed in Austria in mid-March. The teachers of my two children scrambled to find ways to implement “eLearning” in a matter of days. They sent PDF worksheets as large attachments to emails with 30-odd recipients, answered questions in WhatsApp groups, distributed links to online content all over the Internet, and organized the occasional video conference, asking “So, how is everybody?” Like all of us, they have been caught by surprise and find themselves on steep learning curves. Some parents are distraught—they share one laptop between three children and need to do their home office work at night, when none of them is eLearning. Others do not have access to a printer, or are running low on toner or paper. Some children have taken on the roles of translators and IT consultants for their parents, while self-organizing their own education—knowledge and skills that they will not be credited for. Communication with some children has just dropped out entirely.
There has been a lot of work on the (new) digital divide [4,5], but the virus has laid it bare in the midst of our society. Further, in our response to the pandemic we witness firsthand a reconfiguration of that divide, an implicit (and explicit) othering, facilitated through technology. Hard-to-reach children from difficult socioeconomic backgrounds, who we should have the highest interest in lifting up, have just been dealt a(nother) bad hand. In the coming surge of efforts to design roles of technology in education, as no doubt will happen, we will need to negotiate these entanglements between improving learning experiences and creating equal opportunities. There will be no best decisions, only choices and trade-offs in the political arena that is innovation. And, I argue, one of the most productive questions that can guide this innovation is: Who do we want to become through the (educational) tools we bring into this world?
The second example is, at its core, a struggle as old as humanity: between the common good and individual freedom. And, as one would expect, it implicates digital technology at its center. Surveillance capitalism has co-evolved with technology to produce an infrastructure that runs on unprecedented levels of knowledge about the masses, with mechanisms for behavior prediction and manipulation at scale [6]. Now this infrastructure can potentially inform our response to the pandemic. In Austria, the former state-owned telecommunications provider produced aggregated data about people’s mobility for the government, after their first lockdown measures. While the public largely supported the lockdown, this use of information was perceived as suspicious. Of course, Google provides a similar analysis for all the countries of the world and uses movement data to chart the least busy times to shop in supermarkets. Helpful now, no doubt, but also a simple repurposing of information that the company collected for different reasons. It is interesting to note that, at least in Austria, the public seems to be wary of the state using that information, while private companies seem to seize the opportunity for whitewashing their practices.
A related function is contact tracing. In Austria, the Red Cross teamed up with Accenture and a private insurer to produce an app, Stopp Corona, that uses Bluetooth and ultrasound to estimate the distance between two mobile phones, performing a digital handshake if they are close enough. IDs are exchanged, and if someone tests positive, the logs allow the tracing of contacts to contain the spreading. The Austrian data-rights NGO epicenter.works analyzed the app and came to the conclusion that it does many things right—it was developed with privacy and security in mind. But no independent audits have yet been conducted and questions around the involvement of private companies are being asked. There are also fears that, while officially denied, use of the app will become quasi-compulsory to be able to participate in the slow reopening of public life. Meanwhile, a broad European alliance of research institutes and technology providers have teamed up in the Pan-European, Privacy-Preserving Proximity Tracing (PEPP-PT) project. And, in an unusual alliance, Google and Apple are collaborating to build contact tracing into their mobile operating systems in similar ways.
Next to medical testing, such data may become indispensable for making informed decisions around the far-reaching changes in our societies. The nature of the sociotechnical infrastructure that we build to produce this data determines what we can know, who it will discriminate against, and what we become through it—again, an ethico-onto-epistemological question [2]. This entanglement is being acknowledged, as we witness a new quality of debate that recognizes that technology is deeply political. While the paradigm of surveillance capitalism has rampaged through our societies largely unchecked, with far-reaching consequences for our democratic structures, questions about whose interests are being served with tracing apps and how this is reconfiguring power are starting to be asked. As in the educational context above, there will be no objectively “correct” design decisions in using big data for keeping pandemics at bay. And as with the notorious trolley problem, there is no correct answer to how much privacy we may want to give up for saving how many lives. These will be choices that, I argue, need to be negotiated. We also need to find appropriate formats for people to participate in this process of agonistic struggle for desirable (technological) futures [7]. And we should be guided by the question of what the technology we bring into this world will make us and if this is who we want to become.
It may well be that public health becomes the next national security—an inherently elusive, yet indisputable desire of people that is being misused to justify technological surveillance. Like 9/11, we might see the coronavirus serving as the scapegoat to implement modes of mass behavior manipulation by private companies. However, current public discourse offers glimpses of hope that society might have come to realize something in this pandemic: that digital technology is not just a tool; that innovation is a political arena in which we can participate; that technology creators are political actors who cannot be allowed to be above democratic accountability; and that we can have a voice in shaping technological futures—as they shape who we become through them. In a very posthuman, relational way, the virus may have shifted our relationship with technology.
Endnotes
1. Frauenberger, C. Entanglement HCI the next wave? ACM Trans. Comput.- Hum. Interact. 27, 1 (2019), 2:1–2:27. DOI: 10.1145/3364998
2. Barad, K. Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Second Printing edition. Duke Univ. Press Books, Durham, NC, 2007.
3. Latour, B. Reassembling the Social: An Introduction to Actor-Network-Theory. Clarendon Lectures in Management Studies. Oxford Univ. Press, Oxford, UK, 2005.
4. Warschauer, M. Technology and Social Inclusion: Rethinking the Digital Divide. MIT Press, 2004.
5. Brandtzæg, P.B., Heim, J., and Karahasanovic’, A. Understanding the new digital divide—A typology of Internet users in Europe. International Journal of Human-Computer Studies 69, 3 (2011), 123–138. DOI: 10.1016/j.ijhcs.2010.11.004
6. Zuboff, S. The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. 1 edition. PublicAffairs, New York, 2019.
7. Mouffe, C. Agonistics: Thinking the World Politically. Verso, 2013.
Posted in:
Covid-19 on Mon, April 27, 2020 - 5:27:42
Christopher Frauenberger
Christopher Frauenberger is a senior researcher at the Human-Computer Interaction Group, TU Wien (Vienna University of Technology). His research focuses on designing technology with and for marginalized user groups, such as those with disabilities. He is committed to participatory design approaches and builds on theories and methods from diverse fields such as the action research, disability studies, philosophy of science, and research ethics.
[email protected]
View All Christopher Frauenberger's Posts
Sustainable practices for the academic business sector: Publish in journals such as ToCHI
Authors:
Kristina Höök,
Rob Comber
Posted: Fri, April 24, 2020 - 3:15:06
ACM Transactions on Computer–Human Interaction (ToCHI) is the premier ACM journal for HCI research. Founded in 1994, the journal has been publishing research at the cutting edge of HCI every second month since 2013. Since taking over the helm at ToCHI in November 2018, we have started to see a number of changes, from the number and types of papers submitted to the journal to trends and practices in our field—and now, amid a global pandemic, in the role and nature of research and academic publishing. We want to use this blog to tell you a little about ToCHI and to encourage you to consider it, and other journals, for publishing your research.
We realize that this comes at a time of difficulty for many people. Many of us now sit at home, trying to get teaching done over the Internet, rescheduling trips and events, and, most of all, worrying about those near and dear, our friends, our colleagues, our countries, and the financial situation. Publishing journal articles may be the least of our worries.
It may be hard to see any positive effects of the pandemic—aside from the drastic drops in air pollution, allowing mother earth to take a deep breath. Yet some of the challenges and opportunities we now face preceded the current crisis. Many have long argued that we need to stop traveling and instead conduct meetings and conferences over the Internet. Innovative solutions and full-scale tests of how to host conferences are suddenly being devised without much prior experience, pushing us into solutions and behaviors that we have been longing for. At the same time, for those of us spending our days videoconferencing, it has become clear how much more research on CSCW, UIST, and user practices is needed before this will become the better option. In short, it is clear that HCI work is badly needed!—as we start working more and more remotely in order to reduce carbon emissions; as we integrate digital interactions into every area of life in the Internet of Things era; as we shift toward whole-body interactions through novel interactive materials; and as we engage with AI and big data as a path to controlling and managing whole infrastructures efficiently. In our offices, we are trying to move everything online, from traditional lectures to co-design, from supervision meetings to after-work social events.
And we are learning. New practices are arising.
And so a window of opportunity for debating our own profession, including our publication strategies, opens. HCI is a conference-driven field, as is the whole field of computer science. This has had many positive effects on our young field: By meeting regularly, we have fostered a global understanding of what we expect in terms of quality, research methods, understanding of different cultures, and interdisciplinarity. But maybe we are now at a point in time where we can—and perhaps must—shift more of our work toward journal publications on the one end, and popular accounts of our insights in magazines, such as Interactions, on the other end?
While we can see many negative consequences of not meeting as often to share ideas, we might ask what, apart from reducing the environmental impact of our work, could be the benefits from such a shift? As editors of ToCHI, we see every day how journal articles give us an opportunity to provide more—more substance, a better grounding in the literature, a shift toward deeper reflection. This answers to a lack that HCI has struggled with for a long time. As many have argued before, our field needs to start building more coherence. The different subfields of HCI need to start building on one another’s work, developing research programs or even whole paradigms. We need to shift beyond importing theories and start forming our own. We need to probe the interaction design concepts and technologies we develop, testing their validity and reach, developing some coherence in our research methods and the knowledge we create.
How much substance are we talking about when we shift from conference to journal articles? In our experience, a good ToCHI paper often builds on work done over a longer time period. Instead of one short user study, there will be several studies, or a study done over a longer term. Instead of one exploration of a novel interaction technology, there will be several iterations. Instead of a limited literature review, there will be a substantial review providing an analytical framework that shifts a whole research topic forward. ToCHI publishes work that describes user studies done over months to properly observe behavior shifts or design work done over several years, building a whole new interaction model or a design program.
How do we know that there is substance in a submission? As ToCHI aims to be the flagship journal of the HCI field, we put high demands on the quality of papers we publish. The associate editors of ToCHI are leaders in our field with extensive knowledge of their respective subfields. We rely on their expertise to a larger degree than in, for example, the CHI conference review process. An associate editor of ToCHI will engage at least three reviewers, but will then, based on their input, make an independent decision on whether to accept, ask for revisions on, or reject an article.
It has often been argued that journals are too slow to cater to the needs of a rapidly changing field. Like many other journals, ToCHI has worked really hard on changing this. Our aim is to provide authors with a decision within eight to nine weeks from the time an article is submitted (average is currently 48.2 days). In most cases, revisions are needed, so how fast the next step progresses depends on the author. Papers require a median of two rounds of revision, though over 40 percent are completed with one or no revision cycle (see Table 1). Each revision cycle adds about four months to the process, while the average time from first submission to publication is 11 months. These days, as soon as the final, camera-ready version is submitted, it will immediately be published in the ACM Digital Library. With the exception of special issues, there is no longer any need to wait for the whole issue to be completed before individual articles become available. In effect, this means that the time from submission to seeing your paper in the ACM Digital Library might be as short as 80 days.
![](/images/uploads/blogs/Table1_web.jpg)
Table 1. Average number of days per revision cycle.
ToCHI has been increasing in size during the past few years. The growth in the number of articles submitted per year is shown in Table 2, and the trend line shows a continued growth of 14 percent year on year into the future.
![](/images/uploads/blogs/Table2_web.jpg)
Table 2. The number of ToCHI submissions per year.
ToCHI is currently aiming to broaden our scope, from being more focused on the technical side of HCI to include all of the HCI subdisciplines. This is reflected in the composition of associate editors. It is also reflected in what papers have recently been submitted and published.
Finally, ToCHI authors can choose to present their work at one of the ACM SIGCHI conferences. Most have in the past picked the CHI conference. This has given authors the best of both worlds, as CHI lets them meet their community and debate their results. But to address the sustainability concerns outlined above, we now aim to find new ways of engaging authors and readers in debate. We are discussing including commentaries on published papers, more special issues, and perhaps a blog post in Interactions on the impact of certain ToCHI papers.
If you are considering submitting to ToCHI or in general want to know more, please feel free to contact us!
Finally, take care everyone! Put you and your family first!
—Kristina Höök, editor in chief of ToCHI, and Rob Comber, associate editor and information director of ToCHI
Posted in:
Covid-19 on Fri, April 24, 2020 - 3:15:06
Kristina Höök
Kristina Höök is a professor in interaction design at the Royal Institute of Technology (KTH). She is a SIGCHI ACM CHI Academy member since 2020, an ACM Distinguished Scientist since 2014, and editor in chief of ToCHI, but most of all fascinated and devoted to developing ideas around interaction design, aesthetics and movement—or, as she frames it: soma design.
[email protected]
View All Kristina Höök's Posts
Rob Comber
Rob Comber is an HCI researcher and associate professor in communication at KTH Royal Institute of Technology. His research focuses on issues related to the democracy of technology, including social and environmental sustainability, social justice, and feminism, and to specific applications of computing technology, including civic society, food, and social media.
[email protected]
View All Rob Comber's Posts
Workshops are now required to be conducted remotely—is this a bad thing?
Authors:
Vikram Singh
Posted: Thu, April 23, 2020 - 2:22:55
There's a palpable sense of anxiety in the recent global shift to home working. Both bosses and workers are feeling the loss of the benefits of in-person gatherings, such as meetings, workshops, and casual chats. In the UX industry, the loss seems all the more urgent. Some have argued that design and research workshops are irreparably hobbled by the requirement to conduct them remotely. Workshop facilitators are forced to use “whiteboarding” tools such as Miro or Figma, which provide online collaboration with real-time edits.
There's no question that by conducting workshops remotely facilitators lose out on paralanguage—the emotive and gestural communication we take for granted in person. But the disadvantages are more than made up for by the fact that workshops are now conducted via computer, and thus intrinsically bring with them computer-mediated benefits.
It isn't that these benefits fill in for paralanguage; rather, they enhance remote workshops in ways that are lacking in in-person workshops. They also correct for challenges that appear endemic to how we have traditionally run workshops.
First, online workshops have very low barriers to entry. Donald Schon argued that design is a conversation with materials [1]. In design workshops, facilitators will often enact this idea by getting users to sketch, group Post-its, or play with a fungible material such as LEGO or clay. The tangibility of these materials can be very effective at embodying thought and catalyzing new ideas, yet it can be difficult for non-designers—or those who are anxious or shy—to participate.
But with collaborative whiteboarding tools such as Miro, Figma, or even Google Draw, users can copy and paste text, group items, and even sketch out wireframes using very basic point, click, and drag interactions. The barrier to entry of these tools is extremely low for anyone with basic computer literacy. While these interactions lack the haptics of in-person workshops, they have the advantage of simplicity and extreme flexibility and scalability for anyone with basic computer knowledge. Copying, resizing, and modifying are all mere clicks away. What's more, participants who may be shy about drawing may find that the constraints imposed by the tools (e.g., all boxes look the same for every user) are beneficial, as they iron out differences in the various skill levels of participants—at least for basic sketches.
A second point is that online tools allow participants to be embodied in the design practice in an immediate way that isn’t possible with in-person workshops. Design researcher Nigel Cross discusses [2] three different phases of design—gathering information, sketching, and reflecting—as part of Schon's conversation with materials. This conversation with materials becomes highly sharpened in online tools, as participants can switch between these phases extremely quickly. Participants group, annotate, and sketch while allowing everyone else to see what they are doing, and immediately reflect on activities completed. Their actions are embodied within the tools in a way in which it is little effort to engage and disengage in different activities. I recently ran a persona workshop where each participant was able to place notes against each persona, then immediately openly reflect with the group on what they were doing, without mandated turn-taking or phases.
This is a more laborious activity in in-person workshops, where the doing and thinking phases are sharply distinguished. There are often exact note-taking, sketching, grouping, and analyzing phases, meaning it is more challenging for participants to individually act on personal and collective feedback.
A third point is related to the above—the tools themselves are structured with a high degree of freedom. In other words, the structure of the tools does not enforce or imply a workflow or conceptual structure for organizing information. In his book The Stuff of Bits, Paul Dourish describes how the properties of software constrain, enable, limit, and shape the way those representations can be created, transmitted, stored, and manipulated [3]. He discusses how Excel (which, in my experience, is extensively used for research and design tasks) defines what workers are committing too, how the information must be structured, and the level of granularity of the information. However, with new tools such as Miro and Figma, there is no structure—the workspaces constitute an infinite, empty canvas on which users can add anything: screenshots, text, shapes, etc. While constraints can be helpful for creativity, here they need not be mandated by the software’s materiality, as happened so commonly in older tools, but rather are imposed by a facilitator to induce creativity. As such, workshops that use these tools can oftentimes have as much creative freedom as in-person workshops.
Finally, the sensemaking attributes of remote workshops are much higher than in-person workshops. Sensemaking, loosely defined here as finding conceptual frameworks for information, is expedited in online tools by allowing participants to perform any number of actions on workshop artifacts to increase their understanding. A participant may zoom in and out on communal sketches or diagrams, or may even copy artifacts elsewhere for them to play around with. In other words, the sensemaking is subjective. While the objective of workshops is to form a consensus around information, facilitating individualized sensemaking can help users contribute to a group-level sensemaking consensus.
In some ways, it can be argued that we are trapped by COVID-19. And in many ways we are. We simply can’t be physically next to each other, and it is impossible to compensate for the interpersonal benefits that are intrinsic to in-person workshops. Any attempt to replicate the feeling of sitting next to someone in a workshop is bound to be a poor imitation.
But intellectually and creatively, it’s important to examine how the digital tools that we have developed for design and research open new avenues for creation and analysis, and don’t just act as a lesser replacement for being able to sit next to another person.
Endnotes
1. Schon, Donald A. Designing as reflective conversation with the materials of a design situation. Research in Engineering Design 3, 3 (1992), 131–147.
2. Cross, N. Design Thinking: Understanding How Designers Think and Work. Berg, 2011.
3. Dourish, P. The Stuff of Bits: An Essay on the Materialities of Information. MIT Press, 2017.
Posted in:
Covid-19 on Thu, April 23, 2020 - 2:22:55
Vikram Singh
Vikram Singh is the head of UX at Lightful in London. He has an M.Sc. in human-centred systems from City, University of London and regularly writes about technology and how we interact with it.
@wordsandsuch
View All Vikram Singh's Posts
Insights from videochat research in the context of Covid-19
Authors:
Danielle Lottridge
Posted: Mon, April 20, 2020 - 4:16:25
Covid-19 has made videochat even more important than ever, with usage of Zoom and apps like Houseparty soaring. While at Yahoo Inc., my colleagues and I spent a few years studying how teenagers interact online. In this post, I’ll share insights from our research that are relevant to today's reality of online interaction under lockdown.
You can influence others’ videochat attentiveness with your own behavior
Our research shows teens respond to each other’s signals for attentiveness. Looking away from the screen or pausing the video signaled a reduction in attention, while asking questions and calling out signaled an increase in attention; videochat partners matched that signaled attention level [1]. What this means is that your behavior during videochat can influence others. You can signal to others that the videochat will be for co-presence (while you each do other things), or you can rally others to pay attention by signaling your full attention visually and aurally (and perhaps even suggest ending the meeting early if you get through your agenda!)
Have empathy for context switches—they may be more disruptive for others than for you
Many people are working from home and juggling multiple roles, both professional and personal. Sometimes context “bursts” into videochat, especially in the form of sudden noise like dogs barking or a family member calling out, which we found was embarrassing for some teens caught within context overlaps [1]. You can use these bursts as reminders to be understanding and supportive of your colleagues’ other commitments. This may be especially relevant for parents, but anybody’s life can spill in front of the camera!
Videochat can support shifts from focused interaction to social co-presence, and there’s something great about that
Our research found that teens regularly engaged in videochat that included long periods of silence or focus elsewhere, such as scrolling social media feeds or playing games, amidst spontaneous bouts of conversation [1]. When people are vulnerable to social isolation, we encourage you to try idly hanging out on videochat without actively talking or trying to entertain.
Feeling lonely? Consider livestreaming for a new form of social co-presence
Unlike videochat, which tends to be intimate and by invitation, livestreaming has more of an open-door policy, welcoming anybody to watch. Our project on teens’ livestreaming found that they used it for social co-presence [2]. They chose livestreaming because they could start it any time without having to rely on or wait for others to join the call. There are privacy concerns, but two ways that teens deal with that is 1) to vet incoming participants or 2) to aim the camera at an activity, such as cooking, rather than having the camera on themselves.
Wishing everyone health and safety during this difficult time. We hope that these tips are useful in thinking about new ways to use technology to interact with your communities and colleagues.
Endnotes
1. Suh, M., Bentley, F., and Lottridge, D. "It's Kind of Boring Looking at Just the Face": How Teens Multitask During Mobile Videochat. Proc. of the ACM on Human-Computer Interaction 2 (CSCW), (2018), 1–23.
2. Lottridge, D., Bentley, F., Wheeler, M., Lee, J., Cheung, J., Ong, K., and Rowley, C. Third-wave livestreaming: Teens' long form selfie. Proc. of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM, New York, 2017, 1–12.
Posted in:
Covid-19 on Mon, April 20, 2020 - 4:16:25
Danielle Lottridge
Danielle Lottridge completed a Ph.D. in human factors engineering at the University of Toronto, then migrated south to do a postdoc at Stanford University in California. She worked in the Silicon Valley tech industry for several years before moving to New Zealand and into computer science at the University of Auckland.
[email protected]
View All Danielle Lottridge's Posts
Coronavirus and the carnivalesque: What speculative methods can tell us about Covid-19
Authors:
Cally Gatehouse
Posted: Mon, April 20, 2020 - 10:31:01
Like many HCI researchers, I’ve found that the coronavirus pandemic and its consequences has made it almost impossible to continue doing research as normal. Not only am I suddenly remote from the resources and communities that I work within, but I find myself newly cautious about the speculative methods that I use. It seems irresponsible to speculate at a time like this when we’re not able to predict what the end of each week might bring. Asking questions of what shape our shared futures might take feels fraught when so many lives are on hold and at risk during lockdowns. However, as the pandemic has unfolded, I have started to recognize uncanny similarities to my experiences of doing speculative research. Much like speculative research, these events seem to ask us to renegotiate our sense of what is possible, probable, plausible, preferable on almost a daily basis. And while the events that I am normally concerned with involve myself and a handful of other actors rather than the global scale of the coronavirus, I have started to wonder if speculative research might offer some concepts that can help us to make sense of this current moment in which the world feels turned upside down.
In my research, I often use a variety of material, visual and performative devices in pursuit of a carnivalesque unsettling of the authority of what-is to dictate what-may-be. This draws on Bakhtin’s description of the carnival as time and place for “working out, in a concretely sensuous, half-real and half-playacted form, a new mode of interrelationship between individuals [1].” Carnivals, by standing in contrast to normal life, are spaces in which social hierarchies can be questioned and reconfigured. Previously, I have written about how participatory speculative workshops took on exactly such a carnivalesque atmosphere to create a space and time in which we could collectively reconsider what it would mean to take young LGBT people’s experience of hate crime seriously [2]. This upending of the normal world enables us to imbue parodic and unconventional propositions with a sense of provisional seriousness. While these events do not banish existing social hierarchies, they give heightened sense that these hierarchies are open to revision.
Coronavirus and the measures taken to slow its spread have created a similar sense that normal rules have been suspended temporarily. However, the new rules of these events are in some senses an inversion of the carnivalesque. Bakhtin’s account of the carnivalesque has four distinct elements: free and familiar contact amongst people, eccentric behaviour, profanity, and carnivalistic mésalliance (the bringing together of opposites such as light and dark, serious and silly, life and death, etc.) [3]. The temporary regime that coronavirus necessitates with limited physical contact between people, strict policing of deviance from the new rules of social distancing, and the careful control of all things bodily is a darker mirror image of these characteristics. Most important, rather than a carnivalesque inversion of existing hierarchies, we are seeing inequalities like those that result from the digital divide, precarious and low paid work, and unequal access to housing and healthcare play out in ever more striking terms.
But even in this inverted carnivalesque, there remains some of the same inventive potential to produce newly improvised forms of social relationships. Pubs, exercise classes, and religious services are moving online. We are also finding new ways to enjoy the limited public spaces we still have access to, from spectacular mass performances like singing from apartment balconies to clapping to show support for healthcare workers. On a personal scale, we are all learning how to negotiate public spaces while keeping a safe distance. Normally trivial or mundane activities like going to the supermarket bring a new sense of risk, but people are also finding new ways to derive pleasure or comfort from whatever access to outdoor space is available to them. In my neighborhood, I’ve seen sunbathers sit on flat roofs, garages turned into gyms, and badminton games played across garden fences. I saw a woman visiting what I took to be her daughter and grandson, trapped at the garden gate by social distancing. The visitor was throwing a ball the length of the short garden path for the little boy to kick back to her. This seemed a little risky, right on the edge of physical contact, but the boy was laughing with delight.
These kind of deeply ambivalent experiences are perhaps the strongest affinity between what we’re currently experiencing and the carnivalesque. Both are experiences in which we are forced to acknowledge the ways that horror and joy coexist. The experience of lockdown is streets that feel like a peaceful Sunday afternoon all week long and the dread of daily reports of a rising death toll. It is a sudden expansion of police powers and the invention of new social safety nets. It is being painfully cut off from one another while finding new ways to connect to those physically distant. It is by creating these new ambivalences that gives what we’re currently experiencing a speculative power. Things that seemed impossible are suddenly a reality: Air pollution drops as we reconsider what economic activity is really “essential,” homeless people are housed overnight, and universal basic income has gone from a fringe proposal to a national policy in Spain. Perhaps troublingly for HCI, we are also seeing a growing acceptance that the necessity of intrusive surveillance technology outweighs privacy concerns.
As HCI has increasingly engaged with the broader social, political, and environmental implications of our work, we are increasingly confronted with problems that far exceed our individual or collective capacity to respond as HCI researchers. Coronavirus is arguably one of these problems. Many HCI researchers, including myself, have turned to Donna Haraway’s rallying call to “stay with the trouble” [4] as a way to navigate these new territories. How do we stay with this particular moment of trouble? At some point this crisis will end and we will find ourselves in a post-corona world, one that will look and feel very different to the world we knew before. What will persist of this moment in which “business as usual” became impossible is hard to predict. However, looking back through the lens of the carnivalesque ambiguity, we can begin to account for the ways in which this time brought together elements that are both painful and pleasurable, ugly and beautiful, isolating and uniting, in a host of unexpected ways. Understanding how these discordant aspects have come together in new ways can help us design for not just the world we have, but for the other possible worlds that these strange times allow us to glimpse.
Perhaps this is why I am cautious about speculative methods, particularly in moments like this. Speculation produces novelty but offers no guarantee that we will like the novelty that it produces. Indeed, the hope and the risk of speculative methods is that they will produce something that requires entirely new values with which to judge it. In a similar way, these times, as troubling as they are, sharpen our senses to the way in which Karen Barad reminds us that what is “out of sight may be out of reach but not necessarily out of touch” [5]. While we may be physically distant at the moment, the ways in which we are inextricably interdependent have never been more obvious. While it is far too soon to fully account for what new values coronavirus will produce, speculative methods offer some insights into how these risks can be shared and how we can work to develop our collective capacity to keep responding to these events even when solutions remain beyond our grasp.
Endnotes
1. Folch-Serra, M. Place, voice, space: Mikhail Bakhtin’s dialogical landscape. Environment and Planning D: Society and Space 8, 3 (1990), 255–274; https://doi.org/10.1068/d080255
2. Gatehouse, C. A Hauntology of farticipatory speculation. Proc. of the 16th Participatory Design Conference 2020 - Participation(s) Otherwise - Vol 1; https://doi.org/10.1145/3385010.3385024
3. Bakhtin, M. Problems of Dostoevsky’s Poetics. U of Minnesota Press, 2013.
4. Haraway, D.J. Staying with the Trouble: Making Kin in the Chthulucene. Duke University Press, 2016.
5. Barad, K. Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Duke Univ. Press, 2007.
Posted in:
Covid-19 on Mon, April 20, 2020 - 10:31:01
Cally Gatehouse
Cally Gatehouse is a lecturer in communication design at Northumbria University. She is a design-researcher, with a background in graphic and communication design. Her research uses feminist STS to frame and develop an understanding of critical and speculative design research as a means of "staying with the trouble."
[email protected]
View All Cally Gatehouse's Posts
Hit by the pandemic, war, and sanctions: Building resilience to face the digital divide in education
Authors:
Rojin Vishkaie
Posted: Fri, April 10, 2020 - 5:15:21
![](/images/uploads/blogs/Vishkaie_img2_sm.png)
Globally, education is being impacted by the effects of the COVID-19 (coronavirus) pandemic. Many countries are issuing executive orders regarding the physical closure of schools—some for the entire year—to prevent the introduction and spread of COVID-19 into their respective communities. In response to the rapidly changing educational climate, the development of distance-learning programs, in which students and instructors connect via ICT in different locations, appears to be a work-in-progress—even for highly digital and developed countries. As a U.S.-based secondary school teacher recently told me, “Students' education can't stop simply because they’re impacted by the pandemic, but also online education is new for many families and schools right now, so it is going to take some work to prepare for this rapidly changing situation.”
As a result of the somewhat forced and rapid changing of the educational climate, the use of cloud-based online learning platforms, 5G technology, mixed reality, interactive apps, synchronous face-to-face video, and live radio and television broadcasts have quickly become the prevalent choice for educational delivery in the U.S., China, and Japan. Other countries in Asia and Europe use standard asynchronous online learning tools, such as reading material via Google Classroom and email. These technologies have enabled “ubiquitous learning” experiences for learners across these regions, particularly in the growing trend of decentralized homeschooling during the pandemic.
While the U.S. and China have created and own the vast majority of the wealth in the today's digital economy, the rest of the world, particularly countries impacted by sanctions and war, are trailing considerably behind. “This trajectory is likely to continue, further contributing to rising inequality,” writes UN Secretary-General António Guterres in the preface to a report by United Nations Conference on Trade and Development (UNCTAD). “We must work to close the digital divide, where more than half the world has limited or no access to the Internet.”
The term digital divide, which describes a gap in access to and use of ICT, came into formal usage in the early 21st century [1], but its underlying issues had been studied previously in the late 20th century. The notion of “Do Artifacts Have Politics?” put forth by Langdon Winner in 1980 [2] argues that technology can embody specific forms of power and authority in socioeconomic contexts. And Rob Kling’s “social informatics” [3] (1996) looks at the socioeconomics of the technological artifacts and human social context constituted by ICT. Particularly during the unprecedented COVID-19 pandemic, school closures have magnified already-existing socioeconomic and political disparities within the education system, especially for the most vulnerable and marginalized, revealing inequities in access to resources and issues related to privilege, power, and control in certain regions of the world.
In this context, in countries such as Iran, which are affected by economic sanctions and wars, in addition to experiencing one of the world’s largest outbreaks of the virus, the impact of the pandemic on its education system has been signficant. One example: A teacher from the city of Hamidiyeh, in the Ahvaz province, uses the fridge at her house instead of a whiteboard to teach math, and then sends the videos to her students.
Similarly, Syria, another region impacted by war and sanctions for nearly a decade, has a large segment of the population at high risk of novel coronavirus, while also facing the immense challenges of its decimated infrastructure.
The unfortunate side effects of continued sanctions and wars in countries such as Iran, Syria, and Sudan have slowed their ability to keep pace with the evolving classroom-based teaching approaches and the transition to online-based classrooms. Coupled with COVID-19, there is the potential for a negative impact on the educational landscape and a widening of the digital divide for these countries and others around the world.
Considering the wide scope of the still-expanding digital divide, future trajectories can be envisioned for transforming the learning experiences of students across the world to mitigate its impact. Particularly, it is important to support initiatives aimed at narrowing the gap for more vulnerable and disadvantaged countries, who are socioeconomically impacted by both the global obstacle of pandemic disease as well as extreme sanctions and war. To reduce the enormous consequences of these overwhelming challenges on global education, the following goals must be accomplished:
i. Providing inclusive, universal access
Government as well as non- and for-profit organizations must work hand in hand to enable more students across the globe to have universal access to digital devices and the Internet for low-income households at subsidized rates. This also includes providing equal levels of service and networks to rural and underserved communities so that all students, regardless of socioeconomic status, can participate in remote learning. Inclusiveness also means ensuring that young girls are trained with the necessary digital skills, including the knowledge and skills they need to stay safe online.
ii. Developing digital literacy
Having access to computers and the Internet is a crucial necessity for education globally, but that alone is not enough. The incredible power of digital technology for education must also be embraced by training and preserving additional and more qualified staff, alongside new technologies to promote the best application of these resources. In addition, digital literacy as a doorway to socioeconomic and political literacy should educate students for a digital future that is inclusive, sustainable, and collaborative.
iii. Building resilience into the education
Demonstrating the ability to build internal resilience assets such as problem-solving, self-efficacy, empathy, inclusion, and self-awareness in digital tools and systems has the potential to enhance student outcomes. Actively engaging with the information that students are learning will provide an opportunity for them to perform better.
Endnotes
1. Bates, C. Digital divide. In Global Social Issues: An Encyclopedia. C.G. Bates and J. Ciment, eds. Routledge. London, UK, 2013.
2. Winner, L. Do Artifacts Have Politics?. Daedalus 109, 1, Modern Technology: Problem or Opportunity? (Winter 1980), 121–136.
3. Kling, R. Social informatics: A new perspective on social research about information and communication technologies. Prometheus 18, 3 (2000), 245–264.
Posted in:
Covid-19 on Fri, April 10, 2020 - 5:15:21
Rojin Vishkaie
Rojin Vishkaie is a senior user experience Researcher in Amazon. Her work focuses on human-centered design and evaluation of technology, with recent use cases in devices, mixed-reality education, museums, and children’s gaming and toys. Opinions expressed are solely her own and do not express the views or opinions of her employer.
http://rojinvishkaie.com [email protected]
View All Rojin Vishkaie's Posts
Mobile tracking and privacy in the coronavirus pandemic
Authors:
Montathar Faraon
Posted: Thu, April 09, 2020 - 10:08:59
The worldwide pandemic of coronavirus disease (COVID-19) is putting pressure on and testing societies, companies, and citizens. Life, health, and jobs are at stake. In this critical moment, the HCI community is working together to design, test, and execute ideas, prototypes, and solutions with the aim to strengthen our resistance in the fight against the ongoing pandemic. HCI is contributing with abilities, expertise, and methods to design intuitive solutions that are accessible, understandable, and usable.
The coronavirus disease has had far-reaching consequences for public health, financial markets, and everyday life. With currently over 1.5 million infected and 90,000 deaths, the disease has left few people unaffected by its impact. Technological developments in transportation during the past decades have contributed to the rapid movement of people and, in turn, to the worldwide spread of the disease. Likewise, researchers, designers, and developers around the world are racing to understand, design, and develop digital technology to trace and mitigate the spread.
Contact tracing, in combination with social, or more appropriately, physical distancing, has been viewed by governments as a useful asset in controlling the spread of the disease. A video produced through a collaboration between Tectonix and X-Mode Social, who utilized anonymized location data of mobile devices, showed how far the disease could spread if physical distancing is ignored. Novel ideas for contact tracing are considered essential because, unlike other diseases, COVID-19 appears to spread more easily before any symptoms become apparent.
One of the early stories concerning contact tracing came out of Singapore. The government managed to curb the spread of the disease through mobile tracking by using a smartphone app called TraceTogether. The app uses Bluetooth technology to track users’ proximity to other people and alert those who come in contact with someone who has tested positive. The government also sent daily updates to its citizens that included the current number of cases, suspected areas of outbreaks, and hygiene information to avert infection.
In Russia, the local government of Moscow rolled out a QR code system to allow residents to leave their homes and a smartphone app called Social Monitoring to track patients’ movements. Residents need to register on the government’s online portal, which requires both phone number and photography verification. The online portal could then be used to generate a new QR code every time a resident needs to leave their apartment. If stopped by the police, residents would be required to show the QR code.
In the U.S., companies such as Google and Facebook are responding to the pandemic by using mobile location data to aid public health officials in understanding people’s movement and to what degree people are following physical distancing recommendations set forth by the government. Google released Community Mobility Reports, which provide information about mobility trends in places such as retail, grocery, parks, transit stations, workplaces, and residences. Likewise, Facebook shared mobile location data with academics, as part of its Disease Prevention Maps program, to better understand the success of measures aimed to decrease mobility. Similar to previous initiatives, the Massachusetts Institute of Technology (MIT) has published an app called Private Kit: Safe Paths to notify people if they have likely come in contact with a person who has tested positive for COVID-19.
Because of the different privacy standards around the world, it is difficult to take advantage of and adopt mobile tracking technology between countries. Using the tools offered by the Singapore government would, for example, in the U.S. violate privacy laws like the Health Insurance Portability and Accountability Act (HIPAA) and in Europe the General Data Protection Regulation (GDPR).
To address privacy and data-protection concerns, a mobile tracking initiative has been launched in Europe, which consists of 130 members across eight countries. The initiative is called Pan-European Privacy-Preserving Proximity Tracing, or PEPP-PT. It aims to create an app that takes advantage of Bluetooth signals to detect within certain limits if users may be close enough to infect each other (e.g., closer than 2 meters for more than 15 minutes). Unlike the invasive surveillance technology now being used by governments, the initiative adheres to European privacy and data-protection laws by encrypting and anonymizing personal information.
Mobile-tracking apps could provide more accurate data during a pandemic in order to reduce transmission. They are, however, not without any challenges. A challenge with mobile-tracking apps is to make enough people install them in order to make them useful. Even if 30 to 50 percent of a population installs a mobile-tracking app, it could have a long-term impact. Continuous use of contact tracing through mobile tracking, together with physical distancing, may need to be combined with scaled-up testing and hygiene measures to avoid a resurgence of the disease.
The global HCI community has united to work passionately on initiatives that make use of our devices to combat this pandemic. Across the world, HCI has played an essential role in a series of virtual hackathons that are being arranged to design novel tools to address the challenges with COVID-19 (HackTheCrisis in Sweden, the MIT COVID-19 Challenge in the U.S., and the pan-EU EUvsVirus). In the end, defeating the disease requires collective efforts in order to protect ourselves and our loved ones.
Posted in:
Covid-19 on Thu, April 09, 2020 - 10:08:59
Montathar Faraon
Montathar Faraon is an interaction design researcher with an interest in co-creation, participatory processes, ICTs for democracy, and concept-driven design. He is an assistant professor in the Department of Design of Kristianstad University, Sweden.
[email protected]
View All Montathar Faraon's Posts
Reflections on helping research partners who support those made vulnerable during the crisis
Authors:
Angelika Strohmayer
Posted: Tue, April 07, 2020 - 12:04:11
Over the past few days, I have had many conversations with friends and colleagues who work with charities, libraries, and other organizations who support some of the most vulnerable people in society. As you might well be aware, these organizations are grappling with appropriate and timely responses to the developing global COVID-19 pandemic. Many of us are thinking and talking about ways in which we, as researchers, practitioners, and people, can support our research partners in these trying times. This is while keeping in mind that staff in these services are working on overdrive at the moment; trying to keep up with ongoing changes to all of our lives while also maintaining contact with those who they support.
Like many others, staff who support victim-survivors of domestic violence, those experiencing homelessness, families with children in care, people who are in addiction recovery, or others who may have various intersecting complex needs are working from home. Many will have had to suspend the in-person drop-in services and pause ongoing group work programs they usually facilitate, and instead are now working hard to figure out how to best use technologies to carry on with this work remotely. As technology researchers and designers, we may be uniquely placed to support organizations at a time when many are using digital technologies to carry out their work, with many adopting any such technologies for the first time.
The research partners I have worked with are developing strategies in small teams and then sharing the learning across their wider organizations. They are making use of what's available to them: mobile phones, email, video calls, and free platforms that allow them to communicate with those they support. However, there is of course the issue that many of the people they support don't have access to reliable internet connections, and that they may not have laptops, tablets, or smartphones—7 percent of households in the U.K. have no internet access at all. Some of the people who are supported by these organizations are now at greater risk of experiencing violence from their partners, and others may be at risk of perpetrating more violence toward their partners as anxieties are on the rise and coupled with social distancing measures in response to the crisis; we know that natural disasters and diseases often lead to increased number of domestic violence reports. Others may not have homes at all to self-isolate and are instead sleeping rough or in overcrowded emergency accommodation. Despite these barriers, many people are working hard to stay in touch with those most vulnerable and those at greatest risk, and they're interested in finding creative ways of doing this.
From the conversations I've had with colleagues who work in this space on a regular basis, and this may come as no surprise, I've learned that many of us want to help out. But I've also learned that we're not sure about how to best lend our support.
We design technologies for a living, and may have spent years working with partners to do research and develop innovative tools: We have some skills to share. But at the same time, we must be conscious that even when we have good and longstanding relationships with these organizations, we are not party to their crisis meetings and strategies, nor are we knowledgeable about their existing contingency planning, and we only have partial views of their full service delivery.
One way of helping is very simple but may be difficult to fully come to terms with: We should pause our in-person fieldwork. Many of us have already done this. Just like us researchers, our partners will be upset about the projects that have had to be paused. Especially when we work closely with partners in participatory ways, the projects are often as valuable to them as they are to us; they care about them as much as we do. At least in my case, this meant learning how to balance conversations between managing immediate crisis needs and looking toward the time after the crisis.
Earlier this week, I talked to one of my collaborators who is in a managerial role of a national charity that supports women and children. This meeting had been organized to discuss an ongoing project before the crisis hit the U.K., but we decided to keep the time in our calendars even though the project was now on hold. As we were about to end the meeting, I repeated that she should simply send anything that needed doing my way. Unexpectedly, she said that conversations like the one we had just had were very helpful. She then also thanked me for an invitation I had tentatively sent her on Twitter for a 20-minute session organized by fempower.tech to find our inner calm. She had joined in the facilitated meditation session that morning, providing her a less anxious start to the day.
The video chat she had had with me and the 20 minutes of guided meditation were a starkly different kind of meeting to the other daily crisis updates, calls, and chats she has with the staff she supports. They were a relief from the anxiety that is caused by this ongoing pandemic. So maybe it is our role as researchers and designers not to “do” too much, but to acknowledge our partners' human needs and to support them in unconventional ways.
So what can we actually do to help out? We can offer our help. But when we do this, we have to acknowledge this is an unprecedented situation and that our partners may already be overwhelmed by offers of support, or may not know how our skills could be useful. As researchers, we have to understand this and weigh up how supportive our offers of support actually are. This may also be our opportunity to pause our “research” thoughts and instead enact solidarity and help out in other ways that are suitable.
We can offer up our expertise without overpromising. And we can offer to help by talking through potential digital responses to the developing needs and concerns they are facing. But simultaneously, we must not overburden staff who are already working beyond their normal workloads while assimilating to working from home, caring for their own families and loved ones, and coming to terms with the anxiety and grief due to the pandemic.
When offering support, providing examples of what we could do to help out can be immensely useful. This might include things such as writing up and designing materials, documenting practices to share as models of good practice, or being a sounding board for staff to figure out how to work with the technologies that are available to them. Before doing this, it's important to read any public response to the crisis the partner may have put out to tailor our offers. We should be looking to reduce pressures on services and think of ways we can use our research skills and provide clear avenues for our support.
At the same time, though, we also have to acknowledge and appreciate that our own workplaces will have certain expectations of us, and that our funders have relatively strict timelines. While some universities are supporting their staff during this transition, many seem to be expecting the impossible of their staff and students: to continue as normal. Sadly, as is so often the case in academia, the research disruptions I talk about here and that COVID-19 is causing will have the greatest effects on early career researchers, precariously employed staff, and Ph.D. students' job and career prospects. We must find ways to support those most vulnerable in our workplaces informally but also through institutional support.
To work toward building understanding, we can build community. HCI is an interdisciplinary field-one that lends itself greatly to building community across disciplines, forms of expertise, and types of organizations. We must remember that nobody knows what the “right” way of going about working in this crisis is, because nobody has been through such a situation before. But we can still learn from previous experiences. We can ask one another for help, and should be sharing resources and information on things we can do to help. Some starting points might be the Rapid Response Research Toolkit, this Google Doc on how to do fieldwork during a pandemic, or this more specific document for HCI researchers.
Having conversations among ourselves as researchers to share experiences and ideas has been helpful for me as well to figure out how to best get in touch with those who have shared so much time with me over the years of doing research together. Formalized formats, such as the weekly webinars NORTHLab at Northumbria University in the U.K. have been hosting, are incredibly useful in learning more about all the different ways in which research can be useful in this time of uncertainty. But much more informal conversations I have had during the weekly digital tea breaks fempower.tech have been organizing on a Thursday, or even scheduled chats I've had with colleagues, have also been immensely helpful for me to get to grips with what is going on, which has ultimately helped me make decisions about how I am able to lend a hand.
Coming back to my conversations with partners, perhaps another way of helping out might be the simple act of inviting our collaborators to social virtual hangouts. To encourage, promote, and enact compassionate and proactive solidarity; to offer a virtual cup of tea to them, just like they have offered so many physical cups of tea to ourselves and those whom they support.
Posted in:
Covid-19 on Tue, April 07, 2020 - 12:04:11
Angelika Strohmayer
Angelika Strohmayer is an interdisciplinary technology researcher, working closely with third sector organizations and other stakeholders to creatively integrate digital technologies in service delivery and advocacy work. She aims to work collaboratively on in-the-world projects that engage people at all stages of the research process to engender change toward a more just world.
[email protected]
View All Angelika Strohmayer's Posts
Feeding the futures of human-food interaction
Authors:
Marketa Dolejsova,
Hilary Davis,
Ferran Altarriba Bertran,
Danielle Wilde
Posted: Fri, April 03, 2020 - 3:59:48
Human-food interaction (HFI) is a burgeoning research area that traverses multiple HCI disciplines and draws on diverse methods and approaches to bring focus to the interplay between humans, food, and technology. Recent years have seen an increase in technology products and services designed for human-food practices. Examples include June, an oven with integrated HD cameras and a WiFi connection to enable remote-controlled cooking; PlantJammer, an AI-based recipe recommender that suggests “surprising” vegan dishes from leftovers; HAPIfork, which vibrates and blinks when you eat too quickly; and DNAfit, a service that uses consumer genome data to suggest a personalized diet. Such technologies turn everyday food practices into data-driven events that can be tracked, quantified, and managed online. Often wrapped in techno-optimism, they propose what seem like straightforward solutions for diverse food problems: from everyday challenges with cooking, shopping, and dieting to systemic issues of malnutrition and unsustainable food production. While on the one hand offering visions of more efficient food futures, this techno-deterministic approach to human-food practices presents risks to individual consumers as well as food systems at large. Such risks include the uncertain safety of food products “created” by algorithms, the limited security of personal, often sensitive, health-related data shared via food-tech services, and the possible negative impacts of automation on social food practices and traditions. Smart recipe recommenders, for example, can suggest ingredient combinations that are surprising but unsafe to eat. Personal genomic data shared via DNA-diet personalization services might get misused by third parties such as healthcare insurance companies. Smart utensils might improve one’s diet but they can also disturb shared mealtimes.
Every innovation brings the potential for new problems. Yet these concerns receive only peripheral attention in HFI literature. To date, HFI projects that propose to fix, speed up, ease, or otherwise make interactions with food more efficient far outweigh those reflecting upon the broader, and often challenging, social circumstances of food-tech innovation [1]. We believe the field of HFI would benefit from more critical engagement with the social, cultural, environmental, and political implications of augmenting food practices with technology. Motivated by concerns about the opportunities and challenges in food-technology innovation, we formed an HFI community network, Feeding Food Futures, to focus on this issue. Within the network, we undertake research, develop theory, and organize events and conference workshops to consider desirable future directions of the field. In this article, we focus on three selected workshops to discuss the opportunities and challenges that technology brings to the table, and propose possible responses in HFI design and research.
HFI Workshops: A Conduit for Critical Debate
We report our observations from three HFI workshops, undertaken at CHI, Montreal; DIS, Hong Kong; and DIS, San Diego [2,3,4,5]. These workshops brought together diverse participants to reflect on HFI issues through hands-on, performative activities and group discussion. At all workshops, food was both primary theme and edible material, enabling practice-based, sensory-rich reflection. Activities involved collaborative crafting and tasting as well as foraging and speculating, supported by various food design props and kits including: Food Tarot cards to provoke future food imaginaries and the HFI Lit Review App to search and categorize the corpus of existing HFI research publications. We also brought, bought, and found various food-related boundary objects and made artifacts in-situ, including a DIY paper constructed from foraged food waste and personalized menus customized to DNA test results. Each workshop gathered participants of diverse backgrounds and used a variety of HFI methods and techniques: ethnographic research in food communities, experimental food design, food-oriented performance art, DIY food biohacking, as well as the prototyping of practical food-tech products. These diverse, at times contrasting, approaches to HFI supported polarized, friendly debates about the desired role of digital technology in food cultures and the contributions we might expect from HFI research. We highlight six debates that stood out in our workshops, and whose importance was recognized by participants. These debates highlight important issues within HFI and can help us to springboard desirable future developments of the field.
Debate #1: Human agency vs. technological efficiency
At CHI 2018, participants introduced an AI-based recipe recommender that suggests ingredient combinations using deep learning and a recipe/image embedding technique [6]. The recommender aims to enhance human gastronomic creativity by suggesting surprising and spectacular recipes. It can suggest extravagant flavor combinations—supposedly beyond the limit of human imagination—as a provocative starting point for food experimentation. For some participants, such data-driven automation presents an opportunity for exciting culinary practices. Others suggested a fine line between enhancing and reducing, even removing, users’ creative engagement with food preparation. They suggested that using the recommender puts people on autopilot, as the algorithm becomes the chef. While gains are made in efficiency, the playful, creative, educational, and sensory elements of cooking may be lost. The debate foregrounded the need to maintain a careful balance between human agency and technological efficiency in AI-based food experiences.
At DIS 2018, participants cooked pancakes in two ways: 1) using PancakeBot, a machine that prints pancakes based on drawings made in custom software; and 2) using a stove and a frypan (Figure 1). With PancakeBot, the cooks’ engagement was mediated and constrained largely to preparation: They imagined, then drew a shape on rudimentary software, positioned a plunger containing the batter, and hit a virtual button to begin the “fully automated” pancake-making process; then they monitored the performance of the robotic chef. In contrast, using the frypan required embodied, in-the-moment, improvisational engagement with food materials and cooking equipment. The cooks often overlooked the fact that their activities were mediated through tools; rather, they seemed guided by their senses. Reflecting on these differences, we determined that future digital cooking technologies should include mechanisms to enable human intervention in emergent, improvisational, and embodied ways that go beyond simply instructing a machine to prepare food.
![](/images/uploads/blogs/Figure1_forWeb.jpg)
Figure 1. PancakeBot.
Debate #2: Digital technology vs. the materiality of food
With PancakeBot, the machine’s affordances shifted the way we engaged with the organoleptic qualities of the pancakes—the taste, color, odor, texture and other qualities inherent to the materiality of food and its ability to stimulate our senses. The machine’s functionality oriented our attention toward the visual aesthetics of our anticipated pancakes, to the detriment of all else. Excited by what the technology seemed to promise, we adapted the pancake-batter recipe to achieve better flow through the machine’s extrusion mechanisms, disregarding the impact of this adjustment on taste. Further, the spectacular PancakeBot influenced our activities when cooking pancakes using the frypan: After using PancakeBot, we began to prioritize the shape of our pancakes in the pan, irrespective of the impact on taste. This shift to privilege visual aesthetics over other organoleptic qualities demonstrates a key risk of inserting digital technology into material practice. Depending on how technologies are designed, they can cause users to disregard important sensual qualities that sit at the core of material cultures. On recognizing this tendency, we determined that digital enhancements should add to, rather than substitute for, the material richness of food experiences and privilege the complex, sensual nature of food and eating over technological capabilities.
Debate #3: New and clean vs. old and messy
At CHI 2018, one participant presented “DIY Kit for Supermarket ‘Chateaus’ of Hybrid Wines” [7], a wine-fermentation kit that enables users to make wine from table grapes as an alternative to standardized off-the-shelf wines (Figure 2). The kit provoked discussion about the contrast between “new” food technologies aiming for “clean” food practices and “old” traditional food techniques, such as wild fermentation, that support “messy” practices and experimental human-food entanglements. While we agreed that food-tech designers need to ensure the safety of their designs, we also recognized the need to support and revive messy, experimental, and playful approaches to food and wine making. We felt new food technologies should nurture consumers’ curiosity for experimentation and prompt rediscovery of traditional and diverse food knowledge, rather than merely supporting efficiency and safety through standardization.
![](/images/uploads/blogs/Figure2_forWeb.jpg)
Figure 2. DIY wine kit.
At DIS 2018, a rich discussion ensued about how one’s grandmother’s pasta recipe is much more than a combination of ingredients—it has cultural, social, and emotional meaning. As demonstrated by Mother’s Hand Taste (Son-Mat) by Jiwon Woo—an artwork that examines how the bacteria on a person’s hands impacts the flavor of their cooking—human-food practices can also be a collaboration with bacteria. Such collaboration is expressed in commonly recognized ways (e.g., making cheese, yogurt, wine, kimchi, and other fermented foods) and in perhaps more subtle ways (e.g., recognizing the cook’s microbiota as an impactful ingredient—what Koreans call the “hand taste” of the food). At DIS 2018, we used Steven Reiss’s “16 desires” [8] as a roadmap to investigate the values that shape our food choices and experiences. We recognized that desires are complex and nuanced, whereas engagement with desires through food-tech is often simplistic. This realization exposed a critical limitation of food technologies: It is not only the material messiness of food that should be taken into consideration when designing food technologies, but also the conceptual messiness and complex idiosyncratic nature of desire that must be considered.
Debate #4: Techno-determinism vs. contextual sensitivities
Food standardization and simplification was further unpacked at DIS 2019, through a DNA-based menu—a dinner for two—introduced as a boundary object. Gene-based personalized nutrition (PN) is a growing trend that involves diet customization using personal genotypic data. The dinner for two was based on DNA test results obtained via a commercial genetic testing service. The meals contained the same ingredients in different proportions—adjusted according to DNA predispositions. Diner 1’s DNA test suggested an increased risk for type-2 diabetes. Therefore, she had a smaller portion of potatoes and plain blueberries instead of blueberry cake for dessert, although she said she would much prefer eating the cake. Her test also indicated a higher susceptibility to alcohol addiction. Her wine glass was thus served half-full. In contrast, Diner 2 had an open wine tap, as her DNA showed low risk of alcoholism. This provocative menu initiated a debate about the determinism of PN, wherein supposedly precise biodata is given precedence over taste preferences, or social and cultural background. Participants highlighted potential negative impacts on social food traditions, asking: Will we still eat together when our food practices are personalized and mutually incompatible? Further, personal and sensitive data exposed by the menu raised questions about privacy and misuse of data by third parties, such as health insurers or pharmaceutical companies. This example highlights the need for HFI to carefully consider the impact of technological interventions on the varied contextual sensitivities embedded in food practices.
Debate #5: Human-food interactions are socioculturally situated
At CHI 2018, we mapped food-tech trends onto the Montreal foodscape using our Food Tarot tool: a card deck presenting 22 imagined diet tribes to illustrate emerging food-tech practices (Figure 3). For instance, the tribe of Petri Dishers only eat in-vitro grown meat; Datavores are Quantified Self aficionados who eat according to self-tracking data.
![](/images/uploads/blogs/Figure3_forWeb.jpg)
Figure 3. Food Tarot cards.
Inspired by the Food Tarot, local participants mapped the cards with local venues—for example, Gut Gardeners with a local fermentation workshop and Food NeoPunks with the Loop Juice shop that makes juice from leftover fruits [9]. They went to the selected venues, foraged for paper-based food waste, and crafted a locavore version of Food Tarot cards; then asked the other participants to forage paper waste at CHI and craft a collective CHI food-waste card. While crafting with the foraged trash (Figure 4), we discussed the situated nature of human-food practices, highlighting that such practices cannot be interpreted from a research lab or a design studio but rather should be explored through fieldwork “in the wild.”
![](/images/uploads/blogs/Figure4_forWeb.jpg)
Figure 4. Making a Food Tarot card from CHI paper waste.
Debate #6: Innovation and inequity
At DIS 2019, we conducted a “walkshop” to forage for local food items and dining experiences. The conference location, in the resort area of San Diego SeaWorld, provided a controversial context for this exercise: Local dining options involved expensive hotel restaurants or fast-food chains and pizza parlors. While eating pizza and sipping soda from gigantic plastic cups, we talked about unequal socioeconomic access to “good” (healthy, sustainable) food products and food technologies designed to support “good” food practices (Figure 5). We acknowledged our privilege, having “the luxury of choice” in this extreme food landscape. Many food-tech products are available only to people from privileged socioeconomic backgrounds. Non-access due to income, education, or digital illiteracy can marginalize individuals and social groups. Food-tech innovation thus risks extending existing, and creating new, socioeconomic inequalities.
![](/images/uploads/blogs/Figure5_forWeb.jpg)
Figure 5. HFI walkshop in San Diego.
Thoughts on the Future of HFI Design and Research
During the DIS 2019 walkshop, participants documented their experiences through written notes, sketches, photos, and found or bought food items and we collaboratively crafted an HFI Zine (Figure 6). The zine has been extended by the workshop organizers to serve as a condensed summary of discussions and debates drawn from all three workshops. Not a manifesto or a fixed set of guidelines but rather a humble set of ideas, the zine offers suggestions on what HFI design and research could do in the future. For example:
To leverage and support food as a creative, material, equitable, and situated sociocultural practice, HFI should:
- Open spaces for experimentation and learning, rather than deliver quick-fix solutions catering to consumers’ convenience
- Support broad socio-economic access and help reduce inequalities on the global food market rather than expanding them
- Take place ‘in the wild’ to reflect contextual sensitivities of local food cultures and practices
- Reflect the socio-culturally diverse, complex, sensory-rich nature of human-food practices, rather than prioritising standardisation by default
- Be culturally diverse
- Complement existing food traditions, rather than replace them
- Keep consumers’ active and creative engagements central to food-tech practices
- Be mindful of consumer privacy
- Be concerned with local human-food-tech interactions, as well as systemic, larger-scale implications of food-tech innovation.
![](/images/uploads/blogs/Figure6_forWeb.jpg)
Figure 6. HFI Zine.
Feeding Food Futures: Community and Critical Discussion
The workshops discussed here illustrate how new food technologies can have ambivalent impacts on food cultures. Food-tech products and services designed to improve practices and solve problems often create new problems of their own. This is not to say that we should strive for technology-free food futures. Food-tech innovation can make important and necessary contributions to food culture; technology can be incorporated meaningfully into existing social practices. With environmental uncertainty and public health crises, food practices need to change and technological innovation can facilitate the needed changes. However, we must be mindful of the challenges, and carefully consider the sociocultural impacts that food-tech innovation may have on food-related practices.
The debates we highlight here reflect important issues that we believe HFI researchers should consider as food-tech innovation moves forward. We recognize continuity in the conversations we initiated within the workshops, and acknowledge the importance of increasing the diversity of stakeholders at the table. To that end, we will continue to promote events where such discussions can take place. Our long-term aim is to nurture a multidisciplinary HFI community that considers perspectives of food-oriented researchers, designers, practitioners, and (human and nonhuman) eaters of diverse backgrounds. With these goals in mind, we initiated the Feeding Food Futures (FFF) network to support critical, experimental, in-the-wild HFI research and discussion. Our intention is for FFF to encompass a broad spectrum of HFI-focused initiatives, such as community workshops, an HFI summer school, and further co-authored publications. Critically, we do not see ourselves as having a monopoly on FFF. The success of FFF hinges on involving diverse scholars and practitioners. We will only feed the future of food in sustainable ways if we encompass a broad set of perspectives. We thus extend an invitation to interested others to join us on this journey. We look forward to future collaborations to critically reflect on the future of human-food-technology ecosystems and to collectively shape a vibrant, rich, and diverse foundation for HFI.
Acknowledgments
We thank our workshop co-authors and participants, past, present and future, who are key to shaping FFF and HFI activities and discussions.
Endnotes
1. Altarriba Bertran, F.*, Wilde, D.*, Segura, E.M., Pañella, O.G., León, L.B., Duval, J., and Isbister, K. Chasing play potentials in food culture to inspire technology design. Proc. of the 2019 Annual Symposium on Computer-Human Interaction in Play. (*joint first authorship)
2.Altarriba Bertran, F.*, Jhaveri, S., Lutz, R., Isbister, K., & Wilde, D.* Making sense of human-food interaction. Proc. of the 2019 CHI Conference on Human Factors in Computing Systems. (*joint first authorship)
3. Dolejšová, M., Khot, R.A., Davis, H., Ferdous, H.S., and Quitmeyer, A. Designing recipes for digital food futures. Extended Abstracts of the 2018 Conference on Human Factors in Computing Systems.
4. Dolejšová, M., Altarriba Bertran, F., Wilde, D., and Davis, H. Crafting and tasting issues in everyday human-food interactions. 2019 ACM Conference Companion Publication on Designing Interactive Systems.
5. Vannucci, E., Altarriba Bertran, F., Marshall, J., and Wilde, D. Handmaking food ideals: Crafting the design of future food-related technologies. 2018 ACM Conference Companion Publication on Designing Interactive Systems.
6. Biswas, A., Mawhorter, P., Ofli, F., Marin, J., Weber, I., and Torralba, A. Human-recipe interaction via learned semantic embeddings. Presented at the 2018 CHI Conference Workshop on Designing Recipes for Digital Food Futures.
7. Špačková, I. DIY kit for supermarket “chateaus” of hybrid wines. Presented at the 2018 CHI Conference Workshop on Designing Recipes for Digital Food Futures.
8. Reiss, S. Multifaceted nature of intrinsic motivation: The theory of 16 basic desires. Review of General Psychology 8, 3 (2004),179–193.
9. Doonan, N., Tudge, P., and Szanto, D. Digital food cards: YUL forage. Presented at the 2018 CHI Conference Workshop on Designing Recipes for Digital Food Futures.
Posted in:
on Fri, April 03, 2020 - 3:59:48
Marketa Dolejsova
Markéta Dolejšová is a design researcher with background in HCI and food studies. Her work combines experimental and participatory methods to investigate the role of digial technology in food practices and cultures. She is currently a research fellow in New Media Studies at the Charles University in Prague.
[email protected]
View All Marketa Dolejsova's Posts
Hilary Davis
Hilary Davis is an HCI-based, Senior Research Fellow in Melbourne, Australia. Her work investigates the role digital technology plays in people’s work, social activities and home lives. She is interested in digital storytelling, digital cookbooks, and how digital technologies generally, can positively impact on familial relationships at mealtimes.
[email protected]
View All Hilary Davis's Posts
Ferran Altarriba Bertran
Ferran Altarriba Bertran is a playful interaction designer and researcher, and currently a Ph.D. candidate at the University of California Santa Cruz. His research explores how technology could afford increasingly playful ways of engaging in mundane activities, with a focus on food practices.
[email protected]
View All Ferran Altarriba Bertran's Posts
Danielle Wilde
Danielle Wilde is associate professor of embodied design at the University of Southern Denmark. She leads critical participatory design research in food and climate futures and social and ecological sustainability, engaging with wicked problems that cut across disciplines and cultures to rethink practices, policies, and technologies through a bottom-up approach.
[email protected]
View All Danielle Wilde's Posts
Post Comment
@ElizabethHenry (2024 08 21)
RocketPlay’s loyalty program is another excellent way to get value from your gameplay. As you play, you earn loyalty points that can be exchanged for bonus funds, free spins, or other rewards https://rocketplay.co.nz/. The more you play, the higher you climb in the loyalty tiers, unlocking bigger and better rewards as you go. VIP players can enjoy exclusive offers, personalized bonuses, and even dedicated account managers who can tailor promotions to suit their preferences. To stay informed about the latest promotions, it’s a good idea to subscribe to RocketPlay’s newsletter or enable notifications if you’re playing on mobile. This way, you won’t miss out on limited-time offers or special events. Additionally, regularly checking the promotions page on the RocketPlay website ensures that you’re always up-to-date with the current deals.
@luciferdonghua (2024 08 22)
The info provided is appropriate and I want to Thank you for posting this. https://luciferdonghua.com.co/ is a Chinese Donghua website. Watch free donghuas here.