ACM Interactions

This article describes what was done to run a virtual Doctoral Consortium at CHI 2020. The event was originally planned as an in-person, two-day event to take place in Honolulu, Hawaii, on April 25–26, 2020. Because of the Covid-19 pandemic, the decision was taken in March that the CHI conference would not go ahead as a physical event. Most events were canceled; however, the Doctoral Consortium continued as a self-contained track and was swiftly reorganized to successfully run as a virtual event on April 28.

Here, we describe the key steps involved in organizing the Doctoral Consortium as a virtual event, from the technology involved to how to schedule activities. Given the expected long-term impact of the Covid-19 pandemic on conferences for the coming year, we hope that this article provides useful information for the organizers of similar events.

Why Doctoral Consortiums are Important and Must Continue

Doctoral consortiums are valuable and important community-forming events for Ph.D. students. CHI’s Doctoral Consortium format and goals are typical: Students apply by submitting application materials to help the chairs select a diverse group of students to be invited to attend. Care is taken to select students from different institutions, and during the conference, a two-day workshop takes place in which students present their work and give feedback to each other.

One of the most important features of a Doctoral Consortium is that students develop and establish their network of peers from the broader international research community beyond their immediate university lab. The connections made during a Doctoral Consortium often last, as students go on to become established members of their research community and continue to cross paths and collaborate over the years. This kind of community-building activity is very important and must continue, despite the challenges of running virtual events during the current pandemic.

Initial Planning for CHI 2020 Doctoral Consortium

The Doctoral Consortium track at CHI is extremely competitive. This year, 85 students applied and 20 were accepted. Students were required to submit a proposal describing their research along with contextual information about their Ph.D. program (full details on the call are available here: https://chi2020.acm.org/authors/doctoral-consortium/). The selection process considered both the quality of the research as well as additional factors to identify a set of students that were diverse in terms of backgrounds and topics.

By February, 20 students had been selected to participate in the Doctoral Consortium and external funds had been secured to help support travel expenses to attend the conference in person. In addition to the three co-chairs, three mentors had also been invited to join the consortium to give advice and feedback to students. These mentors were invited fairly late in the process, throughout January and February, and were selected from respected members of the CHI research community. As outlined below, mentors play a key role in supporting and facilitating a smoothly running Doctoral Consortium, even more so when it comes to running this as a virtual event.

Transitioning to a Virtual Doctoral Consortium

The decision to run the Doctoral Consortium as a virtual event was made only six weeks before the event. While we did have a list of students who had been accepted, there was still much to do.

The first priority was to identify whether the students who applied still had the capacity and interest to take part in a virtual event. At this time, many countries were beginning to issue mandatory lockdown and stay-at-home orders. The sudden and unexpected transition was challenging for many, and it was unclear whether the students would be able to participate. There were also secondary concerns, such as identifying the time zone that participants would be able to join and what kind of activities they were most interested in. To answer these initial questions, a brief survey was constructed and sent to participants at the end of March.

Working Across Time Zones

Participants were spread across a wide range of time zones (e.g., Oceania, Asia, Europe, and North America). One of the first major decisions was to decide how long the event should be and at what time it should run. The two main options considered were: 1) having one longer event with core overlapping hours, and 2) having two separate shorter events. Our thinking on this decision was informed by reflections from Simon Buckingham Shum on organizing the Doctoral Consortium at LAK ’20 [1]. We decided to split our Doctoral Consortium into two separate four-hour events:

• DC-A: Tuesday, April 28, 2020, 9:00–13:00 UTC
• DC-B: Tuesday, April 28, 2020, 16:00–20:00 UTC

Participants naturally clustered into two groups of 10 based on their preferred meeting time. It’s worth highlighting here that people’s preferences were not based simply on their local time zone. Instead, participants joined the group that suited them best: Some preferred to get up early, while others preferred to stay up late.

A further benefit of running the event as two separate short events meant that each meeting had fewer participants, potentially making communication in the virtual meeting more manageable. Of course, a downside of this decision is that the entire group of participants never came together as one as they would have during an in-person conference.

Scheduling Activities

The next priority was to establish a basic schedule for activities during the two four-hour sessions. Each session had an identical template structure:

• Welcome and introductions (10 min)
• Talk 1 (7 min)
• Talk 2 (7 min)
• Breakout group discussion: two rooms, one for each speaker (15 mins)
• Talk 3 (7 min)
• Talk 4 (7 min)
• Breakout group discussion: two rooms, one for each speaker (15 mins)
• Break (10 min)
• Talk 5 (7 min)
• Talk 6 (7 min)
• Breakout group discussion: two rooms, one for each speaker (15 mins)
• Talk 7 (7 min)
• Talk 8 (7 min)
• Breakout group discussion: two rooms, one for each speaker (15 mins)
• Break (10 min)
• Talk 9 (7 min)
• Talk 10 (7 min)
• Social time (1 hr)

A shared and editable document was sent to participants as soon as possible to help them plan for the meeting. Students were allocated a talk slot in the schedule; this was done in a random order without any special effort given to group students into thematic clusters based on topic. Inevitably there were some requests to switch sessions soon after the initial draft schedule was shared; these requests were all accommodated.

Along with the schedule, all of the students’ extended abstracts were also shared using Google Drive (which had to be done because the papers had not yet appeared in the ACM Digital Library). It was important to share these papers as early as possible, as it gave all of the participants an opportunity to dive deeper and learn about each other’s work prior to the meeting. Finally, speakers were asked to provide a link to slides ahead of the meeting; this worked as a backup should a presenter run into difficulties with screen sharing during their talk (more on that soon).

Welcome and Introductions

The meeting started with a welcome. Every person in the meeting gave a short introduction: name, affiliation, current location and time of day, and a brief overview of their research topic. Introductions were made in a random order based on the layout of participants on the chair’s screen. This meant that mentors and students were interleaved. This welcome event was scheduled to take 10 minutes; however, in both meetings it ran over considerably, taking 20 minutes. It was the only part of the meeting to run over, suggesting that more time should have been allocated. Taking the time for introductions is very important for a successful virtual meeting, as it allows each participant time to speak to the group.

Short Talks

Participants were asked to prepare a five-minute presentation covering three points:

• What’s your research question?
• What work have you done so far?
• What work still needs to be done to complete your Ph.D.?

We opted for short talks, so as to keep down the overall duration of each meeting and to enable longer breakout group discussions. From our personal experience, shorter talks seem to work better in this context, as it’s hard to hold people’s attention online. We considered asking participants to prepare a prerecorded talk, but in the end decided against it due to concerns that it would place too many extra demands on participants ahead of the meeting. In the end, all of the Doctoral Consortium participants were able to give strong and compelling talks about their work within the given time limit.

On the Importance of Mentors

Mentors play a critical role in the smooth running of a Doctoral Consortium; their role is even more vital in a virtual event. Initially three mentors were recruited. But with the event now split in two, more help was clearly needed. Within a couple of days we had successfully recruited a diverse group of 10 mentors who were experts in different areas of HCI research.

Several of the newly appointed mentors were outstanding early-career researchers, who themselves had been members of the CHI Doctoral Consortium over the past few years. There was enthusiasm from this group to give back and also to share advice on how to navigate the steps immediately after completing a Ph.D. Partnering early career researchers with more established senior researchers made for an exciting mix of perspectives on advising the Doctoral Consortium participants.

Technology for Running a Virtual Doctoral Consortium

We relied on four tools to run the event:

• Zoom for hosting the video call, sharing slides, and setting up breakout rooms (see Figure 1 for participants in the two sessions). A paid account was used to enable longer meetings. We used Zoom’s Meeting mode rather than the Webinar mode, as this allowed for more interaction between participants. Participants were encouraged to have their video on with their microphone muted unless they were speaking. Security for virtual meetings is an important concern, and this was reinforced to us by reports of Zoombombing taking place during another CHI-related event the day before our meeting (see Barry Brown’s reflections on how this was handled). We therefore took care to set a password for the meeting and share the link only with meeting participants directly (i.e., we did not put the link on a public website). Finally, it’s worth noting that the role of chairing a virtual meeting is extra demanding, as attention is needed for managing all the tools and ensuring everyone has access. For example, setting up and assigning people to breakout rooms takes time and requires attention; it is difficult to carefully listen to a talk while doing this simultaneously.


• Slack for text communication and sharing notes and other resources. All participants were invited to join a few weeks before the event. Slack was very useful for enabling conversations to occur both before and after the meeting. Several mentors reported that they shared written feedback with students and used this tool to backchannel with students on extended questions. While Zoom enables in-meeting chat, this feature operates only during the meeting and cannot easily be accessed by all participants once the meeting has ended.


• Google Docs for sharing the schedule and links to slides and papers.


• Email for essential communication and reminders. Despite having alternative forms of communication such as Slack, email is still a universal tool for reaching people. Emails were sent by the chair to both students and mentors to share important information (e.g., schedule and meeting link). Reminders were sent one week and 24 hours before the meeting.

Figure 1. Participants in the two sessions of the virtual Doctoral Consortium at CHI 2020.

We’re All Working from Home Now

One of the requirements for taking part in a virtual meeting is that participants have access to a good, stable Internet connection and their own computers with a camera and a microphone. However, there are well-documented issues with speed inequities between different societal groups and locations. During the meeting, some participants did experience issues with poor Internet speeds and had to work around this by turning off their camera or by stopping screen sharing. We planned for such events by asking all speakers to share a copy of their slides ahead of time in case screen sharing became impossible.

While participants should ideally find a quiet space, free from interruption, many live in homes with others and have to work in shared living spaces, where background interruptions and distractions are likely to occur. Everyone appreciates a level of understanding on these issues.

Accessibility

There are well-established arrangements in place to provide accessibility services at physical conferences; these must be retained during the transition to virtual events. Our event included sign language interpretation; one participant coordinated with the interpreters and shared meeting details ahead of the event. Interpreters require frequent breaks, so they often work in small teams and switch throughout the meeting. Special attention is needed when assigning breakout rooms to make sure that the interpreters are assigned to the same virtual room as the people they are supporting during the meeting.

One thing that we regret not doing is enabling closed captioning throughout the meeting for all participants. Several participants reported that this would have been useful, since it would have created a script that could be consulted when something was missed. In Zoom, an assigned meeting participant can manually add closed captions, or a third-party service can be connected to stream captions. Other meeting platforms, such as Microsoft Teams and Google Meet, have integrated automatic closed-captioning services that may be sufficient for some participants but are not equivalent to having a human transcriptionist.

Don’t Record

Just because you can record a digital event does not mean that you should. Doctoral Consortiums are intended to be safe spaces in which students can give and receive feedback from their peers—private, invitation-only events. In keeping with the spirit of a traditional in-person Doctoral Consortium, the event was not recorded.

Breakout Rooms

The schedule was structured so that there was a pattern of two seven-minute talks followed by a 15-minute breakout room discussion. One breakout room was set (and named for) each of the speakers. Prior to the meeting, two mentors were assigned to each speaker; all other participants were randomly assigned to one of the two groups. Mentor-mentee assignments were shared with all participants several days before the event so that mentors could prepare by reading the extended abstracts of their mentees ahead of time. Mentors gave their preferences for who they would like to work with (usually based on a topicality fit), and these preferences were almost always met.

Mentors played a key role in chairing the breakout-room discussions. Pre-assigning them to students was beneficial for the smooth running of the virtual meeting. The chair of the meeting can set up the breakout rooms—however, once they’re launched, participants leave the main meeting space and go to the virtual breakout rooms. Mentors chaired these breakout meetings and often started the conversation by asking their mentee about the areas where they would most value feedback.

In Zoom, there are options to automatically set the duration of breakout rooms and to set how much warning participants get to wrap up their conversation before being returned to the main meeting space. Using these features kept everything running on time.

Ending on a Social Hour

The meetings ended with an open-ended discussion with all participants. One limitation of the virtual format is that these informal discussions are more difficult to manage than when run at an in-person event. There are challenges in turn-taking associated with virtual meeting tools. The option of breakout rooms was given but not taken up. Instead, an informal Q&A between students and mentors developed.

At conferences, social events usually follow the main meeting (e.g., an evening meal together at a restaurant). In planning the virtual event, several participants said that they would be interested in retaining this social element, for instance, by staying on the call to have a virtual beer together. However, on the day this did not happen. Maybe a four-hour virtual meeting is simply too exhausting?

Conclusion

Doctoral consortiums are important training and networking events for Ph.D. students. They must continue despite the challenges of conducting conferences as virtual events during the Covid-19 pandemic. In this article, we have reflected on our experience of rapidly reorganizing the CHI 2020 Doctoral Consortium to a virtual event over a six-week period. In the end, it was an enjoyable and successful event that retained many of the core features of an in-person Doctoral Consortium. Time will tell whether the longer-term objective of helping this cohort of students develop those deep and long-lasting connections was achieved.

List of Participants

Co-Chairs:

• Duncan Brumby, University College London
• Koji Yatani, University of Tokyo
• Leah Findlater, University of Washington

Mentors:

• Ed Cutrell, Microsoft Research
• Anna Feit, ETH Zurich
• Simone Kriglstein, University of Vienna
• Neha Kumar, Georgia Institute of Technology
• Kai Kunze, Keio University
• Shaun Lawson, Northumbria University
• Pedro Lopes, University of Chicago
• Bektur Ryskeldiev, University of Tsukuba
• Katta Spiel, Vienna University of Technology
• Maria Wolters, University of Edinburgh

Students:

• Rahul Arora, University of Toronto. Creative Expression with Immersive 3D Interactions; https://doi.org/10.1145/3334480.3375028 
• Manshul Belani, IIIT-Delhi. Evaluating Virtual Reality as a Medium for Vocational Skill Training; https://doi.org/10.1145/3334480.3375027
• Fabio Catania. Politecnico di Milano. Conversational Technology and Natural Language Visualization for Children’s Learning; https://doi.org/10.1145/3334480.3375041
• Samantha W. T. Chan, The University of Auckland. Biosignal-Sensitive Memory Improvement and Support Systems; https://doi.org/10.1145/3334480.3375031
• Ana Paula Chaves, Northern Arizona University. Should my Chatbot be Register-Specific? Designing Appropriate Utterances for Tourism; https://doi.org/10.1145/3334480.3375033
• Amy Yo Sue Chen, Simon Fraser University. Giving Form to Temporality: Extending Design Practices and Methodologies for “Slow” Interaction; https://doi.org/10.1145/3334480.3375024
• Mayara Costa Figueiredo, University of California, Irvine. Self-Tracking for Fertility Care: A Holistic Approach; https://doi.org/10.1145/3334480.3375029
• Alexander Eiselmayer, University of Zurich. Supporting the Design and Analysis of HCI Experiments; https://doi.org/10.1145/3334480.3375038
• Yasmin Felberbaum, University of Haifa. Enhancing Physical Activity of Walking and Dual-Tasking among Older Adults Using Personalized Navigation Practice; https://doi.org/10.1145/3334480.3375034
• Rakibul Hasan, Indiana University. Reducing Privacy Risks in the Context of Sharing Photos Online; https://doi.org/10.1145/3334480.3375040
• Jialun Aaron Jiang, University of Colorado Boulder. Identifying and Addressing Design and Policy Challenges in Online Content Moderation; https://doi.org/10.1145/3334480.3375030
• Awais Hameed Khan, The University of Queensland. Participatory Design Tools: Exploring the Value of Design through Materiality; https://doi.org/10.1145/3334480.3375023
• Doris Jung-Lin Lee, University of California, Berkeley. Towards an Integrated Solution for Intelligent Visual Data Discovery; https://doi.org/10.1145/3334480.3375035
• Matthew Seita, Rochester Institute of Technology. Designing Automatic Speech Recognition Technologies to Improve Accessibility for Deaf and Hard-of-Hearing People in Small Group Meetings; https://doi.org/10.1145/3334480.3375039
• William Seymour, University of Oxford. A Design Philosophy for Agents in the Smart Home; https://doi.org/10.1145/3334480.3375032
• Carlos E. Tejada, University of Copenhagen. Print-and-Play: 3D-printed Interactive Objects Without Assembly or Calibration; https://doi.org/10.1145/3334480.3375025
• Milka Trajkova. Indiana University, Indianapolis. Designing AI-Based Feedback for Ballet Learning; https://doi.org/10.1145/3334480.3375036
• Yan Wang, RMIT University. Understanding the Design of Playful Gustosonic Experiences; https://doi.org/10.1145/3334480.3375022
• Daricia Wilkinson, Clemson University. All Experiences Matter: Designing Safety Tools for Vulnerable Groups in the Caribbean; https://doi.org/10.1145/3334480.3375037
• Mohan Zalake, University of Florida. Advisor: Agent-based Intervention Leveraging Individual Differences to Support Mental Wellbeing of College Students; https://doi.org/10.1145/3334480.3375026

Duncan Brumby

Koji Yatani

Leah Findlater