ACM Interactions

This blog began in January 2013. There was a quid pro quo: You take the time to read my informal posts on a range of topics, I post observations only after convincing myself that they are viable. So far, only one has not held up, the first, January 2013’s “Wrong about MOOCs?” The third anniversary of the blog and the post is an occasion to review what went wrong.

In 2012, MOOCs made headlines. The concept and acronym for “massive open online course” were around earlier, but this was the year that the Coursera, Udacity, and edX platforms were founded by leading researchers from top universities. Little else was discussed in July 2012 at the biannual Snowbird conference of computer science deans and department chairs. In the opening keynote, Stanford’s President John Hennessy forecast that MOOCs would quickly decimate institutions of higher learning, leaving only a handful of research universities. He announced that by embracing MOOCs, he would see that Stanford was among the survivors.

I was sceptical. Completion rates were low. Institutions don’t change quickly. But by the time I wrote my first blog post several months later, I had drunk the Kool-Aid. Why?

There was a fateful roulette wheel spin. I randomly selected a MOOC to examine, Charles Severance’s "Internet History, Technology, and Security." I was impressed—and assumed it was typical. Yes, completion rates were low, but as we learned with the Internet and Web, attrition doesn’t matter at all when growth is exponential, and the potential for collecting data and sharing practices made contagion seem promising. The problem was that I stopped with Severance’s course. Had I examined others, I would have found that he was exceptional: experienced, talented, and dedicated. I over-generalized.

The student demographic data presented in Severance’s final lecture might have raised a flag. Most were college graduates. I thought, “Well, history is more interesting to people who lived through it.” But MOOCs still attract people who finished formal school studies. They have not made the strong inroads into undergraduate education that Hennessy and many others, including me, expected.

I overlooked changes in the undergraduate experience since my student days decades earlier. Many of us had arrived in college less informed about career possibilities. We were exploring. More students today arrive with specific career paths, no doubt for good reason. More of them work as they study. Focusing on university requirements, they may complain about the number of required courses and not volunteer to take additional online courses.

And high schools! I thought that secondary school students would take MOOCs in favored topics and outperform college or university students, transforming their image of higher education and forcing change. This was wrong for several reasons. With few undergraduate role models in MOOCs, comparisons aren’t possible. More to the point, I went to school before AP courses existed. At that time a MOOC could have been a godsend. Today, ambitious high school students pile on AP courses and take classes at community colleges. The existing system isn’t perfect, but it won’t change quickly, and it leaves no time for MOOCs.

I concluded the essay by looking ahead nine months: “The major MOOC platforms launched in 2012 claim a few million students, but … if we count students who do the first assignment and are still participating after a week, as we do in traditional courses, enrollment is much lower... The 2013-2014 academic year will provide a sense of how this will develop… The novelty effect will be gone. Better practices will have been promulgated. If there are fewer than 10 million students, the sceptics were right. More than a hundred million? Those who haven’t yet thought hard about this will wish they had.”

How did it turn out? About 10 million at the end of 2013 and 20 million at the end of 2015. This is fine growth, but it appears to be linear. Therefore, overly generous attendance criteria and low completion rates matter. MOOCs are primarily supplemental education, not replacement. Hennessy retracted his forecast. The university administrator panic of 2012 subsided.

Cheating turned out to be more of an issue than I expected. I thought that applicants listing MOOCs would be more carefully screened to confirm that they knew the material. Interviewers could inspect MOOC content or contact instructors. For an applicant to claim knowledge and then have their ignorance exposed in an interview would indicate mendacity or poor scholarship. However, employers would rather other people do the screening, and certification became part of the business model for MOOC providers. It could save prospective employers from having to confirm knowledge, but that provides an incentive to cheat. Careful studies show significant levels of cheating. Some of it is sophisticated. Just as a game player can log on as multiple characters who give accumulated materials to one among them who then quickly becomes powerful, some people log on as multiple students, guessing at test questions to determine the correct answers, which are fed to the real “student” who does well. One person automated this process, acing courses without doing any work at all. A researcher associated with a MOOC provider noted sheepishly that this student showed considerable talent.

Although some MOOC platform founders have moved on, their companies are active, and other large-scale online education efforts have surfaced. A sustainable niche has formed. Research and applied experimentation improves practice; innovative instructors are advancing the medium. Growth may be gradual, with existing institutions accommodating rather than being disrupted by the changes.

MOOCs as a context for research

MOOCs are a great setting for experimental research. With hundreds or thousands of participants, students can often be randomly assigned to different conditions with ease and no ethical downsides. Small-group projects can employ various criteria for grouping students to try different interventions. Students must agree in advance to participate, but instructors can devise assignments and interventions, all of which might improve performance, and find out which ones do, thus not unfairly disadvantaging anyone. For example, will homogeneous or diverse groups do better? Studies in other settings have found that it can vary based on the nature of the collaboration and the dimensions on which homogeneity or heterogeneity are measured. Such issues can be explored far more easily and rapidly in MOOCs than in similar classroom research of the past. Three years ago I felt that MOOCs would thrive because of the ability to iterate and move forward. I underestimated the workload in preparing a good online course, which may leave little time to add a significant research component. But a research literature is appearing.

Reflections on blogging

At the end of each year, I’ve reflected on the blogging experience. When I began in 2013 I thought I had about a year’s accumulation of thoughts that could be useful to others. The discipline of posting monthly was perfect—forcing me to budget time and developing a habit of noticing an email exchange or news story that could be developed into a suitable post. Each essay has been work, but only occasionally hard work; mostly it has been fun. Had I not committed to a monthly post, the activity would have shifted to a back burner and then fallen off the stovetop altogether, as it did for most of my fellow bloggers. There is not much feedback and reinforcement for blogging, so it has to be intrinsically motivated. When the goal is to be useful for others, this is not encouraging.

I have enjoyed many Interactions blog posts by others and wish there were more. They are not peer-reviewed. This can be a benefit, one can ask friends or respected colleagues to comment, but it makes blogging an expenditure of time that does not figure in widely recognized productivity metrics. For me, it is a way to organize thoughts and consider where I could focus more rigorously in the future. And the posts are sometimes read by some people—you, for example.

Jonathan Grudin