Jana Fedtke, Mohammed Ibahrine, Yuting Wang, Bouziane Zaid
Social networks, search engines, and analytics through algorithms have gradually become a part of our digital existence. As people navigate the digital world in their work lives and private spheres, more and more data is collected, stored, and analyzed. This increasing datafication has become a concern with regard to issues of citizens' rights, consumer privacy, and data ethics.
In this article, we explore how the Netflix docudrama The Social Dilemma (2020)  positions such ethical concerns surrounding data politics in the context of what Shoshana Zuboff calls surveillance capitalism , an economic system that capitalizes on humans and their data for commercial purposes. We further examine how the film presents the inherent power hierarchies that give a few tech designers the power to control and manipulate the feelings and choices of large segments of the consumers who use their products. As its title indicates, The Social Dilemma critiques the dramatic shift from social networks being used to connect people to the "social dilemma" of these networks being misused to abuse people. In this sense, users become "just zombies for their profit," as Sandy Parakilas  of the Center for Humane Technology laments in the film .
The Social Dilemma provides a compelling narrative, with interviews with industry insiders and former executives that highlight the manipulative nature of social media and the questionable business practices behind these platforms and social networks. It is particularly effective in its combination of interviews and commentaries with fictionalized scenes that depict the experiences of teenagers and families managing screen-based addictive behaviors. The narrative also provides various examples of fake news, and of the connection between young people's mental health and their social media use. Additionally, The Social Dilemma includes examples of the political fallout caused by social media campaigns that promote fake news and encourage people to act irrationally. The film addresses the moral responsibility that social networks and digital platforms have, highlighting the ethical concerns that have emerged over the past few years and providing a call to action for platforms and networks to rethink their business models in light of screen-based addictive behaviors and fake news, among other issues.
Here, we delineate the critique of social media and other platforms that The Social Dilemma advances. We also consider suggested reforms that would begin to privilege the users over the companies' interests in what some scholars have called data justice. In an effort to understand datafication better and transform society into a more just playing field of various actors, technology might be designed to serve people rather than exploit their vulnerabilities.
The Social Dilemma features interviews with tech designers and former executives at giants such as Google, Facebook, YouTube, Twitter, Apple, Instagram, TikTok, Snapchat, and Pinterest. Interviewees include Tristan Harris, Jaron Lanier, Roger McNamee, Sandy Parakilas, Chamath Palihapitiya, Sean Parker, Randima Fernando, Cathy O'Neil, and Bailey Richardson. Virtually all of them comment on the manipulative nature of the design and architecture of social networks and the rise of a platform economy. Their comments establish a critique of algorithm-driven politics on these networks as they have morphed from tools to find a friend or donor into manipulative platforms that peddle fake news and other addictive content.
A customer's attention has become the prime currency. How long can the platform keep someone engaged and how can this attention and time be monetized for the purposes of programmatic and behavioral advertising? As Tristan Harris points out, "There's the engagement goal: to drive up your usage, to keep you scrolling. There's the growth goal: to keep you coming back and inviting as many friends and getting them to invite more friends. And then there's the advertising goal: to make sure that, as all that's happening, we're making as much money as possible from advertising" . Harris repeats an often-cited saying: "If you're not paying for the product, then you're the product" . Shoshana Zuboff might disagree with this sentiment, as she argues that the product is our behavioral analytics, not the customers themselves . Similarly, Jaron Lanier puts the focus on people's behavior: "It's the gradual, slight, imperceptible change in your own behavior and perception that is the product" . Lanier criticizes the manipulative nature of these contemporary business practices: "We've put deceit and sneakiness at the absolute center of everything we do" . These quotes illustrate the negative connotations that some tech designers now associate with the practices of social networks.
In the age of attention economics, where human attention is considered both a scarce commodity and a valuable resource, technology architects intentionally design addictive user interfaces to trick people into spending more time on social media and other platforms, where they then become Parakilas's metaphorical zombies. These practices affect people's real-world behavior and emotions, as Sean Parker highlights when he says that these designs are "exploiting a vulnerability in human psychology" . Along similar lines, Randima Fernando points to the unfair fight of machine against human being. He addresses the exponential growth in computer processing power while "our physiology, our brains, have evolved not at all" . Roger McNamee illustrates this unequal position between humans and Als: "You're effectively playing against this artificial intelligence that knows everything about you, can anticipate your next move, and you know literally nothing about it, except that there are cat videos and birthdays on it. That's not a fair fight" .
A customer's attention has become the prime currency.
While The Social Dilemma criticizes important issues in contemporary data politics, it does so from a one-sided perspective, as it includes only the voices of former executives and excludes current industry representatives. It does not feature activists and data scholars who have long expressed concern with regard to privacy, surveillance, and the potential abuse of personal data. From among the several businesses that stand accused of questionable business practices, only Facebook  has responded to some of the claims in The Social Dilemma. The company claims that the docudrama "buries the substance in sensationalism. Rather than offer a nuanced look at technology, it gives a distorted view of how social media platforms work to create a convenient scapegoat for what are difficult and complex societal problems" . Facebook highlights its own efforts in combating fake news and rejects several claims regarding addiction and customer data being "the product."
Shoshana Zuboff, professor emerita of Harvard Business School, offers her observations on surveillance capitalism in the film. Zuboff shows how businesses want to sell certainty, for which they need predictions, which in turn requires data. Social networks and other digital business models are ideally positioned, she says, as "a marketplace that trades exclusively in human futures…. We now have markets that trade in human futures at scale, and those markets have produced the trillions of dollars that have made the Internet companies the richest companies in the history of humanity" . They are part of what Zuboff calls surveillance capitalism, which she defines in her influential 2019 book as "a new economic order that claims human experience as free raw material for hidden commercial practices of extraction, prediction, and sales" and "a parasitic economic logic in which the production of goods and services is subordinated to a new global architecture of behavioral modification" . According to Zuboff, surveillance capitalism is "the foundational framework of a surveillance economy," which poses a threat to humanity and constitutes "an expropriation of critical human rights that is best understood as a coup from above: an overthrow of the people's sovereignty" .
Social media and other networks follow the logic of surveillance capitalism, which, Zuboff says, "unilaterally claims human experience as free raw material for translation into behavioral data. Although some of these data are applied to product or service improvement, the rest are declared as a proprietary behavioral surplus, fed into advanced manufacturing processes known as 'machine intelligence,' and fabricated into prediction products that anticipate what you will do now, soon, and later. Finally, these prediction products are traded in a new kind of marketplace for behavioral predictions that I call behavioral futures markets" . With reference to Orwell's Big Brother, Zuboff calls the digital apparatus that governs and perpetuates surveillance capitalism Big Other, "the sensate, computational, connected puppet that renders, monitors, computes, and modifies human behavior. Big Other combines these functions of knowing and doing to achieve a pervasive and unprecedented means of behavioral modification" .
Yet how do we counter the traps of data collection, the violation of privacy, and the mechanisms of surveillance capitalism? The business models we have described so far are successful for the companies, so there is no real incentive for them to change their policies. There is no financial incentive, either. Privacy laws in the digital sphere are still in their infancy. At the very end, The Social Dilemma promotes a few simple ways for individuals to resist ever-present surveillance: uninstall apps, choose safe search engines, don't follow recommendations, fact-check information, and limit your screen time. Jason Morgese  suggests three reforms that social media platforms could introduce: ending censorship, not treating users like products, and encouraging connections without clickbait. Current practices indicate that the platforms are not ready to embrace such sweeping changes. Tristan Harris says it will take "massive public pressure" to make the companies change their policies. It remains to be seen whether such changes will happen and whether The Social Dilemma might have contributed to such developments. There is a certain irony that the film was released exclusively on Netflix, a platform that may be leveraging algorithmic manipulation tactics similar to those of social media networks. Ultimately, regulation might be needed to ensure customer rights and to help digital citizens navigate the world of data collection.
Social media and other networks follow the logic of surveillance capitalism.
Maria Petrescu and Anjala Krishen have pointed out that "the regulation of digital information privacy is dependent on national differences and systems and difficult to implement and enforce across jurisdictional boundaries" . The centrality of national interest and the fear of infiltration by foreign nonstate and state actors and agencies also prevent current global collaborations. One recent example of a national attempt at regulating the influence of social platforms and other networks is Australia's efforts to force tech companies such as Facebook and Google to pay for sharing news content.
Similarly, the European Union aims to curb the power of tech giants within its boundaries with the General Data Protection Regulation (GDPR), introduced in 2018. According to Zuboff, "the regulations introduce several key new substantive and procedural features" , including an obligation to notify users when their data is breached and a more precise definition of "consent" to limit the Internet giants' abuse of personal data use. The regulations should also include a ban on making users' information public by default, to design systems with privacy concerns at their core, to give users the right to permanently delete their personal data, and to limit the decision-making power of automated systems .
The approach of the EU strengthens consumer rights and puts the burden of data protection on the tech companies. Data extraction would no longer simply serve the interests of a few; technology would serve the people, guaranteeing individual rights rather than privileging the profits of a few companies. This idea of changing data politics is also evident in what Lina Dencik, Arne Hintz, and Jonathan Cable call data justice . The authors explore attitudes toward digital surveillance, arguing for the necessity of a "(re)conceptualization of resistance to data collection and use that can address the implications of this data-driven form of governance in relation to broader social justice agendas" . In this context, they propose the concept of data justice, which "would help contextualize datafication, connect it to social and economic justice concerns, and thereby contribute to transforming the role of data politics in current civil society practice and, potentially, public debate" . The authors examine data justice predominantly in the context of activism, but it also applies to the future of data collection on social networks and other platforms.
The Social Dilemma does not specify exactly what such regulation would be. The interviewees refer to industries such as telecom and human organ markets in their argument for the need for regulation, but except for one proposal about taxing data collection processing to limit the amount of data the companies collect, they do not propose concrete regulatory measures for social media. In the future, a regulatory model similar to the one many countries have in place for broadcast media might also be applied to the digital context. The Internet giants could operate under some level of oversight similar to that exercised by institutions such as Ofcom (the Office of Communications) in the U.K. Since companies such as Google and Facebook serve billions of people and have a global reach, the question of public interest plays an important role in this context. These companies must be held accountable by institutions that watch out for the public interest on a global level, possibly a mix of intergovernmental and civil society organizations such as UNESCO or the Internet Governance Forum. These institutions could play a role in the formation of a multistakeholder global organization to exercise oversight of the Internet giants.
Modern technology is a product of capitalism, but The Social Dilemma explains how technology has become an ideology with the ability to turn humans into so-called techno-serfs. While some futurists foresee a utopian society based on technological advances, the language and attitudes of many interviewees in The Social Dilemma suggest a more dystopian perspective. It would be ideal to maximize the benefits of technology and minimize its risk potential. The dangers of technology for society are becoming more pronounced and will become even more complex with the imminent convergence of nanotechnology, biotechnology, information technology, and cognitive technologies (NBIC).
For the past decade, social networking sites have been heralded as technologies of freedom and change. As they gradually matured and built their user bases, they have become known for hijacking personal information, invading people's privacy, and controlling how people think and behave. The Social Dilemma sparked a global debate on how to regulate the algorithm-driven analytics of social networking sites. The rapid process of the datafication of democracies, economies, cultures, and people is unstoppable unless a multistakeholder, multifaceted, and global governance framework based on the universal values of fairness, accountability, transparency, and ethics (FATE) can be proposed to achieve data justice. Fairness, accountability, and transparency are the holy grail of design ethics in the algorithmic age.
1. Orlowski, J., director. The Social Dilemma. Netflix, 2020; https://www.humanetech.com/the-social-dilemma
4. Facebook. What 'The Social Dilemma' gets wrong. Facebook, 2020; https://about.fb.com/wp-content/uploads/2020/10/What-The-Social-Dilemma-Gets-Wrong.pdf
5. Morgese, J. 3 reforms social media platforms should make in light of 'The Social Dilemma.' Tech Crunch. Oct. 22, 2020; https://techcrunch.com/2020/10/22/3-reforms-social-media-platforms-should-make-in-light-of-the-social-dilemma/
6. Petrescu, M. and Krishen, A.S. The dilemma of social media algorithms and analytics. Journal of Marketing Analytics 8 (2020), 187–188; https://link.springer.com/article/10.1057/s41270-020-00094-4
7. Dencik, L., Hintz, A., and Cable, J. Towards data justice: Bridging anti-surveillance and social justice activism. In Data Politics: Worlds, Subjects, Rights. D. Bigo, E. Isin, and E. Ruppert, eds. Routledge, 2020, 167–186.
Jana Fedtke's research and teaching interests include transnational literatures with a focus on South Asia; gender studies; and postcolonial literatures. Her work has been published in Online Information Review, Asian Studies, Journalism Practice, South Asian History and Culture, South Asian Review, and Asexualities: Feminist and Queer Perspectives (Routledge). firstname.lastname@example.org
Mohammed Ibahrine is a professor of innovation and entrepreneurship at the American University of Sharjah. The second edition of his book New Media and Neo-Islamism was published in 2012. His main research interests focus on the technologies of the Fourth Industrial Revolution, design thinking, innovation, and entrepreneurship. email@example.com
Yuting Wang is a professor of sociology at the American University of Sharjah. She is the author of Between Islam and the American Dream: An Immigrant Muslim Community in Post-9/11 America (Routledge, 2014) and Chinese in Dubai: Money, Pride, and Soul-Searching (Brill, 2020). She has published widely on Islam and Muslims in China, transnational religious networks, social changes in the Arabian Gulf, the Chinese expatriate community in the UAE, and Sino-UAE relations in the era of Belt and Road. firstname.lastname@example.org
Bouziane Zaid is an associate professor of media and communication at the University of Sharjah. His research interests are in the areas of media policy advocacy, media law, public service media, digital rights, and corporate communication. He has authored and coauthored two books and numerous journal articles, country reports, and book chapters. email@example.com
©2022 ACM 1072-5520/22/03 $15.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2022 ACM, Inc.