Seeing the Consequences

Why do we need to change the system?

ALPHA VERSION - Updated August 17, 2021

Download As PDF

Objective

Understand the harms the attention economy and persuasive technology have produced, and why those harms are the result of a broken system.

Overview

Persuasive technologies are constantly getting better at taking advantage of our vulnerabilities to keep us engaged and influence our behavior. It’s the result of a race for attention that is creating disastrous consequences for our wellbeing, democracy, and ability to solve our biggest problems.

In this issue guide, you’ll learn:

  • The harms persuasive technology has caused for individuals and communities.
  • How those harms result from a larger system.
  • Why changing our individual behavior isn’t enough.

By understanding the harms of persuasive technology, as well as the systemic forces that influence it, we can start seeing how to build a world built on humane technology that operates for the common good.

Introduction

You’ve learned how the social media technology we use every day is shaped by a race for our attention to keep us engaged and to influence our behavior. You’ve also looked at how all that leads to disastrous effects on our mental health

As it turns out, these products lead to a bunch of other big problems. But before we look at those, and think about the role of technology in a better future, we have to first reflect on what a good future means to us.

When you imagine your ideal future, what do you see? 

That question might bring up goals you have for your career, relationships with friends and family, or for personal growth. It may also bring up hopes you have for our world: fairness, sustainability, peace, and happiness.

What would it take to make this future a reality?

We live in a time when many societies are reckoning with massive challenges such as growing wealth inequality, catastrophic impacts of climate change, discrimination, and a global pandemic we were utterly unprepared for.

As the rising generation inheriting these problems and building the future, you need technology that strengthens our ability to come together and focus on big challenges.

Is today’s social media technology helping or hurting that goal? To answer that, let’s first look at our personal experiences with it.

🤔 CONSIDER: 

At the end of the Social Media and the Brain issue guide, you tracked your experiences on social media, focusing on how each experience made you feel.

Look back over the observations you recorded. If you’re working in a classroom or group, share your observations with others.

  • Did you notice certain themes come up across all of your usage?
  • What was surprising about what you observed?
  • Consider that your experience is one among billions. What do you think other users' charts would look like?
  • When you look closely at the technology you use every day, is it helping you become the person who you want to be?

In this final Issue Guide, we’ll zoom out from our individual experiences to discuss the consequences of social media for our society as a whole. We’ll learn about the thinking behind our broader social and economic systems, which drive many of the harms we experience. Then we’ll consider how they could be improved.

Question

1

How does persuasive technology amplify societal problems?

“I felt so insecure about myself. My abilities, my looks, my roots, my potential… I was comparing my life with people around me and people I saw on social media.”
– Nathan, 21, Midlaren, Netherlands
“I realized I was becoming more hateful and less open minded.”
– Madison, 23, Louisville, KY
“If you've never experienced addiction, a small warning, it sucks. I mean that literally: it sucks you in and prevents you from being happy, reaching your dreams, or living life.”
– Mahika, 15, San Francisco, CA

Nathan, Madison, and Mahika’s stories are examples of the ways in which technology shaped by the attention economy is resulting in painful experiences of insecurity, distortion, and addiction. As discussed in the Attention Economy Issue Guide and on the Center for Humane Technology’s Ledger of Harms, research shows that these problems are being felt throughout society. 

Many of these problems have existed in various forms for generations. What’s different now is that they’re being amplified by AI-powered technology used around-the-clock by billions of people around the world. For instance:

  • Fake news spreads six times faster than true news.¹ Misinformation has always been around, but on platforms that thrive on engagement, unexpected, attention-grabbing misinformation is widely shared. 
  • The level of social media use on a given day is linked to weaker memory function the next day.² There have always been companies competing for attention, from TV to magazines to billboards. But the frequency and strength of attention hijacking that happens on social media hurts our memory and focus.
  • The outcomes of elections around the world are being more easily manipulated via social media.³ A politician can now deliver customized, emotionally resonant messages to different groups, even if those messages contradict each other, because most people never find out about the contradiction.
  • AI algorithms have shown significant stereotypical bias by gender, race, profession, and religion. Society has long struggled with these biases, but when they’re embedded in the algorithms that shape platforms, they can become even more prevalent. Watch this bonus clip from The Social Dilemma to explore the topic more.

In response to public outcry, Facebook, Twitter, YouTube, and similar platforms have begun to invest heavily in programs designed to track and counteract organized hate and misinformation, address bias, and counteract many of these harms. But so long as their products are incentivized to lift up the posts that get people worked up, variations of these problems will continue to emerge.

We need technology that is accountable to the communities it serves. As long as our attention is highly profitable, and a small number of companies are trying to capture and control the attention of everyone in the world, that accountability will be impossible.

Question

2

How does persuasive technology keep us from solving problems?

“...the big question of human history, and the first question of human history, is how do you get hundreds and then thousands and then finally hundreds of millions of humans to cooperate?”
– Yuval Noah Harari, historian and best-selling author of Sapiens: A Brief History of Humankind, on Your Undivided Attention

What does social media do to our capacity to collaborate, especially on very hard, very big problems like climate change? When it shows us reasons to be outraged more than reasons to trust one another, that capacity goes down. When it shows different groups radically different views of reality, that capacity goes down. When it contributes to anxiety, insecurity, and an inability to focus, that capacity goes down.

Social media is diminishing our capacity for shared understanding and to make wise, informed choices.

To bring about a better future, we need to collaborate to solve difficult problems. The distortions of our social media environment makes this work, the work of democracy, very difficult. When we lose trust in one another, fewer people vote, fewer people step into public service in good faith, fewer people engage in the difficult but necessary work of listening and finding compromise, and fewer people show up to support and shape our communities. 

When shared understanding shatters, democracy shatters. 

Watch this bonus clip from The Social Dilemma to learn more about how social media platforms have been used to undermine democracies around the world.

Question

3

Why do we need to think systemically?

When we see that a product or industry is generating a whole bunch of problems, what’s the best way to handle them? 

One path is to solve each problem as it arises. But if similar problems are coming up again and again, we can end up in an exhausting, never-ending battle to fix them. In a situation like that, we need to see if there’s a root cause generating all the fires, because fixing that root cause in the system might be a much more efficient solution. That’s called a systemic solution.

Consider fast food. About 85 million adults in the U.S. consume fast food every day. Why? Because it’s affordable, quick, and takes advantage of our bias for salty, fatty, and sweet foods (which are energy-rich and helped us survive in much tougher pre-modern times, when energy was scarce). 

Fast foods are carefully designed and tested to be addictive to drive sales. For many people, that addictive quality makes resisting fast food very difficult, leading to a crisis of obesity, high blood pressure, Type 2 diabetes and heart disease that’s spreading globally.

If we want to address problems like obesity and heart disease we can (and should) build more hospitals and research better medical treatments. But doing so is extremely expensive, especially if millions more people each year are eating food that’s worse for our bodies. 

In this example, examples of more systemic solutions are:

  • Adding a tax to fast food and using that revenue to make healthy food cheaper.
  • Adding nutrition facts and recommendations to menus so people understand the risks in their diets.
  • Education to help people realize that just because something tastes good in the short term, it can be harmful in the long term.   

These types of systemic solutions can reduce the overall number of dangerous outcomes that require expensive medical treatment.

Similarly, if we want to address the harms of social media we can and should directly address problems like misinformation, hate speech, cyberbullying and addiction. But we should also address the mechanisms that cause these problems by changing the way that algorithms are trained, products are built, and the technology industry is regulated. 

Ultimately we will need to change the way we think about consumption if we are to build technology that helps us lead healthy and balanced lives. 

It may seem initially that fast food is “what we want”... but who really wants to get addicted and to struggle with health challenges? Is it a fair fight when companies are scientifically engineering foods that are designed to addict us? When it comes to food, a system designed to get us to consume more winds up looking very different than a system designed to make us feel healthy and balanced. 

What is true for food is also true for social media. At the root is a way of thinking, an idea that people consuming things makes them happy. This leads to a set of mechanisms designed to give people engaging content, things like recommendation algorithms and product features. These mechanisms lead to patterns in the kinds of information that people see on social media. Things like COVID-19 misinformation and hate speech are huge problems in large part because they are a highly engaging form of content. People will consume a lot of them even if doing so doesn’t make them happy or healthy in the long run.

🤔 CONSIDER: 

  • What similarities do you see in the ways of thinking that drive how social media and fast food make money? 
  • What do the consequences of social media and fast food have in common?

Question

4

Why do we need to shift from current paradigms?

Whether it's the health issues of fast food, or the harms of social media, most of our biggest challenges in industrialized countries are driven by some important underlying paradigms, or ways of thinking:

  • More is always better.
  • What someone chooses in the short term is “what they want.”
  • Nature is a stock of resources to be converted to human purposes.

These paradigms show up everywhere: in our economic systems (which reward bigger companies, but without considering many of the harms they create), in the way we treat plants and animals, in the ways we spend time socially, and in many other decisions we make. 

🤔 CONSIDER: 

  • What are some of the ways that these paradigms show up in the design of social media applications?
  • When are these paradigms not true?
    ○ When is more not better?
    ○ When are short-term choices people make not what they want?
    ○ When does treating nature as a stock of resources for human purposes backfire?

Company CEOs and Boards of Directors are required to make short-term profits for shareholders even when it leads to harmful consequences for well-being, other living beings, or our environment. This is, in part, the result of paradigms that tell us that “more is better” and “nature is a stock of resources to be converted to human purposes.” 

As a result of all this, companies compete to find the fastest legal paths to profitability and growth. Almost always, that means using the latest technologies to find new ways of driving consumption, which also drives more rapid extraction and harm of our natural resources.

That approach can look good in the short term, but consistently leads to big problems in the longer term. That’s exactly what’s happened with social media, fast food, deforestation, carbon dioxide generation, and much more:

  • McDonald’s sells about 50 million burgers each day.
  • Global food production accounts for 25% of all greenhouse gas emissions; 7.5% of emissions are from meat and fish production alone.
  • We cut down about 10 million hectares of trees each year—that’s an area of about 24,700,000 football fields.

Actions like these at massive scale have put the world in a dangerous situation—one that will be inherited by your generation.

Extreme weather is accelerating, destroying homes and displacing millions worldwide. We are using our planet’s resources far faster than they will ever replenish. And we are destroying the delicate ecological balance that allows humans to thrive.

“We live in a world in which a tree is worth more, financially, dead than alive, in a world in which a whale is worth more dead than alive. For so long as our economy works in that way and corporations go unregulated, they're going to continue to destroy trees, to kill whales, to mine the earth, and to continue to pull oil out of the ground, even though we know it is destroying the planet and we know that it's going to leave a worse world for future generations... What's frightening, and what hopefully is the last straw that will make us wake up as a civilization to how flawed this theory has been in the first place, is to see that now we're the tree, we're the whale... We're seeing corporations using powerful artificial intelligence to outsmart us and figure out how to pull our attention toward the things they want us to look at, rather than the things that are most consistent with our goals and our values and our lives.”
– Justin Rosenstein, co-inventor of the Facebook Like button, in The Social Dilemma

Question

5

What new paradigms do we need to shift towards?

To protect your future, we need changes and we need them quickly. Unsustainable old paradigms need to be replaced with new ones, like:

  • Growth must be balanced with responsibility. For example, a business shouldn’t be able to extract a resource (like a tree, or our attention) so much that the resource can’t replenish (deforestation, or addiction). 
  • People often need support to make informed choices in their own long-term interest. We often get caught up in the moment, but see more clearly in hindsight. 
  • People thrive when they can live a balanced life filled with the relationships and activities that they find most meaningful. Too few of these and we can feel bored or unfulfilled, but too many lead us to overwhelm.

Changing these types of fundamental assumptions that society is built on might seem impossible. But it’s also impossible to keep pretending that the Earth has infinite resources or that our democracies will continue to function when we can no longer tell what’s true or not.

So all of us who understand what’s at stake have to demand change. As a young person, your voice is perhaps the most important, because your future is most at stake.

🤔 CONSIDER: 

Can you think of a historical example of a paradigm that has shifted? Of a deeply held, almost unquestionable belief that was challenged and then replaced? How about a time in your own life when you’ve made this kind of shift?

  • What did it take to make this shift come about?

Go Deeper

  • Listen to historian Yuval Noah Harari on the Your Undivided Attention podcast. In this wide-ranging conversation, Yuval explains how technology and democracy have evolved together over the course of human history, from Paleolithic tribes to city states to kingdoms to nation states. So where do we go from here? “In almost all the conversations I have,” Yuval says, “we get stuck in dystopia and we never explore the no less problematic questions of what happens when we avoid dystopia.”

Footnotes

  1. Vosoughi, S., Roy, D., & Aral, S., 2018. Science (https://science.sciencemag.org/content/359/6380/1146.full)
    Via: Ledger of Harms (https://ledger.humanetech.com/#study_122)
  2. Sharifian, N., & Zahodne, L. B., 2019. Journal of Gerontology: Series B (https://www.researchgate.net/publication/339290745_Social_Media_Bytes_Daily_Associations_Between_Social_Media_Use_and_Everyday_Memory_Failures_Across_the_Adult_Life_Span) Via: Ledger of Harms (https://ledger.humanetech.com/#study_154)
  3. Confessore, N., Dance, G., Harris, R., & Hansen, M., 2018. New York Times (https://www.nytimes.com/interactive/2018/01/27/technology/social-media-bots.html)
    Via: Ledger of Harms (https://ledger.humanetech.com/#study_44)
  4. Source: Nadeem, M., Bethke, A., & Reddy, S., 2020. Computation and Language ↗ (https://arxiv.org/abs/2004.09456)
    Via: Ledger of Harms (https://ledger.humanetech.com/#study_147)
  5. https://www.cdc.gov/nchs/products/databriefs/db322.htm
  6. https://ourworldindata.org/food-ghg-emissions

What’s Next

Use Center for Humane Technology’s Action Guides to help shift your relationship with social media and demand technology that enables a humane future:

Please Share Your Feedback

How do you feel about the quality of this section?
vote-seeing-the-consequences