Truth, Disrupted

In March of 2018 President Trump’s tweets claiming that Amazon pays “little or no taxes to state & local governments” sent the company’s stock toward its worst monthly performance in two years. Trump had his facts wrong — and the stock price has since recovered — but the incident highlights an unsettling problem: Companies are profoundly vulnerable to misinformation spreading on social media. Unsurprisingly, the mainstream media has focused primarily on whether false news affected the 2016 U.S. presidential election. But the truth is that nobody is safe from this kind of damage. The spread of falsity has implications for our democracies, our economies, our businesses, and even our national security. We must make a concerted effort to understand and address its spread.

For the past three years Soroush Vosoughi, Deb Roy, and I have studied the spread of false news online. (We use the label “false news” because “fake news” has become so polarizing: Politicians now use that phrase to describe news stories that don’t support their positions.) The data we collected in a recent study spanned Twitter’s history from its inception, in 2006, to 2017. We collected 126,000 tweet cascades (chains of retweets with a common origin) that traveled through the Twittersphere during this period and verified the truth or falsehood of the content that was spreading. We then compared the dynamics of how true versus false news spreads online. On March 9 Science magazine published the results of our research as its cover story.

Reality Wars:
Series reprint

download PDF
buy copies

What we found was both surprising and disturbing. False news traveled farther, faster, deeper, and more broadly than the truth in every category of information, sometimes by an order of magnitude, and false political news traveled farther, faster, deeper, and more broadly than any other type.

The importance of understanding this phenomenon is difficult to overstate. And, in all likelihood, the problem will get worse before it gets better, because the technology for manipulating video and audio is improving, making distortions of reality more convincing and more difficult to detect. The good news, though, is that researchers, AI experts, and social media platforms themselves are taking the issue seriously and digging into both the nature of the problem and potential solutions.

In this article I’ll examine how we might contain the spread of falsity. A successful fight will require four interrelated approaches — educating the players, changing their incentives, improving technological tools, and (the right amount of) governmental oversight — and the answers to five key questions:

• How can we educate people to spot and resist falsity?
• How can we disincentivize the spread of falsity and incentivize the spread of good-faith communication and truth?
• How can technological tools — algorithms in particular — be used to contain false information?
• How can regulators usefully weigh in without destroying the economic and social value created by social media?
• And, perhaps most important, Who gets to decide what’s true and what’s false?

But before getting into solutions, let’s take a closer look at why we should care about this issue.

Why False News Is Dangerous

People have been telling lies and spreading rumors since the beginning of recorded history — probably for as long as they’ve known how to talk to one another. But things are quite different today. Social media, which increases the speed and breadth with which information spreads, became extraordinarily powerful in a very short time. Twitter, founded in 2006, has 336 million active users worldwide, and Facebook, founded in 2004, has 2.19 billion. Those platforms have become the primary source of news for many people. But because they have, to date, made a conscious choice not to vet the quality of the content they distribute, virtually no safeguards exist when it comes to truth and falsity online.

The fact that false information spreads so easily on social media platforms is not only a nuisance, of course; it can also be dangerous and costly. Obviously, false news threatens the integrity of our elections and democracies. With Robert Mueller’s Russia-related indictments fresh in our minds, it was chilling to watch the undercover video of Alexander Nix, the former CEO of Cambridge Analytica, describing how his company used fabricated stories, propagated online, to influence global elections. The jury is still out on to what degree false news affected the U.S. presidential election and elections in Europe and Africa. But most experts agree: The spread of falsity online is a serious concern for the democratic process.

False news can also lead to the misallocation of resources when law enforcement and first responders rely even in part on social media to collect information during a terror attack. Misinformation flowed freely at the time of the Boston Marathon bombing, for example, when the MIT campus that Soroush, Deb, and I call our professional home was on lockdown as the site of an ongoing terrorism investigation. Information was scarce, and we didn’t know which parts of campus were safe. We turned to Twitter for updates and found that in addition to the true breaking news, which was more up-to-date than any television broadcasts, a lot of false information was spreading and misdirecting law enforcement. If attackers know that law enforcement relies on social media, they can use it to proactively thwart police responses. When social media is a source of intelligence, polluting it can become a weapon.

Loading...

Misinformation affects our economy, our investments, and the value of individual businesses. In 2014 a false tweet claiming that Barack Obama had been injured in an explosion wiped out $130 billion of equity value in a single day. Modern investment funds use social media sentiment to inform their algorithmic trading practices. When false news infiltrates such media, these automated traders consume and trade on that information. No one has robustly measured the losses generated by algorithmic trading based on false news, but anecdotal examples suggest that social media’s impact on the economy may be large.

False information can distort virtually any aspect of running a business: It can misalign investments; reduce returns; and derail demand forecasts, inventory models, and planning. It can also damage reputations (and with them market valuations). President Trump’s tweets about Amazon are just one example of false claims that hurt a corporation. The fact-checking site Snopes keeps a list of “hot 50” rumors that is updated with alarming regularity; it has included a 2008 rumor that United Airlines was filing for bankruptcy, a 2017 report that Starbucks would give out Frappuccinos to undocumented workers, and another 2017 report claiming that Indra Nooyi, the CEO of PepsiCo, had told supporters of the newly elected U.S. president to “take their business elsewhere.” As with algorithmic trading, we have no solid estimates of how much damage misinformation has caused individual businesses. But the frequency of such events suggests that it is a real and growing problem.

Why Does False News Spread?

Some findings from our research cast doubt on seemingly obvious explanations for why false news travels so quickly and broadly. For example, you might assume that social media heavy hitters are behind the successful spread of false news, but our data revealed the opposite. Those who spread false news were significantly less connected than those who spread true news — they had fewer followers, followed fewer people, were less active on Twitter, were “verified” less often, and had been on Twitter a shorter time. All this suggests that falsity diffuses farther and faster despite the differences between the two groups, not because of them.

You might also imagine, from reading newspaper stories (and congressional testimony), that bots are the key factor driving the diffusion of false information. But according to our data, robots accelerated the spread of true and false news at approximately the same rate — suggesting that false news spreads faster because humans are more likely to spread it.

People’s susceptibility to falsity might be better explained by what’s called the novelty hypothesis. According to this theory, novelty attracts attention and encourages sharing by conveying status on people who seem more in the know. In our study false news was indeed more novel than the truth, and people were more likely to share novel information. (Think of “novelty” as “different from what the tweeter is used to seeing.”) False rumors also inspired greater surprise and disgust in replies, whereas the truth inspired greater anticipation, joy, and trust. People may simply be more likely to share surprising, salacious news.

Fighting False News

One broad way to combat falsity online is to consider the consumer’s perspective. A demand-side approach might, for example, attempt to educate consumers by providing them with information about the quality of a story or a tweet. A supply-side approach would tackle the problem at its source — for example, by creating disincentives for social media and content providers to publish and spread false news. The two approaches are complementary, and both will be essential. Algorithms will undoubtedly be crucial in executing either one. In addition, governments will need to figure out how they can play a useful — not destructive — oversight role.

Protect and educate consumers. Suppose we could assess and communicate the accuracy of the information and news available on social media by labeling it. We already do this with food. In most countries packaged food is extensively labeled. We know how many calories it contains, how many grams of sugar and protein and trans fats it has. We even know whether it’s organic or free-range and whether it was produced in a facility that also processes wheat or peanuts. (That information wasn’t always so readily available. It is now because consumers mobilized and, in response, governments wrote and enforced regulations.)

But when we consume news — particularly online — we have far less information. We don’t know whether the source tends to disseminate true or false information. We don’t know whether a particular story is likely to be true or false. We don’t even know how the news was produced — whether the publisher requires three independent sources to run with a fact or just one. We don’t know how many reporters worked on the story, how many interviews they conducted, or how long they investigated.

This is an important avenue to pursue, but it raises several questions that lack easy answers:

Do we know how to accurately identify false news? In our research we used Twitter cascades whose veracity had been assessed by six independent fact-checking organizations. We then used students working independently at MIT and Wellesley to check for bias in how the fact-checkers had chosen those cascades.

Obviously that process would be difficult to scale. Therefore, building algorithms that can more efficiently predict the veracity of content will be critical. In his PhD thesis at MIT, which Deb Roy supervised and I advised, Soroush Vosoughi developed one of the first algorithms for automatically detecting and predicting the veracity of rumors spreading on Twitter in real time. The classifier uses semantic and syntactic features to identify rumors with 91% accuracy and predicts the veracity of those rumors in real time, with 75% accuracy, using their linguistic style, the characteristics of people involved in propagating them, and the propagation dynamics. I have seen newer research in this area that also looks promising. But we’re far from having an agreed-upon methodology for identifying false news.

Who gets to decide what’s true or false? This is a critical question — but it doesn’t have a clear answer. Should we leave this up to platforms like Facebook and Twitter? Should regulators decide? Should we rely on fact-checking organizations like the ones we looked to in our research? Should we establish some sort of independent commission? It seems impossible to guarantee that fact-checkers and commissions would not be politicized. A lot more thinking — and research — needs to happen before we have good answers.

Responding to pressure around this issue, Facebook recently announced that it will be adding an “about this article” button to news posts on its site. The button will lead to more information, including related articles on the topic, stories recently posted by the same publisher, and a link to the publisher’s Wikipedia page. This may be a move in the right direction, but we clearly need more.

Would accurate labeling actually slow the spread of false news? Scientific evidence on the effectiveness of labeling is currently inconclusive. Some research shows that labeling false news decreases the spread of misinformation. Other research shows that it actually increases the spread. Little of the work in this area has yet been published or peer reviewed. So we need experimentation to learn how labels can most effectively curtail the spread of falsity. For example, would a “veracity score” work better than a link to the publisher’s Wikipedia page?

Change the incentives for content creators, advertisers, and social media companies. The social media advertising ecosystem depends on the spread of content. The more attention content gets, the more value it creates and the more advertising revenue it earns. So in a sense, current digital advertising business models incentivize the spread of false news — because, as we’ve seen, misinformation travels farther, faster, deeper, and more broadly than accurate news does. It has been widely reported that the production of fake news in Macedonia during the 2016 U.S. presidential election was motivated less by political incentives than by economic ones. The producers simply found that they could earn more advertising revenue from false stories than from true ones.

Economic incentives to circulate misinformation are, of course, short-term and short-sighted. In the long run, the spread of false news degrades platforms, hurts advertisers, and diminishes the credibility of honest content creators — a fact of which most of them are well aware.

We’re in the very early stages of thinking about how suppliers might want (or need) to change. That said, I see two broad possibilities:

1. Blow up the ecosystem’s business model. A radical approach would be to replace the ad-driven business model of Facebook and other social media companies with a subscription model — the rationale being that prioritizing clicks and engagement promotes sensational, divisive, and false content. If users simply paid a monthly subscription fee, the argument goes, incentives could be realigned with users’ best interests.

This possibility is already on Facebook’s radar. On the Today show COO Sheryl Sandberg floated the idea of turning Facebook into a “freemium” model, in which users would pay to turn off ads and, presumably, data collection. Economically speaking, that might be possible: Freemium models are, after all, pervasive. Pandora lets you listen free with ads but charges for ad-free listening. The New York Times allows you to read 10 free articles a month, after which you pay.

But is that realistic? Facebook and a few other large companies have together created an extraordinarily valuable economic sector. It’s difficult to imagine that they’d walk away from the moneymaking machine they’ve created without heavy pressure from regulators (which I’ll discuss below).

Practically Speaking, Why Does “the Truth” Matter?

Having a shared understanding of the facts and how they were arrived at is central to virtually any collective human endeavor, online or offline. Scientists build knowledge by gathering evidence to support — or invalidate — hypotheses. Lawyers argue cases on the basis of what they can prove beyond a reasonable doubt. Stock fund managers decide to buy or sell in light of facts that can be ascertained about a company’s performance. Standards of evidence and reporting exist in each of these domains. And nowhere does the need for commonly understood facts hold more true than inside companies, or within a network of business partners. Managers make daily decisions that implement commercial strategy. According to most if not all decision theory, the success of those decisions depends on whether the actions taken make sense when they meet the reality in which they are implemented. If Tesla ramps up production in response to false demand predictions, its business can be crippled. If the market capitalizes a company like Theranos on the basis of lies about its technology, large-scale investments are wasted and value is destroyed.

Of course, the understanding of truth evolves in all these realms. Scientific findings aren’t considered solid until they’ve been replicated — and are generally either refined or disproved in the process. What’s considered proof in a courtroom changes as forensic science and higher court rulings change. Business executives once viewed their experience and intuition as the most important tools they had when making big decisions, but today the data and analytics behind evidence-based decision making often replace that gut sense.

Even with all those caveats, it’s difficult to escape this reality: We need the truth to ensure that our decisions make sense in the world in which they are enacted and that we are on the same page when we coordinate or collaborate.

Even if we could wave a wand and reinvent the business models underlying social media platform companies, would it be wise? Many long-tail publishers that produce diverse, nonmainstream content couldn’t survive without ad revenues, because only larger players have the scale to make subscription models work. Rather than paying for multiple services, consumers would most likely choose one subscription in each content category (news, sports, opinion, and so on). Consequently, the market for content production would probably shrink, concentrating in the hands of a few large companies. The damage to society from this loss of information diversity could be great.

The change wouldn’t hurt only publishers and their consumers. Social media’s digital marketing ecosystem sustains many other companies and is responsible for a significant number of jobs and amount of output across brands, agencies, trading desks, demand-side platforms, ad exchanges, ad networks, and supply-side platforms.

Another important consideration is that moving to a subscription model could accelerate inequality. According to a 2017 PEW study, nearly 70% of American adults “get at least some of their news on social media,” and 70% of the ones who use Twitter say they get news there. In addition to news, Facebook users have access to relationships and networks, which are essential to getting jobs and managing economic opportunities. The move to a subscription model would almost certainly constrain access for those who could not afford it.

A shift to subscriptions would also risk exacerbating what I call the privacy gap — the unequal distribution of privacy across society. Only richer users would be able to pay, say, $9.99 a month for Facebook. If we believe that privacy is valuable and that protecting our personal data is important, how do we feel about a society in which the rich buy freedom from surveillance while the poor are required to trade their privacy for access to information and jobs?

Bottom line: A radical change to the social media ecosystem is unlikely, at least in the United States. Nor should it happen, in my judgment: Heavy-handed regulation would risk destroying the value these companies have created for consumers and shareholders and unleashing a whole slew of unintended negative consequences. However, there is a less radical, more realistic alternative.

2. Adjust the system but don’t blow it up. Platform companies have the power to reduce the reach of false news. They can choose to work with other parties to educate and protect consumers — doing their best to detect and label false news in the hope that users will be more reluctant to share it.

This type of intervention would depend on tweaking various algorithms currently used in the social media ecosystem and would require cooperation from a variety of stakeholders. News-feed algorithms determine what users see in their news feeds; trending algorithms identify and boost the reach and popularity of the most engaging content (for better or worse); ad-targeting models and APIs allow advertisers, political campaigns, and (apparently) foreign actors to direct content at specific audiences — for example, those most susceptible to false news on a particular topic. Assuming a relatively accurate veracity scoring system, these algorithms could be modified to reduce the spread of falsity online.

Another important approach would be in information design and testing. The psychological effects of information design in the presentation of news online will affect how we consume, respond to, and share content. If that design took falsity into account, it could help reduce the spread of it.

Regulation

The issue of government oversight of the social media giants has been all over the news recently. Mark Zuckerberg testified before Congress in March and April of 2018 and before the European Parliament in May. And in May the EU rolled out its General Data Protection Regulation (GDPR), a tough, far-reaching regulatory framework for protecting consumer privacy. Grand conclusions about the right way to approach data privacy and antitrust are beyond the scope of this article. But regulators need to consult digital economy experts and tread carefully.

Some U.S. lawmakers seemed convinced that Facebook must be regulated because it is too large to self-regulate, while others seem not to understand the nuances of how regulation could affect society and the economy. Many of the implications are not obvious. Asking “Should we regulate?” is meaningless. The better question is “How can regulation preserve the positive effects of social media while constraining its negative consequences?”

False news seems especially difficult to regulate, because one question inevitably arises: Who will have the power to decide what information should be disseminated? This is perhaps the most important question facing democracies in the information age. We don’t give the government the right to censor the news media; do we want to give it the right to control how information is disseminated online? Dictators and other authoritarian leaders could easily use that power to entrench themselves. Malaysia recently mandated a six-year prison term for purveyors of false news. Such draconian measures could easily be used to silence opposition and promote repression.

Then there’s the question of how much responsibility to give social media companies for what flows through their channels. The U.S. Communications Decency Act (CDA) of 1996 established that platforms like Facebook were to be considered pass-through entities not responsible for what users post online. This was seen as protecting freedom of speech and a boon to the freedom of the internet. But the law is now being challenged in an attempt to carve out known instances of free speech in which the harm clearly outweighs the benefits. For example, the Senate recently passed the Stop Enabling Sex Traffickers Act, which makes Facebook responsible for sex trafficking ads on its platform, by an overwhelming 97–2 vote.

The CDA will be key to making platforms more responsible for their impact on the world. How it is used is critical, because regulating speech can become a slippery slope. As lawmakers take up similar challenges in the future, the trade-offs between benefits and harm will be weighed and reweighed.

The regulation of political speech online provides another example of the complexity of this issue. Maryland just passed legislation that requires social media platforms to track all political ads and the users being targeted, bars foreign currency from use in Maryland elections, and empowers the Board of Elections to investigate complaints about online ads or voter suppression. Facebook and Twitter have both endorsed the Honest Ads Act (which would implement similar restrictions at the federal level) and have already adopted most of its provisions. But some are concerned that the Maryand law will be found unconstitutional on First Amendment grounds. And, of course, regulation could put political speech under the scrutiny of political actors, who have biases and self-interests.

Regulating the collection of personal data is also complicated. It’s easy to forget how many industries and social services rely on such collection. For example, our entire credit rating system collects, uses, and sells granular data to advertisers for the purpose of targeting. Consumer access to credit cards, mortgages, health care, travel, social services, and education requires the collection of private data. Regulators should weigh the privacy benefits of restricting data collection against the harm that such restrictions could do to these essential services.

Restricting data collection, making it opt-in, or implementing APIs that enable data portability is bound to have unintended consequences. For example, a good argument can be made that data protection and competition trade off. Data portability might enable competition by giving new companies access to established players’ social data (and, therefore, their network effects). But Facebook is currently being criticized for just such data portability, having shared data with global hardware firms, mobile providers, and developers of new applications, in China and elsewhere. We need ideas about how to ensure data protection and competition simultaneously. Otherwise, strengthening one could weaken the other.

Even if high levels of security could be guaranteed, it’s not clear that portability would necessarily improve competition. For example, it’s possible that allowing consumers to take their social network data from one platform to another would enable competition between platforms — but it’s also possible that consumers would opt in to sharing their data only with big players they trust, not with startups they don’t recognize. Imposing universal privacy constraints now, after Facebook and others have built data monopolies, could simply serve to restrain Facebook’s competition, increasing its market power. Should restrictions on data collection and use be combined with antitrust actions? We all ought to watch closely as GDPR continues to roll out, because its simultaneous implementation of privacy constraints and data portability may foreshadow what happens next in the United States.

As a researcher, I’m particularly concerned about the unintended consequences of what I call the transparency paradox: Right now Facebook is under tremendous pressure to reveal more about how targeted advertising works, how its news-feed algorithms work, how its trending algorithms work, and how Russia or any other entity can spread propaganda and false news on the network. But at the same time, it is facing strong pressure to wall up its data, increase security, and protect users’ privacy. Facebook has been working to help scholars assess the impact of social media on elections; the initiative it has developed in collaboration with Gary King of Harvard and Nate Persily of Stanford, for example, provides a new model for industry-academic collaboration and is a welcome step. But there is a real risk that the company will overreact to the Cambridge Analytica affair and put unnecessary restrictions on what it shares, which could affect the conduct of the very research that’s so desperately needed.

Challenges Ahead

Stemming the tide of falsity will be no easy task. The first and perhaps most difficult challenge is that nearly every solution depends on defining what is true or false. For labels to inform us or for algorithms to block the spread of false news, we must determine where the line between truth and falsity stands — and grant somebody the right to make the call. That’s a tough problem to solve.

Second, misinformation is a moving target. As we develop designs to fight falsity, those interested in promoting it will adapt. Some of the most sophisticated falsehoods we studied were a category of story that we called mixed. Mixed stories contain information that is partially true and partially false. They hide falsity in a cloak of truth, which makes them harder to detect and more difficult for consumers to ignore. If falsity is strictly policed, intentionally blended stories will become more prevalent.

Third, we ain’t seen nothing yet. The falsity we are reckoning with today is nowhere near as sophisticated and insidious as what we will see in the near future. In 2016 Russia used text-based political messages with doctored photos in an effort to manipulate the U.S. presidential election. But the false news of the future will manifest as synthesized media — fake audio and video made to look and sound real. Imagine fake videos of politicians engaged in career-ending behavior — which may do more damage in the long run than politicians’ calling actual career-ending behavior fake news, which already happens. The development of commercial-grade media synthesizers will change our perceptions of reality and our standards of truth. The news that “deepfakes,” one of the most notorious media synthesizers, intends to democratize his software, making it easy to access and easy to use, is not a good sign. Detecting, labeling, and blocking the spread of synthesized media will be one of the most important challenges of the impending wave of designer falsity.

Finally, we must not shut down independent investigations into how social platforms are affecting society. Right now the way people react to microtargeted ads may be the best indication of how they will respond to political manipulation. Their reactions are not insignificant, but they’re not impressive, either: Base conversion rates are in the .01% to 0.1% range. Given such low numbers, is it really likely that Cambridge Analytica achieved conversion rates between 5% and 7%, as the whistleblower Christopher Wylie testified? We don’t know — but we need to know that and a whole lot more if we intend to effectively address the challenges that lie ahead for our global information ecosystem.

. . .

The rise of falsity threatens to create what the Atlantic staff writer Franklin Foer calls “the end of reality.” We live in a world where foreign governments distribute falsehoods to manipulate elections and disrupt democracies, politicians defend themselves by labeling opposition as fake news, and new technologies create convincing artificial and virtual realities that can compete with our own. If these trends succeed in separating what is real from our collective perception of it, we’ll be in grave trouble. Platforms, scientists, and regulators need to work together to preserve and promote the truth before we are in a full-blown battle for reality.