Introduction

Fun fact: You can make data (appear to) support almost any argument you want depending on how you present it. In short, how numerical information is presented can have a huge effect on the conclusions we (think) derive from it. For this reason, whenever we encounter numerical data we should keep in mind the Relativity criteria: We should always ask, “relative to what?”.

Part 1: Framing Effect

Case 1

You have a mild chronic medical condition and your doctor gives you the following advice:

You can take medicine X for which 80% of people experience no significant side effects.

How likely would you be to take the medicine? Most people are quite likely to.

However, suppose your doctor says,

“I can offer you medicine Y but 20% of people end up with side effects.”

How likely are you to take medicine Y? When people hear the possibility of side effects, they are less likely than in the first case to take the medicine.

As you may have noticed, the information in both scenarios is exactly the same. So, why do we have a different response? When information is presented only in terms of benefits or in a positive light people are risk-takers. This explains why, in the first case, our intuitive response is to try the medicine. However, when information is presented in terms of costs or negative effects, people are risk-averse. This explains why, in the second case we are more hesitant to take the medicine.

Another common example would be betting odds:

Case 2

Suppose you go to a casino. The game says that that you have a 10% chance of winning. Hey, 10% is not bad! Depending on how much money the game costs, you might try it.

Suppose, on the other hand, the game says there’s a 90% chance you’ll lose. In this case you’re probably much less likely to gamble.

We can explain the differing reactions to the mathematically equivalent information in the same way as Case 1. A 10% chance of winning is a positive frame (i.e., positive=we’re willing to engage in risk-taking behavior). But a 90% chance of losing is a negative frame (i.e., negative=we’re risk averse). It’s no coincidence that casinos advertise the odds of winning rather than of losing.

Case 3: (Tversky & Kahneman 1981)

You’re the head of the infectious disease unit at a hospital. 600 people have come in with a rare infection. Which of the two possible treatments should you give the 600 patients?

Treatment A is predicted to save 200 people

vs

Treatment B is predicted to give a 33% chance of saving all 600 people, 66% possibility of saving no one.

If you chose Treatment A, you’re in the majority. We’ll look at why in a moment…

Case 4: (Tversky & Kahneman, 1981)

Everything is the same as above except this time:

With Treatment A it’s predicted that 400 people will die.

or

With Treatment B there’s a 33% chance that no people will die and a 66% probability that all 600 will die.

Which did you choose?

Most people choose Treatment B.

Here’s the big reveal. Treatments A and B in both Case 3 and Case 4 are exactly the same, only the information has been presented differently. Let’s look at Case 3 first. When choices are framed as probable absolute benefits (i.e., expressed as an absolute number) vs relative risk, we are risk-averse. We want to preserve rather than risk what we perceive as the guaranteed benefit. That’s why in Case 3 about 75% of people choose Treatment A despite the fact that both treatments give the exact same outcome. (A 33% chance of saving 600 people=predicting the treatment will save 200 people).

However, when choices are framed as negative absolute costs (i.e., expressed as an absolute number) vs probabilistic benefit, we prefer risk-seeking to avoid guaranteed loses. So, when the choice is framed as in is in Case 4, we choose B (about 78% choose B) despite both treatments giving the exact same outcome. (A 66% chance that 600 people will die = predicting that 400 people will die).

Notice the size of the framing effect. Both Case 3 and 4 present the exact same information yet we go from 75% of people selecting Treatment A in the first frame to 78% selecting B in the second frame! Understanding the massive size of framing effects leads us to learning about…

How to Fool and Be Fooled:

If you understand the psychology of the (powerful) framing effect, it’s easy to see how it can be used to manipulate people’s views. This is a common tactic of alt-med, pseudoscience, politicians, and science deniers. Let’s look at a few common examples.

Anti-Vaccine Example:

Anti-vaxxers use framing effect in conjunction with slanting by omission.

In a typical anti-vaccine article you’ll see a claim like this:

A: Around 2000 people have gotten sick because of vaccines. (negative frame)

Since negative frames make us risk-averse the way the information is presented makes us weary of vaccines.

Let’s now put vaccines in a positive frame.

B: Several billion have avoided getting sick and losing their lives because of vaccines. (positive frame)

Notice how our reaction to vaccines changes depending on the frame we are presented.

Of course, if you only count the costs any policy or treatment will look bad. And if you only look at the benefits any policy will look good. Evaluating any treatment or policy should involve considering both costs and benefits.

Furthermore, we can also think of the above examples of framing as a misleading comparison because of omitted comparison classes (Relativity). That is, we need to compare costs to benefits but we also need to compare net benefits to alternative treatments/policies. In this case, the comparison class is no vaccinations. When we compare the net benefits of vaccinating to the net benefits of not vaccinating, the former is a clear winner. Asking of vaccinating, “relative to what?” allows us to better assess the information.

Global Warming Denial Example

Anthropogenic global warming (AGW) denialists use the same frame from Case 4. The general public hears all the doom and gloom of global warming (negative absolute frame). And the denialists exploit the probabilistic nature of climate models. In doing so, they pit absolute negative loses against the possibility of positive outcomes or (at least, less-negative outcomes). We know from Case 4, that when an absolute negative frame is pitted against one that is positive relative risk, we are more inclined to favor the latter.

Notice, also, that the degree of probability that climate models are inaccurate is never specified. By failing to quantify the relative probabilities of error compared to the probability that the models are correct within a certain range, our brains fall for the trick. If the comparison were expressed as like compared to like: E.g., as probability of the climate models being correct vs the probability of them being wrong, we wouldn’t be so easily fooled. This article explains the relative success of climate models in predicting temperature increases.

Fun Fact! The same hired-gun scientists that worked for tobacco companies to manufacture doubt about the harms of smoking are the same scientists the the oil companies hired to manufacture doubt about climate change. Read the book, Merchants of Doubt (also now a movie) for a complete history.

At this point you should notice that strategies were the same. In the early 1950s the general public started to hear about many of the harms of smoking. In response, the tobacco companies, for four decades worked hard to create doubt about the harms. Notice the frame they created was the same as in Case 4:

Absolute harms (negative) vs probability of no harm.

We know from above that people will be more disposed to favor the probability of no harm (or of a benefit) when it’s contrasted harms expressed as absolutes . This is in large part how the tobacco industry managed to fool the public despite the fact that the harms of tobacco were known to the public in 1954 (and much earlier to the tobacco companies).

Political Example:

A: The administration just reduced the budget by 486 million dollars! (positive)

B: The administration just eliminated all of the 486 million in funding for legal services for the poor. (negative)

Depending on the political bias in your news feed, you’ll see one of these stories describing the same numerical facts. In both cases, the way it is presented it will reinforce your existing position on the current administration.

Conclusion of Part 1

The moral of the story is that you need to be on the alert as to how numbers and probabilities are framed. You might approve or disapprove of the exact same information depending on how its presented to you. One easy way to avoid being mislead is to apply the Relativity criteria. When you’re presented with positive information, ask “relative to what?” and this should get your brain thinking about negative information as well as help you identify the relevant comparison class. The same goes for if you’re presented with negative information.

Aside from having the power to intentionally mislead us, the framing effect actually raises important ethical problems in politics and medicine. Since there is no neutral way to present information, and the frame we choose will necessarily influence people’s choices. Philosophers, economists, and psychologists are engaged in a current debate on how we ought to present information to people in a way that respects their autonomy. If you’re interesting in reading about this topic, here’s a pdf of the book, Nudge, that started the main debate. The first two chapters are well worth the read.

Practice for Part 1

Now that you have a basic understanding of framing effects, try a little practice: Change the frame from negative to positive and/or probabilistic to absolute.

1. “I wouldn’t take Ami’s Critical Thinking course, there’s a 10% chance of failing.”

2. Thousands of people die from chemotherapy every year.

3. If you join my multi-level marketing company there’s a chance that you’ll be rich beyond your wildest dreams.

4. There’s a chance that aliens are watching us right now!

More Ways to Fool and Be Fooled Using Framing Effect: Hazard vs Risk Based Policy

Hazard-based policy seeks to eliminate anything that is potentially hazardous without taking into account the relative harmfulness or benefits of alternatives. For example, it’s possible to fall down the stairs. A harm-based policy would eliminate all stairs. But is this a reasonable approach?

Example: How to argue like a science denialist/pseudoscientist using harm-based policy

• Over 1 Million injuries occur each year as the result of stairway falls.
• Staircase and stairway accidents constitute the second leading cause of accidental injury, second only to motor vehicle accidents.
• Each year, there are 12,000 stairway accident deaths.

Ban all stairs now!!!11!!1!!

A hazard-based policy fails the Relativity criteria as well as the total evidence requirement. Before we ban stairs, what other information do we need to know?

1. What are the hazards associated with whatever we use instead of stairs.

2. What are the benefits we’re giving up by not using stairs?

In other words, when someone makes an argument for hazard-based policy we should it evaluate it according to three criteria:

1. What are the costs and benefits of whatever we will use instead of the technology/policy we’re eliminating.
2. What benefits are we giving up?
3. Are there ways to mitigate the costs without losing the benefits?

A risk-based policy puts in place regulations on how, where, when, and by whom the technology or item in question is used in order to mitigate risk without losing the benefit of the technology. What are some ways we can keep our stairs and mitigate their risks?

In-Class Practice:

Apply the following evaluative criteria to the cases below:

1. Cars present a hazard to just about everyone and are one of the leading causes of death. It follows that they should be banned.

2. The IARC has declared that bacon is a class 2A carcinogen. Since cancer is a hazard, it follows that bacon ought to be banned.

3. Some people think glyphosate (a herbicide) should be banned. One argument is that glyphosate (Round Up) has been declared a possible carcinogen (2A Class) by the International Agency for Research on Cancer (IARC). That is, glyphosate presents a possible hazard. For some people, it follows that it should be banned.

Part 2: Absolute Numbers vs Percentages.

In the second presidential debate, Trump made the following argument: The US economy is doing poorly because India and China are growing their GDP at around 5% while the US is only growing at 1%.

India GDP data (Links to an external site.)

USA GDP data (Links to an external site.)

Why might this be a misleading claim?

Suppose you are Trump’s political rival. How can you use the same data and argue for the opposite conclusion? Based on the above economic data alone, which country would you rather live in?

Depending on whether we express a value as a percentage or as an absolute value we can tell contradictory stories using the exact same data! From the point of view of critical thinking it means that whenever we are given data (i.e., just the “cold hard facts”) we should consider both as an absolute number and as a percentage gain/increase. Also, as we saw in the unit on misleading comparisons, it’s extremely important to consider what the appropriate comparison classes are.

Indias GDP is 1.877 Trillion and it grew by 5% annual rate. How is the value of the absolute growth? To calculate the absolute value of the change:

Current total- (current total/1+decimal value of percent change)=Absolute value of increase.

To calculate the absolute value of India’s economic grow we go

1.877 trillion-(1.877 trillion/1.05)= 89.380 billion.

What about the absolute value of the growth of the US economy? US GDP is 16.77 trillion and grew by an annual rate of 2.2%. Let’s be charitable and go with Trump’s number of 1% anyway.

16 770 000 000 000-(16 770 000 000 000/1.01) = 166 039 000 000 or 166 billion.

Which country grew more? India, which grew by 89.38 billion or the US which grew by 166 billion? Our answer will have much to do with how we present the information.

We can take our analysis one step further and calculate GDP/person. At the end of the day what matters is how GDP growth affects the well-being of individual citizens.

US GDP/capita is 53,041.98 USD (2013) while for India it’s 1,498.87 USD ‎(2013).  From the point of view of economic well-being, where would you rather live? Who’s “winning”?

The point here is that the exact same numerical information can lead us to wildly different conclusions depending on whether it’s presented as an absolute number or as a percent. As a critical thinker, you cannot take numbers as they are presented to you–even if they come from a reliable source. You must consider how they are being presented.

Let’s try another example:

Suppose in Ohio there were 500 more property crimes than last year which had 25 000. Also, two years ago property crime was increasing at a rate of 7%/year.

A. If you are running for governor against the incumbent, what’s the best way to present the statistics?

B. If you are the incumbent governor, how should you present the statistics?

For A, it’s simple. Run lots of television adds saying that under Governor/Mayor so-and-so property crimes have increase by 500 in just the last year!!!11!!!

For B, we’re going to need some simple math to convert the absolute increase into a percent.

Method: 

To convert absolute numbers to a percent:

Net Increase or Decrease/Total number of units)x100 = % Increase or Decrease

For our example above I go:

(500 property crimes/25 000 total units) x 100=2% annual increase in property crimes.

If I am the incumbent, I’ll take out ads that say Governor/Mayor so-and-so has successfully stopped crime and drastically reduced property crime rates down to just 2%!!11!!!11

But why stop there? Notice that the absolute difference between annual crime rates is 5% (7%-2%). 5% doesn’t sound that impressive. However, if I express that absolute change in rates as a percent, I’ll sound like Batman.

5 as a percent of 7 is 71% ((5/7)x100=71%). Now, I can announce that under my administration, the crime rate is 71% of what it used to be. I’m basically the best crime fighter there’s ever been.

How to Fool and Be Fooled

It’s very common for people to let their guard down when they encounter numbers in an argument. Numbers are ‘objective’, after all. However, as we’ve seen I can make numbers support just about any conclusion I want depending on how I present them. Don’t be lulled into complacency!

In reputable sources, numbers will be presented as both a percent and as an absolute number. Context will also be given my comparing those numbers to the relevant comparison classes. Any time you encounter an article that expresses numerical values only one way or without any context, assume someone is trying to fool you.

Conversely, if you want to fool someone, now you know how to do it!

In-Class Practice:

1. Read the following article:

a) Imagine you are writing for a publication against cutting funding for meals on wheels. How would you present the numbers? Hint: Google the total federal budget.

b) Imagine you want to persuade the public that ending funding to Meals on Wheels is a good thing. How would you present the numbers?

Homework

In each question, the numerical information is presented in a way that favors a particular point of view.

(a) Convert the numerical information into a form that supports the opposite point of view. That is, convert absolute numbers to percents and percents to absolute numbers.

(b) Suggest which frame least distorts the information. To do this you’ll need to identify the relevant comparison class.

Example:

Around 1.3 million people were over-paid with food stamps last year!!! (due to either fraud or administrative error). The information presented this way leads the audience to believe fraud is rampant. Translate the statistics into a percent to tell a story that promotes the food stamp program as effective in preventing fraud. 47 million people/year receive food stamps. Hint: Express the amount of fraud/overpayment as a percent and present it in a headline that supports the integrity of the food stamp program. Link (Links to an external site.)

(a) 1.3 million/47 million=2.77%

Food Stamp Program Boasts 2.77% Fraud Rate!!!

(b) The least misleading frame is to express the figures as a percent. Any program–whether government or industry–will have cases of fraud. Expressing figures as absolute numbers will bias against the size of the program and won’t tell us about the relative efficiency of fraud-prevention. Good comparison classes would be other large government and industry programs. A 2.77% fraud rate is pretty low and reducing it to zero would probably cost more than the fraud it eliminates.

Questions

1. Suppose you’re the CEO of a company that did 10 billion dollars in annual sales last year. Last year the economy grew by 5% but your sales only grew by 3% compared to the previous year. You have a shareholder meeting coming up and these numbers don’t look good. Your sales haven’t kept pace with the economy. Thinking about absolute numbers vs percentages, how can you present the sales numbers to the shareholder meeting in a positive way? Hint: Translate your annual sales into an absolute number.

2. (Only) 32% of BGSU undergrads finish their degree on time (within 4 years) (Links to an external site.). Suppose you are the head of marketing for BGSU and want to make the statistic seem more positive. Suppose there are about 10 000 undergrads. Convert the percentage into an absolute number to show how successful BGSU is a graduating students within 4 years. Present it in a positive headline for a shiny brochure.

3. Most people don’t realize this but 97% of people facing felony charges don’t go to court. (Links to an external site.) They are resolved through plea bargain. Prosecutors have tremendous power in the court system. Because of their power they can offer felons the following deal: Confess, avoid trial and get a sentence below the mandatory sentence or go to trial and risk getting the maximum sentence. Because the risk of a long sentence is so great, over 2 million people in prison got there without ever going to trial. Instead 97% of them made a plea bargain. It turns out that about 2% of the people who made plea bargains were actually innocent but didn’t want to risk going to trial and getting 15 years to life. They preferred to take a 3-5 year sentence rather than risk 15-life–even though they were innocent. It’s also important to note that until there’s a decision to go to court the defense doesn’t have access to the prosecutor’s evidence. That means, there’s no way for a defendant to evaluate how strong their case will be in court.

If you wanted to defend the current practice you might appeal to the 2% number or present it differently: 98% of felons who chose a plea bargain are genuinely guilty! This sounds good. Now, suppose you want to make the argument that the system is actually unjust. Thinking about the concept of absolute numbers vs percentages, how could you present the same data in a way that would favor the argument that they system is unjust? Hint: Calculate the absolute number of innocent people in prison who are there because they didn’t want to risk getting a maximum sentence. Present the number is a provocative headline.

4. Here are some statistics on Syrian refugees and US immigration. Obama has set a goal to accept 10 000 Syrian refugees currently residing in Jordanian refugee camps. There are about 10 million Syrian refugees and 500 000 Syrian refugees in Jordan are awaiting resettlement to a new country.

(a) Suppose you write a pro-Syrian immigration blog. Thinking about absolute numbers vs percentages, how could present the numbers in a way that makes it seem like the US isn’t doing very much to help refugees? Present them in a headline that conveys this point of view.

(b) If you were writing an anti-immigration blog, how would you present the numbers? Present them in a headline that conveys this point of view.