11/17/2008

How to Lie with Statistics

Whats This All About?

Statistics are commonly used to deceive Technically true, but deceptive Preys on fear of numbers Math is hard! --Barbie

Ways to Lie to Voters

0) Fabrication
Just make things up

Can be very effective: False impression of accuracy Figures never lie, but liars figure. Make the son of a bitch deny it. There are three kinds of lies: Lies, Damned Lies, and Statistics. --attributed to Benjamin Disraeli Swift Boat Veterans for Truth Lyndon Johnson:

Need to know how to lie with statistics, to keep from being lied to with statistics.

Also talking about ways to say things that are true, but misleading

11/17/2008

"I guess I should warn you that if I turn out to be particularly clear, you've probably misunderstood me.
-Alan Greenspan at his 1988 confirmation hearings.

Siblings
Sibling Distribution

Ways to Lie to Consumers: Omission

Leave Things Out
E.g., use mean, but mode may better tell the story

Most people have 0,1,2 Few people with huge families

60 50

Median

# Respon ndants

Pull mean up

40

Mean
30 20 10 0 0 2 4 6 8 10

Number of Siblings

11/17/2008

Age (of students and faculty)

Diverse Population Problem
35

Other Examples
Tax Cut 111 million taxpayers will save, on average, $1,586 off their taxes. Facts: 1) 25% receive NO cut (drops mean to $1,217)

Age Distribution
Median
30

Students, mostly 19-22 (Much) older faculty Nobody at mean age

Very bad description

# of Respondants

25 20 15 10 5 0

Mean

2) Median cut: $470 Half of all taxpayers get $470 or less

20

30

40

50

60

Age

Ways to Lie to Consumers: Exaggeration

Make Something of Nothing
The Fifth Dentist Problem Four out of five dentists surveyed How many dentists total? Five not a good sample Leave out the sample size, and you can prove just about anything Four out of five cards drawn from this deck were black! Sounds badbut, Numbers dont lie, thats where we come in! Fear of big numbers: My opponent wants to spend $2 million on [something]

$2 million = 1/1,000,000th of the budget Need to put big numbers in context

11/17/2008

Guys Rule!
Gender Distribution
Honest presentation Full scale shown Bars same width, color Slightly more male students Not a big difference
100 % of Respondants

Example: Guys Rule!

More Survey Data
56 % of Res spondants

Gender Distribution
54 52 50 48 46

Scale axes to blow up small differences

75

50

Nothing false in graph, but creates false impression

25

0 Male Female

44 Male Female

Average Income
150,000 100,000 50,000 0 19 20-29 30-39 40+ 140,000 120,000 100,000 80,000 60,000 40,000 20,000 0 19

Average Income

20-29

30-39

40-49

50-59

60+

Average Income

Average Income

Average Income
300,000 250,000 200,000 150,000 100,000 50,000 0 19 20-29 30-39 40+ 300,000 250,000 200,000 150,000 100,000 50,000 0 19

Average Income A I

20-29

30-39

40-49

50-59

60+

Average Income

Average Income

11/17/2008

Frequency Never Rarely Sometimes Weekly

Bully Others 55.7% 25.0 10.6 8.8

Bullied 58.9 24.2 8.5 8.4

Bully Others Good of bad statement: About the same percentage of kids bully others and are bullied Bullied: The claim that 41.7% of students face a degree of bullying is not a lie However, it would have been equally true to claim that only 16.9% of students were victims of bullying sometimes or weekly

This map shows how people in different states of the US like pizza. (These data were made up). The code for the state color is: Red States = People Love Pizza Yellow States = People Like Pizza Purple States = People Hate Pizza
The map really doesn't say very much. We don't know
how it was determined that people like pizza...were people asked if they liked pizza? Were people asked how much pizza they ate in a week? a month? a year? Was the number of pizzas purchased at stores in different states counted? Was the number of pizza restaurants in different states counted?

What to Do?
Questions to ask about any statistic: 1) Who created it?
Do they have an agenda?

2) Why was it created?

Research or politics?

We also do not know

if there are any real differences between how much people like pizza in the different states. How much do people love pizza in California? What is the difference between how much people love pizza in Utah compared to how much they like pizza in Nevada?

3) How was it created?

Methodology

There are no scales or measurements to indicate any of this information. Although this type of graphic gives almost no information, it is used frequently in many popular magazines.

Another example, who cares if in our elections a state is blue or red since it is the electoral votes that count

11/17/2008

What to Do
Questions to ask about any statistic:

1st thing to look for is bias

4) Whats missing?
Is there hidden context?

Conscious bias
Direct misstatements Ambiguous statement Selection of favorable data
Suppression of unfavorable Units of measurement may be shifted

5) Is it relevant?
Avoid misdirection

6) Does it make sense?

If it sounds ridiculous, it probably is

Who says so? How do they know?

A biased sample, or one that has been selected improperly or has selected itself

Did somebody change the subject?

Are there enough cases to add up to any significance? Look for a measure of reliability (sources for error) What is missing:
Look out for average: variety unspecified, a matter where mean and median might be expected to differ substantially Percentages are given and raw figures are missing The factor that caused change to occur, implying that some other more desired factor is responsible (Ex. Looking at total deaths rather than death rate, dont forget there are more people now than there used to be)

Watch out for a switch somewhere between the raw figure and the conclusion Strange things crop out when figures are based on what people say -- even about thi things th t seem t b objective f t that to be bj ti facts

Does it make sense?

Use common sense The trend-to-now may be a fact, but the future trend represents no more than an educated guess

11/17/2008

A lot of presented data is truthful Most people are willing to present unbiased statistics We are learning how to read and interpret data (judgment)