Confirmation Bias

This is one of my favorite activities to start a semester with because it really gets students thinking about what it means to be a scientist. It's also incredibly fun and it shocks lots of students. You wouldn't believe how many students I've seen drop their mouths wide open when they find out what happened! It can start you off to a great semester.

Whatis the Instructor'sSequenceRule?

The following is an exercise we will complete in class to illustrate an issue that is important when conducting research. The instructor has a rule in mind that makes sequences of 3 numbers. In a sequence, unlike a set, order matters. Your goal is to figure out that rule in her head! But you can't simply ask him or her. To determine his or her rule, you can write a sequence of three numbers in the "sequence" column. The instructor will walk around the room putting either a :-) or a :-( in the "fits Instructor's rule?" column. A :-) means the sequence you created does fit her rule. A :-( means the sequence does not fit her rule. After the instructor writes a :-) or :-( for you, you should guess his or her rule; write it out as a sentence or phrase. Then estimate how sure you are of your guess by writing a percent (100% for completely sure and 0% for completely unsure) in the "How Sure?" column. the instructor will walk around the room writing :-) or :-( until most students are confident that they know her rule. We will do the first sequence together as an example.

Sequence

Fits the instructor'sRule?

Guessthe instructor'sRule

HowSure?

2,4,6

:-)

______________________ _______%

__,__,__

______________________ _______%

__,__,__

______________________ _______%

__,__,__

______________________ _______%

__,__,__

______________________ _______%

__,__,__

______________________ _______%

Please feel free to contact me if you would like my class handout with the above on it. I always go through the first guess very carefully with students because the task confuses some students. Explicitly say the following: I gave you your first guess of my rule for you. It's 2,4,6. See how it's in the sequence column. Now imagine I have been walking around the room and I just put a happy or a sad face in you "Fits the instructor's Rule?" column. Since I put a smiling face, you know the sequence 2,4,6 fits my rule. Now you should guess my rule and write down your guess as a sentence or phrase in the "Guess the instructor's Rule" column. Now estimate how sure you are that your guess really is the rule in my mind. Now we start again. First you guess a sequence and wait until I stop by. Once I put a smiling or sad face on your paper, you can complete the rest of the column and make another guess. Once you reach 100% as how sure your are, please turn your paper over so I can move quickly pass you while we wait for everbody else to get near as sure as you.

I have to admit, the first few times I taught this I was shocked that almost everybody in the class believed they absolutely knew my rule in only 5 guesses! Now poll your students. Ask how many have the rule counting up by two's. Then ask what other rules people have and, after each student gives their rule ask if anybody else has the rule. Then tell them the rule and how many students in the class have the right rule. I say it like this: It turns out that in our class of 30 students only 2 students actually guessed the rule in my head! My rule was simple "the numbers increase." Ask rhetorically: So how is it that all of you thought you were so confident you knew what was going on in my head when you really didn't? Put the following protypical student response sheet on an overhead so you can follow through most people's reasoning.

Sequence

Fits the instructor'sRule?

Guessthe instructor'sRule

HowSure?

2,4,6

:-)

counting up by two's

50%

6,8,10

:-)

counting up by two's

60%

20,22,24

:-)

counting up by two's

70%

3,5,7

:-)

counting up by two's

80%

25,27,29

:-)

counting up by two's

90%

200,202,204

:-)

counting up by two's

100%

Go through the table pointing out how they were thinking. At the same time point out how this is just like a scientific experiment to 'get at the rules inside my head!' Point at row 2 column 1 which has 2,4,6 written in it. At first you just took a guess made a causal observation ... Point at row 2 column 2 which has :-) written in it. and you found out it fit some rule out in the world. Your were making a scientific observation and you're trying to figure out what rule governs some phenomena. In this case the phenomena is just "What will make the instructor happy?" but it could have been anything like "What is the origin of the universe?" Point at row 2 column 3. So you made a hypothesis ... Point at row 2 column 4. and you were kind of confident but still not sure your understood the scientific phenomena. Point at row 3 column 1. So you made another guess which is just like running another experiment. Point at row 3 column 2. and you found out your experiment showed what you expected! Point at row 3 column 3. so you kept the same hypothesis Point at row 3 column 4. and got a bit more confidence. Point at row 4 column 1. you ran another experiment. Point at row 4 column 2. saw it worked! Point at row 4 column 3. kept your hypothesis Point at row 4

column 4. and got even more confidence. Point at row 6 column 4. and it just continued with more experiments showing the same thing until you got sure. But you're wrong! Why were you so sure of a wrong hypothesis? Most likely a student will volunteer that he or she felt she had lots of evidence. Maybe another student will tell you that they were thinking in a box and only really trying what they knew would work. Remember to make your students think rather than giving away the answer too quickly. If it's early in the semester you might have to wait several minutes for an answer (they need to be sure you're not going to feel they're stupid). When you get an answer be sure to praise the student's thoughfulness (that will encourage all the students to know you really do value their thoughts). Rephrase the student's responses so you get at how almost everybody in the class had a confirmation bias. That is, you wanted to run studies where the result would demonstrate that you're right rather than risk running a study they expect will be wrong. But if you did run studies you think might show you that you're wrong you might eventually find out the true answer! Put up the following overhead

Sequence

Fits the instructor'sRule?

Guessthe instructor'sRule

HowSure?

2,4,6

:-)

counting up by two's

50%

10,20,30

:-)

counting up by multiples

60%

100,500,894

:-)

counting up with all even numbers

70%

1,9,20

:-)

counting up

80%

27,13,4

:-(

counting up

90%

55,2,999

:-(

counting up

100%

Go through the above overhead just like you did the last one but explicitly explain how this scientist actively sought to falsify his or her hypotheses! A confirmation bias is only one of many things that make doing science well incredibly difficult. All of us have confirmation biases sometimes but one nice thing about our field is that other people have different intuitions from you so if you miss you're confirmation bias, they might not! Even so, it's important for you to remember that just because a scientist says something, even with lots of evidence, that doesn't we don't have to critically think about the experiments and the results. Throughout this class, and hopefully also in all of your science classes, I hope you will keep in mind that you have a right, even an obligation, to question every study put before you. That's what makes you a true student of science.