Research methods

1) Observational vs Experimental

Observational research:

 the investigator does not deliberately make something happen, but instead sets out to
describe some aspect of nature as he finds it
 observes and describes carefully, without attempting to influence events
 Study natural variation
 E.g.: what percentage of the Dutch children has ADHD? (frequency)
 Do children with ADHD bully more? (association)
 can also be used to test a specific hypotheses

Experimental research:

 the investigator intervenes to make something happen

 Manipulate (some variables) the cause and study the effect
 E.g.: the effect of stress on short term memory (causal)
 Advantages
 Causal claims are possible (!)
 High degree of control on the participants
 Disadvantage
 Artificial environment
 Not all manipulations are (ethically) possible

The difference is that the pollster is not investigating the effect if his intervention. Indeed, he
assumes that the intervention will have no effect at all, but only allow him to find out
something about the subject

Observational methods

1. Case studies
 you study one or a few cases
 Broca’s speech area:
o expressive aphasia: the ability to produce speech
o he studied “Tan” who had trouble producing speech though he could
understand it well
o suffer from the failure of a specialized memory mechanism-> cannot
remember how to use their speech apparatus to form words
o Broca’s area: part of the brain specialised for speech production
 Advantages:
 o High degree of depth
o Generates a lot of new ideas for further research
o Show that some phenomenon can occur (e.g., prosopagnosia-inability
to see faces)

 Disadvantages:
o Generalizability
o Alternative explanations
o Ad-hoc reasoning

2. Direct observation
 a researcher may observe the subjects’ behaviour directly, from outside the
situation
o Natural environment
 E.g., observing how children play in a class room
o In a ‘created’ environment
 E.g., a group of applicants for a certain job are first given a
negotiation task
 Stickleback fish:
o ethology: the study of animal and human behaviour in its natural setting
o reaction is triggered by a characteristic releasing stimulus
 mother-infant interactions, book carrying in humans
 Advantages:
o You observe exactly where you are interested in (generalizability)
 Disadvantages
o Less control on the participants as compared to experiment

3. Surveys
 Interviews and questionnaires
 Why? → Some behaviour is hard or impractical to directly observe
o E.g.: Depression, alcoholism
o E.g.: Opinions, Attitudes
 it can be small-scale or large, but individual cases will not be the focus of
attention
 The Bennington studies
o incoming class conservative later liberal
o social pressure produced it, social support maintained it
o comparison group -> more liberal views in Bennington
 Advantages
o Topics that are hard to study directly
 o Quick and easy
o A lot of information
 Disadvantages
o Possibility of subjectivity in the data
o Non-response (questionnaires only)
o Not a lot of depth (questionnaires only)

4. Participant observation
 the scientists participate in a group’s activities while also studying them
 The Seekers
o cognitive dissonance
o if the actions have already occurred and cannot be changed, we may
modify our beliefs instead, to bring them into line with our actions
o strong pressure
 Advantages
o Valuable information that is not easily accessible
 Disadvantages
o Observer is very intensely involved

Problems with observational research

1. Sampling bias
 an unrepresentative or biased sample of the population
 “self-selected” samples: who have strong opinions, eager to express them
 Solution:
o Random sampling
 Every person in the population has the same probability of
being selected in the sample. -> draw the sample at random
from that population
 E.g.: list a all the member of the population, assign numbers,
draw random sample of numbers
 Difficult if an overview of the population is lacking
 E.g., Dutch house wives/men
 E.g., Visitors of a given supermarket
 Impractical
o If an overview of the population is lacking:
 Multistage sampling
 Province -> city -> street -> house
 Systematic sampling
 E.g., each fifth person that leaves a supermarket
o More practical sampling schemes:
  Purposive sampling
 Search for participants that meet requirements
 Convenience sampling
 A sample that is easily accessible
 Students
o When is sampling bias
o Relatively unimportant:
 Principle of ‘subjects like these’
 If we use a convenience sample, we can reasonably
expect the result to apply to people who are similar to
those subjects in all the ways that matter. We can
generalize. People are much alike in the way their mind
works
 I.e., a “Proof of principle” -> demonstrate something
 E.g., research into the effect of stress on short term memory
 Replication important to study generalizability
o Important:
 If there is a concrete population you want to study
 Always with frequency claims
o E.g. Alcohol abuse at secondary schools in The
Netherlands

2. Observer effects
 the subject may be affected by the presence of an observer, or by something
that he does while observing
 they may not behave as they normally do
 Participant side: People behave differently if they know that they are being
observed
o E.g., Observing children in a class room
o E.g., videotaping family dynamics in someone’s house
 Observer side: The observer affects the results unintended
o E.g., “Rosenthal effect” -> smart rats
o E.g., ‘Clever Hans’
o How the questioner “observer” puts the question affects the replies
 Solutions
o Participant side:
 Hiding: One way mirror, avoid influencing them, “cloak of
legitimacy”
 Waiting out, concealing the presence of them
 Deception: pretend to be converts and actually join
 As in e.g., participant observation
 Ethical questions
 o Observer side:
 Blind observers
The things are there,  Be sure that the observers have no expectations
the observer created o Both:
the effect.  Unobtrusive measures
 Study consequences (traces) of behaviour
You see things that are  E.g., the records of the coffee machine in an office
not there

3. Observer bias:
 You see something that is not there due to your expectations. Distort how we
see it happening
 Solutions:
o Blind observers
o Use objective measures
 E.g.,: time measures, unobtrusive measures, ask about concrete
behaviour / feelings
 Use multiple observers
 ‘inter observer reliability’ » Degree to which two or
more observers agree

4. Inferential bias
 Drawing a conclusion that is not supported by the data
 Correlation does not imply causality:
1. A causes B (watching violent TV series causes problem behaviour)
2. B causes A (problem behaviour results in watching more violent TV
series)
3. The correlation is due to a ‘third variable’ (attention)
o A correlation (r) is a number between -1 and 1
o The correlation coefficient indicates the degree to which two variable
are linear associated
o The closer the correlation approaches -1 or 1, the stronger the linear
relation.