Introduction to Surveys

Definition of survey

Study involving a subset (sample) of

individuals selected from a larger
population
where variables are observed or measured
at a single point in time
aggregated over all individuals in the
sample
to obtain a summary statistic

1
Observational studies

exposure outcome cohort

Cause-effect

case-control
outcome exposure
Effect-cause

outcome exposure cross-sectional

Extension of cross-sectional design

Longitudinal
multiple cross-sectional surveys
can estimate incidence
if results between multiple surveys differ
true difference?
samples very different?
Panel design
multiple cross-sectional surveys using exactly
same sample
Pseudo-cohort
cohort: one age group over time
pseudo-cohort: different age groups studied cross-
sectionally at one point in time

2
Survey
To investigate
if taxi drivers are more likely to use seat belts
than drivers of private automobiles

To determine the community prevalence of

malnutrition
measles
Tuberculosis
HIV

Steps in designing a survey

Define survey objectives
Identify study population
Identify and define study variables
Establish plan of analysis
Determine sample size
Choose sampling method
Decide on mode of data collection
Design, field test and revise questionnaire
Train interviewers, conduct field work
Check all interview forms for error
Enter, tabulate and analyse the results

3
Survey objectives

Formulate the objectives of the survey:

What are you interested in finding out?

Who do you want to study?

When do you want to do the survey?

What do you expect to learn and why?

Attributes of SMART objectives

S pecific
M easurable
A ction oriented
R ealistic
T ime frame

4
Example 1: Objectives

What? What are the knowledge, attitudes

and practices (KAP) to HIV/AIDS
Who? of adults > 18 years of age
Where? in Mwananyamala
When? in year 2006
Why? prior to the start of a new health
education programme

Example 2: Objectives

To measure and compare the prevalence of

smoking among MOH and Ministry of
Agriculture employees in Tabora

To measure and compare the proportion of

non-smokers in MOH and MOA exposed to
tobacco smoke in their working environment
and to determine the extent of the exposure

5
Steps in designing a survey
Define survey objectives
Identify study population
Identify and define study variables
Establish plan of analysis
Determine sample size
Choose sampling method
Decide on mode of data collection
Design, field test and revise questionnaire
Train interviewers, conduct field work
Check all interview forms for error
Enter, tabulate and analyse the results

Study population

(Target-) Population

Sample (study population)

6
Study population
Objectives target population
Study population must be representative of
target population
Time: seasonality, day of the week, time of the
day
Place: urban, rural
Persons: age, sex, other characteristics
A non-representative sample will produce
biased results (validity )

Steps in designing a survey

Define survey objectives
Identify study population
Identify and define study variables
Establish plan of analysis
Determine sample size
Choose sampling method
Decide on mode of data collection
Design, field test and revise questionnaire
Train interviewers, conduct field work
Check all interview forms for error
Enter, tabulate and analyse the results

7
Study variables

Identification
as many as necessary but as few as possible
each must pay its own way
if of no use in analysis, do not include variable
Requirements
clear definition
method of measuring should be:
repeatable
valid

Identify study variables

Objective: To measure and compare the prevalence of

smoking among MOH and MOA employees in Tabora
Variables:
Smokers
in MOH

in MOA

Definition:
A smoker is someone who, at the time of the survey, smokes
cigarettes or any other tobacco products either daily or
occasionally.
daily: smokes any tobacco product 1 / day
occassionally: smokes, but not every day

8
Steps in designing a survey
Define survey objectives
Identify study population
Identify and define study variables
Establish plan of analysis
Determine sample size
Choose sampling method
Decide on mode of data collection
Design, field test and revise questionnaire
Train interviewers, conduct field work
Check all interview forms for error
Enter, tabulate and analyse the results

Establish plan of analysis (POA)

How will the key variable(s) be measured
to meet the objectives?
Proportion, mean, ratio, ...
Level of measurement influences
questionnaire design
type of statistical analysis
What additional variables will need to be
measured?
How will the results be used in the analysis
(dummy tables)?

9
 Plan of analysis
To measure and compare the prevalence of smoking among MOH and MOA
staff in Tabora

Variables obtained from all study participants

Number (%) of persons smoking
- daily
- occasionally
Number (%) of persons smoking cigarettes, pipefuls of tobacco, or cigars /
cigarillos
Number (%) of nonsmokers who are
- neversmokers
- exsmokers (and time since quitting)

Variable obtained from smokers

Average number of cigarettes, pipefuls of tobacco, cigars / cigarillos smoked per
day or week

Dummy table - describe sample

Baseline characteristic of study population in UNHCR and UN
--------------------------------------------------------------------------------------------------------
MOH MOA
N=X N=Y
--------------------------------------------------------------------------------------------------------
number (percent)
Age*
30 years or younger
31 - 45 years
46 years or older
Female
Work status
Staff member
Short Term Professional
Short Term Consultant
Others

10
Dummy table - describe variables
Prevalence of smoking among staff of UNHCR and UN
------------------------------------------------------------------------------------------------------
MOH MOA
N=X N=Y
------------------------------------------------------------------------------------------------------
number, percent
Smokers
smoking daily
smoking occasionally
total n=x n=y

Nonsmokers
neversmokers
exsmokers
quit 1 month ago
quit 2-6 months ago

Steps in designing a survey

Define survey objectives
Identify study population
Identify and define study variables
Establish plan of analysis
Determine sample size
Choose sampling method
Decide on mode of data collection
Design, field test and revise questionnaire
Train interviewers, conduct field work
Check all interview forms for error
Enter, tabulate and analyse the results

11
Estimating sample size
Indicate expected frequency of factor of interest
Decide on desired precision of the estimate
Decide on acceptable risk that estimate will fall
outside its real population value
Adjust for estimated design effect
Adjust for expected response rate
(Adjust for population size, if sample > 10% of
target population)

Methods used in probability samples

Simple random sampling

Systematic sampling
Stratified sampling
Cluster sampling
Multi-stage sampling

12
Steps in designing a survey
Define survey objectives
Identify study population
Identify and define study variables
Establish plan of analysis
Determine sample size
Choose sampling method
Decide on mode of data collection
Design, field test and revise questionnaire
Train interviewers, conduct field work
Check all interview forms for error
Enter, tabulate and analyse the results

Method of data collection

Personal interview
Telephone
Self administered
Questionnaire
Mail
Computer direct
E-mail

13
What method to choose?

Nature of questions
What is the research question to be
addressed?
Convenience
Which method is most readily available?
Cost
How much money is available?

Personal interviews
face-to-face
Advantages
population coverage
can clarify issues, allows complex questions
longer interviews tolerated (2-3 hours)
usually high response rate (~90%)
Disadvantages
expensive
slow
intrusive
requires interviewing skills
interviewer bias

14
Self-administered questionnaires
(mail or hand distributed)
Advantages
cheap
no interviewer bias
long list of response categories possible
sensitive questions
Disadvantages
slow
does not allow clarification
poor response rate (particularly in populations
of lower education and literacy level)
population coverage?
who completes questionnaire??

15
Telephone surveys
Averages advantages and disadvantages of
personal and mail surveys
Advantages
relatively fast
sampling through random digit dialling
Disadvantages
need telephone
problem with answering machines, mobile
telephones
no one at home during the day. Limits calling time
to a window of 6-9 p.m.
interview time < 20 minutes

16
Summary interviewing methods

Personal Telephone Self

Cheap x + ++
Speed x + xx
Population coverage + xx xx
Complex questions + x xx
Sensitive questions x + ++
Length of interview + x xx
Response rate + x xx

17
Steps in designing a survey
Define survey objectives
Identify study population
Identify and define study variables
Establish plan of analysis
Determine sample size
Choose sampling method
Decide on mode of data collection
Design, field test and revise questionnaire
Train interviewers, conduct field work
Check all interview forms for error
Enter, tabulate and analyse the results

Preparing for data analysis

Remember GIGO principle

Garbage in - garbage out
The computer cannot make data better than
they are
Coding
Data entry
Data cleaning and editing

18
Data analysis

Type of data analysis is dependent on the

of outcome variable
Continuous variables
Categorical variables

Points about survey data analysis

Target population in a survey often finite

Most software packages assume:
infinite population sizes
sampling with replacement
both conditions often not upheld in surveys
Software specially developed for surveys
SUDAAN (Survey data analysis)
Stata
Epi Info CSurvey

19
Ethical issues
Informed consent
purpose of study explained?
consent given by individual
to be included?
Confidentiality
an individual should not be identifiable
Service provision

No survey without service (?)

20
Response rate
Dependent on
length and mode of interview
educational level
perceived threat of questions
warm versus cold interviews
warm = interviewee has prior knowledge about
survey
Non-responders are almost always
different from responders
To reduce non-response bias
follow up non-responders at least once

Use of results

Biggest fault in survey design: Results are

not disseminated or used.
Why was the suryey carried out in the first
place?

21