Quantitative research methods

Assoc Prof Julie Pallant School of Rural Health University of Melbourne

Research process
General research topic Aims and research questions Research design

Data collection

Data analysis

Writing the report

Research process
Determine research aims and objectives Choose research design
Overall structure or plan of research

Select specific methods of study

Techniques to collect data (interview, questionnaires) Sample size and methods of sampling Measurement tools (scales, questions) Approaches to data processing and analysis

Two main approaches

Throughout the PHCRED training program we will include sessions on: Quantitative methods Qualitative methods The focus of todays session is quantitative methods

Deductive approaches
generates hypotheses and formally tests these with data quantitative scientific methods

Deduction researcher starts with a theory,

Positivism dominant philosophy underlying

It assumes that phenomena are measurable using the deductive principles of the scientific method Methodologies under this paradigm are referred to as quantitative

Quantitative research
Quantitative research often aimed at testing a research hypothesis. Involve the analysis of numerical data. Hypotheses = propositions about relationships between variables or differences between groups
Eg. Do patients treated with drug A show greater improvement than those treated with Drug B

Other quantitative studies aim to describe characteristics of a sample

Eg. health needs of a local community

Quantitative techniques
Two main approaches: Surveys and questionnaires Experimental designs and intervention studies

Part 1
Surveys and questionnaires

Difference between surveys and questionnaires

Survey: a series of self report measures, administered through an interview or a written questionnaire. Goal is to produce a snapshot of opinions, attitudes, behaviours of a group of people at a given time Often administered as an interview (phone, face to face)
Stangor, 1998, p.97

Surveys and questionnaires

Questionnaire: a set of fixed format, self report items that is completed by respondents at their own pace May produce more honest responses to sensitive topics Cheaper than face to face interviews
Stangor, 1998, p.97

Survey method
What type of research questions can be addressed using survey method? Commonly used in health research to: Establish attitudes, opinions, beliefs concerning health related issues Study characteristics of populations on health related variables (eg. drug use patterns, utilization of health care) To collect information about demographic characteristics of a population Results can be used as bases for hypotheses and theories

Stages in questionnaire design

1. 2. 3. 4. 5. 6. 7. 8. Determine aims of the questionnaire Select appropriate question styles Design questions Pilot test questions Revise questions Administer questionnaire Analyze data Report the results

1. Determine aim of questionnaire

Write down main purpose of study Identify research questions Identify any hypotheses Identify type of information you will need to obtain Consider how the information will be used Prepare an overall questionnaire plan

2. Select appropriate question style

The type of information you wish to collect will determine the type of question style that is suitable Open and closed formats: Open format: respondents can formulate their own answers
Eg. What are the main sources of stress in your life? __________________________

Closed format: respondents are provided with a number of alternative answers

Eg. Gender: male ___ female ___

2. Select appropriate question style

Need to consider the response formats and level of measurement Interval level How many years of formal education have you completed? _______ Ordinal level What is the highest level of qualification you have completed: ___High school certificate ___Undergraduate ___Postgraduate Nominal Since leaving school which of the following activities have you been involved in: ___ further study ___ working full time ___ home duties

2. Select appropriate question style

Closed format questions Should your responses be exclusive (only one answer) or non-exclusive (more than one answer is possible) Consider the implications for coding and analysis

2. Select appropriate question style

Numerical rating scales Likert scales Semantic differential scales Visual analogue scales Rankings See handout Chap 7 Constructing questionnaires in De Vaus D. Surveys in Social Research (5th ed) p. 102-106

2. Select appropriate question style

Numerical rating scales Issues to consider: Number of response options Labels to be used on anchors Inclusion of middle, non-committal answer Not applicable, no opinion, dont know option

3. Design questions
Need to be very careful during the question design stage Consult text books for guidelines See: Chap 7 Constructing questionnaires in De Vaus D. Surveys in Social Research (5th ed)

3. Design questions
Principles of question design
Reliability consistency of response. A person would answer the same way, if given the question on different occasions Ambiguous wording may produce unreliable responses

3. Design questions
Principles of question design
Validity - A valid question is one that measures what we think it does Does the question actually measure what we are asking? Or is it affected by other factors? Eg. does an IQ test actually measure intelligence? (or is the score affected by cultural background, social class etc)

3. Design questions
Avoid: Leading questions Ambiguous questions Double-barelled questions Negative questions Extreme words (all, never, always) Implicit questions Complex vocabulary Patronizing tone Long complex questions Offensive questions

3. Design questions
Response sets

Social desirability
People may answer questions to look good in the eyes of the interviewers Socially desirable behaviours (eg. exercise) are over-reported and undesirable attitudes and behaviours are under-reported

3. Design questions
Response sets

Acquiescent responding
Tendency to respond to questions in a positive manner More common in:
face to face interviews, agree/disagree questions among low education respondents

3. Design questions
Questionnaire layout Clear instructions Order of questions logical flow Use of space Length Coding See handout Chap 7 Constructing questionnaires in De Vaus D. Surveys in Social Research (5th ed) p. 108-112

4. Pilot test questions

Test drive your questions on a small sample of the type of people you intend to study Administer questionnaire and ask for feedback Check for ambiguous questions, poor instructions, response options not included, pages missed

5. Revise questions
Adjust questions and instructions to incorporate feedback from pilot sample Keep a list of alterations, and reasons for changes Check and recheck layout, all instructions, questions, and response options Check that all responses can be coded and analyzed - will the questions provide the data that you need?

6. Administer questionnaire
How you will obtain respondents? What sampling technique will you use? How many respondents do you want/need? Methods of administration Face to face interviews Telephone interviews Postal questionnaires Internet surveys

6. Administer questionnaire
Need to consider strengths and weaknesses of different approaches Response rates Representativeness Quality, honesty, completeness of answers See Chapter 8 Administering questionnaires in De Vaus D.A. Surveys in Social Research

6. Administer questionnaire
Response rate Affected by: Nature of the sample Length of the questionnaire Topic covered Procedures used to recruit participants Data collection approach (interviews, mail out) See De Vaus Chapter 8 for ways to improve response rates to each time of data collection approach

6. Administer questionnaire
Response rate

General population

Highest face to face interviews Lowest postal surveys Similar response levels across interviews, phone and mail, depending on relevance of study to participants Anonymous internet and mail surveys may have higher response rates

Specific groups

Sensitive topics

6. Administer questionnaire
Method of data collection may affect the representativeness of the sample: Need to consider possible sources of bias Is there a problem accessing some people? (eg. those without a phone) Who actually fills out the mailed questionnaire? Characteristics of non-responders

6. Administer questionnaire
Quality of answers Interviews can result in socially desirable responses Need to consider characteristics of interviewer (gender, race, class) in relation to interviewee. Responses to postal and internet surveys can be influenced by the presence of other people

7. Analyze data
You must consider how you will analyze the responses to your questionnaire This will help with the design of the questionnaire Consider how you will report the findings graphs, descriptive statistics, more advanced inferential statistics

8. Report results
The way that you report your findings will depend on the purpose of the study, and the audience You may need more than one report to meet the needs of different audiences (eg. organizational needs vs academic journal article)

Part 2
Experimental designs and intervention studies

Experiments
Enables researchers to demonstrate how manipulating one set of variables (independent variable) produce systematic changes in another set of variables (outcome or dependent variables)
See Chapter 5 in Polgar and Thomas, 2008

Common elements
Definition of population to generalize to Selection of sample representative of the population Procedures to assign cases to each group (intervention/ control) Administration of intervention (treatment) Measurement of outcomes

Different experimental designs

Post test design Pre-test/post-test design Repeated measures Common experimental technique in the medical literature Randomized control trial (RCT)

Post test only design

Weak design Involves measuring outcome of interest after the intervention No pretest scores so it is not possible to assess change

Pre test/ Post test design

Measurements are taken both prior to the intervention and again afterwards Allows detection of change in individuals Not able to assume that the change is due just to the intervention (may be other factors involved) Just administering the measures may change behaviour (eg. questions on eating habits, practice effects)

Repeated measures designs

Each person may receive a number of different treatments and is assessed a number of different times

Randomized control trials

Patients are enrolled in the study and then randomly assigned to either
(a) the intervention group (b) control group

Intervention group receive the experimental treatment Control group receives placebo, or standard care Considered gold standard for studying interventions as they minimize bias

Epidemiological research designs

Epidemiology The study of factors that determine the occurrence and distribution of disease in a population Population oriented epidemiology studies community origins of health problems (eg. nutrition, environment) Clinical epidemiology studies patients in health care settings to improve diagnosis and treatment of diseases

Epidemiological research designs

See Chapter 5 in Epidemiology Biostatistics and Preventive Medicine Cross sectional surveys Ecologic studies Cohort studies (retrospective, prospective) Case-control studies

Cross sectional surveys

Survey of a population at a single point in time Quick and easy way to determine prevalence of risk factors of disease for a given population Data only collected on 1 occasion so cannot determine causal relationship

Ecologic studies
Studies the frequency with which some characteristic (eg. smoking) and an outcome of interest (eg. lung cancer) occur in the same geographic area Useful for generating hypothesis, but not establishing causal relationship Eg. this approach identified the co-occurrence of fluoride in the water and lower levels of teeth cavities in children

Cohort studies
Prospective cohort study Group of people assessed for risk factors and followed over time to see if they develop a specific disease Retrospective cohort study Investigator goes back in time to define a risk group (eg. people exposed to atomic bomb) and follows group to present day to see what diseases have occurred

Case-control studies
Investigator selects the case group (people with disease) and the control group (people without disease) and compares the groups in their frequency of past exposure to possible risk factors

Activities for today

Break into small groups and discuss the following issues: What would be advantages and disadvantages of a quantitative approach in relation to your topic? Discuss how you might implement a quantitative approach to address your research question Write some questionnaire items that might be used to address your research question. Use a variety of different of question styles.

Activities for today

What information would be obtained from these questions how would you report the results Discuss how the questionnaire might be administered (mail, internet, phone). Weigh up the advantages and disadvantages of the different approaches Consider how you might ensure a good response rate