Objec&ves

Understand the concept of Research Design Identify the characteristics of some common

quantitative research designs Dene and discuss research control, and internal and external validity as these terms relate to research design Critique a written research report in terms of design appropriateness and accuracy of design implementation

What is Research Design?

the overall approach best suited to answer specic

questions and achieve the stated purpose of a research project. (Powers & Knapp, 2011).

the overall method of the research study. Should ow from the stated purpose Should provide a plan to answer the research questions and test the stated hypothesis (Schmidt & Brown, 2012).

Review What is a Variable?

Variable the operationalization of a concept Classifying variables/Variable types

Independent (IV) The variable that is thought to be the cause In experimental research, the researcher manipulates this variable to see how that eects the results Dependent (DV) The outcome that is being studied. This is the variable that is of principle interest in the study

More on Variables
Other variable types Intervening A variable that is between an IV and a DV in a causal sequence and so eects the causal relationship

Extraneous Variable A potentially confounding variable

that is not of interest to the study

Extraneous variables must be controlled because they can

confuse or confound the relationship between the independent and dependent variable

Failure to control extraneous variables leads to bias which results in invalid results

Quan&ta&ve Designs: 2 Broad Categories

Nonexperimental Designs
Describe measurable

Experimental Designs
Examine causality Does X cause Y?

phenomenon Explain relationships among variables Describe dierences among variables Predicting relationships and dierences among variables

Quan&ta&ve Design: Important Concepts

Causality
Denition: The relationship that

Probability
The likelihood or chance that

exists between a cause and its eect (Schmidt & Brown 2012). 3 conditions for establishing causality:
There must be a temporal

an event will occur in a given situation (Schmidt & Brown, 2012)

In research on humans,

relationship

The relationship between

Example: X must precede Y in time

variables must be strong No other variables or conditions can account for the relationship between X & Y

causality is never completely certain Idea of probability or likelihood of a cause eect relationship is more common in nursing research

Cause, Eect, Probability Its not that simple!

Theoretical Cause and

Eect Relationship:

Reality: Any eect (Y)

is the result of multiple contributing causes

Quan&ta&ve Design: Important Concepts

Control
The researchers attempt to

Manipulation
Control/intervention of the

clarify the relationship between an independent and a dependent variable by eliminating other things (variables) that might aect the relationship
Eliminating outside

inuences by close control of conditions Using statistics to eliminate the eects of other variables

independent variable by the researcher Dening characteristic of an experimental study Done by creating dierent experimental conditions
Examples

Dividing subjects into two or more groups subjected to dierent conditions

Controlling Bias in Research Studies

Bias results when extraneous variables inuence the

relationship between the IV and the DV Controlling for bias:

Randomization Selection, assignment or arrangement

of elements by chance in an attempt to eliminate or ameliorate the eects of extraneous Two major methods

Random Sampling Random Assignment

Methods of Randomiza&on
Random Sampling
Methods that assure that

Random Assignment
Methods that assure that

every person in the population has an equal chance of being in the sample

This assures that sample is

every subject in the study has an equal chance to be in either the experimental or the control group
Increases the likelihood that

similar to the population it is taken from

Examples of randomization Coins Table of random numbers Computer generated lists

extraneous variable are evenly distributed between the two groups and do not disproportionately aect one of the groups

Two more important terms

Between Groups
Research study compares the

Within Groups
Compares a group with itself on

two groups of subjects on some variable(s).

Examples:

Experimental studies Studies comparing two pre- existing groups on a certain variable

some variable(s) at more than one point in time. Often the group is measured pre and post intervention
Examples:

BMI of persons with Type 2 Diabetes versus persons without Diabetes Incidence of lung cancer among smokers versus non- smokers

Hemoglobin A1C of individuals with Type 2 Diabetes before and after 3 months of Metformin Self-esteem of teen girls before and after participating in a targeted after-school intervention

Study Validity
Validity: The extent to which the results of the study
Accuracy of the study ndings Validity is ensured by the researcher through careful

are true, logical, reasonable & justiable based on the evidence presented in the study
methods which control for extraneous variables

Careful selection of subjects Meticulous attention to study procedures/protocols Accurate measurement of study variables Correct statistical methods Accurate interpretation of results

4 Types of Study Validity (Schmidt & Brown, 2012)*

* *

Threats to Validity
A threat (to validity) is anything that weakens the

internal or external validity of a study

Nurses need to be able to evaluate studies to

determine if the results are true. (no or minimal threats to study validity)

What threatens Internal Validity?

History Something outside the study changes and this changes results (dependent variable) This problem is most likely to aects research that takes place over time Control group is best defense Testing Measurement itself aects results

Repeated testing causes familiarity with test Use of pretest aects post-test results
16

What threatens Internal Validity?

Instrumentation A change in the way a variable is measured during the course of a study can aect validity

Timing of measurements How the variable is measured (type of instrument) Change in data collectors

Subject mortality Loss of subjects due to any factor related to the dependent variable

Death Study withdrawal Loss to follow-up

Selection bias Subjects in experimental and control groups are not similar enough This problem aects studies without random selection the most
17

What threatens Internal Validity?

Maturation Subjects change due to the passage of time rather than due to the experimental intervention

This problem is most likely to aects research that takes

Aging Growth Disease progression

Statistical Conclusion Validity Type II errors: Researchers conclude that there is no dierence or relationship, when there truly is one

place over time

This is a risk with small sample sizes

Unreliable measurement instruments

What threatens External Validity?

Reactivity Hawthorne Eect

Subjects behavior dierently because they know they are in a study This is usually unconscious on their part

Experimenter Eect Some characteristic of the researcher or data collector aects the results Example: The person administering the intervention is pretty and nice so the subjects try to please her Double-blind study can help ameliorate this eect
19

What threatens External Validity?

Construct Validity Instruments used to measure traits must be evaluated to make sure that they are measuring that trait alone

Example: Measures of anxiety might overlap with measures of depression

Novelty Eect (not in your book) Subjects respond dierently because they know the intervention or treatment is new or dierent It is the novelty not the actual intervention that is causing change
20

What threatens External Validity?

Selection Eects Samples that do not represent the appropriate population limit external validity

Interaction Eects Treatment and subject selection

This is minimized by the researcher selecting an appropriate sample for the target population, using random selection of subjects and random assignment to groups

The IV may not aect all individuals in the way it does in the study. This may be due to dierences between the study sample and other groups The results of the study may be related to the specic setting and may not be applicable to other settings The results of the study may be aected by events occurring at the same time as the study but outside of the study

Treatment and setting

Treatment and history

Validity Paradox
Internal Validity Internal Validity External Validity External Validity
22

Fundamentals of Design: Review: What is Research Design?

Research Design is the overall plan for obtaining

answers to the research questions or for testing the research hypotheses.

Spells out basic strategies to insure that results are

accurate, interpretable and useful

Sample/sampling Data collection and management Planned statistical analysis Measures taken to maintain rigor and control threats to validity

25

More Review
The research design must t the study purpose,

question and/or hypothesis

26

How are Research Designs Classied?

By Time Element Retrospective Longitudinal/Prospective Cross-Sectional By Type Nonexperimental Experimental

Quasi-Experimental

By Function The research design must t the study purpose, question and/or hypothesis

What is the researcher trying to do?

27

Classica&on by Time Element

Retrospective/Ex Post Facto Designs

Data collected about past events May be more than one data point

Looks at what happened to understand a current phenomenon

Longitudinal/Prospective Designs

Data collected at more than one point in time beginning at the start of the study Repeated Measures same instruments used to collect data a all measurement points allowing for comparison over time. Allows researcher to examine things in real time Data collected all at one point in time Cannot be experimental Theory important to the interpretation of these studies Whats happening NOW!

Cross-Sectional

Does not give any information about changes over time

28

More on Designs X Time Elements

Retrospective/Ex Post Facto Designs Case-Control Studies

Use data on patients with a condition that is collected by patient history to determine a cause
HIV/AIDS, Smoking, Type 2 Diabetes, Hypertension

Longitudinal/Prospective Cohort Studies follows group(s) forward over time and learns what conditions develop in the groups

Crossover study Same group of subjects receive more than one

Women who take HRT vs women who dont Persons with diabetes whose sugar is tightly controlled versus those whose sugar is not

Cross-Sectional Designs Cohort Comparison Studies look at more than one group of subjects at the same point in time

treatment over time or serve as their own control groups

Sexual behavior in teens who go to public school versus religious schools Depression in women who had a TOP versus those who have not Weight in students who eat in the dining halls versus those who cook for themselves

29

Classica&on by Type
Nonexperimental (Descriptive) Studies as is No manipulation of variables Can be qualitative or quantitative Experimental & Quasi-experimental Study what would happen if Involves manipulation of variable(s) Always quantitative Control important Always prospective/longitudinal
30

Nonexperimental Designs
Function is description Help us understand things as they are currently Describe phenomenon Explain relationships and dierences Predict relationships and dierences Do not claim to indicate cause and eect No manipulation of independent variable Research control still important Can be quantitative or qualitative
31

Nonexperimental Designs
Descriptive Designs Correlational Designs
Qualitative or quantitative Cross-sectional, retrospective
Descriptive Correlational describe

or longitudinal/prospective Explorative when little is know Comparative compares 2 or more group Survey Data obtained by self-report, such as by phone or in-person survey

variables in 2 groups or relationships among 2 variables Predictive Correlational tests relationships between one or more variables
Very theory based Non-directional or directional

Model-Testing Correlational Tests

hypotheses used Variables referred to as research variables or predictor and outcome variables rather than IV and DV
Relationships among multiple variables

hypothesized theoretical model

in the model are tested all at once or in pairs Resulting diagram shows relationships and is called a Path Model.

32

Sample Path Model: Psychological Adjustment to Breast Cancer Diagnosis

Epping-Jordan, J. et al. (1999). Psychological adjustment to breast Cancer: Process of emotional distress. Health Psychology, 18(4), 315-326.

34

Experimental Designs
3 Essential Elements of True Experimental Design 1. Randomization

Random selection Random assignment

2.

Control
Extraneous variables are controlled

3.

Manipulation of the IV

35

 What is Research Control?

Attempts by research to control conditions in order

to decrease variability (change) in the D.V. that is caused by events or factors independent of the study design
Extrinsic Factors

related to environment/research situation related to characteristics of subjects themselves

Intrinsic Factors

36

What is elements must be Controlled?

Areas of Research Control Sampling

Research Conditions Same for all subjects Data Collection Careful choice of reliable and valid instruments Use of comparison or control groups Uniformity of data collection agents and conditions Data collectors are all trained to do the same things Data Management
4/4/13 37

Inclusion & exclusion criteria Random sampling Random assignment to groups

6 Types of True Experimental Design

1. 2-group, pretest-post test 2. 2-group, posttest only 3. Solomon 4-group design 2 groups get intervention and two are control See next slide 4. Multiple experimental groups 2 experimental groups with one control group 5. Factorial Tests >1 intervention can be tested at same time 6. Crossover Design 2 treatments and subjects receive both treatments in random order

Common Experimental Designs

Common Experimental Designs

Quasi-Experimental Designs
Do not meet all the essential components of

experimental design

Manipulation of the IV is always included May not include: Randomization Some elements of research control may be missing

3 Common Quasi-Experimental Designs Nonequivalent control group pretest-posttest Time series designs Preexperimental designs

Common Quasi-Experimental Designs

43

Review of Classica&on of Quan&ta&ve Design

4/4/13

44

Understanding Samples and Sampling Procedures

4/4/13

46

Objec&ves: Samples and Sampling

Evaluate a research report for appropriateness of

sample and sampling methods Evaluate research for adequacy of sample size Utilize principles of evidence-based nursing to determine if the research ndings are applicable to a clinical problem based on sample composition

4/4/13

47

What i s a popula&on? The total number of In research terms:

persons inhabiting a country, city, or any district or area.
(www.dictionary.com)

An entire set of

people, or hospitals or whatever is of particular interest to a researcher (Powers &

Knapp, 2011)

Sometimes called the

study population or the target population

48

4/4/13

Examples of Popula&ons that might be used for Studies??

4/4/13

49

Study/Target Popula&on
Dened by a set of

specic characteristics that they have in common

Can you think of some

examples of populations from the research articles you have read for this class?
50

4/4/13

Accessible Popula&on
Term used to indicate the part of the population that

a researcher has access to Example:

Study population: All men over age 50 with

hypertension Accessible population: Men over age 50 with hypertension who attend a large clinic in a certain city

4/4/13

51

Studying Popula&ons
A researcher wants to study the growth and

development of very premature infants during their rst 5 years of life. 5,000 very premature infants are born in her state every year. What do you think she does?

4/4/13

52

How is sampling like making soup?

4/4/13

53

What is a sample?
A subset of a population that is studied to learn

something about the population Ideally the sample has the same specic characteristics as the population
REPRESENTATIVE

A sample that meets this criterion is called

In this illustration, which circle represents the population and which represents a of the population?

4/4/13

54

Generalizing: Sample to Popula&on

In Quantitative Research you study a sample to learn

about a population

What type of statistics allow you to take the results

you got using a sample and generalize (accurately apply) them to the population?

4/4/13

55

1948 Election: Truman holding newspaper lauding his defeat after he won the election.

Famous Sampling Error

How do you think this happened?

4/4/13

56

 The poll only sampled rich white people with phones. Guess who voted for Truman?

4/4/13

57

Sampling techniques are used to:

Select a sample that is as like the population as possible Avoid sampling bias Control for extraneous variables extraneous variables

(Review: extraneous variables are those that are not being measured in the study but that might aect study results). Example: A researcher wants to measure the eect of an educational program related to sugar intake on teens BMI and blood sugar levels Examples of extraneous variables: 1. Unknown to her, a beverage distributor begins giving out free sweetened iced tea during after-school activities. 2. Some of the teens have family members with Type 2 diabetes and she does not know this.
58

4/4/13

Goal of Sampling
Quantitative Research To decrease variability among the subjects. Subjects should be as alike as possible to avoid pre- existing dierences that might aect results Qualitative Research Tries to increase variability among the subjects to learn as much as possible about a topic. Try to get subjects who represent as many dierent aspects of the subject under study in order to get all possible views.

Purposive or Theoretical Sampling

59

4/4/13

Sampling Decisions are Cri&cal in Quan&ta&ve Research

Once target/study population is known, the

researcher tries to choose the sample that is best for the study.
Most like or representative of the population

Sampling criteria are used to select sample that is as

representative as possible

4/4/13

60

Inclusion vs Exclusion
Inclusion Criteria Characteristics that a person must have to be included as a subject

E.g. age, specic gender, pre-existing condition, English-speaking

Exclusion Criteria Characteristics that, if present, exclude the person from participation

E.g pre-existing condition, already being treated, age, pregnancy

Inclusion and Exclusion criteria are determined by the

researcher based on what is being studied

Also sometimes for practical reasons

4/4/13

61

Sample Types in Quan&ta&ve Research

Sampling method for Quantitative Research Quantitative Research

Probability Sampling All individuals in the population have an equal chance of being included Uses some method of selecting subjects at random Non-Probability Sampling Not everyone has the same chance of being included Used when random sampling not possible May result in less than representative sample

4/4/13

62

Probability Sampling
Random sampling should result in a sample like the

study population

Characteristics (age, gender, disease status) distributed

equally among the groups

4/4/13

63

Quan&ta&ve Sampling: Probability Sampling

Simple Random samples are computer-generated or table of random numbers or similar Stratied Random Subjects are divided by some characteristic (e.g. age, clinic, race) and then random sampling is done within the strata Cluster Sampling Random selection of subjects from within known clusters of individuals with the characteristic

Example: subjects with prostate cancer from 6 large cancer centers. Each clinic group is a cluster. The subjects are then randomized within each cluster.

Systematic Sampling Uses a list of all members of an accessible population. Population size divided by sample size and then every Kth individual selected from the list. (Not true random sample as not all have equal chance of inclusion)
4/4/13 64

Quan&ta&ve Sampling: Non-Probability Sampling

Unlikely to get a truly representative sample with

non-probability sampling

Convenience Sample Uses all available subjects Example: all patients who attend medical clinic on a certain date Quota Sample Similar to convenience sampling but uses a strategy to make sure certain subject types are included Example: Make sure distribution of ages or genders meets a criteria
4/4/13 65

When a probability sample is not possible:

Random Assignment to Groups Next best thing to random sampling Uses a non-probability sample Randomly assigns subjects in the sample to groups

4/4/13

66

 Size Matters! Larger samples are more likely to be representative of (like) population of interest Larger samples are more likely to result in statistical signicance.

Sample Size in Quan&ta&ve Research

Power Analysis A statistical procedure that allows the researcher to compute the minimum # needed in sample to detect a real dierence or relationship

4/4/13

Results using smaller samples more likely to be the result of chance.

Look for this when reading quantitative research.

67

Sampling Bias Poor sampling can skew the results of the study.
Random sampling is best way to avoid bias

3 potential sources of bias in sampling Researcher (or subject recruiter) chooses certain types of subjects

Mostly unconscious. May select people they feel comfortable approaching for example

Self-selection by individuals in the population People who are interested in the research, or who want to support research Limited response rate (survey research) Similar to self-selection in a way. People who are more interested will complete and mail in survey.
4/4/13

68

What about Mortality

Subject Mortality = loss of subjects during the

course of a study for any reason

What do you think can cause subject mortality?

4/4/13

69

Mortality and Subject Bias

Attrition may indicate the Researcher should

existence of sampling bias.

Self-selection Failure to complete

discuss attrition in reporting study and explain potential reasons for it.

survey or study

4/4/13

70

Evalua&ng Quan&ta&ve Samples for EBN

1. Was there an identiable section on sampling in

the research report/article? 2. Was enough information given to understand how and why sample chosen? 3. Is there enough information given about the sample to tell if the research applies to your clinical population? 4. For quantitative research:
large enough.

Is there information that tells me that the sample is

4/4/13

71