Educational Research:

Data analysis and interpretation

Inferential statistics
 Statistics...

A set of mathematical procedures for

describing, synthesizing, analyzing, and
interpreting quantitative data
the selection of an appropriate
statistical technique is determined
by the research design, hypothesis,
and the data collected
 inferential statistics...
mathematical tools that permit the
researcher to generalize to a
population of individuals based
upon information obtained from a
limited number of research
participants
 Review of Sampling
Population group of people, communities, or organizations
studied. Includes all possible objects of study.

Sampling frame list of people/organizations etc. in the

population who can be chosen for participation in the study.
Most sampling frames do not include all people in the
population (example phone book)

Sample part of the population. Reduced down to

manageable size. Ideally we would want to draw a sample
that is representative of the population in terms of certain key
characteristics (for example, gender and age).
 Important information about samples
For qualitative research, we are looking at specific
situations. It may not be important to have a
representative sample. We often use nonprobability
sampling with qualitative methods (snowball,
purposive, or convenience samples).
For most types of quantitative research we do want a
sample that is representative of the population. We
will want to generalize our findings from the sample
to the population.
To generalize means that we can
say that we would expect to have
the same findings if we studied
everyone in the population as we
did when we looked at the
sample (within a certain degree
of probability)
 In studies in which we will generalize from
the sample to the population
We must have a sample that is similar or the same on specific
dimensions as the population.
We will want to use inferential statistics to analyze our data so
that we can infer that findings from a sample are the same as
those we would get from the population.
Theoretically, we must have a normal distribution in order to
use inferential statistics.
We will use sampling methods in which every respondent has
a known probability of selection (probability sampling)
The best type of sampling method to use with inferential
statistics is that in which each participant has an equal
probability of selection (random sampling).
 Exceptions to this Rule
The population under study is small enough that
everyone can be selected for participation (this still
allows you to use inferential statistics)
Certain types of applied research using quantitative
methods such as community needs assessments and
some types of surveys in which it is simply important
to have as many people respond as possible.
However, we will not be able to generalize our
findings to the population.
 Types of sampling

Non-probability samples

Probability samples
 Non probability samples
Convenience samples (ease of access)
sample is selected from elements of a population that
are easily accessible
Snowball sampling (friend of friend.etc.)
Purposive sampling (judgemental)
You chose who you think should be in the
study
Quota sample
 Non probability samples
Probability of being chosen is unknown
Cheaper- but unable to generalise
potential for bias
 Probability samples

Random sampling
Each subject has a known probability of being
selected
Allows application of statistical sampling
theory to results to:
Generalise
Test hypotheses
 Probability samples are the best

Ensure
Representativeness
Precision
Methods used in probability samples

Simple random sampling

Systematic sampling
Stratified sampling
Multi-stage sampling
Cluster sampling
Simple random sampling
 Table of random numbers

684257954125632140
582032154785962024
362333254789120325
985263017424503686
 Systematic sampling

Sampling fraction
Ratio between sample size and population size
Systematic sampling
 Cluster sampling
Cluster: a group of sampling units close to each
other i.e. crowding together in the same area or
neighborhood
 Cluster sampling
Section 1 Section 2

Section 3

Section 5

Section 4
 Stratified sampling
Multi-stage sampling
 sampling error...
the differences in samples due to
random fluctuations within the
population
 Errors in sample

Systematic error (or bias)

Inaccurate response (information bias)
Selection bias

Sampling error (random error)

 Type 1 error
The probability of finding a difference with our
sample compared to population, and there
really isnt one.

Known as the (or type 1 error)

Usually set at 5% (or 0.05)

 Type 2 error

The probability of not finding a difference

that actually exists between our sample
compared to the population

Known as the (or type 2 error)

Power is (1- ) and is usually 80%

 Sample size

Quantitative Qualitative

Z 2 2 Z 2 (1 )
n n
D2 D2

(12 22 )xF 2 P (1 - P) F
n n
D 2
D2
 Qualitative Research
-is primarily exploratory research. It is used to gain an
understanding of underlying reasons, opinions, and motivations.
It provides insights into the problem or helps to develop ideas or
hypotheses for potential quantitative research.
-Qualitative Research is also used to uncover trends in thought
and opinions, and dive deeper into the problem. Qualitative data
collection methods vary using unstructured or semi-structured
techniques.
-Some common methods include focus groups (group
discussions), individual interviews, and
participation/observations. The sample size is typically small, and
respondents are selected to fulfill a given quota.
 Quantitative Research
is used to quantify the problem by way of generating
numerical data or data that can be transformed into useable
statistics.
It is used to quantify attitudes, opinions, behaviors, and other
defined variables.
Quantitative data collection methods are much more
structured than Qualitative data collection methods.
Quantitative data collection methods include various forms of
surveys online surveys, paper surveys, mobile surveys and
kiosk surveys, face-to-face interviews, telephone interviews,
longitudinal studies, website interceptors, online polls, and
systematic observations.
 Problem 1
A study is to be performed to determine a
certain parameter in a community. From a
previous study a sd of 46 was obtained.
If a sample error of up to 4 is to be accepted.
How many subjects should be included in this
study at 99% level of confidence?
 Answer
2
Z 2
n
D 2

2.582 x 462
n 880.3 ~ 881
4 2
 Problem 2
A study is to be done to determine effect of 2
drugs (A and B) on blood glucose level. From
previous studies using those drugs, Sd of BGL
of 8 and 12 g/dl were obtained respectively.
A significant level of 95% and a power of 90%
is required to detect a mean difference
between the two groups of 3 g/dl. How many
subjects should be include in each group?
 Answer
( )xF
2 2
n 2
1 2
D

(8 12 )x10.5
2 2

n 2
242.6 ~ 243
3
in each group
 Problem 3
It was desired to estimate proportion of anaemic
children in a certain preparatory school. In a
similar study at another school a proportion of
30 % was detected.
Compute the minimal sample size required at a
confidence limit of 95% and accepting a
difference of up to 4% of the true population.
 Answer
Z (1 )
2
n 2
D

1.96 x 0.3(1 0.3)

2
n 2
504 .21 ~ 505
(0.04)
 Problem 4
In previous studies, percentage of hypertensives
among Diabetics was 70% and among non
diabetics was 40% in a certain community.
A researcher wants to perform a comparative
study for hypertension among diabetics and
non-diabetics at a confidence limit 95% and
power 80%, What is the minimal sample to be
taken from each group with 4% accepted
difference of true value?
 Answer
2 P (1 - P) F
n
D2

2 x 0.55 (1 - 0.55) x7.8

n 2
2413 .2
0.04
Precision
Cost
sampling errors vary in size
but are normally distributed around
the population mean (M)
and take the shape of a bell curve
 standard error...
the standard deviation of the sample
means (SEx)
tells the researcher by how much
the researcher would expect the
sample means to differ if the
researcher used other samples
from the same population
 but...
the researcher does not have to
select a large number of samples
from a population to estimate the
standard error
 a mathematical formula can be used to
estimate the standard error...
SD .

SEx = N - 1
a smaller standard error indicates
less sampling error
the major factor affecting the size
of the standard error of the mean is
sample size
but, the size of the population
standard deviation also affects the
standard error of the mean
 Important concepts about sampling
distributions:
If a sample is representative of the population, the mean (on
a variable of interest) for the sample and the population
should be the same.
However, there will be some variation in the value of sample
means due to random or sampling error. This refers to things
you cant necessarily control in a study or when you collect a
sample.
The amount of variation that exists among sample means
from a population is called the standard error of the mean.
Standard error decreases as sample size increases.
 The null hypothesis (H0)...

the statement that the difference between

two sample means is due to random,
chance, sampling error
indicates that there is no true
difference or relationship between
parameters in the populations
 the null hypothesis differs in most instances
from the research hypothesis (H1)

which states that one method is

expected to be more effective than
another
 rejecting the null hypothesis provides
evidence (but not proof) that the treatment
had an effect
in other words, that the difference
between dependent variables is
due to something other than
random, chance, sampling error
 The research question, then, is:
whether to accept the null
hypothesis or to reject it
 There are four possibilities:
1. The null hypothesis is true and the
researcher concludes that it is true
A = Ba correct decision
2. The null hypothesis is false and the
researcher concludes that it is false
A Ba correct decision
3. The null hypothesis is true but the
researcher concludes that it is false
A = Ban incorrect decision
4. The null hypothesis is false but the
researcher concludes that it is true
A Ban incorrect decision
 Decisions concerning rejecting the null
hypothesis

The true status of the null hypothesis

null hypothesis
The researchers decision about the

True False

True Correct Incorrect

False Incorrect Correct

 Decisions concerning rejecting the null
hypothesis

The true status of the null hypothesis

null hypothesis
The researchers decision about the

True False

Type II Error
True Correct

Type I Error
False Correct
 researchers use a test of significance to
determine whether to reject or fail to reject
the null hypothesis
involves pre-selecting a level of
probability, (e.g., = .05) that
serves as the criterion to determine
whether to reject or fail to reject the
null hypothesis
 Steps in using inferential statistics

1. select the test of significance

2. determine whether significance test
will be two-tailed or one tailed
3. select (alpha), the probability level
4. compute the test of significance
5. consult table to determine the
significance of the results
 Tests of significance...

statistical formulas that enable the

researcher to determine if there was a real
difference between the sample means
different tests of significance account for
different factors including: the scale of
measurement represented by the data; method
of participant selection, number of groups
being compared, and, the number of
independent variables
the researcher must first decide whether a
parametric or nonparametric test must be
selected
 parametric test...
assumes that the variable measured
is normally distributed in the
population
the data must represent an interval
or ratio scale of measurement
the selection of participants is
independent
the variances of the population
comparison groups are equal
a more powerful test in that it is
more likely to reject a null hypothesis
that is false, that is, the researcher is
less likely to commit a Type II error
used when the data represent a
interval or ratio scale
 nonparametric test...
makes no assumption about the
distribution of the variable in the
population, that is, the shape of the
distribution
used when the data represent a
nominal or ordinal scale, when a
parametric assumption has been
greatly violated, or when the nature
of the distribution is not known
a less powerful test in that it is less
likely to reject a null hypothesis at a
given level of significance
usually requires a larger sample size
to reach the same level of
significance as a parametric test
 The most common tests of significance

t-test
ANOVA
Chi Square
 t-test...
used to determine whether two
means are significantly different at
a selected probability level
adjusts for the fact that the
distribution of scores for small
samples becomes increasingly
different from the normal distribution
as sample sizes become increasingly
smaller
the strategy of the t-test is to
compare the actual mean
difference observed to the
difference expected by chance
forms a ratio where the numerator is
the difference between the sample
means and the denominator is the
chance difference that would be
expected if the null hypothesis were
true
after the numerator is divided by the
denominator, the resulting t value is
compared to the appropriate t table
value, depending on the probability
level and the degrees of freedom
if the t value is equal to or greater
than the table value, then the null
hypothesis is rejected because the
difference is greater than would be
expected due to chance
there are two types of t-tests: the
t-test for independent samples
(randomly formed) and the t-test for
nonindependent samples
(nonrandomly formed, e.g.,
matching, performance on a
pre-/post- test, different treatments)
 ANOVA...
used to determine whether two or
more means are significantly
different at a selected probability
level
avoids the need to compute duplicate
t-tests to compare groups
the strategy of ANOVA is that total
variation, or variance, can be
divided into two sources: a)
treatment variance (between
groups, variance caused by the
treatment groups) and error
variance (within groups
variance)
forms a ratio, the F ratio, with the
treatment variance as the
numerator (between group
variance) and error variance as the
denominator (within group
variance)
the assumption is that randomly
formed groups of participants are
chosen and are essentially the
same at the beginning of a study
on a measure of the dependent
variable
at the studys end, the question is
whether the variance between the
groups differs from the error
variance by more than what would
be expected by chance
if the treatment variance is
sufficiently larger than the error
variance, a significant F ratio
results, that is, the null hypothesis
is rejected and it is concluded that
the treatment had a significant
effect on the dependent variable
if the treatment variance is not
sufficiently larger than the error
variance, an insignificant F ratio
results, that is, the null hypothesis
is accepted and it is concluded
that the treatment had no
significant effect on the dependent
variable
when the F ratio is significant and
more than two means are involved,
researchers use multiple
comparison procedures (e.g.,
Scheff test, Tukeys HSD test,
Duncans multiple range test)
 FANOVA...
used when a research study uses
a factorial design to investigate
two or more independent variables
and the interactions between them
provides a separate F ratio for each
independent variable and each
interaction
 Multiple Regression...
a prediction equation that includes
more than one predictor
predictors are variables known to
individually predict (correlate with) the
criterion to make a more accurate
prediction
 Chi Square (2)...
a nonparametric test of significance
appropriate for nominal or ordinal
data that can be converted to
frequencies
compares the proportions actually
observed (O) to the proportions
expected (E) to see if they are
significantly different
the chi square value increases as
the difference between observed
and expected frequencies
increases
ANCOVA can also be used to
increase the power of a statistical
test by reducing within-group
(error) variance, that is, to make a
correct decision to reject the null
hypothesis
One- and two- tailed tests of significance...

tests of significance that indicate the

direction in which a difference may occur

the word tail indicates the area of

rejection beneath the normal curve
 A = B
no difference between means; the
direction can be positive or negative
direction can be in either tail of the
normal curve
called a two-tailed test
divides the level between the two
tails of the normal curve
 A > B or A < B
there is a difference between means;
the direction is either positive or
negative
called a one-tailed test
the level is found in one tail of the
normal curve
 Degrees of freedom (df)...

a statistical concept indicating that one

degree of freedom is lost each time a
population parameter is estimated on the
basis of a sample of data from the population

indicates that there is no true

difference or relationship between
parameters in the populations
 the ability for the sample means to vary
which is dependent upon the number of
participants and the number of groups
 for example: as the number of participants
increases (df) the value needed to reject
the null hypothesis becomes smaller
Identifying the Appropriate Statistical Test of
Difference

One-way chi-square
One variable

Two variables
(1 IV with 2 levels; 1 DV) t-test

Two variables
(1 IV with 2+ levels; 1 DV) ANOVA

Three or more variables ANOVA