Statistical tests

1
 Comparing means (parametric)
• Means (just comparing means manually, options are there to go for
ANOVA)

• One sample T test (comparing a sample mean against a figure available

from some (secondary) sources (t = [(x – μ)/(s/√n)], df = n-1)

• Independent samples T test (this is used to compare groups of participants

that are independent of each other) (t = [(x1 – x2)/√(s12/n1 + s22/n2)], df =
n1+n2-2)

• Paired samples T test (this is used to compare groups that are related in
some way) (t = [ ΣD/√(nΣD2 – (ΣD)2)/(n-1)], df = n-1)

• ANOVA (comparing means against a particular variable/criteria)

2
Post Hoc tests with one-way ANOVA
Post hoc tests are designed for situations in which the researcher has already obtained
a significant overall F-test with a factor that consists of three or more means and
additional exploration of the differences among means is needed to provide specific
information on which means are significantly different from each other:

1. Least Significant Difference (LSD): it uses t-tests to all possible pairs of

group means to reveal which pairs are significantly making a difference
comparing to the other pairs

2. Duncan’s Multiple Range test and the Newman-Keuls test provide different
critical difference values for particular comparisons of means depending on
how adjacent the means are

3
 Continued…..
3. Student-Newman-Keuls (SNK), Ryan-Einot-Gabriel-Welsh F (REGWF), Ryan-Einot-Gabriel-
Welsh Q (REGWQ) and Duncan methods: After ordering the group means from lowest to
highest, the two most extreme means are tested for a significant difference using a critical
value. If not significant then the testing stops, unless it continues with the next most extreme
set. The process continues for all such pairs. Comparing to all these tests “LSD” is a better
test to get the significant pair-wise mean differences

4. Bonferroni and Sidak: The Bonferroni simply calculates a new pair-wise alpha to keep the
family-wise alpha value at 0.05 (or another specified value) (αB = αFWE/c), where αB is the
new alpha based on the Bonferroni test that should be used to evaluate each comparison or
significance test, αFWE is the family-wise error rate as computed in the first formula, and c is
the number of comparisons

4
 Comparing inter-dependence
(non-parametric)
Nonparametric tests are often used when certain assumptions (such as normality) about
the underlying population are questionable or not known and data are nominal or
ordinal (such as ranking) rather than direct measurements:
• Chi-square (data are categorical, proportions are given across one or more
dimensions)(χ2 = ∑ (Oij – Eij)2/Eij, df = (row – 1)*(column – 1))
Likelihood ratio chi square (G2) = 2 ∑Oij log (Oij/Eij), df = (row – 1)*(column – 1)
Linear by linear association (Mantel-Haenszel Test: χ2MH = (W−1)r2 , df = 1, W =
the grand total value, r = correlation coefficient)
• Runs test (data can be categorical or scale but randomness is tested for better
sample representation)
Note: Chi square test is used both for hypothesis testing and testing a distribution for its
goodness of fit

5
 Runs test (continued)
For a series of responses/respondent representation (gender, education etc.):

Deviation (D) = response value (X) – cut-off (M) (M = median)

If two successive signs are respectively +ve (including “0”) and –ve or vice versa then it is a run

Total run “R” = no of sign changes + 1 (approximately follows normal distributions)

Further, no of negative signs = na, if D < 0; no of positive signs = np, if D >= 0; n (sample size) = na + np

μr = (2na np / na + np) + 1

σr = √[2na np(2na np – na – np)/((na + np)2(na + np -1))]

Z = (R – μr)/ σr, if n >= 50

Else (for n < 50):

Zc = (R − μr + 0.5)/σr if R − μr ≤ 0.5

(R − μr −0.5)/σr if R − μr ≥ 0.5

0 if |R−μr| < 0.5

6
 Explanations of some major tests
• McNemar's chi-square is designed to test a dependent variable (yes or no type or binary), for same
individuals measured twice at different occasions (e.g. time), to make a pair in some way (also before and
after scenario)

• Cocharn’s Q is designed to test a dependent variable (yes or no type or binary), for same individuals
measured three or more no of times at different occasions (e.g. time), to make a pair in some way (also
before and after scenario)

• The Kolmogorov-Smirnov test (KS test in short) is used to test whether or not the sample of data is
consistent with a specified distribution function. When there are two independent samples of data, it is used
to test whether or not these two samples may reasonably be assumed to come from the same distribution

• The Wilcoxon Mann-Whitney Test is designed to test whether two populations have identical distribution
functions about the location (median) (normally distribution assumption not necessary). The Wilcoxon
Mann-Whitney Test is used in place of the independent samples t-test when the normality assumption is
questionable

7
 Continued.....
• The Wilcoxon Signed Ranks test (more powerful than Sigh test) and Sign tests are designed to test a
hypothesis about the location (median) of a population distribution. It often involves the use of matched pairs
e.g. before and after data (dependent samples), in which case it tests for a median difference of zero

• The Kruskal-Wallis test is used to compare three or more independent samples. It is used to test the null
hypothesis that all populations have identical distribution functions against the alternative hypothesis that at
least two of the samples differ only with respect to location (median)

• Friedman test and Kendall's W test: In one application the same quantitative variable is measured at two or
more different times from the same sample (or from two or more samples that have been matched on one or
more important variables). In the other application, two or more comparable quantitative variables are
measured from the same sample (usually at the same time). In both applications, Friedman's test is used to
compare the distributions of the two or more quantitative variables

8
 Summary of statistical tests
Sample characteristics

Level of 1 2 Samples K Samples ( > 2) Correlation

measurement Sample
Independent Dependent Independent Dependent

Categorical or Chi Chi square McNemar’s Chi square Cochran’s Q

Nominal square or chi square
binomial

Rank or One sample Mann Wilcoxon Kruskal -Wallis Friedman Spearman’s

Ordinal Kolmogorov Whitney U; Signed Ranks H test; correlation
-Smirnov Kolmogorov- (matched Kendall’s W ( rho)
test Smirnov two pairs ); Sign test
sample test test
Parametric z test or t test between t test within 1 way ANOVA 1 way Pearson’s
(Interval & t test groups groups between groups ANOVA correlation
Ratio) (within or (r)
repeated
measure)
Factorial (2 way) ANOVA

Source: Compiled from Plonskey (2001)

9