Use of Statistical Software for Statistical Analyses

Dr. Adamu Onu Consultant Family Medicine & Biostatistician

Outline
Use of computers in research

Hypothesis testing
Type I error Type II error

Which statistical test?

Statistical software packages

Use of Computers in Research

The use of computer software for statistical analyses presupposes the use of computers.

What is a computer?
A computer is a programmable machine designed

to automatically carry out a sequence of arithmetic or logical operations. The particular sequence of operations can be changed readily, allowing the computer to solve more than one kind of problem.

Characteristics of the computer

SPEED; a lot can be accomplished in shorter time

when compared with handwriting, organization, editing and corrections of documents. STORAGE ability to store large amount of documents with ease of retrieval. ACCURACY consistent nearly error free performance. AUTOMATION- perform tasks with keyboard flicks. DILIGENCE carry out tasks perfectly as instructed. Obey commands as programmed but remember GIGO

What is research?
Systematic collection, analysis and interpretation

of data to answer a question or solve a problem.

Types of research

Basic Research
Generates new knowledge and technologies to deal with major unresolved problems.

Applied Research
Identifies priority problems, designs and evaluates policies and programs that will deliver the greatest benefit using optimally the available resources

Determining risk factor disease relationship and comparing treatments.

7

e.g. operations research and health systems research

Randomized Controlled Trial Qualitative Research Methods Quantitative Descriptive Observational Analytical Experimental Nonrandomized Control Trial

Conceptual

Research problem & hypothesis Literature review

Design & Planning

Empirical Analytical Dissemination
9

Research design; Sample size Questionnaire and pilot study/pretesting

Collecting & preparing data for analysis Data storage data entry, editing, data management.

Hundreds of hours can be saved by using the appropriate analysis software

Preparing Tables and Visual Data Displays, Publishing, Presentations

Uses of computer software in all phases of research

Software plays an integral role
Throughout the process of research. Ability to handle large samples. For storage of research information in computer or

other auxiliary memories like CD, DVD, external hard drives, memory card and flash drive. Dissemination of research findings.

10

Caveats
Computers makes research work faster, easier,

11

more accurate and more reliable while minimizing errors. However remember garbage in and garbage out. Computers only carry out instructions and cannot think. We should be aware of the abilities and limitations of the computer software for optimum results. Plan well.
25/06/2012

Hypothesis testing

12

4 steps of hypothesis testing

Decision (optional)

Null & Alternative Hypothesis

Test Statistic

p Value & Conclusion

13

Step A: Null & alternative hypotheses

The first step of hypothesis testing is to convert

the research question into null and alterative hypotheses. The null hypothesis (H0)is a claim of no difference. The opposing hypothesis is the alternative hypothesis (H1). The alternative hypothesis is a claim of a difference in the population
This is the hypothesis the researcher often hopes to

bolster.
It is important to keep in mind that the null and
14

alternative hypotheses reference population values, and not observed statistics.

Step B: Test statistic

We calculate a test statistic from the data.

There are different types of test statistics

depending on:
the nature of the data, and the null and alternative hypotheses.

The test statistic will compare the observed

15

sample statistic to an expected population parameter. Large test statistics indicate data are far from expected, providing evidence against the null hypothesis and in favor of the alternative hypothesis.

Step C: p Value & conclusion

The test statistic is converted to a conditional

probability called a P-value. The P- value answers the question If the null hypothesis were true, what is the probability of observing the current data or data that is more extreme? Small p values provide evidence against the null hypothesis because they say the observed data are unlikely when the null hypothesis is true.

16

Step C: p Value & conclusion

We apply the following conventions:
When p value > .10 the observed difference is

not significant When p value .10 the observed difference is marginally significant When p value .05 the observed difference is significant When p value .01 the observed difference is highly significant
Use of significant in this context means the

observed difference is not likely due to chance. It does not mean of important or meaningful.
17

Step D: Decision
Alpha () is a probability threshold for a decision.

If P , we will reject the null hypothesis. Otherwise it will be retained for want of evidence.

18

Fallacies of statistical hypothesis testing

Failure to reject the null hypothesis leads to its

acceptance.
(WRONG! Failure to reject the null hypothesis

implies insufficient evidence for its rejection.)

The p value is the probability that the null

hypothesis is incorrect.
(WRONG! The p value is the probability of the

current data or data that is more extreme assuming H0 is true.)

= .05 is a standard with an objective basis.
(WRONG! = .05 is merely a convention that has
19

taken on unwise mechanical use. There is no sharp distinction between significant and insignificant

Fallacies of statistical hypothesis testing

Small p values indicate large effects.
(WRONG! p values tell you next to nothing about

the size of an effect.)

Data show a theory to be true or false.
(WRONG! Data can at best serve to bolster or

refute a theory or claim.)

Statistical significance implies importance.
(WRONG! WRONG! WRONG! Statistical

significance says very little about the importance of a relation.)

20

Vocabulary
Null hypothesis (H0)
A statement that declares the observed difference is

due to chance. It is the hypothesis the researcher hopes to reject.

Alternative hypothesis (H1)
The opposite of the null hypothesis. The hypothesis

the researcher hopes to bolster.

Alpha ()
The probability the researcher is willing to take in

falsely rejecting a true null hypothesis.

Test statistic
21

A statistic used to test the null hypothesis.

Vocabulary
P-value
A probability statement that answers the question If

the null hypothesis were true, what is the probability of observing the current data or data that is more extreme than the current data?. It is the probability of the data conditional on the truth of H0. It is NOT the probability that the null hypothesis is true.
Type I error
A rejection of a true null hypothesis; a false alarm,

i.e. a false positive

Type II error
22

A retention of an incorrect null hypothesis; failure to

Vocabulary
Confidence (1 - )
The complement of alpha.

Beta ()
The probability of a type II error; probability of a

retaining a false null hypothesis.

Power (1 - )
The complement of ; the probability of avoiding a

type II error; the probability of rejecting a false null hypothesis.

23

Defendant innocent Guilty verdict Not guilty verdict

Defendant guilty

Type I Error Correct

Correct Type II Error

24

Null Hypothesis True Reject Null Hypothesis Fail to reject Null Hypothesis

Null Hypothesis False

Type I Error Correct

Correct Type II Error

25

Which statistical test?

An approximate answer to the right question is worth a great deal more than a precise answer to the wrong question. -John Tukey

26

 Major objective for doing research is to draw

inferences The choice of an appropriate statistical test is based on a sound research proposal Test statistics are fundamental to statistical inferences Good answers come from good questions not from esoteric analysis Schoolman et al, (1968)

27

4 steps of hypothesis testing

Decision (optional)

Null & Alternative Hypothesis

Test Statistic

p Value & Conclusion

28

Step B: Test statistic

We calculate a test statistic from the data.

There are different types of test statistics

depending on:
the nature of the data, and the null and alternative hypotheses.

The test statistic will compare the observed

29

sample statistic to an expected population parameter. Large test statistics indicate data are far from expected, providing evidence against the null hypothesis and in favor of the alternative hypothesis.

Choosing the right statistic test

Goals & Objectives of the Research

Research Design
Quality of Data Number of groups of observations Distribution of Data

30

Classification of test statistic based on type of data

Continuous data

Ordinal Data
Dichotomous & Nominal Data Combination of variables

31

Tests for continuous data

32

Tests for ordinal data

33

Test statistics for nominal data

34

Test statistics for combinations of data

Continuous/Dichotomous
Unpaired Student t test Paired Paired t test

Continuous/Nominal
One-way analysis of variance

Continuous/Continuous
Pearson correlation coefficient Linear regression

Ordinal/Dichotomous
Unpaired Mann-Whitney U test; Chi-square test
35

for linear trend Paired Wilcoxon test

Test statistics for combinations of data

Ordinal/Nominal
Kruskall-Wallis test

Ordinal/Ordinal
Spearman Correlation Coefficient (rho) Kendall Correlation Coefficient (tau)

Ordinal/Continuous
Spearman Correlation Coefficient (rho) Kendall Correlation Coefficient (tau)

Chi-square test

36

Test statistics for combinations of data

Dichotomous/Dichotomous
Unpaired Chi-square test; Fisher exact test Paired McNemar Chi-square test

Dichotomous/Nominal
Chi-square test

Nominal/Nominal
Chi-square test

37

Test statistics for combinations of data dependent & independent

Continuous/All are categorical
ANOVA

Continuous/Some categorical, some continuous

ANCOVA

Continuous/All continuous
Multiple linear regression

Dichotomous/ All categorical

Logistic regression

Nominal/ All categorical

Log-linear

Nominal/ All categorical

Discriminate Function Analysis
38

Time-to-event data
Kaplan Meier survival curve

Log-rank test, Mantel-Haenszel test

Cox proportional hazards regression

39

Non-parametric tests
Many statistical tests are based upon the

assumption that the data are sampled from a normal distribution. These tests are referred to as parametric tests. Commonly used parametric tests are
Mean, SD Student t test, unpaired t test, paired t test ANOVA, Pearson correlation, Linear regression

Non-parametric tests include

Median, IQR Wilcoxon test, Mann-Whitney test
40

Kruskall-Wallis, Spearman correlation, Non-

Parametric vs. non-parametric tests - does it matter?

Large sample.
What happens when you use a parametric test with

data from a non-Gaussian population? The central limit theorem ensures that parametric tests work well with large samples even if the population is non-Gaussian. Unless the population distribution is really weird, you are probably safe choosing a parametric test when there are at least two dozen data points in each group.

41

Parametric vs. non-parametric tests - does it matter?

Large sample.
What happens when you use a nonparametric test

with data from a Gaussian population? Nonparametric tests work well with large samples from Gaussian populations. The P values tend to be a bit too large, but the discrepancy is small. In other words, nonparametric tests are only slightly less powerful than parametric tests with large samples.

42

Parametric vs. non-parametric tests - does it matter?

Small samples
What happens when you use a parametric test with

data from non-Gaussian populations? You can't rely on the central limit theorem, so the P value may be inaccurate.
Small samples.
When you use a nonparametric test with data from

a Gaussian population, the P values tend to be too high. The nonparametric tests lack statistical power with small samples.
43

One- or two-sided p value?

You must choose whether you wish to calculate a

44

one- or two-sided P value. The P value is calculated for the null hypothesis that the two population parameters are equal, and any discrepancy between the two sample statistics is due to chance. If this null hypothesis is true, the one-sided P value is the probability that the two sample statistics would differ as much as was observed (or further) in the direction specified by the hypothesis just by chance, even though the means of the overall populations are actually equal.

One- or two-sided p value?

The two-sided P value also includes the

probability that the sample statistics would differ that much in the opposite direction. The two-sided P value is twice the one-sided P value. A one-sided P value is appropriate when you can state with certainty (and before collecting any data) that
there either will be no difference between the

means or that the difference will go in a direction you can specify in advance (i.e., you have specified which group will have the larger mean).
45

One- or two-sided p value?

If you cannot specify the direction of any

46

difference before collecting data, then a two-sided P value is more appropriate. If in doubt, select a two-sided P value. If you select a one-sided test, you should do so before collecting any data and you need to state the direction of your experimental hypothesis. If the data go the other way, you must be willing to attribute that difference (or association or correlation) to chance, no matter how striking the data. If you would be intrigued, even a little, by data

 The formulation of a problem is often more

essential than its solution which may be merely a matter of mathematical or experimental skill. - Albert Einstein

47

Statistics Software Packages

48

Free Open Source or Public Domain

R
A free implementation of the S language.

Epi Info 3.5.3 Epi Info 7 OpenEpi

A web-based, open source, operating-independent

series of programs for use in epidemiology and statistics based on JavaScript and HTML
PSPP
A free software replacement for SPSS
49

Proprietary
SAS

SPSS
Stata

50

Workflow

Raw Data

Statistical methods

Paper, Slides, Posters

51

Organize raw data code, rename & label variables eliminate outliers

Descriptive analysis figures descriptive tables

Develop methods for analysis choose tests write program code

Get results run program code

Write paper mix results with text & explanations

52

Aims of statistical data analysis

Limit work and time

Get interesting results

Replicability
for us, to understand our data and our methods

after we get back to work after a short break for our friends (coauthors), so that they can understand what we are doing for our enemies we should always (even years after) be able to prove our results exactly

53

Practical Demos

54