Correlation and Hypothesis Testing

Correlation
Edpsy 580
Carolyn J. Anderson
Department of Educational Psychology
I L L I N O I S
UNIVERSITY OF ILLINOIS AT URBRANA - CHAMPAIGN
Correlation
Slide 1 of 38
Overview: Correlation & Regression
Outline: Pearson Correlation

Coefficient
Pearson correlation coefficient
Simple Linear Regression.
Definition & Properties

Inference & the Correlation
What and why?
How (interpretation, estimation & diagnostics).
Statistical Inference.
Comments regarding interpretation.
Coefficient
Fishers Z -Transformation
Correlation
Bi-variate regression
Multiple regression
General Linear Model
Slide 2 of 38
Outline: Pearson Correlation Coefficient

Coefficient
Definition & Properties.
Statistical Inference

Coefficient
Correlation
t-test that correlation equals 0.
Fishers Z-Transformation.
Confidence intervals for .
Test of Ho : = K.
Test of Ho : 1 = 2 (2 independent populations).
Slide 3 of 38
Correlation: Definition & Properties

Coefficient
Correlation: Definition &
Properties
Scatter Diagram & Summary
Statistics
Definition: Correlation
Pearson Product Moment Correlation
Two numerical variables measured on same individual,

(Xi , Yi ) for i = 1, . . . , n. e.g.,
coefficient
How r Works
Examples of Different r s
Height and weight.
Math and science scores.
Salary and merit.
High school GPA and college GPA.
Cost of wine and annual rainfall.
Conservative Party donors and people who buy garden

bulbs by mail.
Non-Linear Relationships
Properties: Correlation
Coefficient
Coefficient
Coefficient
Correlation
Slide 4 of 38
Scatter Diagram & Summary Statistics

Coefficient
Properties
Statistics
coefficient
How r Works
Coefficient
Coefficient
Coefficient
Correlation
Slide 5 of 38
Definition: Correlation coefficient
(Greek rho) = population correlation.

Coefficient
r = sample correlation.

Properties
Formal definition
Statistics
coefficient
How r Works
Coefficient
Coefficient

Coefficient
Correlation
sxy
cov(X, Y )
=
sx sy
sx sy
Pn
1
i Y )
X)(Y
q
q
Pn
Pn
1
1
2
2
i=1 (Xi X)
i=1 (Yi Y )
n1
n1
Pn
i=1 (Xi X)(Yi Y )

qP
qP
n
n
2
2
(X
X)
i
i=1
i=1 (Yi Y )
n1
i=1 (Xi
It measures the extent to which two random variables are

linearly related.
Slide 6 of 38
How r Works
r=
Coefficient
n
i=1 (Xi
n
i=1 (Xi
X)(Y
i Y)
2
X)
n
i=1 (Yi
Y )2
zxi zyi
i=1

Properties
Statistics
coefficient
How r Works
Coefficient
Coefficient
Coefficient
Correlation
Slide 7 of 38
Examples of Different rs

Coefficient
Properties
Statistics
coefficient
How r Works
Coefficient
Coefficient
Coefficient
Correlation
Slide 8 of 38
Examples of Different rs

Coefficient
Properties
Statistics
coefficient
How r Works
Coefficient
Coefficient
Coefficient
Correlation
Slide 9 of 38

Coefficient
Properties
Statistics
coefficient
How r Works
Coefficient
Coefficient
r=0

Coefficient
Correlation
Slide 10 of 38
Properties: Correlation Coefficient
1 r +1

Coefficient
Properties
Statistics
coefficient
How r Works
Coefficient
Coefficient
1 r < 0 small values of X go with large values of Y and

large values of X go with small values of Y .
0 < r +1 large values of X go with large values of Y and
small values of X go with small values of Y .
r = 0 No linear relationship.
r measures the strength of the relationship (magnitude)

between two variables and the direction of the relationship
(sign).

Coefficient
Correlation
Slide 11 of 38
Properties: Correlation Coefficient

Coefficient

Properties
r measures linear relationship.

Linear transformations of X and/or Y do not change the size
(magnitude) of r. Linear transformations do not change the
direction (sign) as long as
X = aX + b
Statistics
coefficient
How r Works
Coefficient
Coefficient
where a > 0 (e.g., z scores).

In a scatter plot, a linear transformation(s) (where a > 0)
simply corresponds to relabelling axis (axes).

Coefficient
Correlation
Slide 12 of 38
Inference & the Correlation Coefficient

Coefficient

Coefficient
Coefficient
The Bivariate Normal
Distribution
Example:
dependent
Preliminaries: bivariate normal distribution.

This is a generalization of the normal distribution for two
random variables
(say X and Y ).
The parameters of the bivariate normal distribution are:
x ,
= 0&
x2 ,
y ,
y2 ,
and xy
Hypothesis Testing
Hypothesis Testing
Example Hypothesis Testing
for
Alternative Method
It looks like a bell or a little hill.

MatLab program.
Alternative Method
Alternative Method
Alternative Method
Computing Correlations: SAS
Correlation
Slide 13 of 38
The Bivariate Normal Distribution
If X and Y have a bivariate normal distribution, then

2
X N (x , x
)
Y N (y , y2 )
xy measures how related X and Y are.
If X and Y are bivariate normal and xy = 0,

then X and Y are statistically independent.
If X and Y are statistically independent,

then xy = 0.
The case where xy = 0 and the (joint) distribution of X and Y is

not bivariate normal does not imply that X and Y are statistically
independent.

Coefficient
Coefficient
Coefficient
Distribution
Example:
dependent
= 0&
Hypothesis Testing
Hypothesis Testing
for
Alternative Method
Alternative Method
Alternative Method
Alternative Method
Correlation
Slide 14 of 38
Example: = 0 & dependent

Coefficient
Coefficient
Coefficient
Distribution
Example:
dependent
= 0&
Hypothesis Testing
Hypothesis Testing
r=0
Marginal distributions of X and Y are not normal:
for
Alternative Method
Alternative Method
Alternative Method
Alternative Method
Correlation
Slide 15 of 38
Hypothesis Testing

Coefficient
Statistical Hypotheses: The most common case,

Ho : = 0 versus
Ha : 6= 0
Assumptions:
X and Y are random variables whose joint distribution is
bivariate normal.*** qualification.
Observations are independent.

Coefficient
Coefficient
Distribution
Example:
dependent
= 0&
Hypothesis Testing
Hypothesis Testing
for
Alternative Method
Alternative Method
Alternative Method
Alternative Method
Correlation
Slide 16 of 38
Hypothesis Testing

Coefficient
(1r 2 )
n2

Coefficient
Coefficient
Distribution
Example:
dependent
= 0&
Hypothesis Testing
Hypothesis Testing
Test Statistic: Given the assumptions

above and Ho : = 0,
r
t= q
Sampling Distribution of the test statistic is Students t with

= n 2.
Note: the test statistic depends on both r and the sample
size n. So for a given -level, you do not have to compute
the test statistic. . . just find the critical value for r.
for
Alternative Method
Alternative Method
Alternative Method
Alternative Method
Correlation
Slide 17 of 38
Example Hypothesis Testing for

Coefficient

Coefficient
Coefficient
Distribution
Example:
dependent
= 0&
6002
Hypothesis Testing
Hypothesis Testing
for
Alternative Method
Alternative Method
High School & Beyond: Reading scores and Motivation

Ho : read,mot = 0 vs Ha : read,mot 6= 0.
Test statistic
.21061
.21061
=p
t= q
= 5.269
(1.210612 )
.9556/598
For = 600 = 2 = 598, p value = P (|t| 5.269) < .001;

therefore, Reject Ho .
Conclusion: The data provide evidence that there is a linear
relationship between reading and motivation.
Alternative Method
Alternative Method
Correlation
Slide 18 of 38
Alternative Method

Coefficient
Find the critical r and compare to the observed r.

Will reject Ho : = 0 vs Ha : 6= 0 whenever
observed
tn2 .025 tn2
or
observed
tn2 .975 tn2

Coefficient
Coefficient
Distribution
Example:
dependent
= 0&
Take
t=
Hypothesis Testing
Hypothesis Testing
for
Alternative Method
r
(1r2 )
n2
=r
(n 2)
(1 r2 )
and r as a function of t.
Alternative Method
Alternative Method
Alternative Method
Correlation
Slide 19 of 38
Alternative Method

Coefficient
Coefficient
Coefficient
Distribution
Example:
dependent
= 0&
Hypothesis Testing
Hypothesis Testing
for
Alternative Method
t= q
r
(1r 2 )
(n2)
Alternative Method
(n 2)
= rp
(1 r)2
Square both sides and solve for r:
(n 2)
1 r2
t2
r2
r2 )
t
(n 2)
r2
t2
(n 2)
r2 (1 + t2 /(n 2))
t2
(n 2)(1 + t2 /(n 2))
2 (1
Alternative Method
Alternative Method
Correlation
Slide 20 of 38
Alternative Method
So
For our HSB example:

1.9639
1.9639
=
= .08
rcrit = r

2
601.85
598 1 + (1.9639)
598

Coefficient
Coefficient
Coefficient
Distribution
Example:
dependent
= 0&
Hypothesis Testing
Hypothesis Testing
for
Alternative Method
tcrit
rcrit = p
(n 2)(1 + t2crit /(n 2))
Any correlation > .08 (or < .08) would be significant for
n = 600.
Note: Statistical significance does not imply importance.
Alternative Method
Alternative Method
Alternative Method
Correlation
Slide 21 of 38
Alternative Method

Coefficient
Coefficient
Coefficient
Distribution
Example:
dependent
More correlations from the HSB data where N = 600,

= .05, and rcrit = .0877:
Locus of
Self
control
concept Motivation
Reading
.38
.06
.21
(p-value)
(< .01)
(.14)
(< .01)
= 0&
Hypothesis Testing
Science
(p-value)
Hypothesis Testing
for
Alternative Method
Alternative Method
Alternative Method
Alternative Method
.32
(< .01)
.07
(.09)
.12
(< .01)
Note p-value= Prob(|r| r given = 0), i.e., Ho : = 0.

Correlation
Slide 22 of 38
SAS program command:

PROC CORR;
VAR rdg sci;
With locus concpt mot;
or
PROC CORR;
VAR rdg sci locus concpt mot;
ASSIST
ANALYST
Interactive data analysis

Coefficient
Coefficient
Coefficient
Distribution
Example:
dependent
= 0&
Hypothesis Testing
Hypothesis Testing
for
Alternative Method
Alternative Method
Alternative Method
Alternative Method
Correlation
Slide 23 of 38
Fishers Z-Transformation

Coefficient
Why? When 6= 0, the sampling distribution for r is skewed.
Fishers Z-Transformation is a function of r whose sampling

distribution of the transformed value is close to normal.
Can compute confidence intervals and a variety of tests

using Fishers Z.
Requirement: For the distribution of Z to be approximately

normal,

Coefficient
How Fishers
Z -Transformation Works
. . . and for Smaller
Samples. . .
Sampling Distribution of
Fishers Z
Using Fishers Z
Example Using Fishers Z
Confidence Interval for
Fishers Z in SAS
Variables from a bivariate normal distribution.
Sample size should be n 10 (and larger if question the

bivariate normal assumption).
Two Independent Group Test

Example Two Independent
Group Test
Group Test
Correlation
Slide 24 of 38
Fishers Z-Transformation:
1
Z = ln
2

Coefficient
Coefficient
1+r
1r
where
r is the sample correlation
How Fishers
Z is the transformed value of r
Samples. . .
Fishers Z
ln is the natural logarithm.
Note: the natural logarithm has base equal to

exp = e = 2.718281828; that is,
Using Fishers Z
Fishers Z in SAS
Group Test
if
expa = x then
ln(x) = a
Group Test
Correlation
Slide 25 of 38

Coefficient
Taking the logarithm of numbers has the effect of

compressing the differences or space between the larger
values and stretching the space between smaller values.
If a distribution is positively skewed, the taking the logarithm

has the effect of making the distribution more symmetric.
How Fishers Z-Transformation works. . .

Coefficient
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Fishers Z in SAS
Group Test
Group Test
Correlation
Slide 26 of 38
How Fishers Z-Transformation Works

Coefficient
Coefficient
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Fishers Z in SAS
Group Test
Group Test
Correlation
Slide 27 of 38
. . . and for Smaller Samples. . .

Coefficient
Coefficient
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Fishers Z in SAS
Group Test
Group Test
Correlation
Slide 28 of 38
Sampling Distribution of Fishers Z

Coefficient

Coefficient
IF Observations
Are from a bivariate normal distribution.
Are independent across individuals.
n 10
2
THEN the sampling distribution of Z is N (Z , Z
) where
E(Z) = Z = Z
How Fishers
2
Z
Samples. . .
Fishers Z
Using Fishers Z

Fishers Z in SAS
Group Test
Group Test
Correlation
1
ln
2
1
n3
1+
1
+
2(n 1)
is the bias factor, which in SAS you can

The value 2(n1)
request that a bias adjustment be used (in confidence
intervals).
2
Z and Z
are independent of each other.
The transformation of r is known as the inverse of the
hyperbolic tangent of r.
Slide 29 of 38
Using Fishers Z
HSB data: Are there relationships between psychological

variables and achievement: motivation and reading?
Observed correlation, r = .21061.
If the true population correlation coefficient is > 0, then the

sampling distribution of r will be skewed.
Use Fishers Z transformation,

1 + .21061
1
1
1
= ln(1.53360) = (.4276) = .2138
Z = ln
2
1 .21061
2
2
The standard deviation,

1
= .04093
Z =
600 3

Coefficient
Coefficient
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Fishers Z in SAS
Group Test
Group Test
Correlation
Slide 30 of 38

Coefficient
Suppose want to test Ho : = .25 vs Ha : 6= .25.
Need the value of Z for = .25,

1 + .25
1
= .2554
Z.25 = ln
2
1 .25
Test statistic is

Coefficient
How Fishers
Zobs Znull
.0416
.2138 .2554
z=
=
= 1.016
=
Z
.04093
.04093
Samples. . .
Fishers Z
Using Fishers Z
Retain Ho . . . .
Note: a lower case z is used for the test statistic and upper
case Z is denotes Fishers Z-transformed value of r.
Fishers Z in SAS
Group Test
Group Test
Correlation
Slide 31 of 38

Coefficient
Another use for Fishers Z-transformation.
Suppose we want a 95% CI for correlation between

motivation and reading scores.
Steps:

Coefficient
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Fishers Z in SAS
Group Test
1. Transform the sample correlation: Zobs = .2138.

2. Compute the (1 )% CI for Z
Zobs
z/2 Z
.2138 1.96(.04093) = (.13, .29)

3. Un-transform the end points of the CI above.
Group Test
Correlation
Slide 32 of 38
Reversing the Fisher Z transformation. . . a little algebra gives

Coefficient
e2Z 1
r = 2Z
e +1

Coefficient
Our example
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Fishers Z in SAS
Group Test
rlower
rupper
e2(.1336) 1
.3063
2.71828.2672 1
=
= .1328
=
.2672
2(.1336)
2.71828
+1
2.3063
e
+1
.8004
e2(.2940) 1
2.71828.5880 1
=
= .2858
=
2.71828.5880 + 1
2.8004
e2(.2940) + 1
The 95% confidence interval for between motivation and

reading scores is (.13, .29).
Group Test
Correlation
Slide 33 of 38
Fishers Z in SAS

Coefficient
TITLE Testing Ho: rho=0 using Fisher-Z transformation;

proc corr data=hsb fisher;
var mot rdg;
RUN;

Coefficient
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Fishers Z in SAS
Group Test
TITLE Ho : rho= .25 , No bias adjustment;

proc corr data=hsb fisher(rho0=.25 biasadj=no alpha=.05);
var mot rdg;
RUN;
TITLE Ho : rho= .25 , With bias adjustment;
proc corr data=hsb fisher(rho0=.25 biasadj=yes alpha=.05);
var mot rdg;
RUN;
Group Test
Correlation
Slide 34 of 38
- slide -
34-1
Test whether the correlation from 2 independent groups are

the same or different.
The same procedure that we used for testing difference

between mean for large samples.
Statistical hypotheses:

Coefficient
Coefficient
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Ho : 1 = 2
vs
Ha : 1 6= 2
Assumptions:
Observations are independent within and between

populations
The joint distribution of the two variables in each

population is bivariate normal.

Fishers Z in SAS
Group Test
Group Test
Correlation
Slide 35 of 38
Test Statistic:

Coefficient
z=

Coefficient
How Fishers
Z1 Z2
Z1 Z2
where
Z1 and Z2 are Fisher Z-transformations of the sample
correlations, r1 and r2 , from the two groups.
Samples. . .
Fishers Z
Using Fishers Z
Standard deviation,
r
q
2 + 2 =
Z1 Z2 = Z
Z2
1
1
1
+
n1 3 n2 3
Why?
Fishers Z in SAS
Group Test
Sampling distribution of the test statistic is N (0, 1).
Group Test
Correlation
Slide 36 of 38
Example Two Independent Group Test
Is the relationship between writing scores and locus of

control the same or different for male and female high school
students?
The data: nmale = 327 and rmale = .40196

nf emale = 273 and rf emale = .28250
Statistical hypotheses:
Ho : male = f emale vs Ha : male 6= f emale
Assumptions:

Coefficient
Coefficient
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Scores come from bivariate normal populations.
Independence within and between groups.

Fishers Z in SAS
Group Test
. . . so whats dependent?
Group Test
Correlation
Slide 37 of 38
Example Two Independent Group Test

Test Statistic:
Coefficient
Zf emale
Coefficient
Zmale
How Fishers
Samples. . .
Fishers Z
Using Fishers Z
Fishers Z in SAS
Group Test
(Zm Zf )
z
1 + .28250
1
1
ln
= (.5808) = .29040
2
1 .28250
2

1 + .40196
1
1
ln
= (.85197) = .425980
=
2
1 .40196
2
s
r
1
1
1
1
+
=
+
= .08240
=
nmale 3 nf emale 3
324 270
.42598 .29040
= 1.645
.08240
Conclusion: Retain Ho for = .05. The difference between the

correlations more likely to be due to chance than reflect real a
difference.
Group Test
Correlation
Slide 38 of 38

Correlation and Hypothesis Testing

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Correlation and Hypothesis Testing

Uploaded by

Copyright:

Available Formats

Correlation

Overview: Correlation & Regression

Outline: Pearson Correlation

Pearson correlation coefficient

Simple Linear Regression.

Definition & Properties

What and why?

How (interpretation, estimation & diagnostics).

Comments regarding interpretation.

General Linear Model

Outline: Pearson Correlation Coefficient

Outline: Pearson Correlation

Definition & Properties.

Definition & Properties

t-test that correlation equals 0.

Confidence intervals for .

Test of Ho : 1 = 2 (2 independent populations).

Correlation: Definition & Properties

Outline: Pearson Correlation

Pearson Product Moment Correlation

Two numerical variables measured on same individual,

Height and weight.

Math and science scores.

Salary and merit.

High school GPA and college GPA.

Cost of wine and annual rainfall.

Conservative Party donors and people who buy garden

Scatter Diagram & Summary Statistics

Outline: Pearson Correlation

Definition: Correlation coefficient

(Greek rho) = population correlation.

Outline: Pearson Correlation

Definition & Properties

Inference & the Correlation

i=1 (Xi X)(Yi Y )

It measures the extent to which two random variables are

Definition & Properties

Outline: Pearson Correlation

Outline: Pearson Correlation

Outline: Pearson Correlation

Inference & the Correlation

Properties: Correlation Coefficient

Outline: Pearson Correlation

1 r < 0 small values of X go with large values of Y and

r measures the strength of the relationship (magnitude)

Inference & the Correlation

Properties: Correlation Coefficient

Outline: Pearson Correlation

Definition & Properties

r measures linear relationship.

where a > 0 (e.g., z scores).

Inference & the Correlation

Inference & the Correlation Coefficient

Outline: Pearson Correlation

Definition & Properties

Preliminaries: bivariate normal distribution.

It looks like a bell or a little hill.

The Bivariate Normal Distribution

If X and Y have a bivariate normal distribution, then

If X and Y are bivariate normal and xy = 0,

If X and Y are statistically independent,

The case where xy = 0 and the (joint) distribution of X and Y is

Outline: Pearson Correlation

Example: = 0 & dependent