Mixed

Mixed and Multilevel Models
Geert Verbeke
Biostatistical Centre, K.U.Leuven
geert.verbeke@med.kuleuven.be
http://perswww.kuleuven.be/geert verbeke
Master in Statistics
Contents
I Introduction 1
1 Introductory material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 Course motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
II Linear Mixed Models 18
3 The Captopril data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4 The lizard data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
5 The paired t-test revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Introduction to Biostatistics i
6 The growth curves data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
7 The linear mixed model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
8 The rat data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
9 The BLSA prostate data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
10 The Leuven diabetes project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
11 Estimation of Random Eects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
12 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
III Generalized Linear Mixed Models 178
13 The toenail data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
14 The Leuven diabetes project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
15 The Epilepsy data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
16 The hierarchical versus marginal model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Introduction to Biostatistics ii
IV Non-linear Mixed Models 248
17 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
18 The orange trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
19 The Theophylline data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
20 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Introduction to Biostatistics iii
Part I
Introduction
Introduction to Biostatistics 1
Chapter 1
Introductory material
Related references
Course material
Software
Course evaluation
1.1 Related references
Aerts, M., Geys, H., Molenberghs, G., and Ryan, L.M. (2002). Topics in
Modelling of Clustered Data. London: Chapman & Hall.
Brown, H. and Prescott, R. (1999). Applied Mixed Models in Medicine. New
York: John Wiley & Sons.
Crowder, M.J. and Hand, D.J. (1990). Analysis of Repeated Measures. London:
Chapman & Hall.
Davidian, M. and Giltinan, D.M. (1995). Nonlinear Models For Repeated
Measurement Data. London: Chapman & Hall.
Davis, C.S. (2002). Statistical Methods for the Analysis of Repeated
Measurements. New York: Springer.
Demidenko, E. (2004). Mixed Models: Theory and Applications. New York: John
Wiley & Sons.
Diggle, P.J., Heagerty, P.J., Liang, K.Y. and Zeger, S.L. (2002). Analysis of
Longitudinal Data. (2nd edition). Oxford: Oxford University Press.
Fitzmaurice, G., Davidian, M., Verbeke, G. and Molenberghs G. (2008).
Longitudinal Data Analysis. Handbooks of Modern Statistical Methods series, #1.
London: Chapman & Hall/CRC
Fitzmaurice, G.M., Laird, N.M., and Ware, J.H. (2004). Applied Longitudinal
Analysis. New York: John Wiley & Sons.
Goldstein, H. (1979). The Design and Analysis of Longitudinal Studies. London:
Academic Press.
Goldstein, H. (1995). Multilevel Statistical Models. London: Edward Arnold.
Hand, D.J. and Crowder, M.J. (1995). Practical Longitudinal Data Analysis.
London: Chapman & Hall.
Hedeker, D. and Gibbons, R.D. (2006). Longitudinal Data Analysis. New York:
John Wiley & Sons.
Jones, B. and Kenward, M.G. (1989). Design and Analysis of Crossover Trials.
London: Chapman & Hall.
Kshirsagar, A.M. and Smith, W.B. (1995). Growth Curves. New York: Marcel
Dekker.
Leyland, A.H. and Goldstein, H. (2001) Multilevel Modelling of Health Statistics.
Chichester: John Wiley & Sons.
Lindsey, J.K. (1993). Models for Repeated Measurements. Oxford: Oxford
University Press.
Littell, R.C., Milliken, G.A., Stroup, W.W., Wolnger, R.D., and Schabenberger,
O. (2005). SAS for Mixed Models (2nd ed.). Cary: SAS Press.
Longford, N.T. (1993). Random Coecient Models. Oxford: Oxford University
Press.
McCullagh, P. and Nelder, J.A. (1989). Generalized Linear Models (second
edition). London: Chapman & Hall.
Molenberghs, G. and Verbeke, G. (2005). Models for Discrete Longitudinal Data.
New York: Springer-Verlag.
Pinheiro, J.C. and Bates D.M. (2000). Mixed eects models in S and S-Plus.
New York: Springer.
Searle, S.R., Casella, G., and McCulloch, C.E. (1992). Variance Components.
New-York: Wiley.
Senn, S.J. (1993). Cross-over Trials in Clinical Research. Chichester: Wiley.
Verbeke, G. and Molenberghs, G. (1997). Linear Mixed Models In Practice: A
SAS Oriented Approach, Lecture Notes in Statistics 126. New-York: Springer.
Verbeke, G. and Molenberghs, G. (2000). Linear Mixed Models for Longitudinal
Data. Springer Series in Statistics. New-York: Springer.
Vonesh, E.F. and Chinchilli, V.M. (1997). Linear and Non-linear Models for the
Analysis of Repeated Measurements. Basel: Marcel Dekker.
Weiss, R.E. (2005). Modeling Longitudinal Data. New York: Springer.
West, B.T., Welch, K.B., and Galecki, A.T. (2007). Linear Mixed Models: A
Practical Guide Using Statistical Software. Boca Raton: Chapman & Hall/CRC.
1.2 Course material
Copies of the course notes: Toledo
Data sets analysed in the course: Toledo
Books:
Verbeke, G. and Molenberghs, G. (2000). Linear Mixed Models for
Longitudinal Data. Springer Series in Statistics. New-York: Springer.
Molenberghs, G. and Verbeke, G. (2005). Models for Discrete Longitudinal
Data. New York: Springer-Verlag.
1.3 Software
Many software packages nowadays allow tting of mixed or multilevel models
In this course, SAS will be used:
PROC MIXED
PROC GLIMMIX
PROC NLMIXED
SAS is the most exible in terms of models that can be tted
SAS is most up to date with the statistical literature
1.4 Course evaluation
Take-home assignment
Data analysis and reporting in teams
Report submitted before nal examination
Oral defense of the report
Chapter 2
Course motivation
Hierarchical data
Correlated data
Overview of model families
2.1 Hierarchical data
Hierarchical data are obtained when the sample is
taken at multiple, hierarchically ordered, levels.
Examples:
Measurements taken on patients, at multiple visits after their treatment
Growth curves of children, animals, plants, . . .
Survey in which all members from each of a sample of families are questioned
Survey in which 10 habitants from each of a sample of cities are questioned
Exam results from students from a sample of schools
. . .
These are examples of two-level data structures, but extensions to multiple levels
are possible:
10 cities
In each: 5 schools
In each: 2 classes
In each: 5 students
Each student given the test twice
Terminologies:
Repeated measures
Longitudinal data
Multilevel data
. . .
2.2 Correlated data
2.2.1 Example: Longitudinal body weight example
Consider a body weight experiment in which body weight is measured on a daily
basis, for a sample of participants
It is natural to assume body weights from dierent subjects to be independent
from each other
Body weights measured on the same subject are expected to be correlated
Should this correlation be accounted for in analysis ?
If yes, how ?
2.2.2 Example: Comparing BMI between males and females
Suppose interest is in comparing the average BMI between males and females,
based on 100 observations from each population
Natural analysis: Two-sample, unpaired t-test
Suppose the 100 males and 100 females are married couples
The BMI of spouses is likely to be correlated
Natural analysis: Paired t-test
2.2.3 Conclusion
Hierarchical data structures often yield data which cannot be assumed independent
From a statistical perspective, the key issue in modelling hierarchical data is how
to account for the association between observations
Alternative terminology:
Repeated measures
Longitudinal data
Multilevel data
Correlated data
. . .
2.3 Overview of model families
Since hierarchical data are correlated, all traditional models in statistics need a
counterpart for correlated data
Many dierent models have been proposed in the statistical literature
We focus on mixed models which explicitly model the various levels in the data
structure
Cross-sectional data Hierarchical data
Linear regression models Linear mixed models
Generalized linear models Generalized linear mixed models
Non-linear regression models Non-linear mixed models
Part II
Linear Mixed Models
Chapter 3
The Captopril data
Example
Paired t-test
Paired versus unpaired t-test
Conclusion
3.1 Example
15 patients with hypertension
The response of interest is the supine blood pressure, before and after treatment
with CAPTOPRIL
Research question:
How does treatment aect BP ?
Dataset Captopril
Before After
Patient SBP DBP SBP DBP
1 210 130 201 125
2 169 122 165 121
3 187 124 166 121
4 160 104 157 106
5 167 112 147 101
6 176 101 145 85
7 185 121 168 98
8 206 124 180 105
9 173 115 147 103
10 146 102 136 98
11 174 98 151 90
12 201 119 168 98
13 198 106 179 110
14 148 107 129 103
15 154 100 131 82
Average (mm Hg)
Diastolic before: 112.3
Diastolic after: 103.1
Systolic before: 176.9
Systolic after: 158.0
3.2 Paired t-test
Lets focus on the analysis of the diastolic BP:
Average (mm Hg)
Diastolic before: 112.3
Diastolic after: 103.1
There is an average decrease of more than 9 mmHG
The classical analysis of paired data is based on comparisons within subjects:
i
= Y
i1
Y
i2
, i = 1, . . . , 15
A positive
i
corresponds to a decrease of the BP, while a negative
i
is
equivalent to an increase.
Testing for treatment eect is now equivalent to testing whether the average
dierence
equals zero.
Statistica output:
Hence, the average change in BP is signicantly dierent from zero (p = 0.001).
3.3 Paired versus unpaired t-test
What if the Captopril data were analysed using an unpaired t-test ?
Results from unpaired and paired t-tests, respectively:
Unpaired:
Paired:
Although both tests lead to a signicant result, there is a serious dierence in
p-values, showing that ignoring the paired nature of the data can lead to wrong
conclusions.
3.4 Conclusion
15 2 measurements = 30 1 measurement
The correlation cannot be ignored in the analyses
In the paired t-test, the correlation problem is circumvented by taking
within-subject dierences
i
= Y
i1
Y
i2
, i = 1, . . . , 15
How to extend this to:
multiple measurements per subject ?
include covariate information ?
multiple levels in the data structure ?
Chapter 4
The lizard data
Example
Two-way ANOVA
Mixed models
Fitting mixed models in SAS
The hierarchical versus marginal model
4.1 Example
Data on 102 lizards
Response of interest: Number of dorsal shells
Research question:
Is number of dorsal shells gender-related ?
Graphically:
Two-sample t-test:
Hence, the small observed dierence is not signicant (p = 0.1024).
A typical aspect of the data is that some animals have the same mother.
We have 102 lizards from 29 mothers
Mother eects might be present
Hence a comparison between male and female animals should be based on
within-mother comparisons.
Graphically:
Observations:
Much between-mother variability
Often, males (considerably) higher than females
In cases where females higher than males, small dierences
Hence the non-signicant t-test result may be due to the between-mother
variability
This is an example of clustered data: Observations are clustered within mothers
It is to be expected that measurements within mothers are more alike than
measurements from dierent mothers.
We expect correlated observations within mothers and independent observations
between mothers.
How to correct for dierences between mothers ?
4.2 Two-way ANOVA
An obvious rst choice to test for a sex eect, correcting for mother eects, is
2-way ANOVA with factors sex and mother.
The mother eect then represents the variability between mothers.
Let Y
ijk
be the kth outcome in the jth gender group for the ith mother.
Our two-way ANOVA model then equals:
Y
ijk
= +
i
+
j
+
ijk
,
Parameter interpretation:
Overall mean
Gender eect
j
Mother eect
i
The parameter of interest is
2
1
, the average dierence between males and
females
Since the model is overparameterized, restrictions are needed, e.g.,
i
=

j

j
= 0
Residual distribution:
ijk
N(0,
2
res
)
In order to better reect the multilevel nature of the data, we will use an
alternative parameterization of the same model, with one index for each level
in the data structure.
Let Y
ij
be the jth measurement on the ith mother, and let x
ij
be 0 for males and
1 for females.
The model then equals:
Y
ij
= +
i
+ x
ij
+
ij
The parameter of interest is , the average dierence between males and females
We still need restrictions on the parameters
i
, e.g.,

i
i
= 0
Residual distribution:
ij
N(0,
2
res
)
Graphically:
i i
Mother number
DORS
average mother i
average mother i
average mother i
SAS program:
proc glm data = lizard;
class mothc;
model dors = sex mothc;
run;
Relevant SAS output:
Class Level Information
Class Levels Values
MOTHC 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
19 20 21 22 23 24 25 26 27 28 29 30
Dependent Variable: DORS
Sum of
Source DF Squares Mean Square F Value Pr > F
Model 29 268.4685062 9.2575347 3.98 <.0001
Error 72 167.3746310 2.3246477
Corrected Total 101 435.8431373
R-Square Coeff Var Root MSE DORS Mean
0.615975 4.351352 1.524680 35.03922
Source DF Type III SS Mean Square
SEX 1 16.7253690 16.7253690
MOTHC 28 256.9378690 9.1763525
Source F Value Pr > F
SEX 7.19 0.0091
MOTHC 3.95 <.0001
Note the highly signicant mother eect.
We now also obtain a signicant gender eect.
Many degrees of freedom are spent to the estimation of the mother eect, which
is not even of interest.
4.3 Mixed models
Note the dierent nature of the two factors:
SEX: denes 2 groups of interest
MOTHER: denes 29 groups not of real interest. A new sample would imply
other mothers.
In practice, one therefore considers the factor mother as a random factor.
The factor sex is a xed eect.
Thus the model is a mixed model.
In general, models can contain multiple xed and/or random factors.
The model is still of the form:
Y
ij
= +
i
+ x
ij
+
ij
But the fact that mothers can be assumed to be randomly selected from a
population of mothers is reected in the additional assumption
i
N(0,
2
moth
)
Note that we still have that the
i
have mean zero. Before, we had the restriction
i
= 0
The normality assumption for the
i
is natural and mathematically convenient,
but not necessarily realistic.
Finally, all
i
and
ij
are assumed independent.
4.4 Fitting mixed models in SAS
Mixed model with sex as xed and mother as random eect:
proc mixed data = lizard;
class mothc;
model dors = sex / solution;
random mothc;
run;
Fixed eects are specied in the MODEL statement.
Random eects are specied in the RANDOM statement.
Relevant SAS-output:
Iteration History
Iteration Evaluations -2 Res Log Like Criterion
0 1 436.17789431
1 3 407.96849207 0.00072385
2 1 407.88032382 0.00001530
3 1 407.87858406 0.00000001
Convergence criteria met.
Solution for Fixed Effects
Standard
Effect Estimate Error DF t Value Pr > |t|
Intercept 35.4904 0.3422 28 103.71 <.0001
SEX -0.8289 0.3220 72 -2.57 0.0121
Covariance Parameter
Estimates
Cov Parm Estimate
MOTHC 1.7799
Residual 2.2501
Type 3 Tests of Fixed Effects
Num Den
Effect DF DF F Value Pr > F
SEX 1 72 6.63 0.0121
Estimation method is iterative
Note the signicant dierence between male and female animals (p = 0.0121)
With the t-test, ignoring the mother eect, this was p = 0.1024.
The average dierence between males and females is estimated as

= 0.8289
Covariance parameter estimates:

2
moth
represents the variability between mothers:

2
moth
= 1.78

2
res
represents the variability within mothers:

2
res
= 2.25
4.5 The hierarchical versus marginal model
Our mixed model was given by
Y
ij
= +
i
+ x
ij
+
ij
,
i
N(0,
2
moth
),
ij
N(0,
2
res
), independent
The above model can be rewritten as
Y
ij
|
i
N( +
i
+ x
ij
,
2
res
), independent
i
N(0,
2
moth
), independent
Each equation then corresponds to one level in the multilevel data structure
The model is therefore called the hierarchical model
The hierarchical model implies a specic marginal model, i.e., the model which
describes the marginal distribution of the outcomes:
Normal distribution
Mean:
E(Y
ij
) = + x
ij
Variance:
Var(Y
ij
) = Var( +
i
+ x
ij
+
ij
) = Var(
i
+
ij
)
= Var(
i
) + Var(
ij
) =
2
moth
+
2
res
Covariance between observations from dierent mothers i and i
:
Cov(Y
ij
, Y
i
k
) = Cov( +
i
+ x
ij
+
ij
, +
i
+ x
i
k
+
i
k
)
= Cov(
i
,
i
) + Cov(
i
,
i
k
) + Cov(
ij
,
i
) + Cov(
ij
,
i
k
)
= 0
Covariance between observations j and k from the same mother i (j = k):
Cov(Y
ij
, Y
ik
) = Cov( +
i
+ x
ij
+
ij
, +
i
+ x
ik
+
ik
)
= Cov(
i
,
i
) + Cov(
i
,
ik
) + Cov(
ij
,
i
) + Cov(
ij
,
ik
)
= Var(
i
) =
2
moth
The total variability, correcting for gender dierences is decomposed as
within-cluster variability and between-cluster variability:
2
=
2
moth
+
2
res
4.03 = 1.78 + 2.25
The mother factor explains 1.78/4.03 = 44% of the total variability, after
correction for gender
Observations from dierent mothers are assumed independent
Observations from the same mother are correlated with correlation coecient
I
= Corr(Y
ij
, Y
ik
) =

2
moth
2
moth
+
2
res
=
1.78
1.78 + 2.25
= 0.44
The correlation
I
is called intraclass correlation
Note how the mixed model accounts for the correlation in the data through the
random eects
i
.
The correlation will be high in cases with much between-cluster variability, relative
to the within-cluster variability
The correlation will be low in cases with little between-cluster variability, relative
to the within-cluster variability
Graphically:
i i
Mother number
DORS
average mother i
average mother i
average mother i
Much between-cluster variability implies that observations from the same cluster
are more alike than observations from dierent clusters
Chapter 5
The paired t-test revisited
Example: The Captopril data
Analysis in SAS
Conclusion
5.1 Example: The Captopril data
A paired t-test analysis of the Captopril data yields:
An alternative analysis could be based on a mixed model
Let Y
ij
be the observation for the ith subject, taken at time point t
j
= 0, 1:
t
j
=
_
_
0 if before treatment
1 if after treatment
The mixed model is then of the form:
Y
ij
= +
i
+ t
j
+
ij
,
i
N(0,
2
subj
),
ij
N(0,
2
res
), independent
The
i
are subject-specic eects, reecting that some patients naturally have
higher BPs than others, irrespective of the treatment
Assuming that subjects are randomly sampled from a population of patients, it is
natural to assume the
i
to be random.
The
i
reect the variability between patients
5.2 Analysis in SAS
SAS program:
proc mixed data=capto;
class subject;
model y = time / solution;
random subject;
run;
Relevant SAS-output:
Class Level Information
Class Levels Values
subject 15 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Estimates
Cov Parm Estimate
subject 96.5476
Residual 37.1048
Standard
Intercept 112.33 2.9850 14 37.63 <.0001
time -9.2667 2.2243 14 -4.17 0.0010
Num Den
time 1 14 17.36 0.0010
The average dierence in BP is estimated as

= 9.27
We obtain the same result as with the paired t-test:
F = t
2
=
_
_
9.27
8.61
15
_
_
2
= 17.36, 14 degrees of freedom

2
subj
represents the variability between patients:

2
subj
= 96.55

2
res
represents the variability within patients:

2
res
= 37.10
The mixed model can again be viewed as a hierarchical model:
Y
ij
|
i
N( +
i
+ t
j
,
2
res
), independent
i
N(0,
2
subj
), independent
The implied marginal model is again a normal one:
Expectation E(Y
ij
) = + t
j
Variance
2
= Var(Y
ij
) =
2
subj
+
2
res
Observations from dierent patients independent
Observations from the same patient correlated:
I
= Corr(Y
i1
, Y
i2
) =

2
subj
2
subj
+
2
res
In our example, the total variability, not explained by the systematic treatment
eect, equals:
2
=
2
subj
+
2
res
= 96.55 + 37.10 = 133.65
The between-subject variability accounts for 96.55/133.65 = 72.24% of all
variability
The within-subject correlation is given by
I
=

2
subj
2
subj
+
2
res
=
96.55
96.55 + 37.10
= 0.7224
The above intraclass correlation does not equal the Pearson correlation between
the BP before and after treatment, which equals = 0.7343.
The reason for this dierence is that the Pearson correlation does not assume the
variances of the BP before and after treatment to be equal.
The mixed model used assumes constant variance:
2
= Var(Y
i1
) = Var(Y
i2
) =
2
subj
+
2
res
= 133.65, = 11.56
Summary statistics for both measurements:
Simple Statistics
Variable N Mean Std Dev Sum Minimum Maximum
Before 15 112.33333 10.47219 1685 98.00000 130.00000
After 15 103.06667 12.55540 1546 82.00000 125.00000
Note again that the correlation arises from the large amount of between-subject
variability, relative to the within-subject variability:
I
= Corr(Y
i1
, Y
i2
) =

2
subj
2
subj
+
2
res
Graphically:

Strong correlation Weak correlation
5.4 Conclusion
The simplest example of clustered data are paired observations, typically analyzed
using a paired t-test.
Traditionally, the within-pair correlation is circumvented by taking within-pair
dierences
i
= Y
i1
Y
i2
which are then analysed using a one-sample t-test
Hence, mixed models can be viewed as an extension of the paired t-test to:
more than 2 observations per cluster
unbalanced data: unequal number of measurements per cluster
models with covariates, e.g., sex, or others
models with multiple random eects (see later)
Chapter 6
The growth curves data
Example
The model
Analysis in SAS
ESTIMATE and CONTRAST statements
6.1 Example
Taken from Goldstein (1979).
Research question:
Is growth related to height of
mother ?
The height of 20 schoolgirls, with small, medium, or tall mothers, was measured
over a 4-year period:
Mothers height Children numbers
Small mothers < 155 cm 1 6
Medium mothers [155cm; 164cm] 7 13
Tall mothers > 164 cm 14 20
Individual proles:
Remarks:
Almost perfect linear relation between Age and Height
Much variability between girls
Little variability within girls
Fixed number of measurements per subject
Measurements taken at xed time points
6.2 The model
We will assume a linear relation between Age and Height, possibly dierent for the
dierent groups.
With cross-sectional data, the appropriate model would be an ANCOVA model:
Covariate Age
Factor Group
Interaction Age*Group
With longitudinal data, the observations are clustered within children, implying
within-child correlation
Correction for the variability between children is done through a random child
eect.
As before, let Y
ij
be the jth measurement of height for the ith cluster (child),
taken at time t
j
(age). Our model is then of the form:
Y
ij
=
_
1
+ b
i
+
2
t
j
+
ij
, if short mother
3
+ b
i
+
4
t
j
+
ij
, if medium mother
5
+ b
i
+
6
t
j
+
ij
, if tall mother
As before, it is assumed that random eects b
i
are normal with mean zero and
variance
2
child
.
The errors
ij
are normal with mean zero and variance
2
res
.
6.3 Analysis in SAS
SAS program:
proc mixed data = growth;
class group child;
model height = age group age*group / solution;
random child;
run;
Estimates
Cov Parm Estimate
CHILD 8.9603
Residual 0.7696
Standard
Effect GROUP Estimate Error DF t Value Pr > |t|
Intercept 83.1229 1.4162 17 58.69 <.0001
AGE 6.2486 0.1049 77 59.59 <.0001
GROUP 1 -1.8229 2.0846 77 -0.87 0.3846
GROUP 2 -0.1486 2.0028 77 -0.07 0.9411
GROUP 3 0 . . . .
AGE*GROUP 1 -0.9786 0.1543 77 -6.34 <.0001
AGE*GROUP 2 -0.6814 0.1483 77 -4.60 <.0001
AGE*GROUP 3 0 . . . .
Num Den
AGE 1 77 8385.15 <.0001
GROUP 2 77 0.46 0.6330
AGE*GROUP 2 77 21.66 <.0001
The hypothesis of interest is H
0
:
2
=
4
=
6
, which corresponds to testing the
interaction Age*Group
We nd a highly signicant dierence between the slopes from the three groups
(p < 0.0001)

2
child
represents the variability between children:

2
child
= 8.96

2
res
represents the variability within children:

2
res
= 0.77
Y
ij
|b
i

_
_
N(
1
+ b
i
+
2
t
j
,
2
res
), if short mother
N(
3
+ b
i
+
4
t
j
,
2
res
), if medium mother
N(
5
+ b
i
+
6
t
j
,
2
res
), if tall mother
Expectation
E(Y
ij
) =
_
1
+
2
t
j
, if short mother
3
+
4
t
j
, if medium mother
5
+
6
t
j
, if tall mother
Variance
2
= Var(Y
ij
) =
2
child
+
2
res
Observations from dierent children independent
Observations from the same child correlated:
I
= Corr(Y
i1
, Y
i2
) =

2
child
2
child
+
2
res
In our example, the total variability, not explained by the systematic trends, equals:
2
=
2
child
+
2
res
= 8.96 + 0.77 = 9.73
The between-child variability accounts for 8.96/9.73 = 92% of all variability
The within-child correlation is given by
I
=

2
child
2
child
+
2
res
=
8.96
8.96 + 0.77
= 0.9209
6.5 ESTIMATE and CONTRAST statements
As in many other SAS procedures, ESTIMATE and CONTRAST statements can
be used to obtain inferences about specic contrasts of the xed eects.
Slopes for each group separately, as well as pairwise comparisons are obtained
using the following program:
proc mixed data=growth;
class child group;
model height = group age*group / noint solution;
random child;
contrast small-medium group*age 1 -1 0;
contrast small-tall group*age 1 0 -1;
contrast medium-tall group*age 0 1 -1;
estimate small group*age 1 0 0 / cl;
estimate medium group*age 0 1 0 / cl;
estimate tall group*age 0 0 1 / cl;
run;
Note the dierent parameterization for the xed eects, when compared to the
original program:
class group child;
random child;
run;
Standard
Effect GROUP Estimate Error DF t Value Pr > |t|
GROUP 1 81.3000 1.5297 77 53.15 <.0001
GROUP 2 82.9743 1.4162 77 58.59 <.0001
GROUP 3 83.1229 1.4162 77 58.69 <.0001
AGE*GROUP 1 5.2700 0.1133 77 46.53 <.0001
AGE*GROUP 2 5.5671 0.1049 77 53.10 <.0001
AGE*GROUP 3 6.2486 0.1049 77 59.59 <.0001
Num Den
GROUP 3 77 3234.13 <.0001
AGE*GROUP 3 77 2845.30 <.0001
Estimates
Standard
Label Estimate Error DF t Value Pr > |t| Alpha Lower Upper
small 5.2700 0.1133 77 46.53 <.0001 0.05 5.0445 5.4955
medium 5.5671 0.1049 77 53.10 <.0001 0.05 5.3584 5.7759
tall 6.2486 0.1049 77 59.59 <.0001 0.05 6.0398 6.4574
Contrasts
Num Den
Label DF DF F Value Pr > F
small-medium 1 77 3.71 0.0579
small-tall 1 77 40.20 <.0001
medium-tall 1 77 21.12 <.0001
The new parameterization implies completely dierent tests.
For example, the tests reported for the Age*Group eect, under both
parameterizations correspond to the hypotheses:
model height = age group age*group; model height = group age*group / noint;

H
0
:
2
=
4
=
6
H
0
:
2
=
4
=
6
= 0
The dierence between the slopes is mainly explained from the dierence between
the third group on one hand, and the other two groups on the other hand.
Chapter 7
The linear mixed model
Random intercepts model
Remarks
The linear mixed model
Analysis in SAS
Conclusion and terminology
7.1 Random intercepts model
The model, used to describe the growth curves, was:
Y
ij
=
_
_
(
1
+ b
i
) +
2
t
ij
+
ij
, if short mother
(
3
+ b
i
) +
4
t
ij
+
ij
, if medium mother
(
5
+ b
i
) +
6
t
ij
+
ij
, if tall mother
This can be interpreted as a ANCOVA model, but with child-specic intercepts b
i
Such a b
i
represents the deviation of the intercept of a specic child from the
average intercept in the group to which that child belongs, i.e., deviation from
1
,
2
, or
3
.
An alternative way to t a random intercepts model in PROC MIXED is:
class group child;
random intercept / subject=child;
run;
The results are identical to those discussed earlier.
From now on, the mixed model can also be interpreted as a subject-specic
regression model, i.e., a regression model with subject-specic regression
parameters.
7.2 Remarks
The growth-curve dataset is an example of a longitudinal dataset
In longitudinal data, there is a natural ordering of the measurements within
clusters
The ordering is of primary interest
Our random-intercepts model implies very strong assumptions:
Parallel proles within all 3 groups
Constant variance
2
=
2
child
+
2
res
Constant correlation within children:
2
child
/(
2
child
+
2
res
)
Hence, the marginal model implicitly assumes that the variance remains constant
over time and that the correlation is the same between any two measurements
from the same subject
In the case of longitudinal data, this is often not realistic
For example, the covariance and correlation matrix of the residuals from the
ANCOVA model equal:
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
8.7041 9.6119 11.4005 10.2351 8.5174
9.6119 11.3896 13.1437 11.9719 10.2474
11.4005 13.1437 15.8781 14.3981 12.6611
10.2351 11.9719 14.3981 13.4490 12.0644
8.5174 10.2474 12.6611 12.0644 12.0655
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1.0000 0.9654 0.9697 0.9460 0.8311
0.9654 1.0000 0.9774 0.9673 0.8742
0.9697 0.9774 1.0000 0.9853 0.9147
0.9460 0.9673 0.9853 1.0000 0.9471
0.8311 0.8742 0.9147 0.9471 1.0000
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
This is the key motivation to further extend our mixed model
7.3 The linear mixed model
One way to extend the random-intercepts model is to also allow the slopes to be
subject-specic:
Y
ij
=
_
_
(
1
+ b
1i
) + (
2
+ b
2i
)t
ij
+
ij
, if short mother
(
3
+ b
1i
) + (
4
+ b
2i
)t
ij
+
ij
, if medium mother
(
5
+ b
1i
) + (
6
+ b
2i
)t
ij
+
ij
, if tall mother
As before, the random eects are assumed to be normally distributed with mean
zero:
b
i
= (b
1i
, b
2i
)
N(0, D)
The residuals
ij
are still i.i.d. N(0,
2
), independent of the random eects b
i
.
D then equals the 2 2 covariance matrix of the random eects:
D =
_
_
_
_
_
_
_
_
d
11
d
12
d
12
d
22
_
_
_
_
_
_
_
_
Interpretation of the parameters:
d
11
equals the variance of the intercepts b
1i
d
22
equals the variance of the slopes b
2i
d
12
equals the covariance between the intercepts b
1i
and the slopes b
2i
.
The correlation between the intercepts and slopes then equals:
Corr(b
1i
, b
2i
) =
d
12
d
11
d
22
7.4 Analysis in SAS
SAS program:
proc mixed data=growth;
class child group;
model height=age group age*group;
random intercept age / type=un subject=child g gcorr;
run;
As before, xed eects are to be specied in the MODEL statement, while
random eects are specied in the RANDOM statement.
The option type=un requires an unstructured covariance D, i.e., two variances
d
11
and d
22
, and one covariance d
12
, with only restriction that D is positive
(semi-)denite.
The options g and gcorr require the printout of the matrix D (in SAS termed
G) and associated correlation matrix.
Covariance Parameter Estimates
Cov Parm Subject Estimate
UN(1,1) CHILD 7.6028
UN(2,1) CHILD -0.4437
UN(2,2) CHILD 0.1331
Residual 0.4758
Estimated G Matrix
Row Effect CHILD Col1 Col2
1 Intercept 1 7.6028 -0.4437
2 AGE 1 -0.4437 0.1331
Estimated G Correlation Matrix
Row Effect CHILD Col1 Col2
1 Intercept 1 1.0000 -0.4412
2 AGE 1 -0.4412 1.0000
Num Den
AGE 1 17 3572.36 <.0001
GROUP 2 60 0.60 0.5514
AGE*GROUP 2 60 9.23 0.0003
We still get a highly signicant interaction term.
Covariance parameters:
d
11
represents the variability in subject-specic intercepts:

d
11
= 7.6028
d
22
represents the variability in subject-specic slopes:

d
22
= 0.1331
d
12
represents the covariance between subject-specic intercepts and slopes:
d
12
= 0.4437
the correlation between subject-specic intercepts and slopes is estimated as:
Corr(b
1i
, b
2i
) =
d
12
_
d
11
_
d
22
= 0.4412

2
represents the variability within children:

2
= 0.4758
Note the dierences in test results for the xed eects, when compared to those
from the earlier random intercepts model:
NOW
Num Den
AGE 1 17 3572.36 <.0001
GROUP 2 60 0.60 0.5514
AGE*GROUP 2 60 9.23 0.0003
BEFORE
Num Den
AGE 1 77 8385.15 <.0001
GROUP 2 77 0.46 0.6330
AGE*GROUP 2 77 21.66 <.0001
Y
ij
|b
i

_
_
N[(
1
+ b
1i
) + (
2
+ b
2i
)t
j
,
2
], if short mother
N[(
3
+ b
1i
) + (
4
+ b
2i
)t
j
,
2
], if medium mother
N[(
5
+ b
1i
) + (
6
+ b
2i
)t
j
,
2
], if tall mother
The expectation is the same as under the random intercepts model:
E(Y
ij
) =
_
1
+
2
t
j
, if short mother
3
+
4
t
j
, if medium mother
5
+
6
t
j
, if tall mother
Variance:
Var(Y
ij
) = Var(
1
+ b
1i
+
2
t
j
+ b
2i
t
j
+
ij
)
= Var(b
1i
+ b
2i
t
j
+
ij
)
= Var(b
2i
t
j
) + 2Cov(b
1i
, b
2i
t
j
) + Var(b
1i
) + Var(
ij
)
= d
22
t
2
j
+ 2d
12
t
j
+ d
11
+
2
Covariance between observations from dierent children i and i
:
Cov(Y
ij
, Y
i
k
)
= Cov(
1
+ b
1i
+
2
t
j
+ b
2i
t
j
+
ij
,
1
+ b
1i
+
2
t
k
+ b
2i
t
k
+
i
k
)
= Cov(b
1i
+ b
2i
t
j
+
ij
, b
1i
+ b
2i
t
k
+
i
k
)
= 0
Covariance between observations j and k from the same child i (j = k):
Cov(Y
ij
, Y
ik
)
= Cov(
1
+ b
1i
+
2
t
j
+ b
2i
t
j
+
ij
,
1
+ b
1i
+
2
t
k
+ b
2i
t
k
+
ik
)
= Cov(b
1i
+ b
2i
t
j
+
ij
, b
1i
+ b
2i
t
k
+
ik
)
= Cov(b
1i
, b
1i
) + Cov(b
1i
, b
2i
t
k
) + Cov(b
2i
t
j
, b
1i
) + Cov(b
2i
t
j
, b
2i
t
k
)
= Var(b
1i
) + Cov(b
1i
, b
2i
)t
k
+ Cov(b
2i
, b
1i
)t
j
+ Var(b
2i
, b
2i
)t
j
t
k
= d
22
t
j
t
k
+ d
12
(t
j
+ t
k
) + d
11
Correlation between observations j and k from the same child i (j = k):
Corr(Y
ij
, Y
ik
) =
d
22
t
j
t
k
+ d
12
(t
j
+ t
k
) + d
11
d
22
t
2
j
+ 2d
12
t
j
+ d
11
+
2
_
d
22
t
2
k
+ 2d
12
t
k
+ d
11
+
2
Note how extending the random intercepts model with random slopes yields a
more exible covariance structure.
Further extension of the random eects structure would allow for even more
exible variance and correlations functions.
Note, however, that the covariance structure, implied by the random-eects
model, is not necessarily a good description for the data set at hand.
For example, the tted variance function for the growth curves equals:
Var(Y
ij
) =

d
22
t
2
j
+ 2
d
12
t
j
+

d
11
+

2
= 0.1331t
2
j
+ 2(0.4437)t
j
+ 7.6028 + 0.4758
In SAS, the tted covariance and correlation matrices can be obtained from the
v and vcorr options in the RANDOM statement:
random intercept age / type=un subject=child v vcorr;
Fitted covariance and correlation matrices:
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
7.5442 7.4230 7.7776 8.1322 8.4869
7.4230 8.3865 8.3983 8.8860 9.3737
7.7776 8.3983 9.4949 9.6398 10.2606
8.1322 8.8860 9.6398 10.8694 11.1474
8.4869 9.3737 10.2606 11.1474 12.5101
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1.0000 0.9332 0.9190 0.8981 0.8736
0.9332 1.0000 0.9411 0.9307 0.9151
0.9190 0.9411 1.0000 0.9489 0.9414
0.8981 0.9307 0.9489 1.0000 0.9560
0.8736 0.9151 0.9414 0.9560 1.0000
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
The observed covariance and correlation matrix of the residuals from the
ANCOVA model equal:
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
8.7041 9.6119 11.4005 10.2351 8.5174
9.6119 11.3896 13.1437 11.9719 10.2474
11.4005 13.1437 15.8781 14.3981 12.6611
10.2351 11.9719 14.3981 13.4490 12.0644
8.5174 10.2474 12.6611 12.0644 12.0655
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1.0000 0.9654 0.9697 0.9460 0.8311
0.9654 1.0000 0.9774 0.9673 0.8742
0.9697 0.9774 1.0000 0.9853 0.9147
0.9460 0.9673 0.9853 1.0000 0.9471
0.8311 0.8742 0.9147 0.9471 1.0000
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
Graphically:
Obviously, the variance cannot be described by a quadratic function with
postitive curvature.
One way to further extend the marginal covariance structure is to add random
eects to the model, e.g., random coecients for Age
2
:
random intercept age age*age / type=un subject=child v vcorr;
New tted covariance matrix, compared to observed covariance from ANCOVA
residuals:
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
8.2014 10.0364 10.9352 10.6322 9.1276
10.0364 12.9310 13.8330 13.5387 11.7826
10.9352 13.8330 15.4548 15.0042 13.2776
10.6322 13.5387 15.0042 15.2944 13.6127
9.1276 11.7826 13.2776 13.6127 13.0531
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
8.7041 9.6119 11.4005 10.2351 8.5174
9.6119 11.3896 13.1437 11.9719 10.2474
11.4005 13.1437 15.8781 14.3981 12.6611
10.2351 11.9719 14.3981 13.4490 12.0644
8.5174 10.2474 12.6611 12.0644 12.0655
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
The tted variance function is now 4 degree polynomial:
Var(Y
ij
) = d
33
t
4
j
+ 2d
23
t
3
j
+ d
22
t
2
j
+ 2d
13
t
2
j
+ 2d
12
t
j
+ d
11
+
2
Estimated random-eects covariance:
Estimated G Matrix
Row Effect CHILD Col1 Col2 Col3
1 Intercept 1 96.3384 -33.4752 2.0725
2 AGE 1 -33.4752 11.5273 -0.7160
3 AGE*AGE 1 2.0725 -0.7160 0.04508
Graphically:
New tted correlation matrix, compared to observed correlation from ANCOVA
residuals:
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1.0000 0.9746 0.9713 0.9493 0.8822
0.9746 1.0000 0.9785 0.9627 0.9069
0.9713 0.9785 1.0000 0.9759 0.9348
0.9493 0.9627 0.9759 1.0000 0.9634
0.8822 0.9069 0.9348 0.9634 1.0000
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1.0000 0.9654 0.9697 0.9460 0.8311
0.9654 1.0000 0.9774 0.9673 0.8742
0.9697 0.9774 1.0000 0.9853 0.9147
0.9460 0.9673 0.9853 1.0000 0.9471
0.8311 0.8742 0.9147 0.9471 1.0000
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
Conclusion:
The role of random eects is to model the variance and
association structure
Adding the quadratic random Age eect again implies some changes in the tests
for the xed eects, when compared to those from the previous model with only
random intercepts and linear Age eects:
NOW
Num Den
AGE 1 17 3594.53 <.0001
GROUP 2 40 2.21 0.1228
AGE*GROUP 2 40 9.39 0.0005
BEFORE
Num Den
AGE 1 17 3572.36 <.0001
GROUP 2 60 0.60 0.5514
AGE*GROUP 2 60 9.23 0.0003
7.6 Conclusion and terminology
The linear mixed model is a linear regression model with two sets of regression
parameters:
Fixed eects
Random eects b
i
N(0, D)
The xed eects are used to model the average outcome
The random eects are used to model the covariance structure
All parameters in D, jointly with the residual variance
2
, are called variance
components
Chapter 8
The rat data
Example
A linear mixed model
Fitting the model in SAS
8.1 Example
Research question (Dentistry, K.U.Leuven):
How does craniofacial growth
depend on testosteron production ?
Randomized experiment in which 50 male Wistar rats are randomized to:
Control (15 rats)
Low dose of Decapeptyl (18 rats)
High dose of Decapeptyl (17 rats)
Treatment starts at the age of 45 days; measurements taken every 10 days, from
day 50 on.
The responses are distances (pixels) between well dened points on x-ray pictures
of the skull of each rat:
Measurements with respect to the roof, base and height of the skull. Here, we
consider only one response, reecting the height of the skull.
Individual proles:
Complication: Dropout due to anaesthesia (56%):
# Observations
Age (days) Control Low High Total
50 15 18 17 50
60 13 17 16 46
70 13 15 15 43
80 10 15 13 38
90 7 12 10 29
100 4 10 10 24
110 4 8 10 22
Remarks:
Much variability between rats, much less variability within rats
Fixed number of measurements scheduled per subject, but not all
measurements available due to dropout, for known reason.
Measurements taken at xed time points
8.2 A linear mixed model
Since linear mixed models assume a linear regression for each cluster separately,
they can also be used for unbalanced data, i.e., data with unequal number of
measurements per cluster.
Note that this was also the case for the lizard data.
Individual proles show very similar evolutions for all rats (apart from
measurement error)
This suggests a random-intercepts model
Non-linearity can be accounted for by using a logarithmic transformation of the
time scale:
Age
ij
t
ij
= ln[1 + (Age
ij
45)/10)]
We then get the following model:
Y
ij
= (
0
+ b
i
) + (
1
L
i
+
2
H
i
+
3
C
i
)t
ij
+
ij
=
_
0
+ b
i
+
1
t
ij
+
ij
, if low dose
0
+ b
i
+
2
t
ij
+
ij
, if high dose
0
+ b
i
+
3
t
ij
+
ij
, if control.
L
i
, H
i
, and C
i
are indicator variables:
L
i
=
_
_
1 if low dose
0 otherwise
H
i
=
_
_
1 if high dose
0 otherwise
C
i
=
_
_
1 if control
0 otherwise

0
: average response at the start of the treatment (independent of treatment)

1
,
2
, and
3
: average time eect for each treatment group
b
i
: subject-specic intercepts
8.3 Fitting the model in SAS
The following SAS program can be used:
data rats; proc mixed data = rats ;
set rats; class treat rat;
t=log(1+(age-45)/10); model y = treat*t / solution;
run; random intercept / type=un subject=rat g;
contrast treatment effect treat*t 1 -1 0, treat*t 1 0 -1;
run;
Note the parameterization of the xed eects
UN(1,1) RAT 3.5649
Residual 1.4448
Standard
Effect TREAT Estimate Error DF t Value Pr > |t|
Intercept 68.6074 0.3312 49 207.13 <.0001
t*TREAT con 7.3138 0.2808 199 26.05 <.0001
t*TREAT hig 6.8711 0.2276 199 30.19 <.0001
t*TREAT low 7.5069 0.2252 199 33.34 <.0001
Num Den
t*TREAT 3 199 734.11 <.0001
Contrasts
Num Den
Label DF DF F Value Pr > F
treatment effect 2 199 2.32 0.1013
Note the dierence between the test for t*treat and the test for the treatment
eect
A lot of variability between rats, while little variability within rats:

2
rat
= 3.565 represents the variability between rats

2
res
= 1.445 represents the variability within rats
No signicant dierence between the treatment groups with respect to the
average evolution over time (p = 0.1013)
As before, the variance and correlation structure need to be explored to check
model t.
Chapter 9
The BLSA prostate data
Example
A linear mixed model
Fitting the model in SAS
9.1 Example
References:
Carter et al (1992, Cancer Research).
Carter et al (1992, Journal of the American Medical Association).
Morrell et al (1995, Journal of the American Statistical Association).
Pearson et al (1994, Statistics in Medicine).
Prostate disease is one of the most common and most costly medical problems in
the United States
Important to look for markers which can detect the disease at an early stage
Prostate-Specic Antigen is an enzyme produced by both normal and cancerous
prostate cells
PSA level is related to the volume of prostate tissue.
Problem: Patients with Benign Prostatic Hyperplasia also have an increased PSA
level
Overlap in PSA distribution for cancer and BPH cases seriously complicates the
detection of prostate cancer.
Research question (hypothesis based on clinical practice):
Can longitudinal PSA proles be used to detect prostate
cancer in an early stage ?
A retrospective case-control study based on frozen serum samples:
16 control patients
20 BPH cases
14 local cancer cases
4 metastatic cancer cases
Complication: No perfect match for age at diagnosis and years of follow-up
possible
Hence, analyses will have to correct for these age dierences between the
diagnostic groups.
Individual proles:
Remarks:
Much variability between subjects
Little variability within subjects
Highly unbalanced data
9.2 A linear mixed model
A model for the prostate data:
ln(PSA
ij
+ 1) =
1
Age
i
+
2
C
i
+
3
B
i
+
4
L
i
+
5
M
i
+(
6
Age
i
+
7
C
i
+
8
B
i
+
9
L
i
+
10
M
i
) t
ij
+(
11
Age
i
+
12
C
i
+
13
B
i
+
14
L
i
+
15
M
i
) t
2
ij
+ b
1i
+ b
2i
t
ij
+ b
3i
t
2
ij
+
ij
.
C
i
, B
i
, L
i
, M
i
are indicators for the 4 diagnostic groups.
Average age-corrected quadratic proles for all groups, modeled through the
xed eects
Random eects b
1i
, b
2i
, and b
3i
allowing subject-specic evolutions to dier
from the average in that diagnostic group, even correcting for age dierences
9.3 Fitting the model in SAS
SAS program:
proc mixed data=prostate;
class id group;
model lnpsa = group age group*time age*time group*time2 age*time2 / noint solution;
random intercept time time2 / type=un subject=id g gcorr ;
run;
Note again the particular parameterization for the xed eects
Relevant SAS output: Covariance Parameter Estimates
UN(1,1) XRAY 0.4518
UN(2,1) XRAY -0.5178
UN(2,2) XRAY 0.9153
UN(3,1) XRAY 0.1625
UN(3,2) XRAY -0.3356
UN(3,3) XRAY 0.1308
Residual 0.02820
Estimated G Matrix
Effect XRAY Col1 Col2 Col3
Intercept 19 0.4518 -0.5178 0.1625
time 19 -0.5178 0.9153 -0.3356
time2 19 0.1625 -0.3356 0.1308
Estimated G Correlation Matrix
Effect XRAY Col1 Col2 Col3
Intercept 19 1.0000 -0.8053 0.6686
time 19 -0.8053 1.0000 -0.9700
time2 19 0.6686 -0.9700 1.0000
Standard
Effect group Estimate Error DF t Value Pr > |t|
group 1 -1.0984 0.9763 299 -1.13 0.2615
group 2 -0.5228 1.0895 299 -0.48 0.6317
group 3 0.2964 1.0587 299 0.28 0.7797
group 4 1.5494 1.0856 299 1.43 0.1546
AGEDIAG 0.02655 0.01423 299 1.87 0.0631
time*group 1 0.5681 1.4725 299 0.39 0.6999
time*group 2 0.3956 1.6377 299 0.24 0.8093
time*group 3 -1.0359 1.5928 299 -0.65 0.5159
time*group 4 -1.6049 1.6258 299 -0.99 0.3244
AGEDIAG*time -0.01117 0.02142 299 -0.52 0.6026
time2*group 1 -0.1295 0.6100 299 -0.21 0.8320
time2*group 2 -0.1585 0.6723 299 -0.24 0.8138
time2*group 3 0.3419 0.6563 299 0.52 0.6028
time2*group 4 0.3951 0.6660 299 0.59 0.5535
AGEDIAG*time2 0.002259 0.008829 299 0.26 0.7982
Num Den
group 4 299 15.90 <.0001
AGEDIAG 1 299 3.48 0.0631
time*group 4 299 7.85 <.0001
AGEDIAG*time 1 299 0.27 0.6026
time2*group 4 299 4.44 0.0017
AGEDIAG*time2 1 299 0.07 0.7982
Note the very strong correlations between random eects
CONTRAST statements can be used to test for group dierences
Based on the xed eects, tted average proles can be plotted (at median age at
diagnosis):
Chapter 10
The Leuven diabetes project
Introduction
A variety of multilevel models
Including covariates at various levels
10.1 Introduction: the DPL project
The Diabetes Project Leuven
In Belgium, general practitioners (GPs) cannot rely on structured assistance of
dieticians or diabetes nurse educators in their practice.
The DPL intends to study the eect of implementing a structured model for
chronic diabetes care on patients clinical outcomes.
GPs will be oered assistance and can redirect patients to the diabetes care team,
consisting of a nurse educator, a dietician, an ophthalmologist and an internal
medicine doctor.
In DPL, two programs were implemented and GPs were randomized to one of two
groups:
LIP: Low Intervention Program (group A)
HIP: High Intervention Program (group R)
We consider the analysis of GPs in the HIP group:
61 GPs
1577 patients
number of patients per GP varies between 5 and 138, with a median of 47
Patients were measured twice:
When the program was initiated (time T0)
After one year (time T1)
The outcome studied here is HbA1c, glycosylated hemoglobin:
Molecule in red blood cells that attaches to glucose (blood sugar)
High values reect more glucose in blood
In diabetes patients, HbA1c gives a good estimate of how well diabetes is
being managed over the last 2 or 3 months
Non-diabetics have values between 4% and 6%
HbA1c above 7% means diabetes is poorly controlled, implying higher risk for
long-term complications.
10.2 A variety of multilevel models
Let Y
ijk
be the kth measurement of HbA1, for the jth patient, of the ith GP
We have 3-level data, hence random eects can enter the models at various levels
Several models for studying the longitudinal evolutions will be illustrated and
compared:
No random eects
Random GP eects
Random patient eects
Random eects for GP and patient
10.2.1 Model 1: No random eects
Y
ijk
=
0
+
1
t
k
+
ijk
,
ijk
N(0,
2
res
)
SAS program:
proc mixed data=dpla; proc glm data=dpla;
model hba1c = time / solution; model hba1c = time / solution;
run; run;
Relevant output:
Estimates
Cov Parm Estimate
Residual 1.2309
Standard
Intercept 7.1357 0.02823 2966 252.81 <.0001
time -0.3899 0.04076 2966 -9.57 <.0001
10.2.2 Model 2: Random GP eects
Y
ijk
=
0
+
1
t
k
+ a
i
+
ijk
, a
i
N(0,
2
GP
),
ijk
N(0,
2
res
)
SAS program:
proc mixed data=dpla;
class mdnr;
model hba1c = time / solution;
random intercept / subject=mdnr;
run;
Relevant output:
Intercept mdnr 0.07093
Residual 1.1709
Standard
Intercept 7.1695 0.04519 60 158.66 <.0001
time -0.3873 0.03978 2906 -9.73 <.0001
10.2.3 Model 3: Random patient eects
Y
ijk
=
0
+
1
t
k
+ b
j(i)
+
ijk
, b
j(i)
N(0,
2
PAT
),
ijk
N(0,
2
res
)
SAS program:
/* unique patient numbers */ /* patients numbered within GPs */
proc mixed data=dpla; proc mixed data=dpla;
class md_patient; class mdnr patientnr;
random intercept / subject=md_patient; random intercept / subject=patientnr(mdnr);
run; run;
Relevant output:
Intercept patientnr(mdnr) 0.6675
Residual 0.5831
Standard
Intercept 7.1392 0.02838 1571 251.54 <.0001
time -0.3785 0.02851 1395 -13.28 <.0001
10.2.4 Model 4: Random GP and patient eects
Y
ijk
=
0
+
1
t
k
+ a
i
+ b
j(i)
+
ijk
,
a
i
N(0,
2
GP
), b
j(i)
N(0,
2
PAT
),
ijk
N(0,
2
res
)
SAS program:
proc mixed data=dpla; proc mixed data=dpla;
class mdnr patientnr; class mdnr patientnr;
random intercept / subject=mdnr; random mdnr patientnr(mdnr);
random intercept / subject=patientnr(mdnr); run;
run;
Relevant output:
Residual 0.5837
Standard
Intercept 7.1668 0.04241 60 169.00 <.0001
time -0.3780 0.02851 1395 -13.26 <.0001
10.2.5 Summary of results
Model 1 Model 2 Model 3 Model 4
Parameter Est. (s.e.) Est. (s.e.) Est. (s.e.) Est. (s.e.)
Fixed eects:
0
7.1357(0.0282) 7.1695(0.0452) 7.1392(0.0284) 7.1668(0.0424)
1
0.3899(0.0408) 0.3873(0.0398) 0.3785(0.0286) 0.3780(0.0285)
Variance components:
2
GP
0.0709 0.0544
2
PAT
0.6675 0.6171
2
res
1.2309 1.1709 0.5831 0.5837
No standard errors reported for variance components, since standard Z-tests do
not produce correct tests (see later)
The various models use dierent decompositions of the total variability:
Model 1:

2
=

2
res
= 1.2309
Model 2:

2
=

2
GP
+

2
res
= 0.0709 + 1.1709 = 1.2418
Model 3:

2
=

2
PAT
+

2
res
= 0.6675 + 0.5831 = 1.2506
Model 4:

2
=

2
GP
+

2
PAT
+

2
res
= 0.0544 + 0.6171 + 0.5837 = 1.2552
Inclusion of random eects has little eect on estimation of xed eects but has
severe impact on the standard errors:
Larger standard errors for between-cluster eects (intercept
0
)
Smaller standard errors for within-cluster eects (time
1
)
There is a signicant decrease in HbA1c, under all models
The models also imply specic correlation structures.
For example, the marginal association structure implied by Model 4 equals:
Observations from dierent GPs i and i
, i = i
, are not correlated:
Corr(Y
ijk
, Y
i
k
) = 0
Observations from same GP but dierent patients j and j
, j = j
, are
correlated:
Corr(Y
ijk
, Y
ij
k
) =
2
GP
2
GP
+

2
PAT
+

2
res
=
0.0544
1.2552
= 0.0433
Observations k and k
, k = k
, from same patient are correlated:
Corr(Y
ijk
, Y
ijk
) =
2
GP
+

2
PAT
2
GP
+

2
PAT
+

2
res
=
0.0544 + 0.6171
1.2552
= 0.5350
10.3 Including covariates at various levels
Additional covariates can be added to explain variability at the dierent levels, or
to study what patient and/or GP characteristics are related to time trends.
We exend Model 4 with the following covariates:
At GP level: Practice form (one, two, more)
At patient level:
BMI at baseline
Whether or not patient is a newly diagnosed diabetic (1: yes, 0: no)
We will investigate the eect of each covariate separately.
Obviously, models with multiple covariates are possible as well.
10.3.1 Model 5: Correcting for dierent practice forms
SAS program:
class mdnr patientnr practice;
model hba1c = practice time time*practice / solution;
random intercept / subject=patientnr(mdnr);
run;
Relevant output:
Residual 0.5838
Num Den
practice 2 1393 3.08 0.0461
time 1 1393 162.59 <.0001
time*practice 2 1393 1.28 0.2786
Standard
Effect practice Estimate Error DF t Value Pr > |t|
Intercept 7.1408 0.07505 58 95.14 <.0001
practice Mor -0.1158 0.1105 1393 -1.05 0.2948
practice One 0.1407 0.1001 1393 1.40 0.1603
practice Two 0 . . . .
time -0.3659 0.04976 1393 -7.35 <.0001
time*practice Mor 0.05052 0.07467 1393 0.68 0.4988
time*practice One -0.06162 0.06680 1393 -0.92 0.3565
time*practice Two 0 . . . .
Since Time is included as a continuous covariate, the main eect of Practice
reects dierences at baseline between the various practice forms
We nd a signicant dierence at baseline (p = 0.0461) with lower average values
of HbA1c the more GPs work together in group practices.
This dierence does not change over time, as the practice form has no signicant
eect on the change over time (p = 0.2786).
10.3.2 Model 6: Correcting for dierent BMI at baseline
SAS program:
class mdnr patientnr;
model hba1c = bmi0 time time*bmi0 / solution;
run;
Relevant output:
Residual 0.5468
Num Den
bmi0 1 1323 10.55 0.0012
time 1 1323 0.94 0.3320
bmi0*time 1 1323 1.63 0.2023
Standard
Intercept 6.6325 0.1654 60 40.09 <.0001
bmi0 0.01750 0.005386 1323 3.25 0.0012
time -0.1573 0.1621 1323 -0.97 0.3320
bmi0*time -0.00685 0.005367 1323 -1.28 0.2023
We nd a signicantly higher baseline value for HbA1c as the intial BMI is larger
(p = 0.0012)
The average time trend is not signicantly related to the intial BMI level
(p = 0.2023).
10.3.3 Model 7: Correcting for new diagnosis
SAS program:
class mdnr patientnr ;
model hba1c = new0 time time*new0 / solution;
run;
Relevant output:
Residual 0.5385
Num Den
new0 1 1321 14.67 0.0001
time 1 1321 115.15 <.0001
new0*time 1 1321 52.82 <.0001
Standard
Intercept 7.1172 0.04367 60 162.97 <.0001
new0 0.4030 0.1052 1321 3.83 0.0001
time -0.3148 0.02934 1321 -10.73 <.0001
new0*time -0.7762 0.1068 1321 -7.27 <.0001
Since Time is included as a continuous covariate, the main eect of New0 reects
dierences at baseline between newly diagnosed diabetics and others.
We nd a signicant dierence at baseline (p = 0.0001) with higher average
values of HbA1c for newly diagnosed patients.
The newly diagnosed patients have a steeper decrease in HbA1c than the others
(p < 0.0001).
We can test whether, after one year, both groups are at the same level:
estimate equal at T1 new0 1 time*new0 1;
Additional output:
Estimates
Standard
Label Estimate Error DF t Value Pr > |t|
equal at T1 -0.3732 0.1106 1321 -3.38 0.0008
Hence, after one year, the newly diagnosed diabetics have lower average values for
HbA1c (p = 0.0008)
Graphically:
This can be explained from the fact that the disease gets worse over time, hence
HbA1c is more dicult to keep under control.
We therefore repeat the analysis, correcting for the number of years patients have
been diabetic (0 for newly diagnosed patients).
model hba1c = duration new0 time time*new0 / solution;
estimate equal at T1 new0 1 time*new0 1;
run;
Relevant output:
Residual 0.5401
Num Den
duration 1 1320 67.83 <.0001
new0 1 1320 35.88 <.0001
time 1 1320 114.40 <.0001
new0*time 1 1320 52.87 <.0001
Standard
Intercept 6.8737 0.05181 60 132.67 <.0001
duration 0.03096 0.003759 1320 8.24 <.0001
new0 0.6432 0.1074 1320 5.99 <.0001
time -0.3141 0.02937 1320 -10.70 <.0001
new0*time -0.7773 0.1069 1320 -7.27 <.0001
Estimates
Standard
equal at T1 -0.1340 0.1125 1320 -1.19 0.2338
The dierence after one year, between newly diagnosed diabetics and others is no
longer signicant (p = 0.2338) after correction for the number of years patients
have been diabetic.
Chapter 11
Estimation of Random Eects
Empirical Bayes estimation
Example: Leuven diabetes project
Example: Prostate data
Average versus cluster-specic prediction
11.1 Empirical Bayes estimation
Random eects reect how specic clusters deviate from the population average
For example, for the Leuven diabetes project, Model 4 for the outcome Y
ijk
being
the kth measurement of HbA1, for the jth patient, of the ith GP, was given by:
Y
ijk
=
0
+
1
t
k
+ a
i
+ b
j(i)
+
ijk
,
a
i
N(0,
2
GP
), b
j(i)
N(0,
2
PAT
),
ijk
N(0,
2
res
)
The parameter a
i
expresses how the average HbA1c level of patients treated by
GP i diers from the overall population average.
The parameter b
j(i)
expresses how the average of patient j treated by GP i
dieres from the average of that specic GP.
Estimation of the random eects can be helpful for detecting outlying proles or
clusters
Since these parameters are assumed to be stochastic, Bayesian methods are
applied.
Posterior means:
a
i
= E(a
i
| Y
ijk
, j, k)
b
j(i)
= E(b
j(i)
| Y
ijk
, k)
The so-obtained estimates are called Empirical Bayes (EB) estimates. They
are the expected random eects, conditionally on the observed data for that
specic cluster
In practice histograms and/or scatterplots of EB estimates are used to detect
outlying clusters
11.2 Example: Leuven diabetes project
We re-consider Model 4 given by:
Y
ijk
=
0
+
1
t
k
+ a
i
+ b
j(i)
+
ijk
The parameters a
i
and b
j(i)
represent GP and patient eects, respectively.
Histograms and scatterplots will be used to study the EB estimates for a
i
and b
j(i)
SAS program for calculation of EB estimates:
model hba1c = time / solution;
random intercept / subject=mdnr solution;
random intercept / subject=patientnr(mdnr) solution;
ods listing exclude solutionr;
ods output solutionr=out;
run;
The ODS statements are used to write the EB estimates into a SAS output data
set, and to prevent SAS from printing them in the output window.
EB estimates:
Obs mdnr gpeb
1 2 0.21976
2 5 0.02113
3 6 0.18372
4 7 0.01447
5 8 0.08236
6 10 0.15512
7 11 -0.07875
8 13 -0.25004
9 14 -0.06225
. ... .......
59 155 -0.01637
60 156 0.26761
61 165 -0.23516
Obs mdnr patientnr patienteb
1 2 1 0.00163
2 2 2 -0.03232
3 2 3 2.10624
.. . .. .......
25 2 25 -0.91490
26 2 26 0.13741
27 5 1 -0.06719
28 5 2 -0.61032
29 5 3 0.44199
.... ... .. .......
1569 165 22 -0.43632
1570 165 23 -1.18312
1571 165 24 0.17469
1572 165 25 0.44626
Histograms of both sets of EB estimates:
We notice some patients with extremely large HbA1c values. The largest
estimated b
j(i)
is

b
2(140)
= 3.46
This patient was newly diagnosed, with initial BMI equal to 26.40, intial HbA1c
equal to 14.3%, and no follow-up measurement after one year
The initial HbA1c level of 14.3% is extremely high:
Scatterplot of EB estimates for patients versus GPs:
Plots can also be made in relation with patient or GP characteristics:

Note how the patient with the largest estimate for b
j(i)
was treated by the GP
with the largest estimated a
i
.
It is therefore worthwhile to repeat the analysis with this subject removed from
the data
The estimate

a
140
drops from 0.40 to 0.27 and four other GPs have now a higher
estimate for their eect a
i
.
EB estimates can also be calculated based on other models which include patient
or GP characteristics as covariates/factors
Extreme EB estimates then reect that a specic GP or patient within GP is
outlying, while this extreme behaviour cannot be explained by the covariates in
the model.
11.3 Example: Prostate data
We re-consider the model
ln(PSA
ij
+ 1)
=
1
Age
i
+
2
C
i
+
3
B
i
+
4
L
i
+
5
M
i
+(
6
Age
i
+
7
C
i
+
8
B
i
+
9
L
i
+
10
M
i
) t
ij
+(
11
Age
i
+
12
C
i
+
13
B
i
+
14
L
i
+
15
M
i
) t
2
ij
+ b
1i
+ b
2i
t
ij
+ b
3i
t
2
ij
+
ij
.
Again, histograms and scatterplots of components of

b
i
can be used to detect
model deviations or subjects with exceptional evolutions over time
Strong negative correlations in agreement with correlation matrix corresponding to
tted D:
D
corr
=
_
_
_
_
_
_
_
_
_
_
_
_
_
_
1.000 0.805 0.669
0.805 1.000 0.970
0.669 0.970 1.000
_
_
_
_
_
_
_
_
_
_
_
_
_
_
Histograms and scatterplots show outliers
Subjects #22, #28, #39, and #45, have highest four slopes for time
2
and
smallest four slopes for time, i.e., with the strongest (quadratic) growth.
Subjects #22, #28 and #39 have been further examined and have been shown to
be metastatic cancer cases which were misclassied as local cancer cases.
Subject #45 is the metastatic cancer case with the strongest growth
11.4 Average versus cluster-specic prediction
Once the EB estimates have been calculated, predictions can be obtained both at
the cluster level, as well as on the population average level.
Re-consider Model 4 for the Leuven diabetes project:
Y
ijk
=
0
+
1
t
k
+ a
i
+ b
j(i)
+
ijk
Predictions:
On population average level:
E(Y
ijk
) =

0
+

1
t
k
On cluster level:
Y
ijk
=

0
+

1
t
k
+

a
i
+

b
j(i)
11.5 Example: Leuven diabetes project
SAS program for predictions under Model 4:
model hba1c = time / solution outpm=predmean outp=pred;
run;
proc print data=predmean;
proc print data=pred;
run;
The option predmean requests calculation of the predicted means
The option pred requests calculation of predictions at cluster level
Table of predicted means
(option predmean):
Obs hba1c mdnr patientnr time Pred
1 6.4 2 1 0 7.16685
2 8.0 2 1 1 6.78883
3 6.7 2 2 0 7.16685
4 7.6 2 2 1 6.78883
5 11.2 2 3 0 7.16685
6 9.4 2 3 1 6.78883
7 6.8 2 4 0 7.16685
8 6.8 2 4 1 6.78883
2965 7.5 165 24 0 7.16685
2966 6.5 165 24 1 6.78883
2967 7.7 165 25 0 7.16685
2968 7.1 165 25 1 6.78883
Table with predictions on subject level
(option pred):
Obs hba1c mdnr patientnr time Pred
1 6.4 2 1 0 7.38824
2 8.0 2 1 1 7.01022
3 6.7 2 2 0 7.35429
4 7.6 2 2 1 6.97628
5 11.2 2 3 0 9.49285
6 9.4 2 3 1 9.11484
7 6.8 2 4 0 7.11667
8 6.8 2 4 1 6.73866
2965 7.5 165 24 0 7.10638
2966 6.5 165 24 1 6.72837
2967 7.7 165 25 0 7.37795
2968 7.1 165 25 1 6.99994
Components needed to calculate predictions:
Standard
Intercept 7.1668 0.04241 60 169.00 <.0001
time -0.3780 0.02851 1395 -13.26 <.0001
Obs mdnr gpeb
1 2 0.21976
Obs mdnr patientnr patienteb
1 2 1 0.00163
2 2 2 -0.03232
3 2 3 2.10624
Population average HbA1c values at baseline and after one year:
E(Y
ij1
) = 7.1668 0.3780 0 = 7.1668
E(Y
ij2
) = 7.1668 0.3780 1 = 6.7888
Subject-specic predicted HbA1c values for rst three patients treated by GP 2:
Y
211
= 7.1668 + 0.2198 + 0.0016 = 7.3882
Y
212
= 6.7888 + 0.2198 + 0.0016 = 7.0102
Y
221
= 7.1668 + 0.2198 0.0323 = 7.3543
Y
222
= 6.7888 + 0.2198 0.0323 = 6.9763
Y
231
= 7.1668 + 0.2198 + 2.1062 = 9.4929
Y
232
= 6.7888 + 0.2198 + 2.1062 = 9.1148
Chapter 12
Concluding remarks
Introduction
Tests for variance components
Distributional assumptions for random eects
Missing data issues
12.1 Introduction
Many examples of linear mixed models for longitudinal or clustered data have been
discussed
Most emphasis was on model formulation, SAS implementation, and
interpretation of results
A number of issues have not been discussed:
Estimation methods (ML, REML, . . . )
Inference (F-test, t-test, LR test, Wald test, . . . )
Model checking
Inuence analysis
. . .
These topics are much more dicult and technical than in classical linear models
for cross-sectional data, and are therefore outside the scope of this course
Three illustrations are given:
Tests for variance components
Distributional assumptions for random eects
Missing data issues
All aspects discussed here equally well apply to generalized linear mixed models
and non-linear mixed models.
12.2 Tests for variance components
In a number of situations, it might be of interest to test whether variance
components equal zero.
For example, consider the Leuven diabetes project, it may be of interest to know
whether there is any variability between GPs
As before, let Y
ijk
being the kth measurement of HbA1, for the jth patient, of the
ith GP.
Model 4 was given by:
Y
ijk
=
0
+
1
t
k
+ a
i
+ b
j(i)
+
ijk
,
a
i
N(0,
2
GP
), b
j(i)
N(0,
2
PAT
),
ijk
N(0,
2
res
)
Absence of any heterogeneity between GPs would be reected in
2
GP
= 0
It is therefore of interest to test H
0
:
2
GP
= 0 versus H
A
:
2
GP
> 0
The default output from SAS is:
Residual 0.5837
In contrast to, e.g., xed eects, SAS does not report standard errors,
test-statistics, nor p-values
These can be requested by specifying the covtest option in the PROC MIXED
statement:
proc mixed data=dpla covtest;
The output for the covariance parameters then becomes:
Standard Z
Cov Parm Subject Estimate Error Value Pr Z
Intercept mdnr 0.05439 0.01858 2.93 0.0017
Intercept patientnr(mdnr) 0.6171 0.03720 16.59 <.0001
Residual 0.5837 0.02256 25.87 <.0001
The reported p-values are based on the N(0, 1) approximation to the Z-statistic,
which cannot reect the correct sampling variability in the estimation of the
variance components as these are estimated under the restriction of being positive.
This so-called boundary problem requires correction of the classical p-values.
The correction depends on the model, and sometimes requires simulation methods.
In the above example, the correction reduces to halving the reported p-values.
As another example, consider the previous analysis of the growth curves, with
random intercepts, and random linear as well as quadratic Age eects.
The reported tests for the variance components are:
Standard Z
Cov Parm Subject Estimate Error Value Pr Z
UN(1,1) CHILD 96.3384 58.5931 1.64 0.0501
UN(2,1) CHILD -33.4752 17.5533 -1.91 0.0565
UN(2,2) CHILD 11.5273 5.3639 2.15 0.0158
UN(3,1) CHILD 2.0725 1.0800 1.92 0.0550
UN(3,2) CHILD -0.7160 0.3313 -2.16 0.0307
UN(3,3) CHILD 0.04508 0.02069 2.18 0.0147
Residual 0.2655 0.05936 4.47 <.0001
Apart from the boundary problem, the p-value reported for UN(3,3) corresponds
to the hypothesis H
0
: d
33
= 0.
Under H
0
, the random-eects covariance matrix D is of the form:
D =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
d
11
d
12
d
13
d
12
d
22
d
23
d
13
d
23
0
_
_
_
_
_
_
_
_
_
_
_
_
_
_
As a covariance matrix, D needs to be positive (semi-)denite. Hence the only
meaningful hypothesis to test would be H
0
: d
13
= d
23
= d
33
= 0, implying that
D is of the form
D =
_
_
_
_
_
_
_
_
_
_
_
_
_
_
d
11
d
12
0
d
12
d
22
0
0 0 0
_
_
_
_
_
_
_
_
_
_
_
_
_
_
Conclusion:
The default variance components tests often
do not test meaningful hypotheses,
and/or report wrong p-values
SAS only reports Wald tests for variance components.
However the above discussed problems equally well apply to Likelihood Ratio and
Score tests, as the three are asymptotically equivalent.
12.3 Distributional assumptions for random eects
We continue the analysis of the Leuven diabetes project, with Model 4
Histograms of EB estimates of GP and patient eects were:
The histograms seem to suggest that the normality assumption for the random
eects a
i
and b
j(i)
is questionable.
However, one should realize that the precision with which a
i
and b
j(i)
are
estimated depends on many aspects, and can vary from patient to patient and
from GP to GP
So, the above histograms do not necessarily reect non-normality of the random
eects a
i
and b
j(i)
.
Outlying EB estimates can be the reection of a random eect estimated with
very little precision.
The dierences in precision can be corrected for by standardizing the EB estimates.
However, standardized EB estimates still do not necessarily reect the correct
random eects distribution.
Too illustrate this, consider a small simulation example:
1000 proles with 5 measurements, balanced
1000 random intercepts sampled from
1
2
N(2, 1) +
1
2
N(2, 1)
Error components
ij
with variance
2
= 30
Data analysed assuming normality for the intercepts
Histogram of sampled intercepts and empirical Bayes estimates:
Apparently, the model assumption sometimes forces the estimates to satisfy the
assumption.
Conclusion:
The normality assumption for random eects cannot be
tested within the context of the linear mixed model.
Model extensions are needed.
Fortunately, inferences about the xed eects are very robust with respect to
model deviations, provided the data set contains sucient independent clusters:
Lizard data: sucient mothers
Rat data: sucient rats
Growth curves: sucient children
Leuven diabetes project: sucient GPs
12.4 Missing data issues
A key feature of mixed models is that they can be used to model unbalanced data.
In the context of longitudinal data, this includes situations where not all subjects
have the same number of repeated measurements, or where subjects are measured
at dierent time points.
Mixed models are therefore often used in contexts with missing data, e.g., subjects
left the study prematurely.
However although mixed models can technically handle such unbalanced data sets,
the obtained results can be severely biased in cases where missingness is related to
the outcome studied.
General principle:
Dropout related to the outcome can
imply biased results
Unrelated dropout:
Subjects moving
Subjects dying of other causes
Lost blood samples
. . .
If dropout is unrelated to the outcome, the obtained sample can be considered as
a random sub-sample, which is still a random sample from the original population
Related dropout:
Best patients most likely to drop out:
Over-pessimistic
Worst patients most likely to drop out:
Over-optimistic
Best patients most likely to drop out, but dropout rate dependent on
treatment:
Biased estimation of treatment eect
Part III
Generalized Linear Mixed Models
Chapter 13
The toenail data
Example
Logistic regression
A logistic mixed model
Analysis in SAS
13.1 Example
Toenail Dermatophyte Onychomycosis: Common toenail infection, dicult to
treat, aecting more than 2% of population.
Classical treatments with antifungal compounds need to be administered until the
whole nail has grown out healthy.
New compounds have been developed which reduce treatment to 3 months
Randomized, double-blind, parallel group, multicenter study for the comparison of
two such new compounds (A and B) for oral treatment.
The multicenter nature will be ignored here. An example will be given later, in the
context of the Leuven Diabetes Project.
Research question:
Severity relative to treatment of TDO ?
2 189 patients randomized, 36 centers
Focus here on patients for which the target nail was one of the big toenails
= 150 and 148 patients only
48 weeks of total follow up (12 months)
12 weeks of treatment (3 months)
measurements at months 0, 1, 2, 3, 6, 9, 12.
Frequencies at each visit:
Complication: Dropout (24%)
# Observations
Time (months) Treatment A Treatment B Total
0 150 148 298
1 149 142 291
2 146 138 284
3 140 131 271
6 131 124 255
9 120 109 229
12 118 108 226
The toenail data set is an example of a longitudinal study, with unbalanced binary
data
13.2 Logistic regression
As in earlier examples, the toenail data are clustered within study participants
As before, let Y
ij
denote the jth measurement taken on the ith patient
Ignoring the clustering, a typical analysis for studying the relation between Y
ij
and
some known covariates such as time and treatment would be based on logistic
regression.
We then assume a Bernoulli distribution: Y
ij
Bernoulli(
ij
)

ij
is the probability for outcome Y
ij
to be a success, i.e.,
ij
= P(Y
ij
= 1).
A logistic relation is assumed between
ij
and the covariates:
logit(
ij
) = log
_
_
_
_
ij
1
ij
_
_
_
_ =
0
+
1
T
i
+
2
t
ij
+
3
T
i
t
ij
Notation:
T
i
: treatment indicator for subject i
t
ij
: time point at which jth measurement is taken for ith subject
More complex models can be considered as well (e.g. including polynomial time
eects, including covariates, . . . ).
In SAS, the model can be tted as follows:
proc genmod data=toenail descending;
model y = treatn time treatn*time / dist=binomial ;
run;
Selected output:
The GENMOD Procedure
Model Information
Data Set WORK.TOENAIL
Distribution Binomial
Link Function Logit
Dependent Variable Y
Observations Used 1908
The GENMOD Procedure
Response Profile
Ordered Total
Value Y Frequency
1 1 408
2 0 1500
PROC GENMOD is modeling the probability that Y=1.
Criteria For Assessing Goodness Of Fit
Criterion DF Value Value/DF
Deviance 1904 1811.8260 0.9516
Scaled Deviance 1904 1811.8260 0.9516
Pearson Chi-Square 1904 1995.2107 1.0479
Scaled Pearson X2 1904 1995.2107 1.0479
Log Likelihood -905.9130
Algorithm converged.
Analysis Of Parameter Estimates
Standard Wald 95% Chi-
Parameter DF Estimate Error Confidence Limits Square Pr > ChiSq
Intercept 1 -0.5571 0.1090 -0.7708 -0.3433 26.10 <.0001
treatn 1 0.0240 0.1565 -0.2827 0.3307 0.02 0.8780
time 1 -0.1769 0.0246 -0.2251 -0.1288 51.91 <.0001
treatn*time 1 -0.0783 0.0394 -0.1556 -0.0010 3.95 0.0470
Scale 0 1.0000 0.0000 1.0000 1.0000
NOTE: The scale parameter was held fixed.
We nd a signicant dierence in evolution between the two treatment groups
(p = 0.0470), where group 1 shows more improvement than treatment group 0.
13.3 A logistic mixed model
As for the linear models, the clustering can be accounted for by including random
eects
We then explicitly model the belief that not all clusters satisfy the same model
with the same parameter values, but intercepts and/or slopes are allowed to be
cluster specic.
For example, a logistic random-intercepts model is obtained as:
Y
ij
Bernoulli(
ij
)
logit(
ij
) = log
_
_
_
_
ij
1
ij
_
_
_
_ =
0
+ b
i
+
1
T
i
+
2
t
ij
+
3
T
i
t
ij
As before, the b
i
are assumed to be normally distributed: b
i
N(0,
2
b
)
A logistic model with random intercepts and random slopes is obtained as:
Y
ij
Bernoulli(
ij
)
logit(
ij
) = log
_
_
_
_
ij
1
ij
_
_
_
_ = (
0
+ b
1i
) +
1
T
i
+ (
2
+ b
2i
)t
ij
+
3
T
i
t
ij
As before, the random eects are assumed to follow a bivariate normal distribution
with mean zero:
b
i
= (b
1i
, b
2i
)
N(0, D)
The logistic mixed model is an example of a generalized linear mixed model
(GLMM)
A number of estimation methods is available for tting GLMMs:
Laplace approximation
Marginal quasi-likelihood (MQL)
Penalized quasi-likelihood (PQL)
(Adaptive) Gaussian quadrature
. . .
Dierent estimation methods can lead to (strong) dierences in the results
In this course, PQL will be used.
Next to the estimation of the xed eects ( parameters) and variance
components (elements in D), empirical Bayes (EB) estimates for the random
eects b
i
can be calculated as well.
13.4 Analysis in SAS
Logistic mixed models can be tted within the GLIMMIX procedure.
Up to SAS version 9.1, the GLIMMIX procedure is not part of the standard SAS
package. It can be downloaded from the SAS website. Once installed, it remains
available for future SAS sessions.
http://www.sas.com/apps/demosdownloads/setupcat.jsp?cat=SAS%2FSTAT+Software
We consider the random-intercepts model:
Y
ij
Bernoulli(
ij
), log
_
_
_
_
ij
1
ij
_
_
_
_ =
0
+ b
i
+
1
T
i
+
2
t
ij
+
3
T
i
t
ij
The model specication in GLIMMIX is very similar to the way linear mixed
models were specied in the MIXED procedure.
SAS program for the random-intercepts model:
proc glimmix data=test;
class idnum;
model onyresp (event=1) = treatn time treatn*time / dist=binary solution;
random intercept / subject=idnum;
run;
Selected output:
Response Profile
Ordered Total
Value onyresp Frequency
1 0 1500
2 1 408
The GLIMMIX procedure is modeling the probability that onyresp=1.
Standard
Cov Parm Subject Estimate Error
Intercept idnum 4.7116 0.6031
Solutions for Fixed Effects
Standard
Intercept -0.7239 0.2370 292 -3.05 0.0025
treatn 0.000918 0.3363 1612 0.00 0.9978
time -0.2883 0.03349 1612 -8.61 <.0001
treatn*time -0.1106 0.05366 1612 -2.06 0.0395
Type III Tests of Fixed Effects
Num Den
treatn 1 1612 0.00 0.9978
time 1 1612 74.10 <.0001
treatn*time 1 1612 4.25 0.0395
Ignoring the clustering, the dierence in slopes between the two treatment groups
was estimated as 0.0783 (p = 0.0470).
Under the random-intercepts model this becomes 0.1106 (p = 0.0395).
The variance between subjects is estimated as

2
b
= 4.7116
Our model assumes dierent xed intercepts and slopes for both groups.
Direct estimation of these can be done based on the following reparameterization
of the same model:
Y
ij
Bernoulli(
ij
), log
_
_
_
_
ij
1
ij
_
_
_
_ =
_
1
+ b
i
+
2
t
ij
, Treatment A
3
+ b
i
+
4
t
ij
, Treatment B
The corresponding SAS code becomes:
class idnum treatn;
model onyresp (event=1) = treatn treatn*time / noint dist=binary solution;
random intercept / subject=idnum;
estimate difference slopes treatn*time 1 -1;
run;
The ESTIMATE statement is used to estimate and test the dierence between the
slopes
2
and
4
of both treatment groups.
Selected output:
Standard
Intercept idnum 4.7116 0.6031
Num Den
treatn 2 1612 9.26 0.0001
time*treatn 2 1612 82.30 <.0001
Standard
Effect treatn Estimate Error DF t Value Pr > |t|
treatn 0 -0.7239 0.2370 1612 -3.05 0.0023
treatn 1 -0.7230 0.2386 1612 -3.03 0.0025
time*treatn 0 -0.2883 0.03349 1612 -8.61 <.0001
time*treatn 1 -0.3989 0.04193 1612 -9.51 <.0001
Estimates
Standard
difference slopes 0.1106 0.05366 1612 2.06 0.0395
Note again that the standard reported F-tests test whether both intercepts and
both slopes are equal to zero, respectively.
Summary of model t:
Eect Parameter Estimate (s.e.)
Intercept group A
1
0.7239 (0.2370)
Intercept group B
3
0.7230 (0.2386)
Slope group A
2
0.2883 (0.0335)
Slope group B
4
0.3989 (0.0419)
Variance random intercepts
2
b
4.7116 (0.6031)
As an example, a logistic mixed model with uncorrelated random intercepts and
slopes can be tted with the following SAS code:
proc glimmix data=test ;
class idnum treatn;
model onyresp (event=1) = treatn treatn*time / noint dist=binary solution;
random intercept timetrans / type=un(1) subject=idnum;
estimate difference slopes treatn*time 1 -1;
run;
The option type=un(1) species that the covariance matrix D should be
diagonal.
Selected output:
Cov Standard
Parm Subject Estimate Error
UN(1,1) idnum 4.6285 0.6143
UN(2,1) idnum 0 .
UN(2,2) idnum 0.0747 0.0184
Standard
Effect treatn Estimate Error DF t Value Pr > |t|
treatn 0 -0.6653 0.2375 1325 -2.80 0.0052
treatn 1 -0.6144 0.2396 1325 -2.56 0.0105
time*treatn 0 -0.3528 0.04997 1325 -7.06 <.0001
time*treatn 1 -0.4983 0.05952 1325 -8.37 <.0001
Estimates
Standard
difference slopes 0.1455 0.07771 1325 1.87 0.0615
Under the random-intercepts model, the dierence in slopes between the two
treatment groups was estimated as 0.1106 (p = 0.0395).
Under the model with uncorrelated random intercepts and slopes, this becomes
0.1455 (p = 0.0615).
Note that the estimation of the xed eects is also aected by including the
random slopes:
Random Random
intercepts intercepts & slopes
Eect Parameter Estimate (s.e.) Estimate (s.e.)
Intercept group A
1
0.7239 (0.2370) 0.6653 (0.2375)
Intercept group B
3
0.7230 (0.2386) 0.6144 (0.2396)
Slope group A
2
0.2883 (0.0335) 0.3528 (0.0450)
Slope group B
4
0.3989 (0.0419) 0.4983 (0.0596)
Variance random intercepts d
11
4.7116 (0.6031) 4.6285 (0.6143)
Variance random slopes d
22
0.0747 (0.0184)
Chapter 14
The Leuven diabetes project
Example
A three-level logistic mixed regression model
Analysis in SAS
14.1 Example
Linear mixed models were used earlier to study the evolution of HbA1c in DPL
participants, correcting for the clustered nature of the data:
within general practioners (GPs)
within subjects
A related outcome of scientic interest is whether the GP is able to keep the
HbA1c level under control, i.e., to keep it below 7%
Hence, the derived outcome of interest is dened as:
Y =
_
_
1 if HbA1c < 7%
0 if HbA1c 7%
14.2 A three-level logistic mixed regression model
Let Y
ijk
be the kth binary outcome measure for patient j of GP i
Ignoring potential important covariates, a model which accounts for the clustering
of the outcomes within patients and GPs is a three-level logistic mixed model:
Y
ijk
Bernoulli(
ijk
)
logit(
ijk
) = log
_
_

ijk
1
ijk
_
_
=
0
+
1
t
k
+ a
i
+ b
j(i)
,
a
i
N(0,
2
GP
), b
j(i)
N(0,
2
PAT
)
The GP eects b
j(i)
represent the fact that some GPs are more succesful in
controling the HbA1c level of their patients than others.
The patient eects a
i
represent the fact that controling the HbA1c level is not
equally easy for all patients
The SAS code to t the three-level logistic mixed model equals:
proc glimmix data=dpla;
model target (event=1) = time / dist=binary solution;
run;
The opion solution is added to the RANDOM statement to request calculation of
the EB estimates
As in the MIXED procedure, ODS statements can be used to save the EB
estimates into an output data set, rather than print them in the output screen.
Selected SAS output:
Standard
Intercept mdnr 0.1399 0.05275
Intercept patientnr(mdnr) 1.1154 0.1308
Num Den
time 1 1395 59.07 <.0001
Standard
Intercept 0.1662 0.07960 60 2.09 0.0410
time 0.6240 0.08119 1395 7.69 <.0001
As for the continuous outcome, we observe far more variability between patients
than between GPs;
Between-GP variability:

2
GP
= 0.1399
Between-patient variability:

2
PAT
= 1.1154
Overall, the probability of reaching the target HbA1c (< 7%) increases over time
(p < 0.0001)
Histograms of both sets of EB estimates:
The histogram of EB estimates

b
j(i)
of patient eects suggests three clusters of
patients, with approximate cut-os for

b
j(i)
equal to 0.6 and 0.1
These clusters reect the possible patient-specic proles:
Patients with Y
0
= Y
1
= 0 are expected to have small predicted probabilities
for reaching the target. Their prediction

b
j(i)
is expected to be very small
(negative).
Patients with Y
0
= Y
1
= 1 are expected to have large predicted probabilities for
reaching the target. Their prediction

b
j(i)
is expected to be very large
(positive).
Patients who change Y
0
= 0 Y
1
= 1 or Y
0
= 1 Y
1
= 0 are expected to
have intermediate predicted probabilities for reaching the target. Their
prediction

b
j(i)
is expected to be of a moderate level.
This can be quantied in the following cross-classication:
Y prole

b
j(i)
< 0.6 0.6

b
j(i)
< 0.1 0.1

b
j(i)
0 0 345 0 0
0 1 0 275 0
1 1 0 0 677
All patients with HbA1c at target at the start of the study have their HbA1c at
target one year later as well.
Scatterplot of patient eects versus GP eects:
For each GP, we observe at most 7 dierent values for the EB estimates for the
patients treated by that GP.
These 7 values correspond to the 7 dierent response proles that can be
observed: 0 0, 0 1, 1 1, 0 , 1 , 0, and 1.
The negative trends observed in the scatterplot are also a side eect of the
discrete nature of the outcomes.
Consider two patients, j
1
and j
2
, treated by dierent GPs, i
1
and i
2
, with the
same response prole, e.g., 1 1
Their subject-specic models are given by:
logit(
i
1
j
1
k
) =
0
+
1
t
k
+ a
i
1
+ b
j
1
(i
1
)
, for patient j
1
logit(
i
2
j
2
k
) =
0
+
1
t
k
+ a
i
2
+ b
j
2
(i
2
)
, for patient j
2
Since both patients have the same data, we expect their predicted probabilities to
be the same at all time points, implying
a
i
1
+ b
j
1
(i
1
)
= a
i
2
+ b
j
2
(i
2
)
Hence, we expect the sum a
i
+ b
j(i)
of GP and patient eects to be constant,
explaining the strong negative relation between the estimates

a
i
and

b
j(i)
.
Chapter 15
The Epilepsy data
Example
Poisson regression
A Poisson mixed model
Analysis in SAS
15.1 Example
Randomized, double-blind, parallel group multi-center study for the comparison of
placebo with a new anti-epileptic drug (AED), in combination with one or two
other (standard) AEDs.
Randomization after a 12-week stabilization period.
45 patients in placebo group, 44 in active (new) treatment group
Double-blind weekly measurements during 16 weeks.
Afterwards, patients enter a long-term open-extension study, with some patients
followed for up to 27 weeks
The outcome of interest is the number of epileptic seizures experienced during the
last week, i.e., since the last time the outcome was measured.
Number of observations and histogram of the weekly outcome measurements:
# Observations
Week Placebo Treatment Total
1 45 44 89
5 42 42 84
10 41 40 81
15 40 38 78
16 40 37 77
17 18 17 35
20 2 8 10
27 0 3 3
Average and median proles for both treatments:
Unstable behavior due to extreme values and few observations past week 20.
15.2 Poisson regression
As in earlier examples, the data are clustered within study participants
Ignoring the clustering, a typical analysis for studying the relation between a count
outcome Y
ij
and some known covariates such as time and treatment would consist
of Poisson regression.
We then assume a Poisson distribution: Y
ij
Poisson(
ij
)
The parameter
ij
is the expected (average) count, i.e.,
ij
= E(Y
ij
).
A logarithmic relation is assumed between
ij
and the covariates:
log(
ij
) =
_
1
+
2
t
ij
if placebo (group 0)
3
+
4
t
ij
if treated (group 1).
More complex models can be considered as well (e.g. including polynomial time
eects, including covariates, . . . ).
In SAS, the model can be tted as follows:
proc genmod data=test;
class trt;
model nseizw = trt trt*time / noint dist=poisson ;
estimate slope difference trt*time 1 -1 ;
run;
The ESTIMATE statement has been added to estimate the dierence between the
two slopes.
Selected output:
Analysis Of Parameter Estimates
Standard Wald 95% Confidence Chi-
Parameter DF Estimate Error Limits Square Pr > ChiSq
Intercept 0 0.0000 0.0000 0.0000 0.0000 . .
trt 0 1 1.2662 0.0424 1.1832 1.3493 892.71 <.0001
trt 1 1 1.4531 0.0383 1.3781 1.5282 1439.57 <.0001
time*trt 0 1 -0.0133 0.0043 -0.0218 -0.0048 9.48 0.0021
time*trt 1 1 -0.0328 0.0038 -0.0403 -0.0253 73.86 <.0001
Scale 0 1.0000 0.0000 1.0000 1.0000
Contrast Estimate Results
Standard Chi-
Label Estimate Error Alpha Confidence Limits Square Pr > ChiSq
slope difference 0.0195 0.0058 0.05 0.0081 0.0308 11.34 0.0008
We nd a signicant dierence in the evolution in the two treatment groups
(p = 0.0008), where group 1 shows more improvement than treatment group 0.
15.3 A Poisson mixed model
Correction for the clustered nature of the data can again be based on the inclusion
of random eects which model the within-patient correlation.
For example, consider the random-intercepts model:
Y
ij
Poisson(
ij
), log(
ij
) =
_
1
+ b
i
+
2
t
ij
3
+ b
i
+
4
t
ij
As before, the subject-specic intercepts b
i
are assumed to follow a normal
distribution N(0,
2
b
).
Other random eects (slopes) can be introduced as well.
SAS code for the random-intercepts model:
class id trt;
model nseizw = trt trt*time / noint dist=poisson solution;
random intercept / subject=id;
run;
Selected SAS output:
Standard
Intercept id 1.1462 0.1835
Num Den
trt 2 1328 20.64 <.0001
time*trt 2 1328 9.14 0.0001
Standard
Effect trt Estimate Error DF t Value Pr > |t|
trt 0 0.8413 0.1668 1328 5.04 <.0001
trt 1 0.6732 0.1692 1328 3.98 <.0001
time*trt 0 -0.01430 0.004404 1328 -3.25 0.0012
time*trt 1 -0.01200 0.004317 1328 -2.78 0.0055
Estimates
Standard
slope difference -0.00230 0.006167 1328 -0.37 0.7094
In contrast to the signicant interaction obtained before, ignoring the longitudinal
nature of the data (p = 0.0008), we no longer nd a dierence between the two
slopes
2
and
4
(p = 0.7094).
The between-patient variability is estimated to be

2
b
= 1.1462.
Summary of model t:
Eect Parameter Estimate (s.e.)
Intercept Placebo
1
0.8413 (0.1668)
Intercept Active
3
0.6732 (0.1692)
Slope Placebo
2
0.0143 (0.0044)
Slope Active
4
0.0120 (0.0043)
Variance random intercepts
2
b
1.1462 (0.1835)
A mixed model with subject-specic intercepts as well as time eects would be:
Y
ij
Poisson(
ij
), log(
ij
) =
_
1
+ b
1i
+ (
2
+ b
2i
)t
ij
3
+ b
1i
+ (
4
+ b
2i
)t
ij
As before, the random eects are assumed to follow a bivariate normal distribution
with mean zero:
b
i
= (b
1i
, b
2i
)
N(0, D)
New SAS code:
class id trt;
model nseizw = trt trt*time / noint dist=poisson solution;
random intercept time / type=un subject=id solution;
Selected output:
Cov Standard
Parm Subject Estimate Error
UN(1,1) id 1.2577 0.2173
UN(2,1) id -0.01891 0.008784
UN(2,2) id 0.002419 0.000565
Num Den
trt 2 1241 20.86 <.0001
time*trt 2 1241 5.08 0.0064
Standard
Effect trt Estimate Error DF t Value Pr > |t|
trt 0 0.9251 0.1768 1241 5.23 <.0001
trt 1 0.6844 0.1807 1241 3.79 0.0002
time*trt 0 -0.02687 0.009787 1241 -2.75 0.0061
time*trt 1 -0.01616 0.009976 1241 -1.62 0.1056
Estimates
Standard
slope difference -0.01071 0.01397 1241 -0.77 0.4436
Under the random-intercepts model, the dierence in slopes between the two
treatment groups was estimated as 0.0023 (p = 0.7094).
Under the current model, this becomes 0.0107 (p = 0.4436).
Note that the estimation of the xed eects is also aected by including the
random slopes:
Random Random
intercepts intercepts & slopes
Eect Parameter Estimate (s.e.) Estimate (s.e.)
Intercept Placebo
1
0.8413 (0.1668) 0.9251 (0.1768)
Intercept Active
3
0.6732 (0.1692) 0.6844 (0.1807)
Slope Placebo
2
0.0143 (0.0044) 0.0269 (0.0098)
Slope Active
4
0.0120 (0.0043) 0.0162 (0.0100)
Variance random intercepts d
11
1.1462 (0.1835) 1.2577 (0.2173)
Covariance random intercepts & slopes d
12
0.0189 (0.0088)
Variance random slopes d
22
0.0024 (0.0006)
Scatterplot of estimated subject-specic intercepts and slopes:
Chapter 16
Parameter interpretation in the GLMM
Marginalizing the mixed model: The toenail data
Marginalizing the mixed model: The epilepsy data
16.1 Parameter interpretation in the GLMM
Let us re-consider one of the linear mixed models, used before for the model
growth curve data:
Y
ij
|b
i

_
_
N(
1
+ b
i
+
2
t
j
,
2
res
), if short mother
N(
3
+ b
i
+
4
t
j
,
2
res
), if medium mother
N(
5
+ b
i
+
6
t
j
,
2
res
), if tall mother
This hierchical model implied a very specic marginal model, with mean:
E(Y
ij
) =
_
1
+
2
t
j
, if short mother
3
+
4
t
j
, if medium mother
5
+
6
t
j
, if tall mother
Hence, the xed eects have a subject-specic interpretation as well as a
population-average interpretation.
Let us now consider the logistic random-intercepts model
Y
ij
Bernoulli(
ij
), log
_
_
_
_
ij
1
ij
_
_
_
_ =
0
+ b
i
+
1
t
ij
Equivalently, we have
E(Y
ij
|b
i
) =
ij
=
exp(
0
+ b
i
+
1
t)
1 + exp(
0
+ b
i
+
1
t)
The above model assumes a logistic evolution of the success probability of each
patients, all curves having the same slope
1
, but dierent intercepts
0
+ b
i
.
Graphically:
The average subject, i.e., the subject with intercept b
i
= 0, has success probability
given by
E(Y
ij
|b
i
= 0) =
exp(
0
+ 0 +
1
t)
1 + exp(
0
+ 0 +
1
t)
The marginal population-average evolution is obtained from averaging over the
random eects:
E(Y
ij
) = E [E(Y
ij
|b
i
)] = E
_
_
exp(
0
+ b
i
+
1
t)
1 + exp(
0
+ b
i
+
1
t)
_
_ =
exp(
0
+ 0 +
1
t)
1 + exp(
0
+ 0 +
1
t)
Conclusion:
Average evolution = Evolution average subject
Parameters in the mixed model have a subject-specic interpretation, not a
population-averaged one.
The problem arises from the fact that, E[g(Y )] = g[E(Y )], unless for linear
functions, such as in the case of linear mixed models:
Conditional mean: E(Y
i
|b
i
) = X
i
+ Z
i
b
i
Average subject: E(Y
i
|b
i
= 0) = X
i
Marginal mean: E(Y

i
) = X
i

Calculation of the marginal average population requires computation of
E(Y
ij
) = E [E(Y
ij
|b
i
)] = E
_
_
exp(
0
+ b
i
+
1
t)
1 + exp(
0
+ b
i
+
1
t)
_
_
=
_
exp(
0
+ b
i
+
1
t)
1 + exp(
0
+ b
i
+
1
t)
f(b
i
) db
i
This can be done using numerical integration methods, or using sampling based
averaging.
Note that what has been explained here in the context of logistic mixed models
equally well applies to every other generalized linear or non-linear model.
16.2 Marginalizing the mixed model: The toenail data
We re-consider the logistic mixed model with random intercepts.
The tted model is given by:
Y
ij
Bernoulli(
ij
),
ij
=
_
_
exp(0.7239+b
i
0.2883t
ij
)
1+exp(0.7239+b
i
0.2883t
ij
)
, Treatment A
exp(0.7230+b
i
0.3989t
ij
)
1+exp(0.7230+b
i
0.3989t
ij
)
, Treatment B
The random eects b
i
are normally distributed with mean 0 and variance
2
b
= 4.7116.
The marginal evolution in both groups is obtained from integrating over the
random eects b
i
N(0, 4.7116):
ij
=
_
_
_
exp(0.7239+b
i
0.2883t
ij
)
1+exp(0.7239+b
i
0.2883t
ij
)
f(b
i
) db
i
, Treatment A
_
exp(0.7230+b
i
0.3989t
ij
)
1+exp(0.7230+b
i
0.3989t
ij
)
f(b
i
) db
i
, Treatment B
SAS code:
data h;
do treat=0 to 1 by 1;
do subject=1 to 1000 by 1;
b=sqrt(4.7116)*rannor(-1) ;
do t=0 to 12 by 0.1;
if treat=0 then y=exp(-0.7239 + b -0.2883*t)/(1+ exp(-0.7239 + b -0.2883*t));
else y=exp(-0.7230 + b -0.3989*t)/(1+ exp(-0.7230 + b -0.3989*t));
output;
end;
end;
end;
proc sort data=h;
by t treat;
run;
proc means data=h;
var y;
by t treat;
output out=out;
run;
dm "dlgprtsetup orient=L nodisplay";
filename fig c:/filename.eps;
goptions reset=all interpol=join ftext=swiss device=pslepsfc
gsfname=fig gsfmode=replace targetdevice=winprtc;
proc gplot data=out;
plot y*t=treat / haxis=axis1 vaxis=axis2 legend=legend1;
axis1 label=(h=2 Time) value=(h=1.5) order=(0 to 14 by 1) minor=none;
axis2 label=(h=2 A=90 P(Y=1)) value=(h=1.5) order=(0 to 0.4 by 0.1) minor=none;
legend1 label=(h=1.5 Treatment: ) value=(h=1.5 A B);
title h=2.5 Marginal average evolutions (GLMM);
symbol1 c=red i=join w=20 l=1 mode=include;
symbol2 c=blue i=join w=20 l=1 mode=include;
where _stat_=MEAN;
run;quit;run;
Result:
The evolution of average subjects, i.e., subjects with b
i
= 0 is given by:
ij
=
_
_
exp(0.7239+00.2883t
ij
)
1+exp(0.7239+00.2883t
ij
)
, Treatment A
exp(0.7230+00.3989t
ij
)
1+exp(0.7230+00.3989t
ij
)
, Treatment B
16.3 Marginalizing the mixed model: The epilepsy data
We re-consider the tted Poisson mixed model with random intercepts and slopes:
Y
ij
Poisson(
ij
)
ij
=
_
_
exp[0.9251 + b
1i
+ (0.0269 + b
2i
)t
ij
] if placebo (group 0)
exp[0.6844 + b
1i
+ (0.0162 + b
2i
)t
ij
] if treated (group 1).
The random-eects vector b
i
= (b
1i
, b
2i
)
is N(0, D) distributed, with tted D:
D =
_
_
_
_
_
_
_
_
1.2577 0.0189
0.0189 0.0024
_
_
_
_
_
_
_
_
.
The non-linear link function again implies that the parameters only have a
subject-specic interpretation.
Subject-specic proles for 20 randomly selected subjects, together with their
average evolution:
The marginal average evolutions are obtained from integrating over the random
eects b
i
:
E(Y
ij
) = E [E(Y
ij
|b
i
)]
=
_
_
_ _
exp[0.9251 + b
1i
+ (0.0269 + b
2i
)t
ij
] f(b
1i
, b
2i
) db
1i
db
2i
_ _
exp[0.6844 + b
1i
+ (0.0162 + b
2i
)t
ij
] f(b
1i
, b
2i
) db
1i
db
2i
As before, the integration can be approximated using sampling based averaging,
which requires generating multivariate random vectors b
i
N(0,

D)
Most software packages only allow generating univariate standard normals, which
can then be transformed using the cholesky decomposition of the covariance

D
The cholesky decomposition L of

D is the upper triangular matrix such that
L
L =

D
In SAS, L can easily be calculated using the IML procedure:
proc iml;
d={1.2577 -0.0189 , -0.0189 0.0024};
l=root(d);
print d; print l;
quit;
Output:
D L
1.2577 -0.0189 1.1214722 -0.016853
-0.0189 0.0024 0 0.0459998
Let b
1i
and b
2i
be independent and standard normal distributed, and
b
i
= (b
1i
, b
2i
)

We then have that
b
i
L
b
i
=
_
_
_
_
_
_
_
_
1.1215 b
1i
+ 0 b
2i
0.0169 b
1i
+ 0.0460 b
2i
_
_
_
_
_
_
_
_
N(0, L
IL) = N(0,

D)
The SAS code for sampling based averaging:
data h;
do treat=0 to 1 by 1;
do subject=1 to 1000 by 1;
b1=rannor(-1);
b2=rannor(-1);
ranint=1.1215*b1;
ranslope=-0.0169*b1 + 0.0460*b2;
do t=0 to 27 by 0.1;
if treat=0 then y=exp(0.9251+ranint +(-0.0269+ranslope)*t);
else y=exp(0.6844+ranint +(-0.0162+ranslope)*t);
output;
end;
end;
end;
proc sort data=h;
by t treat;
run;
proc means data=h;
var y;
by t treat;
output out=out;
run;
dm "dlgprtsetup orient=L nodisplay";
filename fig c:/filename.eps;
goptions reset=all interpol=join ftext=swiss device=pslepsfc
gsfname=fig gsfmode=replace targetdevice=winprtc ;
proc gplot data=out;
plot y*t=treat / haxis=axis1 vaxis=axis2 legend=legend1;
axis1 label=(h=2.5 Time (weeks)) value=(h=1.5) order=(0 to 25 by 5) minor=none;
axis2 label=(h=2.5 A=90 E(Y)) value=(h=1.5) order=(0 to 6 by 1) minor=none;
legend1 label=(h=1.5 Treatment: ) value=(h=1.5 Placebo Treated);
title h=3 Marginal average evolutions (GLMM);
symbol1 c=red i=join w=20 l=1 mode=include;
symbol2 c=blue i=join w=20 l=1 mode=include;
where _stat_=MEAN;
run;quit;run;
Result:
The evolution of average subjects, i.e., subjects with b
i
= 0 is given by:
ij
=
_
_
exp[0.9251 + 0 + (0.0269 + 0)t
ij
] if placebo (group 0)
exp[0.6844 + 0 + (0.0162 + 0)t
ij
] if treated (group 1).
Part IV
Non-linear Mixed Models
Chapter 17
Introduction
Linear and generalized linear mixed models revisited
Non-linear mixed models
17.1 Linear and generalized linear mixed models revisited
In linear mixed models, the mean is modeled as a linear function of regression
parameters and random eects. For example,
E(Y
ij
|b
1i
, b
2i
) = (
1
+ b
1i
) + (
2
+ b
2i
)t
ij
+
ij
In generalized linear mixed models, apart from a link function, the mean is again
modeled as a linear function of regression parameters and random eects:
Binary data, for example
E(Y
ij
|b
1i
, b
2i
) =
exp[(
1
+ b
1i
) + (
2
+ b
2i
)t
ij
]
1 + exp[(
1
+ b
1i
) + (
2
+ b
2i
)t
ij
]
Count data, for example
E(Y
ij
|b
1i
, b
2i
) = exp[(
1
+ b
1i
) + (
2
+ b
2i
)t
ij
]
In some applications, models are needed, in which the mean is no longer modeled
as a function of a linear predictor.
These are called non-linear mixed models.
17.2 Non-linear mixed models
In non-linear mixed models, it is assumed that the conditional mean of Y
ij
, given a
vector b
i
of random eects is modeled as:
E(Y
ij
|b
i
) = h(x
ij
, , b
i
)
The vector x
ij
contains known covariates
The vectors and b
i
contain xed and random eects, respectively
As before, the random eects are assumed to be normally distributed, with mean
0 and covariance D.
Non-linear mixed models can be tted within the SAS procedure NLMIXED.
Chapter 18
The orange trees
Introduction
Analysis in SAS
18.1 Introduction
We consider an experiment in which the trunk circumference (in mm) is measured
for 5 orange trees, on 7 dierent occasions.
Data:
Response
Day Tree 1 Tree 2 Tree 3 Tree 4 Tree 5
118 30 33 30 32 30
484 58 69 51 62 49
664 87 111 75 112 81
1004 115 156 108 167 125
1231 120 172 115 179 142
1372 142 203 139 209 174
1582 145 203 140 214 177
Individual proles:
The following non-linear mixed model has been proposed in the statistical
literature:
Y
ij
=

1
+ b
i
1 + exp[(t
ij

2
)/
3
]
+
ij
, b
i
N(0,
2
b
),
ij
N(0,
2
)
In SAS PROC NLMIXED, the model can be tted using either of the following
equivalent programs:
proc nlmixed data=tree;
parms beta1=190 beta2=700 beta3=350 sigmab=10 sigma=10;
num = b + beta1;
ex = exp(-(day-beta2)/beta3);
den = 1 + ex;
model y ~ normal(num/den,sigma**2);
random b ~ normal(0,sigmab**2) subject=tree;
run;
num = b;
ex = exp(-(day-beta2)/beta3);
den = 1 + ex;
model y ~ normal(num/den,sigma**2);
random b ~ normal(beta1,sigmab**2) subject=tree;
run;
Selected output:
Parameter Estimates
Standard
Parameter Estimate Error DF t Value Pr > |t| Alpha Lower Upper Gradient
beta1 192.05 15.6577 4 12.27 0.0003 0.05 148.58 235.53 -6.8E-8
beta2 727.91 35.2487 4 20.65 <.0001 0.05 630.04 825.77 1.383E-8
beta3 348.07 27.0798 4 12.85 0.0002 0.05 272.89 423.26 3.375E-8
sigmab 31.6463 10.2614 4 3.08 0.0368 0.05 3.1560 60.1366 -3.4E-7
sigma 7.8430 1.0125 4 7.75 0.0015 0.05 5.0318 10.6542 -1.92E-6
Empirical Bayes estimates, and subject-specic predictions can be obtained as
follows:
num = b + beta1;
den = 1 + exp(-(day-beta2)/beta3);
ratio = num/den;den = 1 + ex;
model y ~ normal(ratio,sigma**2);
random b ~ normal(0,sigmab**2) subject=tree out=eb;
predict ratio out=ratio;
run;
We can now compare the observed data to the subject-specic predictions
y
ij
=
1
+

b
i
1 + exp[(t
ij

2
)/
3
]
Chapter 19
The Theophylline data
Introduction
Analysis in SAS
19.1 Introduction
Pharmacokinetics (PK) is the study of the time course of a drug concentration
in the body, i.e., what the body does to the drug.
Pharmacodynamics (PD) is the study of the relationship of the drug
concentration and pharmacologic eects, i.e., what a drug does to the body.
We consider the PK study in which longitudinally measured blood concentrations
of the anti-asthmatic, orally administered, agent Theophylline are studied
12 subjects, dose at t = 0
Blood samples at 10 time points over the following 25 hours
Outcome of interest: Theophylline concentration
The blood concentration depends on:
Absorption: the process of a substance entering the body
Elimination: the process of a substance being removed from the body
Clearance: the volume of blood cleared of drug, in the kidneys, per unit time
In the literature, a one-compartment open model with rst-order absorption and
elimination has been proposed (t > 0):
Y
ij
= C
i
(t
ij
) =
k
ai
k
ei
d
i
C
i
(k
ai
k
ei
)
[exp(k
ei
t
ij
) exp(k
ai
t
ij
)] +
ij
k
ai
: fractional absorption rate for subject i
k
ei
: fractional elimination rate for subject i
C
i
: clearance for subject i
In order to restrict k
ai
, k
ei
, and C
i
to be positive, the model is re-parameterized
as:
C
i
= exp(
1
+ b
1i
),
k
a,i
= exp(
2
+ b
2i
),
k
e,i
= exp(
3
+ b
3i
).
b
1i
, b
2i
, and b
3i
are assumed multivariate normal with mean 0 and covariance D
NLMIXED code:
proc nlmixed data=theoph;
parms beta1=-3.22 beta2=0.47 beta3=-2.45
d11=0.03 d12=0 d22=0.4 d13=0 d23=0 d33=0.03 s2=0.5;
cl = exp(beta1 + b1);
ka = exp(beta2 + b2);
ke = exp(beta3 + b3);
pred = dose*ke*ka*(exp(-ke*time)-exp(-ka*time))/cl/(ka-ke);
model conc ~ normal(pred,s2);
random b1 b2 b3 ~ normal([0,0,0],[d11,d12,d22,d13,d23,d33]) subject=subject;
predict pred out=theopred; run;
Note that very accurate starting values are needed for the various parameters.
Otherwise the numerical optimization procedure does not reach convergence
Starting values can be obtained from tting non-linear regression models to all
subjects separately, from tting simplied mixed models, or from trying several
optimization algorithms.
Results:
Parameter Estimate (s.e.)
Fixed eects:
1
(Cl) -3.277 (0.046)
2
(k
a
) 0.537 (0.063)
3
(k
e
) -2.454 (0.064)
Parameter Estimate (s.e.)
Residual variance:
2
0.623 (0.083)
Random-eect (co-)variances:
d
11
0.057 (0.022)
d
12
-0.012 (0.018)
d
22
0.264 (0.054)
d
13
0.030 (0.020)
d
23
-0.025 (0.017)
d
33
0.035 (0.017)
There seems only weak evidence for correlation between the random eects
b
1i
, b
2i
, b
3i
A model with uncorrelated random eects can be tted by replacing the previous
PARMS and RANDOM statements by:
parms beta1=-3.22 beta2=0.47 beta3=-2.45 d11=0.03 d22=0.4 d33=0.03 s2=0.5;
random b1 b2 b3 ~ normal([0,0,0],[d11,0,d22,0,0,d33]) subject=subject;
The increase in approximate log-likelihood is only 0.9.
This supports our initial statement of weak correlation between the various
random eecs
As an informal check of our model t, we can compare the tted prole for each
subject with its observed data.
Chapter 20
Remarks
Marginalizing non-linear mixed models
Generalized linear mixed models in NLMIXED
Generalized non-linear mixed models in NLMIXED
20.1 Marginalizing the mixed model
It has been discussed that parameters in generalized linear mixed models have
subject-specic rather than population-average interpretations.
The same holds for non-linear mixed models.
For example, in the Theophylline analysis, the marginal average evolution equals:
E(Y
ij
) = E{E(Y
ij
|b
i
)}
= E
_
_
k
ai
k
ei
d
i
C
i
(k
ai
k
ei
)
[exp(k
ei
t
ij
) exp(k
ai
t
ij
)]
b
i
_
_
= E
_
_
exp(
2
+ b
2i
) exp(
3
+ b
3i
)d
i
exp(
1
+ b
1i
)[exp(
2
+ b
2i
) exp(
3
+ b
3i
)]
[exp(exp(
3
+ b
3i
)t
ij
) exp(exp(
2
+ b
2i
)t
ij
)]
b
1i
, b
2i
, b
3i
_
_
This requires integration over the three-dimensional random-eects distribution of
(b
1i
, b
2i
, b
3i
).
As before, this can be done using numerical integration methods, or by sample
averaging.
In some special cases, the xed eects do represent the average evolution.
For example, for the model used to describe the orange tree data, we have
E(Y
ij
) = E{E(Y
ij
|b
i
)} = E
_
1
+ b
i
1 + exp[(t
ij

2
)/
3
]
b
i
_
_
=

1
1 + exp[(t
ij

2
)/
3
]
Whenever the random eects appear in a linear way in the model, no numerical
integration methods are needed for marginalizing the non-linear mixed model.
20.2 Generalized linear mixed models in NLMIXED
Generalized linear mixed models can also be considered as non-linear mixed
models: The mean is a non-linear function of the covariates:
Binary data, for example
E(Y
ij
|b
1i
, b
2i
) =
exp[(
1
+ b
1i
) + (
2
+ b
2i
)t
ij
]
1 + exp[(
1
+ b
1i
) + (
2
+ b
2i
)t
ij
]
Count data, for example
E(Y
ij
|b
1i
, b
2i
) = exp[(
1
+ b
1i
) + (
2
+ b
2i
)t
ij
]
These are special cases as, apart from the link function, there is still a linear
predictor of the form x
ij
+ z
ij
b
i
Generalized linear mixed models can also be tted using the NLMIXED procedure.
For example, the above logistic mixed can be tted as:
proc nlmixed data=dataset;
parms beta1=-3.22 beta2=0.47 d11=0.03 d12=0 d22=0.4;
teta = beta1 + b1 + beta2*time + b2*time;
expteta = exp(teta);
p = expteta/(1+expteta);
model y ~ binary(p);
random b1 b2 ~ normal([0,0],[d11,d12,d22]) subject=subject;
For example, the above Poisson mixed can be tted as:
proc nlmixed data=dataset;
parms beta1=-3.22 beta2=0.47 d11=0.03 d12=0 d22=0.4;
teta = beta1 + b1 + beta2*time + b2*time;
lambda=exp(teta);
model y ~ poisson(lambda);
random b1 b2 ~ normal([0,0],[d11,d12,d22]) subject=subject;
20.3 Generalized non-linear mixed models in NLMIXED
The linear predictor in generalized linear mixed models can be replaced by any
function of the covariates, xed eects, and random eects.
For example, in dose-response models, the following logistic mixed model with
non-linear predictor is sometimes used:
Y
ij
Bernoulli(
ij
)
logit(
ij
) = log
_
_
_
_
ij
1
ij
_
_
_
_ =
1
+ b
i
+
2
dose
3
i
Such models can equally well be tted using the NLMIXED procedure

Mixed

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Mixed

Uploaded by

Copyright:

Available Formats

Mixed and Multilevel Models

, are not correlated:

, from same patient are correlated:

Introduction to Biostatistics 149

Marginal mean: E(Y

Introduction to Biostatistics 233

is N(0, D) distributed, with tted D:

Introduction to Biostatistics 243

You might also like