Newsom 1

USP 656 Multilevel Regression

Winter 2006
kjkgkgkgkk (i.e., 0 mg, 5 mg, an !0 mg". #owever, i$ t%e resear&%er wants to make in$eren&es
'e(on t%e )arti&ular values o$ t%e &g%$g$g$gs$gs$gs$g$$$$$$$$$$kkkkk$g%g%'le use in t%e stu(,
a ranom e$$e&ts moel is use. * &ommon e+am)le woul 'e t%e use o$ )u'li& art works
re)resenting low, moerate, an &v'&'&'&v'&v'
Fixed and Random Coefficients in Multilevel Regression
,%e ranom vs. $i+e %g$g$g$g$g$g$g$kan e$$e&ts is im)ortant in multilevel regression. -n
multilevel regression moels, 'ot% level.! &v&v'&v'v&' to var( ranoml( a&ross grou)s. /e&ause o$
t%e assum)tions a'out t%eir error istri'utions, we &all t%eir varian&es, oo=var(U0j) an 11=var(U1j),
0ranom &oe$$i&ients.1
,%is means t%at we &an t%ink a'out 0j an 1j as akin to t%e ranom variables - es&ri'e
a'ove. -nstea o$ attem)ting to generali2e 'e(on t%e )arti&ular values o$ t%e ine)enent varia'le,
we are attem)ting to generali2e 'e(on t%e )arti&ular grou)s in t%e stu(. 3or instan&e, we ma( %ave
50 &om)anies, 'ut we wis% to generali2e to a larger universe o$ &om)anies w%en we e+amine t%e
means (inter&e)ts" or t%e +.( relations%i) (slo)es".
-n #4M, varian&es $or t%e inter&e)ts an slo)es are estimate '( e$ault. ,%at is, w%en (ou
$irst &onstru&t a moel, U0j an U1j are estimate '( e$ault. #owever, t%e resear&%er %as t%e &%oi&e
o$ setting U0j an U1j to 'e 2ero. /( setting t%em to 2ero, we are testing a moel in w%i&% we assume
0j an 1j o not var( ranoml( a&ross grou)s. -n $a&t, t%eir varian&e is assume to 'e 2ero, so t%e(
are assume to 'e &onstant or 0nonvar(ing1 a&ross grou)s. 3or e+am)le, $i+e, nonvar(ing inter&e)ts
woul im)l( t%e grou) average $or t%e e)enent varia'le is assume to 'e e5ual in ea&% grou).
*lt%oug% we t()i&all( re$er to t%is &onstraint as 0$i+ing t%e inter&e)ts1 or 0$i+ing t%e slo)es,1 t%e term is
somew%at loosel( a))lie, 'e&ause we are reall( assuming t%e( are $i+e and nonvarying.
Intercept Model and Random Effects ANOVA
-n t%e sim)lest #4M moel, t%ere are no )rei&tors. We %ave one level.! e5uation an one level.2
e5uation6
0 ij j ij
Y R = +
w%ere Y is t%e e)enent varia'le, 0j is t%e inter&e)t, an Rij is t%e resiual or error. 0j &an 'e
t%oug%t o$ as t%e mean o$ ea&% grou). ,%e level.2 e5uation also %as no )rei&tors in it7s sim)lest
$orm6
0 00 0 j j
U = +
w%ere
0j
is t%e e)enent varia'le,
00
is t%e level.2 inter&e)t, an U
0j
is t%e level.2 error. -n t%is
e5uation, 00 re)resents t%e gran mean or t%e mean o$ t%e inter&e)ts. U0j re)resents t%e eviation o$
ea&% mean $rom t%e gran mean. W%en t%e average eviation is large, t%ere are large grou)
i$$eren&es.
-t &an 'e s%own t%at t%is is reall( t%e *N89* moel. *N89* e+amines t%e eviations o$ grou) means
$rom t%e gran mean. /e&ause we assume t%at t%e grou) means, re)resente '( 0j an, t%us, t%eir
eviations are ranoml( var(ing, t%is moel is e5uivalent to t%e random effects *N89* moel. We
&an rewrite our two #4M e5uations as a single e5uation, i$ we )lug in t%e level.2 terms into level.!
e5uation.
00 0 ij j ij
Y U R = + +
-n t%is e5uation, 00 re)resents t%e gran mean, t%e U0j term re)resents t%e eviations $rom t%e mean
(or mean i$$eren&es or treatment e$$e&t, i$ (ou woul like", an Rij re)resents t%e wit%in.grou) error or
variation. -n more mat%emati&all( oriente *N89* te+t'ooks, (ou will see an *N89* moel written
somet%ing like6
Newsom 2
USP 656 Multilevel Regression
Winter 2006
ij j ij
Y e = + +
-$ t%e moel is an ranom e$$e&ts *N89*, j is assume to %ave a ranom istri'ution. -n multilevel
regression, 'e&ause we assume U0j to var( ranoml(, t%e sim)le #4M moel wit% no level.! or level.2
)rei&tors is e5uivalent to t%e ranom e$$e&ts *N89* moel.
W%en we a )rei&tors to t%e level.! e5uation, t%e( are &ovariates an t%e moel 'e&omes a
ranom e$$e&ts *N:89* in w%i&% t%e means are ajuste $or t%e &ovariate.
Newsom 3
USP 656 Multilevel Regression
Winter 2006
Summary a!le
Random vs"
Fixed
#efinition Example $se in Multilevel
Regression
Varia!les Random variable6 (!" is assume to 'e measure wit%
measurement error. ,%e s&ores are a $un&tion o$ a true
s&ore an ranom error; (2" t%e values &ome $rom an
are intene to generali2e to a mu&% larger )o)ulation
o$ )ossi'le values wit% a &ertain )ro'a'ilit( istri'ution
(e.g., normal istri'ution"; (<" t%e num'er o$ values in
t%e stu( is small relative to t%e values o$ t%e varia'le as
it a))ears in t%e )o)ulation it is rawn $rom. Fixed
variable6 (!" assume to 'e measure wit%out
measurement error; (2" esire generali2ation to
)o)ulation or ot%er stuies is to t%e same values; (<"
t%e varia'le use in t%e stu( &ontains all or most o$ t%e
varia'le7s values in t%e )o)ulation.
-t is im)ortant to istinguis% 'etween a varia'le t%at is
varying an a varia'le t%at is random. * $i+e varia'le
&an %ave i$$erent values, it is not ne&essaril( invariant
(e5ual" a&ross grou)s.
Random variable:
)%otogra)%s
re)resenting iniviuals
wit% i$$ering levels o$
attra&tiveness
mani)ulate in an
e+)eriment, &ensus
tra&ks
Fixed variable:
gener, ra&e, or
intervention vs. &ontrol
grou).
Prei&tor
varia'les in M4R
generall(
assume to 'e
$i+e
Effects Random effect6 (!" i$$erent statisti&al moel o$
regression or *N89* moel w%i&% assumes t%at an
ine)enent varia'le is ranom; (2" generall( use i$
t%e levels o$ t%e ine)enent varia'le are t%oug%t to 'e
a small su'set o$ t%e )ossi'le values w%i&% one wis%es
to generali2e to; (<" will )ro'a'l( )rou&e larger
stanar errors (less )ower$ul". Fixed effect6 (!"
statisti&al moel t()i&all( use in regression an *N89*
assuming ine)enent varia'le is $i+e; (2"
generali2ation o$ t%e results a))l( to similar values o$
ine)enent varia'le in t%e )o)ulation or in ot%er
stuies; (<" will )ro'a'l( )rou&e smaller stanar
errors (more )ower$ul".
Random effect6
ranom e$$e&ts *N89*,
ranom e$$e&ts
regression
Fixed effect6 $i+e
e$$e&ts *N89*, $i+e
e$$e&ts regression
-nter&e)t onl(
moels in M4R
are e5uivalent to
ranom e$$e&ts
*N89* or
*N:89*.
Coefficients Random coefficient: term a))lies onl( to M4R
anal(ses in w%i&% inter&e)ts, slo)es, an varian&es &an
'e assume to 'e ranom. M4R anal(ses most
t()i&all( assume ranom &oe$$i&ients. 8ne &an
&on&e)tuali2e t%e &oe$$i&ients o'taine $rom t%e level.!
regressions as a t()e o$ ranom varia'le w%i&% &omes
$rom an generali2es to a istri'ution o$ )ossi'le values.
=rou)s are &on&eive o$ as a su'set o$ t%e )ossi'le
grou)s.
Fixed coefficient: a &oe$$i&ient &an 'e $i+e to 'e non.
var(ing (invariant" a&ross grou)s '( setting its 'etween
grou) varian&e to 2ero.
Ranom &oe$$i&ients must 'e varia'le a&ross grou)s.
:on&e)tuall(, $i+e &oe$$i&ients ma( 'e invariant or
var(ing a&ross grou)s.
Random coefficient6
t%e level.2 )rei&tor,
average in&ome, is
use to )rei&t s&%ool
)er$orman&e in ea&%
s&%ool. -nter&e)t
values $or s&%ool
)er$orman&e are
assume to 'e a
sam)le o$ t%e inter&e)ts
$rom a larger )o)ulation
o$ s&%ools.
Fixed coefficient6
slo)es or inter&e)ts
&onstraine to 'e e5ual
over i$$erent s&%ools.
/ot% use in
M4R. Slo)es an
inter&e)t values
&an 'e
&onsiere to 'e
$i+e or ranom,
e)ening on
resear&%ers
assum)tions an
%ow t%e moel is
s)e&i$ie. ,%e
varian&e o$ t%e
slo)es or
inter&e)ts are
&onsiere
ranom
&oe$$i&ients.