Professional Documents
Culture Documents
Xavier DHaultfuille
CREST-INSEE
Outline
Model and motivation
Inference in quantile regressions
Additional properties
Quantile regression in practice
Quantile IV
Quantile restrictions in nonlinear models
Outline
Model and motivation
Inference in quantile regressions
Additional properties
Quantile regression in practice
Quantile IV
Quantile restrictions in nonlinear models
Prologue: quantiles
I
q(U)
q(U)
Prologue: quantiles
I
(1)
q (Y |X ) = inf{u/FY |X (u|X ) }.
I
The model
In the location scale model, OLS = but running OLS we miss the
fact that the effect of X differs according to quantiles of the
Quantile Regression
unobserved variable .
1500
1000
500
Food Expenditure
1000
2000
3000
4000
5000
Household Income
Data taken from Engels (1857) and Koenker and Hallock (2001). Seven estimated quantile regression lines for
Engel Curves for Food: This gure plots data taken from
Ernst Engel's 1857
study of the dependence of households' food ex-
Figure
2.2. The median is indicated by the dashed line while the OLS estimate is the dotted line.
different values
of quantiles.
.
In this case, q (Y |X ) = X 0 1p
so instead of , we estimate 1p
It is independent of and will typically be close to . If some
components of are independent of (homogenous effects), the
contamination does not affect their estimation.
Outline
Model and motivation
Inference in quantile regressions
Additional properties
Quantile regression in practice
Quantile IV
Quantile restrictions in nonlinear models
Proposition
Consider the check function (u) = ( 1{u < 0})u. Then:
q (Y ) arg min E [ (Y a)] .
a
fY (y )dy = FY (a) .
Thus,integrating over P ,
(x 7 q (Y |X = x)) = arg min E [ (Y h(X ))] .
h(.)
1X
|Yi Xi0 |.
n
i=1
Identification
I
Consistency
I
Theorem
(van der Vaart, 1998, Theorem 5.7) Let denote the set of parameters
and supposethat for all > 0:
n
1 X
P
sup
(Ui , ) E ((U1 , )) 0,
(5)
n
i=1
inf
/d(,0 )
E ((U1 , ))
>
E ((U1 , 0 )).
(6)
Consistency
I
Consistency
Example: the standard Glivenko-Cantelli theorem. Let us consider the
functions (x, t) = 1{x t}. Then, if Y1 is continuous:
n
1 X
P
sup
(Yi , t) E ((Y1 , t)) 0.
n
tR
i=1
Proof: fix > 0 and consider t0 = < ... < tK = such that
F (tk ) F (tk1 ) < . Then for all t [tk1 , tk ],
Fn (t) F (t) Fn (tk ) F (tk1 ) Fn (tk ) F (tk ) +
Similarly, Fn (t) F (t) Fn (tk1 ) F (tk1 ) . Thus,
|Fn (t) F (t)| max{|Fn (tk ) F (tk )|, |Fn (tk1 ) F (tk1 )|} + .
Consistency
As a result,
sup |Fn (t) F (t)|
tR
max
i{0,...,K }
By the weak law of large numbers, the maximum tends to zero. The
result follows
This proof can be generalized to classes of functions different from
(1{. t})tR . A -bracket in Lr is a set of functions f with l f u,
1/r
R
< . For a
where l and u are two functions satisfying
|u l|r dF
given class of functions F, define the bracketing number N[ ] (, F, Lr ) as
the minimum number of -brackets needed to cover F.
Proposition
(van der Vaart, 1998, Theorem 19.4) Suppose that for all > 0,
N[ ] (, F, L1 ) < . Then
n
1 X
P
sup
f (Xi ) E (f (X1 )) 0.
f F n
i=1
Consistency
The proposition applies to many cases, see van der Vaart (1998), chapter
19, for examples. In particular, it holds with parametric families satisfying
|(Ui , 1 ) (Ui , 2 )| m(Ui )||1 2 ||, E (m(U1 )) < .
(7)
In quantile regression,
| (Y X 0 1 ) (Y X 0 2 )|
max(, 1 )|X 0 (1 2 )|
Thus (7) holds provided that E (||X ||) < . This establishes consistency
of b since we can then apply the theorem above.
Asymptotic normality
I
(8)
i=1
b we get
Then expanding around ,
"
#
n
n
1 X 2
1 X
b 0 )+oP (||
b 0 ||).
0=
(Ui , 0 )+
(Ui , 0 ) (
n
n
0
i=1
i=1
Hence, provided that one can show that ||b 0 || = OP (1/ n), we
have
" n
#
n
1 X 2
1 X
b
(U
,
)
n(
)
=
(Ui , 0 ) + oP (1).
i 0
0
n
0
n
i=1
i=1
Asymptotic normality
I
By the weak law of large numbers, the central limit theorem and
Slutskis lemma, we get:
L
n b 0 N (0, J 1 HJ 1 ),
h 2
i
where J = E
and H = V (
0 (Ui , 0 )
(Ui , 0 )). This kind of
variance is often called a sandwich formula.
The first order condition(8) may not hold exactly either. However,
0 can be replaced by oP 1n , which will be sufficient subsequently.
Asymptotic normality
Two key ideas for these kinds of situations:
I
7 Q()
I
h(Ui , ) is not
i differentiable at 0 ,
Even if 7
differentiable.
b + Q 0 ()
e n(b 0 ).
= Gn ()
(9)
h
i
b and Gn () = 1 Pn
where e (0 , )
i=1 (Ui , ) Q() . Gn is
n
Asymptotic normality
I
Proposition
(van der Vaart, Theorem 19.5) Gn , as a process indexed by f F,
converges to a continuous gaussian process if
Z 1q
ln N[ ] (, F, L2 )d < .
0
Asymptotic normality
I
K
.
d
Thus the bracketing integral is finite and one can apply the previous
theorem.
I
L
0
1
0
1
b
n 0 N 0, Q (0 ) V
(Ui , 0 ) Q (0 )
Asymptotic normality
I
Besides,
V
(Ui , 0 )
Asymptotic normality
I
Finally, we get:
1
1
L
n b N 0, (1 )E f |X (0|X )XX 0
E XX 0 E f |X (0|X )XX 0
.
I
(1 )
1
E [XX 0 ] .
f (q ())2
This formula is similar to the one for the OLS estimator, except that
2 is replaced by (1 )/f (q ())2 . In general, as we let 1
or 0, f (q )2 becomes very small and thus b becomes imprecise.
This is logical since data are often more dispersed at the tails.
Asymptotic normality
I
b , in
Remark 2: this result applies in particular to simple quantiles q
which case we have:
(1 )
L
n (b
q q ) N 0, 2
.
fY (q )
I
I
I
F1
( )
f (F1 ( ))
F 1 ( + h) F1 ( h)
.
= lim
h0
2h
n
1 X
1{|b
i | hn }Xi Xi0 .
2nhn
(11)
i=1
Note that we can replace the uniform kernel 1{|u| 1}/2 in (11)
by any other density function.
Confidence interval on :
q
q
bas , b + z1/2 V
bas ,
IC = b z1/2 V
where z1/2 is the 1 /2-th quantile of the N (0, 1) distribution.
1
g
g
b
( )Vas ( )
g (b ),
0
Vas
=
B
1 X b
b 2.
( b )
B
b=1
, q1/2
].
IC1 = [q/2
Pr (Tn ( ) z1 )
1 .
I
t T .
Outline
Model and motivation
Inference in quantile regressions
Additional properties
Quantile regression in practice
Quantile IV
Quantile restrictions in nonlinear models
(12)
misspecification, i.e. q (Y |X ) 6= X 0 ;
finite sample errors.
1{x 0 u y }du.
(13)
1.0
0.0
0.2
0.4
0.6
0.8
Q
Q*
0.0
0.2
0.4
0.6
u
0.8
1.0
19
B Rearranged curves
5000
10000
15000
Nonveterans
Veterans
5000
10000
Annual earnings
15000
Nonveterans
Veterans
Annual earnings
20000
A Original curves
0.2
0.4
0.6
Quantile index
0.8
0.2
0.4
0.6
0.8
Quantile index
Outline
Model and motivation
Inference in quantile regressions
Additional properties
Quantile regression in practice
Quantile IV
Quantile restrictions in nonlinear models
Computation of b .
I
10 u + (1 )10 v
s.t. X + u v Y = 0,
Computation of b .
I
xRn
(14)
Then one can show that (i) S is a convex polyhedron and (ii) if
solutions exist, then they are vertices of S.
xR
n
X
ln xk
s.t. B x = d.
(15)
k=1
Software programs
I
Software programs
I
Software programs
I
library(quantreg)
rq(y ~ x1 + x2, tau = (single quantile or vector of
quantiles), data=(dataset), method=("br" or "fn"))
I
An example
I
An example
I
SAS code:
ods graphics on;
proc quantreg data=birth weights ci=sparsity/iid alg=interior(tolerance=1e-4);
model birth weight = boy married black age age2 high school some college
college prenatal second prenatal third no prenatal smoker
nb cigarettes /quantile= 0.05 to 0.95 by 0.05 plot quantplot;
run;
ods graphics off;
Stata code:
sqreg birth weigh boy married black age age2 high school some college prenatal second
prenatal third no prenatal smoker nb cigarettes, quantiles(0.05 0.1 0.2 0.3 0.4
0.5 0.6 0.7 0.8 0.9 0.95)
An example
Quantile and Objective Function
Quantile
Objective Function
Predicted Value at Mean
0.1
31108564.261
2727.4037
Parameter Estimates
Parameter
Intercept
boy
married
black
age
age2
high_school
some_college
college
prenatal_second
prenatal_third
no_prenatal
smoker
nb_cigarettes
DF Estimate
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2150.419
83.8925
64.9045
-251.465
38.3584
-0.6657
6.5725
36.6800
76.1075
-4.1840
22.2022
-472.532
-156.928
-5.8266
Standard
Error
41.9615
3.8034
4.9650
5.4947
3.0443
0.0523
5.7090
6.4022
6.7700
5.9940
12.2669
19.1648
10.6564
0.8140
95% Confidence
Limits
2068.176
76.4380
55.1734
-262.234
32.3916
-0.7682
-4.6170
24.1319
62.8384
-15.9321
-1.8405
-510.095
-177.815
-7.4221
2232.662
91.3471
74.6357
-240.696
44.3251
-0.5631
17.7620
49.2281
89.3765
7.5641
46.2449
-434.970
-136.042
-4.2311
<.0001
<.0001
<.0001
<.0001
<.0001
<.0001
0.2496
<.0001
<.0001
0.4852
0.0703
<.0001
<.0001
<.0001
An example
An example
An example
An example
Outline
Model and motivation
Inference in quantile regressions
Additional properties
Quantile regression in practice
Quantile IV
Quantile restrictions in nonlinear models
Motivation
I
I
I
P(X 0 U X 0 |Z )
P(U |Z ) = .
(16)
"
#0
"
#
n
n
1X
1X
0
0
b
= arg min
g (Zi ) 1{Yi Xi }
g (Zi ) 1{Yi Xi } .
Wn
n i=1
n i=1
I
n
n
i=1
i=1
P
Then, if Wn W , we can show that b is consistent and
asymptotically normal, with:
L
n b 0 N 0, J 1 HJ 1 ,
Tn (0 ) =
!0
n
1 X
g (Zi )( Bi (0 )) Wn
n i=1
!
n
1 X
g (Zi )( Bi (0 )) ,
n i=1
q ( |Z ) = 0.
In other words,
(0 , 0) = arg min E [ (Y X00 0 X10 Z00 )]
(17)
(,)
b = arg min Wn (j )
j=1...J
b ).
and b = (b
I
.
P(Z0 = 0|X1 )
P(Z0 = 1|X1 )
of earnings)
IV
14.6
35.5
23.1
18.4
10.1
7.4
Outline
Model and motivation
Inference in quantile regressions
Additional properties
Quantile regression in practice
Quantile IV
Quantile restrictions in nonlinear models
Introduction
(18)
Proposition
Let g be an increasing, left continuous function, then
g (q (Y )) = q (g (Y )).
Proof: recall that q (g (Y )) = inf{x R/Fg (Y ) (x) }. we have
P(Y q (Y )) P(g (Y ) g (q (Y ))).
Thus, g (q (Y )) q (g (Y )). Conversely, let u = q (g (Y )) and
g (v ) = sup{x/g (x) v }. Then
P(g (Y ) u) P(Y g (u)).
As a result, g (u) q (Y ). Because g is left continuous,
g (g (u)) u. Thus, q (g (Y )) = u g (q (Y )), which ends the proof.
q (|X ) = 0.
x 0 0
dF|X =x (u),
R
and the model imposes that udF|X =x (u) = 0.
Consider 6= 0 . For all x, it is possible (exercise...) to build
a distribution function Gx 6= F|X =x such that:
Z
dGx (u) = P(Y = 1|X = x)
x 0
Z
udGx (u) = 0.
i=1
n
X
n
X
i=1
max
Vect(0 )
W (),
n
X
Yi max(0, Xi0 ) .
i=1
X
X
1
b = arg min
(Yi Xi0 ) +
(Yi ) .
n
0
0
i/Xi 0
i/Xi <0
bj
bj