Akdeniz Erol 2003

This article was downloaded by:[ANKOS Consortium]
On: 25 January 2008

Access Details: [subscription number 772815469]
Publisher: Taylor & Francis
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Communications in Statistics -
Theory and Methods
Publication details, including instructions for authors and subscription information:
http://www.informaworld.com/smpp/title~content=t713597238
Mean Squared Error Matrix Comparisons of Some
Biased Estimators in Linear Regression
Fikri Akdeniz
a
; Hamza Erol
a
a
Department of Statistics, Faculty of Arts and Sciences, University of ukurova,
Adana, Turkey
Online Publication Date: 12 January 2003
To cite this Article: Akdeniz, Fikri and Erol, Hamza (2003) 'Mean Squared Error
Matrix Comparisons of Some Biased Estimators in Linear Regression',
Communications in Statistics - Theory and Methods, 32:12, 2389 - 2413
To link to this article: DOI: 10.1081/STA-120025385
URL: http://dx.doi.org/10.1081/STA-120025385
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article maybe used for research, teaching and private study purposes. Any substantial or systematic reproduction,
re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly
forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents will be
complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be
independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or
arising out of the use of this material.
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.
MARCEL DEKKER, INC. 270 MADISON AVENUE NEW YORK, NY 10016
COMMUNICATIONS IN STATISTICS
Theory and Methods
Vol. 32, No. 12, pp. 23892413, 2003
Mean Squared Error Matrix Comparisons of
Some Biased Estimators in Linear Regression
Fikri Akdeniz
*
and Hamza Erol
Department of Statistics, Faculty of Arts and Sciences,
University of C ukurova, Adana, Turkey
ABSTRACT
Consider the linear regression model y X[ u in the usual nota-
tion. In the presence of multicollinearity certain biased estimators like
the ordinary ridge regression estimator
^
[[
k
X
0
X kI
1
X
0
y
and the Liu estimator
^
[[
d
X
0
X I
1
X
0
y d
^
[[ introduced by
Liu (Liu, Ke Jian. (1993). A new class of biased estimate in
linear regression. Communications in Statistics-Theory and Methods
22(2):393402) or improved ridge and Liu estimators are used to out-
perform the ordinary least squares estimates in the linear regression
model. In this article we compare the (almost unbiased) generalized
ridge regression estimator with the (almost unbiased) generalized Liu
estimator in the matrix mean square error sense.
*Correspondence: Fikri Akdeniz, Department of Statistics, Faculty of Arts and
Sciences, University of C ukurova, 01330 Adana, Turkey; Fax: +90-322-3386070;
E-mail: akdeniz@mail.cu.edu.tr.
2389
DOI: 10.1081/STA-120025385 0361-0926 (Print); 1532-415X (Online)
Copyright & 2003 by Marcel Dekker, Inc. www.dekker.com
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

Key Words: Correlation matrix; Liu estimator; Mean square error;
Multicollinearity; Ridge regression estimator.
1. INTRODUCTION
The problem of multicollinearity and its statistical consequences for
a linear regression model are very well-known in statistics.
Multicollinearity is dened as the existence of nearly linear dependency
among column vectors of the design matrix X in the linear model
y X[u. The best way of explaining the structure of multicollinearity
is to look over the eigenvalues and eigenvectors of the matrix X
0
X.
In case of multicollinearity we know that when the correlation matrix
has one or more small eigenvalues, the estimates of the regression
coecients can be large in absolute value. Severe multicollinearity may
make the estimates so unstable that they are practically useless. In this
situation, the matrix X
0
X becomes near singular or ill-conditioned and
least squares estimator
^
[[ will pass undue sensitivity to the data. To
overcome this, dierent remedial actions have been proposed. One of
biased estimators of [ used when collinearity is present in the data, the
ordinary ridge regression (ORR) estimator, is dened by
^
[[
k
X
0
X kI
1
X
0
y,
where k>0. When k 0 we have the ordinary least squares estimator
^
[[ of
[. The name ridge regression was rst used by Hoerl and Kennard
(1970a, 1970b).
To combat near multicollinearity, Liu (1993) combined the Stein
(1956) estimator with the ORR estimator. The estimator
^
[[
d
due to Liu
(1993), is called the Liu estimator by Akdeniz and Kac iranlar (1995),
(see also Gruber, 1998 p. 48 and Kac iranlar et al., 1999), is dened for
each parameter d 2 1, 1 as follows:
^
[[
d
: X
0
X I
1
X
0
y d
^
[[.
However, Liu (1993) did not compare the mean square error proper-
ties of his estimator to that of the ORR estimator. Sakalliog lu et al.,
(2001) compared the mean square error (MSE) matrices of
^
[[
k
and
^
[[
d
.
Akdeniz et al., (1999) proposed the improved Liu estimator of [
by forming a convex combination of the ordinary least squares (OLS)
estimator and the generalized Liu estimator. The performance of the
proposed estimator as compared with the generalized Liu estimator
2390 Akdeniz and Erol
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

and the almost unbiased generalized Liu estimator in terms of the mean
square error criterion.
In this article under review several estimators of the parameter vector
in the multiple linear regression model are compared in terms of their
respective mean square error matrices. These include the generalized
ridge regression estimator, the generalized Liu estimator, and so-called
almost unbiased versions of these estimators. The article starts with an
introductory section containing a brief review of the literature on these
estimators. In Sec. 2 the general multiple linear regression model is
dened, and expressions are provided for the estimators to be compared
in the remainder of the atricle. The mean square error matrix of an
estimator is also dened in this section. In Sec. 3 two theorems are
presented and proved. These theorems establish comparative results on
the mean square error matrices of the generalized ridge regression
estimator and the generalized Liu estimator. It is important to note
that the biasing parameters in these estimators are assumed to be
nonstochastic to make the problem mathematically tractable in this
section. In practice, these biasing parameters are typically estimated
from the available data. Section 4 is largely a replicate of Sec. 3, but
now the almost unbiased generalized ridge regression estimator and the
almost unbiased generalized Liu estimator are compared in terms of their
mean square error matrices. The rst part of Sec. 5 contains a summary
of dierent estimates that have been proposed in the literature for the
biasing parameters of the previously discussed estimators. This is
followed by application of the theorems of Secs. 3 and 4 to these cases.
The article closes in Sec. 6 with an extensive discussion of an example.
2. THE MODEL AND ESTIMATORS
Let us consider the general linear regression model
y Z, u 2.1
where Z is an n p matrix with full column rank of observations on
nonstochastic regressors, y is an n 1 vector of observations on the depen-
dent variable, , is an p 1 vector of unknown regression coecients and u
is an n 1 vector of independent and identically distributed randomerrors
with E(u) 0 and Euu
0
o
2
I
n
.
It is common practice in ridge regression to standardize the Z matrix
so that Z
0
Z is in correlation form. We also standardize y so that both Z
and y are in the standardized form.
MSE Matrix Comparisons 2391
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

It is well known that a linear regression model can be transformed to
a canonical form by orthogonal transformation. Let T be an orthogonal
matrix such that
T
0
Z
0
ZT 2.2
where is a p p diagonal matrix whose elements z
1
, z
2
, . . . , z
p
are the
eigenvalues of Z
0
Z. Making use of the matrix T, we obtain a canonical
form of the model (2.1):
y X[ u 2.3
where X :ZT and [ :T
0
,. Then, working with the model (2.3) the
ordinary least squares estimator of [ is written as
^
[[ X
0
X
1
X
0
y,
or
^
[[
1
X
0
y. 2.4
The presence of multicollinearity among regression variables in linear
regression analysis may cause highly unstable least squares estimates of
the regression parameters. The statisticians thus face the problem of
choosing between the least squares estimator and a biased estimator.
They also face the problem of choosing between two biased estimators.
The following biased estimators are considered throughout the article.
Ordinary Ridge Regression (ORR) Estimator and
Generalized Ridge Regression (GRR) Estimator
^
[[
k
kI
1
X
0
y, k > 0 2.5
^
[[
K
K
1
X
0
y K
1
X
0
X
^
[[ K
1
^
[[, 2.6
where Kdiag(k
i
) is a diagonal matrix of biasing factors, k
i
>0,
i 1, 2, . . . , p, (see Hoerl and Kennard, 1970a; 1970b).
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

Almost Unbiased Ridge (AUR) and Almost Unbiased
Generalized Ridge Regression (AUGRR) Estimators
[[
k
I k
2
kI
2
^
[[ and

[[
K
I K
1
K
2
^
[[,
2.7
(see Singh et al., 1986).
Liu Estimator and Generalized Liu (GL) Estimator
The Liu estimator
^
[[
d
, is dened for each parameter d 2 1, 1,
see Kac iranlar et al., 1999) as follows:
^
[[
d
I
1
X
0
y d
^
[[ 2.8
or
^
[[
d
I
1
dI
1
X
0
y I
1
dI
^
[[
^
[[ I
1
1 d
^
[[
^
[[
D
I
1
X
0
y D
^
[[ 2.9
or
^
[[
D
I
1
D
1
X
0
y I
1
D
^
[[
^
[[ I
1
I D
^
[[
where Ddiag(d
i
) is a diagonal matrix of the biasing parameters
d
i
, d
i
2 1, 1, i 1, 2, . . . , p (Akdeniz and Kac iranlar, 1995).
Almost Unbiased Liu (AUL) and Almost Unbiased
Generalized Liu (AUGL) Estimators
[[
d
I I
2
1 d
2
1
X
0
y
and
[[
D
I I
2
I D
2
1
X
0
y 2.10
(see Akdeniz and Kac iranlar, 1995).
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

If we want to estimate accuracy of linear estimators Ly of [ by means
of the total mean square error (TMSE), which has the structure
[
0
H[ ho
2
and it is given by
TMSE E Ly [
0
Ly [
_ _
[
0
X
0
L
0
I X
0
L
0
I
0
[ o
2
trLL
0
2.11
The estimators of TMSE are used for comparison of linear estima-
tors (see Gnot et al., 1995).
Another measure of goodness of an estimator is generalized mean
square error (gmse)
gmse
[[, [ E
[[ [
0
B
[[ [ , 2.12
where B is a nonnegative denite (n.n.d.) matrix. Dene the mean square
error matrices of any two estimators

[[
1
and

[[
2
of [ by
M
[[
j
, [ E
[[
j
[
[[
j
[
0
, j 1, 2. 2.13
Theobald (1974) proves that gmse
[[
1
> gmse
[[
2
for all n.n.d.
matrices B if and only if M
[[
1
, [ M
[[
2
, [ is n.n.d. matrix. Thus the
superiority of

[[
2
over

[[
1
with respect to the mean square error criteria can
easily be examined by comparing the mean square error (MSE) matrices.
It is evident that the above mentioned estimators are homogeneous
linear. For any competing homogeneous linear estimators

[[
j
A
j
y,
j 1, 2 we get MSE matrices
M
~
[[
j
, [ Cov
~
[[
j
Bias
~
[[
j
Bias
~
[[
j
0
, j 1, 2. 2.14
where Bias
[[
j
E
[[
j
[, j 1, 2. For notational convenience let us
set d
j
Bias
~
[[
j
A
j
X I [, j 1, 2 and the dierence of the covar-
iance matrices becomes
D Cov
~
[[
1
Cov
~
[[
2
o
2
A
1
A
0
1
A
2
A
0
2
.
Then we get
M
[[
1
, [ M
[[
2
, [ D d
1
d
0
1
d
2
d
0
2
. 2.15
In order to inspect whether
~
[[
1
,
~
[[
2
M
[[
1
, [ M
[[
2
, [ is
positive denite (p.d.) we may conne ourselves the following case:
D Cov
~
[[
1
Cov
~
[[
2
> 0 p.d.. 2.16
As d
1
d
0
1
Bias
[[
1
Bias
[[
1
0
0, it is easy to see that D>0 implies
D d
1
d
0
1
> 0 (see Trenkler and Toutenburg, 1990; Rao and Toutenburg,
1995). Hence the problem of deciding whether
~
[[
1
,
~
[[
2
> 0 reduces to
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

that of deciding whether a matrix type A aa
0
is positive denite when A
is positive denite. Then we have the following result.
Theorem 2.1. (Farebrother (1976)) Suppose that A is p.d., a is a compatible
column vector . Then A aa
0
> 0 ( p.d.) if and only if a
0
A
1
a - 1. From
this we have the following result.
Theorem 2.2. (Trenkler and Toutenburg (1990)) Let

[[
j
A
j
y, j 1, 2, be
two competing homogeneous linear estimators of [. Suppose the dierence
D Cov
~
[[
1
Cov
~
[[
2
of the covariance matrices of the estimators
~
[[
1
and
~
[[
2
is positive denite. Then
~
[[
1
,
~
[[
2
M
[[
1
, [ M
[[
2
, [ is p.d.
(
~
[[
2
is strongly superior to
~
[[
1
) if and only if
d
0
2
D d
1
d
0
1
1
d
2
- 1. 2.17
As the MSE contains all relevant information about the quality of an
estimator, comparisons between dierent estimators may be made on the
basis of their MSE matrices. The MSE matrices and biases for the
estimators given in Eqs. (2.6) and (2.9) are respectively,
M
^
[[
K
, [ K
1
K
1
K
1
K[[
0
K K
1
2.18
Bias
^
[[
K
K
1
K[ 2.19
M
^
[[
D
, [ I
1
D
1
D I
1
I
1
I D[[
0
I D I
1
2.20
Bias
^
[[
D
I
1
I D[ 2.21
3. MSE MATRIX COMPARISONS OF GRR
ESTIMATOR AND GL ESTIMATOR
Our objective is to examine the dierence of the mean square error
matrices of two estimators. Let us consider two competing homogeneous
linear estimators
^
[[
K
K
1
X
0
y C
1
y, 3.1
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

and
^
[[
D
I
1
D
1
X
0
y C
2
y. 3.2
Then it is easy to see that
Cov
^
[[
K
o
2
C
1
C
0
1
and Bias
^
[[
K
C
1
X I [,
and
Cov
^
[[
D
o
2
C
2
C
0
2
and Bias
^
[[
D
C
2
X I [.
The following results are derived on the assumption that certain
crucial constants (biasing parameters) in the estimators have known
values. Let now d
i
s be xed for the moment and let be
0 - d
i
- 1, i 1, 2, . . . , p. Thus we have the following theorem.
Theorem 3.1. Let us consider two biased competing homogeneous linear
estimators
^
[[
K
and
^
[[
D
of [.
(a) Let 0 - k
i
- k
i
- 1. Then M
^
[[
K
, [ M
^
[[
D
, [ is p.d. if and
only if
Bias
^
[[
D
0
o
2
C
1
C
0
1
C
2
C
0
2
Bias
^
[[
K
Bias
^
[[
K
1
Bias
^
[[
D
- 1.
(b) Let 0 - k
i
- k
i
- 1. Then M
^
[[
D
, [ M
^
[[
K
, [, is p.d. if and
only if
Bias
^
[[
K
0
o
2
C
2
C
0
2
C
1
C
0
1
Bias
^
[[
D
Bias
^
[[
D
1
Bias
^
[[
K
- 1,
where k
i
1 d
i
z
i
,z
i
d
i
, i 1, 2, . . . , p.
Proof. Using the estimators
^
[[
K
and
^
[[
D
, we obtain
Cov
^
[[
K
Cov
^
[[
D
o
2
C
1
C
0
1
C
2
C
0
2
o
2
K
1
K
1
I
1
D
1
D I
1
o
2
diag
z
i
k
i
z
i
2

z
i
d
i
2
z
i
1
2
z
i
_ _
p
i1
_ _
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

C
1
C
0
1
C
2
C
0
2
will be p.d. if and only if
z
2
i
1 z
i
2
z
i
k
i
2
d
i
z
i
2
> 0
or
z
i
1 z
i
z
i
k
i
d
i
z
i
> 0. 3.3
This inequality requires that C
1
C
0
1
C
2
C
0
2
is p.d. for 0 - k
i
- k
i

1 d
i
z
i
,z
i
d
i
- 1 and C
2
C
0
2
C
1
C
0
1
is p.d. for 0 - k
i
- k
i
- 1. By
Theorem 2.2 the proof is completed.
If k
1
k
2
k
p
k and d
1
d
2
d
p
d, then we have
the following inequalities:
0 - k -

kk
i
1 d z
i
,z
i
d - 1and0 -

kk
i
- k - 1respectively.
Let now k
i
s be xed for the moment and let be 0<k
i
<1,
i 1, 2, . . . , p. Analogous to Theorem 3.1, we shall state without proof
the following theorem.
Theorem 3.2.
(a) Let 0 - d
i
- d
i
- 1. Then M
^
[[
D
, [ M
^
[[
K
, [ is p.d. if and
only if
Bias
^
[[
K
0
o
2
C
2
C
0
2
C
1
C
0
1
Bias
^
[[
D
Bias
^
[[
D
1
Bias
^
[[
K
- 1.
(b) Let 0 - d
i
- d
i
- 1. Then M
^
[[
K
, [ M
^
[[
D
, [ is p.d. if and
only if
Bias
^
[[
D
0
o
2
C
1
C
0
1
C
2
C
0
2
Bias
^
[[
K
Bias
^
[[
K
1
Bias
^
[[
D
- 1.
where d
i
1 k
i
z
i
,z
i
k
i
, i 1, 2, . . . , p.
If k
1
k
2
. . . k
p
k and d
1
d
2
. . . d
p
d, then we have the
following inequalities:
0 -

dd
i
1 kz
i
,z
i
k - d - 1and0 - d -

dd
i
- 1respectively.
4. MSE MATRIX COMPARISONS OF (AUGRR)
ESTIMATOR,
e
bb
K
AND (AUGL) ESTIMATOR,
e
bb
D
Let us consider two competing homogeneous linear estimators
[[
k
I K
1
K
2
1
X
0
y A
1
y 4.1
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

and
[[
D
I I
2
I D
2
1
X
0
y A
2
y 4.2
Then it is easy to see that
Cov
[[
k
o
2
A
1
A
0
1
and Bias
[[
k
A
1
X I [,
and
Cov
[[
D
o
2
A
2
A
0
2
and Bias
[[
D
A
2
X I [.
Let now k
i
s be xed for the moment and let be 0 - k
i
- 1,
i 1, 2, . . . , p. Thus we have the following theorem.
Theorem 4.1.
(a) Let 0 - d
i
- d
i
- 1. Then M
[[
D
, [ M
[[
K
, [ is p.d. if and
only if
Bias
[[
0
o
2
A
2
A
0
2
A
1
A
0
1
Bias
[[
D
Bias
[[
1
Bias
[[
K
- 1.
(b) Let 0 - d
i
- d
i
- 1. Then M
[[
K
, [ M
[[
D
, [ is p.d. if and
only if
Bias
[[
0
o
2
A
1
A
0
1
A
2
A
0
2
Bias
[[
K
Bias
[[
1
Bias
[[
D
- 1,
where d
i
1 k
i
z
i
,z
i
k
i
, i 1, 2, . . . , p.
Proof. Using the estimators

[[
K
and

[[
D
, we obtain
Cov
[[
D
Cov
[[
o
2
A
2
A
0
2
A
1
A
0
1
o
2
fI I
2
I D
2
1
I I
2
I D
2
0
I K
2
K
2
1
I K
2
K
2
0
g
o
2
diag
z
i
d
i
2
2 z
i
d
i
2
z
i
1 z
i
4

z
i
z
i
2k
i
2
z
i
k
i
4
_ _
p
i1
_ _
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

A
2
A
0
2
A
1
A
0
1
will be p.d. if and only if
z
i
d
i
2
2 z
i
d
i
2
z
i
k
i
4
z
2
i
z
i
2k
i
2
1 z
i
4
> 0,
or
z
i
d
i
2 z
i
d
i
z
i
k
i
2
z
i
z
i
2k
i
1 z
i
2
> 0. 4.3
This inequality requires that A
2
A
0
2
A
1
A
0
1
is p.d. for 0 - d
i
- d
i
- 1
and A
1
A
0
1
A
2
A
0
2
is p.d. for 0 - d
i
- d
i
- 1. By Theorem 2.2 the proof
is completed.
Let now d
i
s be xed for the moment and let be 0 - d
i
- 1,
i 1, 2, . . . , p. Analogous to Theorem 4.1, we shall state without
proof the following theorem.
Theorem 4.2.
(a) Let 0 - k
i
- k
i
- 1. Then M
[[
D
, [ M
[[
D
, [ is p.d. if and
only if
Bias
[[
0
o
2
A
1
A
0
1
A
2
A
0
2
Bias
[[
K
Bias
[[
1
Bias
[[
D
- 1.
(b) Let 0 - k
i
- k
i
- 1. Then M
[[
D
, [ M
[[
K
, [ is p.d. if and
only if
Bias
[[
0
o
2
A
2
A
0
2
A
1
A
0
1
Bias
[[
D
Bias
[[
1
Bias
[[
K
- 1,
where k
i
1 d
i
z
i
,z
i
d
i
, i 1, 2, . . . , p.
All results reported above are based on the assumption that k
i
s
(or d
i
s) are nonstochastic. The theoretical results indicate that the ordin-
ary ridge regression estimator is not always better than the Liu estimator,
(see Sakalliog lu et al., 2001). Similarly generalized ridge regression esti-
mator is not always better than generalized Liu estimator. AUGRR
estimator also is not always better than the AUGL estimator. Which
estimator is best seems to depend on the unknown parameters [ and o
2
and also on the choice of k
i
i 1, 2, . . . , p, in the generalized ridge
regression estimator, d
i
i 1, 2, . . . , p, in the generalized Liu estimator.
This makes it dicult to know how to make use of the results in practice.
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

For practical purposes, we have to replace these unknown parameters by
some suitable estimates.
5. SELECTION OF THE BIASING PARAMETERS AND
THE OPERATIONAL ESTIMATORS
A very important issue in the study of ridge regression is how to nd
appropriate parameters k
i
i 1, 2, . . . , p. The nonnegative elements, k
i
,
are often referred to as biasing constants which may either be
nonstochastic or may depend on the observed data. The choice of values
for these biasing constants has been one of the most dicult problems
confronting the study of the generalized ridge regression. However, when
k is data dependent, Monte Carlo studies are required for a more
denitive comparison of the two estimators as well as the determination
of the best rule for selecting k, (see McDonald and Galarneau, 1975).
Optimal values for the k
i
s in Eq. (2.6) can be considered to be those
k
i
s that minimize
Q
1
E
^
[[
K
[
0
^
[[
K
[.
With a certain amount of algebra, Q
1
may be expressed as
Q
1

p
i1
o
2
z
i
[
2
i
k
2
i

z
i
k
i
2
.
Hoerl and Kennard (1970a, 1970b) have shown that the values of k
i
which minimize the mean square error of
^
[[
K
are given by
k
iopt

o
2
[
2
i
, i 1, 2, . . . , p. 5.1
Moreover, Hocking et al., (1976) showed that when these optimal k
i
values in Eq. (5.1) are known, the generalized ridge regression estimator
is superior to all estimators within the class of biased estimators they
considered. Because these values are unknown, Hoerl and Kennard
(1970a, 1970b) propose replacing the unknown quantities o
2
and [
i
by
their OLS estimates:
^
kk
i

^ oo
2
^
[[
2
i
. 5.2
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

Hoerl et al., (1975) proposed the use of the harmonic mean of the k
i
values; the arithmetic average will not be a good choice because very
small [
i
that have no predicting power would generate a very large k
resulting in too much bias. The harmonic mean k
h
is calculated using
the following formula:
k
h

1
1,p
p
i1
1,k
i
po
2
[
0
[
.
Unfortunately, this interval for k is of limited practical value since its
upper bound depends on the true values of the parameters being
estimated. A widely used practice is to set
^
kk
h

^
kk
HKB

p ^ oo
2
^
[[
0
^
[[
. 5.3
However, in the presence of multicollinearity,
^
[[
0
[ is likely to be large
leading to the use of a very small k value. It is shown that
E
^
[[
0
^
[[ [
0
[ o
2
traceX
0
X
1
. Thus,
^
[[
0
^
[[ is larger than [
0
[ and the
poorer the conditioning of X
0
X the larger the dierence between
^
[[
0
^
[[
and [
0
[. The squared length of the ridge regression estimator based on
^
kk
HKB
p ^ oo
2
,
^
[
0
[
0
^
[[, i.e.,
~
[[
^
kk
HKB
, is going to be smaller than
^
[[
0
^
[[. It seems
reasonable to expect an improvement if the
^
[[ in
^
kk
HKB
replaced by
~
[[
^
kk
HKB
. As k increases we know that
~
[[
0
~
[[
, the squared length of

the ridge estimator, decreases.
Lawless and Wang estimates are given by
^
kk
LW

p
p
i1
z
i
,
^
kk
i
p ^ oo
2
p
i1
z
i
^
[[
2
i
5.4
(see Lawless and Wang, 1976). Because the optimal k
LW
depends upon
the unknown quantities, an adaptive optimal k is instead obtained.
Hemmerle and Brantle estimates are given by
^
kk
HB
i

z
i
^ oo
2
z
i
^
[[
2
i
^ oo
2
if e
i0
- 1
1 if e
i0
1
_
_
5.5
where
e
i0

^ oo
2
z
i
^
[[
2
i
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

(see Hemmerle and Brantle, 1978). Troskie and Chalton estimates are
given by
^
kk
TC
i

z
i
^ oo
2
z
i
^
[[
2
i
^ oo
2
5.6
(see Troskie and Chalton, 1996). Firinguetti (1999) proposed a new
estimate of the optimal shrinkage parameters, k
i
o
2
,[
2
i
, namely:
^
kk
F
i

z
i
^ oo
2
n p ^ oo
2
z
i
^
[[
2
i
. 5.7
Replacing k
i
in K by
^
kk
i
to form
^
KK and substituting it in place of K in
Eq. (2.6) leads to an adaptive estimator of [:
^
[[
^
KK

^
KK
1
X
0
y. 5.8
Optimal values for the d
i
s in Eq. (2.9) can be considered to be those
d
i
s that minimize
Q
2
E
^
[[
D
[
0
^
[[
D
[.
With a certain amount of algebra, Q
2
may be expressed as
Q
2

p
i1
o
2
z
i
d
i
2
z
i
1 d
i
2
[
2
i

z
i
z
i
1
2
. 5.9
Q
2
is minimized at
d
iopt

z
i
[
2
i
o
2
z
i
[
2
i
o
2
. 5.10
Substituting d
i(opt)
in place of d
i
i 1, 2, . . . , p in
^
[[
D
provides an
estimator which is useless for the simple reason that o
2
and [
i
are
unknown. Thus, substituting o
2
and [
i
by their unbiased estimates ^ oo
2
and
^
[[
i
we obtain the estimate of d
i
:
^
dd
iOLS

z
i
^
[[
2
i
^ oo
2
z
i
^
[[
2
i
^ oo
2
1
1 z
i
^ oo
2
z
i
^
[[
2
i
^ oo
2
. 5.11
Unfortunately, we do not have any guarantee that these values are
always positive or between 0 and 1. We call this estimator the OLS-based
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

minimum mean square error estimator for d
i(opt)
. Thus we get a feasible
version of the Liu stimator.
Now, consider the mean square error matrix of

[[
K
as,
M
[[
K
, [ I U
2
1
I U
2
0
U
2
[[
0
U
02
5.12
where U K
1
K. Considering only the i th diagonal element of Eq.
(5.12) as, M
i
[[
K
, [ we have
M
i
[[
K
, [ o
2
1 u
2
i

2
z
i
u
4
i
[
2
i
5.13a
and ordinary mean square error (mse) of

[[
K
is
mse
[[
K
trM
[[
D
, [
p
i1
1 u
2
i

2
o
2
z
i
p
i1
u
4
i
[
2
i
5.13b
where u
i
k
i
,z
i
k
i
. Nomura (1988) has shown that the optimal value
of k
i
for which the M
i
[[
K
, [ is minimum is
k
iopt

o
2
z
i
[
2
i
o
2
_
[
2
i
. 5.14
If the individual k
i
in Eq. (5.14) are combined in some way to obtain
a single value of k, Nomura (1988) suggested that a reasonable choice is
~
kk
N

p ^ oo
2
p
i1

^
[[
2
i
,
_
1
1 z
i
^
[[
2
i
, ^ oo
2
_
_
.
Similarly, we write the MSE matrix of

[[
D
from Eq. (2.9) as
M
[[
D
, [ o
2
I
2
1
I
2
0

2
[[
0
02
5.15
where I
1
I D. Let us consider the i th term of
M
[[
D
, [ i.e.,
M
i
[[
D
, [ 1 o
2
i

2
o
2
z
i
o
4
i
[
2
i
5.16a
and ordinary mean square error of

[[
D
is
mse
[[
D
trM
[[
D
, [
p
i1
1 o
2
i

2
o
2
z
i
p
i1
o
4
i
[
2
i
5.16b
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

where o
i
1 d
i
,1 z
i
. Akdeniz et al., (1999) have shown that the
optimal choice of the parameter d
i
for which the M
i
[[
D
, [ is minimum is
d
iopt
1
o
2
1 z
i
2
o
2
z
i
[
2
i
_ _
_
. 5.17
Substituting the OLS estimators for o
2
and [
i
s in Eqs. (5.14) and
(5.17), we obtain operational biasing parameters of the almost unbiased
generalized ridge regression estimator and the almost unbiased general-
ized Liu estimator respectively.
6. NUMERICAL EXAMPLE
To illustrate our theoretical results in this last section we consider a
dataset which was discussed in Gruber (1998). Data found in economics
are often multicollinear. Table 6.1 gives Total National Research and
Development Expendituresas a Percent of Gross National Product by
Country: 19721986. It represents the relationship between the
dependent variable Y the percentage spent by the United States and the
four other independent variables X
1
, X
2
, X
3
, and X
4
. The variable
X
1
represents the percent spent by France, X
2
that spent by
West Germany, X
3
that spent by Japan, and X
4
that spent by the former
Soviet Union.
Table 6.1. Total National Research and Development
Expendituresas a Percent of Gross National Product
by Country: 19721986.
Year Y X
1
X
2
X
3
X
4
1972 2.3 1.9 2.2 1.9 3.7
1975 2.2 1.8 2.2 2.0 3.8
1979 2.2 1.8 2.4 2.1 3.6
1980 2.3 1.8 2.4 2.2 3.8
1981 2.4 2.0 2.5 2.3 3.8
1982 2.5 2.1 2.6 2.4 3.7
1983 2.6 2.1 2.6 2.6 3.8
1984 2.6 2.2 2.6 2.6 4.0
1985 2.7 2.3 2.8 2.8 3.7
1986 2.7 2.3 2.7 2.8 3.8
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

The model includes the intercept term. The matrix X
0
X has
eigenvalues z
1
312.932, z
2
0.75337, z
3
0.04531, z
4
0.03717,
z
5
0.00186. Observe that: the last three eigenvalues are very small.
The condition number is very large. K z
max
,z
min
168243. The
matrix X
0
X is almost singular, that is, det(X
0
X) 0.000739. The total
variance, o
2
trX
0
X
1
o
2
5
i1
1,z
i
587.569o
2
, which is 117 times
what it would be for an orthogonal system. The least squares estimates
of the regression coecients are:
b X
0
X
1
X
0
y
b
0
b
1
b
2
b
3
b
4
_
_
_
_
_
_
_
_
0.6921
0.6258
0.1154
0.2866
0.0256
_
_
_
_
_
_
_
_
_
Most authors recommend standardizing the data so that the X
0
X
matrix is in the formof a correlation matrix. An advantage of standardiza-
tion of the data is that the regression coecients will then be expressed in
comparable numerical units. The standardization is accomplished by
transforming the linear model y X[u to y
s
X
s
[
s
u. The standardi-
zed matrix X
s
and the standardized vector y
s
are
X
s

0.2164 0.5000 0.4845 0.02203
0.3828 0.5000 0.3814 0.0944
0.3828 0.1667 0.2783 0.5350
0.3828 0.1667 0.1752 0.0944
0.0499 0.0000 0.0722 0.0944
0.1165 0.1667 0.0309 0.2203
0.1165 0.1667 0.2371 0.0944
0.2829 0.1667 0.2371 0.7237
0.4494 0.5000 0.4433 0.2203
0.4494 0.3333 0.4433 0.0944
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
, y
s

0.2554
0.4256
0.4256
0.2554
0.0851
0.0851
0.2554
0.2554
0.4256
0.4256
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
_
.
Notice that to three decimal places
X
0
s
X
s

1.000 0.888 0.925 0.309
0.888 1.000 0.962 0.157
0.925 0.962 1.000 0.328
0.309 0.157 0.328 1.000
_
_
_
_
_
_
_
.
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

Another advantage of standardizing the X matrix is that it can be
seen which variables are highly correlated. The corresponding least
squares estimator is
b
s
X
0
s
X
s
1
X
0
s
y
s

0.6401
0.1180
0.4737
0.0139
_
_
_
_
_
_
.
and b
0
s
b
s
0.6482. For the standardized data since there are ten observa-
tions and four parameters, we obtain
^ oo
2
s

y
s
X
s
b
s
0
y
s
X
s
b
s
n p
0.003932.
The four eigenvalues of X
0
s
X
s
are 2.95743, 0.91272, 0.10984, and
0.02021. Since the smallest eigenvalue is not zero, the factors do dene
a 4-dimensional space in the mathematical sense. The 4 4 matrix T is
the matrix of normalized eigenvectors, is a 4 4 diagonal matrix
of eigenvalues of X
0
s
X
s
such that X
0
s
X
s
TT
0
. Then X
X
s
T and
T
0
[
s
[
so that y
s
X
s
[
s
u X
s
TT
0
[
s
u X
u and
[
0
[
[
0
s
[
s
, where
T
0.558 0.069 0.818 0.119
0.554 0.255 0.490 0.623
0.574 0.070 0.286 0.764
0.229 0.962 0.092 0.118
_
_
_
_
_
_
and
X
0
X

2.95743 0.00000 0.00000 0.00000
0.00000 0.91272 0.00000 0.00000
0.00000 0.00000 0.10984 0.00000
0.00000 0.00000 0.00000 0.02021
_
_
_
_
_
_
.
In orthogonal coordinates the least squares estimates of the regres-
sion coecients are:
^
[[

1
X
0
y
s

0.567064
0.034263
0.444731
0.357295
_
_
_
_
_
_
and
^
[[
0
^
[[
0.6482.
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

Using the values
^
kk
i
^ oo
2
,
^
[[
2
i
obtained by Hoerl and Kennard method,
the initial value of the K matrix is
^
KK
0.0122 0.0000 0.0000 0.0000
0.0000 3.3709 0.0000 0.0000
0.0000 0.0000 0.0199 0.0000
0.0000 0.0000 0.0000 0.0309
_
_
_
_
_
_
.
The corresponding operational generalized ridge regression
estimator is
[[
KK

KK
1
[[

0.5647
0.0073
0.3762
0.1411
_
_
_
_
.
Data-Based k
Using the Eqs. (5.3) and (5.4) estimators of k can be obtained for
ordinary ridge regression estimator. By using Eq. (5.3), we have
^
kk
h

^
kk
HKB
40.003932,0.6482 0.0243. Using Eq. (5.4) we get
^
kk
LW
0.0161.
Our objective is to compare the traces of the estimated mean square
error matrices of
^
[[
k
and
^
[[
d
. The trace of the mean square error matrix of
the ordinary ridge regression estimator
^
[[
k
is given by
mse
^
[[
k
trM
^
[[
k
, [
p
i1
z
i
o
2
k
2
[
2
i
z
i
k
2
and the trace of the mean square error matrix of the Liu estimator
^
[[
d
is
given by
mse
^
[[
d
trM
^
[[
d
, [
p
i1
z
i
d
2
o
2
1 d
2
z
i
[
2
i
z
i
1 z
i

2
By substituting [ and o
2
by their OLS estimates
^
[[ and ^ oo
2
, we get the
estimates for Eqs. (2.5) and (2.8) and their estimated mse values.
Numerical results are summarized in Tables 6.2 and 6.3. For practical
purposes let us consider the ordinary ridge regression (i.e.,
k
1
k
2
k
p
k): various values of k and the corresponding
ridge regression estimators are shown in Table 6.2 below.
Let us consider the ordinary Liu estimator (i.e., d
1
d
2

d
p
d): various values of d and the corresponding Liu estimators are
shown in Table 6.3 below.
Figure 1 shows the decrease (increase) in the squared length of the
coecient vector with k (with d).
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

In Fig. 2 is shown an estimated mse values of the estimators. From
Table 6.2, Table 6.3, and Fig. 2, we reach the following conclusions: we
have min( mse mse(
^
[[
k
))=0.1135346 for k= 0.0280204 and min( mse mse(
^
[[
d
))=
0.13842 for d 0.5709824.
Nonstochastic Selection of the Biasing Parameters
(i) Let d0.98 be xed. Using

kk
i
1 d z
i
,z
i
d ,
i 1, 2, 3, 4 in Theorem 3.1, we nd the following values of
kk
i
: 0.015; 0.0096; 0.0020; 0.0004. Comparing mse mse
^
[[
d0.98
with mse mse

^
[[
k0.0003
for k 0.0003- min
kk
i
0.0004 we see
that mse mse
^
[[
d0.98
0.2272 is smaller than mse mse
^
[[
k0.0003

0.2302 as stated in Theorem 3.1(a) (see also Fig. 2). As
for mse mse(
^
[[
d0.98
) with mse mse(
^
[[
k0.02
) for max
kk
i
- k 0.02,
we have the following inequality from Theorem 3.1(b):
mse mse
^
[[
d0.98
0.2272 > mse mse
^
[[
k0.02
0.1165. We have
mse mse
^
[[
d0.98
0.2272 mse mse
^
[[
kk
for k
1
0.00045 and
Table 6.2.
^
[[,
^
[[
0
K
k
^
[[
K
k and estimated mse
^
[[
K
values.
^
[[
k0
^
[[
k0.0161
^
[[
k0.0243
^
[[
k0.05
^
[[
k0.20
^
[[
k0.50
^
[[
k0.10
0.5671 0.5640 0.5625 0.5577 0.5311 0.4851 0.4238
0.0343 0.0337 0.0334 0.0325 0.0281 0.0222 0.0164
0.4447 0.3879 0.3641 0.305 0.1576 0.0800 0.0440
0.3573 0.1989 0.1622 0.1028 0.0328 0.0138 0.0071
^
[[
0
k
^
[[
k
0.6482 0.5092 0.4764 0.4160 0.3088 0.2424 0.1819
mse mse
^
[[
k
0.2360 0.1213 0.1141 0.1224 0.1992 0.2620 0.3062
Table 6.3.
^
[[
d
,
^
[[
0
d
^
[[
d
and estimated mse
^
[[
d
values.
^
[[
d0.01
^
[[
d0.05
^
[[
d0.20
^
[[
d0.50
^
[[
d0.95
^
[[
d0.98
^
[[
d0.10
0.4252 0.4309 0.4524 0.4954 0.5598 0.5641 0.5671
0.0165 0.0173 0.0199 0.0253 0.0334 0.0340 0.0343
0.0480 0.0640 0.1241 0.2444 0.4292 0.4367 0.4447
0.0106 0.0246 0.0791 0.1822 0.3398 0.3503 0.3573
^
[[
0
k
^
[[
k
0.1835 0.1907 0.2264 0.3390 0.6142 0.6328 0.6482
mse mse
^
[[
d
0.3004 0.2780 0.2088 0.1408 0.2147 0.2272 0.2360
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8


Ridge (k)
Liu (d)
Ridge
Liu
d d k k

'
'
0.10 0.30 0.50 0.70 0.90 0.00 0.20 0.40 0.60 0.80 1.00
0.10
0.30
0.50
0.70
0.00
0.20
0.40
0.60
0.80
Figure 1. Estimated squared length of coecient vector.
Ridge (k)
Liu (d)
Liu
Ridge
mse
0.10 0.30 0.50 0.70 0.90 0.00 0.20 0.40 0.60 0.80 1.00
0.13
0.18
0.23
0.28
0.33
0.10
0.15
0.20
0.25
0.30
0.35
Figure 2. Estimated mse values of the estimators
^
[[
k
and
^
[[
d
.
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

k
2
0.2977973, then mse mse
^
[[
d0.98
> mse mse
^
[[
k
for max
kk
i
-
k - k
2
.
(ii) Let k0.02 be xed. Using
dd
i
1 kz
i
,z
i
k, i 1, 2, 3, 4
in Theorem 3.2, we nd the following values of

dd
i
: 0.9734;
0.9590; 0.8290; 0.4924. In this case mse mse
^
[[
k
0.1165 -
mse mse
^
[[
d
0.2272 for 0 - max
dd
i
0.9734 - d 0.98 as
stated in Theorem 3.2 (a). Since the condition in Theorem 3.2
(b) does not hold, mse mse
^
[[
d0.40
0.1530 is not smaller than
mse mse
^
[[
k0.02
0.1165 for 0 - d 0.40 - min
dd
i
0.4924).
(iii) k 0.20 be xed. We nd
dd
i
1 0.20z
i
,z
i
0.20: 0.7493;
0.6562; 0.2836; 0.0734 using Theorem 3.2. mse mse
^
[[
k0.20

0.1992 and mse mse
^
[[
d0.80
0.1666 for d 0.80 > max
dd
i

0.7493. Since the condition in Theorem 3.2 (a) does not hold,
mse mse
^
[[
k0.20
is not smaller than mse mse
^
[[
d0.80
. We have
mse mse
^
[[
k0.20
0.1992 and mse mse
^
[[
d0.05
0.2780 for d
0.05 - min
dd
i
0.0734. Since the condition in Theorem 3.2
(b) does not hold, mse mse
^
[[
k0.20
mse mse
^
[[
d0.05
is not positive.
But we have mse mse
^
[[
k0.20
0.1992 mse mse
^
[[
dd
for d
1

0.2259032 and d
2
0.9088346. Then mse mse
^
[[
d-d
mse mse
^
[[
k0.20
and mse mse
^
[[
d>d
2
mse mse
^
[[
k0.20
is positive.
0.10 0.30 0.50 0.70 0.90 0.00 0.20 0.40 0.60 0.80 1.00
0.13
0.18
0.23
0.28
0.33
0.10
0.15
0.20
0.25
0.30
0.35
Ridge (k)
Liu (d)
AULiu
AURidge
mse
~
[[
k
and
~
[[
d
.
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

Thus mse mse
^
[[
k0.20
mse mse
^
[[
d
is positive for max
dd
i
- d -
d
2
0.9088. One can also make similar comparisons using
Theorems 4.1, 4.2, and Figs. 3 and 4.
ACKNOWLEDGMENT
The authors are thankful to an anonymous referee for the useful
comments and suggestions.
REFERENCES
Akdeniz, F., Erol, H., O
ztu rk, F. (1999). MSE comparisons of some

biased estimators in linear regression model. Journal of Applied
Statistical Science 9(1):7385.
Akdeniz, F., Kac iranlar, S. (1995). On the almost unbiased generalized
Liu estimator and unbiased estimation of the bias and
mse. Communications in StatisticsTheory and Methods 24(7):
17891797.
Ridge (k)
Liu (d)
AULiu
Ridge
mse
AURidge
Liu
0.10 0.30 0.50 0.70 0.90 0.00 0.20 0.40 0.60 0.80 1.00
0.13
0.18
0.23
0.28
0.33
0.10
0.15
0.20
0.25
0.30
0.35
^
[[
k
,
^
[[
d
,
~
[[
k
, and
~
[[
d
.
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

Farebrother, R. W. (1976). Further results on the mean square error
of ridge regression. Journal of the Royal Statistical Society B.38:
248250.
Firinguetti, L. (1999). A generalized ridge regression estimator and its
nite sample properties. Communications in StatisticsTheory and
Methods 28(5):12171229.
Gnot, S., Trenkler, G., Zmyslony, R. (1995). Non-negative minimum
biased quadratic estimation in the linear regression models.
Journal of Multivariate Analysis 4(1):113125.
Gruber, M. H. J. (1998). Improving Eciency by Shrinkage: The James
Stein and Ridge Regression Estimators. New York: Marcel Dekker.
Hemmerle, W. J., Brantle, T. F. (1978). An explicit and constrained
generalized ridge estimation. Technometrics 20:109119.
Hocking, R. R., Speed, F. M., Lynn, M. J. (1976). A class of biased
estimators in linear regression. Technometrics 18:425437.
Hoerl, A. E., Kennard, R. (1970a). Ridge regression: biased estimation
for non-orthogonal problems. Technometrics 12:5567.
Hoerl, A. E., Kennard, R. W. (1970b). Ridge regression: application for
non-orthogonal problems. Technometrics 12:6982.
Hoerl, A. E., Kennard, R., Baldwin, K. F. (1975). Ridge regression: some
simulations. Communications in StatisticsTheory and Methods
4(2):105123.
Kac iranlar, S., Sakalliog lu, S., Akdeniz, F., Styan, G. P. H., Werner, H. J.
(1999). A new biased estimator in linear regression and a detailed
analysis of the widely-analyzed dataset on Portlandcement.
Sankhya: The Indian Journal of Statistics 61(Series B):443459.
Lawless, J. F., Wang, P. (1976). A simulation study of ridge and other
regression estimators. Communications in StatisticsTheory and
Methods A5:307323.
Liu, Ke Jian. (1993). A new class of biased estimate in linear regression.
Communications in StatisticsTheory and Methods 22(2):393402.
McDonald, G. C., Galarneau, D. I. (1975). A Monte Carlo evaluation of
some ridge type estimators. Journal of the American Statistical
Association 70:407416.
Nomura, M. (1988). On the almost unbiased ridge regression estimator.
Communications in StatisticsSimulation and Computation 17:
729743.
Rao, C. R., Toutenburg, H. (1995). Linear Models Least Squares and
Alternatives. New York: Springer-Verlag.
Sakalliog lu, S., Kac iranlar, S., Akdeniz, F. (2001). Mean squared error
comparisons of some biased regression estimators. Communications
in StatisticsTheory and Methods 30(2):347361.
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8

Singh, B., Chaubey, Y. P., Dwivedi, T. D. (1986). An almost unbiased
ridge estimator. Sankhya: The Indian Journal of Statistics 48(Series
B):342346:
Stein, C. (1956). Inadmissibility of usual estimator for the mean of a
multi-variate normal distribution. Proceedings of the Third
Berkeley Symposium on Mathematical Statistics and Probability,
Berkeley: University of California Press, 197206.
Theobald, C. M. (1974). Generalizations of mean square error applied
to ridge regression. Journal of the Royal Statistical Society
B.36:103106.
Trenkler, G., Toutenburg, H. (1990). Mean squared error matrix
comparisons between biased estimators an overview of recent
results. Statistical Papers 31:165179.
Troskie, C. G., Chalton, D. O. (1996). A Bayesian estimate for the
constants in ridge regression. South African Statistical Journal
30:119137.
D
o
w
n
l
o
a
d
e
d

B
y
:

[
A
N
K
O
S

C
o
n
s
o
r
t
i
u
m
]

A
t
:

1
4
:
1
5

2
5

J
a
n
u
a
r
y

2
0
0
8


Akdeniz Erol 2003

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Akdeniz Erol 2003

Uploaded by

Copyright:

Available Formats

This article was downloaded by:[ANKOS Consortium]

On: 25 January 2008

, the squared length of

with mse mse

ztu rk, F. (1999). MSE comparisons of some

You might also like