IOSR Journal of Mathematics (IOSR-JM)

e-ISSN: 2278-5728, p-ISSN: 2319-765X. Volume 11, Issue 1 Ver. II (Jan - Feb. 2015), PP 24-33
www.iosrjournals.org

Bayesian Inferences for Two Parameter Weibull Distribution

Kipkoech W. Cheruiyot1, Abel Ouko2, Emily Kirimi3
1

Maasai Mara University, Kenya

Department of Mathematics and Physical Sciences
2
The East African University, Kenya
Department of Mathematics
3
Technical University of Kenya
Department of Mathematics and Statistics

Abstract: In this paper, Bayesian estimation using diffuse (vague) priors is carried out for the parameters of a
two parameter Weibull distribution. Expressions for the marginal posterior densities in this case are not
available in closed form. Approximate Bayesian methods based on Lindley (1980) formula and Tierney and
Kadane (1986) Laplace approach are used to obtain expressions for posterior densities. A comparison based on
posterior and asymptotic variances is done using simulated data. The results obtained indicate that, the
posterior variances for scale parameter obtained by Laplace method are smaller than both the Lindley
approximation and asymptotic variances of their MLE counterparts.
Keywords: Weibull distribution, Lindley approximation, Laplace approximation, Maximum Likelihood
Estimates

I.

Introduction

The Weibull distribution is one of the most widely used distributions in reliability and survival analysis
because of various shapes assumed by the probability density functions (p.d.f) and the hazard function. The
Weibull distribution has been used effectively in analyzing lifetime data particularly when the data are censored
which is very common in survival data and life testing experiments. The Weibull distribution was derived from
the problem of material strength and it has been widely used as a lifetime model. Weibull distribution
corresponds to a family of distribution that covers a wide range of distributions that goes from the normal model
to the exponential model making it applicable in different areas for instance in fatigue life, strength of materials,
genetic research, quality control and reliability analysis. The probability density function (p.d.f) for the two
parameter Weibull distribution is given by

x
fT ( x, , )

II.

exp

x0
(1.1)

Likelihood Based Estimation of Parameters of Weibull Distribution.

2.1 The Maximum Likelihood Estimation

Let X1 , X 2 ,..., X n be independent random samples of size n from Weibull distribution with the p.d.f given by
(1.1). Differentiating with respect to

and

n
n
n
dl
x
x
0 n ln ln( xi ) ( i ) ln( i ) 0

d
i
i 1
n n xi
dl
( ) 0
0
i 1
d

From (2.2) we obtained the maximum likelihood estimates of

(2.1)
(2.2)

as

1 n
x
n i i

and equating to zero we obtain and

(2.3)

Substituting (2.3) into (2.1), yields an expression in terms of

DOI: 10.9790/5728-11122433

only as given by

www.iosrjournals.org

24 | Page

Bayesian Inferences for Two Parameter Weibull Distribution

x ln x
i

i 1

x
i 1

1 n
ln xi
n i 1

(2.4)

The maximum likelihood estimate for

is obtained from (2.4) with the aid of standard iterative procedures.

2.2 Variance and Covariance Estimates

The asymptotic variance-covariance matrix of and are obtained by inverting information matrix with
elements that are negatives of expected values of second order derivatives of logarithms of the likelihood
functions. Cohen (1965) suggested that in the present situation it is appropriate to approximate the expected
values by their maximum likelihood estimates. Accordingly, we have as the approximate variance-covariance
matrix with elements
1

2l
2

2l

When

and

2l

var(

)
cov(

2l
2
cov( ) var( )

are independent, the covariance of the above matrix is zero.

When

is known the asymptotic variances for is obtained by

2
var( ) 2
n
III.

Bayesian Estimation of Parameters of Weibull Distribution

3.1 Bayesian Approach.

Let X1 , X 2 ,..., X n be a random sample from a population with a two parameter Weibull distribution given by

f (x / , ) =

x 1
x
( ) exp {-( ) }

(3.1)

The likelihood function is the given by

n

L( , ; x) n n xi 1 exp{ ( i ) }
x

i 1

i 1

The log likelihood function is given by

n

i 1

i 1

l log L n log n log ( 1) log xi ( i )

x

Suppose that we are ignorant about the parameters ( , ) so that the diffuse (vague) prior used is

( , )

(3.2)

The joint posterior distribution is then given by

g ( , / x) L( x / ) ( , )
n
1 n n n
x
g ( , / x)

exp{

( i ) }

i 1
i 1
The marginal p.d.f of x is given by

f ( x)

x
n n
0 1 exp{ ( ) }d d
i

i 1

Similarly, the marginal posterior p.d.fs of

posterior expectations of and as
DOI: 10.9790/5728-11122433

(3.3)

i 1

and

are

(3.4)

required in order to compute the corresponding

www.iosrjournals.org

25 | Page

Bayesian Inferences for Two Parameter Weibull Distribution

E ( / x) g ( / x)

L( x / ) ( )
L( x / ) ( )

(3.5)

and

E ( / x) g ( / x)

L( x / ) ( )
L( x / ) ( )

(3.6)

respectively
Since the Bayes estimate for and involve evaluating ratios of two mathematically intractable integrals,
appropriate Bayesian approximations are applied. Assuming is known, the likelihood function for is given
by
n

L( x / ) n exp{ ( i ) }
x

i 1

The log likelihood function for

(3.7)

is given by
n

l log L n log ( i )
x

i 1

Since

is known, the prior density for

is given by

1
( )

The posterior density for

is given by

g ( / x) L( x / ) ( )
n
1
g ( / x) ( ) n exp{ xi }

i 1

is obtained by
E ( / x) g ( / x)d

Therefore the posterior expectation for

E ( / x)

L( x / ) ( )
(3.8)

L( x / ) ( )

3.2 Laplace Approximation

Since the Bayes estimate of involve ratio of two mathematically intractable integrals, Tierney and Kadane,
(1986) proposed to estimate (3.8) as follows

e
E ( / x)
e

nL

nL

(3.9)

where

L log ( )

l
n

n
1
x
log( ) log 1n ( i )

(3.10)

i 1

and

L log ( ) log
DOI: 10.9790/5728-11122433

l
n
www.iosrjournals.org

26 | Page

Bayesian Inferences for Two Parameter Weibull Distribution

n
1
x
log log( ) log 1n ( i )

(3.11)

i 1

of L is obtained by differentiating L with respect to once and equating to zero, that is,

The posterior mode

dL
0
d
1

n

( ) 0
xi

i 1

Giving

in terms of as
1

xi

n(1 ) i 1

The posterior mode (local maximum) for

to zero to get

L is obtained by differentiating L with respect to

log log( ) log

1
n

(3.12)
and equating

( )
xi

i 1

( x ) 0
n i 1
n

Giving

in terms of as
1

1 n

xi
n i 1

The and
mode given by
2

(3.13)

are equal to the minus the inverse of the second derivative of the log posterior density at its

L" ( )

d 2 L 1 ( 1) n xi
2
( )
d 2

n 2 i 1

1 ( 1) n xi
L" ( ) 2
( )

n 2 i 1
Hence
2

n
n ( 1) xi
i 1

2
1
( 1)

(3.14)

Also

d 2 L

d 2

n ( 1) xi

12
DOI: 10.9790/5728-11122433

i 1

( 2)

L" ( )
www.iosrjournals.org

27 | Page

Bayesian Inferences for Two Parameter Weibull Distribution

2

12

1 n
n xi
i 1

(3.15)

Thus, the Laplace approximation of (3.8) is given by

E ( / x) ( ) exp{n( L ( ) L( ))}

(3.16)

where
1

xi

n(1 ) i 1

and
1

1 n

xi
n i 1

2
L( x / ) ( )
2
E ( / x)
L( x / ) ( )

E (

e
/ x)
e

nL

nL

(3.17)

where

1
1 n x
L log( ) log ( i )

n i 1
and

1
1 n x
L log 2 log( ) log ( i )

n i 1

The posterior mode of L and L are given by

1

1
x
i
n(1 ) i 1

(3.18)

and
1

1
x
i
n( 1) i 1

(3.19)

respectively
The

12 and 12 of L and L respectively are given by

1
12
L" ( )

1 ( 1) n xi
L" ( ) 2
( )

n 2 i 1

Hence

DOI: 10.9790/5728-11122433

www.iosrjournals.org

28 | Page

Bayesian Inferences for Two Parameter Weibull Distribution

2

n
n ( 1) xi
i 1

2
1
( 1)

(3.20)

And

12

L1" ( )
1 ( 1) n xi
L1"
( )

n 2 i 1

Hence
2

12

x
n( 1) i
i 1

( 1)

(3.21)

Thus, the Laplace approximation of (3.17) is

E ( 2 2 / x) ( ) exp{n( L (1 ) L(1 ))}

Hence the posterior variance of

(3.22)

is

V ( / x) E ( 2 / x) [ E ( / x)]2

(3.23)

3.3 Lindley (1980) Approximation

Lindley (1980) developed a multidimensional linear Bayes estimate of an arbitrary function as an
approximation of an asymptotic expansion of the ratio of two integrals which cannot be expressed in a closed
form given by

E U ( ) / x

U ( )V ( ) exp L( ) d

V ( ) exp L( ) d

Expanding L( ) and U ( ) V ( ) by Taylors series about the MLE of , Lindley (1980) Bayesian
approximation of two parameter case is given by

1
1
E U ( ) / x U (U ij 2U j j ) Lijkl ijklU terms of order n 2 and smaller.
2 i j
2 i j k l

1
U U11 11 U12 12 U 21 21 U 22 22 1 U1 11 U 2 21 2 U 2 22 U1 12
2
2
2
2
1 L30 (U1 11 U 2 11 12 ) L21[3U1 11 12 U 2 ( 11 22 2 12 )] L12 [3U 2 22 21 U1 ( 22 11 2 21 )]

2 L03 (U1 12 22 U 2 222 )

all evaluated at MLE of

(3.24)
(see, Lindley, 1980). For two parameter Weibull distribution we have,

( , ) is ( , )
( ) log V ( ) log ( )
i j L
Lij
i j
U
U
2U
Ui
; Uj
and Uij

The MLE of

DOI: 10.9790/5728-11122433

www.iosrjournals.org

29 | Page

Bayesian Inferences for Two Parameter Weibull Distribution

ij (i, j )th

element in the inverse of matrix { Lij } evaluated at

( , ), i, j 1, 2

The quantities Lij ' s are the higher order derivatives of log-likelihood function given by

l
n n xi

( ) L10

i 1

(3.25)

l n ( 1)
2 2 ( xi ) L20
2

i 1
2

(3.26)

l
2n
3 ( 1)(3 2) ( xi ) L30
3

i 1
3

(3.27)

l
n
( xi ) ln( xi ) 1 ( xi ) L11

i 1
i 1
2

(3.28)

l
n
2 2 ( xi ) (21) ( xi ) ln( xi ) ( 21) ( xi ) L21
2

i 1
i 1
i 1
3

(3.29)

l
2
2 ( xi ) 42 ( xi ) ln( xi ) 22 ( xi ) ln( xi ) 2 ( xi ) ln( xi ) 2 ( xi ) ln( xi ) L22
2
2

i 1
i 1
i 1
i 1
i 1
4

l
n
2 ( xi ) ln( xi )
2

i 1

L02

(3.32)

(3.31)

3
3l 2n n xi
xi

ln(
)
L03

3 3 i 1
n
n
2
3l
xi
xi
xi
xi

(
)
ln(
)

( ) ln( ) L12
2

i 1
i 1

(3.30)

n
n
l n
x
x
n ln ln( xi ) ( i ) ln( i ) L01

i
i
2

(3.33)

(3.34)

Fishers information matrix is given by

L
I 20
L11

L11

L02

Thus

I 1

L20
1

L20 L02 L11L11 L11

L11 11 12

L02 21 22

(3.35)

where
n
n
n n
x
L20 L02 L11L11 2 (21) ( i ) 2
i 1
i 1

xi

2
n n xi xi 1 n xi
(ln( )) ( ) ln( ) ( )
i 1
i 1


xi

Next, we have

log ( ) ln( )

Hence

1
Since

is known, we let

and 2

DOI: 10.9790/5728-11122433

www.iosrjournals.org

30 | Page

Bayesian Inferences for Two Parameter Weibull Distribution

such that

1 ; U2
0 and U ij 0 ; i,j=1,2

The quantities Lij ' s are the higher order derivatives of log-likelihood function. Because is known, the
U1

following derivatives are used to obtain Bayes estimates of

l
2n
x
3 ( 1)(3 2) ( i ) L30
3

i 1
3

ij (i, j )th

is obtained by inverting minus second derivative of log likelihood function with respect to

evaluated at ( ), i 1, 2
Therefore, Bayes estimate of using (3.24) is given by

1
E ( / x) 11 L30 112
2

Then

(3.36)

E ( 2 / x) 2 2 11 L30 112 11

Hence the posterior variance for

(3.37)

is given by

Var ( / x) E 2 / x E ( / x)

IV.

(3.38)

Results

Simulation experiments were carried out using R software to compare the performance of the Bayes
and MLE estimates of the two parameter Weibull distribution. We assumed the shape parameter is known.
We performed simulation experiment with different sample sizes (10, 30, 50 and 100), drawn from a Weibull
distribution for different values of shape parameter (1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5 and 6). We specified
the true value of the scale parameter to be fixed at one ( 1) for all the sample sizes. We computed
estimates based on two different Bayesian methods, that is, Tierney and Kadane, (1986) Laplace approximation
method, Lindley (1980) approximation and the method of Maximum likelihood estimation.

Figure 1: Graph of Estimates of

DOI: 10.9790/5728-11122433

against Lambda ( ) for a Sample of n=10 for affixed

www.iosrjournals.org

1
31 | Page

Bayesian Inferences for Two Parameter Weibull Distribution

Figure 2: Graph of Estimates of

against Lambda ( ) for a sample of n=100 for affixed

Figure 3: Graph of variance for sample size 10 against values of Lambda ( ).

Figure 4: Graph of variance for sample size 100 against values of Lambda ( ).
DOI: 10.9790/5728-11122433

www.iosrjournals.org

32 | Page

Bayesian Inferences for Two Parameter Weibull Distribution

V.

Discussions

Figure 1 show the estimates of under varying size of , Bayes estimates obtained by Lindley (1980)
and Tierney and Kadane, (1986) Laplace approximation are found to be larger than the MLE counter parts.
Lindley (1980) approximation is found to overestimate the scale parameter ; however the three methods
demonstrate the tendency for their estimates and variances to perform better for larger sample size. This is
shown in Figure 2 when the sample size is increased to 100. As the sample size increases the MLE and Bayes
estimates becomes more consistent and accurate.
Figure 3 show the variances of the estimates of for varying size of shape parameter for a sample of
10. It is observed that posterior variances of estimates of are smaller than the asymptotic variances of MLE
hence more precise and accurate. However, both variances tend to converge to zero as the value of get larger.
For the sample of 10 Tierney and Kadane, (1986) Laplace approximation is seen to perform slightly better than
Lindley (1980), since it stabilizes faster.
Figure 4 show that Tierney and Kadane, (1986) Laplace approximation performed better for larger
sample of 100. The variance of MLE are observed to stabilize faster when the sample size and shape parameter
increase.
It is also observed that the Bayesian method generally performed better for both small and larger value
of than the MLE counter parts. Lindley (1980) and Laplace methods tend to perform almost similarly for
smaller . But the Tierney and Kadane, (1986) Laplace approximation is found to produce better results than
both MLE and Lindley (1980) for larger

and larger samples sizes.

VI.

Conclusion

We have shown Bayesian techniques for estimating the scale parameter of the two parameter Weibull
distribution which produces estimates with smaller variances than the MLE. Tierney and Kadane, (1986)
Laplace approximation which requires the second derivatives in its computation is found to be more accurate
than the Lindley (1980) which requires third derivatives in its computation. This is in line with Tierney et al
(1989) findings, that Laplace method is more accurate than the third derivative method of Lindley (1980). Even
though the two Bayesian methods are better than the MLE counter parts, they have their own limitations.
Lindley (1980) approximation requires existence of MLE in its computation. This appears as if it is an
adjustment to the MLE to reduce variability. On the other hand, Laplace approximation requires existence of a
unimodal distribution in its computation, hence difficult to use in cases of a multi modal distribution.

VII.

Recommendation

In this study, it is noted that the posterior variances of Bayes estimates are smaller than asymptotic
variances. Comparing the two Bayesian methods, Tierney and Kadane, (1986) Laplace approximation method
has smaller variance than the Lindley (1980) approximation technique hence more precise and accurate. Laplace
approximation does not require explicit third order derivatives in its computation which are required in Lindley
(1980) approximation method hence simple to compute. We therefore recommend further work to be done on
two parameter Weibull distribution when both scale and shape parameters are unknown to investigate accuracy
of the two Bayesian methods.

References
[1].
[2].
[3].
[4].

Cohen, A.C. (1965): Maximum Likelihood Estimation in the Weibull Distribution Based on Complete and Censored Samples.
Technometrics, 7, 579-588.
Lindley, D.V. (1980). Approximate Bayesian Method, Trabajos Estadistica, 31, 223-237.
Tierney L, Kass, R.E. and Kadane, J.B. (1989): Fully exponential Laplace approximations to expectations and variances of nonpositive functions. Journal of American Statistical Association, 84, 710-716.
Tierney L. and Kadane, J.B. (1986): Accurate Approximations for Posterior Moments and Marginal Densities. Journal of American
Statistical Association, 81, 82-86.

DOI: 10.9790/5728-11122433

www.iosrjournals.org

33 | Page