Pitman estimators: an asymptotic variance revisited

Alexander Novikov, University of Technology, Sydney Nino Kordzakhia, Macquarie University, Sydney

Abstract. We provide an analytic expression for the variance of ratio of integral functionals which arises as an asymptotic variance of Pitman estimators for a location parameter of independent identical distributed observations. The expression is obtained in terms of derivatives of a logarithmic moment of the integral functional of Limit Likelihood Ratio Process (LLRP). In the particular case, when the LLRP is a geometric Brownian motion we show that the established expression leads to the representation of the asymptotic variance of Pitman estimator in terms of the Riemann zeta-function.

1. Introduction. The Pitman estimator (Pitman (1939) [?]) for a location parameter of independent random variables (r.v.) i , i = 1, ..., n with a density f (x) = f (x ) has the form (n) n uLu du n) L( := f (i u). (1) n = (n) , u Lu du i=1

Existence and admissibility of Pitman estimators have been discussed in [?]. The integrals in (??) exist for sucently large n under (for example) the condition E |i |p < for some p > 0. An exposition of some important propertes of Pitman estimators is presented in the texbook by Borovkov [?], see also [?]. The form of a limit distribution for the Pitman estimator as n has been derived by Ibragimov & Hasminskii (1970) [?], (1981) [?], who have shown that under some regularity assumptions uLu du d n ) as n , n ( = Lu du

where Lu is a limit likelihood ratio process (LLRP) and the parameter depends on a form of discontinuities of derivatives of the underlying density f (x). The structure of the LLRP Lu , or equivalently, the structure of the process Yu := log(Lu ) has been described by Ibragimov & Hasminskii [?], [?] in several important cases. In particular, it has been shown in [?] that under some regularity conditions 1 H Yu = W u |u|2H , 2 1 (2)

H where Wu is a fractional Brownian motion (fBm) with the Hurst parameter H (0, 1], H H E (Wu ) = 0, E |Wu WsH |2 = |u s|2H , u R = (, ), s R. 1 The form (??) for Yu with H ( 2 , 1] appears, for example, when f (x) is a density of Weibull-like distributions, see monographs by Ibragimov & Hasminskii (1981) [?], Kutoyants (2004) [?]. The r.v. with Yu specied in (??) appears also as a limit in a change 1 point problem for a Brownian motion (the case H = 2 , [?], [?]) and in a cusp estimation 1 problem in ergodic diusion processes (H (0, 2 ), Fujii (2010) [?]). Note the distribution of the random variable is unknown in an explicit form even for the case of a standard . Brownian motion i.e. when H = 1 2 In this paper we discuss the problem of calculation of the variance V ar( ) only for the 1 case (??) with H [ 2 , 1]. However, we expect that our method (see details in Section 2 and Remarks below) may be used to nd analytical or numerical expressions for V ar( ) also in other limit schemes (for example, in the case when Yu is a Levy process, see (??) below). in (??) was studied by several authors. It was rstly Ibragimov The special case H = 1 2 & Hasminskii (1975) [?] who obtained through simulation that

V ar( ) = 19.5 0.5 . Golubev (1979) [?] using the Feyman-Kac formulae found an analytical expression for V ar( ) in terms of integrals of products of Bessel functions and then using numerical integration obtained V ar( ) = 19.276 0.06 . Rubin & Song (1995) [?] using some of results by Golubev (1979) [?] and Ibragimov & Hasminskii (1975) [?] found that V ar( ) = 16Zeta[3] = 19.2329... (3)

where Zeta[k ] is the Riemann-zeta function. 1 , 1] in (??) the In this paper we show (see Theorem 2 in Section 2) that for the case H [ 2 value of V ar( ) can be found in terms of derivatives of the expectation of log( 0 (em1 u Lu + em2 u Lu )du+ )) with respect to parameters m1 and m2 . Then we illustrate the application 1 (see Section 3) where the distribution of functionals of this result in the case H = 2 mu and other similar cases which e Lu du is well known. The general case with H > 1 2 0 involve Levy processes are still under progress (see Remarks below). Before proving the main result in Theorem 2 we present the following theorem about existence of exponential moments of r.v. | |2H . Theorem 1. If H > 541 then r.v. | |2H is exponentially bounded i.e. there exists constant > 0 such that E exp{| |2H } < . This result and some elements from its proof will be used below in the proof of Theorem 2. Proof. Set qt =

Lt Lu du

Since qt is a (posteriori) density function, by convexity arguments (Jensens inequality) we have for any c 0

exp{| |

2H

} c +
1 ). 2

e|t| qt dt,

2H

(the constant c = 0 for the case H E exp(| | Note

0 1 2H

Due to symmetry of WsH we have also

) c + 2
0

et E (qt )dt.

2H

e This implies

t2H

E (qt )dt e E
0

qt dt = e .

E exp(| |2H ) c + e + 2
1

et E (qt )dt.

2H

To estimate E (qt ) note

t

Lu du
0 0

H exp(Wu

u2H 1 }du t exp{ 2 t

t

t H (Wu 0

u2H )du}, t > 0, 2 u2H )du}. 2

(4)

and hence qt Lt
t 0

Lu du

1 t2H 1 exp{ t 2 t

H WtH (Wu 0

Thus we obtain for t 1

1

t2H

E (qt )dt
1

t2H

t2H 1 E exp{ 2 t

t H (Wu WtH 0 t

u2H )du}dt = 2

exp{t
1

2H

H 1 t2H }E exp{ 2H + 1 t

H Wu du}dt, 0 d

H WtH = WtH where the last equation is due to the property Wu u , u [0, t]. t H The random process 0 Ws ds is a zero-mean Gaussian process with the variance t t

V ar(
0

WsH ds) = E (
0

WsH ds)2 =

1 2

t 0 0

[s2H + u2H |u s|2H ]dsdu =

t2H +2 . 2H + 2

Therefore
1

et E (qt )dt
1

2H

exp{t2H exp{t2H t2H (

H 1 t2H + 2 V ar( 2H + 1 2t

WsH ds)}dt
0

=
1

H 1 1 )}dt, 2H + 1 2 2H + 2

which is nite for some positive when < H 1 1 4H 2 + 2H 1 = 2H + 1 2 2H + 2 2(2H + 2)(2H + 1) 3

It remains to note that the 4H 2 + 2H 1 is positive for H > 541 = 0.309... . 2. The main result. Recall that we discuss here the case that the process Yu = log(Lu ) 1 has the representation (??)), H [ 2 , 1]. For the proof of Theorem 2 we use the following representation: 1 V ar( ) = E ( 2
u2 Lu du ). L du u

(5)

This was proved by Golubev [?] for the case H [ 1 , 1] in (??). 2 It will be convenient to use the following parametrised processes:

1 (m) =
0

emu Lu du, 2 (m) =

0

emu Lu du,

1 (m) =
0

uemu Lu du, 2 (m) =

0

uemu Lu du,

where m 0 is an auxiliary parameter. With use of these notations we obtain

Lu du = 1 (0) + 2 (0),

uLu du = 1 (0) 2 (0),

i (m) = and =
d

i (m), i = 1, 2 m (6)

1 (0) 2 (0) . 1 (0) + 2 (0)

H H = W Note that since Wu u (in distribution) we have

1 (0) = 2 (0), 1 (0) = 2 (0). This implies that has a symmetric distribution and E ( ) = 0. To formulate the main result we use the following function

(7)

g (m1 , m2 ) := E (log(1 (m1 ) + 2 (m2 ))) = E (log(

0

(em1 u Lu + em2 u Lu )du))

where m1 > 0 and m2 > 0 are parameters. Theorem 2. The function g (m1 , m2 ) is twice continuously dierentiable and V ar( ) = 2 lim
m1 0,m2 0

2 g (m1 , m2 ) 2 g (m1 , m2 ) ]. m2 m1 m2 1

Proof. First note that the function g (m1 , m2 ) is nite because due the inequality log(x) x

g (m1 , m2 ) E
0

(em1 u Lu + em2 u Lu )du) = 4

1 1 + < m1 m2

and, obviously, g (m1 , m2 ) > due to the estimate (??). The variance V ar( ) = E ( 2 ) < (see e.g. Theorem 1) and in view of (??) we have V ar( ) = E ( = 2E ( 1 (0) 2 (0) 2 ) = 1 (0) + 2 (0) (8)

1 (0)2 (0) 1 (0)2 2 (0)2 ) + E ( ) + E ( )]. (1 (0) + 2 (0))2 (1 (0) + 2 (0))2 (1 (0) + 2 (0))2

Note that the last two terms are equal due the property of symmetry (??). To nd the rst expectation in the RHS of (??), rstly note that the random functions i (m) are continuously dierentiable. By direct calculations we obtain Q(m1 , m2 ) := 2 (log(1 (m1 ) + 2 (m2 )) 1 (m1 )2 (m2 ) . = (1 (m1 ) + 2 (m2 ))2 m1 m2 (9)

(recall we assume m1 > 0, m2 > 0). Applying the Holder inequality we have Q(m1 , m2 ) ( 1 (m1 ) 2 2 (m2 ) 2 ) +( ). 1 (m1 ) 2 (m2 ) (10)

Now we show that the right-hand side of this inequality is uniformly bounded by an integrable random variable. We shall use some of calculations presented above in the prove of Theorem 1. Set emt Lt pt (m) = emu Lu du
0

Since pt (m) is a density function, by CauchyBunyakovskySchwarz inequality we have 1 (m) 2 ) ( 1 (m)

t2 pt (m)dt
0

To estimate pt (m) we apply the following inequalities pt (m) and thus obtain

emt Lt
t mu e Lu du 0

t exp{ 1 t
t

Lt t H (Wu 0

u2H )du} 2

t2 pt (m)dt
0 0

t exp{

t2H 1 + 2 t
t

H + (WtH Wu 0

u2H )du}dt = 2

t exp{

H 1 t2H + 2H + 1 t

H (WtH Wu )du}dt =: Z . 0

2 (m) 2 The similar estimate is valid for the second term ( ) in (??). As seen from the proof 2 (m) of Lemma 1, H 1 1 E (Z ) = t exp{t2H ( )}dt < 2H + 1 2 2H + 2 0

1 at least for the case H 2 . This implies that the random function Q(m1 , m2 ) is unformly bounded by an integrable r.v. which does not dependent of parameters m1 and m2 . Applying the expectation to the both parts of (??) and well-known theorems about differentiability of expectations over parameters we can conclude that the function g (m1 , m2 ) has a continuous mixed derivative and

E (Q(m1 , m2 )) = E (

1 (m1 )2 (m2 ) 2 g (m1 , m2 ) . ) = (1 (m1 ) + 2 (m2 ))2 m1 m2

(11)

Finding the second term in (??) is similar but a little bit more tricky. One can check by direct calculations that u2 em1 u Lu du 2 log(1 (m1 ) + 2 (m2 )) 1 (m1 )2 0 = . + m2 (1 (m1 ) + 2 (m2 ))2 1 (m1 ) + 2 (m2 ) 1 This implies 2 g (m1 , m2 ) 2 log(1 (m1 ) + 2 (m2 )) = E ( )= m2 m2 1 1 2 m1 u ue Lu du 1 (m1 )2 0 = E ( ) + E ( ). (1 (m1 ) + 2 (m2 ))2 1 (m1 ) + 2 (m2 ) Now combining all above equations we obtain E( 1 (m1 ) 2 (m2 ) 2 2 g (m1 , m2 ) 2 g (m1 , m2 ) ) = 2 + 2 1 (m1 ) + 2 (m2 ) + m2 m1 m2 1 2 m1 u ue Lu du +2E ( 0 ). 1 (m1 ) + 2 (m2 ) + 1 (m1 ) 2 (m2 ) 2 1 (m1 ) 2 2 (m2 ) 2 ) 2( ) + 2( ), 1 (m1 ) + 2 (m2 ) 1 (m1 ) 2 (m2 )

Note (

where as shown above the right-hand side is unformly bounded by an integrable r.v. which does not dependent of parameters m1 and m2 . By Lebesgues theorem on majorised convergence and the theorem on monotone convergence, we obtain V ar( ) = lim
m1 0,m2 0

E(

1 (m1 ) 2 (m2 ) 2 ) = 1 (m1 ) + 2 (m2 )

lim
m1 0,m2 0

u2 Lu du 2 g (m1 , m2 ) 2 g (m1 , m2 ) 0 ) + 2 E ( ). m2 m1 m2 1 (0) + 2 (0) 1

(12)

On the other side, due to the result of Golubev (??) and the symmetry of fBm, the formula in (??) can be rewritten as follows u2 Lu du V ar( ) = E ( ). 1 (0) + 2 (0) 6
0

To complete the proof we need only to substitute the last expression in (??) and thus we have proved Theorem 1. Remarks. 1. Another form of the limit process Lu or, equivalently, the process Yu appears in the change-point problems for Poisson processes (see e.g. Dachian and Negri (2011) [?]) and autoregressive processes (Chan & Kutoyants (2010) [?]). In those settings the authors obtained the following representation
+ Yu = Ju I {u 0} + Ju I {u < 0},

(13)

+ are independent Levy processes with some specic Levy measures for and Ju where Ju jumps. In particular, as shown in [?],

d Ju =

+ Ju

1 k , k N ( , 1). 2 k=1

Nu

The representation (??) holds also for the case (??) under the condition E (ezJu ) = E (e(1z)Ju ) for all z (0, 1) (the private communication of A. Gushchin, 2011). 2. One can see that the RHS of (??) can be written in the following form E (log( 1 V ar( ) = lim 2 m0
2 emu Lu du)) +

.
2

Thus V ar( ) can be found when the expectation E (log( emu Lu du)) is known in an explicit form as a function of m. To our knowledge there are not results of this type in the current literature. Contrary, there are many results on distributions of integral exponentials like 0 emu+Ju du where Ju is a Levy process, see e.g. [?], [?], [?]. 3. Other representations for V ar( ) can be given under some additional assumptions. For example, if for some > 0

E log(
0

eu Lu du) < 2 E (log(

emu Lu du)) . m2

(14)

then V ar( ) = lim

m0

(15)

The proof of this formula can be obtained similarly to the proof of Theorem 2. 1 3. Example. Calculation of V ar( ) for the case of H = 2 . We present here an elementary derivation of the explicit formula (??) based on the result of Theorem 2 and the well-known result on the distribution of exponential functional of a standard Bm. For the case under consideration

1 (m) =
0

eWu 7

1/2

(1/2+m)u

du

and

1 P( < x) = 1 (m)

x 0

2(2y )2m 2y e dy (2m + 1)

i.e. 1 (m) has the reciprocal Gamma distribution: 1 Gamma[2m + 1, 2] 1 (m) (16)

(see e.g. Dufresne (1990) [?] ). Note that due to the properties of a standard Bm, the random variables 1 (m1 ) and 2 (m2 ) are independent. To nd the function g (m1 , m2 ), note

E (log(1/1 (m))) =
0

log(x)

2(2x)2m e2x dx = (0, 2m + 1) log(2) (2m + 1)

for any m 0 (this can be checked with Mathematica@ ) where the function (n, z ) is the (n + 1)th derivatives of logarithm of the Gamma function (z ): (n, z ) = P olyGamma[n, z ] = dn+1 log((z )). dz n+1

Due to (??) and the fact that sums of independent Gamma r.v.s is a Gamma r.v., we have 1/1 (m1 ) + 1/2 (m2 ) = Gamma[2m1 + 2m2 + 2, 2] and hence E (log(1/1 (m1 ) + 1/2 (m2 ))) = (0, 2m1 + 2m2 + 2) log(2). Hence E (log(1 (m1 ) + 2 (m2 ))) = E (log(1/1 (m1 ) + 1/2 (m2 ))) E (log(1/1 (m1 )) E log(1/1/2 (m2 ))) = (0, 2m1 + 2m2 + 2) (0, 2m1 + 1) (0, 2m2 + 1) + log(2). Now we are ready to calculate V ar( ) using Theorem 1 with = 0. We have 2 E (log(1 (m1 ) + 2 (m2 ))) = 4(2, 2m1 + 2m2 + 2) 4(2, 2m1 + 1), 2 m1 2 E (log(1 (m1 ) + 2 (m2 ))) = 4(2, 2m1 + 2m2 + 2), m1 m2 hence 2 g (m1 , m2 ) 2 g (m1 , m2 ) V ar( ) = 2[ ]| = 8(2, 2)8(2, 1)8(2, 2) = 8(2, 1). m2 m1 m2 m1 =0,m2 =0 1 It remains to note that (k, 1) = (1)k+1 k !Zeta[k + 1] 8
d

for any k=1,2,.. (formula 6.4.2 in Abramowitz & Stegan (1972) [?]) and so V ar( ) = 8(2, 1) = 16Zeta[3]. Remark. One can check that condition (??) does hold and so it is also possible to obtain the last result using formula (??).

Acknowledgment. The authors are thankful to Yuri Kutoyants for the suggestion to study the problem of calculations of asymptotic variances of Pitman estimators. The authors are indebted to Alexander Gushchin for discussions and suggestions which allowed to improve the presentation of the paper.

References
[1] Abramowitz, M. and Stegun, I.: Handbook of Mathematical Functions, Tenth Printing, 1972. [2] Bertoin, J. and Yor, M.: Exponential functionals of Levy processes. Probab. Survey. 2, 191- 212, 2005. [3] Borovkov, A. A.: Mathematical statistics. Gordon and Breach Science Publishers, Amsterdam, 1998. ISBN: 90-5699-018-7 [4] Carmona, P. Petit, F. and Yor, M. : Exponential functionals of Levy processes. Levy processes, 4155, Birkhauser Boston, Boston, MA, 2001. [5] Chan, H. and Kutoyants, Y. : On Threshold Estimation for Some TAR Models, preprint, 2010. [6] Dachian, Sergue and Negri, Ilia.: On compound Poisson processes arising in changepoint type statistical models as limiting likelihood ratios. Stat. Inference Stoch. Process. 14 (2011), no. 3, 255271. [7] Dufresne, D. : The distribution of a perpetuity, with applications to risk theory and pension funding. Scand. Actuar. J. 1990, no. 1-2, 3979. [8] Fujii, T.: An extension of cusp estimation problem in ergodic diusion processes. Statist. Probab. Lett. 80 (2010), no. 9-10, 779 783. [9] Gjessing, H. and Paulsen, J.: Present value distributions with applications to ruin theory and stochastic equations. Stochastic Process. Appl. 71 (1997), no. 1, 123144. [10] Golubev, G. K. : Computation of eciency of maximum-likelihood estimate when observing a discontinuous signal in white noise. (Russian) Problems Inform. Transmission 15 (1979), no. 3, 6169. 9

[11] Ibragimov, I. and Hasminskii, R. : The asymptotic behavior of generalized Bayesian estimates, Dokl. Akad. Nauk SSSR, v. 194 1970, 257260. [12] Ibragimov, I. and Hasminskii, R. : Estimation of the parameter of a discontinuous signal in Gaussian white noise. (Russian) Problemy Peredaci Informacii 11 (1975), no. 3, 3143. [13] Ibragimov, I. and Hasminskii, R. : Statistical estimation. Asymptotic theory. SpringerVerlag, New York-Berlin, 1981. [14] Kutoyants, Y. : Statistical inference for ergodic diusion processes. Springer Series in Statistics. Springer-Verlag London, Ltd., London, 2004. [15] Pitman, E.J.G.: The estimation of the location and scale parameters of a continuous population of any given form. Biometrika 30, (1939) 391-421 [16] Port, S. C. and Stone, C. J.: Fisher information and the Pitman estimator of a location parameter. Ann. Statist. 2 (1974), no. 2, 225247. [17] Rubin, H. and Song, K.: Exact computation of the asymptotic eciency of maximum likelihood estimators of a discontinuous signal in a Gaussian white noise. Ann. Statist. 23 (1995), no. 3, 732739. [18] Stein, C.: The admissibility of Pitmans estimator for a single location parameter.Ann. Math. Statist. 30 (1959), 518522.

10