Professional Documents
Culture Documents
Independent Sums
P [A B] = P [A]P [B].
P [X A, Y B] = P [X A]P [Y B].
51
52 CHAPTER 3. INDEPENDENT SUMS
The important thing to note is that if X and Y are independent and one
knows their distributions and , then their joint distribution is automati-
cally determined as the product measure.
If X and Y are independent random variables having and for their
distributions, the distribution of the sum Z = X +Y is determined as follows.
First we construct the product measure on RR and then consider the
induced distribution of the function f (x, y) = x + y. This distribution, called
the convolution of and , is denoted by . An elementary calculation
using Fubinis theorem provides the following identities.
Z Z
( )(A) = (A x) d = (A x) d (3.1)
Z Z Z
exp[ i t x ]d( ) = exp[ i t (x + y) ] d d
Z Z
= exp[ i t x ] d exp[ i t x ] d
3.1. INDEPENDENCE AND CONVOLUTION 53
or equivalently
(t) = (t) (t) (3.2)
which provides a direct way of calculating the distributions of sums of inde-
pendent random variables by the use of characteristic functions.
Exercise 3.2. If X and Y are independent show that for any two measurable
functions f and g, f (X) and g(Y ) are independent.
Exercise 3.3. Use Fubinis theorem to show that if X and Y are independent
and if f and g are measurable functions with both E[|f (X)|] and E[|g(Y )|]
finite then
E[f (X)g(Y )] = E[f (X)]E[g(Y )].
Exercise 3.4. Show that if X and Y are any two random variables then
E(X + Y ) = E(X) + E(Y ). If X and Y are two independent random
variables then show that
Var(X + Y ) = Var(X) + Var(Y )
where
Var(X) = E [X E[X]]2 = E[X 2 ] [E[X]]2 .
If X1 , X2 , , Xn are n independent random variables, then the distri-
bution of their sum Sn = X1 + X2 + + Xn can be computed in terms of
the distributions of the summands. If j is the distribution of Xj , then the
distribution of n of Sn is given by the convolution n = 1 2 n
that can be calculated inductively by j+1 = j j+1 . In terms of their
characteristic functions n (t) = 1 (t)2 (t) n (t). The first two moments
of Sn are computed easily.
E(Sn ) = E(X1 ) + E(X2 ) + E(Xn )
and
Var(Sn ) = E[Sn E(Sn )]2
X
= E[Xj E(Xj )]2
j
X
+2 E[Xi E(Xi )][Xj E(Xj )].
1i<jn
54 CHAPTER 3. INDEPENDENT SUMS
with g(x) = (x np)2 and in (3.6) have used the fact that Sn = X1 + X2 +
+ Xn where the Xi are independent and have the simple distribution
3.2. WEAK LAW OF LARGE NUMBERS 55
Sn = X1 + X2 + + Xn
we have
Sn
lim P | m| = 0
n n
Actually it is enough to assume that E|Xi | < and the existence of the
second moment is not needed. We will provide two proofs of the statement
Theorem 3.3. If X1 , X2 , Xn are independent and identically distributed
with a finite first moment and E(Xi ) = m, then X1 +X2n++Xn converges to m
in probability as n .
Proof. 1. Let C be a large constant and let us define XiC as the truncated
random variable XiC = Xi if |Xi | C and XiC = 0 otherwise. Let YiC =
Xi XiC so that Xi = XiC + YiC . Then
1 X 1 X C 1 X C
Xi = X + Y
n 1in n 1in i n 1in i
= nC + nC .
If we denote by aC = E(XiC ) and bC = E(YiC ) we always have m =
aC + bC . Consider the quantity
1 X
n = E[| Xi m|]
n 1in
= E[|nC + nC m|]
E[|nC aC |] + E[|nC bC |]
12
2
E[|n aC | ] + 2E[|YiC |].
C
(3.8)
Exercise 3.7. In the case of the Binomial distribution with p = 12 , use Stir-
lings formula
n! ' 2 en nn+12
to estimate the probability
X n 1
rnx
r 2n
and show that it decays geometrically in n. Can you calculate the geometric
ratio X n1
n 1
(x) = lim
n
rnx
r 2n
explicitly as a function of x for x > 12 ?
58 CHAPTER 3. INDEPENDENT SUMS
This is called the Strong Law of Large Numbers. Strong laws are statements
that hold for almost all .
Let us look at functions of the form fn = An . It is easy to verify that
fn 0 in probability if and only if P (An ) 0. On the other hand
then
P : lim An () = 0 = 1.
n
is the same as
n=1 j=n Aj , or the event that infinitely many of the events
{Aj } occcur.
Exercise 3.8. Prove the following variant of the monotone convergence the-
orem.
P If fn () 0 are measurble functions the set E = { : S() =
n fn () < } is measurable
P and S() is a measurable function on E. If
each fn is integrable
P and n E[fn ] < then P [E] = 1, S() is integrable
and E[S()] = n E[fn ()].
P P
Proof. By the previous exercise if n P (An ) < , then n An () = S()
is finite almost everywhere and
X
E(S()) = P (An ) < .
n
If an infinite series has a finite sum then the n-th term must go to 0, thereby
proving
P the direct part. To prove the converse we need to show that if
n P (An ) = , then limm P (n=m An ) > 0. We can use independence
and the continuity of probability under monotone limits, to calculate for
every m,
P (
n=m An ) = 1 P (n=m An )
c
Y
= 1 (1 P (An )) (by independence)
n=m
P
P (An )
1e m
= 1
and we are done. We have used the inequality 1 x ex familiar in the
study of infinite products.
Another digression that we want to make into measure theory at this point
is to discuss Kolmogorovs consistency theorem. How do we know that there
are probability spaces that admit a sequence of independent identically dis-
tributed random variables with specified distributions? By the construction
of product measures that we outlined earlier we can construct a measure on
Rn for every n which is the joint distribution of the first n random variables.
Let us denote by Pn this probability measure on Rn . They are consistent
in the sense that if we project in the natural way from Rn+1 Rn , Pn+1
projects to Pn . Such a family is called a consistent family of finite dimen-
sional distributions. We look at the space = R consisting of all real
sequences = {xn : n 1} with a natural -field generated by the field
F of finite dimensional cylinder sets of the form B = { : (x1 , , xn ) A}
where A varies over Borel sets in Rn and varies over positive integers.
60 CHAPTER 3. INDEPENDENT SUMS
converges with probability 1. The basic steps are the following inequalities
due to Kolomogorov and Levy that control the behaviour of sums of inde-
pendent random variables. They both deal with the problem of estimating
X
k
Tn () = sup |Sk ()| = sup | Xj ()|
1kn 1kn j=1
62 CHAPTER 3. INDEPENDENT SUMS
Proof. Clearly (iii) (ii) (i) are trivial. We will establish (i) (ii)
(iii).
(i) (ii). The characteristic functions j (t) of Xj are such that
Y
(t) = j (t)
i=1
Y
n
lim
n
j (t) = 1
m m+1
lim P {|Sn Sm | } = 0
n
m
establishing (ii).
(ii) (iii). To establish (iii), because of Exercise 3.11 below, we need only
show that for every > 0
lim
n
P sup |S k S m | =0
m m<kn
Exercise 3.10. Prove the inequality 1 cos 2t 4(1 cos t) for all real t.
Deduce the inequality 1 Real (2t) 4[1 Real (t)], valid for any char-
acteristic function. Conclude that if a sequence of characteristic functions
converges to 1 in an interval around 0, then it converges to 1 for all real t.
3.4. SERIES OF INDEPENDENT RANDOM VARIABLES 65
lim P {|Sn Sm | } = 0
n
m
for every > 0 then there is a limiting random variable S() such that
P lim Sn () = S() = 1.
n
X
S() = Xi ()
i=1
1 X
n
lim P sup |Sk Sm | lim E(Xi2 )
n
m m<kn
n
m
2 j=m+1
1 X
n
= lim Var(Xi ) = 0.
n
m
2 j=m+1
66 CHAPTER 3. INDEPENDENT SUMS
Therefore
lim P { sup |Sk Sm | } 0.
n
m m<kn
lim P {|Sn Sm | } = 0
n
m
X
1 2
j2 ` + (` + C)2 ].
j=1
Proof. We define
(
Xn if |Xn | n
Yn =
0 if |Xn | > n
lim bn = 0
n
3.5. STRONG LAW OF LARGE NUMBERS 69
and
X cn X E[Y 2 ] XZ x2
2
2
n
= 2
d
n n |x|n n
n n
Z
n
X Z
2 1
= x 2
d C |x| d <
nx
n
converge almost surely. It is elementary to verify that for any series n xnn
that converges, x1 ++x
n
n
0 as n . We therefore conclude that
X1 + + Xn b1 + + bn
P lim =0 =1
n n n
Since bn 0 as n , the theorem is proved.
Exercise 3.14. Let X be a nonnegative random variable. Then
X
E[X] 1 P [Xn n] E[X]
n=1
P
In particular E[X] < if and only if n P [X n] < .
Exercise 3.15. If for a sequence of i.i.d. random variables X1 , , Xn , ,
the strong law of large numbers holds with some limit, i.e.
Sn
P [ lim = ]= 1
n n
for some random variable , which may or may not be a constant with prob-
ability 1, then show that necessarily E|Xi| < . Consequently = E(Xi )
with probabilty 1.
One may ask why the limit cannot be a proper random variable. There
is a general theorem that forbids it called Kolmogorovs Zero-One law. Let
us look at the space of real sequences {xn : n 1}. We have the -field B,
the product -field on . In addition we have the sub -fields Bn generated
by {xj : j n}. Bn are with n and B = n Bn which is also a -field is
called the tail -field. The typical set in B is a set depending only on the
tail behavior of the sequence. For example the sets { : xn is bounded },
{ : lim supn xn = 1} are in B whereas { : supn |xn | = 1} is not.
70 CHAPTER 3. INDEPENDENT SUMS
1 x2
p(x) = exp[ 2 ]. (3.11)
2
3.6. CENTRAL LIMIT THEOREM. 71
t
n (t) = [( )]n
n
2 t2
(t) = 1 + o (t2 )
2
to conclude that
t 2 t2 1
( ) = 1 +o( )
n 2n n
and it then follows that
2 t2
lim n (t) = (t) = exp[ ].
n 2
Since (t) is the characteristic function of the normal distribution with den-
sity p(x) given by equation (3.11), we are done.
Exercise 3.17. A more direct proof is possible in some special cases. For
instance if each Xi = 1 with probability 12 , Sn can take the values n 2k
with 0 k n,
1 n
P [Sn = 2k n] = n
2 k
and
X
Sn 1 n
P [a b] = n .
n 2 k
k:a n2knb n
Actually for the proof of the central limit theorem we do not need the
random variables {Xj } to have identical distributions. Let us suppose that
they all have zero means and that the variance of Xj is j2 . Define s2n =
72 CHAPTER 3. INDEPENDENT SUMS
for each > 0 is sufficient for the central limit theorem to hold.
Proof. The first step in proving this limit theorem as well as other limit
theorems that we will prove is to rewrite
and
Y
n
n (t) = n,j (t).
j=1
and
X
n
sup sup |n,j (t) 1| < .
n |t|T j=1
X
n
lim sup log n,j (t) [n,j (t) 1]
n |t|T
j=1
X
n
lim sup C |n,j (t) 1|2
n |t|T
j=1
X
n
C lim sup sup |n,j (t) 1| sup |n,j (t) 1|
n |t|T 1jn |t|T j=1
=0
by the expansion
We see that
Z
sup n,j (t) 1 = sup exp[i t x ] 1 dn,j
|t|T |t|T
Z
x
= sup exp[i t ] 1 dj
|t|T sn
Z
x x
= sup exp[i t ] 1 i t dj (3.12)
|t|T sn sn
Z 2
x
CT dj (3.13)
s2n
Z Z
x2 x2
= CT 2
d j + C T 2
dj
|x|<sn sn |x|sn sn
Z
2 1
CT + CT 2 x2 dj . (3.14)
sn |x|sn
We have used the mean zero condition in deriving equation 3.12 and
the estimate |eix 1 ix| cx2 to get to the equation 3.13. If we let
n , by Lindebergs condition, the second term of equation (3.14) goes
to 0. Therefore
lim sup sup sup n,j (t) 1 2 CT .
n 1jkn |t|T
X Xn Z
1 X 2
n n
x2
sup n,j (t) 1 CT dj CT 2 = CT
|t|T j=1 j=1
s2n sn j=1 j
X
n t2
lim sup (n,j (t) 1) +
n |t|T 2
j=1
Xn
2 t2
lim sup n,j (t) 1 + j
n |t|T
j=1
2s2n
Xn Z
x x t2 2
x
= lim sup exp[i t ] 1 i t + 2 dj
n |t|T sn sn 2sn
j=1
Xn Z
x x t 2 2
x
lim sup exp[i t ] 1 i t + 2 dj
n |t|T sn sn 2sn
j=1 |x|<sn
X n Z
x x t2 2
x
+ lim sup exp[i t ] 1 i t + 2 dj
n |t|T sn sn 2sn
j=1 |x|sn
Xn Z
|x|3
lim CT 3
dj
n
j=1 |x|<s n
s n
X n Z
x2
+ lim CT 2
dj
n
j=1 |x|s n
s n
X n Z
x2
CT lim sup 2
dj
n
j=1
s n
X n Z
x2
+ lim CT 2
dj
n
j=1 |x|s n
s n
= CT
Remark 3.3. The key step in the proof of the central limit theorem under
Lindebergs condition, as well as in other limit theorems for sums of inde-
pendent random variables, is the analysis of products
n (t) = kj=1
n
n,j (t).
The idea is to replace each n,j (t) by exp [n,j (t) 1], changing the product
to the exponential of a sum. Although each n,j (t) is close to 1, making
76 CHAPTER 3. INDEPENDENT SUMS
the idea Preasonable, in order for the idea to work one has to show that
the sum kj=1 n
|n,j (t) 1|2 is negligible. This requires the boundedness of
Pkn
j=1 |n,j (t) 1|. One has to use the mean 0 condition or some suitable
centering condition to cancel the first term in the expansion of n,j (t) 1
and control the rest from sums of the variances.
Exercise 3.18. Lyapunovs condition is the following: for some > 0
n Z
1 X
lim |x|2+ dj = 0.
n s2+
n j=1
As we stated in the previous section, we want to study the behavior of the sum
of a large number of independent random variables. We have kn independent
random variables {Xn,j : 1 j kn } with respective distributions {n,j }.
Pn
We are interested in the distribution n of Zn = kj=1 Xn,j . One important
assumption that we will make on the random variables {Xn,j } is that no
single one is significant. More precisely for every > 0,
= .
Therefore
lim sup |an,j | = 0.
n 1jkn
0
This means that n,j are uniformly infinitesimal just as n,j were. Let us
suppose that n is so large that sup1jkn |an,j | 14 . The advantage in going
0
from n,j to n,j is that the latter are better centered and we can calculate
Z
a0n,j = 0
x dn,j
|x|1
Z
= (x an,j ) dn,j
|xan,j |1
Z
= x dn,j an,j n,j [ |x an,j | 1 ]
|xan,j |1
Z
= x dn,j an,j + n,j [ |x an,j | > 1 ]
|xan,j |1
3 1
|a0n,j | Cn,j [ |x| 0
] Cn,j [ |x| ].
4 2
3.7. ACCOMPANYING LAWS. 79
In other words we may assume without loss of generality that n,j satisfy the
bound
1
|an,j | Cn,j [ |x| ] (3.16)
2
0
and forget all about the change from n,j to n,j . We will drop the primes
and stay with just n,j . Then, just as in the proof of the Lindeberg theorem,
we proceed to estimate
lim sup log n (t) log n (t)
n |t|T
Xkn
lim sup log n,j (t) (n,j (t) 1)]
n |t|T
j=1
X
kn
lim sup log n,j (t) (n,j (t) 1)
n |t|T
j=1
X
kn
lim sup C |n,j (t) 1|2
n |t|T
j=1
= 0.
X
kn
exp (n,j (t) 1)) + itAn = exp[fn (t)]
j=1
have a limit, which is again a characteristic function. Since the limiting char-
acteristic function is continuous and equals 1 at t = 0, and the convergence
is uniform near 0, on some small interval |t| T0 we have the bound
sup sup 1 Re fn (t) C
n |t|T0
80 CHAPTER 3. INDEPENDENT SUMS
or equivalently
kn Z
X
sup sup (1 cos t x ) dn,j C
n |t|T0 j=1
kn Z
X
sup sup (1 cos t x ) dn,j CT .
n |t|T j=1
If we integrate the inequality with respect to t over the interval [T, T ] and
divide by 2T , we get
kn Z
X sin T x
sup (1 ) dn,j CT
n
j=1
Tx
X
kn
sup n,j [ |x| ] C <
n
j=1
kn Z
X
sup x2 dn,j C < .
n
j=1 |x|1
3.7. ACCOMPANYING LAWS. 81
Y
kn
2
lim |n (t)| = lim |n,j (t)|2
n n
j=1
X
kn
sup sup [1 |n,j (t)|2 ] C0 <
n |t|T0 j=1
82 CHAPTER 3. INDEPENDENT SUMS
X
kn
sup sup [1 |n,j (t)|2 ] CT <
n |t|T j=1
X
kn
sup |n,j |2 [ |x| ] C < (3.18)
n
j=1
kn Z Z
X
sup (x y)2dn,j (x) dn,j (y) C < . (3.19)
n
j=1 |xy|2
X
kn
sup n,j [ x : |x| ] C < . (3.20)
n
j=1
One can now derive (3.17) from (3.20) and (3.21) as in the earlier part.
Exercise 3.20. Let kn = n2 and n,j = 1 for 1 j n2 . n = n and show
n
that without centering n n converges to a different limit.
Exercise 3.22. Show that for any 0, the Poisson distribution with pa-
rameter
en n
p (n) = for n 0
n!
is infinitely divisible.
Exercise 3.23. Show that any probabilty distribution supported on a finite
set {x1 , . . . , xk } with
[{xj }] = pj
Pk
and pj 0, j=1 pj = 1 is infinitely divisible if and only if it is degenrate,
i.e. [{xj }] = 1 for some j.
Exercise 3.24. Show that for any nonnegative finite measure with total
mass a, the distribution
X
()j
e(F ) = ea
j=0
j!
with characteristic function
Z
[
e(F )(t) = exp[ (eitx 1)d]
|(x) x| C|x|3
is a characteristic function for any measure M with finite total mass. In fact it
is the characteristic function of an infinitely divisible probability distribution.
It is not necessary that M be a finite measure for to make sense. M could
be infinite, but in such a way that it is finite on {x : |x| } for every > 0,
and near 0 it integrates x2 i.e.,
|ei t x 1 i t x | CT x2
for |t| T , (t) (t) uniformly on bounded intervals where (t) is given
by the integral
Z
(t) = exp [ ei t x 1 i t (x) ] dM + i t a
Z
x2
dM < (3.25)
1 + x2
Theorem 3.20. For every admissible Levy measure M, 2 > 0 and real a
Z
2 t2
(t) = exp [ ei t x 1 i t (x) ] dM + i t a
2
For some ( and therefore for every) ` > 0 such that ` are continuity points
for M, i.e., M{ `} = 0
Z ` Z `
2 2 2 2
lim n + x dMn = + x dM . (3.27)
n ` `
an a as n . (3.28)
3.8. INFINITELY DIVISIBLE DISTRIBUTIONS. 87
Proof. Let us prove the sufficiency first. Condition (3.26) implies that for
every ` such that ` are continuity points of M
Z Z
lim [eitx
1 i t (x) ] dMn = [ ei t x 1 i t (x) ] dM
n |x|` |x|`
Z `
t2 x2
lim lim sup [ ei t x 1 i t (x) + ] dMn
`0 n ` 2
Z `
t2 x2
[eitx
1 i t (x) + ] dM
` 2
= 0
Z
n2 t2
lim + [e itx
1 i t (x)] dMn
n 2
Z
2 t2
= + [e itx
1 i t (x)] dM .
2
and Z Z
` `
3
|x| dMn ` |x|2 dMn .
` `
Z `
2 2
sup n + |x| dMn < . (3.31)
n `
Consequently
(t)
lim = 12 = 22
t t2
leaving us with
Z
(t) = [e itx
1 i t (x) ] dM1 + i t a1
Z
= [e itx
1 i t (x) ] dM2 + i t a2
(t + s) + (t s)
H(s, t) = (t)
2
we get Z Z
itx
e (1 cos s x)dM1 = ei t x (1 cos s x)dM2
for all t and s. Since we can and do assume that M{0} = 0 for any admissible
Levy measure M we have M1 = M2 . If we know that 12 = 22 and M1 = M2
it is easy to see that a1 must equal a2 .
Finally
Applications.
and X
a = lim an,j
n
j
where Z
an,j = x dn,j
|x|1
Z 2 Z 2 Z
|an,j | =
2
x dn,j = x dn,j n,j [ |x| > 1 ] |x|2 dn,j
|x|1 |x|>1
and
X
kn X Z
2 2
|an,j | |x| dn,j sup n,j [ |x| > 1 ]
1jkn
j=1 1jkn
n2 sup n,j [ |x| > 1 ]
1jkn
0.
Pkn
Because j=1 |an,j |2 0 as n we must have
XZ
2
= lim |x|2 dn,j
n |x|`
Exercise
P 3.26. What happens in the Poisson limit theorem (applicationSn1) if
n
n = j pn,j as n ? Can you show that the distribution of
n
converges to the standard Normal distribution?
for any a. But Z is a random variable that does not depend on the par-
ticular values of X1 , , Xn and is therefore a set in the tail -field. By
Kolmogorovs zero-one law P Z a must be either 0 or 1. Since it cannot
be 0 it must be 1.
We will not prove this theorem in the most general case which assumes
only the existence of two moments. We will assume instead that E[|X|2+ ] <
for some > 0. We shall first reduce the proof to an estimate on the
Sn
tail behavior of the distributions of n
by a careful application of the Borel-
Cantelli Lemma. This estimate is obvious if X1 , , Xn , are themselves
normally distributed and we will show how to extend it to a large class of
distributions that satisfy the additional moment
condition. It is clear that
we are interested in showing that for > 2,
p
P Sn n log log n infinitely often = 0.
It would
be sufficient because of Borel-Cantelli lemma to show that for any
> 2,
X p
P Sn n log log n < .
n
E[Sn2 ]
sup P |Sj | (kn1)
1jkn [(kn1)]2
kn
=
[(kn1)]2
kn
= 2
kn1 log log kn1
= o(1) as n . (3.34)
3.9. LAWS OF THE ITERATED LOGARITHM. 95
By choosing small enough so that > 2 it is sufficient to show
that for any 0 > 2,
X
0
P Skn (kn1 ) < .
n
(kn1 )
By picking sufficiently close to 1, ( so that 0 > 2), because (kn )
=
1 we can reduce this to the convergence of
X
P Skn (kn ) < (3.35)
n
for all > 2.
2
If we use the estimate P [X a] exp[ a2 ] that is valid for the standard
normal distribution, we can verify 3.35.
X 2 ((kn ))2
exp <
n
2 kn
for any > 2.
To prove the lower bound we select again a subsequece, kn = [n ] with
some > 1, and look at Yn = Skn+1 Skn , which are now independent
random variables. The tail probability of the Normal distribution has the
lower bound
Z
1 x2
P [X a] = exp[ ]dx
2 a 2
Z
1 x2
exp[ x](x + 1)dx
2 a 2
1 (a + 1)2
exp[ ].
2 2
If we assume Normal like tail probabilities we can conclude that
X X
1 (kn+1) 2
P Yn (kn+1) exp [1 + p ] = +
n n
2 (n+1 n )
2
provided 2(1) < 1 and conclude by the Borel-Cantelli lemma, that Yn =
Skn+1 Skn exceeds (kn+1) infinitely often for such . On the other hand
96 CHAPTER 3. INDEPENDENT SUMS
and therefore,
s
Sn 2( 1) 2
P lim sup = 1.
n (n)
Z
S 1 x2
sup P { a} exp[ ] dx Cn
n
(3.36)
a n a 2 2
for some > 0 in the central limit theorem. Such an error estimate is
provided in the following theorem
Theorem 3.26. (Berry-Esseen theorem). Assume that the i.i.d. se-
quence {Xj } with mean zero and variance one satisfies an additional moment
condition E|X|2+ < for some > 0. Then for some > 0 the estimate
(3.36) holds.
3.9. LAWS OF THE ITERATED LOGARITHM. 97
Lemma 3.28. If , are two probability measures with zero mean having
(), () for respective characteristic functions. Then
Z Z
1 ei a y sin h y
fa,h (x) d( )(x) = [(y) (y)] dy
2 iy hy
where fa,h (x) = fa,,h (x), is given by
0 for < x a h
fa,h (x) = xa+h
for a h x a + h
2h
1 for a + h x < .
98 CHAPTER 3. INDEPENDENT SUMS
Proof. We just let b in the previous lemma. Since |(y) (y)| = o(|y|),
there is no problem in applying the Riemann-Lebesgue Lemma. We now
proceed with the proof of the theorem.
Z
[[a, )] fah,h (x) d(x) [[a 2h, )]
and Z
[[a, )] fah,h (x) d(x) [[a 2h, )].
Z
sup |[[a, )] [[a, )]| sup fah,h (x) d( )(x)
a a
+ 2hC
Z
1 | sin h y |
|(y) (y)| dy
2 h y2
+ 2hC. (3.37)
y2 |y|2+
|n (y) exp[ ]| C if |y| n 2+ .
2 n
3.9. LAWS OF THE ITERATED LOGARITHM. 99
Therefore for = 2+
Z 2
n (y) exp[ y ] | sin h y | dy
2 h y2
Z
y 2 | sin h y |
= n (y) exp[ ] dy
|y|n h y2
Z
2
+ n (y) exp[ y ] | sin h y | dy
|y|n h y2
Z Z
C |y| dy
dy +
|y|n |y|
h 2
|y|n n
n(+1) + n
C
h
C
=
hn +2
Substituting this bound in 3.37 we get
C
sup |n [[a, )] [[a, )]| C1 h + .
a h n 2+