EE5110: Probability Foundations in Signal Processing July-November 2014

Lecture 7: Borel Sets and Lebesgue Measure

Lecturer: Dr. Krishna Jagannathan Scribes: Ravi Kolla, Aseem Sharma, Vishakh Hegde
In this lecture, we discuss the case where the sample space is uncountable. This case is more involved than
the case of a countable sample space, mainly because it is often not possible to assign probabilities to all
subsets of . Instead, we are forced to work with a smaller -algebra, called the Borel -algebra.
7.1 Uncountable sample spaces
Consider the experiment of picking a real number from [0,1], such that every number is equally likely to
be picked. Then = [0, 1]. It is quite apparent that a simple strategy of assigning probabilities to singleton
subsets of the sample space gets into diculties quite quickly. Indeed,
(i) If we assign some positive probability to each elementary outcome, then the probability of events such
as A = {1,
1
2
,
1
3
, } is unbounded.
(ii) If we assign zero probability to each elementary outcome, this alone would not be much helpful in
determining the probability of a subset such as
_
1
2
,
2
3

, since probability measures are not additive over

uncountable disjoint unions.
Thus, we need a dierent approach to assign probabilities to subsets of uncountable sample spaces, such as
= [0, 1]. Intuitively, we would like our uniform measure on [0, 1] to posses the following two properties.
(i) ((a, b)) = ((a, b]) = ([a, b)) = ([a, b])
(ii) Translational Invariance. That is, if A [0, 1], then (Ax) = (A) where, the set Ax is dened
as
Ax = {a +x|a A, a +x 1} {a +x 1|a A, a +x > 1}
However, the following impossibility result asserts that there is no way to consistently dene a uniform
measure on all subsets of [0, 1].
Theorem 7.1 (Impossibility Result) There does not exist a denition of a measure (A) for all subsets
of [0, 1] satisfying (i) and (ii).
Proof: Refer proposition 1.2.6 in [1].
Therefore, we must compromise, and consider a smaller -algebra that contains certain nice subsets of
the sample space [0, 1]. These nice subsets are the intervals, and the resulting -algebra is called the
Borel -algebra. Before dening Borel sets, we introduce the concept of generating -algebras from a given
collection of subsets.
7-1
7-2 Lecture 7: Borel Sets and Lebesgue Measure
7.1.1 Generated -algebra and Borel sets
The -algebra generated by a collection of subsets of the sample space is the smallest -algebra that contains
the collection. More formally, we have the following theorem.
Theorem 7.2 Let C be an arbitrary collection of subsets of , then there exists a smallest -algebra, denoted
by (C), that contains all elements of C. That is, if H is any -algebra such that C H, then (C) H.
(C) is called the -algebra generated by C.
Proof: Let {F
i
, i I} denote the collection of all -algebras that contain C. Clearly, the collection
{F
i
, i I} is non-empty, since it contains at least the power set, 2

. Consider the intersection

iI
F
i
.
Since the intersection of -algebras results in a -algebra (homework problem!) and the intersection contains
C, it follows that
iI
F
i
is a -algebra that contains C. Finally, if C H, then H is one of F
i
s for some
i I. Hence
iI
F
i
is the smallest -algebra generated by C.
Intuitively, we can think of C as being the collection of subsets of which are of interest to us. Then, (C)
is the smallest -algebra containing all the subsets in C.
We are now ready to dene Borel sets.
Denition 7.3
(a) Consider = (0, 1]. Let C
0
be the collection of all open intervals in (0, 1]. Then (C
0
) , the - algebra
generated by C
0
, is called the Borel - algebra. It is denoted by B ((0, 1]).
(b) An element of B ((0, 1]) is called a Borel-measurable set, or simply a Borel set.
Thus, every open interval in (0, 1] is a Borel set. We next prove that every single-element set in (0, 1] is a
Borel set.
Lemma 7.4 Every singleton set {b}, 0 < b 1, is a Borel set, i.e., {b} B ((0, 1]).
Proof: Consider the set
_
b
1
n
, b +
1
n
_
, n 1. By the denition of Borel sets,
_
b
1
n
, b +
1
n
_
B ((0, 1]) .
Using the properties of -algebra,
_
b
1
n
, b +
1
n
_
c
B ((0, 1])
=

_
n=1
_
b
1
n
, b +
1
n
_
c
B ((0, 1])
=
_

n=1
_
b
1
n
, b +
1
n
_
_
c
B ((0, 1])
=

n=1
_
b
1
n
, b +
1
n
_
B ((0, 1]) . (7.1)
Lecture 7: Borel Sets and Lebesgue Measure 7-3
Next, we claim that
{b} =

n=1
_
b
1
n
, b +
1
n
_
. (7.2)
i.e., b is the only element in

n=1
_
b
1
n
, b +
1
n
_
. We prove this by contradiction. Let h be an element
in

n=1
_
b
1
n
, b +
1
n
_
other than b. For every such h, there exists a large enough n
0
such that h /
_
b
1
n0
, b +
1
n0
_
. This implies h /

n=1
_
b
1
n
, b +
1
n
_
. Using (7.1) and (7.2), thus, proves that {b}
B ((0, 1]).
As an immediate consequence to this lemma, we see that every half open interval, (a, b], is a Borel set. This
follows from the fact that
(a, b] = (a, b) {b},
and the fact that a countable union of Borel sets is a Borel set. For the same reason, every closed interval,
[a, b], is a Borel set.
Note: Arbitrary union of open sets is always an open set, but innite intersections of open sets need not be
open.
Further reading for the enthusiastic: (try Wikipedia for a start)
Non-Borel sets
Non-measurable sets (Vitali set)
Banach-Tarski paradox (a bizzare phenomenon about cutting up the surface of a sphere.
See https://www.youtube.com/watch?v=Tk4ubu7BlSk)
We now present a formal procedure to dene a probability measure on a general measurable space (, F).
Specifying the probability measure for all the elements of F directly is dicult, so we start with a smaller
collection F
0
of interesting subsets of , which need not be a -algebra. We should take F
0
to be rich
enough, so that the -algebra it generates is same as F. Then we dene a function P
0
: F
0
[0, 1],
such that it corresponds to the probabilities we would like to assign to the interesting subsets in F
0
. Under
certain conditions, this function P
0
can be extended to a legitimate probability measure on (, F) by using
the following fundamental theorem from measure theory.
Theorem 7.5 (Caratheodorys extension theorem) Let F
0
be an algebra of subsets of , and let F =
(F
0
) be the -algebra that it generates. Suppose that P
0
is a mapping from F
0
to [0, 1] that satises
P
0
() = 1, as well as countable on F
0
.
Then, P
0
can be extended uniquely to a probability measure on (, F). That is, there exists a unique proba-
bility measure P on (, F) such that P(A) = P
0
(A) for all A F
0
.
Proof: Refer appendix A of [2]
We use this theorem to dene a uniform measure on (0, 1], which is also called the Lebesgue measure.
7-4 Lecture 7: Borel Sets and Lebesgue Measure
7.1.2 The Lebesgue measure
Consider = (0, 1]. Let F
0
consist of the empty set and all sets that are nite unions of the intervals of the
form (a, b]. A typical element of this set is of the form
F = (a
1
, b
1
] (a
2
, b
2
] . . . (a
n
, b
n
]
where, 0 a
1
< b
1
a
2
< b
2
. . . a
n
< b
n
and n N.
Lemma 7.6
a) F
0
is an algebra
b) F
0
is not a -algebra
c) (F
0
) = B
Proof:
a) By denition, F
0
. Also,
C
= (0, 1] F
0
. The complement of (a
1
, b
1
] (a
2
, b
2
] is (0, a
1
] (b
1
, a
2
]
(b
2
, 1], which also belongs to F
0
. Furthermore, the union of nitely many sets each of which are nite
unions of the intervals of the form (a, b] , is also a set which is the union of nite number of intervals,
and thus belongs to F
0
.
b) To see this, note that
_
0,
n
n+1
_
F
0
for every n, but

n=1
_
0,
n
n+1
_
= (0, 1) / F
0
.
c) First, the null set is clearly a Borel set. Next, we have already seen that every interval of the form
(a, b] is a Borel set. Hence, every element of F
0
(other than the null set), which is a nite union of
such intervals, is also a Borel set. Therefore, F
0
B. This implies (F
0
) B.
Next we show that B (F
0
). For any interval of the form (a, b) in C
0
, we can write (a, b) =
_

n=1
_
a, b
1
n

_
. Since every interval of the form
_
a, b
1
n

F
0
, a countable number of unions
of such intervals belongs to (F
0
). Therefore, (a, b) (F
0
) and consequently, C
0
(F
0
). This
gives (C
0
) (F
0
). Using the fact that (C
0
) = B proves the required result.
For every F F
0
of the form
F = (a
1
, b
1
] (a
2
, b
2
] . . . (a
n
, b
n
] ,
we dene a function P
0
: F
0
(0, 1] such that
P
0
(F) =
n

i=1
(b
i
a
i
).
Note that P
0
() = P
0
((0, 1]) = 1. Also, if (a
1
, b
1
] , (a
2
, b
2
] , . . . , (a
n
, b
n
] are disjoint sets, then
P
0
_
n
_
i=1
((a
i
, b
i
])
_
=
n

i=1
(P
0
(a
i
, b
i
])
=
n

i=1
(b
i
a
i
)
Lecture 7: Borel Sets and Lebesgue Measure 7-5
implying nite additivity of P
0
. It turns out that P
0
is countably additive as well i.e., if (a, b] =

i=1
(a
i
, b
i
],
where the intervals (a
i
, b
i
] are disjoint, then b a =

i=1
(a
i
, b
i
). The proof is non-trivial and beyond the
scope of this course (see [2] for a proof). Thus, in view of Theorem 7.5, there exists a unique probability
measure P on ((0, 1] , B) which is the same as P
0
on F
0
. This unique probability measure on (0, 1] is called
the Lebesgue or uniform measure.
The Lebesgue measure formalizes the notion of length. This suggests that the Lebesgue measure of a singleton
should be zero. This can be shown as follows. Let b (0, 1]. Using (7.2), we write
P({b}) = P
_

n=1
_
b
1
n
, b +
1
n
_

_
Let A
n
=
_
b
1
n
, b +
1
n
_
. For each n, the lebesgue measure of A
n
is
P(A
n
)
2
n
(7.3)
Since A
n
is a decreasing sequence of nested sets,
P({b}) =P
_

n=1
A
n
_
= lim
n
P(A
n
)
lim
n
2
n
=0
where the second equality follows from the continuity of probability measures.
Since any countable set is a countable union of singletons, the probability of a countable set is zero. For
example, under the uniform measure on (0, 1], the probability of the set of rationals is zero, since the rational
numbers in (0, 1] form a countable set.
For = (0, 1], the Lebesgue measure is also a probability measure. For other intervals (for example =
(0, 2]), it will only be a nite measure, which can be normalized as appropriate to obtain a uniform probability
measure.
Denition 7.7 Let (, F, P) be a probability space. An event A is said to occur almost surely (a.s) if
P(A) = 1.
Caution: P(A) = 1 does not mean A = .
Measure of Cantor set: Consider the cantor set K. It is created by repeatedly removing the open middle
thirds of a set of line segments. We can start by considering the [0, 1] interval. We now remove the middle
open one-third (
1
3
,
2
3
) to obtain [0,
1
3
] [
2
3
, 1]. We then remove the middle open one-third of the two intervals.
We continue this deletion process innitely many times to obtain a set of closed disjoint sets.
To nd the measure of this set, we rst consider its complement. It contains countable number of dis-
joint intervals (
1
3
,
2
3
), (
1
9
,
2
9
), (
7
9
,
8
9
) etc. The interval of measure
1
3
occurs once, the intervals of measure
1
9
occurs twice and so on in a geometric progression. Hence we have:
P(K
c
) =
1
3
+ 2
1
9
+ 4
1
27
+ =
1
3
1
2
3
= 1.
7-6 Lecture 7: Borel Sets and Lebesgue Measure
Therefore P(K) = 0. It is very interesting to note that though the Cantor set is equicardinal with (0, 1], its
Lebesgue measure is equal to 0 while the Lebesgue measure of (0, 1] is equal to 1.
We now extend the denition of Lebesgue measure on [0, 1] to the real line, R. We rst look at the denition
of a Borel set on R. This can be done in several ways, as shown below.
Denition 7.8 Borel sets on R:
Let C be a collection of open intervals in R. Then B(R) = (C) is the Borel set on R.
Let D be a collection of semi-innite intervals {(, x]; x R}, then (D) = B(R).
A R is said to be a Borel set on R, if A (n, n + 1] is a Borel set on (n, n + 1] n Z.
Exercise: Verify that the three statements are equivalent denitions of Borel sets on R.
Denition 7.9 Lebesgue measure of A R:
(A) =

n=
P
n
(A (n, n + 1])
Theorem 7.10 (R, B(R), ) is an innite measure space.
Proof: We need to prove following:
(R) =
() = 0
The countable additivity property
We see that
P
n
(R (n, n + 1]) = 1, n I
Hence we have
(R) =

n=
P
n
(R (n, n + 1])
=

n=
1 =
Now consider (n, n + 1]. This is a null set for all n. Hence we have,
P
n
( (n, n + 1]) = 0, n I
which implies,
() =

n=
P
n
( (n, n + 1]) = 0
Lecture 7: Borel Sets and Lebesgue Measure 7-7
We now need to prove the countable additivity property. For this we consider A
i
B(R) such that the
sequence A
1
, A
2
, . . . , A
n
, . . . are arbitrary pairwise disjoint sets in B(R). Therefore we obtain,
(

_
i=1
A
i
) =

n=
P
n
(

_
i=1
A
i
(n, n + 1])
=

n=

i=1
P
n
(A
i
(n, n + 1])
=

i=1

n=
P
n
(A
i
(n, n + 1])
The second equality above comes from the fact that the probability measure has countable additivity prop-
erty. The last equality above comes from the fact that the summations can be interchanged (from Fubinis
theorem). We also have the following:
(A
i
) =

n=
P
n
(A
i
(n, n + 1])
We now immediately see that
(

_
i=1
A
i
) =

i=1
(A
i
))
This shows countable additivity of Lebesgue measure.
References
[1] Rosenthal, J. S. (2006). A rst look at rigorous probability theory (Vol. 2). Singapore: World Scientic.
[2] Williams, D. (1991). Probability with martingales. Cambridge university press.