NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 1

MODULE 1
PROBABILITY
LECTURE 2
Topics
1.2 AXIOMATIC APPROACH TO PROBABILITY AND
PROPERTIES OF PROBABILITY MEASURE
1.2.1 I nclusion-Exclusion Formula

In the following section we will discuss the modern approach to probability theory where
we will not be concerned with how probabilities are assigned to suitably chosen subsets
of . Rather we will define the concept of probability for certain types of subsets using
a set of axioms that are consistent with properties (i)-(iii) of classical (or relative
frequency) method. We will also study various properties of probability measures.
1.2 AXIOMATIC APPROACH TO PROBABILITY AND
PROPERTIES OF PROBABILITY MEASURE
We begin this section with the following definitions.
Definition 2.1
(i) A set whose elements are themselves set is called a class of sets. A class of sets
will be usually denoted by script letters , , , . For example =
1, 1, 3, 2, 5, 6;
(ii) Let be a class of sets. A function : is called a set function. In other
words, a real-valued function whose domain is a class of sets is called a set
function.
As stated above, in many situations, it may not be possible to assign probabilities to all
subsets of the sample space such that properties (i)-(iii) of classical (or relative
frequency) method are satisfied. Therefore one begins with assigning probabilities to
members of an appropriately chosen class of subsets of (e.g., if = , then may
be class of all open intervals in ; if is a countable set, then may be class of all
singletons , ). We call the members of as basic sets. Starting from the basic
sets in assignment of probabilities is extended, in an intuitively justified manner, to as
many subsets of as possible keeping in mind that properties (i)-(iii) of classical (or

NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 2

relative frequency) method are not violated. Let us denote by the class of sets for
which the probability assignments can be finally done. We call the class as event space
and elements of are called events. It will be reasonable to assume that satisfies the
following properties: (i) , (ii)

= , and (iii)

, =
1,2,

=1
. This leads to introduction of the following definition.
Definition 2.2
A sigma-field ( -field) of subsets of is a class of subsets of satisfying the
following properties:
(i) ;
(ii)

= (closed under complements);

(iii)

, = 1, 2,

=1
(closed under countably infinite unions).
Remark 2.1
(i) We expect the event space to be a -field;
(ii) Suppose that is a -field of subsets of . Then,
(a) since =

(b)
1
,
2
,

=1
since

=1
=

=1

;
(c) , =

and ;
(d)
1
,
2
, ,

, for some ,

=1
and

=1
(take

+1
=
+2
= = so that

=1
=

=1
or
+1
=
+2
=
= so that

=1
=

=1
);
(e) although the power set of is a -field of subsets of , in
general, a -field may not contain all subsets of .
Example 2.1
(i) = , is a sigma field, called the trivial sigma-field;
(ii) Suppose that . Then = ,

, , is a -field of subsets of . It is
the smallest sigma-field containing the set ;
(iii) Arbitrary intersection of -fields is a -field (see Problem 3 (i));
(iv) Let be a class of subsets of and let

be the collection of all -

fields that contain . Then
=

is a -field and it is the smallest -field that contains class (called the -
field generated by and is denoted by ()) (see Problem 3 (iii));
(v) Let = and let be the class of all open intervals in . Then
1
= is
called the Borel -field on . The Borel -field in

(denoted by

) is the

NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 3

-field generated by class of all open rectangles in

. A set

is called
a Borel set in

; here

= {(
1
, ,

): <

< , = 1, , }
denotes the -dimensional Euclidean space;
(vi)
1
contains all singletons and hence all countable subsets of
=
1

, +
1

=1
.
Let be an appropriately chosen class of basic subsets of for which the probabilities
can be assigned to begin with (e.g., if = then may be class of all open intervals in
; if is a countable set then may be class of all singletons , ). It turns out (a
topic for an advanced course in probability theory) that, for an appropriately chosen class
of basic sets, the assignment of probabilities that is consistent with properties (i)-(iii) of
classical (or relative frequency) method can be extended in an unique manner from to
(), the smallest -field containing the class . Therefore, generally the domain of a
probability measure is taken to be (), the -field generated by the class of basic
subsets of . We have stated before that we will not care about how assignment of
probabilities to various members of event space (a -field of subsets of ) is done.
Rather we will be interested in properties of probability measure defined on event space
.
Let be a sample space associated with a random experiment and let be the event
space (a -field of subsets of ). Recall that members of are called events. Now we
provide a mathematical definition of probability based on a set of axioms.
Definition 2.3
(i) Let be a -field of subsets of . A probability function (or a probability
measure) is a set function , defined on , satisfying the following three axioms:

(a) 0, ; (Axiom 1: Non negativity);
(b) If
1
,
2
, is a countably infinite collection of mutually exclusive events
i. e.,

, = 1, 2, ,

= , then

=1
=

1=1
; (Axiom 2: Countably infinite additive)
(c) = 1 (Axiom 3: Probability of the sample space is 1).

(ii) The triplet , , is called a probability space.



NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 4

Remark 2.2
(i) Note that if
1
,
2
, is a countably infinite collection of sets in a -field then

=1
and, therefore, (

=1
) is well defined;
(ii) In any probability space , , we have = 1 (or = 0; see Theorem
2.1 (i) proved later) but if = 1 (or = 0), for some , then it does
not mean that = ( or = ) (see Problem 14 (ii)).
(iii) In general not all subsets of are events, i.e., not all subsets of are elements of
.
(iv) When is countable it is possible to assign probabilities to all subsets of using
Axiom 2 provided we can assign probabilities to singleton subsets of . To
illustrate this let =
1
,
2
, or =
1
, ,

, for some n and let

, = 1, 2, , so that 0

1, = 1,2, (see Theorem 2.1 (iii)

below) and

=1

=1
=

=1
= = 1. Then, for any
,
=

.
:

Thus in this case we may take = , the power set of . It is worth
mentioning here that if is countable and = (class of all
singleton subsets of ) is the class of basic sets for which the assignment of the
probabilities can be done, to begin with, then () = (see Problem 5 (ii)).
(v) Due to some inconsistency problems, assignment of probabilities for all subsets of
is not possible when is continuum (e.g., if contains an interval).
Theorem 2.1
Let , , be a probability space. Then
(i) = 0;
(ii)

, = 1, 2, . , and

= ,

=1
=

=1

(finite additivity);
(iii) , 0 1 and

= 1 ;
(iv)
1
,
2
and
1

2

2

1
=
2

1
and
1

2

(monotonicity of probability measures);
(v)
1
,
2

1

2
=
1
+
2

1

2
.
Proof.
(i) Let
1
= and

= , = 2, 3, . Then
1
= 1, (Axiom 3),

, =
1, 2, ,
1
=

=1
and

= , . Therefore,

NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 5

1 =
1
=

=1


=

(using Axiom 2)

=1


= 1 +

=2

=2
= 0

= 0.

(ii) Let

= , = +1, +2, . Then

, = 1, 2, ,

= , and

= 0, = +1, +2, . Therefore,

=1
=

=1

=

using Axiom 2

=1

=

=1
.
(iii) Let . Then =

and

= . Therefore
1 =
=

= +

(using (ii))
1 and

= 1 (since (

) [0,1])
0 1 and

= 1 .
(iv) Let
1
,
2
and let
1

2
. Then
2

1
,
2
=
1

2

1
and

1

2

1
= .


NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 6


Figure 2.1
Therefore,

2
=
1

2

1

=
1
+
2

1
(using (ii))

2

1
=
2

1
.
As
2

1
0, it follows that
1

2
.
(v) Let
1
,
2
. Then
2

1
,
1

2

1
= and
1

2
=
1

1
.

Figure 2.2
Therefore,

1

2
=
1

2

1

=
1
+
2

1
(using (ii)) (2.1)
Also
1

2

2

1
= and
2
=
1

2

2

1
. Therefore,

NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 7



Figure 2.3

2
=
1

2

2

1

=
1

2
+
2

1
(using (ii))

2

1
=
2

1

2
(2.2)
Using (2.1) and (2.2), we get

1

2
=
1
+
2

1

2
.

1.2.1 I nclusion-Exclusion Formula

Theorem 2.2
Let , , be a probability space and let
1
,
2
, ,

, 2. Then

=1
=
,

=1
,
where
1,
=

=1
and, for 2, 3, , ,

,
= 1
1

.
1
1
<<

Proof. We will use the principle of mathematical induction. Using Theorem 2.1 (v), we
have

1

2
=
1
+
2

1

2


NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 8

=
1,2
+
2,2
,
where
1,2
=
1
+
2
and
2,2
=
1

2
. Thus the result is true for = 2.
Now suppose that the result is true for 2, 3, , for some positive integer
2. Then

+1
=1
=

=1

+1

=

=1
+
+1

=1

+1
(using the result for = 2)
=

=1
+
+1

+1

=1

=
,

=1
+
+1

+1

=1
using the result for = (2.3)
Let

+1
, = 1, . . Then

+1

=1
=

=1

=
,

=1
(again using the result for = ) , (2.4)
where

1,
=

=1
=

+1

=1
and, for 2, 3, , ,


,
= 1
1

1
1
<
2
<<

= 1
1

+1

1
1
<
2
<<

.


NPTEL- Probability and Distributions

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 9

Using (2.4) in (2.3), we get

+1
=1
=
1,
+
+1
+
2,

1,
++
,

1,

,
.
Note that
1,
+
+1
=
1,+1
,
,

1,
=
,+1
, = 2,3, , , and

,
=
+1,+1
. Therefore,

+1
=1
=
1,+1
+
,+1
+1
=2
=
,+1
+1
=1
.

Remark 2.3
(i) Let
1
,
2
, . Then

1

2

3
=
1
+
2
+
3

1,3

1

2
+
1

3
+
2

3

2,3
+
1

2

3

3,3

=
1,3

2,3
+
3,3
,
where
1,3
=
1,3
,
2,3
=
2,3
and
3,3
=
3,3
.
In general,

=1
=
1,

2,
+
3,
+1
1

,
,
where

,
=

,
, if is odd

,
, if is even
, = 1, 2, .
(ii) We have
1
1

2
=
1
+
2

1

2


1

2

1
+
2
1.
The above inequality is known as Bonferronis inequality.