Professional Documents
Culture Documents
Florian Herzog
2013
Probability space
Probability space
A probability space W is a unique triple W = {Ω, F , P }:
• Ω is its sample space
• F its σ -algebra of events
• P its probability measure
Remarks: (1) The sample space Ω is the set of all possible samples or elementary
events ω : Ω = {ω | ω ∈ Ω}.
(2)The σ -algebra F is the set of all of the considered events A, i.e., subsets of
Ω: F = {A | A ⊆ Ω, A ∈ F }.
(3) The probability measure P assigns a probability P (A) to every event
A ∈ F : P : F → [0, 1].
The sample space Ω is sometimes called the universe of all samples or possible
outcomes ω .
Example 1. Sample space
• Toss of a coin (with head and tail): Ω = {H, T }.
• Two tosses of a coin: Ω = {HH, HT, T H, T T }.
• A cubic die: Ω = {ω1, ω2, ω3, ω4, ω5, ω6}.
• The positive integers: Ω = {1, 2, 3, . . . }.
• The reals: Ω = {ω | ω ∈ R}.
Note that the ω s are a mathematical construct and have per se no real or
scientific meaning. The ω s in the die example refer to the numbers of dots
observed when the die is thrown.
P : F → [0, 1],
Not every subset of [0, 1] has a determinable length ⇒ collect the ones with a
determinable length in F . Such a mathematical construct, which has additional,
desirable properties, is called σ -algebra.
Definition 2. σ -algebra
A collection F of subsets of Ω is called a σ -algebra on Ω if
• Ω ∈ F and ∅ ∈ F (∅ denotes the empty set)
• If A ∈ F then Ω\A = Ac ∈ F : The complementary subset of A is also
in Ω
S∞
• For all Ai ∈ F : i=1 Ai ∈ F
The intuition behind it: collect all events in the σ -algebra F , make sure that
by performing countably many elementary set operation (∪, ∩,c ) on elements
of F yields again an element in F (closeness).
The pair {Ω, F } is called measure space.
The Borel σ -algebra includes all subsets of R which are of interest in practical
applications (scientific or engineering).
Definition 4. Borel σ -algebra B(R))
The Borel σ -algebra B(R) is the smallest σ -algebra containing all open
intervals in R. The sets in B(R) are called Borel sets. The extension to the
multi-dimensional case, B(Rn), is straightforward.
• (−∞, a), (b, ∞), (−∞, a) ∪ (b, ∞)
• [a, b] = (−∞, a) ∪ (b, ∞),
S∞ S∞
• (−∞, a] = n=1[a − n, a] and [b, ∞) = n=1[b, b + n],
• (a, b] = (−∞, b] ∩ (a, ∞),
T∞
• {a} = n=1(a − n1 , a + n1 ),
Sn
• {a1, · · · , an} = k=1 ak .
Definition 5. Measure
Let F be the σ -algebra of Ω and therefore (Ω, F ) be a measurable space.
The map
µ : F → [0, ∞]
is called a measure on (Ω, F ) if µ is countably additive. The measure µ
is countably additive (or σ -additive) if µ(∅) = 0 and for every sequence of
S
disjoint sets (Fi : i ∈ N) in F with F = i∈N Fi we have
X
µ(F ) = µ(Fi).
i∈N
The measure of length on the straight line is known as the Lebesgue measure.
Definition 6. Lebesgue measure on B(R)
The Lebesgue measure on B(R), denoted by λ, is defined as the measure on
(R, B(R)) which assigns the measure of each interval to be its length.
Examples:
• Lebesgue measure of one point: λ({a}) = 0.
P∞
• Lebesgue measure of countably many points: λ(A) = i=1 λ({ai }) = 0.
• The Lebesgue measure of a set containing uncountably many points:
– zero
– positive and finite
– infinite
y ∈ R ⇒ {ω ∈ Ω : f (ω) ≤ y} ∈ F
This means that once we know the (random) value X(ω) we know which of
the events in F have happened.
• F = {∅, Ω}: only constant functions are measurable
• F = 2Ω: all functions are measurable
X(ω) : Ω 7→ R R
6
t
j X(ω) ∈ R
ω A2
A1
(−∞, a] ∈ B
A3 Y
Ω A4 X −1 : B 7→ F
y ∈ R ⇒ {ω ∈ Ω : f (ω) ≤ y} ∈ F
f +dP
R
The Rintegral above may be finite or infinite. It is not defined if Ω
and Ω f −dP are both infinite.
The most important concept of the Lebesgue integral is that the limit of
approximate sums (as the Riemann integral): for Ω = R:
f (x) f (x)
6 6
- x - x
∆x
There are more functions which are Lebesgue integrable than Riemann integra-
ble.
X : Ω → R.
for all A ∈ F .
λx −λ
f (x) = e , x = 0, 1, 2, . . . .
x!
1 x−µ
− 1
f (x) = √ e 2 σ
2πσ
Γ( ν+12 )
1 (x − µ)2 − 12 (ν+1)
f (x) = √ 1+
Γ( ν2 ) πνσ ν σ2
Γ( ν+n ) 1 − 1 (ν+n)
2 T −1 2
f (x) = ν
p 1 + (x − µ) Σ (x − µ) .
n
Γ( 2 ) (πν) det (Σ) ν
• The chi square distribution with degree-of-freedom (dof) n has the following
density
−x x
n−2
e 2 2 2
f (x) =
2Γ( n2 )
which is abreviated as Z ∼ χ2(n) and where Γ denotes the gamma
function.
• A chi square distributed random variable Y is created by
n
X 2
Y = Xi
i=1
X
Y =q ,
Z
ν
1 − |x−µ|
p(x) = e σ
2σ
with mean µ and diffusion σ . The variance of this distribution is given as
2σ 2.
With this definition at hand, it does not matter what the sample Ω is. The
calculations for the two familiar cases of a finite Ω and Ω ≡ R with continuous
random variables remain the same.
Definition 14. Variance of a random variable
The variance of a random variable X, defined on a probability space (Ω, F , P ),
is defined by:
Z
2 2 2 2
var(X) = E[(X − E[X]) ] = (X − E[X]) dP = E[X ] − E[X] .
Ω
P (A ∩ B) P (B|A)P (A)
P (A|B) = = , P (B) > 0.
P (B) P (B)
A A∩B B
E(XIB )
E(X|B) = , P (B) > 0.
P (B)
X
Y
- ω
G1 G2 G3 G4 G5
• E(E(X|F )) = E(X).
• If X is F -measurable, then E(X|F ) = X .
• Linearity: E(αX1 + βX2|F ) = αE(X1|F ) + βE(X2|F ).
• Positivity: If X ≥ 0 almost surely, then E(X|F ) ≥ 0.
• Tower property: If G is a sub-σ -algebra of F , then
E(ZX|G) = Z · E(X|G).