1 views

Uploaded by ajengpn

t

- 121_vgGray R. - Probability, Random Processes, And Ergodic Properties
- mathgen-1962475817
- Introduction-Course Outline MAE419
- Probability Theory
- Lecture8_Infinite Coin Toss
- Real Analysis Lecture Notes.pdf
- Data Structures and Algorithms in Java (6e) - Useful Mathematical Facts.pdf
- A General Rudin-Carleson Theorem - Errett Bishop (AMS 13-11-1960)
- Lecture 5&6
- stat_TD
- 1 Probability
- 10W 23plan345 Syllabus
- 9781849962315-c1 (4)
- stochastic lectures
- Homeomorphisms of Hashimoto Topologies
- Chap 2 Probability Distribution
- Communication Theory Sample
- fseries
- Galloway (a Treatise on Probability_7ed Encyclopaedia Britannica ) (1839)
- lect1a

You are on page 1of 36

Florian Herzog

2013

Probability space

Probability space

A probability space W is a unique triple W = {Ω, F , P }:

• Ω is its sample space

• F its σ -algebra of events

• P its probability measure

Remarks: (1) The sample space Ω is the set of all possible samples or elementary

events ω : Ω = {ω | ω ∈ Ω}.

(2)The σ -algebra F is the set of all of the considered events A, i.e., subsets of

Ω: F = {A | A ⊆ Ω, A ∈ F }.

(3) The probability measure P assigns a probability P (A) to every event

A ∈ F : P : F → [0, 1].

Sample space

The sample space Ω is sometimes called the universe of all samples or possible

outcomes ω .

Example 1. Sample space

• Toss of a coin (with head and tail): Ω = {H, T }.

• Two tosses of a coin: Ω = {HH, HT, T H, T T }.

• A cubic die: Ω = {ω1, ω2, ω3, ω4, ω5, ω6}.

• The positive integers: Ω = {1, 2, 3, . . . }.

• The reals: Ω = {ω | ω ∈ R}.

Note that the ω s are a mathematical construct and have per se no real or

scientific meaning. The ω s in the die example refer to the numbers of dots

observed when the die is thrown.

Event

subset A, then the event A is said to have occurred. The set of all subsets of

the sample space are denoted by 2Ω.

Example 2. Events

• Head in the coin toss: A = {H}.

• Odd number in the roll of a die: A = {ω1, ω3, ω5}.

• An integer smaller than 5: A = {1, 2, 3, 4}, where Ω = {1, 2, 3, . . . }.

• A real number between 0 and 1: A = [0, 1], where Ω = {ω | ω ∈ R}.

We denote the complementary event of A by Ac = Ω\A. When it is possible

to determine whether an event A has occurred or not, we must also be able to

determine whether Ac has occurred or not.

Probability Measure I

A probability measure P on the countable sample space Ω is a set function

P : F → [0, 1],

• P(Ω) = 1.

• P(ωi) = pi.

• If A1, A2, A3, ... ∈ F are mutually disjoint, then

∞

[ ∞

X

P Ai = P (Ai).

i=1 i=1

Probability

• Sample space: Ω = {ω1, . . . , ωn}, finite!

• Events: F = 2Ω: All subsets of Ω

P

• Probability: P (ωi) = pi ⇒ P (A ∈ Ω) = pi

w i ∈A

Probability axioms of Kolmogorov (1931) for elementary probability:

• P(Ω) = 1.

• If A ∈ Ω then P (A) ≥ 0.

• If A1, A2, A3, ... ∈ Ω are mutually disjoint, then

∞

[ ∞

X

P Ai = P (Ai).

i=1 i=1

Uncountable sample spaces

• Obviously, P(ω) = 0.

• Intuitively, P([0, a]) = a, basic concept: length!

Answer: No! (e.g. Vitali sets, Banach-Tarski paradox)

Answer: No!

Answer: A lot!

Fundamental mathematical tools

Not every subset of [0, 1] has a determinable length ⇒ collect the ones with a

determinable length in F . Such a mathematical construct, which has additional,

desirable properties, is called σ -algebra.

Definition 2. σ -algebra

A collection F of subsets of Ω is called a σ -algebra on Ω if

• Ω ∈ F and ∅ ∈ F (∅ denotes the empty set)

• If A ∈ F then Ω\A = Ac ∈ F : The complementary subset of A is also

in Ω

S∞

• For all Ai ∈ F : i=1 Ai ∈ F

The intuition behind it: collect all events in the σ -algebra F , make sure that

by performing countably many elementary set operation (∪, ∩,c ) on elements

of F yields again an element in F (closeness).

The pair {Ω, F } is called measure space.

Example of σ-algebra

• Ω = {HH, HT, T H, T T } = {ω1, ω2, ω3, ω4}

• Fmin = {∅, Ω} = {∅, {ω1, ω2, ω3, ω4}}.

• F1 = {∅, {ω1, ω2}, {ω3, ω4}, {ω1, ω2, ω3, ω4}}.

• Fmax = {∅, {ω1}, {ω2}, {ω3}, {ω4}, {ω1, ω2}, {ω1, ω3}, {ω1, ω4}, {ω2, ω3},

{ω2, ω4}, {ω3, ω4}, {ω1, ω2, ω3}, {ω1, ω2, ω4}, {ω1, ω3, ω4},

{ω2, ω3, ω4}, Ω}.

Generated σ-algebras

Let C be a class of subsets of Ω. The σ -algebra generated by C , denoted

by σ(C), is the smallest σ -algebra F which includes all elements of C , i.e.,

C ∈ F.

we then just work with the σ -algebra generated by A and have avoided all the

measure theoretic technicalities.

Borel σ-algebra

The Borel σ -algebra includes all subsets of R which are of interest in practical

applications (scientific or engineering).

Definition 4. Borel σ -algebra B(R))

The Borel σ -algebra B(R) is the smallest σ -algebra containing all open

intervals in R. The sets in B(R) are called Borel sets. The extension to the

multi-dimensional case, B(Rn), is straightforward.

• (−∞, a), (b, ∞), (−∞, a) ∪ (b, ∞)

• [a, b] = (−∞, a) ∪ (b, ∞),

S∞ S∞

• (−∞, a] = n=1[a − n, a] and [b, ∞) = n=1[b, b + n],

• (a, b] = (−∞, b] ∩ (a, ∞),

T∞

• {a} = n=1(a − n1 , a + n1 ),

Sn

• {a1, · · · , an} = k=1 ak .

Measure

Definition 5. Measure

Let F be the σ -algebra of Ω and therefore (Ω, F ) be a measurable space.

The map

µ : F → [0, ∞]

is called a measure on (Ω, F ) if µ is countably additive. The measure µ

is countably additive (or σ -additive) if µ(∅) = 0 and for every sequence of

S

disjoint sets (Fi : i ∈ N) in F with F = i∈N Fi we have

X

µ(F ) = µ(Fi).

i∈N

have

µ(F ∪ G) = µ(F ) + µ(G) if and only if F ∩ G = ∅

The triple (Ω, F , µ) is called a measure space.

Lebesgue Measure

The measure of length on the straight line is known as the Lebesgue measure.

Definition 6. Lebesgue measure on B(R)

The Lebesgue measure on B(R), denoted by λ, is defined as the measure on

(R, B(R)) which assigns the measure of each interval to be its length.

Examples:

• Lebesgue measure of one point: λ({a}) = 0.

P∞

• Lebesgue measure of countably many points: λ(A) = i=1 λ({ai }) = 0.

• The Lebesgue measure of a set containing uncountably many points:

– zero

– positive and finite

– infinite

Probability Measure

A probability measure P on the sample space Ω with σ -algebra F is a set

function

P : F → [0, 1],

satisfying the following conditions

• P(Ω) = 1.

• If A ∈ F then P (A) ≥ 0.

• If A1, A2, A3, ... ∈ F are mutually disjoint, then

∞

[ ∞

X

P Ai = P (Ai).

i=1 i=1

F-measurable functions

The function f : Ω → R defined on (Ω, F , P ) is called F -measurable if

−1

f (B) = {ω ∈ Ω : f (ω) ∈ B} ∈ F for all B ∈ B(R),

easier to work with following equivalent condition:

y ∈ R ⇒ {ω ∈ Ω : f (ω) ≤ y} ∈ F

This means that once we know the (random) value X(ω) we know which of

the events in F have happened.

• F = {∅, Ω}: only constant functions are measurable

• F = 2Ω: all functions are measurable

F-measurable functions

X(ω) : Ω 7→ R R

6

t

j X(ω) ∈ R

ω A2

A1

(−∞, a] ∈ B

A3 Y

Ω A4 X −1 : B 7→ F

F-measurable functions - Example

We only know, whether an even or and odd number has shown up:

F = {∅, {ω1, ω3, ω5}, {ω2, ω4, ω6}, Ω} = σ({ω1, ω3, ω5})

1, if ω = ω1, ω2, ω3;

f (ω) =

−1, if ω = ω4, ω5, ω6.

y ∈ R ⇒ {ω ∈ Ω : f (ω) ≤ y} ∈ F

/ F ⇒ f is not F -measurable.

Lebesgue integral I

(Ω, F ) a measure space, µ : Ω → R a measure, f : Ω → R is F -measurable.

• If f is a simple function, i.e., f (x) = ci, for all x ∈ Ai , ci ∈ R

Z n

X

f dµ = ciµ(Ai).

Ω i=1

fn with fn(x) ≤ fn+1(x) which converges to f : limn→∞ fn(x) = f (x).

With this sequence, the Lebesgue integral is defined by

Z Z

f dµ = lim fndµ.

Ω n→∞ Ω

Lebesgue integral II

(Ω, F ) a measure space, µ : Ω → R a measure, f : Ω → R is F -measurable.

• If f is an arbitrary, measurable function, we have f = f + − f − with

+ −

f (x) = max(f (x), 0) and f (x) = max(−f (x), 0),

Z Z Z

+ −

f dµ = f dP − f dP.

Ω Ω Ω

f +dP

R

The Rintegral above may be finite or infinite. It is not defined if Ω

and Ω f −dP are both infinite.

Riemann vs. Lebesgue

The most important concept of the Lebesgue integral is that the limit of

approximate sums (as the Riemann integral): for Ω = R:

f (x) f (x)

6 6

- x - x

∆x

Riemann vs. Lebesgue integral

Let f be a bounded and continuous function on [x1, x2] except at a countable

number of points in [x1, x2]. Then both the Riemann and the Lebesgue integral

with Lebesgue measure µ exist and are the same:

Z x2 Z

f (x) dx = f dµ.

x1 [x1 ,x2 ]

There are more functions which are Lebesgue integrable than Riemann integra-

ble.

Popular example for Riemann vs. Lebesgue

0, x ∈ Q;

f (x) =

1, x ∈ R\Q.

f (x)dx

0

does not exist, since lower and upper sum do not converge to the same value.

However, the Lebesgue integral

Z

f dλ = 1

[0,1]

Random Variable

A real-valued random variable X is a F -measurable function defined on a

probability space (Ω, F , P ) mapping its sample space Ω into the real line R:

X : Ω → R.

Distribution function

The distribution function of a random variable X, defined on a probability space

(Ω, F , P ), is defined by:

Density function

Closely related to the distribution function

R be a nonnegative function, satisfying R f dλ = 1. The function f is called

a density function (with respect to the Lebesgue measure) and the associated

probability measure for a random variable X , defined on (Ω, F , P ), is

Z

P ({ω : ω ∈ A}) = f dλ.

A

for all A ∈ F .

Important Densities I

λx −λ

f (x) = e , x = 0, 1, 2, . . . .

x!

1 x−µ

− 1

f (x) = √ e 2 σ

2πσ

• Multivariate normal density (x, µ ∈ Rn; Σ ∈ Rn×n):

−2

f (x) = p e .

(2π)ndet (Σ)

Important Densities II

Γ( ν+12 )

1 (x − µ)2 − 12 (ν+1)

f (x) = √ 1+

Γ( ν2 ) πνσ ν σ2

Rn×n):

Γ( ν+n ) 1 − 1 (ν+n)

2 T −1 2

f (x) = ν

p 1 + (x − µ) Σ (x − µ) .

n

Γ( 2 ) (πν) det (Σ) ν

Important Densities II

• The chi square distribution with degree-of-freedom (dof) n has the following

density

−x x

n−2

e 2 2 2

f (x) =

2Γ( n2 )

which is abreviated as Z ∼ χ2(n) and where Γ denotes the gamma

function.

• A chi square distributed random variable Y is created by

n

X 2

Y = Xi

i=1

N (0, 1).

Important Densities III

X

Y =q ,

Z

ν

• Another important density is the Laplace distribution:

1 − |x−µ|

p(x) = e σ

2σ

with mean µ and diffusion σ . The variance of this distribution is given as

2σ 2.

Expectation & Variance

The expectation of a random variable X, defined on a probability space

(Ω, F , P ), is defined by:

Z Z

E[X] = XdP = xf dλ.

Ω Ω

With this definition at hand, it does not matter what the sample Ω is. The

calculations for the two familiar cases of a finite Ω and Ω ≡ R with continuous

random variables remain the same.

Definition 14. Variance of a random variable

The variance of a random variable X, defined on a probability space (Ω, F , P ),

is defined by:

Z

2 2 2 2

var(X) = E[(X − E[X]) ] = (X − E[X]) dP = E[X ] − E[X] .

Ω

Normally distributed random variables

variables with parameters µ and σ is often found in the literature. The

following properties are useful when dealing with normally distributed random

variables:

• If X ∼ N (µ, σ 2)) and Y = aX + b, then Y ∼ N (aµ + b, a2σ 2)).

• If X1 ∼ N (µ1, σ12)) and X2 ∼ N (µ2, σ22)) then

X1 + X2 ∼ N (µ1 + µ2, σ12 + σ22) (if X1 and X2 are independent)

Conditional Expectation I

P (A ∩ B) P (B|A)P (A)

P (A|B) = = , P (B) > 0.

P (B) P (B)

A A∩B B

E(XIB )

E(X|B) = , P (B) > 0.

P (B)

Conditional Expectation II

Definition 15. Conditional expectation

Let X be a random variable defined on the probability space (Ω, F , P ) with

E[|X|] < ∞. Furthermore let G be a sub-σ -algebra of F (G ⊆ F ). Then

there exists a random variable Y with the following properties:

1. Y is G -measurable.

2. E[|Y |] < ∞.

3. For all sets G in G we have

Z Z

Y dP = XdP, for all G ∈ G.

G G

Conditional Expectation: Example

Y = E[X|G]

X(ω)

6

X

Y

- ω

G1 G2 G3 G4 G5

Y = E[X|{∅, Ω}] = Ω XdP = E[X].

Conditional Expectation: Properties

• E(E(X|F )) = E(X).

• If X is F -measurable, then E(X|F ) = X .

• Linearity: E(αX1 + βX2|F ) = αE(X1|F ) + βE(X2|F ).

• Positivity: If X ≥ 0 almost surely, then E(X|F ) ≥ 0.

• Tower property: If G is a sub-σ -algebra of F , then

E(ZX|G) = Z · E(X|G).

Summary

operations

• Borel σ -algebra: all the events of practical importance in R

• Lebesgue measure: defined as the length of an interval

• Density: transforms Lebesgue measure in a probability measure

• Measurable function: the σ -algebra of the probability space is ”rich” enough

• Random variable X : a measurable function X : Ω 7→ R

• Expectation, Variance

• Conditional expectation is a piecewise linear approximation of the underlying

random variable.

- 121_vgGray R. - Probability, Random Processes, And Ergodic PropertiesUploaded bydavid
- mathgen-1962475817Uploaded byAndrewHung
- Introduction-Course Outline MAE419Uploaded byAsrafil Anuar
- Probability TheoryUploaded byVaibhav P. Bhagwat
- Lecture8_Infinite Coin TossUploaded byAnonymous uuSF7AN4
- Real Analysis Lecture Notes.pdfUploaded byphoeberamos
- Data Structures and Algorithms in Java (6e) - Useful Mathematical Facts.pdfUploaded bySAP Fan
- A General Rudin-Carleson Theorem - Errett Bishop (AMS 13-11-1960)Uploaded byLeandro Az
- Lecture 5&6Uploaded byHimura
- stat_TDUploaded byBruno Gomes
- 1 ProbabilityUploaded byMandar Priya Phatak
- 10W 23plan345 SyllabusUploaded bykoechkc
- 9781849962315-c1 (4)Uploaded byAgam Reddy M
- stochastic lecturesUploaded byomidbundy
- Homeomorphisms of Hashimoto TopologiesUploaded byscribd4tavo
- Chap 2 Probability DistributionUploaded byAmmar Fitri
- Communication Theory SampleUploaded byManoj1508Aravind
- fseriesUploaded byDave Ojino
- Galloway (a Treatise on Probability_7ed Encyclopaedia Britannica ) (1839)Uploaded byMarcelo Silvano de Camargo
- lect1aUploaded bydaspranab
- Syllabi_MTechUploaded bySudhir Kumar
- ExchangeableGraphs1.pdfUploaded byAnonymous QHPi951U
- Probability Theory - VaradhanUploaded byTom Hen
- Probability Distribution Homework Help1Uploaded byStatisticsAssignmentExperts
- Decision TheoryUploaded byarupbhatt
- Inde-Depe Events ExamplesUploaded byjally
- Stat 130 Module 1 b SlidesUploaded byambonulan
- StatsUploaded byTusharr Ahuja
- oil warsUploaded byLeonardo Prakoso Soekandar
- Notes4Uploaded byMichael Garcia

- 27372_metnumUploaded byajengpn
- Uji NormalitasUploaded byajengpn
- Transformation Method for Bivariate CaseUploaded byajengpn
- Transformation Method for Bivariate CaseUploaded byajengpn
- 36743-O0VVH01Uploaded byajengpn
- Bab v Aplikasi Pd Orde DuaUploaded bymantapto
- 9.pdfUploaded bydimitry kurenlynko
- x1.txtUploaded byajengpn
- y.docxUploaded byajengpn
- KP_COVERUploaded byajengpn
- persamaan-differensial-biasaUploaded byajengpn
- yUploaded byajengpn
- kalman filter general introdutionUploaded byKira Chiến ND
- MI37slides KalmanUploaded byAndreiSandu
- coba-coba1Uploaded byajengpn
- Remedial Sejarah Ajeng PuspitaningrumUploaded byajengpn
- Biologi SMA Kelas 1 oleh Ari SulistyoriniUploaded byDwi Puji Astini
- Seperti Yang Kita Tahgu Bwsahwa Singapura Termasuk Salah Satu Negara Maju Di Asia Selain JepangUploaded byajengpn
- The Theory of AtomUploaded byajengpn
- Tugas Teori Peluang Distribusi DiskritUploaded byajengpn
- Pendapat Dan Teori Masuknya Islam Di IndonesiaUploaded byajengpn
- Laporan Tugas Teori PeluangUploaded byajengpn
- P. Umum -SUSM 2013Uploaded bywarjoko
- Perang DinginUploaded byajengpn
- UNDANGUploaded bySinggih Prana Putra
- Pengaruh Dan Fasilitas Kegiatan Ekstrakulikuler Di SMA Negeri 1 Balikpapan Bagi Siswa (Autosaved)Uploaded byajengpn
- Label RuanganUploaded byajengpn
- Peranan Organisasi Islam Pada Masa Kemerdekaan Dan Pembangunan NasionalUploaded byajengpn
- Tugas OpalUploaded byajengpn

- Prob NotesUploaded byRoger Figueroa Quintero
- theory_of_probability_Zitcovic.pdfUploaded byJamesBond GuitarPlayer
- ergodicUploaded byalin444444
- Cantor Function107Uploaded byJuliano Deividy B. Santos
- Measures on Metric Spaces and Their Weak ConvergenceUploaded bypieropieru
- analisisUploaded byMat Pérez
- An epsilon of room Pages from year three of a mathematical blogUploaded byPiratereader
- CL202 FundamentalsUploaded byShubham Titare
- Probability and MeasureUploaded byChiara Lattanzi
- AE2015 Lecture Notes Ch4(2)Uploaded byEmilio
- Probabilty notes at IITUploaded byrapsjade
- Stochastic Processes and ApplicationsUploaded byApratim Bhattacharya
- Stochastic ProcessesUploaded byBrenda Wamala Lutaaya
- OMN-201612-110654-1-Course_notes-v1.pdfUploaded bycjmaura
- Borel Sets and Lebesgue MeasureUploaded byread41
- [a. a. Kirillov] Elements of the Theory of RepreseUploaded bywillmate
- Integration in Banach SpacesUploaded byjhorckham
- Basic Ergodic Theory NotesUploaded byAgus Leonardi
- Stats StanfordUploaded byAlexey Shevlaykov
- Probability theory sigma field and measuresUploaded bygalaxystar
- Math 891Uploaded byAravindVR
- [James R. Munkres] a Short Course on the Lebesgue (BookFi.org)Uploaded byJosé Miguel Flores
- stml-42-prev.pdfUploaded byAna-Maria V Ciobanu
- Questions of RegularityUploaded byZeba Mullick
- Absolute Measurable Spaces Encyclopedia of Mathematics and Its ApplicationsUploaded bymario
- Constructions and extensions of measuresUploaded byiscopear
- Probability NotesUploaded byTomas Kojar
- Real Prjct11Uploaded byMuzna Zainab
- Open Problems in TopologyUploaded byValentín Mijoevich
- Sigma Field Notes 770Uploaded bydevils365