1

HIOINFOH^ATICS ^Sc
IHOHAHILIT ANO STATISTICS FOH^lLA SHEET
SET THEOHY DEFIITIOS AD HES\LTS
Lvonls 1 ano 1 aio nutuaIIy excIusIve if 1 1 = O (lho enty set).
Lvonls 1

, ..., 1
I
foim a artItIon of ovonl 1 _ o if
(a) 1
i
1
)
= O foi all i ano (L)
I
_
i
1
i
= 1

' 1

' ... ' 1

I
= 1.
THE H\LES OF IHOHAHILITY: Foi any ovonls 1 ano 1 in samplo spaco o,
(1) 0 _ 1(1) _ 1
(2) 1(o) = 1
(8) If 1 1 = O, lhon 1(1 ' 1) = 1(1) 1(1)
CoroIIarIes :
1(1

) = 1 1(1), 1(O) = 0
If 1

, ..., 1
I
aio ovonls such lhal 1
i
1
)
= O foi all i, , lhon
1
_
I
_
i
1
i
_
= 1(1

) 1(1

) ... 1(1
I
)
.
If 1 1 ,= O, lhon 1(1 ' 1) = 1(1) 1(1) 1(1 1)
CODITIOAL IHOHAHILITY
1(1[1) is lho pioLaLilily lhal lho ovonl 1 occuis, givon lhal 1 Las occuiioo, foi an ovonl 1 such lhal
I(1) 0, ano
1(1[1) =
1(1 1)
1(1)
1ho pioLaLilily of lho IntersectIon of ovonls 1

, ..., 1
I
is givon Ly lho cLaIn ruIe
1(1

... 1
I
) = 1(1

)1(1

[1

)1(1

[1

)...1(1
I
[1

... 1
I
)
Lvonls 1 ano 1 aio Indeendent if
1(1[1) = 1(1) so lhal 1(1 1) = 1(1)1(1).
THEOHEA OF TOTAL IHOHAHILITY : If ovonls 1

, ..., 1
I
foim a paililion of ovonl 1 _ o
1(1) =
I

i
1(1[1
i
)1(1
i
)
HAYES THEOHEA: If ovonls 1

, ..., 1
I
foim a paililion of ovonl 1 _ o,
1(1
i
[1) =
1(1[1
i
)1(1
i
)
1(1)
=
1(1[1
i
)1(1
i
)
I

)
1(1[1
)
)1(1
)
)
2
DISCHETE IHOHAHILITY DISTHIH\TIOS
1ho pioLaLilily oisliiLulion of a J.: ianoom vaiiaLlo A is oosciiLoo Ly lho roLaLIIIty nass
IunctIon )
A
, spociloo Ly
)
A
(a) = 1[A = a[ a = a

, a

, ..., a
a
, ...
Iiopoilios of lho mass funclion :
(i) )
A
(a
i
) _ 0 (ii)

i
)
A
(a
i
) = 1
1ho cumulalivo oisliiLulion funclion oi c.o.f., 1
A
, is oolnoo Ly
1
A
(a) = 1[A _ a[ a
Funoamonlal iolalionship Lolwoon )
A
ano 1
A
:
1
A
(a) =

a

a
)
A
(a
i
)
)
A
(a

) = 1
A
(a

)
)
A
(a
i
) = 1
A
(a
i
) 1
A
(a
i
) foi i _ 2
COTI\O\S IHOHAHILITY DISTHIH\TIOS:
1ho pioLaLilily oisliiLulion of a n.n&& ianoom vaiiaLlo A is oolnoo Ly lho conlinuous cunuIatIve
dIstrILutIon IunctIon oi c.d.I., 1
A
, spociloo Ly
1
A
(a) = 1[A _ a[ foi a
1ho roLaLIIIty densIty IunctIon, oi .d.I., )
A
, is oolnoo Ly
)
A
(a) =
d
da
1
A
(a) so lhal 1
A
(a) =
_
a

)
A
(t) dt
IroertIes oI tLe densIty IunctIon
(i) )
A
(a) _ 0 a (ii)
_

)
A
(a)da = 1.
EXIECTATIO AD YAHIACE
Foi a dIscrete ianoom vaiiaLlo A laking valuos in sol wilh mass funclion )
A
, lho exectatIon of
A is oolnoo Ly
1
)

[A[ =

a
a)
A
(a)
Foi a contInuous ianoom vaiiaLlo A laking valuos in inloival wilh pof )
A
, lho oxpoclalion of A is
oolnoo Ly
1
)

[A[ =
_

a)
A
(a) da.
1ho varIance of A is oolnoo Ly
\ ar
)

[(A 1
)

[A[)

[ = 1
)

[A

[ 1
)

[A[

.
8
DISCHETE IHOHAHILITY DISTHIH\TIOS
TLe HernouIII DIstrILutIon A ~ 1cr:on||i(0)
Iango : = 0, 1
Iaiamoloi : 0 [0, 1[
Mass funclion :
)
A
(a) = 0
a
(1 0)
a
a 0, 1
TLe HInonIaI DIstrILutIon A ~ 1i:o:ia|(:, 0)
Iango : = 0, 1, ..., :
Iaiamolois : : 7

, 0 [0, 1[
Mass funclion :
)
A
(a) =
_
:
a
_
0
a
(1 0)
aa
=
:!
a!(: a)!
0
a
(1 0)
aa
a 0, 1, ..., :
TLe CeonetrIc DIstrILutIon A ~ Gco:ctric(0)
Iango : = 1, 2, ...
Iaiamoloi : 0 (0, 1[
Mass funclion :
)
A
(a) = (1 0)
a
0 a 1, 2, ...
DisliiLulion funclion
1
A
(a) = 1 (1 0)
a
a 1, 2, ...
TLe egatIve HInonIaI DIstrILutIon A ~ cp1i:(:, 0)
Iango : = :, : 1, : 2, ...
Iaiamoloi : : 7

, 0 (0, 1[
Mass funclion :
)
A
(a) =
_
a 1
: 1
_
0
a
(1 0)
aa
a :, : 1, : 2, ... .
TLe IoIsson DIstrILutIon A ~ 1oi--o:(`)
Iango : = 0, 1, 2, ...
Iaiamoloi : `

Mass funclion :
)
A
(a) =
`
a
a!
c
A
a 0, 1, 2, ...
4
COTI\O\S IHOHAHILITY DISTHIH\TIOS
TLe ExonentIaI DIstrILutIon A ~ 1ajo:c:tia|(`)
Iango : =

Iaiamoloi : ` 0
Donsily funclion :
)
A
(a) = `c
Aa
a _ 0
DisliiLulion funclion:
)
A
(a) = 1 c
Aa
a _ 0
TLe Canna DIstrILutIon A ~ Ga::a(c, a)
Iango : =

Iaiamolois : c, a 0
Donsily funclion :
)
A
(a) =
a
c
I(c)
a
c
c
oa
a _ 0 whoio I(c) =
_

t
c
c
|
dt c 0.
If c 1, I(c) = (c 1)I(c 1), so if c = 1, 2, ..., I(c) = (c 1)!.
If c = 1, 2, ..., lhon lho Ga::a(c2, 12) oisliiLulion is known as lho CLI-squared dIstrILutIon
wilh c degrees oI Ireedon, oonoloo

c
.
If A

, A

~ 1ajo:c:tia|(`) aio inooponoonl, lhon 1 = A

~ Ga::a(2, `).
TLe ornaI DIstrILutIon A ~ (j, o

)
Iango : =
Iaiamolois : < j < , o 0
Donsily funclion :
)
A
(a) =
_
1
2o

oxp
_

1
2o

(a j)

_
< a <
If j = 0, o = 1, lhon 1 ~ (0, 1) has a standard noimal oisliiLulion
If A ~ (0, 1), ano 1 = oA j, lhon 1 ~ (j, o

)
If A ~ (0, 1), ano 1 = A

, lhon 1 ~ Ga::a(12, 12) =

.
If A ~ (0, 1) ano 1 ~

c
aio inooponoonl ianoom vaiiaLlos, lhon ianoom vaiiaLlo T = A
_
1c
has a t dIstrILutIon wilh c degrees oI Ireedon.
THE IOISSO IHOCESS
In lho Ioisson piocoss moool foi ovonls lhal occui al ianoom in conlinuous limo wilh conslanl ialo `,
lhoio aio lhioo iolaloo pioLaLilily oisliiLulion iosulls
lho numLois of ovonls occuiiing in oisjoinl inloivals of longlhs t

, t

, t

, ... aio inooponoonl ianoom

vaiiaLlos A

, A

, A

, ... wilh A
i
~ 1oi--o:(`t
i
)
lho limos Lolwoon lho occuiioncos of ovonls aio inooponoonl conlinuous ianoom vaiiaLlos T

, T

, T

, ...
wilh T
i
~ 1ajo:c:tia| (`)
lho limo of lho :lh ovonl is a conlinuous ianoom vaiiaLlo 1
a
wilh 1
a
~ Ga::a (:, `)

THE CETHAL LIAIT THEOHEA

THEOHEA: Supposo A

, ..., A
a
aio i.i.o. ianoom vaiiaLlos wilh 1
)

[A
i
[ = j, \ ar
)

[A
i
[ = o

. If
7
a
is oolnoo Ly
7
a
=
a

i
A
i
:j
_
:o

1hon, as : , 7
a
7 ~ (0, 1) IrresectIve of lho oisliiLulion of A

, ..., A
a
.
AAXIA\A LIKELIHOOD IFEHECE
Supposo a samplo a

, ..., a
a
has Loon oLlainoo fiom a pioLaLilily moool spociloo Ly mass oi oonsily
funclion )(a; 0) ooponoing on paiamoloi(s) 0 lying in paiamoloi spaco O. 1ho naxInun IIkeIILood
estInate oi n.I.e. is piooucoo as follows;
STEI 1 Wiilo oown lho IIkeIILood IunctIon
1(0) =
a

i
)(a
i
; 0)
STEI 2 1ako lho naluial log of lho likolihooo, ano collocl loims involving 0.
STEI 3 Fino lho valuo of 0,

0, foi which |op1(0) is maximizoo in O.
STEI 4 Voiify lhal

0 maximizos |op1(0).
SAAILIC DISTHIH\TIOS
THEOHEA If A

, ..., A
a
aio i.i.o. (j, o

) ianoom vaiiaLlos, lhon if

A =
1
:
a

i
A
i
o

=
1
:
a

i
(A
i

A)

=
1
: 1
a

i
(A
i

A)

aio lho nean, varIance, ano adjusted varIance, lhon il can Lo shown lhal
(1) : A ~
_
j,
o

:
_
(2) :
(: 1)-

a
(8) :

A ano -

aio slalislically inooponoonl.

6
HYIOTHESIS TESTIC FOH OHAAL DATA
OE-SAAILE TESTS
Supposo a

, ..., a
a
~
_
j, o

_
, wilh oLsoivoo samplo moan ano aojusloo vaiianco a, -

. 1o losl lho
LyotLesIs
H

: j = c
H

: j ,= c
if o is known, uso lho Z-test
. =
a c
o
_
:
~ (0, 1) if H

is 1IUL.
If o is unknown, uso lho T-test
t =
( a c)
-
_
:
~ otndc:t(: 1) if H

is 1IUL
whoio t
a
is lho otndc:t (: 1) oisliiLulion.
1o losl H

: o

= c, calculalo losl slalislic

=
(: 1)-

c
~

a
if H

is 1IUL
TVO-SAAILE TESTS
Foi lwo oala samplos of sizo :

ano :

, whoio a

ano a

aio lho samplo moans, ano -

ano -

aio lho
aojusloo samplo vaiiancos; lo losl lho hypolhosis
H

: j

= j

: j

,= j

if o

= o

= o is knovn uso lho slalislic ., oolnoo Ly

. =
a

o
_
1
:

1
:

~ (0, 1) if H

is 1IUL
If o

= o

= o is unknovn, uso lho slalislic t, oolnoo Ly

t =
a

-
1
_
1
:

1
:

~ t
aa
if H

is 1IUL
whoio -

1
= ((:

1)-

(:

1)-

)(:

2) is lho ooIed oslimalo of o

.
1o losl lho hypolhosis H

: o

= o

, uso lho 1 slalislic

1 =
-

~ 1i-/cr (:

1, :

1) if H

is 1IUL
7
95 % COFIDECE ITEHYALS FOH IAHAAETEHS
Lol t
I
(j) Lo lho jlh poiconlilo of a Sluoonl t oisliiLulion wilh / oogioos of fioooom.
OE-SAAILE: 0 / Conloonco inloival foi j is
a 1.06o
_
: if o is known
a t
a
(0.07)-
_
: if o is unknown
0 / Conloonco inloival foi o

is
_
(: 1)-

: (: 1)-

whoio c

ano c

aio lho 0.02 ano 0.07 poinls of lho

a
oisliiLulion.
TVO-SAAILE: 0 / Conloonco inloival foi j

is
a

1.06o
_
1
:

1
:

if o is known
a

t
a

(0.07)-
1
_
1
:

1
:

if o is unknown
0 / Conloonco inloival foi o

is
_
-

(c

)
:
-

(c

)
_
whoio c

ano c

aio lho 0.02 ano 0.07 poinls of lho 1i-/cr (:

1, :

1) oisliiLulion.
THE CHI-SQ\AHED AD LIKELIHOOD HATIO TEST
1o losl lho gooonoss-of-ll of a pioLaLilily moool lo a samplo of sizo :, uso lho cLI-squared statIstIc

=
I

i
(O
i
1
i
)

1
i
.
If H

is liuo, lhon

appioximaloly has a wilh / d 1 oogioos of fioooom, whoio d is lho numLoi of

oslimaloo paiamolois.
Foi a conlingoncy laLlo wilh r iows ano c columns, lho

slalislic

=
v

i
c

)
(:
i)
:
i)
)

:
i)
foi a losl of inooponoonco has a null oisliiLulion lhal is chi-squaioo wilh (r 1) (c 1) oogioos of
fioooom, whoio
:
i)
= :
i.
j
)
=
:
i.
:
.)
:
i = 1, ..., r, = 1, ..., c
ano :
i.
is lho lolal of lho ilh iow, :
.)
is lho lolal of lho lh column, ano : is lho lolal numLoi of
oLsoivalions.
1ho Likolihooo Ialio slalislic 11 has lho samo appioximalo null oisliiLulion, ano is oolnoo Ly
11 = 2
v

i
c

)
:
i)
log
:
i)
:
i)
8
CLASSIFICATIO FOH TVO CLASSES (1 = 2)
Lol )

(a) ano )

(a) Lo lho pioLaLilily funclions associaloo wilh a (vocloi) ianoom vaiiaLlo A foi
lwo populalions 1 ano 2. An oLjocl wilh moasuiomonls a musl Lo assignoo lo oilhoi class 1 oi class 2.
Lol oonolo lho samplo spaco. Lol

Lo lhal sol of a valuos foi which wo classify oLjocls inlo class

1 ano

Lo lho iomaining a valuos, foi which wo classify oLjocls inlo class 2.

1ho condItIonaI roLaLIIIty, 1(2[1), of classifying an oLjocl inlo class 2 whon, in facl, il is fiom
class1 is:
1(2[1) =
_

(a) da.
Similaily, lho conoilional pioLaLilily, 1(1[2), of classifying an oLjocl inlo class 1 whon, in facl, il is
fiom class 2 is:
1(1[2) =
_

(a) da
Lol j

Lo lho :.: pioLaLilily of Loing in class 1 ano j

Lo lho :.: pioLaLilily of 2, whoio j

= 1.
1hon,
1 (OLjocl coiioclly classiloo as class 1) = 1(1[1)j

1 (OLjocl misclassiloo as class 1) = 1(1[2)j

1 (OLjocl coiioclly classiloo as class 2) = 1(2[2)j

1 (OLjocl misclassiloo as class 2) = 1(2[1)j

Now supposo lhal lho of misclassilcalion of a class 2 oLjocl as a class 1 oLjocl, ano vico voisa
aio, iospoclivoly.c (1[2) ano c(2[1). 1hon lho oxpocloo cosl of misclassilcalion is lhoiofoio
c(2[1)1(2[1)j

c (1[2) 1(1[2)j

.
1ho iooa is lo chooso lho iogions

ano

so lhal lhis oxpocloo cosl is minimizoo. 1his can Lo

achiovoo Ly compaiing lho piooiclivo pioLaLilily oonsily funclions al oach poinl a

=
_
a :
)

(a)
)

(a)
j

_
c (1[2)
c (2[1)
_

=
_
a :
)

(a)
)

(a)
j

<
c (1[2)
c (2[1)
_
If j

= j

, lhon

=
_
a :
)

(a)
)

(a)
_
c (1[2)
c (2[1)
_
ano if c (1[2) = c (2[1), oquivalonlly

=
_
a :
)

(a)
)

(a)
_
j

_
ano lnally if j

= j

ano c (1[2) = c (2[1) lhon

=
_
a :
)

(a)
)

(a)
_ 1
_
= a : )

(a) _ )

(a)