Professional Documents
Culture Documents
1
Outline
Syntax
Semantics
Parameterized distributions
2
Bayesian networks
A simple, graphical notation for conditional independence assertions
and hence for compact specification of full joint distributions
Syntax:
a set of nodes, one per variable
a directed, acyclic graph (link directly influences)
a conditional distribution for each node given its parents:
P(Xi|P arents(Xi))
In the simplest case, conditional distribution represented as
a conditional probability table (CPT) giving the
distribution over Xi for each combination of parent values
3
Example
Topology of network encodes conditional independence assertions:
Weather Cavity
Toothache Catch
4
Example
Im at work, neighbor John calls to say my alarm is ringing, but neighbor
Mary doesnt call. Sometimes its set off by minor earthquakes. Is there a
burglar?
Variables: Burglar, Earthquake, Alarm, JohnCalls, M aryCalls
Network topology reflects causal knowledge:
A burglar can set the alarm off
An earthquake can set the alarm off
The alarm can cause Mary to call
The alarm can cause John to call
5
Example contd.
P(B) P(E)
Burglary Earthquake
.001 .002
B E P(A|B,E)
T T .95
Alarm
T F .94
F T .29
F F .001
A P(J|A) A P(M|A)
JohnCalls T .90 MaryCalls T .70
F .05 F .01
6
Compactness
A CPT for Boolean Xi with k Boolean parents has
2k rows for the combinations of parent values B E
7
Global semantics
Global semantics defines the full joint distribution
as the product of the local conditional distributions: B E
n A
P (x1, . . . , xn) = i = 1P (xi|parents(Xi))
e.g., P (j m a b e) J M
8
Global semantics
Global semantics defines the full joint distribution
as the product of the local conditional distributions: B E
n A
P (x1, . . . , xn) = i = 1P (xi|parents(Xi))
e.g., P (j m a b e) J M
9
Local semantics
Local semantics: each node is conditionally independent
of its nondescendants given its parents
U1 Um
...
X
Z 1j Z nj
Y1 Yn
...
10
Markov blanket
Each node is conditionally independent of all others given its
Markov blanket: parents + children + childrens parents
U1 Um
...
X
Z 1j Z nj
Y1 Yn
...
11
D-separation
X E Y
(1) Z
(2) Z
(3) Z
12
Example
Battery
Starts
Moves
Are Gas and Radio independent? Given Battery? Ignition? Starts? Moves?
13
Constructing Bayesian networks
Need a method such that a series of locally testable assertions of
conditional independence guarantees the required global semantics
1. Choose an ordering of variables X1, . . . , Xn
2. For i = 1 to n
add Xi to the network
select parents from X1, . . . , Xi1 such that
P(Xi|P arents(Xi)) = P(Xi|X1, . . . , Xi1)
This choice of parents guarantees the global semantics:
n
P(X1, . . . , Xn) = i = 1P(Xi|X1, . . . , Xi1) (chain rule)
n
= i = 1P(Xi|P arents(Xi)) (by construction)
14
Example
Suppose we choose the ordering M , J, A, B, E
MaryCalls
JohnCalls
P (J|M ) = P (J)?
15
Example
Suppose we choose the ordering M , J, A, B, E
MaryCalls
JohnCalls
Alarm
P (J|M ) = P (J)? No
P (A|J, M ) = P (A|J)? P (A|J, M ) = P (A)?
16
Example
Suppose we choose the ordering M , J, A, B, E
MaryCalls
JohnCalls
Alarm
Burglary
P (J|M ) = P (J)? No
P (A|J, M ) = P (A|J)? P (A|J, M ) = P (A)? No
P (B|A, J, M ) = P (B|A)?
P (B|A, J, M ) = P (B)?
17
Example
Suppose we choose the ordering M , J, A, B, E
MaryCalls
JohnCalls
Alarm
Burglary
Earthquake
P (J|M ) = P (J)? No
P (A|J, M ) = P (A|J)? P (A|J, M ) = P (A)? No
P (B|A, J, M ) = P (B|A)? Yes
P (B|A, J, M ) = P (B)? No
P (E|B, A, J, M ) = P (E|A)?
P (E|B, A, J, M ) = P (E|A, B)?
18
Example
Suppose we choose the ordering M , J, A, B, E
MaryCalls
JohnCalls
Alarm
Burglary
Earthquake
P (J|M ) = P (J)? No
P (A|J, M ) = P (A|J)? P (A|J, M ) = P (A)? No
P (B|A, J, M ) = P (B|A)? Yes
P (B|A, J, M ) = P (B)? No
P (E|B, A, J, M ) = P (E|A)? No
P (E|B, A, J, M ) = P (E|A, B)? Yes
19
Example contd.
MaryCalls
JohnCalls
Alarm
Burglary
Earthquake
battery no charging
dead
car wont
lights oil light gas gauge start dipstick
21
Example: Car insurance
SocioEcon
Age
GoodStudent
ExtraCar
Mileage
RiskAversion
VehicleYear
SeniorTrain
DrivingSkill MakeModel
DrivingHist
Antilock
DrivQuality Airbag CarValue HomeBase AntiTheft
Ruggedness Accident
Theft
OwnDamage
Cushioning OwnCost
OtherCost
22
Compact conditional distributions
CPT grows exponentially with number of parents
CPT becomes infinite with continuous-valued parent or child
Solution: canonical distributions that are defined compactly
Deterministic nodes are the simplest case:
X = f (P arents(X)) for some function f
E.g., Boolean functions
N orthAmerican Canadian U S M exican
E.g., numerical relationships among continuous variables
Level
= inflow + precipitation - outflow - evaporation
t
23
Compact conditional distributions contd.
Noisy-OR distributions model multiple noninteracting causes
1) Parents U1 . . . Uk include all causes (can add leak node)
2) Independent failure probability qi for each cause alone
j
P (X|U1 . . . Uj , Uj+1 . . . Uk ) = 1 i = 1qi
Cold F lu M alaria P (F ever) P (F ever)
F F F 0.0 1.0
F F T 0.9 0.1
F T F 0.8 0.2
F T T 0.98 0.02 = 0.2 0.1
T F F 0.4 0.6
T F T 0.94 0.06 = 0.6 0.1
T T F 0.88 0.12 = 0.6 0.2
T T T 0.988 0.012 = 0.6 0.2 0.1
Number of parameters linear in number of parents
24
Hybrid (discrete+continuous) networks
Subsidy? Harvest
Cost
Buys?
= exp
t 2 2 t
26
Continuous child variables
P(Cost|Harvest,Subsidy?=true)
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
10
0 5
5 Harvest
Cost 10 0
27
Discrete variable w/ continuous parents
Probability of Buys? given Cost should be a soft threshold:
1
0.8
P(Buys?=false|Cost=c)
0.6
0.4
0.2
0
0 2 4 6 8 10 12
Cost c
28
Why the probit?
1. Its sort of the right shape
2. Can view as hard threshold whose location is subject to noise
Buys?
29
Discrete variable contd.
Sigmoid (or logit) distribution also used in neural networks:
1
P (Buys? = true | Cost = c) =
1 + exp(2 c+
)
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 2 4 6 8 10 12
Cost c
30
Summary
Bayes nets provide a natural representation for (causally induced)
conditional independence
Topology + CPTs = compact representation of joint distribution
Generally easy for (non)experts to construct
Canonical distributions (e.g., noisy-OR) = compact representation of CPTs
Continuous variables parameterized distributions (e.g., linear Gaussian)
31