Professional Documents
Culture Documents
decision/optimization/value problems
Lecture Outline
uniform
optimization
Markov Chains
Notation & Terminology
time-homogeneous
Markov Chains
Examples
Markov Chain
Markov Chains
Notation & Terminology
dimension ||x||
Then
t
Markov Chains
Fundamental Properties
Theorem:
DEMO
Markov Chains
Fundamental Properties
Proposition:
Corollary:
DEMO
={0,1}m
Definition
Representation:
MCMC Example:
Knapsack Problem
Comments:
MCMC Example:
Knapsack Problem
Resulting pij:
pij=qijmin{1, (j)/(i)} for i j
pii = 1 - ji pij
Remarks:
Target distribution:
(z) = Cb-1exp( b ivi zi )
Algorithm:
(1) choose J{1,...,m} uniformly at random
(2) let y = (z1,...,1-zJ ,...,zm)
(3) if y is not feasible, then Xt+1 = Xt
(4) if y is feasible, set Xt+1 = y with prob. , else Xt+1 = Xt
where = min{1, exp( b ivi (yi-zi)}
MCMC: Optimization
Simulated Annealing
Algorithm:
(1) pick a component i{1,...,m}
(2) sample value of zi from P[Zi |z-i ], set Xt=(z1 ,...,zm )
MCMC: Relevance to
Bayesian Networks
Possible solution:
(1) sample from worlds according to P[Z=z|E=e]
(2) compute fraction of those worlds where Zi=zi
MCMC: Inference in BN
Example
MCMC: Inference in BN
Example
p(h,l) (h,l)=?
h,l
h,l
h,l
h,l
MCMC: Inference in BN
Example
p(h,l) (h,l)