Professional Documents
Culture Documents
p( y | )
p ( )
p ( | y ) p ( y | ) p ( )
posterior
likelihood
prior
Bayes rule
Given data y and parameters , their joint probability can be
written in 2 ways:
p( | y ) p( y ) p( y, )
p( y, ) p( y | ) p( )
Prior
p ( y | ) p ( )
p ( | y )
p( y )
Evidence
Observation of data
yy
Update of beliefs based upon observations, given a
prior state of knowledge
p( | y ) p( y | ) p( )
Univariate Gaussian
Normal densities
p ( ) N ( ; p , p1 )
y e
p ( y | ) N ( y; , e1 )
p( | y) N ( ; , )
1
e p
1 e y p p
Posterior mean =
precision-weighted
combination of prior mean
p ( ) N ( ; p , p1 )
y x e
p( y | ) N ( y; x, e1 )
p( | y) N ( ; , )
1
e x p
2
1 e xy p p
y X e
p( ) N ( ; p , C p )
p( y | ) N ( y; X , Ce )
C X T Ce X C p
p( | y ) N ( ; , C )
1
C X T Ce y C p p
True posterior
p ( | y, m)
iteratively improve
Approximate
posterior
p ( y, | m)
p ( y m)
q ( ) q( i )
i
p ( y , | m)
q ( )
q ( ) log
q
(
)
p
(
|
y
,
m
)
free energy
log p ( y m) q ( ) log
Objective
function
Value of parameter
y X
y
ED ( y X )T ( y X )
ED
0 ols ( X T X ) 1 X T y
y X
ED ( y X )T ( y X )
ED
0 ols ( X T X ) 1 X T y
y X
ED ( y X )T ( y X )
ED
0 ols ( X T X ) 1 X T y
y X
ED ( y X )T ( y X )
ED
0 ols ( X T X ) 1 X T y
y X e
Prior:
y X e
p( 2 ) N k (0, 21 I k )
exp( 2
/ 2)
Prior:
y X e
p( 2 ) N k (0, 21 I k )
exp( 2
/ 2)
Model:
p( 2 ) N k (0, 21 I k )
Prior:
exp( 2
Likelihood:
/ 2)
p ( y , 1 ) p ( yi | , 11 )
i 1
p( yi , 1 ) N ( X i , 11 )
exp( 1 ( yi X i ) 2 / 2)
Model:
p( 2 ) N k (0, 21 I k )
Prior:
exp( 2
Likelihood:
/ 2)
p ( y , 1 ) p ( yi | , 11 )
i 1
p( yi , 1 ) N ( X i , 11 )
exp( 1 ( yi X i ) 2 / 2)
Model:
p( 2 ) N k (0, 21 I k )
Prior:
exp( 2
Likelihood:
/ 2)
p ( y , 1 ) p ( yi | , 11 )
i 1
p( yi , 1 ) N ( X i , 11 )
exp( 1 ( yi X i ) 2 / 2)
Model:
p( 2 ) N k (0, 21 I k )
Prior:
exp( 2
Likelihood:
/ 2)
p ( y , 1 ) p ( yi | , 1 )
i 1
Bayes Rule:
p( y, ) p( y | , ) p( | )
Model:
p( 2 ) N k (0, 21 I k )
Prior:
exp( 2
Likelihood:
/ 2)
p ( y , 1 ) p ( yi | , 1 )
i 1
Bayes Rule:
p( y, ) p( y | , ) p( | )
Posterior:
p | y,
N , C
C 1 X T X 2 I k
1CX T y
Model:
p( 2 ) N k (0, 21 I k )
Prior:
exp( 2
Likelihood:
/ 2)
p ( y , 1 ) p ( yi | , 1 )
i 1
Bayes Rule:
p( y, ) p( y | , ) p( | )
Posterior:
p | y,
N , C
C 1 X T X 2 I k
1CX T y
Model:
p( 2 ) N k (0, 21 I k )
Prior:
exp( 2
Likelihood:
/ 2)
p ( y , 1 ) p ( yi | , 1 )
i 1
Bayes Rule:
p( y, ) p( y | , ) p( | )
Posterior:
p | y,
N , C
C 1 X T X 2 I k
1CX T y
p( | y)
p ( sth | y ) pth
p( | y)
sth
Two thresholds:
activation threshold sth : percentage of whole brain
mean signal (physiologically relevant size of effect)
probability pth that voxels must exceed to be
displayed (e.g. 95%)
p ( y | , , m) p ( | , m)
p ( y , , m)
p ( y , m)
normalizing constant
Model evidence:
p ( y , m) p ( y | , , m) p ( | , m)d
log p ( y | , m)
accuracy (m) complexity(m)
accuracy (m) y X
complexity(m) k log 21 2
aMRI segmentation
PPM of belonging to
grey matter
white matter
CSF
Electric/magnetic
forward model:
neural activityEEG
MEG
LFP
x F ( x, u, )
fMRI
Neural model:
1 state variable per region
bilinear state equation
no propagation delays
inputs
ERPs
Neural model:
8 state variables per region
nonlinear state equation
propagation delays
m2
m3
m4
attention
attention
attention
attention
PPC
stim
V1
PPC
stim
V5
V1
V5
PPC
stim
V1
V5
PPC
stim
V1
V5
attention
models marginal likelihood
ln p y m
15
0.10
PPC
10
1.25
stim
0.26
0.39
0.26
V1 0.13
0.46
m1 m2 m3 m4
V5
estimated
effective synaptic strengths
for best model (m4)
p N 0, 1 L1
Y X
aMRI
prior precision
of GLM coeff
Smooth Y (RFT)
prior precision
of AR coeff
prior precision
of data noise
ML estimate of
VB estimate of
GLM coeff
AR coeff
(correlated noise)
observations
Penny et al 2005
activation
threshold
p pth
p q ( sth )
sth
Probability mass pn
Mean (Cbeta_*.img)
PPM (spmP_*.img)
q( n ) N ( n , n )
Std dev (SDbeta_*.img)
model K
Compute log-evidence
for each model/subject
BMS maps
subject 1
model 1
subject N
rk
PPM
EPM
q (rk )
model K
rk
Compute log-evidence
for each model/subject
model k
Joao et al, 2009
Reminder
Long term memory
Short term memory
IT indices: H,h,I,i
onsets
Missed
trials
H=entropy; h=surprise; I=mutual information; i=mutual surprise
primary visual
cortex
Regions best
explained by
long-term
memory
model
frontal cortex
(executive
control)
Thank-you