SISEA: KALMAN FILTER AND HIDDEN MARKOV MODELS

LAB WORK 2: HIDDEN MARKOV MODELS

Student: HOANG Gia Minh

January 14, 2014

HOANG GIA MINH

1 Introduction
The objective of this lab work is to show how using ingeniously the forward and backward variables permits to detect a potential change of a model, and should it happen to estimate the instant of change?

2 Methodology and simulations

Through this project, we consider a 2-state Hidden Markov Model (HMM) with conditionally Gaussian one-dimensional observations. In other word, the probability density functions of emission are Gaussian with means and variances that depend on the hidden states. In view of the observations (1 , , ), we wish to detect whether there exists or not an instant between the instant 1 and the instant n, such as At the instant 1, the initial law is 0 = (0), Between the instant 1 and the instant , the model is nominal, i.e. the transition matrix is 0 = 0 0 (, )and the densities of emission density is Gaussian with means 0 = ( )and variance 0 = (0), Between the instant + 1 and the instant , the model is nominal, i.e. the transition matrix is 0 = ( )and the densities of emission density is Gaussian with means 0 = ( )and variance 0 = ( ),

Figure 2-1: Studied Hidden Markov Model

0 0 0 0 0 We remind that the likelihood 0 of the nominal model = ( , , , ) given the observations (1 , , ) can be expressed as 0 = =

= 1, ,

using the forward and backward variables corresponding to the alternative model M . We accept the following result: the likelihood ()of the model with an instant of change t given the observations (1 , , )can be expressed as
0, () =

using the forward and backward variables corresponding to the nominal model 0 and corresponding to the alternative model . We also remind that the normalized version of forward variables associating to the nominal model 0 and the modified model M are defined for all instants = 1, , by
0, = 0, 0,

and by

0,

for

HOANG GIA MINH

Respectively, and the normalized version of forward variables associating to the alternative model M are defined for all instants = 1, , by
=

The likelihood () of the model with an instant of change given by the observation (1 , , ) can be expressed by
0, () =

0, = [ ] [ ]

0,

0, 0, = [ ] [ ] [ ]

The likelihood of the alternative model = ( 0 , , , ) given by the observation (1 , , ) can be expressed by
= = [ ] = [ ] [ ] 0,

Then, the likelihood function to estimate a potential instant of change given by the observation (1 , , ) can be expressed by And the log-likelihood function as () 0, = 0 + [ ]

() 0 0, = [ ]

For each of the studied scenarios, given the observations (1 , , ), the task is to estimate the instant of potential change then to compare with the real value (general is unknown) to evaluate the accuracy of the detection using log-likelihood function. The 2-state nominal model 0 = ( 0 , 0 , 0 , 0 ) is defined the following characteristics: 0 = [0.2 0.8] 0.99 0.01 0 = [ ] 0.01 0.99 0 = [2 +2] 0 = [5 5]

CHANGE IN THE MEAN In this scenario, the alternative model = ( 0 , , , ) is defined by the following characteristics: = 0
HOANG GIA MINH

= [10 10]

= 0
3

The 2 hidden state and the observations are visualed in the Figure 2-2. Based on this observation, the forward and backward equation are implemented to detect whether there exists an instant of change. In case of existance, the that instant will be estimated.

Figure 2-2: Visualization of 2 hidden states and observations (change in mean). Based on the observations, we can observe the instant of change. It seems to occur around the instant 410 since we notice the fairly large shift in the observations. The real value of this change is 412 and we will compare the result of the simulation to this value. The technique used in this simulation consists of the following steps: Resolve the forward and backward equation for the nominal model 0 = ( 0, 0 , 0 , 0 ). That is to calculate the log-likelihood function 0 and the normalized versions of the 0 0 forward variables and the backward variables for all instant = 1, , . Resolve the forward and backward equation for the alternative model = ( 0 , , , ). That is to calculate the log-likelihood function and the normalized versions of the forward variables and the backward variables for all instant = 1, , . Plot the log-likelihood function () 0, = 0 + [ ]

To estimate a potential instant of change given by observations (1 , , ), and calculate the estimator of maximum likelihood of the instant of change. The result of this task is represented in the Figure 2-3.

HOANG GIA MINH

Figure 2-3: Log-likelihood function versus instant k (change in mean). As expected, the maximum of log-likelihood function takes place at the instant 412 and its peak value is 2498 (Figure 2-3). We have detected exactly the instant of change. The change in mean is relatively large [from 0 = [2 2], 0 = 4 to = [10 10], = 20], the good detection is expected. We also note that the separation between 2 states influences the performance of the forward and backward equation. The lager, the better. Furthermore, after detecting the instant of change, we would like to analyse the estimation of hidden states using forward equation versus forward / backward equation by plotting these normalized variables in comparison with the real states. And if these variables is closed to the real state, we receive the sterling estimation. This extra task is shown in the Figure 2-4 and Figure 2-5 [note that the blue line is of the nominal model (before the instant of change) and the green line is of the alternative model (after the instant of change)]. From the results, we notice that the forward / backward equation results in better estimation than that of forward one. This can be explained as when we uses backward equation to estimate the precedent instant, we have more observations than when using forward one or more information is used for the estimation. Again, we also see that the estimation after the instant of change is better than that of before instant of change (compare green line with blue line) due to > 0 .

HOANG GIA MINH

Figure 2-4: Implementation of forward equation (change in mean).

Figure 2-5: Implementation of forward / backward equation (change in mean).

CHANGE IN THE VARIANCE

HOANG GIA MINH 6

Now, the alternative model = ( 0 , , , ) is redefined by the following characteristics: = 0 = [10 10] = [0.5 0.5]

The 2 hidden state and the observations are visualed in the Figure 2-6.

Figure 2-6: Visualization of 2 hidden states and observations (change in variance). Based on the observations, we can observe the instant of change but it is more difficult than that in the previous part. It seems to occur around the instant 330 since we notice the observations suddenly fluctuate slightly due to the decrease in variance from 0 = [5 5] to = [0.5 0.5]. The real value of this change is 337 and we will compare the result of the simulation to this value.

HOANG GIA MINH

Figure 2-7: Log-likelihood function versus instant k (change in variance). It can be found on the Figure 2-7 that the instant whose log-likelihood function gets maximum is 335. Compared with the real instant of change ( = 337), we observe there is a tiny deviation. However, this result is good. Based on this simulation, we can conclude that the change in variance is a little bit difficult to detect but it is not the matter. The result again is very good. For the Figure 2.8 and Figure 2.9, the comments is the same as in the previous session. The estimation using forward / backward equation is quite better than that of only forward equation.

HOANG GIA MINH

Figure 2-8: Implementation of forward equation (change in variance).

Figure 2-9: Implementation of forward / backward equation (change in variance).

CHANGE IN THE MEAN AND THE VARIANCE SIMULTANEOUSLY

HOANG GIA MINH 9

Now, the alternative model = ( 0 , , , ) is redefined by the following characteristics: = 0 = [1 1] = [3 3]

The 2 hidden state and the observations are visualed in the Figure 2-10.

Figure 2-10: Visualization of 2 hidden states and observations (change in mean and variance). Based on the observations, we can observe the instant of change but it is still difficult as in the previous part. It seems to occur around the instant 510 since we notice the base that the observation fluctuates is suddenly changed (increased in value). The real value of this change is 510 (luckily we guessed right!) and we will compare the result of the simulation to this value.

HOANG GIA MINH

10

Figure 2-11: Log-likelihood function versus instant k (change in mean and variance). It can be found on the Figure 2-11 that the instant whose log-likelihood function gets maximum is 523. Compared with the real instant of change ( = 510), we observe that there still exist a small deviation. But note that the deviation in this situation is larger than the deviation in the previous section (change in variance). It can be explained by in this case we have the change in both mean and variance simultaneously. For the Figure 2.12 and Figure 2.13, the comments is the same as in the first session. The estimation using forward / backward equation is quite better than that of only forward equation.

HOANG GIA MINH

11

Figure 2-12: Implementation of forward equation (change in mean and variance).

Figure 2-13: Implementation of forward / backward equation (change in mean and variance).

HOANG GIA MINH

12

CHANGE IN THE TRANSITION MATRIX Now, the alternative model = ( 0 , , , ) is redefined by the following characteristics: 0.7 0.3 = ( ) 0.3 0.7 = 0 = 0

The 2 hidden state and the observations are visualed in the Figure 2-14.

Figure 2-14: Visualization of 2 hidden states and observations (change in transition matrix). Based on the observation in the Figure 2-14. It is really hard to detect the instant of change. We try to detect even by feeling. It maybe occurs around the instant 300. We can sense that the form of the observations after this guessed instant is slightly different than that before this guessed one. The real value of this change is 305 and we will compare the result of the simulation to this value.

HOANG GIA MINH

13

Figure 2-15: Log-likelihood function versus instant k (change in transition matrix). Figure 2.15 leads to the result that the detected instant of change is 308. The result is quite good if we compare with the real value = 305. From this simulation, we can see that the change in transition matrix can be detected well by maximizing the log-likelihood function. For the Figure 2.16 and Figure 2.17, the comments is the same as in the first session. The estimation using forward / backward equation is quite better than that of only forward equation.

HOANG GIA MINH

14

Figure 2-16: Implementation of forward equation (change in transition matrix).

Figure 2-17: Implementation of forward / backward equation (change in transition matrix).

HOANG GIA MINH

15

3 Conclusion
The log-likelihood function can be used to detect the instant of change in the Hidden Markov Model. This change can be mean, variance, both mean and variance as well as transition matrix. The accuracy of this function is quite good. It is perfect in detecting the instant of change in mean. For the change in single characteristic, this function can detect with very small deviation from the real instant. However, in case the change in multiple characteristics takes place, the accuracy is reduced but can be acceptable.

4 References
[1] LE GLAND Franois, TP2: Modles de Markov cachs, Dcembre 2013

HOANG GIA MINH

16