MATH 075

Homework #17
Johnson Liu
24 February 2019
Contents
1 Probability Matrix 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Probability Matrix 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 Probability Matrix 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1
MATH 075 Homework #17 Johnson Liu

1 Probability Matrix 1
We have the probability matrix
 
0 0 1 0
1/2 0 0 0
P =
 0
 . (1)
1 0 1
1/2 0 0 0

Figures (1), (2), and (3) show the time series for different initial state vectors. State 2 always
coincides with state 4. The behavior of the system depends on the initial state vector. However,
the time series are periodic in each of the states.

Figure 1: Time series plot with the initial state vector (1/2, 0, 1/2, 0). The line for state 2 is
under the line for state 4.

2
MATH 075 Homework #17 Johnson Liu

Figure 2: Time series plot with the initial state vector (1, 0, 0, 0). The line for state 2 is under
the line for state 4.

Figure 3: Time series plot with the initial state vector (1/4, 1/4, 1/4, 1/4). The line for state 2
is under the line for state 4.

3
MATH 075 Homework #17 Johnson Liu

Figure 4: The eigenvectors of the probability matrix in Equation (1).

Figure 5: The eigenvalues of the probability matrix in Equation (1)

Figures (4) and (5) show the eigenvectors and eigenvalues, respectively, of the probability
matrix in Equation (1). Since two of the eigenvalues are complex, the system will exhibit periodic
behavior. Also, since three of the eigenvalues have a modulus of 1, the system cycles through
three states.

2 Probability Matrix 2
The next probability matrix is
 
1/3 0 1/2 0 1/2 1/2
 0 1/2 0 1/2 1/2 0 
 
1/3 0 1/2 0 0 0 
P =   . (2)
 0 1/2 0 1/2 0 0 

 0 0 0 0 0 0 
1/3 0 0 0 0 1/2

Figures (6) through (9) show the time series for the application of matrix P to different initial
state vectors. Each of the states approach limiting values that depend on the initial conditions.
State 5 always decays to zero probability and stays there as time goes on. States 2 and 4 always
coincide as time goes to infinity while states 3 and 6 coincide as well.

4
MATH 075 Homework #17 Johnson Liu

Figure 6: Time series plot with initial state vector (1/4, 1/5, 1/20, 1/4, 1/5, 1/20). The line for
state 3 is hidden behind the line for state 6.

Figure 7: Time series plot with initial state vector (1, 0, 0, 0, 0). The lines for state 2 and state
4 are hidden behind the line for state 5. The line for state 3 is hidden behind the line for state
6.

5
MATH 075 Homework #17 Johnson Liu

Figure 8: Time series plot with initial state vector (0, 0, 0, 1/2, 1/2).

Figure 9: Time series plot with initial state vector (0, 0, 0, 0, 1). The lines for state 2 and state
4 are hidden behind the line for state 5.

6
MATH 075 Homework #17 Johnson Liu

Figure 10: The eigenvectors for the probability matrix in Equation (2).

Figure 11: The eigenvalues for the probability matrix in Equation (2).

Figures (10) and (11) show the eigenvectors and eigenvalues, respectively, of the probability
matrix in Equation (2). There are no complex eigenvalues so the system does not exhibit periodic
behavior. There are two eigenvalues that are equal to 1. Stationary states can be formed through
linear combinations of these the eigenvectors associated with these eigenvalues. State 5 always
tends towards 0 because the probability of being in state 5 is 0 in every eigenvector. State 2
tends towards the same limiting value as state 4 because they have the same values in all of the
eigenvectors except for the eigenvector associated with the 0 eigenvalue. State 3 always coincide
with state 6 for the same reason. As time goes towards infinity, a linear combination of the
second and fourth eigenvectors form a stationary state. The particular stationary state that is
achieved depends on the initial state distribution vector.

3 Probability Matrix 3
The final probability matrix is
 
1/3 0 1/2 0 1/2 1/2
 0 1/2 1/2 1/2 1/2 0 
 
1/3 0 0 0 0 0 
P =
  . (3)
 0 1/2 0 1/2 0 0 

 0 0 0 0 0 0 
1/3 0 0 0 0 1/2

Figures (12) through (15) show the state distribution time series for the probability matrix
in Equation (3). States 2 and 4 reach the same limiting value as time goes to infinity. This value
is .5 regardless of the initial state vector. The other states all approach 0 as time goes on.

7
MATH 075 Homework #17 Johnson Liu

Figure 12: Time series plot with initial state vector (1/4, 1/5, 1/20, 1/4, 1/5, 1/20).

Figure 13: Time series plot with initial state vector (1, 0, 0, 0, 0).

8
MATH 075 Homework #17 Johnson Liu

Figure 14: Time series plot with initial state vector (0, 0, 0, 1/2, 1/2).

Figure 15: Time series plot with initial state vector (0, 0, 0, 0, 1).

9
MATH 075 Homework #17 Johnson Liu

Figure 16: The eigenvectors for the probability matrix in Equation (3).

Figure 17: The eigenvalues for the probability matrix in Equation (3).

Figures (16) and (17) show the eigenvectors and eigenvalues, respectively, for the probability
matrix in Equation (3). In this case there is only one eigenvalue equal to 1 while the others are
less than 1. The eigenvectors corresponding to the eigenvalues less than 1 contribute less and less
to the state distribution as time goes on. A constant multiple of the eigenvector corresponding
to the eigenvalue that is equal to 1 is the stationary state for this system. States 2 and 4 tend
towards the same limiting value while the other states drop to 0. This occurs because states 2
and 4 have the same non-zero probability in the eigenvector with an eigenvalue of 1 while the
other states all have zero probabilities in this eigenvector.

10