EIGENVALUES AND EIGENVECTORS Consider the following set of coupled, linear, first order, ordinary differential equations-initial value

problems (ODE-IVPs), with constant coefficients:

dx1 dt
dx 2 dt

= a11 x1 + a12 x2 + . . . + a1n xn + b1 ,

x1(0) = x10

= a21 x1 + a22 x2 + . . . + a2 n xn + b2 , x2(0) = x20

dx n dt

= an1 x1 + an 2 x2 + . . . + ann xn + bn , xn(0) = xn0

This can be written in compact notation, as

dx = Ax + b(t ) dt

x(0) = x0

A is an n n matrix of constants. The use of vectors and matrices not only saves a great deal of space and

facilitates calculations, but also emphasizes the similarity between systems of equations and a single equation.
Above equation is encountered quite commonly in engineering practice. o Systems described by lumped-parameter models in chemical engineering,

e.g., continuous-flow stirred tank reactors (CSTRs), distillation, extraction and absorption columns, etc., are quite well described by such equations under dynamic (unsteady) conditions.
o The behavior of batch (and plug flow) reactors under steady conditions.

o These equations are useful in other fields, too, as for example, for

describing the sustained harmonic oscillations of strings, membranes and structural systems (bridges, etc.), in the presence of external forces.
o At a more abstract, mathematical level, the study of the stability of

numerical solutions of ordinary and partial differential equations (PDEs) also requires the solution of (homogeneous) linear ODEs.

dx = Ax + b(t ) dt

x(0) = x0

In solving above, it is necessary to solve, first, the following set of

homogeneous linear ODEs:

dx = Ax dt
In order to solve above, we assume the solution to be of the following form:
x = v et

This is similar to the procedure used to solve single, linear, homogeneous ODEs with constant coefficients. The constant, , and the constant vector, v, are to be determined. On substituting we obtain
v et = A v et

Since et 0, this leads to or,

Av=v

(A - I) v = 0

Thus, in order to solve the system of ODE-IVPs, we must solve the above system of linear algebraic equations.
The above system of linear algebraic equations is encountered quite often

in engineering and science, and has been studied extensively.

This equation is satisfied by several values of i, which are referred to as the eigenvalues of the matrix, A. For each i, there is a corresponding vector, vi. The latter is referred to as

the eigenvector of the coefficient matrix, A, associated with i.

The above system represents a set of linear, homogeneous algebraic

equations (with v = 0 if the rank of [A - I] is n).

The necessary condition for this equation to have non-trivial solutions that

are physically meaningful, is det (A i I) = 0, i = 1, 2, . . . , n.

n-is exist for which det (A - i I) V = 0

This ensures that the rank of (A - iI) is less than n.

Corresponding to each eigenvalue, i, there is an eigenvector, vi. Thus, there are, in general, n solutions, and the general solution of the

homogeneous is a linear combination of these

n t x = ci vi e i i =1

ci are arbitrary constants that are obtained using the initial conditions.

If i = ai + i bi, I = 1, 2, 3, . . . , n. Then

x = Ci v i e ai t [sin bi t + cos bi t ]
i =1

The significance of the eigenvalues, i, can easily be inferred from the above solution.
If we consider xi to represent the deviations of the state variables from

their steady state values, this equation suggests that these deviations will die out asymptotically with time, t, if all the eigenvalues are real and negative.
Similarly, if even one of the is is positive and real, the corresponding

deviation will increase with time, and the system will be unstable.
If any i is complex, the corresponding deviation will be oscillatory in

nature (due to the imaginary part), and its real part will decide whether it will die out or blow up in time.
Thus, i are related to the (asymptotic) stability of a system under (small)

perturbations.

A particular solution of

dx = Ax + b(t ) dt

x(0) = x0

can readily be obtained if b is a constant vector, i.e., it is not a function of t. In this case the particular solution can be obtained by substituting x = k (where k is a constant).
0=Ak+b

or
k = - A-1 b

Therefore, the complete solution is given by

n t x = ci vi e i - A-1 b i =1

We now apply the initial conditions, x(t = 0) = x0. We obtain

x0 = ci vi - A-1 b i =1
n

This comprises a set of n equations in the n unknowns, ci. It can be shown that the ci can be uniquely determined (since vi are eigenvectors, they form a linearly independent set, and so the rank of the set of equations for ci is n.

The Characteristic Polynomial

Consider the n n matrix, A. The eigenvalues of this matrix may be obtained by solving: det (A i I) = 0,
Avi = i vi

i = 1, 2, . . . , n

while the eigenvector, vi, associated with i obtained using (for v 0) The eigenvalue, i, may be real or complex. In expanded form we have
a11 a21 a12 a22 a1n a2n

det (A-I) = f ( )

=0

an1

an 2

ann

We can expand the determinant to give

f ( ) = n + n1 n1 + n2 n2 + ... + 1 + 0 = 0
Clearly, above has n roots (i.e., n-eigenvalues).
Example 1: Obtain the eigenvalues of the following matrix
6 4 7 2 3 A= 0 1 1 1 7 6 4 det (A - I) = det 0 2 3 1 1 1

f() = - 3 + 9 2 - 30 + 32 = 0
= 2, [7 i 15]/2
6

where i = - 1.

The nth order polynomial, f(), can also be written in terms of the principal
minors, Sj(A), as

f() = Pn() = det (A - I) = S j ( A) ()n j j=0 In this equation, So = 1, and Sj(A) is the sum of all possible principal minors of
A, of size j j obtained by striking out an equal number, n j, of rows as well as

columns from A, and taking the remaining j j elements. While doing this, it must be remembered that identical-numbered rows and columns need to be struck off from A in order to obtain the several minors in Sj. The procedure is best illustrated through an example.

Principal minors

a11 a12 a a 22 21 a k 1 a k 2 a i1 a i 2 a n1 a n 2

a1k

a1i

a 2k a kk aik

a 2i a ki aii

a nk

a ni

a1n a 2n a kn ain a nn

Intersection of striking out of rows and columns should be on diagonal in forming the principal minors.

Example 2: Find the characteristic polynomial for the 3 3 matrix, A, in

Example 1: f() = det (A - I) = So (-)3 + S1 (-)2 + S2 (-) + S3 We have S3 = det A = 32

7

This is obtained by deleting 0 rows and 0 columns in A. Only one principal minor results. S2 can be written as S2 =

a11

a12

a 21 a 22

a11

a13

a 31 a 33

a 22 a 32

a 23 a 33

= 12 + 13 + 5 = 30

S2 is the sum of three principal minors, obtained by deleting the third, second and first rows as well as columns, respectively, from A. S1 can be written as S1= a11 + a22 + a33 = 6 + 2 + 1 = 9 This is the sum of three principal minors again, obtained by deleting two rows and two (identically numbered) columns from A. Also, Therefore, Pn() = - 3 + 9 2 - 30 + 32 = 0 The same result as obtained in Example 1.
-----------

So = 1

It can easily be deduced that S1 = aii = sum of diagonal terms trace of A = tr A

i =1 n

Also, Sn = det A Therefore, for a 2 2 matrix P2() = 2 (tr A) + (det A) = 0

We can also write Pn() = 0 in terms of a multiple product involving the n roots,
i, as:
n n n Pn ( ) = ( j ) =0 = ( ) n + j ( ) n 1 + j =1 j =1 j >i

( )
i =1 i j

n2

+. . . . . + j
j =1

A comparison of the above leads to S1 = tr A = i = sum of all the eigenvalues

i =1 n

n Sn = det A = i = product of all the eigenvalues i =1 By a similar comparison, we can obtain

n n S2 = i j j >i i =1

For n = 3, thus, we can write S2 = 12 + 13 + 23

Properties of Eigenvalues and Eigenvectors

An upper bound of the eigenvalues is given by: Max |i| ||A|| where the norm of matrices is discussed in Section 5.6.1. Proof: We can get a bound on ias follows:
Avi = ivi

(vi 0)

Then, vi = ivi = Avi A vi i Therefore, A for all i for any norm. i When the matrix, A, has distinct eigenvalues, j, (i.e., no repeated eigenvalues), then the set of eigenvectors, vj, forms a linearly
independent set.

Proof: Let vi and vj be two eigenvectors of A that correspond to eigenvalues, i and j, respectively. Then vi and vj are not constant multiples of each other, because if they were, then
vi = k vj Avi = k Avj

(a) (b) (c)

Here, k is a non-zero constant. If we pre-multiply Eq. (a) by the matrix, A, then This is equivalent to i v i = k j vj If we now multiply both sides of Eq. (a) by i, we have i vi = k i vj
10

(d)

Subtracting Eq. (c) from Eq. (d), we have k (i - j) vj = 0 Since k 0, and, i j, this means that vj = 0, which contrary to the assumption that vj is an eigenvector. Therefore, the eigenvectors corresponding to
different (distinct)

eigenvalues of A are linearly independent.

If the n n matrix, A, is symmetric, then all n of its eigenvalues, i (i = 1,

2, . . ., n), are real, i.e., the roots of the characteristic polynomial, f() = 0, have n real roots. However, these roots need not necessarily be distinct. If the matrix is not symmetric, then some of the eigenvalues may be
complex.

For a symmetric matrix, A, there are n distinct eigenvectors, vi, one

corresponding to each eigenvalue, i. This is so even if some of the

eigenvalues are not distinct. However, if A is not symmetric, then it may not always be possible to find n linearly independent eigenvectors (if some of the eigenvalues are repetitive). The n distinct eigenvectors of a symmetric matrix, A, are orthogonal to each other and can easily be made orthonormal. Thus, <vi, vj > = vi vj = ij These eigenvectors form a complete basis set.
11

Hence, any n-dimensional vector, x, can be represented in terms of the vi as

x = ci v i i =1
n

where ci = vi x = <x, vi >

Let U = [v1

v2 v3

. . .

vn] be the n n matrix whose columns contain the n

orthonormal eigenvectors of the symmetric matrix, A. It is easy to show

that
UU = I

This means that

U (U*)T = U-1

Such matrices are referred to as unitary matrices. If U is real, then UTU = I, and then UT = U-1. Such matrices are referred to as
orthogonal matrices, rather than unitary matrices. The rows (or columns) of a

unitary matrix form an orthogonal basis.

Example: Consider the following two U matrices:
1 cos x sin x U1 = ; U2 = i 2 sin x cos x 2
i 1 2 2

U1 is orthogonal since it is real, and

1 U1 =

cos x sin x T = U1 cos 2 x + sin 2 x sin x cos x

1

12

Similarly, it may be demonstrated that U2 is unitary.

1 U2

1 i 2 T 2 * U2 = = U = 2 1 i 2 2

Principal Axes Theorem Let A be a self-adjoint (or Hermitian) matrix of order n, [defined by: A
(A*)T = A]. Then, A has o n real eigenvalues, 1, 2, . . ., n, not necessarily distinct o n corresponding distinct eigenvectors, v1, v2 , . . ., vn, that form a complete orthonormal basis set. The components of the eigenvectors

can be complex.
o If the elements of A are real, then the components of the eigenvectors will

also be real.
o Finally, there is an unitary (or orthogonal) matrix U, for which UAU = D diagonal (i)

where
1 0 0 2 D= 0 0 0 0 0 n

By pre-multiplying UAU (= D) by U, and post-multiplying it by U (and using the fact that U = U-1), we can show that
A = UDU

We can only diagonalize a matrix, A, if the eigenvectors form a basis set, i.e., we have n linearly independent eigenvectors. This is always true for symmetric matrices.
13

However, if A is not symmetric, then it may not always be possible to find n linearly independent eigenvectors (if some of the eigenvalues are repetitive), and in that case we will not be able to diagonalize the matrix.
Example: Consider the following symmetric matrix
2 1 0 A= 1 3 1 0 1 2

We can find the eigenvalues of A as follows: f() = -3 +72 - 14 + 8 = 0 [1, 2, 3] = [1, 2, 4]; all real, as expected. The corresponding eigenvectors can be obtained by solving for non-zero vi using
Avi = ivi or (A - iI)vi = 0.

As the (A - iI) matrix is singular, we have to assign one or more components of

vi arbitrarily, and then find the remaining components.

When = 1, we have for (A-I) v1 = 0 as

1 1 0 v11 0 1 2 1 v = 0 v = 1 12 0 1 1 v13 0 Similarly, we obtain

1 1 1

1 1 v1 = 1 ; v 2 = 0 ; v 3 = 1 1

1 2 1

As expected, these eigenvectors are orthogonal to each other, and <v1, v2> = <v1, v3> = <v2, v3> = 0 These vectors are also linearly independent.
14

We can, of course, make these vectors orthonormal by normalizing them to obtain

1 1 1 2 1 1 1 v1 = 1 ; v = 0 ;v = 3 2 2 3 6 1 1 1

If we now form the U matrix as follows (each column of U contains one of the orthonormal eigenvectors):
1 1 3 2 U = 1 3 0 1 1 2 3 1 2 1 6 6

Eigenvalues

then
1 0 0 0 2 0 U AU = D = 0 0 4
T

Also not that U-1 = UT. If all the is are distinct (even for non-symmetric A), then A has n linearly independent eigenvectors. This can be easily demonstrated and is shown below: We can write, in general Rank of (A-I) < n
n

Pn ( ) = det (A I) = (1 )(2 )....(n ) = ( j ) = 0

j =1

When the ith eigenvalue, i, is simple (i.e., i has a multiplicity of 1), we can rewrite Pn() as

Pn ( ) = (i ) ( j ) = 0
j =1 i j

Differentiating it with respect to , and then substituting = i, we get

15

n dPn ( ) = ( j ) 0 d = j =1 i j i

Rank of (A-iI) = n -1

Therefore, the rank of (A - iI) is n - 1 (when i is a distinct eigenvalue). Then, Dimension of null space = n [rank of (A - iI)] = n (n-1) = 1 Therefore, there is one linearly independent eigenvector for each i. as long as it is a distinct eigenvalue. Note that the symmetry of A was not invoked. Hence, if all eigenvalues are distinct, then A has n linearly independent eigenvectors, which form a complete basis set. Therefore, any ndimensional arbitrary vector can be represented as linear combinations of vis.
Example: Obtain the eigenvalues and the corresponding eigenvectors of the

following symmetric matrix

1 1 A= 1 1

We have det (A - I) = 2 (tr A) + det(A) = 2 2 = 0 = 0, 2 (i.e., the symmetric matrix, A, has two distinct eigenvalues). To obtain the eigenvectors corresponding to = 0, we solve for non-trivial v using (A - I) v = 0:
1 1 v1 0 ( A 1I ) v = = 1 1 v2 0

or v1 + v2 = 0 v1 + v2 = 0

16

These two equations are linearly dependent. We have to fix one of the components of v arbitrarily to find the other one. If we fix v2 = , then
1 v = 1

When = 2:
v1 1 1 v1 0 1 1 v = 0 v = v = 2 2 1 1

We obtained two linearly independent eigenvectors for the two distinct eigenvalues.
Example: Now consider the following unsymmetric matrix: A = 1 1
3 4

We have det (A - I) = 2 (tr A) + det(A) = 2 2 + 1 = 0 We now have two eigenvalues that are identical. We solve for non-trivial v using (A - I) v = 0 for = 1:
4 v1 0 2 ( A I ) v = = 1 2 v2 0

= 1, 1

or 2 v1 + 4 v 2 = 0 - v1 2 v 2 = 0 The two equations are linearly dependent (rank = 1, dimension of null space = 1). We have to fix one component of v arbitrarily and evaluate the other one. If we fix v2 = , we obtain
2 v = 1
17

Note that we can only find only one linearly independent eigenvector for the eigenvalue = 1 with a multiplicity of two since dimension of null space is 1 (and therefore, only one independent vector exists). We do not have two eigenvectors although we have two (identical) eigenvalues for the non-symmetric A.

Example: Evaluate the eigenvalues and the eigenvectors of the symmetrical

identity matrix
1 0 0 A= 0 1 0 0 0 1
Rank of (A-I) = 0 Dimension of null space = 3

We have f() = (1 - )3 = 0 = 1 with a multiplicity, p of 3

Since the rank and dimension of null space for the (A-I) matrix are 0 and 3 respectively, we can find three linearly independent eigenvectors for the above symmetric matrix: v1 = [1 0 0]T, v2 = [0 1 0]T, v3 = [0 0 1]T From the principa1axes theorem, we know that for a symmetric matrix,

All the eigenvalues are real, but all of them are not necessarily
distinct.

However, it is always possible to find distinct eigenvectors regardless

of whether eigenvalues are distinct or repetitive.

The corresponding eigenvectors also form a complete orthonormal

basis.
18

This means that we will always be able to find n linearly independent eigenvectors even if all the n eigenvalues are not distinct when matrix is symmetric.

Consider now the following non-symmetric matrix

1 1 0 A= 0 1 1 0 0 1
f() = (1 - )3 = 0

Rank of (A-I) = 2 Dimension of null space = 1

= 1; p (multiplicity) = 3

Since the rank and dimension of null space for the (A-I) matrix are 2 and 1 respectively, the matrix has only one linearly independent eigenvector for the eigenvalue, = 1: v = [1 0 0]T
If a matrix, A, is singular, then one or more (depending on the rank of the

matrix) of the eigenvalues is zero.

We can also find the eigenvectors of A from the non-trivial columns of adj(A - I). Note that when a matrix is singular, A-1 does not exist but adjugate of A exist. A-1 =
1 adj A det A

For non-trivial x from (A-I) x = 0 can be obtained from non-trivial columns of adj(A-I).

19

Example: Consider the unsymmetrical matrix

4 3 A= 1 1

We found = 1 (multiplicity = 2) for this matrix. We have, then,

Eigenvectors 4 2 4 2 adj ( A I ) = adj = 2 1 2 1

Example: Consider the symmetrical matrix:

1 1 A= 1 1
We had obtained = 0 and 2.
1 1 1 1 adj (A I) = adj = 1 1 1 1

Eigenvectors

For = 0:

For = 2:

1 1 1 1 adj ( A I ) = adj = 1 1 1 1

Eigenvectors

However, if repetitive eigenvalues are present and dimension of (A - I) matrix is greater than 1, then sometimes it may not be possible to obtain eigenvectors from adj(A - I) as all components of adj(A - I) matrix may be zero.

20

Similar Matrices

Two n n matrices, A and B, are said to be similar if there exists a non-singular matrix, P, such that B = P-1AP, or, equivalently, A = PBP-1. Matrices A and B are similar in the sense that

their characteristic polynomials are identical (and so are their eigenvalues), their eigenvectors are related through the matrix, P. They are said to be represented by the same linear transformation but with respect to different bases.

We first demonstrate that the two characteristic polynomials, fA() and fB(), are the same. fB() = det (B - I) = det [P-1 A P - P-1 I P] = det [P-1 (A - I) P] = det(P-1) det (A - I) det(P) = det (A - I) = fA() since det(P-1) det(P) = 1 Therefore, A and B have identical eigenvalues. Since fA() = fB(), the coefficients of the characteristic polynomials are the same. This means that trace of A = trace of B and det A = det B When matrices A and B are similar, even though their eigenvalues are identical, their eigenvectors are not. However, there is a one to one correspondence, as shown below. We have
Ax = x P-1Ax = P-1 x P-1A (P P-1) x = P-1 x
21

or, or,

or, or,

B (P-1x) = (P-1 x) Bz=z

Therefore, the eigenvectors of B are given by z = P-1x, where x is the eigenvector of A.

In summary

For a (n n) matrix A, there exist n eigenvalues. The eigenvalues can be obtained from the characteristic polynomial f() = det (A - I) = 0. The eigenvectors can be obtained from either

The non-trivial solution v from (A - I) v = 0, or Non-trivial columns of adj (A - I) = 0.

The eigenvalues and eigenvectors can also be obtained numerically, for example by Inverse Power method. The n eigenvalues may be distinct or repetitive (multiple), and real or
complex.

If all the eigenvalues are distinct (real or complex), then there exist n
eigenvectors, which are linearly independent to each other.

The n eigenvectors, vi, (i = 1, 2, n) are obtained from either (A - iI) vi = 0 or from adj (A - iI) = 0. If we form a (n n) U matrix such that each column of U matrix contains the n eigenvectors, i.e., U = [v1
v2 v3 . . . vn], then U-1 A U = D, where D is a diagonal matrix whose diagonal elements are the n eigenvalues.

22

If all the eigenvalues are not distinct (real or complex), then we may or
may not be able to find n linearly independent eigenvectors. The number of linearly independent eigenvectors that can be found for the

eigenvalue i with multiplicity m will depend on the dimension of null

space for the (A - iI) matrix.

If the dimension of null space for the (A - iI) matrix is 1, for the eigenvalue
i with multiplicity m, then only one linearly independent eigenvector can be

found for corresponding to the eigenvalue i with multiplicity m. If, however, the matrix A is self-adjoint (or symmetric), then

All the eigenvalues of matrix A are always real.

All the eigenvalues may not be distinct, but we will always be able
to find n linearly independent eigenvectors irrespective of whether

the eigenvalues are distinct or repetitive.

The eigenvectors will not only be linearly independent, but they will also be orthogonal.

If we normalize the eigenvectors, and then form the U matrix, then we will find U-1 = UT. That is the U matrix will be a unitary (or
orthogonal) matrix.

Therefore, U-1 A U = D will be same as UT A U = D.

23

However, if the matrix is not symmetric, then it may not always be

possible to find n- linearly independent eigenvectors (particularly, if

some of the eigenvalues are repetitive), and in that case we will not be able to
diagonalize the matrix.
Algebraic multiplicity: Multiplicity in eigenvalues as obtained from the

root of Pn() or f() = 0. Example: In the previous two matrices, algebraic multiplicity was 3 (for = 1, p was equal to 3 for both the matrices).
Geometric

multiplicity: Maximum number of linearly independent

eigenvectors possible associated with all the eigenvalues. Example: Although, algebraic multiplicity was 3 for both cases, we had 3 linearly independent eigenvectors for the first case (symmetric identity matrix) whereas, we had only one linearly independent eigenvectors for the second case. Therefore, geometric multiplicity is 3 for the first case (identity matrix), whereas geometric multiplicity is only 1 for the second case.
When a matrix A (n x n) has a eigenvalue of algebraic multiplicity p but

geometric multiplicity less than p, then, we can form a Jordon-Canonical

Block, J, consisting of all the eigenvalues of A, and can find a non-singular

matrix P such that P-1AP = J.

24

 The above is similar to U-1AU = D except that

Matrix J is not a diagonal matrix but has a different form. Each columns of U matrix consist of n- linearly independent eigenvectors, whereas columns of P matrix consist of as many as linearly independent eigenvectors possible to be obtained from A, plus some generalized eigenvectors.

Note that when matrix A is symmetric, only then U-1 = UT (of course, only

if the eigenvectors were normalized), otherwise U-1 UT. Therefore, we must use the relation U-1AU = D when matrix A is not symmetric but still we can find n-linearly independent eigenvectors.

How to form the Jordon-Canonical Block?

When we have n-eigenvalues all equal, say , (i.e., algebraic multiplicity equal to n), but it is possible to obtain only one linearly independent eigenvector (i.e., geometric multiplicity equal to 1), then
1 0 0 0 0 1 0 0 Jn = 0 0 0 1 0 0 0 0

i.e., at the diagonal position and 1 above . The rest of the elements are zero.

25

 If A is a (n x n) matrix, then, there always exist a non-singular matrix P for

which

0 J n1 (1 ) 0 J n 2 ( 2 ) 1 P AP = 0 0

0 J nr (r ) 0 0 0 0

The eigenvalues 1, 2 , , r need not be distinct. The number of distinct blocks Jnr(r) must be equal to the number of independent eigenvectors. Same eigenvalues may occur in different blocks.
Example: Let A be a (6 x 6) matrix having the following eigenvalues and the

corresponding number of linearly independent eigenvectors: 1 = 3, p = 2 with only 1 linearly independent eigenvector. 1 = 4, p = 3 with only 1 linearly independent eigenvector. 1 = -2, p = 1 with 1 linearly independent eigenvector.

2 0 0 J = 0 0 0

0 0 0 0 0 3 1 0 0 0 0 3 0 0 0 0 0 4 1 0 0 0 0 4 1 0 0 0 0 4

Number of distinct blocks must be equal to the number of independent eigenvectors

We can put 1, 2, etc. at any location on J matrix. But, when we form the P matrix, the columns of P matrix must correspond to the respective location of in J matrix.

26

Example: Let A be a (5 x 5) matrix having the following eigenvalues and the

corresponding number of linearly independent eigenvectors: 1 = 2, p = 4 with only 2 linearly independent eigenvectors. 1 = 3, p = 1 with 1 linearly independent eigenvector.
2 0 J = 0 0 0 1 0 0 0 2 0 0 0 0 2 1 0 0 0 2 0 0 0 0 3

Numbers of distinct blocks are equal to the number of independent eigenvectors. Note that Same eigenvalue is present in two different blocks.

Alternatively,
2 0 J = 0 0 0 0 0 0 0 2 1 0 0 0 2 1 0 0 0 2 0 0 0 0 3

How do we form the non-singular matrix P?

Example: Consider the following matrix 1 1 A= 1 3 P2() = 2 (tr A) + det(A) = 2 4 + 4 = 0 1 2 1 P 1 AP = J = = 0 0 2 Or,

1 1 p11 p12 p11 p12 2 1 = 1 3 p 21 p22 p21 p22 0 2 A P P J
27

= 2, 2

p11 + p21 = 2 p11 - p11 + 3 p21 = 2 p21 p12 + p22 = p11 +2 p12 - p12 +3 p22 = p21 +2 p22

p11 = p21 p11 = p21 p22 = p11 + p12 p12 = p22 - p21

We have only 2 linearly independent equations for the 4 unknowns. Let Then, p11 = 1, and p12 = 0 p21 = 1 and p22 = 1.
Dimension of null space = 2

Therefore, 0 1 0 1 1 = P= and P 1 1 1 1 and 1 0 1 1 1 0 2 1 P 1 AP = 1 3 1 1 = 0 2 1 1

Example: Consider the following matrix

4 3 5 A = 1 0 3 1 2 1 P3() = - 3 + 6 2 + 0 - 32 = 0 For = -2: = -2, 4, 4

We can take any arbitrary multiple

4 3 0 18 18 7 adj ( A I ) = adj 1 2 3 18 = 0 18 18 1 2 3 0 18
28

Therefore, eigenvector, v1,

1 v1( for = 2) = 1 1

For = 4:

4 3 6 6 0 1 adj ( A I ) = adj 1 4 3 = 6 6 0 1 2 3 6 6 0 Therefore, eigenvector, v2, 1 v2 ( for = 4) = 1 1 We have only 2 linearly independent eigenvectors. Therefore, we need to form Jordon-Canonical block and we need to find one generalized eigenvector, p.

A [ v1 v2 p] = [v1 v2 p] J3()
4 3 1 1 p1 1 1 p1 2 0 0 5 or , 1 0 3 1 1 p2 = 1 1 p2 0 4 1 1 2 1 1 1 p3 1 1 p3 0 0 4 2 0 0 p] 0 4 1 0 0 4
=-2

or ,

A [v1 v2

p ] = [v1 v2

A v1 = -2 v1

or, (A + 2 I) v1 = 0

A v2 = 4 v 2

or, (A - 4 I) v2 = 0
29

=4


Or,

A p = v2 + 4 p
(A 4I) p = v2

Generalized eigenvector

1 4 3 p1 1 1 4 3 p2 = 1 1 2 3 p3 1
(A - 4I)

v2

p1 = 1, p2 = 0, p3 = 0

Therefore, the matrix P is 1 1 1 0 1 1 0 1 1 P = 1 1 0 and P 1 = 1 2 1 1 0 2 2 0 and,

P-1 A P = J3()

Whenever we have repeated eigenvalues with geometric multiplicity less than

algebraic multiplicity, then column vectors of the P matrix contains generalized eigenvectors in addition to the normal eigenvectors. Generalized eigenvectors can be obtained from (A jI) pi = pi -1 ; for i = 2, 3, with p1 = Vj where, Vj is the linearly independent eigenvector obtained from j.

30

 Example:

1 = -2, m = 1 with 1 linearly independent eigenvector. 2 = 1, m = 1 with 1 linearly independent eigenvector. 3 = 3, m = 3 with only 1 linearly independent eigenvector. where, m denotes algebraic multiplicity. Then, we can form matrix P as follow
P = [v1 v2 v3 p1 p2]

p1 and p2 can be obtained as follows: (A 3I) p1 = v3

and

(A 3I) p2 = p1

and the Jordon-Canonical block is given by

2 0 J 5 ( ) = 0 0 0
Number of distinct blocks must be equal to the number of independent eigenvectors.

0 0 0 0 1 0 0 0 0 3 1 0 0 0 3 1 0 0 0 3

and we have P-1 A P = J5().

Example:

1 1 A= 1 3 f () = 2 - 4 + 4 = 0 1 1 [A - I] = [A - 2 I] = 1 1
31

= 2, 2

1 1 Adj [A - 2 I] = 1 1 1 Therefore, v1 = 1

v1 was obtained from (A - I) v1 = 0

Therefore, we have just 1 linearly independent eigenvector. The generalized eigenvector p can be obtained from (A - I) p = v1 or,
- p1 + p2 = 1 - p1 + p2 = 1 If we let p1 = 0, then p2 = 1. or, 0 p = 1 1 0 P= 1 1
v1 obtained from (A - I) v1 = 0 p obtained from (A - I) p = v1

1 1 p1 1 = 1 1 1 p2

Therefore, matrix P is given by

1 0 1 1 1 0 2 1 And, P-1 A P = = =J 1 1 0 2 1 3 1 1

32

 Example: Let A be a (5 x 5) matrix having the following eigenvalues and

the corresponding number of linearly independent eigenvectors: 1 = -3, p = 4 with only 2 linearly independent eigenvectors. 1 = 2, p = 1 with 1 linearly independent eigenvector. There many ways we can from the Jordon-canonical block, J, for this example. However, we must form the corresponding P matrix as per we write our J matrix
Arrangement #1: If we form the J matrix as follows

2 0 0 0 0 0 0 0 3 0 J = 0 0 3 1 0 0 3 1 0 0 0 0 0 0 3 Form P matrix as follows

3 distinct blocks for 3 linearly independent eigenvectors

P = [v1
Where,

v2

v3

p1

p2 ]
From problem statement

v1 is obtained from (A - 2 I) v1 = 0. v2 is obtained from (A + 3 I) v2 = 0. v3 is obtained from (A + 3 I) v3 = 0. p1 is obtained from (A + 3 I) p1 = v3. p2 is obtained from (A + 3 I) p2 = p1.

2 linearly independent vectors can be obtained from (A+ 3 I) v2 = 0 and (A+ 3 I) v3 = 0 Null space dimension must be equal to 2.

33

Arrangement #2: If we form the J matrix as follows

3 1 0 0 0 0 3 0 J = 0 0 3 1 0 0 3 0 0 0 0 0 Form P matrix as follows

P = [v1 p1 v2

0 0 0 0 2

3 distinct blocks for 3 linearly independent eigenvectors

p2

v3 ]
From problem statement

Where,
v1 is obtained from (A + 3 I) v1 = 0. p1 is obtained from (A + 3 I) p1 = v1. v2 is obtained from (A + 3 I) v2 = 0. p1 is obtained from (A + 3 I) p2 = v2. v3 is obtained from (A - 2 I) v3 = 0.

2 linearly independent vectors can be obtained from (A+ 3 I) v1 = 0 and (A+ 3 I) v2 = 0 Null space dimension must be equal to 2.

Arrangement #3: If we form the J matrix as follows

3 1 0 3 J = 0 0 0 0 0 0

0 0 2 0 0

0 0 0 0 0 3 1 0 3 0

3 distinct blocks for 3 linearly independent eigenvectors

34

Form P matrix as follows

P = [v1 p1 v2 v3 p2 ]
From problem statement

Where,
v1 is obtained from (A + 3 I) v1 = 0. p1 is obtained from (A + 3 I) p1 = v1. v2 is obtained from (A - 2 I) v2 = 0. v3 is obtained from (A + 3 I) v3 = 0. p2 is obtained from (A + 3 I) p2 = v3.

2 linearly independent vectors can be obtained from (A+ 3 I) v1 = 0 and (A+ 3 I) v3 = 0 Null space dimension must be equal to 2.

In each case if we form the J and P matrix as above we would get

P-1 A P = J.
Summary

One can find generalized eigenvector, p, which, of course, is linearly independent to all other eigenvectors vi, from (A - iI) p = vi. One can then form a (n n) P matrix such that each column of P matrix contains the linearly independent eigenvectors that are possible to be obtained plus the generalized eigenvectors. Then we can have
P-1 A P = J

where J is called Jordon-Canonical block which is a nearly diagonal matrix whose diagonal elements are the n eigenvalues.

35

 Linear Independence of eigenvectors A set of vectors {xj} is said to be linearly independent if the equation

c1 x1 + c2 x2 + ..... + cn xn = 0
where {cj} are constants, is satisfied only when c1 = c2 = . . . . = cn = 0. When A has distinct eigenvalues, j, (i.e., no repeated eigenvalues), then the set of eigenvectors {vj} forms a linearly independent set. Let vi and vj be eigenvectors that belongs to 1 and j, respectively. Then vi and
vj are not constant multiples of each other, because if they were, then vi = k vj

(a)

where k is a nonzero constant. If we pre-multiply (a) by A, then

Avi = k Avj

(b)

which is equivalent to i v i = k j vj If we now multiply both sides of (a) by i, then we have i vi = k i vj Subtracting (c) from (d), we have k (i - j) vj = 0 Since k 0, and, i j, this means that vj = 0, which contrary to the assumption that vj is an eigenvector. Therefore, for different eigenvalues, the corresponding eigenvectors are
linearly independent.

(c)

(d)

36

 Bi-orthogonality property of eigenvectors Consider two eigenvalue problems

Av=v

(1) (2)

and

AT w = w

Since, the condition that equation (2) has nontrivial solution is det (AT - I) = 0 or, a11 a21 a12 a22 a1n a2 n an1 an 2

=0

ann

Since, the value of determinant remains unchanged when its rows and columns are interchanged, we have, = , and (2) becomes
AT w = w

(2a)

Therefore, (1) and (2a) have the same eigenvalues, i = 1, 2, ...., n, but have
different set of eigenvectors. vj of (1) is one of the non-zero columns of

adj [A - j I], while wj of (2a) is one of the non-zero columns of adj [AT - j I]. It will be same only when A is symmetric, i.e., A = AT. However, columns of adj [AT - j I] are rows of adj [A - jI]. Therefore, wj is simply non-zero rows of adj [A - jI]. Hence, both eigenvectors, vj and wj,
can be obtained from the same calculation, adj [A - j I].

As shown earlier, wj, can also be shown to be linearly independent like vj.
37

However, apart from being linearly independent, the vectors, vj and wj, satisfy the bi-orthogonality property. This means that each member of one set is orthogonal to each member of the other set, except for the one with which it has a common eigenvalue. By definition, and
A vj = j v j wiT A = i wiT,

(3) ij (4)

Pre-multiplying (3) by wiT, and post-multiplying (4) by vj, we have

wiT A vj = j wiT vj wiT A vj = i wiT vj
Subtracting the above two equations, we have

(i j) wiT vj = 0 Since, we assumed that i j, we have

wiT vj = <wi, vj> = 0

(5)

That is, wi and vj and are orthogonal to each other for i j. When matrix A is symmetric, A = AT, and therefore, adj [A iI] = adj [AT iI] and
vi = w i viT vj = <vi, vj> = 0 for i j.

Hence, equation (4) becomes Thus, we have an orthogonal set of vectors {vi}, that is, each member of the set is orthogonal to every other member of the set.

38

Example:

Let

1 0 0 A= 1 2 3 0 2 3 = 0, -1, -5
v1

f () = - 3 - 6 2 - 5 = 0 For = 0:

1 0 0 0 0 0 adj [A - I] = adj 1 2 3 = 3 3 3 0 2 3 2 2 2 For = -1:

v2

w1

0 0 0 4 0 0 adj [A - I] = adj 1 1 3 = 2 0 0 0 2 2 2 0 0 For = -5:

4 0 0 0 0 0 adj [A - I] = adj 1 3 3 = 2 8 12 0 2 2 2 8 12

w3

The eigenvectors of A is given by the non-zero columns of adj [A - iI] and are denoted by vi, whereas the eigenvectors of AT is given by the non-zero
rows of adj [A - iI] and are denoted by wi.

39

Therefore, 0 v1 = 3 2 2 v2 = 1 1
0 v3 = 1 1

1 w1 = 1 1
1 w2 = 0 0

1 w3 = 4 6

It can be easily observed that <v1, w2> = <v1, w3> = <v2, w1> = <v2, w3> = <v3, w1> = <v3, w2> = 0 i.e., <vi, wj> = 0 for i j or the vectors vi and wj are bi-orthogonal.

40

 Expansion of an arbitrary vector We have seen that each square matrix A generates two sets of linearly independent vectors via eigenvectors associated with the eigenvalue problems

Av=v

and

AT w = w

The two sets of eigenvectors, {vi} and {wi}, corresponding to eigenvalues, {i}, i = 1, 2, . . . . . . . . , n, are nonzero columns and rows, respectively, of adj (A i

I). In addition, the eigenvectors, {vi} and {wi}, satisfy the bi-orthogonality
criteria <wj, vi > = 0, i j <wj, vi > 0, i = j We can now write any arbitrary vector z as

z = c1 v1 + c2 v2 + c3 v3 + . . . . . + cn vn =

i =1

ci vi

where ci's are constant to be determined. If we take an inner product with wj, we have < wj, z> = < wj, or, cj =
<w j ,z> < w j ,v j >

i =1

ci vi > = cj <wj, vj>

<wj, vi> = 0 except when i = j

and the arbitrary vector can be written as z = c jv j = v < w j ,v j > j j =1 j =1 If A is symmetric, then wj = vj, and z=
n n <w j ,z>

n <v , z > j v <v j ,v j > j j =1

41

 Application eigenvalues-eigenvectors in solving a system of

linear algebraic equations

Consider the following set of n-linear algebraic equations

Ax=b
known vector of constants, and x is a vector of n-unknowns.

(1)

Where A is a non-singular matrix (det A 0) with simple eigenvalues, b is

We can express any arbitrary vector x and b in terms of eigenvectors of A as

x=

i =1

i vi i vi
n

(2)

and

b=

(3)

i =1

where the eigenvalues {i} and eigenvectors {vi} are obtained from

Av=v
i's can be determined from
< w ,b > , i i>

(4)

i = < w i, v

i = 1, 2, . . . . , n

(5)

where {vi} and {wi} are eigenvectors, nonzero columns and rows respectively of adj (A i I), corresponding to eigenvalues, i. Substituting (2) and (3) in (1), and using (4) we have

i =1

i vi = i i vi = i vi
i =1 i =1

Rearranging, we have

i =1

[ i i i ] vi = ci vi = 0
i =1

42

However, the set of eigenvectors {vi} is linearly independent, therefore, the set of constants {ci}, are equal to zero for all i. That is,

i i = i,
i i

i = 1, 2, . . . . , n.

or, the unknown constants, i, is given by

i =

i = 1, 2, . . . . , n.

(6)

and the solution of A x = b is given by

n n x = i vi = i vi i =1 i =1 i

(7)

where i's are obtained from (5), and i and vi are eigenvalues and corresponding eigenvectors of A.
Note that no solution exist for x from (7) when any i = 0. This is expected

because if any of the eigenvalues of A is zero, then det A must be equal to zero, i.e., A is singular, and we know that no solutions exist. Moreover, if = 0 is an eigenvalue, then we have

Av=v=0

[since = 0]

(8)

Where v is the eigenvector corresponding to the zero eigenvalue. Since, nontrivial solution of (8) can occur only if det A = 0, we must have v = 0, which contrary to the definition of eigenvector. Therefore, if det A 0, = 0 can not be an eigenvalue of A.
Example: Solve for x for the following Ax = b problem

Where,

3 0 1 A = 0 3 0 and b = 1 0 3

0 2 2

The eigenvalues are 2, -3, 4 and the eigenvectors are

43

1 v1 = 0 , 1
1 = 2

0 v2 = 1 0
2 = -3

and

1 v3 = 0 1
3 = 4

Note that the matrix is symmetric, therefore,

vi = wi < vi, wj> = 0 for i j.

Now, b = i vi
i =1 n

i can be calculated either from 1 1 0 + 2 1

or from
< vi , b > < vi , vi >

1 0 0 1 + 3 0 = 2 1 2 0

3 Equations 3 Unknowns

i =
Therefore,

Equation (5)

1 = 2 = 3 =

< v1 , b > < v1 , v1 > < v2 ,b > < v2 , v2 > < v3 , b > < v3 , v 3 >

= 22 = 1 = 12 = 2 =2 =1 2
Equation (7)

The solution x is given by

1 2 v + 3 v x= v + 1 2 3 1 2 3

or,

x1 1 1 x2 = 2 0 + x3 1

0 1 1 4 2 1 + 1 0 = 2 3 4 3 0 1 3 4
44

Example: Solve for x for the following Ax = b problem

Where,

4 2 1 A= 4 1 2 and b = 2 2 2

3 3 6

The eigenvalues are -3, -3, 6 and the eigenvectors are

2 v1 = 2 , 1
1 = 6

+ 0.5 v 2 and v 3 =
2, 3 = -3, -3

1 By arbitrarily selecting = 1, = 0, one can obtain v 2 = 1 . 0

By arbitrarily assigning any other values for and , one can obtain another eigenvector. However, we want the eigenvectors to be orthogonal. Therefore, select and such that

+ 0.5 =0 < v 2 , v 3 > = [ 1 1 0]

= 0.25

1 Therefore, if we arbitrarily select = 1, we obtain v 3 = 1 . 4

Now,

b = i vi
i =1

i can be calculated from i = Therefore,

45

< v i ,b > < vi , vi >

1 = 2 = 3 =

< v1 , b > < v1 , v1 >

< v 2 ,b > <v2 ,v2 >

=2
=0

< v 3 ,b > < v3 , v3 >

= 1

The solution x is given by

x = 1 v1 + 2 v 2 + 3 v 3 1 2 3

or,
------------

x1 x = 2 x3

2 1 1 1 2 2 + 0 1 + 1 1 = 1 6 3 3 1 0 4 1

Application of Eigenvalues and Eigenvectors in Solving Linear Ordinary Differential Equations of Initial Value Problems (ODE-IVPs)
We now return to complete the solution of the first-order, linear, ODE-IVPs with constant coefficients:
dx = Ax + b ; dt

x(0) = x0

The complete solution was obtained as

n t x = ci vi e i - A-1 b i =1

with the ci obtained from

x0 = ci vi - A-1 b i =1
We take the inner product with wj, to get < wj, ci vi > = < wj, x0> + < wj, A-1 b> i =1 The only nonzero term in the LHS of this equation is when i = j, and, therefore,
< w j ,xo > + < w j , A 1b > , cj = <w , v j > j
46

j = 1, 2, . . . , n

The final, complete solution is then given by

t n < w j ,xo > + < w j , A 1b > v e j A 1b x= <w , v j > j j j=1

If A is symmetric, then wj = vj, and the above equation reduces to

t n < v j ,xo > + < v j , A 1b > v e j A 1b x= <v ,v j > j j j=1

Example:
dy1 dt dy 2 dt

= y2 = 1000 y1 1001 y 2 + 5

y1 (0) = 1 y 2 (0) = 1

or,
d dt

1 y1 0 y = 1000 1001 2

y1 0 y + 2 5

1 y (0) = 1

or,
dy dt

= Ay + b

y( 0) = y 0

where
1 0 A= and 1000 1001 0 b= 5

f() = 2 - (tr A) + det A = 2 + 1001 + 1000 = 0

1 1

= -1, - 1000

For = -1, adj [A + I] = adj

1000 1 = 1 1000 1000 1000 1 1000

v1 = , w 1 = 1 1

47

For = - 1000, adj [A + 1000I] = adj

1 1 1 1000 = 1000 1 1000 1000 1

v2 = , w2 = 1000 1

The general solution of

dy dt

= Ay + b

y( 0) = y 0

is given by
y = ci v i e i t A 1 b = c1 v1 e 1t + c2 v 2 e 2 t A 1 b
i =1 n

or, or,

1 1 1000t 1 1001 1 0 1000 y = c1 e t + c2 e 1000 0 1 1000 5

e t e1000t 0.005 + y = c1 t + c2 1000t 0 e 1000 e 1

Apply initial conditions, at t = 0, y(0) = y0 = 1 i.e.,

1 1 1 = c1 1 + c2 1 0.005 1000 + 0

or,

1 = - c1 - c2 + 0.005 -1 = c1 + 1000 c2

1 1 c1 0.995 1 1000 c = 1 2 c1 c = 2
1 1000 999 1 994 1 0.995 999 1 = 0.005 1 999

or,

48

The constants, ci, can also be determined from

<w1,y o > + <w1,A1b> c1 = = <w1, v1> 1 0.005 [1000 1] + [1000 1]

1 [1000 1]

= 994
999

c2 =

<w 2 ,y o > + <w 2 ,A1b> = < w1 , v 2 >

1 0.005 [1 1] + [1 1]

0 1 [1 1] 1000

= 0.005
999

Therefore, ------------------

t 0.005 e 1000t 0.005 y1 994 e = + y 999 t e 999 1000 e 1000t 0 2

We now discuss the solution of the first-order, linear, ODE-IVPs with constant coefficients using similarity transformation

ODE-IVPs of the form dy/dt = Ay

Consider a single linear differential equation of the type
dy = ay ; dt

y ( 0) = y o

where a is a constant. The solution of the above equation can be easily obtained:
y

yo

y = a dt
0

dy

or,

y = e at y o

Consider, now, a system of linear ODE-IVPs with constant coefficients:

dy1 = a11 y1 + a12 y 2 + . . . . + a1n y n ; y1 ( 0) = y1o dt dy2 = a 21 y1 + a 22 y 2 + . . . . + a 2n y n ; y 2 ( 0) = y 2o dt
49


dyn = a n1 y1 + a n 2 y 2 + . . . . + a nn y n ; y n ( 0) = y no dt

The above set of equations can be written in matrix form as

dy dt

= Ay ;

y( 0) = y 0

where
y1 y y = 2 ; yn a11 a12 a a A = 21 22 a a n1 n 2 a1n y1o y a2n ; y (0) = 2o ann yno

Using analogy, we can write the solution of this equation as

y = e At y o

We now show that above is, indeed, a solution. We have

2 2 e At = I + At + A t + . . . . 2 2 2 (this is similar to e at = 1 + at + a t + . . . . ). We have 2 dy d A 2 t 2 + . . y = + + y o e At = d I A t o 2 dt dt dt
or ,

32 22 dy = A + A 2t + A t + . . . yo = A I + At + A t + . . yo 2 2 dt

= A e At y o = Ay

[ ]

Alternatively, we can simplify the equation, dt = Ay , by using matrices, U or

dy

P. We have seen earlier that if we have n linearly independent eigenvectors for a

matrix, A, then

U-1 A U = D
50

where D is a diagonal matrix containing the eigenvalues of A. On the other hand, if we do not have n linearly independent eigenvectors for A, we can use

P-1 A P = J
The columns of U contain the linearly independent eigenvectors of A while the columns of P contain all the linearly independent eigenvectors possible to be obtained from A, plus some generalized eigenvectors. Therefore,
2 2 e At = I + At + A t + . . . . 2

2 = U I U 1 + U D U 1 t + U D 2 U 1 t2 + . . .
2 = U I + D t + D 2 t + . . . U 1 2

= U e D t U 1

where,
e 1t 0 t D e = 0
0 2t e 0 0 0 n t e

Therefore, the solution

dy = Ay ; dt y (0) = y o

can be written in terms of the eigenvalues and the corresponding eigenvectors of

A as
y = e At y o

= U e D t U 1 y o

The above can also be derived as follows:

51

We define y Uz. This satisfies constants. Hence, we have

dy = U dz , since the elements of U are dt dt

dy = Ay ; y (0) = y o dt

or,
U dz = A Uz ; Uz (0) = y o dt

We pre-multiply this equation by U-1 to get

U -1U dz = U -1A Uz ; U -1Uz (0) = U -1y o dt

or, where, zo U-1 yo.

dz = Dz ; z (0) = z o dt

These equations can be represented by

dz1 = 1z1 ; z1 (0) = z1o dt

dz 2 = 2 z 2 ; z 2 (0) = z 2o dt

dz n = n z n ; z n (0) = z no dt

Since, dti depends only on zi, this set of equations is uncoupled and each of these can be solved individually (instead of simultaneously).The solution of this system is
t z1 (t) = z10 e 1 t z 2 (t) = z 20 e 2

dz

52


t z n (t) = z n0 e n

or,
t z1(t) e 1 z (t) 2 = 0 z n (t) 0 0 t e 2 0 0 z1o z 0 2o n t z no e

or,

z(t) = exp(Dt) zo
But,

y(t) = U z(t)
So
y1(t) z1(t) y (t) z (t) 2 = u u 2 1 2 un y n (t) z n (t)

or,
1t y1(t) e y (t) 2 = [U ] 0 y (t) n 0 0 t e 2 0 y1o 0 -1 y 2o 0 [U ] t y no e n

Therefore, solution of coupled linear ODE-IVPs can be obtained by using similarity transformation (U matrix) to decouple the ODE-IVPs as given above.

53

Still further, note that since

U-1 A U = D
We have,

A = U D U-1
Or, And therefore,

A2 = (U D U-1) (U D U-1) = U D2 U-1 An = U Dn U-1

Example: Consider the following ODE-IVPs

dy 1 = 3y + y ; 1 3 dt y1(0) = 0

dy 2 = 3 y ; 2 dt
dy 3 = y + 3y ; 1 3 dt

y2 (0) = 2
y3 (0) = 2

or, where

dy = Ay ; y (0) = y o dt

y 3 0 1 0 1 A = 0 3 0 ; y = y 2 ; y o = 2 y 1 0 3 2 3

The eigenvalues and eigenvectors of A can easily be obtained as: i = 4, -3, 2 The U matrix containing the corresponding eigenvectors is given by
1 U = 0 1 1 = 4 0 1 0 1 0 1 3 = 2

2 = -3

54

Since A is symmetric, its eigenvectors are orthogonal to each other. If we normalize the eigenvectors, then UT = U-1. Normalization of the eigenvectors gives
1 1 0 2 2 U= 0 1 0 1 1 0 2 2

Then,
1 1 0 2 2 U-1 = UT = 0 1 0 1 0 1 2 2

1 1 0 2 2 U-1 yo = 0 1 0 1 1 2 0 2

0 2 2 = 2 2 2

Therefore, y(t) is given by

y(t) = U [eDt] U-1 yo

or
1 y1 (t ) 1 2 0 2 y (t ) = 0 1 0 2 1 1 y t ( ) 3 2 0 2

e 4t 0 0

0 e 3t 0

0 2 0 2 e 2t 2

or,
y1 (t ) y (t ) = 2 y3 ( t )

e 4t e 2t 3t 2e e 4t + e 2t

Note that the above differential equation has eigenvalues of 4, -3, and 2. These appear in the exponential terms in the final solution and control the steady state behavior.
55

For large value of t (i.e., near steady state most of the time we are interested in such steady state solutions only), y(t) approaches e4tu1, where u1 is the eigenvector corresponding to the dominant eigenvalue, l, and blows up (tends to infinity). This is why it is often important to know only the dominant eigenvalue of A and its associated eigenvector. The steady state solution is stable if all the eigenvalues have negative real parts. Otherwise, the steady state solution is unstable in the sense that it approaches infinity as time, t, approaches infinity. Also, it is to be noted here that the most rapidly decreasing solution corresponds to the eigenvalue, -3, while the most rapidly increasing solution corresponds to the eigenvalue, 4. The problem (ODE-IVP) is said to be stiff if it has eigenvalues such that one has a very large magnitude, while another has a very small magnitude, i.e., l >> n. This is represented mathematically in terms of a stiffness ratio, (max / min). Large values of this ratio lead to computational problems, as discussed later when numerical solution of ODE-IVPs are discussed.

-----Example: Consider the following reaction network in an isothermal batch

reactor A k1 k2 D with CA(0) = CA0, CB(0) = CB0, CC(0) = CC0, CD(0) = CD0. The corresponding model equations (ODE-IVPs) are:
56

k3 k4

dC A = ( k1 + k 2 ) C A dt dCB = k1C A k 3CB + k 4 CC dt dCC = k 3CB - k 4 CC dt dCD = k2 CA dt

or,
C A d CB = dt CC CD 0 ( k1 + k 2 ) k3 k1 0 k3 0 k2 0 k4 k4 0 0 0 0 0 C A C B CC CD

or,

dy = Ky ; y (0) = y o dt
The rank of matrix, K, is always less than n (n = 4, in this example) because of the law of conservation of mass (mass cannot be created or destroyed) must be satisfied, i.e.,

dC A dCB dCC dCD + + + =0 dt dt dt dt

One of the above equations, thus, must be a linear combination of the others. Also, since, the determinant of K is zero, one of its eigenvalues must be zero. We leave the further solution of this problem as an exercise. --------

57

Example: Consider the following matrix

2 1 A= 1 2
(i) Determine U and D matrix such that U-1 A U = D. (ii) Using above information, solve the equation
d x = A x; dt

x(t = 0) = x 0

d i.e., dt

x1 2 1 x1 x1 (t = 0) 1 = x 1 2 x ; x ( t = 0 ) = 0 2 2 2
= 1, 3

The eigenvalues of A is given by f() = 2 - 4 + 3 = 0 The eigenvector for = 1:

1 1 1 1 1 v v = adj[A I] = adj = = 1 1 1 1 1

The eigenvector for = 3:

1 1 1 1 1 = v = adj[A I] = adj v = 1 1 1 1 1
1 0 Therefore, D = 0 3 1 1 U= 1 1
1 2 1 2

and

1 2 Since, A is symmetric, normalize U matrix: U = 1 2

Then, U-1 = UT =
1 2 1 2

1 2 1 2

58

Now, let x = Uy, then

d x = Ax dt

dy = D y , dt

y(0) = U-1 x(0)

1 2 1 2

d y1 = 1 0 y1 ; 0 3 y dt 2 y2

y1 (0) y ( 0 ) = 2

1 1 2 = 1 0 2
1 2

1 2 1 2

Solve

dy1/dt = -y1, y1 = C exp[-t]

1 2

y1(0) =

Substituting initial conditions, we have Therefore, y1 =

1 2

=C

exp[-t]

Solve

dy2/dt = -3 y2 , y2 = C exp[-3t]
1 2

y2(0) =

1 2

Substituting initial conditions, we have Therefore, y2 =

1 2

=C

exp[-3t]

Substituting back, we have

1 x1 2 x = Uy = 1 2 2 ------1 2 1 2

1 et 1 t e + e 3t 2 = 2 1 e 3t 1 e t + e 3t 2 2

] ]

The above example is when we have distinct eigenvalues and n-linearly independent eigenvectors such that we can form U matrix.

59

However, when eigenvalues are repetitive and we need to use generalized eigenvector(s) to form P matrix, coupled ODE-IVPs can not be completely decoupled but it is possible to nearly de-couple as shown in the next example.
Example: Consider the following matrix

1 1 A= 1 3
(i) Determine P and J matrix such that P-1 A P = J. (ii) Using above information, solve the equation
d x = Ax; dt

x(t = 0) = x 0

1 1 x1 x1(t = 0) 0 d x1 i.e., dt = x ; x ( t = 0 ) = 1 x 1 3 2 2 2
The eigenvalues are given by f() = 2 - 4 + 4 = 0 The eigenvector for = 2: = 2, 2

1 1 1 1 1 = v = adj[ A I ] = adj v = 1 1 1 1 1
Determine generalized eigenvector from [A - I] p = v

1 1 p1 1 0 = p = 1 1 p2 1 1
0 1 P= ; 1 1 0 1 P 1 = 1 1

2 1 Therefore, J = ; 0 2

Now, let x = Py, then

d x = Ax dt

dy = Jy , y(0) = P-1 x(0) dt

60

d dt

y1 2 1 y1 = ; y 0 2 2 y2
dy2/dt = -2 y2 , y2 = C exp[-2t]

y1 (0) 1 0 0 0 = = ( 0 ) y 1 1 1 1 2
y2(0) = -1

Solve first

Note that the second equation is function of y2 only. Substituting initial conditions, we have -1 = C Therefore, Solve now substituted. Substituting initial conditions, we have 0 = C Therefore, y1 = t exp[-2t] y2 = - exp[-2t] dy1/dt = -2 y1 - y2, = -2 y1 + exp[-2t] y1 = C exp[-2t] + t exp[-2t] Note that first equation is now function of y1 only as solution of y2 can be y1(0) = 0

0 te 2t x1 1 te 2t Py = = = x 1 1 2t 2t + e 2t e te 2

61

ODE-IVPs of the form dy/dt = Ay, with all eigenvalues of A being equal
We again consider the ODE-IVP:
dy = Ay ; y (0) = y o dt

but, with the eigenvalues of A being all equal, say . We need to use the P matrix now. We substitute y = Pz in the above equation to get
P dz = A Pz dt

or,
dz = P 1 AP z = J z ; z (0) = P 1y o dt

or,

z1 1 z 0 2 d = dt z n 1 0 0 0 0 zn
This can be expanded to
dz 1 =z +z 1 2 dt

0 0 z1 z 1 0 2 0 1 z n 1 0 zn

dz 2 = z +z 2 3 dt

dz n 1 = z n 1 + z n dt

dz n = zn dt

The solution of the above set of ODE-IVPs is given by zn = zn(0) et zn-1 = zn-1(0) et + zn(0) t et
62

zn-2 = zn-2(0) et + zn-1(0) t et + zn(0) (t2/2) et z1 = z1(0) et + z2(0) t et +. . . + zn(0) [tn-1/(n-1)!] et or, in matrix form, by

1 0 = e t z n 1 0 z n 0 z1 z2
or,

t 1 0 0

t2 2 t 0 0

t n 1 (n 1)! t n 2 (n 2)! t 1

z1(0) z (0) 2 z (0) n 1 z n (0)

z = K zo
This leads to

y = P z = P K zo = P K P-1 yo

63

Higher-order IVPs
Let us now consider the following higher-order IVP with constant coefficients:
d3y dt 3 d2y +a 2 dt dy + b dt

+ cy = d ;

dy ( o ) y(o) = , dt

d 2 y(o) , 2 dt

We can convert this equation into a set of first order ODE-IVPs using the procedure described below. We define y = y1 then
dy dy1 = = y2 dt dt
d 2 y dy 2 = = y3 dt dt 2 d 3 y dy 3 = dt dt 3

Substituting the above equations in the original equation, we have

dy3 dt

= ay3 by 2 cy1 + d

or,

1 0 y1 0 y1 0 d 0 1 y = 0 y2 + 0 dt 2 y3 c b a y3 d
This may be written in matrix form as

dy = Ay + e ; y (0) = dt

64

Note that the vector, e, is a constant (and not a function of time). We can transform this equation into a homogeneous one by using the concept of the steady state (SS; represented by subscript, S) solution. We have at steady state:
dy S = 0 = A yS + e dt

or

yS = - A-1e = [d/c 0 0]T

We now subtract the SS equation from the original one to obtain

d(y y s ) = A (y ys ) dt
If we define x y - yS, we have

dx = Ax ; x(0) = x o dt
and the solution is

x(t) = U e Dt U 1 x o
or

y (t) = y s + U e Dt U 1 y o y os = A 1 e+ U e Dt U 1 y o y os

65

ODE-IVPs of the form dy/dt = Ay + B(t)

We now consider the more general equation dy/dt y' = Ay + b(t) where b is a function of time. Using y = Uz, we obtain

U z' = y' = Ay + b(t) = A U z + b(t)

On pre-multiplying both sides of this equation with U-1, we obtain

U-1 U z' = U-1 A U z + U-1 b(t)

or,

z' = D z + U-1 b(t) D z + g(t)

Here, g(t) U-1b(t), and D is a diagonal matrix whose entries are the eigenvalues, 1, 2, . . . , n, of A, arranged in the same order as the eigenvectors, u1, u2, . . . , un, (appearing as columns in U). The above is a system of n-uncoupled ODE-IVPs in z1(t), z2(t), . . . , zn(t), and may be solved individually. The ith equation may be written as z'i (t) = dzi/dt = i zi (t) + gi (t); i = 1, 2, . . . , n Above has the following solution

t t t s z i (t) = c i e i + e i e i g i (s) ds ; i = 1, 2, . . . , n to
This may be used with y = Uz to give the solution, y(t).

66

Example: Solve

y1'' +102 y1' + 200 y1 = t, y1(0) = 1, y1'(0) = - 2 y1(0) = 1 y2(0) = - 2

Define then

y 1 ' = y2 y1'' = y2' = - 102 y2 - 200 y1 + t

or,

d dt

1 y1 0 y1 0 = y 200 102 y + t 2 2
= A y + g(t )

y1 (0) 1 y ( 0) = 2 2

or, where

dy dt

y( 0) = y 0

y y = 1 ; y2
Eigenvalues of A:

1 0 A= ; 200 102

0 g= t

1 ; y0 = 2

f() = 2 - (tr A) + det A = 2 +102 + 200 = 0 Eigenvectors of A for = -2:

= -2, -100

1 100 1 2 adj [A + 2I] = adj = 200 200 100 2

Eigenvectors of A for = -100:

1 v1 = 2

1 2 1 100 adj [A + 100I] = adj = 200 2 200 100

The matrix U, U-1 and D are given by

1 v2 = 100

1 1 U= 2 100
so that U-1 A U = D.

1 1 100 U 1 = 98 2 1

0 2 D= 0 100

67

Substituting y = U z in y' = Ay + g(t), we have y' = U z' = A U z + g(t) Or,

z' = U-1 A U z + U-1 g(t) = D z + U-1 g(t)

Or,

d dt

0 z1 2 z = 0 100 2

z1 1 100 1 0 z + 98 2 1 t 2

Or,

0 z1 1 t 2 = 98 t 0 100 z 2
y0 = U z0

Since, y = U z, we have or, or,

z0 = U-1 y0

1 1 1 1 100 z0 = 98 2 = 0 2 1
dz1 dt t = 2 z1 98
t = 100 z2 + 98

Therefore, we have

z1 (0) = 1
z 2 ( 0) = 0
Note that

and

dz 2 dt

or, z1 = c1e

2t

t 1 e 2t se 2 s ds 98 0

ay e ax 1 ye dy = a x a

or,

1 2t 1+ e 2t = c1e 2t 98 4

Applying initial conditions: at t = 0, z1 = -1, we have -1 = c1 - (1/98) (0) Therefore, c1 = -1

1 z1 = e 2t 392 2t 1 + e 2t

]
68

Similarly, z2 can be obtained as

z2 =

9.8 x105

[100t 1 + e ]
100t

Substituting z in y = U z, we can get desired solution y.

69