Professional Documents
Culture Documents
AXEL RUHE1
April 2, 2007
1
Department of Numerical Analysis and Computer Science, Royal Institute of Tech-
nology, SE-10044 Stockholm , Sweden (ruhe@kth.se). Material given to graduate
course in Numerical Linear Algebra.
2
Preface
This is some of the material I find worth covering in a course for graduate stu-
dents in Scientific Computation who will need to understand algorithms based
on linear algebra and use them in different applications.
For the theory, I like the exposition by Stewart and Sun [5], but you can find
most of the material in the text book by Demmel [2]. Algorithms, as the state of
the art was in year 2000, are described in the Templates for Eigenvalue book [1].
A more detailed description of the algorithms involved is given in the book series
of Stewart, see [4]! We also need to follow some of the later developments, global
algorithms based on matrix sign functions, fast algorithms for tridiagonals and
high relative precision algorithms to name three interesting topics.
Stockholm April 2, 2007
Axel Ruhe
Chapter 1
Eigenvalues
x(t) = Σk αk xk eλk t .
A = UTUH
3
4 CHAPTER 1. EIGENVALUES
Ax = xλ1
AU1 = [xλ1 , AXn−1 ]
λ A1,n−1
U1H AU1 = 1
0 An−1,n−1
The zeros in the lower left are there because U is unitary, its first column x is
orthogonal to the remaining columns in Xn−1 .
We have now made the first step, repeat the same procedure on the lower
right block An−1,n−1 giving λ2 and U2 , and continue down to the right, until we
have an upper triangular matrix T and a unitary U = U1 U2 . . . Un−1 , with parts
of the unit matrix added to each Uk , to make it n × n. This ends the proof.
AQ = QH (1.2)
The theorem is proved by the Arnoldi algorithm started at the given first
column q1 . If it runs its full course, k = n, the matrix A is non-derogatory,
having only one eigenvector for each eigenvalue and q1 has components from all
eigenvectors. Otherwize the minimal polynomial of A at q1 has degree k.
We use the Hessenberg uniqueness theorem to explain the Householder re-
duction to Hessenberg form and the implicitly shifted QR algorithm. It is also
the basis of implicitly restarted Arnoldi and rational Krylov algorithms.
An important consequence of the Hessenberg uniqueness theorem is that we
can use the theory of Krylov sequences to explain the behavior of any algorithm
involving transformation to Hessenberg form, also those that do not explicitly
let the matrix operate on a sequence of vectors.
We can also say that any decomposition into a matrix of lower dimension is
a kind of Krylov sequence algorithm.
End.
We see that
Ak+1 = Rk Qk = QH
k Ak Qk (1.4)
an orthogonal similarity. The accumulation makes the final
Uk Sk = Q1 Q2 . . . Qk Rk Rk−1 . . . R1
= Q1 . . . Qk−1 Ak Rk−1 . . . R1 (QR-factorization)
= Q1 . . . Qk−1 Ak QH
k−1 Qk−1 Rk−1 . . . R1
= Q1 . . . Qk−2 Ak−1 Qk−1 Rk−1 . . . R1 (by (1.4))
= A1 Q1 . . . Qk−2 Qk−1 Rk−1 . . . R1
= A21 Q1 . . . Qk−2 Rk−2 . . . R1 (repeat!)
= Ak1
From this factorization we can see how the columns of the accumulated trans-
formation matrix Uk approach those of the Schur decomposition. Look at the
first column,
Ak e1 = Uk Sk e1 = (U k )1 s1,1
the result of k iterations of the power method started at the basis vector e1 .
Likewise the last column is related to the inverse power method since,
the result of k iterations of inverse iteration with AH started at the last basis
vector en .
It is slightly more complicated to show that the whole transformation con-
verges to the unitary matrix of Schur vectors. This assumes that the eigenvalues
are ordered after strictly decreasing absolute values, see [4] p. 78 Th 2.2!
Let us formulate a simple variant of the shifted algorithm.
1.3. ITERATIVE EIGENVALUE ALGORITHMS 7
1. Choose shift σk
4. Accumulate Uk+1 = Uk Qk
End.
The new Ak+1 is still an orthogonal similarity of Ak
sk (λ) = (λ − σ1 )(λ − σ2 ) . . . (λ − σk )
xk = Axk−1 (1.5)
8 CHAPTER 1. EIGENVALUES
We get the connection to the Krylov space when we note that it expands in the
direction of the gradient of the Rayleigh quotient:
2
g(x) = ρ (x) = (Ax − ρ(x)Bx) (1.9)
xH Bx