Day 1 Algebraic

Algebraic Methods for Computed Tomography
Per Christian Hansen
DTU Compute
Department of Applied Mathematics and Computer Science
Technical University of Denmark
Per Christian Hansen Algebraic Methods for CT 1 / 50

About Me
Professor of Scientific Computing (since 1996).

Research: inverse problems, tomography, regularization algorithms,
matrix computations, image deblurring, Matlab software, . . .
Author of several Matlab software packages.
Author of four books.
Head of the project High-Definition Tomography, funded by an ERC

Advanced Research Grant (www.compute.dtu.dk/pcha/HDtomo).

Plan for Today
1 A bit of motivation.
2 The algebraic formulation.
3 Matrix notation and interpretation.
4 Kaczmarzs method (also known as ART) fully sequential.
5 Cimminos method and vatiants fully simultaneous.
6 More linear algebra: null space and least squares.
7 The optimization viewpoint.
Points to take home today:
Linear algebra provides a concise framework for formulating the
algorithms.
Convergence analysis of iterative algebraic methods:
Kaczmarzs method = ART converges for consistent problems only.
Cimminos method always converges.
Be careful with the null space.
Least squares problems always have a solution.
The optimization viewpoint leads to a broad class of iterative methods.
FBP: Filtered Back Projection
This is the classical method for 2D reconstructions.
There are similar methods for 3D, such as FDK.
Many year of use lots of practical experience.
The FBP method is very fast (it uses the Fast Fourier Transform)!
The FBP method has low memory requirements.
With many data, FBP gives very good results.
Example with 3% noise:

FBP Versus Algebraic Methods
Limited data, or nonuniform distribution of projection angles or rays

artifacts appear in FBP reconstructions.
Difficult to incorporate constraints (e.g., nonnegativity) in FBP.
Algebraic methods are more flexible and adaptive.
Same example with 3% noise and projection angles 15 , 30 , . . . , 180 :
Algebraic Reconstruction Technique, box constraints (pixel values [0,1]).

Another Motivating Example: Missing Data
Irregularly spaced angles & missing angles also cause difficulties for FBP:

The Simplest Algebraic Problem
One unknown, no noise:
Now with noise in the data compute a weighted average:
We know from statistics that solutions variance is inversely proportional to

the number of data. So more data is better.
Let us immediately continue with a 2 2 image . . .

A Sudoku Problem
Four unknowns, four rays system of linear equations A x = b:
Unfortunately there are infinitely many solutions, with k R:
(There is an arbitrary component in the null space of the matrix A.)

More Data Gives a Unique Solution
With enough rays the problem has a unique solution.
Here, one more ray is enough to ensure a full-rank matrix:
The difficulties associated with the discretized tomography problem are

closely linked with properties of the coefficient matrix A:
The sensitivity of the solution to the data errors is characterized by
the condition number = kAk2 kA1 k2 (not discussed today).
The uniqueness of the solution is characterized by the rank of A, the
number of linearly independent row or columns.
A Look at the System Matrix
Recall that our algebraic model is a system of linear equations
Ax = b
with a very sparse system matrix A. Example: 5 5 pixels and 9 rays:

The System Matrix is Very Sparse
Another example: 256 256 pixels and 180 projections with 362 rays each.
The system matrix A is 65, 160 65, 536 and has 4.27 109 elements.
There are 15, 018, 524 nonzero elements corresponding to a fill of 0.35%.

Algebraic Reconstruction Methods
In principle, all we need to do in the algebraic formulation is to solve

the large sparse linear system A x = b:
Math: x = A1 b, MATLAB: x = A\b.
How hard can that be?

Actually, this can be a formidable task if we try do use a traditional
approach such as Gaussian elimination.
Researchers in tomography have therefore focused on the use of
iterative solvers and they have rediscovered many methods
developed by mathematicians . . .
In tomography they are called algebraic reconstruction methods.
They are much more flexible than FBP, but at a higher
computational cost!

Some Algebraic Reconstruction Methods
Fully Sequential Methods

Kaczmarzs method + variants.
These are row-action methods: they update the solution using one row
of A at a time.
Fast convergence.
Fully Simultaneous Methods
Landweber, Cimmino, CAV, DROP, SART, SIRT, . . .
These methods use all the rows of A simultaneously in one iteration
(i.e., they are based on matrix multiplications).
Slower convergence.
Krylov subspace methods (rarely used in tomography)
CGLS, LSQR, GMRES, . . .
These methods are also based on matrix multiplications.

Matrix Notation and Interpretation
Notation:

r 1

| | |
A = c 1 c 2 c n = ..
,

.
| | | r m
The matrix A maps the discretized absorption coefficients (the vector x) to
the data in the detector pixels (the elements of the vector b) via:

b1 r1 x
b2 r2 x
b = . = A x = x1 c 1 + x2 c 2 + + xn c n = . .

.. | {z } ..
linear combination of columns
bm rm x
The ith row of A maps x to detector element i via the ith ray:
n
X
bi = r i x = aij xj , i = 1, 2, . . . , m.
j=1

Example of Column Interpretation
A 32 32 image has four nonzero pixels with intensities 1, 0.8, 0.6, 0.4.
In the vector x these four pixels correspond to entries 468, 618, 206, 793.
Hence the sinogram, represented as a vector b, takes the form
b = 0.6 c 206 + 1.0 c 468 + 0.8 c 618 + 0.4 c 793 .

Forward and Back Again
ray i

Forward projection. Consider ray i:
x1 x4 x7

aij = length of ray i in pixel j

r i = [ ai1 ai2 0 ai4 0 0 ai7 0 0 ]
x2

x5 x8
bi = r i x = ai1 x1 + ai2 x2 + ai4 x4 + ai7 x7
The forward projection b = A x maps all image x3 x6 x9
pixels xj along the ray to the detector data bi .
Back projection. Another name for multipli- ray i

T
cation with A :

m m bi ai1 ai4 bi ai7
bi ai1 bi
X X bi ai2
T
A b= bi r i =
..

i=1 i=1 . bi ai2 0 0
Each data bi is distributed back along the
ith ray to the pixels, with weights = aij : 0 0 0
bi r i = [ bi ai1 bi ai2 0 bi ai4 0 0 bi ai7 0 0 ]T
Geometric Interpretation of A x = b
r1 x = a11 x1 + a12 x2 + + a1n xn = b1
r2 x = a21 x1 + a22 x2 + + a2n xn = b2
..
.
rm x = am1 x1 + am2 x2 + + amn xn = bm .
Each equation r i x = bi defines an affine hyperplane in Rn :

x3
6 R3
x2
6 R2 x2
"
ai1 x1 + ai2 x2 = bi "
" S

" S
" S
S S x1
-
S
S
S S
S S
"
x S ai1 x1 +
"ai2 x2 + ai3 x3 = bi
-1
"
S "
S"
Geometric Interpretation of the Solution
Assuming that the solution to A x = b is unique, it is the point x Rm

where all the m affine hyperplanes intersect.
Example with m = n = 2:
x2
6 R2
Q a11 x1 + a12 x2 = b1
Q

Q
Q
Q

Q
QQ a x + a x = b

Q 21 1 22 2 2
Q
Q
Q-

x1

Kaczmarzs Method = Algebraic Reconstruction Technique
A simple, efficient iterative method based on the geometric interpretation.
In each iteration, and in a cyclic fashion, compute the new iteration vector
such that precisely one of the equations is satisfied.
This is achieved by projecting the current iteration vector x on one of the
hyperplanes r i x = bi for i = 1, 2, . . . , m, 1, 2, . . . , m, 1, 2, . . .
Originally proposed in 1937, and independently suggested under the name

ART by Gordon, Bender & Herman in 1970 for tomographic reconstruction.
Orthogonal Projection on Affine Hyperplane
ri
z
6

Hi = {x Rn | r i x = bi }

:
P (z)

i

O r
The orthogonal projection Pi (z) of an arbitrary point z on the affine

hyperplane Hi defined by r i x = bi is given by:
bi r i z
Pi (z) = z + ri, kr i k22 = r i r i .
kr i k22
In words, we scale the row vector r i by (bi r i z)/kr i k22 and add it to z.

Kaczmarzs Method
We thus obtain the following algebraic formulation:
Basic Kaczmarz algorithm

x (0) = initial vector
for k = 0, 1, 2, . . .
i = k (mod m)
bi r i x (k)
x (k+1) = Pi x (k) = x (k) +

ri
kr i k22
end
Each time we have performed m iterations of this algorithm, we have

performed one sweep over the rows of A. Other choices of sweeps:
Symmetric Kaczmarz: i = 1, 2, . . . , m1, m, m1, . . . , 3, 2.
Randomized Kaczmarz: select row i randomly, possibly with
probability proportional to the row norm kr i k2 .

Convergence Issues
The convergence of Kaczmarzs method is quite obvious from the graph on

slide 19 but can we say more?
Difficulty: the ordering of the rows of A influences the convergence rate:

1.0 1.0 2.0
1.0 1.1 2.1
x =
1.0 3.0 4.0
1.0 3.7 4.7
The ordering 1324 is preferable: almost twice as fast.

Convergence of Kaczmarzs Method
One way to avoid the difficulty associated with influence of the ordering of
the rows is to assume that we select the rows randomly.
For simplicity, assume that A is invertible and that all rows of A are scaled
to unit 2-norm. Then the expected value E() of the error norm satisfies:
1 k (0)

E kx (k)
xk22 1 kx xk22 , k = 1, 2, . . . ,
n 2
where x = A1 b and = kAk2 kA1 k2 . This is linear convergence.

When is large we have
1 k

k
1 2
1 .
n n 2
After k = n steps, corresp. to one sweep, the reduction factor is 1 1/2 .

Note that there are often orderings for which the convergence is faster!
Cyclic Convergence
So far we have assumed that there is a unique solution x = A1 b that

satisfies A x = b, i.e., all the affine hyperplanes associated with the rows
of A intersect in a single point.
What happens when this is not true? cyclic convergence:
m = 3, n = 2
Kaczmarzs method can be brought to converge to a unique point, and we

will discuss the modified algorithm later today.

From Sequential to Simultaneous Updates
Karzmarzs method accesses the rows sequentially. Cimminos method
accesses the rows simultaneously and computes the next iteration vector as
the average of the all the projections of the previous iteration vector:
m m
1 X 1 X (k) bi r i x (k)
x (k+1) = Pi x (k) =

x + ri
m m kr i k22
i=1 i=1
m
(k) 1 X bi r i x (k)
= x + ri.
m kr i k22
i=1
P1 (x (k) )
r
H2 J
Q Jrx (k)
r
Q
Q
Q x (k+1)
Q
Q
Q r
P2 (x (k) ) Q
Q
H1 QQ

Matrix formulation of Cimminos Method
We can write the updating in our matrix-vector formalism as follows

m
(k+1) (k) 1 X bi r i x (k)
x = x + ri
m kr i k22
i=1
(k)

b 1 r 1 x
1 r1 r m ..
= x (k) + 2

2 .
m kr 1 k 2 kr m k 2

bm r m x (k)
T
kr 1 k2

r1 2 r1
1
= x (k) + ... .. . (k)
b .. x

.
m
rm kr m k2
2 rm
= x (k) + AT M 1 b A x (k) ,

where we introduced the diagonal matrix M = diag mkr i k22 .


Cimminos Method
We thus obtain the following formulation:
Basic Cimmino algorithm

for k = 0, 1, 2, . . .
x (k+1) = x (k) + AT M 1 b A x (k)

end
Note that one iteration here involves all the rows of A, while one iteration
in Kaczmarzs method involves a single row.
Therefore, the computational work in one Cimmino iteration is equivalent
to m iterations (a sweep over all the rows) in Kaczmarzs basic algorithm.
The issue of finding a good row ordering is, of course, absent from
Cimminos method.

Convergence Study
Assume x (0) = 0 and let I denote the n n identity matrix; then:
k
X
x (k+1) = (I AT M 1 A)j AT M 1 b
j=0

= I (I AT M 1 A)k+1 (AT M 1 A)1 AT M 1 b.
If A is invertible then
(AT M 1 A)1 AT M 1 b = A1 M AT AT M 1 b = A1 b.
Moreover, the largest eigenvalue of the symmetric matrix I AT M 1 A is

strictly smaller than one, and therefore

I (I AT M 1 A)k+1 I for k .
Hence the iterates x (k) converge to the solution x = A1 b.

Convergence of Cimminos Method
To simplify the result, assume that A is invertible and that the rows of A
are scaled such that kAk22 = m. Then
k
2
kx (k)
xk22 1 kx (0) xk22
1 + 2
where x = A1 b, = kAk2 kA1 k2 , and we have linear convergence.
When 1 then we have the approximate upper bound
kx (k) xk22 < 2 k (0) 2

(1 2/ ) kx xk2 ,
showing that in each iteration the error is reduced by a factor 1 2/2 .
This is almost the same factor as in one sweep through the rows of A in
Kaczmarzs method.

Rectangular and/or Rank Deficient Matrices
In tomography, A Rmn is almost always a rectangular matrix: m 6= n.

It is also very common that A does not have full rank.
We need to set the stage for treating such matrices.
The rank r of A is the number of linearly independent rows (equal to the

number of linearly independent columns), and r min(m, n).
The range R(A) is the linear subspace spanned by the columns of A:
R(A) {u Rm | u = 1 c 1 + 2 c 2 + + n c n , arbitrary j }. (1)
The null space N (A) is the linear subspace of all vectors mapped to zero:
N (A) {v Rn | A v = 0}. (2)
The dimensions of the two subspaces are r and nr , respectively.

A Small Example
Consider the 3 3 matrix

1 2 3
A = 4 5 6 .
7 8 9
This matrix has rank r = 2 since the middle row is the average of the first
and third rows which are linearly independent.
The range R(A) and null space N (A) consist of all vectors of the forms

1 3 1 + 32 1
1 4 + 2 6 = 41 + 62
and 3 2 ,

7 9 71 + 92 1
respectively, for arbitrary 1 , 2 , and 3 .

A Small Example, Continued
Consider two linear systems with the matrix A from the previous example:

1 2 3 14 1 2 3 6
4 5 6 x = 20 and 4 5 6 x = 15 .

7 8 9 50 7 8 9 24
The left system has no solution because b

/ R(A); no matter which linear
combination of the columns of A we create, we can never form this b.
The right system has infinitely many solutions; any vector of the form

1 1
x = 1 + 2 , arbitrary

1 1
satisfies this equation. The arbitrary component is in the null space N (A).

Null Space Artifacts in Tomography I
Image has 16 16 pixels and we use 16 horizontal and 16 vertical X-rays

very under-determined system.
Left: three different vectors in the null space N (A).

Right: the exact (ground truth) image x and the reconstruction.
One pixel at the intersection of the vertical and horizontal strips has a
large and incorrect value, and the values of the horizontal and vertical
strips are slightly too low.

Null Space Artifacts in Tomography II
The mage has 29 29 pixels and we use projection angles in [50 , 130 ]
A is 841 841 and rank deficient; the dimension of N (A) is 24.
Both reconstructions are imperfect, and the vertical structure of the second
test image is almost completely lost in the reconstruction.
Null Space Artifacts in Tomography III
Same example the 24 images that span the null space N (A) represent
information about missing vertical structures in the reconstruction:
This illustrates that intuition and mathematics go hand-in-hand.

Consistent and Inconsistent Systems
A system is consistent if there exists at least one x such that A x = b, i.e.,

such that b is a linear combination of the columns c i of A.
This is equivalent to the requirement b R(A).
Otherwise the system is inconsistent, b

/ R(A), as shown below.
b
>

R(A)

* c3

: c2

- c1

Overview of Systems: m n
Full rank Rank deficient
m<n r =m r <m
Underdetermined Always consistent. Can be inconsistent.
Always infinitely No solution or
=
many solutions. infinitely many
solutions.
m=n r =m=n r <m=n

Square Always consistent. Can be inconsistent.
Always a No solution or
unique solution. infinitely many
= solutions.
The system is inconsistent when b / R(A).

There is a unique solution only if r = m = n.
Overview 0f Systems: m > n
Full rank Rank deficient

m>n r =n r <n
Overdetermined Can be inconsistent. Can be inconsistent.
No solution or No solution or
= a unique ininitely many
solution. solutions
The system is inconsistent when b

/ R(A).
There is a unique solution only if r = n and the system is consistent.

The Least Squares Solution
We must define a unique solution for inconsistent systems!
Assume that b = A x + e and e is zero-mean Gaussian noise. The best
linear unbiased estimate of x is the solution to the least squares problem:
x LS = arg min 1/2 kb A xk22 ,
x
and x LS is unique when r = n. Geometrically, this corresponds to finding

x LS such that A x LS is orthogonal to the residual vector b A x LS .
b
>6

R(A)

1

A x LS


Computing the Least Squares Solution
The requirement that A x LS (b A x LS ) leads to:

T
xT T T

A x LS b A x LS = 0 LS A b A A x LS = 0
which means that x LS is the solution to the normal equations:
AT A x = AT b x LS = (AT A)1 AT b.
x LS exists and is unique when AT A is invertible, which is the case when

r = n (i.e., the system is over-determined and A has full rank).
Bonus info: the matrix A = (AT A)1 AT is called the pseudoinverse

(or Moore-Penrose inverse) of A.

The Minimum-Norm Least Squares Solution
If r < n we can define a unique minimum-norm least squares solution by:
x 0LS = arg min kxk2 subject to AT A x = AT b.

x
Example. Consider again the problem

1 2 3 14
4 5 6 x = 20 with r = 2 < n = 3,
7 8 9 50
x LS is not unique, and all least squares solutions have the form

3 1
x LS = 2 + 2 , arbitrary.
1 1
The minimum-norm least squares solution x 0LS is obtained by setting = 0.

Weighted Least Squares Solutions
Recall our definition of the diagonal matrix M = diag mkr i k22 .

We also define the weighted least squares problem
min 1/2 kM 1/2 (A x b)k22 (AT M 1 A) x = AT M 1 b

x
and the corresponding solution x LS,M = (AT M 1 A)1 AT M 1 b.

Similarly we define the minimum-norm weighted least squares solution
x 0LS,M = arg min kxk2 subject to AT M 1 A x = AT M 1 b.

x
The full picture of Cimminos method:

r = n = m: convergence to A1 b.
r = n < m and b R(A): convergence to x LS .
r = n < m and b / R(A): convergence to x LS,M .
r < min(m, n): convergence to x 0LS,M .
The Optimization Viewpoint
Karczmarz, Cimmino and similar algebraic iterative methods as usually
considered as solvers for systems of linear equations.
But it is more convenient to consider them as optimization methods.
Within this framework we can easily handle common extensions:
We can introduce a relaxation parameter or step length parameter
in the algorithm which controls the size of the updating and, as a
consequence, the convergence of the method:
a constant , or
a parameter k that changes with the iterations.
We can also, in each updating step, incorporate a projection PC on a
suitably chosen convex set C that reflects prior knowledge, such as
the positive orthant Rn+ nonnegative solutions,
the n-dimensional box [0, 1]n solution elements in [0,1].
We can introduce other norms than the 2-norm k k2 , which can
improve the robustness of the method.
Example: Robust Solutions with the 1-norm
The 1-norm is well suited for handling outliers in the data:
m
X
min kA x bk1 , kA x bk1 = |r i x bi |.
x
i=1
Consider two over-determined noisy problems with the same matrix:

1 1 1 6 6.0001 6.0001
1 2 4 17 17.0285 17.2850

1 3 9 34 33.9971 33.9971
A=
1
, A x =
57 ,
b=
57.0061 ,
bo =
57.0061 .

4 16
1 5 25 86 85.9965 85.9965
1 6 36 121 120.9958 120.9958
Least squares solutions: x LS and x oLS ; 1-norm solutions: x 1 and x o1 :

1.0041 1.0811 0.9932 0.9932
x LS = 2.0051 , x oLS = 2.0151 , x 1 = 2.0087 , x o1 = 2.0088 .
2.9989 2.9943 2.9986 2.9986

The Least Squares Problem Revisited
The objective function and its gradient
F(x) = 1/2 kb A xk22 , F(x) = AT (b A x).
The method of steepest descent for minx F(x), with starting vector x (0) ,
performs the updates
x (k+1) = x (k) k F(x) = x (k) + k AT (b A x),
where k is a step-length parameter that can depend on the iteration.

Also known as Landwebers method corresponds to M = I in Cimmino.
Cimminos method corresponds to the weighted problem:
FM (x) = 1/2 kM 1/2 (b A x)k22 , FM (x) = AT M 1 (b A x).

Incorporating Simple Constraints
We can include constraints on the elements of the reconstructed image.
Assume that we can write the constraint as x C, where C is a convex set;
this includes two very common special cases:
Non-negativity constraints. The set C = Rn+ corresponds to
xi 0, i = 1, 2, . . . , n.
Box constraints. The set C = [0, 1]n (n-dimensional box) corresponds to
0 xi 1, i = 1, 2, . . . , n.

The Projected Algorithms
Both algorithms below solve minxC kM 1/2 (b A x)k2 .
Projected gradient algorithm (k < 2/kAT MAk2 )

for k = 0, 1, 2, . . .
x (k+1) = PC x (k) + k AT M 1 (b A x (k) )

end
Projected incremental gradient (Kaczmarz) algorithm (k < 2)
for k = 0, 1, 2, . . .
i = k (mod m)

(k+1) (k) bi r i x
x = PC x + k ri
kr i k22
end
Iteration-Dependent Relaxation Parameter k
The basic Kaczmarz algorithm gives a cyclic and non-convergent behavior.
Consider the example from slide 24 with:

k = 0.8 (independent of k) and k = 1/ k, k = 0, 1, 2, . . .
The rightmost plot is a zoom of the middle plot.

With a fixed k < 1 we still have a cyclic non-convergent behavior.

With the diminishing relaxation parameter k = 1/ k 0 as k
the iterates converge to the weighted least squares solution x LS,M .
Overview of Convergence (see also slides 3738)
What the unconstr. methods converge to, with starting vector x (0) = 0:
Kac: Kaczmarzs method with a fixed relaxation parameter.
Kd: Kaczmarzs method with a diminishing parameter.
Cim: Cimminos method with a fixed relaxation parameter.
r < min(m, n) r = min(m, n)

b R(A) b / R(A) b R(A) b / R(A)
m<n x 0LS = x 0LS,M
m=n x 0LS = x 0LS,M A1 b
Kac: cyclic Kac: cyclic
m>n x 0LS = x 0LS,M Kd: x 0LS,M x LS = x LS,M Kd: x LS,M
Cim: x 0LS,M Cim: x LS,M
/ R(A), x 0LS,M 6= x 0LS and x LS,M 6= x LS .
NB: for over-det. systems with b

Last Slide: Other Projected Gradient Algorithms
Many algorithm proposed in the literature (Landweber, CAV, DROP,
SART, SIRT, . . . ) are special cases of the following general formulation.
General projected gradient algorithm (k < 2)
for k = 0, 1, 2, . . .
x (k+1) = PC x (k) + k D 1 AT M 1 (b A x (k) )

end
Of particular interest is the method SIRT in which:
kc j k1 = m
P
D = diag kc j k1 , aij ,
Pni=1
M = diag kr i k1 , kr i k1 = j=1 aij
SIRT is implemented in the ASTRA software, to be discussed tomorrow.

Day 1 Algebraic

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Day 1 Algebraic

Uploaded by

Copyright:

Available Formats

Algebraic Methods for Computed Tomography

Per Christian Hansen

Per Christian Hansen Algebraic Methods for CT 1 / 50

Professor of Scientific Computing (since 1996).

Head of the project High-Definition Tomography, funded by an ERC

Per Christian Hansen Algebraic Methods for CT 2 / 50

Per Christian Hansen Algebraic Methods for CT 4 / 50

Limited data, or nonuniform distribution of projection angles or rays

Algebraic Reconstruction Technique, box constraints (pixel values [0,1]).

Per Christian Hansen Algebraic Methods for CT 6 / 50

One unknown, no noise:

Now with noise in the data compute a weighted average:

We know from statistics that solutions variance is inversely proportional to

Per Christian Hansen Algebraic Methods for CT 7 / 50

Unfortunately there are infinitely many solutions, with k R:

(There is an arbitrary component in the null space of the matrix A.)

The difficulties associated with the discretized tomography problem are

Per Christian Hansen Algebraic Methods for CT 10 / 50

Per Christian Hansen Algebraic Methods for CT 11 / 50

In principle, all we need to do in the algebraic formulation is to solve

Math: x = A1 b, MATLAB: x = A\b.

How hard can that be?

Per Christian Hansen Algebraic Methods for CT 12 / 50

Fully Sequential Methods

Per Christian Hansen Algebraic Methods for CT 13 / 50

Per Christian Hansen Algebraic Methods for CT 14 / 50

b = 0.6 c 206 + 1.0 c 468 + 0.8 c 618 + 0.4 c 793 .

Per Christian Hansen Algebraic Methods for CT 15 / 50

Back projection. Another name for multipli- ray i

Each equation r i x = bi defines an affine hyperplane in Rn :

Assuming that the solution to A x = b is unique, it is the point x Rm

Per Christian Hansen Algebraic Methods for CT 18 / 50

Originally proposed in 1937, and independently suggested under the name

The orthogonal projection Pi (z) of an arbitrary point z on the affine

Per Christian Hansen Algebraic Methods for CT 20 / 50

We thus obtain the following algebraic formulation:

Basic Kaczmarz algorithm

Each time we have performed m iterations of this algorithm, we have

Per Christian Hansen Algebraic Methods for CT 21 / 50

The convergence of Kaczmarzs method is quite obvious from the graph on

Difficulty: the ordering of the rows of A influences the convergence rate:

The ordering 1324 is preferable: almost twice as fast.

Per Christian Hansen Algebraic Methods for CT 22 / 50

where x = A1 b and = kAk2 kA1 k2 . This is linear convergence.

After k = n steps, corresp. to one sweep, the reduction factor is 1 1/2 .

So far we have assumed that there is a unique solution x = A1 b that

Kaczmarzs method can be brought to converge to a unique point, and we

Per Christian Hansen Algebraic Methods for CT 24 / 50

Per Christian Hansen Algebraic Methods for CT 25 / 50

We can write the updating in our matrix-vector formalism as follows

where we introduced the diagonal matrix M = diag mkr i k22 .

Per Christian Hansen Algebraic Methods for CT 26 / 50

We thus obtain the following formulation:

Basic Cimmino algorithm

Per Christian Hansen Algebraic Methods for CT 27 / 50

Moreover, the largest eigenvalue of the symmetric matrix I AT M 1 A is

Hence the iterates x (k) converge to the solution x = A1 b.

where x = A1 b, = kAk2 kA1 k2 , and we have linear convergence.

When  1 then we have the approximate upper bound

kx (k) xk22 < 2 k (0) 2

Per Christian Hansen Algebraic Methods for CT 29 / 50

In tomography, A Rmn is almost always a rectangular matrix: m 6= n.

The rank r of A is the number of linearly independent rows (equal to the

The range R(A) is the linear subspace spanned by the columns of A:

Back projection. Another name for multipli- ray i

When 1 then we have the approximate upper bound