Numerical Methods: Fixed Point Iteration http://wwwf.imperial.ac.uk/metric/metric_public/numerical_methods/iter...

To print higher-resolution math symbols, click the

Hi-Res Fonts for Printing button on the jsMath control panel.

Figure 1: The graphs of y = x (black) and y = cos x (blue) intersect

Equations don't have to become very complicated before symbolic solution methods give out.
Consider for example the equation

x = cos x

It quite clearly has at least one solution between 0 and 2; the graphs of y = x and y = cos x
intersect. However, no algebraic methods exist for solving this equation.

Figure: Spiral towards the root for −1 g( ) 0, and zigzag towards the root for

1 of 4 5/16/2018, 9:36 PM
Numerical Methods: Fixed Point Iteration http://wwwf.imperial.ac.uk/metric/metric_public/numerical_methods/iter...

0 g( 1

Figure 3: Spiral away from the root for g ( ) −1, and zigzag away from the root for
g( ) 1

But observe the following. Let's suppose we start with an initial estimate of 0 75 as a solution
of our equation. Now:

cos(0 75) = 0 731689

cos(0 731689) = 0 744047
cos(0 744047) = 0 735734
cos(0 735734) = 0 741339
cos(0 741339) = 0 737565
cos(0 737565) = 0 740108
cos(0 740108) = 0 738396
cos(0 737565) = 0 740108
cos(0 740108) = 0 738396
cos(0 738396) = 0 739549
cos(0 739549) = 0 738772
cos(0 738772) = 0 739296

and it's clear that our solution is approximately 0 739. However, this approach (known as fixed
point iteration) doesn't always work. Consider, for example, the equation

x = 1 8 cos x

If we start with x = 1 0 as a first approximation to the solution, this is what happens:

2 of 4 5/16/2018, 9:36 PM
Numerical Methods: Fixed Point Iteration http://wwwf.imperial.ac.uk/metric/metric_public/numerical_methods/iter...

1 8 cos(1 0) = 0 972544
1 8 cos(0 972544) = 1 01376
1 8 cos(1 01376) = 0 951614
1 8 cos(0 951614) = 1 04467
1 8 cos(1 04467) = 0 903944
It's clear that these numbers are getting further away from any root.

So, when does fixed point iteration work and when does it fail? We can get some insight into
that by looking at Taylor series.

Let be a root of the equation

x = g (x)
Now, by Taylor's theorem,

g(x) = g ( ) + (x − )g ( ) +

But g( ) = , so we have

g(x) − g ( ) (x − )

But we're iterating g : that is, evaluating it repeatedly. It follows that if our nth estimate is xn then

xn+1 = g (xn )
So from the above, we have that

(xn+1 − ) g ( ) (xn − )

In other words, the distance between our estimate and the root gets multiplied by g ( )
(approximately) with each iteration. So the iteration converges if g ( ) 1, and diverges if
g( ) 1 (the rare case g ( ) = 1 can correspond either to very slow convergence or to
very slow divergence). We can illustrate this diagrammatically, using the following clever idea.
We plot, on the same axes, y = x and y = g (x), then do the following:

1. If your initial estimate is x0 , start on y = x at the point (x0 x0 ), and set i = 0.

2. Move vertically to the curve y = g (x): this will take you to the point (xi xi+1 ).

3. Move horizontally to the straight line y = x; this will take you to the point (xi+1 xi+1 ).

4. Set i = i + 1 and go back to (2).

3 of 4 5/16/2018, 9:36 PM
Numerical Methods: Fixed Point Iteration http://wwwf.imperial.ac.uk/metric/metric_public/numerical_methods/iter...

The resulting line is either a spiral (if g ( ) 0) or a "staircase'' (if g ( ) 0). It can easily be
seen from the diagrams that the spiral is inwards provided −1 g ( ) 0 and that the
zigzag is towards the root if 0 g ( ) 1. It can also be seen that the spiral is outwards
provided g ( ) −1 and that the zigzag is away from the root if g ( ) 1. Given some
particular equation, there are in general several ways to set it up as a fixed point iteration.
Consider, for example, the equation

x2 = 5
(which can of course be solved symbolically---but forget that for a moment). This can be
rearranged to give

x+5
x=
x+1
suggesting the iteration

xi + 5
xi+1 =
xi + 1
Alternatively, it can be rearranged to give

3x2 − 5
x=
2x
suggesting the iteration

3xi 2 − 5
xi+1 =
2xi
The first of these will work, as

g ( ) = g ( 5)
4
=−
(1 + 5)2
= −0 382
The second of these will fail, as

g ( ) = g ( 5)
3 ( 5)2 + 5
=
2 ( 5)2
=2

4 of 4 5/16/2018, 9:36 PM