A short presentation on Newton-Raphson method

Doan Tran Nguyen Tung 5

Nguyen Quan Ba Hong 6

Students at Faculty of Math and Computer Science.

Ho Chi Minh University of Science, Vietnam
email. nguyenquanbahong@gmail.com
blog. http://hongnguyenquanba.wordpress.com

May 2, 2016

5
Student ID: 1411352
6
Student ID: 1411103
Doan Tran Nguyen Tung 7 Nguyen Quan Ba Hong 8
Newton-Raphson
(Students method
at Faculty of Math and Computer
May 2, 2016
Science. 1Ho
/ 24
Chi
 Outline

1 Introduction

2 The Newton-Raphson method

Geometrical view point
Analytical view point

3 Selected examples

4 The Newton-Raphson method has fallen!

The Newton-Raphson method can go bad

5 Improvements

6 N-R method for several variables

7 Extension to systems of equations

Doan Tran Nguyen Tung 9 Nguyen Quan Ba Hong 10

Newton-Raphson
(Studentsmethod
at Faculty of Math and Computer
May 2, 2016
Science.2 Ho
/ 24Ch
 Historical notes

1 This algorithm was discovered by Sir Issac Newton, who

formulated the result in 1669. Later improved by Joseph Raphson
in 1690, the algorithm is presently known as the Newton-Raphson
method, or more commonly Newton’s method.
2 A method for finding the roots of an arbitrary function that uses
the derivative was first circulated by Isaac Newton in 1669. John
Wallis published Newton’s method in 1685, and in 1690 Joseph
Raphson (1648–1715) published an improved version, essentially
the form in which we use it today.
3 Newton’s work was done in 1669 but published much later.
Numerical methods related to the Newton Method were used by
alKāshī, Viète, Briggs, and Oughtred, all many years before
Newton.

Doan Tran Nguyen Tung 11 Nguyen Quan BaNewton-Raphson

Hong 12 (Students
method
at Faculty of Math and May
Computer
2, 2016Science.
3 /Ho
24 C
 Later studies

1 Raphson, like Newton, seems unaware of the connection between

his method and the derivative. The connection was made about 50
years later (Simpson, Euler), and the Newton Method finally
moved beyond polynomial equations. The familiar geometric
interpretation of the Newton Method may have been first used by
Mourraille (1768). Analysis of the convergence of the Newton
Method had to wait until Fourier and Cauchy in the 1820s.
2 The method was then studied and generalized by other
mathematicians like Simpson (1710-1761), Mourraille (1720-1808),
Cauchy (1789-1857), Kantorovich (1912-1986), ... The important
question of the choice of the starting point was first approached by
Mourraille in 1768 and the difficulty to make this choice is the
main drawback of the algorithm.

Doan Tran Nguyen Tung 13 Nguyen Quan BaNewton-Raphson

Hong 14 (Students
method
at Faculty of Math and May
Computer
2, 2016Science.
4 /Ho
24 C
 Why does N-R method appear?

Why does N-R method appear?

Although the bisection method will always converge on the root,
the rate of convergence is very slow.
A faster method for converging on a single root of a function is the
Newton-Raphson method.
Perhaps it is the most widely used method of all locating formulas.

Being one of iterative methods!

1 An initial estimate of the root. found by drawing a graph of
the function in the neighborhood of the root.
2 The method is based upon a knowledge of the tangent to the curve
near the root.
3 A powerful technique for solving equations numerically, based on
the simple idea of linear approximation.

Doan Tran Nguyen Tung 15 Nguyen Quan BaNewton-Raphson

Hong 16 (Students
method
at Faculty of Math and May
Computer
2, 2016Science.
5 /Ho
24 C
 Steps of N-R method

The method consists of the following steps:

1 Pick a point x0 close to a root. Find the corresponding point
(x0 , f (x0 )) on the curve.
2 Draw the tangent line to the curve at that point, and see where it
crosses the x-axis.
3 The crossing point, x1 , is your next guess. Repeat the process
starting from that point.

Doan Tran Nguyen Tung 17 Nguyen Quan BaNewton-Raphson

Hong 18 (Students
method
at Faculty of Math and May
Computer
2, 2016Science.
6 /Ho
24 C
 Geometrical view point

1 Consider the following diagram showing a function f (x) with a

simple root at x = x∗ whose value is required. Initial analysis has
indicated that the root is approximately located at x = x0 . The
aim of any numerical procedure is to provide a better estimate to
the location of the root.
2 The basic premise of the Newton-Raphson method is the
assumption that the curve in the close neighbourhood of the
simple root at x is approximately a straight line. Hence if we draw
the tangent to the curve at x0 , this tangent will intersect the
x−axis at a point closer to x∗ than is x0

Doan Tran Nguyen Tung 19 Nguyen Quan BaNewton-Raphson

Hong 20 (Students
method
at Faculty of Math and May
Computer
2, 2016Science.
7 /Ho
24 C
 Figure 1

Figure 1: Diagram for the curve

Doan Tran Nguyen Tung 21 Nguyen Quan BaNewton-Raphson

Hong 22 (Students
method
at Faculty of Math and May
Computer
2, 2016Science.
8 /Ho
24 C
 Figure 2

Figure 2: Figure 2.

Doan Tran Nguyen Tung 23 Nguyen Quan BaNewton-Raphson

Hong 24 (Students
method
at Faculty of Math and May
Computer
2, 2016Science.
9 /Ho
24 C
 Establish the formula
From the geometry of this diagram we see that
x1 = x0 − P Q
But from the right-angled triangle PQR we have
RQ
= tan θ = f 0 (x0 )
PQ
and so
RQ f (x0 )
PQ = 0 = 0
f (x0 ) f (x0 )
f (x0 )
x1 = x0 − 0
f (x0 )
If f (x) has a simple root near x0 then a closer estimate to the root is
x1 where
f (x0 )
x1 = x0 − 0
f (x0 )
This formula can be used time and time again giving rise to the
following:
Doan Tran Nguyen Tung 25 Nguyen Quan BaNewton-Raphson
Hong 26 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
10 /Ho
24 C
 The N-R iterative formula

The Newton-Raphson iterative formula

If f (x) has a simple root near xn then a closer estimate to the root is
xn+1 where
f (xn )
xn+1 = xn − 0
f (xn )
This is the Newton-Raphson iterative formula. The iteration is begun
with an initial estimate of the root, x0 , and continued to find x1 , x2 , ...
until a suitably accurate estimate of the position of the root is obtained.

Doan Tran Nguyen Tung 27 Nguyen Quan BaNewton-Raphson

Hong 28 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
11 /Ho
24 C
 Analysis view point

We suppose that f is a C 2 function on a given interval, then using

Taylor’s expansion near x
0 2


f (x + h) = f (x) + hf (x) + O h

and if we stop at the first order (linearization of the equation), we are

looking for a small h such as

f (x + h) = 0 ≈ f (x) + hf 0 (x)

giving
h = − ff0(x)
(x)
f (x) 
x+h=x− f 0 (x)

Doan Tran Nguyen Tung 29 Nguyen Quan BaNewton-Raphson

Hong 30 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
12 /Ho
24 C
 Example for N-R method
We just take one example, you should be try yourself!
Example.
f (x) = x − 2 + ln x has a root near x = 1.5. Use the Newton-Raphson
formula to obtain a better estimate.

Solution. Here x0 = 1.5, f (1.5) = −0.5 + ln 1.5 = −0.0945,

f 0 (x) = 1 + x1 , f 0 (1.5) = 1 + 1.5
1
= 35 . Hence using the formula
−0.0945
x1 = 1.5 − = 1.5567
1.6667
The Newton-Raphson formula can be used again: this time beginning
with 1.5567 as our initial estimate. This time use:
x2 = x1 − ff0(x 1) f (1.5567)
(x1 ) = 1.5567 − f 0 (1.5567)
= 1.5567 − 1.5567−2+ln
1+ 1
1.5567
= 1.5571
1.5567

This is in fact the correct value of the root to 4 d.p. 

Doan Tran Nguyen Tung 31 Nguyen Quan BaNewton-Raphson
Hong 32 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
13 /Ho
24 C
 The N-R method can go bad

The Newton-Raphson method is a GOD TOOL, isn’t it?

To answer that question, we will show some non-trivial examples.
Example 1.
Consider the function defined by

√x , if x 6= 0
(
f (x) = |x|
0, if x = 0

Easy to prove that f is continuous. The derivative of this function is

1
f 0 (x) = p , x 6= 0
2 |x|

Doan Tran Nguyen Tung 33 Nguyen Quan BaNewton-Raphson

Hong 34 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
14 /Ho
24 C
 Example (continue)
If we choose any starting point off the actual root, x1 = a 6= 0, then
√a
|a|
x2 = a − = a − 2a = −a
√1
2 |a|

If follows that 
a, if n odd
xn = 
−a, if n even

Example 2.
Take f : R → R, x 7→ x2 − x + 1 and x0 = 1. As f 0 (x) = 2x − 1,
x1 = 1 − ff0(1)
(1) = 1 − 1
1 = 0 and x 2 = 0 − f (0)
0
f (0) = 0 − 1
−1 = 1. It follows
that 
1, if n is even
xn =
0, if n is odd
Thus {xn } does not converge. 
Doan Tran Nguyen Tung 35 Nguyen Quan BaNewton-Raphson
Hong 36 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
15 /Ho
24 C
 Nothing be perfect!

Once the Newton Method catches scent of the root, it usually

hunts it down with amazing speed. But since the method is based
on local information, namely f (xn ) and f 0 (xn ), the Newton
Method’s sense of smell is deficient.
If the initial estimate is not close enough to the root, the Newton
Method may not converge, or may converge to the wrong root
The successive estimates of the Newton Method may converge to
the root too slowly, or may not converge at all.

Doan Tran Nguyen Tung 37 Nguyen Quan BaNewton-Raphson

Hong 38 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
16 /Ho
24 C
 Drawbacks

The Newton-Raphson methods has some drawbacks.

1 It cannot handle multiple roots.
2 It has slow convergence (compared with newer techniques).
3 The solution may diverge near a point of inflection.
4 The solution might oscillates new local minima or maxima.
5 With near-zero slope, the solution may diverge or reach a different
root.

Doan Tran Nguyen Tung 39 Nguyen Quan BaNewton-Raphson

Hong 40 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
17 /Ho
24 C
 Improvements

We can improve the Newton-Raphson method to (some) following

other iterations.
1 Cubic iteration.
2 Householder’s iteration.
3 High order iteration: Householder’s method.
More details for these iterations, please read attachment file of my
team.

Doan Tran Nguyen Tung 41 Nguyen Quan BaNewton-Raphson

Hong 42 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
18 /Ho
24 C
 N-R method for several variables

Target
Newton’s method may also be used to find a root of a system of two or
more non linear equations

f (x, y) = 0
g (x, y) = 0

where f and g are C 2 functions on a given domain.

Using Taylor’s expansion of the two functions near (x, y) we find

f (x + h, y + k) = f (x, y) + h ∂f ∂f 2 + k2


∂x + k ∂y + O h
∂g ∂g
g (x + h, y + k) = g (x, y) + h ∂x + k ∂y + O h2 + k 2

and if we keep only the first order terms, we are looking for a couple
(h, k) such as
Doan Tran Nguyen Tung 43 Nguyen Quan BaNewton-Raphson
Hong 44 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
19 /Ho
24 C
 f (x + h, y + k) = 0 ≈ f (x, y) + h ∂f
∂x + k ∂f
∂y
∂g ∂g
g (x + h, y + k) = 0 ≈ g (x, y) + h ∂x + k ∂y
hence it’s equivalent to the linear system
! 
∂f ∂f  
∂x ∂y h f (x, y)
∂g ∂g =−
∂x ∂y
k g (x, y)

equivalent to    
h f (x, y)
J (x, y) =−
k g (x, y)
This suggest to define the new process
     
xn+1 xn f (x ,
n ny )
= − J −1 (xn , yn )
yn+1 yn g (xn , yn )
under certain conditions (which are not so easy to check and this is
again the main disadvantage of the method), it’s possible to show that
this process converges to a root of the system. The convergence
remains quadratic.
Doan Tran Nguyen Tung 45 Nguyen Quan BaNewton-Raphson
Hong 46 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
20 /Ho
24 C
 Example. N-R for system of equations

Example
We are looking for a root near (x0 = −0.6, y0 = 0.6) of the following
system
f (x, y) = x3 − 3xy 2 − 1


g (x, y) = 3x2 y − y 3

here the Jacobian and its inverse become

 2
xn − yn2 −2xn yn

J (xn , yn ) = 3
2xn yn  x2n − yn2
2 − y2

x 2x y
n n
J −1 (xn , yn ) = 3(x2 +y
1 n n
n
2 )2
n −2xn yn xn − yn2
2

Doan Tran Nguyen Tung 47 Nguyen Quan BaNewton-Raphson

Hong 48 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
21 /Ho
24 C
 Example (continue
and the process gives

x1 = −0.40000000000000000000, y1 = 0.86296296296296296296
x2 = −0.50478978186242263605, y2 = 0.85646430512069295697
x3 = −0.49988539803643124722, y3 = 0.86603764032215486664
x4 = −0.50000000406150565266, y4 = 0.86602539113638168322
x5 = −0.49999999999999983928, y5 = 0.86602540378443871965
...

Depending on your initial guess Newton’s process may converge to one

of the three roots of the system:
√ ! √ !
1 3 1 3
− , , − ,− , (1, 0)
2 2 2 2

and for some values of (x0 , y0 ) the convergence of the process may be
tricky!
Doan Tran Nguyen Tung 49 Nguyen Quan BaNewton-Raphson
Hong 50 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
22 /Ho
24 C
 Final generalization of this text

For further studying and for references only.

Extension to systems of equations

 f1 (x1 , . . . , xn ) = 0
...
fm (x1 , . . . , xn ) = 0


or
f (x) = 0
The Newton-Raphson method becomes

xn+1 = xn − J −1 (xn ) f (xn ) , n = 0, 1, . . .

Doan Tran Nguyen Tung 51 Nguyen Quan BaNewton-Raphson

Hong 52 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
23 /Ho
24 C
 References
Adi Ben-Israel, A Newton-Raphson Method for the Solution of
Systems of Equations, Technion-Israel Institute of Technology and
Northwestern University, Journal of Mathematical Analysis and
Applications 15, 243-252 (1966).
Helm, Workbook Level 1, The Newton-Raphson Method, March 18,
2004.
Aaron Burton, Newton’s method and fractals.
David M. Bressoud, Newton-Raphson Method, Appendix to A
‘Radical Approach to Real analysis 2nd edition’, June 20, 2006.
Pascal Sebah, Xavier Gourdon, Newton’s method and high order
iterations, October 3, 2001.
Dr. Ibrahim A. Assakkaf, Numerical methods for engineers, Spring
2001.

Doan Tran Nguyen Tung 53 Nguyen Quan BaNewton-Raphson

Hong 54 (Students
method
at Faculty of Math and
May
Computer
2, 2016 Science.
24 /Ho
24 C