Day 4

LECTURE FOUR 054374 NUMERICAL METHODS
Process Analysis
using Numerical Methods
LECTURE FOUR
Solution of a Single Non-linear Equation
1 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR
Lecture Four: Nonlinear Equations

Methods for the solution of a nonlinear equation are at
the heart of many numerical Part One: Basic Building Blocks
methods: from the Solution Ax=b Finite Difference Approximations
solution of M & E
balances, to the
optimization of Interpolation Solution f(x)=0 Line Integrals
chemical processes.
Furthermore, the min/max f(x)
need for numerical

solution of nonlinear Nonlinear
Solution Solution
equations also arises

Regression
of ODE's of BVP's
from the formulation Linear Solution
of other numerical Regression of IVPDE's
methods. Part Two: Applications
1 Daniel R. Lewin, Technion

Lecture Four: What is covered

Introduction to methods for determining roots of the
equation f(x) = 0, involving two steps:
Bracketing, to determine upper and lower bounds to
the region in which the root lies.
Selection and implementation of an appropriate
numerical method to determine the root to adequate
accuracy. Several methods are described in this
chapter, with emphasis on their strengths and
weaknesses:
(a) Bisection (d) Secant method
(b) Successive substitution (e) Reguli Falsi method
(c) Newtons method (f) Deflation
4.0 Motivation (1)

E.g., Compute the specific volume of NH3, , at 10 atm
and 250 K, by solving the van der Waals e-o-s :
a RT a
P + 2 ( b) = RT P = = 10
b 2
RT a
i.e., 10 = 0
b 2
Data:
R = 82 cm3/atm-gmol-K,
a = 4.19106 atm-cm3/gmol
b = 37.2 cm3/gmol.

4.0 Motivation (2)

Accuracy: In solving f(x) = 0, two measures of
accuracy are appropriate:
The error in the nonlinear equation, f(x ) , should be
less than a specified amount. The requirement for
m significant figures therefore dictates that:
f(x ) < 10 m
The requirement that the value of the independent
variable be expressed to m significant figures is
approximately equivalent to the requirement that
two successive iterations change the estimate by
less than 10-m, or
x (k ) x (k 1 ) < 10 m
Lecture Four: Objectives

Given a single nonlinear equation and a prespecified
level of required accuracy, the student should be able
to:
Determine the total number of roots and bounds on

their locations using efficient bracketing
Select and implement the most reliable method, or

combination of methods, to achieve the required
accuracy efficiently and reliably, given the
functional form of the nonlinear equation.

4.1 Bracketing (1)

Root of f(x) said to be bracketed in the interval [a, b] if
f(a) and f(b) have opposite signs: if f(x) continuous, then
at least one root must lie in the interval.
Furthermore, f(x) may have more than one root in

search interval: Need to define distinct bounds (ak, bk)
on each root, k:
Linear, accelerated and adaptive bracketing
4.1 Bracketing (2)

Example 4.1: Bracket the 2 meaningful roots for v-d-W
e-o-s for NH3 at 10 atm, 40 < < 2000 cm3/gmol
Liquid Gas nits
a b ba a b ba
20 40 60 20 1860 1880 20 98
50 40 90 50 1840 1890 50 40
Linear
100 40 140 100 1840 1940 100 20
200 bracketing failed 1840 2040 200 10
Liquid Gas nits
a b ba a b ba
20 40 60 20 1860 2140 280 14
Accelerated50 40 90 50 1840 2290 450 9
100 40 140 100 1540 2140 600 6
200 bracketing failed 1240 2040 800 4

4.2 Bisection (1)

x1 = a
x2 = b
x + x2
1 x= 1
2
if (f(x1 ) f(x ) > 0 )
x1 = x
else
x2 = x
end
( ) (
if f(x ) > 10 m or x2 x1 > 10 m go to 1 )
4.2 Bisection (2)

Convergence:The estimation error associated with the
initial bracketing of the solution is (b - a)/2. Since each
bisection iteration halves estimation error, nits,
depends on the initial bracketing range and the
required accuracy: 10 m
log
nits 10 m b a (4.2)
0.5 nits
ba log(0.5 )
Bisection method:
Main advantage: reliability - solution to a bounded
problem with the prescribed accuracy guaranteed in a
finite number of steps
Main disadvantage: linear convergence rate

4.2 Bisection (3)

Example 4.2: Bisection method gives specific volumes
of liquid and gaseous NH3 at T = 250 K and P = 10 atm
to 4 sig figs as 48.77 and 1867.3 cm3/gmol.
Following accelerated bracketing with an initial step
size of 100 cm3/gmol, these results are achieved in 25
and 22 iterations, respectively.
It is noted that for 4 sig. figs., Eq.(4.2) gives:
liquid: gas:
10 4 10 4
log log
nits 100 = 19.9 nits 600 = 22.5
log(0.5 ) log(0.5)
Explain these results.
4.3 Successive Substitution (1)

Equation to be solved: f(x) = 0 (4.3)
is recast into the form: x = g(x) (4.4)
Eq.(4.4) suggests the recursion formula:
( )
x (k +1 ) = g x (k ) (4.5)
Method:
Make initial guess, x(0)
Apply Eq.(4.5) repeatedly until |x(k+1)- x(k)| <
Convergence of (4.5) implies satisfaction of (4.3)


Convergence properties: Let the value of x that
satisfies Eq.(4.3) be x*. Then, x * = g(x *) (4.6)
Linearizing Eq.(4.5) in the vicinity of x*:
x (k +1 ) x * =
dg(x )
dx x *
(
x (k ) x * )
Thus, (k +1 ) = dg(x ) (k ) (4.7)
dx x *
where (k) = x(k) x* is the numerical error in the kth
iteration. Clearly, for convergence the numerical
error must decrease in each iteration, thus:
(k +1 ) dg(x ) (4.8)
= <1
(k ) dx x*

Furthermore, the rate of convergence increases as the
magnitude of dg(x ) dx decreases. For dg(x ) dx = 0.5,
the local convergence rate in the vicinity of the
solution is equivalent to that of bisection.
Thus, to be superior to bisection, one must ensure that

the function g(x) is derived such that
dg(x )
< 0.5
dx x*


Convergence Properties depend on |dg/dx|
dg(x ) dg(x ) dg(x )

0< <1 0> > 1 >1
dx dx dx

Example 4.3: Find the liquid and gaseous specific
volumes of NH3 using successive substitution.
RT a
Solution: The nonlinear equation: f( ) = 2 P
b
is arranged for successive
substitution:
1 RT a
= g( ) =
p b
Which of the two roots

will be found using this
g(x) ?


dg
Right! - the gas volume can be solved since 0 < << 1
dv
Indeed, a solution is achieved to 4 sig. figs. in only 5
iterations starting from v = 1540 cm3/gmol. The liquid
volume cannot be found with this recursion formula!
k x(k) g(x(k))
0 1540.0 1826.7
1 1826.7 1863.4
2 1863.4 1866.9
3 1866.9 1867.2
4 1867.2 1867.3
5 1867.3 1867.3

To converge, to liquid volume, use a different formula
for g(x), e.g. 2 (RT P(V b))
= g( ) = +b
a
Solution starting at v = 40 cm3/gmol:
k x(k) g(x(k))
0 40.00 45.02
1 45.02 47.08
2 47.08 47.99
7 48.74 48.76
8 48.76 48.77
9 48.77 48.77

4.3 Newtons Method (1)

An iterative algorithm for the solution of f(x) = 0
Given an estimate x(k) in the kth iteration, a Taylor
expansion for f(x) in the vicinity of the estimate,
truncated beyond the linear term, gives:
( ) (
f(x ) = f x (k ) + x x (k ) f x (k ))( )
(4.11)
Extrapolating Eq.(4.11) to f(x) = 0, gives an updated

estimate for x, x(k+1):
x (k +1 ) = x (k )
( )
f x (k ) (4.12)
( )
f x (k )
Newtons Method

Convergence: A sufficient, but not necessary, criterion
is that f(x) and f(x) and do not change sign in the
( )( )
interval [x(0), x*] and that f x (0 ) f x (0 ) > 0 . It is noted
that these conditions are not satisfied for an s-shaped
function and explains why for the method fails here.
Newtons Method
Diverging


Rate of Convergence:Main advantage of Newton's
method -quadratic convergence rate.
If the analytical root of f(x) is x*, then kth estimate,
x(k), is related to the numerical error: x(k) = x* + (k).
Substituting into the Newton step of Eq.(4.12):
(k +1 ) = (k )
f x (k ) ( )
(4.13)
f x (k ) ( )
Taylor series expansions for f x (k ) and f x (k ) : ( ) ( )
( ) ( )
f x (k ) = f x * + (k ) = f(x *) + (k )f(x *) + 21 (k ) f(x *) +

f(x (k ) ) = f(x * + (k ) ) = f(x *) + (k )f(x *) +

2
(k ) f(x *) +
1

2


Since f(x*) = 0 and (k) is presumed small:
( )
f x (k ) (k )f(x *)1 + 21 (k )

f(x *)
f (x * ) f x

( )
(k ) f(x *)1 + (k ) f(x *)

f(x *)
Substituting into Eq.(4.13):
1 + 21 (k ) f(x *) f(x *)
(k +1 ) (k ) = (k )
1 + (k ) f(x *) f(x *)
Using expansion (1 + )1 = 1 + 2 3 + -true for << 1
2
f(x *) ( ) f(x *) + (k ) f(x *)
(k +1) (k ) = (k ) 1 + 21 (k ) 1 k
f(x *) f(x *) f(x *)

f(x *)
(k ) 1 21 (k )
f(x *)
(k +1) 1 (k ) 2 f(x *) ( ) 2
= O k Quadratic
Thus: =

2
f(x *) convergence



Equivalence with Successive Substitution: Recursion
formula that is easy to generate:
( ) ( )
x (k +1 ) = g x (k ) = x (k ) + f x (k ) m (4.17)
Stability of Eq.(4.17) is guaranteed if 1 <

( )
dg x (k)
<1
Hence, 1 < 1 + f x ( )
(k ) m < 1
dx
2
<m<0
( )
f x (k )
Furthermore, it can be seen that selecting the value of
m to be the mid-range value makes, dg(x ) dx = 0 and will
lead to the fastest possible convergence. Then:
( )
x (k +1 ) = g x (k ) = x (k )
( )
f x (k )
( )
f x (k )

Example 4.4: Find the liquid and gaseous specific
volumes of ammonia using Newtons method.
RT a
Solution Equation to be solved is: f( ) = P
b 2
df RT 2a
Derivative w.r.t. v is: ( ) = + 3
d 2
( b)
One Newton step is: (k +1) = (k ) f
(k ) ( )
(4.19)
df (k ) d ( )
Applying Eq.(4.19) for liquid, from lower limit obtained
by accelerated bracketing (40 cm3/gmol), yields a
solution to 4 sig. figs. in 7 iterations.


The solution for the gas specific volume, starting from
an initial guess of 1540 cm3/gmol converges to the
solution to 4 sig. figs. in only 3 iterations!
k x(k) f(x(k))
0 1540.0 1.8745
1 1816.4 0.2523
2 1866.0 5.990810-3
3 1867.3 3.545010-6
4.4 Secant Method (1)

Variant of Newtons method:
x (k +1 ) = x (k )
( )
f x (k )
( ) (k ) (k 1)
f x (k )
f(x ) f(x )
Approximate f(x(k)) by: f(x (k ) ) (k) (k 1 )
x x
Hence:. x (k +1 ) = x (k ) f x ( )(
(k ) x (k ) x (k 1) ) (4.21)
( ) (
f x (k ) f x (k 1) )
While the Secant method does not rely on f(x), two
intializing values, at x = x(0) and x = x(1) must be
supplied. Since these values will in general be somewhat
arbitrary, the initial estimate for the gradient using
could be rather inaccurate.

4.4 Secant Method (2)

Example 4.5: Find the liquid specific volume of ammonia
using the Secant method.
k x(k) f(x(k))
0 40.000 4.6927103
Slow start-up 1 41.000 2.8922103
because of 2 42.606 1.4737103
poor f(x) 3 44.275 7.5002102
estimate 4 46.005 3.3855102
5 47.428 1.3162102
6 48.333 3.7773101
Newton-like 7 48.697 6.1607
quadratic local 8 48.768 3.582610-1
converge as f(x) 9 48.773 3.665710-3
estimate improves 10 48.773 2.209910-6
4.5 Reguli Falsi Method (1)

Both the Newton and Secant methods provide
quadratic convergence in the vicinity of the solution,
but are potentially unstable, depending on the shape of
f(x). Like the Secant method, the Regula Falsi method
does not require computation of the function gradient.
In a single step of the Regula Falsi method, the
estimate of the root, x, is determined by linear
interpolation:
( ) (( ) ( ) )
x = x (R. ) f x (R )
x (R ) x (L )
f x (R ) f x (L )
where it is assumed that , the function values at each
end of the bound on the root, x(R) and x(L) have opposite
signs.


The Regula Falsi algorithm is similar to bisection:
x (L ) = a
x (R ) = b
( )f(x(x(R ) ) xf(x(L)) )
(R ) (L )
1 x = x (R.) f x (R )
if (f(x (R ) ) f(x ) > 0 )

x (R.) = x
else
x (L . ) = x
end
( ) (
if f(x ) > 10 m or x x (R. ) > 10 m go to 1)

Reguli falsi guarantees convergence, but can be slow.
This method performs badly with highly convex or

concave functions.


Example 4.6: Find the liquid specific volume of
ammonia using the regula falsi method .
Solution: Starting from brackets: [40,140], solution
to 4 sig. figs. obtained in 177 iterations!!!
k x(L) f(x(L)) x(R) f(x(R)) x f(x)
0 40.00 4693 140.00 24.36 139.48 24.94
1 40.00 4693 139.48 24.94 138.96 25.54
2 40.00 4693 138.96 25.54 138.42 26.15
3 40.00 4693 138.42 26.15 137.88 26.79
176 40.00 4693 48.773 1.2710-4 48.773 1.0810-4

177 40.00 4693 48.773 1.0810-4 48.773 9.1810-5
4.6 Roots of Polynomials (1)

An important class of nonlinear equations are
polynomial equations:
P(x ) = an xn + an 1xn 1 + + a1x + a0 = 0 (4.23)
If x = x* is a root of Eq.(4.23):
P(x ) = (x x *)(bn 1xn 1 + bn 2xn 2 + + b1x + b0 ) = 0
i.e. P( x ) = bn 1xn + (bn 2 bn 1x *)xn 1 + (bn 3 bn 2x *)xn 2 +
+ (b0 b1x *)x b0 x * = 0
Equating coefficients: an = bn 1
an 1 = bn 2 bn 1x *
an 2 = bn 3 bn 2x *
a1 = b0 b1 x *
a0 = b0 x *


Solving for bi: bn 1 = an
bn 2 = an 1 + bn 1x *
bn 3 = an 2 + bn 2x *
(4.26)
b1 = a2 + b2x *
b0 = a1 + b1x *
The polynomial is then reduced by one degree, or
deflated: n = n 1, and the newly computed
coefficients are renamed: ai = bi , i = 1,2, , n. This
procedure is repeated until the deflated polynomial is
quadratic, whereupon the quadratic solving formula is
used to determine the last two roots, which may be
complex. This procedure is called Synthetic Division.

Solving for bi: bn 1 = an
bn 2 = an 1 + bn 1x *
bn 3 = an 2 + bn 2x *
(4.26)
b1 = a2 + b2x *
b0 = a1 + b1x *
The polynomial is then reduced by one degree, or
deflated: n = n 1, and the newly computed
coefficients are renamed: ai = bi , i = 1,2, , n. This
procedure is repeated until the deflated polynomial is
quadratic, whereupon the quadratic solving formula is
used to determine the last two roots, which may be
complex. This procedure is called Synthetic Division.


Synthetic division is used in conjunction with a root-
solving method such as Newtons method, to recursively
solve for all of the roots of P(x).
The existence of more than one pair of complex roots
is handled by carrying out computations using complex
algebra.
Note that this procedure requires an accurate
estimate of the first root. Subsequently, the first
guess for the next root being determined in the
deflated polynomial is commonly taken as the value of
the previous root found.
Summary
Having completing this chapter, the reader should now
be able to:
Determine the total number of roots and bounds on
their locations using efficient bracketing. It has
been shown that accelerated or adaptive methods
will often provide for more efficient bracketing of
the solution.
Select and implement the most reliable method, or
combination of methods, to achieve the required
accuracy efficiently and reliably, given the
functional form of the nonlinear equation. The
following table summarizes the strengths and
weaknesses of each of the methods covered in this
chapter.

Method Advantages Disadvantages

Bisection Easy to implement and robust Linear convergence
Guarantees a reduction in the
estimation error by 50% in each
iteration
Successive Can be very efficient, depending on May diverge
substitution the recursion formula derived Recursion equation formulation that
guarantees conversion not obvious
Performance depends on recursion
equation
Newton Quadratic local convergence May diverge in cases where the f(x) is
very small or very large
f(x) required
Secant Almost quadratic convergence May diverge in cases where function
Easy to implement gradient is very small or very large
Sensitive to initial gradient estimates
Regula falsi Easy to implement Rate of convergence may be very slow,
Robust guarantees convergence especially for highly convex or concave
functions.
Practical Recommendations
Consider the following recommendations when
attempting to derive a numerical approximation for the
solution of a nonlinear algebraic equation:
Try to graph f(x) to observe its general form, which will
suggest the most appropriate method for its solution.
Always try to bracket the root, to provide reliable limits
to a more efficient method to lock into the solution.
Accelerated or adaptive bracketing is recommended
When an arbitrary nonlinear function, f(x), a winning
combination is a hybrid algorithm that combines bisection
for robustness with Newtons method, for speed. This
would guarantee quadratic convergence at best, but
revert to guaranteed linear convergence to the root, at
worst.

Day 4

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Day 4

Uploaded by

Copyright:

Available Formats

LECTURE FOUR 054374 NUMERICAL METHODS

Solution of a Single Non-linear Equation

1 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

Lecture Four: Nonlinear Equations

need for numerical

equations also arises

from the formulation Linear Solution

of other numerical Regression of IVPDE's

methods. Part Two: Applications

2 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

1 Daniel R. Lewin, Technion

Lecture Four: What is covered

3 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

4.0 Motivation (1)

2 Daniel R. Lewin, Technion

4.0 Motivation (2)

5 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

Lecture Four: Objectives

Determine the total number of roots and bounds on

Select and implement the most reliable method, or

6 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

3 Daniel R. Lewin, Technion

4.1 Bracketing (1)

Furthermore, f(x) may have more than one root in

4.1 Bracketing (2)

8 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

4 Daniel R. Lewin, Technion

4.2 Bisection (1)

4.2 Bisection (2)

10 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

5 Daniel R. Lewin, Technion

4.2 Bisection (3)

11 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

4.3 Successive Substitution (1)

12 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

6 Daniel R. Lewin, Technion

4.3 Successive Substitution (2)

13 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

4.3 Successive Substitution (3)

Thus, to be superior to bisection, one must ensure that

14 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

7 Daniel R. Lewin, Technion

4.3 Successive Substitution (4)

dg(x ) dg(x ) dg(x )

15 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

4.3 Successive Substitution (5)

Which of the two roots

16 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

8 Daniel R. Lewin, Technion

4.3 Successive Substitution (6)

17 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

4.3 Successive Substitution (7)

9 Daniel R. Lewin, Technion

4.3 Newtons Method (1)

Extrapolating Eq.(4.11) to f(x) = 0, gives an updated

19 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

4.3 Newtons Method (2)

20 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

10 Daniel R. Lewin, Technion

4.3 Newtons Method (3)

f(x (k ) ) = f(x * + (k ) ) = f(x *) + (k )f(x *) +

21 NUMERICAL METHODS - (c) Daniel R. Lewin LECTURE FOUR

4.3 Newtons Method (4)

f(x (k ) ) = f(x * + (k ) ) = f(x ) + (k )f(x ) +