Dan Um Jay A

Numerical Analysis
Course No. : AAOC C341

Dr. P. Dhanumjaya
Numerical Analysis p.1/63
Instructor-Incharge
Dr. P. Dhanumjaya
Instructor
Dr. Anil Kumar
Text Book
An Introduction to Numerical Analysis
Author: Devi Prasad
Publisher: Narosa Publishing House, 3rd edition
Reference Books
An Introduction to Numerical Analysis,
Kendall E. Atkinson, John Wiley & Sons
(2004).
Elementary Numerical Analysis, Samuel D.
Conte, Carl de Boor, McGraw-Hill, 3rd edition
(1981).
Numerical Analysis, R. L. Burden, J. D.
Faires, 7th edition, Thomson Learning (2001).
Evaluation Scheme
Test-1: Weightage 25%,
Date & Time: 20-09-2008 @ 2-3PM
Test-2: Weightage 25%,
Date & Time: 01-11-2008 @ 2-3PM
Tut-Test/Quiz/Assignments: Weightage 10%,
Date & Time:
Comprehensive Exam: Weightage 40%,
Date & Time: 06-12-2008 @ 2-5PM
Topics to be covered
Roots of equations.
Solve
f(x) = 0, for x.
. x
f(x)
Root
Linear algebraic equations.
Given the as and the cs, solve
a
11
x
1
+ a
12
x
2
= c
1
,
a
21
x
1
+ a
22
x
2
= c
2
,
for the xs.
x
x
1
2
.
Solution
Curve tting (Interpolation).
x
x ( ) f
.
.
.
.
.
Interpolation
Integration.
I =
b
a
f(x) dx.
Find the area under the curve.
x
x ( ) f
I
Ordinary differential equations.
Given
dy
dt
= f(t, y).
Solve for y as a function of t.
t
y
.
.
t
t
t
slope = f(t
i
,y
i
)
i+1
i
Why you should study Numerical Methods
Numerical methods are extremely powerful
problem solving tools. They are capable of
handling large systems of equations,
nonlinearities and complicated geometries
that are often arises in engineering braches.
They are ingeneral impossible to solve
analytically.
During your career, you may often have
occasion to use commercially available
packages that involve numerical methods.
The intelligent use of these packages is often
predicted on knowledge of the basic theory
underlying the methods.
Many problems cannot be approached using
commercially available packages. If you are
conversant with numerical methods and
computer programming, you can design your
own programs to solve the problems.
Finite Digit
Arithmetic and Errors
We consider how numbers are represented in
computers.
Most computers have an integer mode and a
oating-point mode for representing numbers.
The integer mode is used only to represent
integers.
The oating-point form is used to represent
real numbers.
Note 1
Scientic calculations are usually carried out
in oating-point arithmetic.
An n-digit oating-point number in base has
the form
x = (.d
1
d
2
d
3
d
n
)

e
,
where (.d
1
d
2
d
3
d
n
)
is a -fraction called the

mantissa, and e is an integer called the exponent.
A oating-point number is said to be normalized
if d
1
= 0 or else d
1
= d
2
= d
3
= = 0.
Note 2
Most digital computers use the base 2
(binary) number system or base 8 (octal) or
base 16 (hexadecimal).
Pocket calculators use base 10 (decimal).
Example 1
The conversion of a base 2 number to decimal
(11011.01)
2
= 1 2
4
+ 1 2
3
+ 0 2
2
+ 1 2
1
+ 1 2
0
+ 0 2
1
+ 1 2
2
= 27.25
Floating-Point representations on various
computers
Computer n M = m
..................................
IBM 7094 2 27 27
HP 67 10 10 99
Intel 8087 2 24 127
..................................
Note 3
The exponent e is limited to a range
m < e < M
for certain integers m and M.
Chopping and
Rounding
Most real numbers x cannot be represented
exactly by the oating-point representation.
We approximate by a nearby number
representable in the machine.
Given an arbitrary real number x, we let fl(x)
denote its oating-point representation.
There are two ways of producing fl(x) from x,
chopping and rounding.
Let a real number x be written in the form
x = (0.d
1
d
2
d
3
d
n
d
n+1
)

e
with d
1
= 0.
The chopped machine representation of x is
given by
fl(x) = (0.d
1
d
2
d
3
d
n
)

e
The rounded representation of x is given by
fl(x) = (0.d
1
d
2
d
3
d
n
)

e
,
whenever 0 d
n+1
<

2
and
fl(x) =
(0.d
1
d
2
d
n
)
+ (0.00 01)

e
,
whenever

2
d
n+1
< 1.
Note 4
Note that
(0.d
1
d
2
d
3
d
n
)
=
d
1
1
+
d
2
2
+ +
d
n
n
.
Denition of Errors
The absolute error is dened as the difference
between the true value x and the approximate
value fl(x).
Thus,
|e
x
| = |x fl(x)|.
Absolute error may not reect the reality.
One picks up 995 correct answers from 1000
problems certainly is better than the one that
picks up 95 correct answers from 100 problems,
although both of the errors are 5.
A more realistic error measurement is the relative
error which is dened as
relative error =
absolute error
true value
.
For example, if x = 0, then
relative error in x =
x fl(x)
x
.
Obviously, for different x, the error x fl(x) and
the relative error are different.
How do we then characterize the round-off
errors?.
We seek the upper bounds.
In place of relative error, we often use the
concept of signicant digits.
Let x denotes the true solution and x
denotes
an approximate solution.
We say x
has m signicant digits with respect to

x if the error
x x
has magnitude less than or equal to 5 in the

(m + 1)th digit of x counting to the right from the
rst nonzero digit in x.
Example 2
Let x = 0.33333 and x
= 0.333, then the error

|x x
| = 0.00033.
We say that x
has three signicant digits with

respect to x.
Theorem 1
Let fl(x) be n oating-point representation of
a real number x and
x fl(x)
x
= ,
then
1.
n+1
< 0, if chopping is used
2.
|| <
1
2
n+1
, if rounding is used.
Proof
We take
x = (0.d
1
d
2
d
3
d
n
d
n+1
)

e
then fl(x) when chopping is used is
fl(x) = (0.d
1
d
2
d
3
d
n
)

e
We now consider the positive real number case.
In case of negative real number case there is no
change in the sign of .
Calculating
x fl(x) = (0.000 0d
n+1
)

e
.
Let = 1, then
0 x fl(x) < (0.000 0 )

e
=
n1
+
n2
+
e
=
n1
1
1
e
=
n+e
.
Now
0
x fl(x)
x
<

n+e
(0.d
1
d
2
d
n
d
n+1
)
e
<

n
(0.100 )
=
n+1
.
The second part is similar. Please try by your
own.
Propagation of Errors
Propagated error in an arithmetic operations
(+, , x, /) occurs due to approximate values of
numbers taken by computer with nite digits.
Let x and y are the true or exact values and x
and y
are the corresponding approximate values

respectively.
Then we have
x = x
+ , y = y
+ .
Propagated Relative
Error in Multiplication
We calculate
xy x
= xy (x )(y )
= x + y .
The relative error in multiplication is
r
xy
=
xy x
xy
=

y
+

x

xy
= r
x
+ r
y
r
x
r
y
.
For |r
x
|, |r
y
| 1 then
r
xy
r
x
+ r
y
.
Propagated relative
error in division
The relative error in division is
r
x
y
=
x
y

x
x
y
=
x
y

x
y
x
y
=
y x
xy
.
For |r
y
| 1 then
r
x
y
r
x
r
y
.
Note 5
For both multiplication and division, relative
errors do not propagate rapidly.
Propagated relative error in addition and
subtraction
The relative error in addition is
r
xy
=
(x y) (x
)
x y
=
x x
x
x
x y

y y
y
y
x y
.
This gives
r
xy
= r
x
x
x y
r
y
y
x y
.
Note 6
The relative error in addition and subtraction
r
xy
can be much larger than the relative error in x
and y.
Example 3
Let x = and x
= 3.1416. Similarly, we take

y =
22
7
and y
= 3.1429.
We have
x x
7.35 10
6
,
y y
4.29 10
5
.
The relative error in x and y respectively as
r
x
2.34 10
6
r
y
1.36 10
5
.
But the relative error in x y is
r
xy
0.028.
Note 7
The error in (x y) is quite small but the
relative error in (x y) is much larger than r
x
or r
y
alone.
When subtracting nearly equal quantities, it is
possible to have a large decrease in accuracy.
Example 4
Consider the following quadratic equation
x
2
+ 99.8x 20 = 0.
The actual roots are
x = 100, and 0.2
We now use the root nding formula for quadratic
equations
x =
99.8
(99.8)
2
+ 80
2
.
Using 3-digit arithmetic with rounding, we get
x = 100, and x = 0.1
For obtaing the correct root, we rewrite the
formula as
x =
b +
b
2
4ac
2a
=
4ac
2a(b +
b
2
4ac)
.
Example 5
Find the roots of the quadratic equation
x
2
+ 111.11x + 1.2121 = 0,
using 5-digit arithmetic with chopping.
Solution. Plz try
Generated Error
Let w denote one of the arithmetic operations ,
+, , , /, and let w be the computer version of
the same operation with rounding or chopping.
If the error occurs during the computer version of
arithmetic operation, we say the error is
generated error.
Note that x wy is the number actually computed,
we denote
x wy = fl(xwy), (rounding).
Now the relative error is
r
xwy
=
xwy x
wy
xwy
=
xwy x
wy
xwy
+
x
wy
wy
xwy
.
Taking absolute value on both sides, we get
|r
xwy
| |r
p.e
| +|r
g.e
|
n+1
.
This gives us the upper bound for the total
relative error.
Propagated Error in function evaluations
Let x
be the approximation to x correct to n

signicant decimal digits. Then
x = x
+ .
We have f(x) = f(x
+ ) and using Taylor series

expansion, we obtain
f(x
+ ) = f(x
) + f
(x
) +

2
2!
f
(),
where lies between x and x
.
r
f(x)
=
f(x) f(x
)
f(x)
=
f
(x
) +

2
2!
f
()
f(x)
.
Neglecting higher order terms in , we get
r
f(x)

f
(x
)
f(x)
.
Assuming that f(x) and f(x
) are relatively
close, we obtain
r
f(x)

x
xf
(x
)
f(x
)
. Numerical Analysis p.50/63
We rewrite the above expression as
|r
f(x)
| |r
x
| |
x
(x
)
f(x
)
|.
We denote
k = |
x
(x
)
f(x
)
|,
is called condition number.
Note 8
If k is nearly 1 then error in f(x) is called natural,
and if k is very large then relative error in
evaluation of f(x) will also be large.
Example 6
Let
f(x) =
x,
then f
(x) =
1
2
x
.
The value of k is
k = |
x
(x
)
f(x
)
| =
1
2
.
This says that taking square root is a
well-conditioned process.
Example 7
Let
f(x) =
10
1 x
2
,
then f
(x) =
20x
(1x
2
)
2
.
The value of k is
k =
2x
2
|1 x
2
|
,
and this number can be quite large for |x| near 1.
Thus, for x near 1 or 1, this function is quite
ill-conditioned.
Example 8
Let x
approximate x correct upto n signicant

decimal digits if e
10x
is evaluated for x, where
10 x 9, then what should be the relative
error?.
The relative error if f(x) is
r
f(x)
r
x
x
(x
)
f(x
)
10r
x
x
.
Then
|r
f(x)
|
1
2
10
(n2)+1
.
This shows that f(x
) approximates f(x) correct

upto (n 2) signicant decimal digits.
Errors in Summation
Many numerical methods, especially in linear
algebra, involve summations.
We now look at various aspects of summation in
oating-point arithmetic.
Consider the computation of the sum
S =
m
j=1
x
j
,
where x
1
, x
2
, , x
m
are oating-point numbers.
Dene
S
2
= fl(x
1
+ x
2
) = (x
1
+ x
2
)(1 +
2
),
.
.
.
recursively,
S
r+1
= fl(S
r
+ x
r+1
), r = 2, 3, , m1.
Then
S
r+1
= (S
r
+ x
r+1
) (1 +
r+1
) .
The quantities
2
,
3
, ,
m
satisfy the error
bounds depending on whether chopping or
rounding is used.
Expanding the few sums, we obtain
S
2
(x
1
+ x
2
) =
2
(x
1
+ x
2
),
S
3
(x
1
+ x
2
+ x
3
) (x
1
+ x
2
)
2
+ (x
1
+ x
2
+ x
3
)
3
,
S
4
(x
1
+ x
2
+ x
3
+ x
4
) (x
1
+ x
2
)
2
+ (x
1
+ x
2
+ x
3
)
3
+ (x
1
+ x
2
+ x
3
+ x
4
)
4
. Numerical Analysis p.59/63
Note that we have neglected cross-product terms
j
, since they will be of much smaller
magnitude.
By induction, we have
S
m
1
x
j
(x
1
+ x
2
)
2
+
+ (x
1
+ x
2
+ + x
m
)
m
= x
1
(
2
+
3
+ +
m
)
+ x
2
(
2
+
3
+ +
m
)
+ x
3
(
3
+ +
m
) +
+ x
m
m
.
We observe that the best strategy for addition is
to add from the smallest to the largest.
This is true if the numbers x
i
s are all of same
sign.
Therefore, adding from smallest to largest, we
minimize the effect of chopping or rounding
errors.
Example 9
Evaluate the polynomial
f(x) = 1.107x
3
+ 0.3129x
2
0.0172x + 1.1075,
for x = 0.1234 in nested form, using 5-digit
arithmetic with rounding.
The nested form is
[(1.1071x + 0.3129) x 0.0172] x + 1.1075
Plz complete the remaining part.
THE END
Chapter 2
Nonlinear Equations
Dr. P. Dhanumjaya
Chapter 2Nonlinear Equations p.1/158
Mathematical
Problem
Given a function f, nd a value for x such that
f(x) = 0.
Such an x is called a zero of the function f or a
root of the equation f(x) = 0.
This problem is therefore known as the root
nding problem.
Root nding techniques are generally divided
into two categories.
Simple enclosure methods
Fixed point iteration schemes
All simple enclosure methods are based on the
Intermediate Value Theorem.
These methods essentially work by rst nding
an interval which is guaranteed to contain a root
and then systematically shrinking the size of that
interval.
Intermediate Value
Theorem
Let f be a continuous function over the closed
interval [a, b], and let k be any real number that
lies between the values f(a) and f(b). Then
there exists a real number c with a < c < b such
that f(c) = k.
Proof. For a proof please see any text book on
advanced calculus or real analysis.
Note
Basically, the Intermediate Value Theorem
provides a means for identifying intervals
which enclose the real zeros of continuous
functions.
All that is needed is to nd an iterval such that
the values of the function at the end points of
that interval are of opposite sign.
As long as one end point value is positive and
the other negative, zero is somewhere
between the values, and at least one zero of
the function is guaranteed to to exist on that
interval.
Bisection Method
Suppose, we have used the Intermediate Value
Theorem to locate an interval that contains a
zero of a continuous function.
What do we do next?
Our objective will be to systematically shrink the
size of the interval that contains the root.
Perhaps the simplest and most natural way to
accomplish a reduction in interval size is to cut
the interval in half.
Once this has been done, we determine which
half contains a root, by once again using the
Intermediate Value Theorem, and then repeat
the process on that half.
This techinque is known as the bisection method.
Example 1
Do four iterations of bisection method to nd the
root of
x
3
+ 2x
2
3x 1.
Solution. This function has three simple real
zeros, one each on the intervals:
(3, 2), (1, 0), (1, 2).
For the rst iteration, we have (a
1
, b
1
) = (1, 2) and
we know that
f(a
1
) = 1 < 0, and f(b
1
) = 9 > 0.
The mid point of this rst interval is
p
1
=
a
1
+ b
1
2
= 1.5.
To determine whether the root is contained on
(a
1
, p
1
) = (1, 1.5) or on (p
1
, b
1
) = (1.5, 2), we
calculate
f(p
1
) = 2.375 > 0.
Since f(a
1
) and f(p
1
) are of opposite sign, the
Intermediate Value Theorem tells us the root is
between a
1
and p
1
.
For the next iteration, we therefore take
(a
2
, b
2
) = (a
1
, p
1
) = (1, 1.5).
The mid point of this new interval is
p
2
=
a
2
+ b
2
2
= 1.25.
We note that
f(p
2
) 0.328 > 0,
which is opposite sign from f(a
2
).
Hence, the Intermediate Value Theorem tells us
the root is between a
2
and p
2
, so we take
(a
3
, b
3
) = (a
2
, p
2
) = (1, 1.25).
In the third iteration, we calculate
p
3
=
a
3
+ b
3
2
= 1.125,
and
f(p
3
) = 0.420 < 0.
Here, we nd that f(a
3
) and f(p
3
) are of the
same sign, which implies that the root must lie
some where between p
3
and b
3
.
For the fourth iteration, we take
(a
4
, b
4
) = (p
3
, b
3
) = (1.125, 1.25)
We get
p
4
=
a
4
+ b
4
2
1.1875.
Convergence
Analysis
Under what circumstances will the sequence of
approximations generated by the bisection
method converge to a root of f(x) = 0?.
When the sequence does converge, what is the
speed of convergence?.
Theorem 1
Let f be continuous on the closed interval [a, b]
and suppose that f(a) f(b) < 0. The bisection
method generates a sequence of approximations
p
n
which converges to a root p (a, b) with the
property
[p
n
p[
(b a)
2
n
.
Proof. Since the quantity b a is constant and
2
n
0 as n , establishing the error bound
will be sufcient to prove convergence of the
bisection method sequence.
In a bisection method, for each n, p (a
n
, b
n
) and
p
n
is taken as the midpoint of (a
n
, b
n
).
This implies that p
n
can differ from p by no more
than half the length of (a
n
, b
n
), that is
[p
n
p[
1
2
(b
n
a
n
).
However, again by construction
b
n
a
n
=
1
2
(b
n1
a
n1
) =
1
4
(b
n2
a
n2
)
= =
1
2
n1
(b
1
a
1
).
Recalling that b
1
= b and a
1
= a, this implies
[p
n
p[
(b a)
2
n
.
Note
The condition f(a) f(b) < 0 implies differing
signs at the endpoints of the interval, which
guarantees the existence of a root, but not
uniquess.
There may be more than one root on the
interval and there is no way to know, a priori to
which root the sequence will converge, but it
will converge to one of them.
Denition
Let p
n
be a sequence that converges to a
number p. If there exists a sequence
n
that
converges to zero and a positive constant ,
independent of n, such that
[p
n
p[ [
n
[
for all sufciently large values of n, then p
n
is
said to converge to p with Rate of Convergence
O(
n
).
Note
The bisection method has rate of convergence
O
_
1
2
n
_
.
Stopping condition
The absolute error in the location of the root.
Terminate the iteration when [p
n
p[ < .
The relative error in the location of the root.
Terminate the iteration when [p
n
p[ < [p
n
[.
The test for a root.
Terminate the iteration when [f(p
n
)[ < .
Secant Method
The Secant method computes the next
approximation p
n+1
as the x-intercept of a line
that passes through two points on the graph of f.
No attempt is made to maintain an interval that
contains the root.
The equation of the line is given by
y f(p
n
) =
f(p
n
) f(p
n1
)
p
n
p
n1
(x p
n
),
The x-intercept of a line given by
p
n+1
= p
n
f(p
n
)
p
n
p
n1
f(p
n
) f(p
n1
)
.
Note
The calculation of p
n+1
requires both p
n
and p
n1
,
the secant method needs two starting values p
0
and p
1
to initiate the iteration.
Example
Do four iterations of secant method to nd the
root of
x
3
+ 2x
2
3x 1,
by taking p
0
= 2 and p
1
= 1.
Solution. The secant method produces
p
2
= p
1
f(p
1
)
p
1
p
0
f(p
1
) f(p
0
)
= 1.1
Now, with p
1
= 1 and p
2
= 1.1, the next secant
method approximation is
p
3
= p
2
f(p
2
)
p
2
p
1
f(p
2
) f(p
1
)
= 1.2217294900.
The next four iterations are
p
4
= p
3
f(p
3
)
p
3
p
2
f(p
3
) f(p
2
)
= 1.1964853266
p
5
= p
4
f(p
4
)
p
4
p
3
f(p
4
) f(p
3
)
= 1.1986453684
p
6
= p
5
f(p
5
)
p
5
p
4
f(p
5
) f(p
4
)
= 1.1986913364
p
7
= p
6
f(p
6
)
p
6
p
5
f(p
6
) f(p
5
)
= 1.1986912435.
Order of
Convergence
To determine the order of convergence for the
secant method, we need to derive the
corresponding error evolution equation.
The rst step is to subtract the true root p from
both sides of the formula for p
n+1
, yields
p
n+1
p = p
n
p f(p
n
)
p
n
p
n1
f(p
n
) f(p
n1
)
.
We approximate the function values f(p
n
) and
f(p
n1
) by the second degree Taylor polynomials
f(p
n
) f
(p)(p
n
p) +
f
(p)
2
(p
n
p)
2
,
f(p
n1
) f
(p)(p
n1
p) +
f
(p)
2
(p
n1
p)
2
,
The term f(p
n
) f(p
n1
) is then approximately,
f(p
n
) f(p
n1
) f
(p)(p
n
p
n1
)
+
f
(p)
2
_
(p
n
p)
2
(p
n1
p)
2
_
.
We rewrite as
f(p
n
) f(p
n1
)
(p
n
p
n1
)
_
_
f
(p) +
f
(p)
2
(p
n
+ p
n1
2p)
_
_
.
Then
p
n+1
p
(p
n
p)(p
n1
p)f
(p)
2f
(p) + f
(p)(p
n
+ p
n1
2p)
.
As p
n
and p
n1
approach p, the term in the
denominator involving the second derivative can
be dropped and the leading term in the error is
given by
[e
n+1
[ C[e
n
[ [e
n1
[,
where C =
f
(p)
2f
(p)
.
Let the secant method is of order with
asymptotic error constant , then the error
formula is
[e
n+1
[ [e
n
[
.
The relationship can also be written as
[e
n
[ [e
n1
[
, (or) [e
n1
[
[e
n
[
1
.
Now substituting for [e
n+1
[ and [e
n1
[, we get
[e
n
[
C[e
n
[
[e
n
[
1
.
Equating powers on [e
n
[, we obtain
= 1 +
1
.
The single positive root of the equation is
=
1 +
5
2
.
Hence, the secant method is of order
_
_
1 +
5
2
_
_
1.618.
We now equating the coefcients of [e
n
[ yields
C
1
=
_
_
f
(p)
2f
(p)
_
_
1
.
Example
The equation
f(x) = x
3
+ x
2
3x 3 = 0,
has a root on the interval (1, 2) namely x =
3.
For n 2, compute the ratio
[p
n
p[/[p
n1
p[
1.618
and show that this value
approaches ([f
(p)/2f
(p)[)
0.618
.
Solution. Plz try
Regula-Falsi Method
x
y
a
n
b
n
p
n
p
y=f(x)
The second simple enclosure method is known
as the method of false position.
Method of false position iterately determines a
sequence of root enclosing intervals (a
n
, b
n
) and
a sequence of approximations denote by p
n
.
During each iteration, a single point is selected
from (a
n
, b
n
) to approximate the location of the
root and serve as p
n
.
If p
n
is an accurate enough approximation, the
iterative process is terminated. Otherwise, the
Intermediate Value Theorem is used to
determine whether the root lies on the
subinterval (a
n
, p
n
) or on the subinterval (p
n
, b
n
).
The entire process is then repeated on that
subinterval.
The bisection method simply chooses the
midpoint of the enclosing interval.
The method of false position uses the x-intercept
of the line which passes through the points
(a
n
, f(a
n
)) and (b
n
, f(b
n
)) as p
n
.
The equation of the line which passes through
(a
n
, f(a
n
)) and (b
n
, f(b
n
)) is given by
y f(b
n
) =
f(b
n
) f(a
n
)
b
n
a
n
(x b
n
).
Setting y = 0 and x = p
n
yields
p
n
= b
n
f(b
n
)
b
n
a
n
f(b
n
) f(a
n
)
.
Example
Do four iterations of the method of false position
to nd the root of
x
3
+ 2x
2
3x 1,
in the interval (1, 2).
Solution. For the rst iteration, we have
(a
1
, b
1
) = (1, 2) and we know that f(a
1
) = 1 < 0
and f(b
1
) = 9 > 0.
The rst approximation to the location of the zero
is then
p
1
= b
1
f(b
1
)
b
1
a
1
f(b
1
) f(a
1
)
= 1.1
To determine whether the zero is contained on
(a
1
, p
1
) = (1, 1.1) or on (p
1
, b
1
) = (1.1, 2).
We calculate
f(p
1
) = 0.549 < 0.
Since f(a
1
) and f(p
1
) are of the same sign, the
Intermediate Value Theorem tells that the zero is
between p
1
and b
1
.
For the next iteration, we take
(a
2
, b
2
) = (p
1
, b
1
) = (1.1, 2).
The second approximation to the location of the
zero is
p
2
= b
2
f(b
2
)
b
2
a
2
f(b
2
) f(a
2
)
= 1.151743638.
We calculate
f(p
2
) 0.274 < 0,
which is of the same sign as f(a
2
).
Hence, the Intermediate Value Theorem tells that
the zero is now between p
2
and b
2
.
We take
(a
3
, b
3
) = (p
2
, b
2
) = (1.151743638, 2).
In the third iteration, we get
p
3
= 1.17684091, and f(p
3
) = 0.131 < 0.
Newtons Method
x
y
Tangent
line
y=f(x)
p
p
n+1
p
n
Let p
n
denote the most recent approximation to a
zero p of the function f(x).
Replace f(x) by its tangent line approximation
based at the location x = p
n
and take the
x-intercept of the tangent line as the next
approximation p
n+1
to the root.
Since the tangent line approximation based at
x = p
n
is given by
y f(p
n
) = f
(p
n
)(x p
n
).
Now substituting x = p
n+1
and y = 0, we get
p
n+1
= p
n
f(p
n
)
f
(p
n
)
,
where n = 0, 1, 2, .
Note
We observe that each iteration of Newtons
method, we require two separate function
evaluations:
The evaluation of the function.
The evaluation of the function derivative.
Example
Find the root of the function
f(x) = x
3
+ 2x
2
3x 1 = 0,
on the interval (1, 2) by using the Newtons
method.
Solution. To apply Newtons method, the
derivative of f(x) is needed. We get
f
(x) = 3x
2
+ 4x 3.
With a starting approximation of p
0
= 1, we nd
p
1
= p
0
f(p
0
)
f
(p
0
)
= 1.25,
p
2
= p
1
f(p
1
)
f
(p
1
)
= 1.2009345794,
p
3
= p
2
f(p
2
)
f
(p
2
)
= 1.1986958411,
p
4
= p
3
f(p
3
)
f
(p
3
)
= 1.1986912435.
[p p
4
[ 1.937 10
11
.
Note
In the preceding example, Newtons method
achieved an accuracy of 1.937 10
11
with
only eight function evaluations, four
evaluations of f(x) and four evaluations of
f
(x).
For comparison, starting from the interval
(1, 2), the bisection method needs 36 function
evaluations and the method of false position
needs 31 evaluations to produce similar
accuracy.
Example
f(x) = x
3
+ 2x
2
3x 1,
on the interval (3, 2) using (i) Bisection
method (ii) Secant method (iii) Regula-Falsi
method and (iv) Newtons method.
Compare all the methods with
p = 2.9122291785.
Solution. Plz try.
Example
Consider the function
f(x) = x
3
+ 2x
2
3x 1.
The sequences generated by Newtons method
for p
0
= 1, p
0
= 2 and p
0
= 3.
Inuence of p
0
n p
0
= 1 p
0
= 2 p
0
= 3
1 1.2500000000 1.4705882353 2.0277777778
2 1.2009345794 1.2471326788 1.4845011523
3 1.1986958411 1.2006987324 1.2514517238
4 1.1986912435 1.1986949265 1.2010586170
5 1.1986912435 1.1986963626
6 1.1986912435
Note
In the preceding example, the change of p
0
results only in a variation in the number of
iterations needed to achive convergence.
Example
f(x) = tan(x) x 6.
Here, if we take p
0
= 0.48, the Newtons method
converges to 0.4510472613 in ve iterations.
If we take p
0
= 0.4, then the sequence generated
by Newtons method fails to converge even after
5000 iterations.
Note
Thus, unlike simple enclosure methods,
Newtons method is not guaranteed to
converge for an arbitrary starting point p
0
.
If we take p
0
= 0, then the sequence
converges after 42 iterations to 697.4995475.
This is infact, one of the many zeros of
f(x) = tan(x) x 6.
Note
This shows even when Newtons method
converges, it may converge to a value very far
from p
0
.
Order of
Convergence
Theorem. Let p be the root of the function
f(x) = 0, and assume that f
(p) ,= 0. Let
f(x), f
(x), and f
(x) is continuous in a
neighborhood of the root p of f(x) then
lim
n
[e
n+1
[
[e
n
[
2
,
where =
1
2
|f
(p)|
|f
(p)|
.
Proof. We know that
p
n+1
= p
n
f(p
n
)
f
(p
n
)
.
Now taking
e
n+1
= p p
n+1
= p p
n
+
f(p
n
)
f
(p
n
)
= e
n
+
f(p
n
)
f
(p
n
)
.
We calculate
f(p
n
) = f(p e
n
) = f(p) e
n
f
(p) +
e
2
n
2!
f
(
n
),
where
n
lies between p
n
and p.
Since f(p) = 0, we get
f(p
n
) = f(p e
n
) = e
n
f
(p) +
e
2
n
2!
f
(
n
).
We note that
f
(p
n
) = f
(p) +
n
,
where
n
0 as n .
On substituting for f(p
n
) and f
(p
n
), we get
e
n+1
= e
n
+
e
n
f
(p) +
e
2
n
2
f
(
n
)
f
(p) +
n
.
We rewrite the above equation as
e
n+1
= e
n
+
e
n
f
(p) +
e
2
n
2
[f
(p) +
n
]
f
(p) +
n
,
where
n
0 as n .
We have
e
n+1
=
e
n
n
+
e
2
n
2
[f
(p) +
n
]
f
(p) +
n
.
Now taking
lim
n
e
n+1
e
2
n
=
1
2
f
(p)
f
(p)
.
Hence,
lim
n
e
n+1
e
2
n
= lim
n
[e
n+1
[
[e
n
[
2
=
1
2
[f
(p)[
[f
(p)[
.
Therefore, Newtons method converges
quadratically.
Example
The equation
f(x) = x
3
+ x
2
3x 3 = 0,
has a root on the interval (1, 2), namely x =
3.
For n 1, compute the ratio
|p
n
p|
|p
n1
p|
2
and show
that this value approaches
1
2
|f
(p)|
|f
(p)|
.
Solution. Plz try.
M
ullers Method
y=f(x)
y=p(x)
x x x
2
x
3
x
y
0 1
Note
The methods discussed up to this point allow
us to nd a zero of the function once an
approximation to that zero is known.
These methods are not very satisfactory
when all the zeros of a function are required.
M
ullers method is useful for obtaining both

real and complex roots of a function.
M
ullers method is an extension of the secant

method.
The secant method begins with two initial
approximations x
0
and x
1
and determines the
next approximation x
2
as the intersection of the
x-axis with the line through (x
0
, f(x
0
)) and
(x
1
, f(x
1
)).
M
ullers method uses three initial approximations

x
0
, x
1
and x
2
and determines the next
approximation x
3
by considering the intersection
of the x-axis with the parabola through the points
(x
0
, f(x
0
)), (x
1
, f(x
1
)) and (x
2
, f(x
2
)).
Let
y = a(x x
0
)
2
+ b(x x
0
) + c,
be the equation of the parabola passing through
the points (x
0
, f(x
0
)), (x
1
, f(x
1
)) and (x
2
, f(x
2
)).
The constants a, b and c are determined from the
conditions
f(x
2
) = (x
2
x
0
)
2
+ b(x
2
x
0
) + c,
f(x
1
) = a(x
1
x
0
)
2
+ b(x
1
x
0
) + c,
and
f(x
0
) = c.
Let x
1
x
0
= h
1
, x
0
x
2
= h
2
, we obtain
a =
h
2
f(x
1
) (h
1
+ h
2
)f
(
x
0
) + h
1
f(x
2
)
h
1
h
2
(h
1
+ h
2
)
,
b =
f(x
1
) f(x
0
) ah
2
1
h
1
,
and
c = f(x
0
).
Now, the points of intersection of the parabola
y = a(x x
0
)
2
+ b(x x
0
) + c
with x-axis, i.e., y = 0 are
a(x x
0
)
2
+ b(x x
0
) + c = 0.
Due to roundoff errors caused by the subtraction
of nearly equal numbers, we apply this formula
x x
0
=
2c
b
b
2
4ac
.
The sign in the denominator should be chosen so
that the denominator will be largest in magnitude.
With this choice the next approximation x
3
is
closest to the zero of f(x).
Therefore,
x
3
= x
0
2c
b + sgn(b)
b
2
4ac
.
After nding x
3
, to begin the next iteration, we
use the three recent approximations x
1
, x
2
and
x
3
.
Note
At each step, the method involves the radical
b
2
4ac, so the method gives approximate
complex roots when b
2
4ac < 0.
Example
Do three iterations of M
ullers method to nd the

root of
f(x) = e
x
+ 1 = 0,
starting with the values 1, 0, 1.
Solution. Clearly f(x) has no real roots.
We start three initial guesses x
0
= 1, x
1
= 0
and x
2
= 1, then we derive the iteration table
The iteration table:
n x
n
0 1.00
1 0.00
2 1.00
3 1.0820 + 1.5849i
4 1.2735 + 2.8505i
5 0.3814 + 3.7475i
Example
f(x) = x
3
+ 2x
2
3x 1,
by using the M
ullers method. Here take the

starting points x
0
= 0.5, x
2
= 1 and x
1
= 2
Solution. Plz try
Fixed Point Method
A xed point of the function g(x) is any real
number p for which g(p) = p.
i.e., whose location is xed by g(x).
Mean Value Theorem
If the function f(x) is continuous on the closed
interval [a, b] and differentiable on the open
interval (a, b), then there exists a real number
(a, b) such that
f
() =
f(b) f(a)
b a
.
Proof. Refer any calculus book.
Example
sin 0 = 0,
this implies x = 0 is said to be a xed point of
the function sin x.
g(x) = x
2
,
here x = 0, 1 are two xed points of g(x) = x
2
.
Let
g(x) = e
x
.
The equation e
x
= x cannot be solved by
analytical. Chapter 2Nonlinear Equations p.82/158
2 0 2 4 6 8 10
2
0
2
4
6
8
10
Figure indicates that the existence of a unique
xed point.
Example
Let
g
1
(x) = e
x
,
and
g
2
(x) = x
2
+ 1.
These two functions do not have any xed points.
Question?
Under what conditions a function is guaranteed
to have an unique xed point?.
Theorem
Let g(x) be continuous on the closed interval
[a, b] with g : [a, b] [a, b]. Then g(x) has a xed
point p [a, b].
Furthermore, if g(x) is differentiable on the open
interval (a, b) and there exists a positive constant
k < 1 such that
[g
(x)[ k < 1, for all x (a, b),

then the xed point in [a, b] is unique.
Existence
Assume that g(x) is continuous on the closed
interval [a, b] with g : [a, b] [a, b].
Dene the auxiliary function
h(x) = g(x) x.
Note that h(x) satises the following properties
(i) h(x) is the difference of two functions that are
continuous on [a, b], this implies h(x) is also
continuous on that interval.
(ii) the roots of h(x) are precisely the xed points
of g(x).
Now, since
min
x[a,b]
g(x) a, and max
x[a,b]
g(x) b,
it follows that
h(a) = g(a) a 0, and h(b) = g(b) b 0.
If either h(a) = 0 or h(b) = 0, then we have found
a root of h(x) which is a xed point of g(x) and
we are done.
If neither h(a) = 0 nor h(b) = 0, then
h(b) < 0 < h(a).
Since h(x) is continuous on [a, b] the
Intermediate Value Theorem guarantee the
existence of p [a, b] such that h(p) = 0, which
implies g(p) = p.
Existence of a xed
point
a
b
a
b
y
x
y= x
y= g(x)
Uniqueness
This part of the proof will proceed by
contradiction.
Suppose that p and q are both xed points of the
function g(x) on the interval [a, b] with p ,= q.
By the denition of the xed point g(p) = p and
g(q) = q.
Then
[p q[ = [g(p) g(q)[.
Using the Mean Value Theorem, we get
[p q[ = [g
()(p q)[
= [g
()[ [p q[
k[p q[ < [p q[,
which is a contradiction.
Hence, p = q and the xed point is unique.
Note
The hypotheses of this theorem are sufcient
conditions but not necessary conditions.
It is possible for a function to violate one or more
of the hypotheses, yet still have a xed point.
Example
g(x) = 4x(1 x),
on the interval [0.1, ).
Here,
lim
x
g(x) ,
here g does not map [0.1, ) onto itself. Also
lim
x
[g
(x)[ +.
However, g has xed points at x = 0 and x =
3
4
.
Fixed Point Iteration
If it is known that a function g(x) has a xed
point, one way to approximate the value of that
xed point is to use what is known as a xed
point iteration scheme.
Denition
A xed point iteration scheme to approximate the
xed point p of a function g(x), generates the
sequence p
n
by the rule p
n
= g(p
n1
) for all
n 1, given a starting approximation p
0
.
Connection between xed point problems and
root nding problems.
Every root nding problem can be transformed
into any number of different xed point problems.
Some of these xed point problems will converge
rapidly, some will converge slowly and some will
not converge at all.
The conversion process is actually quite simple.
Take the root nding equation f(x) = 0,
algebraically transform it into an equation of the
form
x =
the expression on the right-hand side of the
resulting function is a corresponding iteration
function g(x).
Example
Find a suitable interval and corresponding
iterative function g(x) such that the xed point
iteration converges to the solution of the equation
f(x) = e
x
x 2 = 0.
Perform four iterations.
Solution. We now nd the suitable interval.
We nd the function values at different points.
We have
f(0) = 1 < 0,
f(1) = 0.28 < 0,
f(2) = 3.389 > 0.
Therefore, root lies on the interval I = [1, 2].
Rewrite the given equation as
x = ln(x + 2) = g(x).
We check the conditions for the existence and
uniqueness of the xed point method:
g(1) = 1.0986 > 1,
g(2) = 1.3862 < 2.
This shows that
g : [1, 2] [1, 2]
and also g(x) is continuous on [1, 2].
Now
g
(x) =
1
x + 2
,
and
max
xI
[g
(x)[ = g
(1) = 0.333 k < 1.

Therefore, the function g(x) satises all the
properties of xed point method.
Hence, the function g(x) has a unique xed point
on the interval [1, 2].
We denote xed point iteration scheme as
x
n
= g(x
n1
), n = 1, 2, 3, .
We choose x
0
= 1, we nd
x
1
= g(x
0
) = g(1) = 1.0986,
x
2
= g(x
1
) = g(1.0986) = 1.13095,
x
3
= g(x
2
) = g(1.13095) = 1.14134,
x
4
= g(x
3
) = g(1.14134) = 1.14465.
Example
f(x) = x
3
+ x
2
3x 3,
has a unique zero on the interval [1, 2].
We approximate this root using xed point
iteration.
We start the equation
x
3
+ x
2
3x 3 = 0.
We express
x =
x
3
+ x
2
3
3
= g
1
(x).
Similarly, we can also nd
g
2
(x) = 1 +
3x + 3
x
2
,
g
3
(x) =
_
3 + 3x x
2
_
1/3
,
g
4
(x) =
_
3 + 3x x
2
x
.
Now, choose suitable g
i
(x), i = 1, 2, 3, 4 such that
it has a unique xed point and using the xed
point iteration nd it.
Theorem
Let g(x) be continuous on the closed interval
[a, b] with g : [a, b] [a, b].
Furthermore, Suppose that g(x) is differentiable
on the open interval (a, b) and there exists a
positive constant k < 1 such that [g
(x)[ k < 1
for all x (a, b).
Then the sequence p
n
generated by
p
n
= g(p
n1
) converges to a xed point of p for
any p
0
[a, b].
Proof. We need to show
[p
n
p[ 0 as n ,
for any starting value p
0
[a, b]. Taking
[p
n
p[ = [g(p
n1
) g(p)[
= [g
()[ [p
n1
p[
k[p
n1
p[
k
2
[p
n2
p[
.
.
.
k
n
[p
0
p[.
Now, since k < 1, we get
lim
n
[p
n
p[ lim
n
k
n
[p
0
p[ = [p
0
p[ lim
n
k
n
= 0.
Order of
Convergence
Let g(x) be a continuous function on the closed
interval [a, b] with g : [a, b] [a, b] and suppose
that g
(x) is continuous on the open interval (a, b)

with [g
(x)[ k < 1 for all x (a, b).

If g
(p) ,= 0 then for any p

0
[a, b], the sequence
p
n
= g(p
n1
) converges only linearly to the xed
point p.
We need to prove if g
(p) ,= 0, then the

convergence is only linear.
In other words, it must be shown that
lim
n
[p
n+1
p[
[p
n
p[
= ,
where is a asymptotic error constant.
Now, taking
[p
n+1
p[ = [g(p
n
) g(p)[
= [g
(c
n
)[ [p
n
p[,
where c
n
is between p
n
and p.
Note that p
n
p, we get c
n
p.
Since g
(x) is continuous on (a, b), we have

lim
n
[g
(c
n
)[ = [g
( lim
n
c
n
)[ = [g
(p)[.
Hence,
lim
n
[p
n+1
p[
[p
n
p[
= [g
(p)[.
This show that the order of convergence of the
sequence p
n
is one with asymptotic error
constant [g
(p)[.
Theorem
If the iteration function g(x) is such that g
(x) is
continuous in some neighborhood of the xed
point p and g
(p) = 0, g
(p) ,= 0, then the xed

point method converges quadratically.
Proof. We calculate
e
n+1
= p p
n+1
= g(p) g(p
n
)
= g(p) g(p e
n
).
Using Taylor series expansion, we get
e
n+1
= g(p)
_
_
g(p) e
n
g
(p) +
e
2
n
2!
g
(
n
)
_
_
,
where
n
lies between p
n
and p. We get
e
n+1
= e
n
g
(p)
e
2
n
2
g
(
n
).
Using the fact g
(p) = 0, we nd
e
n+1
=
e
2
n
2
[g
(p) +
n
] ,
where
n
0 as n .
We calculate
[e
n+1
[
[e
n
[
2
=
1
2
[g
(p) +
n
[.
Finally, we get
lim
n
[e
n+1
[
[e
n
[
2
=
1
2
[g
(p)[.
Example
Use the xed-point iteration to nd a root of the
function
f(x) = e
x
sin x = 0.
Solution. We write f(x) = 0 in the form
x = x + e
x
sin x = g(x).
Since
f(0.5) 0.127 > 0,
f(0.7) 0.147 < 0,
the positive root lies on the interval I = [0.5, 0.7].
To verify g(x) is a convergent iteration function,
we need to check the existence and uniqueness
conditions:
g(0.5) = 0.6271 > 0.5,
g(0.7) = 0.552367 < 0.7,
we obtain
0.5 < g(x) < 0.7, for all x I.
Also
g
(x) = 1 e
x
cos x.
We nd
g
(0.5) = 0.48,
g
(0.7) = 0.26.
Since g(x) is a monotone function on I, we have
[g
(x)[ < 1, for all x I.

Hence, the xed point iteration will converge if x
0
is chosen on I.
Plz complete the remaining part.
Newtons Method for
Multiple Roots
A root p of f(x) is called a multiple root of order
m, if
f(x) = (x p)
m
q(x),
where lim
xp
q(x) ,= 0, and m is a positive
integer.
Theorem
Newtons method converges linearly in case of
multiple root.
But if p
n+1
is taken as
p
n+1
= p
n
m
f(p
n
)
f
(p
n
)
,
where p is a multiple root of f(x) of order m, then
it converges quadratically.
Proof. We dene
g(x) =
_
_
x
f(x)
f
(x)
x ,= p
p x = p,
where f(x) = (x p)
m
q(x), p is multiple root of
order m.
We prove continuity of g(x) at p.
Taking
lim
xp
g(x) = lim
xp
_
_
x
f(x)
f
(x)
_
_
= lim
xp
x lim
xp
(x p)
m
q(x)
m(x p)
m1
q(x) + (x p)
m
q
(x)
.
This implies that
lim
xp
g(x) = p = g(p).
Therefore, g(x) is continuous at p.
We now nd g
(p):
g
(p) = lim
xp
g(x) g(p)
x p
.
Substituting the expression for g(x), we get
g
(p) = lim
xp
_
x
f(x)
f
(x)
_
p
x p
.
Using the expression for f(x), we get
g
(p) = 1
1
m
,= 0,
as m > 1.
We now prove the continuity of g
(x) at p.
We know that
g(x) = x
f(x)
f
(x)
,
when x ,= p. Then
g
(x) =
f(x) f
(x)
[f
(x)]
2
.
Now using the expression for f(x), we get
lim
xp
g
(x) = 1
1
m
= g
(p).
Therefore, g
(x) is continuous at p.
We conclude that the convergence rate is linear.
For obtaining the quadratic convergence, we take
the modied Newtons iteration
p
n+1
= p
n
m
f(p
n
)
f
(p
n
)
.
To prove that this converges quadratically, it is
sufcient to prove that
g
(p) ,= 0.
Please complete the remaining part.
Example
Find the root of the equation
f(x) = x
2
+ 2x + 1 = 0,
using Newtons method for simple roots and
multiple roots.
Solution. Given
f(x) = x
2
+ 2x + 1,
then
f
(x) = 2x + 2.
The iteration formula for nding simple root by
Newtons method is given by
x
n+1
= x
n
f(x
n
)
f
(x
n
)
= x
n
x
2
n
+ 2x
n
+ 1
2(x
n
+ 1)
=
x
n
2

1
2
, n = 0, 1, 2, 3, .
Choosing x
0
= 0, we nd
x
1
= 0.5,
x
2
= 0.75,
x
3
= 0.875,
x
4
= 0.9375,
.
.
.
x
20
= 0.999999,
x
21
= 1,
x
22
= 1.
Since the multiplicity m of the root p = 1 is 2.
Using Newtons iterative formula for multiple root
is given by
x
n+1
= x
n
m
_
_
f(x
n
)
f
(x
n
)
_
_
,
= x
n
2
_
_
x
2
n
+ 2x
n
+ 1
2(x
n
+ 1)
_
_
,
= x
n
(x
n
+ 1),
x
n
= 1.
If we take x
0
= 0, we nd x
1
= 1.
Here, we obtain the root in the rst iteration only.
Hence, Newtons method for multiple root is
much faster than Newtons method for simple
root.
Example
Do three iterations of Newtons method to obtain
the double root of
x
3
2x
2
0.75x + 2.25 = 0,
which is close to 1 such that iterations converges
quadratically.
Solution. We use modied Newtons method
p
n+1
= p
n
2
f(p
n
)
f
(p
n
)
.
We have
f(p
n
) = p
3
n
2p
2
n
0.75p
n
+ 2.25
f
(p
n
) = 3p
2
n
4p
n
0.75
Then
p
n+1
=
p
3
n
+ 0.75p
n
4.5
3p
2
n
4p
n
0.75
.
We calculate
n = 0, p
1
= 1.5714
n = 1, p
2
= 1.5009
n = 2, p
3
= 1.5
System of Nonlinear
Equations
We consider the system of n non-linear
equations
f
1
(x
1
, x
2
, x
3
, , x
n
) = 0,
f
2
(x
1
, x
2
, x
3
, , x
n
) = 0,
.
.
.
f
n
(x
1
, x
2
, x
3
, , x
n
) = 0,
where x
1
, x
2
, , x
n
are n unknowns.
We discuss two methods to solve the above
system of equations.
Newtons Method
We discuss the Newtons method for system of
two non-linear equations:
f(x, y) = 0,
g(x, y) = 0.
Let (x
0
, y
0
) be an initial approximation to the root
of the system.
If (x
0
+ h, y
0
+ k) is the root of the system, then
we must have
f(x
0
+ h, y
0
+ k) = 0,
g(x
0
+ h, y
0
+ k) = 0.
Assuming that f and g are sufciently
differentiable, we expand by Taylors series to
obtain
f
0
+ h
f
x
0
+ k
f
y
0
+ higher order terms = 0,
and
g
0
+ h
g
x
0
+ k
g
y
0
+ higher order terms = 0,
where f
0
= f(x
0
, y
0
) and
f
x
0
=
_
f
x
_
x=x
0
.
Neglecting higher order terms, we obtain the
system of equations
h
f
x
0
+ k
f
y
0
= f
0
,
h
g
x
0
+ k
g
y
0
= g
0
. (1)
We now dene the Jacobian
J(f, g) =
f
x
f
y
g
x
g
y
If the Jacobian does not vanish, then (1) has a

unique solution given by
h =
_
_
gf
y
fg
y
J(f, g)
_
_
(x
0
,y
0
)
,
k =
_
_
fg
x
gf
x
J(f, g)
_
_
(x
0
,y
0
)
.
The new approximations are given by
x
1
= x
0
+ h,
y
1
= y
0
+ k.
The process is to be repeated till we obtain the
roots to the desired accuracy.
We now write the Newtons iteration formula:
x
n+1
= x
n
_
_
fg
y
gf
y
J(f, g)
_
_
(x
n
,y
n
)
,
and
y
n+1
= y
n
_
_
gf
x
fg
x
J(f, g)
_
_
(x
n
,y
n
)
,
where n = 0, 1, 2, 3, .
Example
Solve the system of nonlinear equations
x
2
y
3
x + y 1.125 = 0
x
3
+ y
2
+ x y + 0.875 = 0,
by Newtons method starting with
x
0
= 0.2, y
0
= 0.2.
Fixed Point Method
We consider the system of equations
F(x, y) = 0,
G(x, y) = 0. (2)
We assume that the system (2) have a solution
(, ).
We rewrite the above system (2)
x = f(x, y),
y = g(x, y), (3)
such that (3) have same solution (, ).
Let (x
n
, y
n
) be nth approximations to (, )
computed from the iteration scheme
x
n+1
= f(x
n
, y
n
),
y
n+1
= g(x
n
, y
n
), n = 0, 1, 2, (4)
Note that
= f(, ), (5)
= g(, ). (6)
Now substracting (4) from (6), we get
x
n+1
= f(, ) f(x
n
, y
n
)
= f(x
n
+ x
n
, y
n
+ y
n
)
f(x
n
, y
n
).
Using the Taylors series expansion, we obtain
x
n+1
= f(x
n
, y
n
) + ( x
n
)f
x
(x
n
, y
n
)
+ ( y
n
)f
y
(x
n
, y
n
) + higher order terms
f(x
n
, y
n
)
= ( x
n
)f
x
(x
n
, y
n
) + ( y
n
)f
y
(x
n
, y
n
)
+ higher order terms.
Similarly, we nd
y
n+1
= ( x
n
)g
x
(x
n
, y
n
) + ( y
n
)g
y
(x
n
, y
n
)
+ higher order terms.
Neglecting the higher order terms, we express in
the matrix form
_
_
x
n+1
y
n+1
_
_
=
_
_
f
x
(x
n
, y
n
) f
y
(x
n
, y
n
)
g
x
(x
n
, y
n
) g
y
(x
n
, y
n
)
_
_
_
_
x
n
y
n
_
_
.
In compact form
e
n+1
= Be
n
,
where B is the coefcient matrix.
Theorem
If f
x
and f
y
are continuous in R (rectangular
region, containing (, ) and (x
n
, y
n
) for
n = 0, 1, 2, ) then it is proved that the iteration
scheme is given by
x
n+1
= f(x
n
, y
n
),
y
n+1
= g(x
n
, y
n
), n = 0, 1, 2,
converges to (, ) if
|B|
k < 1,
for some number k, i.e.,
[f
x
[ +[f
y
[ k < 1,
and
[g
x
[ +[g
y
[ k < 1.
Example
Consider the nonlinear system
x = 1 + sin
_
8
(x + y)
_
,
y = 1 cos
_
8
(x y)
_
.
(i) Find a region D 1
2
such that the iteration
x
n+1
= 1 + sin
_
8
(x
n
+ y
n
)
_
,
y
n+1
= 1 cos
_
8
(x
n
y
n
)
_
,
is guaranteed to converge to a unique xed point
for any (x
0
, y
0
) D.
(ii) Determine a bound on the rate of linear
convergence of the method in max norm.
Solution. We have
_
_
x
y
_
_
= H(x, y) =
_
_
f(x, y)
g(x, y)
_
_
=
_
_
1 + sin
_
8
(x + y)
_
1 cos
_
8
(x y)
_
_
_
.
Notice that for any (x, y), we have
H(x, y) [0, 2] [0, 2].
Thus, if R is any closed and bounded set in 1
2
such that [0, 2] [0, 2] R, then H(D) D.
We dene
B =
_
_
f
x
(x, y) f
y
(x, y)
g
x
(x, y) g
y
(x, y)
_
_
=
_
_
8
cos
_
8
(x + y)
_

8
cos
_
8
(x + y)
_
8
sin
_
8
(x y)
_

8
sin
_
8
(x y)
_
_
_
Note that
|B(x, y)|
=
max
4
[ cos
_
8
(x + y)
_
[,

4
[ sin
_
8
(x y)
_
[

4
<
1, for any (x, y) D.
Example
Consider the nonlinear system
x =
3
4
ln
_
1 +
1
2
(x + y)
2
_
,
y =
3
4
ln
_
1 +
1
2
(x y)
2
_
.
(i) Find a region D 1
2
, such that the iteration
x
n+1
=
3
4
ln
_
1 +
1
2
(x
n
+ y
n
)
2
_
,
y
n+1
=
3
4
ln
_
1 +
1
2
(x
n
y
n
)
2
_
,
is guaranteed to converge to the root
(x, y) = (0, 0) for any (x, y) D.
Solution. Plz try.
System of Linear
Equations
2
Contents
1. Gauss Elimination Method
2. Inverse of a Matrix
3. Condition Numbers and Errors
ill-Conditioned System
4. Iterative methods
3
System of Linear Equations
Circuit analysis (Mesh and node equations).
Numerical solution of differential equations (Finite
Difference Method).
Numerical solution of integral equations (Finite
Element Method, Method of Moments).
n n nn n n
n n
n n
b x a x a x a
b x a x a x a
b x a x a x a
= + + +
= + + +
= + + +
2 2 1 1
2 2 2 22 1 21
1 1 2 12 1 11
11 12 1 1 1
21 22 2 2 2
1 2
n
n
n n nn n n
a a a x b
a a a x b
a a a x b

=

4
Notations
The above system of equations can be written in
matrix from as Ax = b.
A is called the coefficient matrix, A = (a
ij
).
b is called the constant vector = (b
1
,b
2
,,b
n
)
T
.
[A|b] is called the augmented matrix.
If , then a multiplier is defined as
1
1
11
, 2, 3, 4,...
i
i
a
m i
a
= =
0
11
a
5
Consistency (Solvability)
The linear system of equations Ax=b has a
solution, or said to be consistent if and only if
Rank{A}=Rank{A|b}
A system is inconsistent when
Rank{A}<Rank{A|b}
Rank{A} is the maximum number of linearly
independent rows or columns of A.
Rank can be found by using ERO (Elementary
Row Operations).
6
Elementary row operations
The following operations applied to the augmented
matrix [A|b], yields an equivalent linear system.
Interchanges: Any two rows can be
interchanged.
Scaling: Multiplying a row by a nonzero
constant.
Replacement: The row can be replaced by the
sum of that row and a nonzero multiple of any
other row.
7
An inconsistent example
5
4
4 2
2 1
2
1
x
x
0 0
2 1
Rank{A}=1
Rank{A|b}=2
ERO:Multiply the first row with
-2 and add to the second row
3
4
0
2
0
1
The system
of equations
are not
solvable
8
Uniqueness of solutions
The system has a unique solution if and
only if Rank{A} = Rank{A|b} = n,
n is the order of the system.
Such systems are called full-rank
systems.
9
Full-rank systems Example
If Rank{A}=n
Det{A} 0 A is nonsingular so invertible
and has a Unique solution.
2
4
1 1
2 1
2
1
x
x
10
Rank deficient matrices
If Rank{A}=m<n
Det{A} = 0 A is singular and not invertible,
an infinite number of solutions (n-m free
variables) under-determined system.
8
4
4 2
2 1
2
1
x
x
Rank{A} = Rank{A|b} = 1
Consistent so solvable.
11
ill-conditioned system of equations
A small change in the entries of matrix A,
causes a large deviation in the solution.
1
2
1 2 3
0.48 0.99 1.47
x
x

=

1
2
1 2 3
0.49 0.99 1.47
x
x

=

1
1
2
1
x
x
0
3
2
1
x
x
12
Ill-conditioned continued.....
A linear system of
equations is said to
be ill-conditioned
if the coefficient
matrix tends to be
singular.
13
Solution Techniques
Direct solution methods
Finds a solution in a finite number of operations by
transforming the system into an equivalent system that
is easier to solve.
Diagonal, upper or lower triangular systems are easier
to solve.
Number of operations is a function of system size n.
Iterative solution methods
Computes succesive approximations of the solution
vector for a given A and b, starting from an initial point
x
0
.
Total number of operations is uncertain, may not
converge.
14
Direct solution Methods
Gaussian Elimination
By using ERO, matrix A is transformed into an
upper triangular matrix (all elements below
diagonal are 0).
Back substitution is used to solve the upper-
triangular system.
n
i
n
i
nn ni n
in ii i
n i
b
b
b
x
x
x
a a a
a a a
a a a

1 1
1
1
1 1 11
ERO
n
i
n
i
nn
in ii
n i
b
b
b
x
x
x
a
a a
a a a
~
~
~
0 0
~ ~
0
1 1 1 1 11

B
a
c
k

s
u
b
s
t
i
t
u
t
i
o
n
15
Back Substitution
After the forward elimination phase, the matrix has
been transformed into upper triangular form.
Equation n just involves x
n
and so we can solve
easily.
Equation n-1 just involves x
n-1
and x
n
, and since we
already know x
n
we can find x
n-1
.
Working our way backwards through the equations
we can find x
n
, x
n-1
,, x
1
.
This is called the back substitution phase of the
Gaussian elimination algorithm.
16
First step of elimination
=
=
=
) 2 (
) 2 (
3
) 2 (
2
) 1 (
1
) 2 ( ) 2 (
3
) 2 (
2
) 2 (
3
) 2 (
33
) 2 (
32
) 2 (
2
) 2 (
23
) 2 (
22
) 1 (
1
) 1 (
13
) 1 (
12
) 1 (
11
) 1 (
11
) 1 (
1 1 ,
) 1 (
11
) 1 (
31 1 , 3
) 1 (
11
) 1 (
21 1 , 2
|
|
|
|
0
0
0
/
/
/
n nn n n
n
n
n
n n
b
b
b
b
a a a
a a a
a a a
a a a a
a a m
a a m
a a m
) 1 (
) 1 (
3
) 1 (
2
) 1 (
1
) 1 ( ) 1 (
3
) 1 (
2
) 1 (
1
) 1 (
3
) 1 (
33
) 1 (
32
) 1 (
31
) 1 (
2
) 1 (
23
) 1 (
22
) 1 (
21
) 1 (
1
) 1 (
13
) 1 (
12
) 1 (
11
|
|
|
|
n nn n n n
n
n
n
b
b
b
b
a a a a
a a a a
a a a a
a a a a
Pivotal element
1 1 1 1 1
, and 0for , 2, 3,...,
ij ij i j i i i i
a a m a b b m b a j i n = = = =
17
Second step of elimination
=
=
) 3 (
) 3 (
3
) 2 (
2
) 1 (
1
) 3 ( ) 3 (
3
) 3 (
3
) 3 (
33
) 2 (
2
) 2 (
23
) 2 (
22
) 1 (
1
) 1 (
13
) 1 (
12
) 1 (
11
) 2 (
22
) 2 (
2 2 ,
) 2 (
22
) 2 (
32 2 , 3
|
|
|
|
0 0
0 0
0
/
/
n nn n
n
n
n
n n
b
b
b
b
a a
a a
a a a
a a a a
a a m
a a m
) 2 (
) 2 (
3
) 2 (
2
) 1 (
1
) 2 ( ) 2 (
3
) 2 (
2
) 2 (
3
) 2 (
33
) 2 (
32
) 2 (
2
) 2 (
23
) 2 (
22
) 1 (
1
) 1 (
13
) 1 (
12
) 1 (
11
|
|
|
|
0
0
0
n nn n n
n
n
n
b
b
b
b
a a a
a a a
a a a
a a a a
Pivotal element
18
Like this (n-1) steps would be done so as the
final matrix would be such that upper triangular
gives the solutions and the numbers below
diagonal entries are the multipliers.
19
Back substitution algorithm

) (
) 1 (
1
) 3 (
3
) 2 (
2
) 1 (
1
1
3
2
1
) (
) 1 (
1
) 1 (
1 1
) 3 (
3
) 3 (
33
) 2 (
2
) 2 (
23
) 2 (
22
) 1 (
1
) 1 (
13
) 1 (
12
) 1 (
11
0 0 0 0
0 0 0
0 0
0
n
n
n
n
n
n
n
nn
n
n n
n
n n
n
n
n
b
b
b
b
b
x
x
x
x
x
a
a a
a a
a a a
a a a a
[ ]
1 , , 2 , 1
1
1
1
) ( ) (
) (
1
1
) 1 (
1
) 1 (
1 1
1
) (
) (
=
=
= =
+ =
n n i x a b
a
x
x a b
a
x
a
b
x
n
i k
k
i
ik
i
i
i
ii
i
n
n
n n
n
n
n
n n
n
n
nn
n
n
n
20
Pivoting
Computer uses finite-precision arithmetic.
A small error is introduced in each arithmetic
operation, error propagates.
When the pivotal element is very small, the
multipliers will be large.
Adding numbers of widely differring magnitude can
lead to loss of significance.
To reduce error, row or column or both
interchanges are made to maximize the magnitude
of the pivot element.
21
Simple pivoting
Suppose at the first step then this number can
not be used to eliminate other numbers of the first
columns of other rows.
To avoid this in first step if use this as pivotal
element at first step.
Similarly if at second step, chose a non-zero
number from 2
nd
column and from 2
nd
row onward.
Avoiding zero for pivotal element is known as
simple pivoting.
In Simple pivoting also, due to finite digit arithmetic,
results are in large errors.
, 0
11
= a
, 0
21
a
, 0
22
= a
22
Example: Without Pivoting
93 . 22
414 . 6
210 . 1 14 . 24
281 . 5 133 . 1
2
1
x
x
8 . 113
414 . 6
7 . 113 000 . 0
281 . 5 133 . 1
2
1
x
x
1
2
0.9956
1.001
x
x

=

31 . 21
133 . 1
14 . 24
21
= = m
4-digit arithmetic
Loss of significance
23
Example: With Partial Pivoting

414 . 6
93 . 22
281 . 5 133 . 1
210 . 1 14 . 24
2
1
x
x

338 . 5
93 . 22
338 . 5 000 . 0
210 . 1 14 . 24
2
1
x
x
000 . 1
000 . 1
2
1
x
x
04693 . 0
14 . 24
133 . 1
21
= = m
24
Partial Pivoting
In some cases, a
kk
(k)
is very close to zero. When we
divide it with a bigger number, rounding errors
arises which causes the damage of exact solution.
Using such an element as the pivot element will
result in gross errors in the further calculations in
the matrix.
To check it, we introduce partial pivoting and
complete pivoting.
25
Partial Pivoting Strategy
For in the GE process at stage k, let
Let j be the row index at which the maximum c
k
is
attained.
If j > k, then interchange k-th and j-th row in A and b.
Now, and this helps in
preventing the growth of elements in A
(k)
of greatly
varying size and causes to reduce the loss of
significance errors.
, 1 1 n k
. |, |
) (
n i k a Max c
k
ik k
=
| | 1, 1,...,
ik
m i k n = +
26
Pivoting procedures
) ( ) ( ) (
) ( ) ( ) (
) ( ) ( ) (
) 3 (
3
) 3 (
3
) 3 (
3
) 3 (
33
) 2 (
2
) 2 (
2
) 2 (
2
) 2 (
23
) 2 (
22
) 1 (
1
) 1 (
1
) 1 (
1
) 1 (
13
) 1 (
12
) 1 (
11
0 0 0
0 0 0
0 0 0
0 0
0
i
nn
i
nj
i
ni
i
jn
i
jj
i
ji
i
in
i
ij
i
ii
n j i
n j i
n j i
a a a
a a a
a a a
a a a a
a a a a a
a a a a a a

Eliminated
part
Pivot column
Pivot
row
27
Partial Row pivoting
) ( ) ( ) (
) ( ) ( ) (
) ( ) ( ) (
) 3 (
3
) 3 (
3
) 3 (
3
) 3 (
33
) 2 (
2
) 2 (
2
) 2 (
2
) 2 (
23
) 2 (
22
) 1 (
1
) 1 (
1
) 1 (
1
) 1 (
13
) 1 (
12
) 1 (
11
0 0 0
0 0 0
0 0 0
0 0
0
i
nn
i
nj
i
ni
i
jn
i
jj
i
ji
i
in
i
ij
i
ii
n j i
n j i
n j i
a a a
a a a
a a a
a a a a
a a a a a
a a a a a a

Interchange
these rows
Largest in magnitude
28
Partial Column pivoting
) ( ) ( ) (
) ( ) ( ) (
) ( ) ( ) (
) 3 (
3
) 3 (
3
) 3 (
3
) 3 (
33
) 2 (
2
) 2 (
2
) 2 (
2
) 2 (
23
) 2 (
22
) 1 (
1
) 1 (
1
) 1 (
1
) 1 (
13
) 1 (
12
) 1 (
11
0 0 0
0 0 0
0 0 0
0 0
0
i
nn
i
nj
i
ni
i
jn
i
jj
i
ji
i
in
i
ij
i
ii
n j i
n j i
n j i
a a a
a a a
a a a
a a a a
a a a a a
a a a a a a

Interchange
these columns
Largest in
magnitude
29
Complete pivoting
) ( ) ( ) (
) ( ) ( ) (
) ( ) ( ) (
) 3 (
3
) 3 (
3
) 3 (
3
) 3 (
33
) 2 (
2
) 2 (
2
) 2 (
2
) 2 (
23
) 2 (
22
) 1 (
1
) 1 (
1
) 1 (
1
) 1 (
13
) 1 (
12
) 1 (
11
0 0 0
0 0 0
0 0 0
0 0
0
i
nn
i
nj
i
ni
i
jn
i
jj
i
ji
i
in
i
ij
i
ii
n j i
n j i
n j i
a a a
a a a
a a a
a a a a
a a a a a
a a a a a a

Largest in
magnitude
Interchange
these columns
Interchange
these rows
30
Remarks
There is a possible change in order of the unknowns
in complete pivoting. So, the order of unknowns must
be reversed after back substitution in GE method.
In practical problems, the error behavior is
essentially the same in both partial pivoting and
complete pivoting.
The complete pivoting take more operations that in
partial. So, we prefer the latter in most practical
algorithms.
Hence, we mean pivoting as partial pivoting.
Chapter-3
Dr. P. Dhanumjaya
Chapter-3 p.1/64
Example 1
Consider the following system
0.003000x
1
+ 59.14x
2
= 59.17,
5.291x
1
6.130x
2
= 46.78,
has the exact solution x
1
= 10.00 and x
2
= 1.00.
We now nd the solution using Gauss elimination
method and four-digit arithmetic with rounding.
Chapter-3 p.2/64
We form an augmented matrix, we obtain
_
_
0.003 59.14
.
.
. 59.17
5.291 6.130
.
.
. 46.78
_
_ .
The multipliers are
m
i1
=
a
i1
a
11
, i = 2, 3, 4, ,
This gives
m
21
=
a
21
a
11
=
5.291
0.003
= 1763.66 1764.
Chapter-3 p.3/64
We now replace
a
ij
= a
ij
m
i1
a
1j
, i, j = 2, 3,
and
a
i1
= 0, i = 2, 3, 4,
We have
_
_
0.003 59.14
.
.
. 59.17
(1764) 104300
.
.
. 104400
_
_ .
Chapter-3 p.4/64
Using back substitution, we get
x
2
1.001, and x
1
10.
This shows that a small value of pivot, any error
in the numerator can be dramatically increased.
Chapter-3 p.5/64
Partial Pivoting
The augmented matrix is
_
_
0.003 59.14
.
.
. 59.17
5.291 6.130
.
.
. 46.78
_
_ .
We take
max{|a
11
|, |a
21
|} = 5.291.
We replace r
1
by r
2
, we get
_
_
5.291 6.130
.
.
. 46.78
0.003 59.14
.
.
. 59.17
_
_ .
Chapter-3 p.6/64
The multiplier is
m
21
=
a
21
a
11
= 0.000567.
We replace
a
22
= a
22
m
21
a
12
,
we get
_
_
5.291 6.130
.
.
. 46.78
(0.000567) 59.14
.
.
. 59.14
_
_ .
Chapter-3 p.7/64
This gives
x
2
= 1.0, and x
1
= 10.0
Chapter-3 p.8/64
Example 2
30.00x
1
+ 591400x
2
= 591700,
5.291x
1
6.130x
2
= 46.78,
_
_
30.0 591400
.
.
. 591700
5.291 6.130
.
.
. 46.78
_
_ .
Chapter-3 p.9/64
The maximum value in the rst column is 30.0
The multiplier is
m
21
=
a
21
a
11
=
5.291
30.0
= 0.1764.
We replace
a
22
= a
22
m
21
a
12
.
Chapter-3 p.10/64
We get
_
_
30.0 591400
.
.
. 591700
(0.1764) 104300
.
.
. 104400
_
_ .
x
2
1.001, and x
1
10.00
This shows that in a row if the coefcients vary
too much, then we get less accuracy.
To avoid this, we use scaling.
Chapter-3 p.11/64
Scaled Partial
Pivoting
Scaling
d
i
= max
1jn
|a
ij
|, i = 1, 2, 3, , n
then divide ith row by d
i
for i = 1, 2, 3, , n.
Choose numerically largest element of the
concerned column as pivotal element.
This is known as scaled partial pivoting.
Chapter-3 p.12/64
Note 1
Note that
scaling is useful only when size of the
coefcients vary too much.
Chapter-3 p.13/64
Example 3
Using Gaussian Elimination with partial pivoting,
scaling and back substitution, solve the following
algebraic system
6.684x
1
+ 2.925x
2
+ 9.835x
3
= 10.75,
5.543x
1
+ 5.953x
2
+ 88.63x
3
= 19.81,
8.375x
1
+ 2.988x
2
+ 8.681x
3
= 24.72.
Chapter-3 p.14/64
Solution
_
_
6.684 2.925 9.835
.
.
. 10.75
5.543 5.953 88.63
.
.
. 19.81
8.375 2.988 8.681
.
.
. 24.72
_
_
.
The scaling factors are
d
1
= 9.835, d
2
= 88.63, d
3
= 0.681.
Chapter-3 p.15/64
After scaled partial pivoting, the resulting matrix
is
_
_
0.96475 0.34419 1
.
.
. 2.8476
0.06254 0.06717 1
.
.
. 0.22351
0.67961 0.29741 1
.
.
. 1.0930
_
_
.
The multipliers are
m
21
=
a
21
a
11
= 0.06483,
m
31
=
a
31
a
11
= 0.70444,
Chapter-3 p.16/64
We replace a
22
, a
32
by
a
22
= a
22
m
21
a
12
,
a
32
= a
32
m
31
a
12
.
The resulting matrix is
_
_
0.96475 0.34419 1
.
.
. 2.8476
0 0.06312 0.93517
.
.
. 0.0389
0 0.05495 0.29556
.
.
. 0.91296
_
_
.
Chapter-3 p.17/64
The multiplier is
m
32
=
a
32
a
22
= 0.87056,
and replace a
33
by
a
33
= a
33
m
32
a
23
.
_
_
0.96475 0.34419 1
.
.
. 2.8476
0 0.06312 0.93517
.
.
. 0.0389
0 0 0.51856
.
.
. 0.94682
_
_
.
Chapter-3 p.18/64
Using the back substitution, we nd
x
3
= 1.82586,
x
2
= 26.4352,
x
1
= 10.4903.
Chapter-3 p.19/64
Example 4
algebraic system
0.995x
1
+ 1.54x
2
+ 4.51x
3
= 43.1,
0.995x
1
+ 2.16x
2
+ 1.19x
3
= 31.6,
0.298x
1
+ 0.577x
2
+ 1.42x
3
= 16.2.
Solution. Plz try.
Chapter-3 p.20/64
Crouts Method
Given any
AX = b,
we express the matrix A as a decomposition of a
lower triangular matrix L and an upper triangular
matrix U such that
LU = A.
The different factorizations may be viewed as
resulting from different choices for the diagonal
elements of either L or U.
Chapter-3 p.21/64
The two most common choices for the diagonal
entries are
l
ii
= 1, for each i = 1, 2, 3, , n,
u
ii
= 1, for each i = 1, 2, 3, , n.
If l
ii
= 1, for each i = 1, 2, 3, , n then this
method is known as the Doolittle decomposition
method.
If u
ii
method is known as the Crout decomposition
method.
Chapter-3 p.22/64
Crouts Method
Let A be an n n matrix. To obtain the Crouts
decomposition of A, we must determine the
entries l
ij
(i j) and u
ij
(i < j) such that
Chapter-3 p.23/64
_
_
l
11
l
21
l
22
l
31
l
32
l
33
.
.
.
.
.
.
.
.
.
l
n1
l
n2
l
n3
.
.
. l
nn
_
_
_
_
1 u
12
u
13
.
.
. u
1n
1 u
23
.
.
. u
2n
1
.
.
. u
3n
.
.
.
.
.
.
1
_
_
=
_
_
a
11
a
12
a
13
.
.
. a
1n
a
21
a
22
a
23
.
.
. a
2n
.
.
.
.
.
.
a
n1
a
n2
a
n3
.
.
. a
nn
_
_
.
Chapter-3 p.24/64
The product of the ith row of L (for
i = 1, 2, 3, , n) with the rst column of U is
simply the element l
i1
.
The value is equated to a
i1
, that is
l
i1
= a
i1
, i = 1, 2, 3, , n.
These equations determine the rst column of L.
Chapter-3 p.25/64
Now multiply the rst row of L with the jth
column of U (for j = 2, 3, 4, , n) and equating
to a
1j
produces the equation
l
11
u
1j
= a
1j
,
this implies
u
1j
=
a
1j
l
11
.
Chapter-3 p.26/64
This determining the rst row of U.
Similarly, we compute one more column of L and
one more row of U.
In particular, the kth column of L and the kth row
of U are determined as
l
ik
= a
ik
k1
j=1
l
ij
u
jk
, i = k, k + 1, k + 2, , n,
u
kj
=
1
l
kk
_
_
a
kj
k1
i=1
l
ki
u
ij
_
_
, j = k + 1, k + 2, , n.
Chapter-3 p.27/64
Once the coefcient matrix has been converted
to its LU decomposition, then
AX = b LUX = b.
Let UX = b
1
then Lb
1
= b. This gives
_
_
l
11
l
21
l
22
l
31
l
32
l
33
.
.
.
.
.
.
.
.
.
l
n1
l
n2
l
n3
.
.
. l
nn
_
_
_
_
b
1
1
b
1
2
.
.
.
b
1
n
_
_
=
_
_
b
1
b
2
.
.
.
b
n
_
_
.
Chapter-3 p.28/64
This gives
b
1
1
=
b
1
l
11
,
and
b
1
i
=
1
l
ii
_
_
b
i
i1
k=1
l
ik
b
1
k
_
_
, i = 2, 3, 4, cdots, n.
Now using back substitution for
UX = b
1
,
Chapter-3 p.29/64
we get X as
x
n
= b
1
n
,
x
j
= b
1
j

n
k=j+1
u
jk
x
k
, j = n 1, n 2, , 1.
Chapter-3 p.30/64
Example 5
Solve the following system of equations by
Crouts method
x
1
+ 2x
2
+ x
3
= 2,
2x
1
+ x
2
10x
3
= 4,
2x
1
+ 3x
2
x
3
= 2.
Here, the coefcient matrix is
A =
_
_
1 2 1
2 1 10
2 3 1
_
_
, b =
_
_
2
4
2
_
_
.
Chapter-3 p.31/64
We express A into as follows
_
_
l
11
0 0
l
21
l
22
0
l
31
l
32
l
33
_
_
_
_
1 u
12
u
13
0 1 u
23
0 0 1
_
_
=
_
_
1 2 1
2 1 10
2 3 1
_
_
.
Chapter-3 p.32/64
This gives
l
11
= 1, l
21
= 2, l
31
= 2,
and
u
12
=
2
l
11
= 2, u
13
= 1.
Similarly, we nd
l
22
= 3, l
32
= 1, l
33
= 1,
and
u
23
= 4.
Chapter-3 p.33/64
Therefore,
L =
_
_
1 0 0
2 3 0
2 1 1
_
_
,
and
U =
_
_
1 2 1
0 1 4
0 0 1
_
_
.
Chapter-3 p.34/64
We need to compute
_
_
1 0 0
2 3 0
2 1 1
_
_
_
_
b
1
1
b
1
2
b
1
3
_
_
=
_
_
2
4
2
_
_
.
This gives
b
1
1
= 2, b
1
2
= 0, b
1
3
= 2.
Chapter-3 p.35/64
Finally, we compute
_
_
1 2 1
0 1 4
0 0 1
_
_
_
_
x
1
x
2
x
3
_
_
=
_
_
2
0
2
_
_
.
This gives
x
3
= 2, x
2
= 8, x
1
= 12.
Chapter-3 p.36/64
Chapter-3
Dr. P. Dhanumjaya
Chapter-3 p.1/96
Example 1
0.003000x
1
+ 59.14x
2
= 59.17,
5.291x
1
6.130x
2
= 46.78,
has the exact solution x
1
= 10.00 and x
2
= 1.00.
We now nd the solution using Gauss elimination
method and four-digit arithmetic with rounding.
Chapter-3 p.2/96
We form an augmented matrix, we obtain
_
_
0.003 59.14
.
.
. 59.17
5.291 6.130
.
.
. 46.78
_
_ .
The multipliers are
m
i1
=
a
i1
a
11
, i = 2, 3, 4, ,
This gives
m
21
=
a
21
a
11
=
5.291
0.003
= 1763.66 1764.
Chapter-3 p.3/96
We now replace
a
ij
= a
ij
m
i1
a
1j
, i, j = 2, 3,
and
a
i1
= 0, i = 2, 3, 4,
We have
_
_
0.003 59.14
.
.
. 59.17
(1764) 104300
.
.
. 104400
_
_ .
Chapter-3 p.4/96
x
2
1.001, and x
1
10.
This shows that a small value of pivot, any error
in the numerator can be dramatically increased.
Chapter-3 p.5/96
Partial Pivoting
_
_
0.003 59.14
.
.
. 59.17
5.291 6.130
.
.
. 46.78
_
_ .
We take
max[a
11
[, [a
21
[ = 5.291.
We replace r
1
by r
2
, we get
_
_
5.291 6.130
.
.
. 46.78
0.003 59.14
.
.
. 59.17
_
_ .
Chapter-3 p.6/96
The multiplier is
m
21
=
a
21
a
11
= 0.000567.
We replace
a
22
= a
22
m
21
a
12
,
we get
_
_
5.291 6.130
.
.
. 46.78
(0.000567) 59.14
.
.
. 59.14
_
_ .
Chapter-3 p.7/96
This gives
x
2
= 1.0, and x
1
= 10.0
Chapter-3 p.8/96
Example 2
30.00x
1
+ 591400x
2
= 591700,
5.291x
1
6.130x
2
= 46.78,
_
_
30.0 591400
.
.
. 591700
5.291 6.130
.
.
. 46.78
_
_ .
Chapter-3 p.9/96
The maximum value in the rst column is 30.0
The multiplier is
m
21
=
a
21
a
11
=
5.291
30.0
= 0.1764.
We replace
a
22
= a
22
m
21
a
12
.
Chapter-3 p.10/96
We get
_
_
30.0 591400
.
.
. 591700
(0.1764) 104300
.
.
. 104400
_
_ .
x
2
1.001, and x
1
10.00
This shows that in a row if the coefcients vary
too much, then we get less accuracy.
To avoid this, we use scaling.
Chapter-3 p.11/96
Scaled Partial
Pivoting
Scaling
d
i
= max
1jn
[a
ij
[, i = 1, 2, 3, , n
then divide ith row by d
i
for i = 1, 2, 3, , n.
Choose numerically largest element of the
concerned column as pivotal element.
This is known as scaled partial pivoting.
Chapter-3 p.12/96
Note 1
Note that
scaling is useful only when size of the
coefcients vary too much.
Chapter-3 p.13/96
Example 3
algebraic system
6.684x
1
+ 2.925x
2
+ 9.835x
3
= 10.75,
5.543x
1
+ 5.953x
2
+ 88.63x
3
= 19.81,
8.375x
1
+ 2.988x
2
+ 8.681x
3
= 24.72.
Chapter-3 p.14/96
Solution
_
_
6.684 2.925 9.835
.
.
. 10.75
5.543 5.953 88.63
.
.
. 19.81
8.375 2.988 8.681
.
.
. 24.72
_
_
.
The scaling factors are
d
1
= 9.835, d
2
= 88.63, d
3
= 0.681.
Chapter-3 p.15/96
After scaled partial pivoting, the resulting matrix
is
_
_
0.96475 0.34419 1
.
.
. 2.8476
0.06254 0.06717 1
.
.
. 0.22351
0.67961 0.29741 1
.
.
. 1.0930
_
_
.
The multipliers are
m
21
=
a
21
a
11
= 0.06483,
m
31
=
a
31
a
11
= 0.70444,
Chapter-3 p.16/96
We replace a
22
, a
32
by
a
22
= a
22
m
21
a
12
,
a
32
= a
32
m
31
a
12
.
_
_
0.96475 0.34419 1
.
.
. 2.8476
0 0.06312 0.93517
.
.
. 0.0389
0 0.05495 0.29556
.
.
. 0.91296
_
_
.
Chapter-3 p.17/96
The multiplier is
m
32
=
a
32
a
22
= 0.87056,
and replace a
33
by
a
33
= a
33
m
32
a
23
.
_
_
0.96475 0.34419 1
.
.
. 2.8476
0 0.06312 0.93517
.
.
. 0.0389
0 0 0.51856
.
.
. 0.94682
_
_
.
Chapter-3 p.18/96
Using the back substitution, we nd
x
3
= 1.82586,
x
2
= 26.4352,
x
1
= 10.4903.
Chapter-3 p.19/96
Example 4
algebraic system
0.995x
1
+ 1.54x
2
+ 4.51x
3
= 43.1,
0.995x
1
+ 2.16x
2
+ 1.19x
3
= 31.6,
0.298x
1
+ 0.577x
2
+ 1.42x
3
= 16.2.
Solution. Plz try.
Chapter-3 p.20/96
Crouts Method
Given any
AX = b,
we express the matrix A as a decomposition of a
lower triangular matrix L and an upper triangular
matrix U such that
LU = A.
The different factorizations may be viewed as
resulting from different choices for the diagonal
elements of either L or U.
Chapter-3 p.21/96
The two most common choices for the diagonal
entries are
l
ii
= 1, for each i = 1, 2, 3, , n,
u
ii
= 1, for each i = 1, 2, 3, , n.
If l
ii
method is known as the Doolittle decomposition
method.
If u
ii
method is known as the Crout decomposition
method.
Chapter-3 p.22/96
Crouts Method
Let A be an n n matrix. To obtain the Crouts
decomposition of A, we must determine the
entries l
ij
(i j) and u
ij
(i < j) such that
Chapter-3 p.23/96
_
_
l
11
l
21
l
22
l
31
l
32
l
33
.
.
.
.
.
.
.
.
.
l
n1
l
n2
l
n3
.
.
. l
nn
_
_
_
_
1 u
12
u
13
.
.
. u
1n
1 u
23
.
.
. u
2n
1
.
.
. u
3n
.
.
.
.
.
.
1
_
_
=
_
_
a
11
a
12
a
13
.
.
. a
1n
a
21
a
22
a
23
.
.
. a
2n
.
.
.
.
.
.
a
n1
a
n2
a
n3
.
.
. a
nn
_
_
.
Chapter-3 p.24/96
The product of the ith row of L (for
i = 1, 2, 3, , n) with the rst column of U is
simply the element l
i1
.
The value is equated to a
i1
, that is
l
i1
= a
i1
, i = 1, 2, 3, , n.
These equations determine the rst column of L.
Chapter-3 p.25/96
Now multiply the rst row of L with the jth
column of U (for j = 2, 3, 4, , n) and equating
to a
1j
produces the equation
l
11
u
1j
= a
1j
,
this implies
u
1j
=
a
1j
l
11
.
Chapter-3 p.26/96
This determining the rst row of U.
Similarly, we compute one more column of L and
one more row of U.
In particular, the kth column of L and the kth row
of U are determined as
l
ik
= a
ik
k1
j=1
l
ij
u
jk
, i = k, k + 1, k + 2, , n,
u
kj
=
1
l
kk
_
_
a
kj
k1
i=1
l
ki
u
ij
_
_
, j = k + 1, k + 2, , n.
Chapter-3 p.27/96
Once the coefcient matrix has been converted
to its LU decomposition, then
AX = b LUX = b.
Let UX = b
1
then Lb
1
= b. This gives
_
_
l
11
l
21
l
22
l
31
l
32
l
33
.
.
.
.
.
.
.
.
.
l
n1
l
n2
l
n3
.
.
. l
nn
_
_
_
_
b
1
1
b
1
2
.
.
.
b
1
n
_
_
=
_
_
b
1
b
2
.
.
.
b
n
_
_
.
Chapter-3 p.28/96
This gives
b
1
1
=
b
1
l
11
,
and
b
1
i
=
1
l
ii
_
_
b
i
i1
k=1
l
ik
b
1
k
_
_
, i = 2, 3, 4, cdots, n.
Now using back substitution for
UX = b
1
,
Chapter-3 p.29/96
we get X as
x
n
= b
1
n
,
x
j
= b
1
j

n
k=j+1
u
jk
x
k
, j = n 1, n 2, , 1.
Chapter-3 p.30/96
Example 5
Solve the following system of equations by
Crouts method
x
1
+ 2x
2
+ x
3
= 2,
2x
1
+ x
2
10x
3
= 4,
2x
1
+ 3x
2
x
3
= 2.
Here, the coefcient matrix is
A =
_
_
1 2 1
2 1 10
2 3 1
_
_
, b =
_
_
2
4
2
_
_
.
Chapter-3 p.31/96
We express A into as follows
_
_
l
11
0 0
l
21
l
22
0
l
31
l
32
l
33
_
_
_
_
1 u
12
u
13
0 1 u
23
0 0 1
_
_
=
_
_
1 2 1
2 1 10
2 3 1
_
_
.
Chapter-3 p.32/96
This gives
l
11
= 1, l
21
= 2, l
31
= 2,
and
u
12
=
2
l
11
= 2, u
13
= 1.
Similarly, we nd
l
22
= 3, l
32
= 1, l
33
= 1,
and
u
23
= 4.
Chapter-3 p.33/96
Therefore,
L =
_
_
1 0 0
2 3 0
2 1 1
_
_
,
and
U =
_
_
1 2 1
0 1 4
0 0 1
_
_
.
Chapter-3 p.34/96
We need to compute
_
_
1 0 0
2 3 0
2 1 1
_
_
_
_
b
1
1
b
1
2
b
1
3
_
_
=
_
_
2
4
2
_
_
.
This gives
b
1
1
= 2, b
1
2
= 0, b
1
3
= 2.
Chapter-3 p.35/96
Finally, we compute
_
_
1 2 1
0 1 4
0 0 1
_
_
_
_
x
1
x
2
x
3
_
_
=
_
_
2
0
2
_
_
.
This gives
x
3
= 2, x
2
= 8, x
1
= 12.
Chapter-3 p.36/96
Vector and Matrix
Norms
To measure the errors introduced during the
solution of a linear system, it will be necessary to
have a means for quantifying the size of a matrix.
This is done using matrix norms.
Chapter-3 p.37/96
Vector Norms
The function | | : 1
n
1 is called a vector
norm if for all X, Y 1
n
and all 1, the
following properties hold:
|X| 0,
|X| = 0, iff X = 0,
|X| = [[ |X| and
|X + Y | |X| +|Y |.
Chapter-3 p.38/96
There are innitely many vector norms that can
be constructed.
We will restrict our attention to the commonly
used norms:
| |
1
(l
1
norm),
| |
2
(l
2
norm,
| |
(l
norm).
Chapter-3 p.39/96
Let X 1
n
, the l
1
-norm of X denoted by |X|
1
, is
dened by
|X|
1
=
n
i=1
[x
i
[.
The l
2
-norm or Euclidean norm of X is denoted
by |X|
2
, is dened by
|X|
2
=
_
_
n
i=1
x
2
i
_
_
1
2
.
Chapter-3 p.40/96
The l
-norm or maximum norm is denoted by

|X|
is dened by
|X|
= max
1in
[x
i
[.
Chapter-3 p.41/96
Example
Consider the three vectors
X
1
= [1 2 3]
T
,
X
2
= [2 0 1 2]
T
,
X
3
= [0 1 4 2 1]
T
.
The maximum norm of each of these vectors is
computed as follows:
|X
1
|
= max[1[, [ 2[, [3[ = 3,

|X
2
|
= max[2[, [0[, [ 1[, [2[ = 2,

|X
3
|
= max[0[, [1[, [ 4[, [2[, [ 1[ = 4.

Chapter-3 p.42/96
The l
2
-norm or Euclidean norm of each vector is
|X
1
|
2
=
_
1
2
+ (2)
2
+ 3
2
=
14 3.74,
|X
2
|
2
=
_
2
2
+ 0
2
+ (1)
2
+ 2
2
=
9 = 3,
|X
3
|
2
=
_
0
2
+ 1
2
+ (4)
2
+ 2
2
+ (1)
2
=
22 4.69.
Chapter-3 p.43/96
Matrix Norms
A matrix norm is a function | | : 1
nn
1 that
for all A, B 1
nn
and all 1 satises
|A| 0,
|A| = 0, iff A = 0,
|A| = [[ |A|,
|A + B| |A| +|B| and
|AB| |A| |B|.
Chapter-3 p.44/96
Let A be an n n matrix then
The l
1
-norm denoted by |A|
1
and dened by
|A|
1
= max
1jn
n
i=1
[a
ij
[.
The l
-norm or maximum norm denoted by

|A|
and dened by
|A|
= max
1in
n
j=1
[a
ij
[.
Chapter-3 p.45/96
The l
2
-norm or Spectral norm denoted by
|A|
2
and dened by
|A|
2
=
_
(A
T
A),
where is the spectral radius and denoted by
(A) = max
i
[
i
[.
Chapter-3 p.46/96
Example
We calculate both the l
2
and the l
norm of the
matrices
A
1
=
_
_
1 2
4 3
_
_
and
A
2
=
_
_
1 0 2
0 1 1
1 1 1
_
_
.
Chapter-3 p.47/96
|A
1
|
= max[1[ +[ 2[, [4[ +[3[ = 7,

and
|A
2
|
= max[1[ +[0[ +[2[, [0[ +[1[ +[ 1[,

[ 1[ +[1[ +[1[ = 3.
To determine the l
2
-norm, we rst compute
A
T
1
A
1
=
_
_
1 4
2 3
_
_
_
_
1 2
4 3
_
_
=
_
_
18 10
10 13
_
_
.
Chapter-3 p.48/96
The eigenvalues are
=
1
2
_
31 5
17
_
.
Hence,
(A
T
1
A
1
) =
1
2
_
31 + 5
17
_
,
and
|A
1
|
2
=
_
1
2
_
31 + 5
17
_
5.08013.
Chapter-3 p.49/96
Similarly, we compute
(A
T
2
A
2
) = 6.24914,
and
|A
2
|
2
=
6.24914 2.49983.
Chapter-3 p.50/96
Error Estimates &
Condition Number
We now address the following questions:
1. Howmuch error can we expect when solving a
system of linear equations using Gaussian
elimination.
2. How does the error depend on the properties
of the coefcient matrix and the right-hand
side vector?.
3. When A and b are known only approximately,
how is the error in the solution related to the
errors in the data?.
Chapter-3 p.51/96
Error estimates
Suppose

X is an approximate solution to the
linear system
AX = b,
whose exact solution is the vector X.
In practice, the exact solution to the system is
unknown, so the error in

X
e =

X X,
cannot be directly computed.
Chapter-3 p.52/96
However, the residual vector
r = A

X b,
can be easily computed.
Chapter-3 p.53/96
Note
The residual measures the amount by which the
approximate solution fails to satisfy the linear
system.
Chapter-3 p.54/96
When r = 0, this implies

X is the exact solution,
so
e = 0.
It seems reasonable to expect that whenever |r|
is small, |e| will be small as well.
Unfortunately, this need not always be the case.
Chapter-3 p.55/96
Example
Consider the linear system
_
_
1 2
0.99 1.99
_
_
_
_
x
1
x
2
_
_
=
_
_
1
1
_
_
,
has X = [1 1]
T
as its exact solution.
Chapter-3 p.56/96
Let the vector

X = [1 0]
T
.
The error
e = [2 1]
T
and
|e|
= 2.
However, the residual associated with

X is
r = A

X b =
_
_
0
0.01
_
_
.
Chapter-3 p.57/96
Then
|r|
= 0.01.
Thus, the error is 200 times larger than the
residual.
Chapter-3 p.58/96
Condition Number
The condition number of a matrix A is dened as
/(/) = |A| |A
1
|.
Chapter-3 p.59/96
Theorem
Let A be a nonsingular matrix,

X be an
approximate solution to the linear system
AX = b, r = A

X b and e =

X X. Then for
any natural matrix norm | |,
1
|A|
|r| |e| |A
1
| |r|,
and
1
|A| |A
1
|
|r|
|b|

|e|
|X|
|A| |A
1
|
|r|
|b|
,
provided X ,= 0 and b ,= 0.
Chapter-3 p.60/96
Proof
We calculate
r = A

X b = A

X AX,
= A(

X X) = Ae.
Equivalently,
e = A
1
r.
Now let | | be any natural matrix norm then
|e| = |A
1
r| |A
1
| |r|.
Chapter-3 p.61/96
From r = Ae, we obtain
|r| = |Ae| |A| |e|,
or
|e|
|r|
|A|
.
Thus
1
|A|
|r| |e| |A
1
| |r|. (1)
Chapter-3 p.62/96
Suppose the X ,= 0 and b ,= 0.
Taking the norm on both sides of AX = b yields
|b| = |AX| |A| |X|.
This implies
1
|X|

|A|
|b|
.
Chapter-3 p.63/96
Similarly, from X = A
1
b, we obtain
1
|A
1
| |b|

1
|X|
.
Then
1
|A
1
| |b|

1
|X|

|A|
|b|
. (2)
Finally, combining (1) and (2) yields,
1
|A| |A
1
|
|r|
|b|

|e|
|X|
|A| |A
1
|
|r|
|b|
.
Chapter-3 p.64/96
This provide lower and upper bounds on the
relative error in an approximate solution to
AX = b,
interms of the relative residual
r
b
and the norms
of A and its inverse.
When /(A) is small (i.e., |A| |A
1
| 1), then
|r|
|b|

|e|
|X|

|r|
|b|
.
Chapter-3 p.65/96
This shows that the relative residual provides a
good measure for the error in the approximate
solution.
If /(A) is large, the relative residual can be a
very poor indicator of the accuracy of the
approximate solution.
Chapter-3 p.66/96
Example
Let
A =
_
_
1 2
0.99 1.99
_
_
and
A
1
=
_
_
199 200
99 100
_
_
.
Then
|A|
= 3, |A
1
|
= 399,
and
/(A) = |A| |A
1
| = 3 399 = 1197.
Chapter-3 p.67/96
Therefore, the relative error in an approximate
solution to a system with A as its coefcient
matrix can be as small as
1
1197
times or as large
as 1197 times the relative residual.
Chapter-3 p.68/96
Suppose the entries in A and b are known only
approximately.
How does the error in the computed solution
depend on the errors in A and b?.
Consider the original system of equations
AX = b.
Assume the actual set of equations being solved
is
A

X = b.
Chapter-3 p.69/96
Let
A =

A + E,
where E is the error matrix. Then
_

A + E
_
X = b,
where X represents true solution of the system.
We now estimate the error X

X.
Chapter-3 p.70/96
Then
X = A
1
b = A
1
(

A

X) = A
1
(A E)

X,
=

X A
1
E

X.
We have
X

X = A
1
E

X.
Taking norm on both sides, we obtain
|X

X| |A
1
| |E| |
X| = |A
1
| |A|
|E|
|A|
|
X|.
Chapter-3 p.71/96
Hence
|X

X|
|
X|
/(A)
|E|
|A|
.
This shows that the relative error in the
computed solution can be as large as the relative
error in the coefcients of A multiplied by the
condition number.
Chapter-3 p.72/96
Iterative Methods
It is natural to ask why we would want or even
need to develop iterative techniques.
For systems of small dimension, there is no
need. Direct techniques will perform very
efciently.
However, linear systems arising from practical
applications will frequently be quite large.
The coefcient matrices associated with these
systems also tend to be sparse.
Chapter-3 p.73/96
For systems with large, sparse coefcient
matrices, direct techniques are often less
efcient than iterative techniques.
Basic iterative techniques for the system of linear
system of equations are analogous to the
xed-point techniques.
The original linear system
AX = b,
which can be interpreted as the root nding
problem.
Chapter-3 p.74/96
Find the n-vector X such that
AX b = 0,
is rst converted to the xed point problem.
Find the n-vector X such that
X = BX + c,
for some matrix B and vector c.
Chapter-3 p.75/96
Starting from some initial approximation to the
solution of the xed point problem X
(0)
, a
sequence of vectors X
(k)
is computed
according to the rule
X
(k+1)
= BX
(k)
+ c,
where the matrix B is called the iteration matrix.
Chapter-3 p.76/96
Example
Consider the system of equations
5x
1
+ x
2
+ 2x
3
= 10,
3x
1
+ 9x
2
+ 4x
3
= 14,
x
1
+ 2x
2
7x
3
= 33.
Now, we express the given system of equations
AX = b in the equivalent form
Chapter-3 p.77/96
X = BX + c,
as
_
_
x
1
x
2
x
3
_
_
=
_
_
0
1
5

2
5
3
9
0
4
9
1
7
2
7
0
_
_
_
_
x
1
x
2
x
3
_
_
+
_
_
2
14
9
33
7
_
_
.
Chapter-3 p.78/96
Strictly Diagonally
Dominant Matrices
An n n matrix A is strictly diagonally row
(column) dominant if for each row (column), the
magnitude of the diagonal element is strictly
larger than sum of the magnitudes of the other
elements on the row (column), that is, if for each
i (j),
[a
ii
[ >
j=1, j=i
[a
ij
[,
_
_
[a
jj
[ >
i=1, i=j
[a
ij
[
_
_
.
Chapter-3 p.79/96
Example
Let
A =
_
_
3 1 1
2 6 3
9 7 20
_
_
.
The matrix is strictly diagonally row dominant.
Since
[3[ = 3 > 2 = [ 1[ +[1[,
[ 6[ = 6 > 5 = [2[ +[3[,
[ 20[ = 20 > 16 = [ 9[ +[7[.
Chapter-3 p.80/96
Example
Let
A =
_
_
3 2 2
2 6 3
9 7 20
_
_
.
The matrix is not strictly diagonally dominant.
Since
[3[ = 3 < 4 = [ 2[ +[2[.
Chapter-3 p.81/96
Symmetric Positive
Denite Matrices
A matrix A is Symmetric Positive Denite if it is
symmetric and
X
T
AX > 0,
for any nonzero vector X.
Chapter-3 p.82/96
Example
Let
A =
_
_
3 1 1
1 4 2
1 2 5
_
_
.
Let X = [x
1
x
2
x
3
]
T
. Then
Chapter-3 p.83/96
X
T
AX = 3x
2
1
+ 4x
2
2
+ 5x
2
3
+ 2x
1
x
2
+ 4x
2
x
3
2x
1
x
3
,
= x
2
1
+ x
2
2
+ x
2
3
+ (x
1
+ x
2
)
2
+ (x
1
x
3
)
2
+ 2(x
2
+ x
3
)
2
,
which is clearly greater than zero for any
non-zero X. Therefore, A is symmetric positive
denite.
Chapter-3 p.84/96
Example
Consider the matrix
A =
_
_
3 2 1
2 3 2
1 2 3
_
_
,
is not symmetric positive denite. Since
X
T
AX = 2[(x
1
x
2
)
2
+ (x
1
x
3
)
2
+ (x
2
x
3
)
2
],
which equals zero for any vector whose
components satisfy x
1
= x
2
= x
3
.
Chapter-3 p.85/96
Theorem
Let AX = b can be written as X = BX + c with
some norm of B, |B| < 1, then X = BX + c has
a unique solution.
Further, the sequence X
(m)
generated by
X
(m+1)
= BX
(m)
+ c,
starting with some initial X
(0)
will converge to the
solution vector .
Proof. We prove by contradiction.
Chapter-3 p.86/96
We consider the homogeneous system X = BX.
Let X ,= 0 is the solution. Then
|X| = |BX| |B| |X|.
Since |B| < 1 implies that |X| < |X| which
contradicts that X ,= 0.
This implies that X = 0.
Chapter-3 p.87/96
This shows that the nonhomogeneous system of
equations
X = BX + c,
has an unique solution.
Let the unique solution X = . Then
= B + c.
Now, we consider the iteration scheme
X
(m+1)
= BX
(m)
+ c, m = 0, 1, 2, ,
Chapter-3 p.88/96
Then
X
(m+1)
= BX
(m)
B = B
_
X
(m)
_
.
Now taking norm on both sides, we obtain
|X
(m+1)
| = |B
_
X
(m)
_
|,
|B| |X
(m)
|,
|B|
2
|X
(m1)
|,
.
.
.
|B|
(m+1)
|X
(0)
|.
Chapter-3 p.89/96
Taking limit on both sides as m , we obtain
lim
m
|X
(m+1)
| = 0.
Since |B| < 1, this implies that
lim
m
X
(m+1)
= .
Hence, the sequence X
(m)
converges to the
solution .
Chapter-3 p.90/96
Jacobi Method
The Jacobi method to solve AX = b is
converges, if the coefcient matrix A is strictly
diagonally row dominant.
Proof. The system of equations in matrix form
_
_
a
11
a
12
a
1n
a
21
a
22
a
2n
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
a
n1
a
n2
a
nn
_
_
_
_
x
1
x
2
.
.
.
.
.
.
x
n
_
_
=
_
_
b
1
b
2
.
.
.
.
.
.
b
n
_
_
.
Chapter-3 p.91/96
We express in the iteration form: from the rst
equation, we get
x
1
= 0x
1
a
12
a
11
x
2
a
13
a
11
x
3

a
1n
a
11
x
n
+
b
1
a
11
.
From the second equation, we get
x
2
=
a
21
a
22
x
1
0x
2
a
23
a
22
x
3

a
2n
a
22
x
n
+
b
2
a
22
.
Chapter-3 p.92/96
Similarly from the ith equation, we obtain
x
i
=
a
i1
a
ii
x
1
a
i2
a
ii
x
2
a
i3
a
ii
x
3

a
in
a
ii
x
n
+
b
i
a
ii
.
Chapter-3 p.93/96
Now, we express in matrix form
X =
_
_
0
a
12
a
11

a
1n
a
11
a
21
a
22
0
a
2n
a
22
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
a
n1
a
nn
a
n2
a
nn
0
_
_
X +
_
_
b
1
a
11
b
2
a
22
.
.
.
.
.
.
b
n
a
nn
_
_
.
That is
X = BX + c.
Chapter-3 p.94/96
The Jacobi iteration scheme is
X
(m+1)
= BX
(m)
+ c,
convergent provided |B| < 1. Now
|B|
= max
1in
n
j=1, j=i
[a
ij
[
[a
ii
[
< 1,
gives
n
j=1, j=i
[a
ij
[ < [a
ii
[, i = 1, 2, 3, , n.
Chapter-3 p.95/96
This shows that A is strictly diagonally row
dominant.
Thus, Jacobi method converges if A is strictly
diagonally row dominant.
Chapter-3 p.96/96
Chapter 7 Eigenvalues and Eigenvectors
Dr. P. Dhanumjaya
Chapter 7 Eigenvalues and Eigenvectors p.1/14
Power Method
This method is used to nd an eigenvalue of
greatest modulus of a matrix A and its
corresponding eigenvector.
Let A be n n real matrix. Assume that A has n
linearly independent eigenvectors x
1
, x
2
, . . . , x
n
corresponding to the eigenvalues
1
,
2
, . . . ,
n
.
Case 1
Let |
1
|> |
2
| |
3
| .. |
n
| i.e.
dominant eigenvalue is not repeated and
1
is
real. Let
Y
0
=
1
x
1
+
2
x
2
+ . . . +
n
x
n
.
Generate the sequence {Y
k
}, where
Y
k
= AY
k1
= A
k
Y
0
.
Therefore,
Y
k
= A
k
Y
0
=
1
k
1
x
1
+
2
k
2
x
2
+ . . . +
n
k
n
x
n
.
Let x
i
= (x
i1
, x
i2
, . . . , x
in
).
Let Y
ki
denote the ith component of y
k
then
Y
ki
=
1
k
1
x
1i
+
2
k
2
x
2i
+ . . . +
n
k
n
x
ni
.
Consider
Y
k+1,i
Y
k,i
=

1
k+1
1
x
1i
+
2
k+1
2
x
2i
+ . . . +
n
k+1
n
x
ni
k
1
x
1i
+
2
k
2
x
2i
+ . . . +
n
k
n
x
ni
We can take
1
= 0 and x
1i
= 0.
Y
k+1,i
Y
k,i
=
1
{
1 + d
2
(
1
)
k+1
+ . . . + d
n
(
1
)
k+1
1 + d
2
(
1
)
k
+ . . . + d
n
(
1
)
k
}
where d
p
=

p
x
pi
1
x
1i
for p = 2, 3, . . . , n.
Therefore
limk
Y
k+1,i
Y
k,i
=
1
.
Therefore we can take
1
=
Y
k+1,i
Y
k,i
for sufciently
large k.
Also
Y
k
=
k
1
{
1
x
1
+
2
(
1
)
k
x
2
+ . . . +
n
(
1
)
k
x
n
}.
This shows that for large k, Y
k
is proportional to
x
1
, which is an eigenvector corresponding to
1
.
However, if |
1
|< 1, || Y
k
||0 as k ,
if |
1
|> 1, || Y
k
||as k .
To avoid this in actual computation we do scaling.
Let
0
=
1
x
1
+
2
x
2
+ . . . +
n
x
n
and Z
1
= A
0
.
Dene
1
= max
r
| (Z
1
)
r
| and
1
=
Z
1
1
.
In general, Z
k+1
= A
k
and
k+1
= max
r
| (Z
k+1
)
r
|. Therefore,
Z
k+1
= A
k
=
1
k
.
k1
. . . .
1
A
k+1
0
.
(Z
k+1
)
i
(
k
)
i
=
(A
k+1
0
)
i
(A
k
0
)
i
=

1
k+1
1
x
1i
+
2
k+1
2
x
2i
+...+
n
k+1
n
x
ni
k
1
x
1i
+
2
k
2
x
2i
+...+
n
k
n
x
ni
Hence
(Z
k+1
)
i
(
k
)
i
1
as k , provided
1
= 0
and x
1i
= 0.
Also Z
k+1
= A
k
=
1
k
.
k1
....
1
A
k+1
0
=

k+1
1
k
.
k1
....
1
{
1
x
1
+
2
x
2
(
1
)
k+1
+. . .+
n
x
n
(
1
)
k+1
}.
Hence Z
k+1
is proportional to x
1
for large k.
Example
A =
2 2 1
4 8 1
1 2 0
Eigen values are = 1, 3, 6.

Suppose
0
= (1, 2, 4)
T
.
z
1
= A
0
= (10, 16, 5)
T
.
1
= 16.
1
= (.625, 1, .3125)
T
z
2
= A
1
= (2.9375, 5.1875, 2.625)
T
.
2
= 5.1875.
2
= (.56626, 1, .506)
T
z
3
= A
2
= (2.62652, 5.22896, 2.56626)
T
.
=
(z
3
)
2
(
2
)
2
= 5.22896 6.
Similarly z
3
or
3
is proportional to (.4, 1, .4)
T
which is the eigen vector corresponding to 6.
Case 2
If numerically largest eigen value
1
is of
multiplicity r > 1.
Suppose
1
= . . . =
r
, |
r
|> |
r+1
| . . . |
n
|.
Case 3
If two eigen values of maximum modulus are real
but of opposite sign.
Example
A =
1 2 2
2 1 2
2 2 1
Eigen values are = 3, 3, 3.

Eigenvalues and Eigenvectors
Dr. P. Dhanumjaya
Eigenvalues and Eigenvectors p.1/14
Jacobi Method
Jacobi method is used to nd the eigenvalues
and eigenvectors of a real and symmetric matrix.
A matrix A = [a
ij
] is called symmetric if A
T
= A.
Similarity Matrices
If A and B are n n matrices, then B is similar to
A if there exists a non-singular matrix P such that
B = PAP
1
.
We note that PAP
1
is called similarity
transformation.
Similarity transformation retains the same
eigenvalues but different eigenvectors.
Note 1
A real symmetric matrix is diagonalizable. That is
If A is any real symmetric matrix, then there
exists an orthogonal matrix P such that
P
T
AP = D (diagonal matrix),
or
Ap = PD.
We note that for orthogonal matrices, we have
P
T
= P
1
.
Elementary
Orthogonal Matrix
Let the matrix O
nn
such that
O
pp
= cos = O
qq
,
O
pq
= sin ,
O
qp
= sin 1 p < q n.
The remaining entries of O are same as that of
the unit matrix is called an elementary
orthogonal matrix.
The Jacobi method consists of obtaining the
matrix P as a product of a sequence of
elementary orthogonal matrices. i.e.,
P = O
1
O
2
O
n
.
In Jacobi method take elementary orthogonal
matrices O
1
, O
2
, , O
n
and generates the
sequence {A
r
} as follows:
Take
A
0
= A,
A
i
= O
T
i
A
i1
O
i
, i = 1, 2, 3, r
until A
(r)
becomes almost diagonal.
Eigenvalues of A can be approximated by the
diagonal entries of A
(r)
.
We now choose O
i
such that a selected
off-diagonal elements of A
(i)
becomes zero.
Suppose at some step, we want to annihilate the
pq element p < q of the current matrix A, where
pq element is numerically largest out of all
off-diagonal entries.
The required transformation matrix O has the
following form:
O
pp
= O
qq
= cos ,
O
pq
= sin , O
qp
= sin ,
and all remaining entries are same as that of
I
nn
, identity matrix.
Let
B = O
T
AO.
Then the elements of B are as follows:
b
pk
= a
pk
cos a
qk
sin ,
b
qk
= a
pk
sin + a
qk
sin ,
b
ip
= a
ip
cos a
iq
sin ,
b
iq
= a
ip
sin + a
iq
cos ,
and
b
ik
= a
ik
,
b
pp
= a
pp
cos
2
+ a
qq
sin
2
2a
pq
sin cos ,
b
qq
= a
pp
sin
2
+ a
qq
cos
2
+ 2a
pq
cos 2.
Note that B is also symmetric. Therefore, in
order to make b
pq
= 0, we must have
1
2
(a
pp
a
qq
) sin 2 + a
pq
cos 2 = 0.
This implies that
tan 2 =
2a
pq
(a
qq
a
pp
)
, if a
qq
= a
pp
,
and
=

4
, if a
qq
= a
pp
.
We now compute the values of sin and cos .
Let = 2a
pq
, = a
qq
a
pp
, and =
2
+
2
.
Now, if
, > 0, sin =

2
,
> 0, < 0, sin =

2
,
< 0, > 0, sin =
+
2
,
< 0, < 0, sin =
+
2
.
if = 0, then sin =
1
2
.
We can compute cos =
1 sin
2
.
We note that since
4
<

4
,
2
< sin
1
2
.
Example 1
Use Jacobi method to nd all eigenvalues and
corresponding eigenvectors of the given matrix
A =
2 2
2 3
2
2
2 1
.
Eigenvalues and
Eigenvectors
2
Eigenvalues & Eigenvectors
Find a scalar and its corresponding nonzero vector x
for a given square matrix A such that
(1)
If A is n x n matrix, then the value , for which the
equation (1) has non trivial solution is called an
eigenvalue of A and corresponding solution vector x is
called eigenvector of A.
Ax x =
3
Example 1
Eigenvector of a 22 Matrix
The vector is an eigenvector of
Corresponding to the eigenvalue =3,
since
1
2

=

x
3 0
8 1
A

=

3 0 1 3
3
8 1 2 6
A

= = =

x x
Examples
Characteristic polynomial
Example
Characteristic polynomial
Computing eigenvalues using characteristic
polynomial is not recommended because of
work in computing coefficients of characteristic polynomial
sensitivity of coefficients of characteristic polynomial
work in solving for roots of characteristic polynomial
Characteristic polynomial is powerful theoretical tool
but usually not useful computationally
Example
Power Iteration
10
The Power Method
Start with some random vector v, ||v||=1
Iterate v=(Av)/||Av||
The eigenvector with largest eigenvalue
tends to dominate
How fast?
Linear convergence, slowed down by close
eigenvalues
11
Power Method
Case 1: Let
(that is dominant eigenvalue is not repeated.)
Suppose A has
1
only eigenvalue of largest modulus
and that
1
is real.
Consider the iteration formula:
x
k+1
= Ax
k
where we start with some initial x
0
, so that:
x
k
= A
k
x
0
Then x
k
converges to the eigenvector x
1
corresponding the eigenvalue
1
.
| | ... | | | | | |
3 2 1 n
>
Example of Power Method
Consider the follow matrix A
=
1 0 0
1 2 0
0 1 4
A
Assume an arbitrary vector x
0
= { 1 1 1}
T
Multiply the matrix by the matrix [A] by {x}
1
3
5
1
1
1
1 0 0
1 2 0
0 1 4
Normalize the result of the product

2 . 0
6 . 0
1
5
1
3
5
0435 . 0
4783 . 0
2174 . 4
0435 . 0
217 . 0
1
1 0 0
1 2 0
0 1 4
0435 . 0
217 . 0
1
6 . 4
2 . 0
1
6 . 4
2 . 0
1
6 . 4
2 . 0
6 . 0
1
1 0 0
1 2 0
0 1 4
0183 . 0
1134 . 0
1
2174 . 4
0435 . 0
4783 . 0
2174 . 4
0103 . 0
2165 . 0
1134 . 4
0183 . 0
1134 . 0
1
1 0 0
1 2 0
0 1 4
0025 . 0
0526 . 0
1
1134 . 4
0103 . 0
2165 . 0
1134 . 4
As you continue to multiple each successive
vector = 4 and the vector u
k
={1 0 0}
T
Power method
The special advantage of the power method
is that the eigenvector corresponds to the
dominant eigenvalue and is generated at the
same time.
The disadvantage is that the method only
supplies one eigenvalue
17
Power Method:
Case 2: If numerically largest eigenvalue is multiple.
Suppose
Example:
(i) x
0
= [2,1]
T
, (ii) x
0
= [1,0]
T
, (iii) x
0
= [0,1]
T
1 2 1
... , | | | | ... | |
r r r n

+
= = = >
4 1 1
, eigenvalues are5, 5with eigenvector
1 6 1
A

=

Power Method:
18
. (1,1) r x eigenvecto and 5 eigenvalue to converges This
99128 . 0
1
, 9913 . 4 ,
9478 . 4
9913 . 4
9913 . 0
1
6 1 -
1 4
9913 . 0
1
, 79167 . 4 ,
75002 . 4
79167 . 4
79167 . 0
1
6 1 -
1 4
79167 . 0
1
, 8 . 4 ,
8 . 3
8 . 4
8 . 0
1
6 1 -
1 4
8 . 0
1
4
5
5
1
, 5 ,
4
5
1
2
6 1 -
1 4
,
6 1 -
1 4
A ,
1
2
T
4 4 4
3 3 3
2 2 2
1 1 1
0 1 0
= =
= =
= =
= =
= =
=
=
x z
x z
x z
x z
Ax z x
19
Power Method:
Case 3: If two eigenvalues of maximal modulus are real but of
opposite signs.
Example :
T T
) 3 , 1 ( x , 2 ; ) 1 , 1 ( x 2, ,
1 - 3
1 1
A = = = =
=
1
0
16
16
0
4 -
4
1 - 3
1 1
1 -
1
4
4 -
4
4
0
1 - 3
1 1
1
0
4
4
0
1 -
1
1 - 3
1 1
1 -
1
3
2
1 0
x
x
x x
Both the vectors are occurring
alternatively, which indicate that
eigenvalues are equal and
opposite in sign.
Power Method
20
= +
=
3 -
1
1
0
2
1 -
1
2
1
1
1
0
2
1 -
1
2
1
0
,
1 -
1
,
1
0
,
1 -
1
1 2
1 2
3 2 1 0
x x
x x
x x x x
3 -
1
2
6
2 -
3 -
1
1 - 3
1 1
1
1
2
2
2
1
1
1 - 3
1 1
21
Power Method
Case 4: Dominant eigenvalue is complex conjugate of each other
Example:
= =
= =
= =
= =
=
=
=
1
0.296 -
0.7
, 17329 . 6 ,
1
0.296 -
0.7
17329 . 6
6.17329
1.827 -
4.327
A
0.615437 -
1
0.36537
, 0588 . 3 ,
0.615437 -
1
0.36537
0588 . 3
1.8824 -
3.0588
1.1176
A
1
0.0588
1
, 25 . 4 ,
1
0.0588
1
25 . 4
4.25
0.25
4.25
A
0
1
0.75
, 4 ,
0
1
0.75
4
0
4
3
A
) 1 , 1 , 1 ( ,
5 - 2 3
4 1 1 -
2 - 2 3
A
4 4 3
3 3 2
2 2 1
1 1 0
0
x x
x x
x x
x x
x
T
22
Power Method:
Suppose numerically largest eigenvalue and
corresponding eigenvectors has been computed by
power method.
Now to compute eigenvalues and corresponding
eigenvectors next to dominant eigenvalue.
Method of exhaustion
23
Method of Exhaustion:
Let
1
: numerically largest eigenvalue and
u
1
= (x
11
,x
12
,,x
1n
) be the corresponding eigenvector
computed by power method.
To find
eigenvector v
1
=(y
11
,y
12
,,y
1n
) of A
T
corresponding to the
same eigenvalue
1
by power method.
Normalized u
1
and v
1
.
such that u
1
.v
1
=1
24
Method of Exhaustion:
Construct a matrix A
1
= A -
1
u
1
v
1
T
.
A
1
has the same eigenvalue and same eigenvector as that
of matrix A, excluding the first eigenvalue, in place of
which there is an eigenvalue equal to zero.
Apply the power method to A
1
to get next eigenvalue that
is second largest eigenvalue.
25
Method of Exhaustion: Example.
Find the second numerically largest eigenvalue and
corresponding eigenvector of the matrix
Normalized eigenvector u
1
= (2/3, 1/3, 2/3)
T
.
A is symmetric matrix i.e. A
T
= A therefore v
1
= u
1
and
u
1
.v
1
=1.
Find A
1
= A -
1
u
1
v
1
T
, and then apply power method.
= = =
=
2
1
2
, 6 For . 6 , 3 , 0 ,
4 0 2
0 2 2
2 2 3
1
u A
26
Power Method
= =
4/3 4/3 - 2/3 -
4/3 - 4/3 2/3
2/3 - 2/3 3 / 1
6
1 1 1
T
v u A A
= =
= =
= =
= =
=
1
1
5 . 0
, 3 ,
1
1
2 / 1
3
1
1
2 / 1
, 3 / 2 ,
1
1
2 / 1
3 / 2
. ) 1 , 1 , 1 (
2 1 1 1 2
1 1 0 1 1
0
x x A z
x x A z
x
T
27
Shifting of the eigenvalues:
Let A x = x ( x 0), where A = (a
ij
)
n x n
matrix.
This gives |A I| = 0 for x 0.
Further, let the matrix B = (b
ij
)
n x n
be such that
B = A I => A = B + I
where is the numerically largest eigenvalue of A (by PM).
Then A x = x gives B x = ( ) x for some x 0.
( ) is an eigenvalue of B corresponding to , eigenvalue of
A with the same eigenvector.
If = , 0 is the eigenvalue of B corresponding to eigenvalue
of A.
28
Shifting of the eigenvalues:
Therefore, we have B x = ( ) x and A x = x.
Thus, numerically largest eigenvalue ( ) of B give the
numerically smallest eigenvalue of A.
Numerically smallest eigenvalue of A is found by choosing
such that ( ) becomes numerically largest.
This fact may be used to find the other eigenvalue and
corresponding eigenvector.
29
Inverse Power Method:
If 0 is numerically smallest eigenvalue of the matrix
A, then
A x = x, A
-1
A x = A
-1
x A
-1
x = (1/) x if 0.
(1/) would be the numerically largest eigenvalue of A
-1
.
To find the inverse of A, Gauss-elimination method will
be used.
Then, use power method to find the numerically largest
eigenvalue (1/) of A
-1
.
Therefore, will be the numerically smallest eigenvalue
of A.
Inverse Power Method
[ ]{ } { } x x A = [ ] [ ]{ } [ ] { } x A x A A
1 1
=
{ } [ ] { } x A x
1
1

=
{ } [ ]{ } x B x =
The algorithm is the same as the Power
method and the eigenvector is not the
eigenvector for the smallest eigenvalue. To
obtain the smallest eigenvalue from the power
method.
1

1
= =
32
1/3 - 0 1/3
2/3 3/5 7/15 -
2/3 - 2/5 - 15 / 13
The inverse of the previous matrix is
Applying the power method to this matrix gives a value for the normalizing
factor of 13.4090 and vector of [0.3163, 0.9254, 1]. For the original matrix,
the eigenvalue is the reciprocal, 0.07457. The eigenvector that corresponds
is the same; no change is needed.
Numerical Integration
If f : [a, b] R is differentiable then, we obtain a new function f
:
[a, b] R, called the derivative of f. Likewise, if f : [a, b] R is
integrable, then we obtain a new function F : [a, b] R dened by
F(x) =
_
x
a
f(t)dt x [a, b].
Observation: If f is nonnegative function, then
_
b
a
f(x)dx is represent
the area under the curve f(x).
26-Aug-2009 (1) Sarvesh Rajput, CC 108 1
Antiderivative
Antiderivative: Let F : [a, b] R be such that f = F
, then F is called
an antiderivative of f.
Recall
Fundamental Theorem of Calculus: Let f : [a, b] R is integrable
and has an antiderivative F, then
_
b
a
f(x)dx = F(b) F(a).
Basic Problems
Difcult to nd an antiderivative of the function (for example f(x) =
e
x
2
)
Function is given in the tabular form.
Newton-Cotes Methods/Formulae
The derivation of Newton-Cotes formula is based on Polynomial
Interpolation.
x x
0
x
1
x
2
... x
n
f(x) f(x
0
) f(x
1
) f(x
3
) ... f(x
n
)
The idea is:
Replace f by p
n
(x) and evaluate
_
b
a
p
n
(x)dx
That is,
_
b
a
f(x)dx
_
b
a
p
n
(x)dx =
_
b
a
n
i=0
l
i
(x)f(x
i
)dx
=
n
i=0
f(x
i
)
_
b
a
l
i
(x)dx
=
n
i=0
A
i
f(x
i
)
Where A
i
=
_
b
a
l
i
(x)dx called weights.
Types of Newton-Cotes Formulae
Trapezoidal Rule (Two pint formula)
Simpsons 1/3 Rule (Three Point formula)
Simpsons 3/8 Rule (Four point formula)
Trapezoidal Rule
Since it is two point formula, it uses the rst order interpolation
polynomial P
1
(x).
_
b
a
f(x)
_
x
1
x
0
P
1
(x)dx
where,
P
1
(x) = f(x
0
) + sf(x
0
)
s =
x x
0
h
Now, dx = h ds at x = x
0
, s = 0 and at x = x
1
, s = 1.
Hence,
_
b
a
f(x)dx
_
1
0
(f(x
0
) + sf(x
0
))hds =
h
2
[f(x
0
) + f(x
1
)]
OR
_
b
a
f(x)dx
b a
2
[f(a) + f(b)]
Error
E
T
=
(b a)
3
12
f
(),
where a < < b
Remark: Proof will be given later.
Basic Simpsons
1
3
Rule
To evaluate
_
b
a
f(x)dx.
f will be replaced by a polynomial of degree 2 which interpolates f at
a,
a+b
2
and b.
Take x
0
= a, x
1
=
a+b
2
, x
2
= b
Now
p
2
(x) = p
2
(x
0
+ sh) = f(x
0
) + s
1
f(x
0
) +
s(s 1)
2

2
f(x
0
)
Where s =
xx
0
h
=
xa
h
and h =
ba
2
.
Hence dx = hds. Also x = a = s = 0 and x = b = s = 2. Thus,
_
b
a
f(x)dx I
s
(f) =
_
b
a
p
2
(x)dx
=
_
2
0
[f(x
0
) + s
1
f(x
0
) +
s(s 1)
2

2
f(x
0
)]h.ds
=
h
3
[f(x
0
) + 4f(x
1
) + f(x
2
)]
=
b a
6
[f(a) + 4f(
b + a
2
) + f(b).]
Error
E
s
=
h
5
f
(4)
()
90
for some (a, b).
Simpsons
3
8
rule
f is replaced by p
3
(x) which interpolates f at x
0
= a, x
1
= a +
h, x
2
= a + 2h, x
3
= a + 3h = b. where h =
ba
3
.
Thus
_
b
a
f(x)dx
_
b
a
[f(x
0
)+s
1
f(x
0
)+
s(s 1)
2

2
f(x
0
)+
s(s 1)(s 2)
3!

3
f(x
0
)]dx
Where s =
xx
0
h
and dx = hds. Also x = a = s = 0 and x = b =
s = 3.
Therefore,
_
b
a
f(x)dx
_
3
0
[f(x
0
)+s
1
f(x
0
)+
s
2
s
2

2
f(x
0
)+
s
3
3s
2
+ 2s
3!

3
f(x
0
)]dx
= h[3f
0
+
9
2
(f
1
f
0
)+
1
2
(9
9
2
)(f
2
2f
1
+f
0
)+
1
6
(
81
4
27+9)(f
3
3f
2
+3f
1
f
0
)]
=
3h
8
[f
0
+ 3f
1
+ 3f
2
+ f
3
]
Thus we get:
_
b
a
f(x)dx
3h
8
[f
0
+ 3f
1
+ 3f
2
+ f
3
]
Error: E
s
=
3h
5
80
f
(4)
(), where a < < b.
Example
Using Trapezoidal and Simpson
1
3
rules nd
_
2
0
x
4
dx and
_
2
0
sinxdx and
nd the upper bound for the error.
Composite Rules
Note that if the integral [a, b] is large, then the error in the Trapezoidal
rule will be large.
Idea
Error can be reduced by dividing the interval [a, b] into equal subinterval
and apply quadrature rules in each subinterval.
Composite Trapezoidal Rule
h =
b a
n
, x
i
= x
0
+ ih
Composite Rule
_
b
a
f(x)dx =
_
x
n
x
0
f(x)dx =
n
i=1
_
x
i
x
i1
f(x)dx
Now apply Trapezoidal rule on each [x
i1
, x
i
], we have
_
b
a
f(x)dx =
h
2
[f(x
0
) + 2 (f(x
1
) + f(x
2
) f(x
n1
)) + f(x
n
)]
Error in composite Trapezoidal rule
We know error in the ith interval [x
i1
, x
i
] is
E
CT
(I) =
(x
i
x
i1
)
3
12
f
(
i
) =
h
3
12
f
(
i
)
where
i
(x
i1
, x
i
)). Hence the error in composite rule is
E
CT
=
h
3
12
n
i=1
f
(
i
)
Using Intermediate value theorem, we can prove that
E
CT
= (b a)
h
2
12
f
(), [a, b]
The Composite Simpsons
1
3
Rule
[a, b] will be will be divided into 2n equal subintervals and we apply
basic Simpsons
1
3
rule on each of the n intervals [x
2i2
, x
2i
] for i =
1, 2, 3, , n.
Thus here h =
ba
2n
.
Then
_
b
a
f(x)dx =
_
b=x
2n
a=x
0
f(x)dx
=
_
x
2
x
0
f(x)dx +
_
x
4
x
2
f(x)dx + +
_
x
2i
x
2i2
f(x)dx + +
_
x
2n
x
2n2
f(x)dx
=
h
3
[f(x
0
) + 4f(x
1
) + f(x
2
)] +
h
3
[f(x
2
) + 4f(x
3
) + f(x
4
)]+
+ +
h
3
[f(x
2n2
) + 4f(x
2n1
) + f(x
2n
)]
=
h
3
{f(x
0
) + 4 [f(x
1
) + f(x
3
) + f(x
5
) + + f(x
2n1
)]+
+2 [f(x
2
) + f(x
4
) + f(x
6
) + + f(x
2n2
)] + f(x
2n
)}
The Error in the Composite Simpsons
1
3
Rule
We know error in the ith interval (x
2i2
, x
2i
) is
E
cs
(I) =
h
5
90
f
(4)
(
i
),
i
(x
2i2
, x
2i
)
Using Intermediate value theorem, the error in composite Simpsons
rule is given by
E
CS
= (b a)
h
4
180
f
(4)
(),
where [a, b]
Example
Evaluate the integral
_
1
1
x
2
exp(x)dx by composite Simpsons
1
3
rule
with spacing h = 0.25
Solution: According to composite Simpsons
1
3
rule:
_
1
1
x
2
exp(x)dx =
h
3
[f(x
0
) + 4f(x
1
) + 2f(x
2
) + 4f(x
3
) + 2f(x
4
)+
+4f(x
5
) + 2f(x
6
) + 4f(x
7
) + f(x
8
)]
Here f(x
0
) = f(1) = 2.7183
f(x
1
) = f(0.75) = 1.1908
f(x
2
) = f(0.5) = 0.4122
f(x
3
) = f(0.25) = 0.0803
f(x
4
) = f(0) = 0
f(x
5
) = f(0.25) = 0.0487
f(x
6
) = f(0.50) = 0.1516
f(x
7
) = f(0.75) = 0.2657
f(x
8
) = f(1) = 0.3679
Substituting these values in the above formula we get:
_
1
1
x
2
exp(x)dx 0.87965
Example
Find the minimum no. of subintervals, used in composite Trapezoidal
and Simpsons 1/3 rule in order to nd the integral
_
1
0
e
x
4
dx such that
the error can not exceed by .00001.
Sol. For the composite Trapezoidal rule, we have
1
3
max
0<<1
|f
()|
12n
2
trap
.00001
For the composite Simpson 1/3 rule, we have
1
4
max
0<<1
|f
(4)
()|
180n
4
simp
.00001
Now,
max
0<<1
|f
()| 3.5, max

0<<1
|f
(4)
()| 95
(Please verify )
Hence
n
trap
= 171, n
simp
= 16
Composite Simpsons
3
8
rule
[a, b] is divided into 3n equal subintervals. (h =
ba
3n
. and we apply
3
8
rule on each of the n intervals [x
3i3
, x
3i
] for i = 1, 2, 3, , n.)
Hence,
_
b
a
f(x)dx
_
x
3
x
0
=a
f(x)dx +
_
x
6
x
3
f(x)dx + +
_
x
3n
=b
x3n
3
f(x)dx
=
3h
8
[f
0
+ 3f
1
+ 3f
2
+ f
3
] +
3h
8
[f
3
+ 3f
4
+ 3f
5
+ f
6
]+
+ +
3h
8
[f
3n3
+ 3f
3n2
+ 3f
3n1
+ f
3n
]
=
3h
8
[f
0
+ 3f
1
+ 3f
2
+ 2f
3
+ 3f
4
+ 3f
5
+ 2f
6
+ 3f
7
+ + 3f
3n1
+ f
3n
]
Remember:
f with sufces of multiple 3 are multiplied by 2.
Others by 3, except the end points.
Example
Use composite simpsons
3
8
rule, nd the velocity after 18 seconds, if a
rocket has acceleration as given in the table:
t = 0 2 4 6 8 10 12 14 16 18
a = 40 60 70 75 80 83 85 87 88 88
f
0
f
1
f
2
f
3
f
4
f
5
f
6
f
7
f
8
f
9
Sol: Velocity v =
3h
8
[f
0
+3f
1
+3f
2
+2f
3
+3f
4
+3f
5
+2f
6
+3f
7
+3f
8
+f
9
]=
3
4
[40+360+370+275+380+383+283+285+387+388+88]
= 1389 units.
The Error in Newton-Cotes formulas
Mean Value Theorem:
If f is continuous function on [a, b] and g(x) is integrable on [a, b]
g(x) does not change sign on [a, b] (Either g(x) 0, x [a, b] or
g(x) 0, x [a, b]).
Then, there exists a number [a, b] such that
_
b
a
f(x)g(x)dx = f()
_
b
a
g(x)dx
Already Known:
The error in the interpolation
E
n
(x) = f(x) p
n
(x) = f[x
0
, x
1
, x
2
, , x
n
, x]
n
(x).
Where
n
(x) =
n
i=0
(x x
i
).
Therefore the error in the integration is :
E(I) =
_
b
a
E
n
(x)dx =
_
b
a
n
(x)f[x
0
, x
1
, x
2
, , x
n
, x]dx
Case:1 If
n
(x) does not change sign in [a, b].
By Mean Value Theorem for integrals, we have:
E(I) =
_
b
a
E
n
(x)dx = f[x
0
, x
1
, x
2
, , x
n
, ]
_
b
a
n
(x)dx
for some [a, b].
=
f
(n+1)
()
(n + 1)!
_
b
a
n
(x)dx
for some (a, b).
Case : 2. If
n
(x) changes sign in [a, b] and
_
b
a

n
(x)dx = 0. Now, if we
take any point x
n+1
[a, b], then
f[x
0
, x
1
, x
2
, , x
n
, x
n+1
, x] = f[x
n+1
, x
0
, x
1
, x
2
, , x
n
, x]
=
f[x
0
, x
1
, x
2
, , x
n
, x] f[x
n+1
, x
0
, x
1
, x
2
, , x
n
]
x x
n+1
Thus,
f[x
0
, x
1
, x
2
, , x
n
, x] = f[x
0
, x
1
, x
2
, , x
n
, x
n+1
]
+ (x x
n+1
)f[x
0
, x
1
, x
2
, , x
n
, x
n+1
, x]
Therefore
E(I) =
_
b
a
n
(x)f[x
0
, x
1
, x
2
, , x
n
, x]dx
= f[x
0
, x
1
, x
2
, , x
n
, x
n+1
]
_
b
a
n
(x)dx
+
_
b
a
n
(x)(x x
n+1
)f[x
0
, x
1
, x
2
, , x
n
, x
n+1
, x]dx
=
_
b
a
n
(x)(x x
n+1
)f[x
0
, x
1
, x
2
, , x
n
, x
n+1
, x]dx
(
_
b
a

n
(x)dx = 0. )
Now, The idea is : Choose x
n+1
such that (x x
n+1
)
n
(x) will not
change sign in [a, b].
Thus again by Mean Value Theorem for integrals, we have
E(I) = f[x
0
, x
1
, x
2
, , x
n
, x
n+1
, ]
_
b
a
(x x
n+1
)
n
(x)dx
for some [a, b].
=
f
(n+2)
()
(n + 2)!
_
b
a
(x x
n+1
)
n
(x)dx
for some (a, b).
Error in Trapezoidal Rule:
Using Interpolation Theory, we have
f(x) = P
1
(x) + f[a, b, x](x a)(x b)
Using mean value theorem, Error is given by
_
b
a
f[a, b, x](x a)(x b)dx = f[a, b, ]
_
b
a
(x a)(x b)dx
= f[a, b, ]
(b a)
3
6
=
(b a)
3
12
f
()
where a < < b and [a, b]
The Error in the basic Simpsons
1
3
Rule
E
s
(I) =
_
b
a
(x x
0
)(x x
1
)(x x
2
)f[x
0
, x
1
, x
2
, x]dx
=
_
b
a
(x a)(x
a + b
2
)(x b)f[a, b,
a + b
2
, x]dx
Here
2
(x) = (xa)(x
a+b
2
)(xb) changes its sign in [a, b]. and hence
we cannot apply the Mean Value theorem for integrals immediately. We
proceed as in Case:2 discussed above.
For this We check:
_
b
a
n
(x)dx
?
= 0
_
b
a
2
(x)dx =
_
b
a
(x x
0
)(x x
1
)(x x
2
)dx
=
_
2
0
hs(s 1)h.(s 2)h.hds = 0
We now introduce the point x
n+1
= x
1
Now (xx
n+1
)
2
(x) = (xx
1
).(xx
0
)(xx
1
)(xx
2
) = (xx
0
)(x
x
1
)
2
(x x
2
) < 0 always in (x
0
, x
2
).
Therefore the error
E
s
(I) =
f
(4)
()
4!
_
b
a
(x x
1
)
2
(x)dx
=
f
(4)
()
4!
_
2
0
s(s 1)
2
(s 2)h
5
ds
=
f
(4)
()
4!
h
5
(
4
15
)
=
h
5
f
(4)
()
90
for some (a, b).
Remark: In the similar way, we can nd the error in Simpsons 3/8
rule..(Please work out your own)...To avoid the calculations, we can nd
the error using Method of undetermined coefcient/parameter.
Method of Undetermined Parameters
The Newton - Cotes integration rules are all of the form
I(f) A
0
f(x
0
) + A
1
f(x
1
) + A
2
f(x
2
) + + A
n
f(x
n
)
Also, note that the weights A
i
s do not depend on the given function.
Hence, if the error is of the form
E(I) = Const f
(r+1)
().
Then the rule must be exact for all polynomials of degree r
Recall:
The error in the integral is:
=
f
(n+1)
()
(n + 1)!
_
b
a
n
(x)dx
Or
=
f
(n+2)
()
(n + 2)!
_
b
a
(x x
n+1
)
n
(x)dx
Or we may need to go for higher derivatives of f.
Therefore
If we wish to construct a rule of the form
I(f) A
0
f(x
0
) + A
1
f(x
1
) + a
2
f(x
2
) + + A
n
f(x
n
)
(n-xed) which is exact for polynomials of degree as high as possible,
i.e., we want
E(I) = Const f
(r+1)
(),
with r as large as possible.
This way of constructing integration rules is called the Method of
Undetermined Parameters.
Example
Suppose we want to derive an integration formula of the form:
_
b
a
f(x)dx = A
0
f(a) + A
1
f(b) + f
().
We assume that:The rule is exact for the polynomials 1, x, x
2
.
Now, taking f(x) = 1, we get b a = A
0
+ A
1
Taking f(x) = x we get
b
2
a
2
2
= A
0
a + A
1
b
Solving the above two equations we get, A
0
= A
1
=
ba
2
.
Thus,
_
b
a
f(x)dx =
b a
2
[f(a) + f(b)] + f
()
Now if we take f(x) = x
2
, we get:
b
3
a
3
3
= (
b a
2
)(a
2
+ b
2
) + 2!
= =
(ba)
3
12
Thus
_
b
a
f(x)dx =
b a
2
[f(a) + f(b)]
(b a)
3
12
f
()
We see that: This is exactly the trapezoidal rule. Similarly, Simpsons
1
3
and
3
8
rules can be derived.
Thus in the Method of Undetermined Parameters
We aim directly for a formula of a preselected type.
Working Method:
We impose certain conditions on a formula of desired form and use
these conditions to determine the values of the unknown coefcients in
the formula.
The Error term in the Simpsons
3
8
-rule, using Method
of Undetermined Parameters
Start with:
_
x
3
x
0
f(x)dx =
3h
8
[f
0
+ 3f
1
+ 3f
2
+ f
3
] + f
(4)
()
for some suitable (x
0
, x
3
).
Takeing f(x) = x
4
in the above integration rule we get:
x
5
3
x
5
0
5
=
3h
8
[x
4
0
+ 3x
4
1
+ 3x
4
2
+ x
4
3
] + 4!
4! =
x
5
3
x
5
0
5

3h
8
[x
4
0
] + 3(x
0
+ h)
4
+ 3(x
0
+ 2h)
4
+ (x
0
+ 3h)
4
=
(x
0
+ 3h)
5
x
5
0
5

3h
8
[x
4
0
] + 3(x
0
+ h)
4
+ 3(x
0
+ 2h)
4
+ (x
0
+ 3h)
4
Without loss of generality, we can take: x
0
= 0.
We have: 4! =
243
5
h
5
3h
5
8
[0 + 3 + 3 16 + 81] Thus
4! =
9
10
h
5
That is,
=
3
80
h
5
Therefore the error in the Simpsons rule is =
3
80
h
5
f
(4)
()
for some suitable (a, b).
Example
Find A
0
, x
1
, A
2
so that the following rule is exact for all polynomials of
degree as high as possible and nd the error term also.
_
2
0
f(x)dx = A
0
f(0) +
4
3
f(x
1
) + A
2
f(2) + E(I)
where E(I) = f
() for some [0, 2].

(Note here that, we have chosen the error to be i terms of the third
derivative of f because there are 3 unknowns. )
Taking f(x) = 1 we get: 2 = A
0
+
4
3
+ A
2
Taking f(x) = x we get: 2 =
4
3
x
1
+ 2A
2
Taking f(x) = 1 we get:
8
3
=
4
3
x
2
1
+ 4A
2
On solving these equations we get: x
1
= 1, A
0
=
1
3
= A
2
Thus the integration rule becomes,
_
2
0
f(x)dx =
1
3
[f(0) + 4f(1) + f(2)] + f
()
Now to nd the error term:
Take f(x) = x
3
we get = 0. Thus the rule is exact up to polynomials
of degree 3.
Therefore: Choose the error term as:
E(I) = f
(4)
()
with 4-th derivative.
Thus the integration rule becomes:
_
2
0
f(x)dx = A
0
f(0) +
4
3
f(x
1
) + A
2
f(2) + f
(4)
()
Now take f(x) = x
4
we get
32
5
=
1
3
[0 + 4 + 2
4
] + 4!
= =
1
90
.
Hence the error term is:
1
90
f
(4)
()
Thus the required integration rule is
_
2
0
f(x)dx =
1
3
[f(0) + 4f(1) + f(2)] +
1
90
f
(4)
()
Note that this is Simpsons
1
3
-rule.
The Error term in the Simpsons
3
8
-rule , using Method
of Undetermined Parameters
Start with:
_
x
3
x
0
f(x)dx =
3h
8
[f
0
+ 3f
1
+ 3f
2
+ f
3
] + f
(4)
()
for some suitable [x
0
, x
3
].
Takeing f(x) = x
4
in the above integration rule we get:
x
5
3
x
5
0
5
=
3h
8
[x
4
0
+ 3x
4
1
+ 3x
4
2
+ x
4
3
] + 4!
4! =
x
5
3
x
5
0
5

3h
8
[x
4
0
] + 3(x
0
+ h)
4
+ 3(x
0
+ 2h)
4
+ (x
0
+ 3h)
4
=
(x
0
+ 3h)
5
x
5
0
5

3h
8
[x
4
0
] + 3(x
0
+ h)
4
+ 3(x
0
+ 2h)
4
+ (x
0
+ 3h)
4
Without loss of generality, we can take: x
0
= 0. We have: 4! =
243
5
h
5
3h
5
8
[0 + 3 + 3 16 + 81]
Thus
4! =
9
10
h
5
That is,
=
3
80
h
5
Therefore the error in the Simpsons rule is =
3
80
h
5
f
(4)
()
for some suitable (a, b).
Recall
The Newton - Cotes integration rules are all of the form
I(f) A
0
f(x
0
) + A
1
f(x
1
) + A
2
f(x
2
) + + A
n
f(x
n
)
Also, note that the weights A
i
s do not depend on the given function.
Hence, if the error is of the form E(I) = Const f
(r+1)
(). Then the
rule must be exact for all polynomials of degree r.
Remark: In these quadrature the points x
i
are xed.
Ques: Can we improve the accuracy by choosing some suitable x
i
Ans: Using Gaussian Quadrature rule one can improve the accuracy.
Example
Find x
0
, x
1
, A
0
, A
1
and so that the following rule is exact for all
polynomials of degree 3.
_
1
1
f(x)dx = A
0
f(x
0
) + A
1
f(x
1
) + f
(4)
()
(There are 4 unknowns and hence we have chosen the 4-th derivative
in the error term.)
Taking f(x) = 1, x, x
2
, x
3
we get:
A
0
+ A
1
= 2
A
0
x
0
+ A
1
x
1
= 0
A
0
x
2
0
+ A
1
x
2
1
=
2
3
A
0
x
3
0
+ A
1
x
3
1
= 0
On solving these equations we get:
A
0
= A
1
= 1 x
0
=
1
3
and x
1
=
1
3
.
Thus the integration rule is:
_
1
1
f(x)dx = f(
1
3
) + f(
1
3
) + f
(4)
().
Now if we take f(x) = x
4
we get
2
5
=
2
9
+ 4!
= =
1
4!
(
8
45
) =
1
135
Thus the expected integration rule is:
_
1
1
f(x)dx = f(
1
3
) + f(
1
3
) +
1
135
f
(4)
().
In general
Giving a positive integer n, we wish to determine 2n + 2 numbers
x
0
, x
1
, x
n
and A
0
, A
1
, , A
n
so that the sum
I(f) A
0
f(x
0
) + A
1
f(x
1
) + A
2
f(x
2
) + + A
n
f(x
n
),
provides the exact value of
_
b
a
f(x)dx for f(x) = 1, x, x
2
, x
2n+1
.
Or What we want is that the quadrature rule is exact for all polynomials
of degree 2n + 1.
Remark: Here we have to solve system of nonlinear equations, which is
some time is not an easy job.
Orthogonal Functions
Two functions f, g : [a, b] R are said to be orthogonal if
_
b
a
f(x)g(x)dx = 0.
If
_
b
a
w(x)f(x)g(x)dx = 0
for some function w(x) > 0, x [a, b] then we say that f and g are
orthogonal with respect to w(x)
We call the function w(x) called as weight function.
In general
Let w : [a, b] (0, ) be given.
Functions f
1
(x), f
2
(x), f
3
(x), dened on [a, b] is said to orthogonal
w.r. to the weight function w(x) if
_
b
a
w(x)f
m
(x)f
n
(x)dx = 0 if m = n
and
_
b
a
w(x)f
m
(x)f
n
(x)dx = 0 if m = n
Gauss Quadrature Method says that:
Theorem:
To evaluate
_
b
a
f(x)dx
1. Suppose we write the integral as
_
b
a
f(x)dx =
_
b
a
w(x)g(x)dx for some
positive function w(x).
and
2. Suppose we are able nd a sequence of orthogonal polynomials
p
0
(x), p
1
(x), p
2
(x), w.r.to the weight function w(x).
Then
Given any k. If we choose the points x
0
, x
1
, x
2
, , x
k
as the zeros of
the polynomial p
k+1
(x) and the coefcients A
i
=
_
b
a
l
i
(x)w(x)dx (i =
0, 1, 2, k) then the integration formula
I(f) A
0
g(x
0
) + A
1
g(x
1
) + + A
k
g(x
k
)
will be exact for all polynomials of degree 2k + 1
where
k
j=0,j=i
(x x
j
)
(x
i
x
j
)
, g(x) =
f(x
w(x)
with error term
E(I) =
f
(2k+2)
()
(2k + 2)!
_
b
a
w(x)
p
2
k+1
(x)
2
k+1
dx
_
(1)
Where
k+1
is the coefcient of x
k+1
in p
k+1
(x).
Legendre Polynomials
[a, b] = [1, 1]
w(x) 1
p
n
(x) =
1
n!2
n
d
n
dx
n
(x
2
1)
n
for n = 0, 1, 2, . (Rodriguess Formula)
Then p
0
(x) = 1, p
1
(x) = x, p
2
(x) =
1
2
(3x
2
1), .
we know
_
1
1
p
m
(x)p
n
(x)dx = 0 m = n and
if m = n, then
_
1
1
p
m
(x)p
n
(x)dx =
2
2n + 1
.
Hence p
0
, p
1
, p
2
are orthogonal on [1, 1].
Remark
The integral
_
b
a
f(x)dx can be convert into
_
1
1
g(t)dt
Choose
x =
b a
2
t +
a + b
2
Then
_
b
a
f(x)dx =
b a
2
_
1
1
f
_
b a
2
t +
a + b
2
_
dt
Two point Gauss-Legendre Quadrature
The case when we take only two points x
0
and x
1
in our integration
formula.
Let
_
1
1
f(x)dx = A
0
f(x
0
) + A
1
f(x
1
) + f
(4)
(), (1, 1).
That is the case k = 1 in the above theorem.
Therefore we take the Legendre polynomial of degree 2, that is p
2
(x) =
1
2
(3x
2
1)
Find the roots of p
2
(x) = 0
p
2
(x) =
1
2
(3x
2
1) = 0 = x =
1
3
,
1
3
Therefore as said in the above theorem we take x
0
=
1
3
and x
1
=
1
3
To fond A
0
and A
1
:
Here w(x) 1.
Thus
A
0
=
_
1
1
w(x)l
0
(x)dx =
_
1
1
1.
x x
1
x
0
x
1
dx
=
_
1
1
3
2
(x
1
3
)dx = 1
A
1
=
_
1
1
w(x)l
1
(x)dx =
_
1
1
1.
x x
0
x
0
1 x
0
dx
=
_
1
1
3
2
(x +
1
3
)dx = 1
Thus the 2-point Gauss-Legendre Quadrature formula takes the form:
_
1
1
f(x)dx f(
1
3
) + f(
1
3
)
Error Term:
E(I) =
f
(4)
()
4!
_
1
1
(3x
2
1)
2
9
dx =
f
(4)
()
24
8
45
=
1
135
f
(4)
()
Three point Gauss-Legendre Quadrature
We use three points x
0
, x
1
and x
2
in our integral formula.
This is the case when k = 2
We take p
3
(x) =
1
2
(5x
3
3x)
Find the roots of p
3
(x) = 0
p
3
(x) =
1
2
(5x
3
3x) = 0 = x =
_
3
5
, 0,
_
3
5
Take x
0
=
_
3
5
, x
1
= 0, x
2
=
_
3
5
To fond A
0
, A
1
and A
2
:
A
0
=
_
1
1
w(x)l
0
(x)dx =
_
1
1
w(x).
(x x
1
)(x x
2
)
(x
0
x
1
)(x
0
x
2
)
=
_
1
1
(x 0)(x
_
3
5
)
(
_
3
5
0)(
_
3
5
_
3
5
)
=
_
1
1
(x
2
_
3
5
x)dx =
5
9
.
A
1
=
_
1
1
w(x)
(x +
_
3
5
)(x
_
3
5
)
(0 +
_
3
5
)(0
_
3
5
)
dx
=
_
1
1
(x
2
3
5
)dx =
8
9
A
2
=
_
1
1
(x +
_
3
5
)(x 0)
(
_
3
5
+
_
3
5
)(
_
3
5
0)
dx =
5
9
Error Term:
E(I) =
f
(6)
()
6!
_
1
1
(
5
2
x
3
3
2
x)
2
(
5
2
)
2
dx
=
f
(6)
()
6!
_
1
1
(x
3
3
5
x)
2
dx
=
1
15750
f
(6)
().
Thus the 3-point Gauss-Legendre quadrature formula takes the form:
_
1
1
f(x)dx =
5
9
f(
3
5
) +
8
9
f(0) +
5
9
f(
3
5
) + E(I)
Example
Using 3-point Gauss-Legendre quadrature, evaluate the integral
_
1
1
x sin x
1+x
2
dx
Solution:
_
1
1
xsin x
1 + x
2
dx =
5
9
f(
3
5
) +
8
9
f(0) +
5
9
f(
3
5
)
=
5
9
[
0.5417744
0.599999
] +
8
9
[0] +
5
9
[
0.5417744
1.599999
] = 0.37623
Remark
The integral
_
b
a
f(x)dx can be convert into
_
1
1
g(u)du
Choose
x =
b a
2
u +
a + b
2
Then
_
b
a
f(x)dx =
b a
2
_
1
1
f
_
b a
2
u +
a + b
2
_
du
Example
Suppose we want to integrate
_
2
0
f(x)dx using 3-point Gauss-Legendre
formula
Let
_
2
0
f(x)dx = A
0
f(x
0
) + A
1
f(x
1
) + A
2
f(x
2
) + f
(6)
()
for some (0, 2).
We use the change of variable: x = u + 1
Then x = 0 = u = 1 and x = 2 = u = 1.
Then we get
_
2
0
f(x)dx =
_
1
1
f(u + 1)du
If we write g(u) = f(u + 1), we get
_
1
1
g(u)du A
0
g(u
0
) + A
1
g(u
1
) + A
2
g(u
2
)
Hence u
0
=
_
3
5
, u
1
= 0, u
2
=
_
3
5
Therefore x
0
= u
0
+ 1 = 1
_
3
5
, x
1
= u
1
+ 1 = 0 + 1 = 1 and x
2
=
u
2
+ 1 = 1 +
_
3
5
Thus
_
2
0
f(x)dx
5
9
f(1
_
3
5
) +
8
9
f(1) +
5
9
f(1 +
_
3
5
)
Recall
Theorem:
To evaluate
_
b
a
f(x)dx
1. Suppose we write the integral as
_
b
a
f(x)dx =
_
b
a
w(x)g(x)dx for some
positive function w(x).
and
2. Suppose we are able nd a sequence of orthogonal polynomials
p
0
(x), p
1
(x), p
2
(x), w.r.to the weight function w(x).
Then
Given any k. If we choose the points x
0
, x
1
, x
2
, , x
k
as the zeros of
the polynomial p
k+1
(x) and the coefcients A
i
=
_
b
a
l
i
(x)w(x)dx (i =
0, 1, 2, k) then the integration formula
I(f) A
0
g(x
0
) + A
1
g(x
1
) + + A
k
g(x
k
)
will be exact for all polynomials of degree 2k + 1
where
l
i
(x) =
k
j=0,j=i
(x x
j
)
(x
i
x
j
)
, g(x) =
f(x)
w(x)
with error term
E(I) =
f
(2k+2)
()
(2k + 2)!
_
b
a
w(x)
p
2
k+1
(x)
2
k+1
dx
_
(2)
Where
k+1
is the coefcient of x
k+1
in p
k+1
(x).
Property:
Each p
k
(x) has has k distinct real zeros all lie in [a, b].
Chebyshev Polynomials
[a, b] = [1, 1]
w(x) =
1
1x
2
We know already know that p
0
(x) = 1, p
1
(x) = x, Now p
n
(x) is
recursively deed as
p
k+1
(x) = 2xp
k
(x) p
k1
(x)
For k = 1, 2, 3, . We have p
2
(x) = 2x
2
1, p
3
(x) = 4x
3
3x .
Observation The coefcient of x
k+1
in p
k+1
is 2
k+1
.
Note that the Chebyshev Polynomials given above form an sequence
of orthogonal polynomials w.r.to the weight function w(x) =
1
1x
2
Hence we can use these polynomials to evaluate integrals of the form
_
1
1
f(x)
1
1 x
2
dx
Gauss-Chebyshev Quadrature 2-point formula
To evaluate
_
1
1
f(x)
1 x
2
dx
Noted above: w(x) =
1
1x
2
.
We want to derive 2-point formula, and hence this is the case when
k = 1 in the main theorem.
Therefore we take p
2
(x) = 2x
2
1.
To nd the roots of p
2
(x) = 2x
2
1 = 0 :
= x =
1
2
,
1
2
Therefore the two-point Chebyshev forula will be of the form:
_
1
1
f(x)
1 x
2
dx A
0
f(
1
2
) + A
1
f(
1
2
)
To nd A
0
and A
1
.
A
0
=
_
1
1
w(x)l
0
(x)dx =
_
1
1
1
1 x
2
(
x x
1
x
0
x
1
)dx
=
1
2
2
_
1
1
2x
1 x
2
dx +
1
2
_
1
1
1
1 x
2
dx
= 0 +

2
.
A
1
=

2
=
_
1
1
w(x)l
1
(x)dx =
_
1
1
1
1 x
2
(
x x
0
x
1
x
0
)dx
=
_
1
1
1
1 x
2
(
x +
_
1
2
2
2
)dx =

2
.
Thus,
_
1
1
f(x)
1 x
2
dx

2
f(
1
2
) +

2
f(
1
2
).
The Error term is: substitute k = 1in (2) we get:
E(I) =
f
(4)
()
4!
_
b
a
1
1 x
2
p
2
(x)
2
4
dx
=

2
3
f
(4)
()
4!
3-point Gauss-Chebyshev Quadrature
This is the case when k = 2.
Take p
3
(x) = 4x
3
3x
Roots of p
3
(x) = 0:
4x
3
3x = 0 = x = 0,
3
2
,
3
2
Thus x
0
= 0 x
1
=

3
2
and x
2
=
3
2
Therefore the Three-point Chebyshev forula will be of the form:
_
1
1
f(x)
1 x
2
dx A
0
f(
3
2
) + A
1
f(0) + A
2
f(
3
2
)
To nd A
0
, A
1
and A
2
:
A
0
=
_
1
1
w(x)l
0
(x)dx =
_
1
1
1
1 x
2
(x 0)(x
3
2
)
(
3
2
0)(
3
2

3
2
)
dx
=
_
1
1
(x
2
3
2
x)
1 x
3
dx =

3
.
Similarly,
A
1
= A
2
=

3
.
Thus,
_
1
1
f(x)
1 x
2
dx

3
[f(
3
2
) + f(0) + f(
3
2
)].
The Error Term: Take k = 2 in (2).
E(I) =
f
(6)
()
6!
_
1
1
1
1 x
2
(4x
3
3x)
2
4
2
dx
=
f
(6)
()
16 6!
_
1
1
(16x
6
24x
4
+ 9x
2
)
1 x
2
dx =
f
(6)
()
2
5
6!
.
Example
Using three-point Gauss-Chebyshev quadrature formula, evaluate the
integral
_
1
1
(1 + x)e
x
1 x
2
dx
Solution:
_
1
1
(1 + x)e
x
1 x
2
dx =

3
[f(
3
2
)+f(0)+f(
3
2
)] =

3
[0.7574258+1+3.24338]
= 5.23683
Laguerre Polynomials
[a, b] = (0, )
w(x) = e
x
p
n
(x) = e
x
d
n
dx
n
(x
n
e
x
) for n = 0, 1, 2, .
weight function w(x) = e
x
.
Therefore, we can use these polynomials to evaluate the integrals of
the form:
_

0
e
x
f(x)dx
First few Laguerre Polynomials
p
0
(x) = 1
p
1
(x) = (x 1)
p
2
(x) = x
2
4x + 2
p
3
(x) = x
3
+ 9x
2
18x + 6
2-point Gauss- Laguerre Rule
Take k=1 and proceed in the similar as in Gauss-Legendre case, we
can prove that
_

0
e
x
f(x)dx =
1
4
[(2+
2)f((2
2))+(2
2)f((2+
2))] +
f
(4)
()
6
.
Example
Evaluate using 2-point Gauss-Laguerre quadrature.
_
0
e
x
sin(2x)
1+x
2
dx
Solution: Here f(x) =
sin(2x)
1+x
2
_

0
e
x
sin(2x)
1 + x
2
dx
1
4
[(2 +
2)f(2
2) + (2
2)f(2 +
2)]
1
4
[3.4142 f(0.585786) + 0.585786 f(3.4142)]
= 0.5915
Hermite Polynomials
[a, b] = (, )
w(x) = e
x
2
p
n
(x) = (1)
n
e
x
2 d
n
dx
n
(e
x
2
) for n = 0, 1, 2, .
Since w(x) = e
x
2
, therefore this orthogonal sequence of polynomials
can be used to evaluate integrals of the form
_

e
x
2
f(x)dx.
First few Hermite Polynomials
Example: P
0
(x) = 1,
P
1
(x) = 2x,
P
2
(x) = 4x
2
2,
P
3
(x) = 8x
3
12x
Three -point Gauss-Hermite Quadrature Rule
This is the case when k = 2.
We take p
3
(x), the Hermite polynomial of degree 3.
To nd the roots of p
3
(x) = 0 :
8x
3
12x = 0 = x =
_
3
2
, 0,
_
3
2
Hence x
0
=
_
3
2
, x
1
= 0, and x
3
=
_
3
2
After computing A
0
, A
1
, A
2
, the three-point Gauss-Hermite Rule takes
the form:
_

e
x
2
f(x)dx
6
[f(
_
3
2
) + 4 f(0) + f(
_
3
2
)]
Error term: Take k = 2 in the equation (2).
E(I) =
f
(6)
()
6!
_

e
x
2 (8x
2
12)
2
8
2
dx
=
f
(6)
()
6!
3
4
=
960
f
(6)
().
Example
Evaluate the integral
_

e
x
2
1 + x
2
dx
, using three-point Gauss-Hermite quadrature
Solution: Here f(x) =
1
1+x
2
.
I(f)
6
[f(
_
3
2
) + 4 f(0) + f(
_
3
2
)] =
4
5
.
Some Properties of orthogonal Sequence of
Polynomials
Let w(x) > 0 for all x.
Let p
0
(x), p
1
(x), p
2
(x), , p
k
(x), be be a sequence of orthogonal
polynomials w.r.to w(x). Then
Property: 1
If p(x) is any polynomial of degree k then p(x) can be uniquely written
as p(x) = d
0
p
0
(x) + d
1
p
1
(x) + + d
k
p
k
(x)
Property: 2
If p(x) is any polynomial of degree < k then p(x) is orthogonal to p
k
(x).
That is
_
b
a
p
k
(x)p(x)w(x)dx = 0.
Property: 3
Each p
k
(x) has has k distinct real zeros all lie in [a, b].
Numerical Differentiation
Basic problems
Derive a formula that approximates the derivative of a function in
terms of linear combination of function values ( Function may be
known )
Approximate the value of a derivative of a function dened by discrete
data.
Solution Approaches ..
Use Taylor Series Expansion.
Pass a polynomial through the given data and differentiate the
interpolating polynomial.
Applications
To solve Ordinary and Partial Differential Equations.
First Derivative..
Let f : [a, b] R, then the derivative of f is another function say
f
: [a, b] R and dened by

f
(c) = lim
h0
f(c + h) f(c)
h
, c (a, b).
Geometrically Speaking f
(c) is the slope of tangent to the curve f(x)

at x = c.
Taylor Series
Derivative of a function at x = x
0
Suppose f has two continuous derivatives. Then, by Taylors Theorem
f(x
0
+ h) = f(x
0
) + hf
(x
0
) +
h
2
2
f
()
where (x
0
, x
0
+ h). Now,
f
(x
0
)
f(x
0
+ h) f(x
0
)
h
Called Forward Formula
Error
Error = |true value approximate value|
|E
D
(f)| max
[a,b]
h
2
|f
()|
Example
Example1. Using Taylor series nd the derivative of f(x) = x
2
at x=1,
with h = .1. Also compute the error.
Other formulae
Backward Formula
f
(x
0
)
f(x
0
+ h) f(x
0
)
h
Central Formula
f
(x
0
)
f(x
0
+ h) f(x
0
h)
2h
Similarly, we can drive (Second Derivative )
f
(x
0
)
f(x
0
+ h) 2f(x
0
) + f(x
0
h)
h
2
Derivative for discrete data using interpolating
polynomial
x x
0
x
1
x
2
... x
n
f(x) f(x
0
) f(x
1
) f(x
3
) ... f(x
n
)
Assumption: x
0
, x
1
x
n
are equispaced i.e., x
i
x
i1
= h. Where
the explicit nature of the function f is not known.
Remark 1: We can used Newtons Forward or Backward formula
depends the location of the point
Remark 2: If data is not equispaced then Lagrange interpolating
polynomial can be used.
Using Newtons forward difference formula
f(x) P
n
(x) = f(x
0
)+sf(x
0
)+
s(s 1)
2!

2
f(x
0
)+
s(s 1)(s 2)
3!

3
f(x
0
)

s(s 1)(s 2) (s n + 1)
n!

n
f(x
0
)
where
x = x
0
+ sh
We use p
n
(x) to calculate the derivatives of f.
That is f
(x) p
n
(x) for all x [x
0
, x
n
].
For a given x,
f
(x) p
n
(x)
=
dp
n
ds
ds
dx
=
1
h
dp
n
ds
Similarly,
f
(x)
d
2
p
n
dx
2
d
dx
(
dp
n
dx
)
=
d
dx
(
dp
n
ds
ds
dx
)
=
1
h
d
dx
(
dp
n
ds
)
=
1
h
(
d
2
p
n
ds
2
1
h
)
=
1
h
2
d
2
p
n
ds
2
Thus in general,
f
(k)
(x) =
1
h
k
d
k
p
n
ds
k
Example 1
Using Taylor series expansion (forward formula) and Newton forward
divided difference, compute rst and second derivative at x = 2 for the
following tabulated function
x 1 2 3 4
f(x) 2 5 7 10
Solution
x f(x)
2
3
1 2
3
2 5 -1
2 2
3 7 1
3
4 10
Here h = 1 Using Taylor series
f
(2) =
f(2 + h) f(2)
h
=
f(3) f(2)
1
= 2
f
(2) =
f(2 + h) 2f(2) + f(2 h)
h
2
=
f(3) 2f(2) + f(1)
1
= 1
Using Newton forward divided difference formula
f(x) P
n
(x) = f(x
0
)+sf(x
0
)+
s(s 1)
2!

2
f(x
0
)+
s(s 1)(s 2)
3!

3
f(x
0
)
f
(x)
1
h
dp
n
ds
=
1
h
f(x
0
) +
2s 1
2!

2
f(x
0
) +
3s
2
6s + 2
3!

3
f(x
0
)
Here x = 2, x
0
= 1 , s = 1 and h = 1
f
(2) = 3
1
2
1
3
= 13/6
f
(x)
1
h
2
d
2
p
n
ds
2
=
1
h
2
2
f(x
0
) + (s 1)
3
f(x
0
)
(2) = 1
Example 2
Calculate f
(4)
(0.15)
x 0.1 0.2 0.3 0.4 0.5 0.6
f(X) 0.425 0.475 0.400 0.450 0.525 0.575
Solution:
Newtons forward difference formula:
p
5
(x) = f(x
0
) + s
1
f(x
0
) +
s
2
s
2

2
f(x
0
) +
s
3
3s
2
+ 2s
6

3
f(x
0
)+
+
s
4
6s
3
+ 11s
2
6s
24

4
f(x
0
) +
s
5
10s
4
+ 35s
3
50s
2
+ 24s
120

5
f(x
0
)
Differentiating this 4-times we get,
d
4
f
dx
4

dp
4
5
dx
4
=
1
h
4
[
4
f(x
0
) +
1
5
(5s 10)
5
f(x
0
)
=
1
h
4
[
4
f(x
0
) + (s 2)
5
f(x
0
)]
=
1
(0.1)
4
[035 + (0.5 2)(0.4)] = 95.00 10
2
x f(x)
1
f
2
f
3
f
4
f
5
f
0.1 0.425
0.050
0.2 0.475 -0.125
-0.075 0.25
0.3 0.400 0.125 -0.35
0.050 -0.100 0.4
0.4 0.450 0.025 0.05
0.075 -0.05
0.5 0.525 -0.025
0.050
0.6 0.575

Dan Um Jay A

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dan Um Jay A

Uploaded by

Copyright:

Available Formats

Numerical Analysis

Course No. : AAOC C341

is a -fraction called the

has m signicant digits with respect to

has magnitude less than or equal to 5 in the

= 0.333, then the error

has three signicant digits with

are the corresponding approximate values

= 3.1416. Similarly, we take

be the approximation to x correct to n

+ ) and using Taylor series

approximate x correct upto n signicant

) approximates f(x) correct

ullers method is useful for obtaining both

ullers method is an extension of the secant

ullers method uses three initial approximations

ullers method to nd the

ullers method. Here take the

(x)[ k < 1, for all x (a, b),

(1) = 0.333 k < 1.

(x) is continuous on the open interval (a, b)

(x)[ k < 1 for all x (a, b).

(p) ,= 0 then for any p

(p) ,= 0, then the

(x) is continuous on (a, b), we have

(p) ,= 0, then the xed

(x)[ < 1, for all x I.

If the Jacobian does not vanish, then (1) has a

-norm or maximum norm is denoted by

= max[1[, [ 2[, [3[ = 3,

= max[2[, [0[, [ 1[, [2[ = 2,

= max[0[, [1[, [ 4[, [2[, [ 1[ = 4.

-norm or maximum norm denoted by

= max[1[ +[ 2[, [4[ +[3[ = 7,

= max[1[ +[0[ +[2[, [0[ +[1[ +[ 1[,

Eigen values are = 1, 3, 6.

Eigen values are = 3, 3, 3.

()| 3.5, max

() for some [0, 2].

: [a, b] R and dened by

(c) is the slope of tangent to the curve f(x)

You might also like