You are on page 1of 24

Ch 12 - Integral Calculus

Chapter 12-b
Integral Calculus - Extra

Isaac Newton

Thomas Simpson

BONUS
Introduction to Numerical
Integration

Ch 12 - Integral Calculus

Numerical Integration
Idea is to do integral in small parts, like the way we presented
integration: a summation.
Numerical methods just try to make it faster and more accurate.
12

10

0
3

11

13

15

Basic Numerical Integration


Idea: Weighted sum of function values to approximate integral

f ( x )dx

f ( xi )

i=0

= c0 f ( x0 ) + c1 f ( x1 ) + LL + cn f ( xn )

f(x)

x0

x1

xn-1

Task: Find appropriate cis (weights) and the xis (nodes).

xn
4

Ch 12 - Integral Calculus

Basic Numerical Integration


We want to find integration of functions of various forms of the
equation known as the Newton-Cotes integration formulas.
Newton-Cotes formula
Assume the value of f(x) defined on [a,b] is known at equally
spaced points xi (i=0,1,...,n), where x0 =a, and xn =b. Then,
b

f(x)dx

f ( xi )

i =1

where xi =h i + x0, with h (the step size) equal to (xnx0)/n =


(ba)/n. The ci's are called weights. The xi's are called nodes. The
precision of the approximation depends on n.
The nodes can be equally spaced or unequally spaced.

Basic Numerical Integration


Error analysis
The error of the approximation is the difference between the
value of the integral and the numerical result:
error = = f(x) dx - i ci f(xi)
The errors are frequently approximated using Taylor series for
f(x). The error analysis gives a strict upper bound on the error,
if the derivatives of f are available.

Ch 12 - Integral Calculus

Newton-Cotes Formula
The weights are derived from the Lagrange polynomials L(x).
They depend only on the xi's; not on the function .
Li ( x ) =

( x x 0 )...( x x i 1 )( x x i +1 )...( x x n )
( x i x 0 )...( x i x i 1 )( x i x i +1 )...( x i x n )

Different methods use different polynomials to get the ci's.


Newton-Cotes Closed Formulae -- Use both end points
Trapezoidal Rule : Linear
Simpsons 1/3-Rule : Quadratic
Simpsons 3/8-Rule : Cubic
Booles Rule : Fourth-order
Newton-Cotes Open Formulae -- Use only interior points
midpoint rule

Trapezoid Rule
Straight-line approximation
The trapezoid rule approximates the region under the graph
of the function f(x) as a trapezoid and calculating its area.

f ( x ) dx

f ( xi ) = c0 f ( x0 ) + c1 f ( x1 )

i=0

h
[ f ( x 0 ) + f ( x 1 )]
2

f(x)

Approximate
integral by the
trapezoids area.

L(x)

x0

x1

Ch 12 - Integral Calculus

Trapezoid Rule - Derivation


We use the Lagrange approximation for function f(x) over the
interval (xn-x0) (Lagrange interpolation), given by

f ( x) f ( x0 ) L0 ( x) + f ( x1 ) L1 ( x) + ... + f ( xn ) Ln ( x)
For the case, n=2, with the interval (x1-x0):
L( x) =
let

x x1
x x0
f ( x0 ) +
f ( x1 )
x 0 x1
x1 x 0

a = x 0 , b = x1, =

x = a

x = b

xa
dx
, d =
; h=ba
ba
h

= 0
L ( ) = (1 ) f ( a ) + ( ) f ( b )
=1
9

Then, we integrate to obtain the trapezoid rule

Trapezoid Rule - Derivation


Integrating:

f ( x ) dx

L ( x ) dx = h L ( ) d
0

= f ( a ) h (1 ) d + f (b ) h d
0

= f ( a ) h (

) + f (b ) h
0

2 1

=
0

h
[ f ( a ) + f (b ) ]
2

=> The weights depend only on h!


This approximation may be poor. The approximation error is:
= f(x) dx - i ci f(xi) = -(b a)3/(12) f (),
[a, b].
Thus, if the integrand is convex i.e., positive second
derivative-, the error is negative. That is, the trapezoidal rule
overestimates the true value of the integral.

10

Ch 12 - Integral Calculus

Trapezoid Rule - Example

Evaluate the integral


Exact solution

xe 2 x dx
4

xe

2x

1
x

dx = e 2 x e 2 x
4
2
0
1
1
= e 2 x ( 2 x 1 ) = 5216 . 926477
4
0

Graph

11

Trapezoid Rule - Example


Evaluate the integral

xe 2 x dx

Trapezoidal Rule: Approximation Error


= -(b a)3/(12) f () ( is a number between a and b).
Lets take =2. Then
= -43/12*[2*exp(2*2)+2*exp(2*2)+4*(2)*exp(2*2)] =-3494.282
Trapezoidal Rule
I =

40
[ f ( 0 ) + f ( 4 )] = 2( 0 + 4 e 8 ) = 23847 .66
2
5216 .926 23847 .66
= 357 .12 %
=
5216 .926

xe 2 x dx

12

Ch 12 - Integral Calculus

Simpsons 1/3-Rule (Keplers Rule)


Approximate the function by a parabola

f ( x )dx

f ( xi ) = c0 f ( x0 ) + c1 f ( x1 ) + c2 f ( x2 )

i =0

h
[ f ( x 0 ) + 4 f ( x 1 ) + f ( x 2 )]
3

f(x)

x0

L(x)

x1

x2 13 x

Simpsons 1/3-Rule - Derivation


Use a quadratic Lagrange interpolation:
( x x 0 )( x x 2 )
( x x 1 )( x x 2 )
f ( x0 ) +
f ( x1 )
( x 0 x 1 )( x 0 x 2 )
( x 1 x 0 )( x 1 x 2 )

L( x ) =
+

( x x 0 )( x x 1 )
f ( x2 )
( x 2 x 0 )( x 2 x 1 )

a+b
2

x
x
ba
dx
1
h=
, =
, d =
2
h
h

x
=
x

x
=
x

1 = 0
x = x = 1
2

let

x0 = a , x2 = b, x1 =

L( ) =

( 1 )
2

f ( x0 ) + ( 1 2 ) f ( x 1 ) +

( + 1 )
2

f ( x2 )

14

Ch 12 - Integral Calculus

Simpsons 1/3-Rule - Derivation


Integrate the Lagrange interpolation

h
2

f(x)dx h L ( )d = f(x 0 )
1

( 1 )d

+ f(x 1 )h ( 1 2 )d + f(x 2 )
0

h
2

( + 1 )d
1

h 3
2
3
) + f(x 1 )h (
)
= f(x 0 ) (

2
3
2
3
1
1
1

h 3
2
)
+ f(x 2 ) (
+
2
3
2
1

f(x)dx =

h
[ f(x
3

) + 4f(x

) + f(x

)]

Again, the weights depend only on h!

15

Thomas Simpson (1710 1761, England)

Simpsons 3/8-Rule
Approximate by a cubic polynomial

f ( x )dx c i f ( x i ) = c 0 f(x 0 ) + c 1 f(x 1 ) + c 2 f(x 2 ) + c 3 f(x 3 )


i =0

3h
[ f ( x 0 ) + 3 f ( x 1 ) + 3 f ( x 2 ) + f ( x 3 )]
8

L(x)

x0

f(x)

x1

x2

x3 16 x

Ch 12 - Integral Calculus

Simpsons 3/8-Rule
Lagrange interpolation
L( x ) =

( x x 1 )( x x 2 )( x x 3 )
f ( x0 )
( x 0 x 1 )( x 0 x 2 )( x 0 x 3 )

( x x 0 )( x x 2 )( x x 3 )
f ( x1 )
( x 1 x 0 )( x 1 x 2 )( x 1 x 3 )

( x x 0 )( x x 1 )( x x 3 )
f ( x2 )
( x 2 x 0 )( x 2 x 1 )( x 2 x 3 )

( x x 0 )( x x 1 )( x x 2 )
f ( x3 )
( x 3 x 0 )( x 3 x 1 )( x 3 x 2 )

Integrate to obtain the rule

f(x)dx

L(x)dx ;

h=

b-a
3

3h
[ f ( x 0 ) + 3 f ( x 1 ) + 3 f ( x 2 ) + f ( x 3 )]
8

17

Example: Simpsons Rules

Evaluate the integral

xe 2 x dx

Simpsons 1/3-Rule: Approximation Error


= -(b a)5/(2880) f (4)() ( is a number between a and b).
Since f (4)() >0, the error is negative (overshooting).
Lets take =2.5. Then,
= -45/(2880) [ exp(2*2.5)*[16*(2.5)+32] =-3799.3769
Simpsons 1/3-Rule
I =

4
0

xe 2 xdx

h
3

[f

(0) + 4 f (2 ) + f (4) ]

2
0 + 4 ( 2 e 4 ) + 4 e 8 = 8 2 4 0 .4 1 1
3
5 2 1 6 .9 2 6 8 2 4 0 .4 1 1
=
= 5 7 .9 6 %
5 2 1 6 .9 2 6
=

18

Ch 12 - Integral Calculus

Example: Simpsons Rules


Evaluate the integral
Simpsons 3/8-Rule
I=

xe 2 x dx

xe 2 x dx

3h
4
8

f (0) + 3 f ( ) + 3 f ( ) + f (4)
8
3
3

3(4/3)
[0 + 3(19.18922) + 3(552.33933) + 11923.832 ] = 6819.209
8
5216.926 6819.209
= 30.71%
=
5216.926
=

Simpsons 3/8-Rule: Approximation Error


= -(b a)5/(6480) f (4)() ( is a number between a and b).
19

Midpoint Rule
Newton-Cotes Open Formula

f ( x )dx ( b a ) f ( x m )

= (b a )f(
where [a,b].

a+b
( b a )3
)+
f ( )
2
24
f(x)

The midpoint
rule amounts to
compute the
area of the
rectangle

xm

Note: This rule does not make any use of the end points.

x
20

Ch 12 - Integral Calculus

Two-point Newton-Cotes Open Formula


Approximate by a straight line

f ( x )dx

3
ba
[ f ( x 1 ) + f ( x 2 )] + ( b a ) f ( )
2
108

f(x)

x0

x1

x2

x3 21 x

Three-point Newton-Cotes Open Formula


Approximate by a parabola

ba
[2 f ( x 1 ) f ( x 2 ) + 2 f ( x 3 )]
3
7 ( b a )5
+
f ( )
23040

f ( x )dx

f(x)

x0

x1

x2

x3

x422 x

Ch 12 - Integral Calculus

Better Numerical Integration


Composite integration
Composite Trapezoidal Rule
Composite Simpsons Rule
Richardson Extrapolation
Romberg integration

23

Composite Trapezoid Rule


To improve the Trapezoid Rule, first splits the interval of
integration [a,b] into N smaller, uniform subintervals, and then
applies the trapezoidal rule on each of them.
Two segments

Three segments

11

13

15

Four segments

11

13

15

13

15

Many segments

24

0
3

11

13

15

11

Ch 12 - Integral Calculus

Composite Trapezoid Rule


Use the Trapezoid Rule in n intervals. Then, add them together.

f(x)dx =

x1

x0

f(x)dx +

x2

x1

f(x)dx + L L +

xn
xn 1

f(x)dx

h
[ f(x 0 ) + f(x 1 )] + h [ f(x 1 ) + f(x 2 )] + L + h [ f(x n 1 ) + f(x n )]
2
2
2
h
= [ f(x 0 ) + 2 f(x 1 ) + L + 2f(x i ) + L + 2 f ( x n 1 ) + f ( x n )]
2
=

f(x)

h=

ba
n
x0

x1

x2

x3

x425 x

Composite Trapezoid Rule


Evaluate the integral

I = xe 2 x dx
0

h
[ f ( 0 ) + f ( 4 )] = 23847.66
= 357.12%
2
h
n = 2 , h = 2 I = [ f ( 0 ) + 2 f ( 2 ) + f ( 4 )] = 12142.23 = 132.75%
2
h
n = 4,h = 1 I = [ f ( 0 ) + 2 f ( 1 ) + 2 f ( 2 )
2
+ 2 f ( 3 ) + f ( 4 )] = 7288.79
= 39.71%
h
n = 8 , h = 0.5 I = [ f ( 0 ) + 2 f ( 0.5 ) + 2 f ( 1 )
2
+ 2 f ( 1.5 ) + 2 f ( 2 ) + 2 f ( 2.5 ) + 2 f ( 3 )
+ 2 f ( 3.5 ) + f ( 4 )] = 5764.76
= 10.50%
h
n = 16 , h = 0.25 I = [ f ( 0 ) + 2 f ( 0.25 ) + 2 f ( 0.5 ) + L
2
+ 2 f ( 3.5 ) + 2 f ( 3.75 ) + f ( 4 )]
= 5355.95
= 2.66%
n = 1,h = 4 I =

26

Ch 12 - Integral Calculus

Composite Trapezoid Rule with Unequal


Segments

Evaluate the integral


I = xe 2 x dx
0
Use the following his: {h1=2, h2=1, h3=0.5, h4=0.5}
I=

f ( x) dx +

f ( x) dx +

3.5

f ( x)dx +

f ( x) dx

3. 5

h1
[ f (0) + f (2)] + h2 [ f (2) + f (3)]
2
2
h
h
+ 3 [ f (3) + f (3.5)] + 4 [ f (3.5) + f ( 4)]
2
2
2
1
0.5 6
= 0 + 2e 4 + 2e 4 + 3e 6 +
3e + 3.5e 7
2
2
2
0.5
+
3.5e 7 + 4e8 = 5971.58
= 14.45%
2
=

] [

27

Composite Simpsons Rule


Piecewise Quadratic approximations

h=

ba
n

f(x)

...
x0 h x1 h x2 h x3 h

x4

xn-2 xn-1

xn28 x

Ch 12 - Integral Calculus

Composite Simpsons Rule


4

Evaluate the integral


I = xe 2 x dx
0
Using n = 2, h = 2
h
I = [ f ( 0 ) + 4 f ( 2 ) + f ( 4 )]
3
2
= 0 + 4( 2 e 4 ) + 4 e 8 = 8240.411 = 57.96%
3

Using n = 4, h = 1

h
[ f ( 0 ) + 4 f ( 1 ) + 2 f ( 2 ) + 4 f ( 3 ) + f ( 4 )]
3
1
= 0 + 4 ( e 2 ) + 2( 2 e 4 ) + 4 ( 3 e 6 ) + 4 e 8
3
= 5670.975 = 8.70%

I=

29

Composite Simpsons Rule with Unequal


Segments
Evaluate the integral
Using h1 = 1.5, h2 = 0.5
I=

I = xe 2 x dx
0

f ( x)dx + f ( x)dx

h1
[ f (0) + 4 f (1.5) + f (3)]
3
h
+ 2 [ f (3) + 4 f (3.5) + f (4)]
3
1.5
0.5 6
=
0 + 4(1.5e 3 ) + 3e 6 +
3e + 4(3.5e 7 ) + 4e 8
3
3
= 5413.23 = 3.76%

]
30

Ch 12 - Integral Calculus

Gaussian Quadratures
Newton-Cotes Formulae
- Nodes: Use evenly-spaced functional values
- Weights: Derived from an approximation required to be equal for
a polynomial of order lower or equal to the degree of the
polynomials used to approximate the function. Given nodes, best!
- Problem: Can explode for large n (Runges phenomenon)
Q: Can we use more efficient weights and nodes? Yes!
Gaussian Quadratures
- Gaussian quadrature sets the nodes and the weights in such a way
that the approximation is exact when f(.) is a low order polynomial.
Best choice for both, nodes and weights!
31

Gaussian Quadratures
In fact, Gaussian quadrature is more general than simple integration,
it computes an approximation to the weighted integral:
b

f(x)c(x)dx

ci f ( xi )

i =1

Gaussian Quadratures
- Select functional values at non-uniformly distributed points to
achieve higher accuracy. The values are not predetermined, but
unknowns to be determined.
- Change of variables => the interval of integration is [-1,1].
- Gauss-Legendre formulae for nodes and weights
- With n nodes, delivers exact answer if f is (2n-1)th-order polynomial,
if f is close to a (2n-1)th-order.
32

Ch 12 - Integral Calculus

Gaussian Quadratures
The Gauss-Legendre quadrature rule is in general stated as:
1

f(x)dx

f ( xi )

i =1

the ci's are called the weights, the xi's are called the quadrature nodes.
The approximation error term, , is called the truncation error of
integration.
The goal is to get an exact answer if f is a (2n-1)th-order
polynomial. (With n=5, we get an exact answer f is a 9th-order
polynomial).
For Gauss-Legendre quadrature, the nodes are chosen to be zeros
of certain Legendre polynomials. They are not trivial to compute. 33

Gaussian Quadratures Nodes and Weights


1

To get right answer for f(x) = 1

f(x)dx =

1dx = 2 =

1
1

i =1

To get right answer for f(x) = x2

x 2 dx = 2 / 3 =

To get right answer for f(x) = xj

i =1

xdx = 0 = c

To get right answer for f(x) = x

c1

c
i

2
i

i =1

for j=0,...., 2n-1

x j dx =

c
i

j
i

i =1

=>A system of 2n equations in 2n unknowns.

34

Ch 12 - Integral Calculus

Gaussian Quadratures Nodes and Weights


By construction we get right answer for
f(x) = 1 (j=0), f(x) = x (j=1), ...., f(x) = xj 1 (j=2n-1),
=> enough to get the right answer for any polynomial of order 2n-1.
Example: n=2 =>a system of 4 equations and 4 unknowns:
f

= x

=1

1
1

1dx = 2 = c 1 + c 2

= x2

= x
3

xdx = 0 = c 1 x 1 + c 2 x 2
x 2 dx =

2
= c 1 x 12 + c 2 x 22
3

x 3 dx = 0 = c 1 x 13 + c 2 x 23

c 1 = 1
c = 1
2
1

x1 =
3

x2 = 3

35

Change of Interval for Gaussian Quadrature


Coordinate transformation from [a,b] to [-1,1]
This can be done by an affine transformation on t and a change
of variables.
ba
b+a
t =
x+
2
2
ba
dt =
dx
2
x = 1 t = a

x = 1 t = b

t1
1

f (t )dt = f (
1

t2

ba
b+a ba
ba n
x+
)(
)dx
ci f ( xi )
2
2
2
2 i =1

36

Ch 12 - Integral Calculus

Gaussian Quadrature on [-1, 1]


Gauss Quadrature General formulation:

f ( x )dx c i f ( x i ) = c 1 f ( x 1 ) + c 2 f ( x 2 ) + L + c n f ( x n )
i =1

n = 2:

f(x)dx

= c1f(x 1 ) + c 2 f(x 2 )
x1
x2
-1
1
For n=2, we have four unknowns (c1, c2, x1, x2). These are found
by assuming that the formula gives exact results for integrating a
general 3rd order polynomial. It can also be done by choosing (c371,
c2, x1, x2) such that it yields exact integral for f(x) = x0, x1, x2, x3.

Gaussian Quadrature on [-1, 1]


1

Case n = 2

f(x)dx = c1f(x 1 ) + c 2 f(x 2 )

Exact integral for f = x0, x1, x2, x3


Four equations for four unknowns

=x
= x2

1dx = 2 = c + c
xdx = 0 = c x + c x
2
x dx = = c x + c x
3

=1

= x3

I=

1
1

1
1

1
1

1 1

2 2

2
1 1

2
2 2

x 3 dx = 0 = c1 x13 + c2 x23

f ( x )dx = f (

1
3

c1 = 1

c 2 = 1
1

x1 =
3

1
x2 =
3

)+ f (

1
3

38

Ch 12 - Integral Calculus

Gaussian Quadrature on [-1, 1]


1

Case n = 3 :

-1

f ( x )dx = c1 f ( x1 ) + c 2 f ( x2 ) + c3 f ( x3 )

x1

x2

x3

Now, choose (c1, c2, c3, x1, x2, x3) such that the method yields
exact integral for f(x) = x0, x1, x2, x3,x4, x5. (Again, (c1, c2, c3,
x1, x2, x3) are calculated by assuming the formula gives exact
expressions for integrating a fifth order polynomial).
39

Gaussian Quadrature on [-1, 1]


1

f = 1 xdx = 2 = c1 + c2 + c3
1
1

f = x xdx = 0 = c1 x1 + c2 x2 + c3 x3
1
1

f = x 2 x 2 dx =
1

2
= c1 x12 + c2 x22 + c3 x32
3

f = x x 3dx = 0 = c1 x13 + c2 x23 + c3 x33


3

1
1

f = x 4 x 4 dx =
1

2
= c1 x14 + c2 x24 + c3 x34
5

c1 = 5 / 9
c = 8 / 9
2
c3 = 5 / 9

x1 = 3 / 5
x2 = 0

x3 = 3 / 5

f = x 5 x 5 dx = 0 = c1 x15 + c2 x25 + c3 x35


1

40

Ch 12 - Integral Calculus

Gaussian Quadrature on [-1, 1]


Approximation formula for n = 3

I=

5
3 8
5
3
f ( ) + f (0) + f ( )
9
5 9
9
5

f ( x)dx =

41

Example (1): Gaussian Quadrature


Evaluate

I =

te 2 t dt = 5216 .926477

Coordinate transformation
t=
I =

ba
b+a
x+
= 2 x + 2 ; dt = 2dx
2
2

te 2 t dt =

( 4 x + 4 ) e 4 x + 4 dx =

f ( x ) dx

Two-point formula (n=2)


4

1
1
4 4 3
4 4+ 3
I = f ( x) dx = f ( ) + f ( ) = (4
)e
+ (4 +
)e
1
3
3
3
3
= 9.167657324 + 3468.376279 = 3477.543936 ( = 33.34%)
1

42

Ch 12 - Integral Calculus

Example (1): Gaussian Quadrature


Three-point formula (n=3)
1

5
8
5
f ( 0.6 ) + f (0) + f ( 0.6 )
1
9
9
9
5
8
5
4 0. 6
4
= (4 4 0.6 )e
+ ( 4 ) e + ( 4 + 4 0 . 6 ) e 4 + 0. 6
9
9
9
5
8
5
= (2.221191545) + (218.3926001) + (8589.142689)
9
9
9
= 4967.106689
( = 4.79%)

I=

f ( x)dx =

Four-point formula (n=4)


I=

f ( x)dx = 0.34785[ f (0.861136) + f (0.861136)]


+ 0.652145[ f (0.339981) + f (0.339981)]
= 5197.54375
( = 0.37%)

43

Example (2): Gaussian Quadrature


Evaluate

I=

1.64

x2
2

dx = .44949742

Coordinate transformation

t=
I=

ba
b+a
x+
= .82 x + .82 = .82(1 + x); dt = .82dx
2
2
1
2

1.64

t2
2

dt =

.82
2

1
1 [.82 (1+ x )]2
e 2
dx
1

.82
2

f ( x)dx

44

Ch 12 - Integral Calculus

Example: Gaussian Quadrature


Two-point formula (n=2)
1 2
1
1
[.82(1+
.82 1
1 .82 2 [.82(1 3 )]
2
f ( ) + f ( ) =
e
e
+

2
2
3
3
2

= 0.32713267 * (0.94171147 + 0.43323413) = .44978962 ( = 0.065%)

I=

.82

f ( x) dx =

)]2

Three-point formula (n=3)


I =

.82
2

f ( x ) dx =

.82 5
8
5

f ( 0 .6 ) + f ( 0 ) + f ( 0 .6 )
9
9
2 9

1
1
1
2
2
2
.82 5 2 [.82 (1 0.6 )]
8 2 [.82 (1 0 )]
5 2 [.82 (1+ 0.6 )]
e
=
+ e
+ e
9
9
2 9
= .32713267 * (0.5461465 9 + 0.63509351 + 0.19271450 )
= 0.44946544 ( = 0 .007 %)

45

Multidimensional Integrals
So far, we concentrated on one-dimensional integrals. To
compute integrals in multiple dimensions, one approach is to phrase
the multiple integral as repeated one-dimensional integrals.
But, eventually, we run into the so-called curse of dimensionality. Four
or more dimensions are complicated and, often, imprecise.
There are two methods that work well:
1.Monte Carlo: Based on repeated function evaluations, not
repeated integrations using one-dimensional methods.
Popular algorithm: Markov chain Monte Carlo (MCMC), which
include the Metropolis-Hastings algorithm and Gibbs sampling.
2. Sparse grids: Based on a one dimensional quadrature rule, but
uses a recursive combination of univariate results.

46

Ch 12 - Integral Calculus

Q: What's the integral of (1/cabin)d(cabin)?


A: A natural log cabin!
47

You might also like