Professional Documents
Culture Documents
Outline
Scientific Computing: An Introductory Survey
Chapter 11 Partial Differential Equations
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
Michael T. Heath
1 / 105
Scientific Computing
2 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
uxy = 2 u/xy
3 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
4 / 105
Example, continued
Advection equation
ut = c ux
where c is nonzero constant
Unique solution is determined by initial condition
u(0, x) = u0 (x),
< x <
Scientific Computing
5 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Characteristics
Scientific Computing
6 / 105
Classification of PDEs
Scientific Computing
7 / 105
Michael T. Heath
Scientific Computing
8 / 105
b2
4ac
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
9 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Time-Dependent Problems
Scientific Computing
10 / 105
Time-Dependent Problems
Time-Independent Problems
Semidiscrete Methods
One way to solve time-dependent PDE numerically is to
discretize in space but leave time variable continuous
0 x 1,
t0
0x1
Scientific Computing
11 / 105
Time-Dependent Problems
Time-Independent Problems
12 / 105
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
Method of Lines
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
t0
u(t, 1) = 0,
i = 1, . . . , n
Scientific Computing
13 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Stiffness
Scientific Computing
14 / 105
Time-Dependent Problems
Time-Independent Problems
2
1
0
0
1 2
c
0
1 2
0
y0 =
y = Ay
2
(x) .. . .
..
..
..
.
.
.
.
.
0
1 2
n
X
j (t)j (x)
Michael T. Heath
Scientific Computing
15 / 105
j=1
Michael T. Heath
Scientific Computing
16 / 105
Time-Dependent Problems
Time-Independent Problems
j0 (t)j (xi ) = c
j=1
n
X
j (t)00j (xi )
j=1
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
18 / 105
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
19 / 105
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
22 / 105
Time-Dependent Problems
Time-Independent Problems
t0
Time-Dependent Problems
Time-Independent Problems
ut = c uxx ,
20 / 105
Michael T. Heath
Scientific Computing
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
u(t, 0) = ,
uk+1
i
uk 2uki + uki1
uk+1
uki
i
= c i+1
,
t
(x)2
t k
= uki + c
u
2uki + uki1 ,
(x)2 i+1
or
i = 1, . . . , n
u(t, 1) =
Stencil : pattern of mesh points involved at each level
Michael T. Heath
Scientific Computing
23 / 105
Michael T. Heath
Scientific Computing
24 / 105
Time-Dependent Problems
Time-Independent Problems
Time-Dependent Problems
Time-Independent Problems
utt = c uxx ,
So we can march numerical solution forward in time using
this explicit difference scheme
t0
O((x)2 ),
u(0, x) = f (x),
u(t, 0) = ,
ut (0, x) = g(x)
u(t, 1) =
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
25 / 105
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
26 / 105
Time-Dependent Problems
Time-Independent Problems
uk+1
i
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
uk 2uki + uki1
uk+1
2uki + uk1
i
i
= c i+1
, or
2
(t)
(x)2
t 2 k
= 2uki uk1
+c
ui+1 2uki + uki1 , i = 1, . . . , n
i
x
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
27 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Stability
Scientific Computing
28 / 105
Time-Dependent Problems
Time-Independent Problems
For ODEs we saw that implicit methods are stable for much
greater range of step sizes, and same is true of implicit
methods for PDEs
Applying backward Euler method to semidiscrete system
for heat equation gives implicit finite difference scheme
t k+1
k+1
+ uk+1
i = 1, . . . , n
ui = uki + c
ui+1 2uk+1
i
i1 ,
2
(x)
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
29 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
30 / 105
Time-Dependent Problems
Time-Independent Problems
Crank-Nicolson Method
Applying trapezoid method to semidiscrete system of
ODEs for heat equation yields implicit Crank-Nicolson
method
t k+1
k+1
k
k
k
ui = uki +c
u
2uk+1
+ uk+1
i
i1 + ui+1 2ui + ui1
2(x)2 i+1
Scientific Computing
31 / 105
Michael T. Heath
Scientific Computing
32 / 105
Time-Dependent Problems
Time-Independent Problems
Convergence
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
Time-Dependent Problems
Time-Independent Problems
33 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Stability
Scientific Computing
34 / 105
Time-Dependent Problems
Time-Independent Problems
CFL Condition
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
35 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
36 / 105
Time-Dependent Problems
Time-Independent Problems
x
t
c
1/ c and 1/ c
unstable
stable
Scientific Computing
37 / 105
Time-Dependent Problems
Time-Independent Problems
Time-Independent Problems
38 / 105
Time-Dependent Problems
Time-Independent Problems
Dirichlet : u is specified
Neumann : ux or uy is specified
Mixed : combinations of these are specified
Scientific Computing
Scientific Computing
Michael T. Heath
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
39 / 105
Michael T. Heath
Scientific Computing
40 / 105
Time-Dependent Problems
Time-Independent Problems
Time-Dependent Problems
Time-Independent Problems
i, j = 1, . . . , n
Scientific Computing
41 / 105
Time-Dependent Problems
Time-Independent Problems
Time-Dependent Problems
Time-Independent Problems
u1,1
0.125
u2,1 0.125
x=
u1,2 = 0.375
u2,2
0.375
Scientific Computing
42 / 105
4 1 1
0
u1,1
0
1
4
0 1
u2,1 = 0 = b
Ax =
1
0
4 1 u1,2 1
1
0 1 1
4
u2,2
Michael T. Heath
Scientific Computing
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
43 / 105
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
44 / 105
Time-Dependent Problems
Time-Independent Problems
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Time-Dependent Problems
Time-Independent Problems
Scientific Computing
Michael T. Heath
Scientific Computing
46 / 105
Direct Methods
Iterative Methods
Comparison of Methods
47 / 105
Michael T. Heath
Scientific Computing
48 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Direct Methods
Iterative Methods
Comparison of Methods
Band Systems
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
49 / 105
Scientific Computing
51 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
52 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
50 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Scientific Computing
Fill
Michael T. Heath
Direct Methods
Iterative Methods
Comparison of Methods
Scientific Computing
53 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
54 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Scientific Computing
55 / 105
Michael T. Heath
Scientific Computing
56 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Direct Methods
Iterative Methods
Comparison of Methods
Scientific Computing
57 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
58 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
59 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
60 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Scientific Computing
61 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
62 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Scientific Computing
63 / 105
Michael T. Heath
Scientific Computing
64 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Direct Methods
Iterative Methods
Comparison of Methods
Splitting
with M nonsingular
xk+1 = xk + B
(b Axk )
xk+1 = M 1 N xk + M 1 b
= (I B 1 A)xk + B 1 b
which is implemented as
M xk+1 = N xk + b
c = B 1 b
It converges if
(I B 1 A) < 1
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
65 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Convergence
Scientific Computing
66 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Jacobi Method
N) < 1
Scientific Computing
67 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
68 / 105
Direct Methods
Iterative Methods
Comparison of Methods
X
(k+1)
(k)
xi
= bi
aij xj /aii , i = 1, . . . , n
j6=i
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Gauss-Seidel Method
Scientific Computing
70 / 105
Direct Methods
Iterative Methods
Comparison of Methods
X
X
(k+1)
(k+1)
(k)
xi
= bi
aij xj
aij xj
/aii
j<i
j>i
Scientific Computing
(k+1)
ui,j
71 / 105
1 (k+1)
(k+1)
(k)
(k)
u
+ ui,j1 + ui+1,j + ui,j+1
4 i1,j
Michael T. Heath
Scientific Computing
72 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Direct Methods
Iterative Methods
Comparison of Methods
Successive Over-Relaxation
= (1 )x
Michael T. Heath
Scientific Computing
(k+1)
+ xGS
x(k) )
73 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
74 / 105
Direct Methods
Iterative Methods
Comparison of Methods
x(k+1) = x(k) + D 1 (b Lx(k+1) U x(k) ) x(k)
= (D + L)1 ((1 )D U ) x(k) + (D + L)1 b
< interactive example >
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
75 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
76 / 105
Direct Methods
Iterative Methods
Comparison of Methods
(x) = 12 xT Ax xT b
attains minimum precisely when Ax = b
x0 = initial guess
r0 = b Ax0
s0 = r 0
for k = 0, 1, 2, . . .
k = rkT rk /sTk Ask
xk+1 = xk + k sk
rk+1 = rk k Ask
T r
T
k+1 = rk+1
k+1 /rk rk
sk+1 = rk+1 + k+1 sk
end
Scientific Computing
77 / 105
Direct Methods
Iterative Methods
Comparison of Methods
78 / 105
Direct Methods
Iterative Methods
Comparison of Methods
SSOR
Incomplete factorization
Polynomial
Approximate inverse
Scientific Computing
Preconditioning
Michael T. Heath
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
79 / 105
Michael T. Heath
Scientific Computing
80 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Generalizations of CG Method
Direct Methods
Iterative Methods
Comparison of Methods
Scientific Computing
81 / 105
Direct Methods
Iterative Methods
Comparison of Methods
x1
0.000
0.000
0.062
0.094
0.109
0.117
0.121
0.123
0.124
0.125
x2
0.000
0.000
0.062
0.094
0.109
0.117
0.121
0.123
0.124
0.125
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Example, continued
x3
0.000
0.250
0.312
0.344
0.359
0.367
0.371
0.373
0.374
0.375
x4
0.000
0.250
0.312
0.344
0.359
0.367
0.371
0.373
0.374
0.375
Scientific Computing
82 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Example, continued
SOR method (with optimal = 1.072 for this problem)
gives following iterates
k
0
1
2
3
4
5
x1
0.000
0.000
0.062
0.109
0.121
0.124
0.125
x2
0.000
0.000
0.094
0.117
0.123
0.125
0.125
x3
0.000
0.250
0.344
0.367
0.373
0.375
0.375
x4
0.000
0.312
0.359
0.371
0.374
0.375
0.375
Michael T. Heath
Scientific Computing
83 / 105
Direct Methods
Iterative Methods
Comparison of Methods
x4
0.000
0.335
0.365
0.373
0.375
0.375
x2
0.000
0.000
0.125
Michael T. Heath
x3
0.000
0.333
0.375
x4
0.000
0.333
0.375
Scientific Computing
84 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Scientific Computing
Gauss-Seidel
0.9206
0.9962
0.9990
0.99996
Optimal SOR
0.5604
0.8840
0.9397
0.98754
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
86 / 105
Direct Methods
Iterative Methods
Comparison of Methods
+1
on average, where
max (A)
min (A)
Jacobi
0.9595
0.9981
0.9995
0.99998
Michael T. Heath
x3
0.000
0.268
0.356
0.371
0.374
0.375
Michael T. Heath
x1
0.000
0.000
0.125
Rate of Convergence
x2
0.000
0.000
0.108
0.121
0.124
0.125
x1
0.000
0.000
0.072
0.119
0.123
0.125
Michael T. Heath
Scientific Computing
88 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Direct Methods
Iterative Methods
Comparison of Methods
Smoothers
Disappointing convergence rates observed for stationary
iterative methods are asymptotic
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
89 / 105
Michael T. Heath
Direct Methods
Iterative Methods
Comparison of Methods
Multigrid Methods
Scientific Computing
90 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Scientific Computing
91 / 105
Residual Equation
Michael T. Heath
Scientific Computing
92 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Two-Grid Algorithm
Michael T. Heath
Direct Methods
Iterative Methods
Comparison of Methods
Scientific Computing
Compute residual r = b Ax
93 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
94 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Cycling Strategies
Michael T. Heath
Scientific Computing
95 / 105
Michael T. Heath
Scientific Computing
96 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Direct Methods
Iterative Methods
Comparison of Methods
Scientific Computing
97 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Scientific Computing
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Scientific Computing
99 / 105
Direct Methods
Iterative Methods
Comparison of Methods
101 / 105
Direct Methods
Iterative Methods
Comparison of Methods
method
Dense Cholesky
Jacobi
Gauss-Seidel
Band Cholesky
Optimal SOR
Sparse Cholesky
Conjugate Gradient
Optimal SSOR
Preconditioned CG
Optimal ADI
Cyclic Reduction
FFT
Multigrid V-cycle
FACR
Full Multigrid
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
Comparison of Methods
100 / 105
Direct Methods
Iterative Methods
Comparison of Methods
2-D
k6
k 4 log k
k 4 log k
k4
k 3 log k
k3
k3
k 2.5 log k
k 2.5
k 2 log2 k
k 2 log k
k 2 log k
k 2 log k
k 2 log log k
k2
3-D
k9
k 5 log k
k 5 log k
k7
k 4 log k
k6
k4
k 3.5 log k
k 3.5
k 3 log2 k
k 3 log k
k 3 log k
k 3 log k
k 3 log log k
k3
Scientific Computing
102 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Scientific Computing
Michael T. Heath
Michael T. Heath
Partial Differential Equations
Numerical Methods for PDEs
Sparse Linear Systems
98 / 105
Direct Methods
Iterative Methods
Comparison of Methods
2-D
3
2
1.5
1.5
1.25
1
3-D
3
2.33
2
1.33
1.17
1
Scientific Computing
103 / 105
Michael T. Heath
Scientific Computing
104 / 105
Direct Methods
Iterative Methods
Comparison of Methods
Michael T. Heath
Scientific Computing
105 / 105