06 Polynomial Chaos

Algorithms for Uncertainty Quantification
Lecture 6: Polynomial Chaos Approximation 1: The

Pseudo-spectral Approach
ST 2018
Tobias Neckel
Scientific Computing in Computer Science
TUM
Repetition of Previous Lecture
• Interpolation concepts
• every continuous function can be interpolated uniquely
• interpolation with Lagrange cardinal polynomials
• interpolation error ⇔ Lebesgue constant
• uniform grid not always a good choice
T. Neckel | Algorithms for Uncertainty Quantification | L6: PC approx. 1: pseudo-spectral approach | ST 2018 2
Repetition of Previous Lecture
• Interpolation concepts
• every continuous function can be interpolated uniquely
• interpolation with Lagrange cardinal polynomials
• interpolation error ⇔ Lebesgue constant
• uniform grid not always a good choice
• Quadrature concepts
• weighted inner product spaces
• orthogonal polynomials
• examples: Lagrange (uniform weight), Hermite (Gaussian weight)
polynomials
• Gaussian quadrature
• nodes: zeros of the underlying orthogonal polynomial
• weights: integral of the Lagrange cardinal polynomial evaluated at the nodes
Concept of Building Block:
• Time: ≈ 90 minutes
• Content
• Polynomial Chaos: Basic concept & pseudo-spectral approach
• Example: Damped linear oscillator
Concept of Building Block:
• Time: ≈ 90 minutes
• Content
Polynomial Chaos: Basic concept & pseudo-spectral approach
•
Example: Damped linear oscillator
•
• Expected Learning Outcomes
• The participants can describe the basic idea of generalized
polynomial chaos (gPC) methods and underlying reasons.
• They are able to list several ways to compute the gPC coefficients
and to describe and relate the pseudo-spectral approach in this
context.
• They can give rough estimates for the dependency of the
approximation orders N and K .
• The participants are able to derive the formulas relating expectation
and variance to the coefficients of the gPC approach.
• They can indicate the relevant changes in the concept if several
instead of one parameters are uncertain, i.e. in the multivariate
context.
Agenda
Topic
Methods based on polynomial chaos approximation
Content
• polynomial chaos expansion
• orthogonal polynomials
• the pseudo-spectral approach
• example: damped linear oscillator
• extension to multivariate polynomials
• summary
Forward Propagation of Uncertainty
stochastic
model f (t, ω)
stochastic input Ω stochastic output Y
Problem
• assumption: f computationally expensive or available as a black-box
• deterministic independent variable: t (placeholder for t, x, . . .)
Forward Propagation of Uncertainty
stochastic
model f (t, ω)
Problem
• assumption: f computationally expensive or available as a black-box
• deterministic independent variable: t (placeholder for t, x, . . .)
What we want
• a good approximation of f (or statistical moments of its output Y ) that is
cheap to evaluate
Forward Propagation of Uncertainty (2)
stochastic
model f (t, ω)
Which method to use?

• remember: standard or improved Monte Carlo sampling
1. sample from the input distribution
2. solve system for each sample
3. compute statistical output properties
• slow convergence in general
• computationally expensive (many samples)
Polynomial Chaos Methods
Analogy: Fourier series

• series of trigonometric functions for periodic functions s(t)
∞
X
s(t) = ŝn sin(. . . )
n=0
Polynomial Chaos Methods
Analogy: Fourier series

• series of trigonometric functions for periodic functions s(t)
∞
X
s(t) = ŝn sin(. . . )
n=0
Polynomial chaos expansion

• idea: approximate f (t, ω) by series of polynomials
∞
X
f (t, ω) = f̂n (t) φn (ω)
n=0
• φn (ω) polynomials of degree n, f̂n (t) coefficients
Polynomial Chaos Methods (2)

• truncate series after N terms
N−1
X
f (t, ω) ≈ f̂n (t) φn (ω)
n=0
• φn (ω) orthogonal
• type of polynomials chosen w.r.t. input distribution ρ(ω)
Polynomial Chaos Methods – Checklist

N−1
X
f (t, ω) ≈ f̂n (t) φn (ω)
n=0
What we need to do
• specify type of polynomials φn (ω)
• compute coefficients f̂n (t)
• choose maximum order N
• compute statistical properties of f (t, ω) based on this approximation

N−1
X
f (t, ω) ≈ f̂n (t) φn (ω)
n=0
What we need to do
• specify type of polynomials φn (ω)
Inner Product of Functions
Remember: Scalar product of vectors

• vectors in Euclidean space: a, b
N
X
< a, b > = a · b = ai bi
i=1
(Weighted) inner product of functions

• Euclidean space → Hilbert space
• vectors → functions: p(ω), q(ω)
• sum → integral with weight function ρ(ω)
Z
< p(ω), q(ω) >ρ = p(ω) q(ω) ρ(ω) dω
supp(ρ)
T. Neckel | Algorithms for Uncertainty Quantification | L6: PC approx. 1: pseudo-spectral approach | ST 201810
Orthogonality
Remember: Orthogonal vectors

• scalar product is zero
< a, b > = 0 ⇐⇒ a ⊥ b
Orthogonal functions
• inner product is zero
< p(ω), q(ω) >ρ =

Z
p(ω) q(ω) ρ(ω) dω = 0 ⇐⇒ p(ω) ⊥ q(ω)
supp(ρ)
Univariate Orthogonal Polynomials
Orthogonal basis
• degree 0 to N − 1: φ0 , φ1 , . . . , φN−1
• orthogonal w.r.t. weight ρ(ω)
Z
< φi (ω), φj (ω) >ρ = φi (ω) φj (ω) ρ(ω) dω = γi δij
(
1, if i = j,
• Kronecker delta δij =
0, if i 6= j.
• normalization constants γi =< φi (ω), φi (ω) >ρ
Univariate Orthogonal Polynomials (2)
Orthogonal basis
Z
< φi (ω), φj (ω) >ρ = φi (ω) φj (ω) ρ(ω) dω = γi δij
Orthonormal basis
• Normalization constants are 1
1
φei = √ φi
γi
• from now on: assume φi (ω) are normalized
< φi (ω), φj (ω) >ρ = δij
Type of Polynomials

N−1
X
f (t, ω) ≈ f̂n (t) φn (ω)
n=0
What type of polynomials to use?

• chosen w.r.t. probability distribution ρ(ω)
• examples:
• Ω ∼ U(−1, 1) → Legendre polynomials
• Ω ∼ N (0, 1): → Hermite polynomials
• the idea can be generalized to all probability distributions
Legendre Polynomials
• orthogonal w.r.t. integral from -1 to 1 with weight function ρ(ω) = 1
2
Z 1
2
φi (ω) φj (ω) ρ(ω) dω = δij
−1 2i + 1
1.0
• φ0 = 1
• φ1 = ω
Legendre polynomials
1 0.5
3ω 2 − 1

• φ2 = 2
1
5ω 3 − 3ω

• φ3 = 2 0.0
1
35ω 4 − 30ω 2 + 3

• φ4 = 8
• ... −0.5
φ0
φ1
φ2
φ3
−1.0 φ4
−1.0 −0.5 0.0 0.5 1.0
x
Hermite Polynomials
• orthogonal w.r.t. integral from −∞ to ∞
2
• weight function ρ(ω) = √1 exp(− ω2 )
2π
Z ∞
φi (ω) φj (ω) ρ(ω) dx = i! δij
−∞
8
• φ0 = 1
6
• φ1 = ω
Hermite polynomials
4
• φ2 = ω 2 − 1
2
• φ3 = ω 3 − 3ω
0
• φ4 = ω 4 − 6ω 2 + 3 −2
• ... φ0
−4 φ1
φ2
−6 φ3
φ4
−8
−3 −2 −1 0 1 2 3
x
Three-Term Recursion
Computation of polynomials
• Stieltjes’ three-term recursion relation
φ−1 (ω) ≡ 0
φ0 (ω) ≡ 1
φn+1 (ω) = (An ω + Bn ) φn (ω) − Cn φn−1 (ω) n≥0
• An , Bn , Cn constants (computed by recursion, depending on weight ρ via

inner product)
• satisfied by all orthogonal polynomials
• numerically stable method for computation of polynomials
• other schemes possible, e.g. Gram-Schmidt algorithm

N−1
X
f (t, ω) ≈ f̂n (t) φn (ω)
n=0
What we need to do
• specify type of polynomials φn (ω) X
Computation of Coefficients
The pseudo-spectral approach
• exploit orthonormality of underlying basis
N−1
X
f̂n (t) φn (ω) = f (t, ω)
n=0
N−1
X
< f̂n (t) φn (ω), φm (ω) >ρ = < f (t, ω), φm (ω) >ρ
n=0
N−1
X
f̂n (t) < φn (ω), φm (ω) >ρ = < f (t, ω), φm (ω) >ρ
| {z }
n=0
δnm
f̂n (t) =< f (t, ω), φn (ω) >ρ
• other approaches: least squares, stochastic Galerkin (next lecture) . . .

The Pseudo-spectral Approach
Computation of coefficients
Z
f̂n (t) =< f (t, ω), φn (ω) >ρ = f (t, ω) φn (ω) ρ(ω) dω
Ω
• possible difficulties:
• f (t, ω) computationally expensive
• f (t, ω) available only as a black box
• solution: use quadrature
• Gaussian quadrature optimal in one dimensional settings
K
X −1
f̂n (t) = f (t, xk ) φn (xk ) wk
k =0
The Pseudo-spectral Approach (2)
Nodes and weights

K −1
• quadrature rule → nodes, weights {xk , wk }k =0
• chosen w.r.t. the input probability distribution ρ
• the underlying model f (t, ω) needs be evaluated only once at xk
Algorithm 1: compute coefficients

Require: N, K , ρ
generate polynomials φi
for k = 0 to K − 1 do
generate xk , wk
evaluate f (t, xk )
for n = 0 to N − 1 do PK −1
compute f̂n (t) = k =0 f (t, xk ) φn (xk ) wk
Polynomial chaos expansion and the pseudo-spectral approach
N−1
X
f (t, ω) ≈ f̂n (t) φn (ω)
n=0
K
X −1
k =0
What we need to do
• compute coefficients f̂n (t) X
• choose maximum order N and number of number of quadrature terms K
Number of Terms and Nodes
Number of quadrature nodes K
• expensive: evaluate f (t, xk )
• K determines computational effort
10 0
10 -1
Number of expansion terms N 10 -2
10 -3
10 -4
• f̂n (t) decay exponentially 10 -5
10 -6
|ci |
• few coefficients sufficient 10 -7
10 -8
• low computational effort once 10 -9
f (t, xk ) known 10 -10
10 -11
• rule of thumb: use N ≈ 1
2K
10 -12 0 1 2 3 4 5 6 7
i
coefficients of oscillator example

Polynomial chaos expansion and pseudo spectral approach
N−1
X
f (t, ω) ≈ f̂n (t) φn (ω)
n=0
K
X −1
k =0
What we need to do
• choose maximum order N and number of quadrature terms K X
Expectation and Variance
Expectation
• remember: φ0 (ω) ≡ 1
• E[φ0 (ω) φn (ω)] = < φ0 (ω), φn (ω) >ρ
Expectation
• E[φ0 (ω) φn (ω)] = < φ0 (ω), φn (ω) >ρ
"N−1 #
X
E[f (t, ω)] ≈ E f̂n (t) φn (ω)
n=0
N−1
X
= f̂n (t) E[1 · φn (ω)]
n=0
N−1
X
= f̂n (t) E[φ0 (ω) φn (ω)] = f̂0 (t)
| {z }
n=0 =δ0n
Expectation
• E[φ0 (ω) φn (ω)] = < φ0 (ω), φn (ω) >ρ
"N−1 #
X
E[f (t, ω)] ≈ E f̂n (t) φn (ω)
n=0
N−1
X
= f̂n (t) E[1 · φn (ω)]
n=0
N−1
X
= f̂n (t) E[φ0 (ω) φn (ω)] = f̂0 (t)
| {z }
n=0 =δ0n
Variance
h i
2
Var [f (t, ω)] = E (f (t, ω) − E[f (t, ω)])

N−1
!2  
N−1
!2 
X X
≈E f̂n (t) φn (ω) − f̂0 (x)  = E  f̂n (t) φn (ω) 
n=0 n=1
N−1
X N−1
X
f̂n2 (t) E φn (ω)2 = f̂n2 (t)

=
n=1 n=1
| {z }
=1
• squared sum → sum of squares

• mixed terms = 0 due to orthogonality: E[φn (ω) φm (ω)] = δnm
Variance
h i
2
Var [f (t, ω)] = E (f (t, ω) − E[f (t, ω)])

N−1
!2  
N−1
!2 
X X
≈E f̂n (t) φn (ω) − f̂0 (x)  = E  f̂n (t) φn (ω) 
n=0 n=1
N−1
X N−1
X
f̂n2 (t) E φn (ω)2 = f̂n2 (t)

=
n=1 n=1
| {z }
=1
• squared sum → sum of squares

• mixed terms = 0 due to orthogonality: E[φn (ω) φm (ω)] = δnm
Polynomial approximation and the pseudo-spectral approach
N−1
X
f (t, ω) ≈ f̂n (t) φn (ω)
n=0
K
X −1
k =0
What we need to do
• choose maximum order N and number of quadrature terms K X
• compute statistical properties of f (t, ω) based on this approximation X
Model Problem – Damped Linear Oscillator
 2
d y dy
 dt 2 (t) + c dt (t) + ky (t) = f cos(ωO t)

y (0) = y0
 dy

dt (0) = y1
Model Problem – Damped Linear Oscillator
 2
d y dy
 dt 2 (t) + c dt (t) + ky (t) = f cos(ωO t)

y (0) = y0
 dy

dt (0) = y1
• c – damping coefficient
• k – spring constant
• f – forcing amplitude
• ωO – frequency
• y0 – initial position
• y1 – initial velocity
Damped Linear Oscillator (2)
• c = 0.100, k = 0.035, f = 0.100, ωO = 1.000, y0 = 0.500, y1 = 0.000
• t ∈ [0, 30]
0.8
displacement
0.6 velocity
0.4
0.2
0.0
0.2
0.4
0 5 10 15 20 25 30
time
The Pseudo-spectral Approach – Example
• assume c ∼ U(0.08, 0.12)
N−1
X
y (T , ω) ≈ ŷn (T )φn (ω)
n=0
K
X −1
ŷn (T ) = y (T , xk ) φn (xk ) wk
k =0
• ŷn (T ) – coefficients
• φn (ω) – Legendre polynomials
• xk , wk – Gauss-Legendre quadrature nodes and weights
The Pseudo-spectral Approach – Example (2)
• T = 15
Deterministic result
• y (T ) = −1.51e − 01
Stochastic results – Monte Carlo sampling

• 100000 samples → E[y (T , ω)] ≈ −1.53e − 01, Var[y (T , ω)] ≈ 7.83e − 04
Stochastic results – the pseudo-spectral approach

• 5 nodes → E[y (T , ω)] ≈ −1.52e − 01, Var[y (T , ω)] ≈ 7.80e − 04
Convergence
• sufficient to use small K and N = 12 K
• oscillator example: stoch. Galerkin with 10 terms as reference
Convergence
• sufficient to use small K and N = 12 K
• oscillator example: stoch. Galerkin with 10 terms as reference
10 -2 10 -2
10 -3 10 -3
relative error in variance
relative error in variance

10 -4 10 -4
10 -5 10 -5
10 -6 10 -6
10 -7 10 -7
10 -8 3 4 5 6 7 10 -8 2 3 4 5 6 7
K N
number of quadrature nodes number of expansion terms (K=10)
Multivariate Polynomial Chaos Expansion
• random vector Ω consisting of independent random variables Ωi ,

i = 1, . . . , d
• multi-index n = (n1 , . . . , nd ) ∈ Nd0
• multivariate polynomials: product of univariate polynomials
φn (ω) = φn1 (ω1 ) · · ·φnd (ωd ),
< φn (ω), φm (ω) >w = δnm , δnm = δn1 m1 · · · δnd md
• multivariate polynomial chaos expansion:
X
f (t, ω) ≈ f̂n (t)φn (ω)
n
• use multivariate pseudo-spectral approach to obtain f̂n :
K
X −1
f̂n (t) = f (t, x k ) φn (x k ) wk
k =0
Multivariate Polynomial Chaos Expansion (2)
• multivariate polynomial chaos expansion

X
f (t, ω) ≈ f̂n (t)φn (ω)
n
• n typically chosen such as n1 + . . . nd ≤ N for a given N

• in this situation, the number of elements of
d+N
{n ∈ Nd0 : n1 + . . . nd ≤ N} =

d := P
• example
• if d = 2, N =4→P = 15
• if d = 3, N =4→P = 35
• if d = 4, N =4→P = 70
• if d = 5, N =4→P = 126
• ...
Multivariate Polynomial Chaos Expansion (2)

X
f (t, ω) ≈ f̂n (t)φn (ω)
n
• n typically chosen such as n1 + . . . nd ≤ N for a given N

• in this situation, the number of elements of
d+N
{n ∈ Nd0 : n1 + . . . nd ≤ N} =

d := P
• example
• if d = 2, N =4→P = 15
• if d = 3, N =4→P = 35
• if d = 4, N =4→P = 70
• if d = 5, N =4→P = 126
• ...
• computational cost: K , only indirectly related to P
Literature
• Chapter 10 in R. C. Smith, Uncertainty Quantification – Theory,

Implementation, and Applications, SIAM, 2014
• D. Xiu, Numerical Methods for Stochastic Computations – A Spectral
Method Approach, Princeton Univ. Press, 2010
Summary
Polynomial chaos methods

• polynomial chaos expansion
• approximate quantity of interest by polynomial series
PN−1
• f (t, ω) ≈ n=0 f̂n (t) φn (ω)
• orthogonal polynomials and polynomial chaos
• inner product 0 for orthogonal polynomials
• < φi (ω), φj (ω) >ρ = δij
• choose polynomial type according to input distribution
• the pseudo-spectral approach
• use quadrature rule to compute coefficients
PK −1
• f̂n ≈ k =0 f (t, xk ) φn (xk ) wk
• model problem: damped linear oscillator

06 Polynomial Chaos

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

06 Polynomial Chaos

Uploaded by

Copyright:

Available Formats

Algorithms for Uncertainty Quantification

Lecture 6: Polynomial Chaos Approximation 1: The

stochastic input Ω stochastic output Y

stochastic input Ω stochastic output Y

stochastic input Ω stochastic output Y

Which method to use?

Analogy: Fourier series

Analogy: Fourier series

Polynomial chaos expansion

• φn (ω) polynomials of degree n, f̂n (t) coefficients

Polynomial chaos expansion

Polynomial chaos expansion

Polynomial chaos expansion

Remember: Scalar product of vectors

(Weighted) inner product of functions

Remember: Orthogonal vectors

< p(ω), q(ω) >ρ =

• from now on: assume φi (ω) are normalized

< φi (ω), φj (ω) >ρ = δij

Polynomial chaos expansion

What type of polynomials to use?

• An , Bn , Cn constants (computed by recursion, depending on weight ρ via

Polynomial chaos expansion

f̂n (t) =< f (t, ω), φn (ω) >ρ

• other approaches: least squares, stochastic Galerkin (next lecture) . . .

Nodes and weights

Algorithm 1: compute coefficients

coefficients of oscillator example

• squared sum → sum of squares

• squared sum → sum of squares

• assume c ∼ U(0.08, 0.12)

Stochastic results – Monte Carlo sampling

Stochastic results – the pseudo-spectral approach

relative error in variance

number of quadrature nodes number of expansion terms (K=10)

• random vector Ω consisting of independent random variables Ωi ,

• multivariate polynomial chaos expansion

• n typically chosen such as n1 + . . . nd ≤ N for a given N

• multivariate polynomial chaos expansion

• n typically chosen such as n1 + . . . nd ≤ N for a given N

• Chapter 10 in R. C. Smith, Uncertainty Quantification – Theory,

Polynomial chaos methods

You might also like