Fourier Series and Transforms

Orthogonal functions
Fourier Series
Discrete Fourier Series
Fourier Transform
Chebyshev polynomials

Scope: we are trying to approximate

an arbitrary function and obtain
basis functions with appropriate
coefficients.
Fourier Theory

Fourier Series
The Problem
we are trying to approximate a function f(x) by another function g n(x)
which consists of a sum over N orthogonal functions (x) weighted by
some coefficients an.
N

f ( x) g N ( x) ai i ( x)
i 0

Fourier Theory

The Problem
... and we are looking for optimal functions in a least squares (l 2) sense ...

f ( x) g

f ( x) g N ( x)
2

1/ 2

( x) dx

Min !

... a good choice for the basis functions (x) are orthogonal functions.
What are orthogonal functions? Two functions f and g are said to be
orthogonal in the interval [a,b] if
b

f ( x) g ( x)dx 0
a

How is this related to the more conceivable concept of orthogonal

vectors? Let us look at the original definition of integrals:
Fourier Theory

Orthogonal Functions
b

f ( x) g ( x)dx lim f ( x) g ( x)x

N

i 1

... where x0=a and xN=b, and xi-xi-1=x ...

If we interpret f(xi) and g(xi) as the ith components of an N component
vector, then this sum corresponds directly to a scalar product of vectors.
The vanishing of the scalar product is the condition for orthogonality of
vectors (or functions).

fi
Fourier Theory

gi

fi gi fi gi 0
i

Periodic functions
Let us assume we have a piecewise continuous function of the form

f ( x 2 ) f ( x)
40

f ( x 2 ) f ( x) x 2

30
20
10
0
-15

-10

-5

10

15

20

... we want to approximate this function with a linear combination of 2

periodic functions:

1, cos( x ), sin( x ), cos( 2 x ), sin( 2 x ),..., cos(nx ), sin( nx )

N
1
f ( x) g N ( x) a0 ak cos(kx) bk sin( kx)
2
k 1
Fourier Theory

Orthogonality
... are these functions orthogonal ?

jk

cos(
jx
)
cos(
kx
)
dx

2 j k 0

jk 0

0 j k , j, k 0

sin(
jx
)
sin(
kx
)
dx

jk 0

cos( jx ) sin(kx)dx 0

j 0, k 0

... YES, and these relations are valid for any interval of length 2.
Now we know that this is an orthogonal basis, but how can we obtain the
coefficients for the basis functions?
from minimising f(x)-g(x)
Fourier Theory

Fourier coefficients
optimal functions g(x) are given if

g n ( x) f ( x)

Min!

or

a k

( x) f ( x)

... with the definition of g(x) we get ...

g n ( x) f ( x)
ak

2
2

ak

N
1

a
cos(
kx
)

b
sin(
kx
)

f
(
x
)

0
k
k

2
k 1

dx

leading to
N
1
g N ( x) a0 ak cos(kx) bk sin( kx)
2
k 1

1
ak

1
bk

Fourier Theory

f ( x) cos(kx)dx,

with

k 0,1,..., N

f ( x) sin(kx)dx,

k 1,2,..., N

Fourier approximation of |x|

... Example ...

f ( x) x ,

leads to the Fourier Serie

g ( x)

1
4 cos( x ) cos(3 x ) cos(5 x )

...

2

12
32
52

.. and for n<4 g(x) looks like

4
3
2
1
0
-20
Fourier Theory

-15

-10

-5

10

15

20

Fourier approximation of x2
... another Example ...

0 x 2

f ( x) x 2 ,

leads to the Fourier Serie

N
4 2
4
4

g N ( x)
2 cos( kx )
sin( kx)
3
k

k 1 k

.. and for N<11, g(x) looks like

40
30
20
10
0
-10
-10

Fourier Theory

-5

10

15

Fourier - discrete functions

... what happens if we know our function f(x) only at the points

2
xi
i
N
it turns out that in this particular case the coefficients are given by
2
ak
N
*

2
bk
N
*

f (x
j 1

) cos( kx j ),

k 0,1,2,...

) sin( kx j ),

k 1,2,3,...

f (x
j 1

.. the so-defined Fourier polynomial is the unique interpolating function to

the function f(xj) with N=2m

1 * m 1 *
1
g ( x ) a 0 a k cos(kx) b*k sin( kx) am* cos(kx)
2
2
k 1
*
m

Fourier Theory

10

Fourier - collocation points

... with the important property that ...

g m* ( xi ) f ( xi )
... in our previous examples ...
3.5
3
2.5
2
1.5
1
0.5
0
-10

-5

10

f(x)=|x| => f(x) - blue ; g(x) - red; xi - +

Fourier Theory

11

Fourier series - convergence

f(x)=x2 => f(x) - blue ; g(x) - red; xi - +
N=4

50
40
30
20
10
0
-10
-10

-5

10

15

10

15

N=8

50
40
30
20
10
0
-10
-10

Fourier Theory

-5

12

Fourier series - convergence

f(x)=x2 => f(x) - blue ; g(x) - red; xi - +
N = 16

50
40
30
20
10
0
-10
-10

-5

10

15

10

15

N = 32

50
40
30
20
10
0
-10
-10

Fourier Theory

-5

13

Gibbs phenomenon
f(x)=x2 => f(x) - blue ; g(x) - red; xi - +

N = 16

N = 64

N = 32

-2

-2

-2

-4

-4

-4

-6

0.5
1
N = 128

1.5

-6

-2

-2

-4

-4

-6

Fourier Theory

0.5

1.5

-6

0.5
1
N = 256

1.5

-6

0.5

1.5

The overshoot for equispaced Fourier

interpolations is 14% of
the step height.
0

0.5

1.5

14

Chebyshev polynomials
We have seen that Fourier series are excellent for interpolating
(and differentiating) periodic functions defined on a regularly
spaced grid. In many circumstances physical phenomena which
are not periodic (in space) and occur in a limited area. This quest
leads to the use of Chebyshev polynomials.
We depart by observing that cos(n) can be expressed by a
polynomial in cos():

cos(2 ) 2 cos 2 1
cos(3 ) 4 cos 3 3 cos
cos(4 ) 8 cos 4 8 cos 2 1
... which leads us to the definition:

Fourier Theory

15

Chebyshev polynomials - definition

cos(n ) Tn (cos( )) Tn ( x),

x cos( ),

x [1,1],

n N

... for the Chebyshev polynomials Tn(x). Note that because of

x=cos() they are defined in the interval [-1,1] (which - however can be extended to ). The first polynomials are

T0 ( x) 1
T1 ( x) x
T2 ( x) 2 x 2 1
T3 ( x) 4 x 3 3 x
T4 ( x) 8 x 4 8 x 2 1
Tn ( x) 1
Fourier Theory

for

where

x [1,1]

and n N 0
16

Chebyshev polynomials - Graphical

The first ten polynomials look like [0, -1]
1

T _ n (x)

0.5

-0.5

-1

0.2

0.4

0.6

0.8

The n-th polynomial has extrema with values 1 or -1 at

x
Fourier Theory

( ext )
k

k
cos(
),
n

k 0,1,2,3,..., n
17

Chebyshev collocation points

These extrema are not equidistant (like the Fourier extrema)

x(k)

x
Fourier Theory

( ext )
k

k
cos(
),
n

k 0,1,2,3,..., n
18

Chebyshev polynomials - orthogonality

... are the Chebyshev polynomials orthogonal?
Chebyshev polynomials are an orthogonal set of functions in the
interval [-1,1] with respect to the weight function 1 / 1 x 2
such that
1

Tk ( x)T j ( x)

/2
1 x2

dx

for

k j

for k j 0 ,
for k j 0

k, j N0

... this can be easily verified noting that

x cos ,
dx sin d
Tk ( x) cos(k ), T j ( x) cos( j )
Fourier Theory

19

Chebyshev polynomials - interpolation

... we are now faced with the same problem as with the Fourier
series. We want to approximate a function f(x), this time not a
periodical function but a function which is defined between [-1,1].
We are looking for gn(x)
n
1
f ( x) g n ( x) c0T0 ( x) ck Tk ( x)
2
k 1

... and we are faced with the problem, how we can determine the
coefficients ck. Again we obtain this by finding the extremum
(minimum)

ck

Fourier Theory

( x) f ( x)

dx

1 x
2

20

Chebyshev polynomials - interpolation

... to obtain ...

2
ck

f ( x)Tk ( x)

dx
1 x

k 0,1,2,..., n

... surprisingly these coefficients can be calculated with FFT

techniques, noting that

2
ck f (cos ) cos kd ,
0

k 0,1,2,..., n

... and the fact that f(cos) is a 2-periodic function ...

1
ck f (cos ) cos kd ,

k 0,1,2,..., n

... which means that the coefficients ck are the Fourier coefficients
ak of the periodic function F()=f(cos )!
Fourier Theory

21

Chebyshev - discrete functions

... what happens if we know our function f(x) only at the points

xi cos i
N
in this particular case the coefficients are given by
2
ck
N
*

f (cos
j 1

) cos( k j ),

k 0,1,2,... N / 2

... leading to the polynomial ...

m
1 *
g ( x ) c0T 0 ck*Tk ( x )
2
k 1
*
m

... with the property

g m* ( x ) f ( x)

Fourier Theory

at

x j cos(j/N)

j 0,1,2,..., N

22

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; gn(x) - red; xi - +
N=8

1
0.8

8 points

0.6
0.4
0.2
0
-1

-0.8

-0.6

-0.4

-0.2

0.2

0.4

0.6

0.8

N = 16

1
0.8
0.6

16 points

0.4
0.2
0
-1

Fourier Theory

-0.8

-0.6

-0.4

-0.2

0.2

0.4

0.6

0.8

23

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; gn(x) - red; xi - +
N = 32

1
0.8

32 points

0.6
0.4
0.2
0
-1

-0.8

-0.6

-0.4

-0.2

0.2

0.4

0.2

0.4

0.6

0.8

N = 128

1
0.8

128 points

0.6
0.4
0.2
0
-1

Fourier Theory

-0.8

-0.6

-0.4

-0.2

0.6

0.8

24

Chebyshev - collocation points - x2

f(x)=x2 => f(x) - blue ; gn(x) - red; xi - +
N=8

1.2
1
0.8

8 points

0.6
0.4
0.2
0
-1

-0.8

-0.6

-0.4

-0.2

0
N = 64

0.2

0.4

0.6

0.8

-0.8

-0.6

-0.4

-0.2

0.2

0.4

0.6

0.8

1.2
1

The interpolating
function gn(x) was
shifted by a small
amount to be
visible at all!

0.8

64 points

0.6
0.4
0.2
0
-1

Fourier Theory

25

Chebyshev vs. Fourier - numerical

Chebyshev

Fourier

N = 16

N = 16
35

0.8

30
25

0.6

20
15

0.4

10
5

0.2

0
0

0.2

0.4

0.6

0.8

-5

f(x)=x2 => f(x) - blue ; gN(x) - red; xi - +

This graph speaks for itself ! Gibbs phenomenon with Chebyshev?
Fourier Theory

26

Chebyshev vs. Fourier - Gibbs

Chebyshev

Fourier

N = 16

1.5

1.5

0.5

0.5

-0.5

-0.5

-1

-1

-1.5
-1

-0.5

N = 16

0.5

-1.5

f(x)=sign(x-) => f(x) - blue ; gN(x) - red; xi - +

Gibbs phenomenon with Chebyshev? YES!
Fourier Theory

27

Chebyshev vs. Fourier - Gibbs

Chebyshev

Fourier

N = 64

N = 64

1.5

1.5

0.5

0.5

-0.5

-0.5

-1

-1

-1.5
-1

-0.5

0.5

-1.5

f(x)=sign(x-) => f(x) - blue ; gN(x) - red; xi - +

Fourier Theory

28

Fourier vs. Chebyshev

Chebyshev

Fourier
2
xi
i
N

collocation points

periodic functions

domain

cos( nx ), sin( nx )

basis functions

g m* ( x)

1 *
a0
2
m 1

a *k cos( kx) b*k sin( kx)

k 1

xi cos

interpolating
function

i
N

limited area [-1,1]

Tn ( x ) cos( n ),
x cos

m
1 *
g ( x ) c0T 0 ck*Tk ( x )
2
k 1
*
m

1 *
am cos( kx)
2

Fourier Theory

29

Fourier vs. Chebyshev (contd)

Chebyshev

Fourier
2
ak
N
*

2
bk
N
*

f (x
j 1

) cos(kx j )
coefficients

f (x
j 1

2
ck
N
*

f (cos
j 1

) cos( k j )

) sin( kx j )

Gibbs phenomenon for

discontinuous functions
Efficient calculation via FFT
infinite domain through
periodicity

limited area calculations

some properties

grid densification at boundaries

coefficients via FFT
excellent convergence at
boundaries
Gibbs phenomenon

Fourier Theory

30