Appendix B : Discrete Fourier Transform

Continuous-time Fourier transform methods are well known for analyzing frequency
characteristics of continuous-time signals. In addition, the inverse transform provides
perfect original signal reconstruction. More specifically, an arbitrary frequency
component in the signal can be extracted by forming the inner-product of the signal with
the corresponding sinusoidal basis function. Due to the orthogonality property of
sinusoidal basis functions, each frequency component, which is called the Fourier
transform coefficient, exclusively contains the desired frequency component. At the same
time, the inverse transform can be performed using the exact procedure of the forward
transform, except that the complex conjugated basis function is used.
Fourier transforms provide a very powerful tool for the mathematical analysis and
synthesis of signals. For continuous signals, the work can be performed using paper and
pencil calculations. However, in image processing (and many other fields) where signals
are digitized and the resulting arrays are large scale, we find that the computations
become digital and thus the discrete Fourier transform (DFT) is used. The advantage
here is that the DFT work can easily be performed with the digital computer using well
known DFT methods along with FFT algorithms.
In this appendix, the basic material needed for applying the DFT and IDFT to discrete
signals is presented.

B.1 One -Dimensional Discrete Fourier Transform

The unitary DFT of a sequence {u ( n), n = 0,  , N −1} is defined as

N −1
1
( B.0) v(k ) =
N
∑u (n)W
n =0
N
kn
, k = 0, 1,  , N −1 ,

where

 2π 
( B.0) WN = exp  − j .
 N 

Let the matrix F1 be defined as follows:

1
 W N 0 W N 0 WN
0
 WN 
0

 0 1 2 N −1 
WN WN WN  WN 
1  0 2 ( N −1) 
( B.0) F1 = WN WN
2
WN
4
 WN .
N 
      
W 0 N − 1 2 ( N −1) ( N −1) 2 
 N WN WN  WN 
Then, equation ( B.0) can be expressed in matrix-vector representation, such as

( B.0) v = F1u ,

where v and u represent vectors whose elements take values of the discrete signals
v (k ) and u (n) , respectively.

The inverse transform is given by

N −1
1
( B.0) u ( n) =
N
∑v(k )W
n =0
N
−kn
, n = 0, 1,  , N −1 .

nk
Because the complex exponential function, WN , is orthogonal and F1 is
symmetric, we have that

−1 *T *
( B.0) F1 = F1 = F1 ,

where the superscript * represents the conjugate of a complex matrix. According to

equation ( B.0), the inverse transform can be expressed in vector-matrix form as

*
( B.0) u = F1 v .

The DFT of equation ( B.0), and the inverse expression of equation ( B.0), form a
transform pair. This transform pair has the property of being unitary because the matrix
F1 is unitary. In other words, the conjugate transpose of F1 is equal to its inverse, as
expressed in equation ( B.0). In many applications, differently scaled DFTs and their
inverses may be used. In such cases, most properties, which will be summarized in the
next section, hold with proper adjustment of a scaling factor.

2
B.4 Properties of the DFT

B.4.1 Periodicity
The extensions of the DFT and its inverse transform are periodic with period N . In
other words, for every k ,

( B.0) v(k + N ) = v(k ) ,

and for every n

( B.0) u (n + N ) = u ( N ) .

B.4.2 Conjugate symmetry

The DFT of a real sequence is conjugate symmetric about N / 2 , where we assume N
is an even number. By applying the periodicity of the complex exponential, such as

N  2π 
WN = exp  − j ⋅ N  = 1 , we obtain
 N 

( B.0) v * ( N − k ) = v( k ) .

N
From equation ( B.0), we see that, for k = 0, , −1 ,
2

N  N 
( B.0) v − k  = v *  + k  ,
 2   2 

and

N  N 
( B.0) v  − k  = v + k  .
2  2 

According to the conjugate symmetry property, only N / 2 DFT coefficients

completely determine the frequency characteristics of a real sequence of length N .
More specifically, the N real sequence

3
( B.0) v (0), Re{v (1)}, Im{v (1)},K , Re { v ( N / 2 − 1) } , Im { v ( N / 2 − 1) } , v ( N / 2 )

completely defines the DFT of the real sequence. It is clear from equation ( B.0) that
v(0) is real and from equation ( B.0) that v ( N / 2) is real.

B.4.3 Relationships between DFT basis and circulant matrices

The basis vectors of the DFT are the orthonormal eigenvectors of any circulant
matrix. The eigenvalues of a circulant matrix are the DFT of its first column. Based on
these properties, the DFT is used to diagonalize any circulant matrix.
To prove that the basis vectors of the DFT are the orthonormal eigenvectors of any
circulant matrix, we must show that

( B.0) Hφk = λk φk ,

where H represents a circulant matrix φk , the k th column of the matrix F1 = F1 ,

*T *

and λk the corresponding eigenvalue. Since H is circulant, it can be represented as

 h0 hN −1 hN −2  h1 
 h h0 hN −1  h2 
 1
( B.0) H =  h2 h1 h0  h3  .
 
     
hN −1 hN −2 hN −3  h0 

*T *
The k th column of the matrix F1 = F1 is represented as

( B.0) φk =
1
N
WN [
0
WN
−k
 WN ]
−( N −1) k T
.

From equations ( B.0) and ( B.0), the m th element of Hφk in ( B.0) is obtained as

N −1
1
em [ Hφk ] = ∑h(m − n)W
T −kn
( B.0) N ,
N n =0

4
where em represents the m th unit vector, for example, e1 = [1 0  0] T . By
changing variables and using the periodicity of the complex exponential function, W N ,
we can rewrite equation ( B.0) as

 N −1 kl  1 − km 
em [ Hφk ] =  ∑ h(l )WN 
T
( B.0) WN  ,
 l =0  N 

which results in equation ( B.0). The eigenvalues of the circulant matrix H are defined
as

N −1
( B.0) λk = ∑h(l )WN kl , for k =0, 1, , N −1 .
l =0

From ( B.0), we see that the eigenvalues of a circulant matrix are the DFT of its first
column.
Furthermore, since equation ( B.0) holds for k =0, 1, , N −1 , we can write

( B.0) H [φ0 φ1  φ N −1 ] = [φ0 φ1  φ N −1 ] Λ ,

where Λ = diag { λ0 λ1  λ N −1 } . We note that [φ0 φ1  φ N −1 ] = F1* , then

expression ( B.0) reduces to

* *
( B.0) HF1 = F1 Λ .

By multiplying F1 to the left hand side of ( B.0), we obtain

*
( B.0) F1 HF1 = Λ .

Equation ( B.0) shows that any circulant matrix can be diagonalized by multiplying
the DFT matrix and its conjugate on the left and the right-hand sides, respectively.

B.5 Two -Dimensional Discrete Fourier Transform

The two-dimensional DFT of an N × N sequence u (m, n) is defined as

5
 N −1 N −1
1
( B.0) v(k , l ) =
N
∑∑u (m, n)W
m =0 n =0
N
km
WN
nl
, for k , l =0, 1,  , N −1

and the inverse transform is

N −1 N −1
1
∑∑v(k , l )W −km −nl
( B.0) u ( m, n) = N WN , for m, n =0, 1,  , N −1 .
N k =0 l =0

There are two major categories of the two-dimensional DFT applications, such as (i)
two-dimensional spatial frequency analysis and filtering and (ii) diagonalization of block
circulant matrices for efficient computation of two-dimensional convolution.

B.5.1 Basis images of the two-dimensional DFT

In order to analyze spatial frequency characteristics of a N × N two-dimensional
signal, we consider N 2 basis images of the same size N × N . If each basis image
contains unique two-dimensional spatial frequency components exclusively, we can
compute the desired frequency component in the given image by finding the inner
product of the given image and the corresponding basis image. If we form N 2 basis
images by taking the outer product of two vectors that are permutations from
{φk , k = 0,  , N − 1} , defined in equation ( B.0), each inner product of the given image
and the corresponding basis image is equal to the two-dimensional DFT.
In matrix notation, equations ( B.0) and ( B.0), respectively, become

( B.0) V = F1UF 1 ,

and

* *
( B.0) U = F1 VF1 ,

where U and V represent N × N matrices whose elements are mapped to two-

dimensional signals u (m, n) and v( k , l ) , respectively.
The two-dimensional DFT basis images are given by

Bkl = φk φl , for k , l =0,  , N −1 .

T
( B.0)

6
We note that equation ( B.0) is mathematically equivalent to the set of inner products of
the given image and basis images defined in equation ( B.0).
Given two-dimensional DFT coefficients, which are elements of V , the two-
dimensional signal can be reconstructed by summation of all basis images weighted by
the given DFT coefficients. This reconstruction process is mathematically equivalent to
equation ( B.0).

B.5.2 Diagonalization of block circulant matrices

Let u and v be the lexicographically ordered vectors for two-dimensional signals
u ( m, n) and v ( k , l ) , respectively. The two-dimensional DFT matrix is defined as

( B.0) F = F1 ⊗F1 ,

where ⊗ represents the Kronecker product of matrices and F1 represents the one-
dimensional DFT matrix. According to the property of Kronecker product, we see that
the two-dimensional DFT matrix is also symmetric, that is,

F T = ( F1 ⊗ F1 ) = F1 ⊗ F1 = F1 ⊗ F1 = F .
T T T
( B.0)

The two-dimensional DFT is written in matrix-vector form as

( B.0) v = Fu .

In order to investigate the orthogonality of the two-dimensional DFT matrix, we have

* *
F *T F = F * F = ( F1 ⊗ F1 ) * ( F1 ⊗ F1 ) = ( F1 ⊗ F1 )( F1 ⊗ F1 )
( B.0) * *
= ( F1 F1 ⊗ F1 F1 ) = I N ⊗ I N = I N 2 ×N 2

From equation ( B.0), we know that

( B.0) F *T = F * = F −1 ,

which yields the following inverse transform

( B.0) u = F *v .

7
 We note that the matrix vector notation of the two-dimensional DFT in equation
(B.30) is equivalent to both equations ( B.0) and ( B.0), and that ( B.0) is equivalent to
both equations ( B.0) and ( B.0).
Consider the two-dimensional circular convolution

N −1 N −1

( B.0) y (m, n ) = ∑∑ h( m − p, n − q)C x ( p, q) ,

p =0 q = 0

where

( B.0) h( m, n) C = h( m mod N , n mod N )

The reason for using the circular convolution in image processing is to deal with
boundary problems. In other words, when processing an image with a convolution
kernel, boundary pixels do not have a sufficient number of neighboring pixels. In order
to compensate for this pixel shortage, we may use one of the following: (1) assign zero
values to nonexistent neighbors, or (2) replicate the value of the outermost existing pixel
to its nonexistent neighbors, or (3) suppose that the input image is periodic with a period
that is the same as the size of the image.
Although none of the three methods give us the ideal solution for boundary problems,
the third method that assumes two-dimensional periodicity allows us to use circular
convolution, which can be diagonlized by using a two-dimensional DFT.
Given ( p, q) , we can obtain the two-dimensional DFT of h( m − p, n − q )C as

N −1 N −1
DFT { h (m − p, n − q)C } = ∑∑ h( m − p, n − q)C WN
mk + nl
( B.0) .
m =0 n = 0

Writing i = m − p , j = n − q , and using equation ( B.0), we can rewrite equation ( B.0) as

( B.0)

N −1− p N −1− q N −1 N −1
WN
pk + ql
∑ ∑
i =− p i =− q
h (i, j )C WN
ik + jl
= WN
pk + ql
∑∑ h(m, n)W
m= 0 n= 0
N
mk + nl
= WN
pk + ql
DFT { h( m, n )} .

8
 Since equation ( B.0) holds for p, q = 0, K , N − 1 , the right side of ( B.0) can be
expressed as

( B.0) ( F1 ⊗ F1 ) hp,q ,

where h p ,q , represents the N 2 ×1 row-ordered vector obtained from the rotated version

of the N × N matrix {h( m, n)} by ( p, q ) . By equating ( B.0) and ( B.0), for

p, q = 0, K , N − 1 , we obtain

( B.0) ( F1 ⊗ F1 ) H = D ( F1 ⊗ F1 ) ,

where H represents the N 2 × N 2 doubly block circulant matrix, and D the diagonal
matrix whose N 2 diagonal elements are equal to the two-dimensional DFT of h( m, n) .
Equation ( B.0) can be rewritten as

( B.0) FH = DF or FHF −1 = FHF * = D ,

which shows that a doubly block circulant matrix is diagonalized by the two-dimensional
unitary DFT.
Furthermore, if a double block circulant matrix is used to represent the two-
dimensional circular convolution, such as

( B.0) y = Hx ,

then the two-dimensional DFT of the output can be obtained by multiplying the DFT
matrix, as

( B.0) Fy = FHx = FHF * Fx = D ⋅ Fx .

Equation ( B.0) can be rewritten as

( B.0) DFT { y (m, n)} = DFT {h( m, n)} ⋅ DFT {x (m, n)} .

After having the DFT of the output, its inverse transform can easily be obtained by
multiplying the conjugate of the two-dimensional DFT matrix.