Fast Fourier Transform Algorithms g

(Efficient Computation of the DFT)

Fast Fourier Transform

References:
[1] John G Proakis D G Manolakis Digital Signal G. Proakis, D.G. Manolakis, Digital Processing- Principles, Algorithms & Applications PHI., 3rd Edition, Chapter-6. , p [2] L. R. Rabiner & B. Gold, Theory and Application of DSP, PHI., Chapter-6

JIIT Noida, Slide 1

JIIT Noida, Slide 2

Review of Discrete Fourier Transform

DFT x(n) X (k ) N

Computation complexity in directly computing DFT

X (k ) = x ( n )W Nkn
n =0 N 1

k = 0,1,....., N 1 , ,

X (k ) = x ( n )W Nkn
n =0

N 1

k = 0,1,....., N 1

Complex Multiplications required to compute one value of X(k) = N Complex Multiplications required to compute N values of X(k) = N2

x(n) =

1 X (k )WNkn N k =0
j 2 N

N 1

n = 0,1....., N 1
Complex Additions required to compute one value of X(k) = N-1
k k WN + N = WN

Here WN = e

WN

k+

N 2

k = WN

Complex Additions required to compute N values of X(k) = N(N-1) =N2-N

JIIT Noida, Slide 3

JIIT Noida, Slide 4

Complex Multiplication => (a + jb)(c + jd ) = (ac bd ) + j (ad + bc) Complex Addition =>

(a + jb) + (c + jd ) = (a + c) + j (b + d )
One complex Addition involves: 2 real additions

Why the FFT ?

If we look at the equation for the Discrete Fourier Transform we will find that it is complicated to work out as it involves many additions and multiplications involving complex numbers. i l i l b Even a simple eight sample signal would require 64 complex multiplications and 56 complex additions to work out the DFT DFT. At this level it is still manageable, however a realistic signal could have 1024 samples which requires over 20,000,000 complex multiplications and additions. As we can see the number of calculations required soon mounts up to unmanageable q p g proportions.

One complex Multiplication involves: 4 real multiplications 2 real additions N2 complex Multiplication involves: 4N2 real multiplications 2N2 real additions

N(N-1) complex Addition involves: 2N2-2N real additions

Total Real Multiplications required to compute N-point DFT = 4N2 N point Total Real Additions required to compute N-Point DFT = 4N2-2N

Computational Complexity = O(N2)

JIIT Noida, Slide 5

JIIT Noida, Slide 6

The Fast Fourier Transform is a simply a method of py laying out the computation, which is much faster for large values of N, where N is the number of samples in the sequence sequence. The idea behind the FFT is the divide and conquer approach, to break up the original N point sample into two (N / 2) sequences. ( ) q This is because a series of smaller problems is easier p to solve than one large one.

The Th DFT requires N2 complex multiplications and i l l i li i d N(N-1) complex additions as opposed to the FFT's approach (N/2*Log2N) of breaking it down into a series of 2 point samples which only require 1 multiplication and 2 additions and the recombination of the points which is minimal. For example Seismic Data contains hundreds of thousands of samples and would take months to evaluate the DFT. Therefore we use the FFT.

JIIT Noida, Slide 7

JIIT Noida, Slide 8

Divide & Conquer Approach to compute N-point DFT

Divide & Conquer approach is not applicable if N is a prime number. In particular approach is very efficient when N is highly composite i.e. N may be written as:

Radix-2 FFT algorithm (Special case of Divide and Conquer Approach)

In radix - 2 FFT algorithms N = 2v.

Length of any sequence can be made equal to 2v by zero padding. e gt o a y seque ce ca ade equa e o padd g.

N = r1r2 .......rv

Where {rj , 1 j v} are prime

If x(n) is a sequence of length N=2v.

Say f l (n) = x(2n)

& g l (n) = x(2n + 1) , n = 0,1,....,

N 1 2

Let us take a case of particular importance in which,

r1 = r2 = ..... = rn = r

N = rv

In such as case DFTs are of size r, so that the computation of the N-point DFT has a regular pattern. The number r is called the radix of FFT.
JIIT Noida, Slide 9 JIIT Noida, Slide 10

Substitute n = 2n in first summation and n=2n+1 in second summation.

Here,

2k N

=e

2 2k N

=e

2 k N 2

k = WN 2

If N=2v then N-1 will be odd and N-2 will be even..

JIIT Noida, Slide 11

JIIT Noida, Slide 12

Also WN Al

k+

N 2

=W W
k N

N 2 N

=W e
k N

2 N N 2

k k = WN e j = WN

JIIT Noida, Slide 13

JIIT Noida, Slide 14

Summary

Decimation in Time
x(n) X (k )
DFT N
Visual Representation of Ti Vi l R t ti f Time D i ti Decimation
N 1 n =0

X (k ) = x ( n )W Nkn
Here f l (n) = x(2n)

k = 0,1,....., N 1
N 1 2

& g l (n) = x(2n + 1) , n = 0,1,....,

k X (k ) = Fl (k ) + WN Gl (k )

N k X k + = Fl (k ) WN Gl (k ) 2

Decimation is the process of breaking down something into it's constituent parts. p Decimation in time involves breaking down a signal in the time domain into smaller signals, each of which is easier t h dl i to handle. If the input (time domain) signal, of N points, is x(n) then the frequency response X(k) can be calculated by using the DFT.
kn X (k ) = x(n)WN n=0 N 1

k = 0,1,...........,

N 1 2

k = 0,1,.....,N 1

JIIT Noida, Slide 15

JIIT Noida, Slide 16

We t t by taking this i l d breaking down i t t W start b t ki thi signal and b ki it d into two equal l parts, made up of the odd and even numbered samples. p g g y This process of decimating the signal can easily be visualised. The first breakup into two N/2 point DFT can be shown as

The Recombine Algebra mentioned in the diagram is just used to combine the samples again in the correct order. The decimation process is taken another stage by breaking down the N/2 point DFTs into N/4 point DFTs DFTs.

JIIT Noida, Slide 17

JIIT Noida, Slide 18

This is repeated again and again until we reach a series of t o ntil e two point DFT's e.g. For an 8 sample signal:

Since the recombination algebra takes N/2 complex multiplications and there are log2 (N) stages, the approx number of complex multiplication is (N/2)log2N. This means that this decimation approach has reduced the number of complex multiplications from N squared (N2) to N/2 log2 (N). At high values of N (i.e., large signals) this is a massive saving. The question still remains: What is the recombination algebra ? The Th recombination uses fl bi i flow graph notation and i h i d is called a butterfly network.

JIIT Noida, Slide 19

JIIT Noida, Slide 20

FFT Algorithm
The FFT has a fairly easy algorithm to implement, and it is sho n step b step in the list belo This version of the shown by below. ersion FFT is the Decimation in Time Method. Pad input sequence, of N samples, with ZERO's until the p q p number of samples is the nearest power of two. e.g. 500 samples are padded to 512 (2^9) Bit reverse the input sequence. e.g. 3 = 011 goes to 110 = 6 Compute (N / 4) two sample DFT's from the shuffled inputs. Compute ( / 2) f C (N four sample DFT's f ' from the two sample DFT's. Compute (N) eight sample DFT's from the four sample DFT s DFT's. Until the all the samples combine into one N-sample DFT.

Shuffled Inputs
The Th process of d i ti th signal i th ti f decimating the i l in the time d domain h i has caused the INPUT samples to be re-ordered. It will be shown that the input sequence must be stored in bit-reversed order for the output to be computed in natural order. For an 8 point signal the original order of the samples is 0, 1, 2, 3, 4, 5, 6, 7 But B t after decimation the order is 0 4 2 6 1 5 3 7 0, 4, 2, 6, 1, 5, 3, At first it may look as if there is no order to this new sequence, BUT if the numbers are represented as binary a patter soon becomes apparent.

1. 2. 3. 4. 4 5. 6.

JIIT Noida, Slide 21

JIIT Noida, Slide 22

Butterfly
The Th FFT b butterfly i a graphical method of showing fl is hi l h d f h i multiplications and additions involving the samples. Standard S d d graph fl h flow notation i used where each circle i is d h h i l with entering arrows is an addition of the two values at the end of the arrows multiplied by a constant constant. The constant is a number which appears beside the arrow, arrow if there is no value then the constant is taken as one. Here s Here's a simple example that multiplies the two inputs and adds them together......

This new sequence is the order that the samples enter the FFT.

JIIT Noida, Slide 23

JIIT Noida, Slide 24

Using this notation we can construct butterfly networks that that, together, perform the FFT. Two examples of Time Decimation Butterflies.

In the above diagram each circle is the sum of it's inputs (addition) and the values which appear next to the arrows are the multiplications.

JIIT Noida, Slide 25

JIIT Noida, Slide 26

Illustration of Radix-2 FFT algorithm (Say N= 23 =8)

x(0) x(1) y Say x = . then, . x(8)

x(0) x(1) x(2) x(3) & g1 = f1 = x( ) x(4) (5) x(6) x(7)

x(0) x(2) then, Now f1 = ( ) x(4) x(6)

x(0) x(2) & g2 = f2 = x( ) x(6) (4)

f1

4-Point DFT

F1
Combine to find 8 point DFT

f2 X g2

2-Point DFT

F2
Combine to find 4 point DFT

F1

g1

4-Point DFT

G1

2-Point DFT

G2

JIIT Noida, Slide 27

JIIT Noida, Slide 28

Similarly
x(1) x(3) then, Now g1 = x(5) x(7) x(1) x(3) & g3 = f3 = x(5) x(7)
x (0) x(4)
x(2) x (6)

F2

2-Point 2 P i t DFT 4-Point DFT

G2

F1

2-Point DFT
F3 G1

f3

F3
2-Point DFT Combine to find p 4 point DFT

G1

x (1 ) x (5)

2-Point DFT

Combine to find 8 point DFT 4-Point DFT

g3

G3
2-Point DFT

x (3) x (7 )

2 Point 2-Point DFT STAGE-1

G3

STAGE-2
JIIT Noida, Slide 30

STAGE-3

JIIT Noida, Slide 29

Flow Graph Notations*: p

Computing 2-Point DFT 2 Point

F2 ( k ) = f 2 (n)W2kn
n=0 1

a+b a Wk aWk

, k =0

F2 (k ) = f 2 (0) + W2k f 2 (1) , k = 0 F2 (0) = f 2 (0) + f 2 (1) F2 (1) = f 2 (0) f 2 (1) Similarly, G2 (0) = g 2 (0) + g 2 (1) G2 (1) = g 2 (0) g 2 (1)
g 2 ( 0)
W20

f 2 ( 0)
W20

F2 (0)

a-b

f 2 (1)

F2 (1)

G2 (0)

* This graph notations are used in [2]. g p [ ]

g 2 (1)

G2 (1)

JIIT Noida, Slide 31

JIIT Noida, Slide 32

Combining two 2-Point DFTs

F1 (k + 2 ) = F2 ( k ) W G 2 ( k )
k 4

Combining two 4 Point DFTs 4-Point

X (k + 4) = F (k) W G (k) 1
k 8 1

F1 (k ) = F2 ( k ) + W 4k G 2 ( k )

F2 (0)

k = 0,1
F1 (0)

X (k) = F (k) +W8kG1(k) 1

G1(0)
1 8 1

, k = 0,1,2,3

X (0) = F1(0) +

F1 ( 0 ) = F2 ( 0 ) + F1 (1) = F2 (1) F1 (3) = F2 (1) F1 ( 2 ) = F2 ( 0 )

G 2 (0)
F2 (1) F1 (1)

X (1) = F1(1) + W G (1) X (2) = F1(2) + W G (2)

2 8 1

F1 ( 0 ) F1 (1)
F1 ( 2 ) F1 ( 3 )
W 80

X (0)
X (1)
X (2) X (3)

+ W 41G 2 (1) G 2 (0) W 41G 2 (1)

X (3) = F1(3) + W83G1(3)

W40
G2 (0)

X (4) = F1(0)
W41
F1 (2)

G1(0)
G1 (0 )

X (5) = F1(1) W81G1(1) X (6) = F1(2) W82G1(2) X (7) = F1(3) W G (3)

3 8 1

W 81 W 82 W 83

X (4)

G 1 (1)
G1 ( 2 ) G1 (3)

X (5 )
X (6) X (7 )

Similarly, Similarly G1(k) may be computed computed.

G2 (1)

F1 (3)

JIIT Noida, Slide 33

JIIT Noida, Slide 34

Flow Diagram for 8-point Radix-2 Decimation in ti D i ti i time FFT algorithm l ith
x(0)
x(4)
x(2)
W20
W20
0 4

SIMILARLY Radix-2 DIF FFT Algorithm can be implemented

X(0)
X(1 )
x ( 0) x (1) x ( 2) x (3) x ( 4) x ( 5) x ( 6) x (7 )

W41

X(2) X(3)
W
0 8

x(6)

Radix-2
Decimation in Frequency

FFT Algorithm

x(1 )
W0 2

W81
W40

X(4) X(5) X(6) X(7)

x(5) x(3)
W20

2 8

X(0) X(4) X(2) X(6) X(1) X(5) X(3) X(7)

Bit Reversing Algorithm

X (0) X (1) X (2) X (3) X (4) X (5) X (6) X (7 )

W41

W83

x(7)
JIIT Noida, Slide 35

JIIT Noida, Slide 36

Computational Complexity of Radix-2 DIT FFT Algorithm It may be noted that v stages are required to compute N=2v point DFT using Radix-2 DIT FFT algorithm. For example to compute 8 point DFT 3 stages are required 8-point required. Let us find the computation required to combine two M/2 point DFTs DFT to get M i DFT M-point DFT,
k X (k ) = Fl (k ) + WM Gl (k )

Computations required to combine two M/2 point DFTs to get M-point DFT M point DFT,
k X (k ) = Fl (k ) + WM Gl (k )

M k X k + = Fl (k ) WM Gl (k ) 2

k = 0,1,...........,

M 1 2

One combining operation requires: M tions Complex Multiplica 2 Complex Additions -M

M X k + 2

k = Fl (k ) WM Gl (k )

k = 0,1,...........,

M 1 2

JIIT Noida, Slide 37

JIIT Noida, Slide 38

Say N=8 (CM : Complex Multiplication, CA : Complex Addition) y p p p

From the discussion it may be concluded that:

N Complex Multiplications : Every stage required 2 Complex Additions : N

x (0) x(4)

2-Point 2 P i t DFT CM = 1 CA = 2

F2

x(2) x (6)

2-Point 2 P i DFT G 2 CM = 1 CA = 2 2-Point DFT F 3 CM = 1 CA = 2 2-Point DFT G3 CM = 1 CA = 2 STAGE-1 CM = 4 CA = 8

4-Point DFT F 1 CM = 2 CA = 4

Combine to find 8 point DFT CM = 4 CA = 8

x (1) x (5 )

x (3) x (7 )

4-Point DFT G 1 CM = 2 CA = 4

Total Number of stages = v = log 2 N N N Complex Multiplications : v = log 2 N Total Computations p 2 2 Complex Additions : Nv = N log 2 N

STAGE-2 CM = 4 CA = 8
JIIT Noida, Slide 39

STAGE-3 CM = 4 CA = 8

JIIT Noida, Slide 40

Therefore, Computations required to N-point DFT using radix-2 DIT FFT algorithms are:
N Complex Multiplications : log 2 N 2 Complex Additions : N log 2 N

Computational Complexity = O(N log2N) C t ti lC l it l

JIIT Noida, Slide 41

JIIT Noida, Slide 42

Eight-point radix-2 decimation-in-frequency FFT

Eight-point radix-2 decimation-in-frequency FFT

JIIT Noida, Slide 43

JIIT Noida, Slide 44

JIIT Noida, Slide 45

JIIT Noida, Slide 46

9-point radix-3 decimation-in-time FFT(for X(0), X(2),X(3) etc X(2) X(3) etc. do yourself)

JIIT Noida, Slide 47

JIIT Noida, Slide 48

9-point radix-3 decimation-in-frequency FFT (for x(0), x(2),x(3) etc. do yourself)

Full decimation-in-time FFT implementation of a 8-point DFT i l t ti f 8 i t

JIIT Noida, Slide 49

JIIT Noida, Slide 50

FFT example
Lets take an example of continuous input signal containing components at 1khz and 2 g g p khz.

Input Signal Waveform

x(n) = sin(2.1000.t ) + 0.5(2.2000.t + 3 / 4) ( ( )

Now we will sample it with frequency 8000 Hz and take p q y 8 point DFT i.e. N=8.

x(n) = xin(nts) =sin(2.1000.nts) +0 5(2.2000.nts +3/ 4) t i (2 1000 t 0.5(2 2000 t

JIIT Noida, Slide 51

JIIT Noida, Slide 52

Values of x(n)
VALUE OF ADDITION SAMPLES AFTER
Now we will give these input values to our previous figure.

x(0) =3.535534e-001 (0) 3 535534 001 x(1) =3.535534e-001 x(2) =6.464466e-001 (3) .060660e 000 x(3) =1.060660e+000 x(4) =3.535534e-001 x(5) =-1.060660e+000 (5) 1 060660e+000 x(6) =-1.353553e+000 x(7) =-3.535534e-001
JIIT Noida, Slide 53 JIIT Noida, Slide 54

Stage 1

Stage 2

Stage 3

0.3535 X(0) (0) 0.3535 X(4) 0.6464 X(2) -1.3535 1 3535 X(6) 0.3535 X(1) -1.0607 X(5) 1.0607 X(3) -0.3535 X(7)

.707
1 -1

A(0)

X(0) W08 X(1) W18 X(2) W28 X(3) W38 W48 W58 W68 W78 X(4) X(5) X(6) X(7)

The output of the second stage FFT are:

0.3535 X(0) 0.3535 0 3535 X(4) 0.6464 0 6464 X(2) -1.3535 X(6) ( ) .707
1 -1

W04 .0

A(1)

A(0)

W14 W24 W34

A(2) A(3)

W04 .0 W14 W24 W34

A(1)

-.707
1 -1

-.707 707
1 -1

A(2) A(3)

1.999 -.707 .707

1 -1

B(0) ( )

W04 W14B(1) W24B(2) W34B(3)

JIIT Noida, Slide 55

1.999

1.414 .707
1 1 -1

1.414

JIIT Noida, Slide 56

Calculating the outputs of the third stage of the FFT to arrive at our final answer:
0.3535 X(1) -1.0607 X(5) 1.0607 1 0607 X(3) -0.3535 X(7) -.707
1 -1

B(0)

A(0)

X(0) ( ) W08 X(1) W18 X(2) W28 X(3) W38 W48 X(4) X(5) X(6) X(7)

W04
( ) W14 B(1)

A(1)

1.414 .707
1 -1

W24 W34

B(2) B(3)

A(2) A(3)

1.414

B(0)

B(1) B(2) B(3) JIIT Noida, Slide 57 JIIT Noida, Slide 58

W58 W68 W78

Result of FFT = DFT

So, the FFT gives us the correct results, and again we pp remind that the FFT is not an approximation to a DFT; it is the DFT with a reduced number of necessary arithmetic operations. We've W ' seen f from the above example that the 8 i h b l h h 8-point FFT example required less effort than the 8-point DFT Example Example.

JIIT Noida, Slide 59

JIIT Noida, Slide 60