EEM.

ASP Advanced Signal Processing Adaptive Filters

Prof John Illingworth Room: 09AB05 J.Illingworth@surrey.ac.uk

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

1 / 26

Lecture Schedule

week 1 2 3 4 5 6 7 8 9 10

10 JI JK JK JK JK JK JK JK JI JI

11 JI JK JK JK JK JK JK JI JI JI

12 JK1 JK JK JK JK JK JK JI JI JI

lecture to be given next Tuesday

John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 2 / 26

Lecture Notes and Textbooks

Lecture Notes Last years notes by Dr Chilton will be copied and made available as will my lecture slides as they are developed. Main textbook Haykin S, Adaptive Filter Theory, Prentice-Hall; 4th International Edition. ISBN: 0-13-048434-2, price: 56. Background Reading Monson H. Hayes, Schaums Outlines Digital Signal Processing, McGraw-Hill, ISBN: 0-07-027389-8, price: 10 Ambardar A, Digital Signal Processing: a modern introduction, Thomson, ISBN: 0-495-08238-4, price: 49 Baher H, Analog and Digital Processing, John Wiley, price: 45

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

3 / 26

Pre-requisite knowledge

Students are expected to have a knowledge of: Fourier Transform, Laplace transform and linear systems theory Discrete Fourier Transform, the z-transform and discrete systems Principles of design of discrete, time-invariant linear lters These were topics covered in the Autumn semester course EEM.MSP: Maths of Signal Processing

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

4 / 26

Impulse Response and Transfer Function

In the time domain y (t) = h(t) x(t) In the frequency domain Y (j) = H(j).X (j) h(t) is the impulse response and H(j) is the transfer function of the system.
John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 5 / 26

where

y (t) = h(t)

when

x(t) = (t)

Z-Transform

X (e ) = X (z) =
n=

x(n)e j

b0 + b1 z + b2 z + . . . + bM z X (z) = = 1 + a1 z 1 + a2 z 2 + . . . + aN z N

b0
k=1 N

1 zk z 1 (1 pk z 1 )

k=1

Any discrete system can be represented by a set of poles (pk ) and zeros (zk ) in the z-plane.
John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 6 / 26

Dierence Equations

Apply a single pulse to system at n = 0. Output is a sequence of pulses that depend on previous input and output values

y (n) = a0 x(n)+a1 x(n 1)+a2 x(n 2)+. . .+b1 y (n 1)+b2 y (n 2)+. . . where ai and bi are constants that describe the system.

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

7 / 26

Inter-relations

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

8 / 26

Statistics of Random Variables

Correlation: RXY = E [XY ] = Covariance: x y p(x, y )dxdy

CXY

= E (X X )(Y Y ) = E (XY X .Y Y .X + X .Y = E [(XY ] X .Y Y .X + X .Y = E [XY ] E (X )E (Y ) = RXY X .Y

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

9 / 26

Linear Correlation
Correlation is the product and sum of (x, y ) function and the joint pdf

Note that value of correlation will depend on placement of pdf in space as well as shape of pdf and that high values may occur purely as a result of placement.

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

10 / 26

Correlation - example

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

11 / 26

Covariance
Centering of pdf before performing correlation produces a result that is less sensitive to location and more representative of the shape of the distribution and whether values are linearly co-related i.e. does high value of one variable imply high value of the other etc

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

12 / 26

Random Process
Random Processes are similar to RVs except that now events map to functions of 1 or more dimensions. In the simplest case 1-D functions of time e.g. a typical random process is specied by X (t, s) where s is an outcome and t is time.

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

13 / 26

Autocorrelation function
To partially describe the second order pdf we can use the autocorrelation function, RXX (t1 , t2 )

RXX (t1 , t2 ) = E [X (t1 )X (t2 )] =

x1 x2 pX1 ,X2 (x1 , x2 ; t1 , t2 )dx1 dx2

Measures the correlation between two RVs which come from the same random process. Generally depends in when t1 and t2 are chosen. Often interested in correlation as a function of distance between the two times i.e. t2 t1 RXX (t, t + ) = E [X (t)X (t + )]

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

14 / 26

Example
For the random amplitude sine wave process then the autocorrelation is found as RXX (t1 , t2 ) = E [X (t1 )X (t2 )] = E A2 sin(0 t1 ) sin(0 t2 ) = 1 sin(0 t1 ) sin(0 t2 ) 3

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

15 / 26

Application
Active Suspension Control

Want to keep height constant despite changing and or unknown road surface, mass in car etc. Need an adaptive system.
2

Figure taken from http://cnx.org/content/m11825/latest/

John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 16 / 26

Magnetic Resonance Imaging (MRI) Application

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

17 / 26

Application

In many electrical applications a signal may be corrupted by mains hum i.e. a deterministic noise at the frequency of the electrical supply. Consider say an ECG trace in a medical context. In the UK mains frequency is nominally 50 HZ. This frequency might be removed by a notch lter centred at 50 HZ. However, the mains frequency might wander by a few Hz i.e. between 47 to 53 Hz. If a non-adaptive notch lter is used then it will have to have a width to cover this range. But, the heart signal may have a signicant component in this range and hence a wide lter is undesirable. A narrow adaptive lter would be a better alternative.

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

18 / 26

System Identication
Determine the parameters an unknown system such as a communications channel or the acoustics of an auditorium. Put lter and system in parallel3 .

If error is small then lter response is same as system.

Figure taken from http://www.mathworks.com/ article on Adaptive Filter Toolbox

John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 19 / 26

Inverse System Identication

Develop a lter that has the inverse response to a given system. Put system and lter in series4 .

When output error is small then lter has inverse characteristics of system. In old telephone systems the copper wires acted as a lter with a response that tailed o at higher frequencies. This response can be compensated by the lter yielding better utilisation of the frequency spectrum.
4

Figure taken from http://www.mathworks.com/ article on Adaptive Filter Toolbox

John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 20 / 26

Noise Cancellation

Input to the lter noise that is correlated with the type of noise known to corrupt the signal5 . The lter will adapt to keep the error as small as possible. The error signal output is the cleaned-up i.e. noise-free signal.

Figure taken from http://www.mathworks.com/ article on Adaptive Filter Toolbox

John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 21 / 26

Prediction
In this application it is necessary to assume that signal varies only slowly over time6 .

Can be used to predict and remove a periodic signal from a random noise signal.

Figure taken from http://www.mathworks.com/ article on Adaptive Filter Toolbox

John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 22 / 26

FIR (non-recursive) and IIR (recursive) Filters

Finite Impulse Response - FIR lter - output depends only on current input and past input values Innite Impulse Response - IIR lter - output depends on current input, past input and past output Example of a recursive lter: yn = xn + yn1 y0 y1 y2 y3 ... = = = = = x0 + y1 x1 + y0 x2 + y1 x3 + y2 ... = = = = x1 + x0 + y1 x2 + x1 + x0 + y1 x3 + x2 + x1 + x0 + y1 ...

Computes the sum of previous values - an integrator lter. Concise description of what is done i.e. yn is computed via sum of two terms xn and yn1 . Compare with an FIR implementation which would have n terms.
John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 23 / 26

Signals and Filters

Several lter structures. Perhaps the simplest is the transversal lter:

Only uses delay elements, multiplier and adds.

M

For M tap lter y (n) =

k=0

wk x(n k)

Adaptive lter involves choosing best values of wk so as to minimise an appropriate cost function.

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

24 / 26

Adaption as intelligent search

Each point in the space has a value related to the error between the observed and the desired output. How can we nd the minimum of the error in the space.

Two principle methods for adaptive lter: Least Mean Square (LMS) method Recursive Least Square (RLS) method
John Illingworth (UniS) EEM.ASP Lecture 1 Spring 2007 25 / 26

Web Resources

Adaptive Filter Course at Connexions web site http://cnx.org. see http://cnx.org/content/col10280/latest/. Adaptive Filter Java teaching Tool.
www.eee.strath.ac.uk/r.w.stewart/adaptivejava/begin.htm

www.eee.strath.ac.uk/r.w.stewart/adaptivejava/java adaptive dsp/jdk1.1/

Digital Filter Tutorial http://www.logix4u.net/diglt.htm

John Illingworth (UniS)

EEM.ASP Lecture 1

Spring 2007

26 / 26