Main Pronect

ABSTRACT
The problem of acoustic echo cancellation is the result of hands-free telephony and
teleconferencing systems. In early telephony the microphone and loudspeaker were separated
and no sound could propagate between the speaker and the microphone. Therefore no echo
would be transmitted back. Using a hands free loudspeaker telephone, however, the sound from
the loudspeaker will be picked up by the microphone and transmitted back to the sender who will
recognize this as an echo. This severely reduces conversation quality, even at very small echo
delays.
This project shows some implementations of acoustic echo cancellation algorithms in
MATLAB and the results of analysis on the broader systems involved. It focuses on Normalized
Least Mean Square(NLMS) and Variable Impulse Response Double Talk Detector (VIRE DTD).
In the case of acoustic echo in telecommunications, the optimal output is an echoed signal that
accurately emulates the unwanted echo signal. This is then used to negate the echo in the return
signal. The better the adaptive filter emulates this echo, the more successful the cancellation will
be. This project examines various techniques and algorithms of adaptive filtering, employing
discrete signal processing in MATLAB.
CHAPTER - 1
BACKGROUND
1.1 Introduction.
The project entitled Acoustic Echo Cancellation using Digital Signal Processing. Acoustic
echo occurs when an audio signal is reverberated in a real environment, resulting in the original
intended signal plus attenuated, time delayed images of this signal.
This project will focus on the occurrence of acoustic echo in telecommunication systems.
Such a system consists of coupled acoustic input and output devices, both of which are active
concurrently. An example of this is a hands-free telephony system. In this scenario the system
has both an active loudspeaker and microphone input operating simultaneously. The system then
acts as both a receiver and transmitter in full duplex mode. When a signal is received by the
system, it is output through the loudspeaker into an acoustic environment. This signal is
reverberated within the environment and returned to the system via the microphone input. These
reverberated signals contain time delayed images of the original signal, which are then returned
to the original sender (Figure 1.1, ak is the attenuation, tk is time delay). The occurrence of
acoustic echo in speech transmission causes signal interference and reduced quality of
communication.
Figure 1.1: Origins of acoustic echo
The method used to cancel the echo signal is known as adaptive filtering. Adaptive filters are
dynamic filters which iteratively alter their characteristics in order to achieve an optimal
desired output. An adaptive filter algorithmically alters its parameters in order to minimise a
function of the difference between the desired output d(n) and its actual output y(n). This
function is known as the cost function of the adaptive algorithm. Figure 1.2 shows a block
diagram of the adaptive echo cancellation system implemented throughout this thesis. Here
the filter H(n) represents the impulse response of the acoustic environment, W(n) represents
the adaptive filter used to cancel the echo signal. The adaptive filter aims to equate its output
y(n) to the desired output d(n) (the signal reverberated within the acoustic environment). At
each iteration the error signal, e(n)=d(n)-y(n), is fed back into the filter, where the filter
characteristics are altered accordingly.
Figure 1.2: Block diagram of an adaptive echo cancellation system.
This project deals with acoustic echo as applies to telecommunication systems, although
the techniques will be applicable to a variety of other disciplines.
Before giving any explanation of adaptive filters we will begin our study with a short
overview of different types of signals and some basic concepts which will help to understand
the adaptive filter.
1.2 Discrete Time Signals

Real world signals, such as speech are analog and continuous. An audio signal, as heard
by our ears is a continuous waveform which derives from air pressure variations fluctuating
at frequencies which we interpret as sound. However, in modern day communication systems
these signals are represented electronically by discrete numeric sequences. In these
sequences, each value represents an instantaneous value of the continuous signal. These
values are taken at regular time periods, known as the sampling period, Ts.
For example, consider a continuous waveform given by x(t). In order to process this
waveform digitally we first must convert this into a discrete time vector. Each value in the
vector represents the instantaneous value of this waveform at integer multiples of the
sampling period. The values of the sequence, x(t) corresponding to the value at n times the
sampling period is denoted as x(n).
x (n) = x (nTs) .. eq 1.1
So the discrete time signals may arise by sampling a continuous-time signal, or they may
be generated by some discrete-time process. Whatever the origin of discrete time signals,
discrete time signal processing systems have many attractive features. Such as charge import
devices, general purpose digital computers and can be used to simulate analog systems
1.3 Random Signals

In many situations, the processes that generate signals are so complex as to make precise
description of a signal is extremely undesirable and difficult. Like many signals in real world
applications, the values of the input vector of the acoustic echo cancellation system are
unknown before they arrive. Also as it is difficult to predict these values, they appear to
behave randomly. So there is a brief examination of random signal .A random signal,
expressed by random variable.
function, x(t), does not have a precise description of its waveform. It may, however, be
possible to express these random processes by statistical or probabilistic models a single
occurrence of a random variable appears to behave unpredictably. But if we take several
occurrences of the variable, each denoted by n, then the random signal is expressed by two
variables, x(t, n).
The main characteristic of a random signal is known as the expectation of a random
signal. It is defined as the mean value across all n occurrences of that random variable,
denoted by E[x(t)], where x(t) is the input random variable. It should be noted that the
number of input occurrences into the acoustic echo cancellation system is always 1.
Throughout this thesis the expectation of an input signal is equal to the actual value of that
signal. However, the E[x(n)] notation shall still be used in order to derive the various
algorithms used in adaptive filtering.
1.4 Stationary Signals

A signal can be considered stationary in the wide sense, if the two following criteria are
met.
1. The mean values, or expectations, of the signal are constant for any shift in time.
m(n) = mx (n + k) ................ eq 1.2
2. The autocorrelation function is also constant over an arbitrary time shift.

xx(n, m ) = xx(n+k ,m+k ) ...eq 1.3
The above implies that the statistical properties of a stationary signal are constant overtime.
In the derivation of adaptive filtering algorithms it is often assumed that the signals input to
the algorithm is stationary. Speech signals are not stationary in the wide sense; however they
do exhibit some temporary stationary behavior.
1.5 Speech Signals

A speech signal consists of three classes of sounds. They are voice, fricative and plosive
sounds. Voiced sounds are caused by excitation of the vocal tract with quasi-periodic pulses
of airflow. Fricative sounds are formed by constricting the vocal tract and passing air through
it, causing turbulence that result in a noise-like sound. Plosive sounds are created by closing
up the vocal tract, building up air behind it then suddenly releasing it, this is heard in the
sound made by the letter Figure shows a discrete time representation of a speech signal. By
looking at it as a whole we can tell that it is non-stationary. That is, its mean values vary with
time and cannot be predicted using the above mathematical models for random processes.
However, a speech signal can be considered as a linear composite of the above three classes
of sound, each of these sounds are stationary and remain fairly constant over intervals of the
order of 30 to 40 ms. The theory behind the derivations of many adaptive filtering algorithms
usually requires the input signal to be stationary. Although speech is non-stationary for all
time, it is an assumption that the short term stationary behavior outlined above will prove
adequate for the adaptive filters to function as desired
Figure 1.3 Representation of Speech Signal
1.5.1 Speech generation

Speech generation and recognition are used to communicate between humans and
machines. Rather than using your hands and eyes, you use your mouth and ears. This is very
convenient when your hands and eyes should be doing something else, such as: driving a car,
performing surgery, or (unfortunately) firing your weapons at the enemy. Two approaches
are used for computer generated speech: digital recording and vocal tract simulation. In
digital recording, the voice of a human speaker is digitized and stored, usually in a
compressed form.
During playback, the stored data are uncompressed and converted back into an analog
signal. An entire hour of recorded speech requires only about three megabytes of storage,
well within the capabilities of even small computer systems. This is the most common
method of digital speech generation used today. Vocal tract simulators are more complicated,
trying to mimic the physical mechanisms by which humans create speech. The human vocal
tract is an acoustic cavity with resonate frequencies determined by the size and shape of the
chambers. Sound originates in the vocal tract in one of two basic ways, called voiced and
fricative sounds. With voiced sounds, vocal cord vibration produces near periodic pulses of
air into the vocal cavities. In comparison, fricative sounds originate from the noisy air
turbulence at narrow constrictions, such as the teeth and lips. Vocal tract simulators operate
by generating digital signals that resemble these two types of excitation. The characteristics
of the resonate chamber are simulated by passing the excitation signal through a digital filter
with similar resonances. This approach was used in one of the very early DSP success
stories, the Speak & Spell, a widely sold electronic learning aid for children.
1.5.2 Speech recognition
The automated recognition of human speech is immensely more difficult than speech
generation. Speech recognition is a classic example of things that the human brain does well,
but digital computers do poorly. Digital computers can store and recall vast amounts of data,
perform mathematical calculations at blazing speeds, and do repetitive tasks without
becoming bored or inefficient. Unfortunately, present day computers perform very poorly
when faced with raw sensory data. Teaching a computer to send you a monthly electric bill is
easy. Teaching the same computer to understand your voice is a major undertaking. Digital
Signal Processing generally approaches the problem of voice recognition in two steps:
Feature extraction followed by feature matching. Each word in the incoming audio signal is
isolated and then analyzed to identify the type of excitation and resonate frequencies. These
parameters are then compared with previous examples of spoken words to identify the closest
match. Often, these systems are limited to only a few hundred words; can only accept speech
with distinct pauses between words; and must be retrained for each individual speaker. While
this is adequate for many commercial
1.6 Noise Sources

Sources of noise exist throughout the environment. One type of noise is due to turbulence
and is therefore totally random and impossible to predict. Engineers like to look at signals,
noise included, in the frequency domain. That is, "How is the noise energy distributed as a
function of frequency?"
These turbulent noises tend to distribute their energy evenly across the frequency bands
and are therefore referred to as "Broadband Noise". Very commonly we come across a word
white noise white noise comes under the category of Broadband Noise. White Noise is a
noise having a frequency spectrum that is continuous and uniform over a specified frequency
band.
Note: White noise has equal power per hertz over the specified frequency band. (Synonym
additive white Gaussian noise) Examples of broadband noise are the low frequency noise
from jet planes and the impulse noise of an explosion.
A large number of environmental noises are different. These "Narrow Band Noises"
concentrate most of their noise energy at specific frequencies. When the source of the noise
is a rotating or repetitive machine, the noise frequencies are all multiples of a basic "Noise
Cycle" and the noise is approximately periodic. This "Tonal Noise" is common in the
environment as man made machinery tends to generate it (along with a smaller amount of
broadband noise) at increasingly high levels.
Examples of sources of narrow band noise include:
Internal Combustion Engines: in transportation and as auxiliary power sources.
Compressors: as auxiliary power sources and in refrigeration units.
Vacuum Pumps: used to transfer bulk materials in many industries (also carpet Cleaning).
Rotating Machines: Imbalances cause vibration and secondary acoustic noise
Power Transformers: Strong magnetic fields cause vibration at harmonics of the
Power line frequency and secondary acoustic noise.
1.7 Correlation Function

The correlation function is a measure of how statistically similar two functions are. The
autocorrelation function of a random signal is defined as the expectation of signals value at
time n multiplied by its complex conjugate value at a different time m.
xx(n,m) = E[x(n)x*(m)] ....eq 1.4

As this thesis deals only with real signals the above equation becomes
xx (n, m) =E[x(n)x(m)] eq1.5

The derivations of adaptive filtering algorithms utilize the autocorrelation matrix, R. For
real signals this is defined as the matrix of expectations of the product of a vector x(n) and its
transpose.
R=E[x(k) xT(k ) ] .eq 1.6
The autocorrelation matrix has the additional property that its trace, i.e. the sum of its
diagonal elements, is equal to the sum of the powers of the values in the input vector As we
will see, sometimes a single value replaces one of the vectors in the autocorrelation matrix, in
this case the correlation function results in a vector. This vector is given by the expectation of
that single value multiplied by the expectation of each of the values in the vector.
Correlation matrices and vectors are based on either cross-correlation or autocorrelation
functions. This simply refers to the signals being used in the function. If it is cross
correlation, the signals are different, if it is autocorrelation; the two signals used in the
function are the same.
a. Characterisation of Linear System
A discrete linear system is a digital implementation of a linear time-invariant system. Its
input is a vector representing the sampled input signal (we will use x[]), its output is a vector
of the same size as the input representing the sampled output signal (y[]). The system itself
has to obey the usual conventions of linearity, particularly that it obeys the principle of
superposition, so that f(a+b) = f(a) + f(b), where f() is the processing performed by the
system and a and b are arbitrary signals. Linear time-invariant systems can be characterised
in the frequency domain by their frequency response.
The frequency response of a system tells us how a sinusoidal input is changed in
magnitude and phase as a function of its frequency. Although it is possible to implement
systems in the frequency domain, it is more typical to implement them in the time domain:
sample by sample. To do this we use the time domain equivalent of the frequency response,
known as the impulse response.

A digital impulse response is just the sequence of values that emerge from a discrete
system when a unit impulse is applied to the input (namely: x=[1 0 0 0 0 0 ]).
If we know the impulse response of a discrete linear system (and it is of finite length) we
can readily implement the system in a simple way. To understand this observe that any
sampled signal is really just a series of scaled impulses whose amplitudes are the
instantaneous amplitudes of the original analogue signal and which occur regularly at the
sampling instants.
Thus if the input signal is just a series of impulses and we know what the system does to
impulses and we know the system obeys the principle of superposition then we can calculate
the output of the system to any input signal. We do this by triggering an impulse response at
each input sample scaled according to the amplitude of the sample. The sum of the
overlapping triggered scaled impulse responses is the output signal. Mathematically this is
called convolution, and can be expressed as:
b. Finite Impulse Response (FIR) Filters

To obtain the impulse response of a system specified in the frequency domain we can use
an inverse Fourier transform to get a sampled impulse response then truncate or window the
impulse response to a suitable length.
MATLAB provides a function
fir1()
for generating a truncated impulse response for the
implementation of low-pass, high-pass and band-pass filters. The call: h = fir1(n,w) generates
a truncated impulse response in h[] of length n samples, corresponding to a low-pass filter
with a cut-off at a normalised frequency w. In MATLAB filter design functions, frequencies
are normalised to half the sampling rate, i.e. a normalised frequency of 0.5 corresponds to
one quarter of the sampling rate (don't ask). Here we generate 1 second of white noise signal
at 10,000 samples/sec and low-pass filter it at 2500Hz with a filter of size 100 samples.
x = rand(1,10000);
soundsc(x,10000);
pause;
h = fir1(100,0.5);
y = conv(h,x);
soundsc(y,10000);
High-pass filters are designed with
h = fir1(n,w,'high');
and band-pass filters are designed with
h = fir1(n,[w1 w2]);
where w1 and w2 are the normalised frequencies of the band edges.
In general you need truncated impulse responses that are long enough to capture the
majority of the energy in the full impulse response. Filters with 100 coefficients are not
uncommon. Filters with sharp edges need longer impulse responses.
c. Infinite length impulse response filters
Finite impulse response filters have a number of good characteristics: they are simple to
understand, have good phase response, and are always stable. However they are inefficient.
Each output sample requires a convolution sum that is the size of the impulse response. That
is for an impulse response of length N, the system implementation will require N multiplyadd operations per output sample.
Increased efficiency can be obtained by allowing the linear system calculation to use not
only the current and past samples of the input signal, but also past samples from the output
signal. Often the calculation performed by a linear system can be simplified if it can re-use
some of the calculation that it performed on previous samples. This is called recursive system
design, in contrast to finite impulse response filters which can be called non-recursive.
Recursive systems are more complex to build, often have messy phase responses, and can be
unstable. However they can be significantly more efficient and have impulses responses of
infinite length.
A recursive system is specified by two vectors a[] and b[]: the b[] coefficients are
convolved with the current and past input samples, while the a[] coefficients are convolved
with the past output samples.
Here is a diagram of the idea:
In this figure the input signal x is processed sample-by-sample into the output signal y. At
the stage shown, we are processing sample number n. To calculate output sample y(n), the
filter multiplies the current and past input samples x(n), x(n-1), x(n-2), x(n-3), , x(n-k) by
the set of b coefficients: b(0), b(1), b(2), b(3), , b(k); and sums them. The filter then
multiplies the past output samples: y(n-1), y(n-2), y(n-3), , y(n-k) by the a coefficients:
a(1), a(2), a(3), , a(k) and sums them. It then combines these to form y(n), according to this
formula:
In MATLAB, this whole process is performed by the filter() function: y = filter(b,a,x)
where as before, x is the input signal, y the output signal, and where b and a are the
coefficients that define the recursive linear system.
The MATLAB signal processing toolbox contains a number of different functions for
deigning recursive low-pass, high-pass and band-pass filters. We shall only refer to one
simple design, the Butterworth filter. This design generates filters with maximally flat
responses in the pass band, although they may not have as steep a cut-off as other designs.
A butterworth low-pass filter can be constructed with

[b,a] = butter(n,w);
Where n is the number of coefficients required in the filter, and w is the normalised cutoff frequency. Typically n is much smaller for recursive filters than for FIR filters, values of
10 to 20 are common.
A butter-worth high-pass filter can be constructed from
[b,a] = butter(n,w,'high');
while a band-pass filter from w1 to w2 can be constructed with
[b,a] = butter(n,[w1 w2]);
In this simple example we filter one second of white noise with a low-pass filter at 2500Hz:
x = rand(1,10000);
soundsc(x,10000);
pause;
[b,a]=butter(10,0.5)
y = filter(b,a,x);
soundsc(y,10000);
d. Frequency Response
To display a graph of the frequency response of a filter, use the function
recursive system with coefficients [b,a], this can be called with freq(b,a,n,Fs);
freqz().
For a
The argument n is the number of points to calculate; 512 is a suitable size. The argument
Fs allows you to scale the graph to any given sample rate. For a non-recursive system, where
only the impulse response is available, use:
freqz(h,1,512,Fs);
By default, freqz() plots both the magnitude response and the phase response. You can plot
just the magnitude response (in decibels) with this code:
[h,f] = freqz(b,a,512,Fs);
plot(f,20*log10(abs(h)));
Here h[] stores the complex response and f[] stores the frequency indices.
CHAPTER - 2
FILTERS
2.1 What is a Filter
The term filter is commonly used to refer to any device or system that take a mixture of
particles/elements (frequency components) from its input and process them according to
some specific rules to generate a corresponding set of particles /elements at its output
OR
A filter can be defined as a piece of software or hardware that takes an input signal and
processes it so as to extract and output certain desired elements of that signal. Filters can be
linear or non linear.
But we are considering the linear filters. However, we do not usually think of something as a
filter unless it can modify the sound in some way. For example, speaker wire is not
considered a filter, but the speaker is (unfortunately). The different vowel sounds in speech
are produced primarily by changing the shape of the mouth cavity, which changes the
resonances and hence the filtering characteristics of the vocal tract. The tone control circuit
in an ordinary car radio is a filter, as are the bass, midrange, and treble boosts in a stereo
preamplifier. Graphic equalizers, reverberators, echo devices, phase shifters, and
speaker crossover networks are further examples of useful filters in audio. There are also
examples of undesirable filtering, such as the uneven reinforcement of certain frequencies in
a room with bad acoustics.'' A well-known signal processing wizard is said to have remarked,
``when you think about it, everything is a filter.''
A digital filter is just a filter that operates on digital signals, such as sound represented
inside a computer. It is a computation, which takes one sequence of numbers (the input
signal) and produces a new sequence of numbers (the filtered output signal). The filters
mentioned in the previous paragraph are not digital only because they operate on signals that
are not digital. It is important to realize that a digital filter can do anything that a real-world
filter can do. That is, all the filters alluded to above can be simulated to an arbitrary degree of
precision digitally. Thus, a digital filter is only a formula for going from one digital signal to
another. It may exist as an equation on paper, as a small loop in a computer subroutine, or as

a handful of integrated circuit chips properly interconnected. Linear Time Invariant Systems
that change the shape of the spectrum are often referred to as frequency shaping filters.
Systems that are designed to pass some frequencies essentially undistorted and significantly
attenuate or eliminate others are referred to as frequency selective filters.
2.2 FIR Filter

Finite impulse response (FIR) filter, operates on discrete-time signals and can be
implemented with a digital signal processor. The convolution is very useful for the design of
FIR filters, since we can approximate it with a finite number of terms, or
.. eqn: 2.1
Different techniques are available for the design of FIR filters, such as a commonly used
technique that utilizes the Fourier series. A very useful feature of an FIR filter is that it can
guarantee linear phase. The linear phase feature can be very useful in applications such as
speech analysis, where phase distortion can be very critical. For example, with linear phase,
all input sinusoidal components are delayed by the same amount. Otherwise, harmonic
distortion can occur.
Figure 2.1 FIR Filter Showing Delay
2.3 IIR Filter

The infinite impulse response (IIR) filter that makes use of the vast knowledge already
acquired with analog filters. The design procedure involves the conversion of an analog filter
to an equivalent discrete filter using the bilinear transformation (BLT) technique. As such,
the BLT procedure converts a transfer function of an analog filter in the s-domain into an
equivalent discrete-time transfer function in the z-domain.
This recursive type of equation represents an infinite impulse response (IIR) filter. The
output depends on the inputs as well as past outputs (with feedback).
2.4 Frequency Selective Filters

Frequency selective filters are a class of filters specifically intended to accurately or
approximately select some band of frequencies and reject others. The frequency selective
filters arise in a variety of situations. For example, if noise is an audio recording in a higher
frequency band than the music or voice on the recording is, it can be removed by frequency
selective filtering. Another important application of frequency selective filters is in
communication systems. Like the basis for amplitude modulation systems is the transmission
of information from many different sources simultaneously by putting the information from
each channel into a separate frequency band and extracting the individual channels or bands
at the receiver using the frequency selective filter. Frequency selective filters for separating
the individual channel and frequency shaping filters for adjusting the quality of the tone from
a major part of any home radio and television receiver. While frequency selectivity is the
only issue of concern in applications, its broad importance has lead to a wildly accepted set
of terms describing the characteristics of frequency selective filters. In particular, while the
nature of the frequencies to be passed by a frequency selective filter varies considerably from
application to application, several basic filters are wildly used and have been given names
indicative of their function
2.5 Type of Frequency Selective Filters

Three types of filters,
1. Low Pass Filter
2. High Pass Filter
3. Band Pass Filter
2.5.1 Low Pass Filter
A low pass filter is a filter that passes low frequencies i.e. frequencies around w=0 and
attenuates or rejects higher frequencies.
2.5.2 High Pass Filter
A high pass filter is a filter that passes high frequencies and attenuates or rejects low ones.
2.5.3 Band Pass Filter

A band pass filter is a filter that passes a band of frequencies and attenuates frequencies
both higher and lower than those in the band that is passed.
In each case the cut off frequencies are the frequencies defining the boundaries between
frequencies that are passed and frequencies that are rejected i.e. the frequencies in the pass
band and stop band.
Specifically, an ideal frequency selective filter is a filter that exactly passes complex
exponentials at one set of frequencies without any distortion and completely rejects
frequencies at all other frequencies. For example an ideal frequency low pass filter with
cutoff frequency wc is an LTI system that passes complex exponentials e^(jwt) for values of
w in the frequencies in the range wc<w<wc and rejects signals at all other frequencies. The
frequency response of an ideal continuous time high pass filter with cut off frequency wc,
and the figure illustrates an ideal continuous time band pass filter with low cutoff frequency
wc1 and upper cutoff frequency wc2.
Note that each of these filters is symmetric about w=0, and thus, there appear to be two
pass bands for the high pass and band pass filters. This is the consequence of having our
adopted the use of complex exponential signal. Note that the characteristics of the continuous
time and discrete time ideal filters differ by virtue of the fact that for discrete time filters the
frequency response must be periodic with period 2pi, with low frequencies near even
multiples of pi and high frequencies near odd multiples of pi.
fig 2.2 (a) Low pass filter
(b) High pass
(c) Band pass
CHAPTER - 3
ADAPTIVE FILTERS
As we begin our study of adaptive filters it may be important to understand the meaning
of the terms adaptive and filter in a very general sense. The adjective adaptive can be
understood by considering a system which is trying to adjust itself so as to respond to some
phenomenon that is taking place in its surrounding. In other words the system tries to adjust
its parameters with a aim of meeting some well define goal or target which depends upon the
state of system as well as its surrounding .this is what adaptation means. Moreover there is a
need to have a set of steps or certain procedure by which this process of adaptation is carried
out. And finally the system that carries out or undergoes the process of adaptation is called
by the more technical name filter.
Depending upon the time required to meet the final target of the adaptation process, which
we will call the convergence time that is available to carry out the adaptation, we can have a
variety of adaptation algorithms and filter structures.
3.1 Adaptive Filter

So after the above explanation we can define adaptive filter as
The purpose of the general adaptive system is to filter the input signal so that it
resembles (in some sense) the desired signal input OR An adaptive FIR or IIR filter
designs itself based on the characteristics of the input signal to the filter and a signal which
represent the desired behavior of the filter on its input.Adaptive filters are a class of filters
that iteratively alter their parameters.
The filter minimizes the error between some desired signal and some reference signal. A
result of the process is a set of N tap values which define the nature of the input signal being
filtered. Now, let's say we want to determine the nature of some channel, say a room in
which a signal is being created and received to a microphone. To do this, it seems that we
would have to have some way of supplying a known signal and comparing it to the same
signal as received into the mic. The LMS filter should, provide a set of taps define the
inverse of the room. After filtering out noise alone (which is another task altogether), the tap
weights could then be implemented as the corresponding filter, inversed and applied to the
received signal thus producing (at least theoretically) the original signal in its original form.
tapsre already inversed! When we feed an adaptive filter a training sequence, it allows it to
adapt so that the filtered signal is as close as
possible to the reference signal (i.e., the training signal). In order to do that, the filter should
define the inverse of whatever the channel is, because
Y(f) = X(f)*H(f)*F(f), . eq 3.1

where H(f) is the "channel," F(f) is the adaptive filter, X(f) is the input to the channel, and
Y(f) is the output from the adaptive filter. So in order to make
Y(f) = X(f),
you must have
1 = H(f)*F(f), eq 3.2
and therefore
F(f) = 1/H(f) eq 3.3
3.2 Adaptive Filter Structure

The most commonly used structure in the implementation of adaptive filter is the
transversal structure. The adaptive filter has a single input, x(n) and output, y(n). The
sequence d(n) is the desired signal. The output, y(n), is generated as a linear combination of
the delayed samples of input sequence, x(n), according to the equation
Where wi (n)s are the filter tap weights (coefficients) and N is the filter length. We refer to
the input samples, x(n-i), for i= 0,1,2..,N-1, as the filter tap inputs. The tap weights which
may vary in time are controlled by the adaptive algorithm.
Figure 3.1 Adaptive Filter Block Diagram
Figure 3.2 Basic Adaptive Filter structure
Designing the filter does not require any other frequency response information or
specification. To define the self learning process the filter uses, you select the adaptive
algorithm used to reduce the error between the output signal y(k) and the desired signal d(k).
When the LMS performance criteria for e(k) have achieved its minimum value through the
iterations of the adapting algorithm, the adaptive filter is finished and its coefficients have
converged to a solution. Now the output from the adaptive filter matches closely the desired
signal d(k). When you change the input data characteristics, sometimes called the filter
environment, the filter adapts to the new environment by generating a new set of coefficients
for the new data. Notice that when e(k) goes to zero and remains there you achieve perfect
adaptation; the ideal result but not likely in the real world.
So the system has six main components to be defined:
Input signal
Desired signal
Output signal
Error signal
FILTER - Filtering process
Adaptive process - Some kind of algorithm
The coefficients of an adaptive filter are adjusted to compensate for changes in input
signal, output signal, or system parameters. Instead of being rigid, an adaptive system can
learn the signal characteristics and track slow changes. An adaptive filter can be very useful
when there is uncertainty about the characteristics of a signal or when these characteristics
change.
3.3 Adaptive Filtering System Configurations

There are four major types of adaptive filtering configurations;
1- Adaptive system identification.
2- Adaptive echo cancellation.
3- Adaptive linear prediction.
4- Adaptive inverse system.
Digital signal processing (DSP) has been a major player in the current technical
advancements such as noise filtering, system identification, and voice prediction. Standard
DSP techniques, however, are not enough to solve these problems quickly and obtain
acceptable results. Adaptive filtering techniques must be implemented to promote accurate
solutions and a timely convergence to that solution A number of adaptive structures have
been used for different applications in adaptive filtering.
All of the above systems are similar in the implementation of the algorithm, but different
in system Configuration. All 4 systems have the same general parts; an input x (n), a desired
ACOUSTIC ECHO CANCELLATION
result d(n), an Output y (n), an adaptive transfer function w(n), and an error signal e(n) which
is the difference Between the desired output u(n) and the actual output y(n). In addition to
these parts, the system Identification and the inverse system configurations have an unknown
linear system u(n) that can Receive an input and give a linear output to the given input.
3.3.1 System Identification
The adaptive system identification is primarily responsible for determining a discrete
estimation of the transfer function for an unknown digital or analog system. The same input
x(n) is applied to both the adaptive filter and the unknown system from which the outputs are
compared (see figure ). The output of the adaptive filter y(n) is subtracted from the output of
the unknown system resulting in a desired signal d(n). The resulting difference is an error
signal e(n) used to manipulate the filter coefficients of the adaptive system trending towards
an error signal of zero. After a number of iterations of this process are performed, and if the
system is designed correctly, the adaptive filters transfer function will converge to, or near
to, the unknown systems transfer function. For this configuration, the error signal does not
have to go to zero, although convergence to zero is the ideal situation, to closely approximate
the given system. There will, however, be a difference between adaptive filter transfer
function and the unknown system transfer function if the error is nonzero and the magnitude
of that difference will be directly related to the magnitude of the error signal. Additionally
the order of the adaptive system will affect the smallest error that the system can obtain. If
there are insufficient coefficients in the adaptive system to model the unknown system, it is
said to be under specified. This condition may cause the error to converge to a nonzero
constant instead of zero. In contrast, if the adaptive filter is over specified, meaning that there
are more coefficients than needed to model the unknown system, the error will converge to
zero, but it will increase the time it takes for the filter to converge
Figure 3.3 Adaptive System Configurations
3.3.2 Adaptive Linear Prediction

Adaptive linear prediction is the third type of adaptive configuration figure 4.1.2. This
Configuration essentially performs two operations. The first operation, if the output is taken
from the error signal e(n), is linear prediction. The adaptive filter coefficients are being
trained to predict, from the statistics of the input signal x(n), what the next input signal will
be. The second operation, if the output is taken from y(n), is a noise filter similar to the
adaptive noise cancellation outlined in the previous section. As in the previous section,
neither the linear prediction output nor the noise cancellation output will converge to an error
of zero. This is true for the linear prediction output because if the error signal did converge to
zero, this would mean that the input signal x(n) is entirely deterministic, in which case we
would not need to transmit any information at all. In the case of the noise filtering output, as
outlined in the previous section, y(n) will converge to the noiseless version of the input
signal.
Fig 3.4 Adaptive Linear Prediction
3.3.3 Adaptive Inverse System Configuration

The adaptive inverse system configuration as shown in figure . The goal of the adaptive
filter here is to model the inverse of the unknown system u(n). This is particularly useful in
adaptive equalization where the goal of the filter is to eliminate any spectral changes that are
caused by a prior system or transmission line. The way this filter works is as follows. The
input x(n) is sent through the unknown filter u(n) and then through the adaptive filter
resulting in an output y(n). The input is also sent through a delay to attain d(n). As the error
signal is converging to zero, the adaptive filter coefficients w(n) are converging to the
inverse of the unknown system u(n). For this configuration, as for the system identification
configuration, the error can theoretically go to zero. This will only be true; however, if the
unknown system consists only of a finite number of poles or
the adaptive filter is an IIR filter. If neither of these conditions is true, the system will
converge only to a constant due to the limited number of zeroes available in an FIR system.
Fig 3.5 Adaptive Inverse system

3.3.4 Noise/Echo cancellation
The second configuration is the adaptive noise cancellation configuration as shown in
figure
In this configuration the input x(n) is compared with a desired signal d(n), which consists of
a signal s(n) corrupted by another noise N0(n). The adaptive filter coefficients adapt to cause
the error signal to be a noiseless version of the signal s(n). Both of the noise signals for this
configuration need to be uncorrelated to the signal s(n). In addition, the noise sources must
be correlated to each other in some way, preferably equal, to get the best results. Do to the
nature of the error signal; the error signal will never become zero. The error signal should
converge to the signal s(n), but not converge to the exact signal. In other words, the
difference between the signal s(n) and the error signal e(n) will always be greater than zero.
The only option is to minimize the difference between those two signals.
Fig 3.6: Adaptive Noise or Echo Cancellation
The basic principle is canceling a sound wave by generating another sound wave exactly
out of
phase with the first one. The superposition of the two waves would result in silence.
3.3.5 Additional Structures

(a) Notch with two weights,
which can be used to notch or cancel/reduce a sinusoidal noise signal. This structure
has Only two weights or coefficients.
(b) Adaptive channel equalization

Used in a modem to reduce channel distortion resulting from the high speed of data
transmission over telephone channels.
Fig 3.7: Adaptive Channel Equalization
3.4 Classes Of Adaptive Filter

3.4.1 Mean Square Adaptive Filter
The aim to minimize a cost function equal to the expectation of the square of the difference
between the desired signal d(n), and the actual output of the adaptive filter y(n)
E[e2(n)]=E[(d(n)-y(n)) 2] eq 3.4
Least Mean Square Algorithm

Normalized Least Mean Square Algorithm
Variable Step Size Least Mean Square Algorithm
Variable Step Size Normalized Least Mean Square Algorithm.
3.4.2 Recursive Least Square Adaptive Filter Algorithm
The aim is to minimize a cost function equal to the weighted sum of the squares of the
difference between the desired and the actual output of the adaptive filter for different time
instances. The cost function is recursive in the sense that unlike the MSE cost function,
weighted previous values of the estimation error are also considered. . the parameter is in
the range of 0< <1. It is known as the forgetting factor as for <1 it causes the previous
values to have an increasingly negligible effect on updating of the filter tap weights The
value of 1/(1- ) is a measure of the memory of the algorithm.
3.5 Performance Measures In Adaptive Systems

There are six major performance measures of Adaptive Systems.
a. Step size parameter
determines the convergence or divergence speed and precision of the adaptive filter
coefficients. If is large, the filter will converge fast, but could diverge if is too large.
When is large, the adaptation is quick, but there will be an increase in the average excess
MSE. This excess MSE may be undesirable result. If is small, the filter will converge
slowly, which is equivalent to the algorithm having long memory, an undesirable quality.
Every application will have a different step size that needs to be adjusted. When choosing a
value, their needs to be balance between speed convergence and the MSE. The value is
decided through trial and error so that speed at which the adaptive filter learns and the excess
MSE is obtained within application requirements. The values differs from simulation to
real-time because of the inherit differences between the systems.
b. Convergence Rate
The convergence rate determines the rate at which the filter converges to its resultant
state. Usually a faster convergence rate is a desired characteristic of an adaptive system.
Convergence rate is not, however, independent of all of the other performance
characteristics. There will be a tradeoff, in other performance criteria, for an improved
convergence rate and there will be a decreased convergence performance for an increase in
other performance. For example, if the convergence rate is increased, the stability
characteristics will decrease, making the system more likely to diverge instead of converge to
the proper solution. Likewise, a decrease in convergence rate can cause the system to become
more stable. This shows that the convergence rate can only be considered in relation to the
other performance metrics, not by itself with no regards to the rest of the system.
c. Minimum Mean Square Error
The minimum mean square error (MSE) is a metric indicating how well a system can
adapt to a given solution. A small minimum MSE is an indication that the adaptive system
has accurately modeled, predicted, adapted and/or converged to a solution for the system. A
very large MSE usually indicates that the adaptive filter cannot accurately model the given
system or the initial state of the adaptive filter is an inadequate starting point to cause the
adaptive filter to converge. There are a number of factors which will help to determine the
minimum MSE including, but not limited to; quantization noise, order of the adaptive
system, measurement noise, and error of the gradient due to the finite step size.
d. Computational Complexity
Computational complexity is particularly important in real time adaptive filter
applications.
When a real time system is being implemented, there are hardware limitations that may affect
the performance of the system. A highly complex algorithm will require much greater
hardware resources than a simplistic algorithm.
e. Stability
Stability is probably the most important performance measure for the adaptive system. By
the nature of the adaptive system, there are very few completely asymptotically stable
systems that can be realized. In most cases the systems that are implemented are marginally
stable, with the stability determined by the initial conditions, transfer function of the system
and the step size of the input.
f. Robustness
The robustness of a system is directly related to the stability of a system. Robustness is a
measure of how well the system can resist both input and quantization noise.Filter Length
The filter length of the adaptive system is inherently tied to many of the other performance
measures. The length of the filter specifies how accurately a given system can be modeled by
the adaptive filter. In addition, the filter length affects the convergence rate, by increasing or
decreasing computation time, it can affect the stability of the system, at certain step sizes, and
it affects the minimum MSE. If a filter length of the system is increased, the number of
computations will increase, decreasing the maximum convergence rate. Conversely, if the
filter length is decreased, the number of computations will decrease, increasing the maximum
convergence rate. For stability, due to an increase in length of the filter for a given system,
you may add additional poles or zeroes that may be smaller than those that already exist. In
this case the maximum step size, or maximum convergence rate, will have to be decrease to
maintain stability. Finally, if the system is under specified, meaning there are not enough
pole and/or zeroes to model the system, the mean square error will converge to a nonzero
constant. If the system is over specified, meaning it has too many poles and/or zeroes for the
system model, it will have the potential to converge to zero, but increased calculations will
affect the maximum convergence rate possible.
CHAPTER - 4

4.1 ECHO
Echo is a phenomenon in which a delayed and distorted version of an original sound or
electrical signal is reflected back to the source.
4.2 Telecommunications
Telecommunications is about transferring information from one location to another. This
includes many forms of information: telephone conversations, television signals, computer
files, and other types of data. To transfer the information, you need a channel between the
two locations. This may be a wire pair, radio signal, optical fiber, etc. Telecommunications
companies receive payment for transferring their customer's information, while they must
pay to establish and maintain the channel. The financial bottom line is simple: the more
information they can pass through a single channel, the more money they make. DSP has
revolutionized the telecommunications industry in many areas:
Signaling tone generation and detection, frequency band shifting, filtering to remove power
line hum, etc. Three specific examples from the telephone network will be discussed here:
multiplexing, compression, and echo control.
4.2.1 Echo control
Echoes are a serious problem in long distance telephone connections. When you speak
into a telephone, a signal representing your voice travels to the connecting receiver, where a
portion of it returns as an echo. If the connection is within a few hundred miles, the elapsed
time for receiving the echo is only a few milliseconds. The human ear is accustomed to
hearing echoes with these small time delays, and the connection sounds quite normal. As the
distance becomes larger, the echo becomes increasingly noticeable and irritating. The delay
can be several hundred milliseconds for intercontinental communications, and is particularity
objectionable. Digital Signal Processing attacks this type of problem by measuring the
returned signal and generating an appropriate anti signal to cancel the offending echo. This
same technique allows speakerphone users to hear and speak at the same time without
fighting audio feedback (squealing). It can also be used to reduce environmental noise by
canceling it with digitally generated anti noise.
4.2.2 Echo Location
A common method of obtaining information about a remote object is to bounce a wave off
of it.
For example, radar operates by transmitting pulses of radio waves, and examining the
received signal for echoes from aircraft. In sonar, sound waves are transmitted through the
water to detect submarines and other submerged objects. Geophysicists have long probed the
earth by setting off explosions and listening for the echoes from deeply buried layers of rock.
While these application shave a common thread, each has its own specific problems and
needs. Digital Signal Processing has produced revolutionary changes in all three areas.
Radar is an acronym for Radio Detection And Ranging. In the simplest radar system, a
radio transmitter produces a pulse of radio frequency energy a few microseconds long. This
pulse is fed into a highly directional antenna, where the resulting radio wave propagates
away at the speed of light. Aircraft in the path of this wave will reflect a small portion of the
energy back toward a receiving antenna, situated near the transmission site. The distance to
the object is calculated from the elapsed time between the transmitted pulse and the received
echo. The direction to the object is found more simply; you know where you pointed the
directional antenna when the echo was received. The operating range of a radar system is
determined by two parameters: how much energy is in the initial pulse, and the noise level of
the radio receiver.
Unfortunately, increasing the energy in the pulse usually requires making the pulse
longer. In turn, the longer pulse reduces the accuracy and precision of the elapsed time
measurement. This result in a conflict between two important parameters: the ability to detect
objects at long range, and the ability to accurately determine an object's distance. DSP has
revolutionized radar in three areas, all of which relate to this basic problem. First, DSP can
compress the pulse after it is received, providing better distance determination without
reducing the operating range. Second, DSP can filter the received signal to decrease the
noise. This increases the range, without degrading the distance determination. Third, DSP
enables the rapid selection and generation of different pulse shapes and lengths. Among other
things, this allows the pulse to be optimized for a particular detection problem. Now the
impressive part: much of this is done at a sampling rate comparable to the radio frequency
used, at high as several hundred megahertz! When it comes to radar, DSP is as much about
high-speed hardware design as it is about algorithms.
4.3 Type Of Echoes

In telephone communication, there are two main types of echo:
1- Hybrid (or line)
2- Acoustic echoes.
4.3.1 Hybrid Echo (or line)
The telephone network echo results from the Impedance mismatch at the hybrids of a
public switched telephony network (PSTN) exchange, where the subscriber two-wire lines
are connected to four-wire lines. If a communication is just between two fixed handset
telephones, then only the network echo will occur.
The hybrid echo can be and, in fact, is created in the hybrid part. So part of the signal
being sent to the hybrid on the 4-wire side is returned back as the echo superimposed on the
signal being received from the hybrid on the 4-wire side. Line echo path delays are very short
and each hybrid has a single echo path. Echo paths dont change or change very slowly over
time because of very slow changes of the electrical circuitry parameters and wire lines
parameters in the network. Each hybrid circuit is slightly different each echo tail is different
as well. Many factors determine the echo path. It is even possible for an echo tail to change
while a circuit is active. This could happen when a second telephone extension is taken offhook in parallel with the first one.
Fig 4.1: Hybrid in a PSTN

Due to these variations in echo tails, it is necessary for an echo canceller to adapt to the tail
continuously. Adaptive Filtering is employed within echo cancellers to this end. The adaptive
filters should converge quickly, but not so quickly that they might diverge under some
conditions. This is especially important when a circuit is first established. The amount of
time it takes the echo canceller to adapt to an echo path is referred to as the "convergence
time".
4.3.2 Acoustic echoes
If communication is between one or more hands-free telephones (or speaker phones), then
acoustic feedback paths are set up between the telephone's loudspeaker and microphone at
each end. In the case of acoustic echo, if the time delay is not long, then the echo can be
perceived as soft reverberation, which adds artistic quality, for example in a concert hall.
However, a strong echo that arrives a few tens of milliseconds or more after the initial direct
sound will be highly undesirable and irritating. This acoustic coupling is due to the reflection
of the loudspeaker's sound waves from walls, ceiling, windows and other objects back to the
microphone. The coupling can also be due to the direct path from the loudspeaker to the
microphone, see Figure.
Adaptive cancellation of such acoustic echo has become very important in hands-free
communication systems, e.g. tele-conference, video-conference and PC telephony systems.
The effects of an echo depend mostly on the time delay between the initial and reflected
sound waves (or sound signals), and the strength of the reflected sounds. In the case of
acoustic echo, if the time delay is not long, then the echo can be perceived as soft
reverberation, which adds artistic quality, for example in a concert hall. However, a strong
echo that arrives a few tens of milliseconds or more after the initial direct sound will be
highly undesirable and irritating. In the case of network echo, a short delayed echo cannot be
distinguished from the normal side-tone of the telephone, which is intentionally inserted to
make the communication channel sounds \alive". However, an echo, with a round trip delay
of more than 40 msec will cause significant disturbances to the talker. Such a long delay is
caused by the propagation time over long distances and/or the digital encoding of the
transmitted signals.
Fig 4.2: Sources of acoustic echo in a room when using a hand-free telephone
Acoustic echo cancellation is a common occurrence in today telecommunication systems.

It occurs when an audio source and sink operate in full duplex mode, an example of this is a
hands-free loudspeaker telephone. In this situation the received signal is output through the
telephone loudspeaker (audio source); this audio signal is then reverberated through the
physical environment and picked up by the systems microphone (audio sink). The effect is
the return to the distant user of time delayed and attenuated images of their original speech
signal. The signal interference caused by acoustic echo is distracting to both users and causes
reduction In the quality of the communication. Adaptive filtering techniques to reduce this
unwanted echo, thus increasing communication quality. These echoes can be very annoying
to callers. A widely used technique to suppress echoes is to employ adaptive echo cancellers.
4.4 Adaptive Echo Cancellers

A technique to remove or cancel echoes is shown in Figure. The echo canceller mimics
the transfer function of the echo path (or room acoustic) to synthesize a replica of the echo,
and then subtracts that replica from the combined echo and near-end speech (or disturbance)
signal to obtain the near end signal alone. However, the transfer function is unknown in
practice, and so it must be identified. The solution to this problem is to use an adaptive filter
The method used to cancel the echo signal is known as adaptive filtering. Adaptive filters are
dynamic filters which iteratively alter their characteristics in order to achieve an optimal
desired output. An adaptive filter algorithmically alters its parameters in order to minimize a
unction of the difference between the desired output d(n) and its actual output y(n).
This function is known as the cost function of the adaptive algorithm. Figure shows a
block diagram of the adaptive echo cancellation system implemented throughout this thesis.
Here the filter H(n) represents the impulse response of the acoustic environment, W(n)
represents the adaptive filter used to cancel the echo signal. The adaptive filter aims to equate
its output y(n) to the desired output d(n) (the signal reverberated within the acoustic
environment). At each iteration the error signal, e(n)=d(n)-y(n), is fed back into the filter,
where the filter characteristics are altered accordingly.
Block diagram of Adaptive Echo Canceller
In a room with no propagation delay and no room impulse response (i.e. a studio with
dampening walls and the microphone placed with no distance from the loudspeaker) the
solution would simply be to subtract the input (far-end talk), which is readily available, from
the output signal picked up by the microphone, which consists of both near-end talk and farend talk. After the subtraction the output signal would consist of near-end talk only. This is

not possible, however, since the room both alters the sound and spreads it over time. Using
IP-telephony, as illustrated in Figure 4, this spread over time will vary according to the delay
in the net and therefore IP telephony introduces even more problems. Due to these problems
the input must be modified accordingly before we subtract it. The problem is that the
parameter after which it should be modified is unknown. This is where the adaptive filtering
technology comes in. The adaptive filter adjusts according to inputs and outputs to form the
parameters mentioned after which the input must be modified if a subtraction is to be useful.
Acoustic echo cancellation also introduces a second problem: To be able to detect when
there is nothing but far-end talk entering the microphone, and when there are other things
entering as well. The adaptation algorithm uses only one measurement, the difference
between the modified input and the real input. If this difference is zero, and no near-end talk
is present, the filter will be an exact copy of the room impulse response and hence work as
intended. Now if, at this time, there is near-end talk, the difference will be equal to the nearend talk and the filter algorithm will notice this as an error in the filter. The filter will
therefore adapt to cancel out the near-end talk as well and as a result the it will cease to work.
The same techniques are used in data networking where there are also problems with echoes.
A filter is something that transforms data to extract useful information from a noisy
environment. In digital filtering there are two primary types; infinite impulse response (IIR)
and finite impulse response (FIR). IIR filters can normally achieve the same filter
characteristics as a FIR filter
using less memory and calculations with the cost of possibly becoming unstable. As the
filters become more complex though, IIR filters needs more parameters and the advantages
are reduced. Because of the high complexity of the many strong and sharp peaks in a room
impulse response the filters that are being used in acoustic echo cancellation are usually of
the FIR type.3 For the filter in an acoustic echo cancellation to work in the real case, with
changing parameters such as different room acoustics and people moving around in the room,
a filter with adapting parameters (taps) is necessary.
Figure 6: Filter adaptation.

There are numerous adaptive algorithms that are applicable in acoustic echo
cancellation such as
least mean squares (LMS), recursive least squares (RLS), affine projection algorithm (AFA)
and different degenerates there of. LMS is an old, simple and proven algorithm which has
turned out to work well in comparison with newer more advanced algorithms. In this project
we use the normalized LMS (NLMS) for the main filter and LMS for the noise generation.
5.1 LMS
In 1959 Widrow and Hoff introduced the LMS algorithm. During the years this has been
the by far most used adaptive filtering algorithm for several reasons; it was the first, it
requires relatively little computation and it works well, at least for slow changes in the filter.
The LMS filter uses a gradient method of steepest decent to adapt its weights to minimize
some function c(i) defined in Mathematical formula
Mathematical formula 1: Function c(n) to be minimized.

where e(n) is defined in Figure 5. In comparison to other algorithms LMS is relatively simple
as it doesnt require correlation function calculation or matrix inversions, for each sample in
the signal LMS only uses two multiplications and two additions.
4.5 Choice Of Algorithm

A wide variety of recursive algorithms have been developed in the literature for the
operation of linear adaptive filters, In the final analysis, the choice of one algorithm over
another is determined by one or more of the following factors:
Rate of Convergence
This is defined as the number of iterations required for the algorithm, in response to
stationary inputs, to converge close enough to the optimum wiener solution in the meansquare error sense. A fast rate of convergence allows the algorithm to adapt rapidly to a
stationary environment of unknown statistics.

Misadjustment
For an algorithm of interest, this parameter provides a quantitative measure of the amount
which the final value of the mean-square error, averaged over an ensemble of adaptive filters,
deviates from the minimum mean-square error produced by the Wiener filter
Tracking
When an adaptive filtering algorithm operates in a non-stationary environment. The
algorithm is required to track statistical variations in the environment. Two contradictory
features, however, influence the tracking performance of the algorithm:
(1) rate of convergence, and
(2) steady-state fluctuation due to algorithm noise.
Robustness
For an adaptive filter to be robust, small disturbances (I.e., disturbances with small
energy) can only result in small estimation errors. The disturbances may arise from a variety
of factors, internal or external to the filter.
Computational requirements:
Here the issues of concern include
(a) The number of operations (i.e., multiplications, divisions, and additions/
subtractions) required to make one complete iteration of the algorithm.
(b) The size of memory locations required to store the data and the program,
(c) The investment required to program the algorithm on a computer.
4.6 Approach To Develop Linear Adaptive Filter

4.6.1 Stochastic Gradient Approach
The stochastic gradient approach uses a tapped-delay line, or transversal filter, as the
structural basis for implementing the linear adaptive filter. For the case of stationary inputs,
the cost function, also referred to as the index of performance, is defined as the mean square
error (i.e., the mean square value of the difference between the desired response and the
transversal filter output).
This cost function is precisely a second order function of the tap weights in the transversal
filter. To develop a recursive algorithm for updating the tap weights of the adaptive
transversal filter, we proceed in two stages, First, we use an iterative procedure to solve the
Wiener Hopf equations (i.e., the Matrix equation defining the optimum Wiener solution); the
iterative procedure is based on the method of steepest descent, which is a well known
technique in optimization theory. This method required the use of a gradient vector, the value
of which depends on two parameters: the correlation Matrix of the tap inputs in the
transversal filter and the cross correlation vector between the desired response and the same
tap inputs. Next, we use instantaneous values for this correlation, so as to derive an estimate
for the gradient vector, making it assume a stochastic character in general. The resulting
algorithm is widely known as the least mean square (LMS) algorithm, , the essence of which
for the case of a transversal filter operating on real valued data may be described as
Where the error signal is defined as the difference between some desired response and the
actual response of the transversal filter produced by the tap input vector.
4.7 Least Mean Square (LMS) Algorithm

The Least Mean Square (LMS) algorithm was first developed by Widrow and Hoff
in1959 through their studies of pattern recognition. From there it has become one of the most
widely used algorithms in adaptive filtering. The LMS algorithm is an important member of
the family of stochastic gradient-based algorithms as it utilizes the gradient vector of the
filter tap weights to converge on the optimal wiener solution. It is well known and widely
used due to its computational simplicity. It is this simplicity that has made it the benchmark
against which all other adaptive filtering algorithms are judged.
The LMS algorithm is a linear adaptive filter algorithm, which in general consists of two
basic processes.
1. A filter process: which involves
a. Computing the output of a linear filter in response to an input signal.
b. Generating an estimation error by comparing this output with a desired response.
2. An adaptive process: which involves the automatic adjustment of the Parameters of the
filter in
accordance with the estimation error.
The combination of these two processes working together constitutes a feedback loop;
First, we have a transversal filter, around which the LMS algorithm is built. This component
is responsible for performing the filtering process. Second, we have a mechanism for
performing the adaptive control process on the tap weights of the transversal filter. With each
iteration of the LMS algorithm, the filter tap weights of the adaptive filter are updated
according to the following formula (Farhang-Boroujeny 1999).
w (n +1) = w(n) + 2ex(n)

Here x(n) is the input vector of time delayed input
values, x(n) = [x(n) x(n-1) x(n-2) .x(n- N+1)] .
The vector
represents the coefficients of the adaptive FIR
filter tap weight vector at time n. The parameter is known as the step size parameter and is
a small positive constant. This step size parameter controls the influence of the updating
factor.
Selection of a suitable value for is imperative to the performance of the LMS algorithm, if
the value is too small the time the adaptive filter takes to converge on the optimal solution
will be too long; if is too large the adaptive filter becomes unstable and its output diverges.
Is the simplest to implement and is stable when the step size parameter is selected
appropriately. This requires prior knowledge of the input signal which is not feasible for the
echo cancellation system.
4.7.1 Derivation of the LMS algorithm

The derivation of the LMS algorithm builds upon the theory of the wiener solution for the
optimal filter tap weights, Wo . It also depends on the steepest-descent algorithm. This is a
formula which updates the filter coefficients using the current tap weight vector and the
current gradient of the cost function with respect to the filter tap weight coefficient vector ,
As the negative gradient vector points in the direction of steepest descent for the Ndimensional quadratic cost function, each recursion shifts the value of the filter coefficients
closer toward their optimum value, which corresponds to the minimum achievable value of
the cost function, (n).The LMS algorithm is a random process implementation of the
steepest descent algorithm. Here the expectation for the error signal is not known so the
instantaneous value is used as an estimate. The steepest descent algorithm then becomes
The gradient of the cost function,(n), can alternatively be expressed in the following
form.
Substituting
into
the
this
steepest
descent algorithm of
Eqn , we arrive at the
recursion for the
LMS adaptive algorithm.
w (n +1) = w(n) + 2ex(n)

4.7.2 Implementation of the LMS algorithm
Each iteration of the LMS algorithm requires 3 distinct steps in this order:
1. The output of the FIR filter, y(n) is calculated using equation
2. The value of the error estimation is calculated using equation
3. The tap weights of the FIR vector are updated in preparation for the next
iteration, by
The main reason for the LMS algorithms popularity in adaptive filtering is its
Computational simplicity, making it easier to implement than all other commonly use
adaptive algorithms. For each iteration the LMS algorithm requires 2N additions and 2N+1
multiplications (N for calculating the output, y(n), one for 2e(n) and an additional N for the
scalar by vector multiplication).
4.8 Normalized Least Mean Square (NLMS) Algorithm

One of the primary disadvantages of the LMS algorithm is having a fixed step size
parameter for every iteration. This requires an understanding of the statistics of the input
signal prior to commencing the adaptive filtering operation. In practice this is rarely
achievable. Even if we assume the only signal to be input to the adaptive echo cancellation
system is speech, there are still many factors such as signal input power and amplitude which
will affect its performance. The normalized least mean square algorithm (NLMS) is an
extension of the LMS algorithm which bypasses this issue by selecting a different step size
value, (n), for each iteration of the algorithm. This step size is proportional to the inverse of
the total expected energy of the instantaneous values of the coefficients of the input vector
x(n). This sum of the expected energies of the input samples is also equivalent to the dot
product of the input vector with itself, and the trace of input vectors auto-correlation matrix.
The recursion formula for the NLMS algorithm is stated in equation
Simple to implement and computationally efficient. Shows very good attenuation and
variable step size allows stable performance with non-stationary signals. This was the
obvious choice for real time implementation.
4.8.1 Derivation of the NLMS algorithm
This derivation of the normalized least mean square algorithm .To derive the NLMS
algorithm we consider the standard LMS recursion, for which we select a variable step size
parameter, (n). This parameter is selected so that the error value, e + (n), will be minimized
using the updated filter tap weights, w(n+1), and the current input vector, x(n).
This (n) is then substituted into the standard LMS recursion replacing , resulting
in the following.
Here the value of is a small positive constant in order to avoid division by zero when the
values of the input vector are zero. This was not implemented in the real time implementation
as in practice the input signal is never allowed to reach zero due to noise from the
microphone and from the AD codec on the Texas Instruments DSK. The parameter is a
constant step size value used to alter the convergence rate of the NLMS algorithm, it is
within the range of 0<<2, usually being equal to 1.
4.8.2 Implementation of the NLMS algorithm

The NLMS algorithm has been implemented in Mat lab and in a real time application
using the Texas Instruments TMS320C6711 Development Kit. As the step size parameter is
chosen based on the current input values, the NLMS algorithm shows far greater stability
with unknown signals. This combined with good convergence speed and relative
computational simplicity makes the NLMS algorithm ideal for the real time adaptive echo
cancellation system. The code for both the Matlab and TI Code composer studio applications
can be found in appendices A and B respectively. As the NLMS is an extension of the

standard LMS algorithm, the NLMS algorithms practical implementation is very similar to
that of the LMS algorithm. Each iteration of the NLMS algorithm requires these steps in the
following order:1. The output of the adaptive filter is calculated.
2. An error signal is calculated as the difference between the desired signal and
the filter output.
3. The step size value for the input vector is calculated.
4. The filter tap weights are updated in preparation for the next iteration. Each iteration
of the NLMS algorithm requires 3N+1 multiplications, this is only N more than the
standard LMS algorithm .This is an acceptable increase considering the gains in
stability and echo attenuation achieved.
CHAPTER - 5
INTRODUCTION TO MATLAB
5.1 What Is MATLAB?
MATLAB
is a high-performance language for technical computing. It integrates
computation, visualization, and programming in an easy-to-use environment where problems

and solutions are expressed in familiar mathematical notation. Typical uses include
Typical uses of MATLAB
Math and computation
Algorithm development
Data acquisition
Modeling, simulation, and prototyping
Data analysis, exploration, and visualization
Scientific and engineering graphics
Application development, including graphical user interface building.
The main features of MATLAB
Advance algorithm for high performance numerical computation ,especially in the
Field matrix algebra
A large collection of predefined mathematical functions and the ability to define

ones own functions.
Two-and three dimensional graphics for plotting and displaying data
A complete online help system
MATLAB
MATLAB
Programming language
User written / Built in functions
Graphics
2-D graphics
3-D graphics
Color and lighting
Animation
Computation
Linear algebra
Signal processing
Quadrature
Etc
Tool boxes
1. Signal processing
2. Image processing
3. Control systems
4. Neural Networks
5. Communications
6. Robust control
7. Statistics
Fig 5.1 : Features of MATLAB
External interface
Interface with C and
FORTRAN
Programs
Features and capabilities of MATLAB

MATLAB is an interactive system whose basic data element is an array that does not
require dimensioning. This allows you to solve many technical computing problems,
especially those with matrix and vector formulations, in a fraction of the time it would take to
write a program in a scalar non interactive language such as C or FORTRAN.
The name MATLAB stands for matrix laboratory. MATLAB was originally written to
provide easy access to matrix software developed by the LINPACK and EISPACK projects.
Today, MATLAB engines incorporate the LAPACK and BLAS libraries, embedding the
state of the art in software for matrix computation.
MATLAB has evolved over a period of years with input from many users. In university
environments, it is the standard instructional tool for introductory and advanced courses in
mathematics, engineering, and science. In industry, MATLAB is the tool of choice for highproductivity research, development, and analysis.
MATLAB features a family of add-on application-specific solutions called toolboxes.
Very important to most users of MATLAB, toolboxes allow you to learn and apply
specialized technology. Toolboxes are comprehensive collections of MATLAB functions
(M-files) that extend the MATLAB environment to solve particular classes of problems.
Areas in which toolboxes are available include signal processing, control systems, neural
networks, fuzzy logic, wavelets, simulation, and many others.
5.2 The MATLAB System:

The MATLAB system consists of five main parts:
Development Environment:
This is the set of tools and facilities that help you use MATLAB functions and files. Many
of these tools are graphical user interfaces. It includes the MATLAB desktop and Command
Window, a command history, an editor and debugger, and browsers for viewing help, the
workspace, files, and the search path.
The MATLAB Mathematical Function:
This is a vast collection of computational algorithms ranging from elementary functions
like sum, sine, cosine, and complex arithmetic, to more sophisticated functions like matrix
inverse, matrix Eigen values, Bessel functions, and fast Fourier transforms.
The MATLAB Language:
This is a high-level matrix/array language with control flow statements, functions, data
structures, input/output, and object-oriented programming features. It allows both
"programming in the small" to rapidly create quick and dirty throw-away programs, and
"programming in the large" to create complete large and complex application programs.
Graphics:
MATLAB has extensive facilities for displaying vectors and matrices as graphs, as well as
annotating and printing these graphs. It includes high-level functions for two-dimensional
and three-dimensional data visualization, image processing, animation, and presentation
graphics. It also includes low-level functions that allow you to fully customize the
appearance of graphics as well as to build complete graphical user interfaces on your
MATLAB applications.
The MATLAB Application Program Interface (API):

This is a library that allows you to write C and Fortran programs that interact with
MATLAB. It includes facilities for calling routines from MATLAB (dynamic linking),
calling MATLAB as a computational engine, and for reading and writing MAT-files.
5.3 MATLAB working environment:

MATLAB DESKTOP:MATLAB Desktop is the main MATLAB application window. The desktop contains five
sub windows, the command window, the workspace browser, the current directory window,
the command history window, and one or more figure windows, which are shown only when
the user displays a graphic.
The command window is where the user types MATLAB commands and expressions at
the prompt (>>) and where the output of those commands is displayed. MATLAB defines the
workspace as the set of variables that the user creates in a work session. The workspace
browser shows these variables and some information about them. Double clicking on a
variable in the workspace browser launches the Array Editor, which can be used to obtain
information and income instances edit certain properties of the variable.
The current Directory tab above the workspace tab shows the contents of the current
directory, whose path is shown in the current directory window For example, in the windows
operating system the path might be as follows: C:\MATLAB\Work, indicating that directory
work is a subdirectory of the main directory MATLAB; WHICH IS INSTALLED IN
DRIVE C. clicking on the arrow in the current directory window shows a list of recently used
paths. Clicking on the button to the right of the window allows the user to change the current
directory.
MATLAB uses a search path to find M-files and other MATLAB related files, which are
organize in directories in the computer file system. Any file run in MATLAB must reside in
the current directory or in a directory that is on search path. By default, the files supplied
with MATLAB and math works toolboxes are included in the search path. The easiest way to
see which directories are on the search path The easiest way to see which directories are soon
the search paths, or to add or modify a search path, is to select set path from the File menu
the desktop, and then use the set path dialog box. It is good practice to add any commonly
used directories to the search path to avoid repeatedly having the change the current
directory.
The Command History Window contains a record of the commands a user has entered in
the command window, including both current and previous MATLAB sessions. Previously
entered MATLAB commands can be selected and re-executed from the command history
window by right clicking on a command or sequence of commands. This action launches a
menu from which to select various options in addition to executing the commands.
5.4 IMPLEMENTATIONS
Arithmetic operations
Entering Matrices
The best way for you to get started with MATLAB is to learn how to handle matrices.
Start MATLAB and follow along with each example.
You can enter matrices into MATLAB in several different ways:
Enter an explicit list of elements.
Load matrices from external data files.
Generate matrices using built-in functions.
Create matrices with your own functions in M-files.
Start by entering Drers matrix as a list of its elements. You only have to follow a few basic
conventions:
Separate the elements of a row with blanks or commas.
Use a semicolon, to indicate the end of each row.
Surround the entire list of elements with square brackets, [ ].
To enter matrix, simply type in the Command Window
A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1]
MATLAB displays the matrix you just entered:
A=
16 3 2 13
5 10 11 8
9 6 7 12
4 15 14 1
This matrix matches the numbers in the engraving. Once you have entered the matrix, it is
automatically remembered in the MATLAB workspace. You can refer to it simply as A. Now
that you have A in the workspace,
Sum, transpose, and diag
You are probably already aware that the special properties of a magic square have to do
with the various ways of summing its elements. If you take the sum along any row or

Main Pronect

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Main Pronect

Uploaded by

Copyright:

Available Formats

ABSTRACT

Figure 1.1: Origins of acoustic echo

Figure 1.2: Block diagram of an adaptive echo cancellation system.

1.2 Discrete Time Signals

1.3 Random Signals

1.4 Stationary Signals

2. The autocorrelation function is also constant over an arbitrary time shift.

1.5 Speech Signals

Figure 1.3 Representation of Speech Signal

1.5.1 Speech generation

1.6 Noise Sources

1.7 Correlation Function

time n multiplied by its complex conjugate value at a different time m.

xx(n,m) = E[x(n)x*(m)] ....eq 1.4

xx (n, m) =E[x(n)x(m)] eq1.5

known as the impulse response.

b. Finite Impulse Response (FIR) Filters

for generating a truncated impulse response for the

A butterworth low-pass filter can be constructed with

another. It may exist as an equation on paper, as a small loop in a computer subroutine, or as

2.2 FIR Filter

Figure 2.1 FIR Filter Showing Delay

2.3 IIR Filter

2.4 Frequency Selective Filters

2.5 Type of Frequency Selective Filters

2.5.3 Band Pass Filter

fig 2.2 (a) Low pass filter

(b) High pass

(c) Band pass

3.1 Adaptive Filter

Y(f) = X(f)*H(f)*F(f), . eq 3.1

3.2 Adaptive Filter Structure

Figure 3.1 Adaptive Filter Block Diagram

Figure 3.2 Basic Adaptive Filter structure

3.3 Adaptive Filtering System Configurations

ACOUSTIC ECHO CANCELLATION

ACOUSTIC ECHO CANCELLATION

Figure 3.3 Adaptive System Configurations

3.3.2 Adaptive Linear Prediction

ACOUSTIC ECHO CANCELLATION

Fig 3.4 Adaptive Linear Prediction

3.3.3 Adaptive Inverse System Configuration

ACOUSTIC ECHO CANCELLATION

Fig 3.5 Adaptive Inverse system

Fig 3.6: Adaptive Noise or Echo Cancellation

ACOUSTIC ECHO CANCELLATION

3.3.5 Additional Structures

(b) Adaptive channel equalization

Fig 3.7: Adaptive Channel Equalization

3.4 Classes Of Adaptive Filter

ACOUSTIC ECHO CANCELLATION

Least Mean Square Algorithm

3.5 Performance Measures In Adaptive Systems

ACOUSTIC ECHO CANCELLATION

ACOUSTIC ECHO CANCELLATION

ACOUSTIC ECHO CANCELLATION

ACOUSTIC ECHO CANCELLATION

ACOUSTIC ECHO CANCELLATION

ACOUSTIC ECHO CANCELLATION

4.3 Type Of Echoes

Fig 4.1: Hybrid in a PSTN

ACOUSTIC ECHO CANCELLATION

ACOUSTIC ECHO CANCELLATION

Acoustic echo cancellation is a common occurrence in today telecommunication systems.

Y(f) = X(f)H(f)F(f), . eq 3.1