Adaptive Filters Theory

Wiener Filters
(Multiple Linear Regression Model)

Submitted By:
Komal Ahmad
1. Computer Experiment: Consider an all pole system (the unknown system) of
fourth order, whose input and noise corrupted output are accessible. Model the
unknown system by a Multiple Linear Regression Model. Design a Wiener filter that
identifies the unknown system. Show the effect of increasing order of Wiener filter on
the cost function for the following cases
(a) Under fitted model.
(b) Critically fitted model.
(c) Over fitted model.
clc;close all
v=0.5*(rand(1,1000)-0.5); %v=noise vector (20 uniformly
distributed rv's with zero mean)
u=randn(1,1000); %u=data vector entering the system
and the weiner filter(20 normally distributed with zero mean)
b=1;a=[1 0.75 -0.25 -0.5]; %coefficinets of unknown AR
process M=4
unknwnsys_op=filter(b,a,u); %unknown system output
dn=unknwnsys_op+v; %desired output

max_reg_lg=20; %max.lags for regression model

N=length(u); %length of input data vector
jmin=zeros(1,max_reg_lg);

for lg=1:max_reg_lg
us=zeros(lg,N);
for m=1:lg
for n=1:N+1-m
us(m,n)=u(n-1+m);
end
end
r1=us*u';
r=r1'./N; %autocorrelation of input signal
R=toeplitz(r); %autocorrelation matrix
pdx=xcorr(u,dn,'biased'); %cross correlation btwn input
signal and desired output
p=pdx(1,(N-lg)+1:N);
w=inv(R)*p'; %filter taps

N=length(dn);
lag=1;
dns=zeros(lag,N);
for m=1:lag
for n=1:N+1-m
dns(m,n)=dn(n-1+m);
end
end
r1=dns*dn';
var_dn=r1'./N; %variance of desired output
jmin(1,lg)=var_dn-p*w %cost function matrix for various
orders
end

m=0:max_reg_lg
jmo=var_dn;
jmin=horzcat(jmo,jmin);
stem(m,jmin); %plot of cost function against
various filter orders
When unknown system is IIR:

M<18: Under Fitted

M>18: Over Fitted
Minimum mean-square Jmin(M) 2.5

1.5

M=18
0.5 Critically Fitted

0
0 5 10 15 20 25
Wiener filter length M

 When under fitted, jmin or minimum mean square error doesn’t reach the
variance of the noise signal infact it is greater than the variance of the noise and maximum when
M=0.
 When critically fitted, jmin=σv2 i.e. min. mean square error is equal to
variance of the noise signal. In our case min. mean square error is equal to 0.02 which is also the
variance of noise.
 When over fitted, jmin stays the same for higher order of M of filter taps.

When unknown system is FIR:

2
M<4:Under Fitted
1.8
M>4:Over Fitted

1.6
Minimum mean-square Jmin(M)

1.4

1.2

0.8

0.6
M=4
0.4 Critically Fitted

0.2

0
0 5 10 15 20 25
Wiener filter length M
2. Computer Experiment: Refer to Article 2.7 (Example of Wiener Filtering) of the
text book. Reproduce the results of this article.
function[wo,jm]=wienerfilter(R,p,var_d,var_v,M)
%R=4x4 autocorrelation matrix,d_var=variance of desired signal...
%..,var_v=variance of noise signal,M=filter order(4)
wo=zeros(M,M);
for m=1:M
wo(1:m,m)=inv(R(1:m,1:m))*p(1:m,1); %optimum filter
taps matrix.M=1,2,3,4
end
jmo=var_d; %optimum filter tap
when M=0
jm=var_d-(p(1:m,1))'*wo; %matrix of min.cost
func
jm=horzcat(jmo,jm);
figure; %plot of cost
function
m=0:M;stem(m,jm);xlabel('Wiener filter length M');ylabel('Minimum mean-
square Jmin(M)');

R = [1.1 0.5; 0.5 1.1]; % Input

autocorrelation matrix for M=2
p = [0.5272; -0.4458]; % Cross-correlation
vector for M=2
w0 = 0:0.1:4; % Range of first tap
weight values
w1 = -4:0.1:0; % Range of second
tap weight values
J = zeros(length(w0),length(w1));
% Clear MSE values at all (w0, w1)
for m = 1:length(w0)
for n = 1:length(w1)
w = [w0(m) w1(n)]';
J(m,n) = var_d-2*p'*w + w'*R*w;
% Compute MSE values at all (w0, w1)
end % points, store in J
matrix
end
figure; meshc(w0,w1,J'); % Plot combination
of mesh and contour
view([-130 22]); % Set the angle to
view 3-D plot
hold on; w_opt = inv(R)*p;
plot(w_opt(1),w_opt(2),'o');xlabel('w_0');ylabel('w_1');zlabel('Cost
function J');
title('Error performance surface of the two-tap transversal filter')

w0 = 0:0.1:2; % Range of first tap

weight values
w1 = -2:0.1:0;
figure;
[x,y]=meshgrid(w0,w1);
J=0.9486-1.0544*x+0.8961*y+x.*y+1.1*x.^2+1.1*y.^2;
contour(x,y,J,17); %no.of contour
curves is 17
axis equal tight;
xlabel('w_0');ylabel('w_1');
title('Contour plots of the error performance surface');
 1

0.9

Minimum mean-square Jmin(M) 0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 0.5 1 1.5 2 2.5 3 3.5 4
Wiener filter length M

Error performance surface of the two-tap transversal filter

15

10
Cost function J

0
4
3 -4
-3
2
-2
1 -1
0 0
w1
w0
 Contour plots of the error performance surface
0

-0.2

-0.4

-0.6

-0.8

-1
w1

-1.2

-1.4

-1.6

-1.8

-2
0 0.5 1 1.5 2
w0