Assignment2 Report

CV Assignment 2
Kartik Dutta
201302018
Question 1&2. PROCEDURE

The DLT and RANSAC implementation is very similar to that of the first assignment. Except that, now we are taking two
special cases. One is when the camera center can be arbitrary and its rotation arbitrary but all images are along the XY
plane. Therefore, z = 0. Hence, the transformation from world to image co-ordinates becomes:
transpose([x y w]) = [p1 p2 p4]*transpose([X Y 1])
=> transpose([x y w]) = H1*transpose([X Y 1])
=> x1 = H1*X_world
Similarly, the transformation of the same world point in another image is:
x2 = H2*X_world =>
x2 = H2*inv(H1)*H1*X_world
=> x2 = (H2*inv(H1))*x1
=> x2 = H21*x1
Here, H21 is called the Homography matrix that transforms a point in image 1 to a point in image2. It is a 3x3 matrix.
Now, the 2nd case involves when the world points can be anywhere but the two cameras must share a common camera
center.
In DLT, we have to minimise the norm of A*P. In this case, A is a (2N)x9, while P is a 9x1 matrix which is the 9 elements
of the Homography matrix. The rest of the procedure is the same as Assignment 1.
However, we have to remember to normalise the image in order to not get too large singular values while doing SVD.
The following two normalisations are performed - 1. The origin is shifted to the centroid of the selected points. This is
done by subtracting the mean of the distribution. 2. The points should be scaled such that the average distance of them
from the origin is sqrt(2).
Instead of selecting the points manually, we use SIFT Features to find the point correspondences and use them to
compute the Homography matrix. The SIFT code used is the one released by David Lowe.
OBSERVATIONS
1. Since there a large number of points (313, to be exact) select by SIFT, DLT seems to do pretty bad. The per point error
is very high and un-acceptable. Since RANSAC randomises the selection and computes the H matrix for the least error, it
gives a very good result. This complies with the known fact that RANSAC is very good in the elimination of noise.
2. Some of the SIFT feature points were minor variations in the sky (in this image). The were getting matched to their
reflections on the glass windows of the building. Computing Homography in such situations with objects having
shadows, reflections, changes in illumination etc. is a difficult task.
RESULTS
Homography matrix estimated using Sift features & DLT
-0.4410 0.2451 495.2773
-0.7081 0.8412 491.2968
-0.0019 0.0005 1.9699
Error per point during DLT

254.9326
Homography matrix estimated using Sift features & RANSAC
0.5994 -0.3331 -673.0809
0.9623 -1.1431 -667.6715
0.0026 -0.0007 -2.6771
Error per point during RANSAC
3.6473e-12

In this part, we manually have to select the point correspondences between the two images. Since these points are good
features and on an average are better than the automatically selected ones, we are supposed to get better or atleast
equivalent results as compared to the automatic selection.
OBSERVATIONS
1. From the results below, we can see that the Homography matrix is similar using both DLT and RANSAC. If good feature
points are selected, DLT does well. Also, if these good feature points are few in number, DLT does better than RANSAC
(as we can see from the per point error in the results).
2. When the number of points are large, RANSAC does well in removing the noise and thereby gives a better accuracy.
While humans might select a few points, the computer will select plenty and more noise can be introduced in this
process.
3. Error in manual selection of points can result in the selection of bad features. This might result in unnecessary noise
and, therefore in a bad homography. But, manual selection has the advantage of selecting those features that are
visually distinct according to the human visual system (which is one of the benchmarks). While the
computer sees the image as pixels, we already see it in terms of features. Moreover, the computer might recognise
points that might be features, but irrelevant in identifying the image (from a human point of view). For e.g.: particular
set of clouds or a patch of grass. Humans might consider these as noise.
4. While the human eyes can recognise various distinct features easily, we lack an attention to detail that the computer
does. It may select those points that are unique features missed out by us and thus, can describe the image better.
RESULTS
Homography matrix - using DLT & Manually selected Points
0.4588 0.0463 155.3330
-0.1043 0.5798 5.1197
-0.0003 0.0000 0.6116
Error per point during DLT

0.4106
Homography matrix - using RANSAC & Manually selected points
0.4437 0.0213 163.7131
-0.1036 0.5664 7.7679
-0.0003 -0.0000 0.6344
Error per point during RANSAC
2.5124
Input and Output Images
SIFT Matching
Homography computation using DLT and SIFT
Homography computation using RANSAC and SIFT
Homography computation using DLT and Manual Point Selection
Homography computation using RANSAC and Manual Point Selection
Code
function q2a()
im1 = imread('building1.jpg');
H = dlt(im1, im2);
disp(H);
end
function new_P = dlt(im1, im2)
[x1,x2] = helper(rgb2gray(im1),rgb2gray(im2));
y2 =
y1 =
temp
x1 =
x2 =
x1(:,2);
x2(:,2);
= x1;
x2(:,1);
temp(:,1);
%Normalizing
[x_col, y_row, N1] = normalize_image(x1, y1);
[x_dash, y_dash, N2] = normalize_image(x2, y2);
%Writing world points as image 2's points
world = [];
for i = 1:size(x_dash, 1)
temp = [x_dash(i) y_dash(i) 1];
world = [world; temp];
end
%Writing the 'A' matrix in AxP = [0]

A = [];
for i = 1:size(world, 1)
temp = [
world(i, :, :) 0 0 0 -1*x_col(i)*world(i, :, :);
0 0 0 world(i, :, :) -1*y_row(i)*world(i, :, :);
];
A = [A; temp];
end
%Doing SVD
[U, D, V] = svd(A);
%Projection matrix is the last col of V since D is least for
last col
%Hence, we can minimize A*P
P = V(:, 9);
new_P = [P(1:3)'; P(4:6)'; P(7:9)'];
new_P = inv(N1)*new_P*N2;
op = [];
load('Images_point.mat');
x_col = x1;
y_row = y1;
world = [];
for i = 1:size(x2, 1)
temp = [x2(i) y2(i) 1];
end
%Q5 - Overlay Mesh Network
answ = new_P*world(i, :)';
for j = 1:3
answ(j) = answ(j)/answ(3);
end
op = [op; answ'];
end
%Run Homography matrix over world i.e., points from image2
(above) and
%overlay on image1 (below).

figure, imagesc(im1), colormap(gray), hold on
plot(op(:, 1), op(:, 2), 'r')
title('Image1 with Homography (Transformation) over
Image2');
end
%Function to normalize all the image co-ordinates
function [x_col, y_row, N] = normalize_image(x_col, y_row)
x_m = mean2(x_col);
y_m = mean2(y_row);
x_new = x_col - x_m;
y_new = y_row - y_m;
D = sqrt(x_new.^2 + y_new.^2);
scale = sqrt(2)/mean(D(:));
N = [
scale
0
-scale*x_m
0
scale -scale*y_m
0
0
1
];
p = [x_col y_row ones(size(x_col))]';
norm_points = N*p;
x_col = norm_points(1, :)';
y_row = norm_points(2, :)';
end
function q2b()
[x1,x2] = helper(rgb2gray(im1),rgb2gray(im2));
y2 =
y1 =
temp
x1 =
x2 =
x1(:,2);
x2(:,2);
= x1;
x2(:,1);
temp(:,1);
world = [];
temp = [x2(i) y2(i) 1];

end
old_world = world;
old_x = x1;
old_y = y1;
%Normalizing
world = [];
end
no_pt = 5;
total = size(x_col, 1);
iter = 50;
min_err = 9999999;
final_P = zeros(3, 3);
for k = 1:iter
idx = randperm(total, no_pt);
A = [];
for i = 1:no_pt
temp = [
];
A = [A; temp];
end
%Doing SVD
[U, D, V] = svd(A);
%Projection matrix is the last col of V since D is least
for last col
P = V(:, 9);
new_P = [P(1:3)'; P(4:6)'; P(7:9)'];
err = 0;
%Finding error
for i = 1:size(old_world, 1)
answ = new_P*old_world(i, :)';
for j = 1:3
end
err_x = abs(answ(1) - old_x(i));
err_y = abs(answ(2) - old_y(i));
err = err + err_x + err_y;
end
if err < min_err
final_P = new_P;
min_err = err;
end
end
op = [];
x_col = old_x;
y_row = old_y;
world = old_world;
answ = final_P*world(i, :)';
for j = 1:3
end
op = [op; answ'];
end
(above) and

plot(op(:, 1), op(:, 2), 'r')
Image2');
disp(final_P);
end
x_m = mean2(x_col);
y_m = mean2(y_row);
N = [
scale
0
-scale*x_m
0
scale -scale*y_m
0
0
1
];
norm_points = N*p;
end
function [x_col, y_row, N] = dumb(x_col, y_row)
N = eye(3);
end
function q3a()
H = dlt(im1, im2);
disp(H);
end
function new_P = dlt(im1, im2)
%Normalizing
world = [];
end
A = [];
temp = [
];
A = [A; temp];
end
%Doing SVD
[U, D, V] = svd(A);
%Projection matrix is the last col of V since D is least for
last col
P = V(:, 9);
new_P = [P(1:3)'; P(4:6)'; P(7:9)'];
op = [];
x_col = x1;
y_row = y1;
world = [];
temp = [x2(i) y2(i) 1];
end
answ = new_P*world(i, :)';
for j = 1:3
end
op = [op; answ'];
end
(above) and
plot(op(:, 1), op(:, 2), 'g')
Image2');
end
x_m = mean2(x_col);
y_m = mean2(y_row);
N = [
scale
0
-scale*x_m
0
scale -scale*y_m
0
0
1
];
norm_points = N*p;
end
function q3b()
world = [];
temp = [x2(i) y2(i) 1];
end
old_world = world;
old_x = x1;
old_y = y1;
%Normalizing
world = [];
end
no_pt = 5;
total = size(x_col, 1);
iter = 50;
min_err = 9999999;
final_P = zeros(3, 3);
for k = 1:iter
idx = randperm(total, no_pt);
A = [];
for i = 1:no_pt
temp = [
];
A = [A; temp];
end
%Doing SVD
[U, D, V] = svd(A);
%Projection matrix is the last col of V since D is least
for last col
P = V(:, 9);
new_P = [P(1:3)'; P(4:6)'; P(7:9)'];
err = 0;
%Finding error
for i = 1:size(old_world, 1)
answ = new_P*old_world(i, :)';
for j = 1:3
end
err_x = abs(answ(1) - old_x(i));
err_y = abs(answ(2) - old_y(i));
err = err + err_x + err_y;
end
if err < min_err
final_P = new_P;
min_err = err;
end
end
op = [];
x_col = old_x;
y_row = old_y;
world = old_world;
answ = final_P*world(i, :)';
for j = 1:3
end
op = [op; answ'];
end
(above) and
plot(op(:, 1), op(:, 2), 'g')
Image2');
disp(final_P);
end
x_m = mean2(x_col);
y_m = mean2(y_row);
N = [
scale
0
-scale*x_m
0
scale -scale*y_m
0
0
1
];
norm_points = N*p;
end

First, we create the final image, which has the same number of rows as the anchor image and the number of columns is
equal to the sum of the anchor and the second image. The first half of the final image has the anchor image. Next, a
Homography is calculated. An inverse mapping procedure is followed here in which we iterate through each pixel of the
final image and see what value should be present there. We take an inverse of the Homography matrix and find what
pixel value should be present in that location. For multiple images, we loop through the above procedure, stitching 2
images at a time.
OBSERVATIONS
1. While computing the inverse mapping, we can get values that are out of bounds. We need to take care of these cases
Code
clear; clc
im1 = imread('plane-1.JPG');
sizearray = [size(im1,1),size(im2,1)];
imf = zeros(size(im2,1),(size(im1,2)+size(im2,2)),3);
%imf(:) = 255;
figure; imshow(im1)
[x y] = getpts;
pts1 = [x y];
figure; imshow(im2)
[x y] = getpts;
pts2 = [x y];
Homo1 = homo(pts2,pts1)
for i=1:size(im1,1)
for j=1:size(im1,2)
imf(i,j,:) = im1(i,j,:);
end
end
for i=1:size(imf,1)
for j=1:size(imf,2)
pnew = inv(Homo1)*[j,i,1]';
pnewx = pnew(1)/pnew(3);
pnewy = pnew(2)/pnew(3);
if(pnewx >= 1 && pnewx <= size(im2,2) && pnewy >= 1 &&
pnewy <= size(im2,1))
imf(i,j,:) = im2(floor(pnewy),floor(pnewx),:);
end
end
end
figure,imshow(uint8(imf))
im_final = zeros(size(im1,1),(size(imf,2)+size(im3,2)),3);
imf = uint8(imf);
imshow(imf)
[x y] = getpts;
pts1 = [x y];
imshow(im3)
[x y] = getpts;
pts2 = [x y];
Homo2 = homo(pts2,pts1);
for i=1:size(imf,1)
for j=1:size(imf,2)
im_final(i,j,:) = imf(i,j,:);
end
end
for i=1:size(im_final,1)
for j=1:size(im_final,2)
pnew = inv(Homo2)*[j,i,1]';
pnewx = pnew(1)/pnew(3);
pnewy = pnew(2)/pnew(3);
if(pnewx >= 1 && pnewx <= size(im3,2) && pnewy >= 1 &&
pnewy <= size(im3,1))
im_final(i,j,:) = im3(floor(pnewy),floor(pnewx),:);
end
end
end
function [ H ] = homo( pts1,pts2 )
%Computes 2D-2D homography between two images given a set of
points for
%matching points in 2 images
%
mean_pt = mean(pts1);
tx = [1 0 -mean_pt(1);0 1 -mean_pt(2);0 0 1];
mean_pt_re = repmat(mean_pt,size(pts1,1),1);
diff = pts1 - mean_pt_re;
dist = sqrt(sum(diff.^2,2));
mean_dist = mean(dist);
sx = [sqrt(2)/mean_dist 0 0;0 sqrt(2)/mean_dist 0;0 0 1];
T1 = sx*tx;
buff = ones(size(pts1,1),1);
pts1_homo = [pts1,buff];
pts1_norm = T1*pts1_homo';
%% FOR PTS1 END %%
%% FOR PTS2 %%
mean_pt = mean(pts2);
tx = [1 0 -mean_pt(1);0 1 -mean_pt(2);0 0 1];
mean_pt_re = repmat(mean_pt,size(pts2,1),1);
diff = pts1 - mean_pt_re;
dist = sqrt(sum(diff.^2,2));
mean_dist = mean(dist);
sx = [sqrt(2)/mean_dist 0 0;0 sqrt(2)/mean_dist 0;0 0 1];

T2 = sx*tx;
buff = ones(size(pts1,1),1);
pts2_homo = [pts2,buff];
pts2_norm = T2*pts2_homo';
%% FOR PTS2 END %%
A = zeros(2*size(pts1,1),9);
count = 0;
for i=1:2:size(A,1)
count = count + 1;
A(i,:) = [0 0 0 -pts2_norm(3,count)*pts1_norm(1,count) ...
-pts2_norm(3,count)*pts1_norm(2,count) pts2_norm(3,count)*pts1_norm(3,count)...
pts2_norm(2,count)*pts1_norm(1,count)
pts2_norm(2,count)*pts1_norm(2,count)...
pts2_norm(2,count)*pts1_norm(3,count)];
A(i+1,:) = [pts2_norm(3,count)*pts1_norm(1,count)
pts2_norm(3,count)*pts1_norm(2,count)...
pts2_norm(3,count)*pts1_norm(3,count) 0 0 0 pts2_norm(1,count)*pts1_norm(1,count) ...
-pts2_norm(1,count)*pts1_norm(2,count) pts2_norm(1,count)*pts1_norm(3,count)];
end
[U S V] = svd(A);
P_col = V(:,end);
P = [ P_col(1:3)'; P_col(4:6)'; P_col(7:9)'];
H = inv(T2)*P*T1;
end
Question 6. PROCEDURE
I took 2 of images of distinct planar name boards from different orientations. For each of the image with a perspective
distortion, I corrected it by estimating the homography between these images and a fronto-parallel view by manually
specifying the corners of a rectangular region in the image. For estimating the homography I used the DLT Procedure
mentioned above and then using a double nested for loop I applied the homography to all points of the original image,
in order to construct the fronto-parallel view.
Observations
While performing the mapping, some of the points will map to outside the original images i.e., have negative coordinate values or values larger than the size of the image. Those points are ignored. There exists no mapping to certain
points of the final image. They are the black regions along the corners.
Code
I1 = (imread('mr2.jpg'));
I2 = (imread('mr2.jpg'));
I2 = 0*I2;
% For mr2
x2=[187,90,1;49,404,1;302,158,1;150,480,1];
x1=[1,1,1;1,596,1;391,1,1;391,596,1];
% For mr1
%x2 = [53 35 1;326 45 1;13 780 1;253 780 1];
%x1 = [1 1 1; 300 1 1; 1 790 1; 300 780 1];
[x1,t1]= normalize_image(x1);
[x2,t2]= normalize_image(x2);
A=[];
for i = 1:size(x1,1)
temp=[[x1(i,:) [0 0 0] -1*x2(i,1)*x1(i,:)];[[0 0 0] x1(i,:)
-1*x2(i,2)*x1(i,:)]];
A = [A;temp];
end
[u,D,v]=svd(A);
final=[v(1,9) v(2,9) v(3,9);v(4,9) v(5,9) v(6,9);v(7,9) v(8,9)
v(9,9)];
final= inv(t1)*final*t2;
I2 = compute_mapping(I1,I2,final,size(I1,1),size(I1,2));
figure; imshow(I2);
function [x_final, N] = normalize_image(x)
x_col = x(:,1);
y_row = x(:,2);
x_m = mean2(x_col);
y_m = mean2(y_row);
N = [
scale
0
-scale*x_m
0
scale -scale*y_m
0
0
1
];
norm_points = N*p;
x_final = [x_col y_row ones(size(x_col,1),1)];
end
function [I2] = compute_mapping(I1,I2,final,size1,size2)
for i = 1:size1
for j= 1:size2
npoint=final*[i;j;1];
npoint(1,1)= npoint(1,1)./npoint(3,1);
npoint(2,1)= npoint(2,1)./npoint(3,1);
npoint(3,1)= 1;
if(npoint(1,1)<=size1 && npoint(1,1)>= 1 &&
npoint(2,1)<=size2 && npoint(2,1)>= 1)
I2(i,j,:) =
I1(int32(npoint(1,1)),int32(npoint(2,1)),:);
end
end
end
end
Question 7. PROCEDURE
I took a set of sport images and logos. For each of the logo images I estimated the homography by DLT by manually
calculating and using the coordinates of its current 4 corner points and the 4 corner coordinates in the sport image
where I want the logo to appear. Then using a doubly nested for loop, I I applied the homography to a select points in
the Sports image, while iterating over all points in the logo image.
OBSERVATIONS
In order to ensure that we dont lose any details in either image while mapping one onto another, we can take an
average at the overlapping points. In this case, since we are performing a mapping onto a specific set of co-ordinates, we
shouldnt get a resultant co-ordinates as out of bounds.
Code
clear all; clc;
I1=imread('logo2.jpg');
I2=imread('sport3.jpg');
% For logo.jpg
x1 = [1 1 1; 1 150 1; 250 1 1; 250 150 1];
%For logo2.jpg
%x1 = [ 1 1 1; 1 74 1; 97 1 1; 97 74 1];
%For sport3.jpg
x2 =
[110 1965 1 ;110 2075 1; 360 1965 1;360 2075 1];
%For sport1.jpg
%x2 = [284 487 1; 284 637 1; 534 487 1; 534 637 1];
%For sport2.jpg
%x2 = [34 197 1; 34 271 1; 131 197 1; 131 271 1];
A = [];
for i = 1:4
temp=[[x1(i,:) [0 0 0] -1*x2(i,1)*x1(i,:)];[[0 0 0] x1(i,:)
-1*x2(i,2)*x1(i,:)]];
A=[A;temp];
end
[u,D,v]=svd(A);
final=[v(1,9) v(2,9) v(3,9);v(4,9) v(5,9) v(6,9);v(7,9) v(8,9)
v(9,9)];
for i = 1:size(I1,1)
for j= 1:size(I1,2)
npoint=final*[i;j;1];
npoint(1,1)=npoint(1,1)./npoint(3,1);
npoint(2,1)=npoint(2,1)./npoint(3,1);
npoint(3,1)=1;
if(npoint(1,1)<= size(I2,1) && npoint(1,1)>= 1 &&
npoint(2,1)<=size(I2,2) && npoint(2,1)>= 1)
I2(int32(npoint(1,1)),int32(npoint(2,1)),:) =
I1(i,j,:);
end
end
end
figure;imshow(I2);

Assignment2 Report

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Assignment2 Report

Uploaded by

Copyright:

Available Formats

CV Assignment 2

Question 1&2. PROCEDURE

Error per point during DLT

Question 1&3. PROCEDURE

Error per point during DLT

Input and Output Images

Homography computation using DLT and SIFT

Homography computation using RANSAC and SIFT

Homography computation using DLT and Manual Point Selection

Homography computation using RANSAC and Manual Point Selection

%Writing the 'A' matrix in AxP = [0]

%overlay on image1 (below).

temp = [x2(i) y2(i) 1];

figure, imagesc(im1), colormap(gray), hold on

Question 4&5. PROCEDURE

Input and Output Images

sx = [sqrt(2)/mean_dist 0 0;0 sqrt(2)/mean_dist 0;0 0 1];

Input and Output Images

Input and Output Images

[110 1965 1 ;110 2075 1; 360 1965 1;360 2075 1];

You might also like