University of Louisville Dr. Aly A.

Farag
Electrical and Computer Engineering Spring 2008

ECE 619/645 Computer Vision

Lab # 3 Geometric Stereo Reconstruction - Part 2: Reconstruction
(Issued 2/28 Due 4/10)

A) Matching (Correspondence)

1- Area (correlation)-based Stereo

In correlation based methods the elements to be matched are image windows of fixed size, and
a similarity criterion is used to measure the correlation between corresponding windows in the stereo
pair. The corresponding element is chosen when the similarity criterion is maximized within a search
region. For input stereo pair of images Il and Ir , select a WH window and for each point pl in the left
image find its corresponding point pr in the right image by computing the disparity d=[dx, dy]T that
maximizes one of the following similarity measures (where m= (W-1)/2, n = (H-1)/2 ):

1- The Normalized Sum of Squared Difference (NSSD)

n m

(I
i= n j = m
l ( x + i , y + j ) I r ( x + i + d x , y + j + d y )) 2
C1 ( x , y , d ) =
n m m m

I l ( x + i, y + j ) 2
i= n j = m
I
i= m j = m
r (x + i + d x , y + j + d y )2

2- Normalized Cross Correlation (NCC)

n m

I ( x + i, y + j ) I
l r ( x + i + d , y + j)
C 2 ( x, y , d ) = i = n i = m
n m m m

I l ( x + i, y + j ) 2
i = n j = m
I
i= m j = m
r (x + i + d x , y + j + d y )2

3- The NSSD with subtracted mean intensity:

([I ( x + i, y + j ) I ( x, y)] [I ]
n m

l l r (x + i + d x , y + j + d y ) Il (x + d x , y + d y ) )2
i= n j = m
C 3 ( x, y , d ) =
[I ( x + i, y + j ) I ( x, y)] [I ]
n m m m
2 2
l l r (x + i + d x , y + j + d y ) Il (x + d x , y + d y )
i = n j = m i = m j = m

4- The (NCC) with subtracted mean intensity defined as:

1
University of Louisville Dr. Aly A. Farag
Electrical and Computer Engineering Spring 2008

[I ( x + i, y + j ) I ( x, y)][I ]
n m

l l r (x + i + d x , y + j + d y ) Il (x + d x , y + d y )
i = n j = m
C 4 ( x, y , d ) =
[I ( x + i, y + j ) I ( x, y)] [I ]
n m m m
2 2
l l r (x + i + d x , y + j + d y ) Il (x + d x , y + d y )
i = n j = m i = m j = m

where I l ( x, y ) and I r ( x, y ) are the left and right mean of intensity values within a correlation window
centered at (x,y), respectively. For the following stereo pair of images and its corresponding depth map
(http://cat.middlebury.edu/stereo/data.html)

Set #1

Set #2

Set #3

2
University of Louisville Dr. Aly A. Farag
Electrical and Computer Engineering Spring 2008

Set #4

1. Use the preceding similarity measures to find the depth map images. Compare with the given
ground truth depth maps. You may use the root mean square error as a quality measure.

2. For all the above, include in your report the run time taken by the algorithm.

B) 3-D Reconstruction using stereo

Given corresponded pair of point (xl,yl) and (xr,yr) and assuming calibrated image (M1 and M2 are
known), the following set of equations can be written to solve for the unknown 3-D point (X, Y, Z):

X ( xl m31 m11 ) + Y ( xl m32 m12 ) + Z (m33 xl m13 ) = m14 xl m34

X ( y l m31 m21 ) + Y ( y l m32 m22 ) + Z (m33 yl m23 ) = m24 y l m34

m11
X ( x r m31 m12
) + Y ( x r m32 ) + Z (m33
x r m13
) = m14
x r m34

X ( y r m31 m22
m21 ) + Y ( y r m32 ) + Z (m33
y r m23
) = m24 y r m34

These equations can be arranged in the form Cl=b

xl m31 m11 xl m32 m12 m33 xl m13 m14 xl m34

y m m X
l 31 y l m32 m22 m33 y l
m23 m24 y l m34
21
Y =
m11
x r m31 m12
x r m32 xr
m33 m14
m13 x r m34

Z
m21
y l m31 m22
y l m32 yl
m33
m23 y r m34
m24

l can be found as:

l = (C T C ) C T b
1

1- Find the calibration matrices of the left and right cameras using the images Calib_left, and
Calib_right.

3
University of Louisville Dr. Aly A. Farag
Electrical and Computer Engineering Spring 2008

2- Use this reconstruction method to reconstruct the calibration pattern.

3- Using the CardEye system at the CVIP lab (or any stereo system that can capture a stereo
pair), generate an image the birdhouse object by the three cameras. From these images that
could be named I_cam1 I_cam2, and I_cam3, reconstruct the shape of the birdhouse.

References:

1. E. Trucco and A. Verri, Introductory Techniques for 3-D Computer Vision, Prentice Hall Inc.,
1998.

2. http://cat.middlebury.edu/stereo/data.html

Acknowledgements: This Laboratory evolved over various iterations of offering of ECE

619(ECE645). Special thanks are due to Dr. Moumen Ahmed and Dr. Ahmed Eid who helped in
putting this document together during 2002-2005. Their PhD work is readily available on the CVIP
Lab website www.cvip.uofl.edu. The Laboratory was also implemented on the CardEye vision system
by Asem Ali and was maintained throughout 2005-2008.