Computer Vision Fundamentals: Assignment 3 Assigned: Tuesday 2nd October 2012 Due: Friday 12th October 2012

Instructor: Sohaib Khan TA: Aamer Zaheer, Shabbir Marzban and Anzak Aleem

All additional tasks for MS students can be attempted by BS students for extra credit. Total points 220 (with extra credit)
BASIC PROBLEMS (80 points) 1. Given a 3D rotation matrix, describe at least two ways of visualizing (i.e. an intuitive understanding) of the transformation. 2. Construct a rotation matrix that rotates the 3D space about n = (0.89, 0.45, 0.59) by 0.785 radians. Represent this rotation matrix in the form of a rotation about X-axis followed by a rotation about Z-axis, followed by a rotation about Y-axis. What are these rotation matrices and angles? 3. Find out the axis of rotation and angle of rotation for the following P3 rotation matrix:

0.3276 0.8638 0.3827 0.5850 0.1326 0.8001 0.7419 0.4860 0.4619 0 0 0

0 0 0 1

4. A) Construct a rotation matrix that rotates the 3D space about n1 = (0.111, -0.0095, 0.9938) by 0.5 radians followed by a rotation about n2 = (0.5689, -0.6396, -0.5170) by 1.5 radians. B) Consider the rotation matrix resulting from part A. What are its axis and angle of rotation? 5. Do parallel planes remain parallel under P3 Rigid transformation (i.e. rotation plus translation), Scaling (may be different in each direction; in addition to rigid part) and Affine transformation? Do orthogonal planes remain orthogonal under these transformations? Prove! (Separately for the three transformation types). 6. Prove that under the planarity assumption for the world, the image seen by two orthographic cameras is related by the affine transformation. 7. A camera is located such that its center is at (10,7,15) it is looking towards (6,9,8) (optical axis passes through this point). Additionally, its up vector (a unit vector pointing in the direction of world Y-axis) lies on the yz-plane of the camera and makes an acute angle with the y-axis of the camera. Compute the rotation and translation that will align camera axes (xyz) to the world axes (XYZ) and camera center to the world origin. Also set up the camera matrix for identity K matrix. Note 1: See the figure (1) on the right. It is NOT drawn to scale / alignment Note 2: In the figure, the camera y-axis, z-axis and Up vector lie on the same plane (i.e. the yz plane). Figure 1 8. A camera is located such that its gimbal is at (0, 2, 2). The offset of the gimbal to the center of the image plane is (0.02, 0.01, 0.03). The camera has been rotated (with gimbal as center of rotation) about X-axis by 135 followed by rotation about Z-axis by 90. The focal length of the camera is 0.03. Assume an ideal pinhole camera. Find the image coordinates of the point that has world coordinates (1, 1, 0.2).

9. If an object in the world undergoes rigid motion, will the area of the object remain the same in the image under orthographic projection? Explain your answer. 10. A camera is mounted in a vehicle. The initial position of the vehicle such that the camera is at the world origin, with positive X-axis going out of the left side of the vehicle, positive Y-axis pointed upwards and positive Z-axis in front of the vehicle, as shown. The Z-axis of the world is aligned with the North of the compass. When the vehicle turns, it can be assumed that it turns about its center of mass, which is m above the ground. The gimble vector, from the point of rotation of the camera to the image center is negligible. The location of the camera is [0, 1, 0] (in meters) with respect to the center of mass of the vehicle. The camera is oriented such that it is vertically upright; it is looking at a point on the road directly in front of the vehicle and is tilted downwards at an angle of 30. The vehicle travels to a location that has coordinates 200m along X-axis and 100m along Z-axis along a level road. At that moment, the vehicle is headed NW. If the focal length of the camera is 2cm, compute the camera model for this scenario. (You may write the expression in terms of product of matrices). 11. [For MS Students] Show that for a pinhole camera, three collinear points in the 3-D space are mapped to three collinear points on the image plane. 12. [For MS Students] Show that every rotation matrix (regardless of the dimensionality of the space) is necessarily an orthogonal matrix. Moreover, show that every rotation matrix must also have a determinant of 1.

DESIGN PROBLEMS (60 points)

Figure (a)

Figure (b)

13. A parking garage owner wishes to protect against false damage claims by customers, and has decided to invest in a camera system to take the pictures of every car as it enters and exits the garage. The pictures will be archived for future evidence. If an owner later claims that the car was scratched or damaged while parked, the owner should be able to verify or refute that claim by looking at the archived pictures. He has hired you for help. Figure (a) shows the arrangement of one camera that will be used to cover the entire right side of the vehicle. Figure (b) shows the same arrangement from the top. (The figures are NOT drawn to scale). When the front wheels of the vehicle pass over the sensor, the camera takes the image. At that moment, the optical axis of the camera is looking approximately at the center of the vehicle. The owner has provided the following estimates: The vehicle can be assumed to be of 2m width traveling anywhere in a lane which is 4m wide. The camera is mounted such that its center of lens is 2m from the edge of the lane. The typical length of a vehicle that enters the garage is 4m. The requirement of the owner is that for every vehicle that passes over the sensor, the image should be of such a high quality that a scratch of 2mm thickness should be clearly visible in the image. The image should also contain the whole vehicle in all cases. You may assume that for a feature to be clearly visible, it must map to at least one pixel in the image. a. What is the minimum horizontal resolution of the camera that is needed? That is, what is the minimum number of pixels that the camera must have in the horizontal direction to meet the requirement specifications? b. The camera that is available has adjustable focal length. It has enough number of pixels that you need, as computed in part (a), fitted in a CCD array of width 40mm. What focal length can you set to meet the requirement specifications? 14. For security at an airport, you are asked as a consultant to design a system which can be installed at the entrance of the airport and will take the complete image of the under-side of a vehicle. The cameras which have a horizontal and vertical field of view of 75 will be installed on the road looking upwards. They will take the video of an incoming car at 30 frames per second. The video will later be mosaiced together to form a single image of the under-side of a vehicle. The speed of the vehicle passing over the camera can be up to 10 km/hr. The height of the vehicle above the camera depends on the type of the vehicle, and can vary from 5 cm to 20 cm. Each image that the camera captures is 1000x1000 pixels.

a. For mosaicing to work well, it is a requirement that there be at least a 50% overlap of content between two consecutive frames. Given the above values, determine whether this constraint of 50% overlap is always met in the above scenario or not. You may assume that the shutter speed is high enough to not generate any motion blur. b. Which parameters may you change in the design of the system to increase the amount of overlap between successive frames? Discuss the effect of each parameter. c. Provide a few different design solutions which will meet the 50% overlap specification, and show the validity of those solutions.

15. In cricket coverage on TV, advertisements are frequently shown painted on the cricket ground. The ads are painted such that from a particular camera viewpoint, they appear as vertical billboards, even though they are really horizontal on the ground plane. An example of such an ad is shown in the figure on the right. We assume that the camera is fixed and cannot move, nor can its focal length be changed. a. What must be the shape of the ad when viewed from vertically above? b. What type of transformation exists between the standard ad and the transformed ad? c. Given that the camera parameters are known, clearly outline a process to determine the coordinates of the ad to be drawn on the ground.

MATLAB Problems (80 points) 16. [For MS Students] Camera Calibration: In this problem, you are required to calibrate a camera. You may use the cameras in the vision lab if needed or the images provided with this assignment. The goal is to calibrate a camera, and then do an experiment to verify the accuracy of the calibration parameters. You may not use any existing calibration toolbox; instead you have to write your own code. You have to submit a written document explaining your approach and results. You may be required to give a demo after the homework is due. Hint: Construct an arbitrary camera matrix (containing all intrinsic and extrinsic parameters) and project some 3D points into an image. Now you have 3D to 2D correspondences needed for camera calibration, use them to recover the camera matrix (K[R|t]) and compare with the one originally used to take the image. Synthetic data is a commonly used technique in computer vision for algorithm verification. Hint: You can build a 3D cube (as shown) with known dimensions using the camera calibration pattern (available with this handout) and pasting it on a boxs faces. Just assume every square to be unit width and carefully define the 3D points assuming the world origin and axes aligned to a cube corner and lines. Take an image of this cube through any camera to form 2D to 3D correspondences (by manually marking the required points in the image). Make sure the 3D points lie on three faces of the cube! Hint: If you cannot calibrate a real camera, provide an analysis of your implementations robustness by adding noise to the correspondences. This will get you some partial credit.

This is a scaled image. The original is provided with the assignment.

17. [For MS Students (All parts)] A) Write a MATLAB script to demonstrate Projective Transformations in P3, like the one provided to you for transformations in P2. You can use a simple cube (i.e. 8 points and 12 line segments connecting the points) to demonstrate effects of different transformation parameters/matrices. You can use the following (optional) hints.

Hint: You can use guide in MATLAB to design a simple user interface ( with a viewing axis, text boxes for all
the parameters and a reset button). Hint: Read help for plot3, hold and view functions. Hint: You can build upon the P2 transformations demo such that both kinds of transformations are available (possibly through a drop-down menu?). B) With the help of above code, describe the actions of all the parameters i.e. translations, rotations, scales, shears as well as projective parameters (total 18 parameters) on the cube. C) Modify the code in part A, such that the points (and parts of lines) that are supposed to be occluded behind a face of the cube are not visible.