3D Computer Vision II

Reminder
Camera Models
Nassir Navab"
based on a course given at UNC by Marc Pollefeys & the book Multiple View Geometry by Hartley & Zisserman"

October 27, 2009"

Outline Camera Models"

Geometric Parameters of a Finite Camera"

Projective Camera Model"

Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"

Decomposition of Camera Matrix"

Cameras at Infinity"

Other Camera Models (Pushbroom and Line Cameras)"

3D Computer Vision II - Camera Models"

2"

Pinhole Camera Model"

Mapping between 3D world and 2D image"

Central projection"
Models are described in matrices with particular properties"

3D Computer Vision II - Camera Models"

3"

Homogeneous Coordinates"

3D Computer Vision II - Camera Models"

4"

Central Projection"

3D Computer Vision II - Camera Models"

5"

Principal Point Offset"

principal point
(perpendicular intersection point of
principal axis and image plane)

where
are the coordinates of the principal point
3D Computer Vision II - Camera Models"

6"

Principal Point Offset"

where

3D Computer Vision II - Camera Models"

is called camera calibration

matrix

7"

Camera Rotation and Translation"

Inhomogeneous
coordinates

where

represents the point in

world coordinates
represents the same point in
camera coordinates
represents the coordinates of the camera
origin in the world coordinate frame

3D Computer Vision II - Camera Models"

8"

Camera Rotation and Translation"

Homogeneous
coordinates

projection to image plane from camera coordinates

projection to image plane from world coordinates

3D Computer Vision II - Camera Models"

9"

Extrinsic and Intrinsic Parameters"

where
projection matrix of a general
pinhole camera with 9 DOF

intrinsic camera parameters with 3 DOF

extrinsic camera parameters with each 3 DOF
(camera orientation, position in world coordinates)
3D Computer Vision II - Camera Models"

10"

Camera Rotation and Translation"

No explicit
camera center

where
from

3D Computer Vision II - Camera Models"

11"

CCD Cameras Non-Square Pixels"

number of pixels per unit distance

4 DOF

10 DOF
3D Computer Vision II - Camera Models"

12"

Skew Parameter"
skew parameter

5 DOF

finite projective camera with

11 DOF

3D Computer Vision II - Camera Models"

13"

Finite Projective Camera Summary"

projection matrix
11 DOF (5+3+3)

non-singular

3D Computer Vision II - Camera Models"

14"

Finite Projective Camera Decomposition of P"

non-singular 3x3 matrix (8 DOF)

decompose projection matrix P in K,R,C

QR matrix
decomposition

3D Computer Vision II - Camera Models"

15"

Finite Projective Camera Summary"

where

Camera matrices P are identical with the set of homogeneous 3x4

matrices for which the left 3x3 sub-matrix is non-singular"

If rank(P) = 3, but rank(M) < 3, then camera at infinity"

if rank(P) < 3 the matrix mapping will be a line or a point and not a
plane (not a 2D image)

3D Computer Vision II - Camera Models"

16"

Outline Camera Models"

Geometric Parameters of a Finite Camera"

Projective Camera Model"

Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"

Decomposition of Camera Matrix"

Cameras at Infinity"

Other Camera Models (Pushbroom and Line Cameras)"

3D Computer Vision II - Camera Models"

17"

Camera Anatomy"

Camera center"
Column vectors"
Principal plane"
Axis plane"
Principal point"
Principal ray"

3D Computer Vision II - Camera Models"

18"

Camera Center"
P has a 1D null-space
we will prove that the 4-vector C is the camera center
points on a line through A and C
since
All 3D points on the line are mapped on the same 2D image point, and
thus the line is a ray through the camera center
Finite cameras:
Infinite cameras:
3D Computer Vision II - Camera Models"

19"

Column Vectors"

Column vectors are the image points which project the

axis directions (X,Y,Z) and the origin
Example for the image
of the y-axis
is the image of the world origin
3D Computer Vision II - Camera Models"

20"

Row Vectors"

Represent geometrically particular world planes.

row vectors

column vectors

3D Computer Vision II - Camera Models"

21"

Row Vectors of the Projection Matrix"

p1 is defined by the camera center and the
line x=0 on the image.
p2 is defined by the camera center and the
line y=0 on the image.

Example p2

respectively for p1
3D Computer Vision II - Camera Models"

22"

Principal Plane"
Plane through camera center and
parallel to the image plane.

points X are imaged on the line at

infinity
if X is on the principle plane
especially
3D Computer Vision II - Camera Models"

23"

Principal Point"
The line through camera center and
perpendicular to principal plane is the
principal axis.
The intersection of the principal
axis with the image plane is the
principal point.
normal direction to the principal plane
principal point
where
and
3D Computer Vision II - Camera Models"

third row of M

Principal Axis Vector"

Ambiguity that principal axis points
towards the front of the camera (positive
direction)

towards the front of the camera

direction unaffected by scaling
since

3D Computer Vision II - Camera Models"

25"

Forward Projection"
Maps a point in space on the image plane

Vanishing points

Only M affects the projection of vanishing points

3D Computer Vision II - Camera Models"

26"

Back-Projection to Rays"
Points on the reconstructed ray
camera center C

(pseudo-inverse)
Ray is the line formed by those two points
intersection of the ray with the
plane at infinity

3D Computer Vision II - Camera Models"

27"

Depth of Points"

w can be interpreted
as the dot product of
the ray CX with the
principal ray direction

(PC=0)
If

then m3 is a unit vector pointing in positive axis direction

Suppose

3D Computer Vision II - Camera Models"

. Then

28"

Depth of Points: Examples"

3D Computer Vision II - Camera Models"

29"

Outline Camera Models"

Geometric Parameters of a Finite Camera"

Projective Camera Model"

Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"

Decomposition of Camera Matrix"

Cameras at Infinity"

Other Camera Models (Pushbroom and Line Cameras)"

3D Computer Vision II - Camera Models"

30"

Camera Matrix Decomposition"

Finding the camera center C

numerically: find right null-space by SVD of P

Algebraically:

where
3D Computer Vision II - Camera Models"

31"

Camera Matrix Decomposition"

Finding the camera center C
Any plane going through C will be a linear combination of the three
planes defined by the rows of P. Therefore:

where
3D Computer Vision II - Camera Models"

32"

Camera Matrix Decomposition"

Finding the camera orientation and internal parameters

using RQ decomposition

Decompose

=(

) =

R -1

R -1 Q -1

Ambiguity removed by enforcing positive diagonal entries

3D Computer Vision II - Camera Models"

33"

When is Skew Non-zero?"

arctan(1/s)
1

for CCD/CMOS, always s=0

Image from image, s0 possible
(non coinciding principal axis)
resulting camera:
where H is a 3x3 homography
3D Computer Vision II - Camera Models"

34"

Euclidean vs. Projective Spaces"

General projective interpretation

Meaningful decomposition in K,R,t requires

Euclidean image and space
Camera center is still valid in projective space
Principal plane requires affine image and space
Principal ray requires affine image and Euclidean space
3D Computer Vision II - Camera Models"

35"

Outline Camera Models"

Geometric Parameters of a Finite Camera"

Projective Camera Model"

Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"

Decomposition of Camera Matrix"

Cameras at Infinity"

Other Camera Models (Pushbroom and Line Cameras)"

3D Computer Vision II - Camera Models"

36"

Cameras at Infinity"

Cameras with their center lying at infinity

M is singular

Two types of cameras at infinity:

Affine and non-affine cameras

3D Computer Vision II - Camera Models"

37"

Affine Cameras"

Definition:
An affine camera is a camera with a camera matrix P in which the
last row p3T is of the form (0,0,0,1)T .

Points at infinity are mapped to points at infinity

3D Computer Vision II - Camera Models"

38"

Affine Cameras"

3D Computer Vision II - Camera Models"

39"

Affine Cameras"

distance of the world origin from the camera center in

direction of the principal ray

modifying p34 corresponds to moving along principal ray

3D Computer Vision II - Camera Models"

40"

Affine Cameras"
Combine tracking back and zooming
magnification factor k=dt/d0
remains fixed

3D Computer Vision II - Camera Models"

41"

Error in Employing Affine Cameras"

point on plane parallel with principal plane and
through origin, then

general points (not on the parallel plane) with

distance from the plane

3D Computer Vision II - Camera Models"

42"

Error in Employing Affine Cameras"

3D Computer Vision II - Camera Models"

Affine Imaging Conditions"

Approximation should only cause small error

1. much smaller than d0
2. points close to principal ray (i.e. small field of view)

3D Computer Vision II - Camera Models"

44"

Decomposition of P "

absorb d0 in K2x2

alternatives, because 8dof (3+3+2), not more

3D Computer Vision II - Camera Models"

45"

Summary of Parallel Projections"

canonical representation

calibration matrix

principal point is not defined

3D Computer Vision II - Camera Models"

46"

Hierarchy of Affine Cameras"

dropping the z-coordinate

orthographic projection

(5dof)

3D Computer Vision II - Camera Models"

47"

Hierarchy of Affine Cameras"

scaled orthographic projection

(6dof)

3D Computer Vision II - Camera Models"

48"

Hierarchy of Affine Cameras"

weak perspective projection

(7dof)

3D Computer Vision II - Camera Models"

49"

Hierarchy of Affine Cameras"

Affine camera

(8dof)
full generality of an affine camera

Affine camera is a projective camera with principal plane at infinity

Affine camera maps parallel world lines to parallel image lines
No center of projection, but direction of projection PAD=0
3D Computer Vision II - Camera Models"

50"

General Camera at Infinity"

M is singular, but last row not zero

Camera center is on plane at infinity

Principal plane is not plane at infinity
Images of points at infinity are in general not mapped to infinity
on the image plane

3D Computer Vision II - Camera Models"

Outline Camera Models"

Geometric Parameters of a Finite Camera"

Projective Camera Model"

Camera Center, Principal Plane, Axis Plane, Principal Point, Principal Ray"

Decomposition of Camera Matrix"

Cameras at Infinity"

Other Camera Models (Pushbroom and Line Cameras)"

3D Computer Vision II - Camera Models"

52"

Pushbroom Cameras"

(11dof)

Straight lines are not mapped to straight lines!

(otherwise it would be a projective camera)
3D Computer Vision II - Camera Models"

53"

Line Cameras"
(5dof)

Null-space PC=0 yields camera center

Also decomposition

3D Computer Vision II - Camera Models"

54"

Summary Camera Models"

Photometric and radiometric properties of a camera"

Geometric parameters of a finite camera"
Projective cameras"
Camera anatomy (camera center, principle plane, principle point, and principle axis)"
Camera matrix decomposition (camera center, orientation, and intrinsic parameter"

Cameras at infinity"
Affine cameras"
Non-affine cameras"

Alternative models (pushbroom cameras, line cameras)"

3D Computer Vision II - Camera Models"

55"

Literature on Camera Models"

Chapter 6 in R. Hartley and A. Zisserman, Multiple View Geometry, 2nd

edition, Cambridge University Press, 2003. "
Chapter 3 in O. Faugeras, Three-dimensional Computer Vision, MIT Press,
1993."
Chapter 2 in E. Trucco and A. Verri, Introductory Techniques for 3-D
Computer Vision, Prentice Hall, 1998."
H. Gernsheim, The Origins of Photography, Thames and Hudson, 1982."
A. Shashua. Geometry and Photometry in 3D Visual Recognition, Ph.D.
Thesis, MIT, Nov. 1992. AITR-1401. "

3D Computer Vision II - Camera Models"

56"