02 CameraGeometry PDF

Zoltan Kato: Computer Vision
Image Formation
Light (Energy) Source
Surface
2. Camera Geometry
Imaging Plane
Camera:
Spec &
Pose
3D
Scene
Pinhole Lens
Computer Vision
World
Optics
Sensor
Signal
2D
Image
Zoltan Kato
http://www.inf.u-szeged.hu/~kato/
Slide adopted from Zhigang Zhu Computer Vision - CSC I6716
Pinhole Camera Model
Focal Length, Field Of View

Consider case with object on the optical axis:
Image Plane
Image plane
Optical Axis
ff
z
viewpoint
f
Pinhole lens
Out of view
Pin-hole is the basis for most graphics and vision
Derived from physical construction of early cameras

Mathematics is very straightforward
3D World projected to 2D Image

Image inverted, size reduced
Image is a 2D plane: No direct depth information
Perspective projection
f called the focal length of the lens
Optical axis: the direction of imaging

Image plane: a plane perpendicular to the optical axis
Center of Projection (pinhole), focal point, viewpoint, nodal point
Focal length: distance from focal point to the image plane
FOV : Field of View viewing angles in horizontal and vertical directions
Increasing f will enlarge figures, but decrease FOV
Equivalent Geometry
Perspective Projection
Consider case with object on the optical axis:
Compute the image coordinates of p in terms of the camera

coordinates of P.
y
p(x, y)
P( X,Y,Z )
More convenient with upright image:

z
Z
Z=f
Origin of camera at center of projection

Z axis along optical axis
Image Plane at Z = f; x || X and y||Y
Projection plane z = f
Equivalent mathematically
Pinhole camera model
Reverse Projection
Given a center of projection and image coordinates of a
point, it is not possible to recover the 3D depth of the
point from a single image.
P(X,Y,Z) can be anywhere along this line
p(x,y)
X f

Y a
Z f

X
Z
Y
Z
X
fX f
Y
Z a fY =
Z
1

X
0
Y
0
Z
1 0
1
All points on this line

have image coordinates (x,y).
In general, at least two images of the same point taken
from two different locations are required to recover depth.
7
Stereo Geometry
Amsterdam : what do you see in this picture?
Object point
line
Kosara,, robert@kosara.net
Photo by Robert Kosara
http://www.kosara.net/gallery/pinholeamsterdam/pic01.html
straight
size
parallelism/angle
Central
Projection
Rays
shape
shape
of planes
Vergence Angle
depth
Depth obtained by triangulation

Correspondence problem: pl and pr must
correspond to the left and right projections of P,
respectively.
Pinhole camera image
size
parallelism/angle
shape
shape
of planes
depth
Amsterdam
Amsterdam
9straight line
10
11
9straight line
size
parallelism/angle
shape
shape
of planes
depth
P(X,Y,Z)
12
Amsterdam
size
parallelism/angle
shape
shape
of planes
depth
13
9straight line
9straight line
Amsterdam
size
parallelism/angle
shape
shape
of planes
depth
parallelism/angle
shape
shape
9
of planes
parallel to image
depth
9straight line
size
parallelism/angle
shape
shape
9
of planes
parallel to image
Depth
stereo
motion
size
structure
15
size
Amsterdam: what do you see?

Amsterdam
9straight line
14
- We see spatial shapes rather than individual pixels

- Knowledge: top-down vision belongs to human
- Stereo &Motion most successful in 3D CV & application
- You can see it but you don't know how
16
Camera Parameters
Y
yim
Image
frame
Pose / Camera
(xim,yim)
xim
CCD
Frame coordinates (xim, yim) pixels

Image coordinates (x,y)
Camera coordinates (X,Y,Z)
World coordinates (Xw,Yw,Zw)
X
Object / World
Zw
Xw
Yw
p (x,y,f)
(0,0)
Size:
(sx,sy)
Pixel
(xim,yim)
oy
xim
X
From frame to image Y fX + Zox fs x
Image center
Z a fY + Zo y =

Directions of axes

Z

1
Pixel size

Pw
Intrinsic Parameters (of the camera and the frame grabber):

link the frame coordinates of an image point with its
corresponding camera coordinates
Extrinsic parameters: define the location and orientation of
the camera coordinate system with respect to the world
coordinate system
ox
fs y
oy
1
Intrinsic Parameters
X
0
Y
0
Z
0
1
(ox,oy): principal point (image center)

(sx,sy): effective size of the pixel
f: focal length
17
18
Intrinsic Parameters (2)

k1 , k2
ox
yim
x
Z
Camera Parameters
(x, y)
Coordinate Systems
Intrinsic Parameters (1)
Camera rotation and translation

X
(xim,yim)
x = xd (1 + k1r 2 + k 2 r 4 )
xim
x
Z
y = yd (1 + k1r 2 + k 2 r 4 )
From World to Camera
Lens Distortions
P = R Pw + T
Modeled as simple radial distortions
Extrinsic Parameters
r2 = xd2+yd2
(xd , yd) distorted points
k1 , k2: distortion coefficients
T
P
Zw
Pw
Xw
Yw
A 3D translation vector, T, describing the relative locations of

the origins of the two coordinate systems
A 3x3 rotation matrix, R, an orthogonal matrix that brings the
corresponding axes of the two systems onto each other
A model with k2 =0 is still accurate for a CCD sensor of

500x500 with ~5 pixels distortion on the outer boundary
yim
(xd, yd)
R 1 = RT , i.e. RRT = RT R = I
19
20
Extrinsic Parameters: [R|T]
Finite projective camera
~
~ ~
X cam = R X - C
x = K[I | 0]X cam
~
x = KR I | C X
X
~
R RC Y R RC
X cam =
=
X
Z
0
1
0
1
1

~
P = K[R | T ]
x = PX
T = RC
fx
K =
s
fy
[ ]
px
p y
1
arctan(1/s)
1
~
P = KR I | -C 11 degree of
~
C Projection
freedom (5+3+3)
non-singular
center in world
coordinate
frame.
s skew
for CCD/CMOS, always s=0
decompose P in K,R,C?
~
P = [M | p 4 ] [K, R ] = RQ(M ) C = M 1p 4
~
C Projection center in world coordinate frame.
{finite cameras}={P4x3 | det M0}
If rank P=3, but rank M<3, then camera is at infinity
21
Camera matrix decomposition
Column vectors
Finding the camera center
PC = 0
Algebraically, C may
be obtained as:
22
0
1
[p 2 ] = [p1p 2 p3p 4 ]
0

0
(use SVD to find null-space)
X = det ([p 2 , p 3 , p 4 ]) Y = det ([p1 , p 3 , p 4 ])
Z = det([p1 , p 2 , p 4 ]) W = det ([p1 , p 2 , p 3 ])
Finding the camera orientation and internal parameters
M = KR
Image points corresponding to X,Y,Z

directions and origin (p4)
(use RQ decomposition ~QR)

(if only QR, invert)
=(
R )-1= R -1 Q -1
23
24
Row vectors
Affine cameras
1T X
x p
y = p 2 T Y
T Z
0 p 3
1
1T X
0 p
y = p 2 T Y
T Z
w p 3
1
note: p1,p2 dependent on image

reparametrization
25
26
Weak Perspective Projection
Weak perspective projection
Average depth Z is much larger than the relative distance

between any two scene points measured along the optical axis
x
P =
y
Y
p(x, y)
P(X,Y,Z)
1T
t1 (7 degrees of freedom)
r
r 2T t
2
1 0 1 / k
X
0
Z=f
A sequence of two transformations
Orthographic projection : parallel rays
Isotropic scaling : f/Z
Linear Model
Preserve angles and shapes
f
X
Z
f
y= Y
Z
x=
27
28
Slides adopted from:
Affine camera
x
PA =
s
y
1T
t1
r
r1T t 2
1 0 1 / k
m11 m12
PA = m21 m22
0
0
m13 t1
m23 t 2 (8dof)
0 1
CS 395/495-25: Spring 2004
IBMR:
1 0 0 0
PA = [3 3 affine]0 1 0 0[4 4 affine]
0 0 0 1
R3R2 and P3P2:

The Projective Camera Matrix
1. Affine camera=camera with principal plane coinciding with

the plan at infinity
2. Affine camera maps parallel lines to parallel lines
3. No center of projection, but direction of projection PAD=0
(point on )
Jack Tumblin
jet@cs.northwestern.edu
29
Cameras Revisited
Plenty of Terminology:
Image Plane or Focal Plane
Focal Distance f
Camera Center C
z
Principal Point
Principal Axis
Principal Plane
Camera Coords (xc,yc,zc)
Image Coords (x,y)
Cameras Revisited
yc
Focal Distance f
Camera Center C
z
Principal Point
Principal Axis
Principal Plane
Image Coords (x,y)
yc
xc
xc
Cameras Revisited
Focal Distance f
Camera Center C
z
Principal Point
Principal Axis
Principal Plane
Image Coords (x,y)
Cameras Revisited
yc
yc
xc
Cameras Revisited
Focal Distance f
Camera Center C
z
Principal Point p
Principal Axis
Principal Plane
Image Coords (x,y)
Focal Distance f
Camera Center C
z
Principal Point p
Principal Axis
Principal Plane
Image Coords (x,y)
xc
Cameras Revisited
yc
xc
Focal Distance f
Camera Center C
z
Principal Point p
Principal Axis
Principal Plane
Image Coords (x,y)
yc
xc
Cameras Revisited
Focal Distance f
Camera Center C
zc
Principal Point p
Principal Axis
Principal Plane
Image Coords (x,y)
Cameras Revisited
yc
C =[0,0,0]
xc
Focal Distance f
Camera Center C
z
Principal Point p
Principal Axis
Principal Plane
Image Coords (x,y)
Goal: Formalize projective 3D2D mapping

Homogeneous coords handles infinities well:
xf s px 0 xc
0 yf py 0 yc
0
0 1 0 zc
1
(3x3 submatrix)
submatrix)
x
y
z
P0 X = x
y
zc
yc
y
p
f
xc
[K 0] = P0
y
z
xc
Basic Camera P0 is a 3x4 matrix:
Projective cameras ( convergent eye rays)

Affine cameras (parallel eye rays)
Composed, controlled as matrix product
(Much Better: P3P2)
Basic Camera P0: P3P2 (or camera R3)
Cameras Revisited
Recall Euclidian R3R2 :

x = f x / z
y = f y / z
yc
C
f
Non-square pixels? change scaling (x, y)

Parallelogram pixels? set nonzero skew s
K matrix: (internal) camera calib. matrix
Complete Camera Matrix P
The Pieces of Camera Matrix P
K matrix: internal camera calib. matrix

RT matrix: external camera calib. matrix
PX =
T matrix: Translate world to cam. origin

R matrix: 3D rotate world to fit cam. axes
Output: x
2D Camera Image
yc
C
xc
zc
(world space)

PX =
xw
x, or yzw
w
tw
xc
yc
zc
P
P = P2T
3T
P
1T
P=
p1 p2 p3 p4
f
xc
p1,p2,p3 == image of x,y,z axis vanishing points

Direction D = [1 0 0 0]T = point on P3s x1 axis, at inifinity
PD = 1st column of P = P1. Repeat for y and z axes.
p4 == image of the world-space origin pt.
yc
Proof: let D = [0 0 0 1]T = direction to world origin

PD = 4th column of P = image of origin pt.
zc
f C
xc
Rows of P matrix: camera planes in world space

row 1 = P1T = image x-axis plane
row 2 = P2T = image y-axis plane
row 1 = P3T = cameras principal plane
zc
xc
yc
zc
Columns of P matrix = image of world-space axes:
(c a m
er a s
p ace
)
Combine: write
(P0RT)X = x
as
PX = x
Input: X
3D World Space
x
w
x, or yzw
w
tw
PX =
yc
x, or
xw
yw
zw
tw
xc
yc
zc
P
P = P2T
3T
P
1T
Rows of P matrix: planes in world space

row 1 = P1 = world plane whose image is x=0
row 2 = P2 = world plane whose image is y=0
row 3 = P3 = cameras principal plane
Why? Recall that in P3,
point X is on plane
if and only if TX = 0, so
zc
yc
f
xc

PX =
x
w
x, or yzw
w
tw
xc
yc
zc
P
P = P2T
3T
P
1T

row 1 = P1 = world plane whose image is x=0
row 2 = P2 = world plane whose image is y=0
zc

PX =
yc
x
w
x, or yzw
w
tw
xc
yc
zc
P
P = P2T
3T
P
1T

row 1 = P1 = images x=0 plane
row 2 = P2 = images y=0 plane
Careful! Shifting the
image origin by px, py
zc
p
shifts the x=0,y=0 planes!

PX =
xc
yc
zc
P
P = P2T
3T
P
1T
PX =
yc
f
xc

row 1 = P1 = images x=0 plane
row 2 = P2 = images y=0 plane
zc
yc
xc
xc
xw
x, or yzw
w
tw
x, or
xw
yw
zw
tw
xc
yc
zc
P
P = P2T
3T
P
1T

row 1 = P1 = image x-axis plane
row 2 = P2 = image y-axis plane
princip. plane P3= [p31 p32 p33 p34]T

its normal direction: [p31 p32 p33 0 ]T
Why is it normal? Its the world-space
P3 direction of the zc axis
Principal
plane P3
yc
zc
p
f
xc

PX =
x
w
x, or yzw
w
tw
xc
yc
zc
m1T
P = m2T p 4
m3T
PX =
M
Principal Axis Vector (zc) in world space:
Normal of principal plane: = m3 = [p31 p32 p33 ]T
P3 Scaling Ambiguous direction!! +/- m3 ?
Solution: use det(M)m3 as front of camera
yc
Principal Point p in image space:

image of (infinity point on zc axis = m3)
Mm3 = p = x0
zc
p
x0
(Zisserman book renames p as x0)
xc
x
w
x, or yzw
w
tw
m1T
P = m2T p 4
m3T
~
M
xc
yc
zc
Where is camera in world space? at C:

Camera center C is at camera origin(xc,yc,zc)=0=C
~
Camera P transforms world-space point C to C=0
~
But how do we find C ? It is the Null Space of P :
~
P C = C = 0 (solve for ~C. SVD works, but here
heres an easier way:)
yc
~
-M-1p4
Cameras Position in the World: C =
zc
p f
xc

02 CameraGeometry PDF

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

02 CameraGeometry PDF

Uploaded by

Copyright:

Available Formats

Zoltan Kato: Computer Vision

Zoltan Kato: Computer Vision

Zoltan Kato: Computer Vision

Pinhole Camera Model

Focal Length, Field Of View

Pin-hole is the basis for most graphics and vision

Derived from physical construction of early cameras

3D World projected to 2D Image

Optical axis: the direction of imaging

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Zoltan Kato: Computer Vision

Zoltan Kato: Computer Vision

Consider case with object on the optical axis:

Compute the image coordinates of p in terms of the camera

More convenient with upright image:

Origin of camera at center of projection

Zoltan Kato: Computer Vision

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Zoltan Kato: Computer Vision

Pinhole camera model

All points on this line

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Zoltan Kato: Computer Vision

Zoltan Kato: Computer Vision

Amsterdam : what do you see in this picture?

Depth obtained by triangulation

Zoltan Kato: Computer Vision

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Zoltan Kato: Computer Vision

Pinhole camera image

Pinhole camera image

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Pinhole camera image

Zoltan Kato: Computer Vision

Zoltan Kato: Computer Vision

Pinhole camera image

Pinhole camera image

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Zoltan Kato: Computer Vision

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Zoltan Kato: Computer Vision

Pinhole camera image

Pinhole camera image

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Amsterdam: what do you see?

- We see spatial shapes rather than individual pixels

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Zoltan Kato: Computer Vision

Zoltan Kato: Computer Vision

Frame coordinates (xim, yim) pixels

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Intrinsic Parameters (of the camera and the frame grabber):

(ox,oy): principal point (image center)

Zoltan Kato: Computer Vision

Slide adopted from Zhigang Zhu Computer Vision - CSC I6716

Zoltan Kato: Computer Vision

Intrinsic Parameters (2)

Intrinsic Parameters (1)

Camera rotation and translation

From World to Camera

Modeled as simple radial distortions