Image Processing:

Low-level Feature Extraction

CMPUT 412
University of Alberta, Computing Science
Winter 2007
Csaba Szepesvari
 Summary
• Low level feature detection
• Edge detection
– Differencing
– 1st order: Roberts, Prewitt, Sobel, Canny
– 2nd order: LoG, Marr-Hildreth
– Others
• Curvature
– differencing
– continuous curve approximation
 What is “low level”?
• No shapes; “higher
spatial” information
– no spatial integration
– local processing
• Examples:
– edges (lines)
– corners (=high
curvature points of
lines)
 Edge detection
• (almost) insensitive to illumination changes
• highlights image contrast
• what is an edge?
– position of “step change” in contrast
• approaches:
– first order differentiation
– second order ..
– other..
Edge detection by finite differences
• Change in intensity:
– Difference adjacent pixels
• Horizontally:
–
– detects vertical edges
• Vertically:
–
– detects horizontal edges
• Combined:
Edge detection by finite differences

uniform thresholding
=> binary edge images
 Symmetric differences
• Taylor series:
First-order edge detection

Roberts (1965) cross-operator

Roberts cross operator

Alternative:
edge direction
edge strength
Prewitt edge detection operator
• Goal: reduce sensitivity to noise
• Method: Averaging

Prewitt (1966)
Results for the Prewitt op.
 Sobel edge detector
• Better smoothing:
– Replace “rectangle” with “Gauss” window
• Smoothing:

• Differencing:

Sobel (1970)
 Sobel cont’d
• Sobel’s operators
– 0. Select a (rect) neighborhood of a point
– 1. smooth image along axis x
– 2. difference image along axis y
• Can reverse the role of x and y
Sobel template examples
Sobel: result on Lena
 Sobel: vectorial results

edge direction
normal to edges
 Canny edge detection
• Goals:
– 1: Low sensitivity to noise
– 2: good localization (small bias in edge position)
– 3: eliminate multiple responses to a single edge
• Methods:
– 1: Gaussian smoothing
– 2,3: Thinning ~ non-maximum supression
– 1: Hystheresis thresholding

Canny (1986)
Canny’s operator
Implementation
Non-maximum supression
Hysteresis thresholding

implemented using recursion (branching, seeds..)

Hysteresis vs. uniform thresholding

Purpose: Select points of (b) and also

those points of (c) which are connected to points of (b)
 Canny in action

Goals:
1: Low sensitivity to noise
2: good localization (small bias in edge position)
3: eliminate multiple responses to a single edge
Second-order edge detection

“zero crossing”
 Laplacian operator

Horizontal
Horizontal + vertical
 Laplacian applied to square

Notes:
-isotropic (rotation invariant)
-sum of template coeffs must be zero (no response on flat areas)
-no smoothing -> sensitive to noise
-no directional information!
-fast
 Laplacian of Gaussian (LoG)
• Reduce noise first by Gaussian smoothing
• Apply Laplacian operator

“Mexican hat”
 Marr-Hildreth operator
• How to detect zero-crossings?

• Compute averages in the 4 quadrants

• If there is a sign difference, there must be
a zero crossing at the center

Marr (1980)
 Marr-Hildreth in action

Notes:
-provides closed edge borders
-fast
-multiscale analysis possible (reconstruction!)
-relationship to human visual system
 Spacek & Petrou operators
• Spacek: yet another “optimal” noise (1986)

smoothing filter (step-edge model):

• Petrou: step-edge replaced by ramps (1991)

Canny vs. Spacek vs. Petrou
 Edge detection in large noise

Binary images obtained by hysteresis thresholding; manually tuned thresholds

 Detecting image curvature
• Curvature:
Rate of change in edge direction
• “Corners”
Rate of change is large
• Straight lines
Rate of change is small
• Corners:
– Important information for recognition/matching
– Huge reduction of data
 Curvature basics
• Parametric 2D curve

• Tangent vector function:

• Speed & direction:

 Curvature
• Curvature: change of direction with
respect to arc length (s):

• Curves parameterized by arc length

maintain a constant speed:
Curvature is change in direction for
constant displacements along the
curve
• If is the gradient direction, then
 Curvature
• Chain rule:

“Curvature function”
 Curvature by differencing

gradient directions of a sequence

of pixels in a curve

smoothing:
Curvature by differencing
Curvature by differencing
 Approximation to a continuous curve

• Fit cubic polynomials to small segments

• point of interest at t=0:

• curvature:

• LMS fit:
Tsai (1994),
Lee (1993)
Continuous curve approx. in action

• Problems:
– Tracing curves is cumbersome (noise)
– Remedy(?): robust fitting, e.g. circles
 Kass (1988) “Snakes” ~ active contours

Curvature by changes in intensity

• Gradient:
• Local approximation of edge at (x,y):

• Curvature:

• Putting it together:
 Alternatives
• Start with to get
Curvature by changes in intensity
 Curvature by autocorrelation
• Moravec’s corner detection

min{ Euv(x,y): (u,v) =(1,0),(1,1),(0,1),(-1,-1)}

• Problem: (u,v) is rather limited
• Extension to any direction =>
Harris (1988)
 Harris detector
• Taylor series expansion of image:

• (u,v) are real-valued (e.g. within circle)

• Previous measure:
Harris detector
 Harris detector: (u,v)=?
• Displace window in the direction of the
edge:
 Harris detector
• Euv(x,y) is quadratic in (u,v):

• Rotate coordinate system:

• , are proportional to the autocorrelation

function along the principal axes =>
Curvature by autocorrelation
SIFT