The future - images everywhere

Digital Picture Frames

PDA/Camera hybrids

Mobile hybrids
 Image Enhancement
To process an image so that the result is
more suitable than the original image for a
specific application.
Improvement of image quality
* Image quality depends on context &
application
*Improvement depends on quality
measure
 Image Enhancement
Involves image denoising & image deblurring

Denoising is the process of reducing image

noise by image smoothing or filtering, while
deblurring

Deblurring is achieved by image sharpening

or inverse filtering.
 Refers to the accentuation or sharpening of image features such as
edges, boundaries, or contrast to make an image more useful for
display and analysis.
Image Enhancement
 Image Enhancement

Aerial image Transformations with

c=1 and = 3, 4 and 5
 From system point of view
f(x,y) g(x,y)
T[f(x,y)]
Point processing

f(x,y) g(x,y)
T[N{f(x,y)}]
Mask processing or
Spatial filter
 Different approaches
Spatial domain
Point processing
Mask processing
(neighborhood processing) (spatial
filters)
Frequency domain
Frequency domain filters
9
 Spatial Domain Enhancement
Procedures that operate directly on the
aggregate of pixels composing an image

g ( x, y ) T [ f ( x, y )]
A neighborhood about (x,y) is defined by using
a square (or rectangular) subimage area
centered at (x,y).
 Spatial Domain Methods

When the neighborhood is 1 x 1 then g depends

only on the value of f at (x,y) and T becomes a
gray-level transformation (or mapping)
function:
s=T(r)
r : gray levels of f(x,y)
s : gray levels of g(x,y) at (x,y)
T : transformation operator
Point processing techniques (e.g. contrast stretching,
thresholding)
 Enhancement by
Point Processing

These are methods based only on the intensity

of single pixels.
r denotes the pixel intensity before processing.

s denotes the pixel intensity after processing.

 Point processing

Simple gray level transformations

Image negatives
Log transformations
Power-law transformations
Contrast stretching
Gray-level slicing
Bit-plane slicing
Histogram processing
Histogram equalization
Histogram matching (specification)
Arithmetic/logic operations
Image averaging

13
 Point Operations

Mapping a given gray level u [0, L] into a gray

level v [0, L] according to v = f(u)
Contrast Stretching
For low-contrast images that occur because of:
Poor or non-uniform lighting
Non-linearity or small dynamic range of sensors
Point Operations

Point operations affect the way images occupy greyscale.

A point operation transforms an input image, producing
an output image in which each pixel grey level is
related in a systematic way to that of the
corresponding input pixel.

Output
grey level

Stretch
Unchanged
A point operation will never
alter the spatial relationships
within an image.
Compress Input
grey level
Gray level transformation
 Image Enhancement in
the
Spatial Domain

Contrast Thresholding
Stretching
Some transformations

s = T(r) = L 1 r
s T r c log1 r
s = cr
s T r mr b
255 if A r B
s T r
0 otherwise

21
Examples of Point Operations

Output Output Output

t Input t1 t2 Input Input

(a) Threshold (b) Window Threshold (c) Contrast Stretch

Output Output Output

Input Input Input

(d) Contrast Compression (e ) Combination (f) Contouring
 Contrast Stretching
Contrast stretching (autoscaling) involves altering the
distribution and range of pixel values
reassignment of pixel values is based on the particular
stretch algorithm
range for each band, is calculated and displayed as a
histogram- Unimodal Gaussian/Multimodal distributions
Upper and lower limits of brightness values typically lie
within only a part (30 to 60%) of the total available range.
narrower limits are stretched to undergo expansion to the
full scale 0-255
Special stretches: Piecewise Linear, Linear with
Saturation, Logarithmic, Exponential, Ramp Cumulative
Distribution Function, Probability Distribution Function,
Sinusoidal Linear with Saturation
 Contrast Stretching

Transformation function
Low contrast image

Result of contrast stretching Result of thresholding

 Contrast stretching

s T r mr b

25
Window-Center adjustment
(monitors dynamic range)
Output gray level

256

0
0 input gray level 60000
(high dynamic range image)
 Image Thresholding
Thresholding means choosing threshold level between the
maximum and minimum intensity of the initial image.
Thresholding is a way to get rid of the effect of noise and to
improve the signal-noise ratio.

30% threshold
lena.gif 10% threshold
 Histogram of raw TM values shows a narrow distribution peak at low
pixel values. We might predict from this that we have a dark, flat image.
This is indeed the case:
Most of the DN values, however, lie between 9 and 65
perform a simple linear stretch: 9 goes to 5 and 65 goes to 255

Most of scene features are discriminable. But the image still is rather dark.
choose new limits in which we take pixels between 5 and 45 and expand
these to 0 to 255. This stretch results in the following histogram and image
 Algebraic operations
A point form of operation (with >1 input image)
Grey level of each output pixel depends only on
grey levels of corresponding pixels in input images
Four major operations:

Addition: Multiplication:

C A B C A B
Subtraction Division:

C A B C A B
Other operations can be defined that involve more than 2
input images, or using (for example) boolean or logic
operators.
 Applications of algebraic operations

Addition
Ensemble averaging to reduce noise
Superimposing one image upon another

Subtraction
Removal of unwanted additive interference
(background suppression)
Motion detection
 Applications (continued)

Multiplication
Removal of unwanted multiplicative interference
(background suppression)
Masking prior to combination by addition
Windowing prior to Fourier transformation

Division
Background suppression (as multiplication)
Special imaging signals (multi-spectral work)
 Image Negatives

Are obtained by using the transformation

function s=T(r).

[0,L-1] the range of gray levels

S= L-1-r
 Image Negatives

Function reverses the order from black to white

so that the intensity of the output image
decreases as the intensity of the input
increases.

Used mainly in medical images and to produce

slides of the screen.
Image Enhancement in the
Spatial Domain

s = T(r) = L 1 r
 Log Transformations

s = c log(1+r)
c: constant

Compresses the dynamic range of images with

large variations in pixel values
Log transformation (Dynamic
range compression)

s T r c log1 r

38
Image Enhancement in Spatial Domain
 Power-Law Transformations

s cr
C, : positive constants

Gamma correction

Basic Point Processing Power- law transformations

Basic gray level transformation Plots of transformation formula

functions for image enhancement s=c 2 for different values of
Power-Law transformation

s = cr

42
 Gray-Level Slicing
To highlight a specific range of gray levels in
an image (e.g. to enhance certain features).

One way is to
display a high value
for all gray levels in
the range of interest
and a low value for
all other gray levels
(binary image).
 Gray-Level Slicing
Second approach is to brighten the
desired range of gray levels but preserve
the background and gray-level tonalities in
the input image
 Bit-Plane Slicing
To highlight the contribution made to the total
image appearance by specific bits.

i.e. Assuming that each pixel is represented

by 8 bits, the image is composed of 8 1-bit
planes.

Plane 0 contains the least significant bit and

plane 7 contains the most significant bit.
 More on bit planes:

Only the higher order bits (top four) contain

visually significant data.

The other bit planes contribute the more subtle

details.

Plane 7 corresponds exactly with an image

thresholded at gray level 128.
50
Bit-Slicing
 Spatial Filtering
Mask processing or Spatial filtering :
the values of f in a predefined neighborhood of a
pixel (x,y) determine the value of g at (x,y).

Through the use of MASK

(or kernels, templates, or windows, or filters).
 Spatial Filtering
Spatial frequency: the number of cycles of change in
image DN values per unit distance (e.g., 10 cycles/mm)
along a particular direction in the image.
An image with only one spatial frequency consists of
equally spaced stripes (raster lines).
Ex:- a blank TV screen has horizontal stripes. This
corresponds to zero frequency in the horizontal direction
and a high spatial frequency in the vertical.
Fine detail in an image involves a larger number of
changes per unit distance than the gross image features.
Convolution filtering is a common mathematical method
of implementing spatial filters.
each pixel value is replaced by the average over a square
area centered on that pixel.
this tends to reduce deviations from local averages and
thus smoothes the image. i.e. low pass filtering
 Filters are MASKS/Templates
What is to be done?
Finite difference filters - intuitively, most pixels in
respond strongly to images look quite a lot
noise like their neighbours
- along the edge theyre
obvious reason: image
similar, across the edge
noise results in pixels theyre not
that look very different - smoothing the image
from their neighbours should help, by forcing
Generally, the larger the pixels different to their
noise the stronger the neighbours (=noise
response pixels?) to look more like
neighbours
 Spatial filtering
Use spatial filters (masks) for linear and
nonlinear image enhancement
How to use mask?
1D (the mask is 3 2 1)

2D

z1 z2 z3 f1 f2 f3
z4 z5 z6 f4 f5 f6
g1 z5 f1 z6 f 2 z8 f 4 z9 f 5
z7 z8 z9 f7 f8 f9 9
g 5 zi f i
i 1
mask image
 Spatial filters
Purpose: Purpose
Blur or noise reduction Highlight fine detail or
Lowpass/Smoothing spatial enhance detail blurred
filtering Highpass/Sharpening spatial
Sum of the mask filter
coefficients is 1 Sum of the mask
Visual effect: reduced noise coefficients is 0
but blurred edge as well Visual effect: enhanced
Smoothing linear filters edges on a dark background
Averaging filter High-boost filtering and
Weighted average (e.g., unsharp masking
Gaussian) Derivative filters
Smoothing nonlinear filters 1st
Order statistics filters (e.g., 2nd
median filter)

58
 Smoothing filters 1 1 1
1/9 x 1 1 1
Purpose:
Blur or noise reduction 1 1 1
Smoothing linear filtering (lowpass spatial filter)
Neighborhood (weighted) averaging 1 2 1
Can use different size of masks 1/16 2 4 2
Sum of the mask coefficients is 1
Drawback: 1 2 1
Order-statistic nonlinear filters
Response is determined by ordering the pixels contained in the image
area covered by the mask
Median filtering
The gray level of each pixel is replaced by the median of its neighbor.
Good at denoising (salt-and-pepper noise/impulse noise)
Max filter
Min filter
Averaging
filter

Median
filter
Image Enhancement in Spatial Domain
 2D filtering

A 2D image f[i,j] can be filtered by a 2D kernel h[u,v] to

produce an output image g[i,j]:

This is called a cross-correlation operation and written:

h is called the filter, kernel, or mask.

 2D Convolution
A convolution operation is a cross-correlation where the filter is
flipped both horizontally and vertically before being applied to the
image:

It is written:

Suppose h is a Gaussian or mean kernel.

How does convolution differ from cross-correlation?
 Properties of Masks
Smoothing Masks
Derivative Masks
- All values are +ve
opposite signs
- Sum to 1 Sum to zero
- Output on smooth Output on smooth
region is unchanged region is zero
- Blurs areas of high Gives high output in
contrast areas of high contrast
Larger mask -> more
- Larger mask -> more
edges detected
smoothing
 Filtering Masks

Mean Filter

Gaussian Filter
Image Enhancement in the
Spatial Domain
 Example1: Averaging

One simple example is smoothing using a 3x3

mask.
f(x-1,y-1) f(x-1,y) f(x-1,y+1)
1/9 1/9 1/9

f(x,y-1) f(x,y) f(x,y+1)

1/9 1/9 1/9

f(x+1,y-1) f(x+1,y) f(x+1,y+1)

1/9 1/9 1/9

1
g ( x, y ) f ( x 1, y 1) f ( x 1, y) f ( x 1, y 1) ... f ( x 1, y 1)
9
Two 3x3 smoothing filter masks
 mean filter product, which Sobel edge enhancement filter
tends to generalize image highlights abrupt discontinuities,
such as rock joints and faults, field
boundaries, and street patterns:

High pass filter image with streets,

highways, streams and ridges,
greatly emphasized
 Noise
Filtering is useful for
noise reduction...
Common types of noise:
Salt and pepper noise:
random occurrences of
black and white pixels
Impulse noise: random
occurrences of white
pixels
Gaussian noise:
variations in intensity
drawn from a Gaussian
normal distribution
 Practical noise reduction

How can we smooth away noise

in a single image?

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 100 130 110 120 110 0 0
0 0 0 110 90 100 90 100 0 0
0 0 0 130 100 90 130 110 0 0
0 0 0 120 100 130 110 120 0 0
0 0 0 90 110 80 120 100 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
 Mean filtering

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20
0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30
0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30
0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30
0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20
0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10
0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Effect of
mean filters
Finite differences responding to noise

Increasingnoise>
(thisiszeromeanadditivegaussiannoise)
Low pass spatial filter (Linear):
Used for smoothing (removal of small details prior to
large object extraction, bridging small gaps in lines)
and noise reduction.

Low-pass (smoothing) spatial filtering

Neighborhood averaging
Results in spatial blurring
Image Pyramid
Lowpass Images

Bandpass Images
Image Averaging

A noisy image:

g ( x, y ) f ( x, y ) n ( x, y )

Averaging M different noisy images:

M
1
g ( x, y )
M
g ( x, y )
i 1
i
 Image Averaging

As M increases, the variability of the pixel

values at each location decreases.

This means that g(x,y) approaches f(x,y)

as the number of noisy images used in
the averaging process increases.

Registering of the images is necessary to

avoid blurring in the output image.
 Image average (cont)

If the noise is uncorrelated and has zero

expectation, then

E g x, y f x, y

1 2
2
g x, y x, y
M
How to generate Gaussian noise?
//This function creates a gaussian random number between -3
and 3//
double gaussrand() {
static double V1, V2, S;
static int phase = 0;
double X;

if (phase == 0) {
do {
double U1 = (double)rand() / RAND_MAX;
double U2 = (double)rand() / RAND_MAX;
V1 = 2 * U1 - 1;
V2 = 2 * U2 - 1;
S = V1 * V1 + V2 * V2;
} while(S >= 1 || S == 0);
X = V1 * sqrt(-2 * log(S) / S);
}
else
X = V2 * sqrt(-2 * log(S) / S);
phase = 1 - phase;
return X;
}
a)Image of galaxy pair NGC 3314
b)Corrupted by zero mean AWGN
with std dev of 64 gray levels
c-f) Averagin with
K=8,16,64,128 noisy images
 Image Smoothing
Given image f( x, y) and smoothing filter h( i, j), image,
smoothing is defined by:
The intensity of pixel (x, y) in the output is obtained
from weighted sum of intensities of pixels at and
surrounding (x, y) in the input.

h( i, j) for different i and j shows the weights, whose

sum is 1.
 Mean Filtering

All weights h( i, j) are equal for mean filtering.

Input pixel Intensities surrounding (x, y) have the
same effect on the intensity at (x, y) in output.
This operation is not rotationally invariant if h is
nonzero in rectangular window but is rotationally
invariant smoothing, h is a circular window
 Smoothing Linear Filters

I J

w(i, j ) f (m i, n j )
i I j J
g ( m, n ) I J

w(i, j )
i I j J

Normalization of coefficient to
ensure 0 g(m,n) L-1
 Median filter (non-linear):
Replace the current
pixel value by the
median pixel value in
a given
neighborhood.

Achieves effective
noise supression.

Preserves the
sharpness of real
boundaries.
 Median Filtering
Output pixel value at (x, y) is obtained from the median of
values at and surrounding (x, y) in input.
It is rotationally invariant, if median of circular rather than
rectangular windows is used.
Median filtering is effective when impulse noise is present in
input image.
 Gaussian Filtering
If the weights are chosen as Gaussian coefficients in image smoothing, it is
known as Gaussian filtering.
Gaussian and mean filtering are effective when image noise has a mean of
zero.
Gaussian filtering is rotationally invariant.
A 2- D Gaussian can be decomposed into two 1- D Gaussians, and filtering
can be carried in 1- D rather than in 2- D:
G( x, y) = G( x) * G( y)
basic effects of (2D) Gaussian filter: smoothing the image and wiping off the
noise
A Gaussian kernel gives less weight to pixels further from the center of the
window

1 2 1
2 4 2
1 2 1
This kernel is an approximation of a
Gaussian function:
 GAUSSIAN MASKS
Gaussian mask is a circularly symmetrical (or isotropic) mask, such that
any cross-section through its centre yields a weight profile in the form of
a Gaussian or normal curve.

r (y-axis) --- the distance (in pixels) from the centre of the mask
(x-axis) --- the "width" of the mask

1-D Gaussian mask

2-D Gaussian mask

a digital mask like this is only an approximation to the mathematical

Gaussian because it has been sampled on a discrete grid and truncated
Smoothing Non noisy image

Noise Canceling
 Histogram

Gray-level histogram is a function showing, for

each gray level, the number of pixels in the
image that have that gray level.

Normalized histogram (probability):

nk hist[k ]

1
f x, y k

pk nk / N
93
The Grey-level Histogram

One of the simplest, yet most useful tools.

Can show up faulty settings in an image
digitiser.
Almost impossible to achieve without
digital hardware.

Number of
pixels

The GLH is a function

showing for each grey
level the number of
Grey level
pixels that have that grey
level.
Histogram - Examples

95
Example

97
 Histogram Equalization
Enhances visibility of gradients
When image histogram has a few peaks separated
by wide valleys, the variation of brightness values
of pixels that correspond to the peaks may not be
visible on the display
Equalization spreads the peaks out and makes the
differences between those pixels more evident.
Neighborhood equalization provides for local
contrast enhancement by modifying the brightness
of each pixel according to brightnesses of
neighboring pixels.
Histogram Equalization
HISTOGRAM METHODS
 Grey Level Histogram Equalisation
In GLH equalisation, a non-linear grey scale
transformation redistributes the grey levels,
producing an image with a flattened histogram.
This can result in a striking contrast
improvement.

Cumulative GL histogram Flattened histogram

Original image
 Histogram equalization

Why and when do we want to use HE?

102
HE Example 2
Original Dendrite image. Equalized Dendrite image.

Original Histogram. Equalized Histogram

Histogram Equalization
 Local Enhancement

Original Bone Marrow image.

Local Equalization.

Original Fingerprint image. Local Equalization

A dark image Histogram
After HE
Original Equalized
Not exactly flat due to the fact that were discretizing.
Histogram for Local Enhancement
HE Derivation (572)
T(r) is single-valued and monotonically
increasing within range of r
T(r) has the same range as r [0, 1]

nj
pr r j
k k
sk T rk
j 0 n j 0

r
s T r pr w dw
0

pr r pr r
ps s 1
ds pr r
dr
 Histogram equalization
Transformation function
s T r pr w dw
r
0 r 1
0

pr(w) is the probability density function (pdf)

The transformation function is the cumulative
distribution function (CDF)
To make the pdf of the transformed image
uniform, i.e. to make the histogram of the
transformed image uniform
 HE Discrete case
k n
sk T rk pr rj
k
j

j 0 n j 0

r hist(r) r s s

0 10 0 10 10 10
hist(s)
1 70 1 80 80 70

2 15 2 95 95 15

3 5 3 100 100 5
 *Histogram specification

Step1: Equalize the levels of the original image

Step2: Specify the desired pdf and obtain the
transformation function
Step3: Apply the inverse transformation
function to the levels obtained in step 1

120
 HS - Example

r hist(r) r s
0 10 0 10
1 70 1 80 r z
2 15 2 95 0 10
3 5 3 100 1 30
2 60
z hist(z) z G(z) 3 65
Specified 10 10 10 10
histogram 15 20 15 30
30 50 30 80
60 15 60 95
Comparison (HE vs. HM/HS)

125
 Image Subtraction
Mask mode radiography
h(x,y) is the mask

g ( x, y ) f ( x, y ) h ( x, y )
 Image Gradients
An image gradient consists of the horizontal
and vertical partial derivatives
I
Derivative Estimators x
Gx
I
-1
Gy I
Horz: -1 1 Vert: 1
y

Problems:
Derivatives enhance noise
We desire some filtering
We prefer odd sized filters, not even
Note relation to high-pass filtering
 Image gradient
The gradient of an image:

The gradient points in the direction of most rapid change in intensity

The gradient direction is given by:

how does this relate to the direction of the edge?

The edge strength is given by the gradient magnitude
 Better Derivative Estimators

Prewitt operator
-1 -1 -1
-1 0 1
Horz: 0 0 0
-1 0 1 Vert:
1 1 1
-1 0 1

Sobel operator

-1 0 1 -1 -2 -1
Horz: -2 0 2 Vert: 0 0 0
-1 0 1 1 2 1
Derivative Masks
Determining edge locations

We are interested in the magnitude of the

gradient at each pixel
2 2
mag (I ) Gx G y
For each pixel:
Determine magnitude based on gradient images
Test against a threshold
 Image Sharpening
This is the reverse of image smoothing. The objective is
to enhance image details.
It is achieved by the deblurring or inverse filtering
operation.
Given blurred image g and blurring filter h, the sharp
image (image before being blurred) is estimated from:

Find Fourier transform of blurred image, divide by Fourier transform

of filter point- by- point, and find inverse Fourier transform of result.
Note that the Fourier transform of filter should not contain any zeros.
 Image Sharpening
Edge Enhancement: first derivatives gives superior edge
detection
Gradient neighborhood operators like Sobel operators
produce an image with pixel brightness proportional to
maximum gradient of brightness in the original image

Boundary sharpening: First and Second Derivative filters,

Hough Transform techniques produce sharpened
boundaries

Edge Detection: boundary between two regions with an

abrupt change in intensity
Derivative Masks (Simple differences) with all
coefficients must add to zero, HPF, Hough Transform
 -1 -1 -1
Sharpening filters 1/9 x -1 8 -1
-1 -1 -1

Purpose
Highlight fine detail or enhance detail that has been blurred
Basic highpass spatial filter
Sum of the mask coefficients is 0
Visual effect: enhanced edges on a dark background
Unsharp masking and High-boost filtering
Derivatives
1st derivative
2nd derivative

135
 Example 2 - Sharpening filter

Examples from http://www.mediacy.com/index.aspx?page=AH_ForensicImageEnhancement200001

136
 Edge Enhancement
sharpening operator Laplacian is not as sensitive to
edges and lines as they are to points, including noise
use of first derivatives gives superior edge detection. But
must be used more than one since this is a directional
derivative

Gradient neighborhood operators such as the Sobel and

Kirsch operators produce an image with pixel brightness
proportional to maximum gradient of brightness in the
original image

The Sobel direction operator codes the angle in grey

scale
Kirsch operator uses 3x3 neighborhood to find first
derivative of brightness in each of eight directions and
keeps the largest value.
 Roberts Operator

Mark edge point only

No information about edge orientation
Work best with binary images
Primary disadvantage:
High sensitivity to noise
Few pixels are used to approximate the gradient
 Roberts Operator (Cont.)

First form of Roberts Operator

I (r , c) I (r 1, c 1) 2 I (r , c 1) I (r 1, c) 2
Second form of Roberts Operator
| I (r , c) I (r 1, c 1) | | I (r , c 1) I (r 1, c) |

1 0 0 1
h1 h2
0 1 1 0
 Sobel Operator

Looks for edges in both horizontal and vertical directions, then

combine the information into a single metric.
The masks are as follows:
1 0 1
1 2 1
x 2 0 2
y 0 0 0
1 0 1
1 2 1

y
1
Edge Magnitude = Edge Direction = tan
x2 y2 x
 Prewitt Operator

Similar to the Sobel, with different mask

coefficients:
1 0 1
1 1 1
x 1 0 1
y 0 0 0
1 0 1
1 1 1

Edge Magnitude(|G|) = Edge Direction = y

1
tan
x2 y2 x
 Laplacian Operators

Edge magnitude is approximated in digital

images by a convolution sum.
The sign of the result (+ or -) from two adjacent
pixels provide edge orientation and tells us
which side of edge brighter
 Why Laplacian?

Gaussian

delta function Laplacian of Gaussian

 Unsharp masking and high-boost
filters

Unsharp masking
To generate the mask: Subtract a blurred version of the image
from itself
Add the mask to the original

Highboost: gmask(x,y) = f(x,y) - f(x,y)

k>1

g(x,y) = f(x,y) + k*gmask(x,y)
Application: input image is very dark

144
 Example 1 - Unsharp Mask

Examples taken from http://www.2live4.com/photoshop-tips.asp 145

 Comparison

Unsharp masking (3x3, 9x9) laplacian sobel 146

Laplacian|Prewitt|Sobel|Roberts
Marr-Hildreth|Canny

Perfect image
Corrupted image
147
 Unsharp Masking
100
200
300 =
400
500

200 400 600 800

+ =
 Image gradient
The gradient of an image:

The gradient points in the direction of most rapid change in intensity

The gradient direction is given by:

how does this relate to the direction of the edge?

The edge strength is given by the gradient
magnitude
Effects of noise
Consider a single row or column of the image
Plotting intensity as a function of position gives a signal

How to compute a derivative?

Where is the edge?

 Solution: smooth first

Where is the edge? Look for peaks in

Derivative theorem of convolution

This saves us one operation:

 Laplacian of Gaussian
Consider

Laplacian of Gaussian
operator

Where is the edge?

Zero-crossings of bottom graph
 MATLAB demo
g = fspecial('gaussian',15,2);
imagesc(g)
surfl(g)
gclown = conv2(clown,g,'same');
imagesc(conv2(clown,[-1 1],'same'));
imagesc(conv2(gclown,[-1 1],'same'));
dx = conv2(g,[-1 1],'same');
imagesc(conv2(clown,dx,'same'));
lg = fspecial('log',15,2);
lclown = conv2(clown,lg,'same');
imagesc(lclown)
imagesc(clown + .2*lclown)
 The Laplacian - Masks

0 1 0 1 1 1

1 -4 1 1 -8 1
To recover the image:

0 1 0 1 1 1 g x , y f x, y 2 f x , y

0 -1 0 -1 -1 -1

-1 4 -1 -1 8 1 g x , y f x, y 2 f x, y
0 -1 0 -1 -1 -1

155
 Laplacian Operators
(Cont.)
Masks for 4 and 8 neighborhoods

0 1 0 1 1 1
1 4 1 1 8 1

0 1 0 1 1 1

Mask with stressed significance of the central pixel or

its neighborhood
1 2 1 2 1 2
2 4 2 1 4 1

1 2 1 2 1 2
First Derivative of Gaussian
 Laplacian of Gaussian Operator

Proposed by Marr/Hildreth
Gaussian Filter for noise elimination
Problem of finding maximum
Normalized magnitude peaks may be flat
Finds zero crossings of Laplacian of image
Does not require thresholding
Motivated by human vision system
 Sharpening Linear Filters
High Boost Filter Laplacian operator:

2
f ( x , y ) 2
f ( x, y )
2 f ( x, y )
x 2 y 2
f ( x 1, y) f ( x 1, y) f ( x, y 1) f ( x, y 1) 4 f ( x, y )

A1
- Derivative filter:
- Use derivatives to
approximate high pass
filters.
- Usually 2nd derivatives are
preferred.
- The most common one is
the Laplacian operator.
High-Boost Filtering
Generalization of unsharp masking
f hb x, y Af x, y f x, y
Here A is called boost coefficient, and
We rewrite the equation as
A 1
f hb x, y A 1 f x, y f x, y f x, y
A 1 f x, y f s x, y
Applicable to any sharpening operation
can be
The filter for fhb becomes
fs f f
2
0 1 0
h 1 A 4 1
0 1 0
 0 1 0
h 1 A 4 1
0 1 0
High-Boost Filtering: Demo
By varying A, better overall brightness can be improved.
original A=0

A=1.7, much
A=1 brighter
 Laplacian Filter for Image Enhancement
f ( x, y ) 2 f ( x, y ) center of mask 0;
g ( x, y )
f ( x, y ) f ( x, y ) center of mask 0.
2

a) Image of North pole of Moon

b) Laplacina Filtered Image
c) Laplacian scaled for display
d) Enhanced Image
Gradient filters
Roberts cross-gradient operator

Sobel operator
Original Saturn
Kirsch.
image. Sobel magnitude.

Sobel direction.
Nonlinear Filters
Will discuss some of them in more detail later for
the purpose of image restoration.
Maximum filter: the output the maximum value
under the mask
Minimum filter: the output the minimum value
under the mask
Rank-order filter:
Elements under the mask are ordered, and a particular value is
returned.
Both maximum and minimum filter are instances of rank-order
Another popular instance is median filter
Edge detection and editing
 2D edge detection filters

Laplacian of Gaussian

Gaussian derivative of Gaussian

is the Laplacian operator:

 Effects of noise

Consider a single row or column of the image

Plotting intensity as a function of position gives a signal

How to compute a derivative?

Where is the edge?

 Solution: smooth first

Where is the edge? Look for peaks in

Laplacian of Gaussian

Laplacian of Gaussian
operator

Where is the edge? Zero-crossings of bottom graph

 2D edge detection filters

Laplacian of Gaussian

Gaussian derivative of Gaussian

is the Laplacian operator:

 Try this in MATLAB
g=fspecial('gaussian',15,2);
imagesc(g);colormap(gray);
surfl(g)
gclown=conv2(clown,g,'same');
imagesc(conv2(clown,[11],'same'));
imagesc(conv2(gclown,[11],'same'));
dx=conv2(g,[11],'same');
imagesc(conv2(clown,dx,'same'));
lg=fspecial('log',15,2);
lclown=conv2(clown,lg,'same');
imagesc(lclown)
imagesc(clown+.2*lclown)
 Color Edge Enhancement
Range operator operates on the R, G, B planes to produce
color edges in the resulting image

image should first be converted to grey scale (Image ->

Mode -> Grey scale) and then the range operator applied.
HSI Range operator finds the two pixels in the neighborhood
which are farthest apart in color space, and then creates an
image in which the central pixel is assigned this difference
as a grey scale value (the image is still a 24 bit RGB image,
and must be converted to 8 bit greyscale)

Range operators include the top hat filter. This is a nonlinear

operator that finds the brightest or darkest pixels in two
different size neighborhoods, and then keeps the extreme
values
 Original image. Rank Range image HSI Range.

Original Muscle image. Top hat (inner rad.=1) Top hat (inner rad.=3)
 Boundary sharpening
Boundaries between regions are the visually most
important part of an image
Boundary types: steps in brightness, dark (or light)
lines that separate regions, subtle changes in color
(either hue or saturation), texture or orientation.
median does not shift boundaries, although it can
round corners
iterated median filter sharpens the local contrast at
blurred edges but does not alter the contrast, while
removing noise from the image.
Original Au_Resn image
Iterated median filtered image
Boundary Detection
Image Enhancement
in Frequency Domain
 Image Processing in Frequency Space
Frequency space has attractive advantages for image processing.
It makes large linear filtering operations (large neighborhood
kernels) much faster
collects information in many to separate signal from noise or
allow measurements that would be very difficult in the spatial
domain.
The common denominator in all of these operations is the FFT or
fast Fourier transform.
Forward FT to perform the conversion from the spatial domain to
the frequency domain. This will replace the original image with
the power spectrum
power spectrum image can be processed like any other grey scale
image
Fourier Transform
Original Mullite image and its FT Power Spectrum.
Understanding and implementing
Fourier transform
 Extension to 2D

in Matlab, check out: imagesc(log(abs(fftshift(fft2(im)))));

 Understanding and implementing
Fourier transform

According to translation

For one complete period,

f x, y 1 F u M / 2, v N / 2
x y

y
(0,0) y (0,0)

0 -255 v

-0 255 (0,0)

x x
u
f(x,y) f(x,y)(-1)x+y |F(u-M/2,v-N/2)|
 Filtering in Frequency Domain: Basic Steps

Basic Steps
1. Multiply pixel f(x,y) of
the input image by
(-1)x+y.
1. Compute F(u,v),
the DFT
3. G(u,v)=F(u,v)H(u,v)
4. g1(x,y)=F-1{G(u,v)}
5. g(x,y) = g1(x,y)*(-1)x+y
Low-pass & High-pass Filtering
Man-made Scene
Can change spectrum, then
reconstruct
Low and High Pass filtering
 The Convolution Theorem
The greatest thing since sliced (banana) bread!

The Fourier transform of the convolution of two

functions is the product of their Fourier
transforms
F[ g h] F[ g ] F[h]
The inverse Fourier transform of the product of
two Fourier transforms is the convolution of the
two inverse Fourier transforms
1 1 1
F [ gh] F [ g ] F [h]
Convolution in spatial domain is equivalent to
multiplication in frequency domain!
 2D convolution theorem
example
f(x,y) |F(sx,sy)|

h(x,y) |H(sx,sy)|

g(x,y) |G(sx,sy)|
Fourier Transform pairs
 Lowpass filtering

Ideal filter 1 if D u , v D0
H u , v
D(u, v): distance from point (u, v) to the origin
0 if D u , v D0
cutoff frequency (D0)
nonphysical
radially symmetric about the origin
Butterworth filter

Gaussian lowpass filter

1
H u, v
1 D u, v / D0
2n

H u, v e D 2 u ,v / 2 D0 2
Power ratio
99.9699.6599.0497.84

100

P u, v / PT u, v
u v
 Highpass filter

Ideal filter
0 if D u , v D0
H u, v
1 if D u , v D0
Butterworth filter

1
H u, v
1 D0 / D u , v
2n

Gaussian highpass filter

H u, v 1 e D 2 u ,v / 2 D0 2
Example
 Low-pass, Band-pass, High-
pass
low-pass:
filters

High-pass / band-pass:
Edges in images
 Notch Filter
0 uv0
F (u , v)
1 otherwise.

The frequency response

F(u,v) has a notch at
origin
(u = v = 0).
Effect: reduce mean
value.
After post-processing
where gray level is
scaled, the mean value
of the displayed image
is no longer 0.
Homomorphic
filtering
 Homomorphic filtering

A simple image model

f(x,y): the intensity is called the gray level for monochrome
image
f(x, y) = i(x, y).r(x, y)
0 < i(x, y) < inf, the illumination
0< r(x, y) < 1, the reflectance
 Homomorphic filter (cont)

f x, y i x, y r x, y
z x, y ln f x, y ln i x, y ln r x, y
F z x, y F ln i x, y F ln r x, y
Z (u , v) Fi (u , v) Fr (u , v)

S u , v H u , v Fi u , v H u , v Fr u , v

s x, y i x, y r x, y

g x, y exp s x, y exp i x, y exp r x, y

 Homomorphic filter (cont)

The illumination component

Slow spatial variations
Low frequency
The reflectance component
Vary abruptly, particularly at the junctions of dissimilar
objects
High frequency
Homomorphic filters
Affect low and high frequencies differently
Compress the low frequency dynamic range
Enhance the contrast in high frequency
 Homomorphic Filter (cont)

H 1 c ( D 2 ( u ,v ) / D02 )
H (u , v) ( H L )[1 e ] L
L 1
Homomorphic filter - example
 Point processing Mask processing Frequency domain
(spatial filters) filters

Simple gray level Smoothing filters (blur

transformations
Image negatives
Log transformations
Power-law
transformations
Contrast stretching
Gray-level slicing Sharpening filters
Bit-plane slicing (highlight details)
Histogram processing
Histogram equalization
*Histogram matching
(specification)
Arithmetic/logic
operations
Image averaging
Smoothing filters (blur
details) details)
Average, weighted
Ideal lowpass filter
average
Butterworth lowpass
Order statistics (e.g.
median) Gaussian lowpass
Sharpening filters (highlight
details)
Unsharp masking
High-boost filters
Unsharp masking
Derivative filters - The
High-boost filters Laplacian
Derivative filters Ideal highpass filter
The Laplacian Butterworth highpass filter
The Gradient Gaussian highpass filter
Homomorphic filtering
THANK
YOU