You are on page 1of 47

Pyramid Methods In

Image Processing
E. H. ADELSON |C. H. ANDERSON | J. R. BERGEN | P. J. BURT | J. M. OGDEN
RCA ENGINEER NOVEMBER,1984
Aashish
Nagpal
2012CS50271

Data Structures for image

Format used to represent image data can be as critical in image processing as


the algorithms applied to the data.

A digital Image can be encoded as:

array of pixel intensities

Raw format

Not suited for most tasks

Data Structures for image

represented by its Fourier transform

Operations applied to transform coefficients instead of pixel values

Good for compression and enhancement tasks but inappropriate for others

Data Structures for image

The retinal image is decomposed into several spatially oriented frequency


channels.

Need for representations that retain spatial localization as well as localization


in the spatialfrequency domain.

Can be achieved by decomposing the image into a set of spatial frequency


bandpass component images

Data Structures for image

Analysis procedure that is applied only at a single scale may miss information
at other scales.

Need to analyze image at different scales .

Computation cost of convolution increases in a multiresolution system as the


kernel size increases from small to large.

Image Pyramids

Consider a task of detecting a target pattern that may appear at any


scale in an image.

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Pyramids

1st Approach: Several copies of the pattern can be constructed at


increasing scales, then each is convolved with the image.

2nd Approach: a pattern of fixed size can be convolved with several


copies of the image represented at correspondingly reduced resolution

Both approaches yield equivalent results but second is more efficient

Image Pyramids

The image pyramid is a data structure designed to support efficient


scaled convolution through reduced image representation.

It consists of sequence of copies of an original image in both the


sample density and resolution are decreased in regular step.

nodes in each level represent information that is localized in both


space and spatial frequency.

Image Pyramids

Gaussian Pyramids

The reduced resolution levels of a pyramid are obtained through a


iterative algorithm.

One such iterative technique is using Gaussian :

Gaussian Pyramids

The first level is the original image.

It is lowpass-filtered and subsampled by a factor of two to obtain the


next pyramid level, G1

G1 is again filtered in the same way and subsampled to get G2.

It is equivalent to convolving the original image with a set of Gaussian


like weighting functions.

Gaussian Pyramids

The convolution acts as a lowpass filter with the band limit reduced with each
level.

The function double in width with each level.

Gaussian Pyramids

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Gaussian Pyramids

Expand function for Gaussian is given by:-

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Laplacian Pyramids

Bandpass rather than lower images are also required for several purposes.

It is obtained by subtracting each Gaussian (lowpass) pyramid level from


the next lower level in the pyramid.

Laplacian Pyramids

It is equivalent to convolving the original image with a difference of


two Gaussians (Laplacian Operator).

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Laplacian Pyramids

It is a complete image representation.

The top pyramid level Ln is first expanded and added to L(n-1) to form
G(n-1) and then this is expanded and added to L(n-2) to recover G(n-2)
and so on.

Image Compression

Pyramid representation contains one third more sample elements than


the original image.
1+1/4+1/16+1/64+.. =4/3

But these sample values tend to be zero and therefore can be


represented by a small number of bits.

Image Compression

Low frequency components refers to the smooth areas in the image


and high frequency components refers to sharp areas like edges.

Humans tend to be more sensitive to errors in low-frequency image


components than in high-frequency components.

Since the zeroth level of the laplacian pyramid have high frequency
components .Thus they can be quantized more coarsely than those in
higher levels

Image Compression

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Analysis

The first approach in which image pyramid can be used is pattern matching.

The task is to locate a particular target pattern that may occur at any scale
within an image.

The pattern is convolved with each level of the pyramid .

Since, the pyramid contains only one-third more sample elements than
original image ,the cost of searching is just one-third more than that of
searching in original image.

Image Analysis

Power of two steps are adequate when the patterns to be located are
simple.

Complex pattern requires a closer match between the scale of the


target pattern and the scale of the pattern as it appears in image.

Variants on the pyramid can be defined with squareroot-of-two and


smaller steps.

Image Analysis

Pyramids are also useful in the estimation of integrated properties within local
image regions e.g. texture ,estimates of image motion.

This is done as a three stage sequence of standard operations:

an appropriate pattern is convolved with the image

A non linear intensity transformation is performed on each sample value like


thresholding to detect the presence of target pattern.

the transformed sample values are integrated within local windows to obtain the
desired local property measures.

Image Analysis

Use of Pyramids:

Stage 1: A feature pattern is then convolved with each level of the pyramid

Stage 2: Resulting correlation values passed through non-linear intensity


transformation.

Stage 3: Each filtered and transformed image becomes the bottom level of a new
Gaussian pyramid

Image Analysis

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Analysis

Texture boundary detection using energy


measures

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Analysis

Pyramids are also helpful in coarse-fine search techniques.

Consider a task of locating a complex pattern within an image.

Rather than convolving the full pattern with the image , we first
convolve reduced resolution pattern with reduced resolution copy of
image which roughly locate possible occurrence of the pattern

Image Analysis

Now, we convolve with higher resolution image to refine the position estimate from
the previous convolution.

Computation is kept to a minimum by restricting the search to neighborhoods of


the points identified at the coarser resolution.

Computation cost is substantially low , especially when size and orientation of the
target pattern and its position are not known.

Image Enhancement

Pyramids may also be used to reduce random noise in an image .

Image is first decomposed into its laplacian pyramid representation.

The samples in each level are then passed through coring function where small values are
set to zero while larger values are retained

The final enhanced image is then obtained by summing the levels of the processed
pyramid

In this way , noise is reduced at each resolution level (Multiresolution Coring)

Image Enhancement

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Enhancement

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Enhancement

Image Pyramids are helpful in creating images with extended depth of field.

Given two or more images focused at different distances.

Combine them in a way that retains the sharp regions of each.

Image enhancement

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Enhancement

The node with the largest amplitude will be in the image that is most
nearly in focus.

In Focus image components are selected node by node in the pyramid

Let LA and LB be Laplacian pyramids for the two original images and LC
be the pyramid for composite image

If |LA(i,j)| > |LB(i,j)| then LC(i,j) = LA(i,j) otherwise LC(i,j) = LB(i,j)


level of pyramid

for each

Image Mosaics

Consider the task to join two images smoothly such that the segment
boundary is not visible.

Direct approach is to catinate the portion of one image with the portion of
another image. But this leaves the boundary as sharp step in gray level.

Another approach is to is to join image components smoothly by averaging


pixel values within a transition zone centered on the join line.

Image Mosaics

Width of the transition zone is the critical parameter.

If it is too narrow ,transition will still be visible as a somewhat blurred step


(blurred-edge effect). This is due to mismatch of low frequencies along the
boundary.

If it is too wide ,features from both images will be visible within the
transition zone (double exposure effect) due to the mismatch in higher
frequencies.

Image Mosaics

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Mosaics

Blurred-Edge Effect

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Mosaics

Double Exposure Effect

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Mosaics

Solution : Decompose the image into spatial frequency bands

A mosaic is constructed for each band using a transition zone which is


comparable to the wavelength of the band.

Final mosaic is obtained by summing the component bandpass mosaics.

Due to the power of laplacian pyramid to separate image into spatial


frequency bands , it is the best tool for this.

Image Mosaics

Laplacian pyramids LA and LB are constructed for the two original images.

A third Laplacian pyramid LS is constructed by copying nodes from the left half
of LA to the corresponding nodes of LS, and nodes in the right half of LB to the
right half of LS.

Along the center line of LS are set equal to the average of corresponding
LA and LB nodes.

Final Mosaic is obtained by expanding and summing the levels of LS

Image Mosaic

Mosaic Using Laplacian Pyramids

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Mosaic

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Mosaic

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Image Mosaic

Pic Credits:
Adelson et al.: Pyramid methods
in Image processing

Conclusion

Pyramids offer a useful representation for numerous tasks.

It is efficient to compute.

The information is available in a format that is convenient to use.

It offers a flexible, convenient multiresolution format that matches the


multiple scales found in the visual scenes.

Thank
You

You might also like