Chapter 7

Digital image processing

Introduction to Remote Sensing
Instructor: Dr. Cheng-Chien Liu
Department of Earth Sciences
National Cheng Kung University

Last updated: 18 November 2004

Outline
Introduction
Image enhancement
Image rectification and restoration
Image classification
Data merging
Hyperspectral image analysis

Introduction
Definition of digital image: f(x,y)
DN: Digital Number
Processing

Origin of DIP
Applications of DIP
Classified by EM spectrum
, X, UV, VNIR, IR, microwave, radiowave
Our focus: VNIR

Sound
Ultrasound,

Components of DIP
Contents of DIP

Enhancement
Histogram
Thresholding: Fig 7.11
Level slicing: Fig 7.12
Contrast stretching: Fig 7.13

Image rectification and restoration

Rectification distortion
Restoration degradation
Source
Digital image acquisition type
Platform
TFOV

Image rectification and restoration

(cont.)
Two-step procedure of geometric
correction
Systematic (predictable)
e.g. eastward rotation of the earth skew distortion
Deskewing offest each successive scan line slightly to the west
parallelogram image

Random (unpredictable)
e.g. random distortions and residual unknown systematic
distortions
Ground control points (GCPs)
Highway intersections, distinct shoreline features,
Two coordinate transformation equations
Distorted-image coordinate Geometrically correct coordinate

Image rectification and restoration

(cont.)
Affine coordinate transform
Six factors
Transformation equation
x = a0 + a1X + a2Y
y = b0 + b1X + b2Y
(x, y): image coordinate
(X, Y): ground coordinate

Six parameters six conditions 3 GCPs

If GCPs > 3 redundancy LS solutions

Image rectification and restoration

(cont.)
Resampling
Fig 7.1: Resampling process
Transform coordinate
Adjust DN value perform after classification

Methods
Nearest neighbor
Bilinear interpolation
Bicubic convolution

Image rectification and restoration

(cont.)
Nearest neighbor
Fig 7.1: a a (shaded pixel)
Fig C.1: implement
Rounding the computed coordinates to the nearest whole
row and column number

Advantage
Computational simplicity

Disadvantage
Disjointed appearance: feature offset spatially up to pixel
(Fig 7.2b)

Image rectification and restoration

(cont.)
Bilinear interpolation
Fig 7.1: a, b, b, b a (shaded pixel)
Takes a distance-weighted average of the DNs of the four nearest pixels

Fig C.2a: implement

Eq. C.2
Eq. C.3

Advantage
Smoother appearing (Fig 7.2c)

Disadvantage
Alter DN values
Performed after image classification procedures

Image rectification and restoration

(cont.)
Bicubic (cubic) interpolation
Fig 7.1: a, b, b, b, c, a (shaded pixel)
Takes a distance-weighted average of the DNs of the four nearest pixels

Fig C.2b: implement

Eq. C.5
Eq. C.6
Eq. C.7

Advantage (Fig 7.2d)

Smoother appearing
Provide a slightly sharper image than the bilinear interpolation image

Disadvantage
Alter DN values
Performed after image classification procedures

Image rectification and restoration

(cont.)
Radiometric correction
Varies with sensors
Mosaics of images taken at different times
require radiometric correction

Influence factors

Scene illumination
Atmospheric correction
Viewing geometry
Instrument response characterstics

Image rectification and restoration

(cont.)
Sun elevation correction
Fig 7.3: seasonal variation
Normalize by calculating pixel brightness values
assuming the sun was at the zenith on each date of
sensing
Multiply by cos 0

Earth-Sun distance correction

Decrease as the square of the Earth-Sun distance
Divided by d2

Combined influence

E0 cos 0
d2

Image rectification and restoration

(cont.)
Atmospheric correction
Atmospheric effects
Attenuate (reduce) the illuminating energy
Scatter and add path radiance
ET
Lp

Combination
Haze compensation minimize Lp
Ltot

Band of zero Lp (e.q.) NIR for clear water

Path length compensation

Off-nadir pixel values are normalized to their nadir
equivalents

Image rectification and restoration

(cont.)
Conversion of DNs to radiance values
Measure over time using different sensors
Different range of reflectance
e.g. land water

Fig 7.4: radiometric response function

Linear
Wavelength-dependent
Characteristics are monitored using onboard calibration lamp

DN = GL + B
G: channel gain (slope)
B: channel offset (intercept)

Fig 7.5: inverse of radiometric response function

LMAX LMIN
Equation
L
DN LMIN
255

LMAX: saturated radiance

LMAX - LMIN: dynamic range for the channel

Image rectification and restoration

(cont.)
Noise
Definition
Sources
Periodic drift, malfunction of a detector, electronic
interference, intermittent hiccups in the data transmission
and recording sequence

Influence
Degrade or mask the information content

Image rectification and restoration

(cont.)
Systematic noise
Striping or banding
e.g. Landsat MSS six detectors drift
Destriping (Fig 7.6)
Compile a set of histograms
Compare their mean and median values identify the problematic detectors
Gray-scale adjustment factors

Line drop
Line drop correction (Fig 7.7)
Replace with values averaged from the above and below

Image rectification and restoration

(cont.)
Random noise
Bit error spikey salt and pepper or
snowy appearance
Moving windows
Fig 7.8: moving window
Fig 7.9: an example of noise suppression algorithm
Fig 7.10: application to a real imagey

Image classification
Overall objective of classification
Automatically categorize all pixels in an image into land cover
classes or themes

Three pattern recognitions

Spectral pattern recognition emphasize in this chapter
Spatial pattern recognition
Temporal pattern recognition

Selection of classification
No single right approach
Depend on
The nature of the data being analyzed
The computational resources available
The intended application of the classified data

Supervised classification
Fig 7.37
A hypothetical example
Five bands: B, G, R, NIR, TIR,
Six land cover types: water, sand, forest, urban, corn, hay

Three basic steps (Fig 7.38)

Training stage
Classification stage
Output stage

Supervised classification (cont.)

Classification stage
Fig 7.39

Pixel observations from selected training sites plotted on scatter

diagram
Use two bands for demonstration, can be applied to any band number
Clouds of points multidimensional descriptions of the spectral
response patterns of each category of cover type to be interpreted

Minimum-Distance-to-Mean classifier
Fig 7.40
Mean vector for each category
Pt 1 Corn
Pt 2 Sand ?!!

Advantage: mathematically simple and computationally efficient

Disadvantage: insensitive to different degrees of variance in the spectral
response data
Not widely used if the spectral classes are close to one another in the
measurement space and have high variance

Supervised classification (cont.)

Classification stage (cont.)
Parallelepiped classifier
Fig 7.41
Range for each category
Pt 1 Hay ?!!
Pt 2 Urban

Advantage: mathematically simple and computationally

efficient
Disadvantage: confuse if correlation or high covariance are
poorly described by the rectangular decision regions
Positive covariance: Corn, Hay, Forest
Negative covariance: Water

Alleviate by use of stepped decision region boundaries (Fig

7.42)

Supervised classification (cont.)

Classification stage (cont.)

Gaussian maximum likelihood classifier

Assumption: the distribution of the cloud of points is Gaussian
distribution
Probability density functions mean vector and covariance matrix
(Fig. 7.43)
Fig 7.44: Ellipsoidal equiprobability contours
Bayesian classifier
A priori probability (anticipated likelihood of occurrence)
Two weighting factors
If suitable data exist for these factors, the Bayesian implementation of the classifier is
preferable

Disadvantage: computational efficiency

Look-up table approach
Reduce the dimensionality (principal or canonical components transform)
Simplify classification computation by separate certain classes a prior
Water is easier to separate by use of NIR/Red ratio

Supervised classification (cont.)

Training stage
Classification automatic work
Assembling the training data manual work
Both an art and a science
Substantial reference data
Thorough knowledge of the geographic area
You are what you eat!
Results of classification are what you train!

Training data
Both representative and complete
All spectral classes constituting each information class must be adequately represented in
the training set statistics used to classify an image
e.g. water (turbid or clear)
e.g. crop (date, type, soil moisture, )
It is common to acquire data from 100+ training areas to represent the spectral variability

Supervised classification (cont.)

Training stage (cont.)
Training area
Delineate boundaries (Fig 7.45)
Carefully located boundaries no edge pixels

Seed pixel
Choose seed pixel statistically based criteria contiguous pixels cluster

Training pixels
Number
At least n+1 pixels for n spectral bands
In practice, 10n to 100n pixels is used

Dispersion representative

Training set refinement

Make sure the sample size is sufficient
Assess the overall quality
Check if all data sets are normally distributed and spectrally pure

Avoid redundancy
Delete or merge

Supervised classification (cont.)

Training stage (cont.)
Training set refinement process
Graphical representation of the spectral response patterns
Fig 7.46: Histograms for data points included in the training areas of hay
Visual check on the normality of the spectral response distribution
Two subclasses: normal and bimodal
Fig 7.47: Coincident spectral plot
Corn/hay overlap for all bands
Band 3 and 5 for hay/corn separation (use scatter plot)
Fig 7.48: SPOT HRV multi-spectral images
Fig 7.49 scatter plot of band 1 versus band 2
Fig 7.50 scatter plot of band 2 versus band 3 less correlated adequate

Quantitative expressions of category separation

Transform divergence: a covariance-weighted distance between category means
Table 7.1: Portion of a divergence matrix (<1500 spectrally similar classes)

Supervised classification (cont.)

Training stage (cont.)
Training set refinement process (cont.)
Self-classification of training set data
Error matrix for training area not for the test area or the overall scene
Tell us how well the classifier can classify the training areas and nothing more
Overall accuracy is perform after the classification and output stage

Interactive preliminary classification

Plate 29: sample interactive preliminary classification procedure

Representative subscene classification

Complete the classification for the test area verify and improve

Summary
Revise with merger, deletion and addition to form the final set of statistics used in
classification
Accept misclassification accuracy of a class that occurs rarely in the scene to preserve the
accuracy over extensive areas
Alternative methods for separating two spectrally similar classes GIS data, visual
interpretation, field check, multi-temporal or spatial pattern recognition procedures,

Unsupervised classification
Unsupervised supervised
Supervised define useful information
categories examine their spectral
separability
Unsupervised determine spectral classes
define their informational utility

Illustration: Fig 7.51

Advantage: the spectral classes are found
automatically (e.g. stressed class)

Unsupervised classification (cont.)

Clustering algorithms
K-means
Locate centers of seed clusters assign all pixels to the cluster with the
closest mean vector revise mean vectors for each clusters
reclassify the image iterative until there is no significant change

Iterative self-organizing data analysis (ISODATA)

Permit the number of clusters to change from on iteration to the next by
Merging: distance < some predefined minimum distance
Splitting: standard deviation > some predefined maximum distance
Deleting: pixel number in a cluster < some specified minimum number

Texture/roughness
Texture: the multidimensional variance observed in a moving window
passed through the image
Moving window variance threshold smooth/rough

Unsupervised classification (cont.)

Poor representation
Roads and other linear features not smooth
Solution hybrid classification

Table 7.2
Outcome 1: ideal result
Outcome 2: subclasses classes
Outcome 3: a more troublesome result
The information categories is spectrally similar and cannot
be differentiated in the given data set

Hybrid classification
Unsupervised training areas

Image sub-areas chosen intentionally to be quite different from

supervised training areas
Supervised regions of homogeneous cover type
Unsupervised contain numerous cover types at various locations throughout
the scene
To identify the spectral classes

Guided clustering

Delineate training areas for class X

Cluster all class X into spectral subclasses X1, X2,
Merge or delete class X signatures
Repeat for all classes
Examine all class signatures and merge/delete signatures
Perform maximum likelihood classification
Aggregate spectral subclasses

Classification of mixed pixels

Mixed pixels

IFOV includes more than one type/feature

Low resolution sensors more serious

Subpixel classification
Spectral mixture analysis

A deterministic method (not a statistical method)

Pure reference spectral signatures
Measured in the lab, in the field, or from the image itself
Endmembers

Basic assumption
The spectral variation in an image is caused by mixtures of a limited number of surface materials
Linear mixture satisfy two basic conditions simultaneously
The sum of the fractional proportions of all potential endmembers Fi = 1
The observed DN for each pixel DN F1 DN ,1 F2 DN , 2 ... FN DN , N E
B band B equations
B+1 equations solve B+1 endmember fractions

Fig 7.52: example of a linear spectral mixture analysis

Drawback: multiple scattering nonlinear mixturemodel

Classification of mixed pixels (cont.)

Subpixel classification (cont.)
Fuzzy classification
A given pixel may have partial membership in more than
one category
Fuzzy clustering
Conceptually similar to the K-means unsupervised classification approach
Hard boundaries fuzzy regions
Membership grade

Fuzzy supervised classification

A classified pixel is assigned a membership grade with respect to its
membership in each information class

The output stage

Image classification output products
end users
Graphic products
Plate 30

Tabular data
Digital information files

Postclassification smoothing
Majority filter
Fig 7.53
(a) original classification salt-and-pepper appearance
(b) 3 x 3 pixel-majority filter
(c) 5 x 5 pixel-majority filter

Spatial pattern recognition

Classification accuracy assessment

Significance

A classification is not complete until its accuracy is assessed

Classification error matrix

Error matrix (confusion matrix, contingency table)

Table 7.3
Omission (exclusion)
Non-diagonal column elements (e.g. 16 sand pixels were omitted)

Commission (inclusion)

Non-diagonal raw elements (e.g. 38 urban pixels + 79 hay pixels were included in corn)

Overall accuracy
Producers accuracy
Indicate how well training set pixels of the given cover type are classified

Users accuracy

Indicate the probability that a pixel classified into a given category actually represents that category on
the ground

Training area accuracies are sometimes used in the literature as an indication of

overall accuracy. They should not be!

Classification accuracy assessment

(cont.)
Sampling considerations
Test area

Different and more extensive than training area

Withhold some training areas for postclassification accuracy assessment

Wall-to-wall comparison
Expensive
Defeat the whole purpose of remote sensing

Random sampling

Collect large sample of randomly distributed points too expensive and

difficult
e.g. 3/4 of Taiwan area is covered by The Central mountain

Only sample those pixels without influence of potential registration error

Several pixels away from field boundaries

Stratified random sampling

Each land cover category Stratum

Classification accuracy assessment

(cont.)
Sampling considerations (cont.)
Sample unit
Individual pixels, clusters of pixels or polygons

Sample number
General area: 50 samples per category
Large area or more than 12 categories: 75 100 samples
per category
Depend on the variability of each category
Wetland need more samples than open water

Data Merging and GIS Integration

RS applications data merging
unlimited variety of data
Multi-resolution data fusion
Plate 1: GIS (soil erodibility + slope
information)

Trend
Boundary between DIP and GIS blurred
Fully integrated spatial analysis systems
norm

Multi-temporal data merging

Same area but different dates composites
visual interpretation
e.g. agricultural crop
Plate 31(a): mapping invasive plant species
NDVI from Landsat-7 ETM+
March 7 blue
April 24 green
October 15 red

GIS-derived wetland boundary eliminate the interpretation of false positive

areas

Plate 31(b): mapping of algae bloom

Enhance the automated land cover classification
Register all spectral bands from all dates into one master data set
More data for classification
Principal components analysis reduce the dimensionality manipulate, store, classify,

Multi-temporal profile

Fig 7.54: greenness. (tp, , Gm, G0)

Change detection procedures

Change detection
Types of interest
Short term phenomena: e.g. snow cover, floodwater
Long tern phenomena: e.g. urban fringe development, desertification

Ideal conditions
Same sensor, spatial resolution, viewing geometry, time of day
An ideal orbit: ROCSAT-2
Anniversary dates
Accurate spatial registration

Environmental factors
Lake level, tidal stage, wind, soil moisture condition,

Change detection procedures (cont.)

Approach
Post-classification comparison
Independent classification and registration
Change with dates

Classification of multi-temporal data sets

A single classification is performed on a combined data sets
Great dimensionality and complexity redundancy

Principal components analysis

Two or more images one multiband image
Uncorrelated principal components areas of change
Difficult to interpret or identify the change
Plate 32: (a) before (b) after (c) principal component image

Change detection procedures (cont.)

Approach (cont.)

Temporal image differencing

One image is subtracted from the other
Change-no change threshold
Add a constant to each difference image for display purpose

Temporal image ratioing

One image is divided by the other
Change-no change threshold
No change area ratio 1

Change vector analysis

Fig 7.55

Change-versus-no-change binary mask

Traditional classification of time 1 image
Two-band (one from time 1 and the other from time 2) algebraic
operation threshold binary mask apply to time 2 image

Change detection procedures (cont.)

Approach (cont.)
Delta transformation
Fig 7.56
(a): no spectral change between two dates
(b): natural variability in the landscape between dates
(c): effect of uniform atmospheric haze differences between dates
(d): effect of sensor drift between dates
(e): brighter or darker pixels indicate land cover change
(f): delta transformation

Fig 7.57: application of delta transformation to Landsat

TM images of forest

Multisensor image merging

Multi-sensor image merging
Plate 33: IHS multisensor image merger of
SPOT HRV, landsat TM and digital
orthophoto data

Multi-spectral scanner + radar image

data

Merging of image data with ancillary

data
Image + DEM
synthetic stereoscopic images
Fig 7.58: synthetic stereopari generated from a single
Landsat MSS image and a DEM
Standard Landsat images fixed, weak stereoscopic effect in the relatively
small areas of overlap between orbit passes

Produce perspective-view images

Fig 7.59: perspective-view image of Mount Fuji

Incorporating GIS data in automated

land cover classification
Useful GIS data (ancillary data)
Soil types, census statistics, ownership
boundaries, zoning districts,

Geographic stratification
Ancillary data geographic stratification
classification
Basis of stratification
Single variable: upland wetland, urban rural
Factors: landscape units or ecoregions that combine several
interrelated variables (e.g. local climate, soil type,
vegetation, landform)

Incorporating GIS data in automated

land cover classification (cont.)
Multi-source image classification
decision rules (user-defined)
Plate 34: a composite land cover classification
A supervised classification of TM image in early May
A supervised classification of TM image in late June
A supervised classification of both dates combined using a
PCA
A wetlands GIS layer
A road DLG (digital line graph)

Table 7.5: basis for sample decision rules