01 - Introduction, Klaus D.

Toennies 1
Pattern Recognition in Image Analysis
Image Analysis: Extraction of knowledge from image data.
Pattern Recognition: Detection and extraction of patterns from data.
Pattern: A subset of data that may be described by some well-defined set of rules.
Patterns may constitute the smallest entity in the data that represent knowledge.
this is a pattern
pixel
01 - Introduction, Klaus D. Toennies 2
Applications of Pattern Recognition
automatic analysis of medical images
automatic inspection of parts on an assembly line
human speech recognition by computers
classification of seismic signals (e.g., for oil and mineral exploration)
selection of tax returns to audit, stocks to buy, people to insure
identification of people from fingerprints, retinal scans, handwriting, ...
automatic inspection of printed circuits, printed characters, handwriting recognition
automatic analysis of satellite pictures (e.g., weather condition, water reserves,
mineral prospects,...)
Pattern recognition (in general as well as applied to images) is mainly a
classification task.
Images often constitute the data source for pattern recognition.
Pattern Recognition and Objects
Patterns in images represent (attributes of) objects
Pattern recognition tasks are
Object detection tasks
to find an unknown number of instances of a known kind of an object
in the image
Object recognition tasks
to recognise a detected object as one of a specific kind
01 - Introduction, Klaus D. Toennies 3
Detection and Recognition
01 - Introduction, Klaus D. Toennies 4
Detection:
segmentation, object
matching, searching
techniques

Recognition:
feature computation/
reduction,
classification,
clustering

Detection and Recognition
May be used independently of each other
In combination as process chain
Detection (and segmentation) separates object detail of objects of some broad
class (e.g. cars)
Recognition separates detected objects in sub-classes (e.g. sports cars,
limousines,)
Some problems need an integrated solution (e.g. separate different car types
directly in an image)
Integrated solution is more human-loke vision but much more difficult
Potentially every pixel contributes to the solution
Features are not guaranteed to belong to an object to be classified
01 - Introduction, Klaus D. Toennies 5
Classical Solution
Pattern recognition as a processing pipeline
(Segmentation)
Feature computation
Classification
01 - Introduction, Klaus D. Toennies 6
01 - Introduction, Klaus D. Toennies 7
Pattern Recognition as a Classification
Task
Features: (f
1
, f
2
, ..., f
n
)
Features: (f
1
, f
2
, ..., f
n
)
Objects to be classified are described by features.
Features are evaluated to separate objects into
different classes.
01 - Introduction, Klaus D. Toennies 8
Feature Detection
Prerequisite for feature detection: Extraction of structures with common features
Segmentation.
Why?
Often, features are not computed from single pixels but from pixel sets. Their
computation is erroneous if feature values change over the set.
Image
Segmentation
Feature
Computation
Classification
Pattern Recognition
Segmentation and Pattern Recognition:
Knowledge on the features to be evaluated
greatly enhances the segmentation success.
01 - Introduction, Klaus D. Toennies 9
Classification
Classification: grouping patterns (samples) according to their features into different
classes.
How do I decide this?
Decide which features are relevant to the problem and decide on a way to
compute them.
Decide on a general classification technique (based on the type of features)
Train a classifier based on samples of the images to be analysed:
- Find out a differentiation between different meanings based on feature
characteristics.
- Find a suitable generalisation of the feature values based on the training set
Estimate the error of the classifier using an independent, representative test data
set.
Apply the classifier to images of the problem domain.
01 - Introduction, Klaus D. Toennies 10
Is Pattern Recognition Equivalent to
Image Analysis?
Doors are rotated with respect to camera
Windows are partially occluded
Scene is 3-d but features are from 2-d
picture.
Feature-based pattern recognition from 3-d scenes:
Features are those of the object and not that of its projection!
Pattern recognition may require 3-d surface reconstruction prior to analysis.
Example: Detect windows and doors.
01 - Introduction, Klaus D. Toennies 11
Useful and not so Useful
Features
This should have similar
appearances!
Useful features pertain to object
Useless features are those pertaining to
imaging the scene (e.g., highlights ...)
01 - Introduction, Klaus D. Toennies 12
Finding Features
Reconstruct 3-d surfaces (= 3-d computer vision)
Induce locations of light sources
Synthesise influences from light sources, surface curvature, camera position
Extract object features after accounting for synthesised influences
This is much too complicated in most real-world applications!
Simpler solutions:
looking at 2-d scenes only for feature-based pattern recognition
compute pose- and illumination-invariant features
using reconstructed 3-d geometric features (fitting methods)
01 - Introduction, Klaus D. Toennies 13
2-d Scenes and Almost 2-d Scenes
Medical images such as CT, MRI, etc.
satellite images and aerial photos
drawings and text images
some images from computer aided manufacturing
01 - Introduction, Klaus D. Toennies 14
These cars belong to the same class while
this one does not
What, if it is truly 3-d?
Illumination-Invariant Features
Additional feature reduction step
remove illumination-dependent aspects (e.g. by computing edges)
will remove object-specific features as well (in this case: object colour)
Reduction is a compromise between suppression of non-relevant features
and reduction of object characteristics
01 - Introduction, Klaus D. Toennies 15
The Processing Chain
Two Feature computation steps
Reduction of semantics that suppresses
non-object semantic
(not just illumination but also pose)
Aggregation of reduced features to
increase expressional power
Segmentation
Grouping of features
01 - Introduction, Klaus D. Toennies 16
Image
Segmentation
Feature
Computation
Classification
Feature
Computation
01 - Introduction, Klaus D. Toennies 17
Generate models of the class and decide on quality
of fit.

Model:
geometrical, 3-d model: fit edges to model
edges.
Picture 2-d model (set of 2-d pictures): fit to
interpolation from pictures.
Why?
Human Vision: There must be
some fast classification that may
be later refined.
Fitting a Model
01 - Introduction, Klaus D. Toennies 18
3-d Model Fitting
Rough classification of an object
3-d model may aid to segmentation in 3-d being pre-requisite to PR
techniques
2-d model may allow for segmentation free classification
Problem:
scene may contain more than just the object (i.e., segmentation before
segmentation) and in may be distorted by artefacts / noise.
Find the important features in an image (saliency of features)
Fitting 2-d to 2-d
Top-Down-Approach
Alternative to segmentation-to-
features-to-object
Does not requires segmentation
Requires appropriate model
Capturing the essence of members
of an object class
Capturing acceptable variability
01 - Introduction, Klaus D. Toennies 19
Simple road model
Top-Down vs. Bottom-Up
Bottom-Up = Classical Pattern Recognition
generate meaningful features
conceptual distance between features and semantics may be large (shape features,
texture features)
feature generation is (largely) domain independent
the more expressive features are the simpler is the classification task
Top-Down = Model Fitting
Generation of a meaning model of objects appearance in an image
Conceptual distance between model and semantics is small
Simple classification in low-dimensional feature space but domain-dependent
Flexible models ease adaptation to a new problem
01 - Introduction, Klaus D. Toennies 20
01 - Introduction, Klaus D. Toennies 21
What this course will be about...
Features
Feature detection, feature representation, feature reduction
Pattern Recognition Techniques
Statistical pattern recognition, neural networks (MLPs), support vector
machines
... combined with a project on picture classification
Aim of the course is to give an overview on techniques and applications in pattern
recognition as applied to image analysis.
01 - Introduction, Klaus D. Toennies 22
Background / Literature
Requirements
Basic knowledge of image processing (what is a filter, a gradient, what is
segmentation, etc.)

Literature
R.O.Duda, P.E.Hart, D.G.Stork, Pattern Classification, 2nd edition, 2001, Wiley and
Sons.
E. Gose, R. Johnsbaugh, S. Jost, Pattern Recognition and Image Analysis, Prentice
Hall, 1996
R.Jain, R.Kasturi, B.G.Schunck, Machine Vision, McGraw-Hill, 1995
E.R.Davies, Machine Vision Theory, Algorithms, Practicalities, 3nd edition,
Academic Press, 2002.
Powerpoint slides of the course (on the net)