Mathematic in Visual Data Processing

by
Hui Ji

The set of lecture notes are from the following references:

1. G. Aubert and P. Kornprobst, Mathematical problems in image processing, Applied Mathematical Sciences, Vol 147, Springer-Verlag, 2006.
2. S. Mallat, A wavelet tour of signal processing, AP Professional, London, 1997.
3. S. Osher and R. Fedkiw, Level set methods and Dynamic implicit surfaces, Spring Verlag, New York, 2003
4. I. Pitas, Digital image processing algorithm and Applications, John
Wiley & Sons, New York, 2000.

Chapter 1

Background
1.1

Overview

Human vision is one of the fundamental perception mechanisms by providing

visual information needed for many tasks. As the Chinese proverb says One
picture is worth a thousand words, it is actually very surprising to realize
the tremendous amount of information contained in a single picture. With
the rapid advances in digital image acquisition devices and the fast growth
in computational capability, a new discipline known as image processing
and computer vision emerged since the sixties in last century. Since then, it
has exhibited a tremendous growth in industial applications and scientific
research. Image processing concerns the transformation of an image to a
digital format and its processing by digital computer. Computer vision is
more related to the description and recognition of the digital image content.
One classification of image processing and computer vision has three distinct
classes:

Figure 1.1: The human visual system can interpret the subtle variation in transparency, shading in this image, foreground flower and blurred background

CHAPTER 1. BACKGROUND
1. low-level vision are essentially digital image processing. Its input and
output are both digital images.
2. middle-level vision has the digital images as the input and low-level
symbolic representation of image content as the output, e.g., representation of object contour.
3. high-level vision take either digital images or low-level symbolic content
as the input and high-level symbolic representation as the output,
which tries to simulate the high level of human visual perception, e.g.,
3D structure of the world, object recognition.

1.2

Digital image processing

This section is devoted to a brief overview of the various topics of digital

image processing. Digital image formation is the first step in any digital
image processing. In typical digital imaging system, the optical signal is first
captured to an electrical signal by a sensing device, followed by a digitizer to
be converted to a digital one. Thus, current digital imaging systems always
introduce some degree of deformations and degradations to the captured
digital images, e.g., geometrical distortion, noise.
Image restoration concerns the reduction of the deformations and degradations introduced during the digital image capturing. The goal of image
restoration is to reconstruct or to recover a better version of the digital image. Image enhancement concerns the improvement of the visual quality of
the digital image. The typical image enhancement techniques usually include
contrast enhancement, digital image sharpening and noise reduction. One
important concept which plays the essential role in image restoration and
enhancement is image frequency content. Digital image transforms are used
to obtain the image frequency content (or related measurements). Thus,
transform theory is an integral part through most digital image processes.
Digital images requires a large amount of memory for their storage. A
color image of size 1024 758 pixels easily needs 2+ MB disk space without any optimization. Thus, the reduction of the storage is important to
many applications. digital image compression take advantages of massive
redundancy existing in most real life images to aggressively compress it.

1.2. DIGITAL IMAGE PROCESSING

(a) Noisy image

(b) de-noised image

Figure 1.2: Removing noise from images

(a) Blurred image

(b) De-blurred image

Figure 1.3: Removing motion blurring from images

CHAPTER 1. BACKGROUND

(a) Bio-mage of cells

(b) Cell segmentation

Figure 1.4: Image segmentation

1.3

Digital image analysis

The first step in digital image analysis is often the detection of image features, e.g., edges, lines, feature points. These low-level image features should
be robust to image noise and also they should be robust to various environmental changes, dependent on the subsequent image analysis tasks. One
fundamental problem in image analysis is the segmentation. The goal of
image segmentation algorithms is to partition an image into multiple segmentations such that the image can be represented in a more meaningful
way. Image segmentation typically is used to locate objects and their boundaries. The result of image segmentation is a set of segments and pixels in
each segment are similar with respect to some characteristic property, e.g.,
color, texture.
Object recognition is another important topic in digital image analysis.
The goal of object recognition is to find a given object in an image or
video sequence. Human visual system can recognize objects in image with
ease, despite the fact that the look of the same object in different images
could vary somewhat in different views, in different sizes or even partially
obstructed from the view. The main challenges in object recognition is to
develop representation scheme for images to match the reliability of human
perception.

1.4. COMPUTER VISION

Figure 1.5: Recognizing horses

(a)

(b)

Figure 1.6: 3D reconstruction from multiple images

1.4

Computer vision

human visual system perceive the 3D structure of the world around us from
2D imagery. In parallel, computer vision focus on how to recover the 3D
shape and appearance of objects from 2D images. Computer vision has
been used in a wide range of applications, which includes: machine inspection, 3D model building, medical imaging, motion capture in animation,
surveillance, etc. While all these applications are extremely important, they
mostly pertains to fairly specified imagery and narrow domain. The current
vision techniques are far away from the dream of having a machinery which
interpret an image at the same level as human beings.
The main reason why why vision is so difficult is because it is an inverse

CHAPTER 1. BACKGROUND

(a)

(b)

Figure 1.7: Some optical illusions and what they might tell us about the
human visual system.
problem which is to recover some unknowns given in-sufficient information to
fully specify the solution. The solver to this inverse problem therefore need
accurate model to disambiguate between solutions. However, the model of
the visual world is overwhelmingly difficult considering the rich complexity
of the real world.

1.5

Related mathematical topics

Image processing has traditionally been an area in applied mathematics research as image transforms play an essential role in most image processing
tasks. Most image transforms used in the theoretical and implementational
tools are liner ones, notably in image filtering, restoration, encoding and
analysis. The basic transforms are Fourier transform with a long history
and wavelet transform which have been popular in last decades. Also, Variational principle and PDE-based approaches see their widely applications
in-developed to solve many image processing tasks, e.g., image segmentation. Many algorithms in image processing and computer vision are tackling
inverse problems to revert image formation processes, which are often closely
related to the optimization techniques.
This note is to give a rigorous treatment of various important mathematical topics which have wide applications. The main math topics will be
covered in this note include:
1. Spectral analysis and digital filtering
2. Gabor transform and wavelets

1.5. RELATED MATHEMATICAL TOPICS

3. Snake and level set method
4. Basics in Regularization
The following applications will be touched lightly in this module:
1. Image deblurring and denoising
2. Image sampling and compression
3. Image registration and image segmentation