Professional Documents
Culture Documents
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Chapter 2: Basics
Audio Technology
Images and Graphics
Video and Animation
Example
To display a 525-line television picture (NTSC) without noticeable degradation with a
Video Graphics Array (VGA) video controller, 640x480 pixels and 256 discrete grey
levels give an array of 307.200 8-bit numbers and a total of 2.457.600 bit.
Page 1
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Image Representation
Color Models
Page 2
Page 3
Page 4
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Color Models
Color Models
Page 5
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Usual scheme:
Y = 0.30 R + 0.59 G + 0.11 B (the color sensitivity of the human eye is considered)
U = c1 (B-Y); V = c2 (R-Y)
c1 , c2 = constants reflecting perception aspects of the human eye and the human brain!
Possible Coding:
YUV signal
Y = 0.30 R + 0.59 G + 0.11 B
U = (B-Y) 0.493 = - 0.148 R - 0.29 G + 0.439 B
V = (R-Y) 0.877 = 0.614 R - 0.517 G - 0.096 B
This is a system of 3 equations for determining Y, U, V from R, G, B or for recalculating
R, G, B from Y, U, V
The resolution of Y is more important than the resolution of U and V
Spend more bits for Y than for U and V (Y : U : V = 4 : 2 : 2)
The weighting factors in the calculation of the Y signal compensate the color perception
misbalance of the human eye
Page 6
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Image Formats
Why Compression?
BMP (Bitmap)
Devide-independent representation of an image: uses RGB color model, without
compression. Color depth up to 24-bit, additional option of specifying a color table to use.
JPEG:
Lossy compression remove some image details to achieve a higher compression rate
by suppressing higher frequencies
Combined with lossless techniques
Trade-Off between file size and quality
JPEG is a joint standard of ISO and ITU-T
In June 1987, an adaptive transformation coding technique based on DCT was
adopted for JPEG
In 1992, JPEG became a ISO international standard
PostScript
Images are described without reference to special properties as e.g. resolution. Nice feature
for printers, but hard to include into documents where you have to know the image size...
JPEG (Joint Photographics Expert Group)
Lots of possible compressions, mostly with loss!
Page 7
Page 8
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
JPEG
Implementation
Independent of image size
Applicable to any image and pixel aspect ratio
Entropy encoding
Data stream is considered to be a simple digital sequence without semantics
Lossless coding, decompression process regenerates the data completely
Used regardless of the medias specific characteristics
Examples: Run-length encoding, Huffman encoding, Arithmetic encoding
Color representation
JPEG applies to color and grey-scaled still images
Image content
Of any complexity, with any statistical characteristics
Properties of JPEG
State-of-the-art regarding compression factor and image quality
Run on as many available standard processors as possible
Compression mechanisms are available as software-only packages or together with
specific hardware support - use of specialized hardware should speed up image
decompression
Encoded data stream has a fixed interchange format
Fast coding is also used for video sequences: Motion JPEG
Page 9
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Compressed
Image
DCT
Pixel
Block, MCU
Predictor
Quantization
(approximation of real
numbers by
rational
numbers)
Entropy
Encoding
Run-length
Huffman
Arithmetic
Page 10
Image Preparation
Analog-to-digital conversion
Image division into blocks of NN pixels
Suitable structuring and ordering of image information
Image Processing - Source Encoding
Transformation from time to frequency domain using DCT
In principle no compression itself but computation of new coefficients as input for
compression process
Quantization
Mapping of real numbers into rational numbers (approximation)
A certain loss of precision will in general be unavoidable
Entropy Encoding
Lossless compression of a sequential digital data stream
Uncompressed
Image
Image
Processing
Hybrid encoding
Used by most multimedia systems
Combination of entropy and source encoding
Examples: JPEG, MPEG, H.261
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Image
Preparation
Source encoding
Semantics of the data are taken into account
Lossy coding (encoded data are not identical with original data)
Degree of compression depends on the data contents
Example: Discrete Cosine Transformation (DCT) as transformation technique of the
spatial domain into the two-dimensional frequency domain
Page 11
Page 12
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
The Principle
Original
Transformation
JPEG is not a single format, but it can be chosen from a number of modes:
Quantization
Quantization
Table
JPEG
Picture
Encode
Huffman,
Run Length Encoding
The opposite
JPEG
Decoder
Dequantization
Retransformation
Lossless mode
Low compression ratio perfect reconstruction of original image
No DCT, but differential encoding by prediction
Original
the details cannot
be reconstructed
Page 13
Hierarchical mode
Accommodates images of different resolutions
Selects its algorithms from the three other modes
Page 14
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
X2
B1
A1 A2
X3
Y2
Y1
AN
D1
B2
D2
X1 = 2 X 2 = 2 X 3
Y3
BM
Y1 = Y2 = Y3
DM
Yi
CN
C1
C2
C3
superpixel
Xi
Ci
Page 15
Page 16
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Hi and Vi must be integers in the range of 1 to 4. This restriction is needed for the
interleaving of components
C1
Example:
Y = 4 pixels,
X = 6 pixels
Y1
C3
C2
X1
X1 = 6, Y1 = 4
H1 = 2
V1 = 2
X2 = 6, Y2 = 2
H2 = 2
V2 = 1
X3 = 3, Y3 = 2
H3 = 1
V3 = 1
Page 17
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Page 18
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Interleaving means: dont process all blocks component by component, but mix data units
from all components
Interleaved data units of different components:
Combination to Minimum Coded Units (MCUs)
If all components have the same resolution
MCU consists of one data unit for each component
If components have different resolutions
1. For each component, regions of data units are determined;
data units in one region are ordered left-to-right and top-to bottom
2. Each component consists of the same number of regions
3. MCU consists of one region in each component
Page 19
Page 20
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
a00 a01
a10 a11
Uncompressed
Image
5
c00
b00 b01
c10
Compressed
Image
d00
Image
Processing
Image
Preparation
3
C1: H1 = 2, V1 = 2
C2: H2 = 2, V2 = 1
C3: H3 = 1, V3 = 2
C4: H4 = 1, V4 = 1
DCT
Pixel
MCUs:
Block, MCU
MCU1 = a00a01a10a11b00b01c00c10d00
MCU2 = a02a03a12a13b02b03c01c11d01
MCU3 = a04a05a14a15b04b05c02c12d02
MCU4 = a20a21a30a31b10b11c20c30d10
where
aij: data units of C1
bij: data units of C2
cij: data units of C3
dij: data units of C4
Hi =
xi
min x j
Vi =
yi
min y j
Page 21
x =0
n 1
y =0
fxy e
2i
m
ux
2i
n
Arithmetic
Page 22
vy
Fuv : = N c u c v f xy cos
x =0 y =0
cu =
fxy
1
2
1
(2x
+ 1 ) u
2N
(2y
cos
+ 1 ) v
,
2N
u ,v {0 ,...,N 1} ,
u=0
u>0
Fuv frequency domain (indicates how fast the information moves inside the rectangle)
Huffman
Run-length
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Entropy
Encoding
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Fu v =
Quantization
(approximation of real
numbers by
rational
numbers)
n 1
F00 is the lowest frequency in both directions, i.e. a measure of the average pixel
value
Fuv with small total frequency (i.e u+v small) are (in general) larger than Fuv with large u+v
Page 23
Page 24
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
fxy = N
u =0 v =0
Example
( 2 x + 1 ) u
( 2 y + 1 ) v
c v Fu v c o s
cos
2N
2N
N=2:
Fu v = c u c v
F00 = N
1
1
2
2
f cos( 0 ) =
x
N
f
2
xy
if N = N then
N =
cos
( 2 x + 1 ) u
4
cos
(2 y + 1 ) v
4
=1
Page 25
Meaning of Coefficients
c uc v
x =0 y =0
xy
cos
( 2 x +1 )u
16
cos
1 for u / v = 0
where cu/v = 2
1 otherwise
Low
Transformation to frequencies
and
+
F00
u,v [ 0,7 ]
High
Low
Low
High
High
88 block
( 2 y +1 )v
16
Page 26
How can DCT be useful for JPEG? - Fuv for larger values of u and v are often very small!
1
4
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
F uv =
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
i.e. 2 f if fxy f
3
f00 cos + f01 cos
+ f10 cos + f11 cos
F01 =
4
4
4
4
2
1
1
1
1
2
2
2
2
2
2
2
2
2
N
u
v
u 3 v
3 u v
3 u 3 v
cos
+ f0 1 c o s
+ f1 0 c o s
+ f1 1 c o s
4
4
4
4
4
4
4
4
1
F00 = [f00 + f01 + f10 + f11 ]
2
1
N2
N
f
2
2
x =0 y =0
= N c 0 c 0 F00 1 1
=
2
2
= c u c v f0 0 c o s
Fuv = cu cv 14 ...
N=8 (Standard):
F10
F01
Low
F12
F23
High
...
Page 27
Page 28
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Uncompressed
Image
Image
Preparation
Page 29
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Compressed
Image
Image
Processing
Pixel
Predictor
Block, MCU
DCT
Quantization
(approximation of real
numbers by
rational
numbers)
Entropy
Encoding
Run-length
Huffman
Arithmetic
Page 30
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Quantization
Observation:
...
N-1
Fuv
smaller
values
1
..
.
Quantization process:
Divide DCT-coefficient value Fuv by an integer number Quv
and round the result to the nearest integer
N-1
most values
are zero
Dequantization: F Q uv Quv = F * uv
(only an approximation of Fuv)
..
...
31
...
7 9
9
..
Quv =
17
F Q uv = [Fuv / Quv ]
Example:
...
Page 31
JPEG application provides a table with 64 entries, each used for quantization of one
DCT-coefficient
each coefficient can be adjusted separately
A high compression factor is achievable on the expense of image quality
large quantization numbers: high data reduction but information loss increases
No default values for quantization tables are specified in JPEG
Page 32
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Example
Example
DC
coefficient
140
144
147
140
140
155
179
175
144
152
140
147
140
148
167
179
186
-18
15
-9
23
-9
-14
-19
11
13
15
17
152
155
136
167
163
162
152
172
21
-34
26
-9
-11
11
14
11
13
15
17
19
168
145
156
160
152
155
136
160
-10
-24
-2
-18
-20
-1
11
13
15
17
19
21
162
148
156
148
140
136
147
162
-8
-5
14
-15
-8
-3
-3
11
13
15
17
19
21
23
-3
10
-11
18
18
15
11
13
15
17
19
21
23
25
-2
-18
-4
-7
13
15
17
19
21
23
25
27
17
19
21
23
25
27
29
19
21
23
25
27
29
31
147
167
140
155
155
140
136
162
136
156
123
167
162
144
140
147
148
155
136
155
152
147
147
136
Fuv
1.
Page 33
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
-3
-1
-7
-1
-2
15
-8
-2
-6
17
3.
2.
Page 34
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Example
Example
F*uv - reconstruction
after dequantization
Effects of Quantization
62
-4
-1
-1
-1
-1
186
-20
14
-9
22
-13
-15
-17
-5
-1
-1
20
-35
27
-11
-13
15
17
-1
-3
-1
-1
-7
-27
-15
-19
-1
-1
-9
13
-15
-1
13
15
-19
21
-1
-17
15
Error in reconstruction:
146
157
161
155
149
162
181
185
152
176
148
176
167
176
186
209
13
161
183
161
193
188
187
180
203
14
15
10
24
29
27
28
19
30
28
25
26
25
25
28
31
169
170
180
178
175
176
169
183
172
174
175
169
163
161
163
186
25
24
18
23
21
33
23
158
188
152
188
181
162
159
190
10
26
19
21
23
25
16
24
23
25
143
180
140
186
186
160
155
171
11
21
12
33
26
22
23
28
157
179
159
172
181
167
175
168
24
17
19
24
16
15
24
24
23
17
29
20
28
32
4.
5.
Quantized Matrix
Correct value
was -11
Reconstructed
image after
performing the
inverse DCT:
6.
instead of -18
Page 35
Page 36
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Problem of Quantization
Result: edges of
blocks can be seen
Uncompressed
Image
Compressed
Image
Image
Processing
Image
Preparation
Pixel
Predictor
Block, MCU
DCT
Quantization
(approximation of real
numbers by
rational
numbers)
Entropy
Encoding
Run-length
Huffman
Arithmetic
Page 37
Page 38
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Example
Entropy Encoding
Initial step: map 88 block of transformed values FQuv to a 64 element vector which
can be further process by entropy encoding
DC-coefficients determine the basic color of the data units in a frame; variation
between DC-coefficients of successive frames is typically small
The DC-coefficient is encoded as difference between the current coefficient and the
previous one
Zig-zag ordering
62
-3
-1
-1
-1
-5
-1
-1
-1
-3
-1
-1
-1
-1
-1
-1
Coefficients with lower frequencies are encoded first, followed by higher frequencies.
Result: sequence of similar data bytes efficient entropy encoding
Page 39
Entropy encoding:
Run-length encoding of zero values of quantized AC-coefficients
Huffman encoding on DC- and AC-coefficients
Page 40
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Run-length Encoding
Run-length Encoding
ABCCCCCCCCDEFGGG
ABC!4DEFGGG
Offset of 4, since for smaller blocks there would be no reduction effect; e.g. with offset 3:
D!0
DDD
(both strings have same length)
Page 41
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Page 42
Huffman Encoding
0000 =
Size i
Amplitude
0001 =
0010 =
10
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Example
1
2
3
4
-1
-3, -2
-7,...,-4
-15,...,-8
i
-2 +1, ...,-2i-1
-1023,...,-512
1
2,3
4,...,7
8,...,15
2i-1,...,2i-1
512,...,1023
0011 =
0100 =
0101 =
0110 =
0111 =
8
9
10
11
12
13
14
4 bits 1-complement
Representation (other
representations are
possible)
15
1000 = 15
1001 = 14
1010 = 13
1011 = 12
1100 = 11
1101 = 10
1110 = 9
1111 = 8
15, 0, 15, 0,
5, 7,
57
35 zeros at all,
followed by a value
represented using 7 bit
With 7
bit, 121 is
64 + 57
The Huffman code is an optimal code using the minimum number of bits for a string
of data with given probabilities per character
Statistical encoding method:
For each character, a probability of occurrence is known by encoder and
decoder
Frequently occurring characters are coded with shorter strings than seldomly
occurring characters
Successive characters are coded independent of each other
Resulting code is prefix free
unique decoding is guaranteed
A binary tree is constructed to determine the Huffman codewords of the characters:
Leaves represent the characters that are to be encoded
Nodes contain the occurrence probability of the characters belonging to the
subtree
Edges of the tree are assigned with 0 and 1
In a second step, the string may be still reduced by Huffman encoding principles
Page 43
Page 44
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Huffman Encoding
p(ADCEB) = 1.00
0
1
p(AB) = 0.63
p(CED) = 0.37
0
Page 45
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
p(D) = 0.14
x
A
B
C
D
E
w(x)
10
11
00
011
010
Page 46
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
p(A) = 0.27
p(E) = 0.07
p(B) = 0.36
p(ED) = 0.21
p(C) = 0.16
Resulting Code:
0
1
2
.
.
14
15
... 10
Amplitude codes are stored in the preceding (run-length, size) coding table
(1,3)
(i,j)
(15,10)
Page 47
Page 48
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Characteristics:
Achieves optimality (coding rate) as the Huffman coding
pA = 0.5
Difference to Huffman: the entire data stream has an assigned probability, which
consists of the probabilities of the contained characters. Coding a character takes
place with consideration of all previous characters.
pB = 0.2
pAA = 0.25
0
The data are coded as an interval of real numbers between 0 and 1. Each value
within the interval can be used as code word.
0.35
0.7
pBA
0.5
0.7
pACB = 0.03
0.425
0.35
pBB pBC
0.6 0.68
pACA = 0.075
pC = 0.3
0.5
pCA
pCB
0.85
pCC
0.91
pACC = 0.045
0.5
0.455
pACAB = 0.015
0.3875
0.35
pACAC = 0.0225
0.4025
0.425
ACAB can be coded by each binary number from the interval [0.3875, 0.4025),
rounded up to -log2(pACAB) = 6.06 i.e. 7 bit, e.g. 0.0110010
Page 49
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Page 50
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
JPEG is not a single format, but it can be chosen from a number of modes:
Most popular mode: sequential display mode with 8 bits/sample and Huffman encoding
Page 51
Page 52
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Step 2
Step 3
Step 1
Step 2
Step 3
Page 53
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
C
A
B
X
0
1
2
3
4
5
6
7
predicted values X
no prediction
A
B
C
A+B-C
A+(B-C)/2
B+(A-C)/2
(A+B)/2
Page 54
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
predictor
Uncompressed data
Predictor
Entropy encoder
Compressed data
Example:
(4,0): X is exactly given by A+B-C
(7,1): X is (A+B)/2+1
Page 55
The Hierarchical mode uses either the lossy DCT-based algorithms or the lossless
compression technique
The idea: encoding of an image at different resolutions
Algorithm:
Image is initially sampled at a low resolution
Subsequently, the resolution is raised and the compressed image is subtracted from the
previous result
The process is repeated until the full resolution of the image is obtained in a compressed
form
Disadvantage:
Requires substantially more storage capacity
Advantage:
Compressed image is immediately available at different resolutions
scaling becomes cheap
Page 56
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
JPEG 2000
Wavelets
Page 57
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Thus, the most important aspects are available with a relatively small
number of bits
Application of Wavelets:
Image compression
Image editing
Animation
Signal processing
...
Page 58
1D Haar Transforms
The simplest example of wavelet technology is the Haar transform. There are
one-dimensional Haar wavelet transforms (which allow the compression of the
representation of piecewise constant functions)
two-dimensional Haar wavelet transforms (for image compression, ...)
higher-dimensional Haar wavelet transforms
Example (one-dimensional):
Simple example to show how we can reduce the amount of bits needed for
representation by transformation and compression
Let a string of pixels be given as follows:
7
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Haar Transforms
This procedure may be continued. The detail values become less and less important
From the transformed strings we can fully reconstruct the original
Very often the details are very small and may be suppressed. In such a case we
cannot exactly reconstruct the original but the errors are comparatively small.
Then we do (recursively):
calculate the average of successive pairs
calculate the distance from the average (detail)
Page 59
Page 60
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Example
9
Example
7
Reconstruction:
6; 0; 2, 1; 1, -1, -1, 1
6=6+0
Averages
+1 -1
-1
+1
8=6+2
0
detail
[9, 7, 3, 5, 6, 8, 6, 4]
[6; 0; 2, 1; 1,-1,-1,1]
Page 61
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
5=6-1
7=8-1
6=5+1
4=5-1
Page 62
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Generalizsation
Generalization
A simple basis of Vj (there are many other alternatives for a basis) is:
j
i ( x ) =
0
03
i +1
i
x j ;
2 j
2
i +1
i
x j ;
2 j
2
for i = 0, ...,
7=6+1
V 0 V 1 V 2 V 3 ...
4=6-2
1
9=8+1
Global average
6=6-0
Details
2j -
43
0
1
Definition:
1
1. The inner product of two functions f, g Vj is defined as f g : = f ( x ) g ( x ) dx
0
2. The L2 norm is defined by
1
u( x) 2 : = < u u > 2 =
1
u( x) dx
2
Page 63
Page 64
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Generalization
Generalization
In the following we will use another basis for Vj than the very simple basis which
consists only of the elementary components shown before
Definition:
j
The elements of a basis of Wj (i.e. linearly independent functions i ( x ) which
span Wj) are called Wavelets.
Immediate consequence:
j
1. i ( x ) together with the basis of Vj are a basis of Vj+1
j
j
2. i ( x ) is orthogonal to k ( x ), i.e .
j
i
( x ) kj ( x )d x = 0
The basis functions of Vj together with the basis functions of Wj form a basis of Vj+1
Page 65
Page 66
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Generalization
Example
Wavelets for Wj (i.e. elements of Vj+1 which are orthogonal to basis elements of Vj)
are denoted as Haar wavelets
Haar wavelets of Wj (there would be other possibilities for basis functions):
i (x):
j
+1
= 1
i + 12
i
if x j ;
2 j
2
1
i + 2 i +1
if x
;
j
2 j
2
o t h e r w is e
12
02
+1
+1
12
-1
-1
32
22
22
Page 67
02
0
for i = 0, ..., 2j - 1
2
3
+1
+1
-1
-1
Page 68
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Example
9
Example
3
Graphical representation:
6x
0x
1
+ 2x
= 9 ( x ) + 7 ( x ) + 3 ( x ) + 5 ( x ) + 6 ( x ) + 8 ( x ) + 6 ( x ) + 4 ( x )
3
0
3
1
3
2
3
3
3
4
3
5
3
6
3
7
1
+ 1x
1
+ 1x
= 8 02 ( x ) + 4 12 ( x ) + 7 22 ( x ) + 5 32 ( x ) + 1 02 ( x ) 1 12 ( x ) 1 22 ( x ) + 1 32 ( x )
1
- 1x
1
- 1x
= 6 01 ( x ) + 6 11 ( x ) + 2 01 ( x ) + 1 11 ( x ) + 1 02 ( x ) 1 12 ( x ) 1 22 ( x ) + 1 32 ( x )
1
+ 1x
= 6 00 ( x ) + 0 00 ( x ) + 2 01 ( x ) + 1 11 ( x ) + 1 02 ( x ) 1 12 ( x ) 1 22 ( x ) + 1 32 ( x )
Page 69
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
00
00
01
11
02
12
22
32
Page 70
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Example
00
W2
1
0
V3
1
1
of
W1
to the basis
Page 71
V3 approximation
V2 approximation
W3 detail coefficients
W2 detail coefficients
W1 detail coefficients
average of f(x)
V4 approximation
V1 approximation
V0 approximation
W0 detail coefficients
Page 72
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Decomposition
All Haar basis functions 0 , 0 , 0 , 1 ,.... are orthogonal to each other (of course
not to itself); i.e. <f|g> = 0 if f g are basis functions
Orthogonality to all other basis functions is not necessarily valid for other systems of
basis functions
In addition to orthogonality we can provide orthonormality, i.e. <f|f> = 1
The basis functions
j
0
i ( x) : = 2 i ( x)
j*
i j* ( x) : = 2 i j ( x)
j
are orthonormal
If the basis functions are orthonormalized, then
the detail coefficients and the
j
average coefficient have to be multiplied by 2 2
In our example we then get:
2
2
01* +
1
2
Page 73
S( x ) = 6 00 + 0 00 + 2 01 +1 11 +1 02 1 12 1 22 + 1 32
= 6 00* + 0 00* +
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Data Compression
f ( x ) = c i ui ( x )
i =1
If we do not change the basis function but reduce the number of coefficients then it is
easy to show that:
If the basis functions are orthonormal, then for m < m coefficients and for the L2norm, i.e. f f =< f(x) f (x)| f(x) f (x) > , the error is minimized if we keep the
coefficients of c1 ,..., cm whose absolute values are the largest ones
Proof: Let 1, ..., m be a permutation of 1, ..., m
We want to reduce the number of coefficients, i.e. we seek for a function f ( x ) which:
1. Is similar to f(x), i.e. f ( x ) f ( x ) for some norm
Page 74
Let
f ( x) = ci ui
i =1
~
f ( x ) - f ( x)
~ +1
i =m
~ +1 j = m
~ +1
i =m
(c )
~ +1
i =m
~ +1
j =m
c < u u >
i =1
Finding the best f ( x ) is a difficult problem if all possible u i ( x ) are taken into account.
Thus we restrict ourselves to the Haar Wavelet basis
Page 75
Page 76
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Data Compression
2D Haar Wavelets
Approximation of f(x) by V3 , W3
3
3
i.e. 8 averages 0 ( x ),...,7 ( x )
3
8 details
0 ( x ),..., 73 ( x )
We have shown how to use one-dimensional Haar wavelets for representation and
compression of one-dimensional functions
Now we generalize to 2D-images
First we show how to apply the (averaging+detail) technique to 2D-images
orthogonal functions!!
(for simplicity the * has
been suppressed)
00 ; 00 ; 01 ; 11 ; 02 ,..., 32 ; 03 ,..., 73
a pixel of 2D-image
16 coefficients
global average
Page 77
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Details of different
resolution
(see previous
examples)
Detail
coefficients
Page 78
x++++
x++++
x++++
x++++
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
2D Haar Wavelets
average
we get 2i 2i basis functions for Vi in 2D by combining the 1D basis functions with each
other.
detail
coefficients
step 2
(for left
half only)
Example for V2 where the 2D-basis will consist of 4x4 = 16 basis functions (since there
are 4 basis functions 00 , 00 , 01 , 11 in V 2 )
step 1
3. Repeat step 1 and 2 for the averages part, i.e. for the left upper corner
4. Do that recursively until only one global average is left over
Remark: This technique works only for square images, i.e. 2n2n pixels
Page 79
Page 80
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Example:
y
11 ( y )
-
+
-
+ means: +1
- means: - 1
0
0
x
00 ( y )
00 ( y )
01 ( y )
11 ( y )
00 ( x )
01 ( x )
11 ( x )
The result is
Page 81
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
01 , 1 ( x,y )
1,1 1 ( x,y )
01 , 0 ( x,y )
1,1 0 ( x,y )
00, 0 ( x,y )
+
-
01 , 1 ( x,y )
+
-
01 , 0 ( x,y )
+
+
-
1,1 1 ( x,y )
00, 0 ( x,y )
01, 1 ( x,y )
1,1 1 ( x,y )
0 , 0 ( x,y )
Chapter 2.2:
Images and Graphics0 , 0 ( x,y )
0
01, 0 ( x,y )
+-
+
-
++
-
+-
+-
+-+
+-+
( x) (y ) ( x) (y ) ( x) (y ) 11( x) 00 (y )
function for
global average
+
-
1,1 0 ( x,y )
+
+
-
Page 82
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
( x,y ) := ( x)(y )
( x,y ) := ( x) (y )
01 ( x ) 11 ( y )
+
-
+ -
+-
01 ( x )
0
0
+
0
0
0
0
0
0
1
0
+-
0
0
function for
global average
1,1 0 ( x,y )
Page 83
00 ( x) 00 (y ) 00 ( x) 00 (y ) 01( x) 00 (y ) 11( x) 00 (y )
Page 84
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Comparison
Decomposition
Page 85
Page 86
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Decomposition
Decomposition
1
0
( x ) :=
i j ( x ) : = ( 2 j x i )
ij ( x ) : = ( 2 j x i )
where
for 0 x 1
otherwise
for 0 x < 12
1
x <1
2
( x ) : = 1 for
0
otherwise
i j ( x ) : = 2 ( 2 j x i )
j
2
i j ( x ) : = 2 ( 2 j x i )
Example:
coefficients
basis functions
(6; 0; 2,1;1,-1,-1,1)
= ( 00 ; 00 ; 01 , 11 ; 02 , 12 , 22 , 32 )
Normalized version:
coefficients
Page 87
basis functions
Chapter 2.2: Images and Graphics
(6; 0;
2
2
1
2
; 12 , -12 , -12 , 12 )
0*
0
1*
0
1*
1
Page 88
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Decomposition
2D Image Compression
C :=
C :=
We normalize first:
22
0
0
3
7
C
C
C
=
= n
h
22
2n
k with
Step3: Find the smallest
!
the L2norm,i.e. f f
i =k +1
n 1
2
The same holds for the averages. In successive runs the other detail levels and the
global average are correctly evaluated by means of successive multiplications with 2
Page 89
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Decomposition
Decomposition
This procedure starts with coefficients C[1], , C[m] and finds the smallest m for
m
which
c i 2 2 where 1 , , m is a permutation of 1, ,m
A fast procedure (by binary search) for finding the threshold which coefficients are
negligible
i = m +1
Page 90
such that c 1 c 2
c m
Page 91
Page 92
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Decomposition
Multiresolution Analysis
Let (without any loss of generality) c1, c2, , cm be already sorted such that
|c1| (max) |c2| |cm| (min)
Let s := squared error by discarding all coefficients smaller than , i.e. all coefficients
on the left side of
Case 1: s < 2
We could even discard (possibly) more than those coefficients! Hence we discard the
part on the left of . We set new,min:= and restart with the new interval.
Still very small error?
12
02
new,min =
max
min
Case 2: s > 2
new interval
By discarding the elements left of the actual we would discard too many elements.
The elements left of must be kept anyway. Set new,max:= and restart.
32
22
new,max =
min
max
new interval
Page 93
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Page 94
Matrix Formulation
01
2
0
2
1
2
2
2
3
1
0
1
1
02
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Matrix Formulation
Example:
Example:
32
1
with: P 2 =
0
0
1 0
1 0
0
and Q2 =
0 1
1
1
0 1
j 1
j 1
mj-1
j 1
j
j
There must be a matrix Pj with (a) ( x ) = ( x ) P
j
j
j-1
where P is a m m matrix
1 ( x ) = 2 ( x ) P 2 and 1 ( x ) = 2 ( x ) Q2
11
12
22
In our example:
1 1
0 1
mj
Page 95
1
0
1
1
=
2 2
0 1
2
2
2
3
= j P j Q j
1
0
0
0
1
1
1
1
0
0
0
1
Page 96
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Matrix Formulation
5. All functions of
<
j 1
(from (b)):
6.
<
j 1
j 1
Matrix Formulation
j 1
The columns of Qj are a basis for the null space. There are many alternatives for Qj, thus
there are many different wavelet bases for a given Wj. The Haar wavelets are defined
by the additional requirement that the number of consecutive nonzero values in Qj per
column is minimal
0
1
1
0
2
Q =
0
1
1
0
( x ).
> = 0
<
j 1
j 1
> Q j = 0
Definition
Orthogonal wavelet basis: all functions (wavelets + scaling functions) are orthogonal to
each other
Semi-orthogonal wavelet basis: wavelets orthogonal to scaling functions but not to
each other
Haar basis
Page 97
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
=
semi-orthogonal
orthogonal; spline basis =
Page 98
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Matrix Formulation
Matrix Formulation
We may want to express f(x) within Vj-1 (a lower resolution version, i.e. less accurate,
with lower number of coefficients, i.e. mj-1 instead as of mj coefficients)
The standard procedure of creating the mj-1 coefficients of Cj-1 out of the mj coefficients
of Cj consists of:
linear filtering
the coefficients of Cj
down sampling
This may be expressed as a matrix equation: Cj-1 = Aj Cj
mj
where Aj is a mj-1 mj matrix
mj-1
Since Cj-1 is smaller then Cj this filtering process looses some detail.
The lost detail may be expressed in a second matrix Dj-1 with Dj-1 = Bj Cj where Bj is a
nj-1 mj matrix
Page 99
Page 100
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Matrix Formulation
Example
Cj-1 = Aj Cj
Dj-1 = Bj Cj
Low resolution of Cj
A2 =
If Aj and Bj are appropriately chosen, Cj may be recovered from Cj-1 and Dj-1:
Cj = Pj Cj-1 + Qj Dj-1
The process of calculating Cj-1 and Dj-1 from Cj is called decomposition of Cj or
<analysis>
The process of calculating Cj from Cj-1 and Dj-1 is called reconstruction of Cj or
<synthesis>
The global procedure looks as follows:
The coefficients Cj can be
j
reconstructed
from:
j-1
1
A
A
A
Cj
Cj-1
Cj-2......C1
C0
C0,D0,D1,D2,...,Dj-1
j-1
This sequence is called
Bj
B1
Dj-1 B
Dj-2
D0
wavelet transform. It has
This recursive procedure is called a filter bank
the same size as Cj.
Page 101
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
B2 =
1
2
1 0
C1 = A2 C2
D1 = B2 C2
j 1
j 1
B j
Thus since
j 1
A
j
B
= P j Q j
j 1
= j P j Q j
i.e.
A
j
B
must be invertible
Page 102
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Multiresolution Analysis
Multiresolution Analysis
(j = 0,1,...)
But: orthogonality comes very often with large support. Thus it may be better to
sacrifice the orthogonality. An example for that are the spline wavelets which:
have minimum support
are not orthogonal to each other (except for degree = 0)
(
x
)
j
(this defines Q )
Remark: Pj and Qj define Aj and Bj since A j
j
j
j = P Q
B
1
2
Page 103
Page 104
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
B-Spline Wavelets
The nonuniform B-spline basis functions for degree d are constructed as follows:
Choose positive integers k and d (k d) and values x 0 ,..., x k + d +1
These values are called knots
Then we define recursively the non-uniform B-spline basis functions of degree d:
However, Haar transforms are not well suited for animation and for curve editing since
they do not have enough smoothness
Therefore, we are interested in wavelets which have several continuous derivatives
Such wavelets can be derived from piecewise polynomial splines (B-splines)
1
N i0 ( x ) : =
0
x i x < x i +1
otherwise
N ir ( x ) : =
x xi
N ir 1 ( x ) +
xi +r xi
x i + r 1 x
x i + r 1 x i 1
N ir+11 ( x )
(i = 0, ..., k; r = 1, ..., d)
Remark: If the denominator is 0 then the whole term is defined to be zero
B-spline scaling functions and B-spline wavelets can be defined for different degrees
d = 0, d = 1, d = 2, ...
The higher the degree the smoother the transform
With degree d, the scaling functions have d - 1 continuous derivatives
d = 0 is the Haar transform
Page 105
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Page 106
2j
+ d 1 and by
1
2
),[ 12 : 1 )
N 03
N 0d ( x ),...,N 2d j + d 1 ( x ) = N dd +1 ( x )
N 02
N
Njd(x)
have
The functions
d - 1 continuous derivatives.
N0d(x), ..., Nid(x) are a basis
for V1(d)
N 13
1
0
0
0
N 12
If Vj(d) denotes the space which is spanned by the B-spline scaling functions of
degree j with 2j uniform intervals, then the spaces V0(d), V1(d), ... are nested:
V0(d) V1(d) V2(d) ...
N 23
1 N1
1
0
1
Degree 0
1 N2
2
N 21
Degree 1
N 33
0
N 32
N 43
Degree 2
Degree 3
Page 107
Page 108
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
In the following we show B-spline scaling functions for j = 3 and for degrees
d = 0 (Haar wavelet functions)
d = 1 (linear splines)
d = 2 (quadratic splines)
d = 3 (cubic splines)
lim f ( x + ) lim f ( x )
0
if x =
1
8
, 28 ,..., 78
8 scaling functions
Page 109
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Page 110
9 scaling functions
8 wavelets
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
0
8 wavelets
Page 111
10 scaling functions
1
8 wavelets
Page 112
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Wavelets in JPEG2000
1
11 scaling functions
8 wavelets
Page 113
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Page 114
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Decomposition
Vertical information
(fine details)
Diagonal information
(finest details)
Horizontal information
(fine details)
Page 115
(a)
(b)
(c)
Page 116
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
But
Graphics Format
Graphics image formats are specified through:
Graphics primitives: lines, rectangles, circles, ellipses, text strings (2D), polyhedron
(3D), vectors
Attributes: line style, line width, color affect.
Graphics primitives and their attributes allow a higher level of an image representation
by abstract representation. The graphics package determines which primitives are
supported.
Formats
IGES (Initial Graphics Exchange Standard), for transferring 2D/3D CAD data
DXF: developed for 2D/3D AutoCAD data
HPGL (Hewlett Packard Graphics Language), developed for addressing plotters
Also possible: mixed formats (vector and raster graphics), e.g. WMF (Windows
Metafile)
Page 117
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Image Synthesis
Image synthesis (generation) is an integral part of all computer graphical user interfaces
and indispensable for visualising 2D, 3D and higher dimensional objects. E.g.:
Raster format
Page 118
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Graphics
(vector) format
Page 119
Page 120
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Graphics Hardware
Application model
Represents data or objects to be pictured (stored in an application database)
Stores graphics image formats and connectivity relationships of the components
Should be application specific and independent of any particular display system
Converts image database representations to the graphics system format
Application program
Handles user inputs by sending commands to the graphics system describing what to
display and how this objects should appear
Graphics system
Intermediary component between application programs and the display
Performs an output transformation on objects in the application model
Performs an input transformation on user actions to the application
Consists of output subroutines collected in a graphics package to display images
Input:
Mouse, keyboard, data tablet, touch-sensitive panel on the screen (2D input)
track-balls, space-balls, data glove, etc (3D and higher-dimensional input)
Output: raster display
Sampled Image
1101100001010010100
0101000101010000001
0000000000000100001
0000111010110111000
0010000100100000010
0100000000000000100
0100000101010101111
Video Controller
Display Controller (DC)
Refresh Buffer
Interaction Data
Mouse
Display Commands
Graphics hardware
Receives input from interaction devices and outputs images to display device
Computer-Host Interface
Page 121
Page 122
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Raster Display
vertical retrace
Construct a line from left to right with a maximal gradient 0 m 1, ( 45), all other
lines are achieved through reflections.
pixel
Page 123
Keyboard
0 < dj
dj = 0
1
2
d j +1
1
2
j +1
= yj
d j +1 >
1
2
j +1
= yj +1
Page 124
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
i+1
yi+2
Yi+2
d i +3 = d i + 2 + m 1 = d i + 3m 1
di+3
Yi
y di
i+1
since d i + 2 > 2
Yi+3
di+2
di+1
Yi+1 = yi + di+1
yi + di+1 = yi-1 + di + m
d i + 2 = d i +1 + m = d i + 2m
yi+3
Yi+1
Yi+1 = Yi + m;
d i +1 = d i + m
i+3
i+2
if yi = yi-1
di + m - 1
if yi = yi-1+ 1
di + m
(and yi = yi-1)
=
di + m - 1(and yi = yi-1 + 1)
y i +1 = y i
1
y i + 2 = y i +1 + 1 since d i + 2 >
y i +3 = y i + 2
since
d i +3
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
1
2
1
if d i >
2
if d i
Changes of yi:
yi+1 =
Page 125
1
2
1
criterion :di >
2
criterion :di
di + m
1
2
1
>
2
yi
if d i +1
yi+1
if d i +1
if Di 0
if Di > 0
and
if Di +1 0
y
y i +1 = i
y i +1 if Di +1 > 0
Page 126
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Dithering
Kind of cheating to achieve more than two intensity levels if only two-level intensities
may be represented resp. to increase the possible color depth is dithering.
Dithering means to use the human eyes impossibility to differentiate between
neighbored pixels if the resolution is too fine.
Example:
m = 0.2 x = 1, y = 0.2,
M = 2y = 0.4, D0 = x = 1
xi
10
11
12
13
14
15
16
17
yi
Di
-1 -0,60 -0,20 0,20 -1,40 -1,00 -0,60 -0,20 0,20 -1,40 -1,00 -0,60 -0,20 0,20 -1,40 -1,00 -0,60
yi
2
1
0
1
8
xi
10 11 12 13
14 15 16 17
Page 127
Page 128
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Image Analysis
Image Analysis
Image analysis is concerned with techniques for extracting descriptions about images:
Computation of perceived brightness and color
Partial or complete recovery of 3D data in a scene
Location of discontinuities corresponding to objects in a scene
Characterisation of the properties of uniform regions in a image
Image Enhancement
Improves image quality by eliminating noise (extraneous or missing pixels) or by
enhancing contrast, i.e. X-ray images, computerized axial tomography (CAT)
Example:
Problems:
Page 129
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Page 130
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Target Detection
Steps
an object
unimproved
Source
of Image
000000010000110000000000111111001101111110010000000001000001110000000000000
Formatting
digital
image data
structure
Conditioning
Labelling
Example:
1
w ith a h =
0
h + m -1
if
ai m
Grouping
i=hm +1
e ls e
Extracting
000001000010001110110110111100010
improved
Filtering with (5,3)-filter
000000000000001111111111111100000
logical data
structure
Target
Image
Matching
Symbolic Representation
Page 131
Page 132
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Image Transformation
Formatting
Capturing of an image and transforming to a digital representation
Conditioning
Based on a model that assumes that the observed image is composed of an
informative pattern modified by uninteresting variations
Estimates informative pattern on the basis of the observed image
Suppresses noise and performs background normalization by suppressing
uninteresting systematic or patterned variations
Applied uniformly and context-independent
Examples
1. Location dependent weakening of Intensity (e.g. effect with lens):
(f ( x,y )) = f ( x,y ) + intensityfunction( x,y )
Labelling
Based on a model that assumes that the informative pattern has structure as a
spatial arrangement of events
Determines in what kind of spatial events each pixel participates
Labelling operations:
edge detection, corner detection
thresholding (e.g. filters significant edges and labels them)
identification
of pixels that participate in various shape primitives
Page 133
Chapter 2.2: Images
and Graphics
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
i
s
Page 134
Grouping
Identifies events by collecting or identifying maximal connected sets of pixels
participating in the same kind of event (neural networks!)
Determines new sets of entities
Changes the logical data structure
Entities of interest after grouping are sets of pixels
E.g. line-fitting is a grouping operation, where edges are grouped into lines
Extracting
Computes for each group of pixels a list of properties:
Centroid, area, orientation, spatial moments, grey tone moments, spatial-grey tone
moments, circumscribing circle, inscribing circle, number of holes in a region, average
curvature in an arc, etc.
Measures topological or spatial relationships between two or more groupings, i.e.
clarifies whether two groupings touch, are spatially close or layered
t(i)
Analogous to spreading,
the greyscale can be
compressed and used
only in partial intervals.
A combination of both is
e.g. useful to manipulate
film materials.
t(i)
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Image Transformation
for 0 i a
r
t( i ) : =
( i a ) for a < i < b
b a
for b i r
r
1 for s < i r
ht( f ) is the new Histogram of hf defined as:
Range of new
greyscale
values
Page 135
Page 136
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Matching
Matching
Determines the interpretation of some related set of image events recognised
previously in the extracting step
Associates events with some given 3D objects or 2D shapes
Template matching is a classical example of a wide variety of matching operations,
compares examined pattern with known and stored models and chooses the best
match
Conclusion
Conditioning, labelling, grouping, extracting and matching constitute a canonical
decomposition of the image recognition problem
Each step prepares and transforms the data to facilitate the next step
On any level the transformation is a unit process and data are prepared for the unit
transformation to the next higher level
Depending on the application, the sequence of steps has more than one level of
recognition and description process
Page 137
Lehrstuhl fr Informatik 4
Kommunikation und verteilte Systeme
Conclusion
Images are represented by...
Resolution (pixel x pixel)
Color depth (bit)
Color model: greyscale, RGB, CYMK
Graphics primitives / vectors in graphics image formats
Lots of image formats, as well lossless as lossy compression schemes
JPEG / JPEG2000 as examples for common formats
Image Synthesis...
Is needed to bring an image to the monitor
Can use dithering to artificially enhance image quality
Image analysis...
Tries to detect patterns in an image representation
Can be used for improving image quality by suppressing noise or rescaling
Can be used for recognize characters / text
Page 139
Grouping
maximal group of events
Formatting
a group
Extracting
properties as size,orientation ...
are detected
Matching
Conditioning
some purification has been done
Meaning detected:
It is letter A
Labelling
Corners structures as corners, edges ...
are detected
Page 138