You are on page 1of 70

IMAGE SCALING ALGORITHMS

By SUMEET SAURAV
INTRODUCTION: THE INTERPOLATION PROBLEM

Assume a unit square between the four nearest pixels.





The target pixel will then be some x fraction and y fraction into
this unit square.
Interpolation problem then consists of finding suitable values
for these fractions.
NO NEW DATA will be added by ANY interpolation
scheme! The best you can do is minimize interpolation
artifacts.
CATEGORIES OF INTERPOLATION METHOD
Interpolation is done by two methods:
By interpolating the pixels based on the characteristics of
local features, such as edge information or neighboring
pixel information.

Based on these differences interpolation method can be
classified into two categories:

A)Adaptive
B)Non-adaptive

Non-adaptive is suitable for real time hardware
implementation.
TYPES OF INTERPOLATION
Nearest Neighbor
Bilinear Interpolation.
Bicubic convolution Interpolation.
Cubic convolution Interpolation
Extended Linear Interpolation
Win Scale
Area Average
Lanczos resampling
Hqx(Pixel art scaling algorithm)
Super scaling
Vectorization

NEAREST NEIGHBOR INTERPOLATION
Every pixel value in the output product is set to the
nearest input pixel value.
Pros:
1)Very simple and fast.
2) No new values are calculated by interpolation
Cons:
1) Some pixels get lost and others are duplicated.
2) Loss of sharpness.
BILINEAR INTERPOLATION
Calculation of the new pixel value is performed by the
weight of the four surrounding pixels.



Pros:
1) Extremas are balanced
2)Image losses sharpness compared to Nearest Neighbor.
Cons:
1) Less contrast compared to Nearest Neighbor.
2) New values are calculated which are not present in the
input product.
CUBIC CONVOLUTION
Calculation of the new pixel value is performed by
weighting the 16 surrounding pixels.
Pros:
1. Extremas are balanced.
2. Image is sharper compared to Bi-linear Interpolation
Cons:
1. Less contrast compared to Nearest Neighbor.
2. New values are calculated which are not present in the
input product
3. Slow, compared to Nearest Neighbor resampling






WINSCALE:AN IMAGE SCALING ALGORITHM
The winscale algorithm uses an area pixel model and a
domain filtering method for image scaling.
The algorithm uses an area-model rather than a point-model
for a pixel evaluating.
The pixel shapes of the original and the scaled image are
treated as rectangles and pixel intensity is evenly distributed
in rectangle area.
New pixel of the scaled image is obtained by weighted-
averaging the pixel values of the original pixels with area
coverage ratio.
A source image is the original image before scaling up/down.
Target image refers to the scaled image.
The region of the target pixel being calculated currently is
called a filter window.
IMAGE UP-DOWN SCALE
If the filter window is perfectly included in one source
pixel region, sampling has the nearest neighbor property:
the pixel value becomes equal to that of source pixel.
This property makes good high frequency response and
reduces blurring effect.
If the filter window is spread over multiple source pixels,
sampling has the linear property like bilinear: smoothness
is added.
Therefore, winscale inherits two good properties of both
nearest and linear: good high frequency response and
good smoothness.
FUNCTIONAL BLOCK DIAGRAM

LINE BUFFER
PRESCALER
WINFILTER
FILTER
COEFFICIENTS
GENERATOR
FILTER WINDOW
INTERPOLATOR
INPUT IMAGE
OUTPUT
IMAGE
EDGE ORIENTED IMAGE SCALING
Efficient area-pixel scaling technique is used.
A simple edge catching technique is adopted to preserve
the image edge features.






When up-scaled the area of the target pixel is less than that
of the source pixel.
A window is put on the current target pixel to calculate its
estimated luminance value.
The number of source pixels overlapped by the
current target pixel window is one, two, or a
maximum of four.
Let the luminance values of the source pixels
overlapped by the window of the current target pixel
at (k,l) is given by

The estimated value of the current target pixel is
given by

The areas of the overlapped regions can be
calculated as:




Where

[A(m, n), A(m 1, n), A(m, n 1), A(m 1, n 1)]
[ ( , ) top(k, l), right(k, l) top(k, l), left(k, l) bottom(k, l),
right(k, l) bottom(k, l)]
left k l
+ + + +
=

( , ) (m, n)
top(k, l) heightA(m, n)
right(k, l) widthA(m 1, n 1)
bottom(k, l) height(m 1, n 1)
left k l widthA =
=
= + +
= + +
The area-pixel scaling algorithm requires total of 13
addition,8 multiplications and one division to calculate one
target pixel.
The proposed low-complexity algorithm uses the
following set of equations:




where
[A'(m, n), A'(m 1, n), A'(m, n 1), A'(m 1, n 1)]
[left'(k, l) top'(k, l), right'(k, l) top'(k, l),
left'(k, l) bottom'(k, l), right'(k, l) bottom'(k, l)]
+ + + +
=

[ '( , ), top'(k, l), right'(k, l), bottom'(k, l)]
Appr[left(k, l), top(k, l), right(k, l), bottom(k, l)]
left k l
=
Here a low-cost edge-catching technique is employed to
preserve the edge features to obtain better visual quality.
The final areas of the overlapped regions are given as





Where the tuning operator is used to tune the areas of
four overlapped regions according to the edge features
obtained by edge-catching technique.
[ ''( , ), ''( 1), ''( , 1), ''(m 1, n 1)]
([A'(m, n), A'(m 1, n), A'(m, n 1), A'(m 1, n 1)]
A m n A m A m n A + + + +
= I + + + +
I
Assuming the width and height of the target pixel
window are denoted as win(w) and win(h)


Here win(h) and win(w) are determined based on
current magnification factors (mf_w for x-direction
and mf_h for y-direction)
The value of mf_w and mf_h is given by


( ) ( ) Asum win w win h =
_ /
_ /
mf w TW SW
mf h TH SH
=
=
'( , ) ( ) '( , )
'( , ) ( ) '( , )
right k l win w lefr k l
bottom k l win h top k l
=
=
The value of left(k,l) is determined from the fig
'( , ) min(SRCright(m, n) ( , ), ( )) left k l WINleft k l win w =
( , ) ( 1, ) WINleft k l WINleft k l = +
DETERMINATION OF AND
Initially we have








Both Sw and Sh are rounded to integers.
(0, 0) ( ) / 2
SRCright(0, 0) Sw
WINtop(0, 0) ( ) / 2
WINleft Sw WINw
Sh WINh
SRCbtm Sh
=
=
=
=
w I
h I
OVERLAPPING METHOD FOR ROUNDING DOWN
ERROR
Let represent the regulating times required for
each row, thus it can be given as
rw
OVERLAPPING METHOD FOR ROUNDING DOWN
ERROR
THE LOW-COST EDGE-CATCHING TECHNIQUE
Sigmoidal signal is taken as image-edge model for
image scaling.
E(k) represents the luminance value of the pixel at
coordinate k.
e

An Image-Edge Model
Local Characteristics
The estimated value of the pixel to be interpolated is
determined by using linear interpolation.
The interpolated value might not match greatly with
E(k).
To solve this problem the distance s is modified to make
approach E(k) for better estimation.
Here an evaluating parameter L is defined to estimate the
local characteristics of the data in the neighborhood of k.
It is given as

L=0 indicates symmetry, so s is unchanged.
L>0 indicates that variation between E(m+1) and
E(m-1) is quicker than between E(m+2) and E(m).
It means that edge is more homogeneous to the
right side.
L<0 indicates the edge is more homogeneous on
the left side so s must be decreased to obtain a
better estimation
The modified equation for estimating the interpolated
pixel value is given by


Where s is calculated with the following equation



The same concept of 1-D edge-catching technique can be
used in the proposed 2-D scaling processor.
Let be the evaluating parameter to estimate the local
characteristics of the current pixel at the coordinate(k,l).
If top(k,l) is greater than or equal to win(h)/2 it means
that A(m,n) is bigger than or equal to A(m,n+1).
Hence the upper row(n) is more important than the
lower row(n+1) to catch edge features.
Thus is given by


If =0 indicates symmetry, so A(m,n) is unchanged.
If >0 indicates that the variation between E(m+1,n)
and E(m-1,n) is quicker than between E(m+2,n) and
E(m,n).
It means that edge is more homogeneous on the right
hand side so we increase A(m+1,n).


If <0 the edge is more homogeneous on the left-
hand side ,thus we decrease A(m+1,n) to obtain a
better estimate.
Based on the above rules the final areas of the
overlapped region is given by:

Where Ac=A(m,n) if and Ac=A(m+1,n) if

If top(k,l) is less than win(h)/2 ,it means that A(m,n) is
smaller than A(m,n+1).Hence the lower row(n+1) is
more important than the upper row(n) to catch the edge
features.
Thus is given by


0 LAs
0 LA>
The final area of the overlapped regions are given
by




Where Ac=A(m,n+1) if and Ac=A(m+1,n+1) if


0 LA>
0 LAs
VLSI ARCHITECTURE
ARCHITECTURE OF REGISTER BANK
ARCHITECTURE OF AREA GENERATOR
ARCHITECTURE OF EDGE CATCHER
ARCHITECTURE OF AREA TUNER
ARCHITECTURE OF TARGET GENERATOR
AREA-AVERAGE ALGORITHM
The algorithm can be done on horizontal and vertical
dimensions separately with minimal correlation which
helps to exploit the parallel capabilities of the FPGA.
The scaling algorithm can be performed stepwise and
without causing image quality loss.
If we need to reduce a image f(assume f =a/b>1) times,
first we magnify each pixel in the original image b times
and then shrink the enlarged image a times.
ALGORITHM IMPLEMENTATION

RESOURCE AND SYSTEM PERFORMANCE
COMPARISON
BI-CUBIC INTERPOLATION ALGORITHM
This algorithm requires 16 source pixels for getting the
interpolated value of the target pixel.
It preserves fine details of the image .




Where

3 2
2
2
( )
1 (1 ) (1 )
2 (1 2 )
3 (1 )
4 (1 )
l round x a x l
t a a a
t a a
t a a a
t a a
= =
= - -
= +
= - +
=
3 2
2
2
( ) b=y-k
t5=-b (1-b) (1-b)*t1
t6=(1-2 )
7 *(1 )
8 *( 1)
k round y
b b
t b b b
t b b
=
- -
- +
= +
=
HARDWARE ARCHITECTURE FOR BI-CUBIC
INTERPOLATION
The main components of the architecture are:
(1) Input Buffer Bank (2) Output Buffer Bank
(3) Off-Chip RAM (4) Parallel Zoom Processors
(5) Control Unit
(6) Interpolation Coefficient Generator
ZOOM PROCESSOR ARCHITECTURE
The main components of the Zoom Processor are:
(1) Pixel demultiplexer
(2) Pixel Registers
(3) Coefficient Demultiplexer
(4) Coefficient Registers
(5) Temporal Registers
(6) Output Pixel Register

THE CONTROL UNIT
The unit synchronizes external memory, input and
output buffers bank, zoom processors and
interpolation coefficient generator.
For each clock cycle, the Control Unit generates t1,
t2, t3 and t4 (vertical coefficient) and t5, t6, t7 and
t8 (Horizontal coefficients).
In the cycle L+4, the Zoom Processor 1 captures t1,
t2, t3 and t4 and in cycle L+5 output the first result.
The control unit indicates which processor reads
the Horizontal coefficients and when all the Zoom
Processors has given one result.
PIPELINE SCHEME FOR ZOOM PROCESSOR
INTERPOLATION COEFFICIENT GENERATOR
The Interpolation Coefficient Generator generates
dynamically the 4 by 4 input mask that will be used
by the Zoom processors to obtain the output pixels.
The Interpolation Coefficient Generator gives a
result each clock cycle after 5 latency cycles.
LINEAR SPLINE INTERPOLATION

Given , fit linear
splines to the data.
(X0, Y0), (X1, Y1), (X3, Y3)..........(Xn, Yn)
), (
) ( ) (
) ( ) (
0
0 1
0 1
0
x x
x x
x f x f
x f x f

+ =
), (
) ( ) (
) (
1
1 2
1 2
1
x x
x x
x f x f
x f

+ =
), (
) ( ) (
) (
1
1
1
1

+ =
n
n n
n n
n
x x
x x
x f x f
x f
1 0
x x x s s
2 1
x x x s s
n n
x x x s s
1
QUADRATIC SPLINE
In these splines, a quadratic polynomial
approximates the data between two consecutive
data points.
Given fit quadratic splines
through the data.
( ) ( ) ( ) ( )
n n n n
y x y x y x y x , , , ,......, , , ,
1 1 1 1 0 0
, ) (
1 1
2
1
c x b x a x f + + =
,
2 2
2
2
c x b x a + + =
,
2
n n n
c x b x a + + =
1 0
x x x s s
2 1
x x x s s
n n
x x x s s
1
TWO-POINT CUBIC SPLINE INTERPOLATION
The most widely used scaling algorithms in usually
occasion are nearest neighbor, bilinear, and bicubic.
Nearest neighbor and bilinear interpolation are simplest
and fastest, but the images come from them are either
blocky or over-smoothed.
The bicubic can get a better image quality than them but
it is more complicated because it needs a 4x4 neighbor.
The nearest neighbor has good high frequency response
and can preserved more edge information of the original
image. But it degrades image quality due to aliasing.
The weakness of bilinear and bicubic is blur effect
causing bad high frequency response.

The proposed algorithm has a good high frequency
response approximated to nearest neighbor and it
is as simple and fast as the bilinear.
It can get an image quality as good as the bicubic.
Interpolation Kernel Curve
EXTENDED |LINEAR INTERPOLATION
The kernel of the linear interpolation is built up of
first-order polynomials and the weighting coefficient
is given by the relation:



The kernel of the cubic convolution interpolation is
built up of third-order polynomials whose weighting
coefficient is given by

KEYS AND BI-CUBIC WEIGHTING COEFFICIENT.
Key determined the value of c=-0.5.




Bi-cubic interpolation uses the constant c=-1 and
the weighting coefficient of the bi-cubic interpolation
is given by
DECOMPOSITION OF THE 2-D BI-CUBIC

THE KERNEL OF EXTENDED LINEAR
INTERPOLATION
It has low complexity as well as it improves the
quality of the linear interpolation.
Here the number of sampling points of the third
order polynomial interpolation is used.
The weighting coefficients are given by



DERIVATION OF EL KERNEL FROM BI-CUBIC
Using the coefficient of the Bi-cubic weighting
coefficient we have



After certain manipulation the above equation can
be modified and the required weighting coefficient
is given by



Where is called the sharpness factor.
|
SHARPNESS FACTOR DECISION

COORDINATE ORIENTATION AND THE INTERVAL
CALCULATION
Here the origin of a source pixel in source
coordinates is (0.5,0.5),although its location in the
memory is at(1,1).
The coordinate of the interpolated pixel in the x-
direction and y-direction is
The vertical coordinate of the interpolated pixel(Ym)
gives the row address of the source pixel which is
determined as follows:



Thereafter the address neighboring pixels are
determined as follows:

Similarly column address can be calculated by
using Xm of the interpolated pixel.
The distance in vertical and horizontal direction is
given by:

REDUCTION OF THE COMPUTATION OF THE
WEIGHTING COEFFICIENT

Vertical Weighting Coefficients
Horizontal Weighting Coefficients
DECOMPOSITION OF 2-D 4X4 INTERPOLATION
Vertical interpolation is performed first and four
virtual pixels are calculated using four 1-D
interpolation. The generated virtual pixel is given
by:


Now from the vertically interpolated virtual pixels
actual interpolated pixel is obtained by horizontal
interpolation using one 1-D horizontal interpolation.
The interpolated pixel is given by:
All the interpolated points within the same row of Q
have the same vertical weighting coefficients.

These same vertical weighting coefficients are
calculated only one time for each row and then
proceeds the calculations of horizontal weighting
coefficients of each interpolated pixel in that row.

The number of memory access time is given by
THE PROPOSED HARDWARE
THE COORDINATE CALCULATION UNIT
THE WEIGHTING COEFFICIENT GENERATOR
The most computation effort in extended linear
interpolation is the calculation of interpolation
weighting coefficients.
THE VERTICAL AND HORIZONTAL INTERPOLATION
UNIT

THE VIRTUAL PIXEL BUFFER
The scale ration determine both input and output data
rates of this buffer.
While scaling up the data rate of vertical interpolator is
less than that of the horizontal interpolator in a period of
time.
During scaling down the converse case happens.

You might also like