Chen 2015

This article has been accepted for publication in a future issue of this journal, but has not been
fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TCSVT.2015.2473255, IEEE Transactions on Circuits and Systems for Video Technology
1
Algorithm and VLSI Architecture of Edge-Directed

Image Upscaling for 4K Display System
Qiubo Chen, Hongbin Sun , Member, IEEE, Xuchong Zhang, Huibin Tao, Jie Yang, Jizhong Zhao, Member, IEEE
and Nanning Zheng, Fellow, IEEE
AbstractHigh-quality and cost-efficient image upscaling de- performance requirements at a reasonable implementation cost
sign is very important to many real-time video processing is interesting for industries and academics.
applications, especially when the display panel resolution reaches Although image upscaling is a conventional and hot topic
ultra high definition. Compared with New Edge-Directed In-
terpolation (NEDI) based implicit edge directional upscaling, in the image processing and computer vision community,
explicit methods require less computational resource and more designing high quality upscaling that can achieve real-time
easily reach real-time performance, especially when the required processing within reasonably efficient computational resources
image definition and upscaling ratio are very high. Nevertheless, is still very challenging. Conventional linear image interpola-
investigation of explicit method applications in video processing tion algorithms, such as nearest-neighbor interpolation, linear
systems remains largely missing arguably because it is commonly
believed that explicit edge-directed interpolation tends to intro- interpolation and cubic convolution interpolation [2], generally
duce unexpected artifacts because of inaccurate detection and use constant convolution kernels for an entire image, and the
hence its image quality is relatively poor. This paper proposes value of each new pixel is obtained by computing a linear
an explicit edge-directed adaptive interpolation method that combination of the original neighboring pixel values. Given
leverages more sophisticated edge detection and orientation esti- that these methods are computationally efficient and can easily
mation algorithms to avoid misinterpolation, thereby providing
similar or even better image quality than those with implicit achieve real-time performance, they have been widely used
methods. Targeting for the real-time 4K video display system, and implemented as VLSI circuits in many applications [3]
the proposed edge-directed image upscaling algorithm is further [7]. The variations of these methods are also exploited to
implemented with an efficient VLSI architecture. Experimental further reduce VLSI implementation cost [8], [9]. However,
results demonstrate that the proposed interpolation algorithm these methods tend to smooth details and may incur unex-
outperforms previous explicit and implicit edge-directed methods
in both objective and subjective tests. The presented VLSI pected image artifacts, such as blurring or zigzag on the edge
implementation further demonstrates that the maximum output region, when the scaling ratio is increased to a certain level.
video sequence of the proposed interpolation method can reach Therefore, it is natural to leverage more advanced and edge-
up to 4K2K@60Hz with a reasonable hardware cost. directed interpolation methods to adapt to local image features,
Index TermsImage upscaling, adaptive interpolation, real- which solves the problem introduced by linear interpolation,
time, VLSI architecture. especially for the edge region.
Several explicit edge-directed interpolation methods [10]
[13] are presented initially. The underlying idea of this kind of
I. I NTRODUCTION edge-directed interpolation is first to explicitly estimate edge
orientation and then accordingly tune the function and co-
W ITH the rapid development of flat panel display (F-
PD) technology, display panel resolution has steadily
increased from high definition (19201080) to ultra high
efficients of the following interpolation. However, previously
proposed explicit methods only quantize edge orientation into
a very finite number of choices (e.g. horizontal, vertical or
definition (4K2K) in various consumer electronic products,
diagonal) and hence the quality improvement is quite limited.
including digital TV and personal mobile devices [1]. Howev-
More importantly, the quality of output image interpolated by
er, the definition of video or image sources has not increased
these explicit methods is significantly affected by the accuracy
at the same pace. Video materials to be displayed may be
of edge pixel detection. Any inaccurate detection, either on the
standard definition or even lower. This inevitably makes the
edge pixel or the orientation of the pixel, will inevitably incur
requirement for high quality image upscaling become increas-
obvious artifacts. As previously proposed explicit methods
ingly stringent, to achieve better user perceptual quality in the
have not effectively addressed the above two issues, the
case of high scaling ratio. More importantly, as these video
interpolated image quality is actually unsatisfied, and may
display systems often have hard real-time requirement and are
be even worse than that of conventional linear interpolation.
extremely cost sensitive, meeting both quality and real-time
This issue causes many researchers tend to easily preclude the
possibility of using explicit edge-directed interpolation, even
Q. Chen, H. Sun, X. Zhang, H. Tao, J. Yang, J. Zhao and N. Zheng are with
the Institute of Artificial Intelligence and Robotics, Xian Jiaotong University, though such interpolation could easily reach real-time perfor-
Xian, Shaanxi, P.R. China, 710049. mance. Therefore, many researchers turn to exploit implicit
Corresponding Author: Hongbin Sun, hsun@mail.xjtu.edu.cn.
edge-directed interpolation methods.
Copyright (c) 2015 IEEE. Personal use of this material is permitted.
However, permission to use this material for any other purposes must be One of the most well known and widely used implicit
obtained from the IEEE by sending an email to pubs-permissions@ieee.org. algorithms is called new edge-directed interpolation (NEDI)
1051-8215 (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
2
[14], which performs good subjective quality by estimating signed to distinguish to-be-interpolated pixels that belong to
and using local covariance coefficients to adapt to the follow- regular edge from others. Finally, to-be-interpolated pixels that
ing interpolation. Many other works have been proposed to belong to regular edge are interpolated by adaptively using
further improve the quality and computational efficiency of different edge directional interpolation kernels, while other
the NEDI method [15], [16]. Nevertheless, these NEDI-based pixels are interpolated using conventional bicubic interpola-
methods are inherently computationally intensive, and hence tion. Furthermore, the proposed algorithm is implemented as
are very difficult to achieve real-time performance. There- a VLSI circuit for real-time 4K video display system, and
fore, reference [17] further presents an iterative curvature- several efficient techniques are exploited to reduce the overall
based interpolation (ICBI) that can avoid artifacts and reduce expense. Experimental results demonstrate that the proposed
computational complexity incurred by NEDI-based methods, explicit edge-directed interpolation outperforms previous ex-
thereby obtaining real-time and high quality image upscaling. plicit and implicit edge-directed methods in both objective
However, real-time implementation of the ICBI method can and subjective tests. Moreover, the proposed method is much
only obtain 4 enlargement of 256 256 color images under more efficient than implicit edge-directed interpolations in
a GPU platform with Compute Unified Device Architecture terms of real-time performance. The presented VLSI circuit
(CUDA) technology, which is very limited and too expensive design further demonstrates that the maximum output video
for most real-time applications. Therefore, balancing between sequence after interpolation via the proposed method can reach
image quality and computational complexity is crucial for up to 4K2K@60Hz, while its hardware cost only takes less
edge-directed image interpolation design, especially in real- than 505 K logic gates and 52 KB memories. The evaluation
time applications. results clearly suggest that the proposed explicit edge-directed
Recently, many super resolution works have also been interpolation is a high quality and cost efficient solution to
proposed to address the image upscaling problem [18][21]. various real-time applications.
Despite the high visual quality, these single-frame or multi- In particular, this work makes the following contributions:
frame super resolution approaches require much more ad- We propose the use of regular edge to significantly
vanced computational capabilities, even compared with implic- improve the robustness of edge detection and orienta-
it edge-directed interpolation. Thus, real-time implementation tion classification in explicit edge-directed interpolation,
of these super resolution approaches and their application thereby enabling more interpolation orientations and pro-
to cost-sensitive display systems are still very challenging. viding much better image quality in the case of high
More importantly, the effectiveness of these super resolution upscaling ratio.
approaches is mostly based on certain assumptions that are We further implement the proposed interpolation with
very difficult to guarantee in practice. For example, multi- a fully pipelined VLSI architecture, which is VESA-
frame super-resolution methods, such as that in [21], can gen- compatible and synchronized with the pixel clock of
erate high resolution output by making use of multiple input output video, and hence can be seamlessly integrated into
images that portray the same scene, assuming the presence of 4K display systems with a relatively very low hardware
sub-pixel shift among the images. Actually, its applicability cost.
to motion pictures is generally limited by the inability of Based on extensive analysis and evaluation among ex-
achieving precise motion tracking of individual objects [22]. plicit and implicit interpolations, as well as super reso-
The image reconstruction of single-frame super resolution also lution approaches, we demonstrate that the explicit edge-
usually relies on an accurate blur model that is very difficult directed interpolation is still a viable and cost-efficient
to estimate in a practical display system [23]. Therefore, the solution for high-quality 4K display systems. This result
practical use of super resolution approaches in display systems will motivate other researchers to exploit more efficient
remains unready unless these critical issues are resolved. solutions to further improve the image upscaling quality
This paper re-visits the explicit edge-directed interpolation for 4K display systems.
and tries to contend that, by employing a more sophisticat- The remainder of this paper is organized as follows: Sec-
ed edge detection and orientation estimation, explicit edge- tion II describes the proposed explicit edge-directed adaptive
directed interpolation is also possible to achieve similar or interpolation. Section III discusses the VLSI architecture of
even better image quality compared with NEDI-based implicit the proposed algorithm. Section IV presents the experimental
edge-directed methods. Moreover, using explicit edge-directed results. Finally, the conclusion is drawn in Section V.
interpolation can significantly improve the computational ef-
ficiency and reduce the implementation cost of interpolation II. P ROPOSED E XPLICIT E DGE -D IRECTED A DAPTIVE
design, thereby making it feasible to realize high image upscal- I NTERPOLATION
ing ratio of high-definition images in real-time applications. In This section presents the proposed edge-directed adaptive
the proposed algorithm, to avoid misinterpolation incurred by interpolation that can avoid artifacts incurred by false or
inaccurate edge detection in previous explicit methods, edge inaccurate edge detection in previous explicit methods, thereby
directional interpolation is employed only to pixels that belong obtaining high quality interpolated images with low computa-
to regular edge. In this paper, regular edge is defined tional complexity. Fig. 1 shows an overview of the proposed
as the edge where all of its neighboring edge pixels have a algorithm, which can be briefly summarized as follows:
similar orientation angle. Adaptive threshold edge detection, First, edge detection is employed to identify edge pixels
orientation classification and regular edge estimation are de- and their orientations. To improve detection robustness,
3 3
To-be-interpolated
processes,which
processes, whichonlyonlyclassify
classifythe thecenter
centerofofedgesedgesas as
edgeedge
Original image
pixel pixels. Edge
pixels. Edgedetection
detectionininthis thiswork
workaimsaimstotoidentify
identifyallallthethe
pixelsthat
pixels thatare
areassociated
associatedtotothe theedge.
edge.All Allthethedetected
detectededge edge
Adaptive threshold Construct pixelsare
pixels arenecessary
necessaryfor forthe
thefollowing
followingorientation
orientation classification
classication
edge detection estimation window and regular
and regular edge
edge estimation.
estimation.For Forthetherst
firststep,
step,horizontal
horizontal
andvertical
and verticalgradients
gradientsare arecalculated
calculatedusing
usingthetheSobel
Sobel operator,
operator,
Orientation Edge pixels & Regular edge
classification their orientations estimation
representedasasGGx xand
represented andGG , foreach
y ,yfor eachpixel
pixelinin
thethe original
original image,
image,
asasillustrated
illustratedininthe
thefollowing.
following.
No Belongs to (i,j)j)==ff(i(i++1,1,j j++1)1)++2 2f (i
GGxx(i, f (i++1,1,
j)j)++f (i ++
f (i j
1, 1, j
1) 1)
regular edge ff(i(i1,1,j j++1)1)2 2f (i
f (i1,1,
j)j)
f (if (i j
1, 1, 1) 1)
j
Yes (i,j)j)==ff(i(i++1,1,j j1)1)++2 2f (i,
GGyy(i, j
f (i, j1)1)++f (i
f (i j
1, 1, 1) 1)
j
Bicubic Adaptive kernel ff(i(i++1,1,j j++1)1)2 2f (i, j+
1)1)
f (if (i 1, 1,
j+ 1) 1)
interpolation interpolation
f (i, j+ j+
(1)(1)
where
whereff(i,(i,j)j)represents
representsthe theimage
imageintensity ofof
intensity pixel
pixel(i,(i, The
j).j). The
Interpolated pixel
gradient magnitude || ~
G(i, j)|| and edge orientation
gradient magnitude ||G(i, j)|| and edge orientation (i, j) (i, j) of of
pixel (i,j)j)can
pixel(i, canbebecalculated
calculatedasasfollows.
follows.
Fig.1.1. Basic
Fig. Basic process
process flow
ow of
of the
the proposed
proposed image
image interpolation.
interpolation.
||||G(i,
~ j)||
G(i, j)||==|G|Gx (i, j)| + |Gy (i,
x (i, j)| + |G
j)|
y (i, j)| (2)(2)
1
(i, j) = tan
(i, j) = tan (G 1(G (i, j)/G
x (i, j)/G (i,
y (i, j))j))
x y
the proposed
the proposed edge
edge detection
detection employs
employs an an adaptive
adaptive thresh-
thresh-
old that
that varies
varies with
with the
the surrounding
surrounding local The
The gradient
gradientmagnitude
magnitudeofofeach eachpixel
pixelis isthen
thencompared
compared
old local features.
features. with a threshold to determine whether the corresponding
The detected edge orientation is then further classied
The detected edge orientation is then further classified
with a threshold to determine whether the corresponding
into eight
eight orientation
orientation angle
angle regions pixel
pixel belongs
belongstotothe theedgeedgeorornot.
not.AsAsmentioned
mentionedabove, above,anan
into regions that
that are
are dened
defined adaptive threshold is employed instead of a xed threshold.
according to the location grid of the image pixels,
according to the location grid of the image pixels, toto adaptive threshold is employed instead of a fixed threshold.
The adaptive threshold for pixel (i, j) is dened as the average
improve the
improve the efficiency
efciency andand robustness
robustness of of the
the following
following The adaptive threshold for pixel (i, j) is defined as the average
estimation and interpolation process. gradient magnitude of all neighboring pixels in a 33 window.
estimation and interpolation process. gradient magnitude of all neighboring pixels in a 33 window.
Regular edge estimation is leveraged to recognize to- i+1,j+1

Regular edge estimation is leveraged to recognize to- i+1,j+1
X ||G(m,
be-interpolated pixels that belong to regular edge and Tadaptive temp (i, j) = ~ n)||/9 (3)
be-interpolated pixels that belong to regular edge and Tadaptive temp (i, j) = ||G(m, n)||/9 (3)
further estimate the edge orientations of these pixels. m=i1,n=j1
further estimate the edge orientations of these pixels. m=i1,n=j1
The accuracy of regular edge estimation is critical to Moreover, the simple use of average gradient magnitude
The accuracy of regular edge estimation is critical to Moreover, the simple use of average gradient magnitude
image quality provided by the following adaptive kernel makes the adaptive threshold highly sensitive to images with
image quality provided by the following adaptive kernel
interpolation.
interpolation. a at orthe
makes adaptiveregion,
textureless threshold wherehighlythe sensitive to images with
gradient magnitude of
Finally, the interpolated pixel is calculated using different a flat or textureless region, where the
each pixel is almost zero. Therefore, another minimum gradient magnitude
value of
Finally, the interpolated pixel is calculated using different
interpolation methods. For those to-be-interpolated pixels
interpolation methods. ofeach pixel threshold
adaptive is almostTzero.isTherefore,
set to avoid another minimum
ineffective detectionvalue
regular For those
theto-be-interpolated
interpolated pixelspixels
min
that belong to edge, are of adaptive threshold T is set to avoid
in a at or textureless region. The nal adaptive threshold
min ineffective detection
that
calculated by adaptively choosing interpolated
belong to regular edge, the pixels are
different interpolation isindened
a flat oras textureless
(4), where the region. The final
empirical valueadaptive
of Tminthreshold
is set
calculated by adaptively
to theirchoosing different interpolation
kernels according edge orientations. Otherwise, as 30. Please note that Tmin is only used toofcompensate
is defined as (4), where the empirical value Tmin is set
kernels according to their edge orientations.
the interpolated pixels are calculated by using Otherwise,
conven- as T 30. Please note that T or textureless is only used to compensate
for adaptive temp in a atmin region, and the
the interpolated pixels are calculated by using conven-
tional bicubic interpolation. for
following experimental results have demonstrated that theand
T adaptive temp in a flat or textureless region, nalthe
tional bicubic interpolation.
performance of the proposed interpolation is insensitive final
following experimental results have demonstrated that the to
A. Adaptive Threshold Edge Detection performance
variations of the
of Tmin . proposed interpolation is insensitive to
A. Adaptive Threshold Edge Detection variations of Tmin .
Edge detection is critical to the performance of explicit Tadaptive (i, j) = max(Tmin , Tadaptive temp (i, j)) (4)
Edge detection
edge-directed image is interpolation,
critical to the performance
since of explicit
false or inaccurate de- Tadaptive (i, j) = max(Tmin , Tadaptive temp (i, j)) (4)
edge-directed image interpolation, since false or inaccurate
tection may directly result in a wrong choice on interpolation de- Finally, edge pixel is marked to those pixels whose gradi-
tection may
function, directly
leading to result in aimage
annoying wrongartifacts.
choice onEdge interpolation
detection ent Finally,
magnitude edgearepixel
beyondis marked
the adaptive to those pixels whose
threshold. The corre-gradi-
function, leading
in previous to methods
explicit annoyingtypically
image artifacts.
processesEdge image detection
gradient magnitude
sponding are beyond
orientation (i, j) isthealso
adaptive threshold. The corre-
recorded.
inents
previous explicit
with a xed methods
threshold, typically
which processes
inevitably suffersimage gradi-
from either sponding orientation (i, j) is also recorded.
j)|| > Tadaptive (i, j)
1 , ||G(i,
incomplete
ents edgethreshold,
with a fixed detection which
or poorinevitably
noise immunity.
suffers from Thus,either
the Edge pixel(i, j) = ~ j)||
G(i,
||
0 1, , ||G(i, j)||>Tadaptive
Tadaptive(i,(i,
j) j)
image quality
incomplete edgeof detection
previous explicit
or pooredge-directed
noise immunity.interpolations
Thus, the Edge pixel(i, j) =
0 , ||G(i, ~ j)|| Tadaptive (i, j) (5)
is relatively
image quality poor. This work
of previous proposes
explicit to employ
edge-directed adaptive
interpolations
(5)
isthreshold edge
relatively detection,
poor. This which can be dynamically
work proposes to employ congured
adaptive
accordingedge
threshold to the local image
detection, which canfeature, thereby signicantly
be dynamically configured B. Orientation Classication
improving tothetheaccuracy
according and robustness
local image of edgesignificantly
feature, thereby detection. B.AsOrientation
described Classification
in the preceding section, the interpolation
Please notethethat
improving the main
accuracy andreason that this
robustness of work does not
edge detection. kernelAs and orientation
described of each
in the to-be-interpolated
preceding section, thepixel depend
interpolation
directlynote
Please use the
thataccurate
the main Canny edge that
reason detection
this approach
work does is due
not on the neighboring
kernel and orientationedgeof pixels and their edge orientations.
each to-be-interpolated pixel depend
to its non-maximum
directly use the accurate suppression
Canny edge anddetection
hysteresis thresholding
approach is due During
on the the edge detection
neighboring process,and
edge pixels precise
their edge
edge orientation
orientations.
to its non-maximum suppression and hysteresis thresholding During the edge detection process, precise edge orientation
4
44
4
Extended
Extended Extended
Extended Extended
Extended
region4
region4
Extended region5
region5
Extended region3
region3
Extended
Extended region4 Extended region5 region3
Extended Extended
region6
region6
Extended region2
region2
Extended
region6 region2 Extended
Extended Extended
Extended
region7
region7
Extended region1
region1
Extended
4545 region7 region1

45
Extended Extended
Extended Extended
region0
region0
Extended region0
region0
Extended
region0 region0
Extended
Extended Extended
Extended
region1
Extended
region1 region7
Extended
region7
Extended
Extended Extended
Extended region1 region7
region2
Extended
region2 Extended
region6
region6
region2 region6
Extended
Extended Extended
Extended Extended
Extended
Extended
region4
region4 Extended
region3
region3 Extended
region5
region5
region4 region3 region5
(a) (a) (b)(b)
(b)
(a) (b)
Fig.Fig.
Fig. 3. Orientation
3.3.3. Orientation
Orientation classication
classication
classification for
for for eight
eight
eight extended
extended
extended angle
angle
angle regions:
regions:
regions: (a)(a)
(a) extended
extended
extended angle
angle
angle region{0,{0,2, 2,4,4,6},
region
region {0, 2, 4, 6}, 6},and
and(b)(b)extended
and (b) extended angleregion
angleregion
angle
extended {1,3,3,5,5,7}.
{1, 3, 5,7}.
{1,
region Each
Each
7}. Each
Fig.extended
extended Orientation
angle
angle classication
region
region takes
takes 4545 for
. eight extended angle regions: (a) extended angle region {0, 2, 4, 6}, and (b) extended angle region {1, 3, 5, 7}. Each
extended angle region takes
extended angle region takes 45 . 45 ..
region4:
region5: region4: region3: regionisisclassied
region classiedinto intothat thatangle
angleregion, region,and andisisrecorded
recordedwith with
region5:
region5: -1 region4:
(2) 9090
region3:
region3: -1 region
region is is classified
classied intointo thatthat angle angle
region, region,andvalue.and
is is recorded
recorded with with
180-tan
-1
tan-1tan
-1
(2)(2) the the region
region number
number instead
instead of its precise
ofof ofitsitsprecise value.
180-tan -1 (2)
180-tan (2) 90 tan (2) the the region
region number
number insteadinstead its precise precise value. value.
region6:
region6: region2:
region2: In Inaddition
addition to tothe theeight eight angle
angle regions,
regions, eightextended
eight extendedangle angle
In In addition
addition to the eight angle regions, eighteight extended angleangle
areto the eight angle regions, extended
region6: -1 region2: -1
180-tan
180-tan -1 (1) tan-1tan
-1
(1)(1) regions
regions are further
further dened
dened fororientation
for orientation classication,
classication, asas
180-tan-1(1) regions inare further defined for for orientation classification, mayas
tan (1) regions
shownare further dened orientation classication,
orientationas
(1)
region1: shown in Fig.
Fig. 3. 3.ThisThis is isbecause
because that
that thetheedgeedge orientation may
shown
shown in Fig. 3. This is because that that the edge orientation may may
locateinin Fig. 3. This is region
because thestrict
edge orientation
region7:
region7: region1:
region7:
180-tan -1
(1/2)
region1:
tan-1tan
-1
-1
(1/2)
locate inthe themarginal
marginal region andandthe the strict region
region division
division
(1/2) locate in in the marginal region and the the strict region division
-1
180-tan-1(1/2)
180-tan (1/2) tan (1/2) locate
has poor the
tolerancemarginal for region
this and
situation. strict
Whereas, region
the division
denition
has
has has poor
poorpoor tolerance
tolerance
tolerance forfor this
this
for situation.
situation.
this situation. Whereas,
Whereas, Whereas, thethe denition
denition
the definition
region0: ofadditional
additionalextended extendedregion regionisisbenecial benecial toto the the following
following
region0 region0:
region0: ofof additional
ofregular
additional extended
extended region region is benecial
is beneficial to the tothefollowing
theorientation
following
region0
region0 0
regularedge edge
edgeestimation. estimation.
estimation.Fig. Fig.
Fig.3(a) 3(a) illustrates
3(a)illustrates
illustrates
0 0
regular
regular edge estimation. Fig. 3(a) illustrates thethe the
orientation
orientation
orientation
classicationfor
classication forthe theextended
extendedangle angleregion region{0, {0,2,2,4,4, 6} 6} andand
classication
classification forfor the the extended
extended angle angleregion region{0, 2, {0,4, 2,6}3,
4,and6}7}. and
Fig.
Fig.3(b) 3(b) illustrates
3(b)illustrates
illustratesthetheextended the extended
extendedangle angle
angle region
region {1, 5,
region1 Fig.Fig. 3(b) illustrates the extended angleregion region{1,{1, 3,{1,3,
5, 3,5,
7}.5,7}.7}.
region1
region7
EachEachextended
extendedangle angleregion regionuses uses the the same same central central line as as
region1 region7
region7 Each extended angle region uses the same central lineline as
Each extended angle
its corresponding angleregion region uses does,the while same centralis line
its range largeras
itsitscorresponding
corresponding angleangleregion regiondoes, does, whilewhile its its
rangerange is larger
is larger
region2 region6 its
andcorresponding
takes 45 . Thus, angleedge region does, while
orientation that its does range not is larger
belong
region2
region2 region6
region6 andtakes
and takes4545 . .Thus,
Thus,edge edgeorientation
orientation thatthat doesdoes notnot belong
belong
and
to
totoone
takes
one angle 45 . Thus,
region may edge locateorientation
in its that
extended does angle not belong
region.
region3
region3 region4 region5 region5 oneangleangleregion regionmay maylocate locatein in itsits extended
extended angleangle region.
region.
region4
region4 region5
Two toTwo
Two one adjacent
angleextended
adjacent
adjacent
extended
region
extended may
angle
angle
locate
angle regions
regions
in its
regions
are overlapped
areextended
areoverlapped
overlapped angle with
with
region.
with
Fig. 2. Orientation classication for eight angle regions. each adjacent
Two other, which extendedmeans angle that one edge orientation
regions are overlapped is located with
Fig.2.2.
Fig.
Fig. 2. Orientation
Orientation classification
Orientation classication for for eight
eight angle
angleregions.
angle regions.
regions.
each
eachother,other,which whichmeans meansthat thatone one edge edge orientation
orientation is located
is located
into
into
each two
other, extended
which angle
means regions
that onesimultaneously.
edge orientation The extensive
is located
intotwo twoextended
extendedangle angle regions
regions simultaneously.
simultaneously. TheThe extensive
extensive
explanation
into
explanation two extendedtototheto the use of
angle regionsorientation
simultaneously.classication The for both
extensive
explanation theuse useofoforientation
orientation classication
classication forfor both both
for each edge pixel is obtained, which can be used to direct angle
explanation regions to and
the extended
use of angle
orientation regions are
classification given in
for the
both
for each
for each edge edge pixel is obtained, obtained, which which can can be be used
usedtotodirect direct angle angleregions
following
regionsand andextended
subsection.
extendedangle angleregions regions areare givengiven in inthethe
for each
the
thefollowing edge
following
following adaptive pixel
adaptiveis obtained,
interpolation,
adaptive interpolation, which can
similar be
to used
most to direct
previously angle
following regions
subsection. and extended angle regions are given in the
the interpolation, similar similarto tomost
mostpreviously
previously following subsection.
the proposed
following
proposed explicit explicit
adaptive
explicit methods. methods.
interpolation,
methods. However, However, similar interpolation
interpolation to most orientation
previously
orientation following subsection.
proposed
isrelatively
relatively nite
However, interpolation orientation C. Regular Edge Estimation
proposed
isis neighboring
explicit
relatively nite nite andandis
methods.
and is
islimited
limitedtointerpolation
However,
limited to
to
the
the
the location
location
location
orientation
grid grid
gridorien-ofofofC.C.RegularRegularEdge EdgeEstimation
Estimation
isneighboring
relatively pixels. pixels.
finite and Thus,
Thus,isdirectly directly
limitedusing using
to the precise
preciselocationedge edge grid
orien- of C. Regular
The Edgedifference
underlying Estimation
difference between the proposed method
neighboring pixels. Thus, directly using precise edge orien- The
The underlying
underlying difference between
between thethe proposed
proposed methodmethodis
tation
neighboring
tation is is
very very
pixels. inefcient
inefcient Thus, and and
directly
may may even
using
even confuse
precise
confuse the the
edge following
following orien- and previously proposed explicit edge-directed interpolation
tation is very inefcient and may even confuse theedgefollowing and The
previouslyunderlying proposed difference
explicit between
edge-directed the proposed
interpolation method
is is
tationprocess.
process. is very Therefore,
Therefore,inefficient the the edge
and
edge mayorientation
even
orientation of
confuse
of each each the
edge following
pixel pixelis is and thatpreviously
adaptive proposed
kernel explicit edge-directed
interpolation is only interpolation
employed to pixels
process. Therefore, the edge orientation ofsimplify
each edge that adaptive
previously kernel interpolation
pixel is that adaptive kernel interpolation is only employed to pixels is
and proposed explicit is only employed
edge-directed to pixels
interpolation
classied
process.
classied Therefore,
into into several
several the angle
edge
angle regions
orientation
regions to to of
simplify each the the
edge following
followingpixel is that that
belongbelong tokernel
totoregular regularinterpolation
edge. Whether aonly
to-be-interpolated pixel
classied
interpolation into severalorientation angleestimation
regions toofsimplify each the following
to-be-interpolated
that
thatbelongs
adaptive
belong to regularedge.
regular edgeedge. Whether
or Whether
not is
a isto-be-interpolated
employedpixel
a to-be-interpolated
judged by leveraging
topixel
pixels
regular
classified
interpolation
interpolation into several
orientation angle regions
estimation of to simplify
each the following
to-be-interpolated belongs
that to
belong
orientation estimation of each to-be-interpolated belongs to regular edge or not is judged by leveraging regular regularto edge
regular or not
edge. is judged
Whether by
a leveraging
to-be-interpolated regular pixel
pixel.
pixel.
interpolation orientation estimation of each to-be-interpolated edge edge estimation.
estimation. Thus, Thus, regularregular edge edge estimation
estimation is is the
the most most
pixel.As shown in Fig. 2, the overall edge orientation is divided edge belongs to
estimation. regular edge
Thus,work, or
regular not is judged
edge estimation by leveraging
is impact
the mostregular
pixel.As shown in Fig. 2, the overall edge orientation is divided critical critical partof of thisthis andand has has a signicant on
As shown in Fig. 2, the overall edge orientation is divided edge part
critical estimation.
part of this
work, regular
Thus,work, and has
a signicant
edge a estimation
signicant
impact is the
impact
on most
intointoeight eightangle angleregions.regions.Please Please note note that that the theeighteight angle anglethe the quality
quality of of the
the interpolated
interpolated image. image. The The estimation
estimation of of on
the the
As eight
into shownangle in Fig. 2, the overall
regions. Please edge notethethat orientation
the eight is dividedangle critical
the quality part of of thethis work,toand
interpolated image.has aThe significant
estimationimpact of theon
regionsareare
regions dened
dened according
according to tothe location location grid grid of
of angle thetheregular regular edge edgerst rst requires
requires construct
to construct the the estimation
estimation windowwindow
into
regions eight
image are angle
pixels, dened regions.
which according
means Please note
to theangles that
location the eight
grid ofcentralthe regularthe quality
edge of
rstintothe interpolated
requires to construct image. The
the estimation estimation
the window of the
image pixels, which means thatthat
thethe angles of the of theeighteight central andand onlyonly takes takes into considerationconsideration the the pixels pixels
inside inside
the window. window.
regions
imagelinesare are
pixels,
are{0{0 defined
which
according
means
1 thattan to the
the angles
1 location
of the
1 grid
eight ,central
of the
regular
and As only
shown edge
takes first requires
into consideration
in Fig. 4, the to construct the
the pixels inside
to-be-interpolated estimation
isthe window
window.
lines , tan, tan
1 (1/2),
(1/2), tan1 1(1),
(1),
tantan 1
(2),(2),
9090 , 180
180

As shown in Fig. 4, the to-be-interpolated pixelpixel represented
is represented
image
linestan1pixels,
1 {0which
are (2), ,
tan
180 1
means 1 that
(1/2),
tan tan
(1), the(1),
180 angles
tan
1 1
tan
1of(1/2)}.
the 90
(2), eight ,
The 180 central
bound- Asand
as only
shown
pi and takes
in 16 Fig. into
nearest4, consideration
the to-be-interpolated
neighboring the
pixels pixels pixel
in inside
the is the window.
represented
original image
tan1 (2), 180 tan
1
1 1 (1), 180 1 (1),
tan 1(1/2)}. The bound- as pi and 16 nearest neighboring pixels in the original image
lines
tan are
(2), {0
180 , tan
tan (1/2), tan
(1),angle
180 regions
tan 1
tan(1/2)}. (2), The, in
90 bound-
180
asAs pi shown
and 16 innearest
Fig. 4, neighboring
the to-be-interpolated pixels in pixel
the is represented
original image
aryary of ofany anytwo two adjacent
adjacent angle regions is is located
located in
the theare are selected to construct
selected to construct the estimation window. the estimation window.
ary
tan 1
of(2),
middle
middle any
and 180
and has

two the
has tan
the
1
adjacent
equal (1),
equal angle
180
angle angle

tan1to(1/2)}.
regions
distance
distance is
to
twotwolocatedThe
central
central bound-
in
lines. the are
lines. as
ThepiThe anddetailed
selected
detailed 16to nearest
construct
processprocess neighboring
ofthe of estimation
the the pixels
proposed
proposed inregular
window.
regular the edgeoriginal
edgeesti-image
esti-
ary
middle of any
Therefore,
Therefore, and two
has
anyany the
edge adjacent
edgeequal angle
angle
orientation
orientation regions
distance
thatthat locates
locates to is
two located
central
in ainspecic
a specic in
lines.
angle the
anglemation areThe selected
mation detailed
algorithm
algorithm to construct
processis described
is described ofthe the estimation
proposed
as Algorithm
as Algorithm window.
regular edge
1, where
1, where vari- esti-
vari-
middle
Therefore, and any has edge the equal angle that
orientation distance
locates tointwo central angle
a specic lines. mation The algorithm
detailed process is described of theas proposed Algorithmregular 1, where edge esti-
vari-
Therefore, any edge orientation that locates in a specific angle mation algorithm is described as Algorithm 1, where vari-
5 55
xi-3
xi-3 xi-2
xi-2 xi-1
xi-1 xi xi xi+1
xi+1 xi+2
xi+2 xi+3
xi+3 xi+4xi+4
P(1,1) P(1,2) P(1,3) P(1,4)
yj-1
yj-1
P(1,1) P(1,2) P(1,3) P(1,4)
P(i-3,j-1)
P(i-3,j-1) P(i+4,j-1)
P(i+4,j-1)
hh
yjyj
v
P(2,1) P(2,2) P(2,3) P(2,4)
v
P(2,1) P(2,2) P(2,3) P(2,4)
pi pi pi
pi
yj+1
yj+1
P(3,1)
P(3,1) P(3,2)
P(3,2) P(3,3)
P(3,3) P(3,4)
P(3,4)
yj+2
yj+2
P(4,1)
P(4,1) P(4,2)
P(4,2) P(4,3)
P(4,3) P(4,4)
P(4,4)
P(i-3,j+2)
P(i-3,j+2) P(i+4,j+2)
P(i+4,j+2)
Fig.
Fig.5.5.
5. Illustration
Illustrationofof
ofsquare
squarebicubic
bicubicinterpolation kernel
interpolation forfor
kernel pixels thatthat
pixels do do
Fig.
not belong Illustration
totoregular edgesquare bicubic
ororbelong interpolation
to to
Class I of kernel
regular for pixels
edge. that do
not belong regular edge belong Class I of regular
not belong to regular edge or belong to Class I of regular edge. edge.
Fig. 4.
Fig.
Fig. 4. Estimation
4. Estimation window
Estimation window
window composed
composed of
composed of 16
of 16 nearest
16 nearestneighboring
nearest neighboringpixels.
neighboring pixels.
pixels.
pixels
pixelsinin
inthetheestimation
estimationwindow
windowshould shouldbelong
belongtoto
tothethe
thesame
same
able edge counts pixels the estimation window should belong same
able
able edge counter
edge counter counts the
counter counts the number
the number of
number of edge
of edge pixels
edge pixelswithin
pixels within
within orientation angle
orientation angle region,
angle region, which
region, which
which isis represented
is represented by
represented by max
by maxmaxori; ori;
the estimation window, array orientation ori;
the estimation window, array region counter[ori] (ori ==
the estimation window, array region
region counter[ori]
counter[ori] (ori
(ori = (3)
(3)the
theedge
edgeorientation
orientationofof
ofallall
allthe
theother
otheredge
edgepixels that
pixels dodo
that notnot
counts the (3) the edge orientation the other edge pixels that do not
000 7)
7) counts
7) the number
counts the number of
number of edge
of edge pixels
edge pixels whose
pixels whose edge
whose edge
edge belong to the region max ori should locate in its extended
belong to to the
the region
region max
max ori should locate
ori should locate in
in its
its extended
extended
orientations belong to
orientations to the corresponding region ori and belong
orientations belongbelong to the the corresponding
corresponding region ori and
region ori and angle region. In other words, the to-be-interpolated pixels are
angle region.
region. In In other
other words,
words, the to-be-interpolated
to-be-interpolated pixels
pixels areare
extended region
extended region counter[ex
counter[ex ori] ori] (ex
(ex ori
ori =
= 00
7)
7) counts
counts angle
removed from regular edge set the when either they do not have
extended region counter[ex ori] (ex ori = 0 7) counts removed from from regular
regular edge
edge setset when
when either
either they
they do
do notnot have
have
the number of
the of edge pixels
pixels whose edge orientations belong removed
enough neighboring edge pixels or the edge orientations of
the number
number of edge edge pixels whosewhose edge
edge orientations
orientations belong
belong enough neighboring
neighboring edge pixels pixels or the the edge
edge orientations
orientations of of
toto the
the corresponding
corresponding extended
extended region
region ex ori. Each edge enough
their neighboring edgeedge pixels are or not concentrated enough.
to the corresponding extended region ex ori. ex ori. Each
Each edge
edge their neighboring
neighboring edge edge pixels
pixels are are not
not concentrated
concentrated enough.
enough.
pixel
pixel is simultaneously associated with two extended angle their
This could effectively avoid false orientation estimation on
pixel is is simultaneously
simultaneously associated
associated withwith two
two extended
extended angle
angle This could
could effectively
effectively avoid
avoid false
false orientation
orientation estimation
estimation on on
regions, as represented
regions, represented by ex ori1 and ex ori2 respec- This
vague edge or complex texture, where adaptive interpolation
regions, as as represented by by ex ori1 and
ex ori1 and ex ori2 respec-
ex ori2 respec- vague edge
edge or or complex
complex texture,
texture, where
where adaptive
adaptive interpolation
interpolation
tively.
tively. It is noted that edge counter, region counter and vague
may easily cause annoying artifacts while conventional linear
tively. It It is
is noted
noted that
that edge counter, region counter and and may easily
easily cause
cause annoying
annoying artifacts
artifacts while
while conventional
conventional linear
linear
extended region counter are reset to 0 for every new to- may
interpolation is more appropriate.
extended
extended region counter are reset to 0 for every new to-
region counter interpolation is is more
more appropriate.
appropriate.
be-interpolated pixel. interpolation
be-interpolated
be-interpolated pixel.pixel.
Algorithm 1 Regular edge estimation algorithm D. Adaptive Kernel Interpolation
Algorithm
Algorithm 11 Regular
Regular edge estimation algorithm D. Adaptive Kernel Interpolation
Input: 16 neighboring pixels in the estimation window of pi In the regular edge estimation process, the to-be-interpolated
Input:
Output:16
Input: neighboring
16Whether
neighboring pixels
pixels toin the
the estimation
in regular edge andwindow
estimation window of
of pi In theareregular edge into
estimation process, two
pi belongs pi
its orientation
Output:
Output: Whether
Whether pi
pi belongs
belongs to
to regular
regular edge
edge and
and its
its orientation
orientation
pixels classied distinguished sets based on
1: for all 16 pixels in the estimation window do
2: for
1:
1: forifallall 16
16 pixels
current pixelin
pixels inisthe
an estimation
the edge pixel window
estimation window
then do
do whether the pixels belong to regular edge. two
pixels are classified
classied into distinguished sets based
To interpolate theon
2:
2:
3: ifif current
retrievepixel
current pixel is
is an
the angle an edge
edge pixel
region and then
pixel then
two extended angle regions pixels that do not belong to regular edge, we simply usethe
whether the pixels belong to regular edge. To interpolate
3:
3: retrieve
retrieve the
the angle
angle region
for the orientation region
of currentand two
two extended
andpixel, extended angle regions
anglethem
and represent regions
as pixels that do
conventional not interpolation
bicubic belong to regular
as shownedge, we simply
in Fig. 5. Let piuse
for
ori,the
for ex orientation
the orientation
ori1, and ex of
of current
current pixel,
pixel, and
and represent
ori2, respectively. represent them
them as
as conventional bicubic interpolation as shown in
represent the to-be-interpolated pixel, P (i+m, j+n) represent Fig. 5. Let pi
4: ori,
edgeex
ori, ex ori1, and
ori1,
counter + +ex
and ori2, respectively.
ex ori2, respectively. represent
the reference thepixels
to-be-interpolated
in the original pixel, P (i+m,
image j+n)
that will represent
participate
4:
4:
5: edge
edge counter
counter +
++ + ++
inthe
thereference pixelsand
in the original image the
thathorizontal
will participate
region counter[ori]
5:
5:
6: region
region
extended counter[ori]
counter[ori]
region counter[ex+++ + ori1] + +
interpolation, m and n represent and
6:
6: extended region
extended counter[ex ori1]+ ++ + in the interpolation,
vertical indices for theand m andpixel
reference n represent
P (i, j) the horizontalthe
respectively, and
7: region counter[ex
region counter[ex ori2]
7:
7:
8: end extended
if
extended region
region counter[ex
counter[ex ori2] + + verticalinterpolation
bicubic indices for the reference
function can pixel P (i, j) respectively,
be illustrated as follows. the
8:
9: end
8: end
end forifif bicubic interpolation function can be illustrated as follows.
9: end
9:
10: nd for
end for
the maximum orientation that the majority of edge pix- pi(x, y) =
10: find
10: the
the maximum
els within
nd estimation
maximum orientation
window belong
orientation that
that theto,majority of edgeit pix-
and represent as pi(x,
2 y)2=
(6)
els
els
max within
ori. estimation
within estimation windowwindow belongbelong to, and represent it as 2 2 P (i + m, j + n) C
X
X m+1 (h) Cn+1 (v) (6)
if edge
11: max
max ori.
ori.counter > 3 && m=1 n=1 P (i + m, j + n) Cm+1 (h) (h) Cn+1 (v) (v)
11: if
11: if edge counter
edge
region counter >
counter[max > 33 && ori]/edge counter > 0.5 &&
&& m=1 n=1
region
extended
region counter[max
region counter[max
counter[max ori]/edge
ori]/edge ori] counter
== edge &&
> 0.5counter where C0 (u) = (u3 + 2 u2 u)/2, C1 (u) = (3 u3
then
extended
extended region
region counter[max
counter[max ori] == edge counter 5where
u2 + C002)/2,
(u) =C(u 33
+ 2 u22 3u)/2, C 1u(u) 2 = (3 u3
+ u)/2, and 3
2 (u) = (3 u + 4 1
12: then
then pi belongs to regular edge and its edge orientation is region 5 u 22
+ 2)/2, C (u) = (3 u 3
3 + 4 u22 + u)/2, and
belongs
ori to
3
C3 (u) = (u u )/2. 2 22 h and v represent the horizontal
12:
12: pi
max
pi belongs to regular
regular edgeedge andand its edge orientation is region C (u) = (u 33
u 22
)/2. h and v represent the horizontal
13: elsemax
max ori ori and33 vertical distance between h pi and vP (i, j) respectively, and
13:
13: elsepi does not belong to regular edge
14: else and vertical
hence h =distancex xibetween and vpi = andyP(i,yj) respectively,
j (the minimum and
15: end
14:
14: piifdoes
pi does notnot belong
belong to to regular
regular edge
edge hence
distance hh = x x and
of neighboringii pixelsvis set as 1).jj Conventional
v = y y (the minimum
15: end
15: end ifif distanceinterpolation
bicubic of neighboring is usedpixels is setthatasdo1).notConventional
for pixels belong to
In the proposed estimation algorithm, the to-be-interpolated bicubicedge
regular interpolation
because these is used for pixels
pixels are eitherthat in
doanotatbelong
region to
In
In the
pixel is proposed
the treated asestimation
proposed a regularalgorithm,
estimation edge pixel the when
to-be-interpolated
it satises orregular edge because
in a complex edge regionthese without
pixels are either in orientation.
a dominant a flat
at region
pixel
all theis treated
following
pixel is treated as as a
three regular edge
requirements pixel when
simultaneously: it satisfies
(1) the
satises or in a complex
Therefore, the use edge region without
of conventional bicubica dominant orientation.
interpolation along
all
numberthe following
of edge
all the following three three
pixels requirements
in its simultaneously:
estimation window (1)
should the
be Therefore,and
horizontal thevertical
use of conventional
orientation can bicubic
safely interpolation
protect the orig-along
above
number a ofcertain
edge
number of edge pixels threshold,
pixels in which
its is set
estimation to 3
window according
should to
be inal feature and
horizontal or texture
verticalduring the interpolation.
orientation While the
can safely protect the use
orig-
our
above following
a certainexperiment;
threshold,
above a certain threshold, (2) more
which is than
set half
to 3 of the
accordingedgeto ofinal
directional
feature or interpolation
texture during tendsthe
to introduce
interpolation.annoying
Whileartifact
the use
our
our following
following experiment;
experiment; (2) more than half of the edge of directional interpolation tends to introduce annoying artifact
6 6
xi-3 xi-2 xxi-1

i-1 xxi i xxi+1
i+1 xxi+2
i+2 xxi+3i+3 xi+4
xi+4 xi-3xi-3 xi-2xi-2 xi-1xi-1 xi xi xi+1xi+1 xi+2xi+2 xi+3xi+3 xi+4xi+4
yj-1
j-1 yj-2
yj-2
P(i-3,j-1)
P(i-3,j-1) P(i+4,j-1)
P(i+4,j-1) P(i-3,j-2)
P(i-3,j-2) P(i+4,j-2)
P(i+4,j-2)
pipixx
yyjj yj-1
yj-1
pi
piyy pipi pix pix
yjyj
yyj+1
j+1 pi pi
piy pi
y

yj+1
yyj+2
j+2 y j+1
P(i-3,j+2) P(i+4,j+2) P(i-3,j+1) P(i+4,j+1)
P(i-3,j+2) P(i+4,j+2) P(i-3,j+1) P(i+4,j+1)
Fig. 6.
Fig.
Fig.
6. Illustration
Illustration of
6. Illustration of parallelogram
of parallelogram bicubic
parallelogram bicubicinterpolation
bicubic interpolationkernel
interpolation kernelfor
kernel for pixels Fig.
forpixels
pixels
7. Illustration of parallelogram bilinear interpolation kernel for pixels
Fig. 7. Illustration of parallelogram bilinear interpolation kernel for pixels
that belong to Class II of regular edge. that belong to Class III of regular edge.
that belong
that belong to
to Class
Class II
II of
of regular
regular edge.
edge. that belong to Class III of regular edge.
pixel pi can be calculated through the following equation:

in this case. Moreover, the conventional bicubic interpolation pixel pi can be calculated through the following equation:
in this case. Moreover, the conventional bicubic interpolation pi(x, y) =
is also referred to as square bicubic interpolation, as its 16 pi(x,y) =
is also referred to as square bicubic interpolation, as its 16
X
neighboring reference pixels are in the square shape as shown P (i + m, j + n) Cm+n+1 (h ) 0 Cn+1 (v ) 0
neighboring reference pixels are in the square shape as shown P (i + m, j + n) Cm+n+1 (h
(h ) Cn+1 (v
(v )
in Fig. 5. P (i+m,j+n)
in Fig. 5. P (i+m,j+n)
(7)
For pixels that belong to regular edge, the adaptive direc- (7)
For pixels
tional that belong
interpolation to regular
is employed to edge, the adaptive
well preserve direc-
the edge where is dened as the pixel set that contains all the
tional interpolation is employed to well preserve the edge wherethat is dened
and texture of the original image. The eight orientation angle pixels belong to theasparallelogram
defined the pixel set that Ccontains
kernel. all the
0 (u), C1 (u),
and texture of the original image. The eight orientation angle pixels that belong to the parallelogram kernel.
regions of regular edge are further classied into three classes, C2 (u) and C3 (u) have the same denition as in (6).0 h C C (u),
and1 (u),
regions of regular edge are further classiedclassified into three classes, 2 (u) and C3 (u) have the same denition
the normalized horizontal and vertical distancesand
as in (6). 0
and each class includes one or two pairs of orientation regions C
v represent definition h
h
and
that are orthogonal to each other. pairs
each class includes one or two TableofI orientation regions represent the normalized
pixel Phorizontal andcanvertical distances
0
demonstrates the v
v
between pi and reference (i, j), which be obtained
that are orthogonal to each other. Table I demonstrates the
details of regular edge classication. The classication is based by calculating the geometric relationship of the pixel grid.obtained
between pi and reference pixel P (i, j), which can be h
details of regular edge
on the means of interpolation classification.
classication. The classification
classication
for the pixels, is based
and each class is by calculating the geometric relationship of the pixel grid. h0
h
and v in Fig. 6 can be calculated as follows.

on the means
associated withofainterpolation for the pixels,
specic interpolation kernel.andByeach class is
adaptively and v v in Fig. 6 can be calculated as follows.
0
associated with a specific
specic interpolation kernel. By adaptively h = distance(P (i, j), pix ) = h + v/ tan = h + v
employing different interpolation kernel to different regular h 0
h = distance(P (i, j), h + v/ v/ tan = h h + v v
employing
edge classes, different interpolationavoid
it can signicantly kernel to different
jagged regular
or zigzagging distance(P (i,pij),
x )pi= )h
y
v = distance(P (i, j), piy ) = v
edge classes, it can significantly
artifacts induced by conventional avoid jagged or zigzagging
signicantly bicubic interpolation along v0 =distance(P (i, j), P (i 1, j + 1)) = v
v v
artifacts
the regularinduced
edge. by Forconventional
pixels whosebicubic interpolation
edge orientations along
belong distance(P (i, j), P (i 1, j + 1)) (8)
the regular edge. For pixels whose edge orientations
to Class I, square bicubic kernel is employed and the interpo- belong (8)
For the to-be-interpolated pixels that belong to Class III,
to Class
lation I, square
is along the bicubic
horizontalkernel
and is employed
vertical and the This
orientations. interpo-
is this For
work theproposes
to-be-interpolated
to employ pixels that belongbilinear
a parallelogram to Class in-III,
lation is along the horizontal and vertical orientations.
coincidentally the same as conventional bicubic interpolation, This is this work kernel.
terpolation proposes Thetoto-be-interpolated
employ a parallelogram bilinear in-
pixel is generated
coincidentally
as illustrated inthe Fig.same
5. as conventional bicubic interpolation, byterpolation
using the kernel. The to-be-interpolated
4 neighboring pixels in the adjacentpixel isparallel-
generated
as illustrated in Fig. 5. by using
ogram array. theClass4 neighboring
III contains pixels
four ininterpolation
the adjacentorienta-
parallel-
tions,
ogram i.e.array.
{tan1 Class
(1/2),IIItan 1
contains
(2), 180
four tan1 (2), 180orienta-
interpolation

TABLE I
C LASSIFICATION FOR I NTERPOLATION OF R EGULAR E DGE .
tan 1
tions,(1/2)},
i.e. {tan and let
1 us tan
(1/2), take the
1
(2),orientation
180 tanof tan
1 1
(2)
(2), 180
TABLE I
C LASSIFICATION FOR I NTERPOLATION OF R EGULAR E DGE .
for
tan example
1
(1/2)}, to demonstrate
and let us the takecalculation of parallelogram
the orientation of tan1 (2)
Interpolation Orientations bilinear
for example interpolation kernel, asthe
to demonstrate shown in Fig. of
calculation 7. parallelogram
The to-be-
Class I {0 , 90Orientations
Interpolation }
interpolated pixel pi cankernel,
bilinear interpolation be calculated
as shown through
in Fig.the7.following
The to-be-
Class II
Class I {tan 1 (1), 180
tan1 (1)}
{0 , 90 } equation:
interpolated pixel pi can be calculated through the following
11 1
Class II
Class III
{tan
{tan (1/2),
(1), 180
180 tan
tan1(1/2),
(1)} equation:
pi(x, y) =
1 (2), 180 11
{tan (1/2), 180 tan (2)}
tan1 tan (1/2),
Class III
tan1 (2), 180 tan1 (2)}
(1pi(x,
v y) )(1
= h )P (i, j) + (1 v )h P (i + 1, j 2)+
0 0 0
v(1 (1
vh
v h
)(1)P(i,
h j + + h
)P1)(i,+j)v (1
v
Pv
(i + 1, j0P
)h
)h 1)+ 1, j 2)+
(i
For to-be-interpolated pixels that belong to Class II, this 0
v (1 h
v 0
h )P (i, j + 1) + v 0
v h 0
h P (i + 1, j 1) (9)
work
Forproposes to employ apixels
to-be-interpolated parallelogram bicubic
that belong to interpolation
Class II, this Similar to the parallelogram bicubic interpolation equation for (9)
kernel, as shown in Fig. 6. The parallelogram bicubic
work proposes to employ a parallelogram bicubic interpolationinterpo-
Class II, h
Similar

and
to the v representbicubic
parallelogram the normalized horizontal
interpolation and for
equation
lation kernel is quite similar to its square bicubic counterpart,
kernel, as shown in Fig. 6. The parallelogram bicubic interpo- vertical distances
Class II, h 0
h and v
between
0 pi and reference pixel P (i,
v represent the normalized horizontal and
j), and
exceptkernel
lation that the
is to-be-interpolated
quite similar to itspixel is generated
square by using
bicubic counterpart, are dened
vertical as follows:
distances between pi and reference pixel P (i, j), and
16 neighboring pixels within the parallelogram array. Class II
except that the to-be-interpolated pixel is generated by using distance(P (i, j), pix )
contains a pair of two orthogonal interpolation orientations, i.e. arehdefined
dened

= as follows: = h
16 neighboring pixels within the parallelogram array. Class II distance(P (i, j), P (i j ) 2))
{tan1 (1), 180 tan1 (1)}, and let us take the orientation
contains a pair of two orthogonal interpolation orientations, i.e. distance(P (i,+j),1,pi
of tan1 (1) for example to demonstrate the calculation of par- h
h
0

==distance(P
x
+=vh (10)
h
{tan1 (1), 180 tan1 (1)}, and let us take the orientation
v distance(P (i,(i,
j),j),
piyP) (i
=+ h 1,j tan
2))
allelogram bicubic interpolation kernel. The to-be-interpolated v= 0 2 h + v (10)
of tan1 (1) for example to demonstrate the calculation of par- v = distance(P (i, j), pi y ) = h h tan + v v
allelogram bicubic interpolation kernel. The to-be-interpolated h + v
= 2 h v
7
7 7
Video data Pixel 1 Pixel

2 2 ... ... Pixel
n n Stage
Video data Pixel 1 Pixel Pixel Stage 1 1 Stage
Stage 2 2 Stage
Stage 3 3 Stage
Stage 4 4 Stage
Stage 5 5
H_active YY 1 clock
H_active 1 clock 11 11 clocks
clocks 6 clocks
6 clocks 4 clocks 16 16
4 clocks clocks
clocks
Hsync
Hsync YY Adaptive
Adaptive
Scanning sequence window threshold Coefficient
Coefficient
Scanning sequence window threshold generation

buffer edge detection generation

buffer edge detection
... ... ...

... ... ...
Interpolation unit
Line buffer
Interpolation unit
Line buffer Regular
Orientation
Orientation Regular Edge
Edge
classification edge
edge
CbCr
classification estimation orientation
CbCr
estimation orientation
Active
Active video
video
Delay
Delay Y selection
Y selection
Line
Line buffer
buffer
CbCr
CbCr
window CbCr
CbCr
Blanking window selection

Blanking buffer selection

buffer
Vsync V_active
Vsync V_active
Fig.
Fig.
Fig. 9.9.9.Pipeline
Pipeline
Pipeline architecture
architecture
architecture of
ofof the
the
the proposed
proposed
proposed edge-directed
edge-directed
edge-directed adaptive
adaptive interpo-
interpo-
Fig.
Fig. 8. 8.Video
Video
Video display
display timing
timing diagram
diagram for VESA
for standard. lation.
lation.
lation.
Fig. 8. display timing diagram for VESA
VESA standard.
standard.
Forthe
For thetheregular
regularedge
regular edgeofofClass ClassIII,III,aa aparallelogram
parallelogrambilinear bilinear (VESA)standard,
(VESA)
(VESA) standard,widely
standard, widelyused
widely usedtoto
used tofacilitate
facilitateconnection
facilitate connectionand
connection and
and
For III, parallelogram bilinear
interpolation
interpolation kernel
kernel is is employed
employed instead
instead ofof the
the parallelogram
parallelogram interactionamong
interaction
interaction amongdifferent
among differentvideo
different videodisplay
video displaymodules.
display modules.The
modules. Thegure
The gure
figure
interpolation kernel is employed instead of the parallelogram
bicubicbecause
bicubic becauseofofthe thefollowing
followingtwo two mainreasons:reasons: (1)asas illustratesthat,
illustrates
illustrates that,aaavideo
that, videoframe
video frameisisiscomposed
frame composedofof
composed ofboth
bothactive
both activevideo
active video
video
bicubic because of the following two mainmain reasons: (1) (1) as
bicubicinterpolation
interpolationrequires requiresatatleast
least4 4pixels
pixelsalong
along theinter- inter- and
andandblanking
blankingarea,
blanking area,and
area, andeach
and eachpixel
each pixelshould
pixel shouldbebe
should betransferred
transferredinin
transferred ina aa
bicubic interpolation requires at least 4 pixels along the
bicubic the inter-
polationorientation,
polation orientation,a alarger largerorientation
orientationangle anglewill willcertainly
certainly raster-scannedmanner
raster-scanned
raster-scanned mannerfrom
manner fromthe
from thetop
the topleft
top lefttoto
left tothethebottom
the bottomright.
bottom right.
right.
polation orientation, a larger orientation angle will certainly
incura alarger
incur largerinterpolation
interpolationwindow, window, whichmeans means moremem- mem- Hsyncand
Hsync andVsyncVsyncare areemployed
employedtoto toindicate
indicatea aanew newvideovideoline line
incur a larger interpolation window, whichwhich means more more mem- Hsync and Vsync are employed indicate new video line
oryspace
ory spacerequirement
requirementand andmore
morecomputational
computationalcomplexity.complexity. and and
andvideo video
videoframe, frame,
frame,and and H
andHH activeactive
activeand and V
andVVactive active indicate
activeindicate
indicatethe the active
theactive
active
ory space requirement and more computational complexity.
Thiswill
This willsignicantly
signicantlyincrease increase thehardwarehardware designcost cost for videopixels
video pixelsinininaaascanning
scanningline lineand andactive
activevideovideolines linesininina avideo
a video
This will significantly increase the the hardware design design cost for for video pixels scanning line and active video lines video
real-time applications.
applications. (2) (2) Bicubic
(2) Bicubic interpolation
Bicubic interpolation along a larger frame,
frame, respectively.
frame, respectively.
respectively.For For
Formost most display
mostdisplay terminals,
displayterminals,
terminals,video video
videoframes frames
real-time interpolation along along aa largerlarger shouldbe betransferred
transferredpixel pixelby bypixelpixelatat ata aadened
denedframe
frames
framerate rate
orientationangle
orientation angleinvolves
involvespixels pixels thatare are veryfarfarfrom from theto-to- should
orientation angle involves pixels that
that are very very far from the the to- should
and
be
pixel
transferred
clock rate.
pixel
These
by pixel
timing
defined
requirements
frame
inherently
rate
be-interpolated
be-interpolated pixel; pixel;
pixel; thus, thus, employing
thus, employing parallelogram
employing parallelogram bicubic
bicubic and
and pixel pixel clockclock rate. rate. These
Thesetiming timingrequirements
requirementsinherently inherently
be-interpolated
interpolation tends to damage the local
parallelogram
feature in the
bicubic
case of favorthe thefullyfullypipelined
pipelinedand andsynchronized
synchronizedarchitecture architectureand and
interpolation tends tends to to damage
damage the favor
interpolation
high frequency texture. the local
local feature
feature in in the
the case
case ofof favor
pose
the
severe
fully pipelined
limitations on
and
the
synchronized
hardware
architecture
implementation
and
of
high frequency
frequency texture.
texture. pose
pose severesevere limitations
limitations on onthe thehardware
hardwareimplementation
implementationofof
high Theadaptive
adaptiveuse useofofsquare squarebicubic,
bicubic, parallelogrambicubic bicubic video upscaling. Thus, meeting certain timing requirements
The video
video upscaling.
upscaling. Thus, Thus,meeting
meetingcertain certaintiming timingrequirements
The adaptive use of bicubic,inparallelogram
squarekernels parallelogram bicubic requirements
and parallelogram bilinear
and parallelogram bilinear kernels in edge-directed interpo- edge-directed interpo- isisisasas importantasasreal-time
as important
important as real-timedoes
real-time doesininvideo video displaysystems.
does in videodisplay
systems.
display systems.
and parallelogram
lation, accordingtobilinear
tothe kernelsofinregular
theoutput
output edge-directed interpo-
edgeestimation,
estimation, Comparedwith withprevious
previousNEDI-based
NEDI-basedtechniques, techniques,the theproposed
proposed
lation, according of regular edge Compared
Compared with previous NEDI-based techniques, the proposed
lation, according
cansignicantly
signicantlyimproveto the
improvethe output of
theimage regular
imagequality edge
qualityofofedge estimation,
edgeregions
regions interpolation methodfollows follows theconventionalconventional interpolation
can interpolation
interpolation method method followsthe the conventionalinterpolation interpolation
canin significantly improve the image quality of edge regions framework,and andcan canbebeeasily
easilyimplemented
implementedasasa afully fully pipelined
in ananinterpolated
interpolated image.Moreover,
image. Moreover, thethecomputational
computational com-
com- framework,
framework, and can be easily implemented as a fullypipelined pipelined
in plexity
an interpolated
andhardware image.design
hardware Moreover,
costoftheofthe computational
the proposedexplicit com-
explicit and synchronizedarchitecture. architecture. Thisisisvery very importanttotomany many
plexity and design cost proposed and
andsynchronized
synchronized architecture.This This is veryimportant important to many
plexity and hardware
edge-directed interpolationdesignis cost oflower
muchlower the proposed thoseofexplicit
thanthose ofother
other consumerand andindustrial
industrialelectronic
electronicapplications
applicationsthat thatrequire
require
edge-directed interpolation is much than consumer
consumer and industrial electronic applications that require
edge-directed interpolation
implicitedge-directed
edge-directedmethods; is much
methods;thus, lower
thus,the than
theproposed those of other
proposedinterpola-
interpola- real-time performanceatata areasonable reasonable cost.
implicit real-time
real-timeperformance
performance at a reasonablecost. cost.
implicit edge-directed
tionisisfeasible
feasibleforforreal-timemethods;
real-time thus, the proposed
implementation interpola-
andapplications.
applications. Fig. 9 illustrates the pipeline architectureofofthe thepro- pro-
tion implementation and Fig.
Fig. 99 illustrates
illustrates the
the pipeline
pipeline architecture
architecture of proposed
the pro-
tion is feasible
Please also for real-time
note that the implementation
proposed adaptive andinterpolation
applications.is posed edge-directed image upscaling design. As the
Please also note that the proposed adaptive interpolation is posed
posed edge-directed
edge-directed image
imageupscaling
upscaling design. AsAsthe theproposed
Please
unlimited
alsoonnote
unlimited onthe that
the scaling
the factor,
scaling proposed
factor,ininadaptive
contrastto
contrast
interpolation
toNEDI-based
NEDI-based is edge-directed
interpolation algorithmisdesign.
algorithm isinherently
inherently proposed
hardware-
hardware-
unlimited
implicit on the
methods scaling
that are factor,
limited into contrast
the to
scaling NEDI-based
factor of two. edge-directed
friendly,it itprovides interpolation
providesa afavorable algorithm
favorablebase is
baseforforVLSI inherently
VLSIimplementation. hardware-
implementation.
implicit methods that are limited to the scaling factor of two. friendly,
implicit methods thatisis are limited
veryto the scaling friendly, it provides aarchitecture
favorable base for VLSI implementation.
Thischaracteristic
This characteristic also
also very important
important forforfactor of two.
interpolation
interpolation The proposed VLSI
The proposed VLSI architecture is roughly pipelined with is roughly pipelined withve ve
This characteristic
implementation in is also
practical very important
applications. for interpolation Thestagesproposed
and VLSI memory
includes architecture is roughly
hierarchy, adaptivepipelined
threshold withedge five
implementation in practical applications. stages and includes memory hierarchy, adaptive threshold edge
implementation in practical applications. stages and includes
detection,orientation memory
orientationclassication, hierarchy,
classication,regular adaptive
regularedge threshold
edgeestimation,
estimation, edge
detection,
detection,
and adaptive orientation
kernel classification,
interpolation. Insideregular
these edge
design estimation,
modules,
III.P ROPOSED
III. P ROPOSEDVLSI VLSIAA RCHITECTURE
RCHITECTURE and adaptive kernel interpolation. Inside these design modules,
III. P ROPOSED VLSI A RCHITECTURE and adaptive
complex kernel interpolation.
arithmetic units, such as Inside these design
multipliers and modules,
adders, are
Imagevisual
visualquality
qualityisisnot
notthe
theonly
onlyissue
issuetotobebeconsidered
considered complex arithmetic units, such as multipliers and adders, are
Image complex
implemented arithmeticas units,
multi-clock such as
pipelined multipliers
circuits and
to adders,
shorten are
the
Image
when visual
choosing quality
the is not the
interpolation only issue
approach, to
the be considered
computational implemented as multi-clock pipelined circuits to shorten the
when choosing the interpolation approach, the computational implemented
delaypath pathand as
andmatch multi-clock
matchthe thepixel pipelined
pixelclock.
clock.The circuits
Theve to
vepipeline shorten
pipelinestages stagesthe
when choosing
complexity the interpolation
should alsobebetaken
takenapproach, the computational
intoaccount,
account, especiallyinin delay
complexity should also into especially delay path and match the pixeldesign clock. have The five pipeline stages
complexity
real-time should also
applications. be
Thistaken into
section account,
discusses especially
real-time in
VLSI ofofthe theproposed
proposed interpolation
interpolation design have a atotal
totalofof3838clock clock
real-time applications. This section discusses real-time VLSI of the proposed
cycles,which whichmeans interpolation
meansthe thedelay design
delaybetween
betweenthe have a
theinput total
inputand of 38
andoutput
output clock
implementation of This
the section
proposed discusses
explicit real-time
edge-directed VLSI
inter- cycles, isis
implementation of the proposed explicit edge-directed inter- cycles, which means the delay between the input and output is
implementation
polationthat
thatcan ofsupport
can the proposed
support 4K2K explicit edge-directed
videodisplay.
display. inter- 3838clocks.
clocks.More Moreimportantly,
importantly,the theoverall
overallVLSI VLSIcircuitcircuitisisfully fully
polation 4K2K video 38 clocks. More
synchronizedtotothe importantly,
thepixel
pixelclock the
clockofofthe overall
theoutput VLSI
outputvideo circuit is
videoinina araster- fully
raster-
polation that can support 4K2K video display. synchronized
synchronized
scannedmanner,
scanned manner, to the
and pixel
and hence
hence clock
the ofproposed
the output
theproposed video in design
architecture
architecture a design
raster-
A.A.Pipeline
PipelineArchitecture
ArchitectureforforVideo
VideoUpscaling
Upscaling scanned
is seamlesslymanner, and
compatible hence with the the
is seamlessly compatible with the VESA video display timing proposed
VESA architecture
video display design
timing
A. Pipeline
In a Architecture
video display for Video
system, imageUpscaling
interpolationisisusually
usually isstandard.
seamlessly compatible with the VESA video display timing
In a video display system, image interpolation standard.
In a video
employed indisplay
the system,
backend image
stages and interpolation
the enlarged
employed in the backend stages and the enlarged video after is usually
video after standard.
ToTosupport
supportthe theVESAVESAstandard
standardand andsynchronize
synchronizetotothe thepixel
pixel
employed in the backend stages and the enlarged
upscaling should meet certain timing standards to connect toto
upscaling should meet certain timing standards video
to after
connect To
clock, support
the the
proposed VESA VLSI standard and
architecture
clock, the proposed VLSI architecture follows the conven- synchronize
follows to
the the
conven-pixel
upscaling
the should
thefollowing
following meetpanels.
display
display certain
panels. timing
Fig.
Fig. standards
8 8illustrates
illustrates tovideo
connect
thevideo
the to
display
display clock,
tional the proposed
line-buffer-based VLSI architecture
framework
tional line-buffer-based framework and designs the memory and follows
designs the
the conven-
memory
the following
timing display
diagram of panels.
Video Fig. 8 illustrates
Electronics the
Standards
timing diagram of Video Electronics Standards Association video display
Association tional
hierarchy
hierarchy line-buffer-based
asastwo-level
two-levelmemory framework
memory and including
buffering,
buffering, designs
including the line
line memory
buffer
buffer
timing diagram of Video Electronics Standards Association hierarchy as two-level memory buffering, including line buffer
8 88
New Linebuffer
input Line
Newinput buffer <<2
<<2 Step1
Step1 Step2
Step2
videoline
video line >>3
>>3
Adder
Adder
SPRAM11 aa
SPRAM Sync_FIFO Windowbuffer
buffer >>4
>>4 Compare
Compare
Sync_FIFO Window >>5 bb b>a
b>a
>>5 |G|G Sign(G
Sign(G x)x)
y| y|
>>6
>>6 Xor
SPRAM22
SPRAM reg1111
reg reg1212
reg ...... reg1717
reg reg
reg1818 Sign(G
Sign(G y)y)
Xor
...
...
...
...
Adder
...
...
...
...
...
...
aa
Adder
...
...
>>5 Compare
Compare
...... >>5 bb b>a
b>a I3I3 4-2
SPRAM99
SPRAM reg8181
reg reg8282
reg reg8787
reg reg
reg8888 |G|G
x| x| >>6
>>6 |Gy| y|
|G 4-2 1 1 3'b100
3'b100
I2IPriority
2 Priority ++ 1 1
MUX
Adder
3bit
MUX
Adder
I1I1 encoder 3bit 00
[2:0]
[2:0]
>>1
>>1 encoder 3'b000
MUX
MUX
Statemachine
machinefor
for I edgeoriori
edge
State aa I0 0 3'b000
Adder
Adder
>>3
>>3 00
sync_FIFOwrite Idle
write Idle Compare
Compare
Vo_clk/2 depth sync_FIFO
Vo_clk Wr_sel 4 4depth
Vo_clk/2 Vo_clk Almost_empty
Almost_empty >>4
>>4 bb b>a
b>a 11
3bit
3bit
Wr_sel |G|G gradoriori
grad
y |
&read_FIFO
&read_FIFO >>5
>>5 | y
FIFO
FIFO
[2:0]
[2:0]
Pixel(2n-1)
Pixel(2n-1) Pixel(2i-1) 0 0
Pixel(2i-1) Almost_empty
Almost_empty
Sub
&read_FIFO
&read_FIFO >>3
>>3 Sub
Mux
Adder
Mux
Adder
Pixel(2n)
Pixel(2n) Pixel(2i) Write
Write Write
Write >>4
>>4 aa ++
Pixel(2i) >>5 Compare
Compare 4'b1000
4'b1000
phase2
phase2 phase1
phase1 >>5 bb b>a
b>a
reg
reg 11 >>6
>>6 |G|G
y| y|
Wr_sel=1
Wr_sel=1 Wr_sel=0
Wr_sel=0
Fig.11.
Fig.
Fig. 11. Orientation classicationcircuit
Orientationclassication
classification circuit design
designofof
circuitdesign eight
ofeight angle
eightangle regions.
angleregions.
regions.
Fig.10.
Fig. 10. Memory
Memoryhierarchy
hierarchyofoftwo-level
two-levelbuffering.
two-level buffering.
buffering.
toreduce
toto reducethe
reduce therequired
the requiredoperating
required operating frequency.
operatingfrequency.
frequency.
andwindow
and windowbuffer.buffer.WeWeare areaware
awarethat,
aware that,the
that, theuse
the useof
use ofline-buffer-
of line-buffer-
line-buffer- Conventionalwindow-based
Conventional window-basedimage image processing architectures
Conventional window-based imageprocessing
processingarchitectures
architectures
based pixel-by-pixel
based pixel-by-pixel processing
processing tends
processing tends to
tends to consume
to consume large
consume large on-
large on-
on- [4],
[4], [5] usually design the line buffer with dual port RAM,
[4], [5][5] usually
usually design
designthe theline linebuffer
bufferwith withdual dualport portRAM, RAM,
chip memory,
chip
chip memory, and
memory, and tile-based
and tile-based or
tile-based or block-based
or block-based architecture
block-based architecture can
architecture can
can because
because the line buffer has to be read and written simulta-
because the the line
line buffer
buffer has hastotobeberead readand andwritten
writtensimulta-simulta-
efciently reduce
efciently
efficiently reduce on-chip
reduce on-chip memory
on-chip memory requirements.
memory requirements. However,
requirements. However,
However, neously. Thus, Thus, tototo build
build the the desired 88 window buffer,
neously.
neously. Thus, build the desired
desired 88 88windowwindowbuffer, buffer,888
tile-basedor
tile-based
tile-based ororblock-based
block-basedarchitecture
block-based architecturedoes
architecture doesnot
does notprocess
not processpixels
process pixels
pixels dual-port
dual-port RAMs
RAMs are
are required.
required. AsAs the
the multi-port
multi-port technique
technique
dual-port RAMs are required. As the multi-port technique
ininaaaraster-scanned
in raster-scannedmanner
raster-scanned mannerand
manner andhence
and hencecan
hence cannot
can notdirectly
not directlysupport
directly support
support tendstoto
tends tosignicantly
signicantlycomplicate
complicate the memory design, the use
tends significantly complicatethe thememory
memorydesign, design,the theuseuse
theVESA
the
the VESAtiming
VESA timingstandard.
timing standard.This
standard. Thismakes
This makesthe
makes thetile-based
the tile-basedor
tile-based orblock-
or block-
block- ofof large
large capacity
capacity dual-port
dual-port RAMRAM inin the
the line
line buffer
buffer isis very
very
of large capacity dual-port RAM in the line buffer is very
based
based design
design infeasible
infeasible toto directly
directly integrate
integrate
based design infeasible to directly integrate with other modules with
with other
other modules
modules inefcient. Accordingly,
Accordingly, this this work proposes
inefcient.
inefficient. Accordingly, this work
work proposesproposestoto design
todesign
designline line
line
inin display
in display processing
display processing systems.
processing systems. Moreover,
systems. Moreover, the
Moreover, the tile-based
the tile-based or
tile-based or
or buffers
buffers with
with 99 single-port
single-port RAMs
RAMs instead
instead of
of 8 8 dual-port
dual-port RAMs,
RAMs,
buffers with 9 single-port RAMs instead of 8 dual-port RAMs,
block-based
block-based architecture
architecture inherently
inherently reduces
reduces on-chip
on-chip memory
memory tosignicantly
signicantlyreduce reducethe the area overhead. This
block-based architecture inherently reduces on-chip memory
byincreasing
increasingexternal
externalmemory
memoryaccesses,accesses,which whichincreases
increasesthe the
toto significantly reduce theareaareaoverhead.
overhead.This Thisisis mainly
ismainly
mainlybased based
based
by increasing
by
memory
external
bandwidth
memory
burden of
accesses,
display
which
processing
increases
SoC
the
and
onon the observation that, although 8 SRAM modules requiretoto
on the
the observation
observation that,
that, although
although 8 8 SRAM
SRAM modules
modules require
require to
memory bandwidth burden of display processing SoC and beberead
readsimultaneously
simultaneouslytoto build the 88 window buffer, only
memory bandwidth burden of display processing SoC and
leads to
leads toto high
high expense
expense for for thethe overall
overall system.
system. Therefore,
Therefore, be read simultaneously tobuild
buildthe the88 88window
windowbuffer, buffer,onlyonly
leads high expense for the overall system. Therefore, one of
one
one of the
of the modules
the modulesisis
modules iswritten
written
writtenwith with
withthe the
thenewnew
newimage image
imageline line
lineatat
ataaa
the
the proposed
proposed VLSI
VLSI architecture
architecture employs
employs a a line-buffer-based
line-buffer-based
the proposed VLSI architecture employs a line-buffer-based
processing framework,
processing framework, while while leveraging
leveraging sophisticated
sophisticated circuitcircuit time. Therefore, an extra SRAM module can be leveragedtoto
time.
time. Therefore,
Therefore, an
an extra
extra SRAM
SRAM module
module can
can be
be leveraged
leveraged to
processing framework, while leveraging sophisticated circuit guarantee that
guarantee
guarantee that the
that the SRAM
the SRAMmodules
SRAM modules
modulesare are
arenot not
notreadread
readand and written
andwritten
written
design
design and
and hardware
hardware sharing
sharing techniques
techniques to
to reduce
reduce implemen-
implemen- simultaneously through
simultaneously through the the ping-pong operation. Thus, the
design and hardware sharing techniques to reduce implemen-
tationcost,
tation cost,as
cost, asdescribed
describedin inthe
thefollowing
followingsubsections.
subsections. simultaneously through the ping-pong
ping-pong operation.operation.Thus, Thus,the the
tation as described in the following subsections. proposed
proposed architecture
architecture can
can effectively
effectively
proposed architecture can effectively support simultaneous support
support simultaneous
simultaneous
reading and
reading
reading and writing
and writing with
writing with
with the the line
the line buffer
line buffer
buffer that that
thatisis internally
isinternally
internally
B.B. Memory
MemoryHierarchy
HierarchyofofTwo-Level
Two-LevelBufferingBuffering designedwith
designed withsingle-port
single-portRAMs. RAMs.
B. Memory Hierarchy of Two-Level Buffering designed with single-port RAMs.
The operating
The operating frequency
frequency ofof SRAM SRAM modules modules isis another another
Given that
Given that image
image interpolation
interpolation isis aa type type ofof window-based
window-based The operating frequency of SRAM modules is another
Given that image interpolation is a type of window-based issue that
issue that should
should be be carefully
carefully addressed.
addressed. InIn the the window-
window-
processing, the
processing, the neighboring
neighboring pixels pixels ofof the the to-be-interpolated
to-be-interpolated issue that should be carefully addressed. In the window-
processing, the neighboring pixels of the to-be-interpolated based processing,
based processing, the the line
line buffer
buffer shouldshould be be accessed
accessed atat the the
pixel require
pixel require toto be be accessed
accessed simultaneously.
simultaneously.To Toachieve
achievethis, this, based processing, the line buffer should be accessed at the
pixel require to be accessed simultaneously. To achieve this, same frequency
same frequency asas the the pixel
pixel clock.
clock. To To support
support the the output
output
the proposed
the proposed VLSI VLSI architecture
architecture uses uses the the memory
memory hierarchyhierarchy same frequency as the pixel clock. To support the output
the proposed VLSI architecture uses the memory hierarchy ofof aa 4K2K@60Hz
4K2K@60Hz video, video, the the pixel
pixel clockclock can canreachreachup uptoto
ofoftwo-level
two-levelbuffering,
buffering,including
includingline lineand andwindow
windowbuffers, buffers,asas of a 4K2K@60Hz video, the pixel clock can reach up to
of two-level buffering, including line and window buffers, as around600
around 600MHz,MHz,whichwhichisisvery verychallenging
challengingfor forlarge-capacity
large-capacity
shownininFig.
shown Fig.10.10.TheThelinelinebuffer
bufferisisaaset setofofRAMsRAMswhich whichcan can around 600 MHz, which is very challenging for large-capacity
shown in Fig. 10. The line buffer is a set of RAMs which can memory design.
memory design. Accordingly,
Accordingly, this this work work proposes
proposes toto use use aa
containseveral
contain severallineslinesofofinput
inputvideo,
video,whereas
whereaswindow windowbuffer bufferisis memory design. Accordingly, this work proposes to use a
contain
aaset
several
setofofshift
lines of (represented
shiftregisters
input video,aswhereas
registers(represented asreg)
reg)that
window
thatcontain
buffer
containpixelspixels
is synchronousFIFO
synchronous FIFO(sync (sync FIFO) FIFO)between
betweenthe theline linebuffer
bufferand and
synchronous FIFO (sync FIFO) between the line buffer and
aininset of shift registers (represented as reg) that contain pixels windowbuffer
window buffertotoreduce
reducethe therequired
requiredoperatingoperatingfrequency
frequencyfor for
thewindow.
the window.Since Sincethe thewindow
windowbuffer bufferconsists
consistsofofregisters,
registers, window buffer to reduce the required operating frequency for
in the window. single-port RAMs.
single-port RAMs. The The design
design ofof the the proposed
proposed sync sync FIFO FIFO
ititcan guaranteeSince
canguarantee instant
instanttheaccess
window
access buffer
totoall
allits consists The
itselements.
elements. ofThe
registers,
size
sizeofof single-port RAMs. The design of the proposed sync FIFO
it can guarantee instant access isis illustrated
illustrated inin Fig. Fig. 10. 10. TheThe syncsync FIFO FIFO and and the the following
following
window
window buffer
buffer isis chosen
chosen toto bebeto88,
all its
88, elements.
where
where the The size
the central
central 44
44 of
is illustrated in Fig. 10. The sync FIFO and the following
window buffer is chosen to be 88, where the central 44 window
window buffer buffer are are synchronized
synchronizedwith withthe thepixel
pixelclock clockofofthe the
isisthe
theestimation
estimationwindow windowfor forregular
regularedge edgeestimation
estimationand andthe the window buffer are synchronized with the pixel clock of the
is the registers
estimation window for regular edge output
outputvideo video(represented
(representedasasvo vo clk),
clk),and andthethelinelinebuffer
bufferworks works
extra
extra registersstorestore the
thereference
reference pixels
pixels forestimation
for adaptive and the
adaptivethreshold
threshold output video (represented as vo clk), and the line buffer works
extra atathalf
halfofofthe theoutput
outputpixel pixelclockclock( (12121vo vo clk).
clk).The Theoperation
operationofof
edge registers
edge detectionstore
detection and the
and reference modules.
interpolation
interpolation pixels
modules. for With
adaptive
With the thethreshold
memory
memory at half of the output pixel clock ( vo clk). The operation of
edge detection and interpolation modules. With the memory the
thesyncsync FIFOFIFOand andthe thecorresponding
corresponding 2 clock clockdomaindomaincrossingcrossing
hierarchy
hierarchyofoftwo-level
two-levelbuffering,
buffering,the theneighboring
neighboringpixels pixels ininan an the sync FIFO and the corresponding clock domain crossing
hierarchy issue
issue can can be be addressed
addressed with with the the state
statemachine
machineillustrated
illustratedinin
88
88arrayarrayof can
cantwo-level
be
beprovided buffering,
provided instantly
instantlythefor
forneighboring
each pixels in an
eachto-be-interpolated
to-be-interpolated issue can be addressed with the state machine illustrated in
88 Fig.
Fig.10. 10.
pixelarray
pixel can be
atat each
each pixelprovided
pixel clock.instantly
clock. This isisfor
This each for
crucial
crucial to-be-interpolated
for thethe pipelined
pipelined Fig. 10.
pixel
and at each
and synchronized pixel
synchronized image clock. This
image interpolation is crucial
interpolation design. for Moreover,
design. the pipelined
Moreover, asas
and
line synchronized
line buffer
buffer occupies
occupies image
aalarge interpolation
large percentage
percentageof design.
ofhardware
hardwareMoreover,
area
areaand as
and C.
C. Orientation
OrientationClassication
ClassicationDesign
Design
line buffer occupies a large percentage of hardware area and C. Orientation Classification Design
has
hasaadirect
directimpact
impacton onthetheperformance
performanceofofcircuit circuitdesign,
design,two two The
The key
key ofof orientation
classication design
design isis toto address
address
has
efcienta direct
efcient impact onare
technologies
technologies the performance
are exploited of
exploited toto optimize circuitits
optimize design,
its cost
cost andtwo
and theThe
the key ofcomplexity
potential
potential orientationinduced
complexity classification
induced by
by tandesign
tan1
1 is
andtodivision
()() and address
division
efficient
efciency,
efciency, technologies
i.e.
i.e. using are exploited
using single-port
single-port RAM
RAM to optimize
instead
instead of its cost and
of dual-port
dual-port the potential
operations. complexity
operations. Accordingly,
Accordingly, thisinduced
this designby tan 1
design proposes ()
proposes an and division
an efcient
efcient
efficiency,
RAM
RAM and i.e.
and further using
further usingsingle-port
using synchronous
synchronous RAM instead of dual-port
rst-in-rst-out
rst-in-rst-out (FIFO)
(FIFO) operations.
technique
techniquethat Accordingly,
that can
candetermine
determinethisthedesign
the proposes
orientation
orientation an efficient
classication
classication for
for
RAM and further using synchronous first-in-first-out (FIFO) technique that can determine the orientation classification for
99
Stage 3 Stage 4 Stage 5

Coefficient generation Delayed_orientation 8 clocks 8 clocks
Orientation Horizontal
3 clocks 3 clocks 1 clock interpolation 1
h,v h,v Bilinear/bicubic Coefficient 4x4
generation coefficient generation selection Coe.
Vertical interpolation
Adder
Orientation Y selection
3 clocks 1 clock 3 clocks
(a)
(a) (b)
(b) (c)
(c) (d)
(d)
Y Y 4x4
6x8 Delay
pre-selection selection Y
Y
Horizontal
interpolation 2
Delayed_orientation CbCr selection
Horizontal
3 clocks 1 clock
Delayed h,v interpolation 3
CbCr CbCr 4x4
6x8 CbCr pre-selection selection CbCr Horizontal
interpolation 4
(e)
(e) (f)
(f) (g)
(g) (h)
(h)
Fig.
Fig. 12.
12. Shared
Shared architecture
architecture design
design for
for adaptive
adaptive kernel
kernel interpolation.
interpolation.
Fig. 13.
Fig. 13. Parts
Partsofofthe
theimage
imagedatabase:
database:(a)
(a)Zebra,
Zebra,(b)
(b)Bench,
Bench,(c)
(c)Goose,
Goose,(d)
(d)
Clock,(e)
Clock, (e)Line,
Line,(f)
(f)Guitar,
Guitar,(g)
(g)Buttery
Butterflyand
and(h)
(h)Lena.
Lena.
each pixel
pixel with
with simple
simple shift
shift and
and comparison
comparison operations,
operations, whilewhile
avoiding
avoiding incurring
incurring complex
complex and and expensive
expensive arithmetic
arithmetic circuits.
circuits. can be
can be easily
easily selected
selected viavia the
theMUX
MUXcircuit
circuitaccording
accordingtotothethe
Lets
Lets take
take the
the orientation
classification of of eight
eight angle
angle regions
regions orientation classication
orientation classification result.
result. For
For interpolation
interpolation coefcient
coefficient
(illustrated
(illustrated in Fig. 2) for example to demonstrate how
in Fig. 2) for example to demonstrate how this
this selection,the
selection, thenormalized
normalizedhorizontal
horizontaldistance
distanceh andvertical
h 0and vertical
technique
technique works.works. distancev
distance 0
v are
arerst
firstcalculated,
calculated,andandthen
thenthe
thebilinear/bicubic
bilinear/bicubic
The
The detail
detail of of the
the proposed
proposed orientation
classification isis interpolationcoefcients
interpolation coefficientsarearegenerated.
generated.TheThenal
finalinterpolation
interpolation
presented
presented in Fig. 11. The angle region that the pixelbelongs
in Fig. 11. The angle region that the pixel belongsto to coefficients are
coefcients are selected
selected among
among these
these bilinear/bicubic
bilinear/bicubic coef-
coef-
is determined
determined by both the magnitude ratio and sign of vertical
by both the magnitude ratio and sign of vertical ficient candidates
cient candidates according
according toto the
the orientation.
orientation. TheThe shared
shared
gradient
gradient G Gyy and
and horizontal
horizontal gradient
gradient G Gxx.. We
We cancan rstly
firstly use
use interpolation
interpolation unitunit consists
consists ofof four
four horizontal
horizontal interpolation
interpolation
|Gyy|| and
and |Gxx | to map the gradients to the first quadrant and
|G | to map the gradients to the rst quadrant and units
units and
and one
one vertical
verticalinterpolation
interpolationunit.
unit.InInparticular,
particular,shared
shared
determine
determine the the orientation
classification of of the
the pixel,
pixel, andand then
then interpolation
interpolationisisnotnotrequired
requiredtotoconsider
considerthetheorientation
orientationand
andcan
can
the result can be extended to the other quadrants
result can be extended to the other quadrants according according interpolate
interpolate all
allthethepixels
pixelswith
withsimple
simplemultipliers
multipliersand andadders.
adders.
to
to the
the sign
sign relationship
relationship between
between G Gyy and
and G Gxx.. To
To eliminate
eliminate the the
tan 1() operation, this design pre-calculates the magnitude
1
tan () operation, this design pre-calculates the magnitude IV. XPERIMENTALRRESULTS
IV. EEXPERIMENTAL ESULTS
ratio
ratio (|G |/|Gxx|)|) at
(|Gyy|/|G at each
each boundary
boundary of of the
the adjacent
adjacent angleangle
regions. To
To evaluate
evaluate the
the proposed
proposed explicit
explicit edge-directed
edge-directed interpo-
interpo-
regions. The orientation classification for each pixel
The orientation classication for each pixel can
can then
then
be lation,
lation, thethe experiment
experiment uses uses aa database
database ofof 32 32 natural
natural im-im-
be determined
determined by by comparing
comparing the the gradient
gradient magnitude
magnitude ratio ratio ofof
the ages11. . Several
ages Several ofof these
these images
images are are classic
classic test
test images,
images,
the pixel
pixel with
with these
these pre-calculated
pre-calculated values.
values. In In addition,
addition, the the
division such
such as as Lena
Lena and and Pepper;
Pepper; the the other
other images
images are are selected
selected
division operation
operation during
during thethe magnitude
magnitude ratio ratio calculation
calculation isis from
converted into multiplication by directly comparing from the the morgueFile
morgueFile online
online archive
archive (http://morguele.com),
(http://morguefile.com),
converted into multiplication by directly comparing |G |Gyy|| with
with which
which are are subject
subject toto thethe license
license agreement
agreement available
available atat
|Gx | K[i], and the multiplication with constant values of
|G x | K[i], and the multiplication with constant values of http://morguele.com/archive/terms.php.
http://morguefile.com/archive/terms.php. Fig. Fig. 13
13shows
showspartsparts
K[i] can
K[i] can be
be further
further approximately
approximately implemented
implemented with with shift
shiftand
and of
addition of the
the selected
selectedimages
imagesininthe thetest
testdatabase.
database.The Theselected
selectedlesfiles
addition circuits.
circuits. are
are RGB
RGB colorcolor images
images with
with aa depth
depth ofof 88 bits
bits per
perchannel.
channel.
These
Thesecolorcolorimages
imagesarearerstly
firstlyconverted
convertedtotoYCbCr
YCbCrcolor colorspace,
space,
D. Adaptive
Adaptive Kernel
Kernel Interpolation
Interpolation Design
Design and
and edge
edge detection
detectionandandregular
regularedge
edgeestimation
estimationare areemployed
employed
The
The straightforward
straightforward designdesign of of adaptive
adaptive kernel
kernel interpolation
interpolation only
only ononthetheYYchannel.
channel.The Theedge
edgeinformation
informationcalculated
calculatedfrom from
tends
tends toto be
be very
very costly,
costly, because
because all all the
the eight
eight directional
directional in-
in- the
the YY channel
channel isis then
then used
usedfor forall
allthree
threeYCbCr
YCbCrchannels
channelsfor for
terpolations
terpolations should
should be be implemented
implemented and and the
to-be-interpolated adaptive
adaptive kernel
kernelinterpolation,
interpolation,and andnally
finallythetheenlarged
enlargedYCbCr YCbCr
pixel
pixel isis adaptively
adaptively selected
selected in in the
the nal
final stage.
stage. Nevertheless,
Nevertheless, images
imagesare arere-converted
re-convertedtotoRGB RGBcolor
colorspacespacefor
forthe
thefollowing
following
this
this paper
paper proposes
proposes aa sharedshared interpolation
interpolation architecture
architecture to to evaluation.
evaluation. Both subjective and objective tests ofthese
Both subjective and objective tests of theseRGBRGB
signicantly
significantly reduce
reduce thethe design
design overhead.
overhead. Essentially,
Essentially, the
the pro-
pro- color
colorimages
imagesare areperformed
performedtotoquantitatively
quantitativelycompare
comparethe thequal-
qual-
posed
posed shared
shared architecture
architecture separates
separates the the interpolation
interpolation circuit
circuit ity
ityofofthe
theimages
imagesgenerated
generatedwithwithdifferent
differentmethods.
methods.Moreover,
Moreover,
into
into two
two parts,
parts, i.e.
i.e. pixel
pixel and
and coefcient
coefficient selection,
selection, andand shared
shared the
theperformance
performanceand andcost
costofofthe
theVLSI
VLSIdesign
designarearepresented
presentedby by
interpolation,
interpolation, where
where pixel
pixel and
and coefcient
coefficient selection
selection adaptively
adaptively synthesizing
synthesizing the circuit with TSMC standard cell andmemory
the circuit with TSMC standard cell and memory
chooses
chooses thethe necessary
necessary neighboring
neighboring pixelspixels andand interpolation
interpolation library.
library.
coefcients
coefficients according
according to to the
the orientation
classification result,
result,
and
and shared
shared interpolation
interpolation simply
simply generates
generates the the new
new pixel
pixel with
with A.
provided
A. Objective
ObjectiveTest Test
provided pixels
pixels and
and coefcients.
coefficients. The
The
The proposed
proposed shared
shared architecture
architecture design
design forfor adaptive
adaptive kernel The objective test
kernel sampling
objective test compares
compares images images obtained
obtained by by down-
down-
interpolation is illustrated in Fig. 12. As the window buffer sampling the
the reference
reference images
images and
and then
then enlarging
enlarging them
them with
with
interpolation is illustrated in Fig. 12. As the window buffer
contains all the neighboring pixels for eight
contains all the neighboring pixels for eight potential inter-potential inter- 11Test
Test images
images asas well
well asas images
images with
with 44 enlargement
enlargement results
resultsfor
forthe
the
polation
polation orientations,
orientations, the the corresponding
corresponding neighboring
neighboring pixels
pixels proposed
proposedalgorithm
algorithmare
areavailable
availableatathttp://gr.xjtu.edu.cn/web/hsun/research
http://gr.xjtu.edu.cn/web/hsun/research
10
TABLE II
PSNR AND SSIM C OMPARISONS AMONG D IFFERENT M ETHODS ( I MAGE U PSCALED BY 2 FACTOR IN B OTH H ORIZONTAL AND V ERTICAL
O RIENTATIONS WITH Tmin S ET TO 30).
Bicubic [2] Chen [12] NEDI [14] iNEDI [15] ICBI [17] Proposed SR [20]
Images
PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM
Zebra 31.10 0.9790 30.19 0.9766 30.32 0.9778 30.72 0.9777 31.16 0.9776 31.16 0.9791 33.47 0.9883
Butterfly 33.01 0.9923 32.92 0.9899 31.72 0.9884 32.89 0.9916 33.47 0.9928 33.23 0.9923 35.69 0.9958
Bee 35.56 0.9917 34.31 0.9895 34.32 0.9815 35.26 0.9905 36.08 0.9920 35.76 0.9918 38.40 0.9965
Whirligig 30.62 0.9867 30.14 0.9850 31.35 0.9859 31.49 0.9876 31.08 0.9866 31.20 0.9873 33.63 0.9944
Sheep 33.92 0.9823 32.93 0.9762 32.79 0.9740 33.44 0.9795 33.93 0.9823 34.14 0.9822 36.74 0.9896
Giraffe 37.35 0.9928 36.50 0.9914 35.99 0.9898 36.37 0.9912 37.49 0.9925 37.55 0.9927 40.19 0.9968
Guitar 30.40 0.9851 29.96 0.9792 30.23 0.9809 31.01 0.9831 31.08 0.9829 31.58 0.9860 33.09 0.9912
Flower 37.32 0.9902 37.29 0.9883 37.06 0.9828 38.09 0.9901 38.29 0.9900 38.11 0.9906 40.33 0.9966
Beads 30.61 0.9733 30.46 0.9723 30.16 0.9720 31.14 0.9737 30.92 0.9721 30.77 0.9736 33.01 0.9877
Lena 32.45 0.9523 32.58 0.9522 32.40 0.9556 32.50 0.9543 32.33 0.9481 32.57 0.9526 36.46 0.9895
Average of
31.03 0.9637 30.48 0.9600 30.38 0.9609 31.07 0.9625 31.08 0.9628 31.35 0.9639 33.69 0.9839
32 images
TABLE III
PSNR AND SSIM C OMPARISONS AMONG D IFFERENT M ETHODS ( I MAGE U PSCALED BY 4 FACTOR IN B OTH H ORIZONTAL AND V ERTICAL
O RIENTATIONS WITH Tmin S ET TO 30).
Bicubic [2] Chen [12] NEDI [14] iNEDI [15] ICBI [17] Proposed SR [20]
Images
PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM
Zebra 23.33 0.8996 24.08 0.8933 24.31 0.9057 25.09 0.9121 24.93 0.9052 24.55 0.9028 26.94 0.9346
Butterfly 24.94 0.9285 24.51 0.9168 24.39 0.9138 24.74 0.9278 24.86 0.9299 24.97 0.9289 27.64 0.9452
Bee 26.63 0.9333 26.95 0.9196 27.01 0.9218 27.54 0.9350 27.69 0.9355 27.69 0.9336 30.23 0.9460
Whirligig 23.51 0.9185 23.93 0.9136 24.45 0.9176 24.34 0.9229 24.23 0.9191 24.28 0.9193 27.18 0.9485
Sheep 27.11 0.8727 27.42 0.8519 27.01 0.8444 27.36 0.8613 27.38 0.8674 27.62 0.8726 29.70 0.8889
Giraffe 29.56 0.9579 30.16 0.9484 29.48 0.9420 30.02 0.9519 30.94 0.9584 30.85 0.9580 32.26 0.9600
Guitar 23.72 0.8772 23.82 0.8645 23.84 0.8723 23.98 0.8850 23.64 0.8736 24.23 0.8819 26.53 0.9210
Flower 28.95 0.9342 29.01 0.9356 29.53 0.9266 30.03 0.9419 29.65 0.9359 29.25 0.9353 32.23 0.9573
Beads 24.35 0.8854 25.06 0.8877 24.22 0.8782 25.14 0.8896 25.17 0.8862 24.90 0.8858 27.40 0.9256
Lena 26.98 0.8667 27.46 0.8633 27.70 0.8751 27.63 0.8748 27.32 0.8605 27.65 0.8677 30.46 0.9325
Average of
24.60 0.8564 24.95 0.8496 24.81 0.8527 25.02 0.8532 25.06 0.8520 25.28 0.8574 27.79 0.9011
32 images
different methods, with the original reference images. The polation is obtained using the MATLAB built-in function;
simulation performs the test on all the three RGB channels. Chens method and database-free SR are implemented by us in
The differences between the upscaled and reference images are MATLAB following the description given in the cited papers;
evaluated by the metrics of peak signal-to-noise ratio (PSNR) and the original MATLAB codes of NEDI, iNEDI and ICBI
and structural similarity (SSIM) [24]. are provided from their corresponding websites declared in
cited papers. According to the experimental results in Table II
The proposed explicit edge-directed adaptive interpolation
and Table III, the PSNR of the proposed interpolation is higher
method is compared with the conventional linear method, i.e.
than the results of other evaluated interpolation methods for
bicubic [2], previous explicit edge-directed method, i.e. Chens
the great majority of test images. We can see that the PSNR
method in [12], and NEDI-based implicit edge-directed meth-
of previous explicit edge-directed interpolation, i.e., Chens
ods, i.e. NEDI [14], the improved NEDI (referred to as iNEDI)
method, is even lower than conventional bicubic interpolation
described in [15] and ICBI [17]. Moreover, these interpolation
in the case of moderate scaling ratio. Nevertheless, the pro-
results are also compared with a single-image database-free
posed interpolation can improve the average PSNR by 0.87
super resolution work (referred to as SR) presented in [20],
dB and 0.33 dB for 2 and 4 upscaling respectively, com-
which is also the only super resolution work that targets for
pared with the evaluated explicit edge-directed interpolation.
real-time TV scaler application according to our literature
Moreover, the average PSNR of 32 test images also shows that
research. The upscaled image results by 2 and 4 factors
the proposed interpolation even outperforms the NEDI-based
in both horizontal and vertical orientations are compared with
implicit methods by at least 0.27 dB and 0.22 dB in the case
the original reference images, and parts of and the average
of 2 and 4 upscaling, respectively. The SSIM comparison
PSNR and SSIM results are listed in Table II and Table III,
result is consistent with PSNR that the SSIM of the proposed
respectively. Among the compared algorithms, bicubic inter-
11
11
explicit
explicit interpolation
interpolation outperforms
outperforms the the NEDI-based
NEDI-based implicit implicit 5.0
Mean Opinion Score

methods
methods by by atat least
least 0.0011
0.0011 and and 0.0054
0.0054 in in the
the case
case ofof 2
2 and
and 4.5 Artifact Blurriness 4.12
44 upscaling
upscaling respectively.
respectively. The The PSNR
PSNR and and SSIMSSIM comparison
comparison 4.0 3.62 3.71
3.45
3.39 3.34
results
results demonstrate
demonstrate the the effectiveness
effectiveness of of edge
edge detection
detection and
and 3.5
2.98
3.23
orientation
orientation estimation
estimation in in the
the proposed
interpolation method.
method.
3.0
2.5
In
Inaddition,
addition,the thePSNR
PSNR and and SSIMSSIM results
results ofof the
the evaluated
evaluated SR 1.93
1.74
2.0
work
workare aresignificantly
signicantly higherhigher than than those
those of of other
other interpolation
interpolation 1.5
methods.
methods. Nevertheless,
Nevertheless, the the mainmain reason
reason for for this
this appealing 1.0
result
result isis mainly
mainly basedbased on on the
the assumption
assumption of of anan accurate
accurate blur Bicubic Chen ICBI Proposed SR
model
model inin the the reconstruction
reconstruction constraint
constraint of of the
the SR SR algorithm.
algorithm.
When
When the the blur
blur model
model isis inaccurate
inaccurate and and different
different from the Fig. 17. Comparison of mean opinion
opinion scores
scores of
of subjective
subjective quality.
quality.
real
real camera
camera blur, blur, the
the performance
performance of of the
the SR SR algorithm
algorithm is
significantly
signicantly degraded,
degraded, typically
typically leading
leading to to over-smoothed
over-smoothed
results.
results. The
The effect
effect ofof the
the blur
blur model
model on on SR
SR performance
performance has
been
beenextensively
extensivelystudiedstudied andand evaluated
evaluated in in [23].
[23]. In In practice,
practice, the
video
video sources
sources for for display
display processing
processing system system can can bebe captured we have declared, and introduces
introduces manymany unacceptable
unacceptable artifacts
artifacts
byby different
different cameras
cameras atat different
different resolutions,
resolutions, and and may
may even in images with high frequency
frequency textures,
textures, e.g.,
e.g., the
the indicated
indicated
bebe edited
edited or or cropped
cropped in in advance.
advance. ItIt is is actually
actually infeasible
infeasible to scale of clock in Fig. 15(b). The The ICBI
ICBI method
method shows
shows goodgood
accurately
accurately estimate
estimate the the blur
blur model
model in in the
the display
display processing
processing visual image quality in the edge
edge and and texture
texture areas
areas butbut suffers
suffers
system. Therefore,
system. Therefore, despite
despite its its high
high performance,
performance, the the applica- from distortion in the case of straight
straight lines
lines with
with high
high intensity
intensity
tion of
tion of the
the SRSR method
method in in aa video
video display
display processing
processing system difference, e.g. the radial lines shown
shown in in Fig.
Fig. 16(c).
16(c). Neverthe-
Neverthe-
isisstill
still very
very challenging.
challenging. less, the proposed explicit edge-directed
method
Inthe
In the proposed
proposed explicit
edge-directed interpolation,
interpolation, all pa- can remove the jaggies and improve
improve the the sharpness
sharpness along
along the
the
rameters are adaptively determined
rameters are adaptively determined by the local by the local feature,
feature, except edge region while avoiding the the potential
potential artifacts
artifacts inin the
the case
case
for
for Tmin
T min. In fact, as compensated by the following regular
. In fact, as compensated by the following of high frequency texture. Therefore,
Therefore, the the proposed
proposed method
method
edge estimation, the
edge estimation, the choice
choice of of TTmin
min has has aa negligible
negligible effect has comparable or even better better visual quality than
visual quality than NEDI-
NEDI-
on the nal interpolation performance.
on the final interpolation performance. In this paper, In this min is
paper, Tmin based implicit edge-directed interpolation.
interpolation. In In addition,
addition, visual
visual
empirically set to 30 in all experiments and evaluations. The
empirically set to 30 in all experiments and evaluations. quality comparison results showshow that,
that, although
although thethe evaluated
evaluated
sensitivity of
sensitivity of average
average PSNR PSNR and and SSIM
SSIM performance
performance to the SR method
SR method can
can provide
provide sharper
sharper edge
edge and
and better
better visual
visual quality
quality
variation of
variation of Tmin T is also examined. Evaluation
min is also examined. Evaluation results results shown by dedicated edge enhancement and texture synthesis modules,
by dedicated edge enhancement and texture synthesis modules,
ininTable
Table IV IV clearly
clearly indicate
indicate that that the
the value
value of of TTmin
min is easy to it also
it also poses
poses zigzag
zigzag artifacts
artifacts along
along the
the edge
edge inin the
the natural
natural
determine and
determine and thethe proposed
interpolation approach
approach is robust image database.
image database.
totothe
the variation
variation of min..
of TTmin
To further verify
verify ourour claim
claim quantitatively,
quantitatively, aa perceptual
perceptual ex- ex-
TABLE IV
TABLE IV periment is
periment is also
also conducted.
conducted. Accordingly,
Accordingly, we we randomly
randomly presentpresent
SSENSITIVITY
ENSITIVITYOF AVERAGE
OFA PSNR AND
VERAGE PSNR SSIM PPERFORMANCE
AND SSIM ERFORMANCE all the
all the enlarged
enlarged test test images
images (by (by aa 4
4 factor)
factor) with
with different
different
TOTHE
TO VALUE
THE V ALUE OF
OF T min..
Tmin
algorithms to
algorithms to 20
20 subjects,
subjects, and and let
let them
them give
give each
each image
image result
result
TTmin
min aa score
score between
between 11 to to 55 according
according to to its
its subjective
subjective quality.
quality.
28
28 29
29 30
30 31
31 32
32
The perceived
The perceived subjective
subjective imageimage quality
quality is is scored
scored as as two
two
2PSNR
PSNR 31.357 31.357 31.358 31.358 31.358 separate factors,
separate factors, i.e.
i.e. artifact
artifact and
and blurriness.
blurriness. Fig.
Fig. 1717 illustrates
illustrates
2 31.357 31.357 31.358 31.358 31.358
2 SSIM 0.9638 0.9639 0.9639 0.9639 0.9638 the results
the results of
of the
the perceptual
perceptual experiment
experiment for for the
the subjective
subjective
2 SSIM 0.9638 0.9639 0.9639 0.9639 0.9638
4PSNR
PSNR 25.281 25.282 25.283 25.282 25.282 test. The
test. The quantitative
quantitative results
results are
are largely
largely consistent
consistent with with the
the
4 25.281 25.282 25.283 25.282 25.282
4 SSIM 0.8574 0.8574 0.8574 0.8574 0.8573 observation by
observation by ourselves.
ourselves. In In particular,
particular, wewe note
note that,
that, artifact
artifact
4 SSIM 0.8574 0.8574 0.8574 0.8574 0.8573
is more
is more important
important than than blurriness
blurriness forfor interpolation
interpolation in in display
display
processing systems. The visual quality (especially the
processing systems. The visual quality (especially the blur
blur
effect) of
effect) of the
the interpolated
interpolated image image can
can bebe enhanced
enhanced by by conven-
conven-
B. Subjective
B. Subjective Test
Test tional image
image quality
tional quality improvement
improvement modules,
modules, such
such asas luminance
luminance
ItItshould
shouldbe benoted
noted that
that methods
methods withwith higher
higher PSNR
PSNR or or SSIM
SSIM peaking and detail enhancement, in the display processing
peaking and detail enhancement, in the display processing
do not necessarily correspond to better visually
do not necessarily correspond to better visually perceived perceived system [25].
system [25]. WeWe should
should alsoalso note
note that
that the
the artifacts
artifacts along
along thethe
quality. Therefore,
quality. Therefore, the the visual
visual quality
quality between
between the
the proposed
proposed edge region in the evaluated SR work is mainly because its
edge region in the evaluated SR work is mainly because its
and reference methods is further compared over
and reference methods is further compared over the selected the selected interpolation procedure
interpolation procedure has has not
not processed
processed the the edge
edge region
region well
well
natural image database. Parts of the visual quality
natural image database. Parts of the visual quality comparison comparison enough, and
enough, and these
these zigzag
zigzag artifacts
artifacts are
are further
further enhanced
enhanced by by the
the
results are
results are shown
shown from
from Fig.
Fig. 14
14 to
to Fig.
Fig. 16.
16. following edge enhancement and texture synthesis
following edge enhancement and texture synthesis procedures. procedures.
Bicubic interpolation can easily induce
Bicubic interpolation can easily induce jaggies jaggies along
along edge
edge With more
With more advanced
advanced interpolation,
interpolation, the the image
image quality
quality of of the
the
regions, as we have pointed out in previous sections.
regions, as we have pointed out in previous sections. Chens Chens evaluated SR can be signicantly improved,
evaluated SR can be significantly improved, thereby further thereby further
explicit edge-directed interpolation
method seems
seems toto outper-
outper- indicating that
indicating that thethe high
high quality
quality edge-directed
interpolation
form bicubic interpolation. However, Chens method suffers remains very critical to the pervasive application of 4K display
form bicubic interpolation. However, Chens method suffers remains very critical to the pervasive application of 4K display
from inaccurate edge detection and orientation estimation as systems.
from inaccurate edge detection and orientation estimation as systems.
12
(a) (b) (c) (d) (e)
Fig. 14. Subjective comparison of different interpolation methods by 4 factor in both horizontal and vertical orientations. The five shown images are results
of Fig. 13(b). (a) Bicubic (b) Chens method (c) ICBI (d) Proposed (e) SR.
(a) (b) (c) (d) (e)
of Fig. 13(d). (a) Bicubic (b) Chens method (c) ICBI (d) Proposed (e) SR.
(a) (b) (c) (d) (e)
of Fig. 13(e). (a) Bicubic (b) Chens method (c) ICBI (d) Proposed (e) SR.
C. VLSI Implementation Result
The proposed VLSI architecture has been designed with

Verilog and synthesized using Synopsys tools with TSMC
CLN65GP standard cell and memory library. The layout of
the proposed design was generated with IC Compiler. The
design layout takes 1.71.7 mm2 , and its photomicrograph
is illustrated in Fig. 18. The design results are summarized
in Table V. The maximum input video size is designed as
19201080@60Hz. The input video format is RGB 4:4:4
format with a depth of 8 bits per channel. The maximum Fig. 18. Design layout.
working frequency of the proposed VLSI circuit is 600 MHz,
which means that the maximum output of the interpolated
video sequence can reach up to 4K2K@60Hz, and the conventional bicubic interpolation in [3] and the single-image
throughput of output video can reach up to 497M pixel/s. database-free super resolution presented in [20]. Both of the
According to the synthesis report, the overall VLSI circuit two VLSI circuits are designed to support the maximum output
design only costs less than 505 K logic gates and 52 KB of 1080P@60Hz HDTV video. The memory consumption
memories. In addition, as we explained in Section III, the of the proposed design (9SPRAM) is actually comparable
proposed VLSI circuit is fully pipelined and synchronized to with conventional bicubic interpolation (6DPRAM). The
the pixel clock of the output video, and hence it can seamlessly main reason that the SR method seems to consume much
support the VESA-compatible display system. less memory resource is because it chooses the tile-based
We further compare the implementation results of the architecture and leverages off-chip memory access to reduce
proposed work with two reference designs, including the on-chip memory usage. However, the major drawback of the
13
TABLE V The proposed interpolation method has been extensively

VLSI I MPLEMENTATION R ESULT C OMPARISON . evaluated by comparisons with several well-known methods,
Bicubic [3] SR [20] Proposed
in terms of objective, subjective quality and real-time perfor-
Technology TSMC 0.13um TSMC 65nm TSMC 65nm
mance. Compared with other methods, the objective and sub-
Bit/pixel 8 (Y only) 24 (RGB) 24 (RGB)
jective qualities of the proposed interpolation are significantly
Clock rate 279 MHz 240 MHz 600 MHz
improved. In the meanwhile, its computational complexity
Max input N/A 960540@60Hz 1080p@60Hz
remains low. Because of its inherent parallelism, the VLSI
Max output 1080p@60Hz 1080p@60Hz 4K2K@60Hz
implementation of the proposed interpolation could reach the
Throughput 124M pixel/s 124M pixel/s 497M pixel/s
maximum output video sequence of 4K2K@60Hz with less
Gate count 30,643 765,886 504,100
than 505 K logic gates and 52 KB memory cost.
6 DPRAM DPRAM 9 SPRAM1
Memory ACKNOWLEDGMENT
N/A 22 KB 51.84 KB
Normalized
N/A 3888 MB/s 373 MB/s
This research was supported by the National Science and
bandwidth2 Technology Major Project of China (No. 2013ZX01033001-
1 The length of each line buffer (SPRAM) equals to the maximum 001).
horizontal resolution of input video.
2 The memory bandwidth is normalized to the processing requirement of
4K2K@60Hz (input video is 1080p@60Hz), and the corresponding R EFERENCES
bandwidth for SR is estimated according to the equations illustrated
in [20]. [1] F. L. Luo, W. Williams, R. M. Rao, R. Narasimha, and M. J. Montpetit,
Trends in signal processing applications and industry technology, IEEE
Signal Processing Magazine, vol. 29, no. 1, pp. 174184, Jan. 2012.
[2] R. G. Keys, Cubic convolution interpolation for digital image process-
tile-based architecture is that it tends to dramatically increase ing, IEEE Transactions on Acoustics, Speech and Signal Processing,
the memory bandwidth. According to the memory bandwidth vol. 29, no. 6, pp. 11531160, Dec. 1981.
analysis in [20], its memory bandwidth requirement should [3] C. C. Lin, M. H. Sheu, H. K. Chiang, C. Liaw, and Z. C. Wu, The
efficient VLSI design of BI-CUBIC convolution interpolation for digital
increase to at least 3888 MB/s to support the 4K2K@60Hz image processing, in Proceedings of IEEE International Symposium on
video. Therefore, it is actually infeasible for this SR work to Circuits and Systems, May 2008, pp. 480483.
support 4K2K@60Hz video processing unless it significantly [4] S. L. Chen, H. Y. Huang, and C. H. Luo, A low-cost high-quality
adaptive scalar for real-time multimedia applications, IEEE Transac-
increases its on-chip memory usage, especially considering tions on Circuits and Systems for Video Technology, vol. 21, no. 11, pp.
that video display processing should share external memory 16001611, Nov. 2011.
bandwidth with microprocessor, video decoder and GPU in [5] S. Chen, VLSI implementation of an adaptive edge-enhanced image
scalar for real-time multimedia applications, IEEE Transactions on
video processing SoC. Nevertheless, the memory bandwidth Circuits and Systems for Video Technology, vol. 23, no. 9, pp. 1510
requirement of the proposed design is only 373 MB/s, be- 1522, Sep. 2013.
cause the proposed VLSI circuit employs the line-buffer- [6] S.-L. Chen, VLSI implementation of a low-cost high-quality image
scaling processor, IEEE Transactions on Circuits and Systems II:
based architecture and only needs to load the input video Express Briefs, vol. 60, no. 1, pp. 3135, Jan. 2013.
(19201080@60Hz) from the off-chip memory line by line [7] P. Y. Chen, C. Y. Lien et al., VLSI implementation of an edge-
once. Regarding the logic part, the proposed interpolation oriented image scaling processor, IEEE Transactions on Very Large
Scale Integration Systems, vol. 17, no. 9, pp. 12751284, Sep. 2009.
circuit is also very cost-efficient that it achieves a 4 higher [8] C.-L. Chen and C.-H. Lai, Iterative linear interpolation based on fuzzy
processing capability with less than 66% logic gates compared gradient model for low-cost VLSI implementation, IEEE Transactions
with the SR design. The VLSI implementation comparison on Very Large Scale Integration Systems, vol. 22, no. 7, pp. 15261538,
Jul. 2014.
results further suggest that the proposed explicit edge-directed [9] C. C. Lin, M. H. Sheu, C. Liaw, and H. K. Chiang, Fast first-
adaptive interpolation algorithm can achieve high image qual- order polynomials convolution interpolation for real-time digital image
ity with relatively very low computational complexity and reconstruction, IEEE Transactions on Circuits and Systems for Video
Technology, vol. 20, no. 9, pp. 12601264, Sep. 2010.
implementation cost, and thus is a viable and cost-efficient [10] S. Battiato, G. Gallo, and F. Stanco, A locally adaptive zooming
solution for real-time video display processing applications. algorithm for digital images, Image and Vision Computing, vol. 20,
no. 11, pp. 805812, Sep. 2002.
[11] Y. T. Lai, C. F. Tzeng, and H. C. Wu, Adaptive image scaling based on
V. C ONCLUSION local edge directions, in Proceedings of IEEE International Conference
This work was motivated by the resolution gap between on Intelligent and Advanced Systems, Jun. 2010, pp. 14.
[12] M. J. Chen, C. H. Huang, and W. L. Lee, A fast edge-oriented algorithm
display panel and video source, where the performance of for image interpolation, Image and Vision Computing, vol. 23, no. 9,
image interpolation becomes increasingly important in deter- pp. 791798, Sep. 2005.
mining the perceptual display quality in a variety of video [13] Q. Wang and R. K. Ward, A new orientation-adaptive interpolation
method, IEEE Transactions on Image Processing, vol. 16, no. 4, pp.
processing applications. In particular, this paper is interested 889900, Apr. 2007.
in designing an image interpolation algorithm to meet both the [14] X. Li and M. Orchard, New edge-directed interpolation, IEEE Trans-
quality and real-time performance requirements at a reasonable actions on Image Processing, vol. 10, no. 10, pp. 15211527, Oct. 2001.
[15] N. Asuni and A. Giachetti, Accuracy improvements and artifacts
implementation cost. In this paper, we propose an explicit removal in edge based image interpolation, in Proceedings of Inter-
edge-directed adaptive interpolation method that can main- national Conference on Computer Vision Theory and Applications, Jan.
tain relatively low computational complexity while achieving 2008, pp. 5865.
[16] X. Zhang, S. Ma, Y. Zhang, L. Zhang, and W. Gao, Nonlocal edge-
similar or even better quality, compared with more advanced directed interpolation, in Proceedings of Advances in Multimedia In-
implicit edge-directed methods. formation Processing, Dec. 2009, pp. 11971207.
14
[17] A. Giachetti and N. Asuni, Real-time artifact-free image upscaling, Xuchong zhang received the master degree in con-
IEEE Transactions on Image Processing, vol. 20, no. 10, pp. 2760 trol engineering from Xian Jiaotong University in
2768, Oct. 2011. 2014. He is currently pursuing the Ph.D. degree in
[18] X. Lu et al., Image super-resolution via double sparsity regularized Control Science and Engineering from the institute
manifold learning, IEEE Transactions on Circuits and Systems for Video of artificial intelligence and robotics, Xian Jiaotong
Technology, vol. 23, no. 12, pp. 20222033, Dec. 2013. University. His current research interests are inte-
[19] Z. Zhu, F. Guo, H. Yu, and C. Chen, Fast single image super-resolution grated circuit and system for video signal processing
via self-example learning and sparse representation, IEEE Transactions and computer vision.
on Multimedia, vol. 16, no. 8, pp. 21782190, Dec. 2014.
[20] Y.-N. Liu, Y.-C. Lin, Y.-L. Huang, and S.-Y. Chien, Algorithm and
architecture design of high-quality video upscaling using database-free
texture synthesis, IEEE Transactions on Circuits and Systems for Video
Technology, vol. 24, no. 7, pp. 12211234, Jul. 2014.
[21] Q. Yuan, L. Zhang, and H. Shen, Multiframe super-resolution employ-
ing a spatially weighted total variation model, IEEE Transactions on
Circuits and Systems for Video Technology, vol. 22, no. 3, pp. 379392, Huibin Tao received Ph.D. degree from the Depart-
Mar. 2012. ment of Electronics and Information Engineering in
[22] D. Robinson and P. Milanfar, Statistical performance analysis of super- Xian Jiaotong University, Xian, China, 2014. Now
resolution, IEEE Transactions on Image Processing, vol. 15, no. 6, pp. He is working in the Institute of Artificial Intel-
14131428, Jun. 2006. ligence and Robotics at Xian Jiaotong University
[23] N. Efrat, D. Glasner, A. Apartsin, B. Nadler, and A. Levin, Accurate as Post Ph.D. His current research interests include
blur models vs. image priors in single image super-resolution, in computer vision, digital communication and VLSI
Proceedings of IEEE International Conference on Computer Vision implementation.
(ICCV), Dec. 2013, pp. 28322839.
[24] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, Image quality assess-
ment: from error visibility to structural similarity, IEEE Transactions
on Image Processing, vol. 13, no. 4, pp. 600612, Apr. 2004.
[25] J. E. Caviedes, Intelligent sharpness enhancement for video post-
processing, in Proceedings of European Signal Processing Conference,
Sep. 2006. Jie Yang received the B.S. degree in electrical engi-
neering and automation from North China Electric
Power University, Beijing, China, in 2011 and the
M.S. degree in Integrated Circuit Design Center
from Xian Jiaotong University, Xian, China, in
2014. Her research interests include algorithm de-
sign and hardware implementation of image and
video processing.
Qiubo Chen received B.S. degree from the Depart-

ment of Electronic and Information Engineering in
Xian Jiaotong University, Xian, China in 2010.
He is currently a Ph.D. student in the Institute of
Artificial Intelligence and Robotics at Xian Jiaotong
University. His current research interests include Jizhong Zhao received the B.S. and M.S. degrees
image and video processing, coding and associated in mathematics from Xian Jiaotong University, X-
VLSI design. ian, China, in 1992 and 1995, respectively. He
received the Ph.D. degree in computer science from
Xian Jiaotong University, in 2001. Currently he is
a Professor in Computer Science and Technology
Department, Xian Jiaotong University. His research
interests include computer software, distributed sys-
tems, and computer architecture. He is a member
of the IEEE, the IEEE Computer Society, and the
ACM.
Hongbin Sun (M11) received B.S. and Ph.D. de-

gree in electrical engineering from Xian Jiaotong
Nanning Zheng (SM93F06) graduated from the
University, Xian, China in 2003 and 2009, respec-
Department of Electrical Engineering, Xian Jiao-
tively. He was a visiting Ph.D. student in electrical,
tong University, Xian, China, in 1975, and received
computer and systems engineering department at
the M.S. degree in information and control engi-
Rensselaer Polytechnic Institute, Troy, NY, USA
neering from Xian Jiaotong University in 1981 and
from 2007 to 2008. He was a post-doc in comput-
the Ph.D. degree in electrical engineering from Keio
er science department at Xian jiaotong University
University, Yokohama, Japan, in 1985.
from 2009 to 2011.
He jointed Xian Jiaotong University in 1975, and
Currently he is an Associate Professor in the
he is currently a Professor and the Director of the In-
school of electronic and information engineering at
stitute of Artificial Intelligence and Robotics, Xian
Xian Jiaotong University. His current research interests include memory
Jiaotong University. His research interests include
hierarchy in computer system, VLSI architecture for video processing and
computer vision, pattern recognition and image processing, and hardware
computer vision, and signal processing system for new memory technology.
implementation of intelligent systems.
Dr. Zheng became a member of the Chinese Academy of Engineering
in 1999, and he is the Chinese Representative on the Governing Board of
the International Association for Pattern Recognition. He also serves as the
President of Chinese Association of Automation.

Chen 2015

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chen 2015

Uploaded by

Copyright:

Available Formats

This article has been accepted for publication in a future issue of this journal, but has not been

Algorithm and VLSI Architecture of Edge-Directed

xi-3 xi-2 xxi-1

pixel pi can be calculated through the following equation:

Video data Pixel 1 Pixel

... ... ...

Stage 3 Stage 4 Stage 5

Mean Opinion Score

(a) (b) (c) (d) (e)

(a) (b) (c) (d) (e)

(a) (b) (c) (d) (e)

C. VLSI Implementation Result

The proposed VLSI architecture has been designed with

TABLE V The proposed interpolation method has been extensively

Qiubo Chen received B.S. degree from the Depart-

Hongbin Sun (M11) received B.S. and Ph.D. de-

You might also like