A Cooperative Network for Contour Grouping

Frank Wlangin I\/la,rc Berthod J osiane Zerubia

INRIA Sophia.-Antipoljs
2004 route des Lucioles
06565 Valbonne Cedex
Fr an ce
Abstract
A ne w i t erat i ve al gori t hm for edge i nt ensi t y i mage
enhancement i s proposed. It uses local cooperation-
i nhi bi t i on processes t o prodzice an edge i mage i n which
t he m,ost r el evad cont ours have reached maxi mal ac-
t i vat i on, and smal l gaps and j unct i ons have been jilled
i n. Imp1 e m e nt at i o n on a i n assively para1 1 e 1 m a chi n e
has provided high speed perf ormances, a n d results of
experiiiieiatatioiis wi t h real scene i mages are presented.
The al gori t hm i s robust i n compl ex edge i mage cont ext ,
and i s perfectly stable under any number of i t erat i ons.
1 Introduction
Contour detection in digital image processing is
a fundamental step towards scene interpretation. A
large class of methods, most widely used to extract
image contours, consists in applying to each pixel a
local discontinuity detector filter, and in thresholding
tlie result to produce an edge map.
The information extracted this way is difficult to
use directly for scene analysis, because there is no
straightforward relationship between digital image in-
tensities and contours or surfaces of tlie scene real
objects. Though, the human visual system performs
scene interpretation using only perceived light inten-
sities. The first step of this interpretation is known
as Perceptual Grouping. During this preattentive
process, regions are merged or splitted, new edges are
inferred, some irrelevant edge elements disappear, fol-
lowing rules first studied by Gestalt psycliophysiol-
ogists (see [SI). I t should be emphasized that these
processes precede the interpretation phase, and thus
involve no high-level knowledge. This step of per-
ceptual orgaiiisation is in our opinion a key coiidition
for the success of interpretation.
2 Contour Grouping
problem for two reasons:
Contour grouping in aa edge image is a difficult
0 the number of possible edge configurations is al-
most infinite, thus deciding what the best group-
ing is relies on very complex rules
0 these rules depend on globa,l contour caracteris-
tics, which are difficult to sxtract
These observations imply t1ia.t a, grouping contour
model must iinpleineiit int,era.c,tions between distant
edge elements. Several a.pproaches have been used for
this purpose. A diffusion plienoinenoii is obtained by
iterating local upda.ting of field elements in Markov
Random Field models (see [ 2 ] , [SI). Here a pixel value
modification through one itera.tion only depends on a
neighbourhood whose size is given by the order of the
random field. As a consequence, the interaction rules
between distant, pixels are defined in an implicit way,
which provides few cont,rol over their definition as long
as a multi-scale approach is not used.
Another solution is to propagate inforination along
the contours. This approa.ch was used in [7] to detect
salient contours. Here more accura.te control over the
iiitera,ction rules is provided; however the time needed
for two pixels to intera.ct is proportional to their dis-
tance along the contour.
The general framework wepropose to use is a model
of local interactions between objects, following rules
depending on a. predefined set of attributes attached to
each object. So a.s to meet time-efficiency constraints
and accurate definition of the interaction rules, wealso
believe that a. multiscale a.pproach should be used.
These choices both provide an efficient way of com-
puting global contour features, and allow accurate de-
sign of the grouping rules, which is in our opinion the
first requirement to obtain good results on real images.
The algorithm described below is the first step to-
wards this model. I t implements local interactions for
the lowest scale only, where objects a.re the pixels of
an edge inmge.
Rehted works include relaxatmion labeling applied
to contour enhancement (see [9]). In our case, using
315
0-8186-2924%7/92 $3.00 @ 1992 IEEB
more coinplex rules lead to better gap filling capa-
bilities as well as perfect stability for any number of
iterations. Significant theoretical advances in coniput-
ing curve orientation and curvature for curve inference
have been made by Parent and Zucker in [GI . Their
results should prove to be useful for the next elements
of our model, when higher scale will require inore elab-
orated curve segment inodelisation. The cooperation-
inhihition concept has also been used by Grossberg
and Riliiigolla (see [3]). In that work, iteration of lo-
cal interactions allowed to implement coiiiplex contour
grouping rules observed in human vision. However
working with inore coiiiplex images coming from real
world scenes lead us to use simpler and thus more ro-
bust rules.
3 Model Overview
interactions a.lready implemented:
Five attributes have been used for the lowest scale
0 the edge element intensity, which is upda.ted a,t
ea.cli iteration
0 two caracteristic coiit,our directions as defined be-
low, used in a cooperat,ioii step
0 two reference values computed over the pixel
neighbourhood, used in an inhibition step.
The algorithm is obta.ined by iterating t.lie following
sequence:
0 evaluation of attributes
0 application of the upda.ting rules.
To a.chieve time efficiency constraint,s, the algo-
rit,lim n7as designed t,o allow inassively parallel coin-
puta.tions, while using oiily t,luee siinple basic opera-
tions:
0 convolut,ion of t,he pixel neiglibourhooil Iby pre-
comput,ed kernels.
0 masinia extra.ction
0 thresholding
Experiinenta.1 results show t1ia.t this choice lead to very
high speed performances.
The a.lgorithni startas with iiimges of edge int,ensi-
ties. They can be obta.inetl for example from an edge
or line detection process applied on a graysmle ima.ge,
followed by a no-ina.xima suppression in t,he contour
direction. The final image is obt,ained by iteration of
a two-step process over the image of edge int,eiisities:
0 Local cooperation: a,t positions where an incom-
ing contour has been det,ected, compute a. best
coiit,iiiuatioii directmion, and upda.te neiglibouriiig
edge intensities accordingly.
0 Inhibition: a no-maxima suppression is performed
at each position, in the maximal reinforcement
direction defined below.
The result, composed of the final edge intensities, is
obtained in less t1ia.n ten it,erations, without any post-
processing.
Two parameters control t,he algorithm:
0 an activation threshold , which selects the po-
sitions that ta.ke pa.rt in t,he cooperation step.
0 a continuation threshold.
A local coopera.tion first takes place between neigh-
bouring edge elements. For each hctive edge ele-
ment (i.e. whose iiitensity is greater than the activa-
tion threshold), a support, due is first computed
for each of 16 predefined directxiom.
16 convolution kernels are defined to compute the
support, values. Latera.1 inhibition is used to avoid
selecting a. high intensity orthogona.1 contour ii1stea.d
of a low intensity but well-oriented one. So the kernels
ha.ve a positive center and a negative surround along
the direction of int,erest,, and dues decrease with the
d is t a.n ce .
The maximal support va.lue is then extracted, and
defines the iiicoining direct,ion . The remaining sup-
port values a.re then weighted according to this incom-
ing direction, so a.s to fa.vor st,raight continuations: the
weights decrea.se from one for the straight continuation
down to 0 a.s tlie angle with tlie incoining direction
becomes inore acute. A second maximum is extracted
over t,liese inoduhted values, and if it is greater than
the cont.inuation threshold, i t defines the continua-
tion direction. The continuation threshold is thus
t.he value below which a posit.ion should be rega.rded
as a real end-point, coiisideriiig the inforiimtions avail-
able at this scale.
These a.ttributes a.re used to update the edge in-
tensity field. Since most positions lie on alrea,dy well-
established contours, we dont want them to create
new a.ctive positions. This is why end-points are first
detect,ed: a. position is defined to be an end-point if
aniong its S nearest neighbours, only one has a non-
zero edge intensity, or t.wo have noli-zero intensities
and are connected in a. konnexi ty pattern, and a
position whose intensity is zero before the coopera-
tion can be updated only by end-points. Each ac-
tive edge element updat,es its neighbours depending
011 the incoining and cont,inuat,ion directtion previously
detected. The updating a.mount only depends on the
intensity of the upda.ting pixel, and is independent
of t,he previous edge element. int,ensity.Fina.lly, Since
there are inhibition terins in the contour kernels, the
316
intensities obtained after the cooperation step a.re set
to zero when negative.
After the cooperation step, which acts as a. hypoth-
esis generator, we have to select the optimal contour
positions. No-maxima suppression along the gra.dient
orientation has often been used for this purpose in
other models. However using a contour orientation
leads to eliminate junction edge elements, as shown in
the left, exa.inple of Figure 1.
A quitme natural approa.ch successfully solves t.liis
problem: let us define the maximal reinforcement. ori-
entation as the orientation for which an edge element
received its inaxiinal contribution in the cooperation
step (right example of Figure 1). This maximal rein-
forceinent orientation is in some wa.y the reason why
a. position 1ia.s a high intensity. The key idea. is to set
the int,ensity of a position to zero only if a neighbour-
ing position obtained a higher intensity for i h e s a me
r e as m. 4 reference value is computed on ea.cli side
of a. positmion ort,liogonally to t,he nia.xiina1 reinforce-
ment orientation, The edge intensities are upclat,ed by
siinply setting to zero those which a.re less t,lian oiie
of t,Iieir reference values. Fina.lly, a global t~liresholtl-
ing is performed on the image to inainta.iii t,lie edge
intensities below 1 (see [4] for further details).
f
ReferenFepixels
---_
Maximal
reinforcement
orientations
Figure 1: Effect of no maxima suppression on
junction pixels
4 Convergence and Stabilisation
One reniarka.ble fea.ture of t,he algorithm is that, t,he
result,ing image rema.ins very sta.ble under any 1111111-
ber of it#erations. This has been achieved by strictly
selecting the edge elements t.1ia.t take part, in t81ieco-
operation step.
Interesting groupings however are obt,a.ined in far
less itera.tions, and all the results presented herein
have been obtained after 10 iterations.
At any given active position, two cases may occur:
0 a coiltinuation cont,our is detected in the neigh-
bourhood of radius 4: since the ra.dius of the
neighbourhood used for upda.ting is 2, the gap
will be filled in 2 or 3 it,eratioiis.
0 if no continuation cont,our is detected, no updat-
ing ta.kes place
This explains why very few itera.tions are required.
More iterations a,re usefull oiily in one mse: a con-
tour formed by a long section of edge elements whose
intensities a.re sma.ller than the a.ctiva.tion threshold,
and with soiiie high intensity edge eleinents which will
propgate t.he activa.tion 2 ~iixels further at each iter-
ation. Thus the choice of 10 itera.tioiis corresponds to
contours with non-zero iiia.ct.ive sections of maximal
length 20, which covers the ma.jority of contours in
most images. If many such contours are expected in
an image, the activa.tion threshold should be lowered.
5 Experimental Results
The algorithm has heen iinpleinented on a. Con-
nection Ma.chine 2 system wi th a Sun4/390 front-end.
The prograin is wri tkn in C, a. parallel extension of
C for the Connection Machine.
\Irit,liout, any opt,imiza.tion, 10 iterations a.re per-
forined in &out 3 seconds wlien using one processor
per pixel in the image (e.g. 128x128 image on the 16k
processor Connection khcliine of I NRI A). This level
of performaace was achieved by liiniting the algorithm
to very ba.sic opera.tions, a.s described above. Writing
the prograin in a lower-level 1a.nguage (like C-Paris
for the Connection Machine) with some optimization
should inalre the algorithm at least twice as fast.
A st,ep edge detect,or has been applied to the gray-
level image of an outdoor scene presented in Figure
2. Deriche algorithm, a recursive implementation of
filt8ers deriver1 from Canny criteria. ([I]), 1:a.s been used
for this purpose. Classica.1 no-maxima suppression in
the gradient direction was then a.pplied to form the
a.lgorithm input image, shown in Figure 3.
The result presented in Figure 4 was obtained af-
ter 10 iterations, and is the exa.ct output of the algo-
ritliin without any thresholding, although low inten-
sities are invisible in print>ecl images. The a.ctiva.tion
threshold was set, to 0.05 a.nd t,he continmtion thresh-
old t.0 0.025.
The relevant cont,our elements all rea.ched sa.tura-
t,ion value 1 ~ while the \ due of isola.ted ones reinained
const,a.nt,, aiitl no t.liicl; lines 1ia.ve been introduced by
t8he cooperatmion process. Most junctions and small
gaps were successfully completed, as long as their
size is less than t,he size of the intera.ction window.
Longer gaps have been int,erpreted as contour end-
points, which is t,he relevant choice at this scale, and
110 further diffusion has talteii pla.ce at the end-points.
317
The top of image 2, where many contours a.re only
one pixel apart, deinonstra,tes the robustness of the
process in a. dense edge image contest (compare Fig-
ure 3 with Figure 4).
I n order to a,ddress the problem of road detection,
the algorithm has a.lso been applied on satellite im-
a.ges, using a. ridge detector to forin the initial image.
Full experimental results are provided in [4].
Perception, David Beardslee and Michael Wertheimer.
Eds., (Princeton, N.J ), 11.55135, 1958.
application of relasation labeling to line and curve en-
hancernent. I EEE Tmnsoctions on Computers, Vol.
C-26(4), Apr. 1977.
[91s, NI , zucker, R, A, ~ ~ ~ ~ ~ ~ , , ~ l , alld A, Rosenfeld,
6 Coiiclusioii
A new iterative algorithm that exhibits power-
full contour enhancement capabilities has been imple-
mented, and has been successfully a.pplied to complex
innges of real scenes. I t achieves a very high speed
efficiency, while being perfectly stable under any nun-
ber of itera.tions. Thi s algorit,hin a.pplied to edge im-
ages grea.tly improves the results of subsequent con-
tour chaining or recognit,ion processes.
However it is pa.rt of a. inore global archit,ect,ure,
which is currently being implemented. Thi s inulti-
scale model should provide an efficient iinpleinentation
of complex rules for long-ra.nge interactions, which are
required for perceptual contour grouping.
Re fe re ii c e s
[l] R. Deriche. Using Ca.nnys criteria to derive a re-
cursively implemented optimal edge detector. Inter-
notional Journal of Computev Ifision, pages 167-187,
1987.
[23 S. Geman and D. Geman. Stochastic relamtion, Gibbs
distributions, and the bayesian restoration of images.
I EEE Transactions on Pattern Analysis and Machine
IntelZiyence, Vol. PAMI-6(6), Nov. 1984.
[3] S. Grossberg and E. Mingolla. Neural dynmiics of per-
ceptual grouping: Test,nres, boundaries, and emergent,
segmentations. Perception 63 Ps ychoph~s i cs , Vol. 38-
2:141-171, 1985.
[71
F. Mangin, J . Zerubia, and M. Bertliod. Local edge
grouping by simple process iteration. Technical Report,
1559, INRIA, November 1991.
J . L. Marrocluin. A markovian raudom field of piece-
wise straight, lines. Bioloyicol Cybernetics, 61:457-465,
1989.
P. Parent and S. W. Zucker. Trace inference, curva-
ture consistency, and curve detection. I EEE Truns-
a.ctions on Padtern Analysis a.nd Madtine Intelligence,
Vol. PAMI-11(8), Aug. 1989.
A. Shaashua and S. Ullman. Structural saliency: The
detection of globally salient, struct,ures using a. locally
connected network. In Proc. NI PS 90, 1990.
Figure 2: Source iinage
Figure 3: All edges detected after edge detection +
no-niasiina supression
[PI M. Wert,lieimer. Untersuchungen zur lelire von der
gestalt ii. Psycho/. For.sch., ( 4) , 1923. Tra.nsla.t,ecl as
Principles of Perceptual Organization, in Readings in
Figure 4: Resulting image after 10 iterations
318