XVIII International Congress of Electronic, Electrical and Systems Engineering

Id: P-074
Pgina 1 de 6


Abstract We consider the design of a CMOS pixel using APS
(active pixel sensor) technology. Additional signal processing
hardware is included at the pixel level, so that a binary stream
representing the image compressed at one bit per pixel is the only
output of the sensor array. The technique chosen for such binary
representation is Floyd-Steinberg error diffusion, which is a well-
known halftoning algorithm. To validate the proposed circuit, we
present electrical simulations of a 5 x 5 pixel block.

Index Terms focal plane image processing; halftoning;
CMOS imager

I. INTRODUCTION

OCAL-PLANE image processing has received a remarkable
amount of attention over the last decade [1], [2]. The same
analog transistors that are used in conventional active-pixel
sensor (APS) imagers can also be used to implement basic
image processing tasks. Because part of the processing is
transferred to the sensor itself, focal-plane image processing
presents several advantages with respect to classical (digital)
image processing, such as an increase in the speed of image
capture, overall hardware miniaturization, and the possibility
of using simpler processors. Recently, algorithms for focal
plane image compression have been studied and reported in
the literature [3], [4], [5].

We introduce a CMOS pixel that includes the feature of
focal-plane image halftoning. We aim at the processing of an
image locally captured by the sensor, through a halftoning
algorithm. Halftoning algorithms [6] are widely used for
example in laser printers. Several halftoning algorithms exist
today [7], [8], but we focus on the classical Floyd-Steinberg
algorithm [9], which is very simple yet efficient and still today
a comparison reference for other algorithms. The captured
image is thus processed yet inside the sensor, before analog-
to-digital conversion is performed. Implementation of
halftoning algorithms in hardware is not commonly found in


This work was supported in part by CNPq/Brazil (processes 478336/2007 -
1 and 477451/2009-8), in part by FAPERJ/Brazil (processes E-26/171.101/
2006 and E-26/102.209/2009), in part by CAPES/Brazil (M.S.Scholarship),
and in part by FUJB/UFRJ/Brazil (ALV2006Program).

the literature and, since no analog hardware implementations
are available for use as a starting point or for comparison
purposes, we consider the simplest possible circuit
implementation. To illustrate the proposed design, we include
simulation results for halftoning of a 5x5 pixel block.

A theoretical description of basic halftoning principles is
provided in Section II. The focal-plane halftoning circuit
design is provided in Section III and simulation results are
shown in Section IV, along with a discussion of those results.

II. THEORETICAL FUNDAMENTALS

The halftoning technique consists in converting continuous-
tone images into images that are solely composed of black and
white dots. Such conversion takes place by means of the
spatial manipulation of black dots as opposed to the white
spaces within them. It is possible to control the size, the
spacing and the density of the dots in order to create the
desired shades of gray. For halftoning, we use the error-
diffusion algorithm proposed by Floyd and Steinberg [9],
which implements an adaptive method based on the
quantization of each pixel according to an statistical analysis
of the input pixel and some of its neighbors. This results in an
stochastic arrangement of output binary pixels. Because of the
analysis of neighboring pixel values, the Floyd-Steinberg
error-diffusion has computational complexity somewhat larger
than the complexity of other halftoning methods, such as
amplitude-modulated techniques [8], if the algorithm is
executed in a digital processor. If the algorithm is
implemented using analog hardware at the focal plane, the
computational cost drops dramatically. This is mostly due to
the parallel structure of the operations. The processing time is
also significantly reduced.







A Pixel for Focal-Plane Halftoning
in CMOS Image Sensors
Saulo A. Nunes and Jos Gabriel R. C. Gomes, Member, IEEE
Electrical Engineering Program Universidade Federal do Rio de Janeiro
Rio de Janeiro, RJ 21941-972, Brazil. E-mails: (saulo,gabriel)@pads.ufrj.br
F
XVIII International Congress of Electronic, Electrical and Systems Engineering



Id: P-074
Pgina 2 de 6


A block diagram representation of the Floyd-Steinberg
algorithm is shown in Fig. 1. It has two functional building
blocks: a quantizer and a filter. The complete algorithm
operation is described by (1), (2), and (3), and the quantizer is
represented by (1). The filter, which is described by (2), is
designed to correctly combine the error information that is
received from the neighboring pixels as they propagate it. The
filter taps are the ones introduced by Floyd and Steinberg in
the case of a left-to-right scan. Figure 2(a) shows the
application of the filter taps in the propagation of the error
from one pixel to its neighbors, and Fig. 2(b) shows the
computation of the error that is diffused into a given pixel.



1, if (x(n) + x ( )) 0
( )
0, else
e
n
y n
>
=

(1)

1
( ) ( )
M
e i e
i
x n b y n i
=
=

(2)
( ( ) ( ))
e e
y y x n x n = (3)





Figure 1. Halftoning algorithm block diagram


Figure 2. Error diffusion among neighboring pixels





III. HALFTONING CIRCUIT DESIGN

All pixels of the APS imager are expected to be identical.
Each pixel has the same specific signal processing circuitry for
focal-plane halftoning and the way the pixels are connected is
shown in Fig. 3. All the circuit elements transistors,
switches, sources and so forth mentioned in this section are
referred to the components shown in Fig. 4. Transistor sizes
are shown in Table 1. For simplicty, our pixel structure is
similar to 3T [1] but with some modifications that are needed
for the appropriate current-mode signal processing. We are not
interested in reading the pixel value as provided by the
photodiode. Instead, we are interested in the final binary
information as processed by the halftoning algorithm over the
entire pixel array.
The voltage that results from integration of photocurrent
generated at the photodiode is converted into a current still
within the pixel, by M2 (with 96 A/V average transconduc-
tance). All signal processing associated with halftoning is
performed in current mode. Current mode signal processing
has two importante advantages: the use of simple circuits, and
the accurate adjustment of multiplier values by means of
transistor sizing.
The analog switches S1 and S2 are non-complementary and
have dummy transistors. They perform current-mode
correlated double sampling (CDS) of the photodiode
integrated charge in order to suppress spurious DC readings.
The M2 current values at the beginning and at the end of the
photodiode charge integration are copied into M4 and M5.
Transistors M4 and M5 then perform the operation I
M6
-I
M4
by
means of the M6-M7 current mirror, which results in the
signal current I
APS
. Figure 5 shows the timing diagram that has
been used to control this operation, in which no data
compression is performed. Only the signal mode is changed,
from voltage mode into current mode. Signal compression is
realized by the circuit that is described next.
The filter is implemented by M8, M9, M10, and M11. The
filter coefficients (Fig. 2) are implemented by the aspect ratio
of these transistors, and the filter output is I
FS
. The I
APS
current
is subtracted from I
FS
and the result of this subtraction goes
into a circuit for absolute value computation [10]. This circuit
computes the absolute value of the input current (denoted by
I
MOD
in Fig. 4) and generates a voltage indicating the sign of
the input current (denoted by V
Y
in Fig. 4). The V
Y
signal
represents the y output that is shown in the block diagram of
Fig. 1 because the absolute value circuit associates to V
Y
a
logical 1 value if I
APS
is positive. A logical 0 is generated
otherwise. This is a comparison against a threshold set to zero,
which is exactly the operation of the quantizer that is shown in
Fig. 1.




(a) (b)
7/16
1/16 5/16 3/16
7/16
1/16 5/16 3/16
XVIII International Congress of Electronic, Electrical and Systems Engineering



Id: P-074
Pgina 3 de 6



+
+
+
+
+
+
1/16
5/16
3/16
7/16
1
7/16
5/16
3/16
1/16

Figure 3. Pixel interconnection diagram

Besides y, we must also generate the input signal for the
Floyd-Steinberg filter. This signal will be used by neighboring
pixels to compute their outputs. The switch S3, the current
source I
REF
, and the current mirror M21-M22 convert the
voltage V
Y
back into a current signal, so that an analog error
current I
Ye
can be generated according to (4):



, V 1
,
REF MOD Y
Ye
MOD
I I if
I
I else
=
=

(4)


It is important to observe that (4) is not the same operation
represented in (1). The goal of (4) is simply converting the y
bit into an analog signal so that the operation in (3) may be
carried out by the analog integrated circuit inside the pixel. In
case y=0, the output current is negative and for that reason one
part of the absolute value circuit is used again to correct the
current direction, making it positive. Thus the second output
of the pixel is the |I
Ye
| current signal, which will be distributed
to the filters in neighboring pixels.

IV. NUMERICAL SIMULATION RESULTS

In order to validate the proposed circuit design method, we
performed numerical simulations of a 55 pixel matrix.

Two different models were used in the simulations: the first
one is the ideal Floyd-Steinberg algorithm (Fig. 1), which was
applied to the Lena image and yielded the result that is shown






in Fig. 6. This shows that the ideal algorithm adequately codes
gray-level images. Besides, this first method was also used to
generate the ideal results expected from 55 pixel blocks (Fig.
7a, Fig. 8a, and Fig. 9a). These ideally generated results are
used as a comparison reference for the results generated using
the second model. This second model is an electrical
description of a circuit implementation of the Floyd-Steinberg
algorithm.

In the electrical model that was used in our simulations,
current sources (described below) in parallel with capacitors
(58 fF, approximately, corresponding to the reverse-biased
junction capacitance of a 10 m 10 m n-well photodiode)
replace the diodes D1 so that photo-generated currents can be
simulated. To simulate different amounts of light in each pixel
of the 55 matrix, a different value is assigned to each current
source. The value of each current was computed with respect
to 55 pixel blocks from the Lena image that is shown, after
error-diffusion in MATLAB, in Fig. 6. The square marks in
Fig. 6 indicate the 55 pixel blocks that are used in the
electrical simulations presented in this paper. For the
conversion of pixel luminance values into current values, we
used (5) [1].



TABLE I. SIZES OF THE TRANSISTORS SHOWN IN FIG. 4
Transistor W(m) L(m) Transistor W(m) L(m)
M1 1 0.35 M16 1 1
M2 1 0.35 M17 1 2
M3 5 5 M18 1 2
M4 5 5 M19 3 1
M5 5 5 M20 1 1
M6 1 2 M21 1 3.4
M7 1 2 M22 1 3.4
M8 3.5 1 M23 1 2
M9 2.5 1 M24 1 2
M10 1 1 M25 1.4 1
M11 0.5 1 M26 4 1
M12 1 2 M27 3 1
M13 1.4 1 M28 1 1
M14 4 1 M29 1 2
M15 3 1 M30 8 1


XVIII International Congress of Electronic, Electrical and Systems Engineering



Id: P-074
Pgina 4 de 6


Figure 4. Overall pixel circui pixel



Figure 5. Photoconversion process control timing diagram

The image sensor has been designed to capture light at
illumination levels (Lo) up to 10.000 lux with photodiodes
having 10
-10
m
2
(i.e. 10 m 10 m). Assuming that the
responsivity [11] of silicon is approximately 0.3, Eq. (5) yields
480 pA as the maximum photo-current that will be generated
in the photodiode. The maximum current level is multiplied by
a matrix of normalized luminance values (between 0 and 1), to
yield a 55 photo-current matrix.

Iph Rph Lo A = (5)

To simplify computations, the results shown in Figs. 7, 8
and 9 were obtained from the green field of the RGB Lena
image instead of the luminance field. The spatial initial
conditions at the upper and left block boundaries were the
same in the MATLAB simulations and in the electrical
simulations with Cadence software. Whenever a neighbor
pixel is not available for incoming error propagation which
happens at the upper row and left column pixels the input is
assumed zero in MATLAB simulations and the input is
connected to ground in electrical simulations. Figure 7(a)
shows the result of the MATLAB halftoning of 55 pixel
block #1 according to (1), (2) and (3). The result obtained
from electrical simulations based on the circuit of Fig. 4 is
shown in Fig. 7(b).


XVIII International Congress of Electronic, Electrical and Systems Engineering



Id: P-074
Pgina 5 de 6



The pixels shown in gray squares have values different from
the MATLAB reference output. Figures 8 and 9 show the
same type of result, obtained with pixel blocks #2 and #3. The
errors correspond to 28% of the pixels in Figs. 7 and 8, and
40% in Fig. 9. The considerably larger amount of errors in
Fig. 9 is probably due to the higher amount of detail in the
pixel block #3, and further tests are being conducted to
address this issue.
We consider the results satisfactory because the probable
major cause for the errors is charge injection in S1 and S2,
which can be treated. Charge injection effects were observed
during the photodiode readout circuit design and some
preliminary changes were made in order to reduce such
effects. The sizes of M3, M4, and M5 were increased (from 2
m 2 m to 5 m 5 m, i.e. approximately six times) so
that their gate capacitances were much larger than the parasitic
capacitances in S1 and S2, which resulted in a significant
reduction of charge injection (also approximately six times) as
shown in Fig. 10. However, the charge injection effects still
lead to considerable signal loss with respect to the ideal
simulations: since the pixels are interconnected, an error in the
binary output of one pixel will accumulate over all the
following neighbors, which also suffer from their own charge
injection effects. In future work, a study of non-ideal effects
caused by charge injection and noise associated with the
reduced ratio between photodiode area and overall pixel area
will be carried out.


Figure 6. Binary halftone of Lena image as computed in software according
to the Floyd and Steinberg error diffusion algorithm. The blocks used for
comparison with electrical simulations are marked with a square box.







Figure 7. Block #1: halftoning simulations in MATLAB (a) and Virtuoso (b).





Figure 8. Block #2: halftoning simulations in MATLAB (a) and Virtuoso (b).





Figure 9. Block #3: halftoning simulations in MATLAB (a) and Virtuoso (b).










Block #1
Block #2
Block #3
(a) (b)
(a) (b)
(a) (b)
XVIII International Congress of Electronic, Electrical and Systems Engineering



Id: P-074
Pgina 6 de 6








Figure 10. Reduction of charge injection effects in S1 and S2 as the sizes of
M3, M4, and M5 are increased to yield larger gate capacitances.


REFERENCES


[1] J. Ohta, Smart CMOS Image Sensors and Applications, CRCPress, 2007
[2] J. Nakamura (Editor), Image Sensors and Signal Processing for Digital
Still Cameras, CRCPress, 2005.
[3] W. D. Len-Salas et al., A CMOS imager with focal plane compression
using predictive coding, IEEE J. Solid-State Circuits, 42(11):2555-
2572, Nov. 2007.
[4] E. Artyomov and O. Yadid-Pecht, Adaptive multiple-resolution CMOS
active pixel sensor, IEEE Trans. Circuits and Systems I: Regular
Papers, 53(10):2178-2186, Oct. 2006.
[5] Y. M. Chi et al., An active pixel CMOS separable transform image
sensor, in Proc. IEEE Int. Symp. Circuits and Systems, Taipei, Taiwan,
May 2009, pp. 1497-1500.
[6] D. Anastassiou, Error Diffusion Coding for A/D Conversion, IEEE
Trans. Circuits and Systems, 36(9):1175-1186, Sep. 1989.
[7] D. L. Lau and G. R. Arce, Modern Digital Halftoning, 2
nd
Edition, CRC
Press, 2008.
[8] K. R. Crounse, T. Roska and L. O. Chua, Some methods for practical
halftoning on the CNN universal machine, in Proc. Fifth IEEE Int.
Workshop on Cellular Neural Networks and Their Applications, April
1998, pp. 337-342.
[9] R. W. Floyd and L. Steinberg, An adaptive algorithm for spatial
grayscale, in Proc. Society for Information Display, 17(2):75-77, 1976.
[10] S. Mehta and R. Etienne-Cummings, A simplified normal optical flow
measurement CMOS camera, IEEE Trans. Circuits and Systems I:
Regular Papers, 53(6):1223-1234, Jun. 2006.
[11] G. C. Holst and T. S. Lomheim, CMOS/CCD Sensors and Camera
Systems, JCD Publishing and SPIE Press, 2007.