1

Image Reconstruction with Tikhonov Regularization

2

Introduction
Our goal is to produce an image from an array of data that hits the sensor below the diffraction grating.
Though this data appears to be senseless, it is really a transformation of the image which created it. As
an arbitrary point source of light travels toward the device, it will interact with the grating in some way
that will create a predictable pattern on the sensor. Since an image is a sum of point sources, the
resulting pattern on the sensor will be the sum of the point spread functions of each point source.
Once the way in which the grating affects any given point source is known, the transformation of
image data into sensor data can be fully represented in matrix form. The matrix maps weighted sums of
image pixels onto each pixel in the sensor. At this point, any image (represented as a vector of pixels)
can be multiplied on the left by this transformation matrix in order to calculate the resulting pattern on
the sensor. Letting vector represent the original image, vector represent the observed data, and
matrix represent the transformation matrix, we can write this idea in the following form:

Translation: observed data = transformation of image
This equation is called the forward equation because it maps the image forward onto the sensor. If we
can come up with the inverse equation, we can run the process in reverse and go from sensor data to
the image.
Just to get the main idea across and not get sidetracked by the details, lets assume that both the image
and the sensed data have the same number of pixels. This makes our lives easier since it implies that
is a square matrix. If our image and sensor data were of different sizes, would not be a square matrix,
and setting up the reconstruction problem would just require a few more matrix algebra steps. With
our simple assumptions, the equation seems to be easy to solve:
If:
Then:

where A
-1
is just the inverse of the matrix A, which can be calculated with Matlab or another similar
program. If this solution seems too good to be true, thats because it is. In practice, our above If
statement is flawed. The measured data is more than just the transformation of the image, its the
transformation of the image plus some noise. This noise can really cause problems, since the noisy
sensor data could be something that could not have been mapped to from any image data. Because of
this, the inverse problem is ill-posed.
Fortunately, Tikhonov regularization gives us a way to make sure that A is invertible, and we can still use
the idea that we had in the above solution. All we have to do is modify A in such a way that its
guaranteed to be invertible while still acting somewhat like the original transformation A. To do this, we
add some small constant to each entry along the main diagonal (top left to bottom right) of the matrix
A. Symbolically, this means:


3

Replace A with
where is some small number and I is the identity matrix (ones along the main diagonal, zeros
elsewhere). Since is small, were not changing A by much, but the addition of this small ridge along
the diagonal of A makes it invertible. With the same reasoning as before, we can now conclude that:

This matrix equation can be directly computed in Matlab to calculate for the image from the sensed
data , under the assumption that and have the same number of pixels. Otherwise, if the size of the
sensed data and the image we are trying to construct are different, the more general equation would
be:

where the transpose matrix A
T
is used to construct a square matrix. Even this more complicated
equation can quickly be computed in Matlab to reconstruct the image.

Creating a Point Spread Function
In Figure 1 below, Matlab turns a point source (1) into some sensed data (2), which is then turned back
into a point source (3). In this case, was arbitrarily chosen to be 0.02. Since I do not have access to the
cool spiral point spread function, I had to make one of my own. With 100-by-100 pixels, the
transformation matrix A has ten million entries, so creating it was slightly challenging. After some trial
and error, I ended up with a function that spreads points, which is good enough to get an idea of how
this kind of math works.
Figure 1: My point spread function acting on a point source




4

Reconstructing Images
Now applying this point spread function to a whole image instead of just a point source (Figure 2), we
can use the same math as before to reconstruct an entire image. Note that this time I added 10%
random noise to the sensed data as well, which is noticeable in the reconstructed image. Had I not used
Tikhonov regularization to reconstruct the image, the computer may not have been able to solve the
inverse problem due to the noise. In fact, I tried to solve it by simply inverting A, and received an error
that the matrix was singular to machine precision (it could not be inverted).
Figure 2: Turning an image into nonsense, and then back into an image
A color picture can be split into its RGB components, so this same process can be applied three times in
order to reconstruct a color image. In Figure 3 below, each column was calculated with information only
from the column to its left, and information about the point spread function. The fact that this is an
inverse problem is clearly visible by the symmetry of the overall process.
Figure 3: Reconstructing a color image


5

QR Codes
QR codes are especially fun to reconstruct, since we already know a good amount of information about
them to begin with. In Figure 4, a QR code is fed into the forward equation and turned into ideal
noiseless sensor data (2a). This sensor data, plus 15% noise (2b) results in noisy sensor data (2c). From
only (2c) and information about the point spread function, an image is reconstructed. The
reconstructed image can be further improved by snapping each pixel to either black and white, since we
know that the image is binary. Finally, many of the out-of-place pixels can be washed away by applying
some known spatial properties to the image (for instance, no one pixel should be surrounded entirely by
others of another color). After this is done, only a few erroneous pixels remain (which could also be
removed with more thorough analysis).
Below each image on the right is shown its error, defined as the reconstructed image minus the original
image, which decreases as the image is cleaned up.
Figure 4: Reconstructing a QR code
Its interesting that the QR code on the far right was computed without using the QR code on the far left
in any way other than to compute the ideal sensed data. Essentially, the original QR code could be
deleted once the sensed data is computed, and the resulting reconstructed image would still be the
same. In the case of a lensless camera, the computer never has access to the original image in the first
place, but is still able to reconstruct the image from the sensed data using the same math.