Report (Project) 232

Adaptive bilateral filter for image enhancement and noise removal
Chapter I
INTRODUCTION TO IMAGE PROCESSING
1.1 INTRODUCTION
Confucius said, A picture is worth a thousand words. Pictures are the most common and convenient means of conveying or transmitting information. Pictures concisely convey information about positions, sizes and inter-relationships between objects. Out of the five senses: sight, hearing, touch, smell and taste which humans use to perceive the environment. In these, sight is the most powerful since receiving and analyzing images forms a large part of the routine cerebral activity of human beings throughout their waking lives. In fact, more than 99% of the activity of the human brain is involved in processing images from the visual cortex. A visual image is rich in information. On an individual or person to person basis images are used to reason, interpret, illustrate, represent, memorize, educate, communicate, evaluate, entertain, etc. Humans have evolved very precise visual skills we can identify a face in an instant, differentiate colours, process a large amount of visual information very quickly. Hence we can agree with what Confucius said.
1.2 IMAGE PROCESSING

Image Processing is a technique to enhance raw images received from cameras/sensors placed on satellites, space probes and aircrafts or pictures taken in normal day-to-day life for various applications. Various techniques have been developed in Image processing during the last four to five decades. Image Processing systems are becoming popular due to easy availability of powerful personal computers, memory devices, graphics software etc.
1.3 METHODS OF IMAGE PROCESSING

The two methods of Image Processing are: Analog image processing Digital image processing
Dept of ECE, RRCE, Bangalore
2011-12
1.3.1 ANALOG IMAGE PROCESSING

Analog Image Processing refers to the alteration of image through electrical means. The most common example is the television image. The television signal is a voltage level which varies in amplitude to represent brightness through the image. By electrically varying the signal, the displayed image appearance is altered. The brightness and contrast controls on a TV set serve to adjust the amplitude and reference of the video signal, resulting in the brightening, darkening and alteration of the brightness range of the displayed image.
1.3.2 DIGITAL IMAGE PROCESSING

Digital image processing refers processing of the image in digital form. Modern cameras may directly take the image in digital form but generally images are originated in optical form. They are captured by video cameras and digitalized. The digitalization process includes sampling, quantization. In this case, digital computers are used to process the image. The image will be converted to digital form using a scanner, digitizer and then process it. It is defined as the subjecting numerical representations of objects to a series of operations in order to obtain a desired result. It starts with one image and produces a modified version of the same image.
1.4 ADVANTAGES OF DIGITAL IMAGE PROCESSING

The advantages of Digital Image Processing methods are:
Digital image processing allows a wider range of algorithms to enhance the images. It can avoid problems such as the building up of noise and signal distortion during Digital image processing may be modeled in the form of multidimensional systems. It preserves original data precision. Digital image processing provides real instant images. Processing is easy, efficient is more. Digital signal processing systems are attractive due to their low cost and reliable. It has good accuracy, small physical sizes and very flexible.
processing.
2011-12
1.5 IMAGE
It is basically defined as 2-D function f(x,y) where x,y are spatial co-ordinates of amplitude f at x,y is called as gray level or intensity at that point as shown in Fig 1.1.
Fig 1.1: An image as a function.
1.5.1 TYPES OF IMAGE

There are basically 2 types of image: Analog image Digital image Analog image: Analog image can be mathematically represented as a continuous range of values representing position and intensity. Analog images are physical images that are created when the film in a camera is exposed to light. The chemical coating on the film strip reacts to the light and burns an image permanently onto the film. Digital image: Digital image is restricted in both spatial co-ordinates and in its allowed intensities. It is composed of finite number of elements each of which has particular location and value called picture elements shortly termed as pixels or pels. The address of a pixel corresponds to its physical coordinates. Each pixel is a sample of an original image, more samples typically provides more accurate representations of the original image. The intensity of each pixel is variable.
1.5.2 DIGITAL IMAGE REPRESENTATION

Dept of ECE, RRCE, Bangalore 3 2011-12
Adaptive bilateral filter for image enhancement and noise removal An image may be continuous with respect to x and y co-ordinates and also in amplitude. Converting such an image to digital form requires quantization as well as sampling. Digitizing the amplitude values is called quantization and digitizing the co-ordinate values is called sampling. The image shown in Fig 1.2 has been divided into N = 16 rows and M = 16 columns, this is termed as sampling. The value assigned to every pixel is the average brightness in the pixel rounded to the nearest integer value.
Fig 1.2: Sampling of a continuous gray scale image. In a 8-bit grayscale image each picture element has an assigned intensity that ranges from 0 to 255. This image will have many shades of gray as in Fig 1.3. A normal grayscale image has 8 bit colour depth = 256 grayscales and this is termed as quantization.
Fig 1.3: Each pixel has a value from 0 (black) to 255 (white).
Images can be represented in the form of a matrix as shown in Eq (1.1) Dept of ECE, RRCE, Bangalore 4 2011-12
Right side of Eq (1.1) is the digital image in the form of matrix. Each element in this matrix is called pixel or pel which stands for picture elements. M and N are used to denote the number of rows and columns.
1.5.3 BITS PER PIXEL

The number of distinct colours can be calculated using the Eq (1.2)
where n is number of bits per pixel. The number of distinct colours that can be represented by a pixel depends on the number of bits per pixel as shown in Table 1.1. BITS PER PIXEL NO. OF COLOURS IN POWERS OF 2
1
NO. OF COLOURS 2 colours (monochrome) 4 colours 8 colours 256 colours 65,536 colours (High colour) 16.8 million colours (True
Bpp 2 bpp 3 bpp 8 bpp 16 bpp 24 bpp
2 22 23 28 216 224
colour)
Table 1.1: List of no. of colours for different bits per pixels.
1.5.3.1 BINARY IMAGE Each pixel is just black or white. Since there are only two possible values for each pixel, we only need one bit per pixel. Such images can therefore be very efficient in terms of Dept of ECE, RRCE, Bangalore 5 2011-12
Adaptive bilateral filter for image enhancement and noise removal storage. Images for which a binary representation may be suitable include text such as printed or handwriting, fingerprints, or architectural plans. In this image, we have only two colours: white for the edges, and black for the background.
Fig 1.4: Binary image. 1.5.3.2 GRAYSCALE IMAGE Each pixel is a shade of gray, normally from 0 (black) to 255 (white). This range means that each pixel can be represented by eight bits, or exactly one byte. This is a very natural range for image. Other gray scale ranges are used, but generally they are a power of 2. Such images arise in medicine (X-rays), images of printed works, and indeed different gray levels are sufficient for the recognition of most natural objects.
Fig 1.5: Gray scale image.
1.5.3.3 COLOUR IMAGE
2011-12
Adaptive bilateral filter for image enhancement and noise removal For colour depths of 15 or more bits per pixel, the depth is normally the sum of the bits allocated to each of the red, green and blue components. High colour usually meaning 16 bpp, normally has five bits for red and blue and six bits for green as the human eye is more sensitive to errors in green than in the other two primary colours. For applications involving transparency, the 16 bits may be divided into five bits each of red, green, and blue, with one bit left for transparency. A 24-bit depth allows 8 bits per component.
Fig 1.6: High colour image.
1.5.4 DIFFERENT FORMATS OF IMAGES Format of the images may be of the following types as shown in Table 1.2.
FORMAT NAME TIFF JPEG GIF BMP PNG XWD DESCRIPTION Tagged Image File Format Joint Photographic Expert Group Graphics interchange Format Windows Bitmap Portable Network Graphics X Window Dump EXTENSIONS .tif, .tiff .jpg, .jpeg .gif .bmp .png .xwd
Table 1.2: List of image formats.

Digitizer Mass Storage
1.6 COMPONENTS OF IMAGE PROCESSING

Image Processor Digital Computer Operator Console
Display
Hard Copy Device
2011-12
Fig 1.7: Block Diagram of a Typical Image Processing System.
Image processing system consists of the following components:-
1.6.1 DIGITIZER
A digitizer converts an image into a numerical representation suitable for input into a digital computer. Some common digitizers are: - microdensitometer, flying spot scanner, image dissector, Videocon camera, photosensitive solid- state arrays.
1.6.2 IMAGE PROCESSOR

An image processor does the functions of image acquisition, storage, pre-processing, segmentation, representation, recognition and interpretation and finally displays or records the resulting image. The following block diagram gives the fundamental sequence involved in an image processing system. As detailed in Fig 1.8, the first step in the process is image acquisition by an imaging sensor in conjunction with a digitizer to digitize the image. The next step is the pre-processing step where the image is improved being fed as an input to the other processes. Pre-processing typically deals with enhancing, removing noise, isolating regions, etc. Segmentation partitions an image into its constituent parts or objects.
The output of segmentation is usually raw pixel data, which consists of either the boundary of the region or the pixels in the region themselves. Representation is the process of transforming the raw pixel data into a form useful for subsequent processing by the computer. Description deals with extracting features that are basic in differentiating one class of objects from another. Recognition assigns a label to an object based on the information provided by Dept of ECE, RRCE, Bangalore 8 2011-12
Adaptive bilateral filter for image enhancement and noise removal its descriptors. Interpretation involves assigning meaning to an ensemble of recognized objects. The knowledge about a problem domain is incorporated into the knowledge base. The knowledge base guides the operation of each processing module and also controls the interaction between the modules. Not all modules need be necessarily present for a specific function. The composition of the image processing system depends on its application.
1.6.3 DIGITAL COMPUTER

Mathematical processing of the digitized image such as convolution, averaging, addition, subtraction, etc. are done by the computer.
1.6.4 MASS STORAGE

The secondary storage devices normally used are floppy disks, CD ROMs etc.
1.6.5 HARD COPY DEVICE

The hard copy device is used to produce a permanent copy of the image and for the storage of the software involved.
1.6.6 OPERATOR CONSOLE

The operator console consists of equipment and arrangements for verification of intermediate results and for alterations in the software as and when required. The operator is also capable of checking for any resulting errors and for the entry of requisite data.
1.7
FUNDAMENTALS OF IMAGE PROCESSING
2011-12
Fig 1.8: Fundamentals of image processing. Fundamentals of image processing are:-
1.7.1 IMAGE ACQUISITION

Digital image acquisition is the creation of digital images, typically from a physical scene. The term is often assumed to imply or include the processing, compression, storage, printing and display of such images. The most usual method is by digital photography with a digital camera but other methods are also employed.
1.7.2 IMAGE ENHANCEMENT

Image enhancement is a subjective process which improves the qualities of an image like improving the image contrast and brightness characteristics by reducing its noise content or by sharpening the details. This just enhances the image and reveals the same information in more understandable image. It does not add any information to it but enhances the image with greater clarity. Image enhancement is the improvement of digital image quality. Image enhancement is rather subjective because it depends strongly on the specific information the user is hoping to extract from the image.
1.7.3 IMAGE RESTORATION

Image restoration like enhancement improves the qualities of image but all the operations are mainly based on known, measured, or degradations of the original image. Dept of ECE, RRCE, Bangalore 10 2011-12
Adaptive bilateral filter for image enhancement and noise removal Image restorations are used to restore images with problems such as geometric distortion, improper focus, repetitive noise, and camera motion. It is used to correct images for known degradations. We may consider image formation as a process which transforms an input distribution into an output distribution.
1.7.4 MORPHOLOGICAL PROCESSING

The word morphology signifies the study of form or structure. In image processing we use mathematical morphology as a means to identify and extract meaningful image descriptors based on properties of form or shape within the image. Morphology encompasses a powerful and important body of methods which can be precisely treated mathematically within the framework of set theory. Morphological operations can be applied to images of all types, but the primary use for morphology is for processing binary images and the key morphological operators are the relatively simple ones called dilation and erosion.
1.7.5 IMAGE SEGMENTATION

Segmentation is the process of partitioning a digital image into multiple segments. The goal of segmentation is to simplify and/or change the representation of an image into something that is more meaningful and easier to analyze. Image segmentation is typically used to locate objects and boundaries such as lines, curves, etc in images. More precisely, image segmentation is the process of assigning a label to every pixel in an image such that pixels with the same label share certain visual characteristics. The result of image segmentation is a set of segments that collectively cover the entire image, or a set of contours extracted from the image. Each of the pixels in a region is similar with respect to some characteristic or computed property, such as colour, intensity, or texture. Adjacent regions are significantly different with respect to the same characteristics.
1.7.6 OBJECT RECOGNITION
11
2011-12
Adaptive bilateral filter for image enhancement and noise removal Object recognition in computer vision, this is the task of finding a given object in an image or video sequence. Humans recognize a multitude of objects in images with little effort, despite the fact that the image of the objects may vary somewhat in different view points, in many different sizes / scale or even when they are translated or rotated. Objects can even be recognized when they are partially obstructed from view. This task is still a challenge for computer vision systems in general.
1.7.7 IMAGE ANALYSIS

Image analysis operations produce numerical or graphical information based on characteristics of the original image. They break into objects and then classify them. They depend on the image statistics. Common operations are extraction and description of scene and image features, automated measurements, and object classification. Image analyze are mainly used in machine vision applications.
1.7.8 IMAGE COMPRESSION

Image compression and decompression reduce the data content necessary to describe the image. Most of the images contain lot of redundant information, compression removes all the redundancies. Because of the compression the size is reduced, so efficiently stored or transported. The compressed image is decompressed when displayed Image compression may be lossy or lossless. Lossless compression is preferred for archival purposes and often for medical imaging, technical drawings, clip art or comics. This is because lossy compression methods, especially when used at low bit rates, introduce compression artifacts. Lossless compression preserves the exact data in the original image. Lossy methods are especially suitable for natural images such as photographs in applications where minor loss of fidelity is acceptable to achieve a substantial reduction in bit rate. The lossy compression that produces imperceptible differences may be called visually lossless. Lossy compression does not represent the original image but provide excellent compression.
1.7.9 COLOUR IMAGE PROCESSING

Adaptive bilateral filter for image enhancement and noise removal Processing of the colour image is called colour image processing. Colour images can be represented by an intensity function C(x, y, ) which depends on the wavelength of the reflected light. (so, for fixed , C(x, y, ) represents a monochrome image).
Fig 1.9: RGB model and CMYK model. For science communication, the two main colour spaces are RGB and CMYK. The RGB colour model relates very closely to the way we perceive colour with the R, G and B receptors in our retinas. RGB uses additive colour mixing and is the basic colour model used in television or any other medium that projects colour with light. It is the basic colour model used in computers and for web graphics, but it cannot be used for print production. The secondary colours of RGB cyan, magenta, and yellow are formed by mixing two of the primary colours (red, green or blue) and excluding the third colour. Red and green combine to make yellow, green and blue to make cyan, and blue and red form magenta. The combination of red, green, and blue in full intensity makes white.
1.8 APPLICATIONS OF DIGITAL IMAGE PROCESSING

Digital image processing has a broad spectrum of applications, such as remote sensing via satellites and other spacecrafts, image transmission and storage for business applications, medical processing, radar, sonar and acoustic image processing, robotics and automated inspection of industrial parts.
1.8.1 BIOMEDICAL IMAGING TECHNIQUES
13
2011-12
Adaptive bilateral filter for image enhancement and noise removal X-rays, cineangiograms, projection images of transaxial tomography and other medical images that occur in radiology, nuclear magnetic resonance (NMR) and ultrasonic scanning. These images may be used for patient screening and monitoring or for detection of tumours or other disease in patients. For medical diagnosis, different types of imaging tools such as X- ray, Ultrasound, computer aided tomography (CT) etc are used. Some of the applications of Biomedical imaging applications are as follows:
HEART DISEASE IDENTIFICATION The important diagnostic features such as size of the heart and its shape are required to know in order to classify the heart diseases. To improve the diagnosis of heart diseases, image analysis techniques are employed to radiographic images.
LUNG DISEASE IDENTIFICATION In X- rays, the regions that appear dark contain air while region that appears lighter are solid tissues. Bones are more radio opaque than tissues. The ribs, the heart, thoracic spine, and the diaphragm that separates the chest cavity from the abdominal cavity are clearly seen on the X-ray film.
Fig 1.10 Diagrams of X- ray, MRI.
1.8.2 SATELLITE IMAGING

Images acquired by satellites are useful in tracking of earth resources; geographical mapping; prediction of agricultural crops, urban growth and weather; flood and fire control; and many other environmental applications. Space image applications include recognition and analysis of objects contained in image obtained from deep space-probe missions.
1.8.3 COMMUNICATION
Adaptive bilateral filter for image enhancement and noise removal Image transmission and storage applications occur in broadcast television, teleconferencing, and transmission of facsimile images for office automation, communication of computer networks, closed-circuit television based security monitoring systems and in military communications.
1.8.4 RADAR IMAGING SYSTEMS

Radar and sonar images are used for detection and recognition of various types of targets or in guidance and maneuvering of aircraft or missile systems.
1.8.5 DOCUMENT PROCESSING

It is used in scanning, and transmission for converting paper documents to a digital image form, compressing the image, and storing it on magnetic tape. It is also used in document reading for automatically detecting and recognizing printed characteristics.
1.8.6 DEFENSE/INTELLIGENCE
It is used in reconnaissance photo-interpretation for automatic interpretation of earth satellite imagery to look for sensitive targets or military threats and target acquisition and guidance for recognizing and tracking targets in real-time smart-bomb and missile-guidance systems.
CHAPTER-II
IMAGE ENHANCEMENT
2.1 INTRODUCTION
The technique used to improve visualization of the image contents. In this chapter we consider more general image enhancement. We introduce the concept of image filtering based on localized image sub regions, outline a range of noise removal filters and explain how filtering can achieve edge detection and edge sharpening effects for image enhancement.
2.2 IMAGE ENHANCEMENT

The basic goal of image enhancement is to process the image so that we can view and assess the visual information it contains with greater clarity. Image enhancement therefore is rather a subjective because it depends strongly on the specific information the user is hoping to extract from the image. The primary condition for image enhancement is that the information that you want to extract, emphasize or restore must exist in the image. Fundamentally, you cannot make something out of nothing and the desired information must not be totally swamped by noise within the image. Perhaps the most accurate and general statement we can make about the goal of image enhancement is simply that the processed image should be more suitable than the original one for the required task or purpose. This makes the evaluation of image enhancement, by its nature, rather subjective and hence it is difficult to quantify its performance apart from its specific domain of application.
2.3 PIXEL NEIGHBOURHOODS

An important measure in images is the concept of connectivity. Many operations in image processing use the concept of a local image neighborhood to define a local area of influence, relevance or interest. Central to this theme of defining the local neighborhood is the notion of pixel connectivity, i.e. deciding which pixels are connected to each other. When we speak of 4-connectivity, only pixels which are N, W, E, S of the given pixel are connected. However, if, in addition, the pixels on the diagonals must also be considered, then we have 8connectivity i.e. N, NW, W, NE, SE, E, SW, S are all connected as in Fig 2.1.
16
2011-12
Fig 2.1: Image neighborhood connectivity. In Fig 2.1 (right) we use this concept to determine whether which regions are connected and use a local connectivity model (here NXN=3X3) to determine if these are separate or the same image feature. Operations performed locally in images such as filtering and edge detection, all consider a given pixel location (i; j) in terms of its local pixel neighborhood indexed as an offset (i k; j k). The majority of image processing techniques now use 8-connectivity by default, which for a reasonable size of neighborhood is often achievable in real time on modern processors for the majority of operations. Filtering operations over a whole image are generally performed as a series of local neighborhood operations using a sliding-window based principle, i.e. each and every pixel in the image is processed based on an operation performed on its local NXN pixel neighborhood.
2.4 ENHANCEMENT VIA IMAGE FILTERING

The main goal of image enhancement is to process an image in some way so as to render it more visually acceptable or pleasing. The removal of noise, the sharpening of image edges and blurring effect so often favored in photographs are all examples of popular enhancement techniques. These and other enhancement operations can be achieved through the process of spatial domain filtering.
17
2011-12
Adaptive bilateral filter for image enhancement and noise removal The term spatial domain is arguably somewhat spurious, but is used to distinguish this procedure from frequency domain procedures. Thus, spatial domain filtering simply indicates that the filtering process takes place directly on the actual pixels of the image itself. Filters act on an image to change the values of the pixels in some specified way and are generally classified into two types: linear and nonlinear.
2.4.1 LINEAR FILTERS

A linear filter is a signal-processing device whose output is linear function of its input. Linear filters in the time domain process time-varying input signals to produce output signals, subject to the constraint of linearity. These results from systems composed solely of components classified as having a linear response. Most filters implemented in analog electronics, in digital signal processing, or in mechanical systems are classified as causal, time invariant, and linear. However the general concept of linear filtering is broader, also used in statistics, data analysis, and mechanical engineering among other fields and technologies. This includes non causal filters and filters in more than one dimension such as would be used in image processing; those filters are subject to different constraints leading to different design methods. A linear time-invariant (LTI) filter can be uniquely specified by its impulse response h, and the output of any filter is mathematically expressed as the convolution of the input with that impulse response. The frequency response, given by the filter's transfer function ,
is an alternative characterization of the filter. The frequency response may be tailored to, for instance, eliminate unwanted frequency components from an input signal, or to limit an amplifier to signals within a particular band of frequencies. There are a number of particularly desirable or useful filter transfer functions. Among the time-domain filters we here consider, there are two general classes of filter transfer functions that can approximate a desired frequency response. Very different mathematical treatments apply to the design of filters termed infinite impulse response (IIR) filters, characteristic of mechanical and analog electronics systems, and finite impulse response (FIR) filters, which can be implemented by discrete time systems such as computers
18
2011-12
2.4.2 NONLINEAR FILTER

A nonlinear filter is a signal-processing device whose output is not a linear function of its input. Terminology concerning the filtering problem may refer to the time domain showing of the signal or to the frequency domain representation of the signal. When referring to filters with adjectives such as "band pass, high pass, and low pass" one has in mind the frequency domain. When resorting to terms like "additive noise", one has in mind the time domain, since the noise that is to be added to the signal is added in the state space representation of the signal. The state space representation is more general and is used for the advanced formulation of the filtering problem as a mathematical problem in probability and statistics of stochastic processes.
2.5 FILTER KERNELS AND THE MECHANICS OF LINEAR FILTERING

In linear spatial filters the new or filtered value of the target pixel is determined as some linear combination of the pixel values in its neighborhood. The specific linear combination of the neighboring pixels that is taken is determined by the filter kernel often called a mask. This is just an array of exactly the same size as the neighborhood containing the weights that are to be assigned to each of the corresponding pixels in the neighborhood of the target pixel. Filtering proceeds by successively positioning the kernel so that the location of its centre pixel coincides with the location of each target pixel, each time the filtered value being calculated by the chosen weighted combination of the neighborhood pixels. This filtering procedure can thus be visualized as sliding the kernel over all locations of interest in the original image (i; j), multiplying the pixels underneath the kernel by the corresponding weights w, calculating the new values by summing the total and copying them to the same locations in a new filtered image Fig 2.2. For this reason, many filter kernels are sometimes described as convolution kernels, it then being implicitly understood that they are applied to the image in the linear fashion described above. Formally, we can express the action of convolution between a kernel and an image in two equivalent ways.
19
2011-12
Fig 2.2: The mechanics of image filtering with an 3X3 kernel filter. The first addresses the row and column indices of the image and the kernel:
Here, the indices i=0, j=0 correspond to the centre pixel of the kernel which is of the size (Imax-Imin+1, Jmax-Jmin+1). A second equivalent approach is to use linear indices:
In this case, Ik(i) represents the neighborhood pixels of the ith image pixel, where k is a linear index running over the neighborhood region according to a row-wise as in Fig 2.2 or columnwise convention. Here, wk are the corresponding kernel values and fi represents the filtered value resulting from original input value Ik(i). The former notation is more explicit, whereas the latter is neater and more compact; but neither is generally recommended over the other. Fig 2.2 illustrates this basic procedure, where the centre pixel of the kernel and the target pixel in the image are indicated by the dark gray shading. The kernel is placed on the image so that the centre and target pixels match. The filtered value of the target pixel f i is then given by a linear combination of the neighborhood pixel values, the specific weights being determined by the kernel values wk. In this specific case the target pixel value of original value 35 is filtered to an output value of 14.
20
2011-12
2.5.1 STEPS FOLLOWED IN LINEAR FILTERING

The steps in linear filtering can be summarized as follows: (1) Define the filter kernel. (2) Slide the kernel over the image so that its centre pixel coincides with each pixel in the image. (3) Multiply the pixels lying beneath the kernel by the corresponding values in the kernel above them and sum the total. (4) Copy the resulting value to the same locations in a new image. In certain applications, we may apply such a linear filter to a selected region rather than the entire image we then speak of region-based filtering. We may also take a slightly more sophisticated approach in which the filter itself can change depending on the distribution of pixel values in the neighborhood, a process termed adaptive filtering.
2.5.2 STEPS FOLLWED IN BOUNDARY REGIONS

Filtering at the boundaries of images also poses challenges. It is reasonable to ask what we should do when a target pixel lies close to the image boundary such that the convolution kernel overlaps the edge of the image. In general, there are three main approaches for dealing with this situation: (1) Simply leave unchanged those target pixels which are located within this boundary region. (2) Perform filtering on only those pixels which lie within the boundary. (3) Fill in the missing pixels within the filter operation by mirroring values over the boundary. Resulting undesirable edge artifacts are generally difficult to overcome and, in general, (2) or (3) is the preferred method. In certain instances, it is acceptable to crop the image meaning that we extract only a reduced-size image in which any edge pixels which have not been adequately filtered are removed entirely.
2.5.3 NONLINEAR SPATIAL FILTERING

Nonlinear spatial filters can easily be devised that operate through exactly the same basic mechanism as we described above for the linear case. The kernel mask slides over the image in the same way as the linear case, the only difference being that the filtered value will be the result of some nonlinear operation on the neighborhood pixels.
21
2011-12
For example, employing the same notation as before, we could define a quadratic filter as:
In this case, the action of the filter will be defined by the three weights which specify the contribution of the second, first- and zeroth-order terms. Nonlinear filters of this kind are not common in image processing. Much more important are order filters, which operate by ranking the pixels in the specified neighborhood and replacing the target pixel by the value corresponding to a chosen rank. In this case, we cannot write down a kernel and an equation of the form of our linear convolution is not applicable. In the following sections, we will present and discuss some of the more important examples of both linear and nonlinear spatial filters.
2.6 FILTERING FOR NOISE REMOVAL

One of the primary uses of both linear and nonlinear filtering in image enhancement is for noise removal. We will now investigate the application of a number of different filters for removing typical noise, such as additive, Salt and Pepper and Gaussian noise. However, we first need to consider generation of some example images with noise added so that we can compare the effectiveness of different approaches to noise removal. These images will form the basis for our comparison of noise removal filters.
Fig 2.3: (a) Original image with (b) Salt and Pepper noise and (c) Gaussian noise.
22
2011-12
2.6.1 MEAN FILTERING

The mean filter is perhaps the simplest linear filter and operates by giving equal weight WK to all pixels in the neighborhood. A weight of WK=1/(NM) is used for an NXM neighborhood and has the effect of smoothing the image, replacing every pixel in the output image with the mean value from its NXM neighborhood. This weighting scheme guarantees that the weights in the kernel sum to one over any given neighborhood size. Mean filters can be used as a method to suppress noise in an image. Another common use is as a preliminary processing step to smooth the image in order that some subsequent processing operation will be more effective.
Fig 2.4: Mean filter (3X3) applied to the (a) original, (b) Salt and Pepper noise and (c) Gaussian noise images. We can see that the mean filtering is reasonably effective at removing the Gaussian noise as in Fig 2.4(c), but at the expense of a loss of high-frequency image detail. Although a significant portion of the Gaussian noise has been removed compared with Fig 2.3(c), it is still visible within the image. Larger kernel sizes will further suppress the Gaussian noise but will result in further degradation of image quality. It is also apparent that mean filtering is not effective for the removal of Salt and Pepper noise as in Fig 2.4(b). In this case, the large deviation of the noise values from typical values in the neighborhood means that they perturb the average value significantly and noise is still very apparent in the filtered result. In the case of Salt and Pepper noise, the noisy high/low pixel values thus act as outliers in the distribution.
23
2011-12
Adaptive bilateral filter for image enhancement and noise removal The main drawbacks of mean filtering are:
It is not robust to large noise deviations in the image. When the mean filter straddles an edge in the image it will cause blurring. For this latter reason, the mean filter can also be used as a general low-pass filter. A common variation on the filter, which can be partially effective in preserving edge details, is to introduce a threshold and only replace the current pixel value with the mean of its neighborhood if the magnitude of the change in pixel value lies below this threshold.
2.6.2 MEDIAN FILTERING

Another commonly used filter is the median filter. Median filtering overcomes the main limitations of the mean filter. As each pixel is addressed, it is replaced by the statistical median of its NXM neighborhood rather than the mean. The median filter is superior to the mean filter in that it is better at preserving sharp high-frequency edges also eliminating noise, especially isolated noise spikes such as Salt and Pepper noise. The median m of a set of numbers is that number for which half of the numbers are less than m and half are greater; it is the midpoint of the sorted distribution of values. As the median is a pixel value drawn from the pixel neighborhood itself, it is more robust to outliers and does not create a new unrealistic pixel value. This helps in preventing edge blurring and loss of image detail. By definition, the median operator requires an ordering of the values in the pixel neighborhood at every pixel location. This increases the computational requirement of the median operator. Fig 2.5 shows the effect of median filtering on the two types of noise i.e. salt and pepper and Gaussian introduced to the images.
Fig 2.5: Median filter (3X3) applied to the (a) original, (b) Salt and Pepper noise and (c) Gaussian noise images.
24
2011-12
Adaptive bilateral filter for image enhancement and noise removal In Fig 2.5(c), we again see the removal of some Gaussian noise at the expense of a slight degradation in image quality. By contrast, the median filter is very good at removing salt and pepper type noise as in Fig 2.5(b), where we see the removal of this high/low impulse-type noise with minimal degradation or loss of detail in the image. This is a key advantage of median filtering.
2.6.3 RANK FILTERING

The median filter is really just a special case of a generalized order or rank filter. The general order filter is a nonlinear filter comprising the following common steps: (1) Define the neighborhood of the target pixel (NXN). (2) Rank them in ascending order (first is lowest value, (NXN)th is highest value). (3) Choose the order of the filter (from 1 to N). (4) Set the filtered value to be equal to the value of the chosen rank pixel. Order filters which select the maximum and minimum values in the defined neighborhood are called maximum and minimum filters. Notably, the Gaussian noise has been largely removed as in Fig 2.6c, but at the expense of image detail quality. The nature of the Salt and Pepper type noise causes its high values to be amplified by the use of a maximum filter.
Fig 2.6: Order filtering applied to the (a) original, (b) Salt and Pepper noise and (c) Gaussian noise images A variation on simple order filtering is conservative smoothing, in which a given pixel is compared with the maximum and minimum values (excluding itself) in the surrounding NXN neighborhood and is replaced only if it lies outside of that range. If the current pixel value is greater than the maximum of its neighbors, then it is replaced by the maximum. Similarly, if it is less than the minimum, then it is replaced by the minimum.
25
2011-12
2.6.4 GAUSSIAN FILTERING

The Gaussian filter is a very important for both for theoretical and practical reasons. Here, we filter the image using a discrete kernel derived from a radially symmetric form of the continuous 2-D Gaussian function defined as follows:
Discrete approximations to this continuous function are specified using two free parameters: (1) The desired size of the kernel (as an NXN filter mask); (2) The value of s, the standard deviation of the Gaussian functions. As there is always the case with linear convolution filters, there is a trade-off between the accurate sampling of the function and the computational time required to implement it. Some examples of discrete Gaussian filters, with varying kernel and standard deviation sizes are shown in Fig 2.7.
Fig 2.7: Gaussian filter kernels 3X3s=1, 11X11 s=2 and 21X21 s=4. Applying the Gaussian filter has the effect of smoothing the image, but it is used in a way that is somewhat different to the mean filter. First, the degree of smoothing is controlled by the choice of the standard deviation parameter s, not by the absolute value of the kernel size. Second, the Gaussian function has a rather special property, namely that its Fourier transform is also a Gaussian function, which makes it very convenient for the frequency-domain analysis of filters.
26
2011-12
Fig 2.8 Gaussian filtering (5X5 with
= 2) applied to the (a) original, (b) Salt and
Pepper noise and (c) Gaussian noise images of Fig 4.3 A Gaussian function with a large value of s is an example of a so-called low-pass filter in which the high spatial frequency content of an image is suppressed. To understand this properly requires a background in the Fourier transform and frequency domain analysis. In all cases, the smoothing effect of the filter degrades high frequency but it also removes to some degree the noise present in both Fig 2.8(b) and (c). Gaussian filtering commonly forms the first stage of an edge-detection algorithm where it is used as a means of noise suppression.
2.7 EDGE ENHANCEMENT

2.7.1 LAPLACIAN EDGE SHARPENING
Laplacian responds only to the fine detail in the image but has a zero response to constant regions and regions of smooth gradient in the image. If, therefore we take the original image and add or subtract the Laplacian, then we may expect to enhance the fine detail in the image artificially. It is common practice just to subtract it from the original, truncating any values which exhibit integer overflow in the common 8-bit representation. Using the Laplacian definition of we can define this as follows:
Laplacian image sharpening is shown in Fig 2.14, where we can see the original image, the Laplacian edges and the sharpened final output. Note that we can see the enhancement of edge contrast in this filtering result, but also an increase in image noise in the sharpened image.
27
2011-12
Fig 2.9: Edge sharpening using the Laplacian operator.
Fig 2.10: Edge sharpening using the LoG operator. In order to overcome this problem we replace the Laplacian operator with the LoG operator. Applying different-size Gaussian kernels to investigate the difference in image sharpening effects can be achieved. The example of Fig 2.14 is again shown in Fig 2.15 using the alternative LoG operator where we can note the reduced levels of noise in the intermediate LoG edge image and the final sharpened result.
2.7.2 THE UNSHARP MASK FILTER

An alternative edge enhancement filter to the Laplacian-based approaches is the unsharp mask filter also known as boost filtering. Unsharp filtering operates by subtracting a smoothed version of an image from the original in order to emphasize or enhance the highfrequency information in the image.
28
2011-12
Adaptive bilateral filter for image enhancement and noise removal First of all, this operator produces an edge image from the original image using the following methodology:
The smoothed version of the image is typically obtained by filtering the original with a mean or a Gaussian filter kernel. The resulting difference image is then added onto the original to affect some degree of sharpening:
Using a given constant scaling factor k that ensures the resulting image is within the proper range and the edges are not over sharp in the resulting image. Generally, k=0.20.7 is acceptable, depending on the level of sharpening required. It is in this secondary stage that gives rise to the alternative name of boost filtering. To understand this approach to sharpening we need to consider two facts. First, the smooth, relatively unchanging regions of the original image will not be changed significantly by the smoothing filter. By contrast, edges and other regions in the image in which the intensity changes rapidly will be affected significantly. If we subtract this smoothed image Ismoothed from the original image Ioriginal, then we get a resulting image Iedges with higher values in the areas affected significantly and low values in the areas where little change occurred. This broadly corresponds to a smoothed edge map of the image. The result of subtracting this smoothed image from the original is an image of higher value pixels in the areas of high contrast change and lower pixel values in the areas of uniformity Iedges. This resulting image can then be added back onto the original, using a specified scaling factor k, to enhance areas of rapid intensity change within the image whilst leaving areas of uniformity largely unchanged. It follows that the degree of enhancement will be determined by the amount of smoothing that is imposed before the subtraction takes place and the fraction of the resulting difference image which is added back to the original. Unsharp filtering is essentially a reformulation of techniques referred to as high boost we are essentially boosting the high-frequency edge information in the image. In Fig 2.16 we see the original image, the edge difference image and a range of enhancement results achieved with different scaling factors k.
29
2011-12
Adaptive bilateral filter for image enhancement and noise removal In the examples shown, improvement in edge detail is visible but the increasing addition of the edge image increases both the sharpness of strong edges and noise apparent within the image. In all cases, image pixels exceeding the 8-bit range 0255 were truncated.
Fig 2.11: Edge sharpening using unsharp mask filter.
30
2011-12
Chapter-III
IMAGE NOISE
3.1 DEFINITION OF NOISE
Image noise is the random variation of brightness or colour information in images produced by the sensor and circuitry of a scanner or digital camera. Image noise can also originate in film grain and in the unavoidable shot noise of an ideal photon detector. Image noise is generally regarded as an undesirable by-product of image capture. Although these unwanted fluctuations became known as "noise" by analogy with unwanted sound they are inaudible and such as dithering.
3.2 TYPES OF NOISES

There are 4 types of noises: Amplifier noise (Gaussian noise). Salt and Pepper noise. Shot noise (Poisson noise). Speckle noise.
3.2.1 AMPLIFIER NOISE (GAUSSIAN NOISE)

The standard model of amplifier noise is additive, Gaussian, independent at each pixel and independent of the signal intensity. In colour cameras where more amplification is used in the blue colour channel than in the green or red channel, there can be more noise in the blue channel . Amplifier noise is a major part of the "read noise" of an image sensor, that is of the constant noise level in dark areas of the image.
31
2011-12
Fig 3.1(a): Gaussian noise Image (Mean= 0, variance 0.05)
Fig 3.1(b): Gaussian noise (Mean= 1.5, variance 10)
3.2.2 SALT-AND-PEPPER NOISE

An image containing salt-and-pepper noise will have dark pixels in bright regions and bright pixels in dark regions. This type of noise can be caused by dead pixels, analog-to-digital converter errors, bit errors in transmission, etc. This can be eliminated in large part by using dark frame subtraction and by interpolating around dark/bright pixels.
Fig 3.2: Salt and Pepper noise.
3.2.3 POISSON NOISE

Poisson noise or shot noise is a type of electronic noise that occurs when the finite number of particles that carry energy, such as electrons in an electronic circuit or photons in an optical device, is small enough to give rise to detectable statistical fluctuations in a measurement.
32
2011-12
Fig 3.4: Poisson noise.
3.3.4 SPECKLE NOISE

Speckle noise is a granular noise that inherently exists in and degrades the quality of the active radar and synthetic aperture radar (SAR) images. Speckle noise in conventional radar results from random fluctuations in the return signal from an object that is no bigger than a single image-processing element. Speckle noise in SAR is generally more serious, causing difficulties for image interpretation. It is caused by coherent processing of backscattered signals from multiple distributed targets.
Fig 3.5: Speckle noise.
33
2011-12
Chapter IV
IMAGE RESTORATION
4.1 GENERAL
Image restoration refers to the genre of techniques that aim to recover a high-quality original image from a degraded version of that image given a specific model for the degradation process. This is in contrast to image enhancement techniques that seek to improve the appearance of an image without reference to a specific model for the degradation process. The restoration framework is particularly valuable because in conjunction with a trainingbased approach, it provides a context within which the free parameters of the restoration algorithm may be optimized. Training-based approaches have been used to develop imaging algorithms for a variety of applications, including image interpolation, image restoration, digital half toning, de-screening, and colour correction. The ingredients that training-based approaches have in common when used for development of imaging algorithms are: 1) A set of training pairs each consisting of an input image and a desired output image. 2) An architecture for the algorithm consisting of free parameters. 3) A cost function under which those free parameters may be optimized.
4.2
PROPOSED PROJECT
In this project, we propose a new training-based approach to image restoration. Once
the restoration algorithm has been fully developed we are however free to apply it to images for which the degradation process is unknown. This puts us back in the domain of enhancement. The success of this broader application of the restoration algorithm will depend on how general is the degradation model under which the algorithm was developed, as well as how robust is the overall structure of the algorithm to deviations from the assumed degradation model. The scope of this project is to deal with images that are appropriate for digital photography.
34
2011-12
4.3
FORMS OF DEGRADATION
The two most common forms of degrading an image suffer a loss of sharpness or blur,
and noise. The degradation model we use consists of a linear, shift-invariant blur followed by additive noise. The problem we are interested in is twofold. First we seek to develop a sharpening method that is fundamentally different from the unsharp mask filter (USM), which sharpens an image by enhancing the high-frequency components of the image. In the spatial domain, the boosted high-frequency components lead to overshoot and undershoot around edges, which causes objectionable ringing or halo artifacts. Our goal is to develop a sharpening algorithm that increases the slope of edges without producing overshoot and undershoot, which renders clean, crisp, and artifact-free edges, thereby improving the overall appearance of the image. The second aspect of the problem we wish to address is noise removal. We want to present a unified solution to both sharpness enhancement and noise removal. In most applications, the degraded image contains both noise and blur. A sharpening algorithm that works well only for noise-free images will not be applicable in these situations. In terms of noise removal, conventional linear filters work well for removing additive Gaussian noise, but they also significantly blur the edge structures of an image. Therefore, a great deal of research has been done on edge-preserving noise reduction. One of the major endeavors in this area has been to utilize rank order information. Due to a lack of the sense of spatial ordering, rank order filters generally do not retain the frequency selective properties of the linear filters and do not suppress Gaussian noise optimally. Hybrid schemes combining both rank order filtering and linear filtering have been proposed in order to take advantage of both approaches. These nonlinear rank order approaches in general improve the edge sharpness, but they are more complex to implement than a spatial linear filter.
4.4 BILATERAL FILTER

The bilateral filter is the framework for our proposed algorithm. Here, we would like to point out that the bilateral filter is essentially a smoothing filter; it does not restore the sharpness of a degraded image. Aleksic et al. modified the bilateral filter to perform both noise removal and sharpening by adding a high-pass filter to the conventional bilateral filter.
35
2011-12
Adaptive bilateral filter for image enhancement and noise removal This filter essentially performs USM sharpening for pixels that are above a preselected high pass threshold. Therefore, it produces halo artifacts as does an USM filter. In contrast to the extensive effort to improve de-noising algorithms, much less has been done for sharpening algorithms. The USM remains the prevalent sharpening tool despite the drawbacks that it has. First, the USM sharpens an image by adding overshoot and undershoot to the edges which produce halo artifacts. Second, when applied to a noisy image, the USM will amplify the noise in smooth regions which significantly impairs the image quality.
4.5 SHARPENING AN IMAGE

4.5.1 SLOPE RESTORATION ALGORITHM
Das and Rangayyan developed an edge sharpness enhancement algorithm to improve the slope of edges. In their algorithm, the edge normal direction is first detected and then a 1D operator is applied to the edge pixels so that the transitions of edges are made steeper. They pointed out that corner artifacts and inferior enhancement for circular regions were limitations of their algorithm. The testing of their algorithm in was limited to bi-level, synthesized images. Tegenbosch et al. proposed a luminance transient improvement (LTI) algorithm which first sharpens the image with a linear sharpening method such as the USM, then detects the 1-D edge profiles, and finally clips between the start and end level of the edges to get rid of the overshoot and undershoot. The 1-D LTI algorithm is implemented in a 2-D image by three alternative methods. 1) Apply LTI in the direction of the edge normal, which involves estimating the local edge orientation. 2) Apply LTI in the horizontal direction first, then in the vertical direction. 3) Apply LTI in parallel, in the horizontal and vertical directions, and then combine the results by a weighted sum depending on the edge orientation. According to the authors, 1st method produces the best image quality, but is computationally expensive. 2nd method is efficient to implement, but has the drawbacks of inhomogeneous enhancement and staircase artifacts. 3rd achieves the best balance between image quality and computational complexity. Results comparing images enhanced by the three proposed methods are given.
36
2011-12
Adaptive bilateral filter for image enhancement and noise removal However, no comparison of the input and enhanced output images is provided, nor are edge profiles shown to demonstrate the effectiveness of the proposed algorithms in restoring edge slopes. We will show that our method is able to improve image sharpness for both natural images and text images, and the edge slopes are significantly increased.
4.5.2 ADAPTIVE SHARPENING AND SMOOTHENING

To address the second problem of the USM filter, locally adaptive sharpening and smoothing algorithms have been proposed. Kotera et al. developed an adaptive sharpening algorithm based on the USM and the classification of the pixels. The histogram of the edge strength is used to classify pixels into smooth regions, soft edges, and hard edges, which are subsequently processed with different sharpening strengths. Kim and Allebach developed an adaptive sharpening and de-noising algorithm- the optimal unsharp mask (OUM). Instead of using a fixed gain for the high-pass filter as in the case of the conventional USM filter, the OUM employs a locally adaptive, which has been trained for different regions of the image with pairs of high quality original and corresponding degraded images. By allowing the gain to be negative, the OUM is capable of both sharpening and de-noising. The degraded images were generated by applying a blur point spread function (PSF) to the original image, and adding noise to the blurred image. The blur PSF was modeled after a hybrid analog and digital imaging system, which involves scanning a silver-halide photograph printed from a negative exposed with a low-cost analog camera. A tone-dependent noise model was used to simulate noise produced in the imaging pipeline. The parameters of our proposed restoration algorithm are optimized in a training-based framework similar to that of the OUM.
37
2011-12
Chapter V
INTRODUCTION TO ADAPTIVE FILTERING

5.1 DEFINITION AND CHARACTERISTICS
Adapting is nothing but to adjust or modify fittingly to adjust oneself to different conditions, environments. In recent years, a growing field of research in adaptive systems has resulted in variety of adaptive automations whose characteristics in limited ways resemble certain characteristics of living systems and adaptive biological process. According to the Random House Dictionary, some of the meanings of adaptation are: 1. The act of adapting. 2. The state of being adapted or adjustment. An adaptive automation is a system whose structure is alterable or adjustable in such way that its behavior or performance improves through contacts with environment. A simple example of an automatic adaptive system is the automatic gain controller (AGC) used in radio and television receivers. The function of this circuit to adjust the sensitivity of the receiver inversely as the average incoming signal strength. The receiver is thus able to adapt to a wide range of input levels and to produce a much narrower range of output intensities. Basic principles of adaptation to explain the design, operating characteristics and application of the simpler forms of adaptive systems; and to describe means for the physical realization. The types of systems discussed include those designed primarily for the purpose of adaptive control and adaptive signal processing. Such systems usually have some or all of the fallowing characteristics: 1. They can automatically adapt in the face of changing environments and changing system requirements. 2. They can be trained to perform specific filtering and decision making tasks. Synthesis of systems having these capabilities can be accomplished automatically through training. In a sense, adaptive system can be programmed by a training process. 3. Because of the above, adaptive systems do not require the elaborate synthesis procedures usually needed for non adaptive systems instead; they tend to be self-designing.
38
2011-12
Adaptive bilateral filter for image enhancement and noise removal 4. They can be extrapolating a model of behavioral to deal with new situation after having been trained on a finite and often small number of training signals or patterns. 5. To a limited extent, they can repair themselves; that is, they can adapt around certain kinds of internal defects. 6. They can usually be described as nonlinear systems with time-varying parameters. 7. Usually, they are more complex and difficult to analyze than non adaptive systems ,but they offer the possibility of substantially increased system performance when input signal characteristics are unknown or time varying.
5.2 AREAS OF APPLICATION

Recent progress in microcircuit design and production has resulted in very compact, economical and reliable signal processor that rival biological nervous systems in size and are clearly superior to biological systems in speed. The result has been a very-growing field of application of all types of digital signal processing, including adaptive processing. Current adaptation for adaptive systems is in such fields as communication, radar, sonar, seismology, mechanical design, navigation systems, and biomedical electronics.
5.3 GENERAL PROPERTIES

The essential and principal property of the adaptive system is its time-varying, selfadjusting performance. The need for such performance may readily be seen by realizing that if a designer develops a system of fixed design which he or she consider optimal, the implications are that the designer has foreseen all possible input conditions, at least statically, and knows that she or he would like the system to do under each of these conditions. The designer has then chosen specific criterion where by performance is to be judged, such as the amount of error between the output of the actual system and that of some selected model or ideal; system. Finally, the designer has chosen the system that appears best according to the performance criterion selected, generally choosing this system from an a priori restricted class of designs (such as linear systems).
39
2011-12
Adaptive bilateral filter for image enhancement and noise removal In many instances, however, the complete range of input conditions may not be known exactly, or even statically; or the conditions may change from time to time. In such circumstances, an adaptive system that continually seeks the optimum within an allowed class of possibilities, using an orderly search process, would give superior performance compared with a system of fixed design. By their nature, adaptive systems must be time varying and nonlinear. Their characteristics depend, among other things, on their input signals. If an input signal x1 is applied, an adaptive system will adapt to it and produce an output let us call it as y1. If another input signal, x2 is applied the system adapt to this second signal, x2,is applied ,system will adapt to it this second signal and will again produce an output let us call it this time as y2. Generally, the form or the structure or the adjustments of the adaptive system will be different for the two different inputs. If the sum of the two inputs is applied to the adaptive system, latter will adapt to this new input but it will produce an output that will generally not be the same as y1+y2, the sum of the outputs that would have corresponded to inputs x1 and x2. In such a case, as illustrated in Fig 5.1 the principles of superposition do not work as it does with linear systems. If a signal is applied to the input of an adaptive system to test it response characteristics, the system adapts to this specific input and there by changes its own form. Thus the adaptive system is inherently difficult to characterize in conventional terms.
Fig 5.1: Adaptive system where y3=y1+y2 if H is linear system.
40
2011-12
Adaptive bilateral filter for image enhancement and noise removal Within the realm of nonlinear systems, adaptive systems cannot be distinguished as belonging to an absolutely clear subset. However they have two features that generally distinguish them form other forms of nonlinear systems. first adaptive systems are adjustable, and their adjustment usually depend on finite- time average signal characteristics rather that on instantaneous values of signals or instantaneous values of signals or instantaneous values of internal system states. Second, the adjustments of adaptive systems are changed purposefully in order to optimize specified performance measures. Certain forms of adaptive systems become linear systems when their adjustments are held constant after adaptation. These may be called linear adaptive systems .They are very useful; they tend to be mathematically tractable; and they are-generally easier to design than other forms of adaptive systems.
5.4 OPEN AND CLOSED LOOP ADAPTATION

Several ways to classify adaptive schemes have been proposed in the literature. it is most convenient here to begin by thinking in terms of open loop and closed loop adaptation .the open loop adaptive process involves making measurement of input or environmental characteristics, applying this information to formula or to a computational algorithm, and using the results to set the adjustment of the adaptive system. Closed loop adaptation, on the other hand, involves automatic experimentation with these adjustments and knowledge of their outcome in order to optimize a measured systems performance. The latter process may be called adaptation by performance feedback. The principles of open-hand and closed loop adaptation are illustrated in Figures 5.2 and 5.3 in both cases it is helpful to envisage the adaptive process as it might be performed manually by human operator or supervisor . In Figures 5.2 (a) and 5.3(a) the supervisor is shown adjusting the controls of the processor by reading a display that registers measurements of the preselected performance criterion. In the open loop system this criterion is a characteristic of an input signal and perhaps other data, and in the closed loop systems the operator is replaced by computational or adaptive algorithms, as suggested by Figures 5.2(b) and 5.3(b). The other data in these Figures may be data about the environment of the adaptive system, or in the closed-loop case, it may be a desired version of the output signal.
41
2011-12
(a)
Fig 5.2: Open loop adaptation: (a) concept; (b) equivalent system. When designing an adaptive process, many factors determine the choice of closed-loop adaptation. The availability of input signals and performance indicating signal is a major consideration. Also, the amount of computing capacity and the type of computer required to implement the open loop and the use of general purpose digital computer, whereas other algorithm could be implemented more economically with special purpose chips or other apparatus .some of these structural consideration are discussed on later chapters. It is difficult to develop general principles to guide all choices, but several advantages and a few disadvantages of closed loop adaptation, which is the main subject of this book, can be pointed out here . Closed loop adaptation has the advantage of being workable in many applications where no analytical synthesis procedure either exists or is known. Closed loop adaptation can also be used effectively in situation where physical system component values are variable or inaccurately known. Closed loop adaptation will find the best choice of component values .in the event of partial system failure, an adaptation mechanism that continually monitors performance will optimize this performance by adjusting and re-optimizing the intact parts .as a result, system reliability can often be improved by the use of performance feedback.
42
2011-12
(a)
(b) Fig 5.3 Closed loop adaptation. The closed loop adaptation process is not always free of difficulties; however in certain situations, performance function does not have unique optima. Automatic optimization is an uncertain process in such situations. In other situations, the closed loop adaptation process, like a closed loop control system, could be unstable .the adaptation process could diverge rather than coverage .in spite of these possibilities, performance feedback is a powerful, widely applicable technique for implementing adaptation. Most of the adaptive process described in this book will be closed\loop process utilizing performance feedback.
43
2011-12
5.5 APPLICATIONS
Let us now consider briefly some applications of the closed-loop, performance feedback concept. We begin by representing the performance feedback process more especially in Fig 5.4. We call the input signal x and define a desired response signal d, which is assumed to represent the desired output of the adaptive system. The signal is, for our purpose here, the other data in Fig 5.3(b) The error signal, E, is the difference between the desired output signal and the actual output signal, y, of the adaptive system. Using the error signal, an adaptive algorithm adjusts the structure of the adaptive system, thus altering its response characteristics by minimizing some measures of the error, thereby closing the performance loop.
Fig 5.4 Signals in closed-loop adaptation.
44
2011-12
Fig 5.5 Examples showing how the configuration in fig 5.4 may be applied : (a) Prediction; (b) System identification (modelling); (c) Equalization (deconvolution, inverse filtering, inverse modelling); (d) Interference cancelling.
45
2011-12
Adaptive bilateral filter for image enhancement and noise removal Some examples of applications of applications are given in Fig 5.5 which illustrates the basic closed-loop adaptive process, is simplified slightly and embedded in each part of this figure, and that the application determines show the desired signal, d, is obtained. The prediction application in Fig 5.5(a) is perhaps the simplest of the four. The desired signal is the input signal, s, and a delayed version of the latter is sent to the adaptive processor, which must therefore try to predict the current input signal in order to have y cancel d and drive e toward zero. Prediction is used in signal encoding and noise reduction. The system identification application in Fig 5.5(b) is also easy to understand. Here a broadband signal, s, is the input to the adaptive proce4ssor as well as to an unknown plant. To reduce e, the adaptive processor tries to emulate the plants transfer characteristic. After adaptation the plant is identified in the sense that its transfer function can be specified as essentially the same as that of the adaptive processor. Adaptive system identification or modelling can be used as such, to model a slowly varying plant whose input and output signals are available. The inverse modelling application is shown in Fig 5.5(c). In this application the adaptive processor attempts to recover a delayed version of the signal, s, which is assumed to have been altered by the slowly varying plant and to contain additive noise. The delay in the Figure is to allow for the delay, or propagations time, through the plant and the adaptive processor. Adaptive equalization could be used to undo the effects of a transducer, a communication channel, or some other system, or to produce an inverse model of an unknown plant. It is also applicable in the design of digital filters, as well as in adaptive control problems, and so on. Finally, Fig 5.5(d) shows the adaptive processor in an interference-cancelling configuration. Here the signal, s, is corrupted by additive noise, n, and a distorted but correlated version of the noise, n, is also available. The goal of the adaptive processor in this case is to produce an output y, that closely resembles n, so that the overall output, e, will closely resemble s. The optimal adaptive processor is that which minimizes the mean square value of e.
5.5 EXAMPLE OF AN ADAPTIVE SYSTEM

Adaptive signal processing is really a very general and basic term which implies the use of time-varying, self-adjusting signal processors. Performance of these systems as being purposeful, useful and sometimes even intelligent in a limited sense.
46
2011-12
Adaptive bilateral filter for image enhancement and noise removal The adaptive predictor diagram in Fig 5.5(a) is redrawn in Fig 5.6 with examples of several conventions of notation. The symbols xk, yk, and ek represent the kth elements in the time series represented by x, y and e. Usually, the time series may be assumed to be obtained by sampling continuous signals. The symbol that in Fig 5.6 the output of the stands for a fixed delay of M time steps, so
block is labelled Xk-m. The symbol Hk represents the
transfer characteristic of the adaptive processor or adaptive filter. The subscript k indicates that the transfer characteristic possibly changes at each sample point. Thus in Fig 5.6 we see that the input signal, xk, is delayed and then filtered to produce yk, and that yk is subtracted from xk to obtain the error, ek. The transfer function, Hk, is adjusted to keep average squared value of this error as small as possible. In this way, the processor always uses past values of x to predict the present value of x while using e to adjust H, and is thus involved in a performance-feedback process.
Fig 5.6 Example of an adaptive system: an adaptive predictor.
47
2011-12
Chapter-VI
ADAPTIVE BILATERAL FILTERS

6.1 BILATERAL FILTER
The bilateral filter proposed by Tomasi and Manduchi in 1998 is a nonlinear filter that smoothes the noise while preserving edge structures. The shift-variant filtering operation of the bilateral filter is given by
where f[m, n] is the restored image, h[m, n; k, l] is the response at [m, n] to an impulse at [k, l], and g[m, n] is the degraded image. The response of a bilateral filter is given by
where [m0, n0] is the center pixel of the window, m0,n0 ={[m, n] : [m, n] [m0 N,m0 + N][n0 N, n0 + N]}. r, d are the standard deviations of the domain and range Gaussian filters, respectively, and
is a normalization factor that assures that the filter preserves average gray value in constant areas of the image. The edge-preserving de-noising bilateral filter adopts a low pass Gaussian filter for both the domain filter and the range filter. The domain low-pass Gaussian filter gives higher weight to pixels that are spatially close to the center pixel. The range low pass Gaussian filter gives higher weight to pixels that are similar to the center pixel in gray value. Combining the range filter and the domain filter, a bilateral filter at an edge pixel becomes an elongated Gaussian filter that is oriented along the edge.
48
2011-12
Adaptive bilateral filter for image enhancement and noise removal This ensures that averaging is done mostly along the edge and is greatly reduced in the gradient direction. This is the reason why the bilateral filter can smooth the noise while preserving edge structures.
6.2 ILLUSTRATION OF BILATERAL FILTER
Fig 6.1: Bilateral and Gaussian filter output images. a) Blurred and noisy 165X156 image b) Bilateral filter restored image (r = 20; d = 2) c) Gaussian filter restored image (d = 2). Fig 6.1(b) shows that a bilateral filters with and removes much of the noise that appears in the degraded image shown in Fig 6.1(a) and preserves the edge structures. In Fig 6.1(c), where the spatial domain Gaussian with is applied alone, the edges are significantly blurred.
Fig 6.2: Impulse response and transfer function at region A.
49
2011-12
Adaptive bilateral filter for image enhancement and noise removal In Fig 6.2(a) represents impulse response of domain Gaussian filter, 6.2(b) represents the impulse response of range Gaussian filter, 6.2(c) represents the impulse response of Bilateral filter and 6.2(d) represents the transfer function of the bilateral filter. The range filter with at the edge pixel A is shown in Fig 6.2(b). Combining the spatial domain Gaussian filter Fig 6.2(a) and the range Gaussian filter Fig 6.2(b) results in the bilateral filter at pixel A shown in Fig. 6.3(c). The transfer function of the bilateral filter shown in Fig 6.2(d) demonstrates that the bilateral filter at pixel A is low pass in one direction, and almost all-pass in the orthogonal direction. This explains, from a frequency domain perspective, why this filter is able to preserve edges while removing noise.
Fig 6.3: The zoomed image of Fig 6.1 at region A. On the other hand, the bilateral filter is essentially a smoothing filter. It does not sharpen edges. As shown in Fig 6.3(a) and (b), the edge rendered by the bilateral filter has the same level of blurriness as in the original degraded image, although the noise is greatly reduced. The results of the bilateral filtering are a significant improvement over a conventional linear low-pass filter. However, in order to enhance the sharpness of an image, we need to make some modifications to this filter. The modification in this bilateral filter is described in next topic.
50
2011-12
6.3 ADAPTIVE BILATERAL FILTERS

In this section, we present a new sharpening and smoothing algorithm: the adaptive bilateral filter (ABF) algorithm its impulse response is given by
The response at
of the proposed shift-variant ABF to an impulse at [m, n] is given and m o ,n o are defined as before,
by (4), shown at the bottom of the page, where and the normalization factor is given by
Thus the bilateral filter values are calculated for the purpose of de-noising and Image sharpening. The ABF retains the general form of a bilateral filter, but contains two important modifications.

An offset is introduced to the range filter in the ABF. Second, both and the width of the range filter r in the ABF are locally adaptive. If = 0 and r is fixed, the ABF will degenerate into a conventional bilateral filter. For the domain filter, a fixed low-pass Gaussian filter with d = 1.0 is adopted in the ABF. The combination of a locally adaptive and r transforms the bilateral filter into a much more powerful filter that is capable of both smoothing and sharpening. Moreover, it sharpens an image by increasing the slope of the edges.
51
2011-12
6.3.1 ROLE OF IN THE ABF
Table 6.1: Locations of the resultant range filters with regard to histogram of the data in
m o ,n o .
52
2011-12
Adaptive bilateral filter for image enhancement and noise removal The range filter can be interpreted as a 1-D filter that processes the histogram of the image. We will illustrate this viewpoint for the window of data enclosed in the red box in the boy portrait images in Table 6.1. We index the images in the table by their [row, column] coordinates. The original degraded image with the red data box is shown in [1, 2], for which the histogram is shown in [1, 3]. For the conventional bilateral filter, the range filter is located on the histogram at the gray value of the current pixel and rolls off as the pixel values fall farther away from the center pixel value as shown in [2, 1]. By adding an offset to the range filter, we are now able to shift the range filter on the histogram, as shown in [3, 1], [4, 1], and [5, 1]. As before, let m o ,n o denote the set of pixels in the 2N+1 X 2N+1 window of pixels centered at . Let MIN, MAX, and MEAN denote the operations of taking the minimum, maximum, and average value of the data in m o ,n o respectively. Let and lets demonstrate the effect of bilateral filtering with a fixed domain Gaussian filter ( d =1.0) and a range filter ( r =20) shifted by the following choices for . No offset (conventional bilateral filter): Shifting towards the MEAN: Shifting away from the MEAN: Shifting away from the MEAN, to the MIN/MAX
The locations of the resultant range filters with regard to the histogram of the data in
m o ,n o are illustrated in Table 3, rows two to five. Here, N=12. As we can see from Table 3,
shifting the range filter towards MEAN ( m o ,n o ) will blur the image [3, 2]. Shifting the range filter away from MEAN ( m o ,n o ) will sharpen the image [4, 2]). In the extreme case, if for every pixel above MEAN ( m o ,n o ), we shift the range filter to MAX ( m o ,n o ), and for every pixel below MEAN ( m o ,n o ), we shift the range filter to MIN ( m o ,n o ), we will see a drastic sharpening effect and the image will appear over-sharpened ([5, 2]).
53
2011-12
Adaptive bilateral filter for image enhancement and noise removal The reason behind these observations is the transformation of the histogram of the input image by the range filter. In our case, the data window m o ,n o marked by the red box in [1, 2] contains an edge. Therefore, the histogram of the data in m o ,n o has two peaks, which correspond to the darker and brighter sides of the edge, respectively ([1, 3]). Any pixels located between the two peaks appear on the slope of the edge. A 3-D plot of the image data in m o ,n o is shown in [1, 4]. The conventional bilateral filter (no shift to the range filter) does not significantly alter the histogram of the data ([2, 3]) and, consequently, does not change the slope of the edge ([2, 4]). Shifting the range filter to MEAN ( m o ,n o ) at each pixel will redistribute the pixels towards the centre of the histogram ([3, 3]). Hence, the slope is reduced ([3, 4]). On the other hand, if we shift the range filter further away from MEAN ( m o ,n o ), pixels will be compressed against the two peaks ([4, 3]). The slope will then be increased ([4, 4]). In the case of operation No. 4, the histogram is further compressed around the two peaks, as shown in [5, 3]. The edge almost becomes a step function, as shown in [5, 4]. In this case, one can also observe an outlier pixel in [5, 4]. This is because the range filter can be very sensitive to noise if not applied correctly. In particular, shifting the range filter based on
m o ,n o is not robust to noise. If the range filter is shifted in the wrong direction, large errors
will result. The domain filter also makes the bilateral filter more robust to noise.
6.3.2 ROLE OF r IN THE ABF

The parameter r of the range filter controls the width of the range filter. It determines how selective the range filter is in choosing the pixels that are similar enough in gray value to be included in the averaging operation. If r is large compared to the range of the data in the window, the range filter will assign similar weight to every pixel in the range. Then, it will not have much effect on the overall bilateral filter.
54
2011-12
Fig 6.4: Impact of r . A fixed d = 1 is used in the bilateral filter. DF: domain filter only; RF: range filter only; BF: bilateral filter. (a) DF: d = 1; (b) RF: r = 5; (c) BF: d = 1 r = 5;(h) DF: d = 1;(i)RF:
r = 50;(j) BF: d = 1, r = 50. On the other hand, a small r will make the range filter
dominate the bilateral filter. Fig. 2 demonstrates this effect. The bilateral filtered image resembles the range filtered image when r =5, and it resembles the domain filtered image when r =50.
6.3.3 SUMMARY OF THE RATIONALE FOR ABF

The pixel dependent offset in the ABF is the key to slope restoration. With , we are able to restore the slope by transforming the local histogram of the image, thus circumventing the cumbersome process of locating edge normals and detecting edge profiles. Since at any pixel in the image, the ABF output is bounded between MIN ( m o ,n o ) and MAX ( m o ,n o ), the ABF, in general, does not produce overshoot and undershoot. By making and r adaptive and jointly optimizing both parameters, we transform the bilateral filter into a much more powerful and versatile filter. To smooth the image at a given pixel, we can shift the range filter towards MEAN ( m o ,n o ), and/or use a large r which enables the spatial Gaussian filter to take charge of the bilateral filtering.
55
2011-12
Adaptive bilateral filter for image enhancement and noise removal To sharpen the image at a given pixel, we can shift the range filter away from the midpoint of the edge slope which will be approximately equal to MEAN ( m o ,n o ), towards MAX ( m o ,n o ) or MIN ( m o ,n o ), depending on the position of the edge pixel on the edge slope. At the same time, we would reduce r accordingly. With a small r , the range filter dominates the bilateral filter and effectively pulls up or pushes down the pixels on the edge slope.
6.4 OPTIMISATION OF ABF PARAMETERS

6.4.1 GENERAL
The parameter optimization is formulated as a minimum mean squared error (MMSE) estimation problem. We classify the pixels into T classes and during the training process estimate the optimal and r for each class that minimizes the overall MSE between the original and restored images. Let P be the total number of training image sets. The kth set (k = 1,2.P) consists of an original image fk[m,n], a degraded image gk[m,n], the class index image Lk[m,n], and the restored image f [m,n]. All four of these images have dimensions Mk*Nk. Let be the set of indices for the pixels in these images given by
^
Also let by
be the set of indices for the pixels belonging to the class i in image k and is given
Given the P training image sets as described above, the optimal parameters satisfy
and
56
2011-12
Where
denotes the L-2 norm of the array A over the index set B, , and .Since
the classes are independent and non overlapping, we can separately estimate the optimal and for each class
The restored image ^ [m, n] is a nonlinear function of the parameters i and r,i . f There is no closed-form solution for and . To find the pair of parameters which
minimizes the MSE for each class, we perform an exhaustive search in the parameter space where The parameter space is uniformly quantized with step sizes The range and the step size of the parameters are
chosen empirically such that they can yield adequate sharpening and smoothing for all types of image structures with a balance between accuracy and computational cost.
6.4.2 FEATURE DESIGN

The feature for pixel classification plays an important role in the success of the training. Despite the high correlation between and the desired sharpening/smoothing effect,
is not an appropriate feature for pixel classification because it is very sensitive to noise.
Our general guidelines for choosing the feature(s) are: 1) be able to reflect the strength of edges, 2) can distinguish the regions we want to process differently, mainly, the regions for smoothing and sharpening, and 3) have some robustness to noise.
The feature we have chosen to use for pixel classification is the strength of the edges measured by a Laplacian of Gaussian (LoG) operator with a 9*9 kernel and Dept of ECE, RRCE, Bangalore 57 , 2011-12
where N=4 and C is given in Eq(6.12)
The impulse response and frequency response of the LoG operator are shown in Fig. 6.5(a) and (b), respectively.
Fig 6.5: LoG response. (a) Impulse response; (b) Frequency response. The LoG operator is a highpass filter. It computes the second derivative of the input image. Therefore, near edges, the magnitude of its response is high; in smooth regions, the magnitude of its response is low; and on the center of an edge, the magnitude of its response is 0.
Fig 6.6: Ramp edge and its LoG response. Dept of ECE, RRCE, Bangalore 58 2011-12
The LoG strength is also used in the OUM for pixel classification. However, the OUM does not use the sign of the LoG output for classification. We choose the LoG output as our pixel classification feature for three major reasons. First, the magnitude of the LoG strength reflects the local edge structure. It can distinguish smooth regions from edge regions, where the optimal filter parameters are most likely to be very different. One special case is the edge centers. The magnitude of the LoG response cannot distinguish the center of the edges and the noisy smooth regions very well because both have small LoG response. However, the ABF response at pixel [m_o, n_o] depends on both the spatial domain and range space distribution of the data in the window m o ,n o .
7X7 data window at pixel A
Impulse response of ABF at pixel A
7X7 data window at pixel B
Impulse response of ABF at pixel B
Fig 6.7: ABF for various input image structures. Dept of ECE, RRCE, Bangalore 59 2011-12
Although, these two region types are not completely separated by the LoG classifier, the very different data structures at the edge centers and in the smooth regions ensure that the ABF with the same and r can satisfy the filtering needs of both types of regions. To illustrate this idea, we consider the three sample pixels shown in Fig. 6.7(a). The pixels A and B are located in a noisy smooth region and on an edge center, respectively. Even though the optimal filter parameters are the same for both pixels, the impulse responses of the ABF are different for pixels A and B. Pixel C shows how * and r *impact the ABF. Without the Offset and the locally adaptive r , the ABF at pixel C would be the same as that in pixel B. In that case no sharpening effect would be achieved at pixel C.
6.4.3 TRAINING IMAGES

We used five high quality, high resolution images for our training set. They are from the six training images used for the OUM. One image was left out of the training set to serve as a test image. The content of the training images cover a variety of scenes. To generate the degraded images, we applied a blur PSF to the original image, and added tone-dependent noise to the blurred image. As stated in the introduction, the scope of this paper is to deal with images that are appropriate for digital photography. These are the images with a reasonable level of quality, from ones digital camera or scanners that one would actually want to print. There is a large market for enhancing the digital photographs from consumers, such as the self serve photo kiosks in the supermarket, and the on-line printing services. These services all apply image enhancing algorithms to the digital photos before printing them. This is the main application we targeted when designing the ABF. Therefore, we adopted an image degradation model that simulates the blur and noise introduced by a real hybrid imaging pipeline. The blur PSF and the tone-dependent noise are modeled after an imaging pipeline which involves capturing the original image on silver halide film with a low cost point-andshoot camera, using a common photo processor to develop the film and print a 4 * 6 in silver halide print, and finally scanning the print to obtain a digital image. The size of the blur PSF of this imaging pipeline is approximately 4 pixels. Here, the size of the PSF is defined as twice the distance in pixels from the origin to the point at which the PSF has dropped to 50% of its value at the origin.
60
2011-12
(a) Test image Kids (3439X2344).
(b) Edge normal profiles [52] taken at an edge
(c) Comparison between ABF and Bilateral images. Fig 6.8: Test images.
61
2011-12
6.4.4 ESTIMATED OPTIMAL AND r
Fig 6.9: Estimated optimal parameters for the ABF. (a) Optimal offset for each class. (b) Optimal r for each class. Fig 6.9(a) shows that * is approximately proportional to the LoG class. This confirms the validity of using the LoG response as the feature for pixel classification. The desired amount of offset is proportional to the magnitude of the LoG response. In additional, the sign of * is the same as that of the LoG class. This means that the offset does indeed serve the function described in Section III-A, i.e., it pushes pixel values in edge regions from the edge slope center value either towards the high or low side of the edge based on whether the pixel is originally above or below the midpoint of the edge slope. Fig 6.9(a) and (b) shows that there are three broad classes of pixels in terms of how * and r * behave as a function of LoG class. In fact, Fig 6.9(a) suggests that * is chosen entirely to yield optimal sharpening of the edges. The choice of whether ABF sharpens or smooths is controlled by r * and the local image structure.
Chapter-7
INTRODUCTION TO MATLAB
7.1 INTRODUCTION
MATLAB stands for matrix laboratory developed by mathworks and the version is R2007b. It is used for technical computing and developing the algorithms, data visualization, data computation, array and matrix manipulations. MATLAB is a data analysis and visualization tool designed to make matrix manipulation as simple as possible. In addition, it has powerful graphics capabilities and its own programming language. It is applicable in the field of image processing, signal processing, control designs and so on. It can be integrated by using the C, C++, java, fortan languages etc.
7.2 MATLABS DEVELOPMENT ENVIRONMENT

MATLAB is started from within the Windows environment by clicking the icon that should be on the desktop. (MATLAB is also available for Linux and MacOS, but these environments are not supported in the laboratories, and these versions have their own licensing arrangements which are not covered by the Universitys licence.)
7.3 MATLAB COMPONENTS

MATLABs IDE has five components:
The Command Window, the Workspace Browser, The Current Directory Window, The Command History Window Figure Windows that are active only to display graphical objects.
7.3.1 COMMAND WINDOW

Adaptive bilateral filter for image enhancement and noise removal The Command window is where commands and expressions are typed, and results are presented as appropriate. The workspace is the set of variables that have been created during a session. They are displayed in the Workspace Browser. Additional information about a variable is available there; some variables can also be edited.
Fig 7.1: MATLAB command window.
7.3.2 CURRENT DIRECTORY WINDOW

The current directory window displays the contents of the current working directory and the paths of previous working directories. The working directory may be altered. MATLAB uses a search path to find files. The search path includes the current directory, all of the installed toolboxes plus any other paths that the user has added via the Set Path dialogue accessed from the File menu.
Fig 7.2: Current directory window.
7.3.3 COMMAND HISTORY WINDOW

Adaptive bilateral filter for image enhancement and noise removal The command history window gives a historical view of the current and previous sessions. Commands appearing here can be re-executed. MATLAB provides an editor for writing scripts. It is invoked by typing edit in the command window. Scripts are stored with the extension .m and are therefore also known as m-files.
Fig 7.3: Command history window.
7.3.4 FIGURE WINDOW

Figure windows are generated by MATLAB to display graphical objects.
Fig 7.4: Figure window.
7.3.5 HELP WINDOW

Adaptive bilateral filter for image enhancement and noise removal MATLAB comes with a vast amount of online help and information. To obtain information on a particular command, you can use help. To get more about the doc function the command used is: >> doc doc >> help doc
Fig 7.5: Help command window.
7.3.6 OVERALL WINDOWS
WORKSPACE
COMMAND WINDOW
FIGURE WINDOW COMMAND HISTORY
Fig 7.6: Overall MATLAB Window.
7.4 IMAGE FORMAT

Adaptive bilateral filter for image enhancement and noise removal An image is a rectangular array of values (pixels). Each pixel represents the measurement of some property of a scene measured over a finite area. The property could be many things, but we usually measure either the average brightness (one value) or the brightness of the image filtered through red, green and blue filters (three values). The values are normally represented by an eight bit integer, giving a range of 256 levels of brightness. The resolution of an image is defined by the number of pixels and number of brightness values. A raw image will take up a lot of storage space. Methods have been defined to compress the image by coding redundant data in a more efficient fashion, or by discarding the perceptually less significant information. MATLAB supports reading all of the common image formats such as .jpg, .png, .bmp.
7.5 MATLAB COMMANDS

7.5.1 IMAGE LOADING
An image is loaded into working memory using the command
>> f = imread (image file name);

The semicolon at the end of the command suppresses MATLAB output. Without it, MATLAB will execute the command and echo the results to the screen. We assign the image to the array f. If no path is specified, MATLAB will look for the image file in the current directory.
7.5.2 IMAGE DISPLAY

The image can be displayed using
>> imshow (f, G)

f is the image to be displayed, G defines the range of intensity levels used to display it. If it is omitted, the default value 256 is used. If the syntax [low, high] is used instead of G, values less than low are displayed as black, and ones greater than high are displayed as white. Finally, if low and high are left out, i.e. use [ ], low is set to the minimum value in the image and high to the maximum one, which is useful for automatically fixing the range of the image if it is very small or vary large.
7.5.3 DISPLAY IMAGE IN A FIGURE WINDOW

Images are usually displayed in a figure window. Dept of ECE, RRCE, Bangalore 67 2011-12
>> figure, imshow (f)

This command will generate a new figure window and display the image in it. Note that multiple functions may be called in a single line, provided they are separated by commas.
7.5.4 WRITING IMAGE

An image array may be written to file using:
>> imwrite (array name, file name)

The format of the file can be inferred from the file extension, or can be specified by a third argument.
7.5.5 IMAGE INFORMATION

Information about an image file may be found by
>> imfinfo filename 7.5.6 TYPE CONVERSION

MATLAB provides functions for changing images from one type to another. The syntax is
>> B = data_class_name (A)

Where data_class_name is one of the data types.
>> B = double (A)

This command will convert image A of some type into image B of double integer type, with possible loss of resolution (values less than zero are fixed at zero, values greater than 255 are truncated to 255.)
7.6 TYPE INTERPRETATION RANGE

An image is usually interpreted as being one of: intensity, binary, indexed or RGB. Dept of ECE, RRCE, Bangalore 68 2011-12
Adaptive bilateral filter for image enhancement and noise removal COMMA NDS Uint8 Uint16 Uint32 Int8 Int16 Int32 Single Double Char Logical DESCRIPTION unsigned 8-bit integer unsigned 16-bit integer unsigned 32-bit integer signed 8-bit integer signed 16-bit integer -signed 32-bit integer single precision floating point number Double precision floating point number Character - values are 0 or 1 Table 7.1: List of data types. An intensity images values represent brightness. A binary images pixels have just two possible values. Indexed images pixel values are treated as the index of a look-up table from which the true value is read. RGB images have three channels; representing intensities in ranges of wavelengths corresponding to red, green and blue illumination (other wavelength ranges and more of them are possible). RANGE [0, 255] [0, 65535] [0, 4294967295] [-128, 127] [-32768, 32767] [-2147483648, 2147483647] [-1038, 1038] [-10308, 10308] 2 bytes 1 byte
Chapter-8
RESULTS AND VALIDATION

8.1 INPUT IMAGES AND OUTPUT IMAGES

Standard images of Einstein, mandrill, academy are used as input images in MATLAB.
8.1.1 WINDOWS BITMAP FORMAT

The characteristics of these images are:
Format: .bmp (Windows Bitmap). Size: Einstein = 256X256 pixels, Mandrill = 256X256 pixels, Academy= 300X400 pixels.
Fig 8.1: Input images of a) Einstein.bmp b) Mandrill.bmp c) Academy.bmp.
Fig 8.2: Output images of a) Einstein.bmp b) Mandrill.bmp c) Academy.bmp.
8.1.2 PORTABLE NETWORK GRAPHICS FORMAT

Format: .png (Portable Network Graphics). 70 2011-12
Size: Einstein = 256X256 pixels. Mandrill = 256X256 pixels. Academy= 300X400 pixels.
Fig 8.3: Input images of a) Einstein.png b) Mandrill.png c) Academy.png.
Fig 8.4: Output images of a) Einstein.png b) Mandrill.png c) Academy.png.
8.1.3 JOINT PHOTOGRAPHIC EXPERT GROUP

Format: .jpg (Joint Photographic Expert Group). Size: Einstein = 256X256 pixels. 71 2011-12
Adaptive bilateral filter for image enhancement and noise removal Mandrill = 256X256 pixels. Academy= 300X400 pixels.
Fig 8.5: Input images of a) Einstein.jpg b) Mandrill.jpg c) Academy.jpg.
Fig 8.6: Output images of a) Einstein.jpg b) Mandrill.jpg c) Academy.jpg. The performance of the ABF is evaluated with three images. The first test image is degraded exactly according to the degradation model. The second test image is obtained from a real imaging pipeline on which the aforementioned degradation model is based. The third test image is outside the context of our degradation model. The results obtained with this image will suggest how robust the ABF is for images that are significantly different from our training images.
Chapter-9
CONCLUSION AND FUTURE WORK
72
2011-12
9.1 CONCLUSION
In this project, we presented an adaptive bilateral filter (ABF). The ABF retains the general form of the bilateral filter, but contains two important modifications. First, an offset is introduced to the range filter in the ABF. Second, both and the width of the range filter in the ABF are locally adaptive. They depend on the local image structure, classified according to the output of the Laplacian of Gaussian (LoG) operator applied to the degraded image. The optimal and are estimated in advance during the development of the algorithm via an off-line training procedure using a set of pairs of original and degraded images. During the training procedure, the MSE between the original and reconstructed images is minimized for each class of pixels. The ABF outperforms the bilateral filter in noise removal. At the same time, it renders much sharper images than the bilateral filter does. As a result, the overall quality of the restored image is significantly improved. Compared with the OUM, an adaptive sharpening/denoising method based on the unsharp mask, the ABF restored edges are as sharp as the OUM restored edges, but without the halo artifacts that the OUM produces. The ABF also achieves better noise suppression than the OUM. The ABF sharpens an image by increasing the slope of the edges. Previous slope restoration algorithms involve complex algorithms to determine edge orientation and edge profiles, or compromise the performance by using 1-D horizontal and vertical projections of the edge profiles. In these approaches, the adjustment of the edge profiles tends to produce artifacts. In the ABF, we present a new approach to slope restoration: restoring edge slope by transforming the histogram of the edges. The ABF is efficient to implement, and provides a more reliable and more robust solution to slope restoration. We have demonstrated that the ABF works well for natural and standard images.
9.2 FUTURE DEVELOPMENT

For future development of the ABF, we would suggest that the following issues be addressed. First, the ABF tends to posterize the image, due to its fundamental mechanism of sharpening an image by pulling up or pushing down pixels along the edge slope. Second, the Dept of ECE, RRCE, Bangalore 73 2011-12
Adaptive bilateral filter for image enhancement and noise removal ABF does not perform as well at corners as it does on lines and spatially slow-varying curves, since the ABF is primarily based on transforming the histogram of the local data, which cannot effectively represent 2-D structures. Finally, in the current design of the ABF, a fixed domain Gaussian filter is used. It would be interesting to see what can be gained by jointly optimizing both the domain and the range filters.
BIBILOGRAPHY
[1] R. C. Gonzalenz and Woods, Digital Image Processing, 3rd edition, Pearson India Private Limited, New Delhi, India 2000. [2] Anil. K. Jain, Fundamentals of Digital Image Processing, 3rd edition, Pearson India private limited, New Delhi, India 2000. Dept of ECE, RRCE, Bangalore 74 2011-12
Adaptive bilateral filter for image enhancement and noise removal [3] Simon Haykin, Adaptive Filter Theory, Pearson India Private Limited, New Delhi, India 2009. [4] S. Paris, P. Kornprobst, J. Tumblin, and F. Durand, A gentle introduction to bilateral filtering and its applications, presented at the ACM SIGGRAPH, 2007. [5] B. Widrow and S D Stearns, Adaptive Signal Processing, Pearson India private limited, New Delhi, India, 2005. [6] W. Pratt, Digital Image Processing, 4th edition, Wiley India private limited, New Delhi, India 2010. [7] M. Aleksic, M. Smirnov, and S. Goma, Novel bilateral filter approach: Image noise reduction with sharpening, in Proc. SPIE Int. Soc. Opt. Eng., 2006, vol. 6069, pp. 60690F1 60690F7. [8] J.Tegenbosch, P. Hofman, and M. Bosma, Improving nonlinear up-scaling by adapting to the local edge orientation, in Proc. SPIE Int. Soc. Opt. Eng., 2004, vol. 5308, pp. 1181 1190.
WEBSITES
[1] http://www.einfodaily.com/piTunez/bilateral [2] http://www.harold.phelippeau.free.fr/articles/p3 [3] http://www.ece.lsu.edu/ipl/papers/SPIE_EI2009_blockingartifacts [4] http://www.sciencedirect.com/science/article/pii/S0165168408000066
75
2011-12

Report (Project) 232

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Report (Project) 232

Uploaded by

Copyright:

Available Formats

Adaptive bilateral filter for image enhancement and noise removal

INTRODUCTION TO IMAGE PROCESSING

1.2 IMAGE PROCESSING

1.3 METHODS OF IMAGE PROCESSING

Dept of ECE, RRCE, Bangalore

Adaptive bilateral filter for image enhancement and noise removal

1.3.1 ANALOG IMAGE PROCESSING

1.3.2 DIGITAL IMAGE PROCESSING

1.4 ADVANTAGES OF DIGITAL IMAGE PROCESSING

Dept of ECE, RRCE, Bangalore

Adaptive bilateral filter for image enhancement and noise removal

Fig 1.1: An image as a function.

1.5.1 TYPES OF IMAGE

1.5.2 DIGITAL IMAGE REPRESENTATION

Adaptive bilateral filter for image enhancement and noise removal

1.5.3 BITS PER PIXEL

Bpp 2 bpp 3 bpp 8 bpp 16 bpp 24 bpp

Fig 1.5: Gray scale image.

1.5.3.3 COLOUR IMAGE

Dept of ECE, RRCE, Bangalore

Fig 1.6: High colour image.

Table 1.2: List of image formats.

1.6 COMPONENTS OF IMAGE PROCESSING

Dept of ECE, RRCE, Bangalore

Hard Copy Device

Fig 1.7: Block Diagram of a Typical Image Processing System.

Adaptive bilateral filter for image enhancement and noise removal

Image processing system consists of the following components:-

1.6.2 IMAGE PROCESSOR

1.6.3 DIGITAL COMPUTER

1.6.4 MASS STORAGE

1.6.5 HARD COPY DEVICE

1.6.6 OPERATOR CONSOLE

FUNDAMENTALS OF IMAGE PROCESSING

Dept of ECE, RRCE, Bangalore

Adaptive bilateral filter for image enhancement and noise removal

Fig 1.8: Fundamentals of image processing. Fundamentals of image processing are:-

1.7.1 IMAGE ACQUISITION

1.7.2 IMAGE ENHANCEMENT

1.7.3 IMAGE RESTORATION

1.7.4 MORPHOLOGICAL PROCESSING

1.7.5 IMAGE SEGMENTATION

1.7.6 OBJECT RECOGNITION

Dept of ECE, RRCE, Bangalore

1.7.7 IMAGE ANALYSIS

1.7.8 IMAGE COMPRESSION

1.7.9 COLOUR IMAGE PROCESSING

1.8 APPLICATIONS OF DIGITAL IMAGE PROCESSING

1.8.1 BIOMEDICAL IMAGING TECHNIQUES

Dept of ECE, RRCE, Bangalore

Fig 1.10 Diagrams of X- ray, MRI.

1.8.2 SATELLITE IMAGING

1.8.4 RADAR IMAGING SYSTEMS

1.8.5 DOCUMENT PROCESSING

Adaptive bilateral filter for image enhancement and noise removal

2.2 IMAGE ENHANCEMENT

2.3 PIXEL NEIGHBOURHOODS

Dept of ECE, RRCE, Bangalore

Adaptive bilateral filter for image enhancement and noise removal

2.4 ENHANCEMENT VIA IMAGE FILTERING

Dept of ECE, RRCE, Bangalore

2.4.1 LINEAR FILTERS

Dept of ECE, RRCE, Bangalore