Image Filtering: Noise Removal, Sharpening, Deblurring

Yao Wang Polytechnic University, Brooklyn, NY11201 http://eeweb.poly.edu/~yao

Outline
Noise removal by averaging filter Noise removal by median filter Sharpening (Edge enhancement) Deblurring

Yao Wang, 2006

EE3414: Image Filtering

Noise Removal (Image Smoothing)

An image may be dirty (with dots, speckles,stains) Noise removal:
To remove speckles/dots on an image Dots can be modeled as impulses (salt-and-pepper or speckle) or continuously varying (Gaussian noise) Can be removed by taking mean or median values of neighboring pixels (e.g. 3x3 window) Equivalent to low-pass filtering

Problem with low-pass filtering

May blur edges More advanced techniques: adaptive, edge preserving

Yao Wang, 2006

EE3414: Image Filtering

Example

Yao Wang, 2006

EE3414: Image Filtering

Averaging Filter
Replace each pixel by the average of pixels in a square window surrounding this pixel Trade-off between noise removal and detail preserving:
Larger window -> can remove noise more effectively, but also blur the details/edges

Yao Wang, 2006

EE3414: Image Filtering

Example: 3x3 average

100 100 100 100 100 100 200 195 200 100 100 205 200 205 100 100 203 200 195 100 100 100 100 100 100

100 100 100 100 100

100 144 167 144 100

100 167 200 166 100

100 145 168 144 100

100 100 100 100 100

Yao Wang, 2006

EE3414: Image Filtering

Example

Yao Wang, 2006

EE3414: Image Filtering

Weighted Averaging Filter

Instead of averaging all the pixel values in the window, give the closer-by pixels higher weighting, and far-away pixels lower weighting.

g (m, n) =

l = L k = L

h( k , l ) s ( m k , n l )

This type of operation for arbitrary weighting matrices is generally called 2-D convolution or filtering. When all the weights are positive, it corresponds to weighted average. Weighted average filter retains low frequency and suppresses high frequency = low-pass filter

Yao Wang, 2006

EE3414: Image Filtering

Graphical Illustration

Yao Wang, 2006

EE3414: Image Filtering

Example Weighting Mask

1 1 1

1 1 1

2 4 2

1 2 1

1 9

1 1

1 16

2 1

All weights must sum to one

Yao Wang, 2006

EE3414: Image Filtering

10

Example: Weighted Average

100 100 1
1 9

100 200 195 200 100

100 205 200 205 100

100 203 200 195 100

100 100 100 100 100

1 16

1 1 1

1 1 1

100 100 100

1 2 1

2 4 2

1 2 1

1 1

100 100 100 100 100

100 144 167 144 100

100 167 200 166 100

100 145 168 144 100

100 100 100 100 100

100 100 100 100 100

100 156 174 156 100

100 176 201 175 100

100 158 175 156 100

100 100 100 100 100

Yao Wang, 2006

EE3414: Image Filtering

11

Filtering in 1-D: a Review

Continuous-Time Signal
Time Domain : g (t ) = s (t ) h(t ) = h( ) s (t )d

Frequency Domain : G ( f ) = S ( f ) H ( f )

Filter frequency response : H ( f ) = h(t )e j 2ft dt

Extension to discrete time 1D signals

Time Domain (linear convolution) : s (n) = s (n) h(n) = h(m) s (n m)
m

Frequency Domain : G ( f ) = S ( f ) H ( f ) Filter frequency response (DTFT) : H ( f ) = h(n)e j 2fn

n

H ( f ) is periodic, only needs to look at the range f (-1/ 2 ,1/ 2 ) 1/2 corresponds to f s / 2

Yao Wang, 2006

EE3414: Image Filtering

12

Filtering in 2D
Weighted averaging = 2D Linear Convolution g (m, n) =
l =l0 l1 k = k0 k1

h(k , l ) s (m k , n l )

In 2D frequency domain G ( f1 , f 2 ) = S ( f1 , f 2 ) H ( f1 , f 2 ) Frequency response of the 2D Filter H ( f1 , f 2 ) =

m =l0 n = k 0

h(m, n)e

l1

k1

j 2 ( f1m + f 2 n )

H ( f1 , f 2 ) is periodic, only needs to look at the square region f1 (1 / 2,1 / 2), f 2 ( 1 / 2,1 / 2).

Yao Wang, 2006

EE3414: Image Filtering

13

Frequency Response of Averaging Filters

Averaging over a 3x3 window
0 0 1 h(m, n) = 0 9 0 0 0 1 1 1 0 0 1 1 1 0 0 1 1 1 0 0 0 0 0 0

H ( f1 , f 2) = 1 9 1 + 9 1 = 9 1 = 9 +

1 j 2 ( f1 ( 1) + f 2 ( 1)) j 2 ( f1 ( 0 )+ f 2 ( 1)) j 2 ( f1 (1) + f 2 ( 1)) e +e +e 9 + e j 2 ( f1 ( 0 ) + f 2 ( 0 )) + e j 2 ( f1 (1) + f 2 ( 0 )) + e j 2 ( f1 ( 0 ) + f 2 (1)) + e j 2 ( f1 (1)+ f 2 (1))

(e (e (e (e

j 2 ( f1 ( 1) + f 2 ( 0 ))

) ( )

j 2 ( f1 ( 1) + f 2 (1))

j 2f1

+ 1 + e j 2f1 e j 2f 2 + + 1 + e j 2f1 e j 2f 2

j 2f1

)(

1 j 2f1 1 e + 1 + e j 2f1 1 + e j 2f1 + 1 + e j 2f1 e j 2f 2 9 9 1 + 1 + e j 2f 2 = (1 + 2 cos 2f1 )(1 + 2 cos 2f 2 ) 9

Yao Wang, 2006

EE3414: Image Filtering

14

H ( f1 , f 2 ) = H ( f1 ) H ( f 2 ) H ( f1 ) = 1 (1 + 2 cos 2f1 ), H ( f 2 ) = 1 (1 + 2 cos 2f 2 ) 3 3

Sketch H(f1)
1

0.5

-0.5 30 20 10 0 0 5 10 15 20 25

Yao Wang, 2006

EE3414: Image Filtering

15

Frequency Response of Weighted Averaging Filters

1 b 1 b b2 H= (1 + b) 2 1 b 1 1 1 b = (1 + b) 2 b [1 b 1]; 1 1 H (u , v) = (b + 2 cos(2u ) )(b + 2 cos(2v) ) /(1 + b) 2

1 0.8 0.6 0.4 0.2 0 30 20 10 0 0 5 10 15 20 25

Yao Wang, 2006

EE3414: Image Filtering

16

Averaging vs. Weighted Averaging

1 1 1 1 1 H = 1 1 1 = 1[1 1 1]; 9 1 1 1 1 H (u , v) = (1 + 2 cos(2u ) )(1 + 2 cos(2v) ) / 9 1 1 1 b = (1 + b) 2 b [1 b 1]; 1 1 H (u , v) = (b + 2 cos(2u ) )(b + 2 cos(2v) ) /(1 + b) 2 1 b 1 H= b b2 2 (1 + b) 1 b

1 0.8 0.6

0.5

0.4 0.2

-0.5 30 20 10 0 0 5 10 15 20 25

0 30 20 10 0 0 5 10 15 20 25

b=2
Yao Wang, 2006 EE3414: Image Filtering 17

Interpretation in Freq Domain

Filter response Low-passed image spectrum Original image spectrum Noise spectrum 0
Noise typically spans entire frequency range, where as natural images have predominantly lower frequency components

Yao Wang, 2006

EE3414: Image Filtering

18

Median Filter
Problem with Averaging Filter
Blur edges and details in an image Not effective for impulse noise (Salt-and-pepper)

Median filter:
Taking the median value instead of the average or weighted average of pixels in the window
Median: sort all the pixels in an increasing order, take the middle one

The window shape does not need to be a square Special shapes can preserve line structures

Order-statistics filter
Instead of taking the mean, rank all pixel values in the window, take the n-th order value. E.g. max or min

Yao Wang, 2006

EE3414: Image Filtering

19

Example: 3x3 Median

100 100 100 100 100 100 200 195 200 100 100 205 200 205 100 100 203 200 195 100 100 100 100 100 100 100 100 100 100 100 100 100 200 100 100 100 200 200 195 100 100 100 200 100 100 100 100 100 100 100

Matlab command: medfilt2(A,[3 3])

Yao Wang, 2006 EE3414: Image Filtering 20

Example

Yao Wang, 2006

EE3414: Image Filtering

21

Matlab Demo: nrfiltdemo

Original Image Corrupted Image Filtered Image

Can choose between mean, median and adaptive (Wiener) filter with different window size
Yao Wang, 2006 EE3414: Image Filtering 22

Noise Removal by Averaging Multiple Images

Yao Wang, 2006

EE3414: Image Filtering

23

Image Sharpening
Sharpening : to enhance line structures or other details in an image Enhanced image = original image + scaled version of the line structures and edges in the image Line structures and edges can be obtained by applying a difference operator (=high pass filter) on the image Combined operation is still a weighted averaging operation, but some weights can be negative, and the sum=1. In frequency domain, the filter has the highemphasis character
Yao Wang, 2006 EE3414: Image Filtering 24

Frequency Domain Interpretation

Filter response (high emphasis)

sharpened image spectrum Original image spectrum

Yao Wang, 2006

EE3414: Image Filtering

25

Highpass Filters
Spatial operation: taking difference between current and averaging (weighted averaging) of nearby pixels
Can be interpreted as weighted averaging = linear convolution Can be used for edge detection

Example filters

0 1 0 0 1 0 1 4 1; 1 4 1; 0 1 0 0 1 0
All coefficients sum to 0!

1 1 1 1 1 1 1 8 1; 1 8 1; 1 1 1 1 1 1

Yao Wang, 2006

EE3414: Image Filtering

26

Example Highpass Filters

0 1 0 1 4 1 0 1 0

1 1 1 1 8 1 1 1 1

8 6 4 2 0 30 20 10 0 0 5 10 15 25 12 10 8 6 4 2 0 30 20 10 0 0 5 10 15 20 20 25

Yao Wang, 2006

EE3414: Image Filtering

27

Example of Highpass Filtering

Original image

Isotropic edge detection

Binary image

Yao Wang, 2006

EE3414: Image Filtering

28

Designing Sharpening Filter Using High Pass Filters

The desired image is the original plus an appropriately scaled high-passed image Sharpening filter

f s = f + f h

hs (m, n) = (m, n) + hh (m, n)

f(x) g(x)=f(x)*hh(x) fs(x)=f(x)+ag(x)

x
Yao Wang, 2006 EE3414: Image Filtering

x
29

Example Sharpening Filters

f(x) g(x)=f(x)*hh(x) fs(x)=f(x)+ag(x)

0 1 0 0 1 0 1 1 H h = 1 4 1 H s = 1 8 1 with = 1. 4 4 0 1 0 0 1 0
1 1 1 1 1 1 1 1 H h = 1 8 1 H s = 1 16 1 with = 1. 8 8 1 1 1 1 1 1
Yao Wang, 2006 EE3414: Image Filtering 30

Example of Sharpening

1 1 1 1 H h = 1 8 1 4 1 1 1

1 1 1 1 H s = 1 16 1 8 1 1 1

Yao Wang, 2006

EE3414: Image Filtering

31

Example of Sharpening

=4

=8

Yao Wang, 2006

EE3414: Image Filtering

32

Challenges of Noise Removal and Image Sharpening

How to smooth the noise without blurring the details too much? How to enhance edges without amplifying noise? Still a active research area

Yao Wang, 2006

EE3414: Image Filtering

33

Wavelet-Domain Filtering

Courtesy of Ivan Selesnick

Yao Wang, 2006 EE3414: Image Filtering 34

Feature Enhancement by Subtraction

Taking an image without injecting a contrast agent first. Then take the image again after the organ is injected some special contrast agent (which go into the bloodstreams only). Then subtract the two images --- A popular technique in medical imaging

Yao Wang, 2006

EE3414: Image Filtering

35

Image Deblurring
Noise removal considered thus far assumes the image is corrupted by additive noise
Each pixel is corrupted by a noise value, independent of neighboring pixels

Image blurring
When the camera moves while taking a picture Or when the object moves Each pixel value is the sum of surrounding pixels blurred image is a filtered version of the original The

Deblurring methods:
Inverse filter: can adversely amplify noise Wiener filter = generalized inverse filter Many advanced adaptive techniques

Yao Wang, 2006

EE3414: Image Filtering

36

Example of Motion Blur

Yao Wang, 2006

EE3414: Image Filtering

37

Inverse Filtering vs. Wiener Filtering

Yao Wang, 2006

EE3414: Image Filtering

38

Constrained Least Squares Filtering

Yao Wang, 2006

EE3414: Image Filtering

39

What Should You Know

How does averaging and weighted averaging filter works? How does median filter works? What method is better for additive Gaussian noise? What method is better for salt-and-pepper noise? How does high-pass filtering and sharpening work? For smoothing and sharpening:
Can perform spatial filtering using given filters Deriving frequency response is not required, but should know the desired shape for the frequency responses in different applications

What is the challenge in noise removal and sharpening? What causes blurring? Principle of deblurring: technical details not required

Yao Wang, 2006

EE3414: Image Filtering

40

References
Gonzalez and Woods, Digital image processing, 2nd edition, Prentice Hall, 2002. Chap 4 Sec 4.3, 4.4; Chap 5 Sec 5.1 5.3 (pages 167-184 and 220-243)

Yao Wang, 2006

EE3414: Image Filtering

41