A Multi-Feature Approach to Smoke Detection

Maria Charmaine M. Templado

Institute of Computer Science University of the Philippines Los Baos College, Laguna, Philippines 4031 + (63)9273629402

Vladimir Y. Mariano
Institute of Computer Science University of the Philippines Los Baos College, Laguna, Philippines 4031 (63-49) 536-2313/2302

charmaine.templado@gmail.com

vymariano@uplb.edu.ph

ABSTRACT
This paper presents a method for smoke detection in video using multiple features. In this method, it is assumed that the camera that captures the video is stationary. The presence of smoke in an image creates significant changes to the regions comprising the image. Smoke increases the luminance of an image region while dark smoke decreases the luminance value. Contrast starts to decrease as smoke gets in the way. The edges of objects within the image loose their sharpness and the color components eventually decrease in saturation. Using these features, a classifier detects the presence of smoke in each region.

Keywords
Smoke, smoke detection, luminance, contrast, edge magnitude, saturation, background image, first frame, region

the scene and by estimating the background and the current images. This method also focused on the change in the edges of the scene because they produce local extrema. The decrease in the local extrema indicates smoke. Toreyin often times used the grayish color found in the image as one indicator of smoke. He also based the results on the changes that happen with regard to intensity and contrast. One of the systems for fire and smoke detection based the detection of smoke in the change in intensity, contrast degradation, edge deterioration, and turbulent dynamics instead of depending on movements within the scene. This Video Imaging Smoke Detection (VISD) is an application of artificial intelligence that analyzes video images to detect smoke and fire at a greater distance, covering larger areas and proving faster detection times. Another fire detection system developed by Dizon [2] used smoke detection in line with fire since smoke always exists before fire build up. This system can detect fire, smoke or both. In smoke detection, Dizon used the object isolation and background extraction. He used the grayish color as indicator that the pixel has smoke.

1. INTRODUCTION
"When there is smoke there is fire." This is one of the common saying indicating bigger or worse things by the presence of some smaller ones related to it. Taking this saying literally, a smoke is present whenever there is fire. Fire may already be too large before it reaches the camera's point of view and at this point, detection may be too late. This is one good reason to develop methods for smoke detection since smoke spreads faster than fire. Given a large open space, smoke can travel in the air several times faster to reach the view of the camera making it possible for early detection. Smoke detection in video may also be several steps ahead compared to smoke sensors when distance and large space is concerned. Video based smoke detection method can detect smoke even without it reaching the sensors. Videos are volume sensors while devices are point sensors. This study was designed to create a method for smoke detection using multi-feature approach. This method tries to detect smoke by comparing the values of the different features. Previous works on smoke detection using video based their results in movements and color analysis. Toreyin et al [5] used Hidden Markov Model to distinguish the moving objects including flame flickers. In [3], Toreyin et al used the spatial wavelet transforms to detect the decrease in high frequency of

2. METHOD 2.1 Method Flow

Smoke detection algorithm uses the luminance, contrast, edge magnitude, and saturation value comparison between the background image and the other image sequences. Figure 1 shows the flow of the detection algorithm.

The formula above shows how the color components, red, green, and blue (RGB), are multiplied to constants to get the luminance of that particular pixel. Basically, the color green is seen brighter than blue, this is the reason why luminance is used to see the adjusted brightness to indicate appropriately the actual colors of an image. This is also the case why green has the largest constant multiplier among the three components. Since the image is divided into regions, the value of the luminance of that region is the average of the accumulated luminance values of all the pixels in it. Luminance values are sorted and are stored in an array which is used in the computation of contrast. Figures 3.1 to 3.3 show how the increase and decrease in luminance affect the appearance of an image.

Figure 1 Smoke Detection Method Flowchart.

2.2 Region Division

Smoke detection in this method use regions as point of reference instead of pixel by pixel traversals. Each image in the sequence is divided into regions of 20x20 pixels, as shown in Figure 2, to minimize the scope while generalizing the feature values. This region division disregards the presence of extreme values by getting the average of the values in a particular region.

Figure3.1 Original Image.

Figure 3.2 Increase in the luminance of the image that resembles the luminance created when there is white smoke.

Figure 2 Region division.

2.3 Luminance
Change in average luminance value indicates that smoke appears in the scene. Since the camera is static, the background of each image is the same all through out the video. In this method, the first frame serves as the background. There must be no other objects present, which will eventually disappear later. Therefore if the first frame is clear, all other changes with the succeeding frames will be identified. The first instance to consider is the change in luminosity. Luminance describes the amount of light that passes through a particular area, and falls within a given solid angle. Figure 3.3 Decrease in the luminance of the image that resembles the luminance whenever dark smoke is present.

2.4 Contrast
The contrast or the luminance contrast is defined to be the relationship between the luminance of a brighter area of interest and that of an adjacent darker area. The decrease in the contrast value is an indication of smoke (see Figure 4). There are several ways of computing the contrast of an image namely

luminance = 0.3 red + 0.59 green + 0.11 blue

the Weber contrast, Simple, and Peak-to-peak contrast (Michelson Contrast, Modulation). The Simple Contrast value is used to specify the difference between the bright and dark parts of the picture just by dividing the maximum luminance value over the minimum value. This contrast is often used in photography and not in real-world luminance. The Weber contrast is used in the context of lighting by dividing the difference of the maximum and minimum luminance by the minimum value. The Peak-to-Peak contrast is the type of contrast that was used in this method wherein it measures the relation between the spread and the sum of two luminance. This contrast is calculated by dividing the difference of the maximum and minimum luminance by the sum of the two. Below is a formula of the Peak-to Peak contrast.

Peak-to-Peak contrast maximumLuminance - minimumLuminance maximumLuminance + minimumLuminance

In this case, Peak-to-Peak contrast was modified. Instead of using the maximum and minimum values, this method uses the ninety-fifth percentile and the fifth percentile as shown in the formula below, to disregard the presence of extreme values.

Figure 5 The flowchart shows how the image undergoes edge detection to calculate the edge gradient magnitude.

Modified Peak-to-Peak contrast

luminance [95%] - luminance [5%] luminance [95%] + luminance [5%] Storing of values will happen every time an image is read; therefore the comparison takes place every time the frame values are calculated. Values for every region of each frame are compared to the first frame.

Figure 6 These figures show the edge gradient magnitude image. The edge magnitude decreases when there is smoke.

2.6 Saturation
The presence of smoke decreases the saturation of an image. Once smoke gets in the way of an object, it decreases the vividness of the objects color. The saturation was computed using the RGBtoHSI conversion as shown in the formula below.

saturation = 1.0 - 3.0 * min (red, min( green, blue) ) (0.00001 + red + green + blue)
This formula of saturation produces value result between 0 and 1. A resulting value of 0 indicates that the region or pixel is closer or in its gray scale while a value of 1 means a fully saturated or no white component region or pixel.

Figure 4 The presence of smoke caused the contrast in the circled area to decrease.

2.7 Smoke Detection 2.5 Edge Magnitude

The edges in an image start to loose sharpness whenever smoke appears. Using the Sobel operator, the gradient of an image is measured in order to emphasize regions of high spatial frequency that corresponds to edges. In this part, the image being processed used the grayscale of the original image. The image is smoothened to reduce noise. Using the respective masks, the Sobel gradients, say x and y, are then computed followed by the computation for the gradient magnitude. Figure 5 shows the process of computing for the gradient magnitude and Figure 6 shows how smoke decreases the number of edges in an image. There may be smoke if the luminance values change. If the luminance of the succeeding frames increases more than an additional value L1 of the average recent values, indications of smoke are at hand. In this method, L1 was set to be equal to 10. There is also the possibility of a region to have smoke when the luminance decreases by a value of L2, where L2 is equal to 10. L1 and L2 are constants added to the luminance value of the first frame. These values may vary depending on the amount of smoke needed to be detected. The luminance decreases once the smoke is darker than the background and increases when it is lighter than the background. There may be smoke when the gradient magnitude decreases. The gradient magnitude must be in between two values, say G1 and G2, as range of the gradient magnitude of the region distinguishing it from ordinary moving objects. G1

is a percentage added to the gradient magnitude of the first frame and G2 is a percentage subtracted to the first frame. The contrast and saturation values of the region decrease when there is smoke but there are times when contrast also increases. This case happens when the smoke is more likely to become solid in appearance and it is much whiter than the background. Each feature values alone are not reliable to detect smoke. They must be combined to suit the conditions of having smoke. The pseudo code below will describe the conditions for smoke detection: if ( luminance of frame 1 + L1 < luminance of other frames OR luminance of frame 1 L2 > luminance of other frames ) AND 90% of the saturation < of the first frame AND contrast is between 300% and 70% of the contrast value of the background AND gradient magnitude is between 250% and 35% of the gradient magnitude value of the background { region has smoke } else region has no smoke saturation of other frames

passes. Increase in saturation increases the possibility of false detections.

3. EXPERIMENTAL RESULTS
This method was tested on a video captured outdoors with enough light to see the smoke created using incense and motor oil. To indicate the changes happening within each image, a region of interest of 160x120 was set. This region of interest was accessed as the frame sequences were read. All the feature values of each frame were plotted to an image of the same dimension. Sample graph images are presented in Figure 7. Figure 8 to 10 shows the results of this method. Figure 8 shows the images with detected smoke. This method can be used both indoors and outdoors provided that enough light is present to see the smoke. False detection occurs in regions where the values of the features are almost the same with the values of regions with smoke. Figure 9 shows false detection where regions were detected as having smoke even if there were none because the feature values of those regions resembles that of a smoke. Misdetection occurs when the amount of smoke in a region is small. This causes the feature values not to increase or decrease unlike what smoke can do. Another reason for misdetection was when the color of the smoke is similar to the background. In Figure 10, the grayish color of the smoke is almost the same with the color of the cement in its background. Too much light can also cause misdetection wherein it makes the smoke undistinguishable from the other portions of the image. Misdetection is also caused by the increase in saturation in cases where the background is dark, most specifically when it is almost black. In cases like this, the saturation increases whenever an object of higher saturation

Figure 7 These figures show the graph of luminance, contrast, edge magnitude, and saturation values of the a region of interest as time passes. The graph show how values change as smoke appears in a given region.

Figure 9 False detections in scenes with the presence of objects possessing the same feature values like the smoke.

Figure 8 These figures show the original images together with the result after the smoke detection method. This detection was done by blocks, rather than per pixel, to lessen the effect of noise and extreme feature values. These images also show how smoke was recognized and detected unlike other ordinary moving objects such as the moving person in the video.

Figure 10 Misdetection due to factors such as similarities between the background and the smoke itself and a small amount of smoke in a given region. Table 1 Detection result of the multi-feature approach to smoke detection. Video no. and description Total no. of frames No. of frames with visible smoke 2535 No. of frames detected as having smoke 2517 Total no. of frames with false detection 142

Video 1
(smoke of an incense inside a can)

2537

Video 2
(smoke of a motor oil)

2418

1195

2374

409

Table 1 is a summary of the detection accuracy of this method. Video 1 showed a smoke of incense poured in burning coal. The smoke that was created was light and not thick. The moving object in the scene was darker than the background which made it undetected. Video 2 used motor oil to produce smoke. This video showed thick white smoke which possessed the same characteristics as a white shirt worn by the man in the scene. This event created much false detection since the thick white smoke and the white shirt possessed similarities in feature values.

5. ACKNOWLEDGMENT
We would like to acknowledge the people who help us accomplish this research. A special thanks to God for providing us with the strength and courage to fulfill this project. We would also like to thank the people who guide us through the process of conducting the study from beginning to end.

6. REFERENCES
[1] Chang Y., et al., Texture-Based Color Image
Segmentation Using Local Contrast Information, Brigham Young University, Provo, Utah, USA.

4. CONCLUSION AND FUTURE WORK

A method for smoke detection was developed which used video as input. This method was basically based on changes in luminosity, contrast, saturation, and edge magnitude values of the succeeding frames compared to the first frame. As the values of the succeeding frames satisfy the conditions, the more it was detected of having smoke. The current method can only detect grayish colored smoke and not colored smokes. The primary consideration in this method was that the first frame, which was set as the background, must be free from objects that will eventually disappear after some series of frames. This method can support videos of smoke that were captured under a fair lighting condition, especially during daytime, both indoors and outdoors but not during nighttime where the images are dark and the smoke could not be clearly seen. Further improvements on the evaluation needs to be extended in cases when there are smoke colored moving objects since these objects can be regarded as thick smoke. Case when the background is dark or almost black must also be considered apart from colored or lighter background because this results to an increase in saturation, therefore increases the possibility of false detection. There is also a need to collect lots of video data to evaluate the efficiency of this method.

[2] Dizon, D.A., Real-Time Fire Detection via Features

Extraction on Video Sequences, Undergraduate Special Problem, University of the Philippines at Los Baos, Philippines, IN (2006).

[3] Toreyin, B.U., et al., Wavelet Based Real-Time Smoke

Detection in Video, Bilkent University, Ankara, Turkey, IN EUSIPCO 05 (2005).

[4] Toreyin, B.U., et al., Contour Based Smoke Detection in

Video Using Wavelets, Bilkent University, Ankara, Turkey, IN EUSIPCO 05 (2005).

[5] Toreyin, B.U., et al., Flame Detection in Video Using

Hidden Markov Models, Bilkent University, Ankara, Turkey, IN ICIP 05 (2005).

[6] Xiong, Z., et al., Video- Based Smoke Detection:

Possibilities, Techniques and Challenges, East Hartford, CT.