International Conference on Emerging Research in Electronics, Computer Science and Technology - 2015
Traffic Signal Time Analysis and Voice - Based App
for Visually Impaired Pedestrians
Basavaraju R
Research Scholar - VTU RRC,
Visvesvraya Technological University,
Bangalore, India
basavaraj u.revanna@gmail.com
Abstract- Analysis of traffic signal and ensure the safety of
visually impaired people is a major challenge in helping disabled
people. In this paper, we propose development of an application
software which can be easily installed on a mobile device
equipped with a camera. This application opens a camera and
captures the traffic timer display upon tapping the app-icon. The
timer image is then processed to segment the digits in them to
identify the actual time in the numeric form. The detected time is
then converted as a voice message and played using the app.
Thus, the visually impaired person can hear the message about
the time left to turn on the pedestrian signal. Getting this alert
message, he/she can safely cross the road. The accuracy of the
proposed algorithm is found to be 100% as it detected all the
digits in every instance of the timer display image.
Index Terms- Android SDK, connected components,
correlation coefficient, histogram, preprocessing, segmentation,
text to speech conversion.
I. INTRODUCTION
It is a well known fact all over the world that the visually
impaired (partially or completely blind) people face difficulty
in crossing the roads. Visually impaired people can travel or
cross the street with the help of other pedestrians, using a white
cane to identity and avoid obstacles, using a dog guide, using
special optical or electronic aids etc [1]. Crossing the
road/traffic signal for visually impaired people involves
following tasks [1]:
Locating the Streets: Understanding when they reach
the street. This may include a combination of cues,
curb, slopes, traffic sound etc.
Street Recognition: Blind people has to recognize
which street they have reached to using their mental
map; may be by counting the steps, number of turns
taken etc.
Intersection Assessment: Obtaining possible
intersection of roads, number of streets present in the
intersection, which exact street they need to travel etc.
Crossing the Road: After getting the above
infonnation, facing towards proper street and then
crossing the road.
However, these tasks are not so easy for blind pedestrians,
especially, when there are traffic signals. Even if a blind
978-1-4673-9563-2/15/$31.00 2015 IEEE
Dr. Chetana Hegde
Associate Professor, Department ofMCA,
RNS Institute of Technology,
Bangalore, India
chetanahegde@ ieee.org
pedestrian analyzes the street which he/she need to head
towards, he/she cannot proceed further when there is a traffic
signal. It is necessary for such a people to understand when
there is a green signal to cross the road.
Given a scenario where traffic signals are possessing timer
display, it would be great if the blind pedestrian is informed
how much time is remaining for green signal to tum-on. If the
blind pedestrian is informed the time remaining through voice,
he/she can wait and then safely cross the road.
Keeping the problems faced by visually impaired
pedestrians in mind, this paper proposes development of an
Android based application involving image processing
techniques. A novel algorithm is proposed to capture the timer
display as an image, process it to recognize the actual time and
to inform the time to the user through voice. The paper is
organized as follows : Section II deals with the related work and
Section III presents the architecture and model. Section IV
describes the implementation of the proposed algorithm and the
perfonnance analysis. Section V concludes the work.
II . RELATED WORK
A brief survey of the related work in the area of various
aids available for blind pedestrians as well as the analysis of
traffic signal is presented in this section.
In 1980's, the scientists at Nottingham University proposed
the idea of having a small device containing plastic/metal cone
to be attached at the pole on the pedestrian crossing [2]. When
a blind person needs to cross the road, he/she has to go near to
that pole and press the cone. Now, the cone starts spinning and
the signal is made to the vehicles to stop. The spinning of a
cone will ensure the pedestrian to cross the read safely. But,
proposal has many disadvantages: (i) The blind pedestrian has
to locate the pole having this cone to use it. (ii) It may not be
possible to set up this device at all the signals. (iii) Improper
functionality of the device, in case of damage, will put the
blind pedestrian in a dangerous situation.
These push button cones have then been enhanced to
generate the locator tone [1, 3]. The locator tone infonns
pedestrians of the need to push a button and provides an
audible cue to the location of the pushbutton. During the street
crossing the locator tone may be audible in the last lane and
provide a cue to the direction of the destination. But, to use this
device also, one has to go near to the device mounted on the
477
 International Conference on Emerging Research in Electronics, Computer Science and Technology - 2015
pole. Moreover, it will be difficult to hear the tone in busy
traffic with lot of vehicles moving around. Also, if there are
intersection of roads, it may be confusing to the pedestrian that
to which road, the tone is beeping the alert.
Researchers at Kyoto Institute of Technology in Japan have
developed an electronic eye system to help the blind people to
cross the road at busy intersections [4]. The system consists of
a camera and a small computer that can detect pedestrian
crossings, including the width of each crossing and the status
of its traffic lights. Instructions are relayed to the user by
means of a voice speech system with a built-in speaker. But,
this requires the blind pedestrian to carry this system with
him/her may in the form of wearable device, which may not be
affordable by every individual. Moreover, this system has been
tested on 196 intersections, out of which 194 have been
detected correctly yielding an accuracy of98.97%.
Several efforts have been made by the researchers [5 - 10]
to monitor the traffic signal by analyzing green/red lights
through image processing techniques and neural networks. But,
none of these researches have focused to aid visually impaired
people. But, the study [11] about number of road accidents
faced by visually impaired people enforces reliable research to
be carried out to aid them.
The proposed method is found to be 100% accurate in
detecting the time left-out in the signal and hence aiding the
visually impaired people to cross the road. The proposed
method is discussed in the following sections.
III. ARCHETECTURE AND MODELLING
Initially, a database has to be setup containing 10 images
depicting every digit from 0 to 9. Given an image containing
timer display (ex. Digital clock), obtaining the time involves
identitying each digit as a major module. Then all the digits
have to be combined to make a sequence to represent the time
as a text, which, then needs to be converted into voice. The
architectural diagram for the entire process is as shown in
Figure 1. The various steps involved are explained hereunder.
Digital Clock
lmage
Acquisition
Database
Figure I. Architectural Diagram
A. Jmage Acquisition and ROJ Detection
The Android based application developed for the proposed
algorithm opens the camera of mobile phone when tapped on
the app-icon. The person holding the mobile device can capture
the image of a timer display at the traffic. Since, the blind
people will be using the device, there is no guarantee that the
image actually contains the time. Hence, through the pattern
matching algorithm, it is ensured that the image contains the
timer display. Also, by enhancing techniques, region of interest
(ROI) is cropped from the original image. For simulation
purpose, images of ordinary digital clock are considered. The
image after extracting ROI is as shown in Figure 2 (a).
(a) ROI extracted (b) Seperator Removed
Figure 2. Image of digital clock display
B. Preprocessing
Preprocessing involves series of steps. Initially, the RGB
image has to be converted into binary image for reducing the
future complexity in the computation. It is a known fact that
the digital clock images or timer displays have hour/minute
seperator in the form of colon symbol. For identitying digits in
the image, we need to remove these seperators. It involves
following steps -
(i) Determining the Connected Components: Initially,
connected components in a binary image are extracted. Let Y
represent a connected component contained in a set A and
assume that a point p of Y is known. Then the equation (1)
containing interative expression yields all the elements of Y -
Xk = (X k _ 1 EBB)nA, k=J, 2, 3, . (1)
Where Xo = P and B is a suitable structuring element. If X k =
X k j , then the algorithm converges and we get Y = X k
(ii) Compute area of each component: Area of each
component is computed by counting number of pixels in the
regaion.
(iii) Remove small objects: Small objects in the set A
containing many connected components are removed if the
area of any component is less than a threshold value.
The above three steps remove the seperator as shown in
Figure 2(b). Later, morphological closing is applied to fuse the
narrow breaks and long thin gulfs, to eleminate small holes and
to fill the gaps in the contour in the image. This operation will
ensure that each digit is one connected component. Closing is
done using the equation -
A. B = (A EB B)eB (2)
Here, A is the image and B is the structuring element. The
image after closing operation is as shown in Figure 3.
03 18 Figure 3. Image after closing
478
 International Conference on Emerging Research in Electronics, Computer Science and Technology - 2015

C. Digit Segmentation
To segment each digit from the whole image, initially,
connected components from the preprocessed image are (5)
extracted. Then, each digit is segregated by iteratively hiding
the connected components and using the bouding box of each
row in the respective region. This procedure is carried out 1=1 1=1
using following steps - Here,
(i) Search for the next unlabeled pixel, p .
(ii) Use a flood-fill algorithm to label all the pixels in the
connected component containing p.
(iii) Repeat steps (i) and (ii) until all the pixels are labelled.
6ooor-----,-----~------~----_r----~----~

The images containing individual digits are saved with

approriate file names. Digits extracted from the preprocessed 5000
image are shown in Figure 4(a) - (d).
4000

3000

2000

1000

Figure 4. Images containing extracted digits 0~--750~--~1700~--~15~0~--~

20~0---~2~5~
0 ----~300
Range of Intensity va lue
D. Pattern Matching
Figure 5. Histogram obtained for the the digit One
This steps involves comparing extracted digits in the
previous step with the stored images in the database. 6000.------r----~------~----~----~~--__,
Histograms for each of the extracted digits is plotted. For
intensity images, the n bins of the histogram are each half-open
5000
intervals of width A/(n-l) . That is, for intensity images having
float values, the / ' bin is the half-open interval given by -
4000
A(p -1.5) A(p - 0.5)
----''''--------'- ::; X ::; -=----'- (3)
n-l n-l 3000
where x is the intensity value. For intensity images having
integer values, the i h bin is the half-open interval given by - 2000

A(p -1.5) _ 32768::; X::; A(p - 0.5) _ 32768 (4) 1000

n-l n-l
where x is the intensity value. The scalar factor A depends on
the data type of image class and it may be one among the
values 1,255 or 65535. Histograms plotted for two digits 1 and
8 are shown in Figure 5 and Figure 6.
Histogram computed for each extracted digit is compared
with every digit image in the database. The correlation
coefficient is computed between every pair of histograms. If HI
and H2 are histrograms of two images, then the correlation
coefficient between them is calculated using the Equation (5).
If the correlation coefficient r is nearer to 1, then we can
say that the two digits are the same. Hence, we can identify the
actual digit stored in the extracted image.
..II
0~---~50~--~1~0~0----~1~50~--~
20~0~-~25~0----~
300
Range of Inten sity valu e
Figure 6. Histogram obtained for the the digit Eight
E. Time Detection and Text to Voice Conversion
The detected digits can be stored as numbers and hence the
time can be recognized and displayed. The result in numeric
format is displayed as -
The time remaining: 3 minutes 18 seconds
But, to aid the blind people, the time has to be informed
through voice. Hence, the time in number format has to be
stored as a vector of strings. This text message is then
converted to speech.

479
 International Conference on Emerging Research in Electronics, Computer Science and Technology - 2015
IV. IMPLEMENTA nON AND PERFORMANCE ANALYSIS
The proposed technique is implemented using different
technologies. The initial steps of image capturing and ROI
detection are done using Java language in Android SDK. Later,
image processing techniques like preprocessing, digit
seperation, pattern matching using histogram and correlation
coefficient are done using MatLab 7.5. Again, for text to voice
conversion, the built-in class TextToSpeech within Android
SDK is used.
The proposed algorithm is tested on 200 digital clock
images as well as real-time traffic timer displays. The images
were acquired with different orientation of the camera. Up to
150 degrees of orientation (angle between camera and the timer
display), the digits were identified correctly. Beyond 150, the
image can not be captured properly and hence the efficiency
reduces. It is obvious that, in real time situation, the camera
orientation do not cross 150, hence the proposed algorithm
gives lOO% accuracy in detecting the time.
V. CONCLUSIONS
In this paper, we propose an efficient way of extracting
exact time left for a traffic signal to tum green/red. The
algorithm focuses on the real-time problem faced by visually
impaired people in crossing the traffic signals. Hence, a mobile
phone installed with the proposed application will act as a
reliable aid to such people. The proposed method is easily
adoptable as it addresses real-life challenges of visually
impaired people.
Though the proposed technique is partially developed using
Android SDK, it can be even implemented using other
Operating Systems as well. Also, the main focus in the
proposed algorithm is to capture the image containing digits
and to identify them. Hence, it can be used in various other
applications also. A video capturing the time in stop-clock can
be processed to get a series of timer count to tum a light on/off
at specific time, for automation of home-appliances and such
other scientific applications.
REFERENCES
[1] Accessible Pedestrian Signals - A Guide to Best Practices:
http://www.apsguide.org/index.cfin
[2] The Secret Button at Pedestrian Crossings - A series on small
but significant insights into disabled life:
http://www.bbc.comlnews/blogs-ouch-22706881
[3] MUTCD (Manual on Uniform Traffic Control Devices),
Developed by U S Department of Transportation in 2009:
http://mutcd.thwa.dot.gov/pdfs12009/mutcd2009edition.pdf
[4] Helping the Blind Cross the Road :
http://www.geek.comlscience/helping-the-blind-cross-the-road-
5569941
[5] D Rosenbaum, J Leitloff, F Kur z, 0 Meynberg and T Reize,
"Real - time Image Processing for Road Traffic Data
Extraction from Aerial Images ", Proc. of ISPRS TC VII
Symposium, IAPRS, vol. XXXVIII, Part 7B, pp. 469 - 474,
2010.
[6] P Gupta, G NPurohit, SPandey, "Traffic Load Computation for
Real Time Traffic Signal Control ", Int. Journal of Engineering
and Advanced Technology (HEAT), vol. 2, no. 4, pp. 891 - 895,
2013 .
[7] M Puthran, S Puthur, R Dharulkar, "Smart Traffic Signal ", Int.
Journal of Comp Sc and Info Tech, vol. 6, no. 2, pp. 1360 -
1363, 2015.
[8] A Salhi, B Minaout, M Fakir, "Robust Automatic Traffic Signs
Recognition using Fast Polygonal Approximation of Digital
Curves and Neural Networks ", Int. Journal of Adv Comp Sc and
Applications, Special Issue on Advances in Vehicular Ad hoc
Networking and Applications, 2014.
http://dx.doi.org/10.14569/Speciallssue.2014.
[9] A. Lorsakul and J. Suthakorn, "Traffic Sign Recognition for
Intelligent Vehicle/Driver Assistance System Using Neural
Network on OpenCV", Proc. of 4th International Conference on
Ubiquitous Robots and Ambient Intelligence (URAl 2007).
[10] V Lepetit, P Fua, "Keypoint Recognition using Randomized
Trees ", IEEE Trans on Pattern Analysis and Machine
Intelligence, vol. 28, no. 9,pp. 1465 - 1479.
[11] B Matthews, D Hibberd and 0 Cartsen, "Road and Street
Crossings for blind and partially sighted people: The
Importance of being certain ", University of Leeds.
http://www.guidedogs.org.uklmedial7187843/road-and-street-
crossings-the-importance-of-being-certain2-2.pdf
480