Automated Road Lane Detection for Intelligent Vehicles

&
DTMF Based control of Home Appliances

DE/TD Project Report

submitted in partial fulfillment of the requirements for
the award of the Degree of

Bachelor of Technology
of
DAYALBAGH EDUCATIONAL INSTITUTE
By
Harsh Shukla Rahul Sharma
Dheeraj Singh Chaudhary Somesh Agarwal
Mradul Singh Amit Kumar

Under the supervision of

Prof. D. Bhagwan Das
Reader

2001-2002
Faculty of Engineering
Dayalbagh Educational Institute
Dayalbagh, Agra 282005

1
 CERTIFICATE

This is to certify that the Design Engineering / Theme Development Project work reported in the
dissertation entitled " Automated Road Lane Detection for Intelligent Vehicles & DTMF
Based control of Home Appliances " submitted by Harsh Shukla, Dheeraj Singh Chaudhary,
Rahul Sharma, Mradul Singh, Amit Kumar and Somesh Agarwal in partial fulfillment of the
requirements for the award of the Degree of Bachelor of Technology to the Dayalbagh
Educational Institute, is a record of the bonafide work carried out under my supervision. Further,
that the matter in this report has not been submitted to any other University / Institute for the award
of any degree or diploma.

Dayalbagh ( Prof D. Bhagwan Das )

10.3.2018 Reader, Faculty of Engineering
Dayalbagh Educational Institute

2
 Acknowledgement

We have taken efforts in this project. However, it would not have been possible without the kind
support and help of many individuals and organizations. I would like to extend my sincere thanks
to all of them. I am highly indebted to Prof. D Bhagwan Das for their guidance and constant
supervision as well as for providing necessary information regarding the project & also for their
support in completing the project. We would like to express our gratitude towards my parents &
our university Dayalbagh Educational Institute for their kind co-operation and encouragement
which help me in completion of this project. I would like to express my special gratitude and thanks
to industry persons, GitHub, Stack Overflow and IEEE Explore for giving me such great insights.
Our thanks and appreciations also go to people who have willingly helped us out with their
abilities.

3
Contents

Design Engineering Methodology ................................................................................................................. 5

Projects Considered ...................................................................................................................................... 6
Project Chosen .............................................................................................................................................. 9
Lane Detection in Intelligent Vehicles ...................................................................................................... 9
Image Capturing .................................................................................................................................... 9
Conversion to Gray Scale .................................................................................................................... 10
Noise Reduction .................................................................................................................................. 11
Edge Detection .................................................................................................................................... 11
Line Detection ..................................................................................................................................... 12
Lane Boundary Scan ............................................................................................................................ 13
Hyperbola Fitting................................................................................................................................. 14
DTMF Based control of Home Appliances .............................................................................................. 15
Circuit Concept and Working .............................................................................................................. 15
Results ......................................................................................................................................................... 17
Further Improvements ................................................................................................................................ 18
Conclusion ................................................................................................................................................... 19
References: ................................................................................................................................................. 20

4
Design Engineering Methodology

With increase in the number of road accidents, it has led to concern over the nature of road
accidents. In most cases, human error is the cause of accidents. Thus, great efforts are required to
aid the driver in driving process to reduce the possibility of human error. The most significant
development for intelligent vehicles is advanced driver assistance system (ADAS). It involves a
camera-assisted system which takes real time images from the surroundings and displays relevant
information to the driver. Various safety provisions are provided to avoid accident and collision
by offering techniques that alert the driver of potential problems. The development of LDWS
(Lane Departure Warning System) technology that recognizes the lane to generate a warning signal
to the lane departure is in progress actively. We have used Hough Transforms to detect the lanes
for the development of LDWS.

Steer-by-Wire is such a technology that replaces the mechanical interface between the driver
steering wheel and the vehicle front wheels. Two actuators are used: the steering wheel actuator
generates a force-feedback on the steering wheel and the front wheels actuator moves the front
wheels. Expected benefits are improved safety (by the suppression of the mechanical link between
driver and wheels), better vehicle maneuverability and handling characteristics. Steer-by-Wire can
also be further exploited to enhance vehicle dynamic behavior. We simulated and tried to work on
PID- Controller for steering control using feedback from the encoder data on shaft.

DTMF is widely used in telecommunication systems. To transmit a particular character on a

telephone line, the two frequencies representing that character are generated as electrical signals,
added and sent down the line. A specified period of silence (40 milliseconds or more) separates
one transmitted DTMF character from another. A receiving device decodes the character being
transmitted by identifying the two frequencies present in each sum it receives. “DTMF Based
Home Automation System” we were able to control our home appliances wirelessly. Other
important feature of this project design was that we are did not use any microcontroller in it.

5
Projects Considered

We explored various lane detection and tracking methods and went into details of performance
analysis of different lane detection and tracking methods. This section also investigates the best
lane detection and tracking algorithms that can be selected for a specific road conditions:

Yim and Oh [1] developed a three feature based lane detection algorithm. The features used are
starting position, orientation and intensity value. In the initial step, a Sobel operator is applied to
get the edge information. The lane boundary is represented as a vector comprising of the three
features. The current lane vector is calculated based on the input image and the previous lane model
vector. Two windows, one for each, is used for left and right boundaries. Assuming N pixel in
each horizontal line, N lane vector candidates are generated. The best candidate is selected based
on the minimum distance from previous lane vector using a weighted distance metric. For
equalization each feature is assigned a different weight. Then a lane inference system is used to
predict the new lane vector. If the road width changes abruptly, the current vector calculated is
discarded and the previous one is taken as current vector.

A lane detection approach for urban environment is proposed by Sehestedt et al. [2]. Since the lane
markers are not clearly visible due to wear and tear, occlusions and due to complex road geometry,
a weak model is used for detecting lane markers. In the inverse perspective mapped image, particle
filter is applied from bottom row to top. The filter is tuned in such a way to track multiple lanes.

Aly [3] presents a real time and robust approach to detect lane markers in urban roads. It first
generates a top view of the road image using inverse perspective mapping for avoiding the
perspective effect. Then the top view is filtered using selective oriented two-dimensional Gaussian
kernel. The filter is tuned specifically for bright lines in dark background with specific width. So,
it has high response to the line markers and retain only the highest values by selecting q% quantile
value from filtered image and removing all values below the threshold. Then the straight lines are
detected using simplified Hough transform, which is followed by RANSAC line fitting which
provides initial guess to the RANSAC spline fitting step. Then a post processing step is done to

6
localize the spline and extend it in the image. The algorithm does not perform tracking. It can
detect any number of lane boundaries in the image not just the current lane.

Kim [4] developed a lane detection and tracking algorithm which can handle challenging scenarios
such as faded lane markers, lane curvatures and splitting lanes. In the initial step, a gradient
detector and an intensity bump detector is used to eliminate the non-lane markers. Artificial Neural
Networks (ANN) is applied on remaining samples for lane detection. The detected lane markers
pixels are grouped using cubic splines. Hypotheses are generated from random set of line
segments. RANSAC algorithm helps in validating the hypotheses. Particle filtering is used for lane
tracking.

Cheng et al. [5] introduced a hierarchical algorithm for lane detection. High dimensional feature
points are extracted based on feature color extraction. It is used to distinguish structured roads
from unstructured roads. Then connected components is applied on the feature points. A feature
vector is constructed for feature points exceeding a threshold. Eigen value Decomposition
Regularized Discriminant Analysis is done to reduce the variance of the model. Then maximum
likelihood Gaussian parameters are estimated. The extracted feature points are used as detected
lanes in structured roads. For unstructured roads, the entire scene is divided based on mean-shift

segmentation. Each region is considered to be homogeneous and lane markers are detected using

Bayes rule.

Initially, color image is converted to gray scale [6] and then shadow removal is performed.
Modified Canny edge detector is used for edge detection and then Hough transform is applied on
the resultant binary image. Horizon line lane boundary scan is implemented using the edge
detection result of Hough transform. The scan starts from the portion where the Hough lines meet
the bottom image border. The scan returns data points corresponding to the edges. Then a
hyperbola pair fitting is done on the collected data points.

7
Borkar et al. [7] proposed lane detection based on Hough transform and iterated matched filters.
RANSAC algorithm is used to avoid outliers due to noise and other artifacts in the road. Kalman
filter is used to track the lanes. The first step in the algorithm is to convert the color image to gray
scale and temporal blurring then on that image inverse perspective mapping (IPM) is applied. An
adaptive threshold is applied on the IPM image to generate a binary image. Each binary image is
split into two halves and each one contains one lane marker. A low resolution Hough transform is
applied on the binary images. A 1- dimensional matched filter is applied at each sample along the
line to find the approximate center of each line. After estimating the center, RANSAC algorithm
is applied to the data points for lane detection.

8
Project Chosen

Lane Detection in Intelligent Vehicles

From all the above approaches considered, keeping in the mind the computational and
infrastructural constraints we narrowed down on the approach given in [6].

A CCD camera is fixed on the front-view mirror to capture the road scene. To simplify the
problem, we assume that it is setup to make the baseline horizontal, which assures the horizon in
the image, is parallel to the X-axis. Otherwise, we can adjust the image using the calibration data
of the camera to make it. Each lane boundary marking, usually a rectangle (or approximate) forms
a pair of edge lines. It was assumed that the input to the algorithm was a 620x480 RGB color
image. Therefore, the first thing the algorithm does is to convert the image to a grayscale image in
order to minimize the processing time. Secondly, as presence of noise in the image will hinder the
correct edge detection. Therefore, we apply (F.H.D.) algorithm [8] to make the edge detection
more accurate. Then the edge detector is used to produce an edge image by using canny filter with
automatic thresholding to obtain the edges, it will reduce the amount of learning data required by
simplifying the image edges considerably. Then edged image sent to the line detector after
detecting the edges which will produces a right and left lane boundary segment. The projected
intersection of these two-line segments is determined and is referred to as the horizon. The lane
boundary scan uses the information in the edge image detected by the Hough transform to perform
the scan. The scan returns a series of points on the right and left side. Finally, pair of hyperbolas
is fitted to these data points to represent the lane boundaries. For visualization purposes the
hyperbolas are displayed on the original color image. The algorithm structure is shown in Figure1

Image Capturing

The input data is a color image sequence taken from a moving vehicle. A color camera is mounted
inside the vehicle at the front-view mirror along the central line. It takes the images of the
environment in front of the vehicle, including the road, vehicles on the road, roadside, and
sometimes incident objects on the road. The on-board computer with image capturing card will

9
capture the images in real time (up to 30 frames/second), and save them in the computer memory.
The lane detection system reads the image sequence from the memory and starts processing. In
order to obtain good estimates of lanes and improve the speed of the algorithm, the original image
size was reduced to 620x480 pixels by Gaussian
pyramid.

Conversion to Gray Scale

To retain the color information and segment the

road from the lane boundaries using the color
information this proved difficulties on edge
detection and also it will affect the processing
time. In practice the road surface can be made up
of many different colors due to shadows,
different pavement style or age, which causes the
color of the road surface and lane markings to
change from one image region to another.
Therefore, color image is converted into
grayscale. However, the processing of grayscale
images becomes minimal as compared to a color
image. This function transforms a 24-bit, three-
channel, color image to an 8- bit, single-channel
grayscale image by forming a weighted sum of
the Red component of the pixel value * 0.3
+Green component of the pixel value * 0.59 + Blue component for the pixel value *0.11 the
output is the gray scale value for the corresponding pixel [9].

10
Noise Reduction

Noise is a real-world problem for all systems and computer vision is no exception. The algorithms
developed must either be noise tolerant or the noise must be eliminated. As presence of noise in
our system will hinder the correct edge detection, so that noise removal is a prerequisite for
efficient edge detection with the help of (F.H.D.) algorithm [10] that removes strong shadows from
a single image. The basic idea is that a shadow has a distinguished boundary. Removing the
shadow boundary from the image derivatives and reconstructing the image should remove the. A
shadow edge image can be created by applying edge-detection on the invariant image and the
original image and selecting the edges that exist in the original image but not in the invariant image
and to reconstruct the shadow free image by removing the edges from the original image using a
pseudo-inverse filter.

Edge Detection

Lane boundaries are defined by sharp contrast between the road surface and painted lines or some
type of non-pavement surface. These sharp contrasts are edges in the image. Therefore, edge
detectors are very important in determining the location of lane boundaries. It also reduces the
amount of learning data required by simplifying the image considerably, if the outline of a road
can be extracted from the image. The edge detector implemented for this algorithm and the one
that produced the best edge images from all the edge detectors evaluated was the ‘canny’ edge
detector [11]. To find the maxima of the partial derivative of the image function I in the direction
orthogonal to the edge direction, and to smooth the signal along the edge direction. It was important
to have the edge detection algorithm that could be able to select thresholds automatically however,

11
the automatic threshold that is used in the default Canny produced far too much edge information.
Making a slight modification to the edge detected produced by canny has given more desirable
results. The only changes necessary were to set the amount of non-edge pixels to the best value
that gives more accurate edges in different conditions of image capturing environment. The canny
edge detector also has a very desirable characteristic in that it does not produce noise like the other
approaches.

Line Detection

The line detector used is a standard Hough transform [12] with a restricted search space. The
standard Hough transforms searches for lines using the equation shown in Figure 3:

Fig 3. Hough Transform

When in reality we can reject any line that falls outside a certain region [15]. For example a
horizontal line is probably not the lane boundary and can be rejected. The restricted Hough
transform was modified to limit the search space to 45° for each side. Also, the input image is split
in half yielding a right and left side of the image. Each the right and left sides are searched
separately returning the most dominant line in the half image that falls with in the 45° window.
The horizon is simply calculated using the left and right Hough lines and projecting them to their
intersection. The horizontal line at this intersection is referred to as the horizon.

12
Lane Boundary Scan

The lane boundary scan phase uses the edge image the Hough lines and the horizon line as input.
The edge image is what is scanned and the edges are the data points it collects. The scan begins
where the projected Hough lines intersect the image border at the bottom of the image. Once that
intersection is found, it is considered the starting point for the left or right search, depending upon
which intersection is at hand. From the starting point, the search begins a certain number of pixels
towards the center of the lane and then proceeds to look for the first edge pixel until reaching a
specified number of pixels after the maximum range. The range will be set to the location of the
previously located edge pixel plus a buffer number of pixels further.

Figure 4. Image coordinates

For the starting condition, the search will already be at the left- or right-most border, as shown in
Figure 5 so the maximum range will be the border itself. The extra buffer zone of the search will
help facilitate the following of outward curves of the lane boundaries. The lane points are
organized into two lists expressed

as L(l) and L(r)

13
 Figure.6 Lane Boundaries Detection.

Hyperbola Fitting

The hyperbola pair fitting phase uses the two vectors of data points from the lane scan as input. A
least squares technique is used to fit a hyperbola to the data. One hyperbola is fit to each of the
vectors of data points; however, they are solved in simultaneously due to the fact that they are a
pair model. The parameters of the two hyperbolas are related because they must converge to the
same point, due to the geometry of the roadway. The formula for expressing the lane boundary as
a hyperbola, given the road boundary point (u, v) in image plane:
𝑘
u = 𝑢−ℎ + 𝑏(𝑣 − ℎ) + 𝑐

u and v are the x- and y-coordinate in the image reference frame, h is the Y-coordinate of the
horizon in the image reference frame, and k, b and c are the parameters of the curve, which can be
calculated from the shape of the lane.

14
DTMF Based control of Home Appliances

We use many different types of communication in control applications to control home appliances,
industrial appliances, and other type of automation. There are two types of communication that is
we generally use - one is wired and other one is wireless. In wireless communication we transmit
signal wirelessly, like using radio frequency (RF) and in wired communication in which we use
wires like copper wire. In this project “DTMF Based Home Automation System” we are going
to control our home appliances wirelessly. Another important feature of this project is, that we are
not going to use any microcontroller in it.

Circuit Concept and Working

DTMF controlled home appliances project works over mobile DTMF technology that exists in
Dial tone. DTMF stands for Dual Tone Multiple Frequency. There are some frequencies that
we used to create DTMF tone. In simple words by adding or mixing two or more frequencies
generates DTMF tone. These frequencies are given below:

15
In Given figure we can see two groups of different frequencies. When one upper and one lower
frequencies mixed then a tone is created that tone we calls Dual Tone Multiple Frequency. In this
project we control ac appliances by pressing dial pad keys like 1, 2, 3, 4, 5 and more.

Here we have connected a cell phone using aux wire to the DTMF decoder circuit. Before
explaining the further working of project, we need to know about the output of DTMF decoder for
every key pressed.

Circuit diagram for DTMF controlled home automation project is shown above. In this circuit
we have used a DTMF decoder namely MT8890 IC which converts dial pad tone into the four bit
digital output. LIGHT, FAN and TV are connected to Q1,Q2 and Q3 of DTMF decoder IC through
a relay driver namely ULN2003. 5 volt SPDT 3 relays are used for controlling LIGHT, FAN and
TV.

16
Results

Our Lane Detection pipeline can be viewed in form of block diagram as given below.

Input Video:

Canny Edge Detection:

17
Resulting image with detected lanes:

Further Improvements

Lane Detection can also be done by ROI(Region of Interest) Segmentation and other Deep
learning techniques. Similarly DTMF can also be extended to controlling home appliances and
even much more complex systems like mobile robots etc.

18
Conclusion

In this DETD project we implemented a real time lane detection algorithm based on video
sequences taken from a vehicle driving on highway was proposed. As mentioned above the system
uses a series of images. Out of these series some of the different frames used are shown in the lane
detection algorithm, (F.H.D.) algorithm conduct image segmentation and remove the shadow of
the road. Since the lanes are normally long and smooth curves, we consider them as straight lines
within a reasonable range for vehicle safety. The lanes were detected using Hough transformation
with restricted search area. The proposed lane detection algorithm can be applied in both painted
and unpainted road, as well as slightly curved and straight road. There remained some problems
in the lane detection due to shadowing and the Hough line and horizon overlaid, then in the lower
left edge with the lane boundary points overlaid and in few cases lane scan fails to track the correct
lane.

19
References:
[1] Y.U. Yim and S.- Y. Oh, "Three-feature based automatic lane detection algorithm (TFALDA)
for autonomous driving," IEEE Trans. Intell. Transp. Syst. , vol. 4, no. 4, pp. 219-225, Dec. 2003.

[2] S.Sehestedt, S. Kodagoda, A. Alempijevic, and G. Dissanayake, "Robust lane detection in

urban environments," in Proc. IEEE Intell. Robots Syst., Oct. 2007, pp. 123-128.

[3] M.Aly, "Real time detection of lane markers in urban streets," in Proc. IEEE Intelligent.
Vehicles. Symposium., Jun.4–6, 2008, pp. 7-12.

[4] Z.Kim, "Robust lane detection and tracking in challenging scenarios," IEEE Trans. Intelligent.
Transp. System, vol. 9, no. 1, pp. 16-26, Mar. 2008.

[5] H.Y. Cheng, C.C. Yu, C. C. Tseng, K. C. Fan,J. N. Hwang, and B. S. Jeng,"Hierarchical lane
detection for different types of roads," In Acoustics, Speech and Signal Processing (ICASSP),
2008 IEEE International Conference on pp. 1349-1352. IEEE, 2008.

[6] A.Assidiq, O.Khalifa, R. Islam, and S. Khan, "Real time lane detection for autonomous
vehicles," in Computer and Communication Engineering, 2008. ICCCE 2008. International
Conference on. IEEE,2008, pp. 82-88.

[7] A.Borkar, M. Hayes, and M.T.Smith, "Robust lane detection and tracking with RANSAC and
Kalman Filter," Proceedings of the IEEE International Conference on Image Processing, pp. 3261-
3264, 2009.

[8] D. Graham, S. D. Finlayson1, D. Hordley1, and D. S, Mark, “Removing Shadows from

Images”, School of Information Systems, 2001, pp. 1-14.

20
[9] Gernot, Hoffmann, “Luminance Models for The Grayscale Conversion”,2001, pp. 1-11.

[10] Kristijan Macek, Brian Williams, Sascha Kolski,” A lane Detection vision module for driver
assistance”, EPFL, 1015Lausanne, 2002, pp. 16.

[12] B. M. John, N. Donald, “Application of the Hough Transform to Lane detection and Following
on High Speed Roads”, signal & system Group, Department of Computer Science, National
University of Ireland, 1999,pp. 1-9

21