HAND GESTURE CONTROLLED ROBOT

The main functions / parts which makes our robot are as follows,
Image Processing
GUI Software
Wireless Video Transmission
Wireless Data Transmission
Robot Control Unit
Image Processing
It is the main part of our project. Through image processing the hand gestures of humans are interfaced with computer for
controlling the robot. The software used for supporting image processing are OpenCV and QT software. The OpenCV is
not a software but contains all the libraries needed for developing image processing software which is developed by Intel.
We are using QT (a platform for developing GUI application software) for the OpenCV libraries to work.
The video that is captured by the webcam is used for filter out the hands and its gestures. For finding the hand gesture
firstly the background is subtracted and converted to binary image then figure out the hand in white and background with
black. Then we find the contours or the largest contour. Then through the largest contour points we draw convex hull.
After that the convexity defects are pointed out and thus we can count the no. of fingers. The above activities are carried
out by library functions. Then the center of the palm is modified with a circle having the largest enclosing area. Its center
point is connected to the mouse cursor. So moving hand will move the mouse and in turn rotate the steering of the robot.
. And the second hand will be used for controlling the camera rotation, forward / backward movement, light controls,
sensor activation.
GUI Software
Along with the interfacing of hand gestures the QT software also is used to develop a GUI which accepts and displays the
video transmitted from the robot. It contains a steering for turning the robot. When rotating the steering the
corresponding degree is send to the microcontroller 8051(AT89C51) interfaced with QT. The other controls are brake,
accelerator, camera 3D rotation, lights, sensors enabling, sensor values displaying etc. sends its assigned address to 8051
and then to robot. The software is developed with C++ and qml code.
Wireless Video Transmission
The wireless video transmission or shortly the TV transmitter is the path way for moving the robot. A vga camera fixed in
the robot sends the low frequency signal to the amplifier circuit using transistor and finally converts into VHF signal for
transmission wirelessly. A TV tuner card connected to the system receives the signal and sends to the GUI software and
display the real time video.
Wireless Data Transmission
Two RF module of frequencies 315MHz and 433MHz are using for data transmission from and to system. The 433MHz RF
transmitter is interfaced with 8051 at PC side with HT640 8bit encoder. All the control signals from the GUI software like
steering turning degrees, brake, accelerator, camera 3D rotation, lights, sensors enabling etc. are received through a USB
cable which is then converted to serial data using FT232(usb to serial converter) and fed to 8051 and there it is converted
to corresponding 8bit binary and then to 433MHz transmitter. Also the 315MHz receiver will receive the sensor data in
8bit binary and decodes by HT648 decoder and sends to GUI through same 8051. Similarly another 8051 in robot side will
receives and transmit data and acts accordingly.
Robot Control Unit
In robot section the camera rotation in up, down, clockwise, anticlockwise and the tire movement are driven by gear
motors of low rpm and are controlled by microcontroller with the help of L293D driver IC or XL08 thyristor. The sensors
like ultrasonic sensor for obstacle and distance measurement, temperature sensor, light sensor, gas sensor, metal detector
etc. are placed in the robot