An Assistive Body Sensor Network Glove for Speech- and Hearing- Impaired Disabilities

Abstract:
This paper presents a hand-gesture based interface for facilitating communication among speech and hearing impaired disabilities. In the system, a wireless sensor glove equipped with flex sensor and a 3D accelerometer or G-sensor is used as the input device. By integrating the speech synthesizer onto an automatic gesture recognition system, users hand gestures can be translated into sounds. In this study, we proposed a hierarchical gesture recognition framework based on the combined use of multivariate Gaussian distribution and a set of rules for model and feature set selection, deriving from a detailed analysis of misclassified gestures in the confusion matrix. To illustrate the practical use of the framework, a gesture recognition experiment has been conducted on American Sign Language (ASL) fingerspelling gestures with two additional gestures representing space and a full stop. This proposed project will be helpful for the speech and hearing impaired disability people. The sensors sensed values will be sent to microcontroller where in pre assigned values of the sensors will be set in the program which will be passed to the voice controlled unit and from there the voice commands can be accessed by the impaired person so that the communication can.

Introduction
Communication is an essential element of human life. It is a process that helps generating mutual understanding and enabling us to live in harmony. For speech and hearing impaired disabilities, non-verbal form of communication is very important. This type of communication exists in various forms, ranging from facial expression, lip motion, sign language, to hand-writing. In particular, sign language, such as body or

hand gestures, is one of the most natural means of communication among deaf and mute people. Since they cannot perceive or generate the acoustic

information,exchanging information is therefore usually made visually.

In general, most hearing people do not have knowledge about specific sign languages. Everyday communication with hearing population, therefore, poses a major challenge to Those with disabilities. To alleviate the communication problem among speech-impaired disabilities, an assistive technology that will provide a more convenient and less time consuming means of communication is required.

In this study, an assistive device for speech and hearing impaired disabilities has been developed based on the Body Sensor Network (BSN) technology. In the proposed system, real-time recognition of American Sign Language (ASL) fingerspelling gestures is performed based on input signals acquired from a wireless sensor gloves. The recognized gestures will then be mapped into corresponding sounds using speech synthesizer. The main focus of this paper is on the hierarchical framework for hand gesture recognition.

Working principle:
The capacitive micro machined three-axis accelerometer produced by Free-scale is based on the principle mentioned above. The accelerometer features signal conditioning, a 1-pole low pass filter, temperature compensation and g-Select which allows for the selection among 4 sensitivities -- 1.5g 2g 4g 6g for user to choose. Zero-g offset full scale span and filter cut-off are factory set and require no external devices. The output will change according to the direction and sensitivities defined before. Therefore, the variation of acceleration of each axis can reflects the contrail and direction of the gesture.

The A/D converter can receive the analog signal sent by the accelerometer. Accelerometer has 4 sensitivities selection --1.5g 2g 4g 6g. From the experiment, the acceleration of shaking hand is between 2G and 5G in general, so the suitable

definition is around 0.3G. Therefore the sensitivity 6G is appropriate in case of more forceful shake. Meanwhile, the A/D converter meets the requirement, of which the reference voltage is 2.56V and the minimum definition is 2.5mV. In this manner the tilt or sensitivity of the sensor can be used to control or generate the signal.

Flex sensor is a component that changes resistance when bent. As a nominal resistance of 10,000 ohms. As the flex sensor is bent in both the direction the resistance gradually decreases. Both the sensors are interfaced to microcontroller and different threshold values are pre assigned and is sent to voice control unit where the sound commands are heard by the impaired person. Based on predefined gestures we can communicate sound signals through voice control unit and can be made to hear to the impaired person, multiple flex sensors can be provided for still more gestures but the complexity of the circuit increases hence we are providing only for few gestures.

Hardware requirement:
      Microcontroller- AT89S52 Accelerometer or G-sensor Flex sensor Power supply RS232 and MAX232 Interfacing circuit

Software used:
 Embedded C  Keil IDE  .Net/SQL

Advantages:
     Output sensitivity is stable. It provides an alternate solution for dumb and deaf people to interact. Excellent linearity over their dynamic range Will provide scope to enhance the features of various devices. Will provide scope to new R&D projects.

Disadvantages:
 Complexity of the circuit increases as the number of sensors increase.  Initial cost is high.

Applications:
 They can be used in biological sciences.  They can be used extensively for speech and hearing impaired disability persons.

Block diagram

Transformer

Rectifier

Filter

regulator