BLUE EYES

HUMAN-OPERATOR MONITORING SYSTEM

NAME OF THE PARTICIPANT 1: P.Manas Krishna BRANCH: ECE NAME OF THE INSTITUTION: KESHAV MEMORIAL INSTITUTE OF TECHNOLOGY CONTACT NUMBER:7893973959 EMAIL ID : manas.krishna18@gmail.com

ABSTRACT Blue Eyes system provides technical means for monitoring and recording the operators basic physiological parameters. The most important parameter is saccadic activity1, which enables the system to monitor the status of the operators visual attention along with head

acceleration, which accompanies large displacement of the visual axis (saccades larger than 15 degrees). Complex industrial environment can create a danger of exposing the operator to toxic substances, which can affect his cardiac, circulatory and pulmonary systems. Thus, on the grounds of plethysmographic signal taken from the forehead skin surface, the system computes heart beat rate and blood oxygenation.

The Blue Eyes system checks above parameters against abnormal (e.g. a low level of blood oxygenation or a high pulse rate) or undesirable (e.g. a longer period of lowered visual attention) values and triggers user-defined alarms when necessary. Quite often in an emergency situation operator speak to themselves expressing their surprise or stating verbally the problem. Therefore, the operators voice, physiological parameters and an overall view of the operating room are recorded. This helps to reconstruct the course of operators work and provides data for long-term analysis.

Blue Eyes consists of a mobile measuring device and a central analytical system. The mobile device is integrated with Bluetooth module providing wireless interface between sensors worn by the operator and the central unit. ID cards assigned to each of the operators and adequate user profiles on the central unit side provide necessary data personalization so different people can use a single mobile device (called hereafter DAU Data Acquisition Unit). The

overall system diagram is shown in Figure 1. The tasks of the mobile Data Acquisition Unit are to maintain Bluetooth connections, to get information from the sensor and sending it over the wireless connection

INTRODUCTION Blue Eyes system provides technical means for monitoring and recording the operators basic physiological parameters. The most important parameter is saccadic activity , which enables the system to monitor the status of the operators visual attention along with head acceleration, which accompanies large displacement of the visual axis (saccades larger than

15degrees). Complex industrial environment can create a danger of exposing the operator to toxic substances, which can affect his cardiac, circulatory and pulmonary systems. Thus, on the grounds of plethysmographic signal taken from the forehead skin surface, the system computes heart beat rate and blood oxygenation. The Blue Eyes system checks above parameters against abnormal (e.g. a low level of blood oxygenation or a high pulse rate) or undesirable (e.g. a longer period of lowered visual attention) values and triggers user-d e f i n e d a l a r m s w h e n n e c e s s a r y . Q u i t e o f t e n i n a n e m e r g e n c y s i t u a t i o n operator speak to themselves expressing their surprise or stating verbally the problem. Therefore, the operators voice, physiological parameters and an overall view of the operating room are recorded. This helps to reconstruct the course of operators work and provides data for long-term analysis. Blue Eyes consists of a mobile measuring device and a central analytical system. The mobile device is integrated with Bluetooth module providing wireless interface between sensors worn by the operator and the c e n t r a l u n i t . I D c a r d s a s s i g n e d to each of the operators and adequate user profiles on the central unit side provide necessary data p e r s o n a l i z a t i o n s o different people can use a single mobile device (called hereafter DAU Data Acquisition Unit). The overall system diagram is shown in Figure 1. The tasks of the mobile Data Acquisition Unit are to maintain Bluetooth connections, to get information from the sensor and sending it over the wireless connection, Saccades are rapid eye jumps to new locations within a visual environment assigned predominantly by the conscious attention process. to deliver the alarm messages sent from the Central S y s t e m U n i t t o t h e operator and handle personalized ID cards. Central System Unit maintains the other side of the Bluetooth connection, buffers incoming sensor data, performs o n - l i n e d a t a analysis, records the conclusions for further exploration a n d provides visualization interface. T h e t a s k o f t h e m o b i l e D a t a A c q u i s i t i o n U n i t a r e t o m a i n t a i n Bluetooth connection, to get information from the sensor and sending it over the wireless connection ,to deliver the alarm messages sent from the Central

System Unit to the operator and handle personalized ID c a r d s . C e n t r a l S y s t e m Unit maintains the other side of the Bluetooth connection, buffers incoming sensor data, performs online data analysis, records the conclusion for further exploration and provides visualization interface.

During the on-line browsing it is possible to watch a list of currently working operators and the status of their mobile devices. Selecting one of the operators enables the supervisor to check the operators current physiological condition (e.g. a pie chart showing active brain involvement)and a history of alarms regarding the operator. All new incoming alerts are displayed immediately so that the supervisor is able to react fast. However, the p r e s e n c e o f t h e h u m a n s u p e r v i s o r i s n o t n e c e s s a r y s i n c e t h e s y s t e m i s equipped with reasoning algorithms and can trigger user-defined actions (e.g.to inform the operators co-workers).D u r i n g o f f - l i n e b r o w s i n g i t i s p o s s i b l e t o r e c o n s t r u c t t h e course of the operators duty with all the physiological parameters, audio and video data. A comprehensive data analysis can be performed enabling the s u p e r v i s o r t o d r a w c o n c l u s i o n s o n o p e r a t o r s o v e r a l l p e r f o r m a n c e a n d competency (e.g. for working night shifts).System administrator is a user that maintains the system. The administrator delivers tools for adding new operators to the database, defining alarm conditions, configuring logging tools and creating new analyzer modules.

While registering new operators the administrator enters appropriate d a t a ( a n d a p h o t o i f a v a i l a b l e ) t o t h e s y s t e m d a t a b a s e a n d p r o g r a m s h i s personal ID card.

Defining alarm conditions 1) The function enables setting up user -defined alarm conditions by writing condition -action rules (e.g. low saccadic activity during a longer period of time

2) Inform operators co-workers, wake him up using the beeper or playing appropriate sound and log the event in the database).D e s i g n i n g n e w a n a l y z e r m o d u l e s - b a s e d o n e a r l i e r recorded data the administrator can create new analyzer m o d u l e t h a t c a n r e c o g n i z e other behaviors than those which are built-in the system. The new modules are created using decision tree induction algorithm. The administrator names the n e w behavior to be recognized and poi nts the data associated w i t h i t . T h e results received from the new modules can be used in alarm conditions. M o n i t o r i n g s e t u p e n a b l e s t h e a d m i n i s t r a t o r to choose the parameters to monitor as well as the a l g o r i t h m s o f t h e d e s i r e d a c c u r a c y t o compute parameter values. Logger setup provides tools for selecting the parameters to be recorded. For audio data sampling frequency can be chosen. As regards the video signal, a delay between storing consecutive frames can be set (e.g. one picture in every two seconds).Database maintenance 3) Here the administrator can remove old o r uninteresting data from the database. The uninteresting data is suggested by the built-in reasoning system.

DATA ACQUISITION UNIT (DAU) This section deals with the hardware part of t he Blue Eyes system with regard to the physiological data sensor, the DAU hardware components and the microcontroller software.

PHYSIOLOGICAL DATA SENSOR To provide the Data Acquisition Unit with necessary physiological data an off-shelf eye movement sensor Jazz Multi sensor is used. It supplies raw digital data regarding eye position, the level of blood oxygenation, acceleration along horizontal and vertical axes and ambient light intensity. Eye movement is measured using direct infrared oculographic transducers. (The eye

movement is s a m p l e d a t 1 k H z , t h e o t h e r p a r a m e t e r s a t 2 5 0 H z . T h e s e n s o r s e n d s approximately 5.2 kB of data per second.)

HARDWARE SPECIFICATION

Microcontrollers (e.g. Atmel 8952 microcontroller)can be used as the core of the Data Acquisition Unit since it is a well-established industrial standard and provides necessary functionalities(i.e. high speed serial port)at allow price. T h e B l u e t o o t h m o d u l e s u p p o r t s s y n c h r o n o u s v o i c e d a t a transmission .The

codec reduces the microcontrollers tasks and lessens the amount of data being sent over the UART. The Bluetooth module performs voice data compression, which results in smaller bandwidth utilization and better sound quality.. 10

PERFORMANCE REQUIREMENTS The portable nature of the mobile unit results in a number of performance requirements. As the device is intended to run on batteries, low p o w e r c o n s u m p t i o n i s t h e m o s t i m p o r t a n t constraint. Moreover, it is necessary to assure proper

timing while receiving and transmitting sensor signals. To make the operation comfortable the device should be light weight and

electrically safe. Finally the use of standard and inexpensive ICs will keep the price of the device at relatively low level. The priority of the central unit is to provide real-time buffering a n d incoming sensor signals and semi -real-time processing of the data w h i c h r e q u i r e s s p e e d Bluetooth Connection Manager is responsible for establishing a n d maintaining connections using all available B l u e t o o t h d e v i c e s . I t periodically inquires new devices in an operating range and checks whether t h e y a r e r e g i s t e r e d in the system database. Only with those devices the Connection Manager will communicate. After establishing a connection an a u t h e n t i c a t i o n procedure occurs. The a u t h e n t i c a t i o n p r o c e s s i s p e r f o r m e d using system PIN code fetched from the database. Once the connection has b e e n authenticated the mobile unit sends a da ta frame c o n t a i n i n g t h e operators identifier. Finally, the Connection Manager adds a SCO link (voice connection) and runs a new dedicated Operator Manager, which will manage the new operators session. Additionally, the Connection Manager maps the o p e r a t o r s identifiers into the Bluetooth connections, so that w h e n t h e operators roam around the covered area a c o n n e c t i o n w i t h a n a p p r o p r i a t e Bluetooth device is established and the data stream is redirected accordingly. The data of each supervised operator is buffered separately in t h e d e d i c a t e d Operator Manager . A t t h e s t a r t u p i t c o m m u n i c a t e s w i t h t h e Operator Data Manager in order to get more detailed personal data. The most important Operator Managers task is to buffer the incoming raw data and to split it into separate data streams related to each of the measured parameters. The raw data is sent to a Logger Module, the split data streams are available f o r t h e other system modules through producerc o n s u m e r q u e u e s . Furthermore, the Operator Manager

provides an interface for sending alert messages to the related operator. Operator Data Manager provides an interface to the operator database enabling the other modules to read or write personal data and system access information.

DESIGN METHODOLOGIES In creating the Blue Eyes system a waterfall software development model was used since it is suitable for u n r e p e a t a b l e a n d explorative projects. During the course of the development UML standard notations were used. They facilitate communication between team members all the ideas are clearly expressed by means of various diagrams, which is a sound base for further development. The results of the functional design phase were documented on use case diagrams. During the lowlevel design stage the whole systems was d i v i d e d i n t o f i v e m a i n modules. Each of them has an independent, well -defined functional interface providing precise description of the services offered to the other modules. All the interfaces a r e d o c u m e n t e d o n U M L class, interaction and state diagrams.

CONCLUSION The Blue Eyes system is developed because of the need for a real-time monitoring system for a human operator. The approach is innovative since it helps supervise the operator not the process, as i t i s i n p r e s e n t l y available solutions. We hope the system in its commercial release will help a v o i d p o t e n t i a l t h r e a t s resulting from human errors, such as weariness, oversight, tiredness or temporal indisposition. However, t h e p r o t o t y p e developed is a good estimation of the possibilities of the final product. The use of a miniature CMOS camera integrated

into the eye movement sensor will enable the system to calculate the point of gaze and observe what the operator is actually looking at. Introducing voice recognition algorithm will facilitate the communication between the operator and the central system and simplify authorization process. Despite considering in the report only the operators working in control rooms, our solution may well be applied to everyday life situations. Assuming the operator is a driver and the supervised process is car driving it is possible to build a simpler embedded on-line system, which will only monitor conscious brain involvement and warn when necessary. As in this case the logging module is redundant, and the Bluetooth technology is becoming more and more popular, the commercial implementation of such a system would be relatively inexpensive. T h e f i n a l t h i n g is to explain the name of our system. Blue Eyes emphasizes the foundations of the project Bluetooth technology a n d t h e m o v e m e n t s o f t h e e y e s B l u e t o o t h p r o v i d e s r e l i a b l e w i r e l e s s communication whereas the eye movements enable us to obtain a lot o f interesting and important information.

References: www.krazytech.com www.Wikipedia.com

www.ijcaonline.org http://techfind.blogspot.in