1.

Introduction
Imagine a world where humans interact with computers. You are sitting in front of your personal computer that can listen, talk, or even scream aloud. It has the ability to gather information about you and interact with you through special techniques like facial recognition, speech recognition, etc. It can even understand your emotions at the touch of the mouse. It verifies your identity, feels your presents, and starts interacting with you. You ask the computer to dial to your friend at his office. It realizes the urgency of the situation through the mouse, dials your friend at his office, and establishes a connection. Human cognition depends primarily on the ability to perceive, interpret, and integrate audiovisuals and sensor information. Adding extraordinary perceptual abilities to computers would enable computers to work together with human beings as intimate partners. Researchers are attempting to add more capabilities to computers that will allow them to interact like humans, recognize human presents, talk, listen, or even guess their feelings. How can we make Computers "see" and "feel"? Blue Eyes uses sensing technology to identify a user's actions and to extract key information. This information is then analyzed to determine the users physical, emotional, or informational state, which in turn can be used to help make the user more productive by performing expected actions or by providing expected information. For example, in future a Blue Eyes-enabled television could become active when the user makes eye contact, at which point the user could then tell the television to "turn on".

2. Blue Eyes System Design

Blue eyes system consists of a mobile measuring device and a central analytical system. The mobile device is integrated with Bluetooth module providing wireless interface between sensors worn by the operator and the central unit. The overall system diagram is shown as below

Figure 1: Overall System Diagram The mobile measuring device is called as Data Acquisition Unit (DAU) and the central analytical system is called Central System Unit (CSU).

3. Features of Blue Eyes System

1. Visual attention monitoring 2. Physiological condition monitoring 3. Operators position detection 4. Wireless data acquisition using Blue tooth technology 5. Real-time user-defined alarm triggering 6. Physiological data, operator's voice and overall view of the control room recording 7. Recorded data playback. Blue eyes will help avoid potential threats resulting from human errors, such as weariness, oversight, tiredness or temporal indisposition. BLUE EYES emphasizes the Bluetooth technology and the movement of the eyes. Bluetooth provides reliable wireless communication whereas the eye movement enables us to obtain a lot of interesting and important information.

4. Functional Design
The functional design describes the functions the system offers to particular users. There are three groups of users: operators, supervisors, system administrators. Operators Operator is a person whose physiological parameters are supervised. The only functions offered to that user are authorization in the system and receiving alarm alerts.

Supervisors Supervisor is a person responsible for analyzing operators condition and performance. The supervisor receives tools for inspecting the present values of the parameters as well as browsing the results of long term analysis. System Administrators System administrator is a user that maintains the system. The administrator delivers tools for adding new operators to the database, defining alarm conditions, configuring logging tools and creating new analyzer modules. The administrator can also do database maintenance-here the administrator can remove old or uninteresting data from the data base

5. Data Acquisition Unit

The task of the Data Acquisition Unit is to maintain Bluetooth connections, to get information from the sensor and sending it over the wireless communication and to deliver the alarm messages sent from central system unit to the operator. The components of DAU are:

Figure 2: DAU-Components Physiological Data Sensor To provide the Data Acquisition Unit with necessary physiological data we purchase an offshelf eye movement sensor-Jazz Multisensor. It supplies raw digital data regarding eye position, the level of blood oxygenation acceleration along horizontal and vertical axes and ambient light intensity. Eye movement can be measured using direct infrared oculographic transducers.

Figure 3: Jazz Multisensor Microcontroller Specifications We choose Atmel 8952 microcontroller to be the core of the Data Acquisition Unit since it is a well-established industrial standard and provides necessary functionality (i.e. high speed serial port) at a low price. Blue Tooth Device Blue tooth module supports synchronous voice data transmission. Additionally, the blue tooth module performs voice data compression, which results in smaller bandwidth utilization and better sound quality. Communication between the blue tooth module and the microcontroller is carried on using standard UART interface MAX232 level shifter. The speed of the UART is set to 115200bps in order to assure that the entire sensor data is delivered in time to the central system unit.

6. Central System Unit

Central System Unit maintains the other side of the Bluetooth connection, buffers incoming sensor data, per forms on-line data analysis, records the conclusions for further exploration and provides visualization interface. It consists of four CSU modules: Connection manager, Data Analysis, Data Logger and Visualization. The modules exchange data using specially designed Single-producer-multi-consumer buffered thread-safe queues. Any number of consumer modules can register to receive the data supplied by a producer. Every single consumer can register at any number of producers, receiving therefore different types of data. Connection Manger

Connection Managers main task is to perform low-level Blue tooth communication using Host Controller Interface Commands. It is designed to co-operate with all available Blue tooth devices. Additionally, Connection Manager authorizes operators, manages their sessions, demultiplexes and buffers raw physiological data. The components of connection manager are: y Transport Layer Manager It hides the details regarding actual Blue Tooth physical transport interface (which can be either RS232 or UART or USB standard) and provides uniform HCI command interface. y Bluetooth Connection Manager It is responsible for establishing and maintaining connections using all available Blue tooth devices. y Operator Data Manager It provides an interface to the operator database enabling the other modules to read or write personal data and system access information. y Operator Manager It communicates with the operator data manager in order to get more detailed personal data. The most important task is to buffer the raw incoming data and to split it into separate data streams related to each of the measured parameters. Furthermore, the operator manager provides an interface for sending alert messages to the related operator.

Figure 4: Components of Connection Manager Data Analysis Module This module performs the analysis of the raw sensor data in order to obtain information about the operators physiological condition. The separately running Data Analysis Module supervises each of the working operators. The module consists of a number of smaller 5

analyzers extracting different types of information. Each of the analyzers registers at the appropriate Operator Manager or another analyzer as a data consumer and, acting as a producer, provides the results of the analysis. Data Logger Module This module provides support for storing the monitored data in order to enable the supervisor to reconstruct and analyze the course of the operators duty. Apart from the raw or processed physiological data, alerts and operators voice are stored. The raw data is supplied by the related Operator Manager module, whereas the Data Analysis module delivers the processed data. The voice data is delivered by a Voice Data Acquisition module. The module registers as an operators voice and optionally processes the sound to be stored .The Loggers task is to add appropriate time stamps to enable the system to reconstruct the voice. Additionally, there is a dedicated video data logger, which records the data supplied by the Video Data Acquisition module. Visualization Module This module provides user interface for the supervisors. It enables them to watch each of the working operators physiological condition along with a preview of selected video source and his related sound stream. All the incoming alarm messages are instantly signaled to the supervisor. Watching all the recorded physiological parameters, alarms, video and audio data the supervisor is able to reconstruct the course of the selected operators duty.

Figure 5: CSU-Intermodule Communication 6

7. Tools Developed During the Design

7.1.Blue Dentist This tool provides support for controlling the currently connected Blue tooth device. Its functions are local device management and connection management.

Figure 6: Blue Dentist

7.2.Blue Capture To test the possibilities and performance of the remaining parts of the Project Kit (computer, camera and database software) we created BLUE CAPTURE. This tool supports capturing video data from various sources (USB web-cam, industrial camera) and storing the data in the database. Additionally, the application performs sound recording. After filtering and removing insignificant fragments (i.e. silence) the audio data is stored in the database. Finally, the program plays the recorded audiovisual stream. During the implementation of the DAU we needed a piece of software to establish and test Blue Tooth communications. We therefore created a tool called Blue capture

Figure7: Blue Capture

8. Applications of Blue Eyes System

1. Blue Eyes technology is able to track a person's pupils. Therefore, if a driver closes his or her eyes for too long, the Tech Mobile sounds an alarm. Developed at IBM's Almaden Research Center in San Jose, Calif., the Blue Eyes technology leverages multimodal ways of interacting with computers or in the future a car or ordinary household devices. Blue Eyes leverages sensing technology to identify and observe a user's actions, and to extract key information, such as where the user is looking or what the user is communicating through both words and gestures. 2. We could see its use in video games where, it could give individual challenges to customers playing videogames. The integration of childrens toys, technologies and computers is enabling new play experiences that were not commercially feasible until recently. The Intel Play QX3 Computer Microscope, the Me2Cam with Fun Fair, and the Computer Sound Morpher are commercially available smart toy products developed by the Intel Smart Toy Lab. One theme that is common across these PCconnected toys is that users interact with them using a combination of visual, audible and tactile input & output modalities. The presentation will provide an overview of 8

the interaction design of these products and pose some unique challenges faced by designers and engineers of such experiences targeted at novice computer users, namely young children. 3. Current interfaces between computers and humans can present information vividly, but have no sense of whether that information is ever viewed or understood. In contrast, new real-time computer vision techniques for perceiving people allows us to create "Face-responsive Displays" and "Perceptive Environments", which can sense and respond to users that are viewing them. Using stereo-vision techniques, we are able to detect, track, and identify users robustly and in real time. This information can make spoken language interface more robust, by selecting the acoustic information from a visually-localized source. Environments can become aware of how many people are present, what activity is occurring, and therefore what display or messaging modalities are most appropriate to use in the current situation. The results of our research will allow the interface between computers and human users to become more natural and intuitive.

9. Benefits of Blue Eyes System

Prevention from dangerous incidents Minimization of Ecological consequences Financial loss A threat to a human life The reconstruction of the course of operators work.

10.Future Enhancements
It is still possible to improve this technology. The use of a miniature CMOS camera integrated into the eye movement sensor will enable the system to calculate the point of gaze and observe what the operator is actually looking at. Introducing voice recognition algorithm will facilitate the communication between the operator and the central system and simplify authorization process.

11.Summary
We have decided to develop the BLUE EYES system because of the need for a real time monitoring system for a human operator. The approach is innovative since it helps supervise the operator not the process, as it is in presently available solutions. We hope the system in its 9

commercial release will help avoid potential threats resulting from human errors, such as weariness, oversight, tiredness or temporal indisposition. However the prototype we have developed is a good estimation of the possibilities of the final product. The final thing is to explain the name of our system. BLUE EYES emphasizes the foundations of the technology-Bluetooth technology and the movement of the eyes. Bluetooth provides reliable wireless communication whereas the eye movement enables us to obtain a lot of interesting and important information.

12.Conclusion
In future it is possible to create a computer which can interact with us as we interact each other with the use of blue eye technology. It seems to be a fiction, but it will be the life lead by BLUE EYES in the very near future. Ordinary household devices -- such as televisions, refrigerators, and ovens -- may be able to do their jobs when we look at them and speak to them.

13.References
[1] Blue Eyes Technology, Computer Edge, Oct.2002, pages 23-27. [2] Blue Eyes Technology and Applications, Business Solutions, Nov 2001, pages 95-99. [3] Bluetooth specification, version 1.0B, Bluetooth SIG, 1999. [4] AT89C52 8-bit Microcontroller Datasheet, Atmel [5] Horowitz P., Hill W., The art of Electronics, Cambridge University Press, 1989. [6] ROK 101 007 Bluetooth Module, Ericsson Microelectronics, 2000. [7] MC145483 3V 13-bit Linear PCM Codec-Filter, Motorola Semiconductor, 1997. [8] AT89C52 8-bit Microcontroller Datasheet, Atmel. [9] ST14C16 memory card IC Datasheet, ST Microelectronics, 1999. [10] Lei Necker R. C., Archer T., Visual C++ Bible, IDG Books, 1999.

10