A

Colloquium Report
On
Sixth Sense Technology

Submitted towards partial fulfillment of the requirement for the award of

Degree of

Bachelor of Technology
In

Computer Science and Engineering

Submitted to Submitted by
Mr N.Lal Ojasvi Bhardwaj
Ms Priyanka Dahiya CSE-E 3rd year

College of Engineering and Technology

Mody University, Lakshmangarh
August, 2016
 ABSTRACT

Sixth Sense is a wearable gestural interface device developed by Mr. Pranav Mistry, a PhD
student in the Fluid Interfaces Group at the MIT Media Lab. It is similar toTelepointer. The Sixth
Sense prototype comprises a pocket projector, a mirror and a camera contained in a pendant like,
wearable device. Both the projector and the camera are connected to a mobile computing device
in the users pocket. The projector projects visual information enabling surfaces, walls and
physical objects around us to be used as interfaces; while the camera recognizes and tracks user's
hand gestures and physical objects using computervision based techniques. The software
program processes the video stream data captured by the camera and tracks the locations of the
colored markers (visual tracking fiducially) at the tip of the users fingers. The movements and
arrangements of these fiducially are interpreted into gestures that act as interaction instructions
for the projected application interfaces. Sixth Sense supports multi-touch and multi-user
interaction.
 Acknowledgement

I would like to express profound gratitude to my mentor Senior Lecturer Mrs. Priyanka
Dahiya for her valuable support, encouragement, supervision, and her useful suggestions
through colloquium work. Her moral support and continuous guidance enabled me to complete
my work successfully.

I wish to express my appreciation to Assistant Professor Mr. Niranjan Lal who helped me to
overcome my dough in doing this colloquium. His constant support and suggestion made my
work easy and proficient.

Ojasvi Bhardwaj

140235

( Computer Science and Engineering )

 CONTENTS

Chapter 1: Introduction ......1

Chapter 2: Motivation.........2
Chapter 3: History3
3.1: Why choose SIXTH SENSE TECHNOLOGY? ......3
Chapter 4: Components ......4
4.1Camera..................................4
4.2 Projector.4
4.3 Mirror5
4.4 Mobile Component5
4.5 Color Maker..5
Chapter 5: Working .........................6
Chapter 6: Application6
6.1 Map Application6
6.2 Taking Pictures..7
6.3 Drawing Application.7
6.4 Making Calls.8
6.5 Interacting with physical objects...8

Chapter 7: Advantage..9
Chapter 8: Disadvantages..10
Future Scope..11
Conclusion ..............................................................................................................................12
References .............................................................................................................................. 13
Appendix
Plagiarism..................................................................................................................................
 Chapter-01

Introduction

We humans, interact with the world using our five senses. But as the name suggests this
technology brings forward an additional sixth sense. A sixth sense is an ESP(Extra Sensory
Perception) that aims at a more developed future with both the physical and digital world
connected without the help of hardware devices. We come a step nearer to this goal of the sixth
sense by the introduction of sixth sense technology.

The sixth sense technology employs a sixth sense which is in reality a wearable pendent like
device. This device is actually a mini projector coupled with a camera and a cell phone, designed
for obeying hand gestures. Using simple hand gestures, touch screens can be obtained from any
surface for various applications.

The origin of sixth sense technology can be traced back to Steve Mann who implemented a
wearable computer in the form of neck-projector coupled with a camera system in 1990.In later
years, following his footsteps, Pranav Mistry ,a young research scientist at MIT came up with
new applications of this technology. Thus, the sixth sense technology was developed at Media
labs in MIT and was coined as Wear Ur World (WUW).

The device has various applications such as the drawing application in which movement of the
index finger is recognized and the user can draw on any surface. The other applications include
mapping, reading newspaper, checking time by drawing a wrist watch etc. Other than being
portable, this device also serves as a computer and saves time spent on searching information.
 Chapter-02

Motivation
When we were thinking of projects for the summers to execute, we thought of making a hand,
which would do exactly what our hand is which could have been something brilliant in robotics.
Then we came across a TED talk by Pranav Mistry where he was talking about how we can
connect the physical world with the digital world using Sixth Sense. It looked simply amazing

how he was able to just show his fingers in front of his camera (phone) and the camera would
click pictures. This just enthralled us and was just something we had to make considering its
sheer brilliance.
 Chapter-03

History

Steve Mann is father of sixth sense who made a wearable computer in 1990. The Sixth Sense
Technology was first implemented as the neck worn projector + camera system. He was a media
lab student at that time.

There after it was used and implemented by an Indian who is the man has become very famous
in the recent Pranav Mistry. There will be a long future rather than the short period of history for
the Sixth Sense technology.

WHY CHOOSE SIXTH SENSE TECHNOLOGY?

This sixth sense technology provides us with the freedom of interacting with the digital world
using hand gestures. This technology has a wide application in the field of artificial intelligence.
This methodology can aid in synthesis of bots that will be able to interact with humans.

This technology enables people to interact in the digital world as if they are interacting in the real
world. The Sixth Sense prototype implements several applications that demonstrate the
usefulness, viability and flexibility of the system.
 Chapter-04
Component
The hardware components are coupled in a pendant like mobile wearable device.

3.1. Camera

3.2. Projector

3.3. Mirror

3.4. Mobile Component

3.5. Color Markers

3.1 CAMERA

If there is any object in view, then the object is recognized by the webcam and using computer-
vision based techniques the users gestures are tracked and it sends the data to the Smartphone.

3.2 PROJECTOR

The projector projects the visual information on any surface, including the object itself or your
hand. The project itself contains a battery inside, with 3 hours of battery life. The projector
showcases the information enabling surfaces, walls and physical objects around us to be used as
interfaces
3.3 MIRROR

The downward-facing projector projects the output image on to the mirror And the mirror
reflects this image on to the desired surface.

3.4 MOBILE COMPONENT

The mobile devices like the Smartphone are used to handle connection to the internet. Using
mobile internet data and voice can be communicated across the world either by transmitting or
by receiving it. Similarly video data can also be processed by a web enabled Smartphone.

3.5 COLOR MAKER

The software program processes the video stream data captured by the camera and tracks the
locations of the colored markers at the tip of the users fingers. The movements and
arrangements of these markers are then interpreted into gestures.
 Chapter-05
Working

The hardware that makes Sixth Sense work is a pendant like mobile wearable interface.
It has a camera, a mirror and a projector and is connected wirelessly to a Bluetooth smart
phone that can slip comfortably into ones pocket.
The camera recognizes individuals, images, pictures, gestures one makes with their
hands.
Information is sent to the Smartphone for processing.
The downward-facing projector projects the output image on to the mirror.
Mirror reflects image on to the desired surface.
Thus, digital information is freed from its confines and placed in the physical world.
 Chapter-06
Application

The sixth sense technology finds a lot of application in the modern world. The sixth sense
devices bridge the gap by bringing the digital world into the real world and in that process
allowing the users to interact with the information without the help of any machine interfaces.
Prototypes of the sixth sense device have demonstrated viability, usefulness and flexibility of this
new technology. According to the words of its developers extend the of use of this new device is
only limited by the imagination of human beings. Some practical applications of the sixth sense
technology is given below

5.1 Map Application

With the help of a map application the user can call upon any map of his/her choice and navigate
through them by projecting the map on to any surface .By using the thumb and index fingers
movements the user can zoom in, zoom out or pan the selected map.
5.2 Taking Picture

Another application of sixth sense devices is the implementation of a gestural camera. This
camera takes the photo of the location user is looking at by detecting the framing gesture. After
taking the desired number of photos we can project them onto any surfaces and then use gestures
to sort through those photos and organize and resize them.

5.3 Drawing Application

The drawing application allows the user you to draw on any surface by tracking the fingertip
movements of the users index finger. The pictures that are drawn by the user can be stored and
replaced on any other surface. The user can also shuffle through various pictures and drawing by
using the hand gesture movements
5.4 Making Call

We can make calls with help of sixth sense device .The sixth sense device is used to project the
keypad into your palm and the using that virtual keypad we can make calls to anyone.

5.5 Interacting with physical objects

The SixthSense system also helps to interact with physical objects we use in a better way. It
augments physical objects by projecting more information about these objects projected on them.
For example, a gesture of drawing a circle on the users wrist projects an analog watch on the
users hand. Similarly a newspaper can show live video news or dynamic information can be
provided on a regular piece of paper

Newspaper showing live feeds watch an analog projected into the wrist
 Chapter-07

Advantages

Portable:

One of the main advantages of the sixth sense devices is its small size and portability. It can be
easily carried around without any difficulty. The prototype of the sixth sense is designed in such
a way that it gives more importance to the portability factor. All the devices are light in weight
and the smart phone can easily fit into the users pocket.

2. Support multi touch and multi user interaction:

Multi touch and multi user interaction is another added feature of the sixth sense devices. Multi
sensing technique allows the user to interact with system with more than one than one finger at a
time.. Sixth sense devices also in-cooperates multi user functionality. This is typically useful for
large interaction scenarios such as interactive table tops and walls.

3. Cost effective:

The cost incurred for the construction of the sixth sense proto type is quiet low. It was made
from parts collected together from common devices. And a typical sixth sense device cost up to
$300. The sixth sense devices have not been made in large scale for commercial purpose. Once
that happens its almost certain that the device will cost much lower than the current price.

4. Connectedness between real world and digital world:

Forming a connection between the real world and the digital world was the main aim of the sixth
sense technology.
 Chapter-08
Disadvantages

We have to give correct instruction.

The projectors runs on batteries for power where regularly have to place.

.
 Chapter-09
Future Scope

As technology changes, many devices hit the market and leave soon if they are not efficient. The
security reasons are to be considered while using sixth sense technology. Authorised access to an
application plays a major role. For instance, while using this sixth sense technology in libraries,
there should be a separate method for faculty and students to access them as it may lead to hoax.
Sixth sense technology is very different from other devices and is a new topic for hackers and
other people. We got rid of coloured markers in recent years. However, we should try to make it
more secure and avoid radio waves which are harmful and could cause brain cancer to children.
Camera and projector can be incorporated into a mobile device than using them separately.

However, once the sixth sense technology comes into existence we can see a new world around
us where laptops, cell phones, cameras and other electronic gadgets have less priority.
 Conclusion
Sixth Sense technology is the science of tomorrow with the aim of connecting the digital world
with the physical world seamlessly, eliminating hardware devices. This combination of devices
and software together create a reality in which the digital world is merged with the physical
world. This technology can be used in a plethora of positive ways, especially by enhancing daily
tasks. It facilitates easily learn information about products while shopping, find instructions
while building or constructing and being able to easily identify areas while travelling are all
extremely positive results of Sixth Sense technology. It is essentially a wearable computer that
can surf the web, make phone calls and even connect to other computing devices. It is more
portable and more interactive than any Smartphone, laptop or tablet available today. It does,
however, pose a threat to society in the form of privacy issues. It allows for the user to easily
record and photograph anything without being notice and it can also allow for the user to gain
available online information about anything just by glancing at it and performing an online
search. This will surely bring up many social, legal, ethical and security concerns. Sixth Sense
technology is remarkable in its intended purpose for daily task efficiency and interactive
computing.
 Reference

[1] S. Sadhana Rao, Sixth Sense Technology, Proceedings of the International Conference on
Communication and Computational Intelligence, India.27 29 December,2010

[2] Bathani Raksha K, Sixth Sense Technology OR WUW (Wear Ur World), Research Expo
International Multidisciplinary Research Journal Volume - II , Issue - II June 2012

[3] Meenakshi Gupta, Shruti Sharma, Virtual Class room using six sense Technology, IOSR
Journal of Computer Engineering (IOSRJCE) Volume 6, Issue 4 (Sep. -Oct. 2012)

[4] Monika Arora, Basic Principles of Sixth Sense Technology, VSRD-IJCSIT, Vol. 2 (8), 2012

[5] Sneha Allu, Advantages and Disadvantages about the technology, blog

[6] Pallavi Sonone, Sixth sense technology, Presentation2012.

[7] Lenin Ravindranath, Venkata N. Padmanabhan ,Piyush Agrawal, Sixth Sense : RFID-based
Enterprise Intelligence, Research paper.

[8] Content about Gesture recognition in wikipedia:

http://en.wikipedia.org/wiki/Gesture_recognition

[9] Working of augmented technology and its working:

http://www.howstuffworks.com/augmented-reality.htm

[10] Information about Gesture recognition given in an article-

http://www.engineersgarage.com/articles/gesturerecognition-technology

[11] Image-http://vision.mas.ecp.fr/Personnel/teboul/vision.php

[12] Library management using RFID technology- http://www.libbest.com/rfid.html

[13] Video- http://www.ted.com/talks/pranav_mistry_the_thrilling_pot

ential_of_sixthsense_technology.html

[14] Article on physical education in future- http://www.pelinks4u.org/articles/pasco4_11.htm

[15] Mohamed Alsheakhali, Ahmed Skaik, Mohammed Aldahdouh, Mahmoud Alhelou,

International Conference on Information , Communication and Signal processing(ICICS), 2011
Plagiarism Report