Augmented Reality

Augmented Reality
CHAPTER 1
INTRODUCTION
Video games have been entertaining us for nearly 30 years, ever since Pong was
introduced to arcades in the early 1970s. Computer graphics have become much more
sophisticated since then, and game graphics are pushing the barriers of photorealism.
Now, researchers and engineers are pulling graphics out of your television screen or
computer display and integrating them into real-world environments. This new
technology, called augmented reality, blurs the line between what's real and what's
computer-generated by enhancing what we see, hear, feel and smell.
On the spectrum between virtual reality, which creates immersive, computergenerated environments, and the real world, augmented reality is closer to the real world.
Augmented reality adds graphics, sounds, haptic feedback and smell to the natural world
as it exists. Both video games and cell phones are driving the development of augmented
reality.
Augmented reality is changing the way we view the world - or at least the way its
users see the world. Picture yourself walking or driving down the street. With augmentedreality displays, which will eventually look much like a normal pair of glasses,
informative graphics will appear in your field of view, and audio will coincide with
whatever you see.
Figure 1.1 The Sixth Sense augmented reality system lets you project a phone pad onto your hand
and phone a friend -- without removing the phone from your pocket.
Department of ECE, BGSIT
Page 1
Augmented Reality
These enhancements will be refreshed continually to reflect the movements of your head.
Similar devices and applications already exist, particularly on smartphones like the
iPhone. Picture shows The SixthSense augmented reality system lets you project a phone
pad onto your hand and phone a friend -- without removing the phone from your pocket.
1.1 What is Mobile Augmented Reality for?
As computers increase in power and decrease in size, new mobile, wearable, and
pervasive computing applications are rapidly becoming feasible, providing people access
to online resources always and everywhere. This new flexibility makes possible new kind
of applications that exploit the person's surrounding context. Augmented reality (AR)
presents a particularly powerful user interface (UI) to context-aware computing
environments. AR systems integrate virtual information into a person's physical
environment so that he or she will perceive that information as existing in their
surroundings. Mobile augmented reality systems (MARS) provide this service without
constraining the individuals whereabouts to a specially equipped area. Ideally, they work
virtually anywhere, adding a palpable layer of information to any environment whenever
desired. By doing so, they hold the potential to revolutionize the way in which
information is presented to people.
Computer-presented material is directly integrated with the real world surrounding

the freely roaming person, who can interact with it to display related information, to pose
and resolve queries, and to collaborate with other people. The world becomes the user
interface
Page 2
Augmented Reality
CHAPTER 2
MOBILE AUGMENTED REALITY
Augmented reality (AR) is a live, direct or indirect, view of a physical, real-world

environment whose elements are augmented by computer-generated sensory input such as
sound, video, graphics or GPS data. It is related to a more general concept called
mediated reality, in which a view of reality is modified (possibly even diminished rather
than augmented) by a computer. As a result, the technology functions by enhancing ones
current perception of reality. By contrast, virtual reality replaces the real world with a
simulated one.
Augmentation is conventionally in real-time and in semantic context with
environmental elements, such as sports scores on TV during a match. With the help of
advanced AR technology (e.g. adding computer vision and object recognition) the
information about the surrounding real world of the user becomes interactive and digitally
manipulable. Artificial information about the environment and its objects can be overlaid
on the real world. The term augmented reality is believed to have been coined in 1990 by
Thomas Caudell, working at Boeing.
Research explores the application of computer-generated imagery in live-video
streams as a way to enhance the perception of the real world. AR technology includes
head-mounted displays and virtual retinal displays for visualization purposes, and
construction of controlled environments containing sensors and actuators.
Augmented Reality is considered as an extension of Virtual Reality. Virtual Reality
(VR) is a virtual space where the player immerses themselves into that exceed the bounds
of physical reality. In the VR, time, physical laws and material properties no longer hold
in contrast to real-world environment. Instead of considering AR and VR as exact
opposite concepts.
AR is about augmenting the real world environment with virtual information
by improving peoples senses and skills. AR mixes virtual characters with the actual
world. There are three common characteristics of AR scenes: the combination of real
world environment with computer characters, interactive scenes, and scenes in 3D.
Page 3
Augmented Reality
2.1 Augmented Reality in Cell Phones
Figure 2.1 iPhone applications using Augmented Reality
While it may be some time before you buy a device like SixthSense, more
primitive versions of augmented reality are already here on some cell phones, particularly
in applications for the iPhone and phones with the Android operating system. In the
Netherlands, cell phone owners can download an application called Layar that uses the
phone's camera and GPS capabilities to gather information about the surrounding area.
Layar then shows information about restaurants or other sites in the area, overlaying this
information on the phone's screen. You can even point the phone at a building, and Layar
will tell you if any companies in that building are hiring, or it might be able to find photos
of the building on Flickr or to locate its history on Wikipedia.
Layar isn't the only application of its type. In August 2009, some iPhone users were
surprised to find an augmented-reality "easter egg" hidden within the Yelp application.
Yelp is known for its user reviews of restaurants and other businesses, but its hidden
augmented-reality component, called Monocle, takes things one step further. Just start up
the Yelp app, shake your iPhone 3GS three times and Monocle activates. Using your
phone's GPS and compass, Monocle will display information about local restaurants,
including ratings and reviews, on your cell phone screen. You can touch one of the
listings to find out more about a particular restaurant.
Page 4
Augmented Reality
There are other augmented reality apps out there for the iPhone and other similar
phones -- and many more in development. Urbanspoon has much of the same
functionality as Yelp's Monocle. Then there's Wikitude, which finds information from
Wikipedia about sites in the area. Underlying most of these applications are a phone's
GPS and compass; by knowing where you are, these applications can make sure to offer
information relevant to you. We're still not quite at the stage of full-on image recognition,
but trust us, people are working on it. We've looked at some of the existing forms of
augmented reality. On the next page, we'll examine some of the other applications of the
technology, such as in video games and military hardware.
2.2 Augmented Reality in Video Games and the Military

Video game companies are quickly hopping aboard the augmented-reality
locomotive. A company called Total Immersion makes software that applies augmented
reality to baseball cards. Simply go online, download the Total Immersion software and
then hold up your baseball card to a webcam. The software recognizes the card (and the
player on it) and then displays related video on your computer screen. Move the card in
your hands -- make sure to keep it in view of the camera - and the 3-D figure on your
screen will perform actions, such as throwing a ball at a target.
Total Immersion's efforts are just the beginning. In the next couple of years, we'll
see games that take augmented reality out into the streets. Consider a scavenger-hunt
game that uses virtual objects. You could use your phone to "place" tokens around town,
and participants would then use their phones (or augmented-reality enabled goggles) to
find these invisible objects.
Demos of many games of this order already exist. There's a "human Pac-Man"
game that allows users to chase after each other in real life while wearing goggles that
make them look like characters in Pac-Man.
Arcane Technologies, a Canadian company, has sold augmented-reality devices to
the U.S. military. The company produces a head-mounted display -- the sort of device that
was supposed to bring us virtual reality -- that superimposes information on your world.
Consider a squad of soldiers in Afghanistan, performing reconnaissance on an opposition
hideout. An AR-enabled head-mounted display could overlay blueprints or a view from a
satellite or overheard drone directly onto the soldiers' field of vision.
Page 5
Augmented Reality
2.3 Augmenting Our World

The basic idea of augmented reality is to superimpose graphics, audio and other
sensory enhancements over a real-world environment in real time. Sounds pretty simple.
Besides, haven't television networks been doing that with graphics for decades? However,
augmented reality is more advanced than any technology you've seen in television
broadcasts, although some new TV effects come close, such as RACEf/x and the superimposed first down line on televised U.S. football games, both created by Sportvision.
But these systems display graphics for only one point of view. Next-generation
augmented-reality systems will display graphics for each viewer's perspective.
Some of the most exciting augmented-reality work is taking place in research labs
at universities around the world. In February 2009, at the TED conference, Pattie Maes
and Pranav Mistry presented their augmented-reality system, which they developed as
part of MIT Media Lab's Fluid Interfaces Group. They call it SixthSense, and it relies on
some basic components that are found in many augmented reality systems:
Camera
Small projector
Smartphone
Mirror
These components are strung together in a lanyard like apparatus that the user
wears around his neck. The user also wears four colored caps on the fingers, and these
caps are used to manipulate the images that the projector emits. SixthSense is remarkable
because it uses these simple, off-the-shelf components that cost around $350. It is also
notable because the projector essentially turns any surface into an interactive screen.
Essentially, the device works by using the camera and mirror to examine the surrounding
world, feeding that image to the phone (which processes the image, gathers GPS
coordinates and pulls data from the Internet), and then projecting information from the
projector onto the surface in front of the user, whether it's a wrist, a wall, or even a
person. Because the user is wearing the camera on his chest, SixthSense will augment
whatever he looks at; for example, if he picks up a can of soup in a grocery store,
SixthSense can find and project onto the soup information about its ingredients, price,
nutritional value -- even customer reviews.
Page 6
Augmented Reality
CHAPTER 3
WORKING OF AUGMENTED REALITY AND ITS

TECHNOLOGY INTRODUCED INTO MOBILE
On the spectrum between virtual reality, which creates immersive, computergenerated environments, and the real world, augmented reality is closer to the real world.
Augmented reality adds graphics, sounds, haptic feedback and smell to the natural world
as it exists. Both video games and cell phones are driving the development of augmented
reality. Everyone from tourists, to soldiers, to someone looking for the closest subway
stop can now benefit from the ability to place computer-generated graphics in their field
of vision.
Augmented reality is changing the way we view the world -- or at least the way its
users see the world. Picture yourself walking or driving down the street. With augmentedreality displays, which will eventually look much like a normal pair of glasses,
informative graphics will appear in your field of view, and audio will coincide with
whatever you see. These enhancements will be refreshed continually to reflect the
movements of your head.
3.1 Hardware
The main hardware components for augmented reality are: processor, display,
sensors and input devices. These elements, specifically CPU, display, camera and MEMS
sensors such as accelerometer, GPS, solid state compass are often present in modern
smartphones, which make them prospective AR platforms.
3.1.1 Computer
The computer analyzes the sensed visual and other data to synthesize and position
augmentations. Camera based systems require powerful CPU and considerable amount of
RAM for processing camera images. Wearable computing systems employ a laptop in a
Page 7
Augmented Reality
backpack configuration. For stationary systems a traditional workstation with a powerful
graphics card. Sound processing hardware could be included in augmented reality
systems.
3.1.2 Tracking System

Modern mobile augmented reality systems use one or more of the following
tracking technologies: digital cameras and/or other optical sensors, accelerometers, GPS,
gyroscopes, solid state compasses, RFID and wireless sensors. These technologies offer
varying levels of accuracy and precision. Most important is the position and orientation of
the user's head. Tracking the user's hand(s) or a handheld input device can provide a
6DOF interaction technique.
3.1.3 Display
There are three major display techniques for Augmented Reality: head-mounted
displays, handheld displays and spatial displays. Some examples of spatial augmented
reality displays include shader lamps, mobile projectors, virtual tables, and smart
projectors, described by O. Bimber and R. Raskar in 2005. Shader lamps, developed by
Raskar et al. in 1999, mimic and augment reality by projecting imagery onto neutral
objects, providing the opportunity to enhance the objects appearance. This can be
accomplished with materials of a simple unit- a projector, camera, and sensor. Handheld
projectors further this goal by enabling cluster configurations of environment sensing,
reducing the need for additional environmental sensing.
Other tangible applications include table and wall projections. One such innovation,
the Extended Virtual Table, separates the virtual from the real by including beam-splitter
mirrors attached to the ceiling at an adjustable angle. Virtual showcases, which employ
beam-splitter mirrors together with multiple graphics displays, provide an interactive
means of simultaneously engaging with the virtual and the real. Altogether, augmented
reality display technology can be applied to improve design and visualization, or function
as scientific simulations, and tools for education or entertainment.
3.1.4 Input Devices

Some systems such as the tinsmith system, employ pinch glove techniques.
Another common techniques is a wand with a button on it. In case of smartphone, phone
Page 8
Augmented Reality
itself could be used as 3D pointing device, with 3D position of the phone restored from
the camera images.
3.2 Software and Algorithms

A key measure of AR systems is how realistically they integrate augmentations
with the real world. The software must derive real world coordinates, independent from
the camera, from camera images. That process is called image registration and is part of
Azuma's definition of Augmented Reality.
Image registration uses different methods of computer vision, mostly related to
video tracking. Many computer vision methods of augmented reality are inherited from
visual odometry. Usually those methods consist of two parts. First detect interest points,
or fiduciary markers, or optical flow in the camera images. First stage can use feature
detection methods like corner detection, blob detection, edge detection or thresholding
and/or other image processing methods.
The second stage restores a real world coordinate system from the data obtained in
the first stage. Some methods assume objects with known geometry (or fiduciary
markers) present in the scene. In some of those cases the scene 3D structure should be
precalculated beforehand. If part of the scene is unknown simultaneous localization and
mapping (SLAM) can map relative positions. If no information about scene geometry is
available, structure from motion methods like bundle adjustment are used. Mathematical
methods used in the second stage include projective (epipolar) geometry, geometric
algebra, and rotation representation with exponential map, kalman and particle filters,
nonlinear optimization, and robust statistics.
Page 9
Augmented Reality
CHAPTER 4
TRACKING AND ORIENTATION
The biggest challenge facing developers of augmented reality is the need to know where
the user is located in reference to his or her surroundings. There's also the additional
problem of tracking the movement of users' eyes and heads. A tracking system has to
recognize these movements and project the graphics related to the real world environment
the user is seeing at any given moment. Currently, both video see through and optical seethrough displays typically have lag in the overlaid material due to the tracking
technologies currently available.
4.1 Indoor Tracking

Tracking is easier in small spaces than in large spaces. Trackers typically have two
parts: one worn by the tracked person or object and the other built into the surrounding
environment, usually within the same room. In optical trackers, the targets--LEDs or
reflectors, for instance--can be attached to the tracked person or object, and an array of
optical sensors can be embedded in the room's ceiling. Alternatively, the tracked users can
wear the sensors, and the targets can be fixed to the ceiling. By calculating the distance to
each visible target, the sensors can determine the user's position and orientation.
Researchers at the University of North Carolina-Chapel Hill have developed a
very precise system that works within 500 square feet. The HiBall Tracking System is an
optoelectronic tracking system made of two parts:
Figure 4.1 LEDs and Reflectors used for tracking in Indoor
Page 10
Augmented Reality
External view of the columbian printer maintainence application. Note that all the
components must be tracked. External view of the columbian printer maintainence
application. Note that all the components must be tracked. The system uses the known
location of the LEDs, the known geometry of the usermounted optical sensors and a
special algorithm to compute and report the user's position and orientation. The system
resolves linear motion of less than .2 millimeters, and angular motions less than .03
degrees. It has an update rate of more than 1500 Hz, and latency is kept at about one
millisecond.
In everyday life, people rely on several senses--including what they see, cues from
their inner ears and gravity's pull on their bodies--to maintain their bearings. In a
similarfashion, "hybrid trackers" draw on several sources of sensory information. For
example, thewearer of an AR display can be equipped with inertial sensors (gyroscopes
and accelerometers) to record changes in head orientation. Combining this information
with data from the optical, video or ultrasonic devices greatly improves the accuracy of
the tracking.
4.2 Outdoor Tracking

Head orientation is determined with a commercially available hybrid tracker that
combines gyroscopes and accelerometers with a magnetometer that measures the earth's
magnetic field. For position tracking, we take advantage of a high-precision version of the
increasingly popular Global Positioning System receiver.
A GPS receiver determines its position by monitoring radio signals from
navigation satellites. GPS receivers have an accuracy of about 10 to 30 meters. An
augmented-realitysystem would be worthless if the graphics projected were of something
10 to 30 meters away from what you were actually looking at. Users can get better results
with a technique known as differential GPS. In this method, the mobile GPS receiver also
monitors signals from another GPS receiver and a radio transmitter at a fixed location on
the earth. This transmitter broadcasts corrections based on the difference between the
stationary GPS antenna's known and computed positions. By using these signals to
correct the satellite signals, differential GPS can reduce the margin of error to less than
one meter. Our system is able to achieve centimeter-level accuracy by employing real-
Page 11
Augmented Reality
time kinematic GPS, a more sophisticated form of differential GPS that also compares the
phases of the signals at the fixed and mobile receivers.
Unfortunately, GPS is not the ultimate answer to position tracking. The satellite
signals are relatively weak and easily blocked by buildings or even foliage. This rules out
useful tracking indoors or in places like midtown Manhattan, where rows of tall buildings
block most of the sky. GPS tracking works well in wide open spaces and relatively low
buildings. GPS provides far too few updates per second and is too inaccurate to support
the precise overlaying of graphics on nearby objects.
Augmented-reality systems place extraordinarily high demands on the accuracy,
resolution, repeatability and speed of tracking technologies. Hardware and software
delays introduce a lag between the user's movement and the update of the display. As a
result, virtual objects will not remain in their proper positions as the user moves about or
turns his or her head. One technique for combating such errors is to equip AR systems
with software that makes short-term predictions about the user's future motions by
extrapolating from previous movements. And in the long run, hybrid trackers that include
computer vision technologies may be able to trigger appropriate graphics overlays when
the devices recognize certain objects in the user's view.
Page 12
Augmented Reality
CHAPTER 5
CLASSIFICATION OF MOBILE AR
5.1 Properties of MOBILE AR User Interface

Mobile AR presents a way for people to interact with computers that is radically
different from the static desktop or mobile office. One of the key characteristics of MARS
is that both virtual and physical objects are part of the UI, and the dynamic context of the
user in the environment can influence what kind of information the computer needs to
present next. This raises several issues:
Control: Unlike a stand-alone desktop UI, where the only way the user can
interact with the presented environment is through a set of well defined
techniques,the MARS UI needs to take into account the unpredictability of the
real world. Forexample, a UI technique might rely on a certain object being in the
user's field of view and not occluded by other information. Neither of the
properties can be guaranteed: the user is free to look away, and other information
could easily get in the way, triggered by the user's own movement or an
unforeseen event (such as another user entering the field of view). Thus, to be
effective, the UI technique either has to relax the non-occlusion requirement, or
has to somehow guarantee on-occlusion in spite of possible contingencies.
Consistency: People have internalized many of the laws of the physical world.
When using a computer, a person can learn the logic of a new UI. As long as these
two worlds are decoupled (as they are in the desktop setting), inconsistencies
between them are often understandable. In the case of MARS, however, we need
to be very careful to design UIs in which the physical and virtual world are
consistent with each other. Need for embedded semantic information: In MARS,
virtual material is overlaid on top of the real world. Thus we need to establish
concrete semantic relationships between virtual and physical objects to
characterize UI behavior. In fact, since many virtual objects are designed to
Page 13
Augmented Reality
annotate the real world, these virtual objects need to store information about the
physical objects to which they refer (or at least have to know how to access that
information). Display space: In terms of the available display space and its best
use, MARS UIs have to deal with a much more complicated task compared to
traditional 2D UIs. Instead of one area of focus (e.g., one desktop display), we
have to deal with a potentially unlimited display space surrounding the user, only
a portion of which is visible at any time. The representation of that portion of
augmented space depends on the user's position, head orientation, personal
preferences (e.g., filter settings) and ongoing interactions with the augmented
world, among other things. Management of virtual information in this space is
made even more difficult by constraints that other pieces of information may
impose. Certain virtual or physical objects may, for example, need to be visible
under all circumstances, and thus place restrictions on the display space that other
elements are allowed to obstruct. The display management problem is further
complicated by the possibility of taking into account multiple displays. MARS, as
a nonexclusive UI to the augmented world, may seamlessly make use of other
kinds of displays, ranging from wall-sized, to desk-top, to hand-held. If such
display devices are available and Telegeoinformatics: Location-Based Computing
and Services 33 accessible to the MARS, questions arise as to which display to
use for what kind of information and how to let the user know about that decision.
Scene dynamics: In a head-tracked UI, the scene will be much more dynamic than
in a stationary UI. In MARS, this is especially true, since in addition to all the
dynamics due to head motion, the system has to consider moving objects in the
real world that might interact visually or audibly with the UI presented on the
headworn display. Also, we have to contend with a potentially large variability in
tracking accuracy over time. Because of these unpredictable dynamics, the spatial
composition of the UI needs to be flexible and the arrangement of UI elements
may need to be changed. On the other hand, traditional UI design wisdom
suggests minimizing dynamic changes in the UI composition (Shneiderman,
1998). One possible solution to this dilemma lies in the careful application of
automated UI management techniques.
Page 14
Augmented Reality
5.2 Classification of Augmented Reality

Four major classes of AR can be distinguished by their display type: Optical
sensor through, Virtual retinal system, Video see-through, Monitor based AR and
Projector Based AR.
The following sections show the corresponding devices and present their main
features.
5.2.1 Optical See through Head Mounted Display

Optical see-through AR uses a transparent head mounted display to show the virtual
environment directly over real world (Figures 5.1 & 5.2). It works by placing optical
combiners in front of the users eye. These combiners are partially transmissive, so that the
user can look directly through them to see the real world. The combiners are also partially
reflective, so that the user sees virtual images bounced off the combiners from headmounted monitors.
Figure 5.1 Optical See Through HMD
Page 15
Augmented Reality
Fig 5.2 Optical See Through-Scheme
A simple approach to optical see-through display employs a mirror beam splitter, a

half-silvered mirror that both reflects and transmits light. If properly oriented in front of
the user's eye, the beam splitter can reflect the image of a computer display into the user's
line of sight yet still allow light from the surrounding world to pass through. Such beam
splitters, which are called combiners, have long been used in "head-up" displays for
fighter-jet pilots (and, more recently, for drivers of luxury cars).
Lenses can be placed between the beam splitter and the computer display to focus
the image so that it appears at comfortable viewing distance. If a display and optics are
provided for each eye, the view can be in stereo. Sony makes a see-through display that
some researchers use, called the Glasstron.
Prime examples of an optical see-through AR system are the various augmented medical
systems. The MIT image guided surgery has concentrated on brain surgery. UNC has
been working with an AR enhanced ultrasound system and other ways to superimpose
radiographic images on a patient. There are many other optical see-through systems, as it
seems to be main direction of AR.
Recent optical see through HMDs are being built for well known companies like SONY
and OLYMPUS and have support for occlusion, varying accommodations. There are very
small prototypes that can be attached to conventional eyeglasses.
Page 16
Augmented Reality
Figure 5.3 Eyeglass display with Virtual reality
5.2.2 Virtual Retinal Display

The VRD (VIRTUAL RETINAL DISPLAY) was invented at the university of
Washington in the Human Interface Technology Lab (HIT) in 1991. The aim was to
produce a full colour, wide field-of-view, high resolution, high brightness, low cost
virtual display. Microvision inc. Has the exclusive licence to commercialize the VRD
technology (Figure 5.4)
Figure 5.4 Virtual Retinal System HMD
Page 17
Augmented Reality
This technology has many potential applications, from head-mounted displays

(HMDs) for military/aerospace applications to medical purposes. The VRD projects a
modulate beam of light directly onto retina of the eye producing a rasterized image (fig
5.5). the viewer has the illusion of seeing image as if he/she stands two feet away in front
of a 14-inch monitor. In reality, the image is on the retina of its eye and not on a screen.
The quality of the image he/she sees is excellent with stereo view, full colour, wide field
of view and no flickering characteristics.
Figure 5.5 Virtual Retinal System Scheme
5.2.3 Video See-Through HMD

In contrast, a video see-through display uses video mixing technology, originally
developed for television special effects, to combine the image from a headworn camera
with synthesized graphics. The merged image is typically presented on an opaque headworn display. With careful design, the camera can be positioned so that its optical path is
close to that of the user's eye; the video image thus approximates what the user would
normally see. As with optical see-through displays, a separate system can be provided for
each eye to support stereo vision.
Video composition can be done in more than one way. A simple way is to use
chroma-keying: a technique used in many video special effects. The background of the
computer graphic images is set to a specific color, say green, which none of the virtual
objects use. Then the combining step replaces all green areas with the corresponding parts
Page 18
Augmented Reality
from the video of the real world. This has the effect of superimposing the virtual objects
over real world. A more sophisticated composition would use depth information. If the
system had depth information at each pixel for the real world images, it could combine
the real and virtual images by a pixel-by-pixel depth comparison. This would allow real
objects to cover virtual objects and vice-versa.
Figure 5.6 Video See-Through HMD
5.2.4 Monitor Based Augmented Reality

Monitor based AR also uses merged video streams but the display is a more
conventional desktop monitor or a hand held display. It is perhaps the least difficult AR
setup, as it eliminates HMD issues. Princeton Video Image , inc. has developed a
technique for merging graphics into real time video streams. Their work is regularly seen
as the first down line in American Football Games. It is also used for placing advertising
logos into various broadcasts.
Page 19
Augmented Reality
Figure 5.7 Monitor Based Scheme.
Figure 5.8 Monitor Based Example.
Page 20
Augmented Reality
CHAPTER 6
APPLICATIONS OF MOBILE AUGMENTED REALITY
6.1 Mobile Augmented Reality Contact Lenses - Create New

World of Visions
Figure 6.1 Mobile AR Contact Lens
Mobile augmented realitycontact lenses may do more that improve your sight.
Someday they could replace your mobile phone and let you communicate visually
anywhere in the world, improve your health and make virtual reality real. Perhaps your
ophthalmologist could perform Lasik surgery, burning a wireless circuit into your cornea?
6.1.1 Mobile Augmented Reality Hits Contact Lens Technology
Babak Parviz at the University of Washington in Seattle, is working on a contact
lens technology that could revolutionize wireless health monitoring and mobile
applications for your iPhone. But dont stop there
Babak Parviz lenses become biosensors that monitor internal body functions. While the
prototype version of the lens is powered by radio waves beaming electricity to a loop
antenna embedded in the contact lens, Parviz thinks a mobile phone or solar cells
(wireless electricity) could generate power for the lenses. Mobile augmented reality could
be just around the corner.
Page 21
Augmented Reality
6.2 iPHONE & Augmented Reality Apps: The Future of Gaming
Figure 6.2 iPHONE Mobile AR applications.
The starting point for this article came from an unconventional place a furniture
app from Harveys that allows you to place the sofa of your dreams (possibly the most
unnecessary use of hyperbole ever there) into a room in your house without actually
having to buy it and lug it into your house. Very simply, the app allows users to see what
particular furniture will look like in their rooms, using an innovative augmented reality
camera, in a hypothetical try-before-you-buy approach to pre-shopping. A great app, by
the way, and one that scratches an itch you never thought you knew existed, which is
always key to anything like this being successful, but hardly one of the great
advancements of the 21st century.
6.3 See the Future: Augmented Reality Head-Up Displays

Beckon
Picture-in-picture may be coming to your next car courtesy of head-up displays that put
more snippets of information in your line of sight. By giving you controlled doses of data
projected onto a reflective rectangle just above the steering wheel, automakers say youll
be safer because your eyes dont wander about the cockpit looking to the center stack and
instrument panel.
Page 22
Augmented Reality
Fig 6.3 Shows object or gives information heading up front.
Imagine an exit or turn arrow that doesnt just point to the right but has the same
angle as the turn youre approaching and expands as you reach the turn. Youll interact
with the displays by arm gestures, voice input, or traditional dashboard buttons and
knobs. Cost will be an issue since current HUDs run more than $1,000.
Fig 6.4 Implementation of head up display in Mercedes.
The Mercedes demo on the show floor took the form of a virtual drive in a
simulator through San Francisco with points of interest circled, the idea being you could
gesture to get more information. The gesture youd use in a busy urban area might be a
raised middle finger because of the potential information overload: restaurant, bar,
jewelry shops, tour bus stops, bridge and tunnel congestion. And thats even before you
wonder whos providing the POI information and is it there because its the best, or
because it pays the automaker the best. This is an issue for the whole industry, not just
Mercedes, whose motto is The Best or Nothing.
Page 23
Augmented Reality
6.4 Wire up your Eyes: 2012 is the Year of Augmented Reality
The term augmented reality, or AR, is used to describe technology that enables
normal sight to be modified or enhanced by a computer, allowing data on an object in the
field of view to be displayed alongside it, or to add objects or menus that arent really
there to augment your normal senses. If this all sounds a long way off to you, you need
look no further than your smartphones app store to find a host of smartphone apps
currently in use that use augmented reality to some extent, using the phones camera.
The Nintendo 3DS uses cards for its augmented reality content. If you place an
AR card on a table the 3DSs camera recognises the card and its orientation and inserts
Nintendo characters onto your screen to do battle on the table top. This works just like
QR codes for your smartphone, as a kind of visual barcode for your devices camera- each
card represents a different character or action. This kind of content will become a
standard feature of handheld gaming in 2012.
6.5 New Augmented Reality Apps Point toward Future Trends

6.5.1 New technologies are evolving to make smartphones really useful.
Augmented reality is a technology that combines the real world with digital
information. It gives users the impression that they are interacting with real and physical
Page 24
Augmented Reality
objects. The technology itself is not exactly new and is being used in a varied of
applications from GPS systems to fitness apps.
Fig 6.5 Feature allowing tactile feedback in Smartphones.
Future
The future of augmented reality seems to be more inclined towards haptics and
tactile feedback (which uses sense of touch).
Earlier, Senseg, which claims mastery over such technology, had demonstrated at
the Consumer Electronics Show, haptic technology that allowed users to feel their apps on
the touchscreen by manipulating an electric field. With the technology, users will now be
able to feel bumps and ridges, and also figure out which areas are more rough than others.
The company believes if users are able to get the feel of anything other than glass, it
would be a better experience. The company demonstrated an Android tablet with a
touchscreen, which had different textures on it. Users get used to such a touchscreen
quickly and may not want to go back to a regular touchscreen. Senseg has deliberately
made the effect subtle so it doesn't distract the users while making its presence clearly
felt. The company is still working on different kind of sensations. Another app from
application developer CrowdOptic may point towards a new trend in augmented reality
apps. The new technology of CrowdOptic focuses on crowds, such as in concert or sports
events. When the camera of the smartphone is pointed at a player during a sporting event,
Page 25
Augmented Reality
it displays real time information about the player and the game. The details and context
can also be shared through different social networks. So far, getting information on
moving objects through augmented reality apps was not possible.
Page 26
Augmented Reality
CONCLUSION AND FUTURE OF MOBILE

AUGMENTED REALITY
In this chapter, we have presented an overview of the field of mobile AR,
Augmented reality still has some challenges to overcome. For example, GPS is only
accurate to within 30 feet (9 meters) and doesn't work as well indoors, although improved
image recognition technology may be able to help.
People may not want to rely on their cell phones, which have small screens on
which to superimpose information. For that reason, wearable devices like SixthSense or
augmented-reality capable contact lenses and glasses will provide users with more
convenient, expansive views of the world around them. Screen real estate will no longer
be an issue. In the near future, you may be able to play a real-time strategy game on your
computer, or you can invite a friend over, put on your AR glasses, and play on the
tabletop in front of you.
There is such a thing as too much information. Just as the "CrackBerry"
phenomenon and Internet addiction are concerns, an overreliance on augmented reality
could mean that people are missing out on what's right in front of them. Some people may
prefer to use their AR iPhone applications rather than an experienced tour guide, even
though a tour guide may be able to offer a level of interaction, an experience and a
personal touch unavailable in a computer program. And there are times when a real
plaque on a building is preferable to a virtual one, which would be accessible only by
people with certain technologies.
There are also privacy concerns. Image-recognition software coupled with AR
will, quite soon, allow us to point our phones at people, even strangers, and instantly see
information from their Facebook, Twitter, Amazon, LinkedIn or other online profiles.
With most of these services people willingly put information about themselves online, but
it may be an unwelcome shock to meet someone, only to have him instantly know so
much about your life and background.
Page 27
Augmented Reality
Despite these concerns, imagine the possibilities: you may learn things about the
city you've lived in for years just by pointing your AR-enabled phone at a nearby park or
building. If you work in construction, you can save on materials by using virtual markers
to designate where a beam should go or which structural support to inspect.
Paleontologists working in shifts to assemble a dinosaur skeleton could leave virtual
"notes" to team members on the bones themselves, artists could produce virtual graffiti
and doctors could overlay a digital image of a patient's X-rays onto a mannequin for
added realism.
The future of augmented reality is clearly bright, even as it already has found its
way into our cell phones and video game systems.
Page 28
Augmented Reality
REFERENCES
[01] M. Uenohara and T. Kanade. Vision-based object registration for real-time image
overlay. In Proc. of the IEEE Conference on Computer Vision, Virtual Reality and
Robotics in Medicine (CVRMed95), pages 1322, 1995.
[02] B. Walter, G. Alppay, E. Lafortune, S. Fernandez, and D. Greenberg. Fitting virtual
lights for non-diffuse walkthroughs. In Proceedings of SIGGRAPH 97, 1997.
[03] P. Wellner. Interacting with paper on the digital desk. Communications of the ACM,
36(7):8796, 1993.
[04] J. Weng, P. Cohen, and N. Rebibo. Motion and structure estimation from stereo
image sequences. IEEE Transactions on Robotics and Automation, 8:362382, June 1992.
[05] J. Weng, T. S. Huang, and N. Ahuja. Motion and structure from two perspective
views: Algorithms, error analysis, and error estimation. IEEE Transactions on Pattern
Analysis and Machine Intelligence, 11(5):451476, 1989. This is a computer vision paper
that has geometry extraction from motion and stereo.
[06] R. Whitaker, C. Crampton, D. Breen, M. Tuceryan, and E. Rose. Object calibration
for augmented reality. In Proceedings of Eurographics95, pages 1527, Maastricht, The
Netherlands, August 1995.
[07] M. Wloka and B. Anderson. Resolving occlusion in augmented reality. In
Proceedings of the ACM Symposium on Interactive 3D Graphics, pages 512, 1995.
[08] Yizhou Yu and Jitendra Malik. Recovering photometric properties of architectural
scenes from photographs. In Proceedings of the SIGGRAPH 98 conference, Annual
Conference Series, pages 207217, 1998.
Page 29

Augmented Reality

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Augmented Reality

Uploaded by

Copyright:

Available Formats

Augmented Reality

Department of ECE, BGSIT

1.1 What is Mobile Augmented Reality for?

Computer-presented material is directly integrated with the real world surrounding

Department of ECE, BGSIT

MOBILE AUGMENTED REALITY