Hybrid Infrared and Visible Light Projection for Location Tracking

Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 Newport, RI

Hybrid Projection
one projector infrared visible

But, why?

Projector-Based Location Discovery

Projector Light sensors

[Lee, UIST04]

Electronics & computer

Projector-Based Location Discovery

calibration free: - no computer vision - no alignment - no manual input Scalable and robust

Moveable Surfaces
[Lee, UIST 2005]

- calibration free - no external tracker - interactive content

Drawbacks
Location Discovery [04] Moveable Surfaces [05]

Incremental tracking
Caustic B&W patterns Momentary movement

Ideally

Full-screen Application Content

Full-screen Location Patterns

Hybrid Projection
one projector infrared visible

for the computer

for the human

Infrared & Visible Projection

Infrared

Visible

Infrared & Visible Projection

Infrared

Visible

10

Infrared & Visible Projection

Infrared

Visible

11

Infrared & Visible Projection

Infrared

Visible

12

Infrared & Visible Projection

Infrared

Visible

13

Infrared & Visible Projection

Infrared

Visible

14

Infrared & Visible Projection

Infrared

Visible

15

Infrared & Visible Projection

Infrared

Visible

16

Infrared & Visible Projection

Infrared

Visible

17

Infrared & Visible Projection

Infrared

Visible

18

How?

Light Source: Lamps

Xenon Arc Lamp

Light Technologies

More Efficient and Better Lifespan

Products emerging on market this year, 2007

IR and Visible Light LEDs

1000 LEDs/mm2
University of Strathclyde, Institute of Photonics

DMD

Lens

LED Array

Projection optics

Our Dev Kit: High-Speed Dev Kit: Production Unit:

180 binary images/s 16,500 binary images/s +50,000 binary images/s

1024x768 area = 20 binary images 60Hz tracking = 2.4% duty cycle of production DMD Required changes to commercial designs would be minimal.

Demo of Capability
Uses a second projector for visible content

Inherent Multi-Stylus Tracking

Non-Planar and Discontinuous Surfaces

Static IR Patterns

Concept Applications
Simulated using external tracking (calibration)

Hand-held projection with photosensitive tags

[Siggraph 2004]

Foldable Interactive Displays [submitted to CHI]

Acknowledgements
Funded in part by the National Science Foundation under grants IIS-0121560 and IIS-0325351

Funded in part by Mitsubishi Electric Research Labs

Johnny Chung Lee

johnny@cs.cmu.edu

Other ways to make invisible patterns

Other non-visible wavelengths Steganography
Color shifting Noise encoding

Bit Timing
Synchronization may be difficult

Camera-Based Tracking
Requires calibration Requires markers for segmentation
IR sensors + transmitter is less power than 4 IR LEDs

Does not provide ID Limitation on the number of points Limitation on tracking rate Limitation on scene/target complexity Resolution is not as scalable Less optically robust
Optical path geometry and variable illumination

Projector vs Camera Tracking

Sensors provide point ID Independent of scene/surface complexity

Space-Labeling Projectors
11 Infrared LED slide projectors
Potentially Low-cost Per axis: 500Hz tracking at 10-bits

Outdoor motion tracking

37

Binary
0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111

Gray
0000 0001 0011 0010 0110 0111 0101 0100 1100 1101 1111 1110 1010 1011 1001 1000

Binary
0000 0001 0010 0011 0100 0101 0110 0111 1000 1001 1010 1011 1100 1101 1110 1111

Gray
0000 0001 0011 0010 0110 0111 0101 0100 1100 1101 1111 1110 1010 1011 1001 1000

Binary

Gray

Binary

Gray

Binary

Gray

Binary

Gray

Binary

Gray

Binary

Gray

Binary

Gray

Binary

Gray

Binary

Gray

Binary

Gray

Binary

Gray

Binary

Gray

Pixart Chip
1024x768 resolution 100Hz tracking 4 points

Only 4 points High power LED points