Álvaro Cassinelli, Stéphane Perrin, Masatoshi Ishikawa

Ishikawa-Namiki-Laboratory
Parallel Processing for Sensory information
University of Tokyo
 I. The problem: Today’s approach
UI : “WIMP” + alphanumeric input
I/O Device: touch sensitive screen+stylus
Input and
interaction with
“ever-shrinking”
portable electronic physical
interaction

devices
nice: input space is merged with
viewing space

…Next step ?

• remove the need for any physical I/O space at all

• remove the need of any additional input device

⇒ ”ergonomics-independent” miniaturization
(*) windows, icons, menus and pointer
How to remove the input space? (non-invasive techniques!)

...By providing the PDA with intelligent sight:

- hand/finger position
- handwritten character recognition
- gestures recognition, etc

Low, medium or high-

level vision
drawbacks:

• May require intensive computation

• Not robust enough
• Sensitive to external illumination conditions/backgrounds...

In a word: passive-vision input versatile but not mature yet…

…in any case overkill for the problem at hand! (“inputspace-less” WIMP)
Closer look at the problem...
-Hand/finger closer than anything else
-fingertip always visible (no occlusion)
-fingertip “shape” simple and stable
what a short-sighted
PDA would actually
see…

proximity sensor...
+
...angular information

should provide sufficient

data for tracking!
⇒ cost-
efficient,
ROBUST,
animal
FAST!
antenna

…but cumbersome,
cane for
annoying?
the blind
The proposed “antenna”:
a directive beam of light

advantages:
- active lighting: provides and control illumination
- real-time, precise 3D measures (telemetry)
- measurement of surface roughness, speed, even biometric data.

And...

- laser can be used as output device (on

any external surface, including the retina:
VRD)

Inconvenients: no haptic feedback (just like passive vision, but visual cues possible).
... but how realistic is the integration of a LASER RADAR on a PDA?

VERY! Thanks to today’s “MOEMS” technology...

The realm of “MOEMS” (micro-opto-electro-mechanical systems):

Electronic circuit, photodetectors, laser diodes sources and mirrors
integrated in the SAME chip.
II. Smart Laser Scanner Prototype

Hardware (discrete components):

•laser diode,
•a pair of steering micro-mirrors,
•single non-imaging photodetector.
Tracking Principle

smart laser
scanner:
laser excursion is
locked around the
area of interest

circular laser
“saccade”

Tracking sequence
repeated every
millisecond.

kHz refreshing rate explains

algorithmic extreme simplicity
Prototype Smart Scanner in Action
2 DOF (using fingertip)

(angular precision: better than tenth of a degree)

3 DOF (using fingertip)

Depth precision: (intensity based) around 1cm up to 2 m

a little farther away …
 Simultaneous tracking

(up to four targets without hardware duplication)

Several users…
… unique user, higher DOF
Achieving 6 DOF (using three tracking points)
Fast 3d tracking
of a ping-pong ball
Interactive laser annotation
UIs based on this 3D Input Device:
Proof-of-principle examples
3D locator

X/Y/Z
translation

Gestural command:

RIGTH LEFT
CLICK CLICK
Interface for visualizing 2D data

Positional command:

)
X/Y (+
M
translation O
ZO

) Gestural command:
(-
M
O
ZO

LEFT RIGHT
CLICK CLICK
Visualization interface for video

Positional command:

X/Y e +)
m
a n(
translation r
F tio
o si
p

Gestural command:
e -)
r am n (
F tio
o si
p

Zoom x2 Zoom /2
Conclusion
The problem: design a 3D input device that does not interfere with
technological miniaturization

Proposed solution: use laser “light antennas” to track one (or more) fingers.

Advantages: •fast (no image processing),

•precise 3D measurements,
•insensitive to external light conditions,
•no markers necessary,
•can be used as output device.
Further Work

MOEMS integration: PDA on a keychain,

PDA on a chest-pin: touching the pin or
activates tracking, a gesture can be used to
ask for emails, and these are read on the
hand...

Study of adaptive saccade shapes (ex: follows the hand contour)

Prototype without mobile parts, using multiple laser beams...

 Speed (3m/s)

Precision (cm)

multi-tracking
see more (up to four fingers)
at both
#102

Interactive laser
annotation
ACR - 5.4.2005