Pattern Recognition in

Ubiquitous Computing

Moongu Jeon
GIST
Outline
„ Introduction to ubiquitous computing
„ Mark Weiser
„ “The Computer for the Twenty-First Century"
Scientific American, pp. 94-10, September 1991.
„ “Some Computer Science issues in Ubiquitous
Computing" Communications of the ACM, July
1993.
„ Augmented reality
„ Pattern recognition problems in Ubicomp
„ Introduction to Speech Recognition
Trend of Technology
Development
Ubiquitous Computing
„ Making computing an integral, invisible part
of the way people live their lives, and
available anytime anyplace.
„ Computers become parts of environment,
and vanish into the background.
„ Integrating computers seamlessly into the
world.
“Ubiquitous” in IT
„ Writing
„ The first IT freed information from the limits of
individual memory
„ Books,magazines, newspapers, street signs,
billboards, shop signs, candy wrappers which
are parts of the environment- ubiquitous
„ Current silicon-based IT
„ Huge number of computers and communication
devices is far from having becoming of the
environment
Two Issues in Ubicomp
„ Location
„ If a computer merely knows what room it is in,
it can adapt its behavior in significant ways
without requiring even a hint of artificial
intelligence
„ Scale (size)
„ Tabs (inch-scale machine): Post-It notes
„ Pads (foot-scale): book or magazine
„ Boards (yard-scale): black (or bulletin) board
Tabs
„ ParcTab
„ Olivetti Cambridge
Research Lab – active
badges
„ Identify and keep track
of users or objects, and
do more tasks.
„ Roy Want, PARC – tab
incorporating a small
display
„ Serves simultaneously
as an active badge,
calendar, diary
Pads
„ Scrap computer (analogous to scrap
paper)
„ Can be grabbed and used anywhere.
„ Have no individual identity.
Boards
„ Large and shared display
„ Video screen
„ Bulletin boards
„ Whiteboards
„ Electronic bookcase
„ LiveBoard (PARC)
Other Issues Ubicomp
„ Cheap, low-power hardware
components.
„ A network that ties them all together.
„ Software for screens and pens
„ Applications.
„ Privacy
„ Computational methods
Augmented Reality
„ The opposite approach from virtual reality.
„ VR encloses people in an artificial world using
computers.
„ AR augments objects in the real world using
computers.
„ Examples
„ Digitaldesk (Wellner 1993)
„ KARMA (Feiner 1993)
„ Flatland (Mynatt 1999) –augmented whiteboard
„ UbiTV, MRWindow, ARTable (Woo 2006)
 Mobile AR

„ What is AR?
„ To enhance the user’s
perception of and interaction
with the real world through
supplementing the real world
with 3D virtual objects that
appear to coexist in the
same space as the real
world

„ We define AR system
„ Blends real and virtual, in a
real environment
„ Interact on-the-fly
„ Augment in 3D
Mobile AR (개념도)
증강을 위한 Invisible마커 인식 ubiTV

사용자
C ubiTV

■▶↑ ↓

증강된 컨트롤 버튼으로

서비스 제어
사용자
B

사용자
선택적 콘텐츠 공유
A

특정 사용자와의 선택적 프로파일에 따라

콘텐츠 공유 개인화된 콘텐츠 증강
Mobile AR in U-Space
Future of Ubicomp
„ A key concept of ubicomp is to use
technology to create a calmer
environment (Weiser 1998).
„ Computer technology should serve
humans as environment that does not
occupy much of human attention,
and should serve humans calmly not
consuming human effort.
Pattern Recognition in
Ubicomp
„ Personalization
„ Need to provide personalized service based on
each person’s service history and preference.
„ Collection of user’s data using wireless sensors
„ Recognition of user’s behavior pattern
„ Object or image recognition to get the
augmented data in U-space.
„ Speech recognition
 Process of Pattern
Recognition

Feature Classifier System

pattern sensor Feature
sensor selection design evaluation
generation
Speech Recognition
„ Isolated word recognition (IWR)
„ Continuous speech recognition (CSR)
„ Speaker-dependent recognition
„ Speaker-independent recognition
„ Dynamic time warping (DTW) - DP
„ finds an optimal match between two sequences
of feature vectors which allows for streched
and compressed sections of the sequence.
Symmetrical DTW
Dynamic Time Warping
„ Global constraints
„ Local constraints
„ Monotonicity - matching paths cannot go
backwards in time
„ End point constraints
„ Starts at (0,0) and ends at (Ι,J) and whose first
transition is to the node (1,1)
„ The cost for the transitions
„ Euclidean distance
DTW Global Constraint
DTW Local Constraints
DTW Local Constraints
„ Sakoe and Chiba local constraints
Example
Test Words
 Feature Generation

„ Reference and test patterns

„ 0.45, 0.4 sec. long
„ 22050 Hz sampling rate
„ Frame size 512 samples long
„ 100 samples overlapped
„ 24, 21, 23 (Number of frames)
 Feature Selection
„ r(j), j=1,…,J -Æ reference pattern
„ t(i), i=1,…,Ι -Æ test pattern
„ xj(n), n=0,…511 -Æ samples for the
jth frame of the reference pattern.
„ Taking DFT
„ Xi(m)=∑xi(n)exp(-j2πmn/512)/5121/2,
m=0,…,511
„ Use the first 50 DFT coeff. as features
„ r(j)=[Xj(0) Xj(1) … Xj(49)]T, j=1,…J
Cost
„ Euclidean distance between r(j) and
t(i) corresponding to node (i,j)
„ d(i,j)≡||r(j)-t(i)||
„ Symmetrical DTW
„ D(i,j) = min[D(i-1,j-1),D(i-1,j),D(i,j-1)]
+d(i,j)
„ Asymmetrical DTW
„ D(i,j) = min[D(i-1,j-2),D(i-1,j-1),D(I-
1,j)]+d(i,j)
Results
„ For the test pattern “love”
„ Dlove = 11.472 (0.221)
„ For the test pattern “kiss”
„ Dkiss = 25.155 (0.559)
„ Dlove < Dkiss -Æ correct