MS

I
Multimedia Databases
Arif Ghafoor
Professor
Electrical and Computer Engineering
Purdue University
MS
I
2
Multimedia Database Laboratory
Facilities at Purdue University
Distributed Multimedia Systems Lab
(ECE)
Ghafoor, Kashyap, Khan
Multimedia Systems Lab (CS)
Aref, Elmagarmid, Prabhakar
Raid Lab (CS)
Bhargava
Video and Image Processing Lab (ECE)
Delp
MS
I
3
On-Going Research Projects in
Multimedia Databases
Multimedia database management
(NSF, Fuji Electric, AT&T)
Video modeling and management
Multimedia document management
Distributed multimedia systems
(NSF, AFRL, IBM, Intel, Siemens)
High-performance multimedia database
architecture for storage management
(NSF, AT&T)

MS
I
4
Example of Event-Based Video
Data (Touch-down Scenario)
MS
I
5
Research Challenges for Video
Databases
Development of video processing techniques
for automatic object identification and motion
tracking
Development of data models with powerful
semantic expressiveness
Designing efficient indexing, classification
and event-based searching techniques with
high degree of precision
Providing suitable video querying
environment
Addressing scalability
MS
I
6
Video Parsing and
Segmentation
Object Recognition
Granularity
L
e
v
e
l

o
f

A
b
s
t
r
a
c
t
i
o
n

Iconic-based
Grouping and
Browsing
Motion Detection
Analysis
Knowledge-based
Higher-level
Semantics
Fine Coarse
Low
High
Semantic Video Data Modeling
Complexity
MS
I
7
Raw
Video
Database
Raw
Image
Database
Physical
Object
Database
Frame
Object Description
Image Features
Spatial-Semantics
of Objects
(human,building,)
Sequence of Frames
(indexed)
Object Identification
and Tracking
Intra/Inter-Frame
Analysis
(Motion Analysis)
Inter-Object Movement
(Analysis)
Semantic Association
(President, Capitol,...)
Spatio-Temporal Semantics:
Formal Specification of
Event/Activity/Episode for
Content-Based Retrieval
Spatial Abstraction
Temporal
Abstraction
Object Definitions
(Events/Concepts)
An Architecture for Video
Database System
MS
I
8
Model Querying
Mechanism &
Attributes
Semantic Modeling
Approch
Video Processing
Approach
Fuzziness
Support
Zhang (HP) Visual Browsing - video parsing No
Liu (Princeton) Visual Browsing Graphical video parsing No
Tanaka (Tokyo) Visual SQL, text
only
Object-oriented Manual No
Weiss (MIT) Algebraic , text
only
Object-oriented Manual No
Golshani (ASU) Algebraic ,
motion only
- Automatic
tracking
No
Chang (Columbia) Sketch & Image
Features
- Auitomatic
tracking
Moderate
Ghafoor (Purdue) Petri-net ,Text
and motion
Spatio-temporal
logic and Petri-
nets
Automatic
tracking
High
Comparison of Different Video
Database Management Projects
MS
I
9
Digestive System
Esophagus
Stomach
Omasum
Abomasum
Protozoa
Salivary Glands
Small Intestine
Large Intestine
Rumen
Reticulum
Exit
Multimedia Browsing Graph
The Ruminant Digestive System

Four compartment stomach that:
- utilizes complex carbohydrates
such as cellulose
- utilizes non-protein nitrogen
sources
- utilizes feeds of no nutritive value
for humans




Rumen

The rumen, the first compartment of the
ruminant stomach, is essentially non-
functional at birth but develops rapidly when
calves are given solid feeds such as hay and
grain.






Sample Multimedia Document
MS
I
10
Document Representation and
Specification Complexity
Media Types and Attributes Complexity
SGML/XML
HTML
PDF
Interleaf/
Word
Framemaker
VRML
MPEG
HyTime
Style Complexity
MS
I
11
Specification
- Reliability
- Resolution
- Rate of Presentation
- Display Area
- Temporal Synchronization ( Intra/Inter )
Meta Data / User Interface
Translation
- End-to-End
Delays
- Jitter Delay
- Bandwidth
- Packet Loss
Rate
- Storage Throughput/
Bandwidth
- Storage Delays
- Distributed
Database
Coordination (QoS)
- CPU Throughput
- Memory Overflow
and Reliability
- Intrusion Detection
- Access Control
Database Security OS Network
Negotiation
Dependency Model Analysis and QoS Adjustment
End-to-End
Run Time
Scheduling
End-to-End Resource Allocation and Scheduling
End-to-End QoP / QoS Management
MS
I
12
Multimedia Document Management
Quality of presentation requirements
Resolution, reliability, rate
Synchronization requirements
Temporal, spatial, and logical structure
specification
Media processing requirements
Coloring, enhancements, dubbing, etc.
Security attributes
MS
I
13
Text
Video Image
Text Video
Audio
Audio
p
1
p
2
p
3
p
4
Parameter Symbol
Duration t
Size s
Deadline p
Allowable Skew a
Display Area A
Rate r
Reliability w
Contents
Petri-Net Based Model for Synchronization and
Quality Specification of Multimedia Documents
MS
I
14
Rumen
Reticulum
Digestive System
Esophagus
Stomach
Omasum
Abomasum
Protozoa
Salivary Glands
Small Intestine
Large Intestine
Image
Audio
Video
Text
Audio
Image
Image
Text
Text
Text
Multimedia Browsing
Environment
MS
I
15
Research Issues for Quality and Performance of
Multimedia Document Management Systems
Document modeling
(Dimensionality/Flexibility/Portability)
Languages (SGML/XML)
Graphical (OCPN)
Transformation among models and
heterogeneity management of models
Development of efficient search and indexing
techniques for multimedia documents using
these models
Clustering of documents
Based on contents for fast search
Controlled access
MS
I
16
Model Spatial/Temporal
Composition
Facility
Logical
Structuring
Content &
QoP
Specification
Facilty
Consistency
Checking for
Scheduling
Meta
Schema
Translation
Complexity
Storage
Placement
Support
Network
Interface
Support
HyTime/
SGML
Both Yes Content Only No Difficult No No
Object-
Oriented
(VODAK)
Both Yes Both No Easy No No
OCPN Both Partial Both Yes Easy Yes Yes
Comparison of Different
Multimedia Document Models
MS
I
17
User
Multimedia
Structuring Module
OODBMS and MQL
Distributed Query Processor
HyTime/SGML/XML Translator
HyTime/SGMLXML Parser -->
Multimedia Query Language
Document Generator
Tool Library
Temporal Synchronization Manager
Multimedia Meta-Data
Video
Database
Text
Database
Image
Database
Audio
Database
To Presentation Device
Software Architecture of a Multimedia
Database Management System
MS
I
18
Presentation Device
Audio DBMS
Network A
Network B
Image
Audio
Audio
Text
Video
DBMS
DBMS
DBMS
DBMS
DBMS
Distributed Multimedia Database
Systems
MS
I
19
Research Issues in Distributed
Multimedia Database Management
Distributed Object Management
Efficient distributed query processing
techniques
Network caching techniques for
composition of distributed objects
Adaptive data filtering techniques to
satisfy diverse user QoS requirements
MS
I
20
Multimedia Database Client
Visual Tool for
Multimedia Document
Generation
Multimedia
Database Interface
Multimedia
Presentation Subsystem
Multimedia Database Server
Multimedia
Database Server
Multimedia
Database Server
...
Text Image Video Audio
Meta
Data
Database
Management
System
Media Server
Subsystem
Database Connectivity
Integrated Multimedia Information Server
D
i
r
e
c
t
o
r
y

M
a
n
a
g
e
m
e
n
t

M
u
l
t
i
m
e
d
i
a

M
e
t
a

D
a
t
a

M
a
n
a
g
e
m
e
n
t

Distributed Query
Processor
API for SBS Network
API for SBS Network
API for SBS Network
Architecture of a Distributed
Multimedia Database Management
MS
I
21
Planned Experiments
Development of video database
management system ( using database
compute servers and storage system)
Development of multimedia document
management system
Development of distributed multimedia
object management system using SBS
architecture