Multimedia Content Description Interface

A Presentation By :-
Kunal Kishore

Cochin University of Science And Technology MPEG-7

 Contant

 Introduction
 Need of MPEG-7
 History of MPEG
 Elements Of MPEG-7
 Parts of MPEG-7
 Application
 Conclusion

Cochin University of Science And Technology MPEG-7

 Introduction
 In September 2001, Moving Picture Experts Group
(MPEG) specified an international standard called
Multimedia Content Description Interface, or MPEG-7.
 MPEG-7 is a standardized description, based on XML
Schema, of various types of AV content.
 Material (e.g., still pictures, graphics, audio, video)
with associated MPEG-7 data can be efficiently indexed
and searched for.

Cochin University of Science And Technology MPEG-7

 Why do we need MPEG-7 ?

Support for Advanced Query

 Fast And accurate acess Visual

Need
 Personalized Content Production and
Consumption
+ Audio
 Content Management’
Sketch
 Automation
Cochin University of Science And Technology MPEG-7
 MPEG- History…1
Established in 1988
 MPEG: Moving Picture Experts Group

 ISO / IEC/JTC1/SC29/WG11

A Working Group of ISO/IEC in charge

of the Development of Standards for
Coded epresentation of Digital Audio
and Video

Cochin University of Science And Technology MPEG-7

 MPEG- History…2
 MPEG-1: Interactive CD and MP3 11 / 1992
 MPEG-2: DTV, STB, DVD 11 / 1994
 MPEG-4: Web and Mobility ver1: 09 /1998, ver2:
11 /1999
 MPEG-7: ??? 08 / 2001

Cochin University of Science And Technology MPEG-7

 Scope of MPEG-7

 The description generation (feature extraction, indexing)

is not a normative part of the standard
 The description consumption (search engine, retrieval
process) is also not a normative part of the standard.
 Just the description is normative
Standardizes the minimum needed to ensure
interoperability

Cochin University of Science And Technology MPEG-7

 MPEG: Main Elements…1

Cochin University of Science And Technology MPEG-7

 MPEG: Main Elements…2
 Descriptors (Ds): representations of features
 To describe various types of features of multimedia
information
 To define the syntax/semantics of each feature
representation
 Color example – specify histogram in RGB color space
 Description Schemes (DSs)
 Specifies the structure and semantics of the relationships
between its components, which may be either Ds or
other DSs.

Cochin University of Science And Technology MPEG-7

 MPEG: Main Elements…3
 Description Definition Language (DDL)
 The language to specify DSs and possibly Ds
 Will allow the creation of new DSs (and possibly
Ds) and the extension modification of existing DSs .
 System tools
 To support multiplexing of description,
synchronization issues, transmission mechanisms,
file format, etc.

Cochin University of Science And Technology MPEG-7

 Parts of MPEG-7 Standered
 Systems
 Description Definition Language
 Visual
 Audio
 Multimedia Description Schemes
 Reference software
 Conformance Testing
Extraction and use of descriptions

Cochin University of Science And Technology MPEG-7

 MPEG-7: Systems

It defines tools to:

 Provide for efficient storage and transport
 Synchronize between content and description
 Manage and protect intellectual property

Cochin University of Science And Technology MPEG-7

Basic Structure of Visual Descriptors

 Grid Layout
 Time Series
 2D-3D Multiple View
 Spatial 2D Coordinates
 Temporal Interpolation

Cochin University of Science And Technology MPEG-7

 MPEG-7 Visual Descriptors

 Color
 Texture
 Shape
 Motion
 Localization
 Face Recognition

Cochin University of Science And Technology MPEG-7

 MPEG-7 : Audio Descriptors

 Sound Effects
 Music Instrument Timbre
 Spoken Content
 Melody Contour

Cochin University of Science And Technology MPEG-7

 Low level Audio Visual descriptors
Video segments Still regions

Color Color
Camera motion Shape
Motion activity Position
Mosaic Texture

Moving regions Audio segments

Color Spoken content
Motion trajectory Spectral
Parametric motion characterization
Spatio-temporal Music: timbre,
shape melody

Cochin University of Science And Technology MPEG-7

Multimedia Description Schemes- MDS
• MPEG-7 Multimedia Description Schemes (DSs) are
metadata structures for describing and annotating audio-
visual (AV) content.
• The DSs provide a standardized way of describing in XML
the important concepts related to AV content description
and content management in order to facilitate searching,
indexing, filtering, and access.
• The DSs are defined using the MPEG-7 Description
Definition Language (DDL), which is based on the XML
Schema Language, and are instantiated as documents or
streams.

Cochin University of Science And Technology MPEG-7

 Organization of MDS tools

Cochin University of Science And Technology MPEG-7

 Example of Segment Tree
SR1:
· Creation, Usage meta
information
· Media description
SR6: · Textual annotation
· Color Histogram · Color histogram, Texture
· Textual annotation
SR2:
· Shape
· Color Histogram
Foreground · Textual annotation
Background

SR5:
· Shape
· Textual annotation
SR3:
· Shape
SR4:
· Color Histogram · Shape
· Textual annotation · Color Histogram
· Textual annotation
19

Cochin University of Science And Technology MPEG-7

 DDL: XML & format
XML was created to structure, store, and transport information.
• It is just plain text. Software that can handle plain text can also
handle XML.
• It is just pure information wrapped in tags. Someone must write a
piece of software to send, receive or display it.
• XML-aware applications can handle the XML tags specially. The
functional meaning of the tags depends on the nature of the
application.

Cochin University of Science And Technology MPEG-7

 Example

Cochin University of Science And Technology MPEG-7

 Binary Format- BiM
 BiM is designed for transmission and storage
 BiM encoders and decoders can deal with any
XML language.
 BiM is a schema oriented encoding scheme
• The binary format is deduced from the schema
definition
• There is no need to define coding tables or a
specific encoding mechanism

Cochin University of Science And Technology MPEG-7

 Application Scenario

Cochin University of Science And Technology MPEG-7

 Additional Application

 Mobile phone
 Personal video recording (PVR) systems
 Content delivery
 Surveillance, remote sensing, visual control
equipment
 Database, server
 Storage and retrieval of audiovisual databases
(image, film, radio archives)

Cochin University of Science And Technology MPEG-7

 Some Industrial Tools

• IBM MPEG-7 Annotation Tool

• Ricoh MovieTool
• Canon MPEG-7 Speech Recognition Engine

Cochin University of Science And Technology MPEG-7

 MPEG-7 Timeline Work plan

Divergence Convergence

1996 1998 1999 2000 2001

International
International
Call
Call for
for proposals
proposals Working
Working draft
draft Committee
Committee draft
draft Standard
Standard
Oct
Oct 1996
1996 Mar
Mar 2000
2000 Oct
Oct 2000
2000 Oct
Oct 2001
2001
Final
Final committee
committee Draft
Draft Draft
Draft international
international
Jan
Jan 2001
2001 Standard
Standard
July
July 2001
2001

Cochin University of Science And Technology MPEG-7

 Conclusion
• MPEG-7:

– AV content description for interoperable application

• Description Definition Language:
– XML Schema (flexibility) + Binary version (efficiency)

• Description Schemes:
– Library of description tools

– Covers a wide range of generic needs

Cochin University of Science And Technology MPEG-7

 Refrences
 Multimedia System Standered & n/w :Puri
 MPEG-7 Awareness event(http://www.mpeg-industry.com)
 http://www.mpeg.chiariglion.org (A MPEG Home Page )
 http://www.mpeg-alliance.com
 http://www.w3.org/1999/05/06-xmlschema-1
http://www.dlib.org/dlib/september99/hunter/09hunter.html

Cochin University of Science And Technology MPEG-7

 Questions ?

Cochin University of Science And Technology MPEG-7

Cochin University of Science And Technology MPEG-7