Di Mascio Bozen 22

Cross Language Information Technologies
Free University of Bozen, Italy
Multi-media Retrieval Systems

based on
User Centered Design Methodology:
from the Content-based to the Semantic-based approach
dott. ing.Tania Di Mascio
EURAC RESEARCH BOZEN
Tania Di Mascio, Eurac Research
Bozen (Italy), 22 december
Outline
Outline
Introduction
Definitions
Open issues
Multi-media Retrieval Systems [20,21]

Content Based Image Retrieval Systems for Vector Images [4,7,9,22]
Ontology Based Text Retrieval Systems for the Semantic Web [4,7,9,22]
Ontology Based Multimedia Retrieval Systems for LIS [22, 23]
Human Computer Interaction [1,3]

Information Visualization
Vector Images Visualization [2,9,22]
Ontology Visualization [5,6]

Usability Evaluation
Usability Evaluation for software system [10,13]
Usability Evaluation for web site [11]
Publications
Introduction
Introduction definitions
Multi-media Information
Text
Retrieval Systems
Content-Based
Images
Logical-Based
Sounds
Combinations (e.g. paint)
Ontology-Based
Ontology Based Text Retrieval Systems

Content Based Image Retrieval Systems
Ontology Based Multimedia Retrieval Systems
Open Issues
Introduction open issues
Multi-media Information Retrieval Systems [20, 21]

Content Based
Ontology Based
Ontology Based
Image
Multimedia
Text
Retrieval Systems Retrieval Systems Retrieval Systems
vector images
LIS
Semantic web
Content Based Image Retrieval Systems for Vector images [4, 7, 9, 22]
Ontology Based Text Retrieval Systems for Semantic Web [8, 12]
Ontology Based Multimedia Retrieval Systems for LIS [22, 23]
Retrieval Issue
Open Issues
Introduction open issues
Human Computer Interaction [1, 3]

visualization
usability
Vector images Visualization [2, 9, 22]

Ontology Visualization [5, 6]
Usability Evaluation for software systems [10, 13]
Usability Evaluation for software Web systems [11]
Interaction Issue
Outline
Outline
Introduction
Multi-media Retrieval Systems [20,21]

Ontology Based Multimedia Retrieval Systems LIS [22, 23]


Usability Evaluation for software system [10,13]
Usability Evaluation for web site [11]
Publications
Content-Based Image Retrieval Systems

Multimedia Retrieval Systems Content Based Image Retrieval Systems
Users
Users Interface
Query Resut
Module
Query Interface
Module
Engine System
Query
Image
Query Image
Indexed
Feature
Extraction
Module
Ranking
Module
Feature
Representation
Module
Similarity
Computation
Module
Feature Process
DB Images
Indexing
Repository
Compairison
Process

Multimedia Retrieval Systems Content Based Image Retrieval Systems Retrieval Issue
CBIR Engine
Query
Image
Query Image
indexed
Feature
Extraction
Module
Ranking
Module
Feature
Representation
Module
Similarity
Computation
Module
Feature Process
DB Images
Indexing
Repository
Compairison
Process
Mathematic and Physic approach to solve the Retrieval Issue


Feature Extraction Module
m
i =0
* xi
= xCM
mi
i =0
m * y
i
i= 0
n
mi
= yCM
i= 0
Center of Mass
Calculation
Process
System Origin
Moving
Process
Discretizing
Process

Feature Representation Module
Feature Descriptor
9 Hu Moments
Regular Moments
9 Zernike Moments
mi ; j =
i
j
(
)
(
)
x
CM
CM
i; j
9 Bamieh Moments

Similarity Computation Module
Similarity Measures
9Euclidean Distance
9 Cross Correlation
DE =
D=
wi (FP FI
N
w F
i =0
Pi
FI i
w F w F
max ( p , x )
D(i, v ) =
1
(
)
min
,
p
x
J =1
9 Discrimination Cost
i =1 N
j =1
Pj
k =1
Ik
vj
ij
vj
ij

Multimedia Retrieval Systems Content Based Image Retrieval Systems Interaction Issue
Tuning Interface

Multimedia Retrieval Systems Content Based Image Retrieval Systems Interaction Issue
Search Interface
VISTO (Visual Image Search TOol)
Ontology-Based Text Retrieval Systems

Multimedia Retrieval Systems Ontology Based Text Retrieval Systems Retrieval Issue
Sewasie Interaction Layer

Ontology
Editor
Tool
OLAP
Tool
Query
Tool
Negotiation
Tool
Sewasie
User
Profile
Data
MetaData
Sewasie Core System
Query
Agent
Brokering
Agent
Internet
Internet
SINode
Repository
SINode
Repository
..
SINode
Repository

Multimedia Retrieval Systems Ontology Based Text Retrieval Systems Retrieval Issue
The intelligence of the Sewasie Query tool is driven by an

ontology.
the User perspective
describes the domain of the

data in the information
system;
defines a vocabulary which
is:
richer than the logical
schema of data,
closer than the users
rich vocabulary.
the System perspective
enables only the

generation of precise and
unambiguous query
expressions.

Multimedia Retrieval Systems Ontology Based Text Retrieval Systems Interaction Issue
Retrieval
Data
Query Tool
Ontology
Editor
Tool
MetaData
Query
Building
Module
Query
Result
Module
Data
OLAP
Visualization
Tool
Sewasie
User
Profile
Single
Item
Negotiation
Communication
Tool

Query Tool Interface

the User perspective
to support the user in

formulating a precise
query in order to best
captures users
information needs;
complete ignorance
the System perspective
to generate a query ready

to be executed by some
evaluation engine
associated to the
information system;
conjunctive query

Query Tool Interface
Information Domains;
Query start;
Query Compose;
Query result;
Ontology-Based Multimedia Retrieval Systems

Multimedia Retrieval Systems Ontology Based Multimedia Retrieval Systems e-LIS

Multimedia Retrieval Systems Ontology Based Multimedia Retrieval Systems Retrieval Issues
1) String
2) Image
ES2:
ES1:(Con
Ori
Loc
Mov not )

Multimedia Retrieval Systems Ontology Based Multimedia Retrieval Systems Interaction Issue
static Interface
dynamic Interface
based on
constraint visualisation
Users can select

parameters
Users are guided in parameters

choosing i.e.
Outline
Outline
Introduzione
Definizione
Problemi Aperti
Multimedia Retrieval Systems [20,21]
Ontology Based Multimedia Retrieval Systems [22, 23]
Usability Evaluation for Software system [10,13]
Usability Evaluation for Web site [11]
Publications
Vector Images Visualization

Human Computer Interaction Information Visualization Ontology Visualization e-LIS
Movement Parameters as animation SVG
Vector Images Visualization

Human Computer Interaction Information Visualization Vector Images Visualization VISTO
Result as Similarity Ranking List
Outline
Outline
Introduzione
Definizione
Problemi Aperti
Ontology Based Multimedia Retrieval Systems [22, 23]
Publications
User Centered Design Methodology

Human Computer Interaction Usability Evaluation
Context analysis
survey evaluation
Usability goals
I prototype
survey evaluation
observational evaluation
(Think Aloud and Verbal Protocols)
II prototype
survey evaluation
controlled experiment
Final prototype
survey evaluation
Product
survey evaluation
User Centered Design Methodology

Human Computer Interaction Usability Evaluation
User-Based Criteria
A common Think Aloud Protocol method schema
Users
Analysing Users
Designing
the experiment
Performing
the experiment
Teaching
Users
Analysing the
usability
Engineers

Human Computer Interaction Usability Evaluation Software System
Usability tests of the Query tool aims at perform:

The effectiveness: it is evaluated by relating the goals or
sub-goals of using the Query tool to the accuracy and
completeness with which these goals can be achieved.
The efficiency: it is evaluated by relating the time spent
to show query result.
The satisfaction: it is evaluated by relating the comfort
and acceptability of the Query tool interface.
Us
ers
Analy
sing
User
s
Perfor
ming
the
experi
ment
Teach
ing
Users
Analysi
ng the
usabilit
y
Human Computer Interaction Usability Evaluation Software Systems
For each query
A complexity tree
Level
l=1
P1
C1
l=2
P1
C1
P1
l=3
l=4
C2
P2
C1
C1
P1
P2
Desig
ning
the
exper
iment
C1
P3
C2
P3
En
gin
eer
s
Us
ers
Desig
ning
the
exper
iment
Analy
sing
User
s
Perfor
ming
the
experi
ment
Teach
ing
Users
Analysi
ng the
usabilit
y
En
gin
eer
s
C1 = 0,1
if edge = add proprierty
P1
C1
C2 = 0,2
P1
if edge = add concept
C1
C2
P1
Pn = n
num _ lel
l =1
l * c1; 2
C1
for each node of level
num _ nod _ per _ level

n =1
pn (l ) * f (n, l )
P1
C1
P2
C1
P3
C2
P2
P3
evr _ num _ suc _ level If l < lmax

f (n, l ) =
1
If l=lmax
Us
ers
Analy
sing
User
s
Perfor
ming
the
experi
ment
Teach
ing
Users
Analysi
ng the
usabilit
y
En
gin
eer
s
For example:
Supplier located in Europe which is a Multinational selling on
the Italian Market
Supplier
1
0,2
Which is a
Multinational
Europe
0,1
selling_on
0,1
l=1
located_in
l=2
l=3
Italian_market
The complexity is: 1*[0.1*1]*2+2*[0.2*1+0.1*2]*0.5+3*[0.1*1]*1 = 0.9

Desig
ning
the
exper
iment
DIEL web-site Systems

Human Computer Interaction Usability Evaluation DIEL Web Systems
Effectiveness measure (*)
TRAD
BOX
Domain Expert
33,4%
20,07%
Non Domain Expert
29,7%
17,02%
Computer Skilled
11,7%
5,2%
Non Computer Skilled
35,3%
21,6%
Outline
Outline
Introduzione



Publications
Publications
Publications
Libri Editati
[1] L. Tarantino and T. Di Mascio. Atti di HCItaly Symposium 2005, September 13 2005, Rome (Italy). Finito di stampare in
Luglio 2005. Studio Grafico Pierpaolo Ceccarelli.
Riviste Internazionali
[2] D. Frigioni, L. Tarantino, T. Di Mascio. On Formal Aspects of Zooming in Geographic Maps. Informatica, Vol. 26, n. 3, pp.
309319, 2002. Slovene Society Informatika.
[3] L. Tarantino and T. Di Mascio. Advances Visual Interfaces: the Focus is on the User. The Knowledge Engineering
Review, Vol. 18:2, pp. 175181, 2003. Cambridge University Press, printed in the United Kingdom.
Conferenze Internazionali
[4] T. Di Mascio and L. Tarantino. Main Features of a CBIR prototype supporting cartoon production. In Proceedings of the
10th Human Computer Interaction International Conference (HCI2003), June 2227 2003, Crete (Greece), Vol. 1, pp. 921
925, 2003. Lawrence Erlbaum Associates.
[5] T. Di Mascio. Assisting User of Semantic Web: the Sewasie Proposal. In Proceedings of the 10th Human Computer
Interaction International Conference (HCI2003), June 2227 2003, Crete (Greece), Adjunct Proceedings, pp. 157159,
2003. Lawrence Erlbaum Associates.
[6] T. Catarci, T. Di Mascio, E. Franconi, G. Santucci and S. Tessaris. An Ontology-based Visual Interface for query
generation support. In Proceedings of International Conference on Ontologies, Databases and Applications of SEmantics
(ODBASE2003), November 0307 2003, Catania (Italy), Adjunct Proceedings, pp. 3234, 2003. Springer-Verlag.
[7] T. Di Mascio, M. Francesconi, D. Frigioni, and L. Tarantino. Tuning a CBIR System for vector images: the interface
support. In Proceedings of the Advanced Visual Interface (AVI2004), May 2528 2004, Gallipoli (Italy), pp. 425428,
2004. ACM Press.
[8] T. Catarci, P. Dongilli, T. Di Mascio, E. Franconi, G. Santucci and S. Tessaris. An Ontology based visual tool for query
formulation support. In Proceedings of European Conference on Artificial Intelligence (ECAI2004), August 2427 2004,
Valencia (Spain), pp. 308312, 2004. IOS Press.
[9] T. Di Mascio, D. Frigioni, and L. Tarantino. A visual environment for tuning a contentbased nvector image retrieval
engine. In Proceedings of the 11th Human Computer Interaction International Conference (HCI2005), June 2227 2005,
Las Vegas (USA), Adjunct Proceedings, 2005. Lawrence Erlbaum Associates.
[10] T. Catarci, T. Di Mascio, P. Dongilli, E. Franconi, G. Santucci, and S. Tessaris. Usability evaluation in the SEWASIE
(SEmantic Webs and AgentS in Integrated Economies) project. In Proceedings of the 11th Human Computer Interaction
International Conference (HCI2005), June 2227 2005, Las Vegas (USA), Vol. 2, 2005. Lawrence Erlbaum Associates.
[11] T. Di Mascio, I. Salvatore, and L. Tarantino. Navigation by zooming in boxes: preliminary evaluation. In
Proceedings of the Advanced Visual Interface (AVI2006), May 2326 2006, Venice (Italy), pp. 262265. ACM
Press.
Publications
Publications
[12] R. Gennari and T. Di Mascio. The e-LIS Project: an Ontology based Approach to an Electronic Dictionary for
Italian Sign Language. In Proceedings of the second Workshop on Formal Ontology (FOMI2006), December 1415
2006, Trento (Italy). IOS Press.
Conferenze Nazionali
[13] T. Catarci, T. Di Mascio, P. Dongilli, E. Franconi, G. Santucci, and S. Tessaris. An Ontology- Based Query Manager:
Usability Evaluation Proceedings of the 4th Italian Symposium on Human Computer Interaction (HCItaly2005), September
13 2003, Roma (Italy), pp.95100.
Rapporti Tecnici di Progetto
[14] E. Franconi, S. Tessaris, T. Catarci, T. Di Mascio, G. Santucci, and G. Vetere. Specification of the tool for the design,
management and storage of the Semantic Information related to the data stored. Technical Report D6.2 of Sewasie
project (IST FP5 Programme-IST-2001-34825), February 2003.
[15] S. Tessaris, R. Bernardi, T. Catarci, T. Di Mascio, E. Franconi, G. Santucci, and G. Vetere. Specification of the tool for
end-user query management. Technical Report D6.3 of Sewasie project (IST FP5 Programme-IST-2001-34825), March
2003.
[16] T. Di Mascio, T. Catarci, G. Santucci, E. Franconi, S. Tessaris, and G. Vetere. Evaluation of existing tools, description
of the user centered design. Technical Report D6.1 of Sewasie project (IST FP5 Programme-IST-2001-34825), May 2003.
[17] D. Montanari, A. Tavernari, T. Di Mascio, A. Jertila, S. Tessaris, H. Roberts, A. Becks and A. Brunner. First tests and
analysys of test results. Technical Report D9.3 of Sewasie project (IST FP5 Programme-IST-2001-34825), May 2004.
[18] A. Tavernari, D. Montanari, T. Di Mascio, A. Jertila, R. Offermanns, A. Becks and C. Seeling. Second tests and analysys
of test results. Technical Report D9.4 of Sewasie project (IST FP5 Programme-IST-2001-34825), February 2005.
[19] T. Di Mascio, T. Catarci, G. Santucci, D. Montanari, A. Tavernari, A. Jertila, E. Franconi, P. Dongilli, J. Heguiabehere
and A. Becks. Evaluation of the system usability. Technical Report D9.5 of Sewasie project (IST FP5 Programme-IST2001-34825), March 2005.
Lavori sottomessi a Riviste Internazionali
[20] T. Di Mascio, D. Frigioni, and L. Tarantino. A CBIR system for vector images: the tuning interface.
[21] T. Di Mascio, S. Tessaris, P. Dongilli, T. Catarci, G. Santucci and E. Franconi. Browsing and Querying in the
Sewasie Environment.
Lavori sottomessi Conferenze Internazionali
[22] T. Di Mascio, L. Laura, V. Mirabella, and L. Tarantino. The interface of VISTO: a CBIR system for vector
images.
[23] G. Marcelli, T. Di Mascio MDSS Discovery Support System.
The End
Grazie!
tania@ing.univaq.it

Di Mascio Bozen 22

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Di Mascio Bozen 22

Uploaded by

Copyright:

Available Formats

Cross Language Information Technologies

Free University of Bozen, Italy

Multi-media Retrieval Systems

Tania Di Mascio, Eurac Research

Bozen (Italy), 22 december

Multi-media Retrieval Systems [20,21]

Human Computer Interaction [1,3]

Vector Images Visualization [2,9,22]

Ontology Visualization [5,6]

Usability Evaluation for software system [10,13]

Usability Evaluation for web site [11]

Tania Di Mascio, Eurac Research

Bozen (Italy), 22 december

Ontology Based Text Retrieval Systems

Bozen (Italy), 22 december

Multi-media Information Retrieval Systems [20, 21]

Bozen (Italy), 22 december

Human Computer Interaction [1, 3]

Vector images Visualization [2, 9, 22]

Bozen (Italy), 22 december

Multi-media Retrieval Systems [20,21]

Human Computer Interaction [1,3]

Vector Images Visualization [2,9,22]

Ontology Visualization [5,6]

Usability Evaluation for software system [10,13]

Usability Evaluation for web site [11]

Tania Di Mascio, Eurac Research

Bozen (Italy), 22 december

Content-Based Image Retrieval Systems

Tania Di Mascio, Eurac Research

Content-Based Image Retrieval Systems

Mathematic and Physic approach to solve the Retrieval Issue

Bozen (Italy), 22 december

Content-Based Image Retrieval Systems

Feature Extraction Module

Tania Di Mascio, Eurac Research

Bozen (Italy), 22 december

Content-Based Image Retrieval Systems

Feature Representation Module

Bozen (Italy), 22 december

Content-Based Image Retrieval Systems

Similarity Computation Module

Tania Di Mascio, Eurac Research

Bozen (Italy), 22 december

Content-Based Image Retrieval Systems

Tania Di Mascio, Eurac Research

Bozen (Italy), 22 december

Content-Based Image Retrieval Systems

Tania Di Mascio, Eurac Research

Bozen (Italy), 22 december

Ontology-Based Text Retrieval Systems

Sewasie Interaction Layer

Sewasie Core System

Tania Di Mascio, Eurac Research

Bozen (Italy), 22 december

Ontology-Based Text Retrieval Systems

The intelligence of the Sewasie Query tool is driven by an

describes the domain of the

the System perspective

enables only the

Bozen (Italy), 22 december

Ontology-Based Text Retrieval Systems

Tania Di Mascio, Eurac Research