Copyright © 2007 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark and document use rules apply.
After reading this document, readers may turn to separate living documents discussing individual multimedia annotation vocabularies, and other relevant tools and resources.
This document targets the developers and researchers in multimedia semantics. It describes the four current OWL/RDF proposals of MPEG-7, as well as a comparison of the different modeling approaches in the context of practical applications. Any harmonizations aiming at providing a single ontology of MPEG-7 will not be done in the framework of this XG.
This section describes the status of this document at the time of its publication.
Other documents may supersede this document. A list of Final Incubator
Group Reports is available. See also the W3C technical reports
index at http://www.w3.org/TR/.
This document was developed by the W3C Multimedia Semantics Incubator Group.
Publication of this document by W3C as part of the W3C Incubator Activity indicates no endorsement of its content by W3C, nor that W3C has, is, or will be allocating any resources to the issues addressed by it. Participation in Incubator Groups and publication of Incubator Group Reports at the W3C site are benefits of W3C Membership.
Incubator Groups have as a goal to produce work that can be implemented on a Royalty Free basis, as defined in the W3C Patent Policy. Participants in this Incubator Group have made no statements about whether they will offer licenses according to the licensing requirements of the W3C Patent Policy for portions of this Incubator Group Report that are subsequently incorporated in a W3C Recommendation.
Discussion of this document is invited on the XG public mailing list public-xg-mmsem@w3.org (public archives). Public comments should include "comments: [MPEG-7]" at the start of the Subject header.
MPEG-7 was developed to provide standardized tools for describing different aspects of multimedia at different levels of abstraction. Its XML-based syntax enables smooth interchange across applications and over the web, but the lack of precise semantics hinders metadata interoperability. Two representative examples include:
<FreeTextAnnotation xml:lang="en">Zinedine Zidane scoring against England.</FreeTextAnnotation>
<KeywordAnnotation xml:lang="en"> <Keyword>Zinedine</Keyword> <Keyword>Zidan</Keyword> <Keyword>scoring</Keyword> <Keyword>England</Keyword> <Keyword>goal</Keyword> </KeywordAnnotation>
<StructuredAnnotation> <Who> <Name xml:lang="en">Zinedine Zidane</Name> </Who> <WhatAction> <Name xml:lang="en">Zinedine Zidane scoring against England.</Name> </WhatAction> </StructuredAnnotation>
<Semantic id="FormalAbstractionDescription"> <SemanticBase xsi:type="AgentObjectType" id="Zidane"> <Label><Name>Zidane </Name></Label> <Agent xsi: ="PersonType"> <Name> <GivenName>Zinedine</GivenName> <FamilyName>Zidane</FamilyName> </Name> </Agent> </SemanticBase> <SemanticBase xsi:type="EventType" id="scoring"> <Label> <Name>Zinedine Zidane scoring against England.</Name> </Label> </SemanticBase> </Semantic>
To alleviate the resulting interoperability issues, efforts have been undertaken to translate MPEG-7 into an ontology and through appropriate frameworks to enable its integration with other ontologies, thus enhancing interoperability. Two main such methodologies include the proposals by Hunter et. al. and Tsinaraki et. al. Both approaches aim to provide a framework for interoperable MPEG-7 compliant multimedia metadata. However, given the continuously growing research interest in formalizing multimedia related semantics and building a common metadata framework, the question of how interoperable these proposals are becomes particularly important.
In the approach proposed by Hunter, the ABC ontology is used as the core one to provide attachment points for integrating mpeg7 and domain specific ontologies. More specifically, the mpeg7:MultimediaContent class (and the subsequent multimedia and segment hierarchy) is defined as a subclass of the abc:Manifestation class, while the corresponding domain ontologies are assumed to be appropriately attached to corresponding ABC classes.
A first observation at this point would be that MPEG-7 includes apart from the structure related description schemes, descriptions on other aspects as well (e.g., the semantic part ones), for which it is not clear how the mapping to ABC should be and how they relate to possibly relevant domain specific definitions. For example, mpeg7:Agent could be mapped to abc:Agent. Assuming a domain specific class o:Person it should be again linked to abc:Agent as equivalent class, subclass or through some property, thus raising issues about the semantics of the mpeg7:Agent and o:Person relation, which in turn reduces interoperability among possible pre-existing MPEG-7 based annotation metadata and newly created ones under the ABC core ontology framework.
Let assume that someone follows the approach by Hunter, using the Multimedia Description Scheme (MDS) part of the MPEG-7 ontology to address the structural aspects, in order to annotate an image depicting Zidane scoring. Assuming a soccer ontology s, the involved classes would be s:goal, s:player, s:scoring and mpeg7:image (at least in a simple case where spatiotemporal decomposition is not taken into account). One possible way to represent this annotation would be using the following statements:
:image01 rdf:type mpeg7:Image :goal01 rdf:type s:Goal :scoring01 rdf:type s:Scoring :image01 mpeg7:depicts :goal01 :goal01 abc:hasAction :scoring01 :scoring01 abc:hasAgent s:_b1 :_b1 :hasName 'Zinedine Zidane'
where additionally the following hold:
mpeg7:Image rdfs:subclass mpeg7:MultimediaContent mpeg7:MultimediaContent rdfs:subclass abc:Manifestation s:Scoring rdfs:subclass abc:Action s:Goal rdfs:subclass abc:Event
Notice that under this framework, having attached this annotation to a specific image region rather than the whole image, i.e.
:region01 rdf:type mpeg7:StillRegion :region01 mpeg7:depicts :goal01
we would be able to retrieve the corresponding image if querying for
images depicting Zinedine Zidane scoring, due to the subclass relation
mpeg7:StillRegion rdfs:subclass mpeg7:Image,
something that is not inherently possible by MPEG-7 itself.
Leaving out individual issues regarding the taken modeling decisions (e.g., should still regions be modeled as a subclass of image or related to the latter through partonomic decomposition relations only), the one sees evidence for the value of using an upper ontology, adequately generic to allow the consistent integration between an MPEG-7 ontology and domain specific ones.
In Tsinaraki on the other hand, the semantic part of MPEG-7 is translated into an ontology that serves as the core one for the attachment of domain specific ontologies, in order to achieve MPEG-7 compliant domain specific annotations. A first observation is that under this approach the initial conceptualization of the domain specific ontologies needs to be "mapped" to the MPEG-7 modeling rationale. Consequently, annotation metadata produced following this approach would not be interoperable with approaches coupling domain specific ontologies with an MPEG-7-like one, following a procedure similar to the one proposed by Hunter.
In this section we will present the possible solutions for the interoperability problems that arise from the different translations/formalisations of the MPEG-7 standard. The specific interoperability problems have been illustrated in the motivating example. There are three approaches in the literature that try to overcome such interoperability problems. These approaches are:
TODO: Michael to describe syntactic (XML, XML-Schema) and semantic (RDF/OWL/rules) aspects.
The aim of this section is not to present the analytical solutions but rather the mechanism to ensure interoperability in MPEG-7 based MM applications. In addition, we will present the interoperability problems that are solved and the new ones that are introduced.
$Id: Overview.html,v 1.3 2007/08/15 00:47:45 rtroncy Exp $