W3C

API for Media Resource 1.0

W3C Working Draft 08 June 2010

This version:
http://www.w3.org/TR/2010/WD-mediaont-api-1.0-20100608
Latest version:
http://www.w3.org/TR/mediaont-api-1.0
Previous version:
http://www.w3.org/TR/2010/WD-mediaont-api-1.0-20100309
Editors:
이원석(WonSuk Lee), Electronics and Telecommunications Research Institute (ETRI)
Florian Stegmaier, University of Passau
Chris Poppe, Ghent University

Abstract

This specification defines a client-side API to access metadata information related to media resources on the Web. The overall purpose of the API is to provide developers with a convenient access to metadata information stored in different metadata formats. The API is described using the interface definition language Web IDL. Thereby, the Media Ontology Core Properties will be used as a pivot vocabulary in the API.

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This is the a Last Call Working Draft of the API for Media Resource 1.0 specification. It has been produced by the Media Annotations Working Group, which is part of the W3C Video on the Web Activity.

The W3C Membership and other interested parties are invited to review the document and send comments through 11 July 2010. Comments must be sent to to public-media-annotation@w3.org mailing list (public archive). Use "[LC Comment API]" in the subject line of your email.

Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of Contents

1. Introduction

This section is non-normative.

This specification defines a client-side API to access metadata information related to media resources on the Web. The overall purpose of the API is to provide developers with a convenient access to metadata information stored in different metadata formats. Thereby, the Media Ontology Core Properties will be used as a pivot vocabulary in the API. The description of relations between these core properties and the metadata formats in scope (1.2 Formats in scope) are stored in the Media Ontology in order to provide cross-community data integration. The API is described using the interface definition language [WEBIDL]. The decision to use Web IDL, which offers bindings for ECMAScript and Java, can be based on the Use Cases and Requirements for Ontology and API for Media Resource 1.0 [MEDIA-ANNOT-REQS]. This document clearly states that the focus for this API lies on multimedia services on the Web.

The API serves as a mediator between a developer and the underlying Ontology for Media Resource 1.0 [MEDIA-ANNOT-ONTOLOGY] with the goal to support interoperability between metadata formats. It offers operations to retrieve particular metadata informations represented in a certain metadata format related to media resources on the Web.

This document is being published with the aspiration to gather wide feedback on the yet available API design.

1.1 Formats in scope

Refers to the formats in scope of Ontology for Media Resource 1.0.

1.2 Formats out of scope

Refers to the Formats out of scope of Ontology for Media Resource 1.0.

1.3 Terminology

In this document the terms "Media Resource", "Property", "Mapping" and "Property value type" are to be interpreted as defined in Section 2 of Ontology for Media Resource 1.0.

2. Design consideration

We consider two scenarios where the API could be implemented: either in a user agent (scenario 1) or as a web service (scenario 2). The two scenarios are shown in the figure.

Scenario 1: User agent
The API is implemented in a user agent (e.g., browser or browser plugin) and exposed as a JavaScript API (using the WebIDL JavaScript binding). The user agent includes the components for metadata access (possibly extraction) and mappings for a supported set of formats. The metadata sources (the media resource and/or metadata document(s)) must be retrievable and access (establish connection, retrieval) of the metadata sources is handled by the user agent.
Scenario 2: Web service
The API is implemented as a Web service. Such an implementation would be typically used by a non-UI client, such as an agent harvesting metadata. However, the API could be also accessed from a user agent, and used the same way as described in scenario 1 by the help of a JavaScript library for accessing the web service. At the back-end of the web service, this scenario also allows supporting a media repository (e.g. content provider's archive database, movie store) from which the user agent could directly retrieve metadata sources and which might have a custom metadata format not supported by a user agent. In contrast to an integrated component (see scenario 1), an implementation of the API in a web service could do more complex mappings on the fly as a component integrated in a user agent, and can be more flexible (e.g., supporting additional formats).

Overview of  different API options.

In both scenarios, the access to the metadata properties needs the following stack of components:

3. API Description

The JavaScript examples in this document will only work if the API is implemented by the browser.

The API exists of a number of interfaces, described using Web IDL. Implementations of the API should provide objects implementing the different interfaces. The entire description can be found in Appendix A. The API is contained within the MediaResource interface within the mawg module. Objects implementing this interface provide the necessary methods to access metadata properties of a Media Resource. The object holds methods to identify the actual Media Resource and the metadata sources. All properties can be accessed through a specific operation getProperty. When an attempt to read a property fails, diagnostics information can be obtained using a diagnosis operation. Subtypes in the API are relevant for those properties mentioned in 4.1.3 Core properties of Ontology for Media Resource 1.0. Lastly, methods are available that allow to iterate through the available metadata. Next, the different interfaces and exposed methods are discussed. Finally, examples of the usage of the API can be found in section 4.

3.1 MediaResource interface

The MediaResource interface offers a number of operations that allow accessing the metadata of a Media Resource.

Example on how to introduce this in HTML5 by making the HTMLMediaElement inherit from the MediaResource interface:

interface HTMLMediaElement : MediaResource, HTMLElement {...}

Example of usage in JavaScript:

mediaResource = document.getElementsByTagName("video")[0];
//mediaResource implements the HTMLMediaElement interface and the MediaResource interface
title = mediaResource.getProperty("title");
...

As shown, the getElementsByTagName returns a HTMLMediaElement which inherits from the MediaResource interface. The getProperty method is part of this interface so the returned element has an implementation of this method. By calling the getProperty method with the argument "title" we can retrieve the title of the corresponding media resource.

The JavaScript examples in this section all assume that getElementsByTagName() returns an object implementing the MediaResource interface.

Next, we give the Web IDL description of the MediaResource interface and describe the different operations that are part of it.

[NoInterfaceObject]
interface MediaResource {
    boolean     selectMAResource (in DOMString mediaResource, in optional MetadataSource[] metadataSources);
    MAObject[]  getProperty (in DOMString propertyName, in optional DOMString fragment, in optional DOMString sourceFormat, in optional DOMString subtype, in optional DOMString language);
    DOMString[] getPropertyNamesWithValues (in optional DOMString sourceFormat, in optional DOMString language, in optional DOMString fragment);
    DOMString[] getSourceFormatsWithValues (in optional DOMString language);
    DOMString   getDiagnosis ();
    DOMString[] getOriginalData (in DOMString sourceFormat);
};

3.1.1 Methods

getDiagnosis

This operation allows to retrieve the status code(e.g., the getProperty operation returning a null value). See Section 4 for details.

No parameters.
No exceptions.
Return type: DOMString
getOriginalData
This operation allows to retrieve the original metadata according to the specified source format.
Parameter Type Nullable Optional Description
sourceFormat DOMString This argument identifies a specific metadata format. If a metadata format is defined, only the metadata available in the corresponding metadata format are retrieved. This parameter is optional.
No exceptions.
Return type: DOMString[]
getProperty
This operation allows to retrieve the value of a certain property. The specific property is passed as an argument and a list of objects is returned that hold the values according to the requested property. These objects implement the MAObject interface, described in (MAObject interface). Depending on the requested property, the returned objects implement a different sub-interface (inheriting from the MAObject interface). Requesting for the title gives back an array of MAObjects implementing the Title interface, requesting the creator results in MAObjects implementing the Creator interface and so on. These interfaces are described in section 3.5 to 3.12.
Parameter Type Nullable Optional Description
propertyName DOMString This argument identifies the property for which the values need to be retrieved. Optional arguments allow to refine the request
fragment DOMString This argument allows to identify the specific media fragment for which the metadata is requested. This parameter is optional.
sourceFormat DOMString This argument identifies a specific metadata format. If a metadata format is defined, only the metadata available in the corresponding metadata format are retrieved. This parameter is optional.
subtype DOMString This argument identifies a subtype. Some properties can be filtered on subtypes. This parameter is optional.
language DOMString This argument allows to identify the language of the metadata. Only if the metadata is available in the specified language, the values are returned. This parameter is optional.
No exceptions.
Return type: MAObject[]
getPropertyNamesWithValues
This operation allows to retrieve all property names for which metadata is available. If this operation returns a property name, this means that a call to the getProperty operation with the corresponding property name will return at least one object.
Parameter Type Nullable Optional Description
sourceFormat DOMString This argument identifies a specific metadata format. If a metadata format is defined, only the metadata available in the corresponding metadata format are retrieved. This parameter is optional.
language DOMString This argument allows to identify the language of the metadata. Only if the metadata is available in the specified language, the values are returned. This parameter is optional.
fragment DOMString This argument allows to identify the specific media fragment for which the metadata is requested. This parameter is optional.
No exceptions.
Return type: DOMString[]
getSourceFormatsWithValues
This operation allows to retrieve all source formats for which metadata is available. If this operation returns a source format, this means that a call to the getProperty operation with the corresponding source format will return at least one object.
Parameter Type Nullable Optional Description
language DOMString This argument allows to identify the language of the metadata. Only if the metadata is available in the specified language, the values are returned. This parameter is optional.
No exceptions.
Return type: DOMString[]
selectMAResource
This operation allows to set the specific Media Resource and metadata sources for which the API applies.
Parameter Type Nullable Optional Description
mediaResource DOMString The mediaResource argument identifies the Media Resource. The implementation of the API should try to find relevant metadata sources for this Media Resource
metadataSources MetadataSource[] This argument is an array of objects, each implementing the MetadataSource interface. This interface holds an uri identifying the metadata source (metadataSource) and the name of the actual metadata format (sourceFormat). This parameter is optional.
No exceptions.
Return type: boolean

3.2 MetadataSource interface

MetadataSource interface is used to identify other metadata sources.

[NoInterfaceObject]
interface MetadataSource {
    attribute DOMString metadataSource;
    attribute DOMString sourceFormat;
};

3.2.1 Attributes

metadataSource of type DOMString
An URI identifying the metadata source.
No exceptions.
sourceFormat of type DOMString
The name of the actual metadata format.
No exceptions.

3.3 Language interface

Language interface is used to identify the language of the metadata.

[NoInterfaceObject]
interface Language {
    attribute DOMString language;
};

3.3.1 Attributes

language of type DOMString
This attribute holds the language of the metadata.
No exceptions.

3.4 MAObject interface

MAObject interface is used as the return type of MediaResource.getProperty method. This is used to hold the values of the requested property. Depending on the requested property, these objects implement a different interface.

[NoInterfaceObject]
interface MAObject {
    attribute DOMString unstructuredValue;
    attribute DOMString uri;
    attribute DOMString sourceFormat;
    attribute DOMString fragmentIdentifier;
    attribute DOMString mappingType;
};

3.4.1 Attributes

fragmentIdentifier of type DOMString
This attribute determines the fragment for which the metadata is relevant.
No exceptions.
mappingType of type DOMString
This attribute specifies the kind of mapping as discussed in the semantic level mappings.
No exceptions.
sourceFormat of type DOMString
This attribute allows to specify the metadata source from which the metadata was retrieved.
No exceptions.
unstructuredValue of type DOMString
If it is not clear how a certain value for a property should be structured, this attribute allows to describe the value in plain text.
No exceptions.
uri of type DOMString
This attribute is provided which should hold an URI. This can for example be used to represent labels as URIs.
No exceptions.

3.5 Identification Properties

3.5.1 Identifier interface

Identifier interface is used as the specific return type of MediaResource.getProperty method which has "Identifier" as a value of propertyName parameter.

[NoInterfaceObject]
interface Identifier : MAObject {
    attribute DOMString value;
    attribute DOMString type;
};
3.5.1.1 Attributes
type of type DOMString
This attribute holds the type of the identifier, which can be filtered on in the getProperty operation. Possible values are " UMID" and "ISAN".
No exceptions.
value of type DOMString
This attribute should represent a URI identifying the resource.
No exceptions.

Example in JavaScript is as below:

image = document.getElementsByTagName("img")[0];
id = image.getProperty("identifier");

/*Resulting in:
id[0].value = "http://www.w3c.org/2008/WebVideo/Annotations/wiki/Image:MAWG-Stockholm-20090626.JPG"
id[0].type = "URI"
id[1].value= "ISAN 0000-3BAB-9352-0000-G-0000-0000-Q"
id[1].type="ISAN"
id[2].value= "urn:uuid:36a87260-1102-11df-8a39-0800200c9a66"
id[2].type="UUID"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=identifier

  • Response (JSON format): { "identifier" : [ {"value" : "http://www.w3c.org/2008/WebVideo/Annotations/wiki/Image:MAWG-Stockholm-20090626.JPG", "type" : "URI" } , { "value" : "ISAN 0000-3BAB-9352-0000-G-0000-0000-Q", "type" : "ISAN" }, { "value" : "urn:uuid:36a87260-1102-11df-8a39-0800200c9a66", "type" : "UUID" } ] }

3.5.2 Title interface

Title interface is used as the specific return type of MediaResource.getProperty method which has "Title" as a value of propertyName parameter.

[NoInterfaceObject]
interface Title : MAObject, Language {
    attribute DOMString value;
    attribute DOMString type;
};
3.5.2.1 Attributes
type of type DOMString
This attribute contains the type of the title, which can be filtered on in the getProperty operation.
No exceptions.
value of type DOMString
This attribute should hold the title as a plain string.
No exceptions.

Example in JavaScript is as below:

song = document.getElementsByTagName("audio")[0];
title = song.getProperty("title");

/*Resulting in:
title[0].value = "Artificial Horizon"
title[0].type = "Album title"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=title

  • Response (JSON format): { "title" : { "value" : "Artificial Horizon" , "type" : "Album title" } }

3.5.3 Language interface

Language interface is used as the specific return type of MediaResource.getProperty method which has "Language" as a value of propertyName parameter. Recommended best practice is to use BCP 47 [BCP-47].

[NoInterfaceObject]
interface Language : MAObject {
    attribute DOMString value;
};
3.5.3.1 Attributes
value of type DOMString
This attribute should represent the language as a plain string.
No exceptions.

Example in JavaScript is as below:

video = document.getElementsByTagName("video")[0];
language = video.getProperty("language");

/*Resulting in:
language[0].value = "en-us"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=language

  • Response (JSON format): { "language" : { "value" : "en-us" } }

3.5.4 Locator interface

Locator interface is used as the specific return type of MediaResource.getProperty method which has "Locator" as a value of propertyName parameter.

[NoInterfaceObject]
interface Locator : MAObject {
    attribute DOMString value;
};
3.5.4.1 Attributes
value of type DOMString
This attribute should hold the URI.
No exceptions.

Example in JavaScript is as below:

image = document.getElementsByTagName("img")[0];
locator = image.getProperty("locator");

/*Resulting in:
locator[0].value = "http://www.w3.org/2008/WebVideo/Annotations/wiki/images/9/93/MAWG-Stockholm-20090626.JPG"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=locator

  • Response (JSON format): { "locator" : { "value" : "http://www.w3.org/2008/WebVideo/Annotations/wiki/images/9/93/MAWG-Stockholm-20090626.JPG" } }

3.6 Creation Properties

3.6.1 Contributor interface

Contributor interface is used as the specific return type of MediaResource.getProperty method which has "Contributor" as a value of propertyName parameter.

[NoInterfaceObject]
interface Contributor : MAObject {
    attribute DOMString id;
    attribute DOMString role;
};
3.6.1.1 Attributes
id of type DOMString
This attribute identifies the contributor.
No exceptions.
role of type DOMString
This attribute defines the role, which can be filtered on in the getProperty operation. For the latter a number of suggested terms are defined:
  • editor (EBU 11.1)

  • actor (EBU 25.9)

  • composer

  • featured_in

  • cinematographer

  • director

  • musicproducer

  • producer

  • screenplayer

  • writer

  • distributor (company)

  • production company

No exceptions.

Example in JavaScript is as below:

video = document.getElementsByTagName("video")[0];
contributor = video.getProperty("contributor");

/*Resulting in:
contributor[0].id = "http://individuals.example.com/Contributor1"
contributor[0].role = "editor"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=contributors

  • Response (JSON format): { "contributors" : { "id" : "http://individuals.example.com/Contributor1", "role" : "editor" } }

3.6.2 Creator interface

Creator interface is used as the specific return type of MediaResource.getProperty method which has "Creator" as a value of propertyName parameter.

[NoInterfaceObject]
interface Creator : MAObject {
    attribute DOMString value;
};
3.6.2.1 Attributes
value of type DOMString
This attribute should hold an identifier for an author.
No exceptions.

Example in JavaScript is as below:

video = document.getElementsByTagName("video")[0];
creator = video.getProperty("creator");

/*Resulting in:
creator[0].value = "http://individuals.example.com/Author1"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=creator

  • Response (JSON format): { "creator" : { "value" : "http://individuals.example.com/Author1" } }

3.6.3 CreateDate interface

CreateDate interface is used as the specific return type of MediaResource.getProperty method which has "CreateDate" as a value of propertyName parameter.

[NoInterfaceObject]
interface Date : MAObject {
    attribute DOMString date;
    attribute DOMString type;
};
3.6.3.1 Attributes
date of type DOMString
The exact format of this attribute is currently undefined.
No exceptions.
type of type DOMString
This attribute define the particular category of creation date (e.g. "release date", "date recorded" and "date edited".), which can be filtered on in the getProperty operation.
No exceptions.

Example in JavaScript is as below:

video = document.getElementsByTagName("video")[0];
createdate = video.getProperty("createdate");

/*Resulting in:
createdate[0].date = "2009-06-26T15:30:00"
createdate[0].type = "date recorded"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=createDate

  • Response (JSON format): { "createDate" : { "date": "2009-06-26T15:30:00", "type" : "date recorded" } }

3.6.4 Location interface

Location interface is used as the specific return type of MediaResource.getProperty method which has "Location" as a value of propertyName parameter.

[NoInterfaceObject]
interface Location : MAObject, Language {
    attribute DOMString name;
    attribute Float     longitude;
    attribute Float     latitude;
    attribute Float     altitude;
    attribute DOMString system;
};
3.6.4.1 Attributes
altitude of type Float
This attribute holds the altitude value from system defined in system.
No exceptions.
latitude of type Float
This attribute holds the latitude value from system defined in system.
No exceptions.
longitude of type Float
This attribute holds the longitude value from system defined in system.
No exceptions.
name of type DOMString
This attribute holds a free text name of the location.
No exceptions.
system of type DOMString
This attribute holds the name of location system.
No exceptions.

Example in JavaScript is as below:

video = document.getElementsByTagName("video")[0];
location = video.getProperty("location");

/*Resulting in:
location[0].name = "San Jose"
location[0].longitude = 37.33986481118008
location[0].latitude = -121.88507080078125
location[0].altitude = 0
location[0].system = "GPS"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=location

  • Response (JSON format): { "location" : { "name" : "San Jose", "longitude" : 37.33986481118008, "latitude" : -121.88507080078125, "altitude" : 0, system : "GPS" } }

3.7 Content Properties

3.7.1 Description interface

Description interface is used as the specific return type of MediaResource.getProperty method which has "Description" as a value of propertyName parameter.

[NoInterfaceObject]
interface Description : MAObject, Language {
    attribute DOMString value;
};
3.7.1.1 Attributes
value of type DOMString
This attribute should hold a description of the content of the media resource.
No exceptions.
image = document.getElementsByTagName("img")[0];
description = image.getProperty("description");

/*Resulting in:
description[0].value = "Group picture of the W3C Media Annotations WG at the face-to-face meeting in Stockholm."
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=description

  • Response (JSON format): { "description" : { "value" : "Group picture of the W3C Media Annotations WG at the face-to-face meeting in Stockholm." } }

3.7.2 Keyword interface

Keyword interface is used as the specific return type of MediaResource.getProperty method which has "Keyword" as a value of propertyName parameter.

[NoInterfaceObject]
interface Keyword : MAObject, Language {
    attribute DOMString value;
};
3.7.2.1 Attributes
value of type DOMString
This attribute should hold one keyword.
No exceptions.
image = document.getElementsByTagName("img")[0];
keyword = image.getProperty("keyword");

/*Resulting in:
keyword[0].value = "W3C Media Annotations WG"
keyword[1].value = "meeting"
keyword[2].value = "group picture"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=keyword

  • Response (JSON format): { "keyword" : [ { "value" : "W3C Media Annotations WG" }, { "value" : "meeting" }, { "value" : "group picture" } ] }

3.7.3 Genre interface

Genre interface is used as the specific return type of MediaResource.getProperty method which has "Genre" as a value of propertyName parameter.

[NoInterfaceObject]
interface Genre : MAObject, Language {
    attribute DOMString value;
};
3.7.3.1 Attributes
value of type DOMString
This attribute should represent the genre of the media resource.
No exceptions.
image = document.getElementsByTagName("img")[0];
genre = image.getProperty("genre");

/*Resulting in:
genre[0].value = "Work"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=genre

  • Response (JSON format): { "genre" : { "value" : "Work" } }

3.7.4 Rating interface

Rating interface is used as the specific return type of MediaResource.getProperty method which has "Rating" as a value of propertyName parameter.

[NoInterfaceObject]
interface Rating : MAObject, Language {
    attribute DOMString issuer;
    attribute short     value;
    attribute short     minimum;
    attribute short     maximum;
    attribute DOMString type;
};
3.7.4.1 Attributes
issuer of type DOMString
This attribute identifies the rating person or organization
No exceptions.
maximum of type short
This attribute specifies the maximum rating.
No exceptions.
minimum of type short
This attribute specifies the minimum rating.
No exceptions.
type of type DOMString
This attribute specifies the type of the rating, which can be filtered on in the getProperty operation. Examples are "Review Rating", "MPAA", "Personal Rating".
No exceptions.
value of type short
This attribute indicates the average rating.
No exceptions.
image = document.getElementsByTagName("img")[0];
rating = image.getProperty("rating");

/*Resulting in:
rating[0].issuer = "http://individuals.example.com/ChrisPoppe"
rating[0].value = 10.0
rating[0].minimum = 0
rating[0].maximum = 10.0
rating[0].type = "Personal Rating"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=rating

  • Response (JSON format): "rating" : { "rating" : { "issuer" : "http://individuals.example.com/ChrisPoppe", "value" : 10.0, "minimum" : 0, "maximum" : 10.0, "type" : "Personal Rating" } }

3.8 Relational Properties

3.8.1 Relation interface

Relation interface is used as the specific return type of MediaResource.getProperty method which has "Relation" as a value of propertyName parameter.

[NoInterfaceObject]
interface Relation : MAObject {
    attribute DOMString id;
    attribute DOMString relationship;
};
3.8.1.1 Attributes
id of type DOMString
This attribute identifies the entitiy.
No exceptions.
relationship of type DOMString
This attribute specifies textual description of the relationship, which can be filtered on in the getProperty operation. For the latter a number of suggested terms are defined:
No exceptions.
image = document.getElementsByTagName("img")[0];
relation = image.getProperty("relation");

/*Resulting in:
relation[0].id = "http://www.w3.org/2008/WebVideo/Annotations/wiki/Image:MAWG-Stockholm-20090626_thumb.JPG"
relation[0].relationship = "re-edit"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=relation

  • Response (JSON format): { "relation" : { "id" : "http://www.w3.org/2008/WebVideo/Annotations/wiki/Image:MAWG-Stockholm-20090626_thumb.JPG" , "relationship" : "re-edit" } }

3.8.2 Collection interface

Collection interface is used as the specific return type of MediaResource.getProperty method which has "Collection" as a value of propertyName parameter.

[NoInterfaceObject]
interface Collection : MAObject, Language {
    attribute DOMString value;
};
3.8.2.1 Attributes
value of type DOMString
This attribute should hold the name of the collection from which the media resource originates.
No exceptions.
image = document.getElementsByTagName("img")[0];
collection = image.getProperty("collection");

/*Resulting in:
collection[0].value = "My Work Pictures"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=collection

  • Response (JSON format): { "collection" : { "value" : "My Work Pictures" } }

3.9 Rights Properties

3.9.2 Policy interface

Policy interface is used as the specific return type of MediaResource.getProperty method which has "Policy" as a value of propertyName parameter.

[NoInterfaceObject]
interface Policy : MAObject, Language {
    attribute DOMString value;
    attribute DOMString link;
    attribute DOMString organization;
    attribute DOMString type;
};
3.9.2.1 Attributes
This attribute holds a link to the license if it is externally available.
No exceptions.
organization of type DOMString
This attribute identifies the organization that issued the license.
No exceptions.
type of type DOMString
This attribute holds the actual type of the policy, which can be filtered on in the getProperty operation. Examples are "license", "access", and "privacy".
No exceptions.
value of type DOMString
This attribute holds the description of the license.
No exceptions.
image = document.getElementsByTagName("img")[0];
policy = image.getProperty("policy");

/*Resulting in:
policy[0].value = "Attribution 2.5"
policy[0].organization = "http://creativecommons.org/licenses/by/2.5"
policy[0].type = "license"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=policy

  • Response (JSON format): { "policy" : { "value": "Attribution 2.5 ", "organization" : "http://creativecommons.org/licenses/by/2.5", "type": "license" } }

3.10 Distribution Properties

3.10.1 Publisher interface

Publisher interface is used as the specific return type of MediaResource.getProperty method which has "Publisher" as a value of propertyName parameter.

[NoInterfaceObject]
interface Publisher : MAObject {
    attribute DOMString value;
};
3.10.1.1 Attributes
value of type DOMString
This attribute should represent the name of the publisher.
No exceptions.
image = document.getElementsByTagName("img")[0];
publisher = image.getProperty("publisher");

/*Resulting in:
publisher[0].value = "http://individuals.example.com/JohnDoe"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=publisher

  • Response (JSON format): { "publisher" : { "value" : "http://individuals.example.com/JohnDoe" } }

3.10.2 TargetAudience interface

TargetAudience interface is used as the specific return type of MediaResource.getProperty method which has "TargetAudience" as a value of propertyName parameter.

[NoInterfaceObject]
interface TargetAudience : MAObject, Language {
    attribute DOMString issuer;
    attribute DOMString classification;
    attribute DOMString type;
};
3.10.2.1 Attributes
classification of type DOMString
This attribute defines the actual classification.
No exceptions.
issuer of type DOMString
This attribute identifies the issuer of the classification.
No exceptions.
type of type DOMString
This attribute specifies the type of the classification, which can be filtered on in the getProperty operation. Examples are "Age group" and "Geographical".
No exceptions.
image = document.getElementsByTagName("img")[0];
targetAudience = image.getProperty("targetaudience");

/*Resulting in:
targetAudience[0].issuer = "http://www.fosi.org/icra"
targetAudience[0].classification = "no nudity"
targetAudience[0].type = "Age Group"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=targetaudience

  • Response (JSON format): { "targetaudience" : { "issuer" : "http://www.fosi.org/icra", "classification" : "no nudity", "type" : "Age Group" } }

3.11 Fragments Properties

3.11.1 Fragments interface

Fragments interface is used as the specific return type of MediaResource.getProperty method which has "Fragments" as a value of propertyName parameter.

[NoInterfaceObject]
interface Fragments : MAObject {
    attribute DOMString role;
    attribute DOMString id;
};
3.11.1.1 Attributes
id of type DOMString
This attribute identifies the fragment.
No exceptions.
role of type DOMString
This attribute holds the role of the fragment, which can be filtered on in the getProperty operation.
No exceptions.
movie = document.getElementsByTagName("movie")[0];
fragments = movie.getProperty("fragments");

/*Resulting in:
fragments[0].role = "Person"
fragments[0].id = "http://www.example.com/movie.mov#xywh=320,320,40,100"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=fragments

  • Response (JSON format): { "fragments" : { "role" : "Person", "identifier" : "http://www.example.com/movie.mov#xywh=320,320,40,100" } }

3.11.2 NamedFragments interface

NamedFragments interface is used as the specific return type of MediaResource.getProperty method which has "NamedFragments" as a value of propertyName parameter.

[NoInterfaceObject]
interface NamedFragments : MAObject {
    attribute DOMString name;
    attribute DOMString id;
};
3.11.2.1 Attributes
id of type DOMString
This attribute identifies the fragment.
No exceptions.
name of type DOMString
This attribute specifies the name given to the fragment.
No exceptions.
image = document.getElementsByTagName("img")[0];
namedFragments = image.getProperty("NamedFragments");

/*Resulting in:
namedFragments[0].name = "Joakim Söderberg"
namedFragments[0].id = "http://www.w3.org/2008/WebVideo/Annotations/wiki/Image:MAWG-Stockholm-20090626.JPG#xywh=1600,550,80,150"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=namedFragments

  • Response (JSON format): { "namedfragments" : { "name" : "Joakim Söderberg", "identifier" : "http://www.w3.org/2008/WebVideo/Annotations/wiki/Image:MAWG-Stockholm-20090626.JPG#xywh=1600,550,80,150" } }

3.12 Technical Properties

3.12.1 FrameSize interface

FrameSize interface is used as the specific return type of MediaResource.getProperty method which has "FrameSize" as a value of propertyName parameter.

[NoInterfaceObject]
interface FrameSize : MAObject {
    attribute unsigned long width;
    attribute unsigned long height;
    attribute DOMString     unit;
};
3.12.1.1 Attributes
height of type unsigned long
This attribute represents the height of the frame.
No exceptions.
unit of type DOMString
This attribute represents the unit of the frame. The default value is pixels.
No exceptions.
width of type unsigned long
This attribute represents the width of the frame.
No exceptions.
image = document.getElementsByTagName("img")[0];
frameSize = image.getProperty("framesize");

/*Resulting in:
frameSize[0].width = 3.072
frameSize[0].height = 2.304
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=frameSize

  • Response (JSON format): { "framesize" : { "width" : 3.072, "height" : 2.304 } }

3.12.2 Compression interface

Compression interface is used as the specific return type of MediaResource.getProperty method which has "Compression" as a value of propertyName parameter.

[NoInterfaceObject]
interface Compression : MAObject, Language {
    attribute DOMString value;
};
3.12.2.1 Attributes
value of type DOMString
This attribute specifies the compression type as a plain string.
No exceptions.
video = document.getElementsByTagName("video")[0];
compression = video.getProperty("compression");

/*Resulting in:
compression[0].value = "H.264/AVC"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=compression

  • Response (JSON format): { "compression" : { "value" : "H.264/AVC" } }

3.12.3 Duration interface

Duration interface is used as the specific return type of MediaResource.getProperty method which has "Duration" as a value of propertyName parameter.

[NoInterfaceObject]
interface Duration : MAObject {
    attribute unsigned long value;
};
3.12.3.1 Attributes
value of type unsigned long
This attribute should represent the duration (in secs).
No exceptions.
video = document.getElementsByTagName("video")[0];
duration = video.getProperty("duration");

/*Resulting in:
duration[0].value = 3600
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=duration

  • Response (JSON format): { "duration" : { "value" : 3600 } }

3.12.4 Format interface

Format interface is used as the specific return type of MediaResource.getProperty method which has "Format" as a value of propertyName parameter.

[NoInterfaceObject]
interface Format : MAObject {
    attribute DOMString value;
};
3.12.4.1 Attributes
value of type DOMString
This attribute specifies the MIME type as a plain string.
No exceptions.
image = document.getElementsByTagName("img")[0];
format = image.getProperty("format");

/*Resulting in:
format[0].value = "image/jpeg"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=format

  • Response (JSON format): { "format" : { "value" : "image/jpeg" } }

3.12.5 Samplingrate interface

Samplingrate interface is used as the specific return type of MediaResource.getProperty method which has "Samplingrate" as a value of propertyName parameter.

[NoInterfaceObject]
interface Samplingrate : MAObject {
    attribute unsigned long value;
};
3.12.5.1 Attributes
value of type unsigned long
This attribute specifies the samplingrate (in Hz).
No exceptions.
audio = document.getElementsByTagName("audio")[0];
samplingrate = audio.getProperty("samplingrate");

/*Resulting in:
samplingrate[0].value = 44100
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=samplingrate

  • Response (JSON format): { "samplingrate" : { "value" : 44100 } }

3.12.6 Framerate interface

Framerate interface is used as the specific return type of MediaResource.getProperty method which has "Framerate" as a value of propertyName parameter.

[NoInterfaceObject]
interface Framerate : MAObject {
    attribute float value;
};
3.12.6.1 Attributes
value of type float
This attribute specifies the framerate (in fps).
No exceptions.
video = document.getElementsByTagName("video")[0];
framerate = video.getProperty("framerate");

/*Resulting in:
framerate[0].value = 30
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=framerate

  • Response (JSON format): { "framerate" : { "value" : 30 } }

3.12.7 AverageBitrate interface

AverageBitrate interface is used as the specific return type of MediaResource.getProperty method which has "AverageBitrate" as a value of propertyName parameter.

[NoInterfaceObject]
interface AverageBitrate : MAObject {
    attribute float value;
};
3.12.7.1 Attributes
value of type float
This attribute specifies the average bitrate (in kbps).
No exceptions.

Example is as below:

video = document.getElementsByTagName("video")[0];
bitrate = video.getProperty("averagebitrate");

/*Resulting in:
bitrate[0].value = 45.06
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=averageBitrate

  • Response (JSON format): { "bitrate" : { "value" : 45.06 } }

3.12.8 NumTracks interface

NumTracks interface is used as the specific return type of MediaResource.getProperty method which has "NumTracks" as a value of propertyName parameter.

[NoInterfaceObject]
interface NumTracks : MAObject {
    attribute unsigned short value;
    attribute DOMString      type;
};
3.12.8.1 Attributes
type of type DOMString
This attribute specifies the type of the tracks e.g., "Songs", "Data streams".
No exceptions.
value of type unsigned short
This attribute specifies the number of tracks.
No exceptions.
video = document.getElementsByTagName("video")[0];
numTracks = video.getProperty("NumTracks");

/*Resulting in:
numTracks[0].value = 2
numTracks[0].type = "Data streams"
*/

Usage as a service:

  • Request: http://example.com/my-media-resource/?ma-query=numtracks

  • Response (JSON format): { "numtracks" : { "value" : 2 } }

4. Usage examples

This part illustrates some examples to show how to use the API using JavaScript in actual implementations. The examples assume that the methods getElementsByTagName() and getElementsById() return an element that implements the MediaResource interface. Moreover, in these examples it is assumed that the implementation of the API knows where to find the metadata that corresponds to a specific media resource (if necessary the location of the metadata can be configured using the selectMAResource method). The implementation should provide the mappings of different metadata formats to the core properties.

Example 1: Return the name of the director of the movie "Apocalypse now".
//assume that the video elements in HTML5 inherit from the MediaResource interface 
video = document.getElementsByTagName("video"); 

//search the video's for the one with title "Apocalypse now" 
for(i=0; i<video.length; i++) 
{ 
        //request for the titles of the video 
        titles = video[i].getProperty("title"); 
        for(j = 0; j<titles.length; j++) 
        { 
                //check if the title matches 
                if(titles[j].value == "Apocalypse Now") 
                { 
                        //request for the director of the video 
                        director = video[i].getProperty("contributor","","","director"); 
                        /*Resulting in: 
                        director[0].id = "Francis Ford Coppola"; 
                        */
                } 
        } 
} 
Example 2: Retrieve the title of the second song from the album "Josuha Tree" by U2.
//somehow get the mediaResource object that represents the corresponding album 
album = document.getElementById("JosuhaTree_U2"); 
//get the id of the second song using the fragments property 
trackid = album.getProperty("Fragments","","","Song")[1].id; 

//use this identifier to get the mediaResource object that represents the track 
mediaResource = new MediaResource(); 
mediaResource.selectMediaResource(trackid); 

//get the title of the track 
title = mediaResource.getProperty("Title")[0].value; 
Example 3: Return the copyright of the movie "Planet of the apes".
//somehow get the mediaResource object that represents the movie (e.g., see first example) 
movie = document.getElementById("Planet_of_the_apes_id"); 
copyright = movie.getProperty("copyright","","","","en-us")[0].statement; 
Example 4: Return the genre of the movie "Apocalypse Now".
//somehow get the mediaResource object that represents the movie (e.g., see first example) 
movie = document.getElementById("Apocalypse_Now_id"); 
genre = movie.getProperty("genre","","","","en-us"); 

/*Resulting in: 
genre[0].value = "Action"; 
genre[1].value = "Drama"; 
*/ 

5. API Status Codes

This section introduces a set of status codes for the defined API to indicate the system behaviour. It uses a subset of the HTML/1.1 [HTTP11] status codes for general informations (e.g., bad request), but also system specific ones (e.g., property not defined in source format). The specific status codes have been arranged in the HTML/1.1 [HTTP11] status codes categories. These codes can be retrieved by calling the getDiagnosis method.

The set of status codes my be enlarged or changed in later versions of this document. The numerical code range for the system specific status codes start with x62 (x = number of HTML/1.1 [HTTP11] status codes category and 62 = MA).

Numerical Code Textual Description Example
200 Ok property delivered correctly
204 No Content property retrieved without content
206 Partial Content only a subset of the available data stored in the result set
262 Syntactic Mapping retrieval used a syntactic mapping
263 Semantic Mapping retrieval used a semantic mapping
264 Structured Return Value result is structured
265 Unstructured Return Value result is unstructured
400 Bad Request syntactical error with respect to the GET method used
404 Not Found the queries resource is not found
415 Unsupported Media Type get duration call on an image data store
462 Property not defined in Source Format location is not defined in MediaRSS
500 Internal Server Error internal library (e.g., extractor) crashes
501 Not Implemented only a subset of GET methods for properties implemented

6. Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words must, must not, required, should, should not, recommended, may, and optional in this specification are to be interpreted as described in [RFC2119].

7. Security Considerations

This section is non-normative.

This specification defines a client-side API to access metadata information related to media resources on the Web. These APIs will provide the methods for getting metadata information which can be in one of different formats, either as separate document or embedded in media resources.

There are related activity and technical documents in W3C such as Policy Requirements [POLICY-REQS] in DAP WG, ODRL 1.1 [ODRL11], P3P 1.1 [P3P11] and PLING Wiki [PLING-WIKI].

A. Web IDL description for API

module mawg {
        interface MediaResource {

                  //Media Resource context
                  boolean selectMAResource(in DOMString mediaResource, in optional MetadataSource[] metadataSources );

                  // Property Access
                  MAObject[] getProperty(in DOMString propertyName, in optional DOMString fragment, in optional DOMString sourceFormat, in optional DOMString subtype,
                                         in optional DOMString language );

                  //Iterating
                  DOMString[] getPropertyNamesWithValues(in optional DOMString sourceFormat,
                                                                in optional DOMString language, in optional DOMString fragment);
                  DOMString[] getSourceFormatsWithValues(in optional DOMString language);
                  DOMString getOriginalData(in DOMString sourceFormat);

                  //Operation for retrieval of the reason of an error
                  DOMString getDiagnosis();

          };

        interface MetadataSource {
                  attribute DOMString metadataSource;
                  attribute DOMString sourceFormat;
          };

        module returnValues {

                interface MAObject {
                        attribute DOMString unstructuredValue;
                          attribute DOMString uri;
                        attribute DOMString sourceFormat;
                        attribute DOMString fragmentIdentifier;
                          attribute DOMString mappingType;
                  };

                interface Language {
                        attribute DOMString language;
                        };


                interface UnsignedLongObject: MAObject {
                        attribute unsigned long value;
                  };

                  interface UnsignedShortObject: MAObject {
                        attribute unsigned short value;
                  };

                  interface FloatObject: MAObject {
                        attribute float value;
                  };

                  interface Identifier: MAObject {
                          attribute DOMString value;
                          attribute DOMString type;
                  };

                  interface Title: MAObject, Language {
                          attribute DOMString value;
                          attribute DOMString type;
                  };

                  interface Language : MAObject {
                        attribute DOMString value;
                };

                interface Locator : MAObject {
                        attribute DOMString value;
                };

                interface Contributor: MAObject {
                        attribute DOMString id;
                        attribute DOMString role;
                };

                interface Creator : MAObject {
                        attribute DOMString value;
                };

                  interface Date: MAObject {
                          attribute DOMString date;
                          attribute DOMString type;
                  };

                  interface Location: MAObject, Language {
                          attribute DOMString name;
                          attribute Float longitude;
                          attribute Float latitude;
                          attribute Float altitiude;
                          attribute DOMString system;
                  };

                interface Description : MAObject, Language {
                        attribute DOMString value;
                };
                  
                interface Keyword : MAObject, Language {
                        attribute DOMString value;
                };

                interface Genre : MAObject, Language {
                        attribute DOMString value;
                };

                interface Rating: MAObject, Language {
                                attribute DOMString issuer;
                        attribute short value;
                        attribute short minimum;
                        attribute short maximum;
                        attribute DOMString context;
                          attribute DOMString type;
                };

                  interface Relation: MAObject, Language {
                        attribute DOMString id;
                        attribute DOMString relationship;
                };

                  interface Copyright: MAObject, Language {
                        attribute DOMString statement;
                        attribute DOMString[] holder;
                };

                interface Policy: MAObject, Language {
                        attribute DOMString statement;
                        attribute DOMString link;                
                        attribute DOMString organization;
                        attribute DOMString type;
                };
                
                interface Publisher: MAObject {
                        attribute DOMString value;
                };

                  interface TargetAudience: MAObject, Language {
                        attribute DOMString issuer;
                        attribute DOMString classification;
                };

                  interface Fragment: MAObject, Language {
                          attribute DOMString role;
                          attribute DOMString id;
                };

                interface NamedFragment: MAObject, Language {
                          attribute DOMString name;
                          attribute DOMString id;
                };

                  interface FrameSize: MAObject {
                          attribute unsigned long width;
                          attribute unsigned long height;
                          attribute DOMString unit;                          
                };
                
                interface Compression : MAObject, Language {
                        attribute DOMString value;
                };
                
                interface Duration : MAObject {
                        attribute unsigned long value;
                };
                
                interface Format : MAObject {
                        attribute DOMString value;
                };

                interface Samplingrate : MAObject {
                        attribute unsigned long value;
                };
                
                interface Framerate : MAObject {
                        attribute float value;
                };

                interface AverageBitrate : MAObject {
                        attribute float value;
                };
                
                interface NumTracks : MAObject {
                        attribute unsigned short value;
                        attribute DOMString type;
                };

        };
};

B. Acknowledgements

This document is the work of the W3C Media Annotations Working Group.

Members of the Working Group are (at the time of writing, and by alphabetical order): Members of the Working Group are (at the time of writing, and by alphabetical order): Werner Bailer (JOANNEUM RESEARCH), Tobias B체rger (University of Innsbruck), Eric Carlson (Apple, Inc.), Pierre-Antoine Champin ((public) Invited expert), Ashish Chawla ((public) Invited expert), Jaime Delgado (Universitat Polit챔cnica de Catalunya), Jean-Pierre EVAIN ((public) Invited expert), Philip J채genstedt (Opera Software), Ralf Klamma ((public) Invited expert), WonSuk Lee (Electronics and Telecommunications Research Institute (ETRI)), V챕ronique Malais챕 (Vrije Universiteit), Erik Mannens (IBBT), Hui Miao (Samsung Electronics Co., Ltd.), Thierry Michel (W3C/ERCIM), Frank Nack (University of Amsterdam), Soohong Daniel Park (Samsung Electronics Co., Ltd.), Silvia Pfeiffer (W3C Invited Experts), Chris Poppe (IBBT), V챠ctor Rodr챠guez (Universitat Polit챔cnica de Catalunya), Felix Sasaki (Potsdam University of Applied Sciences), David Singer (Apple, Inc.), Florian Stegmaier ((public) Invited expert), John Strassner ((public) Invited expert), Joakim S철derberg (ERICSSON), Thai Wey Then (Apple, Inc.), Ruben Tous (Universitat Polit챔cnica de Catalunya), Rapha챘l Troncy (CWI), Vassilis Tzouvaras (K-Space), Davy Van Deursen (IBBT).

The people who have contributed to discussions on public-media-annotation@w3.org are also gratefully acknowledged.

C. References

C.1 Normative references

[MEDIA-ANNOT-ONTOLOGY]
이원석(WonSuk Lee), Tobias Bürger, Felix Sasaki, Véronique Malaisé, Florian Stegmaier and Joakim Söderberg. Ontology for Media Resource 1.0. 09 March 2010. W3C Working Draft. (Work in progress.) URL: http://www.w3.org/TR/2010/WD-mediaont-10-20100309/
[MEDIA-ANNOT-REQS]
WonSuk Lee; Felix Sasaki; Tobias Bürger; Véronique Malaisé. Use Cases and Requirements for Ontology and API for Media Object 1.0. 21 January 2010. W3C Working Draft. (Work in progress.) URL: http://www.w3.org/TR/2010/WD-media-annot-reqs-20100121
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. Internet RFC 2119. URL: http://www.ietf.org/rfc/rfc2119.txt

C.2 Informative references

[BCP-47]
A. Phillips; M. Davis. Tags for Identifying Languages. Internet RFC 5646. URL: http://tools.ietf.org/html/rfc5646
[HTTP11]
R. Fielding; et al. Hypertext Transfer Protocol - HTTP/1.1. June 1999. Internet RFC 2616. URL: http://www.ietf.org/rfc/rfc2616.txt
[ODRL11]
Open Digital Rights Language (ODRL) Version 1.1. W3C Note. 19 September 2002. URL: http://www.w3.org/TR/odrl
[P3P11]
Matthias Schunter; Rigo Wenning. The Platform for Privacy Preferences 1.1 (P3P1.1) Specification. 13 November 2006. W3C Note. URL: http://www.w3.org/TR/2006/NOTE-P3P11-20061113
[PLING-WIKI]
PLING Wiki. URL: http://www.w3.org/Policy/pling/
[POLICY-REQS]
Laura Arribas; Paddy Byers; Marcin Hanclik; Frederick Hirsch; David Rogers. Device API Policy Requirements. 13 April 2010. W3C Editor's Draft. (Work in progress.) URL: http://dev.w3.org/2009/dap/policy-reqs/
[WEBIDL]
Cameron McCormack. Web IDL. 19 December 2008. W3C Working Draft. (Work in progress.) URL: http://www.w3.org/TR/2008/WD-WebIDL-20081219