W3C

Pick Media Intent

W3C Working Draft 12 July 2012

This version:
http://www.w3.org/TR/2012/WD-gallery-20120712/
Latest published version:
http://www.w3.org/TR/gallery/
Latest editor's draft:
http://w3c-test.org/dap/gallery/
Editors:
송정기(Jungkee Song), Samsung Electronics Co., Ltd.
이원석(Wonsuk Lee), Samsung Electronics Co., Ltd.

Abstract

The Pick Media Intent defines a Web Intent [WEBINTENTS] that enables access to a user's media gallery from inside a Web application. It defines both an Intent action/type pair that selects this operation, and the format of the media data that is returned by services implementing this specification.

Status of This Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document recasts the previous pure JavaScript APIs version as an API built using Web Intents, while refining the metadata format based on related media data definitions and practical web-based media services.

This document was published by the Device APIs Working Group as a First Public Working Draft. This document is intended to become a W3C Recommendation. If you wish to make comments regarding this document, please send them to public-device-apis@w3.org (subscribe, archives). All feedback is welcome.

Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of Contents

1. Introduction

This section is non-normative.

On account of a number of media applications and services accessed from various devices, users tend to maintain their media data in multiple sources including local storage of devices and web-based media galleries. For instance, users upload subset of their photos to several different social media galleries with different metadata. For another example, users collect their favourite audio and video resources in several different media galleries which provide inconsistent metadata attributes one another.

Note

A terminology, media object, is used hereunder to denote media data that contain a media content and its related metadata. (See Media dictionary)

This specification enables a Web application to have access to a selected subset of user's media objects, obtained from arbitrary services not known to the Web application. The interactions, brokered using Web Intents [WEBINTENTS] are designed in order to maximise the user's security and privacy. Media objects may be sourced from a plurality of sources — both online and local to the user's device — so long as those sources are registered as Intent services with the user agent. It defines a common media object format which services use to provide data to Web applications in a consistent and interoperable manner.

For a Pick Media Intent request, user permission can be naturally asked by a necessary user authentication process that a service implementer may provide. Also, users can determine the result media objects to bring to the client with the searching and selecting functionalities that a service implementer may provide.

A set of Security and Privacy Considerations are presented for the discretion of both implementers of Pick Media Intent services and recipients of media data (i.e. Web applications).

The following code illustrates how to obtain media object from a user's media gallery:

Example 1
var intent = new Intent({ action:   "http://webintents.org/pick",
                          type:     "http://w3.org/type/media",
                          extras:   { search: "olympic",
                                      filters: ["title", "description", "author", "tags"],
                                      limit: 100 }});
navigator.startActivity(intent, mediaOK, mediaFail);

function mediaOK (mediaObjectArray) {
    // iterate over the array of media objects to do something useful with them
}
function mediaFail (err) {
    // display an error to the user
}

When the above code is run, the user would typically be prompted by her user agent to select a service able to pick media objects (there may be several such services, if she has multiple media galleries). Upon selecting a service, she will be presented with an interface enabling her to choose what media objects are returned to the Web application. Upon completing her choice, an array of the media objects would be returned to the Web application in the mediaOK callback.

2. Conformance

As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.

The key words must, must not, required, should, should not, recommended, may, and optional in this specification are to be interpreted as described in [RFC2119].

There is only one single conformance requirement placed upon the user agent product: a user agent must support Web Intents [WEBINTENTS].

The conformance criteria in this specification apply to a single product: the Media service which exposes a Web Intent service that handles Pick Media Intents as defined in this specification.

The data returned by the Media service is described in this specification using [WEBIDL]. When this data is provided using JavaScript, then the Media service must do so in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification.

3. Security and Privacy Considerations

This section is non-normative.

The Intent defined in this specification can be used to find media objects from user's media galleries. The media objects, in the form of photos, videos, voice recordings with related metadata, may contain user's private information. The distribution of this information could potentially compromise the user's privacy. A conforming implementation of this specification should provide a mechanism that protects the user's privacy and this mechanism should ensure that no such information is retrievable without the user's express permission.

3.1 Privacy considerations for implementers of the Pick Media Intent

This section is non-normative.

A media service should not provide media objects to Web sites without the express permission of the user. Obtaining the user's express permission to access a set of media objects does not imply that the user has granted permission for the same Web site to access more media objects. A media service should take great care to ensure that the user can clearly see which information is about to be shared, and must not share more information than has been requested by the Web application.

A user agent may have prearranged trust relationships with a specific media service that do not require such user interaction.

3.2 Privacy considerations for recipients of media objects

Web sites operators that retrieve media objects using this Intent are denoted as recipients below.

Recipients should only request media objects when necessary, and only use the media objects for the task for which it was provided to them.

Recipients should dispose of media objects once that task is completed, unless expressly permitted to retain it by the user. Recipients should also take measures to protect this information against unauthorised access. If media objects are stored, users should be allowed to update and delete this information.

The recipient of media objects should not retransmit the media objects without the user's express permission. Care should be taken when retransmitting and use of encryption is encouraged.

Recipients should clearly and conspicuously disclose the fact that they are collecting media objects, the purpose of the collection, how long the data is retained, how the data is secured, how the data is shared if it is shared, how users can access, update and delete the data, and any other choices that users have with respect to the data. This disclosure should include an explanation of any exceptions to the guidelines listed above.

Note that even if a user gives permission to share their media objects this can have serious privacy implications for those parties whose media objects are shared, as they may not wish such sharing to occur. This should be considered by Web applications when requesting and using such information.

3.3 Additional implementation considerations

Further to the requirements listed in the previous section, implementers of a user agents are also advised to consider the following aspects that can negatively affect the privacy of their users: in certain cases, users can inadvertently grant permission to disclose their media objects to Web sites. In other cases, the content hosted at a certain URL changes in such a way that the previously granted permissions no longer apply as far as the user is concerned. Or the users might simply change their minds.

Predicting or preventing these situations is inherently difficult. Mitigation and in-depth defensive measures are a user agent's responsibility and not prescribed by this specification. However, in designing these measures, implementers are advised to enable user awareness of information sharing, and to provide easy access to user interfaces that enable revocation of permissions that Web applications have to access this Intent.

4. Intent Description

The action for this Intent is http://webintents.org/pick.

The type for this Intent is http://w3.org/type/media.

When a media service is matched for delivery using these action and type, it must respond in one of two ways:

4.1 Intent Extras

The Pick Media Intent can be instantiated with an extras field that adheres to the following dictionary.

4.1.1 The MediaIntentExtras dictionary

The MediaIntentExtras dictionary describes the options that the invoker can inform to the service as hints in searching the media objects. None of the fields are mandatory.

dictionary MediaIntentExtras {
    DOMString?     search;
    DOMString[]?   filters;
    unsigned long? limit;
};
4.1.1.1 Dictionary MediaIntentExtras Members
filters of type array of DOMString, nullable
An array of string which contain hints to the media service to the DOMString and DOMString[] fields in the Media dictionary that the Web application prefers to use as search criteria. The media service may use the values of the filters to search media objects in the media gallery. If a filter name is provided that the media service does not recognise as a field of the Media dictionary, then it must ignore it.
limit of type unsigned long, nullable
By default a media service may return as many media objects as the user selects. If limit is specified, the media service must not return more than limit media objects. The media service should enforce this limitation in the user interface that it exposes.
A string which provides a hint to the media service to search media objects. The exact manner in which this hint is exploited is entirely up to the media service.

5. Data Format

Upon successful invocation, the media service must return an array of Media dictionaries.

5.1 The Media dictionary

The Media dictionary captures the properties of a media object. The properties defined in this dictionary present the content (URI by default and Blob as option) of the media data and the related metadata. The properties of the dictionary are defined based on the previous work of metadata survey in Media Ontology WG, the core set, [MEDIA-ANNOT-REQS] as well as the common attributes of open APIs provided by typical web-based media services, thereby allowing the data format to be supported across implementations supporting these various media metadata representations.

Additional attributes may be included according to the provisions detailed in Extended Media Properties and Parameters.

dictionary Media {
    MediaContent   content;
    DOMString      description;
    DOMString?     id;
    DOMString      title;
    DOMString      type;
    DOMString?     author;
    DOMString?     category;
    DOMString?     copyright;
    DOMString?     fileName;
    unsigned long? fileSize;
    Position?      location;
    DOMString[]?   tags;
    MediaContent?  thumbnail;
    Date?          createdDate;
    Date?          updatedDate;
    DOMString?     collection;
    unsigned long? duration;
    unsigned long? frameRate;
    DOMString?     language;
    unsigned long? likeCount;
    float?         rating;
    unsigned long? ratingCount;
    DOMString?     resolution;
    unsigned long? samplingRate;
    Date?          publishedDate;
    DOMString?     publisher;
    unsigned long? viewCount;
};

5.1.1 Dictionary Media Members

author of type DOMString, nullable
The author of the media resource in string.
category of type DOMString, nullable
The category (genre) of the content of the media resource.
collection of type DOMString, nullable

The name of the collection or album that the media resource belongs to.

content of type MediaContent
A dictionary containing a URI to the content of the media resource. It optionally contains a Blob of the content.
The copyright statement associated with the media resource.
createdDate of type Date, nullable

The creation date of the media resource in Date type.

description of type DOMString
The text description of the content of the media resource.
duration of type unsigned long, nullable

The actual duration of the resource. The units are defined to be seconds.

fileName of type DOMString, nullable
This field's value is not a url, but is the filename (probably not fully qualified, but simply the name of the leaf file) of the resource being passed as assigned by the user. (See WebIntents/MIME Types)
fileSize of type unsigned long, nullable
The size of the content of the media resource.
frameRate of type unsigned long, nullable

The video frame rate. The units are defined to be frames per second.

id of type DOMString, nullable

A URI identifying a media resource.

Note

The property, identifier, defined in the core set of [Ontology for Media Resources 1.0] can be considered as a representation format.

language of type DOMString, nullable

The language used in the media resource.

likeCount of type unsigned long, nullable

The count of the vote in favour of the media resource.

location of type Position, nullable
The location information related to the content of the media resource. [GEOLOCATION-API]
publishedDate of type Date, nullable

The publishing date of the media resource in Date type.

publisher of type DOMString, nullable

The publisher of the media resource.

rating of type float, nullable

The rating value (e.g., customer rating, review, audience appreciation) of the media resource.

ratingCount of type unsigned long, nullable

The number of ratings made on the media resource.

resolution of type DOMString, nullable

The frame size of the resource (e.g., width and height of 720 and 480 in px is represented as 720*480+px).

Note

If the usage of this property is frequent, a separate dictionary can be defined referring to frameSize property defined in [Ontology for Media Resources 1.0]

samplingRate of type unsigned long, nullable

The audio sampling rate. The units are defined to be samples per second.

tags of type array of DOMString, nullable

The array of strings representing series of tag string.

thumbnail of type MediaContent, nullable
A dictionary containing a URI to the thumbnail of the content of the media resource. It optionally contains a Blob of the thumbnail of the content.
title of type DOMString
The title or name given to the media resource.
type of type DOMString
The MIME type of the media resource (e.g., image/png, video/*) [RFC2046].
updatedDate of type Date, nullable

The last updated date of the media resource in Date type.

viewCount of type unsigned long, nullable

The access count made on the media resource.

5.2 The MediaContent dictionary

The MediaContent dictionary describes a media content reference in detail.

dictionary MediaContent {
    DOMString uri;
    Blob?     blob;
};

5.2.1 Dictionary MediaContent Members

blob of type Blob, nullable
The optional property to contain the media content in Blob format.
uri of type DOMString
The URI to the content of the media resource. This attribute is a mandatory property.

5.3 The MediaError dictionary

If the media service encounters an error then it must return an error (through postFailure()) using the MediaError dictionary.

dictionary MediaError {
    DOMString message;
};

5.3.1 Dictionary MediaError Members

message of type DOMString
A message describing the error.

5.4 Extended Media Properties and Parameters

A media service may extend the dictionaries described in in the Data Formats section with additional fields. If providing an extended field, a media service must prefix its name with X (U+0058 LATIN CAPITAL LETTER X) or use a vendor-specific prefix.

A. Media Service Example

This section is non-normative.

This section describes an example usage of the Pick Media Intent in media service. It shows the use of the values delivered through MediaIntentExtras dictionary, and the way to return the result array of media objects through Media dictionary.

A.1 Accessing MediaIntentExtras Values - Example #1

This section is non-normative.

Media service implementers can retrieve the values delivered in extra field of the intent request. The extra field contains the MediaIntentExtras dictionary:

  <script type="text/javascript">
  if (window.intent) {
      var keyword = window.intent.extra.search;  // a string based hint to search
      var filters = window.intent.extra.filters; // DOMString[] based hint to fields to match
      var limit = window.intent.extra.limit;     // maximum number of objects to return

      // do search relevant media objects based on the extras
      var mediaObjectArray = searchMediaObjects(keyword, filters, limit);
      
      if (mediaObjectArray) {
          window.intent.postResult(mediaObjectArray);
      } else {
          window.intent.postFailure(errorMessage);
      }
  }  
  </script>
          

The use of the properties in MediaIntentExtras dictionary is optional. Service implementers can decide whether these properties are to be used as hints for search.

A.2 Creating Media Object - Example #2

Media service implementers can create media objects in the format of the Media dictionary and make an array of media objects to send to client. (using postResult()):

  <script type="text/javascript">
  if (window.intent) {
      var content = {};
      content.uri = "http://example.com/gallery/image00001.png";
       
      var mediaObject = new Object();
      mediaObject.content = content;
      mediaObject.description = "The city view of Seoul";
      mediaObject.id = ""; 
      mediaObject.title = "City of Seoul";
      mediaObject.type = "image/png";  // MIME type of the media content
      mediaObject.author = "J.";

      var mediaObjectArray = new Array();
      
      mediaObjectArray[0] = mediaObject;
      
      if (mediaObjectArray) {
          window.intent.postResult(mediaObjectArray);
      } else {
          window.intent.postFailure(errorMessage);
      }
  }  
  </script>
          

B. References

B.1 Normative references

[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Internet RFC 2119. URL: http://www.ietf.org/rfc/rfc2119.txt
[WEBIDL]
Cameron McCormack. Web IDL. 27 September 2011. W3C Working Draft. (Work in progress.) URL: http://www.w3.org/TR/2011/WD-WebIDL-20110927/
[WEBINTENTS]
Greg Billock; James Hawkins; Paul Kinlan. Web Intents. Editors' Draft. (Work in progress.) URL: http://dvcs.w3.org/hg/web-intents/raw-file/tip/spec/Overview.html

B.2 Informative references

[GEOLOCATION-API]
Andrei Popescu. Geolocation API Specification. 22 December 2008. W3C Working Draft. (Work in progress.) URL: http://www.w3.org/TR/2008/WD-geolocation-API-20081222/
[MEDIA-ANNOT-REQS]
WonSuk Lee; Felix Sasaki; Tobias Bürger; Véronique Malaisé. Use Cases and Requirements for Ontology and API for Media Object 1.0.W3C Working Draft 21 January 2010. URL: http://www.w3.org/TR/2010/WD-media-annot-reqs-20100121/
[RFC2046]
N. Freed; N. Borenstein. Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types. November 1996. Internet RFC 2046. URL: http://www.ietf.org/rfc/rfc2046.txt