Use cases and requirements for Media Fragments

W3C Working Draft 10 March 2010

This version:
Latest version:
Previous version:
Raphaël Troncy , EURECOM
Erik Mannens , IBBT Multimedia Lab, University of Ghent
Jack Jansen , Center for Mathematics and Computer Science (CWI Amsterdam)
Yves Lafon , W3C
Silvia Pfeiffer , W3C Invited Expert
Davy Van Deursen , IBBT Multimedia Lab, University of Ghent


This document describes use cases and requirements for the development of the Media Fragments 1.0 specification. It includes a technology survey for addressing fragments of multimedia document.

Status of this Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This is the First Public Working Draft of the Use cases and requirements for Media Fragments specification. It has been produced by the Media Fragments Working Group, which is part of the W3C Video on the Web Activity.

A list of changes is available in E Change Log.

A diff-marked version against the previous version of this document is available.

Please send comments about this document to public-media-fragment@w3.org mailing list (public archive).

Publication as a Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. The group does not expect this document to become a W3C Recommendation. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

Table of Contents

1 Introduction
2 Terminology
3 Side Conditions
    3.1 Single Media Resource Definition
    3.2 Existing Standards
    3.3 Unique Resource
    3.4 Valid Resource
    3.5 Parent Resource
    3.6 Single Fragment
    3.7 Relevant Protocols
    3.8 No Recompression
    3.9 Minimize Impact on Existing Infrastructure
    3.10 Focus for Changes
    3.11 Browser Impact
    3.12 Fallback Action
4 Use Cases
    4.1 Linking to and Display of Media Fragments
        4.1.1 Scenario 1: Retrieve only segment of a video
        4.1.2 Scenario 2: Region of an Image
        4.1.3 Scenario 3: Portion of Music
        4.1.4 Scenario 4: Image Region of video over time
    4.2 Browsing and Bookmarking Media Fragments
        4.2.1 Scenario 1: Temporal Video Pagination
        4.2.2 Scenario 2: Audio Passage Bookmark
        4.2.3 Scenario 3: Audio Navigation
        4.2.4 Scenario 4: Caption and chapter tracks for browsing Video
        4.2.5 Scenario 5: Jumping back in time during live streaming
        4.2.6 Scenario 6: Jumping to a particular event in a live stream
    4.3 Recompositing Media Fragments
        4.3.1 Scenario 1: Reframing a photo in a slideshow
        4.3.2 Scenario 2: Mosaic
        4.3.3 Scenario 3: Video Mashup
        4.3.4 Scenario 4: Spatial Video Navigation
        4.3.5 Scenario 5: Selective previews
        4.3.6 Scenario 6: Music Samples
        4.3.7 Scenario 7: Highlighting regions (out-of-scope)
    4.4 Annotating Media Fragments
        4.4.1 Scenario 1: Spatial Tagging of Images
        4.4.2 Scenario 2: Temporal Tagging of Audio and Video
        4.4.3 Scenario 3: Named Anchors
        4.4.4 Scenario 4: Spatial and Temporal Tagging
        4.4.5 Scenario 5: Search Engine
    4.5 Adapting Media Resources
        4.5.1 Scenario 1: Changing Video quality (out-of-scope)
        4.5.2 Scenario 2: Selecting Regions in Images
        4.5.3 Scenario 3: Selecting an Image from a multi-part document (out-of-scope)
        4.5.4 Scenario 4: Retrieving an Image embedded thumbnail (out-of-scope)
        4.5.5 Scenario 5: Switching of Video Transmission
        4.5.6 Scenario 6: Toggle All Audio OFF
        4.5.7 Scenario 7: Toggle specific Audio tracks
        4.5.8 Scenario 8: Video aspect ratio (out-of-scope)
5 Requirements for Media Fragment URIs
    5.1 Requirement r01: Temporal fragments
    5.2 Requirement r02: Spatial fragments
    5.3 Requirement r03: Track fragments
    5.4 Requirement r04: Named fragments
    5.5 Fitness Conditions on Media Containers/Resources


A References
B Evaluation of fitness per media formats
C Technologies Survey
    C.1 Existing URI fragment schemes
        C.1.1 General specification of URI fragments
        C.1.2 Fragment specifications not for audio/video
        C.1.3 Fragment specifications for audio/video
    C.2 Existing applications using proprietary temporal media fragment URI schemes
    C.3 Media fragment specification approaches
        C.3.1 URI based
            C.3.1.1 SVG
                C. Spatial
            C.3.1.2 Temporal URI/Ogg technologies
                C. Temporal
                C. Track
                C. Named
            C.3.1.3 MPEG-21
                C. Temporal
                C. Spatial
                C. Track
                C. Named
        C.3.2 Non-URI-based
            C.3.2.1 SMIL
                C. Temporal
                C. Spatial
                C. Track
                C. Named
            C.3.2.2 MPEG-7
                C. Temporal
                C. Spatial
                C. Track
                C. Named
            C.3.2.3 SVG
                C. Temporal
                C. Spatial
            C.3.2.4 TV-Anytime
                C. Temporal
                C. Named
            C.3.2.5 ImageMaps
                C. Spatial
            C.3.2.6 HTML 5
D Acknowledgements (Non-Normative)
E Change Log (Non-Normative)

1 Introduction

Audio and video resources on the World Wide Web are currently treated as "foreign" objects, which can only be embedded using a plugin that is capable of decoding and interacting with the media resource. Specific media servers are generally required to provide for server-side features such as direct access to time offsets into a video without the need to retrieve the entire resource. Support for such media fragment access varies between different media formats and inhibits standard means of dealing with such content on the Web.

This document collects background information to the Media Fragment URI specification Media Fragments URI 1.0. It contains a collection of side conditions under which the specification was developed. It further contains a large collection of use cases that are either regarded as relevant to this specification or as out-of-scope. From these use cases, it deducts the different required dimensions for the Media Fragment URI specification. Finally, this document finishes with a survey of existing media fragment addressing approaches.

2 Terminology

The keywords MUST, MUST NOT, SHOULD and SHOULD NOT are to be interpreted as defined in RFC 2119.

According to RFC 3986, URIs that contain a fragment are actually not URIs, but URI references relative to the namespace of another URI. In this document, when the term 'media fragment URIs' is used, it actually means 'media fragment URI references'.

3 Side Conditions

This section lists a number of conditions which have directed the development of this specification. These conditions help clarify some of the decisions made, e.g. about what types of use cases are within the realm of this specification and which are outside. Spelling out these side conditions should help increase transparency of the specifications.

3.6 Single Fragment

Editorial note: Werner Bailer  
Not sure that the term 'mask' is the best choice here, e.g. in MPEG-7 mask is used for the opposite, i.e. not a single segment but a segment composed of several unconnected parts.

A media fragment URI should create only a single "mask" onto a media resource and not a collection of potentially overlapping fragments.

4 Use Cases

In which situations do users need media fragment URIs? This section explains the types of user interactions with media resources that media fragment URIs will enable. For each type it shows how media fragment URIs can improve the usefulness, usability, and functionality of online audio and video.

4.1 Linking to and Display of Media Fragments

In this use case, a user is only interested in consuming a fragment of a media resource rather than the complete resource. A media fragment URI allows addressing this part of the resource directly and thus enables the User Agent to receive just the relevant fragment.

4.2 Browsing and Bookmarking Media Fragments

Media resources - audio, video and even images - are often very large resources that users want to explore progressively. Progressive exploration of text is well-known in the Web space under the term "pagination". Pagination in the text space is realized by creating a series of Web pages and enabling paging through them by scripts on a server, each page having their own URI. For large media resources, such pagination can be provided by media fragment URIs, which enable direct access to media fragments.

4.2.4 Scenario 4: Caption and chapter tracks for browsing Video

Silvia has a deaf friend, Elaine, who would like to watch the holiday videos that Silvia is publishing on her website. Silvia has created subtitle tracks for her videos and also a segmentation (e.g. using CMML CMML) with unique identifiers on the clips that she describes. The clips were formed based on locations that Silvia has visited. In this way, Elaine is able to watch the videos by going through the clips and reading the subtitles for those clips that she is interested in. She watches the sections on Korea, Australia, and France, but jumps over the ones of Great Britain and Holland.

4.3 Recompositing Media Fragments

As we enable direct linking to media fragments in a URI, we can also enable simple recompositing of such media fragments. Note that because the media fragments in a composition may possibly originate from different codecs and very different files, we can not realistically expect smooth playback between the fragments.

4.4 Annotating Media Fragments

Media resources typically don't just consist of the binary data. There is often a lot of textual information available that relates to the media resource. Enabling the addressing of media fragments ultimately creates a means to attach annotations to media fragments, for example, using the Ontology for Media Resource 1.0.

4.4.2 Scenario 2: Temporal Tagging of Audio and Video

Editorial note: Silvia 
Time-aligned text such as captions, subtitles in multiple languages, and audio descriptions for audio and video don't have to be created as separate documents and link to each segment through a temporal URI. Such text can be made part of the media resource by the media author or delivered as a separate, but synchronised data stream to the media player. In either case, when it comes to using these with the HTML5 <video> tag, they should be made accessible to the Web page through a javascript API for the video/audio/image element. This needs to be addressed in the HTML5 working group.

4.5 Adapting Media Resources

When addressing a media resource as a user, one often has the desire not to retrieve the full resource, but only a subpart of interest. This may be a temporally or spatially consecutive subpart, but could also be e.g. a smaller bandwidth version of the same resource, a lower framerate video, a image with less colour depth or an audio file with a lower sampling rate. Media adaptation is the general term used for such server-side created versions of media resources.

5 Requirements for Media Fragment URIs

This section describes the list of required media fragment addressing dimensions that have resulted from the use case analysis.

It further analyses what format requirements the media resources has to adhere to in order to allow the extraction of the data that relates to that kind of addressing.

5.5 Fitness Conditions on Media Containers/Resources

There is a large number of media codecs and encapsulation formats that we need to take into account as potential media resources on the Web. This section analyses the general conditions for media formats that make them fit for supporting the different types of fragment URIs.

The extraction of media fragments from a media resource is possible when these media fragments are extractable in the compressed domain. In other words, media fragments need to be expressable in terms of byte ranges referring to the parent resource. Dependent on the fragment axis, there are different requirements regarding the extraction of media fragments (i.e., obtaining media fragments in terms of byte ranges):

Not all media formats fullfil all these requirements. Hence, we distinguish the following categories:

Those media types that are capable of doing what server-side media fragments require are of interest to us. For those that aren't, the fall-back mechanisms apply (e.g., full download and then offsetting). Appendix B Evaluation of fitness per media formats lists a large number of typical formats and determines which we see fit or currently unfit for supporting the different types of media fragment URIs.

Editorial note: Silvia 

We ask for further input into the table in the attachment, in particular where there are question marks.

A References

Continuous Media Markup Language (CMML), Version 2.1. IETF Internet-Draft, March 2006.
[HTML 4.0]
D. Ragett, A. Le Hors and I. Jacobs. HTML Fragment identifiers. W3C Recommendation, December 1999. Available at http://www.w3.org/TR/REC-html40/intro/intro.html#fragment-uri.
[ISO Base Media File Format]
Information technology - Coding of audio-visual objects - Part 12: ISO base media file format. April 2009.
[Media Fragments URI 1.0]
Media Fragments URI 1.0. W3C Working Draft, October 2009.
Information Technology - Multimedia Content Description Interface (MPEG-7). Standard No. ISO/IEC 15938:2001, International Organization for Standardization(ISO), 2001.
Information Technology - Multimedia Framework (MPEG-21). Standard No. ISO/IEC 21000:2002, International Organization for Standardization(ISO), 2002. Available at http://www.chiariglione.org/mpeg/working_documents/mpeg-21/fid/fid-is.zip.
[Ontology for Media Resource 1.0]
W. Lee, T. Bürger, F. Sasaki, V. Malaisé, F. Stegmaier and Joakim Söderberg. Ontology for Media Resource 1.0. W3C Working Draft, June 2009. Available at http://www.w3.org/TR/mediaont-10.
[RFC 2119]
S. Bradner. Key Words for use in RFCs to Indicate Requirement Levels. IETF RFC 2119, March 1997. Available at http://www.ietf.org/rfc/rfc2119.txt.
[RFC 3533]
S. Pfeiffer. The Ogg Encapsulation Format Version 0. IETF RFC 3533, May 2003. Available at http://www.ietf.org/rfc/rfc3533.txt.
[RFC 3986]
T. Berners-Lee, R. Fielding and L. Masinter. Uniform Resource Identifier (URI): Generic Syntax. IETF RFC 3986, January 2005. Available at http://www.ietf.org/rfc/rfc3986.txt.
[RFC 5147]
E. Wilde and M. Duerst. URI Fragment Identifiers for the text/plain Media Type. IETF RFC 5147, April 2008. Available at http://tools.ietf.org/html/rfc5147.
[RFC 5234]
D. Crocker. Augmented BNF for Syntax Specifications: ABNF, IETF RFC 5234, January 2008. Available at http://tools.ietf.org/html/rfc5234.
Rich Open multitrack media Exposition (ROE). Xiph.org Wiki, April 2009.
Ogg Skeleton. Xiph .org Wiki, April 2009.
SMPTE RP 136 Time and Control Codes for 24, 25 or 30 Frame-Per-Second Motion-Picture Systems
J. Ferraiolo. SVG Fragment identifiers. W3C Recommendation, September 2001. Available at http://www.w3.org/TR/2001/REC-SVG-20010904/linking#FragmentIdentifiersSVG.
[Temporal URI]
S. Pfeiffer, C. Parker and A. Pang. Specifying time intervals in URI queries and fragments of time-based Web resources. Internet Draft, March 2005. Available at http://annodex.net/TR/draft-pfeiffer-temporal-fragments-03.html. IETF Internet-Draft, March 2005.
[XPointer Framework]
P. Grosso, E. Maler, J. Marsh and N. Walsh. XPointer Framework. W3C Recommendation, March 2003. Available at http://www.w3.org/TR/xptr-framework/.

B Evaluation of fitness per media formats

In order to get a view on which media formats belong to which fitness category, an overview is provided for key media formats. In the following tables, the 'X' symbol indicates that the media format does not support a particular fragment axis. The tables are separated by video/audio/image codecs and container formats.

Video CodecTrackTemporalSpatialNameRemark
MPEG-1 Videon/afitunfitn/a
H.262/MPEG-2 Videon/afitunfitn/a
MPEG-4 Visualn/afitunfitn/a
H.264/MPEG-4 AVCn/afitfit (in theory)n/aWith Flexible Macroblock Ordening (FMO), it is possible to encode spatial regions independently from each other ...
Motion JPEGn/afitunfitn/a
Motion JPEG2000n/afitfitn/a
Diracn/afitunfitn/aWhen Dirac is stored in the Ogg RFC 3533 container using Skeleton Skeleton, ROE ROE and CMML CMML, track, temporal and named fragments are supported.
Theoran/afitunfitn/aWhen Theora is stored in the Ogg RFC 3533 container using Skeleton Skeleton, ROE ROE and CMML CMML, track, temporal and named fragments are supported.

Audio CodecTrackTemporalSpatialNameRemark
MPEG-1 Audion/afitn/an/a
Vorbisn/afitn/an/aWhen Vorbis is stored in the Ogg RFC 3533 container using Skeleton Skeleton, ROE ROE and CMML CMML, track, temporal and named fragments are supported.
FLACn/afitn/an/aWhen FLAC is stored in the Ogg RFC 3533 container using Skeleton Skeleton, ROE ROE and CMML CMML, track, temporal and named fragments are supported.
Speexn/afitn/an/aWhen Speex is stored in the Ogg RFC 3533 container using Skeleton Skeleton, ROE ROE and CMML CMML, track, temporal and named fragments are supported.
AC-3/Dolby Digitaln/afitn/an/a

Image CodecTrackTemporalSpatialNameRemark
JPEG LSn/an/aunfitn/a
HD Photon/an/afitn/a

Container FormatsTrackTemporalSpatialNameRemark
MOVfitn/an/afit QTText provides named chapters
MP4fitn/an/afit MPEG-4 TimedText provides named sections
3GPfitn/an/afit 3GPP TimedText provides named sections
MPEG-21 FFfitn/an/afit MPEG-21 Digital Item Declaration provides named sections
OGGfit (1)fitn/afit (2)(1) Using ROE ROE and Skeleton Skeleton, track selection is possible; (2) Using ROE, CMML CMML and Skeleton, named addressing of temporal and track fragments is possible
ASFfitn/an/afitMarker objects provide named anchor points
FLVfitn/an/afit cue points provide named anchor points
TIFFfitn/an/afitCan store multiple images (i.e., tracks) in one file, possibility to insert "private tags" (i.e., proprietary information)

C Technologies Survey

C.1 Existing URI fragment schemes

Some existing URI schemes define semantics for fragment identifiers. In this section, we list these URI schemes and provide examples of their fragment identifiers.

C.1.1 General specification of URI fragments

  • URI FragmentRFC 3986
    cited from RFC3986: "The fragment identifier component of a URI allows indirect identification of a secondary resource by reference to a primary resource and additional identifying information. The identified secondary resource may be some portion or subset of the primary resource, some view on representations of the primary resource, or some other resource defined or described by those representations. A fragment identifier component is indicated by the presence of a number sign ("#") character and terminated by the end of the URI."

C.1.2 Fragment specifications not for audio/video

  • HTML named anchorsHTML 4.0
    refers to a specific named anchor within the resource http://www.w3.org/2008/WebVideo/Fragments/wiki/Main_Page
  • XPointer named elementsXPointer Framework
    refers to the element with id equal to 'Rottweiler' in the target XML document http://www.w3schools.com/xlink/dogbreeds.xml
  • text (plain)RFC 5147
    identifies lines 11 to 20 of the text.txt MIME entity
    specifies the region to be viewed of the SVG image http://upload.wikimedia.org/wikipedia/commons/d/d2/Yalta_summit_1945_with_Churchill%2C_Roosevelt%2C_Stalin.jpg

C.1.3 Fragment specifications for audio/video

  • Temporal URI/Ogg technologiesTemporal URI
    specifies a temporal fragment of the OGG Theora video http://example.com/video.ogv starting at 12.3 s and and ending at 21.16 s
  • MPEG-21MPEG-21
    specifies a temporal fragment of the MP4 resource http://www.example.com/myfile.mp4 starting at 10 s and and ending at 30 s

C.2 Existing applications using proprietary temporal media fragment URI schemes

In this section, we list a number of proprietary URI schemes which are able to identify media fragments. Note that all of these schemes only provide support for addressing temporal media fragments.

  • Google Video (announcement)
    Syntax: #50m16s
  • YouTube (announcement)
    Syntax: #t=1m45s

    YouTube also does click-throughs on their embedded videos to time offsets (see announcement), and returns to time offsets when people browse away from a video they half watched (see http://youtube-global.blogspot.com/2009/09/release-notes-91709.html).

  • Archive.org (uses the temporalURI specification Temporal URI)
    Syntax: ?t=74.5
  • Videosurf (announcement)
    Syntax: ?t=140&e=184 (with t=start, e=end)

C.3 Media fragment specification approaches

Media fragment ApproachTemporalSpatialTrackName
URI based
Temporal URI/Ogg technologiesYesNoYesYes

C.3.1 URI based

C.3.1.2 Temporal URI/Ogg technologies
C. Temporal

A Temporal URI Temporal URI is being used to play back temporal fragments in Annodex. The clip's begin and end are specified directly in the URI. When using "#" the URI fragment identfier, it is expected that the media fragment is played after downloading the complete resource, while using "?" URI query parameters, it is expected that the media fragment is extracted on the server and downloaded as a new resource to the client. Linking to such a resource looks as follows:

                <a href="http://example.com/video.ogv#t=12.3/21.16" />
                <a href="http://example.com/video.ogv?t=12.3/21.16" />

It it possible to use different temporal schemes, which give frame-accurate clipping when used correctly:

                <a href="http://example.com/video.ogv?t=npt:12.3/21.16" />
                <a href="http://example.com/video.ogv?t=smpte-25:00:12:33:06/00:21:16:00" />
                <a href="http://example.com/audio.ogv?t=clock:20021107T173045.25Z" />
C. Track

Tracks are an orthogonal concept to time-aligned annotations. Therefore, Xiph/Annodex have invented another way of describing/annotating these. It's only new (since January 2008) and is called: ROE (for Rich Open multitrack media Encapsulation) ROE. With ROE you would describe the composition of your media resource on the server. This file can also be downloaded to a client to find out about the "capabilities" of the file. It is however mainly used for authoring-on-the-fly. Depending on what a client requires, the ROE file can be used to find the different tracks and multiplex them together. Here is an example file:

                <link rel="alternate" type="text/html" href="http://example.com/complete_video.html" />
                <track id="v" provides="video">
                <mediaSource id="v0" src="http://example.com/video.ogv" content-type="video/ogg" />
                <mediaSource id="v1" src="http://example.com/theora.ogv?track=v1" content-type="video/theora" />
                <track id="a" provides="audio">
                <mediaSource id="a1" src="http://example.com/theora.ogv?track=a1" content-type="audio/vorbis" />
                <track id="c1" provides="caption">
                <mediaSource src="http://example.com/cmml1.cmml" content-type="text/cmml" />
                <track id="c2" provides="ticker">
                <mediaSource src="http://example.com/cmml2.cmml" content-type="text/cmml" />

This has not completely been worked through and implemented, but Metavid is using ROE as an export format to describe the different resources available as subpart to one media resource. Note that ROE is also used to create an Ogg Skeleton Skeleton in a final multiplexed file. Thus, the information inherent in ROE goes into the file (at least in theory) and can be used to extract tracks in a URI:

<video src="http://example.com/video.ogv?track=a/v/c1"/>
C. Named

To include outgoing hyperlinks into video, you have to define the time-aligned markup of your video (or audio) stream. For this purpose, Annodex uses CMML CMML. Here is an example CMML file that can be used to include out-going hyperlinks next to or into Ogg RFC 3533 streams. ("next to" means here that the CMML file is kept separate of the Ogg file, but that the client-side player knows to synchronise the two, "into" means that CMML is multiplexed as a timed text codec into the Ogg physical bitstream creating only one file that has to be exchanged). The following defines a CMML clip that has an outgoing hyperlink (this is a partial document extracted from a CMML file):

                <clip id="tic1" start="npt:12.3" end="npt:21.16" title="Introduction">
                <a href="http://example.com/fish.ogv?t=5" >Watch another fish video.</a>
                <meta name="author" content="Frank"/>
                <img src="fish.png"/>
                <body>This is the introduction to the film Joe made about fish.</body>

Note how there is also the possibility of naming a thumbnail, providing metadata, and giving a full description of the clip in the body tag. Interestingly, you can also address into temporal fragments of a CMML CMML file, since it is a representation of a time-continuous data resource:

<a href="http://example.com/sample.cmml?t=npt:4" />

With CMML and ROE you can address into named temporal regions of a CMML file itself:

<a href="http://example.com/sample.cmml?id="tic1" />
C.3.1.3 MPEG-21

Four different schemes are specified in MPEG-21 Part 17 MPEG-21 to address parts of media resources: ffp(), offset(), mp(), and mask():

  • ffp() is applicable for file formats conforming to the ISO Base Media File Format (aka MPEG-4 part 12 or ISO/IEC 14496-12) and is able to identifiy tracks via track_ID located in the iloc and tkhd box respectively

  • offset() is applicable to any digital resource and identifies a range of bytes in a data stream (similar functionality as the HTTP byte range mechanism).

  • mp() is applicable for media resources whose Internet media type (or MIME type) is equal to audio/mpeg, video/mpeg, video/mp4, audio/mp4, or application/mp4 and provides two complementary mechanisms for identifying fragments in a multimedia resource via:

    • a set of so-called dimensions (i.e., temporal, spatial or spatiotemporal) which are independent of the coding/container format: for the temporal dimension, the following time schemes are supported: NPT, SMPTE, MPEG-7, and UTC.

    • a hierarchical logical model of the resource. Such a logical model is dependent on the underlying container format (e.g., audio CD contains a list of tracks). The structures defined in these logical models are accessed with a syntax based on XPath.

  • mask() is applicable for media resources whose Internet media type (or MIME type) is equal to video/mp4 or video/mpeg and addresses a binary mask defined in a resource (binary masks can be achieved through MPEG-4 shape coding). Note that this mask is meant to be applied to a video resource and that the video resource may itself be the resource that contains the mask.

Note that hierarchical combinations of addressing schemes are also possible. The '*' operator is used for this purpose. When two consecutive pointer parts are separated by the '*' operator, the fragments located by the first pointer part (to the left of the '*' operator) are used as a context for evaluating the second pointer part (to the right of the '*' operator).

C.3.2 Non-URI-based

C.3.2.1 SMIL
C. Temporal

Playing temporal fragments out-of-context

SMIL allows you to play only a fragment of the video by using the clipBegin and clipEnd atributes. How this is implemented, though, is out of scope for the SMIL spec (and for http-based URLs it may well mean that implementations get the whole media item and cut it up locally):

It is possible to use different time schemes, which give frame-accurate clipping when used correctly:

Adding metadata to such a fragment is supported since SMIL 3.0:

Referring to temporal fragments in-context

The following piece of code will play back the whole video, and during the interesting section of the video allow clicking on it to follow a link:

It is also possible to have a link to the relevant section of the video. Suppose the following SMIL code is located in http://www.example.com/smilpresentation:

Now, we can link to the media fragment using the following URI:

Jumping to #tic2area will start the video at the beginning of the interesting section. The presentation will not stop at the end, however, it will continue.

C. Spatial

Playing spatial fragments out-of-context

SMIL 3.0 allows playing back only a specific rectangle of the media. The following construct will play back the center quarter of the video:

Assuming the source video is 640x480, the following line plays back the same:

This construct can be combined with the temporal clipping.

It is possible to change the panZoom rectangle over time. The following code fragment will show the full video for 10 seconds, then zoom in on the center quarter over 5 seconds, then show that for the rest of the duration. The video may be scaled up or centered, or something else, depending on SMIL layout, but this is out of scope for the purpose of this investigation.

Referring to spatial fragments in-context

The following bit of code will enable the top-right quarter of the video to be clicked to follow a link. Note the difference in the way the rectangle is specified (left, top, right, bottom) when compared to panZoom (left, top, width, height). This is an unfortunate side-effect of this attribute being compatible with HTML and panZoom being compatible with SVG.

Other shapes are possible, as in HTML and CSS. The spatial and temporal constructs can be combined. The spatial coordinates can be animated, as for panZoom.

C.3.2.2 MPEG-7
C. Temporal
Editorial note: Raphaël 
For all dimensions covered by MPEG-7 the use of indirection should not forgotten. http://www.example.com/mpeg7file.mp7#speaker refers to the "speaker" xml element of this resource. The UA needs to parse this element in order to actually point to this fragment.

A video is divided into VideoSegments that can be described by a timestamp. MediaTimes are described using a MediaTimePoint and MediaDuration, which are the starting time and shot duration respectively. The MediaTimePoint is defined as follows: YYYY-MM-DDThh:mm:ss:nnnFNNN (Y: year, M: month, D: day, T: a separation sign between date and time, h: hours, m: minutes, s: seconds, F: separation sign between n and N, n: number of fractions, N: number of fractions in a second). The MediaDuration is defined as follows: PnDTnHnMnSnNnF with nD number of days, nH number of hours, nM number of minutes, nS number of seconds, nN number of fractions and nF fractions per second. The temporal fragments can also be defined in Time Units or relative compared to a defined time. This MPEG-7 example describes a 'shot1' starting at 6sec 2002/2500 sec and lasts for 9sec 13389/25000 sec.

C. Spatial

Selecting a spatial fragment of the video is also possible, using a SpatialDecomposition-element. This MPEG-7 example describes a spatial (polygonal) mask called "speaker" which is given by the coordinates of the polygon: (40, 300), (40,210), ..., (320,300).

The spatial video fragment can be combined with temporal information thus creating a SpatialTemporalDecomposition-element.

A region of an image can also be described in MPEG-7

C.3.2.5 ImageMaps
C. Spatial

Client-side image maps: The MAP element specifies a client-side image map. An image map is associated with an element via the element's usemap attribute. The MAP element content model includes then either AREA elements or A elements for specifying the geometric regions and the link associated with them. Possible shapes are: rectangle (rect), circle (circle) or arbitrary polygon (poly)

Server-side image maps: When the user activates the link by clicking on the image, the screen coordinates are sent directly to the server where the document resides. Screen coordinates are expressed as screen pixel values relative to the image. The user agent derives a new URI from the URI specified by the href attribute of the A element, by appending ? followed by the x and y coordinates, separated by a comma. For instance, if the user clicks at the location x=10, y=27 then the derived URI is: http://www.example.com/images?10,27

C.3.2.6 HTML 5
Editorial note: Silvia 
Currently, HTML5 relies on the abilities of the used media format for providing media fragment addressing. In future, HTML5 is planning to adopt the fragment URI specifications of this document for providing fragment addressing. Input from the WHAT and HTML working groups is requested.

D Acknowledgements (Non-Normative)

This document is the work of the W3C Media Fragments Working Group. Members of the Working Group are (at the time of writing, and in alphabetical order): Eric Carlson (Apple, Inc.), Michael Hausenblas (DERI Galway at the National University of Ireland, Galway, Ireland), Philip Jägenstedt (Opera Software), Jack Jansen (CWI), Yves Lafon (W3C), Wonsuk Lee (Electronics and Telecommunications Research Institute), Erik Mannens (IBBT), Thierry Michel (W3C), Guillaume (Jean-Louis) Olivrin (Meraka Institute), Soohong Daniel Park (Samsung Electronics Co., Ltd.), Conrad Parker (W3C Invited Experts), Silvia Pfeiffer (W3C Invited Experts), David Singer (Apple, Inc.), Raphaël Troncy (EURECOM), Davy Van Deursen (IBBT)

The people who have contributed to discussions on public-media-fragment@w3.org are also gratefully acknowledged. In particular: Olivier Aubert, Werner Bailer, Pierre-Antoine Champin, Cyril Concolato, Franck Denoual, Martin J. Dürst, Jean Pierre Evain, Ken Harrenstien, Kilroy Hughes, Philip Jägenstedt, Ryo Kawaguchi, Véronique Malaisé, Henrik Nordstrom, Yannick Prié, Yves Raimond, Julian Reschke, Geoffrey Sneddon, Felix Sasaki, Philip Taylor, Christian Timmerer, Jorrit Vermeiren and Munjo Yu.

E Change Log (Non-Normative)

@@This paragraph will be replaced by the change log. DO NOT TOUCH@@