Media Fragments Working Group Teleconference -- 17 Apr 2009

<trackbot> Date: 17 April 2009

Silvia, you miss some amazing food yesterday night, but you will see the pretty pictures taken :-)

you miss as well the longggggggggggggg drink session :-)

trackbot, start telecon

<trackbot> Meeting: Media Fragments Working Group Teleconference

<trackbot> Date: 17 April 2009

<silvia> any changes to the agenda?

<scribe> Meeting: Media Fragments WG, 3rd F2F Meeting - Barcelona (Spain)

yes Silvia, we will start with the session: Implementation Reports

scribe: then talk about the issues
... and finally about the test cases, where we need to kick off the work

Does that suit you ?

<silvia> sure

Silvia, we are waiting for Jack, coming in a few minutes

scribe: and Yves, that he could phone us

<scribe> Scribe: Raphael

<scribe> Scribenick: raphael

1. Quick summary of First day meeting

Raphael: we should have a picture representing the general structure of the media object, the H, H', K, D, etc. we talked about yesterday
... Can we do such a representation valid for most of the formats?

Conrad: yes, for the streamable formats
... I'm happy to take an action to draw such a schema

trackbot, status?

<scribe> ACTION: Conrad to draw a representation of the general structure of a media resource, for streamable formats [recorded in http://www.w3.org/2009/04/17-mediafrag-minutes.html#action01]

<trackbot> Created ACTION-69 - Draw a representation of the general structure of a media resource, for streamable formats [on Conrad Parker - due 2009-04-24].

2. Implementation Reports

a) Davy wants to present and demo a software that allows to slide media files independently of the codec format

b) Jack has some reporting to do on its implemntation

c) Davy wants to talk about RTSP implementation

d) Conrad wants to report on his experience in Annodex

Silvia, Davy will start with some slides that are coming

Davy going through a presentation

Slides at: http://www.w3.org/2008/WebVideo/Fragments/meetings/2009-04-16-f2f_barcelona/Adaptation_logic_for_(server-side)_media_fragment_extraction.pdf

Davy: Slide 6, my PhD work is built on BSD content adaptation, using RDF
... based on a model for media bitstreams encoded in OWL
... Slide 7: Coding formats -> model for media bitstreams -> container formats
... Slide 8: model for media bitsreams represented as a set of classes and properties
... left part is the structure part, while the right part is more semantic
... Slide 9: shows the workflow
... Slide 10: shows an example, two tracks (audio, video) represented in the intermediary model
... assume there is a mapping between one particular codec format and this model
... Track selection, means selection of MediaBitstreams, straightforward
... Temporal selection, means selection of data blocks, based on their timestamps
... problem for Spatial clipping?

Jack: it works only for scalable formats for Spatial Fragment

Davy: Slide 11: general architecture of the NinSuna platform
... the retrieval service is not yet compatible with the media fragment syntax, but we could adapt it
... Slide 12: two remarks, 1/ is the media needs to be 'ingested', i.e. the intermediary model needs to be instantiated and 2/ currently client decides the output format

Time for demo

<mhausenblas> DON'T CLICK on the following link, now:

<mhausenblas> http://respighi.elis.ugent.be/NinSunaWeb

<silvia> what is the link for?

<mhausenblas> silvia: that's davy's demo :)

<mhausenblas> note that the demo requires IE and Silverlight

<silvia> ah, thanks

scribe: and VLC for the playback

<Yves> raphael, no

<Yves> we will put thata on another server

<erik> our demo can be found at http://multimedialab.elis.ugent.be/NinSuna/

<silvia> is there a screencast?

<silvia> erik?

A screencast will be made by the IBBT guys next week

<silvia> ok - those screenshots at the demo site are good, too

I'm uploading the videos on my web page

cofffee break, we are back in 15 min

Videos will be at http://homepages.cwi.nl/~troncy/tmp/spool/video/

<mhausenblas> coffeeeeeeeeeeee

Silvia, we will resume

Raphael: now we will listen to Jack

Jack: I took the Python URL library
... I took this library and add some code of myself
... for parsing the media fragments part
... and here I discover all the issues discovered and reported, now fixed
... and then the problems started, I tried first with the file URL (local resource)
... problem is with synch on video/audio, should we do transcoding or not, etc.
... my goal was to first address: file://myvideo.mp4#t=10,20
... use ffmpeg for slicing the media
... it's hard to use it, the API changes frequently, but the functionalities are very good
... at the C level, it does what we want, but at the command level, it is meant to do transcoding

Guillaume: why not using the Python library G-streams, an API for ffmpeg?

URL lib for Python: http://docs.python.org/library/urllib.html

<guillaume> Gstreamer Python binding library : http://gstreamer.freedesktop.org/modules/gst-python.html

<guillaume> and GStreamer itself http://www.gstreamer.net/

Jack: I will have a working version in a few more days, but it will transcode
... since this is what ffmepg does
... my plan is thus rather going through C++ implementation, to attack ffmeph low layers, and have no transcoding
... I can share the code that does the parsing of the fragment

<mhausenblas> Anyone done or plans to do a client-side implementation of the parsing, etc., say in JavaScript

<mhausenblas> if not, I might have some resources (not myself, for sure, but a PhD of mine) who could do something based on jQuery

<scribe> ACTION: Jack to commit in CVS (code directory) his python code doing the parsing on client side of the media fragment [recorded in http://www.w3.org/2009/04/17-mediafrag-minutes.html#action02]

<trackbot> Created ACTION-70 - Commit in CVS (code directory) his python code doing the parsing on client side of the media fragment [on Jack Jansen - due 2009-04-24].

<scribe> ACTION: Michael to investigate whether he could have an implementation in Javascript that does the client-side media fragments implementation [recorded in http://www.w3.org/2009/04/17-mediafrag-minutes.html#action03]

<trackbot> Created ACTION-71 - Investigate whether he could have an implementation in Javascript that does the client-side media fragments implementation [on Michael Hausenblas - due 2009-04-24].

Conrad will now report on his implementation experience

Davy, first report on RTSP implementation

http://homepages.cwi.nl/~troncy/Talks/2009-03-06-mozcamp/#(3)

Davy: RTSP implementation wiki page, http://www.w3.org/2008/WebVideo/Fragments/wiki/RTSP_implementation
... quick test, I have an implementation of the media fragment over the RTSP protocol
... it works for track and temporal selection
... first communication is the Describe, and the server answers which track is available, etc.
... then for each track, we have a Setup
... so we can have easily a track selection in RTSP
... next command is the play command, where we can specify the temporal clipping
... I don't remember which units is supported, but for sure npt and smpte
... again, I have an implementation that parses the media fragment, and generates a PLAY command
... i do not yet cover the whole grammar of the media fragment, so far just, t= and track=
... implemented in C
... I'm working on covering the whole grammar

Raphael: my (silly?) idea, embed VLC in a web page
... implement on client side, in the browser, the possibility to write down a media fragment URI (rtsp), use the code of Davy to generate the good Describe, Setup and Play commands
... send that through VLC and get the results back, using the VLC plugin in the browser

Davy: should be doable
... my issue is with track selection, how do you know the name of the track

Jack: this is related to the issue raised by Silvia, http://www.w3.org/2008/WebVideo/Fragments/tracker/issues/4
... what I come accoss, is that often, you want not 1 track, but 2 tracks, or all tracks except one
... not very convenient with our current scheme?

Silvia: suggestion to add a comma separation, such as: #track='track1,track2,track3'

Jack: or event having a minus

<silvia> the naming of the tracks is up to the container format - some allow it, others don't

<mhausenblas> Scribenick: mhausenblas

<silvia> for example QuickTime has chapter tracks that should be addressable in the given way with #track

;)

Conrad has essentially said everything yesterday, now no new stuff

<silvia> Jack: mentions that naming of tracks is decided by author and they create the URL

<silvia> Conrad: mentions that with Accept-Language it should be possible to leave the track selection for languages to the server

raphael: conrad ok for you to implement it based on # rather than ?

conrad: will try to implement the hash part as well

raphael: agree, after the first step both # and ? have the same procedure
... conrad would you implement the 'missing link' using the HTTP Link: header draft

conrad: yes, I like Yves idea using it, gonna try

raphael: Yves, silvia any question?

Test Cases

<scribe> Scribenick: erik

yves: give a set of URI's and describe what should be expected

michael: rather have a set of simple cases to start with

<Yves> parsing URI is more or less done (via the code I contributed), so it's more how a cient parse que URI and act based on that

michael: really stress on what the outcome is
... try to make it as automatic as possible

Jack: some temporal testing should at least be able to be done automatically (cfr. via using subtitles)
... some manual video-inspection still has to be done

<conrad> we could add time range validation to http://validator.annodex.org/ :-)

Jack: because we use half-open intervals

<conrad> http://validator.annodex.org/?uri=http%3A%2F%2Fia331343.us.archive.org%2F2%2Fitems%2Fnight_of_the_living_dead%2Fnight_of_the_living_dead.ogv%3Ft%3D10%2C20

michael (on blackboard)

on UA:

<jackjansen> erik, I wanted to say "because we use half-open intervals we can also do some automatic testing on video/audio"

two test cases: URI resolving & Generate HTTP request (also see photo)

on Server:

from HTTP request to HTTP response

back on UA:

comment from jack: this is testing from User Expectation ... shouldn't we just test our Spec?

<Yves> we need to test if it's feasible to get interoperable implementations

<Yves> so we test the spec, but also (parts of) implementations as well

<Yves> we won't test that implementations are 100% correct

<Yves> we just test 100% coverage of the spec by interoperable implementations

michael: try the top-down approach (opposed to Jack's proposal of doing it bottom-up)

raphael: in the end both should come up with the same results though

(human in the loop) on UA

<conrad> on the last step ("human in the loop") we should document the expected behaviour of a UA: that the first frame rendered is the first frame of the interval, that the video image is not broken (ie. the decoder has been primed with the previous i-frame)

TestCase 0: URI#

-> == URI == entire resource

Jack: at very least for every dimension we should define all the test cases where we get back the whole resource & also the ones where we get back an empty resource

<guillaume> see http://www.w3.org/2006/07/SWD/RDFa/testsuite/ for example

<guillaume> http://torrez.us/code/rdfa-js/tests/ and http://rdfa.digitalbazaar.com/rdfa-test-harness/

Yves: there are tools online that check HTTP responses

<Yves> I agree we just need to test that the headers we cared for are ok

michael: we should test our extra HTTP-headers for example ... not the complete HTTP stack

raphael: practically ... just test specific HTTP-headers with name & their possible values

Jack & Conrad: tests are most easily done on server

<guillaume> see http://wiki.csswg.org/test/harness

conrad: two test harnesses ... one for client & one for server

conrad & jack: both test harnesses can be scripted

<conrad> yes

<jackjansen> I can be scripted, but the qeustion is: do I do what you expect.....

<scribe> ACTION: michael to setup template for test cases on wiki [recorded in http://www.w3.org/2009/04/17-mediafrag-minutes.html#action04]

<trackbot> Created ACTION-72 - Setup template for test cases on wiki [on Michael Hausenblas - due 2009-04-24].

raphael: initial 10 test cases ... test case 0 & empty/entire resource & only temporal and npt-units

<conrad> if we are testing a client, the base uri is the uri of the harness

<conrad> if we are testing a server, we need to provide media and test fragment requests to that media

jack: why not only within test harness just the fragment (not the whole URI) to make it automatically testable

erik: what about RTSP?

jack: we do end-to-end test for multiple protocols

raphael: how was the test suite handled within SMIL?

Jack: SMIL 2.0 test suite was xml & then parsed by bits of perl/python
... then put into HTML forms, again some script processing & afterwards put into a document ... all testing was done manually :(
... please, choose the right media to run the tests!

<raphael> invite zakim #mediafrag

<raphael> trackbot, start telecon

<trackbot> Meeting: Media Fragments Working Group Teleconference

<trackbot> Date: 17 April 2009

<raphael> scribenick: raphael

Wrapping up the test cases

Conrad: have a test case with a #foo fragment
... on a non-media resource, if the behaviour is as expected

Dave: test whether the video.mp4#t=10 (as used by Google video) still works as expected

Michael: we should completement with more negative test cases

4. Issues

http://www.w3.org/2008/WebVideo/Fragments/tracker/issues/open

Issue 3: Michael, http://www.w3.org/2008/WebVideo/Fragments/tracker/issues/3

Michael: in RFC3986, the semantics of the fragment is per design not defined
... together with the mime-type, one can learn what the semantics means
... example, HTML, with its registration
... but this is not true for most of the media types
... to register at IANA and IETF the media fragment syntax

Jack: this is not possible, mp4 has already its post-hash syntax
... we could write a cover letter, explaining who we are, and recommend to the mime-type owners what they should register

Michael: there are plenty, is this possible at all?
... it will take too much time

Member only link: http://www.w3.org/2009/02/06-swcg-minutes.html#item03

<mhausenblas> proposal: media/fragment

<mhausenblas> media/fragment+video

<mhausenblas> media/fragment+audio

<mhausenblas> media/fragment+image

Conrad: it's a hack to use the Accept header for that
... we need to do a media specific documentation

Davy: problem, MPEG-21 has done that for MPEG resources, why they would change and adopt our scheme?

Conrad: we have no authority, we could just convince them with existing implementations, and deployment

Michael: I need to take my action seriously and come back with figures
... how many mime type owners we need to talk with ?
... how many mime-types will be affected?

Jack: looking at IANA database, it is very hard to guess who is the owner

Michael: my method, look at all the mime-type concerned (audio/*, video/*, image/*) and look for each whether they have a semantics for the hash
... if NOT, no problem
... if YES, then, identify the owner and evaluate the impact
... evaluate impact meaning detecting clash

Conrad: we should talk to the AVT group in IETF
... charter: http://www.ietf.org/html.charters/avt-charter.html
... group home page: http://tools.ietf.org/wg/avt/
... I think Dave Singer is somehow associated to this group

Dave: can you confirm the asumption of Conrad just above?

Issue 4: http://www.w3.org/2008/WebVideo/Fragments/tracker/issues/4

<dsinger> I am looking for the assumption

Are you associated to some extent to the AVT group in IETF ?

Raphael: I feel the Issue 4 is mixing two things

a) the fact of selecting various tracks, or all tracks except 1

b) pre-selecting track names

Jack: for a) I think it should go to the version 2 of Media Fragments
... because we will end up soon to a presentation language behind the hash
... the problem is we cannot measure what amount of work it represents, nor to the people we need to talk too

<conrad> http://wiki.xiph.org/index.php/ROE

Davy: why not doing like RTSP ...

Raphael: this is the ROE way

Jack: we assume there is a mechanism for which the client can have a description of the media items

Conrad: clarification, the retrieval of the description is not in the media fragments, it is done before

<Yves> http://www.iana.org/assignments/media-types/video/

<Yves> so it includes lots of RFCs (not easy to republish)

<mhausenblas> Michael: re issue 3, I mean it is always good to write the cover letter

<mhausenblas> ... just a matter of how the odds are that we can convince them, looking at the sheer number of potential clashes

<mhausenblas> ... where a clash is defined as a media type that defines frag semantics

Yves, hopefully, there will be 0 or 1 RFC impacted

scribe: we really care when there is a clash

<mhausenblas> ... and these semantics contradict with ours

<Yves> indeed

Jack: no, the syntax will not change
... we pass verbatim the track="value" to the server, and this is up to the server, capabilities of container formats

Conrad: I agree with you
... I think what we can get is just no audio or no video

Jack: I dont know what get 'audio' means in the general case

<scribe> ACTION: Conrad to change the phrasing of the issue 4 (just audio/video)? [recorded in http://www.w3.org/2009/04/17-mediafrag-minutes.html#action05]

<trackbot> Created ACTION-73 - Change the phrasing of the issue 4 (just audio/video)? [on Conrad Parker - due 2009-04-24].

Issue 6: http://www.w3.org/2008/WebVideo/Fragments/tracker/issues/6

Jack: the issue comes during the implementation trial, where I happily do some transcoding

<Yves> my point was... "it is fuzzy"

<mhausenblas> +q to ask for sanity check

<Yves> I can point to... http://tools.ietf.org/html/draft-ietf-httpbis-p6-cache-06#section-3.6

Raphael: yes, thus the action-62, http://www.w3.org/2008/WebVideo/Fragments/tracker/actions/62 now associated

<Yves> Warning: 214 Transformation applied

<Yves> (note that is is also in rfc2616)

<conrad> 4.3.2 Receipt of Warning: 214 Transformation Applied

<conrad> If the response includes a Warning: 214 Transformation Applied HTTP header, proxies must not apply further transformation.

Proposal from Yves: not really implemented, but we could use the warning header to warn the UA there has been some transcoding

<Yves> oh proxies can do further transformations

<Zakim> mhausenblas, you wanted to ask for sanity check

<Yves> it just warns the client that there was a transformation somewhere in the chain

<conrad> i was quoting from http://www.w3.org/TR/ct-guidelines/

<mhausenblas> Michael: is this within our scope?

<mhausenblas> ... would like to see WG saying: YES!

Jack: should we care, as a standardisation group, whether there is transcoding or not

<conrad> for formats that do not encode presentation in-points, we could add an HTTP response header to inform the client of the presentation time at which to enable rendering

Issue 5: http://www.w3.org/2008/WebVideo/Fragments/tracker/issues/5

Jack: this is just a heads up, for us
... we must remember if the server has done some clipping, it must send also the original w/h to the client
... in the case where the spatial cropping has been asked in terms of %

Raphael: my concern, are we stretching too much the definition of a fragment, when the fragment (= spatial clipping) has no single bytes in common from the original resource
... I hope this will be answered by Yve's action to TAG
... perhaps, we will say, this is not anymore a fragment, and recommend to use the ? rather than the # for this use case

Frank: this is an analogy of what is said for spatial but not said for temporal

5. Wrap Up

<davy> scribenick: davy

raphael on the blackboard

raphael: uc & req doc will be split
... use cases and requirements and side conditions will be in a separate document
... technologies survey in another document

guillaume: uc & req should be first, then side conditions

raphael: historically, side conditions were first

mhausenblas: put survey in the uc & req doc

proposal: uc & req doc and the technologies survey in one doc?

<erik> +1

<mhausenblas> +1

guillaume: duplicate side conditions (both in working note and REC docs)

<jackjansen> +1

jackjansen: in the rec, side conditions will be specified more formally

raphael: spec will be rec

<Yves> wg note as well for UC&req&survey

raphael: uc & req will be working note?

all: working note

<mhausenblas> should http://www.w3.org/2005/Incubator/app-backplane/ be in

<scribe> ACTION: raphael and erik to request feedback of other groups such as SYMM, SVG, HTML + WHATWG, WAI, MAWG, TAG, MobileWeb, TimedText, ... [recorded in http://www.w3.org/2009/04/17-mediafrag-minutes.html#action06]

<trackbot> Sorry, couldn't find user - raphael

<scribe> ACTION: Erik and Raphael to request feedback of other groups such as SYMM, SVG, HTML + WHATWG, WAI, MAWG, TAG, MobileWeb, TimedText [recorded in http://www.w3.org/2009/04/17-mediafrag-minutes.html#action07]

<trackbot> Created ACTION-74 - And Raphael to request feedback of other groups such as SYMM, SVG, HTML + WHATWG, WAI, MAWG, TAG, MobileWeb, TimedText [on Erik Mannens - due 2009-04-24].

raphael: planning for the next 6 months
... second public wd within 2-3 months

jackjansen: 2nd wd in september

raphael: if people do not provide feedback, we should find out why
... Goal is to have a 2nd WD early in september

erik: will the document already be split?

raphael: yes, what about test cases?

mhausenblas: would leave it in the wiki for a while

raphael: current policy regarding editorship
... which policy should we have?

jackjansen: list everyone in the group or list everyone contributed a chapter

mhausenblas: if someone writes something in the rec, then he/she is an editor
... have two editors, then the contributers

jackjansen: then we can assume that everyone is a contributor

<Yves> yes, all the WG members at least

raphael: looking at other w3c docs, they use editors and a number of contributors

AOB

<raphael> Raphael: proposal on the blackboard is to have a number of editors (chairs ? team contact? others?) and all group members as contributors

<scribe> ACTION: raphael to split the document and setup the diff feature of xmlspec [recorded in http://www.w3.org/2009/04/17-mediafrag-minutes.html#action08]

<trackbot> Sorry, couldn't find user - raphael

<raphael> trackbot, status?

<mhausenblas> http://www.w3.org/2008/WebVideo/Fragments/wiki/Semantics

<scribe> ACTION: raphaël to split the document and setup the diff feature of xmlspec [recorded in http://www.w3.org/2009/04/17-mediafrag-minutes.html#action09]

<trackbot> Created ACTION-75 - Split the document and setup the diff feature of xmlspec [on Raphaël Troncy - due 2009-04-24].

<raphael> [adjourned]

<raphael> BIG THANKS for the local organizers

Media Fragments Working Group Teleconference

17 Apr 2009

Attendees

Contents

1. Quick summary of First day meeting

2. Implementation Reports

Test Cases

4. Issues

5. Wrap Up

AOB

Summary of Action Items

Scribe.perl diagnostic output