This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
Created attachment 1556 [details] Silvia's diagram The discussion below was started on public-inbandtracks@w3.org on Oct 27. I also share concerns about allowing non-“standard" mappings in the webapp. In addition, wouldn’t UA native mappings consume much less CPU and memory? This still matters on resource constrained devices such as Smart TVs. JP > From: Giuseppe Pascale <giuseppep@opera.com> > Date: Monday, November 3, 2014 at 3:04 AM > > there might be devices/specs (e.g. HbbTV) where dash is supported natively > and NOT via MSE, hence a "standard" mapping may still be useful, to avoid > people come up with their own. > > On Mon, Nov 3, 2014 at 10:33 AM, Silvia Pfeiffer <silviapfeiffer1@gmail.com> wrote: >> On Mon, Nov 3, 2014 at 10:33 AM, Silvia Pfeiffer >>> <silviapfeiffer1@gmail.com> wrote: >>>> On Mon, Nov 3, 2014 at 9:12 AM, Cyril Concolato >>>> <cyril.concolato@telecom-paristech.fr> wrote: >>>> Le 02/11/2014 21:30, Silvia Pfeiffer a écrit : >>>> On 3 Nov 2014 03:42, "Cyril Concolato" >>>> <cyril.concolato@telecom-paristech.fr >>>> <mailto:cyril.concolato@telecom-paristech.fr>> wrote: >>>>> >>>>> Hi Bob, Silvia, >>>>> >>>>> Le 02/11/2014 16:57, Bob Lund a écrit : >>>>>> >>>>>> >>>>>> On 11/2/14, 3:45 AM, "Silvia Pfeiffer" <silviapfeiffer1@gmail.com >>>>>> <mailto:silviapfeiffer1@gmail.com>> wrote: >>>>>> >>>>>>> On Wed, Oct 29, 2014 at 8:48 AM, Cyril Concolato >>>>>>> <cyril.concolato@telecom-paristech.fr >>>>>>> <mailto:cyril.concolato@telecom-paristech.fr>> wrote: >>>>>>>> >>>>>>>>> Le 28/10/2014 05:02, Abello, Jean-Pierre a écrit : >>>>>>>>> It would be awesome if the spec could also define mappings for HLS. >>>>>>>> >>>>>>>> Yes, I do think HLS is another format that we should target. It >>>>>>>> has some commonalities with existing formats that we target: DASH >>>>>>>> for the use of manifests and adaptative streaming aspects; and >>>>>>>> MPEG-2 TS as a segment format. The basics of exposing tracks >>>>>>>> from TS should be reusable. It might need some tweaking compared >>>>>>>> to what we started specifying, for instance to expose ID3 Tag >>>>>>>> PES streams. >>>>>>>> >>>>>>>> In an informal discussion with Eric (in copy), I discovered >>>>>>>> that WebKit already exposes them using some API, see: >>>>>>>> >>>>>>>> >>>>>>>> http://trac.webkit.org/browser/trunk/LayoutTests/http/tests/media/track-i >>>>>>>> n-band-hls-metadata.html >>>>>>> >>>>>>> For the record: I do have some reservations about adding DASH and HLS >>>>>>> support, since browsers do not typically support these formats >>>>>>> natively. >>>>> >>>>> [CC] I've heard that Webkit/GTK has some native support for DASH, but I >>>>> couldn't verify it. I would expect that in the future some browsers have >>>>> native support for DASH or HLS. But I agree with you that support for >>>>> DASH/HLS is different from direct support for TS/MP4/OGG... >>>>> >>>>>>> Actually, HLS is supported in Safari, so it has some excuse, >>>>>>> but DASH is only supported via Media Source Extensions. I have been >>>>>>> worried about that a bit. >>>>>> >>>>>> There has been text added to the spec for DASH using MSE. MSE behavior >>>>>> is that the UA sources tracks based on information in Initialization >>>>>> Segments. The application may specify default track attributes for >>>>>> those tracks, which the UA will use if those same attributes are not >>>>>> sourced from Initialization Segment data. It seems useful to me for >>>>>> the sourcing spec to describe this. >>>>>> >>>>>> On a related note, I plan to submit an MSE bug so that it references >>>>>> the sourcing spec for sourcing tracks as described above. >>>>> >>>>> What about adding a diagram like this one to the introduction: >>>>> http://concolato.wp.mines-telecom.fr/files/2014/11/inband-sourcing.png >>>>> and maybe indicating that some implementations may use one path or the >>>>> other. >>>>> >>>> >>>> Adding a diagram like this is useful, but I don't understand the one you >>>> made. In particular, all the media format parsing is happening in the UAs >>>> and not in an app. >>>> >>> In the diagram, I meant to say that: Browser+MSE+HTML Media+Other API is the >>> UA. Maybe that's not obvious. Can you suggest any change here? >>> Then I tried to carry to options in this diagram: >>> - when processing DASH (or HLS), the manifest is fetched and parsed by the >>> Web App, and then media data is fetched using XHR (other APIs) and passed to >>> the MSE part of the Browser. When it goes in the MSE API part in the >>> diagram, this is meant to say that some parsing is done (eg. MPD, or ISOBMFF >>> sidx ...) and then some part is done in the browser (in the MSE part). >>> - it is perfectly possible to process media in the Web app, it's exactly >>> what I do in mp4box.js. Other projects such as [2] do it for TS. >>> >> >> OK. It might be better to paint individual paths (parallel paths) for >> each of these different paths that a media resource can go through >> from being picked up by the browser all the way to rendering. Then >> identify what APIs are being used. >> >> I'm not very good at drawing - do you want to try this? > > After all, I gave it a try, see attached. > > The red arrows are what we are defining. I've tried to explain the > different paths through the UA APIs. > > Hope that makes sense? > > Cheers, > Silvia.
We'd also like to propose the following URL for testing HLS in-band timed metadata: http://nielsen.guru/id3test/prog_index.m3u8
Hello Silvia, Bob, Cyril, Eric and Giuseppe, I hope you are all doing well. Did you see the Jan 29 announcement about Microsoft adding native support for HLS and DASH in Internet Explorer on Windows 10? http://blogs.msdn.com/b/ie/archive/2015/01/29/simplified-adaptive-video-streaming-announcing-support-for-hls-and-dash-in-windows-10.aspx In the light of this new development, it might make more sense now to add HLS along with DASH in http://dev.w3.org/html5/html-sourcing-inband-tracks Silvia had also proposed back in November to move DASH and HLS into their own dedicated section in the document: > On 11/2/14, 2:45 AM, "Silvia Pfeiffer" <silviapfeiffer1@gmail.com> wrote: > > For the record: I do have some reservations about adding DASH and HLS > support, since browsers do not typically support these formats > natively. Actually, HLS is supported in Safari, so it has some excuse, > but DASH is only supported via Media Source Extensions. I have been > worried about that a bit. > > I think these facts probably need to be explained better in the > document. This is particularly important, since different target > audiences are addressed with the different formats: DASH and HLS > address Web Developers (+Safari), MPEG-4, WebM and Ogg address Browser > developers, and MPEG-2 addresses Web-enabled set-top box vendors > (these are still UA devs). So, DASH and HLS are bascially in a league > of their own and should probably be moved to an appendix or so. > > Silvia. Thanks, JP
Bob, would you be ok with reorganising the spec a bit? I'm concretely thinking about creating two main sections, the first being "Media File Fromats" and containing: * MPEG-2 Transport Streams * MPEG-4 ISOBMFF * WebM * Ogg and the second being "Manifest formats for HTTP Adaptive Streaming" and containing: * DASH * HLS According to https://developer.mozilla.org/en-US/Apps/Build/Audio_and_video_delivery/Live_streaming_web_audio_and_video , both HLS and DASH have some native browser support, so it makes sense to limit to these two formats.
(In reply to Silvia Pfeiffer from comment #3) > Bob, would you be ok with reorganising the spec a bit? > > I'm concretely thinking about creating two main sections, > the first being "Media File Fromats" and containing: > * MPEG-2 Transport Streams > * MPEG-4 ISOBMFF > * WebM > * Ogg > > and the second being "Manifest formats for HTTP Adaptive Streaming" and > containing: > * DASH > * HLS > > According to > https://developer.mozilla.org/en-US/Apps/Build/Audio_and_video_delivery/ > Live_streaming_web_audio_and_video , both HLS and DASH have some native > browser support, so it makes sense to limit to these two formats. Silvia, I think this is a good idea. We need to make sure that the relationship between the media file format and the manifest format is correctly reflected. I don't know about HLS but in DASH tracks and associated attributes are first created by information in the initialization segment of the media file, then from MPD information. Perhaps we should also reorganize the spec so that there are separate sections for inband track data format, e.g. WebVTT, TTML, CEA 608/708, and for cue formats. I find that these are redundantly described in the current spec. I would be happy to work with you on this. Last, can this reorganization wait until I update the spec to resolve bug 27306[1]? I can do this by my eod on Tue 4/7. [1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=27306
Hi Bob and Silvia, are you still planning on re-organizing the spec?
(In reply to JP Abello from comment #5) > Hi Bob and Silvia, are you still planning on re-organizing the spec? I'm somewhat waiting for Bob to finish https://www.w3.org/Bugs/Public/show_bug.cgi?id=27306 . But I've also not had time, so if somebody wants to prepare a pull request, be my guest.