<scribe> scribenick: kaz
Chris: We ran a call for consensus
around Christmas, now closed.
... No comments, so that is approved to go ahead.
... Would like to propose we review the comments by Mark,
... then take the snapshot when that's done, rather than publishing
the Note right away.
... I'm working through the comments right now.
... What do you think?
(no objections)
Chris: Mark has left some really helpful comments.
Chris: Some of this is to do with the
document formatting, some I've addressed already.
... The main point to discuss is the use case section,
... and we've not covered important use cases such as ad
insertion.
... Is there existing text that would fit the draft? If not, I can
write something.
... His other comment mention EBIF. Can include if it leads to
another use case or requirement.
Chris: I'll look at the use cases,
to reformulate them.
... He also mentions the WebVMT case which leads with the
technology rather than the use case.
Rob: I can look at that.
Chris: timed interaction event
... Some of the rest of the comments are about making the document
clearer.
... Would be good to get the item on ad-insertion cues into the
doc,
... one of the big drivers coming from CTA WAVE, so it's important
to include.
... In section 4, he mentions adding CMAF. We talk about emsg in
reference to DASH but not CMAF.
<Zakim> nigel, you wanted to propose that we clarify that they are demonstrating requirements described elsewhere in the doc, when we get around to the query about why various docs are in
Nigel: Mark makes some good points,
he's not sure why some of those documents are there.
... Not all of those are standards, more guidelines.
... They're there because they provide evidence for the
recommendations in section 6.
... But that's not clear. I could suggest a restructuring,
... it would be helpful to make each major section a use case,
describe the evidence for why change is needed,
... and the recommendations related to that.
... For example, synchronization, which isn't mentioned as a use
case, rather the gap analysis.
... At that point, we might want to mention BBC subtitle gudelines,
quote relevant part, make conclusion.
... Also a question about why WebVTT is there.
... HbbTV is about emsg, that's OK.
... This could make it more readable and clearer, and why the
recommendations are there.
Chris: I have made some updates.
There is an open PR, where I'm making changes based on the
comments.
... For example, I describe why I included WebVTT, because you can
use it for out of band events.
... There's an example of it in the HTML spec.
Chris: looks like a spot of
programming
... In response to Mark's comment, I've updated the section
4.6
... should be clearer now
... Regarding BBC subtitle guideline, I've shortened it to the main
point,
... which is about aligning captions with shot changes.
Chris: It's about extracting from
that reference what the motivation is.
... I'm hoping that I can keep much of the existing
structure,
... because it would take work to reorganise.
... So I'm hoping that the changes I'm making will make the
document more cohesive and clearer.
Nigel: OK, we can review the changes.
Chris: What I want to get to is
publishing this doc, then focus on the explainer for WICG,
... which the browser implementers will need, copy from this
doc.
Chris: You can review the changes
from the PR. When it's ready for merge, we can do another
review.
... (going back to the issue 29)
... I'm currently studying the time marches on algorithm.
Chris: On the last call, Nigel, you
mentioned 20ms as a number we could use,
... being half of the typical frame rate, 25 FPS.
... Is this simply a case of reducing the 250ms for timeupdate
events down to 20ms,
... or is it a more involved changed, but we're getting into
design.
Nigel: It is a complicated algorithm,
it's doing a few things at once.
... The safest thing would be to change as little as
possible,
... simplest to reduce the number from 250ms to 20ms,
... which would meet most of the use cases.
... I don't think it's possible to do a lot better.
... If what you want is for the main events to progress smoothly
and handle an arbitrary set of events,
... each could take any time to process, and it's single threaded,
there's a lot of constraints.
... The best you could do is to say that if you author your page so
that nothing takes a long time,
... at least we'll honour that and process the events with better
synchronization than at present.
Chris: Yes, if you have a long
running event handler, the more time there is to miss events.
... I thought maybe a different approach could be to deliver all
the events on the timeline since the handler was last
invoked.
... But really what I want to do here is formulate it as a
requirement statement.
... I've got some draft text, so I'll put that into the document
for review.
... The issue that Jon Piesing raised was related to the HbbTV
implementation, which is a native DASH player.
... There, the UA is responsible for extrcting the events and
creating the cues.
... If you do it in a web app, you can add onenter/onexit handlers,
and time marches on will guarantee all those will be called.
... If instead you're using the cuechanged event and inspecting the
activeCues list, then you'll possibly miss cues.
... Maybe this too much detail for now. We can save the detail for
the next phase, the WICG incubation.
... In addition to Mark's comments, there are many "Editor's Notes"
within the draft, need addressing.
... What do you think about the Note right above 5.1.2?
5.1.2 Synchronization and timing
[[
Editor's note
To support DASH clients implemented in web applications, there is therefore either a need for an API that allows applications to tell the UA which schemes it wants to receive, or the UA should simply expose all event streams to applications. Which of these is preferred?
]]
Chris: This could enable
optimization. HbbTV use opt-in, where applications subscribe to
kinds of events.
... This could be our recommendation in general.
... Do we have a point of view for this, or maybe it doesn't matter
either way?
... If so, could possibly remove some of the recommendations.
-> The API should allow web applications to subscribe to receive specific event types.
6.1 Subscribing to event streams
Nigel: It could be good practice to
subscribe, it's not just optimization for implementations but
optimization for network traffic as well.
... Reduce amount of data on the network. If there are say 20
different event streams available,
... subscription would make sense, reduce mobile bandwidth.
... That's an architectural thing. When that API gets built and who
wants to use it first is another question.
Chris: The next recommendation is about out of band events,
Chris: needed to support WebVMT,
where the events are delivered separately to the video.
... The Webkit implementation should be OK for this.
... For event triggering, there are the different modes coming from
DASH-IF.
... Mode 1, when parsed from container, mode 2, at the right time
on the timeline.
... Not sure if mode 1 is currently supported, but can review that
later.
<tidoust> [I note en passant that Out-of-band events could be useful as well for "Bullet Curtain" events that Song raised]
Chris: and in-band event processing
Chris: In-band tracks to be updated? Or specify elsewhere?
Sourcing In-band Media Resource Tracks from Media Containers into HTML
Chris: For now, I've said we
recommend updating this to handle other kinds of in-band
events,
... and consider splitting it into a registry, with a spec per
media format.
... Another large piece of work to do.
... Any comments or suggestions?
Ali: I'd like to review those
sections.
... There was an MPEG meeting last week, a lot of time spent on
events. There a lot of new recommendations.
... I would like to make sure we are lined up with your
recommendations.
... I'll ask some others to look at your spec.
Chris: Excellent, thank you.
... Can I ask you a question? We refer in our document to the
Carriage of Web Resources in ISO BMFF
... Should we still reference this draft, given we're really
focused on emsg and out of band events. Or should we follow up
separately?
-> https://w3c.github.io/me-media-timed-events/#mpeg-working-draft-on-carriage-of-web-resources-in-iso-bmff https://w3c.github.io/me-media-timed-events/#mpeg-working-draft-on-carriage-of-web-resources-in-iso-bmff
Ali: It may be not so relevant. It's
still a working draft, but it's good idea to keep it here unless
there's a good reason not to include it.
... There could be a future edition.
Chris: I'm thinking of Mark's comment
about explaining why we have these related standards.
... I'll add a note about the current status. At this state it's
not clear that it brings requirements.
Ali: That's true, and it might be the case in future too, but also a good idea to keep an eye on it.
Chris: I agree. If you want to come and talk with the IG about that, very welcome.
Ali: Sure.
Chris: Rob, would you like to give an update on some of the recent conversations we've had?
Rob: Back at TPAC in Lyon, we made a
proposal to take this to WICG and Chris has created a repo for this
work.
... That's been blocked for some time, but now it's moving ahead,
we have a first draft of an explainer document.
Rob: I propose expanding the
structure, to include use cases from the media timed events
document and gap analysis.
... Also to include placeholder sections to allow others to
contribute.
... I identified two areas as new sections needed are: current
implementations
... (we have WebKit implementation currently), but also another
equivalent section for other browsers.
... The gap analysis leads to identified requirements.
... I take on board the comment about use cases, so we can come
back to WebVMT.
... Start with use case, lead to requirements. This is a good
start.
... Chris suggested using his GitHub to can assemble the
document.
... The 5 use cases in the Media Timed Event Note are a good
start,
... refine those to simple testable items. In the end we can show
we've made progress, tests to show we've satisfied the
requirements.
Rob: Thinking of timing
requirements, I don't know how much scope there is to meet those
requirements, part of the investigation process.
... The final thing discussed is the existing TextTrackCue object
in HTML, has an end time which is mandatory.
... In WebVMT streaming use cases you don't know when the end time
is.
... It's helpful to be able to create a cue that extends to the end
of the media, and we don't care when that is.
... Eric has said there's wording already there, but it seems to be
for reading rather than writing, a cue with unknown end time.
... It sounds like a good solution.
Nigel: That's interesting one. A
couple of comments.
... One is not yet knowing the end time, in principle also applies
to live subtitles,
... but are you sure you really need a possibly null end
time?
... Can you architect the system so that even if the cue has a
defined end time, that doesn't necessarily mean that you have to do
something at that end time.
... You could either have a null onexit handler whose time is
irrelevant, or far in the future,
... or you could define a semantic where immediately prior to
firing the onenter) handler of a cue,
... make sure all the onexit handlers of previously entered but not
yet exited cues are handled first, in that order.
... I'm just wondering if it's necessary to have an exit
time?
... The time marches on spec wants to do filtering on the start
time and end time.
Rob: My understanding is that you
specify the start time and end time of the cue.
... If you don't specify the end time, how do you know when to stop
displaying the cue?
... Or how do you refer back to that event to update it at a later
time?
Nigel: There's an issue with live
captions, like VTTCue, where appearance/disappearance is managed by
the UA.
... For live captions, the end time is not known at authoring
time.
... Could architect DataCue differently to avoid that
problem,
... where a new cue arriving effectively modifies the previously
visible cue's end time.
Rob: Looking at it from a metadata
point of view, the use case is drawing a track on a map.
... A common use case is we have video and location coming in, and
we want to update the line on the map.
... We don't want it to disappear, it should continue for the
duration of the media clip.
... We want the whole journey to be displayed at the end of the
media clip.
... The cues are successive sections of a line.
... We could either continually update the end time of a cue, or
just say that it extends to the end.
... Seems simple to me, but I'm open to other suggestions.
Nigel: If your cue represents where
you are in space within a period of time, and you make all the end
events the same,
... that suggests you're at lots of places simultaneously. But
that's not what you mean.
... There's another model, where you could update some data state,
so each new event ends the previous event.
... This could apply in other cases as well, where each cue
automatically ends the previous cue.
... I think that the issue of identifying the location applicable
at each moment during the presentation is separable from the issue
of whether or not to draw the line.
Rob: This is a discussion for offline?
Chris: Another example is
chapterisation of live video, where you don't know where a chapter
ends at the time it starts,
... so you want the ability to start the event and update when it
ends.
... Can we pick this up separately?
Nigel: Yes
Chris: There's discussion thread
between Rob and myself and people at Apple.
... I'm keen to continue discussions in public.
... Ideally this should go under the WICG repo, but for now it's
under my personal GitHub account,
... following Marcos's suggestion. Need to follow up with him about
that.
... He's raised it with Mozilla.
<cpn> https://github.com/mozilla/standards-positions/issues/122
Chris: My overall plan is continuing
this as an IG document, continue on the explainer.
... When the IG doc is ready, circulate that among the browser
guys.
... Next phase is incubation and spec development, needs their
involvement.
... Who will write the API definitions?
... Need to have the right information written down to have
discussions.
... This DataCue explainer is just a copy of the issue we posted to
the WICG Discourse forum,
... feel free to raise issues there.
Frame accurate seeking of HTML5 MediaElement
Chris: There's a long discussion
thread on frame accurate seeking, then goes into frame accurate
rendering,
... interest from non-IG participants as well. I'd like to follow
up on it, but it's not really in the scope specifically for the
current DataCue work.
<nigel> +1 to what cpn said - cue timing precision has no impact on seeking accuracy, the latter is a separate problem.
Chris: Is this something any of you have strong feelings about?
Nigel: You're right, it's a separate
problem. Not sure how much work is really needed.
... There are web apps that do frame accurate seeking, it may be
painful.
... May not be a strong requirement if there are workarounds.
<tidoust> [I don't think that's limited to "seeking" accuracy, though]
Chris: Discussion led to rendering issues.
<tidoust> [I believe I said I'd take a first stab at a document that summarizes the discussion. I still haven't done that but I'm still happy to!]
Chris: AOB?
(none)
Chris: Let's continue with the calls
until the docs are done and we can close the TF.
... Once the MTE discussion moved to the WICG side, maybe we still
want to have calls. Depends how people want to work.
Rob: When is the next call?
Chris: We meet on the 3rd Monday of each month, so Feb. 18.
Rob: OK
Chris: The document should be done by that time.
[adjourned]