Media Timed Events TF -- 16 Mar 2020

<tidoust> scribe: tidoust

<cpn> https://lists.w3.org/Archives/Public/public-web-and-tv/2020Mar/0009.html

Agenda

<scribe> Agenda: https://lists.w3.org/Archives/Public/public-web-and-tv/2020Mar/0009.html

cpn: Wondering if there's further work for the IG task force to do, or whether we should just fold the task force and go on with the DataCue incubation in the WICG.
... It would be good to go through open issues and decide what to do with them on the way to publishing an update of the IG Note
... Would like to talk about cues generated by user agent (~ in-band cues)
... Also report on discussions I've had recently with DASH-IF folks.

Review open issues

<cpn> https://github.com/w3c/me-media-timed-events/issues

cpn: #39 is I believe covered. Done through the HTML issue tracker to report the issue.
... I don't think there's anything more to do on this particular issue;
... I've added text to cover additional use cases that Mark raised

<kaz> draft wot minutes

kaz: FYI, the WoT group just had a meeting and discussed media use cases. You can review the minutes offline.

<cpn> https://github.com/w3c/me-media-timed-events/issues/36

nigel: The issue ended up with two new issues, but that's far from the actual title of the issue.

<kaz> PR 46

cpn: Pending pull request 46 is in response to #39. One part where I wanted Rob to provide feedback.
... Essentially, once PR #46 has been merged, we can close #39.
... With regards to #39 we made a corresponding proposal into the HTML spec.
... The question I have is in relation with the second paragraph, updating event times.
... Is this a proposal that we want to put forward? If it is, should it trigger change on any property update, not only on times?

RobSmith: With the existing interface, you can delete a cue and create a new one with the same identity.
... The design for the cues is immutable currently.

cpn: No, the cues can be changed. At least, the timing can be changed.
... So you don't have to do remove/add, you can just modify the properties directly.
... I made some investigation. As I understand it, you can change any of the cue attributes. There is an implementation bug around an event that I raised with browser vendors.

<Zakim> nigel, you wanted to say there are no semantics associated with the ids and id usage in general in HTML

nigel: It would be unusual in HTML to use an ID for anything. The only place where it's really being used is for CSS selector. The TextTrackCue ID is really just an arbitrary string. The implication would be that the sender of the cues would need to be aware of the state of the client, which worries me a bit.

RobSmith: The purpose of setting the same ID is to be able to track it. Different implementations can do different things. The "onexit" and "onenter" could be linked to the identity. I may ignore them deliberately. Or not.

nigel: OK, I'm still worried about this because there's no uniqueness with the notion of ID.
... Also, why wouldn't you want to update the cue directly instead of removing/adding a new one.

RobSmith: The system didn't seem designed to do that.
... If you consider that to be immutable, then its lifecycle is basically added/deleted.

nigel: It's not the actual model. If it's immutable then it would have to say it.
... The default for objects in DOM is mutable.

RobSmith: But then you have to define the interface to handle that

nigel: Yes, that's what need to be specified. I just note that the TextTrackCue interface does not say that the content cannot be changed. Implementations may not expect that to happen, but that would be a bug.

RobSmith: Such an implementation could easily say that the spec is not clear on the topic.

cpn: In terms of start/end time, then the spec defines behavior with regards to updating them.
... If we talk about other properties, I'm less clear.

RobSmith: Also the problem of the asynchronous process processing the cue, you're never quite sure about what happens.

cpn: When an event is triggered, the attributes tend to be held fixed during the execution of the event handler. Behind the scene, the implementation may be changing things but they will only be visible to the application next time the event handler runs.

RobSmith: You may have a cue that is active when you make the request but by the time you get it, it no longer is active.
... But you may have changed the endTime in the meantime, and the cue becomes active again.

nigel: We have a single-thread loop. So I'm not sure what the problem is.

RobSmith: You may end up with a cue that exits and that enters again. You should certainly flag it.

nigel: Genuine case when an expired cue has its endTime modified and then it starts again. It seems to make sense to trigger "enter" again.
... Second race condition seems more convoluted.

RobSmith: If you make a request, there is some time before it gets serviced.

[more exchanges on updating endTime of an existing cue]

cpn: If you got a cue servicing thing running, you need to allow sufficient time in advance for things not to run into weird situations.

RobSmith: But then you would need a mechanism to say "I cannot serve your request".

nigel: This is quite hypothetical conversation. Some demonstrator would be useful.

RobSmith: I can create a timing diagram

cpn: That would be useful, indeed.

RobSmith: It's an edge condition, but no matter the way you approach the issue, it may still occur.

cpn: Coming back to #46. I have this heading "Updating event times". Is this a recommendation that we'd still want to have?
... If it's the application that handles the changes, then it doesn't need the event.
... This recommendation would need to lead to some technical solution.

<Zakim> nigel, you wanted to say we haven't got an async model for info to be sent to the DOM, have we?

nigel: I think that API sounds really helpful, but I'm missing the context for having real-time updates pushed to the client for this API.
... We're receiving live streamed media. At the moment, TextTrackCue is not part of MSE so the streaming bit is separate. But you can envision a consistent stream coming through MSE and the application would need to handle this.
... If you had this kind of paradigm, then I can completely see the need.

cpn: It could be that in the media encoder on the server side, you're embedding emsg events. Because these things have IDs, you can potentially replace an existing one (to update its properties).
... It's possible that the server ends up pushing an event that the application has already seen. Do we have a mechanism to tell the app that this has happened?
... On this particular thing, is there a requirement for this kind of changing arbitrary attributes? Timing, yes, but it would seem that other changes get raised as events. Proposal would be to trigger an event for any kind of update.

[Exchanges on use cases for updating startTime, for cues that are not yet active, or when seeking back]

Discussions with DASH-IF

cpn: DASH-IF has this parallel activity on DASH events API. Question is how do we collaborate?
... The DASH-IF event group has calls every two weeks and welcome W3C participants.
... If you're interested around the emsg aspects of this and the event processing model more generally, please do join in with the DASH-IF calls, and we will continue to do this Monday call from a W3C perspective.
... We can work in a couple of ways. I can join their call and represent what we're doing here, or you can join the DASH-IF call as well.
... The other conversation I had was with some of the media people at Microsoft, to discuss the DataCue API. This was something that they were not aware at all when I mentioned the API. One of the conclusions that we came to is that one of their people will review the document that we produced and give us implementer feedback on that.
... It's possible that they would be interested, but they will likely want more information about business cases.
... Very promising in terms of getting implementer's engagement.

<cpn> scribenick: cpn

cpn: will join them with the DASH-IF activity

[adjourned]

- DRAFT -

Media Timed Events TF

16 Mar 2020

Attendees

Contents

Agenda

Review open issues

Discussions with DASH-IF

Summary of Action Items

Summary of Resolutions

Scribe.perl diagnostic output