<cpn> scribenick: cpn
Giri: I asked my colleague Charles Lo
to present an initiative from DASH-IF.
... It's work in progress, so we can only go into so much
detail.
... We'll distribute the slides after the call.
Charles: Thanks, Giri
... I'm Charles, from Qualcomm.
<kaz> Charles's slides on DAInty API
Charles: I see others from DASH-IF here, please jump in to clarify anything, if needed.
Charles: There's an interop group at
DASH-IF, DAInty: APIs for interactivity.
... A client could be implemented as a JavaScript library in a web
page, or a streaming client in a particular device platform.
... The origin of this work is a related work item at 3GPP on
service interactivity.
... The goal is to allow users to engage with auxiliary content
alongside live or time-shifted content delivered over 3GPP, either
broadcast or unicast network.
... DAInty provides hooks for interactivity, and also measurement
on the use of interactivity.
... There's a work item description document and a technical
report.
... DASH-IF was requested by 3GPP to support API specification.
Charles: The APIs are between a DASH
client and a service application. The interactivity events could be
delivered over broadcast or unicast HTTP.
... The 3GPP's focus is more on the broadcast case. Event data and
media data is passed over the 3GPP network to the interactive
application.
... Event-type information is handled by the DASH client.
... At 3GPP there's interest in leveraging the DASH client to
record measurement of interactivity usage.
... It's possible for the application to provide this directly, but
3GPP already has a reporting framework, so we want to use this for
application level events.
Charles: There are 3 APIs to be
developed. The first one allows an application to subscribe to DASH
stream messages, which carry interactivity information.
... This could be in-band (emsg) or MPD events.
... An application subscribes to a scheme id and value, and the
client dispatches matching events to the application.
... The second API allows an application subscription to an ISO
BMFF timed metadata track.
... The third is interactivity usage measurement reporting. The
usage measurement is delivered to the DASH client, which delivers
it to the network server.
Charles: The timeline diagram shows
an in-band event message, carrying some app specific
information.
... It's been subscribed to by some application.
... An emsg box, which is defined to be at the beginning of a
segment.
... The payload has a presentation time for the start of the
event.
... The duration could span multiple segments.
... The emsg event must arrive no later than the beginning segment
to which it applies, for the event start and duration to be fully
usable.
... The lower picture shows "event dispatch mode", passing from the
DASH client to the application.
... Two cases are possible: dispatch at the time the event arrives
(mode 1), or dispatch at the event start time (mode 2).
... The application can register with the DASH client which mode is
preferred.
... Mode 1 gives the application more time, but in either case the
application should be able to respond in time.
Charles: The DAInty APIs are intended
both for web apps, and native apps on a particular device platform,
as there are both application environments in the 3GPP model.
... For the web app case, the desire is for interactivity apps to
be defined in DASH.js, an open source DASH player supported by
DASH-IF.
... There are lots of other clients available, but the one
developed under the auspices of the DASH-IF is DASH.js.
... There is already support in DASH.js for SCTE-35, but we want
support for more generic events.
... We're writing in IDL, so not implementation specific. This
allows the IDL to be mapped to a specific language or
platform.
... The present focus of work is on the DASH event API, then later
on the timed metadata track and interactivity usage
reporting.
... Does W3C see value in standardising an IDL description? Could
use WebIDL.
... We have an agressive timeline, aligned with 3GPP, to complete
by end of the year.
... We're well on the way with the API definition.
Kaz: Is it OK to distribute the slides to the M&E IG public mailing list?
Giri: Yes, I will share them after the call.
Kaz: You're working on a concrete API defintion in IDL. Are you interested not only in WebIDL but also TypeScript? Other W3C groups have been struggling to define event APIs in WebIDL.
Charles: We'd be happy to look into
that, if W3C has some guidance.
... We also looked at OMG IDL. Could you provide some more
information?
... We can pick up offline with Giri.
Giri: We did talk about the choice of
IDL. The one experience I had was in Fido Alliance, where we used
it to define native APIs.
... I asked Charles to present on this, it's because of a percieved
need that the web standards have not met.
... We want to feed this into WICG.
Charles: I'm not sure how much WebIDL expertise we have in DASH-IF.
Giri: I'd recommend that when this work is in a sufficient state, DASH-IF should send this to W3C under liaison.
Will: IDL isn't our strength, certianly, but we will send under liaison.
Francois: I'm trying to understand
how the event model differs from the usual Web eventing
model.
... I understand your mode 2 is similar to the media element's
event model, but mode 1 differs.
Charles: Yes, emsg events could
arrive at the beginning of a segment. The emsg box describes the
start time and duration
... Notifying the application when the event is received gives it
additional time.
Will: I can confirm that. We have a
preference for mode 1, so that we can put the events into a helper
library to dispatch at the right time.
... It's never wrong to get an event early, but it is wrong to get
it late.
... We've had a long thread on this. My concern with the document
is with some of the language, such as event API.
... I want to make sure that what's proposed fits, it is specific
to certain players?
... Need to avoid creating a confusing API surface for
developers.
Charles: 3GPP's interest is to have some reference implementation, could be implemented in DASH.js
Will: another comment on language: interactivity is a broad term. Web developers think of click events on UI controls. We need a clear definition.
Charles: Absolutely.
<tidoust> scribenick: tidoust
Chris: You say the event carries a
start time and a duration. Is it possible for an event to be
unbounded?
... So you don't know when it ends in the future.
Charles: In principle, it is. We're basing that on the DASH specification. I don't remember exactly what's in there.
Giri: Internally, we've struggled with TTML on ways to define an unbounded metadata cue. "Display once, keep it on the display until I tell you to get rid of it". We haven't found something in TextTrack that lets us do that out of the box.
Chris: OK, I would have to go back
and read the DASH spec.
... My next question is related to the work we're doing in this
Task Force. We're producing a use cases & requirements document
to gather requirements for user agents that would support these
kind of events.
<kaz> Media Timed Events Use Cases and Requirements document
Chris: We have agreement on generic
use cases, I believe, but need more details on requirements that
you may bring here.
... For example, does the application have to specify the scheme id
and value to receive events?
Charles: There may be many types of events and the application may want to know about specific events that it's interested in, so yes, there's a way to subscribe to events based on the scheme ID URL.
Chris: I think the HbbTV specification has a similar API for exposing events that operates in a same way.
Charles: Is this the part that transfers the event to some TextTrack?
Chris: Yes, that's right.
... In HbbTV, it's always an opt-in from an application
perspective.
... And then the event itself is defined as a DataCue that exposes
the binary data from the event message.
... Is that what you're looking for as well, a generic blob that
the application knows how to interpret?
Charles: [looking up examples of
events in the DASH spec]
... Looking at the DASHEventMessageBox definition, the intent is to
provide the timing information as well as the data to the
application so that the application can make use of it.
Chris: Related to MPD events. Because they're delivered through the MPD, you need an API that allows the application to pass the events into the media player?
Charles: Yes, not explained here, but
we have a set of slides on that as well.
... Both types should be supported.
Chris: I guess it's just that there would need to be an API to allow an application to pass in objects with start time and end time and data, rather than having that extracted from the media segments by client.
Charles: Yes, the DASH client extracts that info, the only difference is where the info resides.
Chris: I'm thinking of this from a
Web platform API point of view, we would want the user agent to
take care of the emsg box parsing, whereas it's application-level
JavaScript that would parse the events embedded in the MPD. The
requirement is the same, but done by different things.
... Another question: If we were to push for a Web platform API to
do this, is the preference still for the Mode 1? As I understand
it, the current model that we have, using TextTracks, is closer to
Mode 2.
Charles: I don't know which one is
the ideal mode. We would want to allow both modes, and allow the
client to subscribe to the mode it's interested in.
... I don't know that we want to specify one or the other.
<cpn> scribenick: cpn
Francois: There's no event when a
change is made to the list of cues, even when not active.
... Maybe it's just a matter of adding an event when the list
changes.
<tidoust> scribenick: tidoust
Chris: Interesting that we don't have something for mode 1 on the Web.
<cpn> scribenick: cpn
Will: I'd support Francois on that.
<tidoust> scribenick: tidoust
Chris: We shouldn't be going too deep into API design, but what you've been describing Charles really helps shape the specific API requirements.
<kaz> issue 4
Chris: What kind of timing requirements did you specify for event dispatch? For example, Mode 2 event dispatch is probably more timing sensitive.
Charles: I think the idea is that the
application developer should be cognizant of the fact that there
may be some sort of delay.
... It probably cannot be done with guarantees within the right
millisecond.
... It would be important that the DASH player provides the current
presentation time when it performs the dispatch.
... This allows synchronization.
Giri: We're almost out of time for
this call. We can continue the discussion offline.
... Thank you, Charles, for this presentation.
<kaz> [adjourned]