14:56:49 RRSAgent has joined #me 14:56:49 logging to https://www.w3.org/2021/10/11-me-irc 14:56:53 Zakim has joined #me 14:57:10 Meeting: Media Timed Events: WebVTT unbounded cues 15:03:20 calvaris has joined #me 15:03:54 ChrisLorenzo has joined #me 15:04:19 zacharycava has joined #me 15:05:41 present+ Chris_Needham, Chris_Lorenzo, Zachary_Cava, Xabier_Rodriguez_Calvar, Rob_Smith 15:05:45 Chair: Chris_Needham 15:06:02 Topic: Agenda 15:07:53 ChrisN: Covered unbounded cues in segmented media last time 15:07:55 ... Any features or syntax changes in WebVMT that should be added to WebVTT? 15:08:30 ... Discuss whether we need to identify cues across WebVTT documents, if so how, and where to specify? 15:09:06 ... Anything more generally on DataCue? 15:09:22 Topic: WebVMT features into WebVTT 15:09:48 Rob: This ties to what's happening with DataCue. One feature worth porting across is aligning metadata with VTT and DataCue 15:10:16 ... Metadata is a JSON and it's amorphous, no formatting. A small amount of restriction on that would make it more useful or interoperable 15:11:10 Chris: In terms of other features, not sure what Gary had in mind 15:11:24 Zack: How far along is WebVMT? 15:12:03 Rob: Not on standards track. Dash-cam market has use cases that would benefit from it, which currently uses proprietary formats 15:12:24 ... A way to export the data allows use of proprietary formats, but users can export to a common format to share data 15:12:37 Larry_Zhao has joined #me 15:13:41 Chris: Let's look at the DataCue API relation. 15:14:43 Rob: DataCue has a 'type' field and content in a 'value' field, which can be anything 15:15:18 ... Too open-ended. In the WebVTT document, it's just a WebVTT document it's only JSON 15:16:00 ... The simple addition of the 'type' such as a URN allows you to recognise what it is 15:17:12 Chris: Examples in the explainer: https://github.com/WICG/datacue/blob/main/explainer.md such as org.id3 15:20:00 ... What's the scope of the type field? Are they globally defined, or do we have types specific to HLS defined in one place, WebVMT types defined else where 15:20:21 Rob: Thinking of an IANA type registration, to stop different uses stepping on each others toes 15:21:53 Zack: The HLS spec includes the date range type. Prior to that, Disney and Hulu implemented their own, and used com.hulu 15:22:06 ... The same functionality as the Apple-defined, but non-conflicting 15:22:37 ... When things get more standardised and more adopted, the ability to change the name is helpful 15:22:42 ... Having a URN makes it flexible 15:22:54 present+ Larry_Zhao 15:24:03 Rob: When to do this? In the dash-cam market there'll be different variants, so having an open format is beneficial. There may be commonality, which may lead to something like mime types being set up 15:26:05 ChrisN: Is it enough to say this cue is WebVMT data, or do you need to be more specific? 15:26:32 calvaris has joined #me 15:27:15 Rob: We don't really have WebVMT data. Started with location, now can be anything. Speed, direction, acceleration. With drones, there's altitude, camera orientation, sensors 15:27:32 ... So the data is really sensor data 15:28:42 ChrisN: Thinking of use cases. In HLS, cues are surfaced by the browser to the web app 15:29:05 ... This the in-band time metadata cues, same with emsg boxes in DASH 15:29:39 ... The other case is where the web app creates cues after reading a WebVMT document 15:30:39 ... Where are the interop points? 15:31:24 Rob: WebVMT was designed for devices without connectivity, autonomous. Recording data such as temperature with minimum overhead 15:31:53 ... Being able to record in a common format allows others to recognise what it is without converting 15:32:57 ChrisN: We can make setting up a registry for data types a part of the proposal 15:34:32 ... Although as a developer I should be able to put any arbitrary data in a DataCue without registering 15:36:03 Zack: SCTE-35 has schemes in the spec, and people use it without registering their scheme. If people use URNs, you'd rarely get conflicts 15:36:58 ... Having a spec with optional registering encourages private adoption, proprietary uses. If then later they need external interop, they'll be able to register at that time 15:38:16 ChrisN: Thinking about DataCue more generally - we've focused a lot on defining the emsg mapping, without really resolving 15:39:08 ... We could perhaps usefully split DataCue into two parts: the API, as it is in WebKit, the second part is the mapping to emsg 15:39:27 ... If there's still interest in emsg, I'd like to come back to that 15:39:49 ... Unclear on the extent of interopt there 15:41:49 ChrisN: If we end up with app-level emsg parsing, we'd still need a DataCue to put the data on the timeline. Current solution is using VTTCue 15:43:39 ... Still need a way to distinguish VTT caption cues from metadata cues 15:43:58 Rob: Can use different TextTracks, one for captions, one for metadata 15:44:40 ... If you have a metadata track, all the JSON objects would have a top-level 'type' and 'data' objects that would allow DataCues to be created 15:45:34 ... It would be easy to inspect the list of cues to extract the ones you're interested in 15:45:51 ChrisN: So the presence of the 'type' field in itself is helpful, which isn't available with VTTCue 15:46:10 ... We can write this into the explainer 15:46:50 Rob: With the current VTTCue workaround, it's a more complex object than a DataCue, as it has presentation stuff we don't need 15:49:41 ChrisN: Mapping of emsg box type information is complex, depends on the DASH-IF event interop work 15:50:28 ... Need to summarise the open questions 15:50:45 Topic: Identifying cues across WebVTT documents 15:51:24 ChrisN: Context is segmented delivery. Each segment would have its VTT document specific to that section 15:56:31 ... Do we need cue identifiers, or identifiers to higher level concepts? 15:56:42 ... eg: { chapter: 1 } 15:57:01 ... or { id: 'some-id', chapter: 1 } 15:57:34 Rob: Do you need identifier at the type level? 15:58:00 Zack: Answer could be driven by need for de-duplication. I'd expect user agents to do the collapsing 15:58:41 ... Up-levelling anything needed to enable de-duplication is important. So pulling the id and type out, to bring them together 15:59:20 ... In DASH, you have event id, start time, payload. The id allows de-duplication across period boundaries without parsing the data itself 16:00:07 Rob: So if the id is at the same level as type, do we get the same issue - cue ids should be unique within a document, but not between documents? 16:01:05 Zack: From a media perspective, i'd expect it to be unique within the track it's operating in. Could be multiple documents making up the track 16:01:27 ... It's not unheard of property. It happens for audio and video, where track descriptions are shared across segments 16:05:41 zacharycava has left #me 16:06:31 ChrisN: Next steps: Follow up with Gary on WebVTT cue ids, revisit emsg mapping proposal, talk with DASH-IF about emsg cue id scope 16:06:35 [adjourned] 16:06:39 rrsagent, draft minutes 16:06:39 I have made the request to generate https://www.w3.org/2021/10/11-me-minutes.html cpn 16:06:44 rrsagent, make log public 16:09:25 s/interopt/interop/ 16:09:27 rrsagent, draft minutes 16:09:27 I have made the request to generate https://www.w3.org/2021/10/11-me-minutes.html cpn 16:09:44 s/Chris:/ChrisN:/ 16:09:46 rrsagent, draft minutes 16:09:46 I have made the request to generate https://www.w3.org/2021/10/11-me-minutes.html cpn 16:10:10 s/Chris:/ChrisN:/ 16:10:13 rrsagent, draft minutes 16:10:13 I have made the request to generate https://www.w3.org/2021/10/11-me-minutes.html cpn 16:10:41 s/Chris:/ChrisN:/ 16:10:43 rrsagent, draft minutes 16:10:43 I have made the request to generate https://www.w3.org/2021/10/11-me-minutes.html cpn 16:21:13 s/eg: { chapter: 1 }/eg: { chapter: 1 } with consistent cue ids across VTT documents/ 16:21:16 rrsagent, draft minutes 16:21:16 I have made the request to generate https://www.w3.org/2021/10/11-me-minutes.html cpn