15:07:57 RRSAgent has joined #datacue 15:08:01 logging to https://www.w3.org/2025/10/14-datacue-irc 15:08:09 Meeting: WICG DataCue 15:08:18 tidoust has joined #datacue 15:08:33 Chris: Welcome everyone, this is the WICG DataCue meeting 15:08:39 scribe: nigel 15:08:41 scribe+ cpn 15:08:42 RobSmith has joined #datacue 15:08:54 eric-carlson has joined #datacue 15:09:13 alicia has joined #datacue 15:09:32 ken has joined #datacue 15:10:31 Topic: TextTrackCue constructor 15:10:49 Rob: Thank you for joining today 15:11:13 ... I'm leading development of Web Video Map Tracks, synchronising location and video 15:11:24 ... Made some good progress this year, so want to give an update and discuss next steps 15:12:07 ... TextTrackCue provides a common interface, common attritubes, startTime and endTime, and a pauseOnExit flag 15:12:22 ... There's a cue content interface, the abstract component 15:12:40 ... VTTCue is for timed text, takes a string, and DataCue which takes type and value 15:12:56 ... DataCue has only one browser implementation, so removed from the HTML spec 15:13:18 ... Currently the constructor isn't exposed, and this prevents custom cues 15:13:44 ... I was puzzled, but now I know why. Browser subclassses werent' possible when TextTrackCue was first designed in 2014 15:14:04 ... TextTrackCue has a constructor, and exposing this would be a small change, easy to implement 15:14:15 ... Constructor access shouldn't be confused with instantiation 15:14:42 wschildbach has joined #datacue 15:14:57 ... The instance is returned by the end of the constructor. An abstract class can be implemented in WebIDL with suitable constructor steps. Errors can be raised during instantiation, as a way to implement an abstract class 15:15:03 present+ 15:15:29 ... The parent class is an input to the constructor. I've discussed with Alicia, foolip, and he gave it a thumbs up 15:15:54 ... Abstract classes and objects shouldn't be confused with abstract functions. TextTrackCue doesn't have abstract functions 15:17:03 ... What does the constructor look like? startTime and endTime are required. HTML gives defaults to the remaining attributes 15:17:18 ... pauseOnExit is false, identifier is empty string 15:17:30 ... That is in the TextTrack section of the HTML spec 15:18:08 ... There could be a second form of the constructor that includes an identifier. That might be unnecessary as the id can be set after construction 15:18:21 ... The proposal here is designed to be as simple as possible 15:18:50 ... I'm aware there's a proposed WebKit enhancement, from 2020 or 2021, to make TextTrackCue a concrete class. Not sure why that is. 15:19:23 ... This is equivalent to DataCue type = undefined, which I feel impedes access to the content 15:19:42 ... I think it was proposed for the same reason as I propose access to the construtor, to enable subclassing 15:20:24 Eric: It was proposed as a way to handle the situation where a site doesn't use VTT, and creates DOM nodes from some other text track cue format 15:20:45 ... That's a challenge for accessibility when it's possible for the user to define their preferred cue styling at the system level. 15:20:57 ... If the browser renders cues, it can apply the user's custom styles 15:21:30 ... If a page converts from a non-browser supported cue format, and inserting DOM nodes into the page, there's no way for the browser to know they're cues, and it can't apply the user's preferred format 15:21:54 ... Our proposed extension to TextTrackCue takes a document fragment, and some additional markup that allows the page to identify the cue and the background 15:22:24 ... WIth that the browser can apply the user's preferred style, and put the node into an isolated shadow DOM that the page can't access, and so use the user's preferred styling as a fingerprinting vector 15:22:44 ... A large complication with WebKit with your proposal is that we have an experimental version of the API in WebKit 15:23:03 ... It has a constructor that has startTime and endTime, and requires a document fragment 15:23:57 Rob: I've seen that. Can it be done with a subclass. In issue 35 I propose a subclass that sits between TextTrackCue and VTTCue, so there's a FragmentCue subclass that are then inherited by VTTCue. Functionally equivalent 15:24:29 Eric: It would be a breaking change to WebKit. You propose moving the implementation from TextTrackCue to a new FragmentCue class 15:24:35 Rob: Yes 15:25:11 ... The proposal to exposing the TextTrackCue constructor would allow anyone to make subclasses 15:26:01 ... The other problem is, the changes are specific to timed text, and inherited for every other sort of cue, e.g., timed metadata, where they don't make sense 15:26:41 Alicia: Eric, couldn't we still make that work. I can think of two possibilities 15:27:18 ... Use the extra argument, and return a new FragmentCue. I assume it's possible as the extra arg is a node. I haven't studied in detail though 15:27:57 ... Other way is having a class in the other direction, instead of a child of TextTrackCue, maybe something like TimeCue, not specific to text, just timed metadata. It makes some sense as a name, and could be possible vector for extension 15:28:15 Eric: If that can be done without breaking existing DataCue, which is used extremely widely 15:28:31 ... It's used for ad insertion and ad metadata 15:28:52 ... Also used for IDv3 metadata in shoutcast streams for radio stations, for example 15:29:38 present- 15:30:01 scribe+ 15:30:02 cpn: If you were designing this from scratch, TimedCue would be the parent, with TextTrackCue and then timed metadata branch where DataCue lives 15:30:10 ... How to determine compatibility on that? 15:30:37 Alicia: Changing DataCue, which inherits TextTrackCue, could be breaking 15:31:10 Eric: Possible to have multiple constructors, e.g., MediaStream has 3 forms of constructor. If we make a TextTrackCue constructor that takes start, endtime and id, 15:31:37 ... Alicia's first proposal would work. It would be an implementation detail in WebKit, but not a breaking change 15:32:21 Rob: There's currently no constructor, and given WebKit's implemntation takes document fragment, it could distinguish [missed] 15:32:31 Eric: Did you suggest having an identifier as the third argument? 15:32:51 Rob: I did, but not necessary, as we can distinguish a cue fragment and a string as third argument 15:32:56 Eric: That wouldn't matter 15:33:17 Rob: I'm trying to make the minimum change, assuming that's better 15:33:39 Eric: If it makes sense to have the id in the constructor, then have that. The amount of work involved shouldn't be the criteria 15:34:04 Rob: My motivation was the reason it was dropped from HTML 15:34:28 Eric: It can be defined in IDL by setting the default value 15:34:59 Rob: So I think stick with just start and end time 15:36:14 Chris: Not sure I understand the first option 15:36:51 Rob: Arguments are passed as an array, they're not explicitly typed, so in the constructor you'd have to count them. 15:37:58 Chris: So would DataCue get a CueFragment that it doesn't use 15:39:08 Eric: I think it makes sense to have the ID, as it's an attribute in TextTrackCue. Marking it in IDL that it's empty when not supplied, takes care of it 15:39:27 ... When script wants to initialise the ID, script doesn't need an additional step 15:39:32 ... to set the cue 15:39:39 s/cue/id/ 15:39:57 Rob: To confirm, we just have the second constructor (with startTime, endTime, id) 15:40:05 ... I'm happy with that 15:40:43 Chris: Need to have other browser interest to add to the spec 15:41:04 Rob: With the constructor, developers can write polyfill 15:41:34 ... If not native to the browser you can delcare it as a class easily. That would encourage adoption by other browsers 15:42:33 Rob: We have a proposed way forward 15:43:01 ... While I was looking at all this, I looked at the WPTs. I noticed that polyfills weren't behaving as expected 15:43:37 ... There are some WPT failures, onenter and onexit. Those fail because of dispatchEvent, which should inherit from EventTarget, but wasn't doing anything 15:43:44 ... It should call synchronously but doesn't 15:43:55 ... It works in Firefox. I raised issues in Chrome and Safarai 15:44:08 Alicia: https://github.com/WebKit/WebKit/pull/50526 15:44:22 Rob: Fixed in Chrome, and there's a PR in WebKit 15:45:00 ... It affects how TextTrackCue can be used, with WebVMT for example we dispatch events directly on the cue, and bypassing TextTrackCue as time marches on doesn't meet our requirements as it's stateful 15:45:44 ... WPTs provide useful insight 15:46:09 ... Tests still fail on unbounded cues 15:46:48 ... [Summarises test result numbers] 15:47:56 ... Exposing the constructor means we can streamline the WPTs by removing the VTTCue dependency from the tests, which made tracking down bugs difficult 15:48:41 ... Community Benefits. Allowing JS access allows support to progress by the community, reducing burden on browser developers 15:49:14 ... It also reduces browser overhead, by streamlining the tests 15:50:19 ... There are working polyfills. In the Sync on the Web CG live streaming use cases, there are polyfills that work on the 3 main browsers. This includes demos to explore design features, such as streaming, multiple tracks and handlers, duplicate cue types. I'll add a custom scheduler demo, as TextTrackCue can be used on its own without TextTrack 15:50:57 ... The Live Screening case, e.g., a music or theatre event, it's streamed live to remote venues 15:51:12 ... The use case is to stream the lighting effects so that lighting can be re-created in the remote venue 15:52:01 ... It includes a DataCue polyfill, and highlights some interesting requirements, e..g, duplicate cue types, many concurrently active cues, and concurrent cue changes 15:52:10 ... It's quite different from timed text 15:52:34 Alicia: It depends, some timed text, e.g., WebVTT allows overlapping cues. Support is a bit flaky though 15:52:48 q? 15:53:08 Eric: How does the lighting control go from data to controlling the lighting? 15:53:56 Ken: For DMX lighting device, it's controlled by USB, for example using WebUSB API 15:54:04 Eric: So it's only working in Chrome 15:54:06 Ken: Yes 15:54:09 q? 15:55:09 Chris: Should we propose spec text change, and if so, where? 15:55:31 Rob: I've been thinking about that. Changes are needed are minimal 15:55:59 ... Philip suggested raising it as an issue in HTML so he can comment directly 15:56:28 ... I think we resolved the conflict between this proposal and WebKit's approach, so no problem raising an HTML spec issue 15:58:28 Eric: I agree. DataCue is orthogonal, but I think it would be extremely useful to have a proposal to reintroduce DataCue, as it's widely used, and it is useful 16:00:48 Chris: Is it the surfacing of media in-band events you're thinking about? 16:00:50 Eric: Yes 16:01:23 Rob: Would it be possible to develop a polyfill, using TextTrackCue? 16:01:53 Eric: There's huge utility in being able to embed metadata in the media file, so you have one single file and the delivery of the metadata is timed with the audio and video samples 16:02:38 Alicia: In the case of MSE, there's lots of JS code parsing the binary data. This constructor proposal allows you to write a DataCue polyfill to handle those, put them in a new track 16:02:54 ... It could motivate other browsers to implement, as it would be a performance improvement 16:03:17 Eric: Our existing DataCue constructor is used with libraries that parse containers for use with MSE 16:03:25 Alicia: Do they work with other browsers? 16:03:32 Eric: I don't know that they do 16:05:01 Alicia: It creates a chance of getting adoption, and enables something that can't be done already 16:05:39 Chris: I'm happy to support work on DataCue through chairing, if there's a willing community 16:05:54 Rob: I was thinking about a breakout at TPAC 16:08:09 Rob: I can move forward, and very happy to have had the discussion 16:08:39 ... Demos, the OGC Testbed, the DMX lighting demo from Ken 16:15:05 Chris: Happy to coordinate on a TPAC breakout, if you want to. I think the goal there would be to encourage other implementations, based on the design we've arrived at today 16:15:20 [adjourned] 16:15:24 rrsagent, draft minutes 16:15:26 I have made the request to generate https://www.w3.org/2025/10/14-datacue-minutes.html cpn 16:15:34 rrsagent, make log public 16:16:43 present: Chris_Needham, Francois_Daoust, Rob_Smith, Eric_Carlson, Kensaku_Komatsu, Alicia_Boya_Garcia, Wolfgang_Schildbach, Nigel_Megitt 16:16:46 rrsagent, draft minutes 16:16:47 I have made the request to generate https://www.w3.org/2025/10/14-datacue-minutes.html cpn 16:16:53 rrsagent, make log public 16:19:09 Chair: Chris_Needham 16:19:12 rrsagent, draft minutes 16:19:13 I have made the request to generate https://www.w3.org/2025/10/14-datacue-minutes.html cpn 18:48:24 Zakim has left #datacue 19:35:12 rrsagent, draft minutes 19:35:13 I have made the request to generate https://www.w3.org/2025/10/14-datacue-minutes.html cpn 19:35:54 i/Rob: Thank you for joining today/Slideset: https://www.w3.org/2011/webtv/wiki/images/a/af/20251014_ProgressTTC.pdf 19:35:56 rrsagent, draft minutes 19:35:58 I have made the request to generate https://www.w3.org/2025/10/14-datacue-minutes.html cpn