13:55:11 RRSAgent has joined #me 13:55:15 logging to https://www.w3.org/2025/03/04-me-irc 13:55:19 meeting: Media and Entertainment IG 13:55:31 agenda: https://lists.w3.org/Archives/Public/public-web-and-tv/2025Mar/0000.html 14:57:36 present+ Kaz_Ashimura, Bernd_Czelhan 14:57:42 rrsagent, make log public 14:57:47 rrsagent, draft minutes 14:57:48 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 14:59:22 ohmata has joined #me 14:59:23 nhk_ryo has joined #me 14:59:25 cpn has joined #me 15:00:34 present+ Chris_Lorenzo, Hisayuki_Ohmata, Ryo_Yasuoka 15:00:55 nigel has joined #me 15:01:45 present+ Chris_Needham 15:01:51 scribe+ cpn 15:01:51 resent+ Francois_Daoust 15:02:16 chair: Chris_Needham, Chris_Lorenzo 15:02:24 rrsagent, draft minutes 15:02:26 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:02:39 present+ Alicia_Boya_Garcia, Tatsuya_Igarashi 15:02:56 rrsagent, draft minutes 15:02:57 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:03:04 present+ Nigel_Megitt 15:03:04 alicia has joined #me 15:03:27 rrsagent, draft minutes 15:03:28 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:03:38 chair+ Tatsuya_Igarashi 15:04:09 s/resent+ F/present+ F/ 15:04:11 present+ Rob_Smith 15:04:21 topic: Agenda 15:04:28 cpn: 2 topics 15:04:45 ... updats on TextTrackCue and DataCue proposals 15:04:58 ... Mile High Video Conf report 15:05:08 scribe+ nigel 15:05:14 i/2 to/scribenick: kaz/ 15:05:19 Previous meeting: https://www.w3.org/2025/02/04-me-minutes.html 15:05:52 Alicia: Text tracks in MSE 15:06:09 JohnRiv has joined #me 15:06:16 Topic: TextTrackCue 15:06:30 Chris: Thank you Rob for the work you've been doing 15:06:48 present+ 15:06:54 Rob: Quick background, I started trying to write a DataCue polyfill 15:07:11 ... I discovered that DataCue is implemented in Webkit and Safari 15:07:57 -> https://github.com/WICG/datacue/issues/35 Proposal to expose HTML TextTrackCue constructor WICG/datacue#35 15:08:02 ... But in doing that I found that I needed to go back to TextTrackCue given what's happened with DataCue, where it's only implemented in Safari so dropped from the HTML 15:08:25 cpn: This was in HTML but then dropped. 15:08:47 .. Since then we ran a task force and produced a report that was focused on these emsg boxes 15:08:57 .. that can be carried in CMAF, and how to surface those to applications. 15:09:21 .. Then all of that work stalled, and it didn't feel there was enough interest to push it forwards. 15:09:30 RobSmith has joined #me 15:09:39 .. Perhaps it was premature - the DASH-IF had a subgroup looked at event processing, which this tied in with. 15:09:42 atai has joined #me 15:09:49 rrsagent, draft minutes 15:09:50 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:09:56 .. At the time they were defining a processing model for these "media in-band events". 15:10:06 .. There ended up being a feeling that it wasn't quite ready for standardisation so we left it. 15:10:24 .. As a result it meant that the use case that Rob is interested in, independent of emsg boxes etc., 15:10:45 .. more to do with associating metadata with video and location information and visualising on maps, 15:10:50 present+ Andreas_Tai, Piers_O'Hanlon, Thasso_Griebel 15:10:52 rrsagent, draft minutes 15:10:53 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:10:59 .. a worthwhile use case, coupled with in-band processing meant that we weren't able to progress 15:11:08 .. the thing that was useful, independently of the not-quite-ready thing. 15:11:26 .. In the discussion that followed Rob came and said he'd like to move forward with something 15:11:37 .. he could use to associate metadata with media times to build applications. 15:11:50 .. What's the simplest thing to do to solve that problem? Can we focus on that? 15:11:56 .. I'm thankful to Rob for helping to move this forwards. 15:12:16 Rob: My particular interest as Chris hinted is for out-of-band metadata. 15:12:32 .. I've been leading the development of WebVMT, a variant of WebVTT which has sort of branched into 15:12:44 .. its own now, published in Sep 2023 as a Group Note, by the Spatial Data on the Web Group, 15:12:58 .. which provides a v1 for implementers to work to, and I can continue with the development. 15:13:12 .. Given the history with Datacue and VTTCue, both inherit from TextTrackCue. 15:13:27 .. I started looking at TextTrackCue as the root of this and whether I could adapt something on top of that, 15:13:38 .. given that DataCue had not been adopted. Could I do a minimal change? 15:13:51 .. I identified that if the constructor for TextTrackCue were available that would solve my problem. 15:14:02 .. I am puzzled about why it isn't, given that it's supposed to be an abstract base class. 15:14:18 .. That was my direction, so I wrote a TextTrackCue polyfill, which I can show you. 15:15:15 .. [shares screen] 5s long video, on Firefox. Video element, with a "count" and "colour" indicator below. 15:16:19 .. Those indicators flicker with different values, all driven by DataCue. 15:16:29 .. The log shows what's going on in the background. 15:16:52 Rob: The log shows the cue enter and exit times, and the cue data content 15:17:03 i|The log|scribenick: cpn| 15:17:06 rrsagent, draft minutes 15:17:07 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:17:17 ... When I move around the timeline, you can see them coming and going 15:17:29 ... I found that DataCue is redundant if you take this approach 15:17:58 ... I made another demo. If you extend TextTrackCue to create a derived cue, it does the same thing as the 'type' attribute 15:18:34 ... This has CountdownCue and ColourCue, which are derived from TextTrackCue. It works across browsers 15:18:54 ... What I'm looking to do is put it forward as a proposal to extend TextTrackCue to operate in this way 15:19:11 ... Alicia has been very helpful, providing technical input (thank you!) 15:19:25 ... Four points have come up in the discussion 15:19:59 piers has joined #me 15:20:06 ... The abstract class, the constructor attributes, there's backward compatibility, and ... 15:20:30 ... The proposal is the smallest change possible, as far as I can tell 15:21:12 q+ 15:22:00 Rob: It has broad applicability, sensors, dashcam, accelerometer, vehicle collision monitoring 15:22:18 ... by parsing the accelerometer data and looking for spikes 15:23:17 Nigel: I want to highlight a discussion in the GitHub about having an abstract base class 15:24:34 cpn: We're limited in how much of the solution we can discuss in an IG. 15:24:43 i|We're|-> https://github.com/WICG/datacue/issues/35 WICG datacue issue 35 - Proposal to expose HTML TextTrackCue constructor| 15:24:53 i|We're|scribenick: nigel| 15:25:00 rrsagent, draft minutes 15:25:02 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:25:08 .. I understand that you've implemented a constructor that prevents instantiation of TextTrackCue 15:25:10 q+ alicia 15:25:14 .. and you're preserving existing behaviours while introducing the extensibility point. 15:25:15 ack ni 15:25:27 alicia: There is already a constructor implemented in WebKit. 15:25:33 .. This has already happened with IDL and everything. 15:25:42 .. I'm not sure about talking about abstract classes and such. 15:25:55 .. In the HTML spec everything is interfaces anyway, and some of them happen to have constructors. 15:26:11 .. Classes are only one way to think about it, but it's not in the spec as far as I know. 15:26:20 i|Thank you Rob|scribenick: cpn| 15:26:21 Rob: Francois has confirmed that in the thread. 15:26:23 rrsagent, draft minutes 15:26:24 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:26:51 i|This was in HTML but|scribenick: nigel| 15:26:52 rrsagent, draft minutes 15:26:53 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:27:04 Rob: I appreciate Francois's input into this. I think what he has explained is not inconsistent with this approach. 15:27:16 .. I accept that there may not be another example of abstract classes in HTML (prepared to be corrected!) 15:27:30 .. but I think it is possible to do, because I have. My demo does exactly that. 15:28:16 .. There's an inheritance test. 15:28:21 Nigel: Can you share the code for that demo? 15:28:22 Nigel: Could you share the code? 15:28:25 Rob: Happy to. 15:28:26 i|I appreciate|-> https://github.com/WICG/datacue/issues/35#issuecomment-2697233375 Francois' response on Issue 35| 15:28:28 rrsagent, draft minutes 15:28:29 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:29:00 s/Nigel: Could you share the code?// 15:29:17 rrsagent, draft minutes 15:29:18 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:29:25 cpn: W3C has a breakouts day in a few weeks, and this could be a good topic for that 15:29:28 .. - people like Marcos and Eric from Apple for example. 15:29:37 .. I'd like to propose that we bring this to one of those breakout sessions. 15:29:50 .. In order to do that we need to have more of an agreed approach among ourselves to start with. 15:30:04 .. Plus how this relates to some of the other TextTrackCue extension proposals that have been put forward 15:30:17 .. to show that this is compatible, e.g. with the HTMLCue idea that Apple put forward. 15:30:38 i|W3C has a|-> https://lists.w3.org/Archives/Member/chairs/2025JanMar/0022.html Call for breakouts day (Member-only)| 15:30:40 rrsagent, draft minutes 15:30:41 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:30:54 .. Targeting one of those breakout sessions on the 26th may require us to have a more agreed understanding. 15:30:57 q+ 15:31:15 .. We could organise a separate focused session to talk about this under WICG participation rather than 15:31:17 .. the IG. 15:31:26 .. So that we're doing it under the community licence. 15:31:34 s|-> https://lists.w3.org/Archives/Member/chairs/2025JanMar/0022.html Call for breakouts day (Member-only)|-> https://github.com/w3c/breakouts-day-2025 Breakouts Day 2025 on GitHub| 15:31:36 rrsagent, draft minutes 15:31:37 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:31:48 .. Try to fit that in in the next week or so with an aim for having something presentable in the 15:31:51 .. wider breakouts. 15:32:05 Rob: I have the demos working and they are compliant with the proposal I put forward. 15:32:26 q? 15:33:03 Nigel: Another option, we could just take the discussion to the breakout 15:33:08 q- 15:33:43 ... Having some external viewpoints might help resolve things 15:34:10 q- alicia 15:34:14 rrsagent, draft minutes 15:34:15 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:34:55 s/- people/people/ 15:34:56 rrsagent, draft minutes 15:34:57 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:35:20 i|Another option|scribenick: cpn| 15:35:21 rrsagent, draft minutes 15:35:22 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:35:31 Chris: Suggest not using the C++ terminology so much, and focus on what WebIDL and web platform capabilities 15:37:17 -> https://github.com/w3c/breakouts-day-2025/wiki/Meeting-Planner-Resources Schedule for breakouts proposals 15:37:39 ... We could hold a meeting to prepare, if needed 15:38:18 [ 12 March: Deadline for submitting initial list of proposals. ] 15:38:48 Rob: I can share the code 15:39:59 Topic: Mile High Video 15:40:54 Piers: This was a couple of weeks ago. I gave a talked about an enhanced approach to respond to low latency, and minimising stalling, 15:41:04 ... by analysing the performance of the download over time, before it ends 15:41:27 ... then we can achieve improved QoE using this approach. Intra-segment information could be provided by the network 15:41:32 s/gave a talked/gave a talk/ 15:41:40 ... fits with some of the edge computing and network quality work 15:41:50 ... This code will go into dash.js in the coming months 15:42:14 https://www.svta.org/2025/02/04/exploring-the-edge-in-the-streaming-video-workflow/ 15:42:14 ... In terms of other work, SVTA are looking at edge functionality with respect to video streaming 15:42:59 ... Initial focus is around edge caching, the open caching initiative. Glenn Deen is organising 15:44:10 ... Potentially things could fit with this interest group. 15:44:17 ... Other topics that came up, more use of GPU for video encoding, also once video data is in the GPU it could also be used for other purposes, such as AI analysis, or quality metrics on the fly 15:44:49 ... It's a new concept that might be enabled, could feed into work on quality monitoring 15:45:35 ... There were other talks about doing quality metrics using predictive VMAF implementation, allowing calculation on the fly 15:46:24 ... Other talks on processing such as super resolution, to enhance the quality of video. It could be done at the edge, to offset the transmission costs 15:47:16 ... Talks from Akamai and Meta on Media over QUIC. Also content-aware transport metrics over QUIC. Adjust the server side config based on the network situation, provided to the client by CMSD to allow adaptation at the client 15:47:45 ... A number of presentations on server-guided ad insertion. This is at the final stages of standardisation at MPEG 15:48:05 ... Related functionality may benefit from some of the edge functionality 15:48:19 ... Ad decision servers and related in-network functionality 15:48:37 ... Any thoughts or questions, let me know 15:49:34 q+ 15:51:19 Chris: Interesting, also relates to Web & Networks IG, looking at cloud/edge. Also research projects looking at edge composition and compute offload 15:51:33 -> https://www.w3.org/groups/cg/cloud-edge-client/ newly created CG on Cloud-Edge-Client Coordination 15:52:04 Kaz: There's a CG working on cloud-edge-client coordination, and the combination of GPU approach and QUIC based handling would be useful 15:52:34 Piers: Also optimising for QUIC delivery, network parameters for congestion control and tuning to the connection 15:53:05 Chris: TPAC breakout on low latency event signalling 15:53:42 Piers: There are different ways to signal having an updated manifest. emsg is one way. Need to deal with the "thundering heard" problem when things happen all at the same point 15:53:58 ... Relevant particularly in live sessions 15:54:08 Kaz: ALso relates to IoT and smart city use cases 15:54:50 Topic: Text Tracks in MSE 15:55:00 s/ALso/Also/ 15:55:09 rrsagent, draft minutes 15:55:11 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 15:55:26 Alicia: Webkit has landed support for them in Safari, and in Webkit gstreamer this week. I found that the MSE spec doesn't seem to handle text tracks very well 15:55:37 ... I found there are no WPTs for TextTracks in MSE at all 15:56:12 ... The spec has some concerning points. It doesn't have the concept of gaps. Workarounds feel like hacks 15:56:28 ... In some text formats, e.g., WebVTT you can have overlapping cues in time 15:56:35 ... Not sure MSE spec handles this properly 15:57:53 ... Overlapping cues only works reliably if one overlaps with another, but if you have a second one, splicing might happen where it shouldn't 15:58:17 ... Also the MSE spec doesn't contemplate having an MP4 or WebM file with just a single text track and no video or audio 15:59:20 ... The SourceBuffer buffered attribute only considers audio and video buffers. It works around having a gap concept for text tracks. What if the SourceBuffer only has text? 15:59:34 ... Could be reasonable to support, but not possible because of how the spec works 16:00:22 ... We were talking about gaps in MSE for another purpose. I've tested Firefox and Chrome, neither supports inband text tracks 16:00:32 i|Not sure MSE|-> https://www.w3.org/TR/media-source/#attributes Media Source Extensions - 5.1 Attributes| 16:00:34 rrsagent, draft minutes 16:00:36 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 16:01:13 Chris: Discussed at last TPAC 16:01:45 Nigel: There was agreement something needs working on 16:02:48 original issue in WebKit to add MSE text tracks https://bugs.webkit.org/show_bug.cgi?id=125161 16:05:15 q+ 16:05:57 Chris: I could gather info on the previous discussions 16:05:58 https://www.w3.org/2025/03/breakouts-day-2025/ 16:06:41 q- 16:06:55 Chris: Could bring to Media WG, or propose for breakouts day 16:08:12 Nigel: The issue about not being able to define gaps could be a VTT thing, TTML has a model where there's always an active document for a period of time 16:08:17 q? 16:08:53 Alicia: VTT cues are ordered, and there's an MP4 "no cue" sample tag 16:09:15 ... I think they also do something to represent overlapping cues 16:09:25 rrsagent, draft minutes 16:09:27 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz 16:09:36 ... A sample may have several cues, not necessarily a 1:1: mapping 16:10:10 [adjourned] 16:10:15 rrsagent, draft minutes 16:10:16 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html cpn 16:10:59 i|Alicia: Text|scribenick: cpn| 16:11:00 rrsagent, draft minutes 16:11:01 I have made the request to generate https://www.w3.org/2025/03/04-me-minutes.html kaz