14:58:51 RRSAgent has joined #me 14:58:55 logging to https://www.w3.org/2025/02/04-me-irc 14:58:56 Zakim has joined #me 15:01:49 meeting: MEIG monthly meeting 15:01:54 igarashi has joined #me 15:02:02 Present+ Nigel_Megitt 15:02:02 present+ 15:02:03 agenda: https://lists.w3.org/Archives/Public/public-web-and-tv/2025Feb/0000.html 15:02:23 present+ Kaz_Ashimura, Chris_Lorenzo, Chris_Needham 15:02:40 Chris has joined #me 15:02:48 present+ Alicia_Boya, Francois_Daoust, Piers_O'Hanlon, Rob_Smith 15:03:08 present+ Louay_Bassbouss 15:03:16 rsagent, make log public 15:03:25 s/rsagent, make log public// 15:03:30 rrsagent, make log public 15:03:36 rrsagent, draft minutes 15:03:37 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:03:55 present+ Daniel_Silhavy 15:04:09 rrsagent, draft minutes 15:04:11 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:04:17 Louay has joined #me 15:04:17 RobSmith has joined #me 15:04:18 JohnRiv has joined #me 15:04:19 present+ Louay_Bassbouss 15:04:27 present+ 15:04:51 alicia has joined #me 15:05:22 chair: Chris_Needham, Chris_Lorenzo, Tatsuya_Igarashi 15:05:29 scribe+ nigel 15:06:13 rrsagent, draft minutes 15:06:14 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:06:29 Agenda: MSE Dynamic addition / removal of source buffers 15:06:32 https://github.com/w3c/media-source/issues/359 15:06:59 cpn: Daniel responded to some of the comments in the reply 15:07:06 rrsagent, draft minutes 15:07:07 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:07:53 s/Agenda: /topic: / 15:07:53 rrsagent, draft minutes 15:07:54 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:08:00 cpn: The use case you've given is different DASH periods where 15:08:08 .. either there's video or no audio, and back to video and audio, or similar 15:08:38 Daniel: Yes, that would be the scenario 15:08:54 cpn: And the workaround you're using is, if there's no audio, that you continue 15:08:57 .. to fill the audio source buffer with dummy data, silence presumably, 15:09:13 .. then you can maintain the continuous throughput so that playback doesn't stall due to 15:09:22 .. an underflowing buffer? 15:09:33 Daniel: Yes that would be more elegant than what we have today, 15:09:48 .. where we tear the whole thing down and set it up again for each period. 15:10:10 .. I agree dummy data would be more elegant but I don't know if anyone does that in their players today. 15:10:28 .. You would need dummy data or reuse part of the previous segment 15:10:34 cpn: Two suggestions. 15:11:06 .. 1. Use different players with different source buffer sets and switch the video player in the page 15:11:22 Daniel: Maybe performance problems hiding and showing players. 15:11:31 .. Not sure about ad blocker behaviour either. 15:11:47 .. If you're in full screen and do the switch, is it possible to continue in full screen? 15:11:59 .. The timing of the switch might not be completely accurate. 15:12:22 cpn: Wonder if the switch timing is a problem when switching media sources? 15:12:37 Daniel: Yes that might also be a point. Would need to check how accurate it would be. 15:12:47 .. Maybe there's a way to request removal of a buffer when empty. 15:12:56 .. Also what about playback through unbuffered ranges. 15:13:10 alicia: In gstreamer we have a solution for this called capped events. 15:13:20 .. Each track has a serialiser stream of buffers which are frames and events 15:13:38 .. Events can be "change the coding parameters for the upcoming part of the video" 15:13:54 i|Dainel responded|-> https://www.w3.org/2025/01/07-me-minutes.html#t07 FYI, related discussion during the MEIG call on Jan 7| 15:13:56 rrsagent, draft minutes 15:13:57 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:14:03 .. Or Gap Events where you are not going to get frames for x seconds but don't consider it an underrun 15:14:11 s/capped/gap/ 15:14:14 .. Explicitly say that there's no data for a period. And they're format agnostic. 15:14:23 .. Could try to do that in MSE if it works for the use cases. 15:14:25 i|Daniel responded|-> https://www.w3.org/2025/01/07-me-minutes.html#t07 FYI, related discussion during the MEIG call on Jan 7| 15:14:41 .. They're just like frames. Instead of AppendBuffer could call AppendGap, and specify a time 15:14:44 rrsagent, draft minutes 15:14:45 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:14:52 .. period in the stream where there is intentionally no data. 15:14:56 .. Would be the correct time and duration as specified. 15:15:06 Daniel: Sounds a bit like Chris's proposal? 15:15:27 cpn: Possibly [pulls up #160] 15:15:43 q+ alicia 15:15:46 .. We're interested in keeping as close to the live edge as possible and having a mode where that's 15:16:05 .. more important than continuous live playback, so if there's a drop out skip ahead to the most recent 15:16:18 .. playback position, to avoid buffering and delay, important for live sports for example. 15:16:42 Daniel: Dave was contributing to dash.js at the time - 2016 was a long time ago! 15:17:07 q- alicia 15:17:30 cpn: Figuring out where this issue got to... 15:17:47 .. In the WG this hasn't been progressed particularly. It was discussed in FOMS 2023 for example 15:18:08 https://github.com/w3c/media-source/issues/160 15:18:56 s|https://github.com/w3c/media-source/issues/160|-> https://github.com/w3c/media-source/issues/160 media-source issue 160 - Support playback through unbuffered ranges, and allow app to provide buffered gap tolerance| 15:19:00 Daniel: I think there was a hacky way to skip the gaps and it didn't work in all browsers. 15:19:02 rrsagent, draft minutes 15:19:03 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:19:16 cpn: We have to be a bit careful in this group - we can't design solutions because we don't have IPR coverage. 15:19:29 .. We can identify issues with existing APIs and be very clear about the requirements we have. 15:19:37 .. Solution design needs to happen in the MediaWG. 15:20:01 .. We could put this onto the Media WG agenda, then we would have the browser implementers 15:20:05 .. in the conversation as well. 15:20:16 Daniel: Would be good from my point of view too. 15:20:37 .. Also want feedback from other player makers, Shaka Player, HLS.js etc 15:20:52 .. Would like to know if these things I'm stumbling upon also affect others. 15:20:58 .. Want feedback from implementers. 15:21:45 cpn: We know e.g. Joey Parrish from Shaka Player. And does Gary Katsevman still work on players? 15:21:58 .. The other place is the video-dev Slack group, which can be a good place to get this kind of feedback. 15:22:13 .. There's a good community of app and player developers there. 15:23:02 .. Common use case where there's a stall and you want to get to latest playhead position. 15:23:24 .. Though this suggests there's sparse data so you want to resume playback and then 15:23:30 .. restart buffering later. 15:24:02 alicia: I do think the two use cases are separate though 15:24:23 .. I've never been super-convinced about low latency MSE, it always felt like a job for WebRTC. 15:24:40 cpn: You would need the underlying media playout to be aware it was in that mode so it could 15:24:52 .. take over without relying on the javascript doing something perhaps. 15:25:20 .. If we go back to #359, more concretely what are our next steps? 15:25:39 .. I think its worth replying to explain the limitations of having multiple video elements. 15:25:57 .. And also describe the latency and timing considerations and our goals, i.e. maintaining continuous 15:26:07 .. throughput and playback, minimising gaps when there is a switchover. 15:26:22 .. We think the switching of a media source would have the same timing issues as switching the media 15:26:25 .. element itself. 15:26:45 .. The full screen thing is interesting because if the video element itself is full screen then 15:26:58 .. I can see how you would potentially have a visible impact when switching. 15:27:12 .. But can there be a full screen container element that has video as a child, to make it smoother? 15:27:32 alicia: Additional complication. What if the event is at the beginning of the video, so we have 15:27:47 .. a video that starts without a video. How would we handle that? Would it be empty? Maybe. 15:28:08 .. But I guess you could still use the initial resolution of the video - that's already a thing in MSE. 15:28:32 .. When you sent the first init segment I think that sets the resolution - may be misremembering. 15:28:40 Daniel: I think there are always interesting edge cases. 15:28:47 .. I just asked in the video-dev channel. 15:28:59 JohnRiv has left #me 15:29:06 .. I'm with you that the solution, and for #160, may be just to tell the buffer that it's okay to be without 15:29:20 .. content for a while, but then you have to handle no video at the beginning. 15:29:35 alicia: Or you could signal "now we're in ads". 15:29:45 cpn: Would the web app want control over the holding image? 15:29:53 alicia: I was suggesting it could be the last image shown. 15:30:10 .. Simplest solution, then can be customised by sending the last image you want to show. 15:30:26 cpn: I was wondering if configuration to select "last good image" or "black screen" would be needed. 15:30:43 alicia: I think it's better to have an explicit gap API than just a flag of ignore underruns. 15:31:00 .. I think it would backfire if people aren't sure how they should handle underruns. 15:31:30 .. For audio-free ads, it needs to be clear that we are not going to have audio for some time. 15:31:44 .. For the live stream maybe the video element needs a way to set a policy of how to handle 15:31:52 .. underruns. 15:31:59 cpn: Good point you might want different behaviour 15:32:31 alicia: Not only MSE, could be a problem in low latency even if not using MSE, 15:32:38 .. so could be something for the video element itself. 15:32:49 Daniel: The Media over QUIC group was doing something similar, 15:33:08 .. dropping video data and pausing video while audio continues, until video data comes. 15:33:24 .. Showing last frame of video. 15:33:49 alicia: What if the javascript code to handle the underruns cannot run because some other intensivefe 15:34:02 .. thing is happening. What would that do to timing? 15:34:08 s/vefe/ve 15:34:33 cpn: The videos I've seen are all over web transport with canvas-based rendering. 15:34:57 Daniel: The use case of keeping the last frame would not be possible - they would face the same 15:35:12 .. limitation with an MSE-based implementation that we are discussing here right now. 15:35:44 .. If we could handle gaps by playing over them then we wouldn't need to remove and add source buffers. 15:36:13 .. Or have an API that allows us to append dummy data to fill the buffer like virtual data or whatever you want to call it. 15:36:38 alicia: Could be as simple a a gap with a dictionary saying the start and end timestamp and timescale. 15:36:56 cpn: How does this differ from #360 multiple source buffers. 15:37:23 daniel: With two source buffers you could pre-buffer the second buffer and then at the segment boundary 15:37:38 .. signal the switch so the player can pick up the already filled source buffer. 15:37:45 alicia: You can already do this, right? 15:37:48 rrsagent, draft minutes 15:37:50 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:38:01 daniel: You could do it with a virtual buffer, yes. 15:38:21 piers: A bit like the two player approach. 15:38:40 alicia: Assume that we are talking about milliseconds if not perfectly aligned? 15:39:02 piers: It's comparable to the two player approach, just taken down one level, having two 15:39:14 .. source buffers instead of two players, and then switching between them. 15:39:54 daniel: Now you can only create one source buffer per type right? 15:40:00 Alicia: There's a VideoTrack API, and according to spec there's an API to switch tracks and in theory you can switch tracks 15:40:01 i|How does this differ from #360|-> https://github.com/w3c/media-source/issues/360 media-source issue 360 - Multiple attachable/detachable SourceBuffers| 15:40:03 rrsagent, draft minutes 15:40:04 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:40:14 ... If that's the case, the problem is in implementatrion 15:40:33 i|There's a Video|scribenick: cpn| 15:40:40 Daniel: https://www.w3.org/TR/media-source-2/#dfn-sourcebuffer-configuration 15:41:10 ... In theory, for this use case we'd need 2 SourceBuffers per track, so 4 overall 15:41:24 Alicia: Not sure how well tested that is 15:41:34 Daniel: When I tried it, it said only one is allowed 15:41:54 Alicia: The Track API doesn't have good cross browser support, WebKit has some support, but experimental at the time in other browsers 15:42:18 Daniel: The spec does mention it as a quality of implementation issue 15:42:47 s/buffer-configuration/buffer-configuration Meia Source Extensions Working Draft - SourceBuffer configuration| 15:42:53 Nigel: Is the result of this conversation, that the requirement should be "must support at least two configurations" so you can switch between them, for both audio and video? 15:42:58 s/configuration|/configuration/ 15:43:00 rrsagent, draft minutes 15:43:01 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:43:24 Nigel: Or put it differently, the requirement is for smooth transitions between configurations that may have different numbers of sources 15:43:26 Daniel: I think the latter 15:43:51 ... Support one per type, but if you want to guarantee smooth transition, e.g., for ad insertion, then benificial to support two SourceBuffers per type 15:44:18 Alicia: It mentions the Tracks API in 3.3 activeSourceBuffers 15:44:37 -> https://www.w3.org/TR/media-source-2/#activesourcebuffers-attribute 3.3 activeSourceBuffers attribute 15:44:44 ... so already covered by the spec 15:45:28 Could say we have several tracks and want to switch between them. Seems already covered by activeSourceBuffers 15:45:38 ... The spec allows playing multiple audio tracks but only one video track 15:45:52 Daniel: I agree it's implicitly there, but to select it you'd have to have multiple tracks 15:46:07 Alicia: Which you do when you have multiple SourceBuffers 15:46:23 Daniel: Could be good to point to this in the GitHub issue 15:46:32 ... Do you support multiple SourceBuffers in WebKit? 15:46:37 Alicia: We do 15:47:17 Piers: It doesn't imply you can switch off the video though? 15:49:18 Alicia: Three features: 1) Signalling gaps in either a track or a SourceBuffer, 2) a way to tell the video element that you care more about being on time than not having underruns, and 3) Implementation of being able to switch between video tracks in an MSE context 15:51:20 Chris: Next steps: document requirements, limitations of multiple media elements, and timing issues with switch MediaSources 15:51:29 ... Also gathering input from other video players 15:51:47 ... Then bring it all to the working group for discussion 15:52:51 Chris: I'm confused about the overlapping segments in #360 15:53:10 q+ 15:53:21 i|Next steps|topic: Next steps| 15:53:36 Daniel: I think it means you play segment 1 to the end, then play parts of segment 2, when segment 1 ends you play segment 2. 15:54:05 Nigel: For ad insertion, you want to cut away from segment 1, then play segment 2 at the time of the ad, but the limitation is you can't replace segments or overwrite 15:54:12 i|I think it means|-> https://github.com/w3c/media-source/issues/360 media-source issue 360 - Multiple attachable/detachable SourceBuffers| 15:54:15 rrsagent, draft minutes 15:54:17 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:54:25 Daniel: In this case you can replace the parts of segment 1 that you don't want to play 15:55:02 Alicia: Often there are deviations between time when things, e.g., due to edit lists 15:55:18 Daniel: We should clarify the diagram. Multiple IDR frames, where everything is lost to the next IDR 15:56:02 Piers: And what kind of media encoding change you have. This could be trying to cope with a situation where the content can't be conditioned (adjusted in length so the segments all fit together). 15:57:11 Topic: DataCue 15:57:31 Rob: I'm looking at geo-tagged video, interseted in timed metadata 15:57:42 ... We proposed DataCue in WICG, which has stalled 15:57:48 i|I'm confused about the over|topic: Multiple attachable/detachable SourceBuffers| 15:57:50 rrsagent, draft minutes 15:57:51 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:57:57 ... I've been doing some research to try to move it forward 15:58:02 ... I wrote a polyfill that works in all browsers 15:58:14 s/topic: Next steps// 15:58:15 rrsagent, draft minutes 15:58:16 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 15:58:42 ... I looked into the history, DataCue was in HTML and then removed 15:58:45 ... The issue is there isn't an exposed TextTrackCue constructor 15:59:11 ... The original design had DataCue and VttCue as derived classes, which have constructors 16:00:14 ... If we could expose the TextTrackCue constructor, it would allow us to create cues of their own making, e.g., DataCue, or other varieties 16:00:39 ... So can be a way to progress this, and allow experimentation with different cue types 16:00:57 ... VTTCue as a specific cue for timed text and DataCue as a generic cue for timed data 16:01:20 ... It would work as an exposed constructor. But the only constructor today is VTTCue 16:01:54 ... And it doesn't have the vital thing, which is a type field, for metadata, so you could have a pub-sub type interface for particular types 16:02:09 rrsagent, draft minutes 16:02:11 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz 16:02:31 q? 16:03:16 Nigel: I think it makes sense for TextTrackCue to be concrete, or for DataCue to have a constructor, and agree with the intent to have something unburdened with the additional stuff from TextTrackCue 16:03:16 s/TextTrackCue/VTTCue/ 16:03:52 Rob: My polyfill is based on VTTCue, but I have another for Safari where TextTrackCue has a constructor, based on a presentation from Eric and Tess from 2020 I think 16:04:03 ... where they were trying to use TextTrackCue directly 16:04:22 ... Exposing the constructor would enable them to do that as well, for the cue they wanted to design 16:04:41 Nigel: Makes sense, I'd ask what the default event handlers should be 16:05:10 ... VTTCue has some default behaviour 16:08:17 Rob: Should I raise an issue? 16:08:25 Chris: Please, in the WICG/DataCue repo 16:09:57 ... Also we can discuss offline, if helpful 16:10:00 ack k 16:10:30 [adjourned] 16:10:43 rrsagent, draft minutes 16:10:45 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html cpn 16:12:17 i|ad|kaz: We should clarify how to organize the discussion around these topics (and possibly some more issues around MSE). Let's talk about that during the ME Chairs call.| 16:12:20 rrsagent, draft minutes 16:12:21 I have made the request to generate https://www.w3.org/2025/02/04-me-minutes.html kaz