21:02:28 RRSAgent has joined #mediawg 21:02:33 logging to https://www.w3.org/2023/05/16-mediawg-irc 21:02:38 Zakim has joined #mediawg 21:03:05 Meeting: Media WG meeting 21:03:08 scribe+ cpn_ 21:03:10 sandersd has joined #mediawg 21:03:26 Agenda: https://github.com/w3c/media-wg/blob/main/meetings/2023-05-16-Media_Working_Group_Teleconference-agenda.md 21:04:48 xiaohan has joined #mediawg 21:06:00 jernoble has joined #mediawg 21:07:53 Has the meeting started? 21:11:24 present+ Dale_Curtis, Dan_Sanders, Xiaohan_Wang, Jer_Noble, Tommy_Steimel 21:12:23 present+ Chris_Needham 21:14:25 Topic: Media Session 21:14:38 jernoble_ has joined #mediawg 21:14:38 Tommy: I'd like to propose a new Media Session Action, enterpictureinpicture 21:15:14 ... There's value in having a MS action to delegate to the website, e.g., if there's more than one video, or if the website uses a canvas-backed video 21:15:44 ... I don't think we need exitpictureinpicture, as the UA can close any PiP window without going through the website 21:16:13 ... Chrome has UI the user can click. We're thinking of also having a way to automatically enering PiP when switching between tabs 21:16:46 ... Two possibilitities: have a boolean or enum to distinguish auto PiP or if the user explicitly wants it, or maybe two separate actions 21:16:55 ... Any feedback on which might make more sense? 21:17:24 Jer: I wonder whether the information you're trying to convey is already available from the Visibility API 21:17:37 ... In the case of auto PiP behaviour? 21:17:55 Tommy: But the page might not also be visible, so may not be enough 21:18:15 Jer: So it's a boundary between visibility and not visibility, and a user action in the UA UI somehow? 21:19:07 ... Would a site want to block auto pip but enable pip more generally? 21:19:28 ... iPad OS has an auto pip feature where any full screen video goes to PiP 21:19:43 ... I haven't seen a request to disable that, except for disabling PiP entirely 21:19:56 ... Not an argument against putting context info in the action handler 21:20:18 ... From my experience as an implementer, it's all or nothing, you either want it or not 21:20:48 ... Do you want a boolean or an enum with reasons? 21:21:12 Tommy: I don't foresee other reasons to need an enum 21:22:07 Jer: They're both user actions, so how to distinguish in an enum? One way out is to not provide context, unless there's a case where someone wants to prevent PiP in once scenario and not the other 21:22:28 ... Suggest considering what the enum values might be, a signal for why the pip started 21:22:39 Tommy: So you'd prefer that to a separate action? 21:22:43 Jer: I think it does 21:23:16 ... When auto-PiP starts, for Safari, there's no time to enter PiP, the user is swiping, entry to PiP has to be simultaneous with other animations 21:23:43 ... Worry that won't complete in time. Could we implement without breaking the animation. Shouldn't block the media session action for enter PiP 21:23:53 ... We might not be able to implement for the same cases as Chrome 21:24:34 Chris: When would Chrome do auto enter PiP 21:25:07 Tommy: We have global controls for manual use, and we're talking about when switching tabs would useful for users 21:25:37 markw__ has joined #mediawg 21:25:42 Chris: So follow up is to look at the enumeration, and what to put in the media session action details 21:26:10 Topic: EME 21:26:26 Xaiohan: This is about mixed encrypted an unencrypted content 21:26:43 ... There's been lots of discussion, I made a recommendation in the GH issue, so want feedbac 21:26:51 ... https://github.com/w3c/encrypted-media/issues/251 21:27:05 ... Right now the spec is unclear, may be a quality of implementation issue 21:27:15 ... It's used with ad insertion where ads are unencrypted 21:27:38 ... on Disney+, they show a logo then switch to encrypted, which can cause a failure 21:28:06 ... The media pipeline doesn't know it will be encrypted later, so when we have a clear pipeline it might be harder to switch to an encrypted pipeline 21:28:27 ... Can handle seamlessly in Chrome, but there are cases in hardware where it's harder to do the switch 21:29:00 ... My recommendation is if the site knows it will be mixed content, it should set the media keys up front, so the UA knows to setup the correct pipeline 21:29:24 ... Doesn't need license exchange, get the keys and set on the media element, then there's a strong hint what to do 21:29:43 ... If media keys is set before playback, it should mixed playback, otherwise it's a quality of implementation issue 21:30:16 ... Then we can advice sites what to do. Improve compatibility / interop as well 21:30:20 ... Any comments? 21:30:53 Jer: To clarify, do you propose adding a note on recommended practice, or is there a normative spec change to always set the media keys before starting playback? 21:31:31 Xiaohan: Current recommendation is to change the spec text, and add a note on the QoI issue 21:31:47 ... If keys are set up from, the implementation *must* support ... etc 21:31:56 s/Xaiohan/Xiaohan/ 21:32:18 ... It's been open issue for a few years 21:32:35 Jer: I see same behaviour with Netflix, but they may be doing something different than Disney+ 21:33:00 Mark: There are corner cases on some platforms where it doesn't work, so I support having this clearer in the spec 21:33:31 Jer: I remember this came up with changeType, changing between clear and protected content 21:33:39 Xiaohan: That's on the MSE side 21:34:07 Jer: On Apple platforms we support protected playback in HLS, elsewhere I assume it's done with MSE 21:34:15 Xiaohan: that's true 21:34:55 ... There are two ways you can start with clear playback. You append the segment, where initial samples are clear. singal that the whole thing is encrypted, that's fine 21:35:21 ... Other case is append all clear buffers, then later encrtpyted. But that can be too late as we already set up the pipeline 21:35:36 Jer: Seems reasonable that setting the keys you can change to encrypted later 21:36:17 Xiaohan: Seems there's agreement. I'll ping you on the bug to agree the exact text 21:36:54 Chris: Timecale for EME draft spec? 21:38:22 Xiaohan: Editors met to discuss scope for V2, no progress, so I'll ping them again to follow up, get a more firm commitment 21:38:43 Chris: Concern that we have implementations but not a spec, and AC reviewers may ask about our intentions 21:39:19 Topic: WebCodecs 21:39:39 Eugene: https://github.com/w3c/webcodecs/issues/92 21:40:14 ... This is to allow when we readback a video frame there'll be a way to convert the pixel format, e.g., YUV to RGB at the same time 21:40:37 ... Currenlty we can use getImageData on the canvas, but that's synchronous, requires a Window context, so inconvenient in workers 21:40:52 ... Other way is manually in WASM, but needs a lot of code, and account for color spacves 21:41:19 ... Since browsers can already render all video frames, they could convert at least to RGB, so not arbitrary conversions 21:41:54 ... Proposal is to add an extra format field in copyTo, and when the UA can do the pixel format conversion, it'll return the size of the buffer 21:42:22 .. if UA doesn't support it will throw an exception. If can determine the buffer size, it could convert during readback of the Video frame 21:42:42 ... Dale pointed out it may not be a good idea to signal if conversion supported by throwing an exception 21:43:06 ... Another option is to introduce an extra method, canCopyTo, for the purpose of checking if a conversion is supported by the UA 21:43:36 ... What do you think? What's the best way if a conversion is OK? 21:44:10 Jer: Converting YUV to RGB is fine if they have the same color space. But decoded frames might not be SRGB, so implies a colorspace conversion 21:44:19 ... Are we doing both color conversion and pixel format conversion? 21:44:32 Eugene: Video is tagged with color space, so it's both 21:45:06 Jer: If you have an HDR image and you want to convert to P3 colorspace, do we need a separate enum? Or a second call? 21:45:39 Dale: For color space, we'll need both format and color space. For Chrome we're thinking of only converting to RGB 21:45:48 Jer: There's different white points and matrices... 21:46:26 Bernard: This shows why canCopyTo might be problematic, needs lots of options 21:46:42 Dale: Has to have some way to indicate if supported or not, either a boolean or an exception 21:46:52 Jer: Is copyTo sync? 21:46:57 Dale: It's async 21:47:16 Jer: So instead of throwing it could return a rejected promise 21:47:52 Eugene: Yes, but ideally we want to know in advance, we do prep to determine buffer size, so if we know that we know if we can do it at all 21:48:15 Jer: With the caveat that the color spaces might not be available 21:49:04 Eugene: True, but we can synchronously determine that. Alternatively, people will do it anyway, e.g., for doing ML models on video frames they want RGB, so they use canvas currently 21:49:19 ... We want to removes those extra steps, async, and in a worker 21:49:35 Jer: The Window requirement for canvas, is that due to lack of OffscreenCanvas? 21:49:45 Eugene: With OffscreenCanvas we're good 21:50:47 Jer: But that only supports RGB, so you wouldn't be able to 21:51:11 Chris: Are looking at Color on the Web CG, they're figuring out some of this stuff 21:51:18 Jer: Yes 21:52:34 Eugene: copyTo will reject if you call it with something unsupported. If nobody objects, we could go with this, I'll follow up with Paul, he was against this approach I think 21:53:01 ... I can prepare a pull request for the spec 21:53:20 Dan: Are there other cases than SRGB where the browser will know what colorspace we want rather than rely on the user? 21:54:00 Jer: Use case is painting to a P3 or HDR canvas and you don't want to reduce to SRGB. Not somehting the UA knows the client in the page may know the intent 21:54:27 ... If canvas will support more color spaces, you may need a convert to RGB 21:54:58 Dan: Odd coded sizes issue 21:55:31 ... Using coded sizes that aren't a multiple of the canvas size. VP9 lets you specify it has an odd coded size, so this happens in practice a lot 21:55:58 ... Proposal to define, and round up. Implies you can sample a half pixel outside the visible rect 21:56:11 ... This is what ffmpeg and VLC will do 21:56:29 ... Is that an OK definition to use throughout WebCodecs, or any concerns? 21:57:05 s/sizes issue/sizes issue: https://github.com/w3c/webcodecs/pull/666/ 21:57:12 Jer: Not hearing any concerns 21:58:56 Eugene: Content hint issue 21:59:13 ... Some codecs benefit from knowing, some codecs take the info into account. IsConfigSupported won't reject if there's a content hint it doesn't like 21:59:40 ... We have a spec, content hints spec, motion, detail, text, etc 22:00:19 Bernard: Motion vs frame rate is more for rate control. In libwebrtc for h264 hints theres QP stuff we already support, but this other things 22:00:32 Jer: Difficulty is coming up with list of hints across all decoders 22:00:49 Bernard: Detail motion, text, text added for AV1 screen content tools 22:00:57 https://w3c.github.io/mst-content-hint/ 22:01:37 Bernard: The only concern is make sure it doesn't conflict with other things like screen coding tools 22:01:51 Eugene: it's an implementation detail 22:02:25 Bernard: They'll change to minimum QP. 22:02:57 ChrisN: Follow up in a WG meeting? 22:03:10 Bernard: Useful to do sooner than next month 22:05:42 Topic: Rechartering and TPAC 22:05:57 ChrisN: AC review is open, please ask your AC rep to review 22:06:27 ... And we've requested meeting time for Monday September 11. Hope to see you there 22:06:38 [adjourned] 22:06:44 rrsagent, draft minutes 22:06:45 I have made the request to generate https://www.w3.org/2023/05/16-mediawg-minutes.html cpn_ 22:06:55 rrsagent, make log public 22:08:28 present+ Berard_Aboba, Mark_Watson 22:08:40 chair: Chris_Needham, Jer_Noble 22:08:42 rrsagent, draft minutes 22:08:44 I have made the request to generate https://www.w3.org/2023/05/16-mediawg-minutes.html cpn_ 22:09:06 regrets: Francois_Daoust 22:09:08 rrsagent, draft minutes 22:09:09 I have made the request to generate https://www.w3.org/2023/05/16-mediawg-minutes.html cpn_