21:01:05 <RRSAgent> RRSAgent has joined #mediawg
21:01:05 <RRSAgent> logging to https://www.w3.org/2020/07/14-mediawg-irc
21:01:08 <Zakim> RRSAgent, make logs Public
21:01:08 <Zakim> please title this meeting ("meeting: ..."), nigel
21:01:38 <nigel> Meeting: Joint meeting between CSS WG and Media WG
21:01:48 <nigel> Agenda: https://github.com/w3c/media-wg/issues/18
21:01:52 <ving> ving has joined #mediawg
21:01:57 <nigel> Present+ Nigel_Megitt
21:02:43 <cpn> Present+ Chris_Needham, Alan_Stearns, Daniel_Holbert, Florian_Rivoal, Gary_Katsevman, Greg_Freedman, Kevin_Babbitt, Jer_Noble, Matt_Wolenetz, Mounir_Lamouri, Peng_Liu, Simon_Fraser, Simon_Thompson, Vi_Nguyen, Theresa_O'Connor, Jimmy_Huang, Rossen_Atanassov
21:02:50 <wolenetz> wolenetz has joined #mediawg
21:03:12 <Rossen_> Rossen_ has joined #mediawg
21:03:57 <cpn> present+ Becca_Hughes
21:03:57 <kbabbitt> kbabbitt has joined #mediawg
21:04:28 <jernoble> jernoble has joined #mediawg
21:05:27 <GregFreedman> GregFreedman has joined #mediawg
21:05:35 <jimmyhuang> jimmyhuang has joined #mediawg
21:05:43 <pengliu> pengliu has joined #mediawg
21:06:44 <jernoble> scribe: jernoble
21:06:44 <jernoble> Topic: Introductions
21:07:55 <florian> Florian Rivoal, Invited Expert
21:08:14 <dholbert> Daniel Holbert, Mozilla
21:08:16 <astearns> Alan Stearns, Adobe
21:08:17 <smfr> smfr has joined #mediawg
21:08:21 <wolenetz> Matt Wolenetz, MSE spec co-editor, Google
21:08:22 <jernoble> Jer Noble, Apple
21:08:24 <GregFreedman> Greg Freedman, Netflix
21:08:29 <smfr> Simon Fraser, Apple
21:08:32 <Rossen_> Rossen Atanassov, Microsoft
21:08:35 <jimmyhuang> Jimmy Huang, Intel
21:08:37 <mounir> Mounir Lamouri, Google
21:08:38 <kbabbitt> Kevin Babbitt, Microsoft
21:08:41 <nigel> Nigel Megitt, BBC, Chair TTWG
21:08:46 <Simon_T_BBC> Simon Thompson, BBC
21:08:46 <hober> Theresa O'Connor, Apple
21:08:47 <ving> Vi Nguyen, Microsoft, MediaCapabilities
21:08:48 <pengliu> Peng Liu, Apple
21:08:59 <cpn> Chris Needham, BBC, Media & Entertainment IG co-chair
21:09:05 <gkatsev> Gary Katsevman, Brightcove, Chair TTWG
21:09:56 <chcunningham> @Google
21:09:04 <jernoble> Topic: Video Media Queries
21:09:04 <jernoble> florian: introduction of topic: some devices have separate planes for video and graphics contents
21:09:29 <jernoble> florian: these planes have separate properties including width, height, and resolution. and the existing queries target only the graphics plane
21:09:47 <chcunningham> chcunningham has joined #mediawg
21:09:52 <jernoble> florian: video sites care mostly about the graphics plane, and would like api to chose an appropriate variant
21:10:16 <jernoble> florian: originally the CSS WG accepted this proposal, but found issues and wondered if these were the correct way of solving this problem
21:11:38 <cpn> present+ Barbara_Hochgesang
21:11:48 <jernoble> florian: MQs need to come with a unit; CSS pixels? CSS Pixels can mean an angular measurement that changes size based on the distance from the screen. Physical pixels? anchor a CSS measurement to a physical measurement.
21:11:53 <beccahughes_> beccahughes_ has joined #mediawg
21:12:30 <jernoble> florian: for the video plane, you may want actual literal pixels, which would include a third definition.
21:12:36 <chcunningham> q+
21:13:12 <jernoble> florian: One possibility would be to make these properties unitless.
21:14:09 <jernoble> florian: this issue has an analogue for images, and the general path has been not to do this through media queries.
21:14:43 <jernoble> florian: images must also be sensitive to changes in resolution as windows move between windows, but also to available bandwidth
21:15:14 <jernoble> florian: for images, the UA will chose the appropriate image variant based on not only resolution, but other factors.
21:16:07 <jernoble> florian: this may not be the same solution available for video, but there could be a .js API in which the page provides the available resolutions and properties, then the UA choses the correct variant
21:16:12 <mounir> q?
21:16:19 <jernoble> florian: were there other arguments pro or con for media queries?
21:16:25 <mounir> ack chcunningham
21:17:30 <jernoble> chcunningham: question: does it not solve the problem by mapping the pixel ratio to a fixed value?
21:17:48 <jernoble> florian: we could make an allowance, but it's not currently allowed.
21:18:25 <jernoble> florian: what does that then mean when people query for the video plane in "em" units?
21:19:00 <jernoble> florian: There's not a number that is based on font sizes that is correct.
21:19:33 <jernoble> chcunningham: if you use the existing mechanism that ratio, you could discover the ratio.
21:19:57 <jernoble> florian: MQs don't allow you to query for the size of the screen, it only allows you to query for whether the screen is within a provided value
21:20:21 <jernoble> florian: maybe this is the wrong tool, since you can't query the value directly.
21:20:54 <jernoble> chcunningham: I don't think that's a dealbreaker, since sites will have a list of formats or variants with specific resolutions, so that works with the way MQs work
21:21:29 <jernoble> chcunningham: w.r.t. the Picture API; this would be a profound change from how media APIs are used on the web.
21:21:55 <jernoble> chcunningham: in MSE and WebCodecs, the page has low level access to variant selection
21:22:13 <jernoble> chcunningham: clients prefer to make these decisions themselves
21:22:37 <mounir> q?
21:22:45 <cpn> q+
21:22:48 <Simon_T_BBC> q+
21:23:08 <jernoble> dholbert: there are also "inch' and "centimeter" units
21:23:25 <jernoble> dholbert: that would be something diffecult to be define; argument for unitless value
21:24:04 <mounir> q?
21:24:07 <jernoble> florian: if we wanted to do this,  we don't want to break all of CSS units, so it would still be a fixed ratio between this value and a CSS inch
21:24:19 <mounir> ack cpn
21:24:52 <jernoble> cpn: There was a proposal to expose these values on the screen object; is there a value to re-visiting that decision?
21:25:37 <jernoble> florian: the query example is only one possibility; we could also just expose those numbers explicitly.
21:25:54 <wolenetz> q+
21:26:05 <jernoble> cpn: a proposal which inverts the current model that gives all the variant selection to the client would be  a big change
21:26:26 <mounir> ack wolenetz
21:26:41 <jernoble> wolenetz: is there already a way for JS apps to determine the resolution where the video will be composited?
21:27:08 <jernoble> florian: I think the MQ was proposed as a way to expose that value.
21:28:01 <jernoble> wolenetz: the current way media works on the web is to expose to clients underlying capabilities and allow the site to chose variants.
21:28:05 <florian> q?
21:28:09 <jernoble> q+
21:28:17 <mounir> ack Simon_T_BBC
21:28:35 <jernoble> Simon_T_BBC: set top boxes should be thought about.
21:29:24 <jernoble> Simon_T_BBC: HDMI capabilities must also be taken into account; some may only be capable of a certain frame rate
21:29:45 <jernoble> Simon_T_BBC: There are some non-screen parameters that are important for deciding which variant to chose.
21:30:24 <jernoble> florian: you may want to make a different tradeoff than just available resolution
21:30:28 <GregFreedman> q+
21:30:37 <chcunningham> q+
21:30:41 <ving> q+
21:30:47 <mounir> ack jernoble
21:31:45 <ving> scribe: ving
21:31:49 <ving> jernoble: as the person who proposed MediaQueries as a possible solution. MQ already provides event mechanism for when that ratio changes when moving windows to different screens. If we did expose something to Screens API, we have to figure out events
21:32:17 <florian> q?
21:32:21 <jernoble> scribe: jernoble
21:32:22 <florian> q+
21:32:26 <mounir> ack GregFreedman
21:32:29 <dholbert> q+
21:33:04 <mounir> ack chcunningham
21:34:09 <jernoble> chcunningham: I understand CSS WG is concerned that this is not the correct solution. But we ran this by the WG before it was proposed. Are we back to the drawing board, or is this a minor issue?
21:35:37 <jernoble> florian: I don't think the CSS WG has concluded that this was impossible; merely uneasy. On further reflection, the way it is spec'd cannot work. It doesn't work for print media. We would need to fix the unit issue to make this possible.
21:36:02 <jernoble> florian: we could fix this, but the questions this raises makes this solution feel "odd".
21:36:35 <jernoble> florian: there's also a desire in the CSS WG that for devices that do not have a separate video plane, these properties should be the same for the graphics plane.
21:37:19 <jernoble> florian: screen properties were deprecated, because leaking non-important information, and those properties have privacy implications
21:37:45 <jernoble> florian: CSS pixels can and will be used incorrectly
21:37:58 <jernoble> florian: we think this should return to the drawing board.
21:38:11 <mounir> q?
21:38:19 <jernoble> florian: the current approach is possible, but not ideal.
21:38:46 <mounir> ack ving
21:38:53 <florian> q-
21:39:22 <jernoble> ving: the Media WG did discuss exposing this to the Screen object; could we get a summary of why Screen is not the ideal proposal?
21:40:16 <jernoble> ving: also, there was a concern about interfaces and cables, but that should be handled through MediaCapabilities API. We're trying to solve the problem of exposing the exact resolution of the video plane on the actual display.
21:41:09 <jernoble> chcunningham: where are you seeing these concerns jernoble listed?
21:41:15 <jernoble> ving: in the notes.
21:42:34 <jernoble> chcunningham: I don't oppose putting this on screen; florian what do you think of having device pixels on the Screen?
21:42:47 <jernoble> florian: my personal feeling is that's ok; we'd need events if that changes
21:43:00 <GregFreedman> q+, maybe my microphone is working
21:43:01 <jernoble> florian: is this a concern for those devices that have a separate video plane?
21:43:40 <dholbert> q-
21:43:48 <jernoble> mounir: I don't think an event is really an issue. The Screen object doesn't have an event. This is an existing problem in that the Screen's properties can change without an event.
21:44:18 <jernoble> chcunningham: if we expose device pixels on the screen object, we're hitting the same reasons why the existing properties are deprecated
21:44:55 <jernoble> florian: if we expose the video device pixel, you'd just get a number that is in device pixels.  You wouldn't have a way to ask for the video plane in "em" unit.s
21:45:04 <jernoble> florian: yes, we'd still have the privacy concern.
21:46:07 <jernoble> GregFreedman: if you put device pixels on the Screen object, you'd still have the problem of the difference between the video and graphics plane. Would you have device pixels for both planes?
21:46:44 <jernoble> GregFreedman: I kind of liked where we landed with the MQ. what we want to do is say: can you support this resolution on the video plane; and that seems like this works with MQ well.
21:46:55 <jernoble> GregFreedman: is the problem that CSS doesn't like the unitless query?
21:47:37 <jernoble> florian: yes, CSS as a language could support it, but if we're on a device that does not have a separate plane, we probably want width: to do the same thing as video-width:
21:47:48 <nigel> q+ to ask for a reminder of the use cases that remain if we use MediaCapabilities to choose which video to get
21:48:02 <jernoble> florian: additionally, that definition also affects print media
21:48:12 <jernoble> florian: so what should you do on print media?
21:48:29 <mounir> ack nigel
21:48:29 <Zakim> nigel, you wanted to ask for a reminder of the use cases that remain if we use MediaCapabilities to choose which video to get
21:49:19 <jernoble> nigel: the use case is originally to detect what variant to display. If MediaCapabilities is the best way to query for that, what's left in CSS ?
21:49:56 <wolenetz> q+ to say that decoder capabilities exceeding display capabilities shouldn't limit what is decodable
21:50:04 <jernoble> GregFreedman: the device may be capable of doing 4k UHD, but if the display is only 1080p SD, there's no reason to pick that variant. so we split it into two APIs: one to query decode, one to query display.
21:50:20 <jernoble> ving: MediaCapabilies also is not screen aware.
21:50:38 <cpn> q+
21:50:51 <jernoble> nigel: could MQs handle that scenario, where there's multiple displays?
21:51:11 <jernoble> hober: we already handle that with the 50%+1 to determine which monitor's properties to return.
21:51:21 <jernoble> chcunningham: that's the implementation in chrome.
21:51:55 <jernoble> cpn: can the UA play this particualar content is answered by the combination of screen and decode capabilities.
21:52:29 <jernoble> cpn: asking these two questions separately doesn't give a complete picture of whether a variant can be successfully played
21:52:47 <jernoble> GregFreedman: MC asks "can we decode this" and MQ asks "should we decode this".
21:53:40 <jernoble> nigel: is the set of things we need to know just the video plane resolution, and the frame rate?
21:54:04 <jernoble> nigel: are there other properties that we need to include?
21:54:11 <jernoble> florian: those do not yet exist?
21:54:27 <mounir> ack wolenetz
21:54:27 <Zakim> wolenetz, you wanted to say that decoder capabilities exceeding display capabilities shouldn't limit what is decodable
21:54:32 <jernoble> nigel: just wondering about design and what info needs to be included.
21:54:46 <jernoble> q+
21:54:49 <cpn> q-
21:55:22 <jernoble> wolenetz: MC already gives you the answer of whether you can decode a given piece of media. Do we try to move some of the properties in MC into MQ?
21:56:10 <jernoble> mounir: oginially there was information in the Screen object.
21:56:39 <jernoble> wolenetz: since MC is already being asked about candidates, maybe MC is the best place to ask more questions
21:57:17 <jernoble> wolenetz: since I'm already using MC, doing that again against a different API is difficult.
21:57:46 <jernoble> chcunningham: I have the exact opposite opinion, respectfully. the Screen API and the MQ API already exists.
21:58:09 <smfr> q+
21:58:14 <jernoble> chcunningham: we never considered putting MC queries into the Screen API
21:58:49 <jernoble> mounir: websites already will react to window size changes, e.g., when going into fullscreen.
21:59:02 <florian> q+ to suggest that the JS API could take the frame rate as input more easily than MQs
21:59:22 <mounir> ack jernoble
21:59:30 <mounir> scribe: mounir
21:59:53 <mounir> jernoble: we want to answer the question whether the video should be upscaled or downscaled
22:00:01 <mounir> jernoble: so maybe we should design an API for that?
22:00:15 <jernoble> scribe: jernoble
22:00:15 <jernoble> smfr: I would be reluctant to add new APIs to the Screen api due to fingerprinting concerns.
22:00:24 <mounir> ack smfr
22:01:14 <mounir> ack florian
22:01:14 <Zakim> florian, you wanted to suggest that the JS API could take the frame rate as input more easily than MQs
22:01:20 <jernoble> florian: if the frame rate matters, it would be more amenable to a Screen API, since you can pass in parameters to that API, but that style of query isn't possible with MQ.
22:02:46 <jernoble> chcunningham: no option seems to be the clear winner; IMO the least of several evils was the CSS MQ options.
22:03:31 <nigel> CSS media queries don't remove the fingerprinting vector
22:03:43 <jernoble> chcunningham: I wonder if there is a way to fix this problem
22:03:56 <jernoble> florian: I'd like to ask Simon to clarify this answer about fingerprinting
22:04:59 <jernoble> smfr: it depends on whether it allows you to enumerate all screens. and depends on whether the API allows you to detect whether users have changed their screen resolution or are using non-standard third party screens
22:05:10 <jernoble> smfr: it's still doable with MQ, but you have to binary search, so it's not as easy.
22:05:36 <jernoble> mounir: what's the next step here? shall we move this to GitHub issues?
22:05:54 <jernoble> chcunningham: we haven't discussed any options beyond "fix MQ" or "move to Screen".
22:06:06 <astearns> lost my webex connection - would there be a way of making a video-playback-specific API that would expose less fingerprinting surface but still be able to answer coarse questions on what can be displayed? How fine-grained do we really need to be for playback?
22:06:40 <jernoble> chcunningham: maybe now that we've had this meeting the CSS WG will come up with an alternate solution that better solves all the use cases
22:07:17 <jernoble> mounir: lets follow up on GitHub; lets try to discuss between MediaWG and CSS WG people there
22:07:43 <jernoble> florian: it's unfortunate that there were fewer voices in the CSS WG than I anticipated; there are conflicting desires in the CSS WG.
22:07:53 <jernoble> florian: I'm unable to represent all those various POV.
22:08:22 <jernoble> florian: but yes, we can explore; either alternative ways of speccing the MQ or alternative API entirely.
22:08:45 <jernoble> chcunningham: thank you florian! and thank you for representing the CSS WG.
22:09:07 <jimmyhuang> jimmyhuang has left #mediawg
22:09:27 <cpn> rrsagent, draft minutes v2
22:09:27 <RRSAgent> I have made the request to generate https://www.w3.org/2020/07/14-mediawg-minutes.html cpn
22:09:41 <cpn> rrsagent, make log public
22:10:25 <cpn> Chair: Mounir, Jer
22:10:27 <cpn> rrsagent, draft minutes v2
22:10:27 <RRSAgent> I have made the request to generate https://www.w3.org/2020/07/14-mediawg-minutes.html cpn