Meeting minutes
https://github.com/w3c/openscreenprotocol/issues/194
[Remote Playback] Capabilities for HDR rendering and display #194
cpn: two parts within the WG: decode capability detection, and display capability detection
cpn: for display capabilities, there's an open discussion within the CSS WG for how to do so
mark_foltz: OpenScreen Protocol can send media to another device, so needs to be able to determine what media capabilities on the remote device exist to select the correct variant
mark_foltz: one of those capabilities is color, which we added as "color profiles", but would like to align the definitions with Media Capabilities
mark_foltz: I put up a pull request to use the "color gamut" capability with the same values as exposed through MC.
https://
mark_foltz: That pull request has been waiting for feedback from the Media WG before merging.
mark_foltz: The second piece was about the display characteristics of the display.
chcunningham: the displays discussion is mostly settled; HDR was added. Another discussion topic was about TV devices, which is not yet settled.
chcunningham: The last remaining issue is about device-pixel-ration when the video and graphics planes have different capabilities
chcunningham:
Media queries: https://
mfoltzgoogle: the capabilities are going to be used by the UA to decide on whether a specific variant is compatible with the remote device
mfoltzgoogle: there doesn't seem to be a need for more color gamuts than are currently specified in the MC API
cpn: how is this information exposed to the page?
mfoltzgoogle: for remote playback, this is internal to the UA
mfoltzgoogle: there's a question about whether this is exposed before remote playback is initiated
jernoble: IIRC remote playback API only sends the current URL being played
… Is there discussion of extending Remote Playback API to allow multiple sources?
<tidoust> [Right, I don't think that the Remote Playback API actually asks the user agent to only send the current URL being played]
jernoble: if you go to remote a set of sources, do we need to spec how the media queries apply to the remote display and not the local?
mfoltz: i'm not sure i have a good answer. depends on the type of remote device you're sending to
… once the two sides have negotated what streams to send, do we communicate that back to the sender? and does that get communicated back to the page?
jernoble: an extension is declarative video elements and sources, and enabling the negotation of those
mfoltz: it's worth tracking that
… I'll ping Chris and others with a PR. If someone sent a discussion in CSS for dynamic range media query, we can look at that
… landing both of those should close this
<mfoltzgoogle> Pull request: https://
Matt: For MSE, which may be one of the children in the media element's source list, is that list sent to the remote side during negotiation?
mfoltz: so you have a media element, with an MSE object url in the source list
… the protocol allows the agent to send a URL of the media source or the demuxed media
… if the remote agent can understand the object url, the implemtnation could do that, but doesn't seem to fit with how MSE works
… Chrome pipes the media fetched in the source buffer to the remote device over the streaming protocol
… The sending site isn't generally aware. They can use the Remote Playback API to know they're remoting, but they still use MSE to transfer the data
Matt: If there's a changeType that involves capability changes, the required set of capabilities may not be detectable up front
… unless you transcode the stream to a single type
mfoltz: a session can include multiple stream types, called encodings. the sender gets a full list of capabilities up front
… if the type of media encoding changes, it knows if it can change to that encoding
… we may have an issue open on that, but it should be handled
cpn: do you also need the video plane resolution?
mfoltz: yes, one property is the native resolution, what you can display without scaling the video, e.g., the panel resolution
… then what are the recommended scaled resolutions. you can enumerate preferred resolutions
… we could look at aligning terminology with CSS. the data is there, but could align
chcunningham: haven't looked at the protocol to know. the concept of getting the native resolution is simple
mfoltz: makes sense to align, but fine either way
chcunningham: you could save words by referencing the CSS spec of a bi-plane device
mfoltz: agree
Use the same codec names as the media APIs
mfoltz: want to align what we ask devices with what's in Media Capabilities
… I don't think there's anything difficult or controversial, but wanted feedback
… In the GH issue, I tried to track down some of the RFCs, extended MIME types that describe the codec and profiles and config attributes
… Do we see any difficulty using these strings to enumerate decoding capabilities?
chcunningham: the strings would be my first choice. Same question came up with WebCodecs, and chose to use those strings
… We made a registry of the strings and the RFCs that define them, to have clear reference
… the strings aren't perfect, but your best best
… Some evolution, VP9 didn't describe the profile, but now does
… Newer strings are more verbose, adding info on bit depth and colour
… MC API breaks those out separately, so we can support newer and older strings
… The people who made the strings also made the codec, so happy to defer to them
mfoltz: Is there an issue where enumerating codecs could result in a large number of them?
chcunningham: MC API the site asks the browser
mfoltz: we want to enumerate, depdends on hardware decoder support. could be a desktop with software support for a number of codecs
… could that result in a large number of strings?
chcunningham: consider the VP9 string, has 10 parts each with 2 or 3 values, so the combination is large
… with software decoding, can have a big list. so only describe the things that matter
… VP9 includes things about the stream, not just things for decode
… need to look at details, but could motivate some cleverness on communicating just the things you care about
… initially i'd just use the pieces you care about from the strings
mfoltz: we might make it possible for agents to drop information after the profile, or use a wildcard in some way
… maybe that's not quite as precise, but could be good enough
eric: that probably violates the RFC that this is based on
mfoltz: don't want to go that route unless we need to
… would be good to have some normative reference to the strings. sounds like webcodecs already did that
<chcunningham> webcodecs codec registry https://
mfoltz: we can probably use that to firm up the codec names to close this issue
AOB
matt: Origin trial for MSE in Workers is open
… Also MSE for WebCodecs, currently working on spec and implementations
cpn: I was on an AC call earlier. Question about whether groups have a feeling about whether they would like to hold a physical meeting. I believe that a survey is upcoming.
… W3C would like to plan logistics if there is a sense that physical meetings are going to be possible.
… Seems hard to make commitments though.
… Next meeting will be February 8th, early time Pacific time. If you feel we need an intermediary call, let us know.