jer: Welcome. Looking at the agenda, any change requested? Couple of changes in the last couple of days. Low-latency video signaling was added today.
mounir: I added WebCodecs, audiobooks, and I moved some things around for external folks to be able to join relevant sessions.
jer: This is the first meeting of
the Media WG. We don't really have a process or norms setup. We
didn't want to settle on anything before we had a chance to
discuss
... When will we have phone calls, F2F, email threads, github
issues, etc?
scottlow: I personally like calls, good way to progress issues.
richard: I prefer video calls sometimes.
jer: Sure, that's through WebEx, video is possible.
richard: We had troubles with webassembly groups, went through Zoom.
jer: Let's try to start with WebEx
narifumi: Guidelines for work mode, communications?
jer: We don't have them yet.
markw: I would be in favor of not
only have calls on topics, but also a regular cadence of
calls.
... Good way to pop issues back to the top.
... Calling for consensus on the calls.
mounir: The other side of that is
that some groups punt discussions to calls.
... Work in WICG went well without calls.
... I could be open to a regular call but don't want to discuss
everything.
markw: I agree with that.
cpn: Just would like the call not to overlap with Media & Entertainment IG monthly calls.
mounir: Something we could do.
Ada has a bot where you can do /agenda on an issue that adds an
issue to the agenda.
... I don't want to have everyone show up in a weekly call just
because the time is booked on your calendar
markw: Yes, not weekly calls. Regular cadence is a way to process backlog and close issues when there's no activity or call for volunteers to address them.
richard: Cancellation when there are no agenda items can be good.
jer: Yes.
... A monthly cadence seems good. I doubt we'll have nothing to
talk after a month.
cpn: Useful to have a dedicated slot on the calendar instead of ad-hoc doodling.
mounir: definitely
... Everyone here ok with a monthly call? And encourage ad-hoc
discussions?
markw: You could have a slot for ad-hoc discussions.
mounir: Yes, encourage ad-hoc discussions, minute the discussions publicly
markw: Right, just make sure that everyone is invited.
mounir: Yes, good point.
markw: Want to emphasize the need to have a triage task during the monthly call.
jer: Yes.
richard: I don't want to take a lot of time on that because you have only one hour.
jer: Thinking about non assigned issues?
markw: yes
jer: So triage could be about
assigning issues and not discussing issues in themselves.
... What about F2F meetings?
... We have a F2F at TPAC, is that enough?
mounir: Most of us go to FOMS. I don't know if we should use that as a second F2F opportunity.
jer: There's an opportunity to do a meeting.
mounir: It would be less exotic, but more convenient for people
markw: I think we should revisit in a couple of months whether we want to have another meeting. Good idea to coordinate with other event otherwise.
mounir: I'm sure 6 months after TPAC, a 1 day meeting is useful
<scribe> ACTION: jer and mounir to talk to FOMS organizers about interest to co-host a Media WG F2F
scott: Note Greg's proposal on GitHub
markw: Question on how formal we want to be. We need to understand what the consensus is.
jer: Actions and resolutions are
a good way to record the TLDR of meetings.
... Question is do you want to do it for everything?
mounir: For GitHub issues, I
think what works well is aggressively cc'ing people.
... I don't see any reason to change that, no one complained
about changes landed so far.
... Are there concrete concerns?
scottlow: I don't think that Greg is concerned about anything in particular, just to have a way to track resolutions.
chcunningham: Some GitHub issues
may be harder/longer than others
... The HDR one is tricky for instance. Substantial addition to
the spec.
... I don't feel strongly that we have to formalize the
process.
scottlow: For new people coming to the group, it may be difficult to understand what's going on.
markw: Good wake up call to have in the issue is to say "I think this is good now"
mounir: Good to point people at pull request.
markw: There may be different alternatives without PR and you want to understant what the consensus is. Proposed resolution seems good.
jer: True.
... I don't know if we can resolve that now.
<scribe> ACTION: jer and mounir to create an issue on the media-wg repo about the resolution process
PROPOSED RESOLUTION: The Media WG will have monthly calls, and encourage ad-hoc meetings on specific issues
RESOLUTION: The Media WG will have monthly calls, and encourage ad-hoc meetings on specific issues
[Side discussion on versioning for MSE and EME and repos]
jer: Deliverables. I am not an
expert on W3C process. We have a number of specs listed in the
charter of the Media WG.
... [reviewing the charter]
... Deliverables include Media Capabilities,
Picture-in-Picutre, Media Session, Media Playback Quality,
Autoplay policy detection, MSE, EME, all of them coming from
the WICG.
... In the future, we may adopt additional normative
deliverables, after incubation in another group.
cpn: One idea we're floating
around is to create a Media Community Group that could act as a
companion to this group for incubation.
... The thing we were talking about yesterday is whether we
could reorganize the IG into a more CG structure and use that
as a place to incubate.
jer: So that could be a potential
home for some of the incubations listed in the charter as
potential normative deliverable.
... Did we fail to include some deliverable?
mounir: One additional comment. For MSE, we may extend with additional features. For EME, only 4 additional features listed in the charter, we cannot do more.
jer: Can we do maintenance?
mounir: Yes.
chcunningham: I've been pushing a new attribute on the video element. More on HTML.
padenot: just cc everybody.
jer: Yes, we have a generic issue that by definition, some of our specs need to monkey patch HTML, We'll need to have joint session with HTML in the future.
markw: Do we have editors assigned for these deliverables?
mounir: Media Capabilities,
chcunningham and I are editors.
... Picture-in-Picture, François Beaufort and I are
editors
... Media Session has Becca Hughes and I as editors
... Media Playback Quality, technically I'm the editor, but I
just copied the text out of MSE.
... Autoplay policy has no one, the plan was to have Paul and
Becca.
... MSE has Matt Wolenetz and folks from Microsoft are no
longer there I think.
... EME has David Dorwin and Mark Watson. David won't continue,
but Joey Parrish will take over.
... Now, let's revisit the list and see what changes we should
be making
jer: Given your chairing responsibilities, should we replace you as editor with other?
mounir: Practically speaking, I
will only do spec reviews. If people are concerned with my
having two hats, no problem dropping from editing, but good to
have 2 editors to ensure some peer review mechanism.
... I am editing 0 spec by myself.
markw: We need to make sure that the other editor doesn't have the same expectation.
mounir: Sure. For Media Capabilities, chcunningham volunteers. Scott, do you want to have someone?
scottlow: Yes. We're interested. Greg would be the one.
PROPOSED RESOLUTION: Add Greg Whitworth as editor of Media Capabilities
RESOLUTION: Add Greg Whitworth as editor of Media Capabilities
mounir: For Picture-in-Picture, anyone interested to join?
jer: Apple is interested in implementing, but cannot take the task.
mounir: Then no change.
... Media Session, I heard that Mozilla is shipping it, any
interest to edit?
padenot: I need to check. I can't commit right now.
<scribe> ACTION: padenot to check whether someone from Mozilla can become editor of Media Session.
mounir: For Media Playback Quality, I think chcunningham might be willing to edit.
chcunningham: Yes, I started to look into it.
mounir: One issue is that the
HTML editors said that they would like to move the spec to
HTML.
... Anyone else on top of chcunningham?
jer: This is a small spec, it may be sufficient to have only one editor. And the editing role might be a pull request against HTML.
<scribe> ACTION: mounir to swap his name with chcunningham as editor of Media Playback Quality
mounir: I think merging with HTML depends on the content of the spec.
jer: That can be a discussion in
the spec.
... Autoplay policy, are we happy with Becca and Paul?
[group nods]
mounir: For MSE, Matt is happy to continue.
scottlow: I may be able to have someone on our side, I need to check.
GregFreedman: I'm expecting to edit either MSE or EME
mounir: So we'll have someone from Netflix, and maybe someone from Microsoft?
markw: Correct.
mounir: And for EME, same thing for Netflix?
markw: Yes.
mounir: My undersanding is that Joey might be taking over David.
scottlow: Same action for me to look into possible candidates.
<scribe> ACTION: markw and GregFreedman to figure out who edits which spec (MSE/EME) among themselves
<scribe> ACTION: scottlow to check for possible editors for MSE and EME
[going back to agenda bashing]
jer: Anything more for agenda bashing?
francois: We should figure out
which of these specs is ready to go to FPWD
... important from a patent policy perspective
... the call for exclusions covers anything that's in the
spec
mounir: i think media
capabilities is ready
... also picture in picture and media session
francois: the next point that triggers IPR commitment is CR
[side discussion on whether Media Capabilities is ready to go to FPWD, or whether HDR support needs to be baked in first]
<scribe> ACTION: jer and mounir to send CfC for publication as FPWD of Media Capabilities, PiP and Media Session the week after TPAC
See Encrypted Media Extensions repo
mounir: Before we start, in the agenda you had vNext and issues.
jer: Yes, it's about understanding what we want to see in the next version of the spec
mounir: OK, that's mostly for MSE, then, because we're constrained on EME.
jer: Yes.
... We can talk about Persitent Usage Record sessions, HDCP
detection, scheme capability detection, and existing session
finding.
markw: And maintenance issue
Joey_Parrish: [present context
for session finding]
... Discussion at FOMS was to separate the two issues
mounir: Joey, can you confirm that you want to take editor's rolve over David?
Joey_Parrish: That's
correct.
... One other thing I'd like is to get rid of robustness
strings.
... Matches what Microsoft has done with com.microsoft.hardware
/ com.microsoft.software
... There seems to be a better way to do that.
jer: I think that would be
considered under maintenance issues.
... Joey, can you file an issue on that?
mounir: Just to be explicit, we should be using the W3C repo for that. The WICG one should move away.
jer: The ability to push new keys on an existing session through some mechanism is captured in an issue, I think. The issue got punted for next release. Part of maintenance as well.
See fork of EME repo with persistent usage record
jer: It was in the spec and got
removed.
... That's a way to have a record of playback to be stored and
replayed cryptographically so that the application can verify
that the key is no longer in use.
markw: The language has been sitting in the WICG repo. The charter says it's our starting point, so we should upstream that.
mounir: Mark, are you driving
this?
... Any concern with moving that to the main spec?
jya: I would prefer not to implement it
markw: I don't think that's mandatory.
mounir: We are talking about persistent usage record
markw: It's an after-the-fact mechanism for fraud detection.
richard: there shouldn't be objections, that's in the charter.
mounir: OK, seems Mark or Greg should prepare a PR.
markw: Yes, we were actually
blocked on ReSpec issues. Now fixed. MSE and EME were built
with old versions of ReSpec.
... There may be work that we can do to switch to the latest
version.
Joey_Parrish: Does this mean we're sticking to ReSpec and not Bikeshed?
markw: For the spec that already
exist, I would stick to the same system.
... I won't do the transition at least.
jer: Joey, you can use Bikeshed for new specs, no problem with that.
mounir: HDCP detection and
encryption scheme capability detection are WICG repos.
... HDCP detection can fill a lot of time, given TAG feedback,
etc. so would move it to the end.
See Encrypted Media: Encryption Scheme Query proposal
jer: Apple Fairplay team would only support CBCS scheme and others CENC. In the meantime, the schemes have been added to the same common file format
markw: The problem still exists.
jer: AV1 is on CBCS exclusively?
markw: As far as AV1 is
concerned, it's a requirement that all encryption patterns get
supported.
... It's understood, I think, that devices that support AV1
need to support all of them.
... The portion you apply the pattern to has to be a multiple
of 16 bytes chunks.
... This is a lower level detail than the pattern, but there is
a question about whether the 16x bytes to which the pattern is
applied is aligned with the start or the end of the OBU.
... Whether we allow both approaches or require one or the
other is an open discussion.
jer: Do you think that it has implications for the spec?
markw: I hope not
jer: Apple is interested in clarifying behavior. Any objection to adding to the EME spec?
Joey_Parrish: Chrome supports both CBCS and CENC. Just no way for an application to query that yet.
Joey_Parrish: Prepared an
explainer.
... It just adds one encryption scheme. Minor change.
... Not entirely settled how default would work.
yongjun: CENC version 1 and
version 3 have different requirements. In the deployment of
Prime videos, we noticed problems.
... Version 1 requires 4 bytes, version 3 requires 5 bytes.
It's not compatible.
... Basically, we have two specs under ISO 23001-7:2016.
... It's nearly impossible to backfill everything in the same
implementation.
... It's always a mix.
jer: Just wondering whether we can mandate a specific version of CENC with the string.
chcunningham: Can we mandate both?
jer: We could, but won't help Amazon.
yongjun: The recommendation is to support everything.
Joey_Parrish: We will be developing a polyfill that allows to query the version supported.
jer: Is the idea that you will try and fail to detect the version?
Joey_Parrish: No, apps hardcode what devices support, the polyfill would abstract that in a library
yongjun: It's a mistake in the CENC specification not to have been backward compatible.
jer: Rather than making a more specific query to seek the version of CENC, be more specific in the section that details CENC that all sections of CENC must be supported.
yongjun: We don't have enough time to backfill everything. There's always a mixed catalog on the backend.
GregFreedman: It seems like if 90% of your catalog is CENC 3, it seems strange to be willing to support CENC 1
chcunningham: Was there a line somewhere that version 3 was a good thing to use when it got introduced, because it would be supported by EME?
yongjun: We always move to the
latest version.
... When we onboard new devices, we make sure that they support
all versions.
markw: The capability problem is
only in one direction there.
... The newer spec has more bytes in the clear.
... The older devices should not have an opinion provided that
at least the first 4 bytes are unencrypted.
... The problem is that there are newer devices that require
more bytes in the clear.
... In practice, that only applies to H.265 streams and not
H.264
yongjun: Microsoft Edge is like this, I think.
mounir: I think we are digressing
here.
... Maybe you should file an issue about this. Can you do
it?
yongjun: Sure.
... This is super important for Microsoft Edge.
mounir: Is there something in the change that people have concerns about?
Joey_Parrish: For backward
compatibility, if the encrypted schemes is not provided by the
application, the behavior is mostly unspecified. No
restriction. The way we would be able to detect support for the
field itself, when the application calls getConfiguration, that
should be in the configuration returned for media keys.
... The idea is that a polyfill could be written to make simple
calls to detect support for the field.
mounir: Everyone happy with that?
scottlow: Yes, as an implementer.
PROPOSED RESOLUTION: Joey_Parrish to prepare a PR to integrate encryption scheme proposal in EME (W3C repo)
RESOLUTION: Joey_Parrish to prepare a PR to integrate encryption scheme proposal in EME (W3C repo)
mounir: My understanding from discussion at FOMS is that we need to get back to the drawing board. Not done yet?
Joey_Parrish: Right. The feedback
I get from Jer is that the design was a bit overloaded as I
wanted to cover two cases.
... Example in Widevine, matching base on the content ID.
GregFreedman: I assume this is active session.
Joey_Parrish: Correct, active not
expired session. In the case of key rotation, expired sessions
would not show up.
... Main purpose of that is that, as application developer, I
don't have a deep understanding of sessions, which may lead to
duplicate sessions, which can be an issue with CDN
sessions.
jer: Seems reasonable as a future direction.
Joey_Parrish: Key IDs are not commonly available on all devices.
GregFreedman: Right. No objection.
Joey_Parrish: Would an API that would give you back key ids instead of sessions be preferrable? I'm open to that.
GregFreedman: I don't feel strongly. It may be preferrable for inspection purpose.
Joey_Parrish: From initialization
data to key IDs instead of from initialization data to
sessions. I think I went for the latter because I wanted to
cover key rotation, but agree based on feedback that it should
be kept separate.
... The only reason to have a session object is [missed]. Other
than that, it's not useful.
richard: There's a limited number of sessions that you can have from an hardware perspective. Is there a way to tell the difference between expired sessions and sessions that exceed the number of available sessions?
jer: It seems to me that there is a better way [missed]
GregFreedman: From a diagnostic standpoint, key id would be more useful. I might know that the session is active, but I do like the idea of having the key ID. Whether or not you have an API to check active sessions is less important from my perspective but it's ok.
PROPOSED RESOLUTION: Expose an API to expose key ID information from initialization data in order to find existing session
Joey_Parrish: You could use the API to do the mapping. I don't have a WICG repo for this yet, only a personal repo.
tidoust: You don't need to have a WICG repo.
jer: It can be a PR against the W3C repo
yongjun: Format of key ID is different. CDM specific.
jer: Yes, ArrayBuffer in the spec.
markw: The Key ID format is defined by the container specification (e.g. CENC). 16 byte string
[discussion on key ID format]
yongjun: We should not infuse an additional format here, that's my point.
PROPOSED RESOLUTION: Expose an API to expose key ID information from initialization data in order to find existing session
RESOLUTION: Expose an API to expose key ID information from initialization data in order to find existing sessions
See Encrypted Media: HDCP Policy Check proposal and TAG Feedback
jer: Content may not be licensed
for playback given HDCP support or not.
... The proposal is to have an API to allow early detection of
which HDCP levels are supported.
mounir: Joey_Parrish wrote a spec for that.
Joey_Parrish: It's a mapping on MediaKeys that returns the version of HDCP. Privacy review suggests concern for fingerprinting. Mitigation is that it is user activated as part of MediaKeys.
jer: 9 different versions of HDCP
that can be supported. Value depends on the connected display,
so danger of exposing more unique info per user.
... It's already possible to gather that information through
trial and error.
... And small number of Web sites that can use this since keys
need to be got first.
markw: Granularity requirement is
basically HD, 4K, perhaps to be refined to add a third
one.
... Slower time to start playback if you don't have such an
early capability detection.
... User consent is for EME at all.
yongjun: Dynamic detection is needed when user plugs another display.
jer: Two things can happen:
unplug a device that supported HDCP requirement, plug a new one
that supports HDCP requirement.
... One solution would be to poll this API to detect
changes.
mounir: Media capabilities has
this idea of adding a change event on the window object, and
you could perhaps use this.
... I believe Mozilla implemented, do you have any
feedback?
jya: I don't know.
chcunningham: CDM that enforces HDCP during the playback. Multiple screens, Widevine policy is I believe to report the lowest protection value when you have a highly-protected display and a low-protected display.
jer: Thing happens, because we
receive user reports.
... We take the same approach.
... The user doesn't understand that a display that used to
work no longer work after plugging another one.
markw: The general problem of reacting to user changes is not one that we see as a priority. We don't object to it, but it's not straightforward to make it a good user experience.
jer: One thing that someone like
Brave or privacy-mode might do is to try to return a single
value for HDCP support.
... I would hate to get in a situation where you couldn't
implement a privacy mitigation without breaking existing
sites.
mounir: Mark, you said we could
get down to 2-3 values.
... Is that possible?
Joey_Parrish: Probably, except it
probably depends on the specific contract requirements. Studios
may update from 2.2 to 2.3, and this value would need to
reflect that.
... If a browser is faking this value for the benefit of
privacy. It's effectively equivalent with fetching the value in
advance and the user plugging in a display in between.
... Any application that cannot handle that is going to break
in corner cases in any case.
markw: Right, I wouldn't say it's
equivalent.
... Difference between a corner case and a case that always
happen.
jer: The fingerprinting of media
capabilities is significantly less for Apple because we run in
a short number of hardware. But that's not the case for
others.
... The HDCP policy is different because it can change
dynamically and dramatically at any time.
Joey_Parrish: We could add an "unknown" value that would allow not to lie.
jer: Returning "unknown" is already a signal compared to lying.
Joey_Parrish: Having implemented
something similar in ShakaPlayer, you must start with SD and
ramp up with keys you're going to receive.
... With lying, it wouldn't mean that you'd be stuck forever,
we'd just do what we're already doing
mounir: I assume EME is HTTPS only, only available on top. No reason for cross-origin iframes should use EME. EME is async API by default, so you can have a prompt. This call itself is aync, so you could have a prompt. Technically, everyone can get this information with Widevine, it's just a bit more work.
jer: Not the case for everyone, though
mounir: Right, but fingerprinting
is not high risk.
... I would add in the spec a note to explain what
implementations could do if they don't want to increase the
fingerprinting surface.
... And we'd just get back to what we're doing that.
markw: Inform sites that rejecting this API is something that can always happen.
jer: Having the ability to reject
this promise is good. Links to Privacy Budget discussion as
well. The call may succeed at one time and fail at other
times.
... Language like that would be appropriate.
Joey_Parrish: Adding some language for fingerprinting mitigation is ok.
GregFreedman: Some HDCP versions are unsecure. 2.0 2.1, anything below 1.4, we'll never query for.
chcunningham: The privacy gains by doing the collapses is minimal.
markw: You could bucket those, no problem on our side.
jer: Any normative reference that we could have not to have to update the list?
mounir: We'd need a registry.
[discussion on registries at W3C, possible process updates, etc.]
<scribe> ACTION: Joey_Parrish to add language to the privacy path
GregFreedman: On Windows, I was told that the media foundation does not recognize 1.4 and reports 1.0
<scribe> ACTION: Joey_Parrish to look into reducing the number of HDCP versions we expose to collapse to those actually used
Joey_Parrish: Fine with the action but I will need some guidance on what values are actually necessary
mounir: About the TAG issue, should we come back with a resolution?
jer: If we enact the actions, then I think we can get back to the TAG for a resolution on these issues.
mounir: Right.
[short discussion on maintenance issues]
Joey_Parrish: The two things mentioned earlier are key rotate and my suggest to kill robustness.
markw: There are a bunch of issues on the repo. Some of which may need to be dropped, or acted upon.
jer: Seems a good thing to do in
a call.
... About key rotation, internal clients of EME at Apple had
issues with adding new keys.
... Why generateKeyRequest cannot be used multiple times?
... There seems to be hardware limitations.
... We closed that early in the first version for lack of
time.
... I'll just re-raise the issue on GitHub, and we'll cover
that later on.
... About removing robustness, any significant pushback?
[none heard]
Joey_Parrish: Issues could reasonably be addressed by introducing things such as .hardware or .software and other properties.
GregFreedman: I know we use robustness in practice, so need time to transition.
Joey_Parrish: Yes, long time starting with intent to deprecate.
mounir: Did you file an issue about that?
Joey_Parrish: Not yet, will do that.
mounir: We want to make sure that we can notify everyone else.
See [Proposal] hint attribute on HTMLMediaElement to configure rendering latency thread on WICG's Discourse
jer: Proposal from chcunningham
to allow client to signal to a user agent that they would
prefer a different style of rendering, e.g. to signal not to
use a jitter buffer.
... Game streaming mentioned as a use case.
... Currently, Chrome has a heuristic and would like to make it
explicit
chcunningham: Correct, and the heuristic is often wrong
jer: The right place for incubation is a CG, but there is overlap of interest with Media WG participants.
chcunningham: [going through the discourse post]
alex: Are we talking about the rendering buffer or the jitter buffer?
chcunningham: The rendering
buffer.
... Proposal is to add an attribute that would cut it down to 1
frame.
alex: Is there a separate proposal for the jitter buffer?
jer: I used the wrong term, I don't know anything about jitter buffer.
alex: Jitter buffer is used in WebRTC.
chcunningham: Through discussion
with Jer, he thought the API was too low-level. Taking leverage
of mediatrack hints, proposal is now to rather take a
declarative approach.
... I did also talk to Harald briefly yesterday, he didn't
think this was a crazy idea.
... He thought we might even use their enums and was open to
adjust it, e.g. to support real-time use cases.
jer: Also, I don't want to impose requirements for this, because I may not be able to implement a 1 frame buffer. Hence the hint approach.
jya: Moving back to MSE, there
would be significant change of behavior, no stop and gap,
etc.
... Would that apply to MSE? It's only when you attach MSE to a
video element that you'd have access to it.
... We would have a different thing in MSE?
chcunningham: Yes.
jer: You can imagine having gaps keeping and low-latency rendering
jya: Could we assume than rather
that being associated to the media element, it should be
associated with the source?
... Just trying to have something that could have a smooth
continuation on MSE, rather than having a second proposal on
MSE.
jer: Right now, it's written as an IDL change. For attributes, you'd have to add the attribute to each source.
jya: Yes, even more flexibility!
mounir: Everyone talking about
hints as if it was a done deal. I'm not a huge fan of it. Much
harder for people to understand what will happen. Developers
will reverse-engineer to understand what's going on behind the
scenes.
... What people care about is the behavior, which user agents
can change with an hint approach.
jer: And I disagree because I don't want to be required to implement a specific behavior.
mounir: The benefit of this is
that we can use this for a lot of stuff, e.g. Audio
focus.
... However, we will end up with a hint that will only be used
for buffering.
... So we'd be stuck with an expected behavior.
markw: You want things to be well-defined IIUC.
mounir: Yes.
chcunningham: Even in Chrome, we make no guarantee.
markw: There may be use cases where you want to specify 2 content hints.
alex: Any other optimizations for these hints?
chcunningham: Only buffering planned.
jer: I can imagine that being used for other things.
alex: If you specify by optimization, risk of ending with 12 different parameters.
mounir: I believe that's
OK.
... If we start changing behavior, people will have
expectations.
jer: Most of media playback is open-ended. It's a kind of a pain but allows for flexibility.
jya: But then you end up having customers that expect a certain behavior when you have a strong player.
[revisiting the agenda to find a slot to continue the discussion]
Jer: This is pretty far along,
there are some issues needing discussion
... The biggest ones are: queries for HDR support and spatial
audio support
Jer: Issues #118 and #119 both allow
applications to determine the ability to decode and display HDR
content
... The API shape has changed
... Allows the UA to understand whether formats can be
displayed and decoded, given color spaces, transfer fucntions,
metadata values
... There's also an API we're considering adding to Screen, to
determine whether it's able to handle HDR
... Is this sufficient for clients and implementers? Is the API
surface acceptable to expose to the web
yongjun: Is this HDR, HDR10, HLG, etc.?
Jer: I avoided encoded specific
product names into the API
... e.g., Dolby Vision
... HLG has a specific transfer function, that's being proposed
to be added
... Does the agent accept a particular transfer function?
MarkW: I think it's more the
ability to decode, if a query responds positively to the query,
you should know it will display correctly, although may not be
the best output you could expect
... For one stream, can I play this? For CSS, selecting between
multiple streams
Yongjun: It's not just HDR capability alone, also frame rate, e.g., HDR at 30 fps not 60 fps
Jer: Media Capabilities already has frame rate
Yongjun: So need to be able to signal these combinations
MarkW: The API takes a specific config, you get a yes/no answer
Jongjun: It's more like a
query
... What about composition? An SDR and an HDR stream, can you
show at the same, e.g, in a picture in picture window?
Jer: That's a problem in the API, regardless of HDR, e.g., can I offload from CPU to GPU?
Jongjun: How to composite SDR and HDR content?
mounir: a design principle of this API from the beginning is that we only answer the question for one video playing regardless of what's happening
MarkW: It's a very difficult problem to cover this in general. You have limited decoder resources being shared between streams
Eric: If you come up with a solution, the answer can change dynamically, there can be a race condition
MarkW: If you want to render multiple streams, you have to constrain things
Yongjun: so what's the scope of Media Capabilities?
Jer: VideoConfiguration has 3
additional parameters: hdrMetadataType, colorGamut,
transferFunction
... should allow all HDR types currently in existence
Jongjun: Also bit depth?
chcunningham: When you have higher bit depth profiles of VP9, you're implicitly saying you support those bit depths, and the rendering
Jongyun: We assume HDR is 10 bit, but also works if 8 bit. It's both the decoding and rendering process, so not sure if 8 bit will be supported across devices
MarkW: The codec profiles are different between 10 bit and 8 bit
Yongjun: Screen and decoder on same device, but what about HDMI, you need separate queries
chcunningham: The definition of rendering is ambiguous, so we recognise applying the transfer function, not a decoding step, also not a display function. The piece between the screen and decoding we call "rendering"
MarkW: There are two kinds of
hardware setups. On PCs and laptops, decode to linear half
float, and then convert to anything on the way out
... If connection to an old monitor on HDMI, it'll be rendered,
so the answer from the API is yes
... If that's the only copy of the content I have, I want it to
display
... If we have both HDR and SDR, it will say yes to both, but
we can check the HDR flag to select which to show
... This API only addresses what can and can't play. Use that
to filter down
Richard: Why does the "maybe" thing happen?
Jer: This is because you could
under-specify the MIME type
... We're moving away from that
MarkW: One tells you whether you could, and one is whether you should
chcunningham: If you have multiple screens, the Screen API gives dimensions, and this changes when you drag windows between screens. Seemed a natural place to expose whether the screen supports HDR
Jongjun: Downgrade from HDR to SDR, we don't use device conversion
Matt: Is canPlayType only used to select HDR or SDR
Jer: No, could also be about available bandwidth
chcunningham: The API tells you what we
can do, and how well we can do it.
... So the UA allows the webapp to make the decision
Jer: The API returns if possible, smooth, and if power efficient
Mark: Other dimensions include
resolution and color gamut
... This API tells you if a resolution can be rendered, could
involve downscaling
... Then you can check the Screen API to select based on
resolution
Jer: Could be polyfilled by an API
Mark: Also color gamut,an SDR in
709, and 2020, query the Screen. But we don't have this, as
wide gamut capability generally comes along with HDR
capability
... Could do it, but doesn't make sense in practice
Jer: Color gamut, should we
specfy rec2020, small difference from P3, but it's there
... I believe with these parameters, can distinguish Dolby
Vision, HDR10, HDR10+, HLG (for now...)
Jongjun: For one title, we have HDR10, 10+, and Dolby Vision
MarkW: Similar, we also have multiple variants
Jongjun: If someone only has one copy, should we use "maybe"?
chcunningham: We should add some examples
<scribe> ACTION: chcunningham to add examples to the Media Capabilities spec
<gregwhitworth> Opened issue #132 for examples
Jer: There are two PRs
Jer: VideoDisplayConfiguration is
newly added, you can pass in width, height, hasHDR, can check
if this is supported by the display
... This API is more limited, to reduce fingerprinting impact.
We hope it's enough for content providers to make decisions
about filtering HDR and non-HDR streams
<gregwhitworth> it is only MQ
<gregwhitworth> matchMedia will work
Jer: Bit depth and color depth for the display is already exposed in CSS
MarkW: Not sure bitdepth on
Screen is that useful. it's difficult to pin down. Most TVs,
the actual panel is still 8 bits
... Asking whether its worthwhile sending 10 bits to the
display is difficult
Jer: Media Queries were used in webkit to ask if display is capable of displaying wide color gamut images
<jya> See the definition of the Screen interface in CSSOM View Module
chcunningham: I agree with MarkW, I wouldn't want bitdepth
Jer: Are width and height just there for embedded players with a separate video layer?
chcunningham: Yes
Greg: Don't want to stream 4K unless there are 4K device pixels
chcunningham: Cast has a proprietary API where you can get the display width and height
Mounir: We have feedback that this is generally useful
chcunningham: As written, it appears like implementations have to support all, but that's not the intent
<mounir> https://github.com/w3c/media-capabilities/issues/118#issuecomment-529878912
Mounir: So in general we're happy with the PR, but there's some wording changes needed
GregW: Can we clarify what's outstanding?
<gregwhitworth> Proposed Resolution: Remove statements that define what the sRGB, etc and allow that to be defined by UA
chcunningham: I'd want to take one last
pass through the PR, need a couple of days for that
... Consensus is what's in the PR is good, feedback on
semantics of the enum values
See PR #123 - Add support for Spatial Audio rendering capability queries
chcunningham: We landed the PR, is there any feedback on that?
Mounir: Re decoding vs rendering, the API can say no to rendering even if decoding is possible
Jer: There's no equivalent to Screen for audio, so no other place for those queries to go
Paul: Web Audio didn't take this
into consideration. We are addressing this, taking into account
modern technologies
... There's a new repo so you can add issues to request
stuff
See Audiobook Profile for Publication Manifest and Publication Manifest
Wendy: I'm from a digital bookseller, and co-chair and editor from the Publication working group
Wendy: on TAG advice, I'm here to tell you about our spec
Wendy: We have two specs:
Publication Manifest, and Audiobook profile
... They're closely relayed. It's a general manifest format for
publications on the web
... We have a profile for specific use cases
... We currently have one profile, for audiobooks
... We keep as much audio content in mind, thinking of
podcasts
... Want to avoid overlap with other work, and be open about
what we're doing
... Manifest format for B2B or B2C. A couple of use cases:
listening to an audio book, a11y, and portability
... Make it flexible for users. Using JSON-LD and schema.org
for metadata. Focus is on identification and information to
user or distributor
... Audiobooks is siloed by retailers, want to break this down
with an open format
... Identifying metadata, reading order. There's a CG working
on synchronised media
... An audio file with other media, such as text or video, as a
more accessible experience: listen and read, or sign
language
... Spec allows for a table of contents, for detailed
navigation data for the user
... It's also for podcasts, want to cover their use cases
... I want to sure our spec works with your APIs, or can we
help make your work easier?
Jer: Is this about the audiobooks profile or the publishing spec in general?
Wendy: Audiobooks in particular, this has strict rules, e.g., can contain only audio files
Jer: Overlap with ePub?
Wendy: We work closely with the ePub3 CG, can be the future of digital publishing post-ePub
chcunningham: We should make sure our
format strings are the same
... Is audio/vnd-wav vendor specific?
<mounir> See MIME types section in Media Capabilities
Mounir: Now we require codecs in the mime type for everything
Wendy: We decided not to restrict
the allowed audio formats, wanted to be future proof
... We can reference this for the right formats
Jer: When you have a client different capabilities, how to select between different formats?
Wendy: Could have a fallback or
high quality or low quality, decided not to do that, will do in
a best practice doc. Approach would be to have separate
manifests
... We have a list of 2.0 things
chcunningham: Let's have the same language for specifying which codecs to use
Mounir: Agree, helps make using APIs together easier
Paul: The temporal fragment - look at the Media Fragments URI spec
Francois: Do you have specific requirements for synchronisation. This is being discussed in the M&E IG
Wendy: That's in a CG, Marisa DeMeglio is an editor
Wendy: That spec is separate, want it to be generally usable
cpn: we talked with
Marisa about this in the past
... one of the reasons why TAG is interested is because of the general
packaging effort
... there is an
effort about media containers -- having a MP4 file synchronised
with the presentation
... there is a draft
MPEG standard nearly finalised and they are now working with
W3c -- that is somewhat controversial
... Dave Singer
(Apple) has been interested in having a workshop around this
topic
Wendy: Dave Singer
has been working with us around packaging for the publishing
industry as they are not as technical
... we talked about
MPEG and HEIF, he gave a presentation and it looked
promising
... for this
industry, it wasn't a clear cut process
... we are still
working on this but this spec will unlikely have anything
around it
... we believe web
packaging is promising for publishing
mfoltzgoogle: in
general, when there is media metadata format, Media Session
should be looked at
... we should look
at mapping them
Francois: The alignment could go both ways. This is based on schema.org, should we align Media Session to that?
Wendy: We use the creators list, author and "read by", rather than narrator
Mounir: In Chrome, we've been
looking at schema.org to see if we can populate the media
session
... Some websites don't use media session, but they do have
schema.org
... In future, Media Session could suggest default values based
on schema.org
... Today, most browsers use file icon, page title, for artist
title, so could use this to provide better information
Richard: Is there overlap regarding security and privacy
Wendy: We're ready for CR, so need to get security and privacy review. As a manifest format, there aren't so many security and privacy concerns
MarkF: Another relevant use case is background fetch for offline use. Can you get the file size from the manifest?
Wendy: The reading order can be a
series of URLs. Depends on UA implementation, whether you
pre-fetch or not. We don't have instructions on that
... In that case you'd download everything
MarkF: Is there enough information in the manifest to be able to check against storage quotas?
Wendy: We could include bitrate,
so that with duration gives a hint. We work with reading
systems as UA rather than browsers
... Can request the first bytes from the audio file to
calculate the size
... We use zip for compression
Mounir: If you have multiple files, bitrate is useful, adjust for quality, as with DASH
jer: do we just want to do triaging of these bugs?
Chris: I think we may want to look at some put issues
[talking about some issues we could look at]
jer: should we walk through them one at a time?
Chris: we should look at the subset I mentioned
jer: let's start with transition() ergonomics #102
<tidoust> Issue #102 Discuss transition() ergonomics
jer: it's about transitioning from one codec to the next and the capability of doing so
Chris: it was done before the MSE
API change
... the explainer had a method called query() at the time and
we had this idea of an API called transition()
... we discussed very complicated solutions like if you can do
A and B, can you do A => B or B => A
... after talking to Joey Parrish, he said it was over
engineered and he wanted all or nothing as a web author
... websites want to know if you can A <=> B or
nothing
... so it seems artificial to say that A => B is doable so
is B => C but not C => A
... the proposition I came up with is to have a boolean in the
result that says whether the transitions from this codec are
allowed
... which calls out cases where a UA cannot transition out of a
specific codec
markw: what is the "pipeline" you are describing in the issue?
Chris: it's the clear pipeline
for example
... software and hardware CDM may have different transition
support
markw: in a hardware CDM context, could some transitions be supported but not others?
Chris: the API will answer only if one cannot transtion from or to a given codec
Greg: can you give an example?
Chris: [didn't hear]
jya: i'm puzzled about the use
case
... even for the hardware codecs, could the UA deal with
tearing down the pipeline and creating another one?
Chris: they may do that but still websites would need to know
jya: i can't find a case where we couldn't transfer somehow
Chris: it is a concerned for
hardware CDM and televisions
... something we heard for example is whether non-smooth
transitions were okay and we think this shouldn't be considered
a transition
jya: you may lose a few frames but you should always be able to transition
Chris: this is implying an architecture
Greg: the use case I'm
envisioning would be to use AV1 for low resolution and VP9 for
high resolutions
... and maybe AV1 is hardware decoded
Chris: if you CDM lives in the chip, it's possible that you can't extract the keys from it and can't transition
markw: is seamless implicit for transitions here?
Chrisyes
markw: what about non-codec changes such as colour, framerate, etc.
Chris: it should be fine as far
as I know, things like 30fps to 60fps transition should always
work
... in Chrome if the decodec can do 60fps, it can implicitly do
30fps, is that not always true?
markw: correct, in some cases transitioning when HDMI framerate matches the video framerate can take several seconds
jya: what about audio transition? [mentions a case where there can be several seconds of silence when transitioning]
Chris: are these scenarios around HDMI common enough to take them into account?
markw: [mentions taking into account tv, and not only browsers]
Chris: Some progress here. I have some call next week with TV makers
jer: happy with the direction the
API is going -- easier to implement and use
... switching to issue #111 and feature detection
<tidoust> Issue #111 - Changes to MediaConfigurations cannot be feature detected in older UAs
jer: one way is to return the
dictionary as it was understood
... for HDR, for example, one could check whether the
HDR-related information are returned to check if it was
understood
Chris: supporting this and was waiting to see if we wanted to have a sequence of configuration as input before going forward
jer: no one else?
... nothing, then we can go to the next issue
Chris: Mark's framerate issue can be brought up
jer: issue #96 and #95
<tidoust> Issue #96 Revert the "DOMString framerate" change from issue #12
jer: previously the framerate was
a double and Mark mentioned that some configuration couldn't be
represented as a double
... so I asked for this change to be reverted
... because I didn't implement the string parsing
... so now we are stuck at how we can represent a rationale
number inside the web platform
mounir: I talked to some folks at Google working on ECMAScript and they said it would get support but would likely take a long time
markw: i didn't want to have a string, I would be fine with something like a double or enum
Chris: doing an enum would be
better than string parsing
... but the original issue mentioning a concern about people
mixing representations using double or rationale
representation
... it doesn't matter to our implementation so it really is
just about the ergonomics of the API
... and have it less error prone
mounir: mark, is this something that you cane about because of API design or Netflix usage?
padenot: same for Web Audio, we
made bad representation mistakes in the past and regreted
them
... issue tracker is full of issues that change the number
slightly
markw: facebook has a number as a common denominator for framerate and audio samples called [flix]
jer: we should work on getting
time values in media land appropriately
... it sounds like web audio has something
padenot: yes, we have a solution [couldn't hear]
<padenot> web audio allows keeping track of the number of samples as an integer, and lets you divide by the sample rate if needed
markw: maybe a helper could help here so the browser can tell if two values are the same to it
mounir: it would really just apply to media capabilities
jer: should we go with the enum?
Chris: I would point out that we shouldn't do enum only
jer: idea here is to do string + enum
mounir: I don't think we should
do an enum. Mark's comment is "Let's represent framerate
correctly on the Web" and doing an enum is just a hack. We'd
need to complete the enum over time.
... I looked a bit into using an interface. Main issue is
operational operations looking ugly. You can do them, but
that's ugly. A Rational interface could be doable, but we
should liaise with Ecmascript.
Chris: Would it be fine not to define mathematical operations?
jer: But people will want to to them to times the framerate
Chris: Will happen with the string as well
jer: Yes, but totally impossible
to do with a string without first parsing the string
... The ideal scenario is to have a rational.
markw: it sounds that people
don't really want enum
... maybe if we just specify as a double but we are very clear
about what that number means
... where it's clear that this number is approximate and not a
precise representation of the framerate
... maybe that takes away the problem
Chris: I see this as a
performance thing: the Apple framerate restriction for
HDR
... 30 and 60 are round numbers but maybe the cut off will be
30.1 someday
markw: is this a performance
issue or something else? could you play a 60fps content at half
speed?
... there is also a lot of restrictions for example with 4k,
you have framerate restrictions for a given profile
RESOLUTION: we can revert the framerate attribute back to a double
Chris: we have the same issue
with bitrate
... this is a performance indicator, not something
precise
... and it is recommended to pass the highest bitrate instead
of the average
markw: maybe the name should be changed for "renderingFramerate" to make this clearer
Chris: let's talk about the
enconding info issue
... it's #126
... Chrome implemented part of the API
<tidoust> Issue #126 Cleanup encodingInfo()
Chris: the proposal came from
someone who works on this
... RTC folks may not implement it
jya: we did implement and launch
it
... for both WebRTC and MediaRecorder
Chris: one issue is that this area do not use the same codec strings as us
jya: we even added web platform
tests
... it's a big improvement compared to what canPlayType is for
media recorder
Chris: for WebRTC it's a bit
strange
... there is ORTC performance API coming
jya: the problem with WebRTC,
when you query it can be irrelevant sometimes because sometimes
you can only do one codec
... in addition, with the negociation, I do not see a big
use
... with MediaRecorder, I believe there is a better use
case
Chris: that's true, though it's
unfortunate that they use the h264 codec string
... a use case I've heard is to avoid re-opening the camera
when downscaling
jer: is the idea that some of this stuff move to WebRTC?
Chris: I'm trying to figure this out
jya: I thought that the idea was to avoid this
See the WebCodecs repo
peter: we've had breakout and
wicg sessions, lots of comments and issues discussion...
... webcodecs relevance to media wg
... media capabilities
... mse vnext ...
... not sure what the overlap will be yet, but maybe some
integration
... as webcodecs matures, may become part of this wg...
... if we want injectable codecs, may want wg reviwe of
shape...
jer: will you also be able to control specific properties of the decoder (configurability of the codecs)
peter: yes, see github issue for
some outline. e.g. some discussion of b frames
... looking for input
mounir: webrtc (harold) looking into injectible codecs. is that tied to this? are we thinking about wasm codecs?
peter: webrtc interested, so far no one looking into it for video stack outside webrtc
jer: vlc would wasm all of
vlc
... already have
peter: power users free to wasm everything
mounir: stack is thinking about
this
... chromeos supports some obscure codecs, would be nice to
make them injectable for security pov
... in chrome stack webrtc and video are two different
pipelines
paul: concern is that webrtc is
modularized.
... will folks be able to do a hybrid solution with web codecs
+ peer connection?
... or will web codecs mean they entirely roll their own webrtc
impl
peter: one idea floated is to
have codec registry
... api for injecting codec would then make that codec visible
to webrtc and video alike
steve: the challenge is each api
that would embed a custom codec would need additional
info
... mse needs to containerize
... webrtc needs codec signalling
... not clear if registry is feasible
peter: alternative is for each API to have its own injection hoook
paul: playback of video could use a media stream from web codecs
mounir: is the a difference how you decode depending on where the video comes from?
jer: yes, same decoders in the end, but very different routes to get there
chcunningham: webrtc may have some duplicate sw decoders - not sure what they take from the platform
mfoltz: for wasm use case, is that a way to support codecs that lack universal x-browser support?
peter: yes, ex: if browser wants
to drop support for an old codec
... also works to add new codecs
mfoltz: what does it look like for player to choose between web codec and wasm?
peter: 2 ways. vlc way: all wasm. or you could inject the codec into video stack (if api had a hook to do so)
mfoltz: i don't see that there's
an extension point to do that yet
... what would it look like for mse to use webcodecs?
peter: designing on fly, something on html element like an attribute that represents the encoder or decoder as a stream
mfoltz: sounds like area for exploration. interesting to have player libraries that could do either way
petere: might instead have a codec object that could handle switching between codecs
richard: considered a way to use for still image
mwatson: some video codecs have metadata embedded in the stream (e.g. hdr sei msgs) - how does webcodecs deal ?
peter: could add this to the output. currently have just is_key_frame, could add other properties
mwatson: so for given codec you define what these will be so receiver knows how to interpret?
peter: could be ways to put codec specific things in there. k-v pairs
mwatson: if I want to feed the
output stream to video element, may need some extra
details
... had a question yesterday about image discussion - what
codecs would be supported for images?
peter: its in webcodecs explainer
mounir: encoding/decoding APIs
are indeed media wg purvue
... may be able to do this even without re-charter
... discussion about creating a media cg as well
... whenever you believe you have something stable enough could
move to wg
francois: agree, we don't need to
change the charter
... just need support of the group
eric: skeptical we can define an interface that allows javascript/wasm to create frames that can be passed down to the platform pipeline
(encoded frames that can be decoded by platform decoder)
eric: particularly for a cross platform way
jya: we've made a demuxer's /
encoders / etc inside the platform that work in a interoperable
way
... so web codecs could work
eric: i'd love to be wrong. support investigation
peter: thanks, all I got :)
jya: you support mpeg ts, but
chrome doesn't, so you find a lot of libraries that fill the
gap in js
... twitch generates mp4s with a single frame
... could simply use webcodecs
jer: maybe an api to control decoder configuration, it should be web codec
chris, we might want something simple here, in advance of web codec that will take some time
jer: a specified behaviour for
this would be rather hard
... should this affect every webcodec as well ?
chris: in webcodec, you're in charge of everything, right ?
steve: yes, indeed
padenot: also yes, you do eveything yourself
chris: hence, this would not have a use in the web codec world
jer: the issue here is the shape
of the API
... for some shape of the api, I can't implement some part of
it in safari, so we would wait for wait codec
... so a higher level API would be preferred, for us
... I'd like an API that would be useful for every UA, but if
not, we can't implement
chris: for someone like twitch, it makes no difference, they are not asking for this
mounir: how exactly having hints help?
jer: it can affect the number of UA that can implement
mounir: how is the videocontenthint something you can do, but the low-latency api something you can't do
jer: for hints, there can affect
some other bits, for example pitch shifting technique, the type
of audio streams, etc.
... it's been historically a problem, especially on mobile, but
this will be covered by the audio focus spec
... we've tried to have a way to be able to slot in new
behaviour
jya: could we focus on doing an API that would be useful for the user instead ?
jer: no matter what the shape of
the API, some of it we won't be able to implement
... but we'd like to do something that can do more that just
changing the buffering strategy
chris: asks jya about opinion
jya: the hint that says that the
video should be low latency is obviously very useful
... we would be able to pass, on a platform level, different
settings to the decoders, for example on Linux and Windows, it
has massive impact for the end user
... I'm still also looking at the MSE side of things, and I'd
like something that covers it both
... it always relate to the type of input you have, after
all
... maybe you have gaps, but you also want to be on time
... I do see value in having something in the area, but I don't
know whether or not it's better to have this on the media
element or on the source
... it's always been the source that drives the rest, in the
end
mounir: I don't understand why this is a problem, to have it on the element
jya: it's also about the ease of
implementation
... the source is created first
... so it would be easier to set the attribute on the
source
... it would allow to have things setup slightly earlier
jer: can you append to a source buffer before it's attache?
jya: no
... we can't probably do either way, but on a implementation
perspective, it's easier. it's also more logical to put it on
the source
mounir: on the other hand that means it can only be done with MSE
jya: as a matter of fact, it's
only with MSE that you can achieve that
... but maybe even with a plain URL src =
... but it also depends on the provider of the media, depends
on the presence of b-frames
... if there is b-frames, then it makes sense to try to do
low-latency, otherwise it's useless
... a side effect of that hint, would be to see frames out of
order
chris: on the source, it would
work just as well maybe, but the way I reason about it, is that
MSE is about demuxing
... but you're right it's a consequence of the source, you have
the source, the demuxer, the decoder
... there are multiple things to do here, and they are not
inherently connected
jya: indeed we also keep 3
hardware frames, but 10 software frames, there is implication
everywhere
... it doesn't matter where it goes, but I'd like to have MSE
brought into the picture
mounir: we have a session tomorrow morning
jer: we control MSE, it's easier to do
mounir: let's not do that because
it's easy
... and it would be approved on HTML anyways
chris: the use case I have in
case for cloud gaming, there is no possibility of gaps, they
don't have gaps/underruns
... for twitch, big MSE users, they won't have gaps, so this
won't care about gaps skipping
jya: their gaps are too small for
them to cause problem
... a few microseconds here and there
padenot: because of rounding and round-tripping between different domain
chris: we con't have consensus, let's continue discussins after MSE
jer: I thought in all this "source" was the source element
jya, no, any source really
scribe: I can't see that use outside of MSE really, it's so natural to see the link between the two
francois: you mention MSE between a temp measure before web codecs
jya: we would still use MSE
jer: how does demuxing work in web codec
padenot: a magic "demux()" function
chris: we're just trying to give more control
mounir: we don't know what the future is, it's best to have something more general
jer: it would make things worse: less resilience, but you don't see any latency benefit because you don't have control over the rest
mounir: I agree, but technically it could be done, and why not put in on the element
jer: if it's useless, let's not do it
chris: it's not about this possibility for src=, it was more natural semantically to put it on the element
mounir: I reckon that leaking this info across object doesn't seem good
chris: let's ask microsoft
mounir: do we have at least a resolution to not do the content type hint
jer: happy to be the odd one out
jya: I don't care where it goes as long as we can get it done
jer: this is going to be
temporary, folks that will use this will jump to web codec as
soon as it's realease and the need for it will disappear
... in a world where there are web codec, this will not be used
at all
chris: this is at least partially
true I think
... stadia would definitely prefer to use web codec if/when
available
... let's bring this up again during the MSE discussion
scott: I don't have a strong opinion right now
RESOLUTION: using a hint specific to low latency instead of content hint