Media WG F2F - Day 1/2

19 Sep 2019


Alexandre Gouaillard (CoSMo Software)
Chris Cunningham (Google)
Chris Needham (BBC)
Eric Carlson (Apple)
Francois Daoust (W3C)
Greg Freedman (Netflix)
Jean-Yves Avenard (Mozilla)
Jer Noble (Apple)
John Riviello (Comcast)
Mark Foltz (Google)
Mark Watson (Netflix)
Michael Rosenzweig (Intel)
Mounir Lamouri (Google)
Narifumi Iwamoto (Intel)
Paul Adenot (Mozilla)
Richard Winterton (Intel)
Rijubrata Bhaumik (Intel)
Scott Low (Microsoft)
Soju Tanaka (ACCESS)
Stephan Steglich (Fraunhofer FOKUS)
Wendy Reid (Rakuten)
Yajun Chen (China Mobile)
Yuki Kawamura (NHK)
Yongjun Wu (Amazon)
Youngsun Ryu (Samsung)
Greg Whitworth (Microsoft)
Joey Parrish (Google)
Vi Nguyen (Microsoft)
Jer, Mounir
tidoust, mounir, cpn, chcunningham, mfoltzgoogle


Agenda bashing

jer: Welcome. Looking at the agenda, any change requested? Couple of changes in the last couple of days. Low-latency video signaling was added today.

mounir: I added WebCodecs, audiobooks, and I moved some things around for external folks to be able to join relevant sessions.

WG boot up

jer: This is the first meeting of the Media WG. We don't really have a process or norms setup. We didn't want to settle on anything before we had a chance to discuss
... When will we have phone calls, F2F, email threads, github issues, etc?

scottlow: I personally like calls, good way to progress issues.

richard: I prefer video calls sometimes.

jer: Sure, that's through WebEx, video is possible.

richard: We had troubles with webassembly groups, went through Zoom.

jer: Let's try to start with WebEx

narifumi: Guidelines for work mode, communications?

jer: We don't have them yet.

markw: I would be in favor of not only have calls on topics, but also a regular cadence of calls.
... Good way to pop issues back to the top.
... Calling for consensus on the calls.

mounir: The other side of that is that some groups punt discussions to calls.
... Work in WICG went well without calls.
... I could be open to a regular call but don't want to discuss everything.

markw: I agree with that.

cpn: Just would like the call not to overlap with Media & Entertainment IG monthly calls.

mounir: Something we could do. Ada has a bot where you can do /agenda on an issue that adds an issue to the agenda.
... I don't want to have everyone show up in a weekly call just because the time is booked on your calendar

markw: Yes, not weekly calls. Regular cadence is a way to process backlog and close issues when there's no activity or call for volunteers to address them.

richard: Cancellation when there are no agenda items can be good.

jer: Yes.
... A monthly cadence seems good. I doubt we'll have nothing to talk after a month.

cpn: Useful to have a dedicated slot on the calendar instead of ad-hoc doodling.

mounir: definitely
... Everyone here ok with a monthly call? And encourage ad-hoc discussions?

markw: You could have a slot for ad-hoc discussions.

mounir: Yes, encourage ad-hoc discussions, minute the discussions publicly

markw: Right, just make sure that everyone is invited.

mounir: Yes, good point.

markw: Want to emphasize the need to have a triage task during the monthly call.

jer: Yes.

richard: I don't want to take a lot of time on that because you have only one hour.

jer: Thinking about non assigned issues?

markw: yes

jer: So triage could be about assigning issues and not discussing issues in themselves.
... What about F2F meetings?
... We have a F2F at TPAC, is that enough?

mounir: Most of us go to FOMS. I don't know if we should use that as a second F2F opportunity.

jer: There's an opportunity to do a meeting.

mounir: It would be less exotic, but more convenient for people

markw: I think we should revisit in a couple of months whether we want to have another meeting. Good idea to coordinate with other event otherwise.

mounir: I'm sure 6 months after TPAC, a 1 day meeting is useful

<scribe> ACTION: jer and mounir to talk to FOMS organizers about interest to co-host a Media WG F2F

scott: Note Greg's proposal on GitHub

markw: Question on how formal we want to be. We need to understand what the consensus is.

jer: Actions and resolutions are a good way to record the TLDR of meetings.
... Question is do you want to do it for everything?

mounir: For GitHub issues, I think what works well is aggressively cc'ing people.
... I don't see any reason to change that, no one complained about changes landed so far.
... Are there concrete concerns?

scottlow: I don't think that Greg is concerned about anything in particular, just to have a way to track resolutions.

chcunningham: Some GitHub issues may be harder/longer than others
... The HDR one is tricky for instance. Substantial addition to the spec.
... I don't feel strongly that we have to formalize the process.

scottlow: For new people coming to the group, it may be difficult to understand what's going on.

markw: Good wake up call to have in the issue is to say "I think this is good now"

mounir: Good to point people at pull request.

markw: There may be different alternatives without PR and you want to understant what the consensus is. Proposed resolution seems good.

jer: True.
... I don't know if we can resolve that now.

<scribe> ACTION: jer and mounir to create an issue on the media-wg repo about the resolution process

PROPOSED RESOLUTION: The Media WG will have monthly calls, and encourage ad-hoc meetings on specific issues

RESOLUTION: The Media WG will have monthly calls, and encourage ad-hoc meetings on specific issues

[Side discussion on versioning for MSE and EME and repos]

jer: Deliverables. I am not an expert on W3C process. We have a number of specs listed in the charter of the Media WG.
... [reviewing the charter]
... Deliverables include Media Capabilities, Picture-in-Picutre, Media Session, Media Playback Quality, Autoplay policy detection, MSE, EME, all of them coming from the WICG.
... In the future, we may adopt additional normative deliverables, after incubation in another group.

cpn: One idea we're floating around is to create a Media Community Group that could act as a companion to this group for incubation.
... The thing we were talking about yesterday is whether we could reorganize the IG into a more CG structure and use that as a place to incubate.

jer: So that could be a potential home for some of the incubations listed in the charter as potential normative deliverable.
... Did we fail to include some deliverable?

mounir: One additional comment. For MSE, we may extend with additional features. For EME, only 4 additional features listed in the charter, we cannot do more.

jer: Can we do maintenance?

mounir: Yes.

chcunningham: I've been pushing a new attribute on the video element. More on HTML.

padenot: just cc everybody.

jer: Yes, we have a generic issue that by definition, some of our specs need to monkey patch HTML, We'll need to have joint session with HTML in the future.

markw: Do we have editors assigned for these deliverables?

mounir: Media Capabilities, chcunningham and I are editors.
... Picture-in-Picture, Fran├žois Beaufort and I are editors
... Media Session has Becca Hughes and I as editors
... Media Playback Quality, technically I'm the editor, but I just copied the text out of MSE.
... Autoplay policy has no one, the plan was to have Paul and Becca.
... MSE has Matt Wolenetz and folks from Microsoft are no longer there I think.
... EME has David Dorwin and Mark Watson. David won't continue, but Joey Parrish will take over.
... Now, let's revisit the list and see what changes we should be making

jer: Given your chairing responsibilities, should we replace you as editor with other?

mounir: Practically speaking, I will only do spec reviews. If people are concerned with my having two hats, no problem dropping from editing, but good to have 2 editors to ensure some peer review mechanism.
... I am editing 0 spec by myself.

markw: We need to make sure that the other editor doesn't have the same expectation.

mounir: Sure. For Media Capabilities, chcunningham volunteers. Scott, do you want to have someone?

scottlow: Yes. We're interested. Greg would be the one.

PROPOSED RESOLUTION: Add Greg Whitworth as editor of Media Capabilities

RESOLUTION: Add Greg Whitworth as editor of Media Capabilities

mounir: For Picture-in-Picture, anyone interested to join?

jer: Apple is interested in implementing, but cannot take the task.

mounir: Then no change.
... Media Session, I heard that Mozilla is shipping it, any interest to edit?

padenot: I need to check. I can't commit right now.

<scribe> ACTION: padenot to check whether someone from Mozilla can become editor of Media Session.

mounir: For Media Playback Quality, I think chcunningham might be willing to edit.

chcunningham: Yes, I started to look into it.

mounir: One issue is that the HTML editors said that they would like to move the spec to HTML.
... Anyone else on top of chcunningham?

jer: This is a small spec, it may be sufficient to have only one editor. And the editing role might be a pull request against HTML.

<scribe> ACTION: mounir to swap his name with chcunningham as editor of Media Playback Quality

mounir: I think merging with HTML depends on the content of the spec.

jer: That can be a discussion in the spec.
... Autoplay policy, are we happy with Becca and Paul?

[group nods]

mounir: For MSE, Matt is happy to continue.

scottlow: I may be able to have someone on our side, I need to check.

GregFreedman: I'm expecting to edit either MSE or EME

mounir: So we'll have someone from Netflix, and maybe someone from Microsoft?

markw: Correct.

mounir: And for EME, same thing for Netflix?

markw: Yes.

mounir: My undersanding is that Joey might be taking over David.

scottlow: Same action for me to look into possible candidates.

<scribe> ACTION: markw and GregFreedman to figure out who edits which spec (MSE/EME) among themselves

<scribe> ACTION: scottlow to check for possible editors for MSE and EME

[going back to agenda bashing]

jer: Anything more for agenda bashing?

francois: We should figure out which of these specs is ready to go to FPWD
... important from a patent policy perspective
... the call for exclusions covers anything that's in the spec

mounir: i think media capabilities is ready
... also picture in picture and media session

francois: the next point that triggers IPR commitment is CR

[side discussion on whether Media Capabilities is ready to go to FPWD, or whether HDR support needs to be baked in first]

<scribe> ACTION: jer and mounir to send CfC for publication as FPWD of Media Capabilities, PiP and Media Session the week after TPAC

EME - Overview

See Encrypted Media Extensions repo

mounir: Before we start, in the agenda you had vNext and issues.

jer: Yes, it's about understanding what we want to see in the next version of the spec

mounir: OK, that's mostly for MSE, then, because we're constrained on EME.

jer: Yes.
... We can talk about Persitent Usage Record sessions, HDCP detection, scheme capability detection, and existing session finding.

markw: And maintenance issue

Joey_Parrish: [present context for session finding]
... Discussion at FOMS was to separate the two issues

mounir: Joey, can you confirm that you want to take editor's rolve over David?

Joey_Parrish: That's correct.
... One other thing I'd like is to get rid of robustness strings.
... Matches what Microsoft has done with com.microsoft.hardware / com.microsoft.software
... There seems to be a better way to do that.

jer: I think that would be considered under maintenance issues.
... Joey, can you file an issue on that?

mounir: Just to be explicit, we should be using the W3C repo for that. The WICG one should move away.

jer: The ability to push new keys on an existing session through some mechanism is captured in an issue, I think. The issue got punted for next release. Part of maintenance as well.

EME - Persistent Usage Record

See fork of EME repo with persistent usage record

jer: It was in the spec and got removed.
... That's a way to have a record of playback to be stored and replayed cryptographically so that the application can verify that the key is no longer in use.

markw: The language has been sitting in the WICG repo. The charter says it's our starting point, so we should upstream that.

mounir: Mark, are you driving this?
... Any concern with moving that to the main spec?

jya: I would prefer not to implement it

markw: I don't think that's mandatory.

mounir: We are talking about persistent usage record

markw: It's an after-the-fact mechanism for fraud detection.

richard: there shouldn't be objections, that's in the charter.

mounir: OK, seems Mark or Greg should prepare a PR.

markw: Yes, we were actually blocked on ReSpec issues. Now fixed. MSE and EME were built with old versions of ReSpec.
... There may be work that we can do to switch to the latest version.

Joey_Parrish: Does this mean we're sticking to ReSpec and not Bikeshed?

markw: For the spec that already exist, I would stick to the same system.
... I won't do the transition at least.

jer: Joey, you can use Bikeshed for new specs, no problem with that.

mounir: HDCP detection and encryption scheme capability detection are WICG repos.
... HDCP detection can fill a lot of time, given TAG feedback, etc. so would move it to the end.

EME - Encryption scheme capability detection

See Encrypted Media: Encryption Scheme Query proposal

jer: Apple Fairplay team would only support CBCS scheme and others CENC. In the meantime, the schemes have been added to the same common file format

markw: The problem still exists.

jer: AV1 is on CBCS exclusively?

markw: As far as AV1 is concerned, it's a requirement that all encryption patterns get supported.
... It's understood, I think, that devices that support AV1 need to support all of them.
... The portion you apply the pattern to has to be a multiple of 16 bytes chunks.
... This is a lower level detail than the pattern, but there is a question about whether the 16x bytes to which the pattern is applied is aligned with the start or the end of the OBU.
... Whether we allow both approaches or require one or the other is an open discussion.

jer: Do you think that it has implications for the spec?

markw: I hope not

jer: Apple is interested in clarifying behavior. Any objection to adding to the EME spec?

Joey_Parrish: Chrome supports both CBCS and CENC. Just no way for an application to query that yet.

Joey_Parrish: Prepared an explainer.
... It just adds one encryption scheme. Minor change.
... Not entirely settled how default would work.

yongjun: CENC version 1 and version 3 have different requirements. In the deployment of Prime videos, we noticed problems.
... Version 1 requires 4 bytes, version 3 requires 5 bytes. It's not compatible.
... Basically, we have two specs under ISO 23001-7:2016.
... It's nearly impossible to backfill everything in the same implementation.
... It's always a mix.

jer: Just wondering whether we can mandate a specific version of CENC with the string.

chcunningham: Can we mandate both?

jer: We could, but won't help Amazon.

yongjun: The recommendation is to support everything.

Joey_Parrish: We will be developing a polyfill that allows to query the version supported.

jer: Is the idea that you will try and fail to detect the version?

Joey_Parrish: No, apps hardcode what devices support, the polyfill would abstract that in a library

yongjun: It's a mistake in the CENC specification not to have been backward compatible.

jer: Rather than making a more specific query to seek the version of CENC, be more specific in the section that details CENC that all sections of CENC must be supported.

yongjun: We don't have enough time to backfill everything. There's always a mixed catalog on the backend.

GregFreedman: It seems like if 90% of your catalog is CENC 3, it seems strange to be willing to support CENC 1

chcunningham: Was there a line somewhere that version 3 was a good thing to use when it got introduced, because it would be supported by EME?

yongjun: We always move to the latest version.
... When we onboard new devices, we make sure that they support all versions.

markw: The capability problem is only in one direction there.
... The newer spec has more bytes in the clear.
... The older devices should not have an opinion provided that at least the first 4 bytes are unencrypted.
... The problem is that there are newer devices that require more bytes in the clear.
... In practice, that only applies to H.265 streams and not H.264

yongjun: Microsoft Edge is like this, I think.

mounir: I think we are digressing here.
... Maybe you should file an issue about this. Can you do it?

yongjun: Sure.
... This is super important for Microsoft Edge.

mounir: Is there something in the change that people have concerns about?

Joey_Parrish: For backward compatibility, if the encrypted schemes is not provided by the application, the behavior is mostly unspecified. No restriction. The way we would be able to detect support for the field itself, when the application calls getConfiguration, that should be in the configuration returned for media keys.
... The idea is that a polyfill could be written to make simple calls to detect support for the field.

mounir: Everyone happy with that?

scottlow: Yes, as an implementer.

PROPOSED RESOLUTION: Joey_Parrish to prepare a PR to integrate encryption scheme proposal in EME (W3C repo)

RESOLUTION: Joey_Parrish to prepare a PR to integrate encryption scheme proposal in EME (W3C repo)

EME - API to find existing sessions

mounir: My understanding from discussion at FOMS is that we need to get back to the drawing board. Not done yet?

Joey_Parrish: Right. The feedback I get from Jer is that the design was a bit overloaded as I wanted to cover two cases.
... Example in Widevine, matching base on the content ID.

GregFreedman: I assume this is active session.

Joey_Parrish: Correct, active not expired session. In the case of key rotation, expired sessions would not show up.
... Main purpose of that is that, as application developer, I don't have a deep understanding of sessions, which may lead to duplicate sessions, which can be an issue with CDN sessions.

jer: Seems reasonable as a future direction.

Joey_Parrish: Key IDs are not commonly available on all devices.

GregFreedman: Right. No objection.

Joey_Parrish: Would an API that would give you back key ids instead of sessions be preferrable? I'm open to that.

GregFreedman: I don't feel strongly. It may be preferrable for inspection purpose.

Joey_Parrish: From initialization data to key IDs instead of from initialization data to sessions. I think I went for the latter because I wanted to cover key rotation, but agree based on feedback that it should be kept separate.
... The only reason to have a session object is [missed]. Other than that, it's not useful.

richard: There's a limited number of sessions that you can have from an hardware perspective. Is there a way to tell the difference between expired sessions and sessions that exceed the number of available sessions?

jer: It seems to me that there is a better way [missed]

GregFreedman: From a diagnostic standpoint, key id would be more useful. I might know that the session is active, but I do like the idea of having the key ID. Whether or not you have an API to check active sessions is less important from my perspective but it's ok.

PROPOSED RESOLUTION: Expose an API to expose key ID information from initialization data in order to find existing session

Joey_Parrish: You could use the API to do the mapping. I don't have a WICG repo for this yet, only a personal repo.

tidoust: You don't need to have a WICG repo.

jer: It can be a PR against the W3C repo

yongjun: Format of key ID is different. CDM specific.

jer: Yes, ArrayBuffer in the spec.

markw: The Key ID format is defined by the container specification (e.g. CENC). 16 byte string

[discussion on key ID format]

yongjun: We should not infuse an additional format here, that's my point.

PROPOSED RESOLUTION: Expose an API to expose key ID information from initialization data in order to find existing session

RESOLUTION: Expose an API to expose key ID information from initialization data in order to find existing sessions

EME - HDCP detection

See Encrypted Media: HDCP Policy Check proposal and TAG Feedback

jer: Content may not be licensed for playback given HDCP support or not.
... The proposal is to have an API to allow early detection of which HDCP levels are supported.

mounir: Joey_Parrish wrote a spec for that.

Joey_Parrish: It's a mapping on MediaKeys that returns the version of HDCP. Privacy review suggests concern for fingerprinting. Mitigation is that it is user activated as part of MediaKeys.

jer: 9 different versions of HDCP that can be supported. Value depends on the connected display, so danger of exposing more unique info per user.
... It's already possible to gather that information through trial and error.
... And small number of Web sites that can use this since keys need to be got first.

markw: Granularity requirement is basically HD, 4K, perhaps to be refined to add a third one.
... Slower time to start playback if you don't have such an early capability detection.
... User consent is for EME at all.

yongjun: Dynamic detection is needed when user plugs another display.

jer: Two things can happen: unplug a device that supported HDCP requirement, plug a new one that supports HDCP requirement.
... One solution would be to poll this API to detect changes.

mounir: Media capabilities has this idea of adding a change event on the window object, and you could perhaps use this.
... I believe Mozilla implemented, do you have any feedback?

jya: I don't know.

chcunningham: CDM that enforces HDCP during the playback. Multiple screens, Widevine policy is I believe to report the lowest protection value when you have a highly-protected display and a low-protected display.

jer: Thing happens, because we receive user reports.
... We take the same approach.
... The user doesn't understand that a display that used to work no longer work after plugging another one.

markw: The general problem of reacting to user changes is not one that we see as a priority. We don't object to it, but it's not straightforward to make it a good user experience.

jer: One thing that someone like Brave or privacy-mode might do is to try to return a single value for HDCP support.
... I would hate to get in a situation where you couldn't implement a privacy mitigation without breaking existing sites.

mounir: Mark, you said we could get down to 2-3 values.
... Is that possible?

Joey_Parrish: Probably, except it probably depends on the specific contract requirements. Studios may update from 2.2 to 2.3, and this value would need to reflect that.
... If a browser is faking this value for the benefit of privacy. It's effectively equivalent with fetching the value in advance and the user plugging in a display in between.
... Any application that cannot handle that is going to break in corner cases in any case.

markw: Right, I wouldn't say it's equivalent.
... Difference between a corner case and a case that always happen.

jer: The fingerprinting of media capabilities is significantly less for Apple because we run in a short number of hardware. But that's not the case for others.
... The HDCP policy is different because it can change dynamically and dramatically at any time.

Joey_Parrish: We could add an "unknown" value that would allow not to lie.

jer: Returning "unknown" is already a signal compared to lying.

Joey_Parrish: Having implemented something similar in ShakaPlayer, you must start with SD and ramp up with keys you're going to receive.
... With lying, it wouldn't mean that you'd be stuck forever, we'd just do what we're already doing

mounir: I assume EME is HTTPS only, only available on top. No reason for cross-origin iframes should use EME. EME is async API by default, so you can have a prompt. This call itself is aync, so you could have a prompt. Technically, everyone can get this information with Widevine, it's just a bit more work.

jer: Not the case for everyone, though

mounir: Right, but fingerprinting is not high risk.
... I would add in the spec a note to explain what implementations could do if they don't want to increase the fingerprinting surface.
... And we'd just get back to what we're doing that.

markw: Inform sites that rejecting this API is something that can always happen.

jer: Having the ability to reject this promise is good. Links to Privacy Budget discussion as well. The call may succeed at one time and fail at other times.
... Language like that would be appropriate.

Joey_Parrish: Adding some language for fingerprinting mitigation is ok.

GregFreedman: Some HDCP versions are unsecure. 2.0 2.1, anything below 1.4, we'll never query for.

chcunningham: The privacy gains by doing the collapses is minimal.

markw: You could bucket those, no problem on our side.

jer: Any normative reference that we could have not to have to update the list?

mounir: We'd need a registry.

[discussion on registries at W3C, possible process updates, etc.]

<scribe> ACTION: Joey_Parrish to add language to the privacy path

GregFreedman: On Windows, I was told that the media foundation does not recognize 1.4 and reports 1.0

<scribe> ACTION: Joey_Parrish to look into reducing the number of HDCP versions we expose to collapse to those actually used

Joey_Parrish: Fine with the action but I will need some guidance on what values are actually necessary

mounir: About the TAG issue, should we come back with a resolution?

jer: If we enact the actions, then I think we can get back to the TAG for a resolution on these issues.

mounir: Right.

[short discussion on maintenance issues]

Joey_Parrish: The two things mentioned earlier are key rotate and my suggest to kill robustness.

markw: There are a bunch of issues on the repo. Some of which may need to be dropped, or acted upon.

jer: Seems a good thing to do in a call.
... About key rotation, internal clients of EME at Apple had issues with adding new keys.
... Why generateKeyRequest cannot be used multiple times?
... There seems to be hardware limitations.
... We closed that early in the first version for lack of time.
... I'll just re-raise the issue on GitHub, and we'll cover that later on.
... About removing robustness, any significant pushback?

[none heard]

Joey_Parrish: Issues could reasonably be addressed by introducing things such as .hardware or .software and other properties.

GregFreedman: I know we use robustness in practice, so need time to transition.

Joey_Parrish: Yes, long time starting with intent to deprecate.

mounir: Did you file an issue about that?

Joey_Parrish: Not yet, will do that.

mounir: We want to make sure that we can notify everyone else.

Low-latency <video> signaling

See [Proposal] hint attribute on HTMLMediaElement to configure rendering latency thread on WICG's Discourse

jer: Proposal from chcunningham to allow client to signal to a user agent that they would prefer a different style of rendering, e.g. to signal not to use a jitter buffer.
... Game streaming mentioned as a use case.
... Currently, Chrome has a heuristic and would like to make it explicit

chcunningham: Correct, and the heuristic is often wrong

jer: The right place for incubation is a CG, but there is overlap of interest with Media WG participants.

chcunningham: [going through the discourse post]

alex: Are we talking about the rendering buffer or the jitter buffer?

chcunningham: The rendering buffer.
... Proposal is to add an attribute that would cut it down to 1 frame.

alex: Is there a separate proposal for the jitter buffer?

jer: I used the wrong term, I don't know anything about jitter buffer.

alex: Jitter buffer is used in WebRTC.

chcunningham: Through discussion with Jer, he thought the API was too low-level. Taking leverage of mediatrack hints, proposal is now to rather take a declarative approach.
... I did also talk to Harald briefly yesterday, he didn't think this was a crazy idea.
... He thought we might even use their enums and was open to adjust it, e.g. to support real-time use cases.

jer: Also, I don't want to impose requirements for this, because I may not be able to implement a 1 frame buffer. Hence the hint approach.

jya: Moving back to MSE, there would be significant change of behavior, no stop and gap, etc.
... Would that apply to MSE? It's only when you attach MSE to a video element that you'd have access to it.
... We would have a different thing in MSE?

chcunningham: Yes.

jer: You can imagine having gaps keeping and low-latency rendering

jya: Could we assume than rather that being associated to the media element, it should be associated with the source?
... Just trying to have something that could have a smooth continuation on MSE, rather than having a second proposal on MSE.

jer: Right now, it's written as an IDL change. For attributes, you'd have to add the attribute to each source.

jya: Yes, even more flexibility!

mounir: Everyone talking about hints as if it was a done deal. I'm not a huge fan of it. Much harder for people to understand what will happen. Developers will reverse-engineer to understand what's going on behind the scenes.
... What people care about is the behavior, which user agents can change with an hint approach.

jer: And I disagree because I don't want to be required to implement a specific behavior.

mounir: The benefit of this is that we can use this for a lot of stuff, e.g. Audio focus.
... However, we will end up with a hint that will only be used for buffering.
... So we'd be stuck with an expected behavior.

markw: You want things to be well-defined IIUC.

mounir: Yes.

chcunningham: Even in Chrome, we make no guarantee.

markw: There may be use cases where you want to specify 2 content hints.

alex: Any other optimizations for these hints?

chcunningham: Only buffering planned.

jer: I can imagine that being used for other things.

alex: If you specify by optimization, risk of ending with 12 different parameters.

mounir: I believe that's OK.
... If we start changing behavior, people will have expectations.

jer: Most of media playback is open-ended. It's a kind of a pain but allows for flexibility.

jya: But then you end up having customers that expect a certain behavior when you have a strong player.

[revisiting the agenda to find a slot to continue the discussion]

Media Capabilities

See Media Capabilities repo

Jer: This is pretty far along, there are some issues needing discussion
... The biggest ones are: queries for HDR support and spatial audio support

Jer: Issues #118 and #119 both allow applications to determine the ability to decode and display HDR content
... The API shape has changed
... Allows the UA to understand whether formats can be displayed and decoded, given color spaces, transfer fucntions, metadata values
... There's also an API we're considering adding to Screen, to determine whether it's able to handle HDR
... Is this sufficient for clients and implementers? Is the API surface acceptable to expose to the web

yongjun: Is this HDR, HDR10, HLG, etc.?

Jer: I avoided encoded specific product names into the API
... e.g., Dolby Vision
... HLG has a specific transfer function, that's being proposed to be added
... Does the agent accept a particular transfer function?

MarkW: I think it's more the ability to decode, if a query responds positively to the query, you should know it will display correctly, although may not be the best output you could expect
... For one stream, can I play this? For CSS, selecting between multiple streams

Yongjun: It's not just HDR capability alone, also frame rate, e.g., HDR at 30 fps not 60 fps

Jer: Media Capabilities already has frame rate

Yongjun: So need to be able to signal these combinations

MarkW: The API takes a specific config, you get a yes/no answer

Jongjun: It's more like a query
... What about composition? An SDR and an HDR stream, can you show at the same, e.g, in a picture in picture window?

Jer: That's a problem in the API, regardless of HDR, e.g., can I offload from CPU to GPU?

Jongjun: How to composite SDR and HDR content?

mounir: a design principle of this API from the beginning is that we only answer the question for one video playing regardless of what's happening

MarkW: It's a very difficult problem to cover this in general. You have limited decoder resources being shared between streams

Eric: If you come up with a solution, the answer can change dynamically, there can be a race condition

MarkW: If you want to render multiple streams, you have to constrain things

Yongjun: so what's the scope of Media Capabilities?

Jer: VideoConfiguration has 3 additional parameters: hdrMetadataType, colorGamut, transferFunction
... should allow all HDR types currently in existence

Jongjun: Also bit depth?

chcunningham: When you have higher bit depth profiles of VP9, you're implicitly saying you support those bit depths, and the rendering

Jongyun: We assume HDR is 10 bit, but also works if 8 bit. It's both the decoding and rendering process, so not sure if 8 bit will be supported across devices

MarkW: The codec profiles are different between 10 bit and 8 bit

Yongjun: Screen and decoder on same device, but what about HDMI, you need separate queries

chcunningham: The definition of rendering is ambiguous, so we recognise applying the transfer function, not a decoding step, also not a display function. The piece between the screen and decoding we call "rendering"

MarkW: There are two kinds of hardware setups. On PCs and laptops, decode to linear half float, and then convert to anything on the way out
... If connection to an old monitor on HDMI, it'll be rendered, so the answer from the API is yes
... If that's the only copy of the content I have, I want it to display
... If we have both HDR and SDR, it will say yes to both, but we can check the HDR flag to select which to show
... This API only addresses what can and can't play. Use that to filter down

Richard: Why does the "maybe" thing happen?

Jer: This is because you could under-specify the MIME type
... We're moving away from that

MarkW: One tells you whether you could, and one is whether you should

chcunningham: If you have multiple screens, the Screen API gives dimensions, and this changes when you drag windows between screens. Seemed a natural place to expose whether the screen supports HDR

Jongjun: Downgrade from HDR to SDR, we don't use device conversion

Matt: Is canPlayType only used to select HDR or SDR

Jer: No, could also be about available bandwidth

chcunningham: The API tells you what we can do, and how well we can do it.
... So the UA allows the webapp to make the decision

Jer: The API returns if possible, smooth, and if power efficient

Mark: Other dimensions include resolution and color gamut
... This API tells you if a resolution can be rendered, could involve downscaling
... Then you can check the Screen API to select based on resolution

Jer: Could be polyfilled by an API

Mark: Also color gamut,an SDR in 709, and 2020, query the Screen. But we don't have this, as wide gamut capability generally comes along with HDR capability
... Could do it, but doesn't make sense in practice

Jer: Color gamut, should we specfy rec2020, small difference from P3, but it's there
... I believe with these parameters, can distinguish Dolby Vision, HDR10, HDR10+, HLG (for now...)

Jongjun: For one title, we have HDR10, 10+, and Dolby Vision

MarkW: Similar, we also have multiple variants

Jongjun: If someone only has one copy, should we use "maybe"?

chcunningham: We should add some examples

<scribe> ACTION: chcunningham to add examples to the Media Capabilities spec

<gregwhitworth> Opened issue #132 for examples

Jer: There are two PRs

Jer: VideoDisplayConfiguration is newly added, you can pass in width, height, hasHDR, can check if this is supported by the display
... This API is more limited, to reduce fingerprinting impact. We hope it's enough for content providers to make decisions about filtering HDR and non-HDR streams

<gregwhitworth> it is only MQ

<gregwhitworth> matchMedia will work

Jer: Bit depth and color depth for the display is already exposed in CSS

MarkW: Not sure bitdepth on Screen is that useful. it's difficult to pin down. Most TVs, the actual panel is still 8 bits
... Asking whether its worthwhile sending 10 bits to the display is difficult

Jer: Media Queries were used in webkit to ask if display is capable of displaying wide color gamut images

<jya> See the definition of the Screen interface in CSSOM View Module

chcunningham: I agree with MarkW, I wouldn't want bitdepth

Jer: Are width and height just there for embedded players with a separate video layer?

chcunningham: Yes

Greg: Don't want to stream 4K unless there are 4K device pixels

chcunningham: Cast has a proprietary API where you can get the display width and height

Mounir: We have feedback that this is generally useful

chcunningham: As written, it appears like implementations have to support all, but that's not the intent

<mounir> https://github.com/w3c/media-capabilities/issues/118#issuecomment-529878912

Mounir: So in general we're happy with the PR, but there's some wording changes needed

GregW: Can we clarify what's outstanding?

<gregwhitworth> Proposed Resolution: Remove statements that define what the sRGB, etc and allow that to be defined by UA

chcunningham: I'd want to take one last pass through the PR, need a couple of days for that
... Consensus is what's in the PR is good, feedback on semantics of the enum values

Media Capabilities: Spatial Audio

See PR #123 - Add support for Spatial Audio rendering capability queries

chcunningham: We landed the PR, is there any feedback on that?

Mounir: Re decoding vs rendering, the API can say no to rendering even if decoding is possible

Jer: There's no equivalent to Screen for audio, so no other place for those queries to go

Paul: Web Audio didn't take this into consideration. We are addressing this, taking into account modern technologies
... There's a new repo so you can add issues to request stuff

Audio Books

See Audiobook Profile for Publication Manifest and Publication Manifest

Wendy: I'm from a digital bookseller, and co-chair and editor from the Publication working group

Wendy: on TAG advice, I'm here to tell you about our spec

Wendy: We have two specs: Publication Manifest, and Audiobook profile
... They're closely relayed. It's a general manifest format for publications on the web
... We have a profile for specific use cases
... We currently have one profile, for audiobooks
... We keep as much audio content in mind, thinking of podcasts
... Want to avoid overlap with other work, and be open about what we're doing
... Manifest format for B2B or B2C. A couple of use cases: listening to an audio book, a11y, and portability
... Make it flexible for users. Using JSON-LD and schema.org for metadata. Focus is on identification and information to user or distributor
... Audiobooks is siloed by retailers, want to break this down with an open format
... Identifying metadata, reading order. There's a CG working on synchronised media
... An audio file with other media, such as text or video, as a more accessible experience: listen and read, or sign language
... Spec allows for a table of contents, for detailed navigation data for the user
... It's also for podcasts, want to cover their use cases
... I want to sure our spec works with your APIs, or can we help make your work easier?

Jer: Is this about the audiobooks profile or the publishing spec in general?

Wendy: Audiobooks in particular, this has strict rules, e.g., can contain only audio files

Jer: Overlap with ePub?

Wendy: We work closely with the ePub3 CG, can be the future of digital publishing post-ePub

chcunningham: We should make sure our format strings are the same
... Is audio/vnd-wav vendor specific?

<mounir> See MIME types section in Media Capabilities

Mounir: Now we require codecs in the mime type for everything

Wendy: We decided not to restrict the allowed audio formats, wanted to be future proof
... We can reference this for the right formats

Jer: When you have a client different capabilities, how to select between different formats?

Wendy: Could have a fallback or high quality or low quality, decided not to do that, will do in a best practice doc. Approach would be to have separate manifests
... We have a list of 2.0 things

chcunningham: Let's have the same language for specifying which codecs to use

Mounir: Agree, helps make using APIs together easier

Paul: The temporal fragment - look at the Media Fragments URI spec

Francois: Do you have specific requirements for synchronisation. This is being discussed in the M&E IG

Wendy: That's in a CG, Marisa DeMeglio is an editor

Wendy: That spec is separate, want it to be generally usable

cpn: we talked with Marisa about this in the past
... one of the reasons why TAG is interested is because of the general packaging effort
... there is an effort about media containers -- having a MP4 file synchronised with the presentation
... there is a draft MPEG standard nearly finalised and they are now working with W3c -- that is somewhat controversial
... Dave Singer (Apple) has been interested in having a workshop around this topic

Wendy: Dave Singer has been working with us around packaging for the publishing industry as they are not as technical
... we talked about MPEG and HEIF, he gave a presentation and it looked promising
... for this industry, it wasn't a clear cut process
... we are still working on this but this spec will unlikely have anything around it
... we believe web packaging is promising for publishing

mfoltzgoogle: in general, when there is media metadata format, Media Session should be looked at
... we should look at mapping them

Francois: The alignment could go both ways. This is based on schema.org, should we align Media Session to that?

Wendy: We use the creators list, author and "read by", rather than narrator

Mounir: In Chrome, we've been looking at schema.org to see if we can populate the media session
... Some websites don't use media session, but they do have schema.org
... In future, Media Session could suggest default values based on schema.org
... Today, most browsers use file icon, page title, for artist title, so could use this to provide better information

Richard: Is there overlap regarding security and privacy

Wendy: We're ready for CR, so need to get security and privacy review. As a manifest format, there aren't so many security and privacy concerns

MarkF: Another relevant use case is background fetch for offline use. Can you get the file size from the manifest?

Wendy: The reading order can be a series of URLs. Depends on UA implementation, whether you pre-fetch or not. We don't have instructions on that
... In that case you'd download everything

MarkF: Is there enough information in the manifest to be able to check against storage quotas?

Wendy: We could include bitrate, so that with duration gives a hint. We work with reading systems as UA rather than browsers
... Can request the first bytes from the audio file to calculate the size
... We use zip for compression

Mounir: If you have multiple files, bitrate is useful, adjust for quality, as with DASH

Media Capabilities: Issues

jer: do we just want to do triaging of these bugs?

Chris: I think we may want to look at some put issues

[talking about some issues we could look at]

jer: should we walk through them one at a time?

Chris: we should look at the subset I mentioned

jer: let's start with transition() ergonomics #102

<tidoust> Issue #102 Discuss transition() ergonomics

jer: it's about transitioning from one codec to the next and the capability of doing so

Chris: it was done before the MSE API change
... the explainer had a method called query() at the time and we had this idea of an API called transition()
... we discussed very complicated solutions like if you can do A and B, can you do A => B or B => A
... after talking to Joey Parrish, he said it was over engineered and he wanted all or nothing as a web author
... websites want to know if you can A <=> B or nothing
... so it seems artificial to say that A => B is doable so is B => C but not C => A
... the proposition I came up with is to have a boolean in the result that says whether the transitions from this codec are allowed
... which calls out cases where a UA cannot transition out of a specific codec

markw: what is the "pipeline" you are describing in the issue?

Chris: it's the clear pipeline for example
... software and hardware CDM may have different transition support

markw: in a hardware CDM context, could some transitions be supported but not others?

Chris: the API will answer only if one cannot transtion from or to a given codec

Greg: can you give an example?

Chris: [didn't hear]

jya: i'm puzzled about the use case
... even for the hardware codecs, could the UA deal with tearing down the pipeline and creating another one?

Chris: they may do that but still websites would need to know

jya: i can't find a case where we couldn't transfer somehow

Chris: it is a concerned for hardware CDM and televisions
... something we heard for example is whether non-smooth transitions were okay and we think this shouldn't be considered a transition

jya: you may lose a few frames but you should always be able to transition

Chris: this is implying an architecture

Greg: the use case I'm envisioning would be to use AV1 for low resolution and VP9 for high resolutions
... and maybe AV1 is hardware decoded

Chris: if you CDM lives in the chip, it's possible that you can't extract the keys from it and can't transition

markw: is seamless implicit for transitions here?


markw: what about non-codec changes such as colour, framerate, etc.

Chris: it should be fine as far as I know, things like 30fps to 60fps transition should always work
... in Chrome if the decodec can do 60fps, it can implicitly do 30fps, is that not always true?

markw: correct, in some cases transitioning when HDMI framerate matches the video framerate can take several seconds

jya: what about audio transition? [mentions a case where there can be several seconds of silence when transitioning]

Chris: are these scenarios around HDMI common enough to take them into account?

markw: [mentions taking into account tv, and not only browsers]

Chris: Some progress here. I have some call next week with TV makers

jer: happy with the direction the API is going -- easier to implement and use
... switching to issue #111 and feature detection

<tidoust> Issue #111 - Changes to MediaConfigurations cannot be feature detected in older UAs

jer: one way is to return the dictionary as it was understood
... for HDR, for example, one could check whether the HDR-related information are returned to check if it was understood

Chris: supporting this and was waiting to see if we wanted to have a sequence of configuration as input before going forward

jer: no one else?
... nothing, then we can go to the next issue

Chris: Mark's framerate issue can be brought up

jer: issue #96 and #95

<tidoust> Issue #96 Revert the "DOMString framerate" change from issue #12

jer: previously the framerate was a double and Mark mentioned that some configuration couldn't be represented as a double
... so I asked for this change to be reverted
... because I didn't implement the string parsing
... so now we are stuck at how we can represent a rationale number inside the web platform

mounir: I talked to some folks at Google working on ECMAScript and they said it would get support but would likely take a long time

markw: i didn't want to have a string, I would be fine with something like a double or enum

Chris: doing an enum would be better than string parsing
... but the original issue mentioning a concern about people mixing representations using double or rationale representation
... it doesn't matter to our implementation so it really is just about the ergonomics of the API
... and have it less error prone

mounir: mark, is this something that you cane about because of API design or Netflix usage?

padenot: same for Web Audio, we made bad representation mistakes in the past and regreted them
... issue tracker is full of issues that change the number slightly

markw: facebook has a number as a common denominator for framerate and audio samples called [flix]

jer: we should work on getting time values in media land appropriately
... it sounds like web audio has something

padenot: yes, we have a solution [couldn't hear]

<padenot> web audio allows keeping track of the number of samples as an integer, and lets you divide by the sample rate if needed

markw: maybe a helper could help here so the browser can tell if two values are the same to it

mounir: it would really just apply to media capabilities

jer: should we go with the enum?

Chris: I would point out that we shouldn't do enum only

jer: idea here is to do string + enum

mounir: I don't think we should do an enum. Mark's comment is "Let's represent framerate correctly on the Web" and doing an enum is just a hack. We'd need to complete the enum over time.
... I looked a bit into using an interface. Main issue is operational operations looking ugly. You can do them, but that's ugly. A Rational interface could be doable, but we should liaise with Ecmascript.

Chris: Would it be fine not to define mathematical operations?

jer: But people will want to to them to times the framerate

Chris: Will happen with the string as well

jer: Yes, but totally impossible to do with a string without first parsing the string
... The ideal scenario is to have a rational.

markw: it sounds that people don't really want enum
... maybe if we just specify as a double but we are very clear about what that number means
... where it's clear that this number is approximate and not a precise representation of the framerate
... maybe that takes away the problem

Chris: I see this as a performance thing: the Apple framerate restriction for HDR
... 30 and 60 are round numbers but maybe the cut off will be 30.1 someday

markw: is this a performance issue or something else? could you play a 60fps content at half speed?
... there is also a lot of restrictions for example with 4k, you have framerate restrictions for a given profile

RESOLUTION: we can revert the framerate attribute back to a double

Chris: we have the same issue with bitrate
... this is a performance indicator, not something precise
... and it is recommended to pass the highest bitrate instead of the average

markw: maybe the name should be changed for "renderingFramerate" to make this clearer

Chris: let's talk about the enconding info issue
... it's #126
... Chrome implemented part of the API

<tidoust> Issue #126 Cleanup encodingInfo()

Chris: the proposal came from someone who works on this
... RTC folks may not implement it

jya: we did implement and launch it
... for both WebRTC and MediaRecorder

Chris: one issue is that this area do not use the same codec strings as us

jya: we even added web platform tests
... it's a big improvement compared to what canPlayType is for media recorder

Chris: for WebRTC it's a bit strange
... there is ORTC performance API coming

jya: the problem with WebRTC, when you query it can be irrelevant sometimes because sometimes you can only do one codec
... in addition, with the negociation, I do not see a big use
... with MediaRecorder, I believe there is a better use case

Chris: that's true, though it's unfortunate that they use the h264 codec string
... a use case I've heard is to avoid re-opening the camera when downscaling

jer: is the idea that some of this stuff move to WebRTC?

Chris: I'm trying to figure this out

jya: I thought that the idea was to avoid this


See the WebCodecs repo

peter: we've had breakout and wicg sessions, lots of comments and issues discussion...
... webcodecs relevance to media wg
... media capabilities
... mse vnext ...
... not sure what the overlap will be yet, but maybe some integration
... as webcodecs matures, may become part of this wg...
... if we want injectable codecs, may want wg reviwe of shape...

jer: will you also be able to control specific properties of the decoder (configurability of the codecs)

peter: yes, see github issue for some outline. e.g. some discussion of b frames
... looking for input

mounir: webrtc (harold) looking into injectible codecs. is that tied to this? are we thinking about wasm codecs?

peter: webrtc interested, so far no one looking into it for video stack outside webrtc

jer: vlc would wasm all of vlc
... already have

peter: power users free to wasm everything

mounir: stack is thinking about this
... chromeos supports some obscure codecs, would be nice to make them injectable for security pov
... in chrome stack webrtc and video are two different pipelines

paul: concern is that webrtc is modularized.
... will folks be able to do a hybrid solution with web codecs + peer connection?
... or will web codecs mean they entirely roll their own webrtc impl

peter: one idea floated is to have codec registry
... api for injecting codec would then make that codec visible to webrtc and video alike

steve: the challenge is each api that would embed a custom codec would need additional info
... mse needs to containerize
... webrtc needs codec signalling
... not clear if registry is feasible

peter: alternative is for each API to have its own injection hoook

paul: playback of video could use a media stream from web codecs

mounir: is the a difference how you decode depending on where the video comes from?

jer: yes, same decoders in the end, but very different routes to get there

chcunningham: webrtc may have some duplicate sw decoders - not sure what they take from the platform

mfoltz: for wasm use case, is that a way to support codecs that lack universal x-browser support?

peter: yes, ex: if browser wants to drop support for an old codec
... also works to add new codecs

mfoltz: what does it look like for player to choose between web codec and wasm?

peter: 2 ways. vlc way: all wasm. or you could inject the codec into video stack (if api had a hook to do so)

mfoltz: i don't see that there's an extension point to do that yet
... what would it look like for mse to use webcodecs?

peter: designing on fly, something on html element like an attribute that represents the encoder or decoder as a stream

mfoltz: sounds like area for exploration. interesting to have player libraries that could do either way

petere: might instead have a codec object that could handle switching between codecs

richard: considered a way to use for still image

mwatson: some video codecs have metadata embedded in the stream (e.g. hdr sei msgs) - how does webcodecs deal ?

peter: could add this to the output. currently have just is_key_frame, could add other properties

mwatson: so for given codec you define what these will be so receiver knows how to interpret?

peter: could be ways to put codec specific things in there. k-v pairs

mwatson: if I want to feed the output stream to video element, may need some extra details
... had a question yesterday about image discussion - what codecs would be supported for images?

peter: its in webcodecs explainer

mounir: encoding/decoding APIs are indeed media wg purvue
... may be able to do this even without re-charter
... discussion about creating a media cg as well
... whenever you believe you have something stable enough could move to wg

francois: agree, we don't need to change the charter
... just need support of the group

eric: skeptical we can define an interface that allows javascript/wasm to create frames that can be passed down to the platform pipeline

(encoded frames that can be decoded by platform decoder)

eric: particularly for a cross platform way

jya: we've made a demuxer's / encoders / etc inside the platform that work in a interoperable way
... so web codecs could work

eric: i'd love to be wrong. support investigation

peter: thanks, all I got :)

jya: you support mpeg ts, but chrome doesn't, so you find a lot of libraries that fill the gap in js
... twitch generates mp4s with a single frame
... could simply use webcodecs

Low-latency <video> signaling (continued)

jer: maybe an api to control decoder configuration, it should be web codec

chris, we might want something simple here, in advance of web codec that will take some time

jer: a specified behaviour for this would be rather hard
... should this affect every webcodec as well ?

chris: in webcodec, you're in charge of everything, right ?

steve: yes, indeed

padenot: also yes, you do eveything yourself

chris: hence, this would not have a use in the web codec world

jer: the issue here is the shape of the API
... for some shape of the api, I can't implement some part of it in safari, so we would wait for wait codec
... so a higher level API would be preferred, for us
... I'd like an API that would be useful for every UA, but if not, we can't implement

chris: for someone like twitch, it makes no difference, they are not asking for this

mounir: how exactly having hints help?

jer: it can affect the number of UA that can implement

mounir: how is the videocontenthint something you can do, but the low-latency api something you can't do

jer: for hints, there can affect some other bits, for example pitch shifting technique, the type of audio streams, etc.
... it's been historically a problem, especially on mobile, but this will be covered by the audio focus spec
... we've tried to have a way to be able to slot in new behaviour

jya: could we focus on doing an API that would be useful for the user instead ?

jer: no matter what the shape of the API, some of it we won't be able to implement
... but we'd like to do something that can do more that just changing the buffering strategy

chris: asks jya about opinion

jya: the hint that says that the video should be low latency is obviously very useful
... we would be able to pass, on a platform level, different settings to the decoders, for example on Linux and Windows, it has massive impact for the end user
... I'm still also looking at the MSE side of things, and I'd like something that covers it both
... it always relate to the type of input you have, after all
... maybe you have gaps, but you also want to be on time
... I do see value in having something in the area, but I don't know whether or not it's better to have this on the media element or on the source
... it's always been the source that drives the rest, in the end

mounir: I don't understand why this is a problem, to have it on the element

jya: it's also about the ease of implementation
... the source is created first
... so it would be easier to set the attribute on the source
... it would allow to have things setup slightly earlier

jer: can you append to a source buffer before it's attache?

jya: no
... we can't probably do either way, but on a implementation perspective, it's easier. it's also more logical to put it on the source

mounir: on the other hand that means it can only be done with MSE

jya: as a matter of fact, it's only with MSE that you can achieve that
... but maybe even with a plain URL src =
... but it also depends on the provider of the media, depends on the presence of b-frames
... if there is b-frames, then it makes sense to try to do low-latency, otherwise it's useless
... a side effect of that hint, would be to see frames out of order

chris: on the source, it would work just as well maybe, but the way I reason about it, is that MSE is about demuxing
... but you're right it's a consequence of the source, you have the source, the demuxer, the decoder
... there are multiple things to do here, and they are not inherently connected

jya: indeed we also keep 3 hardware frames, but 10 software frames, there is implication everywhere
... it doesn't matter where it goes, but I'd like to have MSE brought into the picture

mounir: we have a session tomorrow morning

jer: we control MSE, it's easier to do

mounir: let's not do that because it's easy
... and it would be approved on HTML anyways

chris: the use case I have in case for cloud gaming, there is no possibility of gaps, they don't have gaps/underruns
... for twitch, big MSE users, they won't have gaps, so this won't care about gaps skipping

jya: their gaps are too small for them to cause problem
... a few microseconds here and there

padenot: because of rounding and round-tripping between different domain

chris: we con't have consensus, let's continue discussins after MSE

jer: I thought in all this "source" was the source element

jya, no, any source really

scribe: I can't see that use outside of MSE really, it's so natural to see the link between the two

francois: you mention MSE between a temp measure before web codecs

jya: we would still use MSE

jer: how does demuxing work in web codec

padenot: a magic "demux()" function

chris: we're just trying to give more control

mounir: we don't know what the future is, it's best to have something more general

jer: it would make things worse: less resilience, but you don't see any latency benefit because you don't have control over the rest

mounir: I agree, but technically it could be done, and why not put in on the element

jer: if it's useless, let's not do it

chris: it's not about this possibility for src=, it was more natural semantically to put it on the element

mounir: I reckon that leaking this info across object doesn't seem good

chris: let's ask microsoft

mounir: do we have at least a resolution to not do the content type hint

jer: happy to be the odd one out

jya: I don't care where it goes as long as we can get it done

jer: this is going to be temporary, folks that will use this will jump to web codec as soon as it's realease and the need for it will disappear
... in a world where there are web codec, this will not be used at all

chris: this is at least partially true I think
... stadia would definitely prefer to use web codec if/when available
... let's bring this up again during the MSE discussion

scott: I don't have a strong opinion right now

RESOLUTION: using a hint specific to low latency instead of content hint

Summary of Action Items

[NEW] ACTION: chcunningham to add examples to the Media Capabilities spec
[NEW] ACTION: jer and mounir to create an issue on the media-wg repo about the resolution process
[NEW] ACTION: jer and mounir to send CfC for publication as FPWD of Media Capabilities, PiP and Media Session the week after TPAC
[NEW] ACTION: jer and mounir to talk to FOMS organizers about interest to co-host a Media WG F2F
[NEW] ACTION: Joey_Parrish to add language to the privacy path
[NEW] ACTION: Joey_Parrish to look into reducing the number of HDCP versions we expose to collapse to those actually used
[NEW] ACTION: markw and GregFreedman to figure out who edits which spec (MSE/EME) among themselves
[NEW] ACTION: mounir to swap his name with chcunningham as editor of Media Playback Quality
[NEW] ACTION: padenot to check whether someone from Mozilla can become editor of Media Session.
[NEW] ACTION: scottlow to check for possible editors for MSE and EME

Summary of Resolutions

  1. The Media WG will have monthly calls, and encourage ad-hoc meetings on specific issues
  2. Add Greg Whitworth as editor of Media Capabilities
  3. Joey_Parrish to prepare a PR to integrate encryption scheme proposal in EME (W3C repo)
  4. Expose an API to expose key ID information from initialization data in order to find existing sessions
  5. we can revert the framerate attribute back to a double
  6. using a hint specific to low latency instead of content hint
[End of minutes]

Minutes manually created (not a transcript), formatted by David Booth's scribe.perl version 1.154 (CVS log)
$Date: 2019/10/02 19:40:55 $