Meeting minutes
Bernard: focus is to make progress on our specs, incl getting some of our specs to CR & PR
Bernard: Today, we'll work largely on existing specs and Thursday, new work
WG Status
harald: We've been rechartered to continue to do what we were doing
… finish WebRTC 1.0
… describe requirements from new use cases and address them
<vr000m> Can someone let me in to the google meet?
harald: WebRTC 1.0 should just work across browsers, networks
… we should have low level data access with high perf- think of funny hats, voice compression, background blur
… We're not getting as much demand for ORTC and sons
… WebTransport and WebCodecs are basically where that action has moved
… Insertable streams will also help on that side
… once you can get the information of what's going on up and down the JS API, that should help do what ORTC was set out to do
… In terms of document status
… Media Capture and Streams is a CR, with a few open issues (incl old ones)
… not much has changed, but some new ideas for the in-borwser device picket to deprecate the constraints-picker based mechanism
… but overall, this is working as intended, with lots of products depending on it
… we're trying to push new things to extensions as we can, possibly with some back merge using the process 2020 capabilities to merge new features
… plans is to target Rec by Q1 2021
… next: screen capture
… a key feature in our increasinly virtual environment
… we need to go through TAG review
… a new proposal emerged on tab sharing
… Next spec: WebRTC 1.0, published as final CR (famous last words)
… 10 open issues, mostly editorial, one privacy; most should be easy to fix once we get around to it
… the implementation maps show good progress on interop
… the community sense is that it's being used in billions of hours per day
… we promised a Rec in Q4 2020 (now)
… Next: WebRTC Identity - nothing has happened on it
… it's still referenced from WebRTC, but we will ask forgiveness for it
… In terms of Resources Availables - only 2 non-chairs editors (Henrik and Youenn)
… pull requests coming from them and Jan-Ivar for the most part
… other contributes PRs - thank to you
… we need more workforce as editors
HTA: we can't force anyone to work, but please volunteer if you want stuff to happen
… Other active documents: capture from DOM is heavily used, no recent big changes, need to go through wide review and close open issues
… Likewise, Recorder needs updates - we're missing an editor
… Stats Identifiers is getting active edits, and the issues are under control
… Other documents are more quiet: depth, audio output devices, content hints, priority
… In terms of other activity in W3C: WebCodec, WebTransport (happening in WGs)
… we had a joint meeting with the Media Interest Group which showed interest across the community - no specific proposal emerged out of it though
… lots of people caring deeply about security and privacy issues and want to make sure we write specs that respect security & privacy
… In this meeting, we want to finish 1.0 - we're close to get them shipped, which will allow us to look at new APIs
… the use cases and requirements are key; raw media is kind of the current burning issue for lots of people
… we've taken up a couple of proposals to access raw media and deal with it, which we will discuss on Thursday
… any question before we dive in testing?
Testing and implementation status
DrAlex: this is an update to the presentation from last year, will cover the slides fast
… It remains difficult to compute the coverage of the spec in terms of testing
… our spec is instrumented to keep track of which parts of the spec is covered
Bernard: also, it only focuses on W3C specs, not the IETF specs
DrAlex: correct - particularly relevant when it comes to network testing or simulcast
… If you look at the tests we do have in Web Platform Tests - the number of tests has risen, and the number of passes have increased too
… slow progress since last year though
… we have heard frustration from non-browser vendors in getting their tests merged in WPT
… we need to decide how to rekindle that process, and make sure contributions are indeed processed
… [showing results from running Web Platform Tests in Kite]
… Kite shows 66% pass; Kite can take a more granular approach to counting results than what WPT does by default (one failure per test, not per test file)
… we've gone from 66% to 70.6% in 2020
… improvements across all browsers
… In terms of Simulcast & SVC: WPT can't test peer-to-peer usage, you need network instrumentation to test ICE
… that became a bigger problem in 2015 when simulcast was added to 1.0
… the simulcast loopback by Philip Hancke helped with getting simulcast visibility in WPT
… after several hackathons and sprints on this, I think we're pretty good now
Aboba: the loopback test has helped identify pretty big protocol bugs (e.g. missing MIDs in FF)
DrAlex: right - my fear is that there may be more big bugs hiding behind what WPT can actually test
… we've been able to run reports on WebRTC support across various SFU
… we had a good experience testing browsers with SFU during an IETF hackathon
… but this is a lot of work that has proved difficult to reproduce esp with IETF going virtual
Bernard: when might we go back to this?
DrAlex: hard to tell - SFU vendors aren't W3C members, more focused on protocols than JS
… one of the limitations of testing simulcast via SFU is that it depends on specific approaches from the various vendors
… even finishing basic tests have proved difficult to get done
Tim: I can offer to help talk with SFU vendors if we have a concrete proposal
… there was a proposal to have vancouver hackathon that was overtaken by events
… hopeful we'll get one in November
DrAlex: I thought the Vancouver proposal was specifically targeted for non-JS environments
Tim: there are lots of people interested in having simulcast work, and SFUs have moved on a lot, with much more public APIs
… I think it's worth going back to that group with a concrete proposal
DrAlex: let's do this then
Tim: I'm happy to take the action if there is a clear ask
Carine: the WG is now under the new W3C process 2020
… a few differences in it for Proposed Rec
… the first few criteria are unchanged: two interoperable impl of each features, wide review (already done), close issues that we received during CR, do not make substantive changes
… (CRs are now either Snapshots or Drafts)
… no substantive changes can be made to the last CR Snapshot when going to Proposed Rec (as they would invalidate the patent policy commitments)
… we can remove features marked at risk in the previous CR
… for WebRTC 1.0, no change has been since our last CR snapshot and we have one feature marked at risk
… 10 open issues on webrtc-pc, 4 editorials, 7 ready for pull requests, 4 questions or test suites issues - nothing substantive that would prevent us to go to Proposed Rec
… only received 4 issues in the last month, 2 editorials; there may be open questions on whether other testing issues may emerge (e.g. on simulcast)
… WPT has shown improvements; we've developed an interoperability report showing what has no implementation or only 1 implementation
… (considering Chrome and Edge as a single implementation)
… the major pieces that have only 1 implementation are SCTPTransport, IceTransport, setStreams, DataChannel.onclosing
… to go to PR, we need to document the situation on simulcast for the transition request
… previously we were also tracking progress on IDL interface implementations - that report now is mostly equivalent to what we get from WPT
… the table remains interesting as it maps gaps to specific APIs pieces
… since last year, more green, less orange and red - shows improvements overall
… for consideration to the WG: we have 2 main features without double implementations IceTransport and SctpTransport (setStreams and onclosing are expected to be implemented reasonably shortly)
… the 2 transports will be implemented but in a slower timescale
… Process 2020 allows to fix recommendations post-facto if bugs emerge, so our feeling is that we should recommend Proposed Rec with the current state of implementations
… knowing that IceTransport and DtlsTransport would be part of the spec, are going to be implemented
… if we discover bugs as these implementation emerge, we would be able to get the Rec updated to fix it
… so we're suggesting not to delay WebRTC 1.0 any further and ask the Director to approve that situation
Bernard: we've been talking about the testing problem for a while; WPT limitations have been underlined recently with a breakage in mux/demux code
… not covered in WPT, more of an IETF functionality
… WPT is used both for W3C processes, but also for regression testing in browsers
… Kite can't be run for regression testing
HTA: on that specific bug, once we understood what it was, we were able to produce a WPT test for it
Bernard: how was that managed?
HTA: Fippo mangled the SDP in a smart way
Dom: WPT is an open source project - maybe worth bringing this to the community and see if there is interest in stronger instrumentation for network
Alex: we proposed to bring WPT.fyi results based on Kite, but haven't heard back
Aboba: we have similar issues in WebTransport - we've written little servers that run in WPT
Alex: we could do that for simulcast, but we didn't get any reaction on our simpler proposal, so not holding my breath
Dom: still worth investigating in my opinion if we think the ecosystem would derive significant value
WebRTC Stats
Henrik: updates from last TPAC - we had lots of discussions around enabling simulcast which required moving lots of stats around
… That migration has now been completed
… outbound-rtp objects are generated per simulcasts layers
… track stats have been made obsolete as they can be found elsewhere
… in Chromium, M86 shows that migration completed
… we keep track stats for backwards compat (with the old design)
… the stats now map pretty closely with the object model of the WebRTC API
… only a subset of the new stats types have been implemented - missing remote-outbound-rtp, sender, receiver, transceiver, sctp transport metrics, ice server metrics, csrc (but that's already available in get{Synchronization,Contributing}Sources
… all in all, most things are available but there are some things missing, some are available outside of getStats, some aren't
… In terms of Mandatory stats, we have 66 out of 77 implemented in Chromium M87; FF and Safari have lots of impl, but also many gaps
… over 170 metrics have been implemented, not sure what percentage that represents
… Beyond moving stuff around, not a lot of activity on spec / implementations
… Does that mean we're all good? I think Stats are in pretty good shape, mostly polishing work needed
DrAlex: is simulcast going to work the same with SVC?
Henrik: the simulcast stats don't cover SVC
… we discussed SVC stats at last TPAC, but this is blocked on a proper API for SVC?
DrAlex: so webrtc-SVC?
Varun: we're indeed waiting on the SVC API approval
Florent: we wanted to get experience on the SVC spec stuff before writing the stats for it
Varun: if things are good enough, how do we move to the next stage?
Dom: I think we may want to take advantage of the living standards model from process 2020
Shachar: I would like to see more traction on the SCTP transport stats
Varun: in terms of implementations right?
Henrik: I don't have an update on that - not spending a lot of time on getStats right now, haven't heard lots of demand for it
Jan-Ivar: we're committed to implement MTI stats, but not committed to a specific timeline at the moment
… sctp transport stats can be shimmed from what I can remember
Varun: we've added a couple of metrics from the low level stack (e.g. RTT)
Jan-ivar: but they're not mandatory, right?
Henrik: correct
Varun: we had a privacy review recently which led to removing networkType
Media Capture and Streams
Jan-Ivar: we had a joint meeting with PING (privacy Interest Group) last week
… on the 12 issues they had filed, 4 are still open and 8 were closed
… I propose to review the issues we presented with the solutions
… #640 on revealing label of devices in a more limited way
… we've suggested we can't solve this in the short term for web compat; in the longer term, the in-browser device picker is where we see the solution, being developed in mediacapture-extensions
… we were encouraged to clarify sanitization and the purposes of the labels as being for display only
… on #645, limit enumerateDevices to the type of devices being shared
… we will align with what Chromium implements which is more privacy sensitive
… on #646, we learned that returning empty lists is not web compatible, so won't do this but give clarifications on waht UAs are allowed to do for privacy mitgiations
… #672 getCapabilities privacy - already limited to be usable only during capture; feature at risk, may be revisited later in browser picker
… Beyond privacy issues, looking at regular issues
… #660 handling of rotation for camera capture streams
… current implementations assume that constraints are always in landscape, but getSettings get rotate in portrait mode
… the proposal is to specify this (since that's already what browsers do) and it avoids thorny issues about overconstrainted situations
Youenn: +1
… have you tried applyConstraints too?
Jan-Ivar: I haven't, but I think we should specify it would work the same
… #735 change fitness distance from MAY to SHOULD
… to help with web compat for device selection, for this spec but also image capture
Youenn: MAY vs SHOULD doesn't really change much; I don't think we can do MUST with a better defined algorithm
… PTZ may need more detailed handling
… fitness distance is difficult to use to get the results we want
… I would like to have at least guidelines we agree on, to make that SHOULd stronger
… the additional issue I have with fitness distance in general is that you end up with equal fitness distances across devices
… which may also be the source of interop issues
Jan-Ivar: I think MAY is pretty weak, which may be coming from FF having a permission prompt
… that MAY feels more a bug to me
… FF allows the user to override the desires (constraints) from the app, which we can address with a separate MAY
HTA: another source of input to picking a device is the one identified as the default one
Youenn: the PR looks fine, but there are cases with PTZ where this may not work
Henrik: the goal is to clarify our intent that this is what ought to be done
Jan-Ivar: not much has happened on mediacapture-extensions for the in-browser picker
… we want to move away from in-content device selection
… we wants privacy-by-default in browser picker
… We've seen progress in audio speaker section with selectAudioOutput() in the last year
… that gets you a picker, which if agreed exposes a deviceId to the app which can uses it for setSinkId
… it also allows to work without mic permission
… Safari drove the design of that improvement
… We also added a deviceId member to selectAudioOutput to help with remembering devices
… (which the UA can still put behind a permission prompt, e.g. to deter trackers)
… This design can serve as inspiration for picking mic/cameras, but that problem is more complex
… you may need several cameras, microphones, apps want to guide camera selection (e.g. resolution)
… more importantly, there is a question on how to migrate from current getUserMedia to that new device picker model which gives less power to developers
… the main privacy issues right now are during capture
… we're thinking of an incremental API; FF already has an in-browser picker
… we would provide a new mode ("semantics") called "browser-chooses" vs "user-chooses"
… with the goal of flipping the default over time - doing that would probably create new permissions prompts
… the end goal is to remove labels
Henrik: if you don't flip the default, how could you possibly remove labels?
Jan-Ivar: the first big step is having "user-chooses" in all browsers, at which point all sites can upgrade to user-chooses and skip having an in-content device picker
… the stick is that we would remove labels, which makes the old mode less appelaing
fluffy: +1 on getting better way of doing this - better control of input devices is greatly needed
… but I don't think the stick is going to work as an approach
jan-ivar: the goal is to give motivation for the migration
… the migration path may work on a long timescale
fluffy: but this needs to be developed with the app vendors
jan-ivar: removing the labels is what we need to do to plug the privacy leak
… the API is meant to lessen that pain
Youenn: Web app developers will compare the two UX; if the new one is better, that's a pretty good motivation
TimP: what happens if you do the two last steps to a site that hasn't been updated in 3 years
… would the right thing just happens in terms of UX?
Henrik: the user would get what they want, but the in-app picker would show a stale UI with meaningless labels
Youenn: one downside is having both approaches in a given browser is that there will 2 prompts that users will have to understand
timp: the devices that get exposed in enumrateDevices may change once the default is flipped
Jan-Ivar: there are 2 types of sites: with device management, and without it
Henrik: only returning the chrome-selected device as a migration path would break device updates
Jan-Ivar: we got positive feedback from PING
… we plan to implement selectAudioOutput and use that as input for the new API
Fluffy: so you do not plan to deprecate that in the 1.0 timeframe
Jan-Ivar: correct - longer term
Henrik: the only thing that goes away in 1.0 is that enumerateDevices only returns 1 device per type prior to actual capture
Jan-Ivar: correct, stricter than it used to be
… some push back buttons from a couple of app vendors, but the privacy reasoning has been stronger
Screen Capture
Issue 60 - Tab Capture
Harald: the idea of a tab was not defined; the closest thing seems to be a "browser display surface", which I suggest we refer to this as the correct term
Jan-Ivar: terminology-wise, we've used browsing context instead of "current document"
Henrik: +1 - I think we have language that says we can't change the source, does that apply here?
Jan-Ivar: the source here is the container (the content displayed in it is not the source)
… that's probably what most users would understand too (that capture doesn't end when you hit the back button)
Henrik: what about switching tabs?
Jan-Ivar: I think of this as a "virtual source" - the point here is to avoid confusing users
Henrik: OK, I think that's worth clarifying
Guido: wrt sources, it's linked to the deviceId setting (although not exposed to app developers)
Jan-Ivar: displayMedia doesn't have a device Id, but it exposes labels
… there is some guidance on UX in the spec given the security risks
Youenn: some people are asking to be able to know if a tab is same origin or not, whitelisted or not
… and be able to mute the track if so
… for that, we would need an event that signals the tracks is getting opaque
Image Capture
Jan-Ivar: 2 issues with Pan/Tilt/Zoom
… the spec needs a way to call getUserMedia with Pan/Tilt/Zoom, but without specifying a value for Pan (e.g. don't want to move the camera just to ask permission)
… they invented "true" as a placeholder in stead of a value
… but this didn't work correctly with fitness distance
Youenn: with PTZ being a privileged feature, we're not sure that fitness distance will be the right approach
Jan-Ivar: that's probably covered by the SHOULD
… but the fix still needs to be fixed
… Also unspecified how non-PTZ cameras satisfy default value - e.g. needs Zoom to be adjustable
… two proposals on the able - make values other than 1 for zoom request adjustable zoom; make any value for zoom request adjustable zoom
Youenn: PTZ should be restricted to applyConstraints with specific values
… booleans for device selection make sense
riju: any implementation plans for this?
jan-ivar: no current plan
youenn: no short terms plans, but we welcome patches
jan-ivar: likewise
henrik: to me it's confusing to mix picking and configuring e.g. for zoom
youenn: +1
<TimPanton> +1
jan-ivar: the goal here is to align with the constraints syntax
… that's why I'm more for Proposal B
… (i.e. any value for zoom requests adjustable zoom)
youenn: if we think boolean values for getUserMedia, specific values for applyConstraints, we should do that in the spec
jan-ivar: I think that still implies proposal B
Youenn: so the restriction would be discussed as separate issue?
… sounds reasonable
Media Capture from DOM
Youenn: #24 & #85 to help move media capture from DOM forward
Harald: +1 to Jan-Ivar's proposal