Harald: Welcome to the WebRTC WG meeting
... we'll be talking about WebRTC in the next 2 days
... a lot of time will be spent on current state of WebRTC and its bugs and issues
... we will also discuss testing
... and next steps to bring WebRTC to PR
... by 3pm today we should be done with the current spec, and we'll then switch to NV / next work
... (i.e. things that won't get into the 1.0 spec)
... these are based on requests we've been getting for the past couple of years
... please abide by the IPR policy - in particular, do not make substantive contributions if you have not joined the WG
... After the F2F, we will update the WG drafts
... what's on github reflects the WG current thinking
<dom> [Warning: meeting is being recorded]
Harald: first coffee break at 10
... tomorrow morning we will restart at 8:30am
... we have slots available tomorrow afternoon for topics we can't finish during their allotted time
Harald: we've been doing this work for 7 years now
... our charter sets us to finish WebRTC 1.0 as high priority
... work on an object-based API based on ORTC addressing requirements for new use cases
... that includes now protocols and associated APIs, new data access functions
... People demand that WebRTC 1.0 should just work the same across browsers
... that's true for a subset of cases
... Chrome has switched to Unified Plan in Canary, which broke 1/3 of users
... we're getting closer
... we're also seeing demands for low level access to data - driving some of the new work
... we're also seeing demand to move away from SDP (i.e. ORTC)
... Looking at our core specs: getUserMedia and WebRTC 1.0
... getUserMedia was published as CR a year ago - we need to push an update
... we still have 20 open issues
... interop reasonably good, but not done yet
... were supposed to reach PR this quarter
... WebRTC 1.0 has 46 open issues (30+ recent ones), still lots of interop issues
... plans is to reach Proposed Rec one year from now
... We split out identity in its own CR - with 23 open issues, most of them old
... test suite needs to be separated out of the main WebRTC test suite
... In terms of resources, we have 2 active editors (thanks to the former editors!)
... other specs have other editors
... Resources come from people volunteering to do the work, with the blessing of their org
... we need to make sure we get things done - both finishing specs, and starting work people cares about
... Other docs include: screen share, recorder, stats, depth (quieted down?), audio output devices (seems tsable), content hints, dscp (not getting much momentum)
... we should consider which of these need to be advanced or killed
... Focus of this meeting: finish 1.0 (based on contributions we got in time for this meeting)
... new APIs, raw media access (paving the path for the future)
Youenn: I really like the first item "finish 1.0" - top priority for me
Jan-Ivar: +1
Harald: 3 issues on getUserMedia and audio output
<dom> PR 548 in getUserMedia
Jan-Ivar: our idea to double-key origin for embedded content permissions has been taken over by feature policy
... users don't understand iframes, so asking complex permissions question wasn't very practical
... so the suggestion is to remove the combination language for origins
... it becomes the responsibility of the top level domain to manage delegated permissions with feature policy
Youenn: Safari is already only showing the top level origin in its prompt
... so +1
Henrik: +1
<dom> Should getUserMedia be functional in SecureContext only?
Jan-Ivar: should we restrict getUserMedia to secure contexts only? most browsers only support it in https - I think only Firefox still supports in HTTP: but we're happy to switch it off
... the proposal is to limit to secure contexts only - in HTTP, we could remove the method, or remove the mediaDevices object
... remove the object itself would make sense since the other interfaces don't really make sense out of getUserMedia
... this relates to the audio output issue because of enumerateDevi e
<dom> Should setSinkId be functional in SecureContext only?
Dom: I would suggest removing the whole interface of http - enumerateDevices is a source of fingerprinting
Youenn: +1
Jan-Ivar: I agree too
Bernard: we've been asking people to use getUserMedia in https for a long time
... this would also align with screen capture
Jan-Ivar: what about setSinkId in audio output?
dom: it can't really be used without enumerateDevices, right?
Jan-Ivar: right
Harald: I detect likely consensus to move these to secure contexts
Jan-Ivar: this may break some sites
Harald: browsers have a process to deal with deprecation
RESOLUTION: Move to [SecureContext] for getUserMedia, Audio Output
<dom> Full screen mode
Henrik: dealing with full screen is a bit tricky; full screen mode can happen in different ways (borderless window, existing window getting full screen, new window getting full screen)
... it's hard to tell what to do with applications spawning new windows
... the proposal is: if the current window enters fullscreen, no change (except track gets resized)
... if a new windows get spawned, it MUST not be shared which may mute the current track of the current window
... if you share at the application level, should not be an issue
... One problem of this is that this may break some apps
... issue 81 is seeing how we could allow UA to change sources
Youenn: this is a clarification, not a change, right?
Henrik: this was undefined behavior
Harald: is full-screen mode well-defined across OSes?
Eric: AFAIK, it's OS dependent; there is no well-defined "fullscreen mode" e.g. in MacOS or iOS
Bernard: powerpoint can use multiple displays
Henrik: as long as the UA is clear to the user WHAT they are sharing, this should be fine
... OK with treating switching to full screen as resizing?
Harald: there doesn't seem to be an OS concept of fullscreen anyway
youenn: right - we should make this as a note in the spec (in particular, let's not add MUST if this is already covered for new windows)
Harald: so +1 to an informative note on this
<dom> Define behavior of existing constraints
Jan-Ivar: we have constraints, capabilities and settings
... we need to define the min/max for the constrainable properties
... for width - the max is the width of the source (no upscale), min would be bounded by preserving aspect ratio when downscaling
... aspect ratio is basically fixed, except when a window is being shared and resized
... the pull request also indicates how to handle window resizing
... overconstrained situations don't really make sense in the context of screen sharing
Harald: I think this is a bit under-specified - which resolution will be picked when we get into overconstrained mode
Jan-Ivar: this is covered later
... [downscaling and frame decimation]
... I'm not covering advanced constraints - I would prefer not to implement them for getDisplayMedia, but wouldn't be a hardship either
harald: this looks relatively well-defined, and seems a net improvement to the spec
... would like to suggest merging this
henrik: is this just clarification? or is this new stuff?
jib: it's new because right now it's not defined
<dom> Handling source device pixel ratio
JIB: proposal is to double it & scale down
Henrik: is there a race if this happens while the user resize?
JIB: this is handled by queuing
Bernard: +1 on merging this
Harald: seems like consensus
<dom> Limiting browser sharing to a list of domain/URLs
Jan-Ivar: limiting the selection of tabs per origin can be abused by an attacker
... one of the risk is an attacker including an iframe to a third-party origin, so you don't even want to limit sharing to one's own origin (for top level)
Youenn: does this mean no tab sharing at all?
Jan-Ivar: the spec doesn't disallow tab sharing - but the JavaScript should have no control on what the UA shows when doing tab sharing
Youenn: sounds good
Harald: ok, jan-ivar will close this with a comment explaining the rational
<dom> Unclear how to aggregate windows or how to handle multiple windows/monitors
Henrik: clarifications on how aggregation actually works - e.g. preserving relative sizing
... leaning on leaving it UA-dependent re positioning
Youenn: is there any implementation for this type of aggregation? is there actually need for this?
Henrik: the spec distinguishes between window and app - it's a bit confusing though
Bernard: Edge doesn't support app (nor aggregation)
Henrik: sharing a screen may be simpler
Youenn: or you share multiple windows, each as its own track
Jan-Ivar: Firefox supports application sharing
... think we should keep it - sharing a full screen has security issues with sharing Web content
... Firefox will be downplaying "app" as a user selection
... window will be the default granularity, and we may make exceptions over time
Youenn: my concern is about user understanding; it's easy to understand window, screen, - app not so much
Jan-Ivar: but there are cases where the user's mental model is at the app level (e.g. photoshop with multiple toolbar windows)
Youenn: but if this ends up a UA dependent choice, this could be exposed as a "window" rather than "app"
Henrik: why would the app needs to know if something is being shared as app vs window?
... this seems to be orthogonal from the Web app perspective
Bernard: the only impact I could see is at the level of content hint, but that's orthogonal
Jan-Ivar: this has been in the spec for a long time - let may be check on the list before removing
Harald: I'm hearing the simplest version is to remove constraints entirely; the second is to remove the "app" constraint
... or to leave it alone entirely
Youenn: I'm fine with going to the list, but with a specific proposal
Henrik: I think the UA should be allowed to let the user share what they want, aggregated or not
... but I don't think we need to share any info about that decision to the JS layer
<dom> [4 people for removing constraints]
<dom> [6 people for removing the "app" as a special constraint]
<dom> [1 to keep as is]
<dom> Jan-Ivar: would like to check back with Martin about the original intent
Harald: we'll take it the list, noting that "removing" (app|constraint) had stronger support
<dom> [new strawpoll: strongest support for removing the entire constraint]
... we'll take this to the list then
<dom> Constraint to exclude application audio (echo)
Henrik: unclear if and how audio can be tied to a specific window
... proposal is to make audio source UA-dependent, always optional
Youenn: i.e. if no audio constraint, no audio track; if audio: true, there may be one
Henrik: audio-only requests are always rejected
Dom: are we default to true for video? that seems inconsistent and TAG-frown-worthy
Henrik: good point, but a separate discussion
Youenn: no objection, a bit worried by adding a new feature
Varun: is the audio track synchronized?
Harald: at the RTP level yes
Bernard: my only concern is leaking audio from the system when the user shares a window
Henrik: it's the responsibility of the UA to guide the user
Youenn: e.g. share audio only while sharing the screen?
Henrik: it's a matter of making it clear to the user
Youenn: this is a *screen* capture API - but now we're adding audio?
Jan-Ivar: the problem is one of user expectation
Bernard: there are risks of surveilling a user by sharing just a small window
Henrik: the UA needs to make it clear to the user what they are sharing
Youenn: adding this feature will require guidance in terms of security
... this will delay implementation of the spec
Jan-Ivar: at the minimum, the security section of the spec should call out the risk of over sharing audio
Harald: so I'm hearing we should merge this, and add stuff to the security section
Jan-Ivar: will the audio include other audio constraints (echo cancellation, etc)?
Harald: no, I think we want to keep it simple
<dom> [Coffee Break until 10:30]
<burn> scribenick: burn
(returning from break)
hta: a few more screen share issues before we move on to webrtc-pc
<dom> -> hhttps://github.com/w3c/mediacapture-screen-share/issues/79 Constraint to exclude application audio (echo)
Issue 79
<dom> trackbot, bye
henbos: issues with including all
audio - yourself, tab's audio
... echo cancellation doesn't help.
... proposal is a new constraint that means 'give me everything
EXCEPT the application's audio'
... one way to satisfy this constraint is merely not to provide
any audio
... another is to give audio for various other tabs
... the key is not to share the application audio
hta: anytime you are in a
conference and share the output back to the conference is a
problem
... when app is not the conference it's fine.
... use case is sharing all system audio when that's the
conference
henbos: not a problem if only
sharing another tab's audio. It's sharing your own that's the
issue
... this is a hint from user app to avoid this.
jib: why can't the app itself control this?
henbos: it wants to share screen sharing audio but not its own
jib: why can't the app do this itself by turning audio on and off
henbos: tricky
youenn: couldn't the user set up a mic to record the audio they want and only play that? (not sure that's right)
jib: if sharing full screen and
get system audio, participants might hear themselves. IIUC, if
JS could turn off its own audio this would not be a problem,
but then its own audio wouldn't be played.
... can echo cancellation handle this?
henbos: no. multiple different sources.
jib: no, i meant overloading the echo cancellation constraint for this rather than creating a new one.
hta: the default would probably be on for this.
henbos: wouldn't want on by default.
youenn: tab is doing PC. Browser knows its own audio it's outputting on that tab.
henbos: bad to guess if you have a PC active.
youenn: the browser needs to be smart anyway for an audio conference. If it understands the page already it can just leave out the tab audio.
henbos: what about before the PC is started?
youenn: it can handle that.
henbos: best not to make the implementation remove an arbitrary audio source.
youenn: that's what echo
cancellation is.
... this is new work. Maybe we should see what implementers can
accomplish first without the constraint.
hta: conclusion is let's investigate use of echo cancellation for this before adding a constraint
issue-81
<dom> The user agent should be allowed to change sources after getDisplayMedia resolves
henbos: (use cases from slide)
youenn: gum and gdm are different. This might be worth considering.
dom: intent was not to change without user consent. User consent is not an issue here.
youenn: question is whether js needs to know about the change or not.
jib: the model was that the js
would know so that it could detail devices. By allowing sources
to change that would not happen. But all of that is for
gum.
... As long as we don't violate the gDM model it's okay.
hta: in the interest of time, removing the 'must not change' language seems reasonable. We don't have an API suggestion yet for how to do this.
jib: we don't need a JS API. The
key is not to change security risk level without going back to
the user.
... we might need to reconsider some of the security language
here.
hta: remove 'must not', add guidance.
issue-82
<dom> Should getDisplayMedia() be moved to navigator.mediaDevices
henbos: agree with this.
hta: don't move stuff just to make it look better
jib: we would be happy to move it.
youenn: yes, we could live with it as it is, but it doesn't break things to move it.
bernard: it's already a secure
context. we wouldn't move it. We would just have it in both
places.
... moving it would break existing apps.
dom: but could you live with that?
hta: consensus in room (after show of hands) is to move it
issue-2005/1718
<dom> Track ids
<dom> a=msid should contain sender/receiver ids (#1718)
hta: in your example, are you
intending to send the same track twice? If so, this would be
legal.
... and the JSEP PR was a breaking change after last call.
henbos: since stream ids are still signaled you can figure out which track is which
hta: I disagree with this change.
jib: FF handles this by creating a random ID
henbos: chrome does the same thing.
hta: anytime you use addtrack rather than addtransceiver, the id is transferred correctly.
henbos: yes, but the remote party
doesn't get it.
... one problem is collision on local side. Main problem is
ontrack on remote end will use transceiver with a different
track id (typically)
youenn: this change makes sense to me.
hta: my objection is that we a breaking backwards compat on a feature where we can't tell.
henbos: unified plan already breaks this. When reusing transceivers this doesn't work because ids are assigned early.
hta: could delay assigning id to track until the first time (missed the rest)
henbos: if no transceiver exists,
what do you do. spec is unclear whether you can use track
id.
... since FF and Chrome use randomized ids, may be too late to
change now.
jib: FF randomized ids that go into SDP, on receiving side use the ones in the SDP.
hta: maybe need a test for this.
henbos: let's see what apps do today.
hta: could stop saying anything
about track ids matching on the two sides, or patch them so
that they work for some simple cases.
... (after a poll) only a small number of us care, and they are
in favor of not saying anything about track ids matching on the
two sides.
issue-1930
<dom> Rename sender.transport.transport to sender.transport.iceTransport
hta: agreement to change (no impact to existing implementations)
issue-1940
<dom> transceiver.direction doesn’t respond, if out of sync
hta: this is a bug fix.
steve: will renegotiation work here?
jib: depends on what else the js
does
... you can work around it today by setting direction to
something else and then setting back
henbos: is this mid negotiation?
jib: if you read back direction you could see whether it had changed
issue-1981
<dom> RTCIceTransport selected candidate behavior when changing state
hta: if you got 'disconnected' and wanted to check what you got disconnected from, not clear what you would get.
peter: makes sense for it to be
non-null if ???
... connected state is tied to selectedcandidatepair. Once it
has a selected pair the connected state is tied to it.
hta: so once it has a value it remains at that value until changed. we should add text that it should remain null until connected.
henbos: don't we have well-defined steps on how to update these slots?
bernard: not in this case. There
is no slot.
... proposal corresponds to what is commonly done, where you
only fire if it changes to non-empty.
jib: there is a selectedcandidatepair slot.
steve: yes, but you don't know when it will be updated.
bernard: updating the slot doesn't fire the event.
hta: proposal has consensus, except that unless should be until
peter: if the selected pair goes away you would want to know
bernard: not if it's temporarily coming in and out.
peter: but if you lose it permanently you would want to know.
bernard: when disconnected, can go back to connected. You might not necessarily have lost the pair.
peter: you don't want to go to null if temporary, but would if permanent. You should fire the event in the latter case.
hta: the pair is not nullable
peter: later today will cover a method showing you all the candidate paitrs
henbos: we should fire the event when the state changes
bernard: disconnected is temporary, failed is permanent.
hta: conclusion is that this is complicated and needs further work.
issue-2004
<dom> No procedure for the ICE failed state
peter: could also say 'terminal until ICE restart'
hta: could also say 'can only escape from failed state via ICE restart'
steve: fyi this is the ICE transport state
issue-1982
<dom> Missing normative steps for determining codecs
(general agreement)
issue-2006
<dom> setCodecPreferences and Direction
bernard: alternative 1 is the
status quo.
... so alt 1 would be clarify that this is how it works. alt 2
moves setCodecPreferences() to the sender and receiver.
henbos: question of which codec wins is separate.
bernard: but ordering can be changed and have an effect
hta: SDP O/A does not consider idea of a codec you can send but not receive, but preferences indicate what you prefer to receive and not send.
bernard: No. 3264 lets you not
send using a specific codec.
... the problem is that both direction and preferences affect
what shows up in SDP.
henbos: affects which codecs are in m-lines and the preference order.
bernard: affects what's in createOffer/Answer.
hta: that's the argument for keeping this on transceiver since it's O/A mess.
bernard: we can do that and clarify, or just move it.
henbos: do we want this mechanism
to be able to affect preference ordering.
... if you really care you can just use setPreferences to set
it directly.
bernard: sounds like we want to clarify this.
hta: any test cases yet?
henbos: only basic tests, nothing testing this.
dom: sounds like keeping on transceiver is the way to do because of the O/A dependence
Florent: how do we test in the browser the list of supported codecs.
hta: can simulate by stripping codecs out of the SDP.
henbos: if we keep this on transceiver, changing direction can change which codec you're already sending
bernard: no, only changes what you negotiate.
henbos: yes, exactly
hta: need volunteer to write test case
<Orphis> That's me
Florent volunteered
issue-2009
<dom> Clarify how codecs should be prioritized
Current procedure is what 3264 says.
hta: suggest we obey 3264
here.
... simpler to reason about. Consensus is option a
issue-2088/pr-2018
<dom> Using codecPayloadType with addTransceiver()
henbos: you can always be in a
position where you reuest a codec and don't get it
... the problem here is that you are expressing what you want
in terms of a payload type number. Only makes sense if you are
the offerer.
bernard: payload types can differ
between offerer and answerer
... only diff in approach 2 is you might be able to get better
validation.
... only setCodecPreferences help you in negotiation.
hta: moving remaining
webrtc
... webrtc-pc issues to after lunch.
bernard: approach 1 is simpler to explain
henbos: should be able to
negotiate multiple codecs but say which you prefer
... without the receiver changing it
bernard: as long as the receiver
included both codecs you can change the order again.
... approach 2 allows for more validation with more
complexity.
jib: FF doesn't implement this.
bernard: greater chance for
errors in approach 2
... but you could set up simulcast with multiple codecs
henbos: want to be able to say which out of a set of negotiation codecs will be used.
<dom> scribenick: youenn
bernard: 8 open issues in WPT webrtc
limited progress
vast majority of test uploaded fro Chromium and Mozilla
8 open PRs awaiting review, review PR is not working well
scribe: Need either to increase
PR review or increase PR submission rate
... increase test authors and/or test reviewers.
... No weekly discussion in WebRTC WG to make progress in
WPT
... Need more owners for WPT
bernard: listing key WPT
issues
... fundamental issue about how do we deal with test requiring
generating RTP
... KITE is the only option right now
... Test before commit...
... Only one issue has "Needs Test" labelled since TPAC
2017
... Should we continue with 'test as you commit'?
Harald: sometimes, this is 'test as you implement' and this seems to work well.
Henryk: this is how things are implemented.
Dom: for specification writing, it also makes sense to write tests.
Henryk: Write a test allows discovering issues in the spec.
JanIvar: "test as you commit" is
good as a goal
... for small changes, it is useful to do it as well.
youenn: 'test as you commit' makes sense for stable/maintenance mode
bernard: example is simulcast where it is difficult to write tests.
Dom: 'test as you commit' requires to submit a test PR at the time you merge a spec PR. It does not require to merge the test PR at the same time.
Henryk: Usually we should file a test issue for every commit
Harald: WPT issue?
Henryk: yes
jib: 'test as you commit' is also good as getting WPT reviews linked to PR is good
Bernard: should we schedule a regular WPT issue/PR meeting?
Harald: consensus is yes.
... second issue is who will volunteer?
jib, Henryk, youenn, dom, chairs are volunteering
<dom> +Karthik
<lgrahl> Would volunteer for data channel stuff
youenn: supporting the idea of adding test information for every PR.
<dom> Test volunteers (updated): jib, Henryk, youenn, dom, chairs, Karthik, lgrahl (data channel)
lgrahl presenting slide 73
<dom> More Tests for WebRTC Data Channels #13499
slide 74
WPT test review process seems too long
lgrahl showing PR issues
<dom> (removing white spaces changes reduces the diff to 4339 additional lines, 987 removal https://github.com/web-platform-tests/wpt/pull/13499/files?w=1)
youenn: also mention that review
might not need to be always very high. We can always fix later
on, need to have a good balance
... will look at the PR.
Soares presenting WPT issues July -Oct 2018
Soares presenting wpt.fyi results
Soares presenting coverage status slide
Not much progress since TPAC 2017
Henryk: TODO?
Soares: Can be tested but no test
yet.
... "separate tests by spec" slide
dom: WPT idea is to have one folder per spec.
Harald: can you take an action to create directories?
Dom: can do that but somebody to move the tests.
youenn: can do that.
<dom> https://github.com/web-platform-tests/wpt/issues?q=is%3Aopen+is%3Aissue+label%3Awebrtc
dom: automated labelling based on updated file folder
Soares: "WPT: Some things remain difficult to test automatically" slide
DrAlex: "Specific WebRTC Testing
Issues" slides
... "GetUserMedia Permission prompt"
... not sure about WebDriver support for Firefox, but Chrome,
Safari and Edge have it
youenn: Safari supports a proprietary value for now.
jib: no support yet in Firefox
DrAlex: "Media creation" issues
WPT tests have been rewritten to use WebAudio/Canvas for generating media from JS
youenn: discussion in WPT to use internals API during TPAC
DrAlex: "WPT automation"
slide
... "WPT automation & KITE WPT test" slide
... KITE is advanced in terms of configuring browsers compared
to WPT, example H264 support in Firefox
Hopefully, we can improve WPT so that wpt.fyi results are more meaningful
DrAlex: WPT is about API
interop/conformance. KITE is about browser interop (Firefox
<-> Safari e.g.)
... "KITE Interop SE Grid" slide
... "KITE 2-clients, beyond browsers" slide
... "KITE simulcast" slide
... standalone test for simulcast
... SFU echoing stream to display selected stream in same
page
... "KITE Simulcast Update"
youenn: WebKit, probably Chrome, has a LayoutTest for simulcast that runs in CI and gives some simulcast regression testing
DrAlex: "KITE Update 06/2018"
slide
... joint effort with callstats.io
Validated results for various browsers
Varun: Initial good results but a lot more work to be done to validate browser implementation
DOM: coverage?
Varun: audio and video is well covered, not data channel
DrAlex: network instrumentation
very useful to validate stat provided values
... "WebRTC Testing: INTEL contributions" slide
Jianju: some part overlap with KITE. some cover video quality
DrAlex: 'WebRTC testing:
scientific publications (1/2)
... slide
... we have a system to fully test simulcast efficiency/SFU
evaluation
... no switching in any such case but the tool allows adding
more scenarios
... 'WebRTC testing: scientific publications (2/2)' slide
Bernard presenting
Bernard: summarizing PR
requirements
... "WebRTC Issues" slide
Possibility to reach zero issues in 18 months at the current rate
Bernard: potential area of issues is Simulcast
Bernard asking the WG about whether considering hackathon or virtual meetings devoted to simulcast.
Varun: agreed that specific efforts need to be done in that area, simulcast stats as well
DrAlex: need to get the people
from browsers and SFU around the table to estimate the height
of the montain
... a task force would be good to create there
Varun: simulcast is not WebRTC1.0?
Harald: no, Bernard tried to
split simulcast from webrtc pc but was not improving the
situation
... simulcast is in WebRTC 1.0
Bernard: question to DOM: can we get members and non members in the same room?
dom: question will be whether
they will provide substantive contributions, in which case IPR
commitment will be needed.
... If they only contribute problem, probably not a big
issue.
... Otherwise will need IPR commitment.
Bernard: might be hard for some
people to get commitment from their companies
... simulcast is implemented but not standard. Machinery is
there but API is not.
Dom: concern that it will delay a lot of this spec
Harald: Would need to have all issues on table to seriously work at IETF hackathon in March
Bernard: Do an interim sometimes early January?
Resolution to have virtual interim in January and hackathon in March on simulcast
Bernard: WPT/WebRTC status
... "Simulcast Playground"
... "Testing simulcast in-browser" slide
... You can basically look if the action of the parameters
controlling simulcast actually work
... maxBitrate for instance
DrAlex: not working for SVC.
Bernard: yes, will talk about SVC
tomorrow
... no need for SFU in that tet
... just testing whether simulcast is doing what it should,
starting point
... "How?" slide
... based on SDP munging. Issue is that Safari/Chrome/Firefox
do not have the same SDP
Harald: not signal mid/rid in SDP?
Bernard: no MID header. nobody sends RID today.
Harald: Interop issue with mid?
Bernard: not with header
extension
... but with actual content of the mid
... 'Why?' slide
... getting started is important. Good to identify bugs early
on.
... 'Confluence Status' slide
Florent: 'Simulcast playground'
we built it with Chrome and it is possible
... Munging is for the remote to split simulcast streams
DrAlex: concerned about using the code as is as not standard.
Florent: good for webrtc sample
Henryk: the more we can test in WPT, the better.
Jib: ditto
Sangwhan: (from the TAG)
... why are the test results so red?
Bernard: combination of things:
not everything is implemented and dependencies.
... Lot of little details, infra issues...
... Edge implements a lot of APIs but fails these tests due to
dependencies.
youenn: might be good to have a hackathon related to WPT tests
dom: March Hackathon is about that, not only for simulcast
Bernard: showing confluence
results
... confluence allows to check for "below 2 implementations"
bar
Slide listing these.
Lot more green in confluence than WPT
Bernard: we can remove things but we need to not do that at the last minute.
dom: optional is not going to fly. It should be either remove or implement
Bernard: ICE transport is rewritten but was only implemented in edge.
Move back would take a lot of time.
jib: we plan to implement transport objects but it is low priority
Varun: IceTransport is a good example of why we should not go back. For debugging, this is very useful.
Bernard: Kill SCTP transport?
jib: we plan to implement all transport objects.
sangwhan: is there single-implemented feature that is controversial? Would be good to remove these.
Bernard: simulcast is not controversial in terms of using, but in terms of the amount of work.
Dom: we split IDP for that
purpose.
... Edge is the sole implementer of some of these features
(transport objects) but not controversial.
Bernard: how to measure interoperability?
Harald: Using adapter.js is not good for interop proof
HenryK: first step is making sure we know what each browser is implementing, then go to WPT.
dom: confluence is basically the first step.
Bernard: issue might be that implementors will spend all their time in simulcast.
Harald: Chrome is planning to be fully WebRTC 1.0 spec compliant before PR
Bernard: same target for Edge.
dom: with the timeline.
jib: same plan
<dom> ScribeNick: alexg__
<DrAlex> webrtc Next Version
<dom> ScribeNick: DrAlex
<caribou> scribenick: DrAlex
we are at the requirement phase, as described in the linked document
<dom> WebRTC Next Version Use Cases
the key point was: is it a use case that interest enough people so that people will come up and participate.
is there existing implementations that can give us insights
first parts of the requirement document are improvements, which are dangerous proposal that needs to have enough added value on existing use case to generate interest
there are only 3
multiparty online gaming with voice chat
mobility
and video conferencing with a central server
described in sections 2.1, 2.2, and 2.3
respectively
second part is about new use cases
mainly file sharing and IoT
they are described in sections 3.1 and 3.2
questions remains: is there a real world demand for webrtc data exchange in low power IoT device
as well as Use cases coming from the Native world
funny hats (3.3), machine learning (3.4), VR games (3.5), Secure comms (3.6)
<dom> WebRTC NV Use Cases issues
<lgrahl> o/
<dom> dom: +1 on adopting this and publishing it as a FPWD
<dom> youenn: +1 as well
adopted
the document is adopted as a working group document
RESOLUTION: NV Use cases document adopted as a WG document
<dom> jeff_jaffe: we should also think about it holistically e.g. in terms of the next rechartering and how to package these items
bernard: you have to think in these terms: is it really something that someone would pay you to do
hta: make sure it s possible: would yo use able to do it if someone was paying you to do it
:D
performance challenge of OpenCV.js video pipeline by INTEL
in collaboration with UC Irvine
bernard: what is the difference with tensor flow for example?
<armax> If I may, I'd like to come too to dinner (make it 15 basically)
intel: tensor flow is more GPU,
onpcv CPU, and openCV is based on Web ASM
...: I will not go through all tutorials here today, but just
highlight some.
we have some basic image processing, blurring, canny edge detection
and also video analysis for example.
(face recognition)
<armax> youenn: if you saw my chat, would be great :)
object recognition and tracking needs to be done real-time, and it poses some challenges.
<youenn> armax: Haven't seen it...
<armax> I had to check to uneroll from the meetup first. If I am too late is fine, I may diet :P
<youenn> armax: I tried to reserve a first restaurant, but it is full so need to find one more.
<youenn> armax: want to join as well, so 15 I guess?
<armax> yep
<armax> great :)
example video I/O using webrtc in pence.js
opencv.js
too late for me
<armax> +1 for 9pm being fine (I might be able to keep awake :P )
<youenn> ... for 15
ok
30+ mn by feet from cite internationalle
through the park
<soares> scribenick: soares
jib: other than gc, are there any performance issues
youenn: can we compare the time for pipeline creation vs processing time?
<jib> jib got answer: video copies
youenn: we are not even doing something meaningful. If we are doing meaningful processing can we do it in HD?
ningxin: OpenCV native we can do it
jib: Firefox get OverConstrained error for your demo
Peter presents WebRTC-ICE
peter: 2 purposes: 1. NV style
RTCIceTransport; 2. FlexICE - more ability to control
...: previous f2f - consensus on both in general
... PR webrtc-ice#22
... Implemented in Chrome (NV-style) and Edge
(ORTC-style)
... Next step: finish implement NV-style, data channels
..: FlexICE - is there an urgent customer?
bernard: game developers
harald: status of WPT?
... status of TAG review?
peter: main question for TAG is
feasibility of RTCIceTransport independent of
RTCPeerConnection
...: FlexICE is still unclear
harald: better to have TAG
formally review it
...: for WPT, if you don't know how to write a test, this might
cause issue to the spec
jib: it is still immature to assess the value of webrtc-ice
peter: by itself it is worthless,
but with quic stream etc it will be valuable
... we already have concensus on NV-style and FlexICE
...: most things added in PR #22 applies to both just
fine
... In order to add more features, it need to be an interface
instead of dictionary in webrtc-pc
... I avoided adding methods to Candidate
peter: workaround is to have
object containing additional thing
... FlexICE is orthogonal to NV-style
...: we cover the ICE improvement NV use cases
peter: fork/clone don't really
apply to ICETransport from RTCPeerConnection
...: should disallow ICE transport from RTCPeerConnection from
forking
harald: concensus for Q1: yes
bernard: forking have effects on higher level objects such as DTLS as well
jib: what is FlexICE?
peter: methods added in PR
#22
... for controlling/adding ice candidates and pairs
harald: now I know how ICE candidates can disappear as discussed previously
peter: all have straightforward implementations except for calling nominate more than once
bernard: for ICE transport with pc, what happen with remove candidate?
???: probably ICE restart
peter: calling forks multiple times have different candidates with distinct passwords
bernard: when calling fork, what
does it exactly forking?
... potentially problematic
peter: we got to a point where it is ready to implement, however not clear whether we should implement all
vr000m: can solve issues for ICE restarts
harald: confirm we have more
customers for webrtc-ice, except perhaps fork
... we have use cases for fork, do we have enough use cases for
implementation
bernard: mobile gaming is
everything. fork is just for joining another game
...: can do a lot of stuff without forking. can do a few stuff
better with forking
bernard: best to ask CDN people
dom: concerns about privacy
peter: ORTC already have network information interface. should we use it or go with our own?
harald: conclusion is we will use
network information defined elsewhere
... lack of information for Q2 and Q3