Meeting minutes
Slideset: https://
WebRTC Event Log uploads with the Reporting API
<Zakim> dom, you wanted to ask about spec protocol buffers, alternative of a different reporting mime type
youenn: re protocol buffer, there are conversion algorithms to JSON we could reuse
dom: how well spec'd is the profocol buffer format?
guido: it would have to be part of that new spec, but it has been stable in libwebrtc
dom: rather than hacking the binary string into JSON, we could see if adding another format to the reporting API would be acceptable
Youenn: one use case for JSON is testing (weak), and exposing browser log through the web inspector - having it in JSON would make it easier
Guido: the advantage of keeping the protocol buffer is the compatibility with existing native apps and tools
Jan-Ivar: mozilla's position on the reporting API is positive (although still behind a flag)
… on the API shape, rather than 3 methods, could we add it as configuration option to PC?
Guido: for enabling, sounds good; it would still be good to keep stop/discard
dom: I'm not sure i understand the no-upload option
Guido: it's to use it in a local version
dom: but then that should be left to the developer tool rather the API
Harald: for additional methods, there could be a Log object that could expose the additional methods
dom: note if we have a boolean member, it can't default to true
Youenn: re content of event log, we should make sure privacy safety
… compare it to existing information exposed today
… maybe the asynchronous nature makes it better
… we should ask a review from the PRivacy WG after having identified which new information is available
Guido: I don't think there is anything new, except timing (which can also be detected by the app)
… Chrome Privacy hasn't found issues atm
Youenn: Binary format is worth discussing
… any impact on performance?
Guido: not significant
dom: including on mobile?
Guido: don't expect so - native apps use it on mobile
Youenn: there may be value in allowing to discard some data in case perf is detected as problematic
… can you share more about where this is already used?
Guido: it is in a Chrome extension API (which we're removing)
jan-ivar: the other example of the reporting API feel more like smaller data set
… are we sure a webrtc event log would be an adequate fit?
guido: these would be larger than typical reports, but given that they can be uploaded out of band, it shouldn't be a problem
RESOLUTION: Overall support to explore this, with further discussion on privacy, performance and API shape
Guido: the API would be added to WebRTC extensions, and a separate spec for protocol buffers
Carine: any change we expect from Web Perf on reporting API?
Guido: we'll ask them to review the report spec
SFrame
Improving SFrameTransform API
dom: type required vs default?
youenn: if we can converge on a default, sure
jan-ivar: LGTM
… re default value, no strong opinion; we can always start with required, and see later for a default
harald: the crypto API allows you to construct key from JS and manage keys that JS can't see
… do we want to impose restrictions on which kind of key that can be used in this?
Youenn: the end goal would be if we have an MLS API, they would be integrated into this
… even if the MLS API is added, it might be reasonable to keep JS-visible APIs in this
Harald: we could use the same trick as SRTP and derive a key from DTLS
… this could even be the default
Youenn: this could be an option
… there are sframe users that want control on key derivation, so we should probably support both approaches
… These objects won't change the API key content (except if we use HArald's idea)
… the web app will responsible for deriving keys, ideally using WebCrypto
… there is already a spec to derive an sframe key from an MLS key
RESOLUTION: rough consensus to proceed with the new proposed API shape, proceed with PR
SFrame + RTCRtpScriptTransform
Youenn: I Think we should wait we get a request for this use case, this is mostly a proof of concept to evaluate feasability
Youenn: likewise, mostly looking at feasability, not planning to go there at this time
jan-ivar: another approach might be to have a separate interface
Youenn: right, API shape can be discussed
hta: in a packet case, you'll have 4 or 5 packets input, and 1 frame out in the decoder
… and vice versa for the encoder
Youenn: the packetizer is responsible for sending multiple packets
… we're talking about an SFrame frame
… on the receiver side, a frame would be the equivalent to a packet, send it to the depacketizer
hta: if we don't need the restriction we currently have, let's remove it
… but I Like this overall direction
Encoded Source API update
Peter: this is great! esp the constructor
… looking forward to the implementation
Youenn: we should probably not replicate the scripttransform api which has shown a bit problematic
… (for the encodedsource constructor)
… the encodedsource object in the worker has some APIs - we've discussed whether to have them in scripttransform
Guido: indeed, that's where they come from
Youenn: right - they might be worth factoring out in a mixin interface
Youenn: on the receiverside, there would be a jitter buffer tied to the peerconnection
… would this be useful as a separate object from peerconnection
… if so, what new API would the app be needed e.g. on setting the jitter buffer, if not too complex?
… re WebCodecs for rtcencodedsource objects - we discussed this in the joint meeting with media last year
… maybe we should talk about this with them
… this could be a constructor, transferable
… either tomorrow, or later today
… what do synchronous valid checks mean when enqueuing a frame?
Guido: some could be asynchronous since write is async anyway
… this shouldn't make a difference
jan-ivar: where to file issues?
Guido: webrtc-extensions
… or encoded-transform
Jan-Ivar: there are 3 ways to get something in a worker: postMessage, stream, scripttransform - this would be a 4th way
Guido: we could do something equivalent to transform, but since there are no properties, it's simpler to do the association directly; but we coudl emulate scripttransform
RESOLUTION: rough consensus to make proposals in that direction
Camera intrinsics attributes
Rik: (Meta, Oculus browser)
Guido: a short explainer to explain why it's needed would be a good first step, with the various entries
Youenn: this should come to mediacapture-extensions
… another entry point could be in MediaDeviceInfo for constant values
hta: does this only make sense if the camera and the screen are locked together?
… we also have pan/tilt/zoom
… it seems a bit special-purpose only when you have a hardware-coupled camera
Rik: we could look at how android deal with pan/tilt/zoom
Jan-Ivar: I like the idea; MediaTrackSettings is part of the constrainable patterns
… we need to understand the impact e.g. on getCapabilities
… maybe a bit more discussion on API shape
Rik: the sequences are fixed length, I'm not sure we can express in WebIDL
… I can put more details in the explainer
youenn: if we don't want or need applyConstraints, then mediadeviceinfo feels like a better approach
Dom: worth doing the comparison of the two approaches in the explainer
ErikS: with pan/tilt/zoom, it changes the field of view
Dom: also worth addressing the explainer how it relates to pan/tilt/zoom in mediacapture-image
Youenn: overall, no objection in the direction
rik: this question applies to all other devices with a similar set up exposed to web developers
Mediacapture-transform and track transfer
MarkusH: what is the good use case for transfering the track to the worker?
jan-ivar: a track can be genreated from videocodecs, combine things
… also want to avoid duplicate API that achieve the same things
Youenn: the usual way of doing exchanges between window and worker is postMessage
… for datachannels, when we wanted to process things in the worker, there were 2 options: create a PC in the worker (feasible, but not done)
… or transfer the channel to the worker - simpler and useful
… you can use window to worker, or out of a process (e.g. via sharedworker)
… for MST, it seems natural to re-use that pattern
… we had received use cases from Google to have a track generated in an iframe to another cross-origin context, which can only be done via transfer
… which would be hard to do elsewhere
Guido: transfering from a window to a window is not the same as window to worker
… data channels are data objects
… the objects with data is the stream, not the track
… tracks are the control plane
… forcing to move tracks to the worker to use the feature as it makes much more complicated
… what we're proposing is to re-use a pattern we've used in the past
[discussing conformance of Chrome implementation with stream spec requirements]
youenn: do we have signals of major customers saying they can't use the spec'd approach?
jan-ivar: the shim I provided should help with adoption
hta: fixing the wrong model with a shim is not the right approach
jan-ivar: we're the only one with tight coupling with worker
guido: audio does it for worklet
dom: let's see if we can get developer signals
youenn: how is Google Meet adopting to the split situation?
MarkusH: I can checkout
<dom> s/Subtopic: SFrame + RTCRtpScriptTransform|Subtopic: SFrame + RTCRtpScriptTransform
|[slide 60]|<dom> Related issues: w3c/