W3C

– DRAFT –
Evolved Video Encoding with WebCodecs

25 September 2024

Attendees

Present
-
Regrets
-
Chair
Erik Språng, Eugene Zemtsov
Scribe
PeterThatcher

Meeting minutes

I'm taking notes

[slide 1]

[slide 2]

[slide 3]

[slide 4]

[slide 5]

[slide 6]

[slide 7]

[slide 8]

[slide 9]

[slide 11]

[slide 12]

[slide 12]

Peter: Is there an LTR example?

Erik: Not yet

Peter: How many buffers can I have?

Erik: AV1: 8, VP9: 8, H265: 32, H264: 16, VP8: 3

From many: discussion of making the API be done a different way, using EncodeVideoChunk or some other object

?: What are the benefits vs. scalability modes

Erik: If you come up with something new, you have to go through a standards body, which is painful for what would be one page of code

Benard: a lot more recovery options available with this API

[slide 10]

?: About the example

?: What about things that receive feedback from the other side could be provided?

Erik: This is just the encoder, but you could implement feedback from the other side and then decide things, such as for an LTR

Bernard: RPSI is an example of "I hold this reference", and people have implemented it for the 1:1 case

Bernard: but it doesn't seem to work for a conference

Bernard: Google was working on "LTF" (?)

Lennox: Might be nice to say "hey this is LTR" on the decode side (?)

Lennox: what is the perf on running JS on every frame?

?: Not much compared to encoding

?: You should probably use a Worker

Lennox: B-frames?

Erik: Technically, you can do that if you have side channel info

Erik: But the API would need an extension to be "spec compliant" B-frames

?: Can do B-frames "manually"

Peter: So how does this work?

?: You use buffers[i] instead of i, but you basically have a fixed number of buffers you can save to and reference from

Discussion: Easier to update JS than browsers, but still might need OS updates to get hardware encoder support (?)

[slide 13]

[slide 18]

Elad: Your view seems convincing. Did you consult privacy experts?

Erik: Not yet.

Bernard: Privacy experts do not know about encode/decode

Bernard: Regarding choices: my problem with returning capabilities is that I formulate a query to get a "no", so I like the idea of enumerating

Bernard: Maybe we block enumeration behind some permission

?: Have you looked into MediaCapabilities?

Paul: Might as well extend MediaCapabilities if you're going to extend something.

Erik: Yes, we should look into that

[slide 20]

Paul: One callback or multiple callbacks?

Erik: Probably serveral

Erik: Simulcast doesn't matter; SVC has to be in the right order

Lennox: How does this work with "manual?"

Erik: You would set things differently for each layer

Lennox: need control per layer about quality

? VideoEncoderEncodeOptions that always has QP value, so you can set it

Bernard: what about bitrate?

Erik: We're going CQP route

Erik: target bitrate is much more complex with time, timestamps, etc

Peter: How do I control inter-layer dependencies?

Erik: Use the buffers from the "manual" control from before

Peter: What is available behind the feature flag?

?: Everything from [slide 8]

?: For hardware support: have to wait

?: Anything beyond [slide 8], we don't know yet

Peter: Can you upload examples/demos somewhere?

Erik: Yes

<handellm> generate minutes

<handellm> zakim generate minutes

Minutes manually created (not a transcript), formatted by scribe.perl version 235 (Thu Sep 26 22:53:03 2024 UTC).