22:26:26 <RRSAgent> RRSAgent has joined #mediawg
22:26:26 <RRSAgent> logging to https://www.w3.org/2022/09/15-mediawg-irc
22:26:33 <Zakim> Zakim has joined #mediawg
22:26:54 <dom> Slideset: https://lists.w3.org/Archives/Public/www-archive/2022Sep/att-0008/TPAC_2022_WebRTC_WG___Media_WG___MEIG_Joint_Meeting.pdf
22:27:00 <alwu> does anyone know what the passcode is for the webRTC joint meeting? thanks.
22:27:00 <bneedham> bneedham has joined #mediawg
22:27:15 <sandersd> sandersd has joined #mediawg
22:27:17 <cpn> present+ Chris_Needham, Paul_Adenot, Dominique_Hazael_Massieux, Francois_Daust, Hisayaki_Ohmata, Bernard_Aboba
22:27:17 <bneedham> present+ Bradley_Needham
22:27:59 <cpn> present+ Jake_Holland
22:28:00 <fluffy> fluffy has joined #mediawg
22:28:08 <fluffy> present+ Cullen Jennings
22:28:18 <cpn> present+ Kaz_Ashimura
22:28:20 <TuukkaToivonen> TuukkaToivonen has joined #mediawg
22:28:43 <cpn> present+ Mike_English, Alastor_Wu, Dan_Sandsers
22:28:49 <fluffy> present+ Cullen_Jennings
22:28:54 <cpn> s/Dan_Sandsers/Dan_Sanders/
22:29:14 <cpn> Present+ Tuukka_Toivonen, Pater_Thatcher
22:29:30 <nigel> nigel has joined #mediawg
22:29:50 <kajihata> kajihata has joined #mediawg
22:30:07 <cpn> Present+ Youenn_Fablet
22:30:16 <cpn> Present+ Tatsuya_Igarashi
22:30:36 <herre> herre has joined #mediawg
22:31:21 <cpn> Present+ Brian_Baldino, Daisuke_Kodakima
22:33:06 <dom> Present+ Eric_Carlson, David_Singer, Elad_Alon
22:33:08 <dom> scribe+
22:33:14 <tguilbert> tguilbert has joined #mediawg
22:33:19 <kajihata> present+ Hiroshi_Kajihata
22:33:21 <eugene> eugene has joined #mediawg
22:33:25 <ken> ken has joined #mediawg
22:33:25 <ohmata> ohmata has joined #mediawg
22:33:27 <tguilbert> present+ Thomas_Guilbert
22:33:42 <gendler> gendler has joined #mediawg
22:33:46 <jib> jib has joined #mediawg
22:33:47 <gendler> present+
22:33:58 <Orphis> Orphis has joined #mediawg
22:34:02 <Orphis> present+
22:34:08 <youenn_> youenn_ has joined #mediawg
22:34:10 <eugene> present+ Eugene Zemtsov
22:34:12 <herre> present+ Tony_Herre
22:34:16 <ken> present+
22:34:17 <bialpio> bialpio has joined #mediawg
22:34:17 <ohmata> present+ Hisayuki Ohmata
22:34:19 <klausw> klausw has joined #mediawg
22:34:19 <youenn_> present+ Youenn Fablet
22:34:22 <jholland> jholland has joined #mediawg
22:34:30 <TuukkaToivonen> present+ Tuukka Toivonen
22:34:31 <klausw> present+ klausw
22:34:32 <jib> present+ Jan-Ivar Bruaroey
22:34:34 <jholland> present+ Jake_Holland
22:34:38 <klausw> present+ Klaus Weidner
22:34:38 <englishm> present+ Mike_English
22:34:42 <bialpio> present+
22:34:45 <dom> Topic: Introduction
22:34:53 <dom> [slide 5]
22:34:53 <eric-carlson_> eric-carlson_ has joined #mediawg
22:34:56 <eehakkin> eehakkin has joined #mediawg
22:34:58 <dsinger> dsinger has joined #mediawg
22:35:01 <eric-carlson_> +present
22:35:06 <dom> Bernard: some context for this joint meeting
22:35:08 <eric-carlson_> present+
22:35:09 <dom> [slide 6]
22:35:14 <eeeps> eeeps has joined #mediawg
22:35:21 <dsinger> present+ dsinger
22:35:22 <riju_> present+ rijubrata bahumik
22:35:24 <dom> Bernard: the pandemic has brought a new wave a technology to the mass market
22:35:30 <eehakkin> present+ Eero_Häkkinen
22:35:34 <eeeps> present+ Eric Portis
22:35:53 <dom> ... podcasting (75% of China once/week), video conferencing, video streaming (vod, live streaming)
22:36:07 <dom> ... ...
22:36:24 <hfujisawa> hfujisawa has joined #mediawg
22:36:41 <dom> ... a bunch of these are being compiled in the webrtc-nv use cases and webtransport
22:37:00 <dom> ... they blur the lines between streaming and realtime
22:37:09 <dom> [slide 7]
22:38:01 <dom> [slide 8]
22:38:35 <dom> RRSAgent, draft minutes
22:38:35 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/15-mediawg-minutes.html dom
22:38:39 <dom> RRSAgent, make log public
22:38:42 <dom> [slide 9]
22:40:00 <igarashi_> igarashi_ has joined #mediawg
22:40:14 <igarashi_> present+ Tatsuya_Igarashi
22:40:27 <dom> [slide 10]
22:41:07 <kaz> kaz has joined #mediawg
22:41:30 <kaz> zakim, who is on the call?
22:41:30 <Zakim> Present: Chris_Needham, Paul_Adenot, Dominique_Hazael_Massieux, Francois_Daust, Hisayaki_Ohmata, Bernard_Aboba, Bradley_Needham, Jake_Holland, Cullen, Jennings, Kaz_Ashimura,
22:41:33 <Zakim> ... Mike_English, Alastor_Wu, Dan_Sandsers, Cullen_Jennings, Tuukka_Toivonen, Pater_Thatcher, Youenn_Fablet, Tatsuya_Igarashi, Brian_Baldino, Daisuke_Kodakima, Eric_Carlson,
22:41:33 <Zakim> ... David_Singer, Elad_Alon, Hiroshi_Kajihata, Thomas_Guilbert, gendler, Orphis, Eugene, Zemtsov, Tony_Herre, ken, Hisayuki, Ohmata, Fablet, Toivonen, klausw, Jan-Ivar, Bruaroey,
22:41:33 <Zakim> ... Weidner, bialpio, present, eric-carlson_, dsinger, rijubrata, bahumik, Eero_Häkkinen, Eric, Portis
22:41:52 <eugene> present+ Eugene_Zemtsov
22:42:03 <dom> [slide 11]
22:42:03 <jib> present+ Jan-Ivar_Bruaroey
22:42:54 <dom> Bernard: lots of SdOs involved, and even lots of different groups involved on these topics in W3C itself
22:43:34 <dom> ... as a developer, you have to combine all these APIs together to build an app
22:43:52 <dom> ..; the webcodecs sample code illustrate that as they need to use many different of APIs
22:43:54 <dom> [slide 12]
22:44:01 <kaz> s/..;/.../
22:44:13 <kaz> rrsagent, make log public
22:44:18 <kaz> rrsagent, draft minutes
22:44:18 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/15-mediawg-minutes.html kaz
22:45:08 <dom> [slide 13]
22:45:44 <yonet> yonet has joined #mediawg
22:45:56 <dom> Bernard: the webrtc worked with WHATWG to make the WHATWG streams work better for media use cases
22:46:23 <dom> ... converting video frames from WebCodecs into other formats is another need that has emerged
22:47:02 <dom> ... the support for worker thread for the pipeline is still inconsistent
22:47:35 <kaz> meeting: Web Real-Time Communications Working Group and Media Working Group and Media and Entertainment Interest Group - Joint meeting
22:47:39 <dom> ... separating worker threads for send and receive pipelines is not easy to put in place in practice
22:47:46 <dom> [slide 14]
22:48:32 <dom> Bernard: goal is to identify owners of the identified problems and next steps
22:48:49 <dom> q?
22:48:58 <cpn> q?
22:49:49 <dom> [slide 15]
22:50:01 <kaz> q+
22:50:15 <dom> q?
22:50:20 <dom> ack kaz
22:50:56 <dom> Kaz: Web of Things are thinking of using WebRTC for use in video surveillance cameras
22:51:10 <dom> ... I wonder about security considerations for these types of applications
22:51:38 <dom> Youenn: does this need DRM?
22:51:55 <dom> Kaz: not necessarily, but the cameras would still want to protect their data
22:52:01 <fluffy> q+
22:52:30 <dom> youenn: WebRTC provides hop-by-hop encryption
22:52:32 <Kodajima> Kodajima has joined #mediawg
22:52:44 <dom> ... complemented by end-to-end encryption done in the encoded content
22:53:01 <dom> ... that may be something to consider here as well
22:53:14 <dom> bernard: please file an issue for use cases
22:53:29 <dom> -> https://github.com/w3c/webrtc-nv-use-cases/issues WebRTC NV Use Cases Repo
22:53:31 <dom> q?
22:53:36 <dom> ack fluffy
22:53:38 <kaz> ack k
22:54:27 <dom> fluffy: it would be important to have support for end-to-end encryption - beyond JS-based encryption, it would be good to re-use our existing infrastructure (certificates management, webauthn) to enable E2EE
22:54:33 <cpn> q?
22:54:34 <dom> ... probably anchored into MLS
22:55:06 <dom> Bernard: would that be separate from WebRTC then?
22:55:10 <dom> Cullen: yes
22:55:20 <dom> ... we have a much better understanding of what's needed
22:55:47 <dom> ... we need to make sure our architecture exposes API in the pipeline where the browser doesn't necessarily have access to the underlying data
22:56:06 <dom> Bernard: there is a related issue in the webcodecs repo, filed in the DRM context
22:56:07 <kron> kron has joined #mediawg
22:56:08 <dom> q?
22:56:54 <cpn> scribe+ cpn
22:57:27 <cpn> Dom: Code samples is a good way to identify where the gaps are
22:57:50 <cpn> Action: Set up a place where we can discuss cross-group or cross-API architecture issues
22:58:31 <dom> Topic: Issue 90/Issue 121: Pluggable codecs
22:58:45 <dom> -> https://github.com/w3c/webrtc-encoded-transform/issues/90 Adopt feedback streams in RTCRtpScriptTransformer #90
22:58:54 <youenn> youenn has joined #mediawg
22:58:55 <dom> -> https://github.com/w3c/webrtc-encoded-transform/issues/121 Need APIs for using WebCodec with PeerConnection #121
22:59:08 <dom> [slide 16]
22:59:41 <dom> Harald: once you take a media and you transform it, it no longer has the same format, which is problematic for SDP-based negotiations
22:59:53 <dom> ... this creates issues where the pipeline is configured for the wrong media
23:00:13 <gkatsev> present+ Gary_Katsevman
23:00:34 <dom> ... one property that we need from the communications machine for webrtc is the ability to speak truth that the content being sent doesn't conform to the spec
23:00:58 <dom> ... there is also an issue around packetization/depacketization
23:01:37 <peter> peter has joined #mediawg
23:01:48 <dom> ... we should figure an API so that once you install a transformation in the pipeline how to handle the data in the communications machinery
23:02:05 <dom> ... that should operate in terms of codec description
23:02:28 <kaz> rrsagent, draft minutes
23:02:28 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/15-mediawg-minutes.html kaz
23:02:28 <dom> [slide 17]
23:02:53 <dom> Bernard: this is key to enabling to combine webcodecs & webrtc
23:03:01 <dom> ... also needed for E2EE
23:03:07 <dom> s/[slide 17]//
23:03:25 <dom> Harald: this problem has indeed arisen already in the context of E2EE
23:03:38 <dom> ... also, should this be done in terms of packets or in terms of frames
23:03:38 <kaz> s/Hisayaki/Hisayuki/
23:03:55 <kaz> chair: cpn
23:04:06 <cpn> q?
23:04:16 <dom> Bernard: in terms of next steps, the idea is to sollicitate proposals?
23:04:57 <dom> Youenn: re WebCodecs, the traditional view is that if you're using webcodecs you will use web transport or datachannel to transport the data
23:05:10 <dom> ... what's the use case for using webcodecs inside something very integrated like peerconnection
23:05:36 <dom> ... it's good to provide proposals, but I would focus on use cases and requirements
23:05:52 <dom> Bernard: the key reason is that niether datachannels nor webtransport will generate low latency
23:06:07 <dom> ... getting to 100ms can only be achieved with WebRTC
23:06:16 <kaz> present+ Harald_Alvestrand, Alex_Turner, Brandon_Jones, Brian_Baldino, Chao_He
23:06:17 <dom> Youenn: I agree with that
23:06:32 <dom> ... one of the main benefits of WebCodecs is for the use of hardware encoder/decoder
23:06:45 <dom> ... when you expose that in WebCodecs, we should expose it in PC
23:06:57 <kaz> present+ Dale_Curtis, Dan_Sanders, Eero_Haekkinen
23:07:02 <dom> ... the main benefits would be to finetune encoders/decoders behavior
23:07:10 <eeeps> eeeps has joined #mediawg
23:07:17 <dom> ... not clear what the benefits would be vs native encoder/decoder integration in PC
23:07:27 <kaz> present+ Eric_Portis
23:07:30 <dom> Harald: in a browser, the code supporting the codecs for the 2 are the same
23:07:38 <dom> ... but the capabilities in the 2 APIs aren't
23:07:55 <kaz> present+ Hiroshi_Fujisawa, Hiroshi_Kajihata
23:08:00 <dom> ... the most intensive control surface would be in WebCodecs rather than in WebRTC
23:08:08 <peter> q+
23:08:16 <jib> q+
23:08:16 <dom> ... you control the interface via WebCodecs with an input/output in RTP
23:08:21 <eugene> q+
23:08:23 <dom> ... this requires supporting the integration
23:08:46 <dom> Bernard: WebCodecs hasn't been for long, but they're already exposing a lot more capabilities than WebRTC
23:08:47 <fluffy> q+
23:09:13 <kaz> present+ Johannes_Kron, Klaus_Weidner, Daisuke_Kodajima
23:09:16 <dom> Peter: +1 to Youenn that the same codecs are exposed in both WebRTC & WebCodecs
23:09:36 <tovep> tovep has joined #mediawg
23:09:54 <dom> ... but as we add new low level controls - would we want to add them in both places or in a single place?
23:09:57 <bialpio> present+ Piotr_Bialecki
23:09:57 <dom> ack peter
23:10:01 <dom> ack jib
23:10:02 <kaz> present+ Mike_English, Riju, Sung_young_Son, Thomas_Guilbert, Tuukka_Toivonen
23:10:22 <kaz> rrsagent, draft minutes
23:10:22 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/15-mediawg-minutes.html kaz
23:10:27 <dom> jib: we have multiple APIs that are starting to overlap in usages
23:10:36 <tovep> present+ Tove_Petersson
23:10:43 <dom> ... what might help would be to ask what we prefer this or that API
23:10:54 <dom> ... I'm hoping we can avoid "this API is better than that API"
23:11:05 <cpn> q?
23:11:16 <dom> ... what makes WebCodecs better? if it's more codec support, we can add more codecs to WebRTC
23:11:36 <dom> ... or for WHATWG stream support in WebTransport, that could be exposed in data channels
23:11:38 <kaz> present+ Peter_Thatcher
23:12:11 <dom> ... it may not be necessary to break open WebRTC to benefits from the RTP transport benefits
23:12:17 <dom> ack eug
23:12:17 <peter> q+
23:12:27 <kaz> present+ Tatsuya_Igarashi
23:12:36 <dom> eugene: it's been mentioned that RTCDataChannel and WebTransport are not truly optimized for RTC
23:12:42 <dom> ... can something be done about it?
23:12:53 <dom> ... A/V is nto the only application for real-time transport
23:13:22 <eugene> q-
23:13:27 <dom> Bernard: this is under discussion in a number of IETF WGs
23:13:30 <kaz> present+ Youenn_Fablet
23:13:36 <dom> ... some things can be done but it's still a research problem
23:13:55 <dom> ... in the WebTransport WG, we disucssed interactions with the congestion control algorithms
23:14:06 <dom> s/disucss/discuss/
23:14:18 <dom> ... not sure how much momentum there would be to change congestion control for datachannels
23:14:25 <cpn> q?
23:14:26 <dom> Harald: datachannels are built on SCTP
23:14:35 <dom> ... it's now easier to play with the congestion control for that
23:14:53 <dom> ... understanding congestion control is hard, improving it even harder
23:15:00 <dom> ... unlikely to be solved in the short term, in IETF
23:15:03 <dom> ack fluffy
23:15:48 <dom> fluffy: in terms of congestion control, ReNo CC can be moved to BBRv2
23:16:28 <dom> ... in terms of driving interest on WebCodecs, the next generation of codecs driven by AI are providing substantial improvements to this space
23:16:57 <dom> ... these are proprietary codecs that some would like to deploy at scale in browsers
23:17:14 <dom> Bernard: a lot of these technologies aren't that easy to do in WebCodecs
23:17:18 <dom> ... e.G. AI integration
23:17:29 <cpn> q?
23:17:33 <dom> ... pre/post-processing can be done with webcodecs
23:17:38 <cpn> ack peter
23:17:58 <dom> peter: re congestion control, there are something we could do at the W3C level by exposing more information about what's happening in the transport so that app can adjust what they send
23:18:05 <dom> ... I've volunteered to help with that
23:18:44 <dom> ... on the codecs side, re which API to use - even if we allow "bring your own codec" and an RTP Transport, there is still a question of the jitter buffer which is uniquely exposed in WebRTC (not in WebTransport)
23:19:29 <dom> CPN: additional control surface in WebCodecs?
23:19:48 <dom> Paul: we can add knobs in WebCodecs when they apply to everything
23:20:04 <dom> ... otherwise, we add knobs in struct in the registry
23:20:54 <dom> ... e.g. tuning for drawing vs real-life for video
23:21:00 <dom> ... or packet size for audio
23:21:19 <dom> q?
23:21:36 <dom> CPN: we welcome issues for additional control surface in the webcodecs repo
23:21:59 <dom> youenn: one of the things we discussed was asking higher quality in the area of the video whtere there is e.g. a face
23:22:34 <dom> Topic: Issue 131: Packetization API
23:22:38 <dom> [slide 17]
23:22:53 <dom> -> https://github.com/w3c/webrtc-encoded-transform/issues/131 Should we expose packetization level API to RTCRtpScriptTransform? #131
23:23:11 <dom> Youenn: rtcpeerconnection used to be a blackbox with an input and output
23:23:23 <dom> ... webrtc-encoded-transform opens it up a little bit
23:23:27 <dom> ... it works for some things
23:23:33 <dom> ... but it doesn't cover all use cases
23:23:39 <cpn> q?
23:23:43 <dom> ... we tried to identify use cases where there isn't good support
23:24:06 <dom> ... e.G. when adding redundancy for audio to make it more reliable over transport
23:24:32 <dom> ... but adding redundancy expose the risk of having packets being discarded
23:26:00 <dom> ... Giving more control on the generation packets for video frame would also help in the context of encryption (SPacket)
23:26:54 <dom> ... RTP packets come with RTP headers that aren't exposed directly - it could be nice to give r/w access to apps e.g. to improve the voice-activity header
23:27:35 <cpn> q?
23:27:36 <dom> ... I plan to gather use cases and requirements since there is some interest - hope others will join me
23:27:49 <dom> cpn: where would that be done?
23:27:51 <yonet> yonet has joined #mediawg
23:27:59 <dom> youenn: issue #131 is probably the right place
23:28:20 <cpn> q?
23:29:12 <Bernard> Bernard has joined #mediawg
23:29:26 <dom> fluffy: it's hard to know how to find where the discussions happen; I'm interested
23:29:52 <dom> bernard: this relates to the challenge of keeping track of work happening across all these groups
23:30:40 <dom> fluffy: the architectural discussion is whether packetization is part of the pipeline we need to consider in our cross-group coordination
23:31:04 <cpn> q?
23:31:20 <englishm> fluffy++
23:31:41 <dom> cpn: we need to seed our x-greoup repo with a description of where we are and where we want to go
23:32:09 <jib> q+
23:32:12 <dom> bernard: another possibility would be a workshop to help with these x-groups discussions
23:32:16 <jib> q-
23:33:17 <dom> jib: I played with the API which illustrated that packetization is both codec and transport specific
23:33:23 <eric-carlson> eric-carlson has joined #mediawg
23:33:43 <dom> ... is our quesiton "where should it belong architecturally?"
23:33:47 <dom> bernard: great question
23:34:13 <dom> jake: has anyone talked with IETF on the need to surface the MTU to the encoder?
23:34:28 <dom> youenn: the packetizer has the info on MTU
23:34:38 <dom> ... the question is whether the packetizer would give the info back to the app
23:35:19 <cpn> Action: Create initial architecture description
23:35:26 <dom> Topic: Issue 99 & Issue 141: WebCodecs & WebRTC
23:35:31 <dom> [slide 18]
23:35:48 <dom> -> https://github.com/w3c/webrtc-encoded-transform/issues/99 Encoded frame IDLs share definitions with WebCodecs #99
23:35:52 <Larry_Zhao> Larry_Zhao has joined #mediawg
23:35:58 <dom> -> https://github.com/w3c/webrtc-encoded-transform/issues/141 Relationship to WebCodecs #141
23:36:25 <dom> Youenn: webrtc-encoded-transform and webcodecs share some very similar structures, but with differences incl on mutability of some of the data
23:36:37 <dom> ... it's a little bit inconvenient we have these different models
23:36:59 <dom> ... but that ship has probably sailed
23:37:16 <cpn> q?
23:37:20 <dom> ... for metadata, we're thinking of referring back to webcodecs
23:37:38 <dom> ... this got support from the webcodecs involved in the WebRTC WG meeting on Monday
23:38:16 <dom> Topic: Issue 70: WebCodecs & MediaStream transform
23:38:29 <dom> -> https://github.com/w3c/mediacapture-extensions/issues/70 Issue 70: WebCodecs & MediaStream transform
23:38:35 <dom> [slide 19]
23:39:03 <dom> Youenn: media capture transform allows to grab frames, modify them, and repackage them as a mediastreamtrack
23:39:06 <peter> q+
23:39:08 <dom> ... this is based on the VideoFrame object
23:39:23 <dom> ... cameras nowadays are capable to expose information about e.g. face positions
23:39:32 <dom> ... it's cheap to compute and could be usefully exposed to Web apps
23:39:49 <dom> ... since this is tightly synchronized data, it would be good to package it with the video frames
23:39:56 <nigel> nigel has joined #mediawg
23:40:13 <dom> ... in our dsicussion on Monday, we agreed to have a metadata dictionary in VideoFrame
23:40:30 <dom> ... WebRTC would expose its set of metadata for e.G. face detection
23:40:48 <dom> ... we discussed the possibility to have webapp specific metadata e.g. through JS objects
23:40:56 <dom> ... this could be useful in the context of AR/VR
23:41:24 <dom> ... the initial PR won't go that far, but it should open the way to that direction
23:41:36 <dom> ... the first use case will focus on browser-defined metadata
23:41:58 <Bernard> q+
23:42:17 <peter> q-
23:42:21 <cpn> q+
23:42:21 <dom> ... these would be exposed as the top-level, and there would be a .user sub-object that would need to be serializable
23:42:28 <nigel_> nigel_ has joined #mediawg
23:42:33 <dom> q+ ada
23:42:52 <dom> Bernard: this has application beyond the use cases on the slide
23:42:54 <bialpio> q+
23:43:10 <dom> ... e.g. face detection could help encoders around faces vs blurred background
23:43:24 <dom> ack Bernard
23:43:44 <dom> ... we would need to consider whether the encoder should be expected to look at that metadata
23:43:47 <dom> youenn: +1
23:43:55 <dom> ack cpn
23:44:06 <dom> cpn: how common is face detection in the camera level?
23:44:10 <dom> eric: it's very common
23:44:39 <dom> youenn: we could also expose a face detection blackbox that would transform a camera feed and annotate it with face detection metaata
23:44:41 <Bernard> q-
23:44:48 <dom> ack ada
23:44:58 <dom> Ada: (Immersive Web WG co-chair)
23:45:15 <dom> ... some of of our WG participants are very intereesting in bringing more AR feature to WebRTC
23:45:22 <TuukkaToivonen> TuukkaToivonen has joined #mediawg
23:45:27 <dom> ... they do AR by running WASM on the camera feed from WebRTC
23:45:55 <dom> ... if you were to fire up the various AR systems on the devices, you could surface more metadata e.G. the current position of the device or solid objects in the environments
23:46:25 <cpn> q?
23:46:26 <kaz> q+
23:46:30 <dom> Youenn: very important we get the AR/VR input on this work
23:46:53 <dom> ... we're also thinking of exposing requestVideoFrame to expose some of these metadata as well
23:47:05 <ada> ada has joined #mediawg
23:47:18 <dom> -> https://github.com/w3c/webcodecs/issues/189  WebCodec as pass through to application metadata #189
23:47:29 <dom> bialpio: also from the IWWG
23:47:53 <dom> ... we also have a use case to integrate WebXR animation loop with video frames used in XR session
23:48:10 <dom> ... how would we correlate a video frame from gUM with poses?
23:48:59 <dom> youenn: in WebRTC, accessing frames from the camera is via a track processor in a worker
23:49:13 <dom> ... correlating that with timestamps might work
23:49:15 <Bernard> q+
23:49:23 <cpn> ack bia
23:49:25 <dom> ... would be interesting to see if this is workable with readablestreams
23:49:29 <cpn> q+
23:50:05 <dom> bialpio: in this particular use case, WebXR would be introducing video frames, they could be added as a metatadata into video frames
23:50:55 <dom> ... some devices are doing pose prediction - this might make it tricky to sync with past video frames
23:51:08 <dom> ... the question is what we would be the best place to discuss this?
23:51:24 <dom> youenn: VideoFrame sounds like the place where this is converging
23:51:36 <fluffy> q+
23:51:39 <dom> ... so webcodecs repo in the Media WG would seem good
23:51:48 <cpn> q- cpn
23:51:58 <dom> cpn: also worth discussing in the architectural repo
23:52:18 <dsinger> dsinger has joined #mediawg
23:52:28 <bialpio> q-
23:52:34 <cpn> Action: Include image capture for AR applications and stream correlation in architecture description
23:52:38 <dom> ack kaz
23:52:57 <dom> kaz: spatial data wg also looking at sync between location and time
23:53:06 <cpn> q?
23:53:10 <dom> ... useful to consider sync with video stream as well
23:53:14 <dom> ack Bernard
23:53:21 <dom> Bernard: the synchronization issue has come up several time
23:53:32 <dom> ... the videotrackprocess gives you videoframe with timestamps
23:53:40 <dom> ... how to render that accurately in sync with audio
23:53:50 <dom> ... it's not always clear that you get the sync that you want
23:54:04 <dom> ... is that the appropriate API when dealing with all these operations
23:54:16 <dom> ... what's the right way to render sync'd audio/video?
23:54:28 <dom> ... this is probably worth some sample code to figure it out
23:54:28 <cpn> q?
23:54:32 <dom> ack fluffy
23:55:05 <dom> fluffy: one of the architectural issues is that these various systems are working at different timing with different control loops
23:55:24 <dom> ... how to synchronize them and render them correctly is an architectural issue - it needs the big picture of how it fits together
23:55:46 <dom> Bernard: half of the questions I get is in that space
23:56:18 <cpn> Action: capture synchronization issues in the architecture desctiption
23:56:45 <nigel> nigel has joined #mediawg
23:56:59 <dom> paul: this metadata seems to become a central piece
23:57:13 <riju_> Thanks Youenn for bringing this up.. Others if there's something specific to FaceDetection you want to contribute/ask here's an explainer https://github.com/riju/faceDetection/blob/main/explainer.md
23:57:15 <dom> ... it seems worth focus on this to save trouble down the line
23:57:48 <cpn> q?
23:58:05 <dom> [slide 20]
23:58:36 <dom> Topic: WebCodecs
23:58:41 <jholland> apologies, I have to leave early.  Thanks for an informative session.
23:59:49 <kaz> present+ Masaki_Matsushita
23:59:55 <dom> Subtopic: w3c/webcodecs #198 Emit metadata (SPS,VUI,SEI,...) during decoding