13:55:07 <RRSAgent> RRSAgent has joined #me
13:55:07 <RRSAgent> logging to https://www.w3.org/2020/06/02-me-irc
13:55:13 <Zakim> Zakim has joined #me
13:55:26 <takio> takio has joined #me
13:57:18 <kazho> kazho has joined #me
13:59:43 <RobSmith> RobSmith has joined #me
13:59:47 <kaz> kaz has joined #me
14:01:30 <tidoust> present+ Francois_Daoust
14:01:32 <tidoust> scribe: tidoust
14:01:47 <kaz> present+ Kaz_Ashimura, Andreas_Tai, Barbara_Hochgesang, Chris_Needham, Francois_Daoust, Garret_Snger, Huaqi, John_Riviello
14:01:54 <Garrett_Singer> Garrett_Singer has joined #me
14:01:55 <tidoust> Agenda: https://lists.w3.org/Archives/Public/public-web-and-tv/2020May/0027.html
14:01:55 <JohnRiv> JohnRiv has joined #me
14:01:58 <Larry_Zhao> Larry_Zhao has joined #me
14:02:06 <SteveBecker> SteveBecker has joined #ME
14:02:20 <kaz> present+ Kazuhiro_Hoya, Larry_Zhao, Pierre,
14:02:24 <tidoust> Meeting: Media & Entertainment IG call
14:02:30 <tidoust> RRSagent, makes logs public
14:02:30 <RRSAgent> I'm logging. I don't understand 'makes logs public', tidoust.  Try /msg RRSAgent help
14:02:31 <Igarashi> Igarashi has joined #me
14:02:34 <tidoust> RRSagent, make logs public
14:02:38 <Yajun_Chen> Yajun_Chen has joined #me
14:02:47 <kaz> present+ Rob_Smith, Steve_Becker, Takio_Yamaoka, Will_Law, Fuqiao_Xue
14:02:47 <pal> pal has joined #me
14:03:05 <BarbH> BarbH has joined #ME
14:03:10 <kaz> present+ Yajun_Chen, Yash_Khandelwal
14:03:30 <nigel> nigel has joined #me
14:03:35 <tidoust> Topic: Agenda
14:03:46 <Larry_Zhao_> Larry_Zhao_ has joined #me
14:03:46 <tidoust> cpn: Main topic is Client-side video editing proposal.
14:03:56 <atai> atai has joined #me
14:04:02 <kaz> i/Main/scribenick: tidoust/
14:04:17 <kaz> present+ Tatsuya_Igarashi
14:04:30 <nigel> Present+ Nigel_Megitt_BBC
14:04:30 <tidoust> ... Most of the call today will be about that. We can leave some time in the end for general status update on topics of interest: media integration guidelines, bullet chatting.
14:04:38 <tidoust> ... Also, TPAC plans if time allows.
14:04:50 <tidoust> Topic: Client-side video editing (MediaBlob)
14:05:12 <tidoust> cpn: Welcome folks from Microsoft who are behind the proposal
14:05:35 <Larry_Zhao__> Larry_Zhao__ has joined #me
14:05:39 <kaz> Agenda: https://lists.w3.org/Archives/Public/public-web-and-tv/2020May/0027.html
14:05:51 <tidoust> -> https://github.com/WICG/video-editing/blob/master/readme.md Client-side video editing explainer
14:06:13 <tidoust> -> https://github.com/w3ctag/design-reviews/issues/514 TAG review on Client-side video editing
14:06:19 <kaz> present+ Gary_Katsevman
14:06:19 <gkatsev> present+ Gary_Katsevman
14:06:29 <Johnsim> Johnsim has joined #me
14:06:46 <tidoust> Yash: SDE at Microsoft. Steve Becker is also here.
14:07:04 <kaz> rrsagent, draft minutes
14:07:04 <RRSAgent> I have made the request to generate https://www.w3.org/2020/06/02-me-minutes.html kaz
14:07:14 <tidoust> ... We'll go through current issues, the solution, a case study, and relationship with WebCodecs.
14:07:50 <tidoust> ... JS libraries are not able to take advantage of optimizations.
14:08:24 <tidoust> ... Using MediaRecorder is another path. To get 15mn of video, you need 15mn of recording, which is problematic.
14:08:45 <tidoust> ... With cloud-based processing, process affects the bandwidth since video needs to be sent back and forth.
14:09:01 <tidoust> ... There could be a queue delay too, for instance as happens in Youtube.
14:09:13 <tidoust> ... We are proposing an API called MediaBlob.
14:09:34 <tidoust> ... Simply pass a blob to the constructor who will detect the media stream settings.
14:09:40 <kaz> present+ Estella_Oncins
14:09:44 <tidoust> ... "duration" gives the duration.
14:10:53 <tidoust> ... We're also propositing editing ooperations separately, such as trim, concat (list of MediaBlob to one MediaBlob), split (one > two MediaBlob), finalize
14:11:03 <tidoust> ... [showing code example]
14:11:21 <kaz> s/Huaqi/Huaqi_Shan
14:11:44 <tidoust> ... Creating a MediaBlob and operation, which we can trim, concat, and finalize to run the actual editing operation.
14:12:07 <tidoust> ... The whole thing takes 728ms, in the browser.
14:12:20 <tidoust> ... The API is easy to use with MediaRecorder the File API.
14:12:43 <tidoust> ... No need to have specific knowledge about media concepts, codec licensing issues, etc.
14:13:04 <kaz> present+ Harneet_Sidhana
14:13:19 <tidoust> ... Flipgrid is owned Microsoft company.
14:13:29 <kaz> present+ John_Simmons
14:13:46 <tidoust> ... Flipgrid use of MediaBlob allowed them to multiply video editing capacities by 40.
14:14:14 <Igarashi> Igarashi has joined #me
14:14:16 <Larry_Zhao> Larry_Zhao has joined #me
14:14:20 <tidoust> ... WebCodecs is a powerful API that provides support for live gaming, RTC and others. It provides transcoding. But no support for demux operations.
14:14:36 <kaz> present+ Mounir_Lamouri
14:15:00 <tidoust> ... For instance, with WebCodecs, you may need an external demuxer.
14:15:18 <tidoust> ... And then WebCodecs and an external muxer in the end.
14:15:45 <tidoust> ... With MediaBlob, for transcoding, trimming and so on, we don't need any external demuxer/muxer.
14:16:01 <tidoust> ... MediaBloc exists on the Edge browser, and is under Original Trials.
14:16:03 <kaz> present+ Paul_Adenot
14:16:28 <tidoust> s/MediaBloc/MediaBlob
14:16:38 <kaz> present+ Pierre-Anthony_Lemieux
14:16:38 <Igarashi> Igarashi has joined #me
14:16:51 <Igarashi> q+
14:17:02 <BarbH> Questions - Impact on AV1 versus V9?
14:17:05 <tidoust> cpn: In this group, we've been aware of this work for some time. The trigger for the invitation is you starting the TAG review last week.
14:17:12 <kaz> q?
14:17:31 <BarbH> Questions - Other browser beyond Chromium?
14:17:31 <tidoust> ... I'd like to invite questions, and then we could talk about professional use cases that the IG is currently exploring.
14:17:43 <kaz> q+ will
14:17:46 <kaz> ack will
14:18:04 <tidoust> Will: On MediaBlobOperation, why is that abstraction necessary?
14:18:52 <tidoust> Yash: We wanted to align with things done in WebGL. Another reason was batching, but maybe not necessary.
14:18:55 <BarbH> Question - Have you reviewed in the Media WG and WebGPU CG?
14:19:08 <tidoust> Will: OK, I would argue that the coupling does not look necessary.
14:19:34 <tidoust> ... Also, finalize can be very heavy. It may do transcoding, which may take time. Why not expose that more explicitly?
14:19:42 <tidoust> Yash: That's good feedback.
14:19:59 <tidoust> Will: What sort of containers do you support? I would hope CMAF-based.
14:20:00 <kaz> q?
14:20:17 <tidoust> Yash: Right now, HEIF MP4, [missed]
14:20:30 <cpn> s/[missed]/WebM/
14:21:02 <tidoust> Igarashi: About use cases, can this be applied to file uploading cases? How to create a MediaBlob in these cases?
14:22:15 <tidoust> Yash: I don't have an example right here, but can update the explainer to clarify this situation
14:22:51 <tidoust> Igarashi: This MediaBlobOperation may depend on the media type. Are there some media capabilities detection that may be needed to know whether an operation is supported?
14:23:19 <tidoust> Yash: That is good feedback. If Mime sniffing does not work, the creation operation will fail.
14:23:53 <tidoust> Igarashi: The "finalize" operation may take time. In some cases, it may be useful to reduce the bitrate. Would translating be supported on top of transcoding?
14:24:12 <tidoust> Yash: We did think about it. We're still in the process of deciding whether it would be useful or not.
14:24:37 <tidoust> ... We're looking for feedback on whether that's useful to expose this.
14:24:39 <cpn> q?
14:24:43 <cpn> ack iga
14:24:44 <cpn> q?
14:24:47 <kaz> q+
14:25:03 <tidoust> ack kaz
14:25:44 <tidoust> kaz: Possible additional APIs for "create" and "delete" for MediaBlob, and possibly "send"
14:26:13 <tidoust> Yash: Basically, since we're inheriting the Blob API, all of these should be easy for us.
14:27:01 <tidoust> Igarashi: I think it's possible to use trim and concatenate to cut some portion of media.
14:27:16 <nigel> q+
14:27:26 <tidoust> kaz: My question was more about the object itself, not chunks within the media stream.
14:27:48 <tidoust> Igarashi: split, concat enables any kind of video editing.
14:28:23 <tidoust> Yash: Why do you think we should have a "delete" operation? trim and split seem to be the right answers there. Do you think that "delete" would be useful?
14:28:34 <pal> q+
14:28:44 <tidoust> Igarashi: It depends on performances. If a "delete" operation is much more efficient, then it should be considered.
14:28:48 <kaz> s/stream/stream. so Yash's clarification on inheriting the Blob API is OK by me./
14:28:59 <cpn> ack nigel
14:29:02 <Johnsim> q?
14:29:15 <Johnsim> q+
14:29:36 <tidoust> nigel: A quick reflexion on the conversation. concat takes in a sequence of MediaBlob, rather than the result of MediaBlobOperations.
14:29:39 <kaz> s/"delete" operation?/"delete" operation for part of the media blob in addition to the proposed "trim" and "split"?/
14:30:16 <Igarashi> Igarashi has joined #me
14:30:22 <tidoust> ... It seems you're partly in the world of MediaBlob and partly in the world of MediaBlobOperation.
14:30:50 <tidoust> Yash: The editing only happens when "finalize" gets called.
14:31:29 <tidoust> nigel: So if I want to concat the result of a trim, then I need to call "finalize" to individual MediaBlobs?
14:32:06 <tidoust> ... If the MediaBlob you need to pass to concat is the result of a split, it seems strange that you can't prepare all operations in advance.
14:32:12 <tidoust> Yash: I see your point.
14:32:24 <BarbH> add me to question queue
14:32:41 <cpn> q?
14:32:54 <tidoust> nigel: long long time is a number of milliseconds. Have you thought about rounding issues that you might get with certain frame rates? It seems easy to have a frame rounding error.
14:33:13 <tidoust> ... 29.99ms frame rate is often used.
14:33:26 <tidoust> ... I'm just concerned that there may be rounding errors.
14:33:32 <nigel> s/29.99/29.997
14:33:42 <tidoust> Yash: We haven't thought about that, but that's something we'll look into.
14:33:47 <cpn> ack pal
14:33:57 <cpn> q+
14:34:08 <tidoust> pal: Wanted to echo Nigel's two points. Instead of a sequence of operations on blobs, have you considered a playlist approach?
14:34:17 <takio> q+
14:34:42 <tidoust> ... The idea with the playlist model is that it's easier to pass around and to include multiple tracks instead of just one big track which is a multiplex of mulitple ones.
14:34:51 <nigel> +1 to the playlist idea, also the playlist itself might be a usefully serialisable thing
14:35:05 <tidoust> ... I would also really use rationale numbers for time.
14:35:26 <tidoust> Yash: Thanks, we already have someone who filed an issue around that, we'll look into it.
14:35:35 <kaz> present+ Ali_C_Begen
14:35:49 <kaz> q?
14:36:30 <tidoust> nigel: The playlist itself is something that could be easily serialized/deserialized as well. Imagine that you're doing an editing session and wants to apply it to other media files.
14:36:56 <nigel> -> https://github.com/WICG/video-editing/issues/13 Why does it not make sense just to create playlist? #13
14:37:02 <tidoust> cpn: Presumably, we can raise issues against the GitHub repo to provide feedback.
14:38:19 <tidoust> Johnsim: For media containers, HEIF was scribed (probably incorrectly, bad scribe!). Are image formats supported?
14:38:30 <tidoust> Yash: We're not supporting any image format for now.
14:38:39 <tidoust> Johnsim: So that was a mistake, ok.
14:39:18 <tidoust> BarbH: Besides Chromium, is this something that is also being socialized with other browser vendors, e.g. Apple?
14:39:37 <tidoust> Yash: We have reached out to Apple, not yet with Firefox.
14:39:40 <cpn> q?
14:39:46 <cpn> ack Johnsim
14:40:09 <tidoust> BarbH: Have you gone to the Media WG? The reason I'm suggesting it is that Apple actively participates there, and could give feedback there.
14:40:30 <kaz> w?
14:40:33 <kaz> s/w?//
14:40:34 <kaz> q?
14:40:34 <xfq> xfq has joined #me
14:40:37 <kaz> ack cpn
14:41:02 <tidoust> cpn: For eventual standardization, after incubation, that would probably be the appropriate place to take this. I reached out to Apple for today. Looking at early proposals is what the IG does, so that's the right place to get feedback!
14:41:54 <tidoust> BarbH: Have you also reached out to the tools vendors that are doing video editing today? Adobe is very active in the W3C and they have video editing tools for instance.
14:42:06 <kaz> q+
14:42:14 <tidoust> ... They're not the only ones.
14:42:24 <xfq> rrsagent, draft minutes
14:42:24 <RRSAgent> I have made the request to generate https://www.w3.org/2020/06/02-me-minutes.html xfq
14:42:43 <tidoust> Yash: We have not reached out to video editing tools, but this is something that we will start looking into. Engaging with other browser vendors as well.
14:43:11 <tidoust> pal: Part of what we're doing today is bringing awareness about this proposal to a wider media community.
14:43:34 <tidoust> cpn: Right and Media Production is one topic that the IG is currently exploring.
14:43:49 <tidoust> ... Connections with Adobe would be good.
14:43:58 <tidoust> BarbH: I'll see what I can do.
14:44:04 <cpn> q?
14:44:07 <cpn> ack takio
14:44:18 <tidoust> takio: I filed issue #13 last night. I'd like to confirm that issue.
14:44:25 <xfq> https://github.com/WICG/video-editing/issues/13
14:45:21 <tidoust> ... Quick question: Sometimes, operations need transcoding. Sometimes, they don't, right?
14:45:45 <tidoust> Yash: Right, for trim, we will do possibly partial transcoding to re-create an I-Frame for instance.
14:46:16 <tidoust> takio: The user cannot distinguish whether transcoding happens or not, this is perhaps problematic.
14:46:30 <Igarashi> asynchronous api using promis should be considered
14:46:37 <tidoust> Yash: Thank you, I think we got feedback some feedback here on "finalize".
14:47:09 <tidoust> ... When transcoding takes place, things may take place, we'll add some abort handler.
14:47:13 <cpn> q?
14:47:15 <tidoust> cpn: Progress events would be nice too.
14:47:18 <cpn> ack kaz
14:48:02 <nigel> the need to transcode depends on the input formats as well as the output formats: concat([type 1 blob, type 2 blob]) must transcode at least one. What format do you get at the output?
14:48:06 <tidoust> kaz: TV broadcasters might have some potential opinion from their viewpoint. They may be using hardware-based editors which might have powerful capabilities. They may have opinions on this software-based approach.
14:48:31 <Yajun_Chen> Yajun_Chen has joined #me
14:49:08 <Igarashi> proxy editing would be useful for professional editing
14:49:26 <tidoust> cpn: Right, typically with cloud-based edition, we'll send time points on which to run operations on the server, with the client seeing a "proxy" representation.
14:49:34 <pal> q+
14:49:44 <tidoust> ... I see the main use case being user generated content (Youtube et al.).
14:49:50 <Igarashi> mediaBlob  oiled be useful for proxy editing
14:50:15 <tidoust> pal: In terms of editing, there's surprisingly few differences between professional and "casual" editing: codecs perhaps, and some special effects.
14:50:16 <Igarashi> s/oiled/editing/
14:50:45 <kaz> i/mediaBlob/kaz: yeah, potential gap between "professional use" and "customer use", etc./
14:50:52 <tidoust> ... In terms of playlist, there are a number. One is OTIO, originally from Pixar but now open-source.
14:51:09 <takio> I'm prefer playlist. If do so, we can chose client side or server side transcoding.
14:51:14 <tidoust> ... All of these playlist models look different, but they're all the same at the end of the day.
14:51:34 <tidoust> ... Main differences are support for transitions (for audio and video).
14:51:45 <kaz> s/I'm prefer/I prefer/
14:51:54 <tidoust> ... Other than that, they're all the same. That's also why I'm encouraging exploring a playlist approach.
14:51:56 <takio> thanks kaz.
14:52:01 <kaz> s/can chose/can choose/
14:52:24 <kaz> s/thanks kaz//
14:52:59 <tidoust> cpn: Clearly, there's a lot of interest in this API and also in WebCodecs. If people are interested, maybe we can ask some of the editors behind WebCodecs to have a closer look at the API to see where they are and how it maps to some of the use cases that we have.
14:53:21 <tidoust> ... It seems we have some useful feedback to provide.
14:53:37 <tidoust> pal: What are your next steps?
14:53:50 <tidoust> ... Where would you want the work to happen?
14:54:25 <tidoust> cpn: Are you looking for more input from media companies?
14:55:21 <tidoust> Yash: Immediate next step is figuring out the story with WebCodecs. Google's perspective seems that MediaBlob only provides demuxing/muxing, with the rest being covered by WebCodecs.
14:55:51 <takio> Even if editing into same codec, it will need to be re-encode if GOP is too long...
14:55:59 <tidoust> cpn: Interesting, lots of open questions. Here, we're happy to host these kinds of conversations. My suggestion would be that we continue this discussion certainly among media companies. You're very welcome to be part of it.
14:56:25 <tidoust> pal: Yash, would you like us to try to setup a meeting with the right folks?
14:56:42 <tidoust> Yash: I think that would be good.
14:56:44 <kaz> q+
14:57:01 <tidoust> Harneet: I think we should start by evaluating the classes of scenarios that we're trying to enable.
14:57:09 <pal> q-
14:57:25 <tidoust> ... I don't think that the goal is to replicate everything that Adobe can do in editing tools, for instance.
14:57:34 <tidoust> ... It's more to enable common basic needs.
14:58:09 <Igarashi> proxy editing should be added to the use cases of media blob
14:58:13 <tidoust> ... As a next step, we need to evaluate the pros and cons of MediaBlob vs. WebCodecs.
14:58:39 <tidoust> ... I think there is room for a simple API that provides basic video editing capabilities on the Web.
14:59:07 <tidoust> ... Is there interest in this level of media editing? Or is interest more on high-end video editing capabilities?
14:59:46 <tidoust> ... Is there level of interest from the industry on concat, trim, split?
14:59:56 <pal> q+
15:00:07 <kaz> q- later
15:00:12 <tidoust> ... We don't envision MediaBlob to be used in professional media editing, too complex for the API.
15:00:17 <kaz> ack pal
15:00:48 <tidoust> pal: I'd like to dispel that myth. I believe they can be supported with very minimal changes. The challenge in front of us is can we go a little beyond to cover much broader use cases?
15:00:50 <nigel> +1
15:01:09 <tidoust> Harneet: We'd love to hear about that.
15:01:39 <tidoust> ... We'd like not to require deep media knowledge to be able to use the API.
15:01:52 <tidoust> pal: But that's a good requirement also for professional applications.
15:02:18 <Igarashi> proxy editing does not require professional codes
15:02:38 <tidoust> Harneet: Do you have a discussion area? We'd love to get all the feedback through issues on the GitHub repo.
15:02:50 <tidoust> ... If you can file issues on this repo, that would be great.
15:03:00 <tidoust> cpn: Very happy to do that.
15:03:24 <tidoust> [Side discussion on sharing slides]
15:03:25 <cpn> q?
15:04:23 <kaz> ack k
15:04:23 <tidoust> kaz: We have many stakeholders from media industries, which can help clarify use cases, requirements and gaps. From time to time, we can revisit this discussion.
15:04:39 <tidoust> Topic: Media Integration guidelines
15:05:17 <tidoust> cpn: We're out of time, but note we started some work on guidelines for integration of media APIs on devices with limited hardware capabilities.
15:06:05 <tidoust> JohnRiv: I migrated issues from the original CTA WAVE discussions. I haven't integrated comments from Dan yet.
15:06:29 <tidoust> cpn: OK, so this is a call for participation to get involved in this work!
15:07:04 <tidoust> Topic: Bullet Chatting
15:07:21 <tidoust> Huaqi: The CG completed the use cases and gap analysis.
15:07:38 <tidoust> ... We discussed two rendering ways.
15:07:57 <tidoust> ... CSS way seems to be preferred by vendors, so we're exploring that more thoroughly.
15:08:23 <tidoust> cpn: Thank you.
15:08:48 <tidoust> RRSagent, draft minutes v2
15:08:48 <RRSAgent> I have made the request to generate https://www.w3.org/2020/06/02-me-minutes.html tidoust
15:09:38 <xfq> xfq has joined #me
15:09:50 <xfq> RRSAgent, make minutes
15:09:50 <RRSAgent> I have made the request to generate https://www.w3.org/2020/06/02-me-minutes.html xfq
15:11:17 <xfq> RRSAgent, make minutes v2
15:11:17 <RRSAgent> I have made the request to generate https://www.w3.org/2020/06/02-me-minutes.html xfq
15:12:07 <JohnRiv> FYI, here is the Media Integration Guidelines repo: https://github.com/w3c/me-media-integration-guidelines
15:14:25 <xfq> xfq has joined #me
15:38:22 <atai> atai has left #me
15:41:12 <cpn> Chair: Chris, Pierre, Igarashi
15:41:19 <cpn> rrsagent, make minutes v2
15:41:19 <RRSAgent> I have made the request to generate https://www.w3.org/2020/06/02-me-minutes.html cpn
16:03:53 <tidoust_> tidoust_ has joined #me
17:31:00 <Zakim> Zakim has left #me