Meeting minutes
Introduction
cpn: [Welcomes everyone, shows antitrust slide]
https://
cpn: The mission of our group is to provide an industry forum in W3C
… for discussions related to media and media features on the web in particular,
… and interacting with working groups who develop the features.
… As an Interest Group we do not develop specifications but do
… work on requirements and reviews.
… The scope is the application of web technology relating to media from end to end.
… [shows Introduction slide]
… and related technologies such as timed text, assistive tech etc.
… We've been going since 2011
… [History of major initiatives slide]
… Covered the video element in HTML5, EME and MSE as key capabilities.
… Reformed as MEIG in 2017
… Looked at the development of the media web app platform,
… and the application of web technologies for media consumption and creation.
… [Workflow slide]
… We operate as:
… Typically take input from members and contributors
… Many relationships with other standards bodies and industry groups.
… They bring us requirements.
… We do gap analysis thinking about future web platform evolution.
… The outcome is either new work for existing WGs or
… in new areas, the creation of a CG if we have enough buy-in.
… [Resources slide]
… We have a homepage, Charter, GitHub repository, a public mailing list and a member mailing list
… if needed e.g. for sharing liaison information that can't necessarily be made public.
… One interesting resource is an overview of Medit Technologies for the Web
… that we published a few years ago.
… A one-stop place to see media activities that are happening.
… Unfortunately it hasn't been maintained - would be interesting to bring it up to date.
… If you are interested in working on it I would be happy to work with you on that.
… [Announcements slide]
… We have some new co-chairs.
… First, I thank Igarashi-san and Chris Lorenzo for your work as co-chairs.
… You've been with us for a long time and we very much appreciate the work that you have done.
… Welcome to Wolfgang and Song as joining us as new co-Chairs.
… Look forward to working with you.
… We also have a new W3C Staff contact.
… Thank you to Kaz who has been with us from the beginning and been a key driver
… to all our work here. None of it would be possible, thank you for all that you have done.
… Congratulations on your new position. We look forward to working with you in your Invited Expert capacity.
kaz: Thank you
cpn: Welcome to Roy as our new team contact.
Roy_Ruoxi: I will try my best to support this group and learn from all of you how MEIG works.
Wolfgang: I look forward to working with you.
Song: I'm Song from China Mobile, and work in the media industry and more recently AI
… stuff that is hot everywhere. I offer myself to help Wolfgang and Chris to work with you all.
Agenda
cpn: Before the break, two codec related presentations,
… from Jianhua, Fabio, and Simone from V-Nova about challenges using LC-EVC.
… Then we will revisit Next Generation Audio codecs with Wolfgang and Bernd.
… [slide: Agenda]
… [continues to iterate through agenda]
LCEVC decode support in browser
https://
Jianhua: Today I will talk about LCEVC decode support in browser
… and how to enable playback in the browser environment
… V-Nova is a company based in London,
… mainly providing video compression technology
… LC-EVC, SMPTE VC-6 for contribution and production workflows,
… and PresenZ - a VR format for cinematic presentation
… TV 3.0 overview
… Signed in August this year into law in Brazil.
… Means that VVC and MPEG-H are the mandatory codecs to implement in the next few years.
… The first devices will be in June 2026 before the World Cup.
… Many devices use dash.js and Shaka player as web player solutions.
… Ecosystem development
… 2023: developed and contributed to LCEVCdec and LCEVCdecJS open source projects.
… For the LCEVCdec library, it will be integrated into the media framework for e.g. ffmpeg and gstreamer.
… For the decJS, it will be integrated into dash.js and Shaka player.
s/For 2025: For
… When we talk about LCC playback in browser, our goal is to have an efficient decoder
… available on different platforms
… [How LCEVC Works overview slide]
… We can focus more on one use case as a starting point.
Jianhua: There is a base layer encoding. The LC-EVC is the difference between that and the 4K
… After demux, there's the base layer data, and the enhancement layer. The base layer is decoded in hardware, and the LC-EVC is done in hardware or software
… The major benefit of this form of encoding is to save bitrate
… LC-EVC can be deployed in software on exiting devices
… Sometimes, on less powerful devices, there's a performance challenge
… Let's look at how the data is carried in the MP4 or DASH format
… We contributed to the ISO standard a few years ago
… The idea is to link the enhancement layer to the base layer
… The player needs to know which base layer is linked to which enhancement layer, so there's a new MP4 box, "sbas" to link them
… This is known as dual-track carriage
… There's a similar mechanism in DASH-IF IOP since v4.3. The idea is the base layer and enhancement layer to be described in separate presentations, then use a @dependencyId attribute to link them
… We implemented support in DASH.js and Shaka Player
… Produce a low resolution base layer, and the delta, without producing the high resolution
… Diagram of DASH.js solution
… On the left, the player handles the base video stream and LCEVC enhancement stream
… After downloading the data, it's sent to MSE to demux and decode
… We contributed to DASH.js the lower part. We created a module, a software library to decode the LCEVC bitstrea
… An external MSE model. LCEVC bitstream decoding in WASM. Upsample the base video. From the HTML video tag, after decoding by MSE,
… the data is signalled to the LCEVC module, upsample to 4K
… Base layer is decoded by browser MSE. LCEVC is decoded in software. And construct a final 4K video stream.
… Generally it works well on PC browsers, we demoed at IBC and NAB
… Limitations of this solution. Performance - on STB and smart TV it doesn't work so well. Copy data from MSE to DASh.js outside the library, instead of in the browser
… This architecture issue downgrades the performance
… No way to have DRM protected streams
… Need to be able to protect the video frames in memory
… So, how to enable a native LCEVC decoder architecture?
… Parse manifest, download data. MSE with base and enhancement layer decoding, to perform well
… MSE and EME are designed to work together, so DRM protection shuold be a natural outcome
… So I come here to ask for help
… Questions: how to enable LCEVC decode and playback in a native architecture
… Query if the device can support this codec. Use Media Capabilities API to detect. Syntax for querying support, using the codec string, e.g., avc1.42E01E,lvc1
… Seems simple, but I tried querying with this syntax - can the browser support queries for two codecs. Dolby vision? I got a syntax error
… Reading RFC6381, it says you can use multiple codec strings
… For modern browsers, are there existing implementations of this kind of query?
… After the player has negotiated support with the browser. Two use cases:
… If the input stream is a single segment with dual tracks, a progressive MP4 file or a DASH stream with one representation containing two tracks. How to send to MSE?
… One segment, so call addSourceBuffer with the codec string containing both codecs. So MSE would have to recognise the data in the MP4
… This way of carriage has been standardised in ISO, but no implementation
… Use case 2, two representations in the DASH manifest. Linked to each are two segments. This method saves bitrate as the player only downloads what's necessary
… Create a SourceBuffer for the base layer and another for the enhancement layer
… The browser decodes and makes sure they're synchronised
… Question: Is the above understanding correct?
… SoC support is important. Companies like RealTek and Amlogic have implemented support in their STB and TV SoCs
… We're working closely with manufacturers. on Android, we use Google's Exoplayer to support this use case
… If the SoC has an implementation in the driver, just need to add the codec implementation. A high level codec can just set up a link
… How to get access to the low level decode in the browser?
… People want to use W3C API for playback. Trend is moving towards web based APIs
… Based on the hardware decode, and the user wants to use DASH.js or Shaka Player to play a 4K stream, what needs to be done? How to access the decoder from browser API?
… Is it up to browser vendors, or to TV manufacturers to link the API to the lower level decoder?
<tidoust> [Regarding use case 2 mentioned above, I believe MSE currently does not support merging 2 SourceBuffer into a single video track]
cpn: Would like to hear opinions in the room.
… You're saying that by implementing LCEVC you can save network usage overall
… but you need the ability to decode and render in a browser based environment
… but you have efficiency concerns because you have to decode and render into a canvas,
… apply the enhancement to the canvas data, and that's computationally intensive,
… but certain devices have hardware acceleration support and in theory you could take advantage
… of it to make a more efficient decode pipeline, but you can't use it because the web APIs
… don't provide access to it. Then the question is should we investigate how such support
… should be introduced into the web APIs and what conditions need to be met to allow that.
… Then you have specific questions.
… The first one is about capability detection and how you express a codec string that includes
… the base layer and the enhancement layer and if this is the right syntax,
wschildbach: I think this syntax is not supported, arguably it should be.
… RFC6381 I think is targeted at expressing what's contained in the media and by proxy it has
… become something to use to express capability queries.
… It's not clear how to query more than one codec at the same time.
… I don't think it's generally supported.
cpn: Can we move to the question about MSE? I think this is the core of the question.
… You're proposing that in order to benefit from the bandwidth bitrate reduction you really
… want to look at use case 2 where there are separate source buffers, one for the base layer
… and one for the enhancement layer and allow the implementation to combine them together.
Jianhua: They're both the same really, just different ways to express the data.
tidoust: On use case 2, if I understand MSE, I don't think it's possible for 2 source buffers
… to be merged into 1 video track. It would not work well with the way the specification is written now.
… Use case 1 ddoes not require changes to the spec, but use case 2 oes.
Jianhua: Use case 2 is more up to the MSE implementation.
tidoust: Each source buffer is associated with a video track, and at the moment they are disjoint
… so you can't have two source buffers be merged into a single video track.
alan: If we wanted to start a study mission to do that enhancement, what would be the way to do that in W3C?
cpn: Some of this depends on the end goal. If you want something that works across all
… mainstream desktop and mobile browsers then I would recommend starting discussions with
… the projects that implement those browser engines and get expressions of support
… to say that it looks interesting and we want it as a general capability on the web.
… With support in that direction doing the downstream work of specific API structure design
… can be figured out. You're in the right place, in this group.
… It would be interesting for us to have a document that captures what you have described
… and the existing pain point and the need that you see, and that supports a dialogue
… with the browser engine implementations.
alan: We can head in that direction. We have support from the organisations building
… equipment for the TV 3.0 service. Hopefully we can then bring the browser community
… in after demonstrating the business importance.
<wschildbach> q.
alan: But you're saying that we should come back here about API discussions?
Chris: That's right, this group can't work on specifications, we have to focus on use cases,
… requirements and gap analysis, which is what you've done here.
… For solution design we can form a different kind of group to give us the IPR protections to
… work on draft specifications.
alan: Thank you, appreciate the insight.
igarashi: I've been impressed by the implementation.
… Aside from the issue of W3C API support, I have a question about codec licensing.
… If you are targeting mainstream browser support, as you know we have discussed
… in W3C and in relation to the patent policy, there are requirements that the codec licensing is
… free and previously some codec licensing has been that the decoder is free but the encoder is not.
… There are still requirements - what is the licensing scheme for LCEVC?
alan: V-Nova is committed to transparency [audio cuts out]
… [scribe cannot understand, though audio is working again]
… V-Nova's website has information about licensing.
cpn: Does anyone here want to comment on how what kind of requirements browser implementations
… look for when considering adding support for codecs? What conditions do they place on this, typically?
Chris_bloom: I'm not a strong source here, but a rough generalisation is that after adding support
… it is difficult to remove later, so you probably want wide support in general and then add to
… the browser last. I would recommend getting support all over the place where you can.
cpn: Good indication to get broad support and when there's widespread adoption that's a positive
… indicator. The W3C doesn't set rules on which codecs should or should not be implemented.
… Our specifications tend to be more descriptive in that sense.
… We say how to support implementations but not which codecs must be implemented.
<igarashi> FYI https://
cpn: It's a choice for each browser implementation based on their own criteria like what Chris just mentioned.
Alan: There's a cart and horse problem with the API being limited for implementations currently in play.
… That's something we have to talk about internally.
Song: With the recent experience when I tried to start a discussion between codecs and browsers,
… the copyright and IPR statement is the bottom line for W3C.
… Beside that, in general, codec adoption in industry: it's true that we need to get the potential
… codec validation data to present to most of the members in the group.
… Also EVC is not a standalone codec, so we need to let the members see the value of this kind
… of enhancement framework.
… We also need the developer to see the value of using LCEVC.
… We need some validation or usage data to prove that this is stable or widely adopted in the industry
… even before standardisation. This is my general experience for past work.
alan: Something that may cast it in a different light. LCEVC is a unique technology in that it is
… the first scalable video codec solution that seems to have got significant traction for a television
… service. We've had it for conferencing, but this is a first for television, where the client devices
… are using browsers to create media rendering experiences.
… I don't think it will be the only one, an additional stream might be needed for others as well as LCEVC too.
cpn: That makes it interesting as well, looking at how to generalise into other encodings.
eric-carlson: Stepping back to requirements for a browser to add support for a new video codec.
… For us, for Apple's port of webkit, we typically won't use a software based video codec
… because of the power requirements, so support in hardware is generally a requirement for us.
cpn: Thank you
wschildbach: I've been following the discussion with some interest because there's a more general
… use case for multiple source buffers - Dolby vision has been mentioned.
… The question of what requirements apply is an interesting one.
… We've heard that this is a conversation with browser implementors, which can be outside W3C.
… When it comes to changing semantics of the API, then I think it is in W3C and starts in this group.
cpn: Yes it can be.
cpn: Thank you for this, it has been useful, we're happy to continue the conversation.
… This is a good starting point.
Jianhua: thank you everyone
Next Generation Audio codec API proposal
wschildbach: I work for Dolby labs, and alongside Fraunhofer IIS we have been talking to W3C
… about next generation audio and how personalisation can be achieved.
… Rather than give another presentation to this group about what personalisation and NGA are,
… take a step back, and understand where we are with this journey with W3C.
… I've seen your introduction Chris, where you talk about requirements, gap analysis documents, use cases
… where this has been put into the process. Are we coming to an end of the journey or still
… figuring out if this is a thing. For the group to ponder, and discuss after the break,
… is this something the group needs an output for, or are we yet to agree that we have something
… to do.
… Suggest coming back to this after the break.
cpn: Yes. To add to that, in previous TPACs we've walked through the use cases for personalisation.
… Did we have detail about the gap analysis at that time?
wschildbach: Possibly not enough detail.
… The suggestion was made to put forward a gap analysis, and we sent one to the reflector,
… but did not discuss it in detail. In the last MEIG call I summarised it.
… We didn't give the gap analysis a lot of detail.
cpn: OK let's break, we're now at 10:30, can discuss over coffee.
… [adjourns for the break]
Agenda review for session 2
cpn: The previous presentations overran, so let's review the agenda now.
… Louay, can we hold for 20 minutes?
Louay: Yes, fine for me
cpn: Thank you. Then we'll restart with Next Gen Audio.
Next Generation Audio codec API proposal (cont.)
cpn: Last TPAC there was feedback to talk about a gap analysis.
… We had focused on presenting the use cases.
… There is a document about them, including what an API change might look like
Wolfgang: Right. 2 years ago I think we presented the use cases we want to enable.
… That document is appended to the notes of that meeting. We can share it again.
… The outcome of that meeting - we began talking about it in 2022 in Vancouver, then
… continued in Sevilla. In Anaheim we were asked to provide a gap analysis.
… That is a document to explain why the use cases cannot be fulfilled with the current API,.
… There's a longer Word document that I sent to the reflector, just before the previous IG telco
… a few weeks ago. I presented a slide deck that abridged it.
… If you haven't seen the gap analysis I would encourage you to take a look at it.
… It explains why the use cases cannot be implemented using existing APIs. We can provide more detail.
cpn: We could decide to use that document as the basis for an IG Note published by W3C.
Nigel: The use cases or the gap analysis document?
cpn: Actually both, either as one or two IG Notes.
… The question is if that advances the goal of encouraging implementation.
… It perhaps gives the requirements some more visibility.
… Similar discussion to the LCEVC codecs one, whether this group produces a requirements
… document or not does not guarantee anyone moves forward to the next stage
… but it does give us something to iterate on and gives us more visibility than a mailing list document.
Wolfgang: Yes, it would also be a point of reference.
… We understand that this group has no power to force anyone else to do specific work.
… We can produce a note that shows what we have talked about and where we stand,
… and could produce an end point to our conversations on this topic.
cpn: We could decide to do that. That means that, if we have sufficient motivation and interest
… to do that work, then I would be supportive. We can figure out it that's one or two documents.
cpn: I wondered if you have any thoughts on the Eclipsr codec and if that has similarity to AC4 and MPEG-H
… codecs and if that means we should be looking at a broader set of codecs in the gap analysis.
… I'm not familiar with the details, but my colleague Matt who chairs the Web Audio WG thought
… it would be worth considering.
… How similar or different their capability sets are and if it satisfies the same set of use cases.
Wolfgang: I don't actually know, maybe colleagues from Fraunhofer might now.
… I think Eclipsr does Immersive Audio but I'm not sure if it does personalisation.
… I don't think we are proposing any particular codec though, we want to be codec agnostic.
Bernd: That was a clear requirement for us.
Wolfgang: If the proponents of Eclipsr have particular viewpoints we would welcome them.
… Would it change anything though?
cpn: It depends on who is interested to implement.
Wolfgang: I think implementation is a different issue, not for this group.
… If nothing gets implemented then none of this matters.
… I think that's a different set of conversations, or is it?
cpn: Typically a WG is going to want to have at least one implementer willing to commit
… to not necessarily a specific codec but the API shape.
… At least with expressions of support from others.
… We don't want specifications to be drawn up with no implementer support.
… There would need to be at least one committed, and ideally others giving expressions of support.
Wolfgang: That would be a prerequisite for anyone starting work in the Media WG.
cpn: Exactly, yes.
Wolfgang: Is it a requirement to finish the work in MEIG?
cpn: Not really, no. If implementers are committed to working on it then all you need is a
… definition of the problem space such that we can charter a group around it.
… They can do the spec work then. It can happen independently of this group.
… We're not the only route to creating things.
… It could come out of private conversation with potential implementers.
… Then that would feed through the chartering process into pulling it into a WG.
Nigel: The other approach is to have an incubation group, where you prototype an implementation. When you have something that looks like it works, standardise
cpn: W3C is quite well set up to do incubation groups, either in a community group for the purpose
… or the WICG where all the browser vendors are already signed up to the IPR terms, so it might
… be an easier path than creating a new group.
Igarashi: Implementation is not a requirement to move forward with a standard, it's for interoperability. If a member supports the activity, it can move forward.
… More important is whether people in the IG support the work
Wolfgang: So what this group can give is a statement of support that the technology and potential new activity are useful
… What I'd like from this group is to agree that the gap analysis is correct, i.e., that the use cases cannot be implemented using existing APIs
Igarashi: And that the use cases are beneficial to the web
Wolfgang: I'm looking for support about how we do that, is it a Note or something else?
Chris: Notes are the tools we have, then separately there's advocacy
Chris: To make it concrete we can publish a Note or two Notes that will serve as a point of reference.
… We have done that before.
… At this point that would be a proposed Resolution, for the IG, based on the documents
… already shared, to draft an IG Note detailing the use cases, requirements and gap analysis
… for Next Generation Audio Codecs.
Wolfgang: Should we start that today, or give the group time to review the documents.
Chris: There are some formalities about how we do CfC and see if there are any objections.
kaz: Are we okay with moving ahead as a whole IG or should we set up a task force?
Chris: A task force would be a subset of the IG that forms to work on a specific area of work,
… holds its own meetings, and then reports back.
… I don't know, do you recommend we have one?
kaz: Maybe we could start with a simple draft first and then think about how to proceed after that.
Chris: Task forces help when the group is working on multiple things at the same time.
… At the moment we aren't doing that, so it's a tool available to us but we don't need to do it.
Wolfgang: If everyone agrees already then we can move ahead, or if they're neutral.
Chris: We'll take that as a proposed resolution and issue a CfC
Nigel: I think you don't need a CfC to start the work, it's more for publishing
Chris: Yes we can get going and do a CfC when it is time to publish
Bernd: We don't have to wait another year?
Chris: Absolutely, we can do it by email or in our regular calls
Wolfgang: OK I will work on the first version. Do we put this on GitHub?
Chris: Yes, that's detail we can figure out. There are two formats, Bikeshed and Respec.
… I can help with that part.
… The GitHub is the MEIG's own repo
… When we do the publication step we ask W3C to give it a formal W3C URL
Roy_Ruoxi: I can help with that part.
<JohnRiv> looking at the Process, it appears to me that Nigel is correct that nothing formal is required until we decide to publish the note: https://
Nigel: Experience has been that a repo per document makes things easier with PR Preview etc
kaz: Yes it's easier that way
Roy_Ruoxi: I can set that up
Chris: We'll take Roy's advice about which to use and figure out the repo stuff.
CTA Wave EME Testing
Louay: [shares slides]
Louay: A brief intro to the CTA WAVE test suite
https://
Louay gives over view of test Suite group.
Louay: which tests devices
… Idea is to have mezzanine content that facilitates automatic testing of devices.
… testing is also automatic. There is a testrunner, test implementations in HTML+JS, using MSE and EME APIs to playback content.
… There are a set of test implementations for encrypted and clear content. And an observation framework
… that records audio+video and checks whether content is missing is misaligned.
… (demoes the device testing process)
… test device, recording device, test runner are cross referenced by use of QR codes.
… a lot of failures can be detected, both in audio and video. A/V sync can be tested as well; and there are audio only tests.
… If you want to get started, there is a landing page at CTA WAVE (link in presentation).
… DRM testing is an ongoing activity (currently, only clear key testing is supported).
… CTA WAVE is looking to support commercial DRM systems in testing, and has a call for partners / survey out.
… Key survey insights: CBC and CENC are widely used.
… and Fairplay, Playready, Widevice are dominant DRM.
… CTA WAVE is looking for organizations interested in supporting the activities
Louay presents results of questionaire, answer by answer.
Louay: wide adoption of major DRM systems named above. There is good adoption of CBC encryption (compatible with the Apple ecosystem)
… which is important because it enables CMAF content to be used across content ecosystems
… so you can have CMAF content for DASH and HLS, and better utilize content storage.
cpn: suggests a separate meeting to further discuss open issues
Louay: agrees with call. Often there are implementation issues, which are surface by these tests (and tests are the only way to surface these)
… one example is switch between encrypted and non encrypted content, as can happen in advertisement breaks
… We have basic tests but priority calls need to be made for which test cases to develop further.
cpn: The presentation contains a lot of practical issues and it is interesting for the group to discuss this.
… and we should have some EME implementers in the room, and get their feedback. Could also do spec fixes on the back of this.
cpn: what can we do to help?
Louay: Help identify scope what needs to be implemented. The survey is a good starting point.
… members that use the EME API could contribute input what needs to be tested, could review the tests themselves, or run the tests.
… If they are willing, could also contribute test case implementations.
… If they are DRM vendors, could help with DRM servers.
cpn: This work is very welcome. The test suite is hugely beneficial for practical interop.
… cpn would encourage people to help as Louay suggested.
… it is a good next step to have a call fairly soon.
… we need to check with Louay and SVTA on a joint presentation. Action on cpn.
Media Content Metadata Japanese CG Update
https://
hiroki: gives media content metadata presentation.
… hiroki is chair of Media Content Metadata Japanese CG
<kaz> MCM-JP CG
hiroki: Explains the background of breakout session.
Hiroki: We have a breakout session coming up
<kaz> MCM-JP Breakout during TPAC 2025
Hiroki: We had a breakout at TPAC 2023 to present challenges
… Example issue, operational costs are rising due to adapting for different platforms, and verification on each platform
… The MCM-JP CG was created address the challenges
… In the breakout we'll present the outcomes from one year of the CG, invite feedback, and include live demonstrations
… The mission of the CG is to promote interoperabiltiy of media metadata across industries
… We gather and share case studies and best practices
… Not motivation to move to a different spec unless there's a specific need
… We're collecting scenarios from different industries, look at issues, solutions, best practices for metadata use in each industry
… A report on feasibility of desired scenarios, combining industry knowledge
… Outcomes: We collected over 10 case studies, documenting issues and solutions and metadata used in practice
… We have identified 6 use cases using only existing industry metadata
… We'll have a live demo in the breakout
… Some scenarios can be implemented using existing industry metadata. They'll be introduced in the demo
… [Shares details of the breakout, Tuesday at 08:30 JST, in Floor 3, Room 302]
… We invite other industries, such as publishers. CG members will be at the breakout. All stakeholders are welcome
cpn: You are listing a good collection of use cases. Looking forward to the breakout tomorrow.
… You have participants from japanese publishers and epub community.
kaz: and metadata providers
WebVMT and DataCue for lighting devices
https://
Ken: Media over QUIC and timed metadata. Some use case studies
… QUIC is a protocol in IETF. For browsers, using MoQ, use WebTransport API and WebCodecs
… MoQ enables very low latency. What are the differences with WebRTC?
… For real time streaming, there are two parts: networking part, and the media processing part
… WebRTC has both of these. Fundamentally, only media data can be handled by developers
… MoQ handles packetisation and transport. It is not standardised in the browser, handled in the application
… Many use cases will be realised by MoQ, high quality and low latency A/V, multi-track video, high resolution audio. Things that are hard in WebRTC
… Media and device orchestration. Timed metadata can be handled in MoQ. With timestamps, we can synchronise media data
… Interactive live screen, with a live venue and a screening
… MoQ-ILS. Live streaming video, audio, and DMX (lighting) data to a remote site, synchronised
… About 0.1 seconds latency can be realised
… [Shows the MoQ-ILS stack]
… The jitter buffer is well tuned for low latency. Timestamp alignment for synchronisation is now done
… [Shows demo]
… At the Kobe develop meetup, we showed a demo. A robot is controlled at the venue site, so not only video data
… This requires two way communication. The robot sends feedback to the controller side
Wolfgang: You listed a number of use cases. The use cases that use WebCodecs that are restricted to in the clear content. Does that present a challenge?
Ken: We can use h264 and h265. For browser supported cases, we can use WASM, to have more codecs, and high quality video
Wolfgang: On content protection specifically? Artists might not be willing to use the system...
… Media WG was rechartered to add protection around WebCodecs, not sure?
Ken: EME and low latency a challenge. EME v2 is focused on the frame based approach
Kaz: Is it per-frame or per-second update?
Ken: It's updating every frame. For video, 33 milliseconds synchronisation
Philip: WebVMT and TextTrackCue?
Ken: WebVMT is Video Map Tracks, similar to WebVTT but handles object metadata
… With this timed metadata, geolocation data can be displayed and synchronised to the video
… [Shows a sample WebVMT file]
… DataCue is a JS interface for timed metadata. Difference from TextTrackCue, we can add object data to the track
… WebVMT and DataCue and handle arbitrary data such as lighting devices
… [Shows sample of lighting device in WebVMT and DataCue]
… DMX is used for control of lighting devices
… The data format is simple binary data, 512 bytes maximum. Each value is applied to the channel of the DMX device
… Pan, Tilt, Dimmer, Red, Green, Blue. We can set a start address for each datacue
… Each DMX lighting product has a specific DMX chart for that device
… The challenge is that metadata from the VMT file we create DMX data. so we have to transform the metadata
… Two approaches: Use a binary format of the DMX data as is, or base64 encoded
… Approach2##
… It's easy to transform the data for dedicated device
… [Demo]
… Code is available on GitHub
… For WebVMT, the polyfill works well, and in Chromium based browsers using WebVMT
… [Discussion between Chris and Philip on progressing TextTrackCue constructor and DataCue APIs. Philip mentioned that he hasn't worked on those APIs for a long time, and suggested that a good indication is that if Eric is OK with the direction]
<Zakim> wschildbach, you wanted to ask whether slides / presentation will be available?
kaz: Good discussion that's WoT related. How to handle binary data including streaming data
Ken: Any data, can be included the WebVMT
Kaz: Suggest joining the WoT plugfest!
MEIG / Timed Text Working Group Joint Meeting
Nigel: (Reviews the agenda) Any other business?
(nothing)
TTWG Updates
Nigel: Not much has happened on TTML2. Two profiles being actively worked on. First is for subtitles and captions, v1.3 of IMSC Text Profile
… This is in CR. A couple of changes: introduce the TTML2 feature that allows font variations for superscript and subscript
… The other is to support Japanese character sets better. In collaboration with ARIB
… The only thing we had feedback on, is we have separated the text profile from the image profile in IMSC
… No change, so IMSC 1.3 will be just a text profile
… Japanese feedback, sometimes graphics are used for company logos in subtitles and captions. I'll suggest using reserved Unicode codepoints. That should allow character-like glyphs that can be laid out by the browser
… Goal to get IMSC to Rec soon
… As it's a profile of TTML2, no features introduced. So the test suite is empty
… Any questions?
(none)
Next, DAPT, is for dubbing and audio description. Working towards a new CR Snapshot, the last before Rec
We have a test suite and an implementation report. Awaiting implementing feedback
… It's a good format for transcripts and translations
… A nice stepping stone from transcription to subtitles and captions. It doesn't require styling and positioning
… Any questions on DAPT?
(none)
Nigel: WebVTT is in CR from 2016. Some changes proposed in the last year, to support an attributes block
Dana: Last year we proposed WebVTT as an Interop focus area, it was approved as an investigation area
… Proposed because of low WPT test scores, and promote adoption of VTT across the web
… Investigation area means focus on improving the test suite
… Once that's done it might be a focus area. For now, hard to improve interop score without a test suite
… Started investigation in July. Limited the scope to tests that one browser fails. We've added some new tests as well
… We maintain a spreadsheet of which tests have been investigated so far. Discuss at bimonthly meetings. If we think there's a bug in the test, we file a PR
… Or we file bugs against implementations. Or against the spec
… We propose investigating in 2026
… Calls are an offshoot of TTWG so we don't occupy all the meeting time
Nigel: So they're not W3C meetings?
Dana: No
Nigel: Would be good to have visibility, e.g., minutes
Dana: We have a github repo with meeting notes. Wasn't sure how to bring here
… I'd welcome help on how to do that
Nigel: Let's follow up offline
Dana: So far we've made changes to about 70 tests. We've merged about 7 PRs to the WebVTT WPT repo
… We merged one editorial PR against the spec
… This allowed us to clarify the behaviour tested by 10s of tests
… We have open issues and PRs against the VTT spec to discuss in TTWG
… Issues related to our ability to test more areas of the spec
… One example, that cues move out of the way when the controls are showing
… The tests were written arbitrarily, each browser has uniquely sized controls
… We'll continue working on VTT interoperability, hope to have people join or contribute
… I'd like to ask representatives of web apps, if there are things to allow browsers to render VTT cues. Things not in the spec that should be added?
Nigel: There's a wider discussion to have, MEIG might be a good place. Outside the web, broadcast standards use TTML. So the web is a weird outlier
… CMAF profile requires IMSC, but allows VTT if you want. Weird that the web doesn't support that
DVB Accessibility Implementation Guidelines
https://
Andreas: I'm chair of the DVB accessibility task force, and working for SWR in Germany
… There's a letter, requesting feedback. There's a public draft that anyone can review
… DVB standardises broadcast technology
… Guideline is DVB Bluebook A185.
… It originated in DVB-I, but it's a spec that enables a common UI for broadcast and internet media offerings
… Hides the complexity of getting media services, independent of source, and standardises the internet media offerince
… A standardised portal or interface for content. Many broadcasters and manufacturers working on it. Published as an ETSI TS
… New Bluebook planned next year, A177r7
… The other spec is the metadata used. For accessibilty it defines a model for how to signal a11y features
… We have guidelines. DVB Bluebook A185
… The first Bluebook published in Feb 2025 is a guideline to implement accessibility in DVB-I
… The second, A185r1 is more generic
… This new draft, the main purpose is to document what's in DVB specs
… Different signalling mechanisms, so we describe those for each spec
… Pre-playback signalling, e.g., in EPGs. In playback signalling, to select the different services
… It also adds categorisation of features
… What are we asking from W3C?
… Feedback, is it complete, accurate?
… Different standards using different terminology and descriptions. There's no harmonised way to describe
… so it's understood through the entire chain, end to end
… Nigel is the main contributor
… How it's categorised, and naming of the features
… Audio Description, Sign interpreted video. Hard of hearing subtitles (known as captions)
… Audio related a11y features. Next gen audio or object based audio, or pre-mixed enhanced speech intelligibility, or an enhanced immersion audio mix, or dubbing audio
… New a11y features for media. A programme localised for a certain country. A video with easy to understand language
… Transliteration subtitles - writing the same words using a different writing system
… Content warnings, at programme level, before you watch, or in-programme signalling
Chris: When do you need the feedback?
Andreas: Ideally by end of this month. But we can be a bit flexible. ASAP, but even after November is still valuable
Nigel: Kinds of feedback: is this the right set of features? And is this more useful in a global setting?
… e.g., global alignment on terminology, e.g., in W3C, ITU
… One of the motivations I have writing this, is that over many years I've noticed that people who aren't experts in making media accessible don't know these terms
… And then developers, if they don't know what a feature is, they might not implement, or not implement properly
… The problem might not just be for device manufacturers
Andreas: European norms, also struggle with terminology
… What's used in WCAG vs in DVB not always the same, so good to align on it
Nigel: Yes
ChrisBlume: Explain transliteration subtitles?
Nigel: In Serbia, writing in Cyrillic and in Latin
Andreas: Interest to review? We'll discuss with APAWG in joint meeting this week
… Need media and a11y expertise
Hiroki: I'm excited about this topic, a11y is a key requirement for modern broadcasting services
… As public broadcaster, we align to current web standards, WCAG, including 3.0
… Japan proposed a content discovery system, as example of a common broadcast+broadband schema approach
… Your insight on metadata is a key point
… We've been exploring approaches such as automatically applying WCAG HTML tags using programme information, expressed in schema.org metadata
… I prepare the show metadata in schema.org, Javascript generated. Showed a demo in TPAC breakout session
… I want to continue to discuss this, the metadata vocabulary
… It's important, I think.
cpn: We have a session on Thursday with APA. What's the most useful thing for us to do?
… Review the document, as Nigel described, send feedback on the definitions, terminology,
… anything that's missing. Do we need a meeting to go through it in detail?
Andreas: Whatever we want to do, as always. Nigel and I have worked on it, so the feedback we
… need is from others.
… As we see with APA, it would be good to find people to drive it.
… It is good to get feedback from whoever wants to provide it.
… The wider question is if we should work on a common set of references for media
… accessibility services and metadata.
… In DVB we have different ways to do it. This document is informative, but as I showed before
… in TV-Anytime we have a fixed normative vocabulary. It would be good to bring it all together.
Nigel: I think there are gaps, but also things included that others haven't thought of. One example, is using NGA concepts to make audio accessible
… It might expand horizons of people in the web community
Wolfgang: Yes
… Dialog enhancement, narrative importance. Where would the factor iin?
Nigel: Processing the audio to make dialog easier to understand is one. Other is generating audio to make it more immersive for people who can't see
Wolfgang: Would the document have common ownership between DVB and W3C? is that the idea?
Andreas; THe DVB document is for their own specs
… A more global document doesn't have to be DVB, could be W3C. Look at if there's interest, and then see where is the best place to work on it
Nigel: Some W3C specs are relevant. MAUR, and WCAG3. I think WCAG3 should be focused on barriers to access
… That's a good framing, but can be too abstract for developers
… So write the mitigations for each of the barriers to access. Then you find there's duplication
… This document describes the mitigations, and maps them to the barriers to access. Makes it more concrete
Regulatory changes
Nigel: Two things I'm aware of. FCC changes, and in EU, the EAA. EN 301 549 being updated
… They both potentially impact how we make media accessible on the web
Dana: FCC mandates and end user should be able to change their caption style settings without exiting the web app. They need to be at the system level
… Apps need to show a control that shows the system settings menu
… WebKit proposal to add a method on the HTMLMediaElement to show the caption menu
… To mitigate fingerprinting, the website can't access the preference
… The browser does the work of styling the VTTCue
… The API we propose has optional argument to let the website position where the menu appears, e.g., an anchor node
Nigel: We have time in TTWG to discuss
… Timeline?
Dana: I'd need to check, but August 2026
Andreas: Does it includes metadata?
Dana: Styling, color and font
Andreas: It affects not only web, so TV manufacturers. So good to have alignment
Nigel: Only a requirement in US?
Dana: Yes
Nigel: From BBC perspective, it breaks our subtitles. We include semantic information in colours, e.g., change of speaker
Dana: In Mac or iOS you can check a box to allow the website to override that style
Nigel: The issue is that the colour override is for a single colour, not manipulation of the colour palette to show them differently
Nigel: Any questions about this?
(none)
Andreas: In Europe, EAA, came into force this year
… What does it mean for web standards, web media, and browsers?
… EAA is a follow up to the Web Accessibility Directive, EU 2019/822
… This puts requirements on public sector, and EAA also applies to private sector
… National laws must be at least as strict as EAA. Should now be law in EU contries
… It applies to any product that provides access to AV media: TV and STBs
… For services, any service that provides AV media
… or servies that provide accessibilty services
… Requirements, all comply to WCAG type requirements
… For media, components need to be fully transmitted and displayed with adequate quality, synchronised, and give user control over display
… That means, if you have an MP4 container and IMSC in it, the subtitle needs to be accessible to the user
… The regulation applies to products and services, not browsers directly
… The base technology to implement regulatory requirements
… Interesting for the IG to look at if web standards area ready. I see gaps.
… On the EN 301 549, a harmonised norm can be used as assumption of conformity. If conformant to the norm, i'm also conformant to the EAA
… It should have been ready in June, still working on it
… Plan to be ready next year
… Certain parts on media accessibility in chapter 7
… 7.1 is display, synchronization, personalistion of subtitles, also spoken subtitles
… 7.3 on User controls. Activation via remote control or other interface, at same layer as channe; change or volume control
… Have a more systematic approach? Gaps to be closed?
cpn: You've just quoted some European Norms that use terminology - is it aligned with the
… DVB terminology from the previous topic?
Andreas: No, I think DVB should contact ETSI about that, but I am not sure if they will change.
… They had a very long discussion on subtitles and captions and which to call it.
… In the US "subtitles" means something very different to what it means in Europe.
… A common mapping would be useful.
cpn: Is it as specific as the FCC about user controls, e.g. saying it's a system level thing
Andreas: 2 things - user control means switching on and off.
… For user preferences it only says to give the user choice to change it but it does not say
… where it should be done.
… I think they want to define what needs to be achieved but not how.
Nigel: Synchronization of subtitles and captions. In the work on the EN 301 549 update, this came up. The requirements are slightly tighter
… +/- 100ms of the timestamp
… I sent feedback that this is too coarse. Using TextTrackCue in desktop browsers, it fires within 5-10ms, which is fine
… TV manufacturers say their web stacks, even if they support TextTrackCue, they're more like 150-200ms threshold. The implement using currentTime, which isn't appropriate for this purpose
… A half second subtitle, show 100ms late and clear 100ms early, you half the duration is should be shown fore
Andreas: Blame given to web standards, it was worse years ago but changed now
Wolfgang: Going back to the terminology, subtitles is used differently in different context, so discuss the concept, not the terms
… in different geographies
Real time timed text streams on unmanaged networks
Nigel: Live captions, and streaming of pre-recorded subtitles. How to transport them over the network, including unmanaged networks, e.g., into and out of cloud
… Not talking about distribution to client players. MP4 and DASH/HLS covers that. This is about upstream
… SRT transport is good for transfer on unmanaged networks. Popular for AV media, and now being demonstrated for use with subtitles and captions
… e.g., Syncwords demonstrated using TTML in SRT using the DVB TTML spec, which specifies how to put subtitles into MPEG2 transport streams
Chris: What should this group do?
Nigel: I'm just sharing it. No web standards for sharing upstream of distribution
… It's an application, not a requirement for new stuff
… There's an Audio Description CG meeting, discussing DAPT
[adjourned]