Meeting minutes
New participants
Kaz: first time for Wolfgang?
Walfgang: yes, this is my first time :)
Welcome!
Slideset: https://
Agenda
Chris: TPAC planning
… Liaison statement from MPEG
… Sync on the Web
… DataCue API updates
TPAC 2025 Schedule
Chris: W3C staff preparing for the meeting
… will be held in Kobe on 10-14 Nov
… joint meetings with Timed Text and APA
… may be some schedule conflicts to be resolved
… potential change to the schedule
… GitHub issue 110 there also
DataCue API
Chris: revisit later
Liaison from MPEG
Chris: got a liaison statement from ISO/IEC JTC1/SC 29/WG 2, aka, MPEG
… also got a reminder recently
… questionnaire about market and practical considerations for a next generation video codec
… MPEG is meeting right now in Korea
Kaz's message o the liaison statement (Member-only)
Chris: not sure if they need our official response as W3C
… but does anybody have any additional context?
… interested in the opinions of those participating in MPEG
… any particular insights?
Walfgang: don't have any particular insight myself, but...
… as you mentioned, W3C as a whole doesn't have any particular position about this
Chris: right
Rob: discussion withing OGC
… can paste information here on IRC
<RobSmith> https://
Rob: discussion around next generation video codec still ongoing
Chris: interesting
Rob: that's a CfP which is already past
… encoding images at different level of detail
… image of map and zoomed in the image
… work is done to optimize cloud access
… related to geospatial viewpoint
Chris: ok
… the letter we received mentioned they would welcome input from individuals
Kaz: I contacted them to ask about their deadline, they meet this week in Korea. It might make more sense for us both to have more collaborative discussion
… Someone from Huawei would like to join this call to discuss more
Chris: I'd be happy to organise that. yes
ACTION: kaz and chris to see how to continue this discussion about mpeg questionnaire
Sync on the Web
Ken's slides on "Sync on the Web"
Related article on "NTT Com partners with Yamaha to trial interactive live viewing technology"
Ken: I'll introduce the Sync on the Web CG, where I'm the chair
… The CG is beginning, we are doing some work on immersive interactive live viewing
… The CG is really related to the new media transport technology, MoQ
… It's discussed at IETF
… The MoQT Transport Protocol includes features, but I'll explain the parts related to our sync on the web activity
… Media data and arbitrary data are handled using tracks. Each track is indicated, there's a publish-subscribe system
… Object data is handled as well, e.g., lighting data such as DMX
… Object means a short period of data in a track. which could be a video frame in a video track, and maybe a short period (20ms) of audio in an audio track
… The idea of sync on the web. MoQT can contain metadata inside objects. It's similar to RTP headers, but any data can be contained in the object
… This includes the capture timestamp as well. The subscriber (receiver) can use this to synchronise track data: video, audio, arbitrary data
… Time alignment is done in the subscriber
… Data tracks will be synchronised in the client software
… Demo shows the transport of audio and video, and motion data analysed on the sender side. On the receiver side, it can render avatar data
… We can also send MIDI data, and the receiver can use this to play sounds and effects
… Previously hard to realise, using WebRTC or HLS
… I established the Sync on the Web CG. Current work is use case study and gap analysis, and coordination with related groups such as MEIG
… I'll talk about an exact use case that is our current activity, immersive interactive live viewing
… Live viewing may be a Japanese term, not known so much in English?
… ChatGPT suggested live screening for movie theatres
… Uses satellite communications, real time transfer of audio and video to a movie theatre. People can see the live entertainment show from different movie theatres
… Currently one-way communication, so people can watch a live show, but they can't communicate with the artist, e.g., to send a call and response to the main venue
… It's a frustration from the satellite audience, so we want to introduce some interactive experience for the audience
… I'm collaborating with Yamaha. They're developing interesting technology called GPAP - general purpose audio protocol
… GPAP is a recording technology to record all live stage data: audio, video (sometimes), and lighting (DMX) into one WAV file
… This is realised using Dante, which handles multi-channel data. Yamaha is developing audio formats including lighting data, so not only audio
… GPAP is a recording technology, but I want to introduce live interactive features using GPAP, transport real time GPAP data over MoQ
… Latency is about 0.1 seconds
… Demo. Synchronised data is recorded using GPAP. When recording entertainment, people can enjoy not only the video screen but also the synchronised lighting
… We now introduce real-time communication, interactive live viewing with GPAP
… Diagram of GPAP over MoQ. Audio and DMX data is transported to the sender side. These are separated into audio and data tracks in MoQ
… Each object can include capture timestamp data so that, on the receiver side, we can interleave the data with time alignment
… At the satellite side we can re-generate the GPAP data
… Demo: call and response and audience reaction at the satellite venue is displayed at the back screen
… Here's an article, link: https://
… Gaps for sync on the web and GPAP over MoQ. Currently this cannot work in browsers. The reason is that browsers cannot handle Dante
… On Windows, Dante works with ASIO, not handled in browsers. So we can't realise these services in browsers
… On Mac, because Dante is realised using CoreAudio, the browser can handle the data. But only two types of data in Mac browsers
… To treat the DMX data we have to handle more than two channels so it's hard to realise in current browsers
… I'm planning a local community meetup on 9 November, the day before TPAC. Details not decided, but I'll announce later, and I'd be happy if you could all join the meetup
… Thank you!
Kaz: Thank you for presenting. I was wondering about the data track mechanism, what kind of data, and how many streams can be included?
Ken: The reason the data track is included in MoQ is to treat time related data with video or audio. That is similar to metadata but it can do more. I use DMX lighting data, it has to contain about 1Mbps of data, about the same as audio
Kaz: Also geospatial or position data of each audio object?
Ken: That's an interesting use case for the data track. Such position data would be important to make some kind of immersive audio
Wolfgang: Are you saying MoQ doesn't have a synchronisation mechanism of its own?
Ken: The MoQ spec doesn't have synchronisation, but each object can contain metadata including timestamps
… On top of MoQ we can develop a time alignment service
Wolfgang: So you propose a general purpose timestamping mechanism for any data over MoQ
Ken: Yes, has to be using the timestamp. It's hard to do using WebRTC and HLS
Rob: Definite overlap with the work I'm doing, there's already an arbitrary data sync in WebVMT, which would accommodate this
… What I don't know if it's sufficiently accurate in timing, if the DMX data is similar size to the audio data
… Would be interesting to experiment. Relates to DataCue
Ken: I think the data track can handle any arbitrary data, so VTT or DataCue. Other data formats could be included in MoQ
<RobSmith> https://
Chris: are you using Web Transport for GPAP over MoQ?
Ken: yes, that's true
Chris: prioritization mechanism for track to be resolved?
Ken: now browsers have capability to handle tracks
… what period of data accuracy is important for time alignment
Ken: Accuracy of the time alignment, for video, 33 ms for video frame accuracy
… Browsers can handle this
Chris: for the Dante format...
… is this a media format?
Ken: yes, that's an audio format
Chris: wondering about the relationship with browser's channels
<RobSmith> https://
[[ Dante is the product name for a combination of software, hardware, and network protocols that delivers uncompressed, multi-channel, low-latency digital audio over a standard Ethernet network using Layer 3 IP packets. ]]
Ken: it's an audio operation system
Ken: Dante can handle 128 channels from one audio interface, but from browsers, in AudioWorklet I found that only 2 channels of data can be handled
… If we use 4 or 8 channels of data in Dante we can't use those additional channels. That's the gap, I think
… That's on the Mac. On Windows it's ASIO, not native in Windows and the browser can't handle it
Chris: Could be a topic for the Audio WG or CG
Ken: For big concert venues, the technical people have to manage many tracks of data. In live entertainment cases, TV, to work with Dante would be important
Chris: what's next?
… discussion by the CG?
Ken: maybe some kind of CG meeting
Ken: Need more use cases for sync with media
Chris: if you like, I'd happy to follow up with you
… for additional use cases
Ken: thanks
Chris: to get attention from people for the use cases
… this is really interesting user experience
… synchronize devices at peoples rooms
DataCue API
Chris: final topic for today
Rob: would you like summary?
Chris: yes :)
Rob: I raised a proposal to make a small change to the TextTrackCue constructor
… It has start time and end time, abstract payload, and extended to define the cue types. Relates to the previous presentation
… The change is to expose the inheritance, the naming is strange, TextTrack, but that affects HTMLMediaElement, so renaming that would't bring benefit
… Discussion on constructor access and instantiation
… Benefits in terms of accessibility, TextTrackCue is supported in 95% of browsers, enable community development
… Efficiency, better than VTTCue. There are web platform tests, with variable pass rate
… Proposal to discuss in WICG on 15 July
<nigel> WICG/
Nigel: Eric made a point about compatibility with the Apple proposal, and the PR DataCue #37 was just merged. For me it wouldn't fly as it is now
Chris: I'll try to organise the discussion
Next call
Chris: will be held on August 5
... agenda proposal is welcome
<kaz> [adjourned]