Meeting minutes
Introduction
Chris: Welcome, in this
meeting we'll recap some of the main outcomes from TPAC and talk
about next steps.
… Also a discussion around the Multi-Device Timing
CG, where I saw it was proposed to be closed.
… Anything else?
(nothing)
Slideset:
https://
Next Generation Audio and Video Codecs
Chris: NGA proposal
presented at TPAC MEIG meeting.
… I want to figure out where to go next with
it.
… Outcomes from the meeting: gap analysis for
audio, also consider video.
… Also there was opportunity at TPAC to talk with
people outside of the meeting.
Bernd: Not quite sure about the gap analysis.
Chris: Let's figure out
what it really means
… We can start with use cases.
… Some of this in the document we prepared and you
presented at TPAC, so we have some material already.
Bernd: Should we come
with a proposal? Learn from the industry what's missing on the web
compared to other platforms. We're in contact.
… What's done so far and what's still missing to
deliver something which is a good user experience.
… Have some service provider to show there's the
need.
Chris: This is a conversation TV and media industry is having elsewhere?
Bernd: My understanding
is maybe some service provider can address today's technology
… but I'm missing how should we work on
that.
Chris: We could write a
document to capture the points
… that kind of use cases and gap
analysis
… that's what we can turn into a
document
… more technical analysis too
Bernd: And have a meeting for that purpose?
Chris: Yeah, we can do
that.
… We can use this meeting also, or can have special
topic-focused meetings.
… If we have a lot to talk about in the main MEIG
meeting, we can schedule additional meetings.
… We can be flexible.
Bernd: Sounds
perfect.
… We can already work on the document,
… also dedicated meeting would be good.
Chris: Sounds good to
me.
… Happy to talk with you separately about getting
the media industry input needed.
Bernd: ok.
Chris: I'm not sure
about next-generation video at the moment, I don't know the status
of development.
… That would be interesting, to hear from somebody
with expertise to give an introduction to the
technology
… could also cover that
… if you're designing APIs
… bigger problem with focusing on
… but I think moving up would really require
special knowledge
… might be longer term
… was interesting a couple of people showed
interest
… think some of the materials are already
there
Bernd: yeah
… some requirements there
… real deployment would be helpful
Chris: yeah, industry
support would be good
… any other thoughts?
Niko: One question, a
gap analysis describes a difference between two points.
… What are those two points?
Chris: I see two ways
to approach this. One is between the user needs, and the current
technical capability the Web has,
… so starting from the use case perspective,
looking at what's supported by browsers today and documenting
what's missing.
… The alternative, and the approach we've taken so
far, is more codec-centric.
… We have some particular technology that enables
use cases, so we look at how to bring that to the web.
… So I think we need some use cases and do the
former analysis, e.g., what can be done with Web Audio
today.
… We have some of this already.
Niko: ok, that matches my understanding from TPAC.
Chris: We can follow up this separately on the industry side.
Media WG update
Chris: A few
updates.
… Media WG just agreed to publish the Audio Session
API spec.
… It allows websites to control how multiple audio
sources get mixed, e.g., music and notifications, etc.
… The music volume might be temporary adjusted so
you hear the notification. Feedback on the spec is
welcome.
… WebCodecs, VideoFrame metadata for camera
effects, e.g., background blur
… A lot of the development is driven by RTC use
cases, very low latency streaming.
… Camera effects is an example. Do we need to talk
about requirements from media creation use cases?
… Finally, there was some agreement at TPAC to look
at Text Tracks in MSE,
… so support for timed text carried in media
container files or the video bitstream.
… I'm hoping to organize a dedicated session on
that soon.
… It's something my organization is interested
in.
… Any thoughts/questions?
(nothing)
Multi-Device Timing
Chris: This has been a long term topic, discussion at TPAC 2016.
Ingar: did some work to
generate interesting forum around timing
… starting at TPAC 2016
https://
Ingar: We put this into
a standardization draft, just a pending some interest.
… Should I say something about the initiative
itself?
Chris: Yes
Ingar: Opportunity with
the web, to have timing. It's a collection of dynamic
documents.
… Think about it as a thing for rendering data,
visualisations.
… Anything could be sync'd up and played back
together. Multimedia frameworks would support it, but the web
doesn't have that.
… Hard to use them together. Idea of timing object
is to make a generic spec for a timing model, timing control and
timeline,
… and make it independent of specific
frameworks.
… It's been successful, we use it a lot, to solve
many problems.
… There's interest in syncing things to video or
audio.
… The missing piece isn't much. For what should be
standardised: not a lot.
… The thing that would matter is the media element
having a mode for sync. Media element wasn't made for
that,
… e.g., if you ask it to skip it'll spend some time
doing that, then you'd need to sync again.
… Solution to have a sync mode in the media
element. Instead of passing a command, pass a timestamp vector to
aim for.
… It would open up the important use
cases.
… The multi-device aspect isn't necessary. Timing
is more important.
… Not only entertainment media, also recording and
replaying sensor data.
… Shame there was no interest from
browsers.
Chris: At TPAC 2023
Apple presented the Media Session Coordinator proposal. The use
case is co-viewing of media content.
… Not at the same timing accuracy level as Timing
Object enables.
… But even that didn't get traction with other Web
browser vendors.
Ingar: That's just one
use case for timing,
… but three are thousands of use cases
there.
… Media Session Coordinator bundles together timing
with session management,
… whereas Timing Object focsues on just the sync
part
Chris: Yes, those things are kind of orthogonal.
Ingar: Right
Kaz: I've been working
with MEIG for a while, and also WoT group, and the multi-modal
group 10 years ago.
… Do we need to think about synchronisation just
for web browsers, or maybe also non-browser entities like IoT
sensors, surveillance cameras?
… Could use exsting statechart mechanisms
also?
Ingar: It's not
specific to the web, yes. Even though we named it "multi-device
timing", we didnt' really look at the multi-device part, more the
timing object.
… Having a standard way to do the multi-device part
could be a consideration. But the timing part opens up the use
cases.
Kaz: During TPAC2024, Audio WG people were intersted in precise sync. Could create a document with use cases.
Ingar: Sure, the
findings back then, the limiting factor was the ability to locally
control the media element. We got that down to around 7-10ms on
reasonable devices
… For general sync of timing objects across web
clients, was 1 or 2 ms. Good enough for a lot of applications, or
maybe not for some specific audio apps.
Kaz: Technically, need some precise base clock.
Ingar: You need a
clock, and can rely on the server you connect to. That's the way we
did it, and sync based on network latency, you can get to 1-2 ms
off.
… You shouldn't have to depend on anything, the
important thing is availabiltiy of a sync signal, and see the clock
as an app level object and not an infrstructure resource
Kaz: So Timing Object is like a virtual clock?
Ingar: It's like a logical clock, stopwatch.
TPAC Breakout: Sync on the Web
Chris: There was
related breakout on sync on the Web
… Someone mentioned the Web Audio conference, which
sometimes discusses millisecond-accurate
synchronization.
… APIs for timestamps or latency for
input-generating APIs, Gamepad, USB, MIDI, to enable the app to do
synchronization.
… Some points made for digital twins as use case,
cloud gaming as well
… Discussion on that happens within the Web &
Networks IG
Ingar: There's overlap
there, as long as you have a timing object for event handling
… sometimes people monitor the latency, can be very
complicated but could simplify it.
Chris: A broad question
to the group. Is there a renewed interest in this topic, and
something to follow up?
… Ingar, you have solutions that are working pretty
well.
… Maybe there's not so much to be done
here?
… It seems a shame to close the CG...
Ingar: got a report to sending the content accessible
Chris: All the archives can still be accessed if the group closes.
Ingar: There is still
an opportunity there,
… one thing you could do is going back to the
industry to find people interested
… many type of application domains.
Chris: We heard from
Mr. Komatsu from NTT, on how to do sync using Media over
QUIC.
… This "Sync on the Web" session was a reminder to
think about the topic more generally.
… We should let him know about this discussion and
see what he wants to do next.
Ingar: We talk to
industry from time to time, a media use case is the lottery show,
with video and synchronised text and graphics.
… It seems such an easy thing, but that's pretty
hard for developers to do today, and special cases with media
players on different platforms.
… It's really a showstopper for media industry
business.
Francois: Ian and Dom
are handling the CG procedure and checking the progress of
CGs.
… I wanted to confirm that closing the CG itself
would not lose the content, nothing would be lost.
… We could write a blog post to summarise and point
people to MEIG for future discussion when we revive the
topic.
… I also wanted to note we tried for years to push
browsers to consider the spec and the approach.
… The feedback has always been to expose hooks,
latency measures, but the leave orchestration up to the
application.
Chris: A potential
improvement could be around the currentTime and playbackRate
attributes?
… The HTML spec doesn't set any accuracy or
requirements.
… And the ability for browsers might vary by
platform, e.g., desktop, mobile or TV.
… This may not be a spec issue, more about quality
of implementation.
Ingar: even the sort of
properties
… is that good enough?
… skipping is happening
… that is always a source of noise
Chris: Focusing on that level might be productive. Show the practical data we have on accuracy, raise browser issues.
Ingar: Just having a sort of test with some kind of video content might be useful.
Louay: Regarding
testing, I'm working on the CTA WAVE test suite.
… Video playback tests include the current
time.
… there is a feature on Vision Pro devices called
SharePlay.
… You can select any applications to listen to
music or watch video and synchronize.
Chris: Yes, Media Session Coordinator was essentially bringing that feature to the web.
Ingar: It's important to open up it for anybody, so it's not limited to any vendor specific protocols.
kaz: Another discussion place is Web of Things. I'll talk with WoT chairs also. Time synchronization is not yet defined within the model.
Content Authenticity
Chris: Another topic
from TPAC was Content Authenticity: Originator Profile and C2PA,
which we've discussed here before.
… Discussions are happening about a potential
workshop next year.
CTA WAVE Collaboration
Chris: We had a meeting
in September and at TPAC about their streaming media test
suite.
… We're planning to have a follow up meeting to
look at some MSE specific issues, hopefully soon.
Next meeting
Chris: Same time next month: 3 Dec at 15:00 UTC.
[adjourned]