Meeting minutes
A couple of media-related joint meetings during TPAC
cpn: focus of today MSE issues
… first AOB item: TPAC
… media ig planning meetings to assess future direction of web tech for consumer devices
… in collab w/ CTA WAVE and others
… date: 25 October, more details coming soon
… WG members invited to join
aboba: Also joint meeting w/ WebRTC group
… on 26 October
Media Source Extensions
cpn: today's focus, MSE issues
… thank you for cfc replies
… resolved: MSE in workers is ready to integrate
MSE for WebCodecs (#184)
Media Source Extensions for WebCodecs - Explainer
wolenetz: First issue, #184
… MSE for WebCodecs
wolenetz: looking for authors to try origin trial, and feedback on API
… first promise-based MSE API
… summary: MSE would take WebCodecs chunks rather than containerized bytestreams
… expecting interest from player libraries, hls.js, etc, who could use this to avoid re-muxing
MSE-in-Workers: {Audio,Video,Text}Track{,List} IDL in HTML need additional DedicatedWorker in Exposed (#280)
wolenetz: #280 how to expose HTMLMediaElement tracks in worker.
… currently, tracks interfaces exposed only in Window
… 1) do we need tracks in worker?
… 2) if yes, how to update html5 to go about it
cpn: changes to html should go to whatwg
… question: tracks exist both on HTMLMediaElement and SourceBuffer. What's the difference?
wolenetz: w/ SourceBuffer API, you can ask SB what tracks it specifically provides
… note: currently Chromium doesn't expose tracks even in Window
jer: Generally, I prefer MSE in worker to be as close to on main thread as possible. To facilitate easier adoption
… Probably not a big issue for HTML to expose tracks in Workers
Threading lifetimes (#277)
wolenetz: #277 what happens to <video> when worker is terminated
… do we remove the buffers from <video> that MSE-in-worker was contributing?
… or do we let <video> retain what was buffered
… also, should <video> get some notification that worker termination occurred
… currently, chrome's mse-in-workers drops buffers after worker dies, but not sure if it's right
jernoble: I would behave as if source object is removed from <video>. return to empty network state
wolenetz: or potentially trigger EOS behavior?
jernoble: that sounds more complicated. if source object disappears, most transparent thing is to remove the buffers
… could be detected by network state event
… similar to how <video> behaves w/ src set to empty string
cyril: can things be downloaded, buffered, then wasted if worker dies?
wolenetz: yes that could occur, but generally we expect the app to be driving the termination, so just a question about the short period of time following that
cyril: fine to remove buffers if it's just for a short period.
jernoble: "terminating" the worker would be caused by explicit terminate(), which in theory is only done by the client of the worker
cyril: can the app wait to terminate until it's used all the buffer
wolenetz: yes
cpn: what do we need to spec from interop pov
wolenetz: I like Jer's suggestion to detach as much as possible.
cpn: can you reuse media element by setting a new src / srcObject?
wolenetz: yes
Timing of initial HAVE_METADATA transition (#275)
wolenetz: Regarding #275, currently, HAVE_METADATA is spec'ed to be fired synchronously w/ arrival of init segment
… not feasible with workers. any concerns to relax it to be async
… specifically, bottom of init segment received algorithm: when all SB's have init segment received = true
… set's readyState = HAVE_METADATA
jernoble: I think the time where init segment is processed is probably already arbitrary
wolenetz: so it's probably ok to do async here as well? not blocking the worker on synchronously setting readyState?
jernoble: I agree
janivar: might be good to gather data on current browser behavior
wolenetz: will do
jernoble: do we have a wpt test for this? could use that to track differences
wolenetz: test exists, but it's flaky
jernoble: for webkit, we issue have_metadata before updated event
Object URLs vs srcObject (#156)
wolenetz: Regarding #156, chromium regrettably doesn't support srcObject yet
… the original object URL was "autorevoking"
… Chrome changed behavior to let apps explicitly revoke to improve GC ability to collect
… for MSE-in-workers, we need to keep object URLs alive long enough to be used by worker (can't immediately revoke).
… proposal: for workers, what if we don't revoke until after first use.
jernoble: I initially favored keeping the URL, but now I like srcObject. Still, usage of srcObject is very low because of missing support in other UAs
wolenetz: we haven't decided how src would work w/ MSE-in-workers yet. so far just looked at using srcObject
jernoble: for src url, we could maintain autorevoking behavior on the main thread while behaving differently for use in workers
<cpn> https://
janivar: I'm supportive of srcObject. Firefox hasn't implemented it yet.
jernoble: it seems simpler to create object in worker, do attachment on main thread, and have spec language say <video> informs the MSE-in-worker that they're now attached
… vs earlier proposals that considered creating src url in main thread
janivar: beware, as you "transfer" objects, the typical language is that the source-side is "detached"
wolenetz: Up next I'll be addressed Mark Watsons PR comments and taking a look at Spectre mitigations
cpn: to confirm, for workers, we've aligned on srcObject for attachment?
wolenetz: correct
jernoble: something to consider, should <source> elements have srcObject attribute?
Next meeting
cpn: scheduled for end of october. do we need anything sooner?
cyril: I'd like an earlier session to discuss bytestream issues
jer: how about same time in two weeks?
cyril: in the meantime, please add your thoughts to the GH issues