Meeting minutes
Audio Session preferredSinkId
Youenn: There's a preference in general to have this kind of API. Aligns with iOS. Hard there to have two media elements each going to a different speaker
… So having a more global way to send audio to a selected speaker makes sense on iOS. Are there similar principles on Android?
… There's a question of which WG would develop this API
Sunggook: We have feedback from partners who want to control audio outputs across all frames, cross-origin
… Media can play in cross-origin iframes
… Cross-origin iframes play to the default device, which is the system default
… There's a mismatch, so they want to control via a single API
… Each frame could set its own output via AudioContext setSinkId
… So we provided a new API for the whole app, so as long as they set the default, it affects the align and individual frames
Tommy: Do both proposals accomplish that, or is there some aspect that differs?
Sunggook: Not clear with AudioSession how it handles the cross-origin case, and how frames interact with each other
Youenn: I don't think we went into that details, so it's open to discussion
… It's syntactic sugar where you put it, in AudioSession or on MediaDevices
… The 3P iframes use case makes sense to me
… Good idea to use the heirarchy of iframes to restrain where the audio goes, based on the heirarchy
… That's independent of API shape
… I'd like to fuse the two proposals into one, and discuss which WG works on it
… I think it's tackling the same issues, and have one solution
Tommy: Does AudioSession include cross-origin iframes as part of the same session?
Youenn: Not currently. You have mutually exclusive sessions, so if one is playing, others are suspended
… If you don't set explicitly, you might not have the same heirarchy
… For existing pages, use of AudioSession shouldn't be disruptive
… So if you're not using AudioSession, two iframes shouldn't be disrupted. But setting the audio session type is where things kick in
… If you have an iframe using an AudioContext, under the hood it would be similar to the ambient audio session
Tommy: If we're saying that the website sets its audio output device on the top level audio session, then it should say any iframes should go through he same audio session
… Would it make sense if the user creates more than one audiosession?
Youenn: You create one top level, or refer to the top level. There's one AudioSession per document
Tommy: And is it easy for the top frame to set the output device for the cross origin iframes?
Youenn: It's difficult. 3P iframes would have different ids as they'd be partitioned
… If you have a cross-origin iframe that's a cousin, but not a child? We'd need to define
… You could find the top-level document, have a property there, or something like that
Sunggook: If there's a nested iframe, it could call setSinkId and it affects all, as current behaviour
… Interaction with the existing setSinkId on AudioSession and media elements?
Youenn: AudioSession would be the default. Check the media element sinkId, then go to the AudioSession in the document, then go through the heirarchy, just like for media devices
… That's a potential algorithm
Sunggook: So with nested iframes you'd have to check the parent's parent etc
Youenn: Yes
Alastor: We can have multple AudioSession per iframe. But only one should be active. If we have multiple, each has its own id set, we'll only make one activated
… So in this case we'll encounter that setting sinkId on the AudioSession makes it more complicated, IIUC?
Youenn: If you' don't set the audiosession type, it won't interrupt. You can have multiple active. It's only when you set the type to "playback" it'll suspend the others
… Only this particular iframe will be able to play audio
Alastor: So if I have an embedded iframe that sets the sink id to a specific output, the output will be routed, but if the main page sets it's own id, that woud override it?
Youenn: Depends. If it interrupts, then the iframe might be paused. If not, the sinkid will go to the top level
Alastor: So only one should be playing audio
Youenn: If two are both set to ambient, they don't interrupt each other
… if iframe A sets idA, and iframe B sets idB, then iframe A and all its children go through idA, and iframe B and all its children go through idB. Top level page goes through the default
… We can define these results regardless of where we put the API
… Need to decide which is more natural for web developers. And in future, if you create multiple audio sessions programmatically, is it good to be able to set an id there or not?
… I'd like to concentrate on these questions, to help decide where the API lives
Tommy: If you have a top frame and iframe and each has an audiosession, the iframe can interrupt the top level?
Youenn: If set to playback, yes
Tommy: There isn't a parent-child relationship in general?
Youenn: Not today, no
… If I were to specify it, i'd have a slot in AudioSession, then the algo would go thorugh the heirarchy chain, when it finds one not null, use it
… (or on MediaDevice)
… Not sure we're ready for that yet, it's for the future
… Could be a surprise for developers to put AudioSession. In the spec it has a list of all the audio producers it's tied to
… It's handy for saying that this group of producers goes to that speaker itselft
Chris: This is an argument for putting it into MediaDevice
… MediaDevice allows enumeration, and getUserMedia, and getDisplayMedia
Sunggook: In terms of iframe relationships, I'd think of setPreferredSinkId as a simple global API.
… It's an extension of the exisitng setSinkId, but affecting the whole application
… If we put into AudioSession, the sinkiD may be confused with using setSinkId
Alastor: We have a dedicated setSinkId on mediaelement. We could put onto MediaDevice first, then when we have more experience, revisit AudioSession, as an idea
Youenn: In that world, you'd setSinkId on AudioSession as well?
Alastor: Yes. The AudioSession let's you set the type, so should be able to set the output device there too
… If setSinkId is more fit for mediaDevice, could have a different label for the override, and different granularity
Youenn: That could work. If we have two, then we could further simplify setPreferredSinkId
… If you have a top-level page and two iframes and children, set the preferred id in iframe A and B, do you need the heirarchy rules, or have it as just global
Sunggook: In our proposal it's a global, no heirarchy
Tommy: So an iframe could set it and it affects the top-level frame?
Sunggook: Yes. But the iframe could continue to use setSinkId
Youenn: There's a speaker-selection feature policy. 3P iframes could be allowed this global switch
… I'm wondering how it's working. If one iframe selects speaker A, another B, how to know which to use?
Sunggook: The frames can override the default by calling setSinkId
… if they call setPreferredSinkId, it affects the top level frame
Alastor: I'm concerned an iframe might misuse the API by altering the main frame's behaviour
Sunggook: The top frame would set the permission for the iframe, and the iframes need to be same-origin with the top level
… Use the speaker-selection feature policy, we don't propose creating a new one
Tommy: Any reason not to make it heirarchical, so it affects you + children?
Sungook: We haven't thought about that, as our partners want a global, and it comes from one of their own iframes
Tommy: I assume a heirarchical one should solve their problem too
Sunngook: Heiararchical may be harder for developers to understand, and to debug. Hence why we propose a simple global
Youenn: The concern is two iframes fighting by setting the preferred id
… How do they know which sinkId is currently preferred or in use?
Sunggook: It's up to the developer how they'll provide their interface to the user
Youenn: Not sure if it's easier to debug for web developers. In one case it's time sensitive, in other case there's a heirarchy. Heiarchical might be easier
Sunggook: But would it be clear from the user perspective? They don't know the relationship between frames
Tommy: As a developer I think I'd be confused if not heirarchical, and also if it affects siblings. it should be the top-level frame's decision to set for itself and child iframes
Youenn: Can we restrict this API to only the top-level document? Not sure we have other APIs that do that
Sunggook: We could restrict the API, so only the top level can call the API
… Not sure about other APIs. Same origin APIs can do the same as top-level, which is why we proposed to allow
Chris: Not sure we're ready to conclude today. Suggest thinking about whether the heirarchical aproach satisfies the use cases?
Youenn: Also find an example of an API that allows interaction with page-global state
Suggook: I'd like to avoid heirarchical if we can. We could limit to top-level frame only
Youenn: Could be a good start, as it's constrained
Chris: Suggest we think in this more, and come back to it
… And what about MediaDevices as home for this?
Sunggook: We feel MediaDevice might be right
Chris: I feel like we need to discuss the processing model in general as well
Horizontal reviews
Chris: I'm thinking early review from Accessibility may be beneficial
Youenn: We need to do the privacy and security self reviews
… I can write the privacy and security sections, then do the reviews
Chris: And I can start the accessibility self review
[adjourned]