Audio Working Group Teleconference -- 11 Jul 2013

<trackbot> Date: 11 July 2013

olivier: I'm here. Just dialing in.

<olivier> chrislowis: good!

<padenot> me

<olivier> zakim aabb is crogers

Removing #OldNames ( http://lists.w3.org/Archives/Public/public-audio/2013AprJun/0634.html )

<scribe> scribeNick: chrislowis

olivier: we have a full agenda. We may not cover it all.

<olivier> http://lists.w3.org/Archives/Public/public-audio/2013AprJun/0666.html

We discussed, and mostly agreed on what to do with the section on removed/deprecated names.

I'd like some agreement on when and how this section will be removed, given that there is consensus on removing it.

If you think there's a need to plan this, please come forward.

Otherwise, I'll just give crogers the action to remove it.

crogers: I think at this time of transition while devs are adapting their code, it'll be helpful to keep it in there.

<olivier> (section is https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#OldNames )

crogers: although it's true we have most of the information on the mozilla wiki, I think it'd be valuable to keep it there.
... for now, but I agree that we'll probably remove it when it comes to that time.

Zakim: IPcaller is me

ehsan: I don't have a strong feeling about removing it now - as long as it doesn't make it into any final publication

olivier: not hearing a strong opinion to remove it asap, so I'll defer to crogers to make the call as weditor about when to remove it.

<olivier> ACTION: crogers to remove the #OldNames section from the webaudio spec [recorded in http://www.w3.org/2013/07/11-audio-minutes.html#action01]

<trackbot> Created ACTION-64 - Remove the #OldNames section from the webaudio spec [on Chris Rogers - due 2013-07-18].

olivier: onto the more meaty "fixing the race condition issue".
... ehsan noted that people who have an opinion (roc, in particular), cannot be on the call, so we may not make a decision today. But a quick chat on "how" we're going to solve this, at least.
... there's been a lot of discussion, but there's not a clear consensus emerging.
... there has been suggestions that demos showing behaviour one way or another would help.
... can people give some options to how we might get to consensus.

/me: olivier apologies

ehsan: if we don't want to make any changes in the IDL, I think the only option we have is to neuter the buffers.
... if we're willing to accept changes to the IDL, then jer's proposal about making ArrayBuffer's immutable would fix the problem.
... there's also the performance issue that crogers raised, I think we all agree we want to avoid memcopy's as much as possible
... but I don't think I have enough information on whether the proposals we have would be acceptable.
... I'd like to ask crogers to share some more information on that.

olivier: thanks ehsan, first to gmandyam on the Q

gmandyam: are we talking about defining an extension to the ArrayBuffer type and making that immutable

ehsan: my mistake, I meant the AudioBuffer, not ArrayBuffer.

crogers: I'd like to give my overview of what I think the situation is:
... firstly roc pointed out a security bug in our implementation, which we have a fix for, I don't think there's anything really "race-y" about this in the normal context of the API.
... it's true that it can be if you immediately modify the buffer, but that is an extremely contrived example.
... we have over 2 years of experience from developers, both novice and experienced, and we have never seen any problems around race conditions with this issue.

olivier: you're saying it is possible to get race conditions, but is very contrived. A question: would it ever be desirable to allow it to happen at all?

crogers: normally, you wouldn't want to tempt it into a race condition as such. There would be no predicatable effect that you'd want to hit with this.

olivier: my second question, and bear in mind I'm not up to speed on the detail, you say there would be a negative impact to fixing this (unlikely) race condition. Do you have any way to showcase that negative impact?

crogers: it depends if we are talking about copying buffers of data? If we are, they can be quite large (audio sprites, with lots of audio samples in a single buffer, for example). It's bad practice then to be in a situation where these have to be copied.

olivier: ehsan mentioned that we agree that memcopy is something we want to avoid.

crogers: ok. The second point: getChannelData is designed for simplicity for devs to get access to the data.

<olivier> ak ehsan

crogers: any changes that require the dev to understand when things are copied by adding bells and whistles to the API would be a bad idea.

ehsan: roc had 2 examples. One that showed an AudioBuffer being modified by calls to getChannelData. The second showed modification of the buffer before sending it to a web worker.
... I don't believe the second example is contrived: it's something I did myself in the early days of the spec.
... just because we haven't heard from developers hitting this problem, doesn't mean it isn't a problem.
... if there are edge cases in the API that cause bad effects, there will be people who hit them. While we have the chance, now, to mitigate this we should take this, or it will hurt interoperability. Code might work because of timing issues on my machine, but may not work on another because of a change in hardware, for exampe.
... I don't think we can say that this is a problem that won't occur.
... the other point about the memcopy situation, by adding a stage where the buffer is neutered.

jussi: I agree with ehsan, we should race conditions, even if the examples are contrived, I don't think race conditions are desirable, even if it's a rare case.
... we just want to avoid them in general.

jernoble: two related comments, when we brought up the neutering code path to our performance team they brought up two issues:
... when the JS compiler looks at those ArrayBuffers it can assume that it doesn't change,
... they would like us to come up with a solution that doesn't involve neutering.
... secondly, it would be unfortunate if memcopy is required in the API, if we could arrange for it that it only happens when people do unusual things, such as modifying a buffer when they get it back, then it might be ok.

olivier: we need to decide whether we are going to fix this, if so, how we are going to fix this, and *when* we're going to do it.
... the question of when is important because we want to start to stabalise the API.

ehsan: I have a question to jernoble about the problem with neutering ArrayBuffers. The only buffer that gets neutered is the one that getChannelData has previously returned. If the normal use case is for that ArrayBuffer to not be modified, is that still a problem?
... I wanted to confirm if under this scenario, it'd still be an issue?

jernoble: as far as I understand the proposal. It would require a neutering for normal operations, not just in the exceptional ones.
... I would like to see a solution that is as reasonable for developers and implementors. That's why I proposed the AudioBuffer solution.
... if we were to take a vote on whether we need to fix this right now, put me on the side of not taking it as a big deal for now.

ehsan: I believe we should address this problem.
... on the topic of neutering the AudioBuffers - if we don't do that, we would have to make breaking changes to the API, and we have differing opinions on how big a problem that would be.
... If we want to avoid neutering, but believe we must fix this, then we need to make changes to the IDL.

olivier: I get the feeling we're in a vicious circle here to a certain extent. I think the arguments are clear and have been made very well. I think we need a proposal for how to break the deadlock.

joe: my opinion is that we should make clear that after the effect changes to ArrayBuffer have no defined effect, but that we don't need to explicity prevent the race conditions.

crogers: ehsan was concerned that there might be a case where something sounded usable on one browser and not on another. I don't think there would be a case where that would be an effect that someone would want to exploit.

ehsan: between mobile and desktop though?

crogers: yes, I agree, but I don't see there being any kind of developer code that would be sensible that would "take advantage" of this effect on one browser over another. It's possible, roc wrote it down, but it would just be "wrong code" that would produce unusable results.
... for example, using setTimeout to modify a gainNode's gain.value and seeing that it would have different effects on different browsers.
... this has been out there, and has been used a lot by excited developers, it's true that it's a prefixed version, but we deal with seasoned and experienced developers, and I think that counts for something.

olivier: I'm going to make 2 proposals here:
... I get the sense that the burden is on the proponants of change. 1) I'd like a volunteer to find out if someone out there has an opinion on the potential of the problem of race conditions. If there's a w3c position on whether we should address race conditions at all cost, then that's one way out.

2) I don't hear consensus that roc's demonstration of the race condition is realistic, is there a more convincing example we could come up with?

ehsan: I don't believe anyone else, accept crogers and jernoble on this call, agrees that leaving these race conditions in is acceptable. On public-audio everyone agrees it should be removed.
... I agree with crogers that it is bad practice to write code like this, but you can't argue that people could come to rely on the race condition without realising it.
... you've never seen this kind of condition before, you test it on FF, Chrome on your desktop, and you believe that your code will work everywhere for all time, and that may not be tru.e

olivier: that's a good point. But what we're dealing with is a risk assessment. We know there is a risk, but we should try to concentrate on how bad the risk is, and what we should do about it to get a resolution to the issue.
... I do not have a good way forward to balance the risk, the cost of a fix, and the cost of leaving it there.

ehsan: I do believe that this call is not the right venue to come up with a conclusion on what we should be doing. I don't know what the process should be.
... there's two sides to the argument, we disagree with each other, and I don't know how we're going to make progress unfortunately.

olivier: the typical process is that someone brings an argument that is convincing enough, but that is proving problematic in this case. We've had a good go at it, let's move on for today and carry on on the list, I'll think about it in the meantime.

web midi issue 2 ( https://github.com/WebAudio/web-midi-api/issues/2 )

olivier: next agendum. cwilso can we have a summary of where we are on this issue?

<cwilso> https://github.com/WebAudio/web-midi-api/issues/2#issuecomment-20687489

cwilso: sure, we've shuffled around a few designs, what we've come up with is a map-like concept that will look something like what we have in our build today, but is much more forward-looking in its design.

olivier: jussi and marcosc, are you happy, or happy enough with the map-like concept?

jussi: I'm quite happy with the map-like concept, and much happier with it than my original proposal, so I'm glad
... I think we're coming close to a consensus on this one as well, from what I'm picking up - I'm happy as it's been a long-standing issue.

olivier: so you need more pairs of eyes looking at it now?

jussi: yes, I believe so.
... actually, the biggest concern is getting implementation experience, so the ball is now in implementators court to see how implementable in the real world it is.

olivier: the current implementation uses the old interface, does it cwilso?

cwilso: the one in Chrome Canary uses something that's in one of my editor branches, it's not quite at the proposed stage, but it's getting there.

crogers: I'm not sure I understand the API - could you give an overview?

<olivier> what it looks like -> https://github.com/WebAudio/web-midi-api/issues/2#issuecomment-20687489

cwilso: the easiest way to get up to speed is to go to the github comment link:

cwilso: you can iterate through the values, but you won't use a for loop the way I have it declared with array's you'll populate through it using `for i in`, take a look at jussi's gist for example.

<cwilso> jussi's gist: https://gist.github.com/jussi-kalliokoski/5960415

<cwilso> for ( let [ id, port ] of midiAccess.inputs.items() ) {

<cwilso> let option = document.createElement('option');

<cwilso> option.value = id;

<cwilso> option.innerHTML = port.manufacturer + ' ' + port.name;

<cwilso> options.selected = !!selectedPort && selectedPort.id === id;

<cwilso> select.appendChild(option);

<cwilso> }

<olivier> marcos

marcosc: with ES6 we start getting things like map and so on, so we were thinking about things in an ES5 world. What jussi has done is show how the problem can be addressed using ES6 syntax.

crogers: being an implementor, or one of them, it would be nice to get some developer feedback from webmidi developers. My preference is for something simple, but I can understand the other points of view.

marcosc: it's been difficult to get much feedback from devs, as we haven't had an implementation for them to play with, now with the canary build we can ask developers for a bit more feedback, which is exciting.

crogers: I agree.
... I think the webRTC api has something similar to get audio and video tracks.

olivier: listing the tracks in the webRTC?

gmandyam: that has to be possible as they have track ids, some basic "capabilities"

<crogers> https://svn.webkit.org/repository/webkit/trunk/Source/WebCore/Modules/mediastream/MediaStream.idl

olivier: has there been, in the webRTC group, discussion about how we give developers access to the channels and streams.

gmandyam: I think I'd agree that what they ended up with in webRTC isn't that elegant.
... but I still can't get the group there to agree about whether cloning results in something stuctured or not.

crogers: I was just pointing to webRTC as an example, but we should just consider how to access channels etc.

marcosc: I think now we just need some demos, like we have for web audio, to help get some more feedback.

crogers: I don't object to the map approach, I'd like to see something simpler, but I don't object.

<olivier> RESOLUTION: there is consensus around the map solution for web midi issue 2

olivier: then we should record that we've reached consensus around this issue, and move towards getting more feedback.

cwilso: #77 I'm going to leave open for now.

<olivier> issues for webmidi -> https://gist.github.com/jussi-kalliokoski/5960415

cwilso: #59 needs an example, I'm going to add one.

<olivier> I meant - https://github.com/WebAudio/web-midi-api/issues?state=open

cwilso: origin scope needs to go in, I haven't got that in.
... #2 is discussed
... and virtual ports is marked as a v2 issue.
... Any disagreement to those, bring it up on the list.

olivier: agreed, I'll note those in the summary of this call.
... and we'll give everyone the week to register disagreement.
... thanks everyone for participating. We *are* making progress, even if it's not easy. Speak to you all in 2 weeks.
... adjorned.

<olivier> s,issues for webmidi -> https://gist.github.com/jussi-kalliokoski/5960415,issues for webmidi -> https://github.com/WebAudio/web-midi-api/issues?state=open,

- DRAFT -

Audio Working Group Teleconference

11 Jul 2013

Attendees

Contents

Removing #OldNames ( http://lists.w3.org/Archives/Public/public-audio/2013AprJun/0634.html )

web midi issue 2 ( https://github.com/WebAudio/web-midi-api/issues/2 )

Summary of Action Items

Scribe.perl diagnostic output