17327 2012-06-05 11:19:28 +0000 (ScriptStateInteraction): Script interaction (setting and reading) of most state is underdefined 2014-10-28 17:16:28 +0000 1 1 1 Unclassified AudioWG Web Audio API unspecified PC All CLOSED WONTFIX P2 normal TBD 1 philipj crogers mage olivier.thereaux public-audio dave.null oldest_to_newest 68637 0 philipj 2012-06-05 11:19:28 +0000 Audio-ISSUE-28 (ScriptStateInteraction): Script interaction (setting and reading) of most state is underdefined [Web Audio API] http://www.w3.org/2011/audio/track/issues/28 Raised by: Philip Jägenstedt On product: Web Audio API How scripts observe and modify the state of the audio graph has big consequences for both implementation complexity and ease of use for Web authors. Observing: Can long-running scripts busy-wait and monitor changes to AudioContext.currentTime, AudioBufferSourceNode.playbackState, Oscillator.playbackState and AudioParam.value? (HTMLMediaElement.currentTime is held stable during script execution.) Modifying: When modification are made to the graph, when do they take effect? Should conforming implementations work with long-running scripts that continuously modify some state, or should all modifications made be applied atomically after the script thread has finished. Recommendations: For observing, it's not clear what is the best solution. Showing scripts a stable state is probably simpler to specify, simpler to write a test suite for (since otherwise state may at any time be inconsistent, a big problem in testing HTMLMediaElement) but hard to implement as an after-thought. For modifications, it seems wise to apply all changes as a single "transaction" after the script has finished running. Otherwise, intermediary states could cause audible glitches if GC is run for even a few milliseconds between the two setters. An example would be changing ConvolverNode.buffer and normalize. If the change to normalize takes effect some time after the change to buffer, one could hear a loud pop or gap of silence generated from the inconsistent combination of the new buffer and old normalize value. 68833 1 olivier.thereaux 2012-06-07 15:55:44 +0000 Comment from Chris Rogers, 18 May 2012 I'll need to add more detail into the spec. Here's my thinking: Observing: the states MAY change during script execution. We believe we have good automated test methods for testing things such as .playbackState, and already test them today. Modifying: I believe it's critical to have modifications to the graph state happen immediately due to the real-time nature of audio (think real-time synthesis where we have a MIDI API, playing notes in real-time). We need to respond to the "command" noteOn() (or start() given a better naming) right away as soon as we know for sure that this is what we want to happen. Any delays caused by waiting for the script thread to finish, gc, etc. will adversely affect this model and would be highly undesirable. That said, there could be some advanced uses cases (not yet required or demanded by any Web Audio developers) where more complex and atomic changes to the graph will be necessary. For this, I propose something along the lines of: context.beginAtomicChanges(); ... context.endAtomicChanges(); Once again, I believe I can demonstrate that these atomic changes will be rarely needed so won't overly complicate the JS code for developers in nearly all cases. But we should support them for certain advanced cases, and will need to work out exactly the details... Regarding your example of ConvolverNode -- I don't believe this specific example will be a problem, but I can understand your confusion because I haven't fully explained how these work. Give me some time... 68974 2 mage 2012-06-12 13:03:21 +0000 More comments from the mailing list... Robert O'Callahan, http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0453.html " Chris wrote: > Observing: the states MAY change during script execution. We believe we > have good automated test methods for testing things such as .playbackState, > and already test them today. > It's generally simpler for Web authors if states don't change during script execution (except for changes explicitly made by the script), in particular if scripts always see a consistent snapshot of the state of the graph. That's what MSP does. If we're not going to do this, I think we need a strong argument for not doing it, we'll need a careful description of which states can change asynchronously and how, and we'll need to make sure that scripts can actually safely read and do something with that state (which is often not the case for state that can change asynchronously). Modifying: I believe it's critical to have modifications to the graph state > happen immediately due to the real-time nature of audio (think real-time > synthesis where we have a MIDI API, playing notes in real-time). We need to > respond to the "command" noteOn() (or start() given a better naming) right > away as soon as we know for sure that this is what we want to happen. Any > delays caused by waiting for the script thread to finish, gc, etc. will > adversely affect this model and would be highly undesirable. > I think that deferring changes to happen atomically at the next stable state (in HTML5 terms) makes sense. That's what MSP does. That makes it very simple for Web authors to avoid glitches due to unintended intermediate states, without needing to introduce new atomicity constructs and educate authors to use them correctly. I don't think Chris' objections apply. Authors cannot and should not rely on their API calls happening at particular moments in real time. A GC or other pause happening after noteOn() and before the next stable state could just as easily have happened before the noteOn(). As a matter of general Web application design, scripts should not run long between stable states since that adversely affects almost everything in the page (including rendering etc), so any delays introduced by deferring to the next stable state should be very short. If deferring changes to happen atomically at the next stable state is still considered harmful, I'd like to see more detailed examples of scripts that would be adversely affected. " Philip Jägenstedt, http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0486.html " We strongly agree with Robert. We have negative experiences with race conditions in the HTMLMediaElement API and do not want to implement something that we know will cause subtle and hard-to-reproduce bugs. The specification will require a lot more detail if state changes are reflected immediately, e.g. as in the ConvolverNode example we've given. We suggest defining state modification (both by scripts and the audio graph) in terms of tasks [1] such that e.g. modifying the ConvolverNode properties will queue a task to atomically apply the changes to the underlying audio graph. [1] http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#task-source " Robert O'Callahan, http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0490.html " I suggest batching all changes between each stable state and then applying them atomically. " Philip Jägenstedt, http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0494.html " I think we mean exactly the same thing, did I use the wrong terminology? " Robert O'Callahan, http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0536.html " Queuing a task to perform an action is rather different to performing the action at the next stable state. " 68985 3 philipj 2012-06-12 14:41:48 +0000 Robert is correct, queuing a task doesn't guarantee that the steps will be run before any more scripts are run, awaiting a stable state is better. That concept is defined in http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#await-a-stable-state 113848 4 olivier.thereaux 2014-10-28 17:13:32 +0000 Web Audio API issues have been migrated to Github. See https://github.com/WebAudio/web-audio-api/issues 113991 5 olivier.thereaux 2014-10-28 17:16:28 +0000 Closing. See https://github.com/WebAudio/web-audio-api/issues for up to date list of issues for the Web Audio API.