ISSUE-28: (Bugzilla 17327) Script interaction (setting and reading) of most state is underdefined

ScriptStateInteraction

(Bugzilla 17327) Script interaction (setting and reading) of most state is underdefined

State:
CLOSED
Product:
Web Audio API
Raised by:
Philip J├Ągenstedt
Opened on:
2012-05-15
Description:
How scripts observe and modify the state of the audio graph has big consequences for both implementation complexity and ease of use for Web authors.

Observing:

Can long-running scripts busy-wait and monitor changes to AudioContext.currentTime, AudioBufferSourceNode.playbackState, Oscillator.playbackState and AudioParam.value? (HTMLMediaElement.currentTime is held stable during script execution.)

Modifying:

When modification are made to the graph, when do they take effect? Should conforming implementations work with long-running scripts that continuously modify some state, or should all modifications made be applied atomically after the script thread has finished.

Recommendations:

For observing, it's not clear what is the best solution. Showing scripts a stable state is probably simpler to specify, simpler to write a test suite for (since otherwise state may at any time be inconsistent, a big problem in testing HTMLMediaElement) but hard to implement as an after-thought.

For modifications, it seems wise to apply all changes as a single "transaction" after the script has finished running. Otherwise, intermediary states could cause audible glitches if GC is run for even a few milliseconds between the two setters. An example would be changing ConvolverNode.buffer and normalize. If the change to normalize takes effect some time after the change to buffer, one could hear a loud pop or gap of silence generated from the inconsistent combination of the new buffer and old normalize value.
Related Actions Items:
No related actions
Related emails:
  1. [web-audio-api] (AudioBufferSourceNodeModification): AudioBufferSourceNode modification behavior undefined (#77) (from notifications@github.com on 2013-09-11)
  2. [web-audio-api] (RealtimeAnalyserNode): RealtimeAnalyserNode design (#86) (from notifications@github.com on 2013-09-11)
  3. [web-audio-api] (ConvolverNodeState): ConvolverNode state modification (#122) (from notifications@github.com on 2013-09-11)
  4. [web-audio-api] (OscillatorTypeModification): Oscillator type modification (#136) (from notifications@github.com on 2013-09-11)
  5. [web-audio-api] (AudioBufferSourceNodeState): AudioBufferSourceNode state semantics and transitions undefined (#153) (from notifications@github.com on 2013-09-11)
  6. [web-audio-api] (ScriptStateInteraction): Script interaction (setting and reading) of most state is underdefined (#108) (from notifications@github.com on 2013-09-11)
  7. Re: Next teleconference : 6th June (from Olivier.Thereaux@bbc.co.uk on 2012-06-04)
  8. Re: Next teleconference : 6th June (from mage@opera.com on 2012-06-04)
  9. Re: State of issues as of 2012-05-30 (from philipj@opera.com on 2012-06-01)
  10. Re: [Agenda] W3C Audio WG Teleconference, 30th May 2012 (from philipj@opera.com on 2012-06-01)
  11. Re: [Agenda] W3C Audio WG Teleconference, 30th May 2012 (from robert@ocallahan.org on 2012-05-31)
  12. State of issues as of 2012-05-30 (from olivier.thereaux@bbc.co.uk on 2012-05-30)
  13. Re: [Agenda] W3C Audio WG Teleconference, 30th May 2012 (from olivier.thereaux@bbc.co.uk on 2012-05-30)
  14. Re: Summary of issues (from crogers@google.com on 2012-05-22)
  15. Summary of issues (from mage@opera.com on 2012-05-22)
  16. Audio-ISSUE-81 (OscillatorTypeModification): Oscillator type modification [Web Audio API] (from sysbot+tracker@w3.org on 2012-05-18)
  17. Re: Audio-ISSUE-28 (ScriptStateInteraction): Script interaction (setting and reading) of most state is underdefined [Web Audio API] (from philipj@opera.com on 2012-05-18)
  18. Audio-ISSUE-74 (RealtimeAnalyserNode): RealtimeAnalyserNode design [Web Audio API] (from sysbot+tracker@w3.org on 2012-05-16)
  19. Audio-ISSUE-72 (ConvolverNodeState): ConvolverNode state modification [Web Audio API] (from sysbot+tracker@w3.org on 2012-05-16)
  20. Audio-ISSUE-51 (AudioBufferSourceNodeState): AudioBufferSourceNode state semantics and transitions undefined [Web Audio API] (from sysbot+tracker@w3.org on 2012-05-15)
  21. Audio-ISSUE-50 (AudioBufferSourceNodeModification): AudioBufferSourceNode modification behavior undefined [Web Audio API] (from sysbot+tracker@w3.org on 2012-05-15)
  22. Audio-ISSUE-28 (ScriptStateInteraction): Script interaction (setting and reading) of most state is underdefined [Web Audio API] (from sysbot+tracker@w3.org on 2012-05-15)

Related notes:

I'll need to add more detail into the spec. Here's my thinking:

Observing: the states MAY change during script execution. We believe we have good automated test methods for testing things such as .playbackState, and already test them today.

Modifying: I believe it's critical to have modifications to the graph state happen immediately due to the real-time nature of audio (think real-time synthesis where we have a MIDI API, playing notes in real-time). We need to respond to the "command" noteOn() (or start() given a better naming) right away as soon as we know for sure that this is what we want to happen. Any delays caused by waiting for the script thread to finish, gc, etc. will adversely affect this model and would be highly undesirable.

That said, there could be some advanced uses cases (not yet required or demanded by any Web Audio developers) where more complex and atomic changes to the graph will be necessary. For this, I propose something along the lines of:

context.beginAtomicChanges();
...
context.endAtomicChanges();

Once again, I believe I can demonstrate that these atomic changes will be rarely needed so won't overly complicate the JS code for developers in nearly all cases.
But we should support them for certain advanced cases, and will need to work out exactly the details...

Regarding your example of ConvolverNode -- I don't believe this specific example will be a problem, but I can understand your confusion because I haven't fully explained how these work. Give me some time...

Chris Rogers, 18 May 2012, 23:06:21

Issue under discussion. Setting as OPEN.

Olivier Thereaux, 30 May 2012, 20:15:50

[MikeSmith]: https://www.w3.org/Bugs/Public/show_bug.cgi?id=17327

5 Jun 2012, 13:57:12

Display change log ATOM feed


Matthew Paradis <matthew.paradis@bbc.co.uk>, Raymond Toy <rtoy@google.com>, Chairs, Chris Lilley <chris@w3.org>, Staff Contact
Tracker: documentation, (configuration for this group), originally developed by Dean Jackson, is developed and maintained by the Systems Team <w3t-sys@w3.org>.
$Id: index.php,v 1.326 2018/10/13 17:29:51 vivien Exp $