Bug 17388 - (audioparam): AudioParam constructor
Summary: (audioparam): AudioParam constructor
Status: CLOSED WONTFIX
Alias: None
Product: AudioWG - OBSOLETE - Moved to Github
Classification: Unclassified
Component: Web Audio API - OBSOLETE - See Github (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal
Target Milestone: TBD
Assignee: This bug has no owner yet - up for the taking
QA Contact: This bug has no owner yet - up for the taking
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-06-05 12:09 UTC by Michael[tm] Smith
Modified: 2014-10-28 17:17 UTC (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael[tm] Smith 2012-06-05 12:09:11 UTC
Audio-ISSUE-6 (audioparam): AudioParam constructor [Web Audio API]

http://www.w3.org/2011/audio/track/issues/6

Raised by: Jussi Kalliokoski
On product: Web Audio API

Raised by Jussi, going back to a discussion back in spring 2011:
[1] http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0374.html
[2] http://lists.w3.org/Archives/Public/public-audio/2011AprJun/0085.html
[3] http://lists.w3.org/Archives/Public/public-audio/2011AprJun/0086.html
[4] http://lists.w3.org/Archives/Public/public-audio/2011AprJun/0087.html
[5] http://lists.w3.org/Archives/Public/public-audio/2011AprJun/0089.html
[6] http://lists.w3.org/Archives/Public/public-audio/2011AprJun/0101.html
[7] http://lists.w3.org/Archives/Public/public-audio/2011AprJun/0103.html

The discussion focused on the need for an AudioParam constructor in the context of a JavaScriptAudioNode.

Jussi then suggested

partial interface JavaScriptAudioNode {
    AudioParam createAudioParam(name, defaultValue, minValue, maxValue,
units);
}

partial interface AudioProcessingEvent {
    Float32Array getParamValues(name);
}

With which CRogers agreed, but suggested to pass the AudioParam instead of name to getParamValues().

Thread: http://lists.w3.org/Archives/Public/public-audio/2012JanMar/thread.html#msg412
http://lists.w3.org/Archives/Public/public-audio/2012JanMar/0437.html
Comment 1 Olivier Thereaux 2012-06-07 14:31:24 UTC
From the minutes of the audio wg teleconf on April 2nd: 
« The discussion on making it possible for the javascriptnodes to create audioparams for themselves reached some early consensus on making it a "nice to have", perhaps something to include in a future version of the web audio spec. Al will be gathering more input on this. (See ACTION-43).»

Closed per resolution at the 2012-05-02 audio WG teleconference. We will keep this in a "backlog" of potential features for a next version of the spec.
Comment 2 Olivier Thereaux 2012-06-07 14:31:45 UTC
Comment from Marcus Geelnard, 4 Jun 2012, 08:07:19
«
First of all, sorry for being late with input to this. I hope I did the right thing here by re-opening the issue (?).

Adding to the previous discussions, I strongly feel that this is an important feature of the API.

Without the ability to have AudioParams on JavaScriptAudioNodes, it would not be possible to re-implement the native nodes for instance, e.g. for testing and reference implementations. It also makes it far more difficult to do drop-in replacements in an existing graph (e.g. if a developer wanted to make a custom filter node to replace a BiQuadFilterNode in an existing graph).

Generally speaking, it seems logical that the JavaScriptAudioNode is the most generic node with which you can implement "everything". Hence, support for AudioParam seems like a fundamental requirement.
»
Comment 3 Olivier Thereaux 2012-06-28 13:56:18 UTC
We have seen several requests to bring this back since the April 2nd decision to keep it off the v1 scope. Can't seem to find any strong argument against it (major added complexity? Performance hit?). 

I would like to see pointers or arguments against bringing it back in scope, if there are any.
Comment 4 Marcus Geelnard (Opera) 2012-08-21 12:25:13 UTC
Regarding the suggested interface that returns a Float32Array... Returning a Float32Array would inevitably require GC. A better solution could be to fill out a Float32Array that is passed as an argument. I also think it would be a good idea to provide a method for returning a single value, for lessening the overhead for k-rate parameters.

E.g:

partial interface AudioProcessingEvent {
    void getParamValues(DOMString name, Float32Array values);
    float getParamValue(DOMString name);
}
Comment 5 Jussi Kalliokoski 2012-08-21 12:33:56 UTC
(In reply to comment #4)
> Regarding the suggested interface that returns a Float32Array... Returning a
> Float32Array would inevitably require GC. A better solution could be to fill
> out a Float32Array that is passed as an argument. I also think it would be a
> good idea to provide a method for returning a single value, for lessening the
> overhead for k-rate parameters.
> 
> E.g:
> 
> partial interface AudioProcessingEvent {
>     void getParamValues(DOMString name, Float32Array values);
>     float getParamValue(DOMString name);
> }

Good idea!

I also just realized that Chris Rogers's suggestion for passing the param instead of the name won't play well with audio processing in workers.
Comment 6 Marcus Geelnard (Opera) 2012-08-21 12:52:32 UTC
(In reply to comment #5)
> I also just realized that Chris Rogers's suggestion for passing the param
> instead of the name won't play well with audio processing in workers.

Exactly.

Not quite sure what the best solution would be, but for now the "name" string should do (and you'd get an exception if you try to create two AudioParams with the same name).

If string/hash lookups are deemed too costly to do per-callback, we could introduce something similar to uniform/attribute locations in WebGL (i.e. integers).
Comment 7 Ray Bellis 2012-12-03 12:24:19 UTC
I have an alternate proposal.

We probably don't really need the ability to directly instantiate a standalone AudioParam, we just need to allow a ScriptProcessorNode to own them, hence we could ask for them in the ScriptProcessorNode factory call:

    partial interface AudioContext {
        ScriptProcessorNode createScriptProcessor(unsigned long bufferSize,
            optional unsigned long numberOfInputChannels = 2,
            optional unsigned long numberOfOutputChannels = 2,
            optional unsigned long numberOfAudioParams = 0         // new
         );
    }

As far as I can see the AudioParam minValue, maxValue and units fields are all informational, so I've ignored them.

The ScriptProcessorNode does of course need to be expose these AudioParams so they can receive connections:

    partial interface ScriptProcessorNode {
        readonly attribute AudioParam param[];
    };

IMHO there's no need for direct name support - the developer can add a read-only named attribute that references the appropriate index into the param[] array if it's needed.

Rather than the "getParamValues()" value suggested above, I believe it should be possible to pre-calculate the animation curves and pass their values directly as part of the AudioProcessingEvent interface just as inputBuffer data is now:

    partial interface AudioProcessingEvent {
        readonly attribute AudioBuffer paramBuffer[];
    };

I believe the advantage of this approach is that it would simplify usage with WebWorkers - all of the required data is passed in one go.
Comment 8 Jussi Kalliokoski 2012-12-03 18:03:20 UTC
(In reply to comment #7)
> I have an alternate proposal.
> 
> We probably don't really need the ability to directly instantiate a
> standalone AudioParam, we just need to allow a ScriptProcessorNode to own
> them, hence we could ask for them in the ScriptProcessorNode factory call:
> 
>     partial interface AudioContext {
>         ScriptProcessorNode createScriptProcessor(unsigned long bufferSize,
>             optional unsigned long numberOfInputChannels = 2,
>             optional unsigned long numberOfOutputChannels = 2,
>             optional unsigned long numberOfAudioParams = 0         // new
>          );
>     }
> 
> As far as I can see the AudioParam minValue, maxValue and units fields are
> all informational, so I've ignored them.
> 
> The ScriptProcessorNode does of course need to be expose these AudioParams
> so they can receive connections:
> 
>     partial interface ScriptProcessorNode {
>         readonly attribute AudioParam param[];
>     };
> 
> IMHO there's no need for direct name support - the developer can add a
> read-only named attribute that references the appropriate index into the
> param[] array if it's needed.

I like the idea of having anonymous AudioParams, it does indeed simplify things.

> Rather than the "getParamValues()" value suggested above, I believe it
> should be possible to pre-calculate the animation curves and pass their
> values directly as part of the AudioProcessingEvent interface just as
> inputBuffer data is now:
> 
>     partial interface AudioProcessingEvent {
>         readonly attribute AudioBuffer paramBuffer[];
>     };
> 
> I believe the advantage of this approach is that it would simplify usage
> with WebWorkers - all of the required data is passed in one go.

I'm not sure what you're after with this part. Your suggestion has no way of differentiating between k-rate and a-rate parameter inputs, which has implications on performance. I think that we should just change the previous proposal a bit to accommodate:

partial interface AudioProcessingEvent {
    void getParamValues(uint index, Float32Array values);
    float getParamValue(uint index);
}
Comment 9 Ray Bellis 2012-12-03 23:36:10 UTC
(In reply to comment #8)

> I like the idea of having anonymous AudioParams, it does indeed simplify
> things.

Great :)
 
> I'm not sure what you're after with this part. Your suggestion has no way of
> differentiating between k-rate and a-rate parameter inputs, which has
> implications on performance. I think that we should just change the previous
> proposal a bit to accommodate:
> 
> partial interface AudioProcessingEvent {
>     void getParamValues(uint index, Float32Array values);
>     float getParamValue(uint index);
> }

AIUI, whether an AudioParam is a-rate or k-rate depends on the specification of each type of AudioNode.  It's actually not clear to me that k-rate AudioParams make sense in the context of a ScriptProcessorNode.

I'm not intimately familiar with the internals of any of the WebAudio implementations, but it sounds like your second function prototype wouldn't fit with the requirement that k-rate samples are taken on a 128 sample block boundary, given that a ScriptProcessorNode could easily be processing a block longer than that.
Comment 10 Marcus Geelnard (Opera) 2012-12-04 09:28:07 UTC
(In reply to comment #9)
> (In reply to comment #8)
> > Your suggestion has no way of
> > differentiating between k-rate and a-rate parameter inputs, which has
> > implications on performance. I think that we should just change the previous
> > proposal a bit to accommodate:
> > 
> > partial interface AudioProcessingEvent {
> >     void getParamValues(uint index, Float32Array values);
> >     float getParamValue(uint index);
> > }

I agree.

> AIUI, whether an AudioParam is a-rate or k-rate depends on the specification
> of each type of AudioNode.  It's actually not clear to me that k-rate
> AudioParams make sense in the context of a ScriptProcessorNode.

Well, there are certainly use cases where you'd want to use k-rate params. Just have a look at the existing nodes in the spec, and imagine that you want to re-implement a variant of one of them as a script processor node.

I tend to think of k-rate parameters as constants or slowly varying parameters that you want to control in the same manner as any other (e.g. a-rate) parameter, e.g. using knobs in a UI etc.

It's true that you can achieve k-rate behavior in your script by simply picking the first sample from an a-rate parameter array, which could be a performance win for you algorithm, but on the other hand the audio engine would still have to generate all the a-rate samples for you, which is a potential performance loss.

> I'm not intimately familiar with the internals of any of the WebAudio
> implementations, but it sounds like your second function prototype wouldn't
> fit with the requirement that k-rate samples are taken on a 128 sample block
> boundary, given that a ScriptProcessorNode could easily be processing a
> block longer than that.

True. Either the spec has to be clear on this (i.e. specifying that getParamValue() samples at the beginning of the event buffer), or we need to add another parameter, e.g. like this:

float getParamValue(uint index, double time);

I think that the first option would suffice, though. If you need more fine grained sampling, you're probably looking for an a-rate solution anyway, or you can use getParamValues() and decide the processing batch size yourself.
Comment 11 Olivier Thereaux 2014-10-28 17:14:14 UTC
Web Audio API issues have been migrated to Github. 
See https://github.com/WebAudio/web-audio-api/issues
Comment 12 Olivier Thereaux 2014-10-28 17:17:07 UTC
Closing. See https://github.com/WebAudio/web-audio-api/issues for up to date list of issues for the Web Audio API.