This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 20039 - The callbacks of the decodeAudioData API are underspecified
Summary: The callbacks of the decodeAudioData API are underspecified
Status: CLOSED FIXED
Alias: None
Product: AudioWG
Classification: Unclassified
Component: Web Audio API (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal
Target Milestone: TBD
Assignee: Chris Rogers
QA Contact: public-audio
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-11-22 00:17 UTC by Ehsan Akhgari [:ehsan]
Modified: 2013-01-18 15:52 UTC (History)
1 user (show)

See Also:


Attachments

Description Ehsan Akhgari [:ehsan] 2012-11-22 00:17:58 UTC
Here is a list of problems that come to my mind right now.

1. It's not clear whether the callbacks have to be asynchronous or not.  For example, if for some reason the implementation decides that it's unable to decode the audio right away, should it just call the failure callback directly, or call it the next time that the execution hits the event loop.

2. It's not clear what the AudioBuffer callback passed to the failure callback should contain.  For example, if the audio's type cannot be sniffed, what should be passed to the failure callback?  What if the source buffer doesn't contain valid samples and the decoding operation fails half-way?

3. "The ArrayBuffer can, for example, be loaded from an XMLHttpRequest with the new responseType and response attributes." should probably converted to "The ArrayBuffer can, for example, be loaded from an XMLHttpRequest's response attribute after setting the new responseType to "arraybuffer"."
Comment 1 Chris Rogers 2012-11-29 10:36:59 UTC
(In reply to comment #0)
> Here is a list of problems that come to my mind right now.
> 
> 1. It's not clear whether the callbacks have to be asynchronous or not.  For
> example, if for some reason the implementation decides that it's unable to
> decode the audio right away, should it just call the failure callback
> directly, or call it the next time that the execution hits the event loop.

I'm not sure I completely understand the question.  The decoding is meant to be asynchronous, so in some sense will not happen "right away".  So neither the success nor the error callback will be called directly, but at some later time (a later time in the event loop).  The failure callback should be called if there is an actual error in the decoding, meaning that the audio data is not one of the known and supported formats (depending on what the user agent supports), or that the audio data is corrupt.

> 
> 2. It's not clear what the AudioBuffer callback passed to the failure
> callback should contain.  For example, if the audio's type cannot be
> sniffed, what should be passed to the failure callback?  What if the source
> buffer doesn't contain valid samples and the decoding operation fails
> half-way?

I think it's best that there is *no* AudioBuffer in the failure case.
Comment 2 Ehsan Akhgari [:ehsan] 2012-11-29 16:26:13 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > Here is a list of problems that come to my mind right now.
> > 
> > 1. It's not clear whether the callbacks have to be asynchronous or not.  For
> > example, if for some reason the implementation decides that it's unable to
> > decode the audio right away, should it just call the failure callback
> > directly, or call it the next time that the execution hits the event loop.
> 
> I'm not sure I completely understand the question.  The decoding is meant to
> be asynchronous, so in some sense will not happen "right away".  So neither
> the success nor the error callback will be called directly, but at some
> later time (a later time in the event loop).  The failure callback should be
> called if there is an actual error in the decoding, meaning that the audio
> data is not one of the known and supported formats (depending on what the
> user agent supports), or that the audio data is corrupt.

Let me clarify the question.  Consider this test case for example:

  var cx = new AudioContext();
  var buf = new Uint8Array(100);
  // buf is filled with zeros, and therefore, not a valid audio stream
  var decodeFailed = false;
  cx.decodeAudioData(buf.buffer, function(x) {
    }, function(x) {
      decodeFailed = true;
  });
  alert(decodeFailed);

The question is, should decodeFailed be true or false after the call to decodeAudioData.  The reason that I started to think about this is that in the Gecko implementation, we sniff the buffer synchronously as it's a relatively cheap operation, so in this case we will know immediately that the decoding operation will fail.  Now, we have the option of calling the failure callback immediately which means that decodeFailed will be true when decodeAudioData returns, or calling it the next time that we hit the event loop, which means that decodeFailed will be false when decodeAudioData returns.

> > 2. It's not clear what the AudioBuffer callback passed to the failure
> > callback should contain.  For example, if the audio's type cannot be
> > sniffed, what should be passed to the failure callback?  What if the source
> > buffer doesn't contain valid samples and the decoding operation fails
> > half-way?
> 
> I think it's best that there is *no* AudioBuffer in the failure case.

In that case, we need to change the Web IDL to offer two distinct callback types, like this:

  callback DecodeSuccessCallback = void (AudioBuffer decodedData);
  callback DecodeErrorCallback = void ();

  interface AudioContext {
    // ...
    void decodeAudioData(ArrayBuffer audioData,
                         DecodeSuccessCallback successCallback,
                         optional DecodeErrorCallback errorCallback);
    // ...
  };

I personally think that this makes way more sense.  Please let me know if you want me to change the Web IDL accordingly.  Thanks!
Comment 3 Chris Rogers 2012-11-29 17:28:20 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > (In reply to comment #0)
> > > Here is a list of problems that come to my mind right now.
> > > 
> > > 1. It's not clear whether the callbacks have to be asynchronous or not.  For
> > > example, if for some reason the implementation decides that it's unable to
> > > decode the audio right away, should it just call the failure callback
> > > directly, or call it the next time that the execution hits the event loop.
> > 
> > I'm not sure I completely understand the question.  The decoding is meant to
> > be asynchronous, so in some sense will not happen "right away".  So neither
> > the success nor the error callback will be called directly, but at some
> > later time (a later time in the event loop).  The failure callback should be
> > called if there is an actual error in the decoding, meaning that the audio
> > data is not one of the known and supported formats (depending on what the
> > user agent supports), or that the audio data is corrupt.
> 
> Let me clarify the question.  Consider this test case for example:
> 
>   var cx = new AudioContext();
>   var buf = new Uint8Array(100);
>   // buf is filled with zeros, and therefore, not a valid audio stream
>   var decodeFailed = false;
>   cx.decodeAudioData(buf.buffer, function(x) {
>     }, function(x) {
>       decodeFailed = true;
>   });
>   alert(decodeFailed);
> 
> The question is, should decodeFailed be true or false after the call to
> decodeAudioData.  The reason that I started to think about this is that in
> the Gecko implementation, we sniff the buffer synchronously as it's a
> relatively cheap operation, so in this case we will know immediately that
> the decoding operation will fail.  Now, we have the option of calling the
> failure callback immediately which means that decodeFailed will be true when
> decodeAudioData returns, or calling it the next time that we hit the event
> loop, which means that decodeFailed will be false when decodeAudioData
> returns.

I see.  I would say that decodeFailed will be false, since the callback will
only happen the next time we hit the event loop.

> 
> > > 2. It's not clear what the AudioBuffer callback passed to the failure
> > > callback should contain.  For example, if the audio's type cannot be
> > > sniffed, what should be passed to the failure callback?  What if the source
> > > buffer doesn't contain valid samples and the decoding operation fails
> > > half-way?
> > 
> > I think it's best that there is *no* AudioBuffer in the failure case.
> 
> In that case, we need to change the Web IDL to offer two distinct callback
> types, like this:
> 
>   callback DecodeSuccessCallback = void (AudioBuffer decodedData);
>   callback DecodeErrorCallback = void ();
> 
>   interface AudioContext {
>     // ...
>     void decodeAudioData(ArrayBuffer audioData,
>                          DecodeSuccessCallback successCallback,
>                          optional DecodeErrorCallback errorCallback);
>     // ...
>   };
> 
> I personally think that this makes way more sense.  Please let me know if
> you want me to change the Web IDL accordingly.  Thanks!

Yes, agreed this makes more sense, and feel free to edit the IDL.
Comment 4 Ehsan Akhgari [:ehsan] 2012-11-29 21:00:36 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > (In reply to comment #1)
> > > (In reply to comment #0)
> > > > Here is a list of problems that come to my mind right now.
> > > > 
> > > > 1. It's not clear whether the callbacks have to be asynchronous or not.  For
> > > > example, if for some reason the implementation decides that it's unable to
> > > > decode the audio right away, should it just call the failure callback
> > > > directly, or call it the next time that the execution hits the event loop.
> > > 
> > > I'm not sure I completely understand the question.  The decoding is meant to
> > > be asynchronous, so in some sense will not happen "right away".  So neither
> > > the success nor the error callback will be called directly, but at some
> > > later time (a later time in the event loop).  The failure callback should be
> > > called if there is an actual error in the decoding, meaning that the audio
> > > data is not one of the known and supported formats (depending on what the
> > > user agent supports), or that the audio data is corrupt.
> > 
> > Let me clarify the question.  Consider this test case for example:
> > 
> >   var cx = new AudioContext();
> >   var buf = new Uint8Array(100);
> >   // buf is filled with zeros, and therefore, not a valid audio stream
> >   var decodeFailed = false;
> >   cx.decodeAudioData(buf.buffer, function(x) {
> >     }, function(x) {
> >       decodeFailed = true;
> >   });
> >   alert(decodeFailed);
> > 
> > The question is, should decodeFailed be true or false after the call to
> > decodeAudioData.  The reason that I started to think about this is that in
> > the Gecko implementation, we sniff the buffer synchronously as it's a
> > relatively cheap operation, so in this case we will know immediately that
> > the decoding operation will fail.  Now, we have the option of calling the
> > failure callback immediately which means that decodeFailed will be true when
> > decodeAudioData returns, or calling it the next time that we hit the event
> > loop, which means that decodeFailed will be false when decodeAudioData
> > returns.
> 
> I see.  I would say that decodeFailed will be false, since the callback will
> only happen the next time we hit the event loop.

Sounds good.  Do you want me to edit the spec to reflect this?  (And also my point 3 in comment 0 perhaps?)

> > > > 2. It's not clear what the AudioBuffer callback passed to the failure
> > > > callback should contain.  For example, if the audio's type cannot be
> > > > sniffed, what should be passed to the failure callback?  What if the source
> > > > buffer doesn't contain valid samples and the decoding operation fails
> > > > half-way?
> > > 
> > > I think it's best that there is *no* AudioBuffer in the failure case.
> > 
> > In that case, we need to change the Web IDL to offer two distinct callback
> > types, like this:
> > 
> >   callback DecodeSuccessCallback = void (AudioBuffer decodedData);
> >   callback DecodeErrorCallback = void ();
> > 
> >   interface AudioContext {
> >     // ...
> >     void decodeAudioData(ArrayBuffer audioData,
> >                          DecodeSuccessCallback successCallback,
> >                          optional DecodeErrorCallback errorCallback);
> >     // ...
> >   };
> > 
> > I personally think that this makes way more sense.  Please let me know if
> > you want me to change the Web IDL accordingly.  Thanks!
> 
> Yes, agreed this makes more sense, and feel free to edit the IDL.

Done: https://dvcs.w3.org/hg/audio/rev/cb492d3bd589.
Comment 6 Olivier Thereaux 2013-01-18 15:52:08 UTC
No objection to the changesets in nearly a month. Closing.