Audio Working Group Teleconference -- 01 Jun 2015

<trackbot> Date: 01 June 2015

<ChrisL> oh zakim get a grip

<ChrisL> http://www.w3.org/2015/07/zakim.html

<BillHofmann> scribenick BillHofmann

<BillHofmann> Joe: we can start working back from the goal

<BillHofmann> Joe: in pretty good shape WRT issues in the spec

<BillHofmann> Matt: we should "fan the work out" to be more efficient

<BillHofmann> Joe: wide review will be the easiest thing to accomplish - lots of folks interested in the spec

<BillHofmann> Agenda: two halves - more discussion of issues (Monday); schedule/work plan (Tuesday)

<BillHofmann> Joe: no hard and fast rule about "what will be done" by TPAC

<BillHofmann> ChrisL: used to have Last Call; now you just have to show what you did before CR; used to be that if you did CR and had major changes, you had to go back to LC

<BillHofmann> ChrisL: aim is a document that includes all feedback, response, and feedback makers' response to this

<BillHofmann> Doug: two main issues - finishing the spec; completing a test suite and getting implementation reports

<BillHofmann> Doug: need (for tests) to have a Test Tsar - first identify key features to test as a requirement for CR

<BillHofmann> Doug: need to do implementation reports

<BillHofmann> ChrisL: test can also come when there is a difference in implementations

<BillHofmann> Doug: can issue errata on a spec if we find a bug

<BillHofmann> Matt: how long do you leave things open; etc

<BillHofmann> Doug: realistically, need to have a set of key tests, and at least two passing implementations of each of these

<BillHofmann> Doug: then, 3 months for technical and organizational review (includes steps in there)

<BillHofmann> Doug: SVG has had a group "stay at home F2F" - all on IRC, can jump on the phone - if we had 2-3 of those < TPAC, we could actually get the spec done before TPAC

<BillHofmann> Doug: post-TPAC, concentrate on getting tests in - could be done by a few months after TPAC

<BillHofmann> Roll Call:

<BillHofmann> Doug Scheppers - W3C Staff Rep

<BillHofmann> Bill Hofmann - Dolby

<BillHofmann> Todd Gehring - Dolby

<BillHofmann> (focused on browser technology)

<BillHofmann> Joe Berkowitz - President of Noteflight - co-chair

<BillHofmann> Chris Wilson - Google; spec editor for Web MIDI, ex-WebAudio spec editor

<BillHofmann> Hongchan Choi - Google - works on Blink

<BillHofmann> Chris Lilley - W3C Staff Rep

<BillHofmann> Raymond Toy - Google - works on WebAudio

<BillHofmann> Kawai - Yamaha (AMEI)

<BillHofmann> Matt Paradis - BBC (co-chair)

<BillHofmann> Paul Adenot - Mozilla

<BillHofmann> ---------------

- walking through things needing reviews;

<ChrisL> https://github.com/WebAudio/web-audio-api/labels

<BillHofmann> thanks

<joe> https://github.com/WebAudio/web-audio-api/issues?q=is%3Aopen+is%3Aissue+milestone%3AUncommitted+sort%3Acreated-asc

<BillHofmann> Issue 391 - Specify nominal ranges for AudioParam

<BillHofmann> bug 391 - Specify nominal ranges for AudioParam

<BillHofmann> Assign to Paul, v1, ready for editing

<BillHofmann> Next bugs: biquad filter issues

<BillHofmann> bugs 426, 427, 428, 429, 430, 431, 432 - refer to audio cookbook, however, the implementations don't match - should we make them match, or make the spec match the implementation

<BillHofmann> recommendation is to make the formula match the implementation

<BillHofmann> mark as ready for editing; should acknowledge audio cookbook

<BillHofmann> bug 436 - handling attributes set to null

<shepazu> Audio EQ cookbook is at http://shepazu.github.io/Audio-EQ-Cookbook/audio-eq-cookbook.html

<shepazu> ACTION: rtoyg to add acknowledgment to Robert Bristow-Johnson in Web Audio API spec [recorded in http://www.w3.org/2015/06/01-audio-minutes.html#action01]

<trackbot> Error finding 'rtoyg'. You can review and register nicknames at <http://www.w3.org/2011/audio/track/users>.

<shepazu> ACTION: raymond to add acknowledgment to Robert Bristow-Johnson in Web Audio API spec [recorded in http://www.w3.org/2015/06/01-audio-minutes.html#action02]

<trackbot> Created ACTION-120 - Add acknowledgment to robert bristow-johnson in web audio api spec [on Raymond Toy - due 2015-06-08].

<BillHofmann> proposed action - values will be nullable, but can only assume the value of null in their inited state, otherwise will throw InvalidStateError

<BillHofmann> bug 444

<BillHofmann> proposed: just document - should be oncomplete last

<BillHofmann> bug 447

<BillHofmann> (Difference in loudness from different implementation)

<BillHofmann> mozilla now aligns; but how to spec?

<BillHofmann> Joe: notes issue 91 specifies the behavior we want, pretty much

<BillHofmann> Joe: proposes re-titling to "Define built-in oscillator output more carefully"

<BillHofmann> Joe: output of a built-in oscillator must be equivalent to the output of an equivalent PeriodicWave with normalization.

<BillHofmann> bug 452

<BillHofmann> resolution: fix spec to cite MediaStreamAudioDestinationNote as well...

<BillHofmann> bug 457 - Clarify behavior when DelayNode output is connected back to delayTime AudioParam, creating a cycle

<BillHofmann> resolution: will mute

<BillHofmann> Joe - propose punt to v.Next

<GitHubBot> [web-midi-api] cwilso pushed 1 new commit to gh-pages: https://github.com/WebAudio/web-midi-api/commit/439c238bd40470b9902a7689729bf6a3b3cf120e

<GitHubBot> web-midi-api/gh-pages 439c238 Chris Wilson: Fixes #149.

<BillHofmann> bug 459 - ChannelMergerNode in a cycle has unspecified behaviour

<BillHofmann> resolution: already fixed...

<BillHofmann> (per cwilso)

<shepazu> scribenick: shepazu

issue 460

cwilso: there's a error, it doesn't increase in real time
... it represents the current time of the next block to be processed (the beginning of the next rendering content frame)
... let's just remove the word "realtime"

joe: that's what #460 says
... this is covered in #381
... marked as ready for editing, in v1

issue 464

ChrisL: accept as invalid state error

issue 465

joe: where are we on this? the topic changes
... do we just change them from floats to doubles?
... there's loss in precision going back and forth

rtoyg: the issue is that internally in Chrome, we use floats.... you can't evaluate it in JS, making it hard to write a test for it

padenot: I think we have the same behavior in FF
... seems a bit brutal to just use double, but...

rtoyg: then implementations need to handle it better, and that could be disruptive

joe: WebIDL could be a double, but specs internally could use floats

BillHofmann: might set wrong expectations to developers
... on precision

padenot: instead of flooring, do we round?
... having the param the same as the sample makes sense to me

ChrisL: flip it the other way around
... say the interface type is floats, but say that implementations may promote to double for improved precision of intermediate calculations

joe: leaving it as float makes sense ...
... note to developers that if they are going to go between integer-based units, converting between sample rates and times, to use round rather than truncation

rtoyg: not sure that solves the issue

[quibbling]

rtoyg: when you look at in isolation, yes... but...
... I'll look at it closer, but good enough for now

resolved

issue 468

ChrisL: this is a great case of wide and substantive review

joe: I think this a good issue, but it's a big change, so maybe move to v2?

cwilso: this is absolutely a valid use case
... whether we want to implement it as doing FFT before and after, not sure
... maybe we do need an FFT library
... maybe someone should prototype it as demonstration

<ChrisL> see http://chinpen.net/webaudiofftperf/

cwilso: that pipeline is indeed complicated
... the performance test was using scriptprocessor, because it couldn't do anything else
... this needs to be done in audioworkers
... step 1 should be implementing a native FFT library that works on arrays, not audio specific
... along the lines of the Extensible Web

issue 469

joe: I understand why people want this
... but it seems a bit underdefined
... seems like it's about the performance of the graph

BillHofmann: sigh
... would you spec exposing a latency? seems like it lead to a loop
... maybe we should limit latency?

ChrisL: who are we doing this for? (discussed different use cases)

cwilso: I didn't mention in the bug that they are using a compressor

joe: doesn't make sense to talk about latency in this way if the output is different than the input
... seems like a non-issue with respect to what they mention, should it be separate issues?
... the root is that the original spec talked a lot about spacialization, and not enough about delay

ChrisL: i hadn't know there was this much latency in the graph

padenot: it's because of lookahead

shepazu: should we note this in the spec?

[debate about value of warning devs]

joe: maybe we could have proper latency reporting in v2?

rtoyg: not sure that latency in compressor is even computable

[discussion of pros, cons, and constraints]

ChrisL: maybe a note to say, "when latency is important, developers should calculate this offline"

470, already assigned to padenot

471

<padenot> padenot: maybe it's a equal-power vs. equalpower issue ?

<padenot> cwilso: I'm current fixing it, it's just an editorial issue

<ghaudiobot> [web-audio-api] cwilso pushed 1 new commit to gh-pages: https://github.com/WebAudio/web-audio-api/commit/d9286b46c4feff2bf6f4b2df3f5e414d9abc4a41

<ghaudiobot> web-audio-api/gh-pages d9286b4 Chris Wilson: Fixes #471

<padenot> padenot: it should not be hyphenated

474

<ghaudiobot> [web-audio-api] cwilso pushed 1 new commit to gh-pages: https://github.com/WebAudio/web-audio-api/commit/3392038eb9b207ae23f30f97baae3aab20da01f6

<ghaudiobot> web-audio-api/gh-pages 3392038 Chris Wilson: Fix hyphenated instances of "equal-power" (part of #471)

<ghaudiobot> [web-audio-api] cwilso pushed 1 new commit to gh-pages: https://github.com/WebAudio/web-audio-api/commit/f608f6c2c27f46c1f6865e997b8673e055caf619

<ghaudiobot> web-audio-api/gh-pages f608f6c Chris Wilson: Further hyphenation fixes (#471)

<padenot> joe: Resolution: AudioWorkerNode are implementing postMessage, the behaviour should follow the definition of postMessage

<padenot> padenot: we will need input from people working on script, and proabably worker

477

<padenot> Resolution: "just fix it"

478

<padenot> Resolution: it's already fixed upstream, we can just close it

480

<padenot> cwilso: We should probably use NotSupportedError here, when setting the channel count to something bigger than the supperted range

481

<padenot> Resolution, closing it without change, see comment

482

<padenot> cwilso: the .value is never going to give you something interesting here

<padenot> joe: is the intrinsic value a concept that is defined when the audio is connected to an AudioParam

<padenot> cwilso: it's probably defined as "largely useless"

<padenot> cwilso: (reading spec...)

<padenot> joe: there is another issue that capture the fact that .value needs to be defined, does it returns the "computed value"

<padenot> joe: reading 2.5.3, audio connected don't change .value

<padenot> joe: neither scheduled parameters nor audio input change the .value

<padenot> cwilso: not true, it depends how you read it

<padenot> joe: we could say "it does not matter"

<padenot> cwilso: it's described in the section where it says "when read...."

<padenot> joe: if we leave the spec alone, this particular issue is resolved, but we have the issue about "intrinsic value" meaning

486

<padenot> joe: I can tell you from using this API that computing a time for when it reach 0 is hard

<padenot> ... lengthy discussion about the fact that this is hard ...

<padenot> ... more discussion ...

<padenot> Resolution: we just point to setTargetAtTime, because this is the wrong method for the job

489

<padenot> Resolution: we leave it like that for v1, because it's a crappy API anyways, and works is gonna happen in v.next

<padenot> in the meantime, more useful information can be put in the error console

<padenot> cwilso: it's already specced, just close this

496

<padenot> cwilso: we can't fix that in v1. MediaRecorder is at the possibly wrong level of abstraction for now

<padenot> joe: it's gonna be addressed in the future by reworking encoding and decoding APIs

497

<padenot> Resolution: This is just editorial

<padenot> Topic 503

<padenot> Resolution: if you cancel...(), then you should be able to go back to using .value

<padenot> also, setting the value calls cancelScheduledValues() under the hood

504

<padenot> "just do what the comment says"

<padenot> Topic 506:

<padenot> "that's not webaudio's problem"

<padenot> Topic 509:

<padenot> resolution is to switch to a-rate. engines might perform optimizations

<rtoyg_m> Issue https://github.com/WebAudio/web-audio-api/issues/510

<rtoyg_m> joe: Complicated issue with some confusion from the developer

<rtoyg_m> joe: Shows the complexity of automation methods. We have discussed exposing a way to get automation values.

<rtoyg_m> padenot: Suggests reimplementing audioparams in JS.

<rtoyg_m> padenot: Or function to probe automation value

<rtoyg_m> padenot: holdValueAtTime makes some sense to hold the value.

<rtoyg_m> cwilso: Isn't that the same as cancelValueAtTime?

<rtoyg_m> joe: What does cancelValueAtTime actually do?

<rtoyg_m> cwilso: Technically doesn't work with setTargetValue.

<rtoyg_m> all: It's completely broken.

<rtoyg_m> cwilso: Developer basically has to implement automation to figure out what happens and adjust parameters appropriately. See DJ app.

<rtoyg_m> cwilso: cancelScheduledValues needs to be updated to handle setTargetValue

<rtoyg_m> joe: What do people what really want?

<rtoyg_m> cwilso: We should give an example of ADSR to show how to do it. But basically requires devs to compute automation.

<rtoyg_m> joe: Close this issue since it doesn't add any new info not already covered elsewhere.

<rtoyg_m> https://github.com/WebAudio/web-audio-api/issues/512

<rtoyg_m> padenot: Can't have everything.

<rtoyg_m> joe: Clarify spec to hint that ABSN can be used for any length, not just short samples.

<rtoyg_m> https://github.com/WebAudio/web-audio-api/issues/513

<rtoyg_m> cwilso: jer's answer is correct.

<rtoyg_m> Next issue: all dezippering issues.

<rtoyg_m> See https://github.com/WebAudio/web-audio-api/issues/76 for primary issues

<rtoyg_m> joe: Resolve 76 that dezippering is equivalent to setTargetAtTime, with some differences.

<rtoyg_m> joe: Add new issue that describes how dezippering applies on initial value.

<rtoyg_m> joe: Example: If value is 2, and node starts, no dezippering should happen, and value should start at 2.

<rtoyg_m> rtoyg_m: What about gain nodes that don't have "start" times.

<rtoyg_m> joe: Wouldn't it be poetic to remove dezippering again?

<rtoyg_m> rtoyg_m: Might break things.

<rtoyg_m> cwilso: But what would we break? Needs to be explained better.

<rtoyg_m> joe: Document the current state so we don't break anything and tell people how to do what they want.

<rtoyg_m> cwilso: It happens in bad places (freq/detune on oscillator).

<rtoyg_m> cwilso: Don't bake in current state if we want a spec to last 10 years.

<rtoyg_m> cwilso: Document where we want it or complete remove it. Volume (gain) control is really the only place it makes sense.

<rtoyg_m> joe: Dezippering is too complicated to actually explain.

<rtoyg_m> shepazu: If we remove dezippering, is there any place we really need it?

<rtoyg_m> padenot: delay time needs it.

<ChrisL> see glowpot, here: http://whitefiles.org/rws/r02.htm

<rtoyg_m> padenot: FF does dezippering only for delay time.

<mdjp> note - the glowpot must have been just before my time at the bbc

<rtoyg_m> shepazu: Should we just tell people how to do smooth volume changes?

<rtoyg_m> BillHofmann: LIkes the removal of dezippering.

<rtoyg_m> joe: Not knowing when and how dezippering is a burden on developers.

<rtoyg_m> cwilso: Remove everywhere including delay node. Need to tell developers what todo to make params changes smoothly.

<rtoyg_m> shepazu: Any cost to implementations?

<rtoyg_m> rtoyg_m, cwilso, padenot Very minor implementation cost.

<ChrisL> scribenick: ChrisL

joe: proposed tail-between-legs retraction of previous retraction on dezippering
... needs revision on delaynode

(group): really? gone

OH" this is about how it is implemented, not how it is made"

rtoyg: chrome will change to take into account removal of dezippering (and magical treatment of first value)

joe: volunteers to do the spec edit, removing auto dizippering, how to use automation, and effect on delayNode

BillHofmann: (reads from spec on delayNode)

close all the dizippering issues

BillHofmann: if users are not complaining, and ff does no dezippering, that means it is not an issue

mdjp: (points out parked PR 393 to remove dezippering, already)

Walk through issues marked as Need WG Review

joe: https://github.com/WebAudio/web-audio-api/issues?q=is%3Aopen+is%3Aissue+label%3A%22Needs+WG+review%22

(agenda reshuffle discussion)

self.congratulate() - method not found

<joe> https://github.com/WebAudio/web-audio-api/issues?q=is%3Aopen+is%3Aissue+label%3A%22Needs+WG+review%22

https://github.com/WebAudio/web-audio-api/issues/532

padenot: starting to impl audio worker. dev knowing a lot about workers pointed out many issues
... compared web worker and audio worker spec. listed issues, quoting worker spec
... conept of worker tightly coupled to OS thread
... base inteerface has a ton of stuf, most of which we don't want as its on the audio thread

cwilso: like what?

padenot: fetch, promise, setInterval for example

cwilso: can audio processing threads fork?

padenot: do not want jiting on audio thread

ChrisL: sounds like a base class of an execution context

padenot: yes, but that is a big project

cwilso: yes, audio workers in a shared thread, not a high startup performance cpost or memory cost.
... long lived, file transfers is an issue

padenot: postMessage with transferrable

cwilso: how to have code sitting that knows to do this

padenot: no traditional event loop. execute between blocks
... queue that is emptied at th beginning of each block

cwilso: avoid setTimeout etc. postMessage requires an event loop
... architectual layering here is a proble (as roc pointed out long ago)
... lack input ad output device access. workers we tried to build on top in a web worker with other stuff pushed into it.
... would not lke to see a totally different beast for that
... stilll issue of jiting code

padenot: a few blockers here

joe: expensive things clearly identified, like setup costs
... assume yyou make a global factory with the expensive startup. node creation and running is light
... lack a way for worker script to indicate setup complete, resady to go

padenot: promises

(do we, don't we)

cwilso: promise inside global scope

joe: script cant do stuff asynchronously

cwilso: method called to get a promise that resolves when it is ready

joe: lack a mechanism for deferring own return

cwilso: (deja vu on complexity analysis)
... design not done yet
... can take feedback to TAG, see what they think

padenot: beyond scope of webaudio. similar to graphics synced on vsync
... video workers

BillHofmann: sounds like a base class indeed. do we know exactly what we are asking?

padenot: to multiplex on the same thread, this stops it being as lightweight as

joe: needs to not be tied to a single thread
... running in a same thread is not visible

padenot: deterministic

joe: doset up outside then handoff the thread

cwilso: concerned about hit from new audio worker

BillHofmann: feel like nodeing a new script node that triggers computation

cwilso: eval jit is in thread

padenot: how to spec it is the main issue

<cwilso> s/eval just is in thread/eval jit MAY be in thread?/

padenot: so we need to talk to others
... setTimeOut?

cwilso: needs to look at current time
... needs postMessage
... (fft library)

joe: who will take this further

cwilso: happy to continue

padenot: set of requirements is non-traditional

https://github.com/WebAudio/web-audio-api/issues/445

joe: getUserMedia is input only, not io

https://github.com/WebAudio/web-audio-api/issues/445

joe: returns a device ID

(points to spec)

joe: filtering depends on anti-fingerrinting issues
... also no channelcount, hedphones etc

BillHofmann: they are clearly ommitting, not forgetting

joe: permissionsare not clear, do get a decent default

shepazu: requiring permissions is fine

joe: ca get an array f thsese, inc channel cont

(several) its an opaque ID

joe: need a few more attributes, to count
... if access has been granted, they are ok with adding extra info
... need access to at leasdty one device before getting a list
... (example, asking for headphoes)

mdjp: connecting to default device

BillHofmann: persistent means

shepazu: session persistent

BillHofmann: makes multiple devices worthless

(catch 22 enumeration). constraint structure

scribe: is missing for audio

(reads persistent ids section)

shepazu: needs a clear, public blog post to establish the use cases

joe: their LC is underway

shepazu: accessibility requirement to access one of several audio devices used for different purposes or with different characteristics
... point out the input and output assymmetry

padenot: people are doing all sorts of stuff with webrtc if you give them access

BillHofmann: will help on the blog post aspect

https://github.com/WebAudio/web-audio-api/issues/419

joe: current tme does not change in realtime, its an advance time on eext audio quantum
... want to relate it to a performance, change dom, visual change and be synced to audio
... need mapping functions

cwilso: its estimation, not strict mapping
... driven by separate crystal so changes over time wrt another thread

padenot: drift

cwilso: wanted a window performance now related to current time in audio context
... a pair of times, one in each system

joe: maybe safer, with pause and resume

so do current time only

cwilso: asking with window performance time maps to a given audio thread time

padenot: (bluetooth can add huge drift)

joe: add new audio context attribute for current time, and dom timestamp corresponding to current time
... so we can do the math and make estimates

cwilso: need to smooth jitter to estimate latency

simplify issue 12 by removing the dom timestamp, and point the oter isue at issue 12

padenot: intervals reasonably safe

(discussion of jitter reduction)

cwilso: like a getter not an attribute

joe: readonly

padenot: another absolute time, not a latency quantity. derive it
... good enough synchronization
... normally, you get a consistent number when asking for time. good for start methods, get a consistent result when starting a lot of them

this would be different, less stable

padenot: mostly needed for audio and video sync. playhead offset, needs cross thread calculation. a latency that evolves onthe main thread is not easy to use

joe: so impl difficulties to advance in real time

cwilso: chris had this in a build to smooth jitter but did not expose it

joe: how di i get the dom hires timestamp?

padenot: add the latency
... even with bluetooth and external usb soundcards you get a decent number

joe: 250ms latency common on android

https://github.com/WebAudio/web-audio-api/issues/340 updated

kawai: we can use this to sync with MIDI host?

cwilso: yes. but currentTime does not advance uniformly. WebMIDI uses performance time

https://github.com/WebAudio/web-audio-api/issues/335

progressCallback in decodeAudioData #335

joe: propose a per-decode object rather than adding more stuff to promises

cwilso: this is a crappy api lets not fix it but instead add a better one later. it has an update as least, and is light
... in the long run, avoid it

jernoble: Ick!

padenot: byte buffer to audio/video is complex. so one member that says appendData, wemiss something
... (wonders about resampling, concludes it is fine)

joe: progress event seems like a medusa. dispatch off the audio context (see the thread)
... the progress event will need more machinery

padenot: was full fledged decoding and encoding api, its needed, don't want a short term hack
... want a reall full featured api in v.2

cwilso: get a good overall progress with some smoothing

padenot: not clear we need a stopgap. its fine for v.1

cwilso: decodes lengthy on mobile, is the issue

(infinite spinners vs. progress bars, UX for the win)

joe: agree on defer and do much better one on v2

padenot: better layering too

cwilso: can live with

padenot: can hook to a completion promise?

cwilso: no, fetch fires progress events to itself
... give it an object, and it fires progress events to it
... needs a new event target

joe: creating ab object that will patch events?

cwilso: its a target, not a dispatcher

joe: how do you make one?

cwilso: lots of things are event targets (gives long list)

joe: using callbacks all over this, do that here too? was good suggestion but

cwilso: lots of ways to do this

- DRAFT -

Audio Working Group Teleconference

01 Jun 2015

Attendees

Contents