18:48:40 RRSAgent has joined #audio 18:48:40 logging to http://www.w3.org/2012/06/13-audio-irc 18:48:42 RRSAgent, make logs world 18:48:42 Zakim has joined #audio 18:48:44 Zakim, this will be 28346 18:48:44 ok, trackbot; I see RWC_Audio()3:00PM scheduled to start in 12 minutes 18:48:45 Meeting: Audio Working Group Teleconference 18:48:45 Date: 13 June 2012 18:49:12 Agenda: http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0692.html 18:49:19 Agenda+ Testing 18:49:36 Agenda+ Loop start/stop points (Bugzilla 17390) 18:49:43 Agenda+ rechartering update 18:50:11 Agenda+ MIDI issues (Bugzilla 17416 and 17417) 18:53:09 ot has joined #audio 18:58:06 RWC_Audio()3:00PM has now started 18:58:14 + +1.862.201.aaaa 18:59:51 cwilso has joined #audio 19:00:16 joe has joined #audio 19:00:25 zakim, code? 19:00:25 the conference code is 28346 (tel:+1.617.761.6200 sip:zakim@voip.w3.org), ot 19:00:41 +??P9 19:00:49 zakim, ??P9 is me 19:00:49 +ot; got it 19:00:58 zakim, who is here? 19:00:58 On the phone I see +1.862.201.aaaa, ot 19:00:59 On IRC I see joe, cwilso, ot, Zakim, RRSAgent, MikeSmith, automata, shepazu, kennyluck, trackbot, philipj, paul___irish 19:01:08 zakim, +1.862 is me 19:01:18 +cwilso; got it 19:01:26 + +1.978.314.aabb 19:01:41 + +1.510.334.aacc 19:02:40 zakim, aabc is joe 19:02:40 sorry, ot, I do not recognize a party named 'aabc' 19:02:52 zakim, aabb is joe 19:02:52 +joe; got it 19:03:08 +Doug_Schepers 19:03:14 zakim, aacc is crogers 19:03:14 +crogers; got it 19:04:16 Joebob has joined #audio 19:04:19 jussi has joined #audio 19:04:57 chris_ has joined #audio 19:05:21 +??P14 19:05:51 Zakim, ??P14 is me 19:05:51 +jussi; got it 19:05:59 Zakim, mute me 19:05:59 jussi should now be muted 19:08:02 zakim, take up agendum 1 19:08:02 agendum 1. "Testing" taken up [from ot] 19:08:26 -crogers 19:08:50 +crogers 19:08:52 tmichel has joined #audio 19:10:23 Olivier: several conversations about testing, now seems like a good time to start discussing our approach, agree on a strategy and get a test lead (for the group or for each deliverable) 19:10:37 http://svn.webkit.org/repository/webkit/trunk/LayoutTests/webaudio/ 19:10:45 Doug: testing is very helpful to help where specs are unclear 19:10:53 … also important when you start having several implementations 19:11:17 … it has been useful in other groups to have a "test lead" who helps coordinate the test effort 19:11:30 +??P17 19:11:34 … may be the person who makes the tests, or extracts test requirements from the spec 19:11:44 … or encourages other people to add them 19:11:56 … can think of the test coordinator as the editor for the test 19:12:02 … will need one for each spec 19:12:21 … for the midi API it is likely that one of the two editors can manage the tests 19:12:47 for the web audio API, Chris has a lot of work with the spec, it may be best to have another person in charge of the testing 19:13:18 doug: it also engages more people, and adds checks and balance (Chris is not in charge of everything and wielding too much power) 19:13:48 doug: do we have any volunteer to be test controller for each of the spec 19:14:03 crogers: we already have over 60 tests written -> http://svn.webkit.org/repository/webkit/trunk/LayoutTests/webaudio/ 19:14:10 … any of those can be shared 19:14:34 doug: that's good. one bit of value in the tests is that the implementors can use them to test their own build 19:14:57 … it's importance for the spec interoperability, but in practice it's useful to test the implementations 19:15:07 … so it's helping when each implementor contributes tests 19:15:18 … are these unit tests? 19:15:36 crogers: fairly basic tests, testing the gain node, etc 19:15:40 … pretty targeted 19:15:54 doug: will need hundreds, maybe thousands 19:17:30 Olivier: I've seen simple specs with thousands of tests, testing combination and parameters… those tend to be automated 19:18:13 doug: w3c is developing a test framework, a test harness 19:18:33 https://github.com/w3c/testharness.js 19:18:59 http://w3c-test.org/resources/testharness.js 19:19:08 -crogers 19:19:11 http://lists.w3.org/Archives/Public/public-audio/2012AprJun/0725.html 19:19:20 +crogers 19:20:23 rtoyg has joined #audio 19:21:28 Olivier: wondering if the webkit tests look like the method mentioned by Philip? 19:21:44 crogers: very similar to pixel tests 19:23:23 Olivier: how do you cater for the fact that you don't get a predetermined signal as output 19:23:55 crogers: a human being has to generate the baseline for each rendering engine 19:24:55 … the tests will be shared, but the baseline may not 19:24:58 -crogers 19:25:56 zakim, who is here ? 19:25:56 On the phone I see cwilso, ot, joe, Doug_Schepers, jussi (muted), ??P17 19:25:58 On IRC I see rtoyg, tmichel, chris_, jussi, joe, cwilso, ot, Zakim, RRSAgent, MikeSmith, automata, shepazu, kennyluck, trackbot, philipj, paul___irish 19:25:58 + +1.650.214.aadd 19:26:07 zakim, aadd is crogers 19:26:07 +crogers; got it 19:26:41 crogers: a majority of our tests don't compare against a ref file 19:27:04 … what they do is render audio internally and use js to walk the rendered data and verify that it is correct 19:27:09 … with a very small tolerance 19:27:24 … many of those tests can be shared 19:29:50 olivier: did you have pixel tests with svg, doug? 19:30:03 doug: probably similar to what is called ref test in css 19:30:36 … a reference view, sometimes reproduced with a combination of simpler features 19:31:03 … in svg, we actually had reference images, and we did the testing manually 19:31:21 … hundreds of tests and we had f2f meetings where we ran tests together in a room 19:31:32 … also important for competitive reasons 19:31:43 … for svg2 we plan on doing web tests 19:31:59 crogers: a majority of these tests can be automated on baseline 19:32:03 q+ 19:32:54 q+ 19:33:14 olivier: similar to the graph that Philip sent today 19:33:36 crogers: not for oscillators, but for e.g biquad, yes 19:33:46 … similar to rendering lines 19:33:52 … lines will not be rendered exactly 19:33:59 … the same 19:34:11 … all are an approximation of a mathematical ideal 19:34:34 … most of the audio stuff is not that complicated, it's actually precise 19:35:00 q- 19:35:00 ack she 19:35:15 ack joe 19:35:26 joe: wanted to share a useful testing technique 19:36:30 … we alternate between the baseline and the test 19:36:39 crogers: you mean a diff? 19:37:17 … might not work so well for web audio 19:38:20 doug: I like the idea of having a baseline 19:39:30 crogers: 2 kinds of tests 19:39:42 … one fully automated with js walking the data and comparing 19:39:56 … the other compares the generated audio with a baseline 19:40:12 … bit-exact compariso 19:40:17 s/riso/rison/ 19:40:35 doug: is there some functionality in the API enabling this? 19:40:53 crogers: we had to build an extra tool in our test harness 19:41:37 doug: would that be useful for some audio analysis use cases 19:42:19 crogers: probably not because in a normal environment you can generate the wav file and upload it to a server via xhr 19:42:37 … we just have a specific hook in our test harness 19:42:47 … details would be different in another test tool 19:44:26 ot has joined #audio 19:45:33 -Doug_Schepers 19:46:00 zakim, unmute me 19:46:00 jussi should no longer be muted 19:46:04 jussi, you've posted a link to some tests 19:46:27 jussi: testing that the interfaces are there 19:46:53 https://github.com/jussi-kalliokoski/web-midi-test-suite 19:47:46 jussi: not sure about more complex tests 19:47:58 … don't think you can do automated test 19:48:05 … you'd need a tester with MIDI device 19:48:59 ot: could that be "faked"? 19:49:06 jussi: a virtual midi device could work 19:49:24 cwilson: would be hard to make cross-platform, obviously 19:49:46 jussi: I think OSX lets you do that 19:49:47 crogers: yes I think you can do that 19:50:06 … harder to write that test than to write the MIDI implementation itself 19:50:27 jussi: I had a virtual device in my original proposal, decided to keep it out of v1 19:50:38 … some platforms don't support it at all. 19:51:57 Olivier: I will send a call for volunteers on the list, to see who wants to be in charge of testing for each of the specs 19:52:15 jussi: if noone else volunteers I can be in charge for the MIDI API, for now 19:52:45 zakim, take up agendum 3 19:52:45 agendum 3. "rechartering update" taken up [from ot] 19:53:11 http://www.w3.org/2011/audio/charter/2012/charter-proposed.html 19:55:05 -crogers 19:55:06 -joe 19:55:06 -ot 19:55:09 -cwilso 19:55:11 -jussi 19:55:15 -??P17 19:55:16 RWC_Audio()3:00PM has ended 19:55:16 Attendees were +1.862.201.aaaa, ot, cwilso, +1.978.314.aabb, +1.510.334.aacc, joe, Doug_Schepers, crogers, jussi, +1.650.214.aadd 19:55:42 rrsagent, draft minutes 19:55:42 I have made the request to generate http://www.w3.org/2012/06/13-audio-minutes.html ot 19:55:49 zakim, bye 19:55:49 Zakim has left #audio 19:55:53 rrsagent, bye 19:55:53 I see no action items