oops
didn't watch the time sorry
<krisk> that is fine
<krisk> can't seem to get trackbot and rssagent up
trackbot-ng, start telcon
<krisk> jgraham are you going to participate on IRC or dial in?
<jgraham> I am on IRC
<scribe> scribeNick: plh
<krisk> great
Kris: none
Kris: first one was video example
<krisk> I updated the UA sting to canPlayType and added an image of the rendering
http://test.w3.org/html/tests/submission/Microsoft/video/video.htm
<krisk> james do you have other feedback?
s/all time clocks at zero/all time cloks at zero/
Resolved: video.htm approved
<scribe> ACTION: Kris to move video.htm into the approved test directory
Kris: next time we meet I'd like to have video_00[1-13]
<krisk> Let's plan on asking for the rest of video, canvas and xhtml5 tests
plh: is there a reason why we can't approve them today?
<krisk> How about we state we will post all the video tests as meeting notes
<krisk> If we don't hear negative feedback we can approve all the video tests
<krisk> Which should also include audio0->3
<krisk> http://test.w3.org/html/tests/submission/Microsoft/audio/
<krisk> jgraham do you agree that this is OK?
Resolved: all video* and audio* are approved.
<jgraham> I haven't looked closely
<jgraham> sorry
<krisk> simon peter looked at the tests
James? do you want more time to look at the tests?
<jgraham> I think simon is better qualified than me to review video tests
<krisk> He looked at the tests
<krisk> the one test that will need to be updated is #6
<krisk> see http://lists.w3.org/Archives/Public/public-html-testsuite/2010Jun/0035.html
sounds like we're ok then
<scribe> ACTION: Kris to update video #6 before moving it
XHTML5 and Canvas tests have been submitted
Kris will look at the canvas tests
<krisk> We also have getElementsByClassName, selection api
<krisk> Anne's feedback was that we need to update one test to the actual parser
plh: let's approve the foreign tests except for foreign_content_007 then
<krisk> that woud be #7 http://lists.w3.org/Archives/Public/public-html-testsuite/2010Jun/0017.html
Resolved: foreign_content[1-6] and [8-15] are approved
<krisk> jgraham are you able to look at canvas tests
<krisk> as a group it's good to get the submitted test backlog under control
<jgraham> We already use many of them, so I guess someone already reviewed them. I guess I can get them to look for changes or something
<krisk> that would be good
<krisk> I think a reasonable approach it to set a date that we can all have reviewed the tests
<krisk> Canvas that is since their is ~800 total tests
<krisk> we should get agreement on how we want some of the api's to be organized
<krisk> for example we could just cover the document
<krisk> and a document folder would contain all the api tests for document, for example document.getElementByClassName_001.htm
<krisk> Same goes for the window object
<jgraham> Yes, I think organisation by chapter would be good
<jgraham> (also, the annotated spec that Philip made for the canvas tests is very nice, we should make something similar)
http://test.w3.org/html/tests/approved/
<krisk> I was going to call it be the name of the html5 feature rather than the chapter
<jgraham> Hopefully name of feature and chapter are quite similar
<jgraham> I guess there might be edge cases though
<krisk> I think once we think the chapters (numbers) are not going to change we can rename them to include the chapter
<jgraham> Using chapter-title based rather than chapter-number based naming should be forwards compatible
<krisk> The way I see it is that at some point we create chapters and move tests into that folder
<krisk> OK
<krisk> It's seems to be easy to move tests around - e.g. hg move seems to work just fine and won't lose any data
<krisk> Let's move on to agend item #3
<krisk> jgraham did you have anymore updates planned for domtestcase.js?
<jgraham> krisk: Yes.
<jgraham> I just haven't had time to make them yet...
<krisk> I updated the harness - removed the .name since it's not a standard
<jgraham> Basically some assert functions that check common IDL things
<jgraham> like assertReadonly
<jgraham> and some more things that I have forgotten right now
<krisk> do we need to do them before we agree to have this approved to be what we want to use for async tests
<krisk> the reason I say this is adding a readonly assert should be able to be done later, since it's just a minor add
<jgraham> Yeah I think we can more assertions later
<krisk> ok - since this does help test async stuff like events
<krisk> I can move the script and the sample test then to the approved folder
<krisk> unless you object james
<jgraham> No, I think that is fine as long as we can still make changes to it
Resolved: update domtestcase.js with latest dev version
<krisk> The only issue would be if we want to make some disruptive change that breaks a bunch of tests that is used by domtestcase.js
<krisk> Then this breaking change will need to also update the impacted tests
<krisk> sound fair?
<jgraham> Yes
<jgraham> If people are happy with the basic API
<jgraham> I'm not a good person to judge for obvious reasons
<krisk> I can take an action item to update the wiki with these tests as examples that can cover sync, async, manual (ugh like audio)
<scribe> ACTION: Kris to update the wiki with test information
<krisk> Yes though we need more input from the group
<krisk> Intresting after the meeting Philip Taylor posted results to the list for canvas tests
<krisk> Seems like we can do the same for approved
<krisk> If we think it will help make the spec get to rec and make HTML5 more interoperable
<jgraham> Yes, it would be great to have
<krisk> see http://lists.w3.org/Archives/Public/public-html-testsuite/2010Jun/0036.html
<krisk> I think we could do this...
<krisk> We could have browser vendors submit their results in xml and then have a page load up and transform these results into a result page
<krisk> I can do this once, though the results moving forward would have to come from the vendor
<jgraham> Yeah, something like that would be good although I would choose different specific technologies
<jgraham> :)
<krisk> It's in the open - so sure a vendor could lie...but that would be quite embarassing of the errors were intentional and not a error
<jgraham> Well it is easy to verify any results
<gsnedders> Provided they're public builds, it's easy to check
<jgraham> It doesn't even have to be vendors that submit things really since only public builds should be eligible
<krisk> It could be just submitted to Hg - then it's tracked and will come from a w3c member
<jgraham> Yeah, that sounds like a workable model given some convention for where to submit results files
<krisk> we'll need to not overload the w3c team with this idea
<gsnedders> See what the webapps WG did for widgets
<gsnedders> http://dev.w3.org/2006/waf/widgets/imp-report/
<gsnedders> Just XML files of testfile:testname -> result would do
<krisk> sounds like then opera and microsoft support this idea
<jgraham> Personally I would just use json, since it is basically key-value pairs and xml is overkill
the format isn't critical. xml or json is fine
<jgraham> and I wouldn't format it like the widgets people...
we could accept more than one in fact
<jgraham> The bigger question is how we make a unique id for a test
<jgraham> If we want it to be invariant under directory reorganizations and so on
ah, good point
<jgraham> But maybe that is a non-issue and a path is good enough?
<krisk> I'd rather just stick to using the path
<paulc_> I agree that we might need to handle test suite versions
<jgraham> Regenerating the results is cheap, coming up with complex solutions is hard
<paulc_> As new tests get added or tests get removed, we need to make sure that the results format is robust enough to work.
we could agree to use the path to start with and see how it flies
in terms of formats, as long as I can integrate whatever I get, I don't mind too much
<krisk> Seems like we agree that this is the right direction to head and just have to work out some of the details
it's a matter of transformation for me
<jgraham> Something like {"tests":[{"id":"/path/to/test", "result":"pass"}]} seems simple and extensible
and if start getting too many updates, i'll look into some automatic ways
the test results needs to report the user agent version
<krisk> some of the details would be...
since it may change overtime
<krisk> must include the useragent and build
and OS ?
one vendor might want to report several test results depending on the device/platform
[adjourned]