See also: IRC log
trackbot-ng, start telcon
<trackbot> Sorry... I don't know anything about this channel
<trackbot> If you want to associate this channel with an existing Tracker, please say 'trackbot, associate this channel with #channel' (where #channel is the name of default channel for the group)
<krisk> OK
<krisk> sounds fine from my phone
<krisk> shall we get started?
<scribe> scribeNick: plh
Kris: no bugs
<krisk> That is good since we approved alot of tests and moved them into the approved folder
plh: what about the canvas tests?
<krisk> 1 canvas test will be updated and then moved (from MSFT)
kris: there is one canvas from Microsoft?
s/Microsoft\?/Microsoft/
Kris: I expect to have feedback for 50 tests from Philipp Taylor
plh: I thought that opera was using those tests already
Geoffrey: does it actually make sense to review them all and not just the script to generate them
<jgraham> We are using the tests, but we didn't do documented test-by-test review, so we can't trivially approve them
<krisk> we noticed a few issues -
Geoffrey: the only ones that we looked into are the ones we failed
<krisk> for example http://test.w3.org/html/tests/submission/PhilipTaylor/canvas/context.unrecognised.nullsuffix.html
<krisk> This test is testing jscript null handling for example...
Geoffrey: why would it be? it's testing exactly what null does in this context
<jgraham> That test seems fine to me
<gsnedders> s/Goeffrey/Geoffrey/g
<krisk> so does the HTML5 spec state how the "2d\0" string passed in should be treated in the DOM?
<krisk> I recall that 2 browsers treated 2d\0 as a c string == '2d'
I get "Aborted with exception: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) " in firefox
<krisk> though another set of browsers translated it to 2d0
<krisk> and another left 2d\0 asw 2d\0
<jgraham> It states that you should only return a context for the name "2d". "2d\0" isn't "2d"
<krisk> so should it not be just '2d'?
<jgraham> Whether null in strings should cause it to throw is kinda a WebIDL question
<jgraham> I think
I tried the test in opera, firefox, and chrome and they all fail...
<jgraham> The test is testing that you *don't* get a context object back in this case
<gsnedders> WebKit gives undefined and not null
<jgraham> Chromium passes for me
<jgraham> Opera is just buggy in this case
<krisk> We don't need to debug this right now - but
<krisk> this is the type of critical review we want to do for the tests
<krisk> which takes time and why I don't want to just approve them all right away
plh: so all browsers don't
implement this properly. What do we do? change the browsers or
change the spec?
... if the spec doesn't match reality...
<jgraham> (chromium passes for me)
<gsnedders> WebKit trunk passes
<jgraham> (the opera behaviour is insane)
Goeffrey: the sensitive thing to do is what is in the spec
<jgraham> (the firefox behaviour is defensible)
<krisk> This is a good bug (test)
<gsnedders> s/sensitie/sensible/
<jgraham> (conclusion: what's in the spec is no worse than anything else)
<krisk> but really not many web devs are going to type in '2d\0' and depend upon the error handling
<krisk> It has not happend 'yet' - but it is possible to approve a test that all current browsers fail
<krisk> So one ask may be to have browser vendors focus on tests that they fail - since this is an interoperability bug
<krisk> So I'll send a list out to the list this week, ones that are good and one that may need updates.
conclusion: spec is correct and current browser behaviors aren't
<krisk> Then the group can discuss on the list and move forward
geoffrey: we'll go with Kris's recommendations and may file bug later
<krisk> sounds like a way to move forward
<krisk> Since it's actually painful to run the tests
<krisk> I submitted a way to run the tests http://test.w3.org/html/tests/harness/harness.htm
<krisk> It loads tests from a text file - list of uri's
Kris: load list of uris, build an
array, keep track of passes or fails.
... then generates results
Geoffrey: is there anyway to automate this?
<krisk> I'll still need to do two workitems
Kris: still looking at it
<krisk> one to auto run tests that don't need manual verification (yuk)
<krisk> The list comes for all the tests in the approved folder
<jgraham> We seem to have a depressing number of tests that need manual verification already
<krisk> I agree
http://test.w3.org/html/tests/harness/approvedtests.txt
<krisk> The uris come from all the tests in the approved folder
<krisk> So the other workitem is to turn the approvedtests.txt to xml and same with the results
<jgraham> Maybe we should stop approving tests that require manual work unless it can be demonstrated that the feature cannot be tested in a better way
<jgraham> For many of these javascript tests or reftests could work
<krisk> I think for jscript tests that will be fine feedback
but we don't have a framework to run reftests
<jgraham> I propose we beg borrow or steal from CSSWG if we need reftests
<jgraham> I mean borrow their framework
<jgraham> if we need reftests for HTML features
<jgraham> But hopefully we will need fewer than them since we should have fewer visual things to test
<krisk> I don't think we'll end up with too many ref test for HTML
but do their framework generate test results?
<jgraham> I'm not sure. I need to talk to Tab about it
<krisk> I like the idea of reftests - though making it work across browsers, devices, operating systems, dpi, fonts is tricky
<krisk> For example border == thick
<krisk> how many pixels is thick?
more than thin :)
<krisk> true - but then how about inset?
<krisk> who's inset is correct and wrong?
<krisk> though browsers today are different - but correct
<gsnedders> Where in HTML5 is this defined? :)
<krisk> It's not - which is the point
<jgraham> HTML5 says nothing about the rendering in those cases though
<jgraham> So you can't write a test for it at all
<krisk> yep
<jgraham> (the rendering section is all non-normative)
<krisk> So then it would be easy to store the results in Hg
<gsnedders> (like, rendering it with display: none is conforming)
<krisk> Which with XHR you can look for results...0.xml, 1.xml, etc...
<krisk> I can get the UA and store this into the XML file
<krisk> Though I think we can add a 'Browser Name' value and have this added to the result
<krisk> Do you want to submit a draft of test results? with a fake browser?
<krisk> The data pivot - would be feature, test, results
<krisk> I'd keep the format simple html+css
<krisk> the results are not a test :)
<krisk> any thing else to discuss?
<krisk> see you on the list
<krisk> or on irc
[adjourned]
This is scribe.perl Revision: 1.135 of Date: 2009/03/02 03:52:20 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: RRSAgent_Text_Format (score 1.00) FAILED: s/Microsoft\?/Microsoft/ Succeeded: s/it does make/does it actually make sense/ Succeeded: s/Goeffrey/Geoffrey/g Succeeded: s/Goeffrey/Geoffrey/ FAILED: s/Goeffrey/Geoffrey/g FAILED: s/sensitie/sensible/ Found ScribeNick: plh Inferring Scribes: plh WARNING: No "Present: ... " found! Possibly Present: AAPI F2F Goeffrey Microsoft SW_RIF Team_ VB_VBWG Vision W3C_Global WAI_PFWG XML_ aaaa active conclusion geoffrey gsnedders jgraham kris krisk plh scribeNick trackbot You can indicate people for the Present list like this: <dbooth> Present: dbooth jonathan mary <dbooth> Present+ amy Agenda: http://lists.w3.org/Archives/Public/public-html-testsuite/2010Jul/0001.html WARNING: No meeting chair found! You should specify the meeting chair like this: <dbooth> Chair: dbooth Got date from IRC log name: 13 Jul 2010 Guessing minutes URL: http://www.w3.org/2010/07/13-htmlt-minutes.html People with action items:[End of scribe.perl diagnostic output]