15:02:14 <RRSAgent> RRSAgent has joined #htmlt
15:02:14 <RRSAgent> logging to http://www.w3.org/2011/05/31-htmlt-irc
15:02:14 <Ms2ger> Thanks for reminding me :)
15:02:29 <krisk> RRSAgent, make logs public
15:02:41 <krisk> zakim, who is here?
15:02:41 <Zakim> sorry, krisk, I don't know what conference this is
15:02:42 <Zakim> On IRC I see RRSAgent, Zakim, krisk, plh, Ms2ger, gsnedders, jgraham
15:02:53 <krisk> zakim, this is htmlt
15:02:53 <Zakim> sorry, krisk, I do not see a conference named 'htmlt' in progress or scheduled at this time
15:03:00 <Ms2ger> Still? :)
15:03:32 <plh> zakim, list conferences
15:03:32 <Zakim> I see SW_(HCLS)10:30AM, XML_ET-TF()11:00AM, UW_WebTVIG(Home Net)10:00AM active
15:03:34 <Zakim> also scheduled at this time are DIG_weekly()11:00AM, T&S_XMLSEC()10:00AM, SW_RIF()11:00AM, I18N_ITS IG()11:00AM, RWC_WebEven()11:00AM, VB_VBWG()10:00AM, WAI_UAWG(CHAIRS)10:30AM,
15:03:36 <Zakim> ... SW_(SPARQL)10:00AM, SW_HCLS(COI)11:00AM, WAI_PFWG(HTML_TF)11:00AM
15:03:39 <krisk> I'll dail in to the conf call...though I suspect again this will be on IRC
15:03:55 <krisk> feel free to fix this plh
15:04:05 <plh> hum... maybe I need to renew the bridge or something?
15:04:57 <plh> indeed, no reservation anymore
15:05:04 <plh> kris: did you manage to dial in?
15:05:10 <plh> s/:/,/
15:05:18 <krisk> ...I get a this passcode is not valid
15:05:25 <krisk> ..error
15:05:27 <plh> zakim, room for 5?
15:05:29 <Zakim> ok, plh; conference Team_(htmlt)15:05Z scheduled with code 26632 (CONF2) for 60 minutes until 1605Z; however, please note that capacity is now overbooked
15:05:40 <Zakim> Team_(htmlt)15:05Z has now started
15:05:40 <Zakim> +Plh
15:05:47 <plh> use this one for today
15:05:53 <plh> and I'll renew the bridge reservation
15:06:01 <krisk> thanks
15:06:36 <Zakim> +[Microsoft]
15:06:44 <krisk> Zakim, Microsoft is me
15:06:44 <Zakim> +krisk; got it
15:08:22 <krisk> OK let's get going
15:09:05 <krisk> Agenda: http://lists.w3.org/Archives/Public/public-html-testsuite/2011May/0006.html
15:09:51 <krisk> Item #1 Check for any bugs on approved tests
15:10:02 <krisk> I don't see any new bugs on approved tests (good)
15:10:19 <krisk> Next agenda item...
15:10:49 <krisk> Last time we discussed setting a date to release the test suite
15:11:33 <krisk> To do this we could move to using a branch to fix/stablize any of the tests while another branch is open to new test submissions
15:12:19 <jgraham> Why would we want fixes not to be ported to HEAD?
15:12:48 <jgraham> If we keep the current setup with submitted/approved directories I don't see any need for branches
15:12:57 <Ms2ger> Me neither
15:13:05 <jgraham> The possible use for branches would be to replace submitted/approved directories
15:13:39 <jgraham> Which I think would have some advantages, but I don't see a really good way to do it without having one branch per unapproved testsuite
15:14:08 <jgraham> Which *I* would be happy with, but people less used to hg/dvcs probably less so
15:14:50 <jgraham> (I would like to think of a good way to have one branch for submitted and one branch for approved, but accounting for the commits becomes a bit difficult)
15:15:29 <jgraham> (since you would need to hg transplant approved commits across and I don't think that preserves the hash)
15:16:36 <krisk> Another draw back I can think of (do we care?) if we want to date a test suite release
15:16:56 <plh> we don't care that much in keeping historical test suites, except maybe for the one we'll use to move html5 from CR to PR. And even for that, we can just tag the repository of something. no need to create a separate branch
15:17:15 <jgraham> Yeah, tagging specific revisions is fine
15:17:21 <krisk> Though this could be take care of in another manner - list it on the wiki
15:17:53 <krisk> Though I don't no the Hg command off the top of my head (no pun intended)
15:18:15 <krisk> ..it should be easy for one to sync to a date
15:18:29 <plh> I'm not familiar with mercurial, but with cvs, you were able to tag a set of files. we used to do that for the DOM specifications in fact, so we had the possibility to check out an old version of the spec (whcih we never used I think)
15:18:43 <krisk> ...at least this is easy to do with other source control control mechanisms
15:18:43 <Ms2ger> hg update HTML_20120314
15:18:55 <jgraham> In mercurial you can tag a revision
15:19:29 <krisk> Ah...http://mercurial.selenic.com/wiki/Tag
15:20:26 <krisk> OK then lets not create a branch and then we can 'tag' the release and list this on the wiki
15:20:37 <plh> yes
15:20:43 <krisk> do we all concour?
15:20:55 <jgraham> Yeah
15:21:00 <Ms2ger> +1
15:21:18 <plh> +1
15:21:47 <krisk> Agenda Item #3 Approve the reflection attribute tests See -> http://w3c-test.org/html/tests/submission/AryehGregor/reflection/reflection-metadata.html
15:22:04 <krisk> I have seen no objections on the list
15:22:30 <krisk> so then unless someone objects we should approve these reflection-metadata tests
15:22:55 <Ms2ger> I'd approve them all
15:22:55 <krisk> If someone has an issue they can file a bug like other approved tests
15:23:02 <plh> +1 to approve the tests
15:23:24 <krisk> Any other items that people want to discuss?
15:23:44 <krisk> ms2ger did you look at the last few conf call notes?
15:23:48 <plh> so, what's our schedule for approving other tests?
15:23:48 <jgraham> Yes, I expect there are bugs in them, but I expect we will find them when people start running them
15:23:53 <Ms2ger> No
15:24:24 <jgraham> (another reason that running tests is so important)
15:24:43 <krisk> Let me catch you up (ms2ger)
15:25:32 <krisk> ..you can also look at the list and read the notes/irc logs
15:25:44 <plh> http://www.w3.org/2011/Talks/0516-htmlupdate/#%289%29 says we have 28,858 tests submitted. I'm curious to know what's the plan for approving them...
15:26:05 <Ms2ger> Is that a decimal comma?
15:26:09 <krisk> basically we want to pick a date (Fall?, early winter) and approve all submitted tests
15:26:28 <plh> s/28,858/28858/
15:26:30 <krisk> Then we'll have two months to take fixes to the tests
15:26:33 <jgraham> plh: I am way more concerend with "what is our plan for getting people running the tests"
15:27:01 <krisk> at the end of the two months we will release the test suite with some name (alpha, beta, release #1, etc..)
15:27:28 <jgraham> Approval is theoretically nice but nothing we do is useful if people aren't running the tests
15:27:30 <plh> James, we're trying to address this issue in the testing effort. we're going to deploy the new framework in June in fact
15:27:55 <jgraham> "new framework"?
15:28:02 <plh> James, we'll use with the css framework as the starting point
15:28:28 <jgraham> Hmm, that doesn't sound like it addresses any of the problems that we (Opera) have with running the tests
15:28:42 <jgraham> I don't know what problems others have
15:28:57 <plh> why can't open run the tests?
15:28:59 <Ms2ger> I think PHP will end up being a problem for us
15:29:00 <plh> s/open/opera/
15:29:14 <jgraham> plh: See my mail today
15:29:37 <jgraham> <http://www.w3.org/mid/4DE4C870.7020408@opera.com>
15:29:41 <krisk> can you send a link to the 'my mail today'
15:29:53 <plh> James, I'll look at it
15:29:53 <krisk> I get a 404
15:29:56 <plh> http://lists.w3.org/Archives/Public/public-html-testsuite/2011May/0007.html
15:30:41 <jgraham> I am trying to solve issue 1 internally
15:31:28 <jgraham> Ms2ger: Mozilla use javascript only, right?
15:31:31 <krisk> It's good that vendors are running the tests
15:31:38 <jgraham> With a custom HTTP server?
15:31:59 <Ms2ger> Afaik, yes
15:32:28 <krisk> Mozilla does seem able to run the tests today
15:32:40 <krisk> ..they have sent test results to the list a few times
15:32:44 <jgraham> Interesting. We use PHP ourselves so that is no problem
15:32:52 <Ms2ger> They're not integrated
15:32:57 <plh> James, with the new framework, the goal is to get as many groups as possible to use testharness.js and reftests. if we make it easy for them to do so, they'll be happier to use them
15:33:03 <krisk> ..I have not looked to see if they have opened bugs in their bugzilla DB
15:33:12 <Ms2ger> I sure have
15:33:35 <jgraham> plh: OK. I agree that only having to deal with a small number of test formats is a big win
15:33:38 <plh> btw, php is used by some tests, like XHR
15:33:45 <Ms2ger> Right, that's my point
15:33:55 <plh> (or web performance)
15:34:12 <krisk> keeping the number of test formats is good
15:34:52 <plh> yes, if everyone use the same metadata input, and the same test format, we'll reduce the stress
15:35:07 <jgraham> plh: I think the point is that Mozilla don't use external servers and don't support PHP on the servers they do use
15:35:13 <jgraham> for testing
15:35:26 <Ms2ger> Exactly
15:35:36 <plh> then how do they test things like XHR?
15:35:42 <Ms2ger> Server-side JS
15:35:51 <jgraham> (because aiui "servers they do use" is actually a custom javascript server that runs on the same machine as the browser being tested)
15:35:56 <plh> ah, something like node.js ?
15:36:48 <Ms2ger> With a sufficiently wide definition of "like" :)
15:36:54 <plh> :)
15:36:54 <jgraham> Anyone know what WebKit do?
15:36:57 <krisk> I think that as long as we have a few format then vendors can follow
15:37:06 <krisk> how they choose to follow is their own choice
15:37:23 <krisk> They can use PHP (like others) or another tool to do the same job
15:37:26 <jgraham> krisk: It is a problem if we write tests in a format that vendors can't use
15:37:55 <jgraham> It may be that vendors have to make some changes of course, but we should try to minimise the pain in running the tests
15:38:05 <Ms2ger> I don't expect us to rewrite the XHR tests to not use PHP
15:38:06 <jgraham> Because tests that aren't run aren't useful
15:38:19 <Ms2ger> Or to rewrite our own tests to use PHP, to get them submitted
15:38:37 <jgraham> Ms2ger: Do you expect Mozilla to run the XHR tests?
15:38:48 <Ms2ger> I would hope so
15:38:54 <jgraham> OK
15:39:06 <plh> so, it's a question of which programming language to use for the scripts that are required by the tests...
15:39:15 <Ms2ger> That's one question
15:39:48 <plh> this is separate of the language used by the framework btw. the css framework is written in php.
15:40:06 <Ms2ger> That's separate indeed
15:40:47 <jgraham> Yes, my expectation (that could turn out to be wrong) is that the framework will be irrelevant to vendors
15:40:51 <plh> I mean besides limiting the number of scripts, and ensuring we have the right coverage of languages, I'm not sure there is much we can do
15:41:04 <jgraham> Except insofar as it makes people all write tests in the same formats
15:41:11 <plh> James, correct
15:41:29 <plh> James, I see the framework has only one among possible many
15:42:12 <krisk> Having a consistent format is the key part
15:42:22 <jgraham> plh: Well when designing things we should take vendor needs into account e.g. testharness.js tries to allow the results to be reported in a flexible vendor-specific way
15:42:27 <plh> so far the scripts written are fairly limited. we have those in XHR, and web performance reused them (by copying them)
15:42:37 <krisk> ..this should allow anyone to follow
15:42:48 <plh> James, agreed
15:43:03 <krisk> Ok any other agenda items?
15:43:04 <jgraham> and reftests are better than visual tests for vendors, particularly those with very non-centralised testing setups
15:43:33 <plh> agreed
15:43:36 <jgraham> There are probably going to be other examples in the future
15:49:46 <krisk> Let's adjourn
15:49:48 <krisk> rrsagent, generate minutes
15:49:48 <RRSAgent> I have made the request to generate http://www.w3.org/2011/05/31-htmlt-minutes.html krisk
15:51:46 <Zakim> -Plh
15:51:48 <Zakim> -krisk
15:51:48 <Zakim> Team_(htmlt)15:05Z has ended
15:51:50 <Zakim> Attendees were Plh, krisk
15:53:59 <gsnedders> jgraham et al.: WebKit use PHP for server-side stuff
15:54:35 <jgraham> OK
17:30:18 <Zakim> Zakim has left #htmlt
17:30:53 <Steve> RRSAgent, please excuse us
17:30:53 <RRSAgent> I see no action items