16:06:30 <RRSAgent> RRSAgent has joined #htmlt
16:06:30 <RRSAgent> logging to http://www.w3.org/2010/11/16-htmlt-irc
16:06:39 <plh> zakim, make logs public
16:06:39 <Zakim> I don't understand 'make logs public', plh
16:07:08 <krisk> Agenda http://lists.w3.org/Archives/Public/public-html-testsuite/2010Nov/0060.html
16:07:43 <krisk> Item #1 Bugs on approved tests
16:07:53 <krisk> I started a thread on the list http://lists.w3.org/Archives/Public/public-html-testsuite/2010Nov/0062.html
16:08:32 <plh> http://www.w3.org/Bugs/Public/buglist.cgi?product=HTML%20WG&component=testsuite&resolution=---
16:10:05 <jgraham> Oh, telecon
16:10:06 <krisk> This item we can't close on today - looks like their is not agreement in the group on how a page should behave when the page doesn't have the HTML5 doctype
16:10:17 <krisk> We have a few other bugs as well
16:10:19 <krisk> http://www.w3.org/Bugs/Public/show_bug.cgi?id=11321
16:10:51 <jgraham> who is "their"?
16:11:06 <plh> s/their/there/
16:11:13 <plh> Kris will follow up with Anne on the list
16:11:13 <jgraham> Oh
16:11:26 <plh> we have also http://lists.w3.org/Archives/Public/public-html-testsuite/2010Nov/0038.html
16:11:27 <jgraham> OK. The spec is well defined on that point
16:11:44 <jgraham> But yes, we should have this discussion on-list
16:11:49 <krisk> Seems like a bug in the spec
16:12:06 <plh> http://www.w3.org/Bugs/Public/show_bug.cgi?id=11236
16:12:09 <jgraham> Hmm?
16:12:27 <krisk> alot of content on the web is old and would not be compatible with HTML5
16:12:55 <jgraham> This is not a significant problem for us at least
16:12:57 <krisk> It's also not just a 'browser' vendor choice
16:13:09 <krisk> alot of content authoring systems generate html
16:13:21 <krisk> and they don't generate html5 content
16:13:41 <jgraham> If you think the spec should be changed that should go through the normal HTMLWG Process
16:13:50 <krisk> IMHO - let's not discuss this here....
16:13:51 <jgraham> But per current spec, the test is fine afaict
16:14:24 <krisk> The test still is 'approved'
16:14:38 <jgraham> Bug 11321 is something for me. Maybe we could have a different component in bugzilla for bugs in the framework rather than bugs in tests?
16:15:47 <krisk> if we have alot of test harness bugs sure, but that seems like overkill given the number of bugs
16:15:59 <jgraham> Could be
16:16:13 <krisk> The general feedback from TPAC was that the harness was a bit terse to use
16:16:36 <jgraham> "terse"?
16:16:48 <krisk> cumbersome
16:17:16 <krisk> I think we should be open to changes - in theory more folks will write tests
16:17:35 <jgraham> That could be true. There is the possibility of adding a simpler API, but there are tradeoffs with roubustness
16:17:36 <krisk> james do you think you could incorporate this feedback?
16:17:47 <jgraham> I plan to look at it
16:18:02 <krisk> My worry is that we continue to work on the harness and not tests
16:18:14 <jgraham> And encouraging people to write tests is a goal for sure
16:18:48 <krisk> Given the turnout at TPAC (apple, boeing, Opera, Mozilla, Microsoft) was alot more than normal
16:18:59 <jgraham> From my point of view, the tests are something that get developed alongside other work at Opera. The harness needs special attention because it doesn't fall naturally out of implementation work
16:19:18 <krisk> it seems appropriate to start a requirement ask with a end date
16:19:55 <krisk> It would also let everyone have time from various particpants to send in requirements and provide feedback
16:20:04 <jgraham> We could do that for sure
16:20:30 <krisk> lets do that - target a date say in a month
16:21:03 <jgraham> OK
16:22:03 <krisk> Note that I have an action item from TPAC that is dependent up changes to the harness (test type javascript, manual, ref-test)
16:22:13 <krisk> Next bug http://www.w3.org/Bugs/Public/show_bug.cgi?id=11236
16:23:32 <krisk> seems like a legit bug
16:23:49 <jgraham> That action item seems like it would be affected by my proposal for per-directory metadata
16:24:22 <plh> http://test.w3.org/html/tests/approved/video/video_003.htm
16:24:54 <plh> btw, we're working in changing the domain name for test.w3.org
16:24:57 <plh> to avoid XSS issues
16:26:30 <jgraham> Yeah, the bug is legitimate.  It would be good if bugzilla had a "confirmed" state
16:26:34 <plh> "HTML5 Media Elements: 'application/octet-stream' is not a type the UA cannot render."
16:26:50 <krisk> well a UA may say 'maybe'
16:27:31 <krisk> seems like we should remove the test and have it fixed
16:27:47 <plh> +1
16:27:49 <krisk> let's move on
16:28:14 <jgraham> The relevat UA requirement seems to be """ canPlayType(type) method must return the empty string if type is a type that the user agent knows it cannot render or is the type "application/octet-stream" """ fwiw
16:28:22 <jgraham> *relevant
16:28:35 <krisk> agree
16:28:56 <krisk> Item #2 Approval for the next 25 phillip taylor tests
16:29:04 <krisk> They seem fine to me
16:29:24 <krisk> Though at TPAC concern was raise about the progress of getting them all approved
16:29:37 <krisk> How about we pick a date to be all 'reviewed' 
16:29:55 <jgraham> Yes, in general it is not clear to me how the approval process will scale with large numbers of tests
16:30:19 <jgraham> So you propose that we pick a date and say "and objections by this date or we use the bug system"?
16:30:28 <jgraham> +approve
16:30:49 <krisk> that would be the plan
16:30:56 <jgraham> That wfm
16:31:05 <krisk> what is wfm?
16:31:11 <jgraham> "Works for me"
16:31:22 <jgraham> Apologies for being obscure
16:31:29 <krisk> ms2ger gave some feedback as well
16:31:37 <krisk> though the attahed pdf faults?
16:31:56 <jgraham> That also wfm :)
16:32:22 <jgraham> The PDF was a derivation of the correct answers for the tests based on teh Porter Duff operators and the test input
16:32:41 <jgraham> I believe Philip fixed all the issues 
16:32:50 <krisk> He did
16:33:29 <jgraham> (basically the issues were incorrect rounding in the expected output)
16:33:56 <jgraham> So the PDF is interesting but not essential if you are having difficulties opening it
16:34:42 <krisk> assume it has some value, else it would not be attached
16:35:00 <jgraham> Well, yes the value is understanding where the corrections came from
16:35:47 <krisk> anyhow - lets pick end of the year as a target date to approval all of phillip taylors tests
16:36:12 <krisk> I'll send a email to the list - after that date updates can be done via bugs
16:36:28 <krisk> Item #3 TPAC Follow Up
16:37:12 <krisk> before then plh said we have another set of test results from Midori
16:37:36 <krisk> We should have a single person from each vendor handle their test results
16:37:47 <krisk> I can take care of Microsoft
16:37:49 <plh> we'll need to figure what to do with all those test results
16:38:13 <krisk> I think the question should be...
16:38:13 <plh> at least, the wave of results went down
16:38:35 <krisk> Do they represent the browser vendor?
16:38:57 <krisk> If they are member of the w3c it should be easy to check
16:39:30 <plh> so, are you saying we should accept results from browser vendors?
16:39:44 <jgraham> To be honest I am not sure who is benefiting from getting any results at all at this stage
16:39:48 <plh> we could, but this hasn't been clear to me in the past
16:39:59 <jgraham> Apart from advertising to possible contributers
16:39:59 <krisk> Though not sure if the Midori or the maxthon
16:40:28 <jgraham> We *know* the testsuite is woefully incomplete
16:40:29 <krisk> are members of the w3c - though they build a browser and want to have their results displayed
16:40:51 <krisk> that is why it has all the warnings and disclaimers
16:41:03 <krisk> though it's not like someone can't go run the tests themselves
16:41:14 <jgraham> Of course they can
16:41:28 <krisk> this happens all the time - for example a number of sites post SVG results
16:41:50 <krisk> the reason it's good if it's on the w3c is that then everyone gets to have a voice
16:41:50 <jgraham> So, just to play devil's advocate, given all the health warnings we have to put up at the moment, why not just pull the results from public display for now
16:42:09 <jgraham> Then there would be no concern about people misusing them
16:42:18 <jgraham> What would the drawbacks be?
16:42:57 <krisk> it won't help the spec move forward
16:43:35 <jgraham> In what way? The spec is not blocked on a testsuite at the moment
16:43:36 <krisk> It's good to see that a browser I never heard of participate (Midori)
16:44:25 <jgraham> It is good if they are contributing tests. If they are just running the (incomplete) testsuite then I see no benefits
16:44:27 <krisk> the results are all opt-in 
16:44:58 <jgraham> I don't understand what you mean?
16:45:49 <krisk> If browser vendor doesn't want the results shown then they don't have to have them posted
16:46:24 <jgraham> Oh I see. But there is a considerable difference between no-one posting their results and one or two vendors opting out
16:46:38 <jgraham> Until the testsuite reaches a meaningful size
16:47:01 <krisk> Well it seems that on about a month or so we will have more than 1000 tests
16:47:27 <krisk> which is bigger in size than the SVG test suite and about 1/10 the size of the CSS2.1 test suite
16:47:54 <jgraham> I expect we will end up with 10,000-100,000 if we do a good job
16:48:04 <jgraham> Hopefully in the upper end of that range
16:48:19 <krisk> It's not complete but will have more html5 tests than other resources that I am aware of
16:48:21 <jgraham> So we are talking about a few percent of the total number of tests
16:48:30 <jgraham> That we need
16:49:00 <jgraham> (btw I consider it a priority to get some mechanism for mapping tests to the spec so we can see where we have coverage and where we do not)
16:49:22 <jgraham> (Philip's canvas tests do this nicely and I think it might be possible to adapt that mechanism)
16:49:23 <krisk> Intresitng at TPAC this was discussed - use the classname
16:49:40 <krisk> we just need to make sure that ian doesn't change classnames
16:49:43 <jgraham> I don't think that is fine-grained enough
16:49:51 <jgraham> Or, at least, I'm not sure
16:50:03 <jgraham> and lots of the ids and classes in the spec are auto-generated
16:50:17 <krisk> well I think we need something that is concrete
16:50:27 <jgraham> by the pre-processor and don't appear in the original document
16:50:40 <jgraham> Sure, my suggestion is we reuse the mechanism Philip used
16:50:49 <jgraham> Which is based on a regexp match of the text
16:50:52 <plh> using an annotated spec?
16:50:56 <krisk> The reason we started with 'features' was that even chapters numbers were changing a year ago
16:51:14 <krisk> I think we should choose the classname for a while
16:51:16 <jgraham> Also has the advantage that when something changes the regexp is unlikely to still match
16:51:33 <jgraham> So you get to see which tests probably became invalid
16:51:54 <jgraham> And you can link to an actual conformance requirement rather than a general area of the spec
16:52:00 <krisk> I'll take a peek
16:52:04 <jgraham> and the code is mostly written :)
16:52:29 <krisk> moving on...
16:52:53 <krisk> plh can you make sure that 'new' heads are not possible for Hg?
16:53:13 <plh> hu... I don'
16:53:15 <plh> t know
16:53:16 <krisk> at tpac this came up briefly - when people were learning how to submit tests
16:53:27 <krisk> Jonas should be able to help
16:53:48 <plh> I can try to follow up with Jonas on this
16:54:06 <krisk> I'll send an email to Jonas you and the systems folks - sound good?
16:54:07 <jgraham> http://mercurial.selenic.com/wiki/TipsAndTricks#Prevent_a_push_that_would_create_multiple_heads
16:54:12 <plh> kris, ok
16:54:46 <krisk> jonas was also concerned about the logging and said mozilla has a tool to track changes better
16:54:52 <jgraham> (just need a pre-commit hook)
16:55:13 <plh> James, thank you for the pointer, I'll follow up with the webmaster
16:55:19 <krisk> note I move backwards in the list #3.G -> #3.F....
16:55:32 <plh> kris, re tracking tool, yes I already sent an email to our team about this
16:55:34 <jgraham> What was the actual concern about logging?
16:55:39 <plh> might take a while to get deployed
16:55:46 <krisk> that is fine
16:56:07 <plh> james, the names that appear at http://dvcs.w3.org/hg/html/
16:56:13 <plh> are set by the user, and not the server
16:56:43 <krisk> and could be changed and don't map to the w3c account
16:56:53 <plh> so, we don't know for sure how did the push
16:56:57 <plh> s/how/who/
16:57:38 <jgraham> Oh. OK. I don't have a problem with more logging of course but it seems unlikely to be an issue in practice
16:58:04 <plh> well, folks seemed concerned about this during TPAC
16:58:09 <plh> and Jonas pointed me to  http://hg.mozilla.org/hgcustom/pushlog/
16:58:33 <jgraham> Like I said, I don't mind solving it if the oppertunity cost is low
16:58:43 <krisk> given the number of people pushing right now it's not a problem
16:59:04 <plh> I won't push our system team to ct on this one fast
16:59:11 <plh> php and w3ctest is more important imho
16:59:18 <krisk> though if we get more participants it can be a problem (at least some people that have worked on larger project have ran into this problem)
16:59:19 <jgraham> Yeah
16:59:26 <krisk> agree
16:59:30 <jgraham> PHP and w3test are essential
16:59:36 <plh> agreed
16:59:43 <jgraham> (or s/PHP/Python :)
16:59:50 <krisk> so how is the XSS and PHP stuff coming along?
17:00:06 <plh> w3test should be done pretty fast
17:00:13 <krisk> I recall we agreed to have the name be test-w3.org
17:00:14 <plh> we bought the domain name already
17:00:34 <krisk> for sure it should be registered and have the w3c be the owner
17:00:40 <krisk> what is the name?
17:00:45 <plh> w3c-test.org
17:01:20 <krisk> will we still have the test.w3c-test.org and test2.w3c-test.org?
17:01:28 <plh> yes
17:01:52 <krisk> james not sure how much you talked to anne post TPAC
17:02:12 <plh> for php, I know that our system folks looked into that and are coming up with a solution
17:02:16 <krisk> though it seems that we needs a few specific PHP pages
17:02:22 <plh> dunno about the ETA at this point
17:02:39 <krisk> the group didn't want to create alot of PHP pages
17:02:43 <jgraham> I hve talked to Anne a bit
17:03:07 <krisk> in fact it may not be php - could be any generic cgi script
17:03:27 <plh> http://www.w3.org/2008/webapps/wiki/Testing_Requirements
17:03:30 <krisk> so basically we don't want to enable anyone to just push a new php file to the server
17:04:05 <jgraham> Yes, there are reasonable security concerns
17:04:07 <krisk> the php file would be a manually added to the web servers unlike the rest of the content
17:04:36 <krisk> any other agenda items?
17:04:36 <plh> I'm guessing they'll create /scripts or something like that
17:04:49 <jgraham> It would be useful if it was still VCS backed somehow
17:05:03 <plh> and move the php/python/whatever into /scripts as they are approving them
17:05:15 <plh> james, we can still maintain them under /html
17:05:21 <krisk> It could be maybe a seperate project that only get proped to the web server upon demand
17:05:24 <jgraham> Right. That seems like it should work
17:06:00 <plh> it's just that we'll only execute them when they're place under /scripts
17:06:07 <plh> placed
17:06:08 <jgraham> They are managed in hg but the server only executes them in a special directory which they only reach after review for security problems
17:06:17 <plh> yep
17:06:38 <plh> as long as the review process isn't long, we should be fine
17:07:02 <krisk> let's adjurn
17:07:14 <Zakim> -krisk
17:07:36 <krisk> rrsagent, generate minutes
17:07:36 <RRSAgent> I have made the request to generate http://www.w3.org/2010/11/16-htmlt-minutes.html krisk
17:12:14 <Zakim> disconnecting the lone participant, Plh, in HTML_WG(HTMLT)11:00AM
17:12:17 <Zakim> HTML_WG(HTMLT)11:00AM has ended
17:12:17 <Zakim> Attendees were Plh, krisk
17:19:39 <gsnedders_web> gsnedders_web has joined #htmlt
17:59:51 <plh> plh has left #htmlt
18:41:16 <Zakim> Zakim has left #htmlt