16:01:49 RRSAgent has joined #htmlt 16:01:49 logging to http://www.w3.org/2013/05/21-htmlt-irc 16:02:17 We can't wait a bit to see if Bin from AT&T will attend today 16:02:25 Agenda -> http://lists.w3.org/Archives/Public/public-html-testsuite/2013May/0002.html 16:05:27 Bin_Hu has joined #htmlt 16:05:34 Hi Kris 16:05:40 Sorry that I am late 16:05:58 no problem 16:05:58 present+ Bin_Hu 16:06:00 Have to run now. Sorry. 16:06:09 OK 16:06:28 What's been discussed so far? 16:06:34 Nothing 16:06:39 So that's good :) 16:06:47 :) 16:07:21 The main goal for todays meeting is to start to update the wiki or even the github so that it lists area of the HTML spec that need tests 16:07:51 All of them? 16:08:00 At the April F2F meeting the group walked through the spec and identified areas in the spec that need tests. 16:08:02 Robin summarized what's been discussed in F2F 16:08:32 This should also help for the 'Tokyo' testthewebforward event 16:08:58 James indicated that there are known interoperability issues even on those items Robin summarized as "green" 16:09:23 yes but we have to distinguish two things here 16:09:35 the getting HTML to Rec project, and the testing the web platform project 16:09:59 the summary I made applies to the former, and that should be treated under the loose definition of interoperability that prevails there 16:10:13 also, it should not interfere with plans to build tests for the latter project 16:10:56 I agree, those are 2 different items, and the former shouldn't interfere with plans to build test for the latter project 16:10:56 I think this sums up the distinction: http://lists.w3.org/Archives/Public/public-html/2013May/0074.html 16:11:06 mdyck1 has joined #HTMLT 16:11:10 There are interoperability issues with everything that isn't tested comprehensively 16:11:18 And with most of the stuff that is as well 16:11:24 Ms2ger: indeed 16:11:50 As for recs... You don't really need tests for that 16:12:02 Ms2ger: and that's something we are working on fixing with WPT 16:12:07 So Kris, is our goal here to ratify the former project, i.e. advance HTML to REC? 16:12:25 indeed, you don't *need* tests, but we need at least a modicum of something 16:12:40 and getting to Rec is important — I want that RF commitment done and solid 16:12:51 or to help with plans for the latter project testing the web platform 16:13:22 Bin_Hu: I would say that the priority of the HTTF is to advance HTML to Rec, but that we want to do so in a way that is maximally useful to the broader project 16:13:56 Robin: that makes sense 16:14:18 I don't see that as a priority at all, but that's just me 16:14:48 Ms2ger: I'll assume that so long as it helps for the broader project you don't care either way? 16:15:13 I also presume that you're not opposed to getting RF commitments on HTML :) 16:15:45 If people want to spend their time on it and it doesn't hurt the real work, sure 16:16:36 right, so I guess we're all agreed 16:17:08 It's not like we are going not accept tests for area that are 'green' 16:17:17 Though I'd prefer the time spent on serious tests, of course 16:17:31 Robin: let's say, if we all agree that current summary satisfies the CR exit criteria, but we still have a lot of work to do to achieve the goal of testing the web platform project, 16:18:28 i.e. the "green" area may turn to "yellow", and we need more tests to ratify the interoperabilty 16:19:52 Looks like we all agree that http://www.w3.org/html/wg/tests-cr-exit.html is a good source of information 16:19:57 If this is a correct assumption, do you expect HTTF to take this work (for HTML), or the wider Testing group to take this work? 16:20:16 Now it may not be perfect - it's ok to 'get' better over time... 16:20:36 mdyck1 has joined #HTMLT 16:21:13 ..and use this information to help guide people that want to write tests for the HTML 5 spec 16:21:13 mdyck_web has joined #htmlt 16:22:23 Bin_Hu: I don't really have a strong opinion about who does the work and where it happens, so long as it does 16:22:25 Well, cr exit crteria is the guide for advancing to Rec, but may not be the best source for guiding people to write the tests for HTML5 in testing the web platform 16:23:06 Bin_Hu: my primary goal is that everyone can contribute happily irrespective of whether they care about getting HTML to Rec or not — that way we don't waste energy over sterile debates :) 16:23:39 Bin_Hu: I would expect that this TF focus on the HTML stuff, and the broader work mostly happens on GitHub, in #testing, in #whatwg, etc. 16:23:53 mdyck1 has joined #HTMLT 16:24:10 Robin: agreed that this TF should focus on HTML stuff 16:24:48 Robin: I support your goal. On the other hand, my primary goal is to make sure that test coverage is sufficient 16:26:00 Bin: The best way is to keep with github+webplatform, it clearly has momentum and is not going to go away 16:26:14 indeed 16:26:15 because the last thing we want to see is that in the real operations of diversified devices with various browsers, the apps work on some devices, but not others 16:27:04 and our customer service has to deal with many many complaints 16:27:51 Indeed interoperability is important 16:28:46 Please educate me: with github+webplatform, can we make sure that all potential interoperability issues will be covered by test cases submitted there? 16:28:46 One way to help this is too have a single spot for all 'tests' - fracturing (e.g.. web audio) only makes things much worse 16:29:14 mdyck1 has left #HTMLT 16:30:38 Having a good spec and tests go a long way towards interoperability 16:30:57 It's far better than a poor spec and no tests 16:31:26 It is "mechanical" to have a single spot for all "tests". The more important is - how can we make sure the test has covered all potential issues? 16:32:03 For example take CSS2.1 - if someone implemented the spec and made their implementation pass all the tests in the suite they are highly likely to be interoperable 16:32:22 ..with other browsers on the market that all pass these tests 16:32:55 I would contrast this with the HTML4 spec and test suite 16:33:15 Sure, that's the way to declare victory. But the real question is - does the "suite" cover everything in CSS2.1? 16:34:11 How can we be assured that? 16:35:01 In the ideal manner it's really a cycle ... 16:35:05 Write a spec... 16:35:18 Someone reads the spec and implements a feature... 16:35:32 Then they give feedback to the spec... 16:35:37 The spec improves... 16:36:01 Tests are written that show issues in the spec and/or in browsers... 16:36:13 Spec gets updated again... 16:36:29 OK. Then I can see that it is really like s/w development cycles, ideally 16:36:56 CSS2.1 has alot more tests and a better spec than CSS1 16:37:18 In another words, test case should be developed in parallel with spec development 16:37:44 That's right. I am not specifically concerned about CSS2.1, but use it as an example 16:37:53 HTML5 is in a simalar spot... 16:38:38 For example when IE was working on 'workers' and Indexed DB we needed to implement SCA (structured clonning algorythm) 16:39:05 The spec was quite clear on what do, though other browses just used JSON at the time. 16:39:47 So we wrote tests and submitted them to the HTML WG and people agreed that the spec was good and the tests were correct 16:40:12 I see. So it seems to me that there is no magic way to verify the test coverage, but follow the cycles 16:40:14 So at the time their was no interop - IE used SCA, other pure JSON, others part SCA/JSON 16:40:35 And if a feature is rarely implemented, it is very likely to have interoperability issue 16:40:41 Though fast forward to today no one uses JSON 16:40:48 and viola we have interop 16:41:03 :) 16:41:19 Ms2ger has joined #HTMLT 16:42:07 Kris, do you still plan to go over HTML5 spec and find the test coverage gaps? (not necessarily today, but as a standing work in this TF)? 16:43:07 Well to start I'm going to leverage the work that occured at the F2F and Robin's page (http://www.w3.org/html/wg/tests-cr-exit.html) 16:43:32 But that's at a very high level 16:43:33 So that it's clear to people what tests we need for events like the testhewebforward 16:45:11 Very open to getting better or re-classifing sections/parts of the spec 16:45:16 For example, we did some readings in 2.1 and 2.2.1, and found some gaps in the past meetings 16:45:38 And the CR exit is green though 16:46:14 it's ok to 'get better' and use peoples feedback 16:47:03 I see. 16:47:14 I don't think anyone can argue that a section is 'green' when one has tests that show interop issues exsist 16:50:16 So your plan is to focus on the tests that are clearly needed based on F2F and Robin's page for testthewebforward events 16:51:26 And open to getting better or re-classifying the sections / parts of the spec. But not necessarily as TF's work, and it realies on members' eyes and contributions 17:01:00 I have to go - let's adjorn the meeting for today 17:01:13 RRSAgent, make log public 17:01:23 RRSAgent, generate minutes 17:01:23 I have made the request to generate http://www.w3.org/2013/05/21-htmlt-minutes.html krisk 17:04:57 what is "RF commitment"? 17:20:12 mdyck_web: Royalty Free 17:22:08 (see the patent policy) 18:49:05 tobie has joined #htmlt 19:51:00 gitbot has joined #htmlt 19:51:00 01[13web-platform-tests01] 15hallvors pushed 2 new commits to 06master: 02https://github.com/w3c/web-platform-tests/compare/82f4d310e539...4ee5b706ee8d 19:51:00 13web-platform-tests/06master 147e13f14 15Thiago Marcos P. Santos: Adding Intel XHR timeout tests.... 19:51:00 13web-platform-tests/06master 144ee5b70 15Hallvord R. M. Steen: Merge pull request #91 from tmpsantos/new_xhr_tests... 19:51:00 gitbot has left #HTMLT 20:31:39 Ms2ger has joined #HTMLT 20:42:02 gitbot has joined #htmlt 20:42:02 01[13web-platform-tests01] 15hallvors pushed 2 new commits to 06submission/Microsoft/XHR: 02https://github.com/w3c/web-platform-tests/compare/3ce151cfefa9...319668b74305 20:42:02 13web-platform-tests/06submission/Microsoft/XHR 14116d43a 15Hallvord Reiar M. Steen: fixing merge conflict regarding MANIFEST 20:42:02 13web-platform-tests/06submission/Microsoft/XHR 14319668b 15Hallvord Reiar M. Steen: removing isInitialized workaround for the double readystatechange event that should no longer be sent per spec 20:42:02 gitbot has left #HTMLT 20:49:56 mdyck1 has joined #HTMLT 20:50:16 mdyck1 has left #HTMLT 21:07:25 gitbot has joined #htmlt 21:07:25 01[13web-platform-tests01] 15hallvors pushed 3 new commits to 06submission/Microsoft/XHR: 02https://github.com/w3c/web-platform-tests/compare/319668b74305...b65bc5af0a84 21:07:25 13web-platform-tests/06submission/Microsoft/XHR 14e4e5056 15Hallvord Reiar M. Steen: reversing expected order to align with spec 21:07:25 13web-platform-tests/06submission/Microsoft/XHR 1474c4c6e 15Hallvord Reiar M. Steen: updating helper file to make sure it will work for all new tests 21:07:25 13web-platform-tests/06submission/Microsoft/XHR 14b65bc5a 15Hallvord Reiar M. Steen: minor fixes, changed helper files, whitespace, no double readyState 1 events 21:07:25 gitbot has left #HTMLT