16:01:49 RRSAgent has joined #htmlt
16:01:49 logging to http://www.w3.org/2013/05/21-htmlt-irc
16:02:17 We can't wait a bit to see if Bin from AT&T will attend today
16:02:25 Agenda -> http://lists.w3.org/Archives/Public/public-html-testsuite/2013May/0002.html
16:05:27 Bin_Hu has joined #htmlt
16:05:34 Hi Kris
16:05:40 Sorry that I am late
16:05:58 no problem
16:05:58 present+ Bin_Hu
16:06:00 Have to run now. Sorry.
16:06:09 OK
16:06:28 What's been discussed so far?
16:06:34 Nothing
16:06:39 So that's good :)
16:06:47 :)
16:07:21 The main goal for todays meeting is to start to update the wiki or even the github so that it lists area of the HTML spec that need tests
16:07:51 All of them?
16:08:00 At the April F2F meeting the group walked through the spec and identified areas in the spec that need tests.
16:08:02 Robin summarized what's been discussed in F2F
16:08:32 This should also help for the 'Tokyo' testthewebforward event
16:08:58 James indicated that there are known interoperability issues even on those items Robin summarized as "green"
16:09:23 yes but we have to distinguish two things here
16:09:35 the getting HTML to Rec project, and the testing the web platform project
16:09:59 the summary I made applies to the former, and that should be treated under the loose definition of interoperability that prevails there
16:10:13 also, it should not interfere with plans to build tests for the latter project
16:10:56 I agree, those are 2 different items, and the former shouldn't interfere with plans to build test for the latter project
16:10:56 I think this sums up the distinction: http://lists.w3.org/Archives/Public/public-html/2013May/0074.html
16:11:06 mdyck1 has joined #HTMLT
16:11:10 There are interoperability issues with everything that isn't tested comprehensively
16:11:18 And with most of the stuff that is as well
16:11:24 Ms2ger: indeed
16:11:50 As for recs... You don't really need tests for that
16:12:02 Ms2ger: and that's something we are working on fixing with WPT
16:12:07 So Kris, is our goal here to ratify the former project, i.e. advance HTML to REC?
16:12:25 indeed, you don't *need* tests, but we need at least a modicum of something
16:12:40 and getting to Rec is important — I want that RF commitment done and solid
16:12:51 or to help with plans for the latter project testing the web platform
16:13:22 Bin_Hu: I would say that the priority of the HTTF is to advance HTML to Rec, but that we want to do so in a way that is maximally useful to the broader project
16:13:56 Robin: that makes sense
16:14:18 I don't see that as a priority at all, but that's just me
16:14:48 Ms2ger: I'll assume that so long as it helps for the broader project you don't care either way?
16:15:13 I also presume that you're not opposed to getting RF commitments on HTML :)
16:15:45 If people want to spend their time on it and it doesn't hurt the real work, sure
16:16:36 right, so I guess we're all agreed
16:17:08 It's not like we are going not accept tests for area that are 'green'
16:17:17 Though I'd prefer the time spent on serious tests, of course
16:17:31 Robin: let's say, if we all agree that current summary satisfies the CR exit criteria, but we still have a lot of work to do to achieve the goal of testing the web platform project,
16:18:28 i.e. the "green" area may turn to "yellow", and we need more tests to ratify the interoperabilty
16:19:52 Looks like we all agree that http://www.w3.org/html/wg/tests-cr-exit.html is a good source of information
16:19:57 If this is a correct assumption, do you expect HTTF to take this work (for HTML), or the wider Testing group to take this work?
16:20:16 Now it may not be perfect - it's ok to 'get' better over time...
16:20:36 mdyck1 has joined #HTMLT
16:21:13 ..and use this information to help guide people that want to write tests for the HTML 5 spec
16:21:13 mdyck_web has joined #htmlt
16:22:23 Bin_Hu: I don't really have a strong opinion about who does the work and where it happens, so long as it does
16:22:25 Well, cr exit crteria is the guide for advancing to Rec, but may not be the best source for guiding people to write the tests for HTML5 in testing the web platform
16:23:06 Bin_Hu: my primary goal is that everyone can contribute happily irrespective of whether they care about getting HTML to Rec or not — that way we don't waste energy over sterile debates :)
16:23:39 Bin_Hu: I would expect that this TF focus on the HTML stuff, and the broader work mostly happens on GitHub, in #testing, in #whatwg, etc.
16:23:53 mdyck1 has joined #HTMLT
16:24:10 Robin: agreed that this TF should focus on HTML stuff
16:24:48 Robin: I support your goal. On the other hand, my primary goal is to make sure that test coverage is sufficient
16:26:00 Bin: The best way is to keep with github+webplatform, it clearly has momentum and is not going to go away
16:26:14 indeed
16:26:15 because the last thing we want to see is that in the real operations of diversified devices with various browsers, the apps work on some devices, but not others
16:27:04 and our customer service has to deal with many many complaints
16:27:51 Indeed interoperability is important
16:28:46 Please educate me: with github+webplatform, can we make sure that all potential interoperability issues will be covered by test cases submitted there?
16:28:46 One way to help this is too have a single spot for all 'tests' - fracturing (e.g.. web audio) only makes things much worse
16:29:14 mdyck1 has left #HTMLT
16:30:38 Having a good spec and tests go a long way towards interoperability
16:30:57 It's far better than a poor spec and no tests
16:31:26 It is "mechanical" to have a single spot for all "tests". The more important is - how can we make sure the test has covered all potential issues?
16:32:03 For example take CSS2.1 - if someone implemented the spec and made their implementation pass all the tests in the suite they are highly likely to be interoperable
16:32:22 ..with other browsers on the market that all pass these tests
16:32:55 I would contrast this with the HTML4 spec and test suite
16:33:15 Sure, that's the way to declare victory. But the real question is - does the "suite" cover everything in CSS2.1?
16:34:11 How can we be assured that?
16:35:01 In the ideal manner it's really a cycle ...
16:35:05 Write a spec...
16:35:18 Someone reads the spec and implements a feature...
16:35:32 Then they give feedback to the spec...
16:35:37 The spec improves...
16:36:01 Tests are written that show issues in the spec and/or in browsers...
16:36:13 Spec gets updated again...
16:36:29 OK. Then I can see that it is really like s/w development cycles, ideally
16:36:56 CSS2.1 has alot more tests and a better spec than CSS1
16:37:18 In another words, test case should be developed in parallel with spec development
16:37:44 That's right. I am not specifically concerned about CSS2.1, but use it as an example
16:37:53 HTML5 is in a simalar spot...
16:38:38 For example when IE was working on 'workers' and Indexed DB we needed to implement SCA (structured clonning algorythm)
16:39:05 The spec was quite clear on what do, though other browses just used JSON at the time.
16:39:47 So we wrote tests and submitted them to the HTML WG and people agreed that the spec was good and the tests were correct
16:40:12 I see. So it seems to me that there is no magic way to verify the test coverage, but follow the cycles
16:40:14 So at the time their was no interop - IE used SCA, other pure JSON, others part SCA/JSON
16:40:35 And if a feature is rarely implemented, it is very likely to have interoperability issue
16:40:41 Though fast forward to today no one uses JSON
16:40:48 and viola we have interop
16:41:03 :)
16:41:19 Ms2ger has joined #HTMLT
16:42:07 Kris, do you still plan to go over HTML5 spec and find the test coverage gaps? (not necessarily today, but as a standing work in this TF)?
16:43:07 Well to start I'm going to leverage the work that occured at the F2F and Robin's page (http://www.w3.org/html/wg/tests-cr-exit.html)
16:43:32 But that's at a very high level
16:43:33 So that it's clear to people what tests we need for events like the testhewebforward
16:45:11 Very open to getting better or re-classifing sections/parts of the spec
16:45:16 For example, we did some readings in 2.1 and 2.2.1, and found some gaps in the past meetings
16:45:38 And the CR exit is green though
16:46:14 it's ok to 'get better' and use peoples feedback
16:47:03 I see.
16:47:14 I don't think anyone can argue that a section is 'green' when one has tests that show interop issues exsist
16:50:16 So your plan is to focus on the tests that are clearly needed based on F2F and Robin's page for testthewebforward events
16:51:26 And open to getting better or re-classifying the sections / parts of the spec. But not necessarily as TF's work, and it realies on members' eyes and contributions
17:01:00 I have to go - let's adjorn the meeting for today
17:01:13 RRSAgent, make log public
17:01:23 RRSAgent, generate minutes
17:01:23 I have made the request to generate http://www.w3.org/2013/05/21-htmlt-minutes.html krisk
17:04:57 what is "RF commitment"?
17:20:12 mdyck_web: Royalty Free
17:22:08 (see the patent policy)
18:49:05 tobie has joined #htmlt
19:51:00 gitbot has joined #htmlt
19:51:00 01[13web-platform-tests01] 15hallvors pushed 2 new commits to 06master: 02https://github.com/w3c/web-platform-tests/compare/82f4d310e539...4ee5b706ee8d
19:51:00 13web-platform-tests/06master 147e13f14 15Thiago Marcos P. Santos: Adding Intel XHR timeout tests....
19:51:00 13web-platform-tests/06master 144ee5b70 15Hallvord R. M. Steen: Merge pull request #91 from tmpsantos/new_xhr_tests...
19:51:00 gitbot has left #HTMLT
20:31:39 Ms2ger has joined #HTMLT
20:42:02 gitbot has joined #htmlt
20:42:02 01[13web-platform-tests01] 15hallvors pushed 2 new commits to 06submission/Microsoft/XHR: 02https://github.com/w3c/web-platform-tests/compare/3ce151cfefa9...319668b74305
20:42:02 13web-platform-tests/06submission/Microsoft/XHR 14116d43a 15Hallvord Reiar M. Steen: fixing merge conflict regarding MANIFEST
20:42:02 13web-platform-tests/06submission/Microsoft/XHR 14319668b 15Hallvord Reiar M. Steen: removing isInitialized workaround for the double readystatechange event that should no longer be sent per spec
20:42:02 gitbot has left #HTMLT
20:49:56 mdyck1 has joined #HTMLT
20:50:16 mdyck1 has left #HTMLT
21:07:25 gitbot has joined #htmlt
21:07:25 01[13web-platform-tests01] 15hallvors pushed 3 new commits to 06submission/Microsoft/XHR: 02https://github.com/w3c/web-platform-tests/compare/319668b74305...b65bc5af0a84
21:07:25 13web-platform-tests/06submission/Microsoft/XHR 14e4e5056 15Hallvord Reiar M. Steen: reversing expected order to align with spec
21:07:25 13web-platform-tests/06submission/Microsoft/XHR 1474c4c6e 15Hallvord Reiar M. Steen: updating helper file to make sure it will work for all new tests
21:07:25 13web-platform-tests/06submission/Microsoft/XHR 14b65bc5a 15Hallvord Reiar M. Steen: minor fixes, changed helper files, whitespace, no double readyState 1 events
21:07:25 gitbot has left #HTMLT