14:56:23 <RRSAgent> RRSAgent has joined #htmlt
14:56:23 <RRSAgent> logging to http://www.w3.org/2013/03/26-htmlt-irc
15:01:13 <krisk> Pretty light agenda today, so let's wait a bit to see if others attend
15:01:33 <krisk> Agenda -> http://lists.w3.org/Archives/Public/public-html-testsuite/2013Mar/0018.html
15:02:06 <jgraham> Is this meeting now? I thiught it was in an hour
15:02:14 <jgraham> Not that it matters to me
15:03:09 <jgraham> Well I guess this is the time on the mail you sent out at least
15:04:21 <krisk> Oh yeah I guess I'm a bit early..
15:04:48 <krisk> We can wait till 16:00 UTC
15:06:14 <jgraham> Oh, right I am confused too :)
15:06:38 <jgraham> We are goign for 16:00 UTC?
15:07:04 <krisk> Yes that seems to make the most sense...
15:14:44 <jgraham> 'K. I am never sure if W3C meetings are anchored in UTC or in Boston time.
15:16:44 <mdyck> mdyck has joined #HTMLT
15:17:33 <krisk> I was not aware of a 'standard' per se, though next fall we can try again...
15:17:55 <krisk> Choosing UTC vs Europe vs US time...
15:18:58 <krisk> I don't have a preference
15:22:19 <mdyck> mdyck has joined #HTMLT
15:25:01 <Ms2ger> Ms2ger has joined #HTMLT
15:26:04 <mdyck> mdyck has joined #HTMLT
15:34:54 <mdyck> mdyck has joined #HTMLT
15:43:11 <mdyck> mdyck has joined #HTMLT
15:57:20 <mdyck> mdyck has left #HTMLT
15:58:53 <mdyck1> mdyck1 has joined #HTMLT
15:59:25 <mdyck1> test
15:59:31 <krisk> test passed
15:59:35 <Ms2ger> test failed
16:00:12 <mdyck1> hm, don't have two interoperable implementations.
16:00:35 <Ms2ger> Go back to LC
16:00:35 <jgraham> Test in a quantum superposition of states?
16:00:58 <Bin> Bin has joined #HTMLT
16:01:14 <krisk> Welcome Bin
16:01:28 <krisk> Feel free to introduce yourself
16:02:03 <tobie> krisk: I never got to introduce myself. This is so unfair.
16:02:15 <Ms2ger> Hi all
16:02:18 <Ms2ger> This is tobie
16:02:22 <Ms2ger> He does everything
16:02:39 <krisk> Feel free tobie
16:02:41 <Bin> This is Bin from AT&T
16:02:51 <Bin> Glad to meet everone here
16:03:03 <krisk> Here is the meeting agenda -> http://lists.w3.org/Archives/Public/public-html-testsuite/2013Mar/0018.html
16:03:09 <tobie> Welcome Bin!
16:03:17 <Bin> Thank you Tobie
16:04:05 <krisk> I wanted to use this meeting to start to walk through the spec a classify sections
16:04:21 <Bin> sure.
16:04:34 <Bin> BTW, I couldn't dial in the conference bridge
16:04:37 <Ms2ger> I'm not sure what you mean by that
16:04:46 <Bin> is the conference code 48658?
16:04:58 <Ms2ger> There is no conf call
16:05:04 <Ms2ger> We just hang out on IRC
16:05:13 <Bin> I see. sorry :)
16:05:14 <krisk> Once we have agreement I'll place this on the Wiki so that people have a list of stuff to focus their testing
16:05:16 <tobie> Bin: and tell jokes
16:05:53 <tobie> krisk: ETA for this doc?
16:06:17 <krisk> Like most stuff in the HTML spec it'll take time
16:06:45 <krisk> I sent out an email to the list http://lists.w3.org/Archives/Public/public-html-testsuite/2013Mar/0009.html
16:06:57 <krisk> As a start
16:07:13 <krisk> Starting from the top of the spec and moving down...
16:07:32 <krisk> The first 'section' that has normative requirements is 2.1 terminology
16:07:34 <krisk> http://www.w3.org/TR/html51/infrastructure.html#terminology
16:10:59 <mdyck1> darobin: It would be good if your "Test Suite Coverage Analysis" page showed which version of the spec and test suite it analyzed.
16:11:31 <darobin> mdyck1: yes, we're looking into basically making it map to URLs
16:11:57 <darobin> basically, a URL can be mapped to a TS directory, and you can assess its coverage thus
16:13:09 <krisk> I don't believe the 2.1 section (not 2.1.1, 2.1.2, etc) has any normative requirements
16:13:15 <krisk> Does anyone disagree?
16:14:51 <mdyck1> mdyck1 has joined #HTMLT
16:14:58 <mdyck1> URLs tell you where the data came from (so an improvement), but if the thing at that URL changes over time, you'd also want to indicate *when* the analysis happened (or when you grabbed the data)
16:15:21 <Bin> I agree. Actually, the subsections (DOM trees, Scripting etc.) should be covered by the other part of the specs
16:15:58 <krisk> mdyck1: I'll place a pointer to the spec (dated version) on the wiki
16:16:30 <krisk> Looking at section 2.1.1 and 2.1.2 I also don't see any normative requirements.
16:16:37 <krisk> Does anyone disagree?
16:16:54 <Bin> There are cross-references, and those anchors should be tested in the normative sections where they are actually specified
16:17:09 <Ms2ger> It's all definitions
16:19:04 <mdyck1> coverage analysis says first '2119's is in 2.1.3 dom-trees
16:19:25 <mdyck1> "A   user agent must not mutate the DOM in such situations." presumably
16:19:59 <krisk> I agree and I don't believe we have a test for this normative requirement
16:20:22 <krisk> any disagreement?
16:20:51 <mdyck1> coverage page says zero tests
16:21:11 <krisk> Now looking at 2.1.4 'Scripting'
16:22:01 <krisk> This section also has a normative requirement and no tests to my knowledge
16:22:10 <krisk> "object must operate on the actual underlying "
16:23:01 <Bin> If a DOM object is said to be live, then the attributes and methods on that object must operate on the actual underlying data, not a snapshot of the data.
16:23:19 <krisk> Looks like we have agreement
16:23:29 <krisk> Moving on to 2.1.5 Plugins
16:23:32 <tobie> Are you  planning to go through the whole HTML spec on irc?
16:24:23 <krisk> not today though over time it's part of the HTML task force charter to build a comprehensive test suite
16:24:24 <mdyck1> But a test would have to be with respect to a particular DOM object that the spec says is live, so the test would be tied to the section where that statement is made, not to 2.1.4.
16:24:45 <krisk> One way to build a comprehensive test suite is to enumerate all the normative sections that have no tests
16:25:08 <mdyck1> same with 2.1.3
16:25:13 <krisk> So that we can publicly ask for test contributions with direction
16:27:04 <krisk> mdyck1: we can also agree that though a section has a normative requirement the test should part of another section of the spec
16:27:19 <krisk> I'm open to either approach
16:27:51 <krisk> Looking at section 2.1.5
16:28:01 <krisk> This section has 2 normative requirements
16:28:31 <krisk> 'but that neither act as child browsing contexts of the Document nor introduce any Node objects to the Document's DOM'
16:28:34 <krisk> and...
16:28:42 <krisk> A user agent must not consider the types text/plain and application/octet-stream as having a registered plugin.
16:29:43 <krisk> I don't believe the first one is really an interop issue today
16:30:16 <mdyck1> that first one is part of defn of "plugin"
16:31:04 <Bin> That's right. The first one shouldn't be an issue
16:31:08 <tobie> krisk: if you're defining normative statements by the presence of RFC2119 key words, darobin's work has you covered.
16:31:34 <krisk> tobie: consider this a small step towards progress
16:32:27 <darobin> mdyck1: yes, URLs change over time, but the idea is to have a cron job run the analysis on a regular basis
16:32:36 <krisk> tobie: These intial sections should not be contraversal, though given the number of people we should also be able to say that a section really needs tests
16:33:20 <tobie> krisk: right. I think there's a lot of value in getting the subjective analysis out.
16:33:39 <mdyck1> darobin: good to hear. it'd be nice to see when the cron ran, then. (Gives the reader confidence that the info is up-to-date.)
16:34:09 <krisk> I think we do need a test for the second normative statement
16:34:30 <tobie> krisk: existing interop, etc.
16:34:57 <krisk> Moving on to the last section for today 2.1.6
16:35:17 <darobin> mdyck1: I think that the response to that will be something like "10 minutes ago", or perhaps "1 hour ago", not more
16:36:08 <tobie> mdyck1: we're planning for "uptodateness" to be a given. Not something people have to wonder about.
16:36:16 <tobie> s/panning/aiming/
16:36:52 <darobin> hence the using a cron job rather than having people run it
16:37:00 <darobin> make them machines work hard for us
16:38:01 <krisk> This section indeed has normative requirements
16:38:04 <krisk> 'a pair of code units consisting of a high surrogate followed by a low surrogate must be treated as the single code point represented by the surrogate pair, but isolated surrogates must each be treated as the single code point with the value of the surrogate.'
16:38:08 <Bin> So we don't have to be a machine :)
16:38:21 <mdyck1> re 2.1.5: not sure how you'd test that (e.g.) text/plain *doesn
16:38:37 <mdyck1> doesn't have a registered plugin
16:39:17 <mdyck1> but that's probably not our present concern
16:39:19 <krisk> mdyck1: we can also document that a normative section is actually not test able
16:40:08 <mdyck1> that would be useful.
16:40:39 <krisk> Indeed we should have tests for 'surrogate' pairs
16:40:50 <krisk> ..section 2.1.6
16:41:17 <krisk> That is all I had for todays agenda
16:41:36 <krisk> I'll send out the notes and start to document this on the wiki
16:41:48 <mdyck1> so if we document these things, how do we ensure that the documentation tracks changes in the spec? Or are we just focused on 5.1, and we'll worry about 5.2 if/when it happens?
16:42:38 <Bin> Reading " When a conformance requirement is defined in terms of characters or Unicode code points,..." in 2.1.6
16:42:57 <mdyck1> actually, 5.1 is only a WD, so it'll change anyway
16:43:20 <Bin> Is there a conformance requirement related to characters or Unicode defined in Section 2.2 Conformance Requirement?
16:43:33 <mdyck1> So skip the "Or are we ..." sentence.
16:44:30 <krisk> We can use http://www.w3.org/TR/html5/
16:45:35 <mdyck1> which version does that test suite supposedly target?
16:45:46 <mdyck1> s/that/the/
16:45:55 <krisk> Bin: I was not planning on covering 2.2 - though this is the next section to be targeted
16:46:20 <krisk> the test suite is a deliverable for the HTML5 Spec, it's part of the w3c process
16:47:49 <krisk> I'd expect that a large part of the HTML5 test suite applies to the the HTMl5.1, HTML5.2,etc...
16:49:00 <Bin> krisk: I see. We can cover 2.2. next time, with a note that we may have to revisit the conformance requirement indicated in 2.1.6, and see if there is a duplicate
16:49:25 <mdyck1> mdyck1 has joined #HTMLT
16:49:29 <mdyck1> well, we should presumably be analyzing the same spec as the test suite targets
16:50:40 <jgraham> The testsuite doesn't target a version of the spec. It should target the latest state of the spec
16:51:02 <jgraham> In some cases tests may be backported to the lawyer copy of the spec
16:51:10 <jgraham> But really that's missing the point
16:52:03 <mdyck1> the point being?
16:52:25 <jgraham> Interop
16:53:25 <jgraham> If the spec changed on some point it's not useful to write tests to some old version just because someone happens to be publishing a static copy of that version of the spec for IPR purposes
16:53:30 <krisk> I'm open to look at HTML5 and the current 5.1 WD at the same time
16:54:34 <krisk> since I expect a number of tests will help both the HTML5 and 5.1 WD specs
16:54:36 <mdyck1> mdyck1 has joined #HTMLT
16:54:54 <mdyck1> but the only tool we have to measure interop is the test suite
16:55:03 <mdyck1> isn't it?
16:55:05 <Bin> If HTML5.0 is the one that is being implemented on market, and expected on various browsers, we should focus on testing HTML5.0
16:55:28 <jgraham> It isn't
16:55:29 <Bin> to meet the market need
16:55:37 <krisk> This is a classic problem...
16:55:42 <jgraham> That's the whole point
16:55:56 <krisk> Indeed it's very valueable to have both
16:56:29 <jgraham> If HTML5.0 were being implemented then HTML.current wouldn't have any changes (except new features) from HTML5.0
16:56:49 <jgraham> To the extent that there are differences you should always target the latest version of the spec
16:57:11 <jgraham> (you may choose to ignore *features* that aren't in some older version of course)
16:57:25 <jgraham> (but how you prioritise your time is your own issue)
16:57:45 <jgraham> (and different people will have different priorities/needs there)
16:57:51 <krisk> We are not going to close on this discussion and the meeting is scheduled to end in 3 minutes
16:58:11 <Bin> I am not saying we don't consider 5.1 WD :)
16:58:42 <tobie> In practice, is this really much of an issue?
16:59:01 <tobie> Isn't 5.1 mostly new features?
16:59:31 <krisk> The outcome that will have the most consensus is to just document both, so it's clear when a test is for 5.1 vs 5
16:59:35 <tobie> In which case, ins't 5.0 just 5.1 minus a bunch of tests, identifiable by URL?
17:00:27 <Bin> If that is the case, it is great
17:00:28 <krisk> in theory this should not be hard to accomplish
17:01:03 <krisk> Let's adjourn!
17:01:12 <Bin> We can work on 5.0 first, because it is stable
17:01:36 <krisk> yes, but we'll always be open for tests for 5.1 :)
17:01:45 <Bin> Once we finish 5.0, and 5.1 becomes more stable, we work on 5.1 (the delta part)
17:01:54 <mdyck1> do tests identify that they're 5.1-specific?
17:02:00 <jgraham> I don't know who "we" is
17:02:22 <mdyck1> s/that/whether/
17:02:33 <jgraham> But I expect organisations developing new features to write tests and not care about spec-land politics
17:02:50 <jgraham> mdyck1: Seperate git branches
17:03:34 <mdyck1> ah, ok
17:03:34 <krisk> RRSAgent, make logs public
17:04:17 <krisk> rrsagent, generate minutes
17:04:17 <RRSAgent> I have made the request to generate http://www.w3.org/2013/03/26-htmlt-minutes.html krisk
17:06:17 <mdyck1> tobie: so your earlier question amounts to whether the 5.0->5.1 diff is just the addition of new sections
17:06:38 <jgraham> There is no possible way that will be true
17:06:49 <jgraham> For example
17:06:52 <mdyck1> that's kinda what i suspected
17:07:13 <jgraham> If we add the new appcache stuff that people are discussing, that will change the navigation algorithm
17:07:28 <Ms2ger> Ms2ger has joined #HTMLT
17:07:43 <jgraham> Web Components changes the parser
17:07:46 <jgraham> etc.
17:10:48 <mdyck1> jgraham: earlier you said that the test suite isn't the only tool we have to measure interop, but I'm not sure you said what other tool we have
17:13:34 <jgraham> Earlier when?
17:14:38 <mdyck2> mdyck2 has joined #htmlt
17:15:33 <mdyck1> mdyck1 has joined #HTMLT
17:15:42 <jgraham> 17:13 < jgraham> Earlier when?
17:16:00 <jgraham> (today? two weeks ago? some other time?)
17:16:25 <mdyck1> ah, sorry, i think you were replying to someone else
17:17:36 <jgraham> But in general there *are* other ways to get an idea of interop. I mean people have a felling about more vs less interop today and there mostly isn't a test suite, and certainly no one uses it to gauge interop
17:18:01 <mdyck1> yes, makes more sense if I assume you were replying to Bin (22 min ago)
17:18:15 <jgraham> A testsuite is just *better* than the existing methods (but still a long way from perfect)
17:18:53 <jgraham> Right, I was replying to Bin. Sorry for the confusion
17:19:48 <mdyck1> minutes are at http://www.w3.org/2013/03/26-htmlt-minutes.html
17:21:33 <mdyck1> hm, presentation of minutes is somewhat odd
17:22:57 <Ms2ger> Yeah, it's made for one person minuting the telcon
17:23:47 <mdyck1> ah, that explains it somewhat.
17:24:14 <mdyck1> how does RRSAgent decide what gets indented?
17:25:09 <Ms2ger> Looks like it thought krisk was scribing
17:25:25 <Ms2ger> http://www.w3.org/2013/03/26-htmlt-irc is probably more useful
17:26:02 <mdyck1> oh, i guess everything from non-krisk is indented
17:47:34 <mdyck1> mdyck1 has left #HTMLT
18:18:08 <krisk> IRC link seem more useful, the minutes can be useful if you start to use a bunch of keywords..
18:18:31 <krisk> Chair, scribe, speaker queue, action, bug, etc...
19:29:04 <darobin> darobin has joined #htmlt