07:51:47 <RRSAgent> RRSAgent has joined #er
07:51:47 <RRSAgent> logging to http://www.w3.org/2006/02/28-er-irc
07:52:25 <shadi> meeting: ERT WG face-to-face, Day 2
07:56:14 <JohannesK> JohannesK has joined #er
07:56:36 <shadi> regrets: Chirs, Sandor, CarlosV, Nick
07:56:50 <shadi> present: Chaals, Jim, Johannes, CarlosI, David, Karima (observer), Dirk (observer), Shadi
07:57:24 <shadi> agenda: http://www.w3.org/WAI/ER/2006/02/f2f-agenda
08:21:52 <chaals> chaals has joined #er
09:17:05 <shadi> shadi has joined #er
09:20:26 <JohannesK> JohannesK has joined #er
09:20:43 <JohannesK> scribe: JohannesK
09:20:58 <JohannesK> about EARL Schema doc
09:21:06 <JohannesK> section 1: use cases to guide
09:21:17 <JohannesK> section 1.1: to guide
09:21:24 <JohannesK> section 1.3: to guide
09:21:31 <JohannesK> leave examples in Schema doc
09:21:37 <JohannesK> section 2.8: short example with earl:uri
09:21:45 <JohannesK> about EARL Guide doc
09:21:52 <JohannesK> section 1: add basic information on RDF
09:22:04 <JohannesK> section 5: reference technologies, libraries that process/parse/query RDF in different languages
09:22:11 <JohannesK> DR: how do tools output EARL?
09:22:17 <JohannesK> SAZ: markup validator produces RDF/XML with CSS for persons to look at; some tools write on files, some tools give EARL API to internal database
09:22:22 <JohannesK> section 6: update to location pointer; move examples to their own section in appendix
09:25:46 <shadi> rrsagent, make logs world
10:09:15 <CarlosI> CarlosI has joined #er
10:12:15 <pecorra> pecorra has joined #er
10:16:45 <JibberJim> JibberJim has joined #er
10:16:57 <JibberJim> scribenick: JibberJim
10:17:51 <JibberJim> Topic: ER gets a visit from QA IG!
11:13:30 <SZ> SZ has joined #er
11:15:52 <JibberJim> scribenick: jibberjim
11:15:53 <JibberJim> [Introductions]
11:15:53 <JibberJim> SZ: So we introduce EARL, old draft that we picked up again recently, and looked around for stuff that had been invented elsewhere and kicked them out, split the spec into 2 - a schema for developers, and a guide for how people to use it.  Hope to take the next WD to LC, have a couple of issues on the spec remaining, the guide is less contentious of course.
11:16:01 <JibberJim> SZ: We originally just had a URI for content, then we needed to expand this with httpRequest/Response to define what was tested more.  So we have an HTTP in RDF Note  We took this out of EARL to be more re-usable.
11:16:02 <JibberJim> KD: When do you expect to have it published?
11:16:02 <JibberJim> SZ: Hopefully a publication by end of march, at the moment it just needs editorial clean-up, if you're interested in commenting we can send ti to you in advnace
11:16:02 <JibberJim> OT: I would like to review?
11:16:02 <JibberJim> KD: Does the HTTP in RDF note contain use cases etc.?
11:16:04 <JibberJim> SZ:
11:16:21 <JibberJim> SZ: Currently we've kept a very low profile just defined it because we need it, very little else, the introduction refers to some uses elsewhere...
11:16:21 <JibberJim> SZ: The EARL guide describes using the HTTP namespace.
11:16:21 <JibberJim> JK: Refresh header from HTML META isn't defined anywhere, how do we deal with these important but unknown message headers, with just name/value pairs for RFC 822 messages, probably as another very small WG note.
11:16:21 <JibberJim> JL: Could we not role that into the larger note?
11:16:29 <JibberJim> JK: I don't want to merge it in together as it's not really related.
11:16:30 <JibberJim> SZ: So we have a problem with wanting to modularise everything but still keep high quality.
11:16:30 <JibberJim> OT: Why not make RFC822 your main note, and then have the HTTP just as an example of that.
11:16:30 <JibberJim> JK: There was strong push in the group for dedicated properties that are well defined, which was the motivation for the http.
11:16:31 <JibberJim> SZ: So an issue we have before last call.
11:16:33 <JibberJim> SZ: What is a test?
11:16:38 <JibberJim> SZ: There is a test requirement - something you test for, and a test which is what you actually do.
11:16:39 <JibberJim> KD: Is what you call a requirement a test assertion?
11:16:39 <JibberJim> PC: So in the QA world we have a different view of the word assertion, in the QA world the thing that is tested is an _Assertion_ a differing view.
11:16:42 <JibberJim> SZ: A Requirement is the overreaching aim - "does it meet WCAG 1.0", and to do that you have lots of individual tests.
11:16:45 <JibberJim> SZ: So there two different types of tests we wish to report about.
11:16:47 <JibberJim> FB: So in the Test
11:16:54 <JibberJim> FB: So in the Test FAQ we discuss test and test suites.
11:16:55 <JibberJim> PC: We would use TestAssertion where you use TestRequirement, you can have very high level test requirements e.g. "this parser shall parse XML" or low level "given 2 numbers the program will return the addition or throw an exception".
11:16:55 <JibberJim> PC: A TestAssertion will contain 1 or more testcase.
11:16:56 <JibberJim> PC: I think changing the name assertion would clear up a lot of confusion
11:16:59 <JibberJim> PC: Could someone suggest other names?
11:17:00 <JibberJim> JL: I guess it could be a claim
11:17:02 <JibberJim> FB: That also has problems
11:17:04 <JibberJim> KB: Would the ER people like a review?
11:17:06 <JibberJim> PC: So labels for classes are very flexible, ensuring the model is correct is more important.
11:17:08 <JibberJim> PC: Associated with each requirement there should be tests and test cases.
11:17:10 <JibberJim> PC: You ought to after you've identified the test requirements to meet your spec, you should then work out the number and nature of tests required.  Not all will necessarily be completed.
11:17:13 <JibberJim> PC: There may not be a requirement with a test case.
11:17:19 <JibberJim> JL: So we are mostly interested in reporting, and are happy with both requirements/tests that are open ended, the requirement/test case details are down to the test description language.
11:17:20 <JibberJim> KD: So Dom has an action to create an RDF vocab for tests which may be used in MWI, and there are some people in Oasis also doing this.
11:17:21 <JibberJim> SZ: THe fraunhoffer have also produced a vocab for test suites.
11:17:28 <JibberJim> KD: A problem we have is WG's have very different cultures and want different things from their test suite description languages.
11:17:31 <JibberJim> SZ: As in Earl we just reporting results it's right to just have one testRequirement which is currently open ended, so when people have a test description language, or just use rdf description.
11:17:36 <JibberJim> PC: That should probably be kept outside.
11:17:37 <JibberJim> SZ: We needed something and we would like to point to.
11:17:37 <JibberJim> PC: So it may motivate Dom to get something out and he's likely to produce something in the next 2 weeks.
11:17:37 <JibberJim> SZ: I don't recommend reviewing now, but in 2 weeks we should publish a new draft.
11:17:46 <JibberJim> FB: Is there any co-ordination with WCAG 2.0 people?
11:17:48 <JibberJim> SZ: There's a lot of coordination going on.
11:17:48 <JibberJim> FB: Is there any test framework planned?
11:17:48 <JibberJim> SZ: Not currently, EARL would be used mostly used to report results, not describe testableAssertions.
11:17:54 <JibberJim> PC: Why is there such a focus on Who made the test?
11:17:55 <JibberJim> SZ: There are big use cases in Access testing, where who or what said something is really important, as you don't know if you trust people.
11:17:55 <JibberJim> JL: Do you have a problem with the word Assertor as well as Assertion?
11:17:55 <JibberJim> PC: Certainly there is less problem with Assertor than assertion.
11:18:00 <JibberJim> SZ: So EARL is going to depend on FOAF, the first W3 Group to use a non-standard dependencies.
11:18:02 <JibberJim> SZ: We rely on DublinCore date, and various other DC properties.
11:18:02 <JibberJim> PC: We also point to outside metadata things, re-using is good.
11:18:02 <JibberJim> SZ: We depend on FOAF for people organisations and things, so we've brought libby here to lock her in make sure the things we depend on are stable.
11:18:06 <JibberJim> LB: Agent and Person are very stable, they've been well discussed in the list, and have been widely used and consistent for a long time.
11:18:07 <JibberJim> LB: We in the FOAF group have never had a wider dependency from external pressures so a discussion on what it means.
11:18:09 <JibberJim> LB: We'd also like input on what to do with the spec - take it to the W3 as a note, go to IETF, seperate out stable and unstable.
11:18:13 <JibberJim> SZ: I think it's good it's not a W3c note, encouraging outside use is good to show off using different vocabularies.
11:18:15 <JibberJim> SZ: I would like the document split and versioned so we can point to things.
11:18:17 <JibberJim> LB: There was a big problem using DC during the period when DC went from 1.0 to 1.1 it was very disruptive for the community.
11:18:20 <JibberJim> LB: So we went to the property level of testing/unstable etc.
11:18:22 <JibberJim> SZ: Snapshots would be nice so we can reference something, to stop someone in 3 years from looking at something different.
11:18:25 <JibberJim> KD: Is the FOAF spec going to be available always?
11:18:27 <JibberJim> SZ: There are two issues with stability, both of the spec and uri's and the terms defined in them.
11:18:29 <JibberJim> LB: I will send pointer to the stability of uri's, and update the specification with stable, and discuss with danbri splitting it out more.
11:18:32 <JibberJim> KD: do you expect to have implementations?
11:18:34 <JibberJim> SZ: Yes we expect to have lots, we have many tools outputting EARL already, fewer ones consuming it.
11:18:36 <JibberJim> PC: Is it all in WAI area?
11:18:38 <JibberJim> SZ: Mostly, but also QA.
11:18:40 <JibberJim> JK: there's a Web Compliance Manager which has a standalone version and as a CMS, it's not only an AT tool, but a rule engine supporting lots of things.
11:18:43 <JibberJim> SZ: Another thing for Libby, foaf:Agent is there anything about software?
11:18:45 <JibberJim> KD: There is DOAP which is closely related.
11:18:47 <JibberJim> SZ: all we require on software is a title
11:18:49 <JibberJim> KC: A version would really be useful.
11:18:51 <JibberJim> KD: I think you should try and push it outside, perhaps to Edd inside in DOAP.
11:18:53 <JibberJim> LB: Edd is quite serious about DOAP so referencing it would be good.
11:18:55 <JibberJim> SZ: So we've talked about DOAP before.
11:18:57 <JibberJim> ACTION: JL to talk to DOAP about taking our software outside of us.
11:18:59 <JibberJim> JL: When will QA tools produce EARL?
11:19:01 <JibberJim> OT: I asked Nick to have a look at the EARL of the output, and he said yes.
11:19:03 <JibberJim> OT: So the problem with the mark-up validator is infrequent releases, and the last one was last week.
11:19:05 <JibberJim> OT: The other problem is we're in the middle of a new generation, and most releases are security or performance.
11:19:07 <JibberJim> JL: It should just be a template change I believe so should be quite simple and safe to change.
11:19:09 <JibberJim> OT: No perl needed, just template.
11:19:11 <JibberJim> OT: EARL and XML are poorly documented and unofficial.
11:19:13 <JibberJim> OT: I am pushing to get the SOAP output better documented and there's some resistance.
11:19:15 <JibberJim> SZ: Let's see if we can get students to do it.
11:19:17 <JibberJim> LB: Can I suggest that you arrange a meeting on #swig on IRC to comment on schemas especially things like the HTTP in RDF.
11:19:20 <JibberJim> SZ: Yes I'd like to do that.
11:19:24 <JibberJim> SZ: Lunch, and thanks to the QA people for visiting, and to Libby to come along!
13:03:23 <drooks> drooks has joined #er
13:03:56 <drooks> resuming after lunch...
13:04:18 <drooks> talking about test resquirement and test case... again...
13:05:52 <drooks> CI: carlos has proposed a model whereby the test case points to the requirement
13:06:06 <drooks> CI: as opposed to the requirement pointing to requirement
13:07:13 <drooks> CI: this way a test case can be associated with multiple requirements
13:07:32 <drooks> alfred gilman joins the wg
13:09:34 <shadi> scribe: drooks
13:19:46 <chaals> chaals has joined #er
13:20:05 <chaals> [Wow! made it online]
13:21:41 <drooks> yes there is
13:23:44 <JibberJim> we're discussing tests a bit with Al chaals
13:25:08 <drooks> shadi: conclusion: we have 1 pointer form the assertion to a test thingy and 0 or more pointers from an assertion to other assertions
13:27:52 <drooks> jim: leave assertion relationships implied
13:30:06 <drooks> jim: we need to be able to extract high level and lower level results
13:33:36 <drooks> shadi: question; is thingyoucanpassorfail class with a subclassof test requirement and test case an acceptable solution?
13:37:53 <chaals> [works for me]
13:43:01 <drooks> general concensus is that its a good idea
13:45:02 <JibberJim> chaals, are you still happy to have the action item to write up the thing you can pass or fail and subClass ?
13:46:27 <chaals> yeah.
13:51:59 <drooks> shadi: question: what are views on test method and test evidence? are we just filling a hole? is it going to be done for us by a test language?
13:52:43 <chaals> [don't know that it will be done by a test language either (although it could) but don't see it as necessary for last call]
13:53:12 <drooks> thanks chaals
13:59:44 <JibberJim> Discussion of pointing from one Assertion back to another assertion.
14:00:03 <JibberJim> A poor mans evidence
14:00:43 <chaals> [either do a proper one, or leave it alone...]
14:00:52 <chaals> [suggest leaving it alone]
14:01:27 <JibberJim> DR draws picture of WCAG 1.1 A WCAG 1.1 AA and some requirements and tests.
14:01:27 <JibberJim> wants to be able to point to the FAILED test in the assertion to "explain" why it fails
14:07:19 <CarlosI> CarlosI has joined #er
14:07:54 <JibberJim> JL: suggests you can use dc:hasPort etc. if you really want to expand things beyond implied relationships or in the test description language
14:09:10 <JibberJim> JK: The connection could be modelled in the test situation.
14:10:30 <JibberJim> SZ: In the current EARL we've done it with testcases already.
14:18:03 <JibberJim> SZ: Proposal drop EARL:evidence/methodology.
14:20:19 <chaals> [worky for me]
14:22:18 <JibberJim> RESOLUTION: drop EARL:evidence/methodology
14:24:58 <shadi> http://www.w3.org/WAI/EO/Drafts/eval/tools/advanced
14:25:13 <shadi> rrsagent, make logs world
15:41:22 <pecorra> pecorra has joined #er
16:53:19 <shadi> shadi has joined #er
16:54:08 <shadi> rrsagent, make logs world
16:54:16 <shadi> rrsagent, make minutes
16:54:16 <RRSAgent> I have made the request to generate http://www.w3.org/2006/02/28-er-minutes.html shadi
16:54:23 <shadi> rrsagent, bye
16:54:23 <RRSAgent> I see 1 open action item saved in http://www.w3.org/2006/02/28-er-actions.rdf :
16:54:23 <RRSAgent> ACTION: JL to talk to DOAP about taking our software outside of us. [1]
16:54:23 <RRSAgent>   recorded in http://www.w3.org/2006/02/28-er-irc#T11-18-57