ERT WG face-to-face, Day 2

28 Feb 2006


See also: IRC log


Chaals, Jim, Johannes, CarlosI, David, Karima (observer), Dirk (observer), Shadi
Chirs, Sandor, CarlosV, Nick, Charles
Johannes, David


about EARL Schema doc

section 1: use cases to guide

section 1.1: to guide

section 1.3: to guide

leave examples in Schema doc

section 2.8: short example with earl:uri

about EARL Guide doc

section 1: add basic information on RDF

section 5: reference technologies, libraries that process/parse/query RDF in different languages

DR: how do tools output EARL?

SAZ: markup validator produces RDF/XML with CSS for persons to look at; some tools write on files, some tools give EARL API to internal database

section 6: update to location pointer; move examples to their own section in appendix

<JibberJim> scribenick: JibberJim

ER gets a visit from QA IG!


SZ: So we introduce EARL, old draft that we picked up again recently, and looked around for stuff that had been invented elsewhere and kicked them out, split the spec into 2 - a schema for developers, and a guide for how people to use it. Hope to take the next WD to LC, have a couple of issues on the spec remaining, the guide is less contentious of course.
... We originally just had a URI for content, then we needed to expand this with httpRequest/Response to define what was tested more. So we have an HTTP in RDF Note We took this out of EARL to be more re-usable.

KD: When do you expect to have it published?

SZ: Hopefully a publication by end of march, at the moment it just needs editorial clean-up, if you're interested in commenting we can send ti to you in advnace

OT: I would like to review?

KD: Does the HTTP in RDF note contain use cases etc.?

SZ: ... Currently we've kept a very low profile just defined it because we need it, very little else, the introduction refers to some uses elsewhere...
... The EARL guide describes using the HTTP namespace.

JK: Refresh header from HTML META isn't defined anywhere, how do we deal with these important but unknown message headers, with just name/value pairs for RFC 822 messages, probably as another very small WG note.

JL: Could we not role that into the larger note?

JK: I don't want to merge it in together as it's not really related.

SZ: So we have a problem with wanting to modularise everything but still keep high quality.

OT: Why not make RFC822 your main note, and then have the HTTP just as an example of that.

JK: There was strong push in the group for dedicated properties that are well defined, which was the motivation for the http.

SZ: So an issue we have before last call.
... What is a test?
... There is a test requirement - something you test for, and a test which is what you actually do.

KD: Is what you call a requirement a test assertion?

PC: So in the QA world we have a different view of the word assertion, in the QA world the thing that is tested is an _Assertion_ a differing view.

SZ: A Requirement is the overreaching aim - "does it meet WCAG 1.0", and to do that you have lots of individual tests.
... So there two different types of tests we wish to report about.

FB: So in the Test
... So in the Test FAQ we discuss test and test suites.

PC: We would use TestAssertion where you use TestRequirement, you can have very high level test requirements e.g. "this parser shall parse XML" or low level "given 2 numbers the program will return the addition or throw an exception".
... A TestAssertion will contain 1 or more testcase.
... I think changing the name assertion would clear up a lot of confusion
... Could someone suggest other names?

JL: I guess it could be a claim

FB: That also has problems

KB: Would the ER people like a review?

PC: So labels for classes are very flexible, ensuring the model is correct is more important.
... Associated with each requirement there should be tests and test cases.
... You ought to after you've identified the test requirements to meet your spec, you should then work out the number and nature of tests required. Not all will necessarily be completed.
... There may not be a requirement with a test case.

JL: So we are mostly interested in reporting, and are happy with both requirements/tests that are open ended, the requirement/test case details are down to the test description language.

KD: So Dom has an action to create an RDF vocab for tests which may be used in MWI, and there are some people in Oasis also doing this.

SZ: THe fraunhoffer have also produced a vocab for test suites.

KD: A problem we have is WG's have very different cultures and want different things from their test suite description languages.

SZ: As in Earl we just reporting results it's right to just have one testRequirement which is currently open ended, so when people have a test description language, or just use rdf description.

PC: That should probably be kept outside.

SZ: We needed something and we would like to point to.

PC: So it may motivate Dom to get something out and he's likely to produce something in the next 2 weeks.

SZ: I don't recommend reviewing now, but in 2 weeks we should publish a new draft.

FB: Is there any co-ordination with WCAG 2.0 people?

SZ: There's a lot of coordination going on.

FB: Is there any test framework planned?

SZ: Not currently, EARL would be used mostly used to report results, not describe testableAssertions.

PC: Why is there such a focus on Who made the test?

SZ: There are big use cases in Access testing, where who or what said something is really important, as you don't know if you trust people.

JL: Do you have a problem with the word Assertor as well as Assertion?

PC: Certainly there is less problem with Assertor than assertion.

SZ: So EARL is going to depend on FOAF, the first W3 Group to use a non-standard dependencies.
... We rely on DublinCore date, and various other DC properties.

PC: We also point to outside metadata things, re-using is good.

SZ: We depend on FOAF for people organisations and things, so we've brought libby here to lock her in make sure the things we depend on are stable.

LB: Agent and Person are very stable, they've been well discussed in the list, and have been widely used and consistent for a long time.
... We in the FOAF group have never had a wider dependency from external pressures so a discussion on what it means.
... We'd also like input on what to do with the spec - take it to the W3 as a note, go to IETF, seperate out stable and unstable.

SZ: I think it's good it's not a W3c note, encouraging outside use is good to show off using different vocabularies.
... I would like the document split and versioned so we can point to things.

LB: There was a big problem using DC during the period when DC went from 1.0 to 1.1 it was very disruptive for the community.
... So we went to the property level of testing/unstable etc.

SZ: Snapshots would be nice so we can reference something, to stop someone in 3 years from looking at something different.

KD: Is the FOAF spec going to be available always?

SZ: There are two issues with stability, both of the spec and uri's and the terms defined in them.

LB: I will send pointer to the stability of uri's, and update the specification with stable, and discuss with danbri splitting it out more.

KD: do you expect to have implementations?

SZ: Yes we expect to have lots, we have many tools outputting EARL already, fewer ones consuming it.

PC: Is it all in WAI area?

SZ: Mostly, but also QA.

JK: there's a Web Compliance Manager which has a standalone version and as a CMS, it's not only an AT tool, but a rule engine supporting lots of things.

SZ: Another thing for Libby, foaf:Agent is there anything about software?

KD: There is DOAP which is closely related.

SZ: all we require on software is a title

KC: A version would really be useful.

KD: I think you should try and push it outside, perhaps to Edd inside in DOAP.

LB: Edd is quite serious about DOAP so referencing it would be good.

SZ: So we've talked about DOAP before.

<scribe> ACTION: JL to talk to DOAP about taking our software outside of us. [recorded in http://www.w3.org/2006/02/28-er-minutes.html#action01]

JL: When will QA tools produce EARL?

OT: I asked Nick to have a look at the EARL of the output, and he said yes.
... So the problem with the mark-up validator is infrequent releases, and the last one was last week.
... The other problem is we're in the middle of a new generation, and most releases are security or performance.

JL: It should just be a template change I believe so should be quite simple and safe to change.

OT: No perl needed, just template.
... EARL and XML are poorly documented and unofficial.
... I am pushing to get the SOAP output better documented and there's some resistance.

SZ: Let's see if we can get students to do it.

LB: Can I suggest that you arrange a meeting on #swig on IRC to comment on schemas especially things like the HTTP in RDF.

SZ: Yes I'd like to do that.
... Lunch, and thanks to the QA people for visiting, and to Libby to come along!

<drooks> resuming after lunch...

<drooks> talking about test resquirement and test case... again...

<drooks> CI: carlos has proposed a model whereby the test case points to the requirement

<drooks> CI: as opposed to the requirement pointing to requirement

<drooks> CI: this way a test case can be associated with multiple requirements

<drooks> alfred gilman joins the wg

<shadi> scribe: drooks

<chaals> [Wow! made it online]

yes there is

<JibberJim> we're discussing tests a bit with Al chaals

shadi: conclusion: we have 1 pointer form the assertion to a test thingy and 0 or more pointers from an assertion to other assertions

jim: leave assertion relationships implied
... we need to be able to extract high level and lower level results

shadi: question; is thingyoucanpassorfail class with a subclassof test requirement and test case an acceptable solution?

<chaals> [works for me]

general concensus is that its a good idea

<JibberJim> chaals, are you still happy to have the action item to write up the thing you can pass or fail and subClass ?

<chaals> yeah.

shadi: question: what are views on test method and test evidence? are we just filling a hole? is it going to be done for us by a test language?

<chaals> [don't know that it will be done by a test language either (although it could) but don't see it as necessary for last call]

thanks chaals

<JibberJim> Discussion of pointing from one Assertion back to another assertion.

<JibberJim> A poor mans evidence

<chaals> [either do a proper one, or leave it alone...]

<chaals> [suggest leaving it alone]

<JibberJim> DR draws picture of WCAG 1.1 A WCAG 1.1 AA and some requirements and tests.

<JibberJim> wants to be able to point to the FAILED test in the assertion to "explain" why it fails

<JibberJim> JL: suggests you can use dc:hasPort etc. if you really want to expand things beyond implied relationships or in the test description language

<JibberJim> JK: The connection could be modelled in the test situation.

<JibberJim> SZ: In the current EARL we've done it with testcases already.

<JibberJim> SZ: Proposal drop EARL:evidence/methodology.

<chaals> [worky for me]

<JibberJim> RESOLUTION: drop EARL:evidence/methodology

<shadi> http://www.w3.org/WAI/EO/Drafts/eval/tools/advanced

Summary of Action Items

[NEW] ACTION: JL to talk to DOAP about taking our software outside of us. [recorded in http://www.w3.org/2006/02/28-er-minutes.html#action01]
[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.127 (CVS log)
$Date: 2006/03/02 10:42:57 $