See also: IRC log
about EARL Schema doc
section 1: use cases to guide
section 1.1: to guide
section 1.3: to guide
leave examples in Schema doc
section 2.8: short example with earl:uri
about EARL Guide doc
section 1: add basic information on RDF
section 5: reference technologies, libraries that process/parse/query RDF in different languages
DR: how do tools output EARL?
SAZ: markup validator produces RDF/XML with CSS for persons to look at; some tools write on files, some tools give EARL API to internal database
section 6: update to location pointer; move examples to their own section in appendix
<JibberJim> scribenick: JibberJim
[Introductions]
SZ: So we introduce EARL, old draft that we
picked up again recently, and looked around for stuff that had been invented
elsewhere and kicked them out, split the spec into 2 - a schema for
developers, and a guide for how people to use it. Hope to take the next WD to
LC, have a couple of issues on the spec remaining, the guide is less
contentious of course.
... We originally just had a URI for content, then we needed to expand this
with httpRequest/Response to define what was tested more. So we have an HTTP
in RDF Note We took this out of EARL to be more re-usable.
KD: When do you expect to have it published?
SZ: Hopefully a publication by end of march, at the moment it just needs editorial clean-up, if you're interested in commenting we can send ti to you in advnace
OT: I would like to review?
KD: Does the HTTP in RDF note contain use cases etc.?
SZ: ... Currently we've kept a very low profile
just defined it because we need it, very little else, the introduction refers
to some uses elsewhere...
... The EARL guide describes using the HTTP namespace.
JK: Refresh header from HTML META isn't defined anywhere, how do we deal with these important but unknown message headers, with just name/value pairs for RFC 822 messages, probably as another very small WG note.
JL: Could we not role that into the larger note?
JK: I don't want to merge it in together as it's not really related.
SZ: So we have a problem with wanting to modularise everything but still keep high quality.
OT: Why not make RFC822 your main note, and then have the HTTP just as an example of that.
JK: There was strong push in the group for dedicated properties that are well defined, which was the motivation for the http.
SZ: So an issue we have before last call.
... What is a test?
... There is a test requirement - something you test for, and a test which is
what you actually do.
KD: Is what you call a requirement a test assertion?
PC: So in the QA world we have a different view of the word assertion, in the QA world the thing that is tested is an _Assertion_ a differing view.
SZ: A Requirement is the overreaching aim -
"does it meet WCAG 1.0", and to do that you have lots of individual tests.
... So there two different types of tests we wish to report about.
FB: So in the Test
... So in the Test FAQ we discuss test and test suites.
PC: We would use TestAssertion where you use
TestRequirement, you can have very high level test requirements e.g. "this
parser shall parse XML" or low level "given 2 numbers the program will return
the addition or throw an exception".
... A TestAssertion will contain 1 or more testcase.
... I think changing the name assertion would clear up a lot of confusion
... Could someone suggest other names?
JL: I guess it could be a claim
FB: That also has problems
KB: Would the ER people like a review?
PC: So labels for classes are very flexible,
ensuring the model is correct is more important.
... Associated with each requirement there should be tests and test cases.
... You ought to after you've identified the test requirements to meet your
spec, you should then work out the number and nature of tests required. Not
all will necessarily be completed.
... There may not be a requirement with a test case.
JL: So we are mostly interested in reporting, and are happy with both requirements/tests that are open ended, the requirement/test case details are down to the test description language.
KD: So Dom has an action to create an RDF vocab for tests which may be used in MWI, and there are some people in Oasis also doing this.
SZ: THe fraunhoffer have also produced a vocab for test suites.
KD: A problem we have is WG's have very different cultures and want different things from their test suite description languages.
SZ: As in Earl we just reporting results it's right to just have one testRequirement which is currently open ended, so when people have a test description language, or just use rdf description.
PC: That should probably be kept outside.
SZ: We needed something and we would like to point to.
PC: So it may motivate Dom to get something out and he's likely to produce something in the next 2 weeks.
SZ: I don't recommend reviewing now, but in 2 weeks we should publish a new draft.
FB: Is there any co-ordination with WCAG 2.0 people?
SZ: There's a lot of coordination going on.
FB: Is there any test framework planned?
SZ: Not currently, EARL would be used mostly used to report results, not describe testableAssertions.
PC: Why is there such a focus on Who made the test?
SZ: There are big use cases in Access testing, where who or what said something is really important, as you don't know if you trust people.
JL: Do you have a problem with the word Assertor as well as Assertion?
PC: Certainly there is less problem with Assertor than assertion.
SZ: So EARL is going to depend on FOAF, the
first W3 Group to use a non-standard dependencies.
... We rely on DublinCore date, and various other DC properties.
PC: We also point to outside metadata things, re-using is good.
SZ: We depend on FOAF for people organisations and things, so we've brought libby here to lock her in make sure the things we depend on are stable.
LB: Agent and Person are very stable, they've
been well discussed in the list, and have been widely used and consistent for
a long time.
... We in the FOAF group have never had a wider dependency from external
pressures so a discussion on what it means.
... We'd also like input on what to do with the spec - take it to the W3 as a
note, go to IETF, seperate out stable and unstable.
SZ: I think it's good it's not a W3c note,
encouraging outside use is good to show off using different vocabularies.
... I would like the document split and versioned so we can point to
things.
LB: There was a big problem using DC during the
period when DC went from 1.0 to 1.1 it was very disruptive for the
community.
... So we went to the property level of testing/unstable etc.
SZ: Snapshots would be nice so we can reference something, to stop someone in 3 years from looking at something different.
KD: Is the FOAF spec going to be available always?
SZ: There are two issues with stability, both of the spec and uri's and the terms defined in them.
LB: I will send pointer to the stability of uri's, and update the specification with stable, and discuss with danbri splitting it out more.
KD: do you expect to have implementations?
SZ: Yes we expect to have lots, we have many tools outputting EARL already, fewer ones consuming it.
PC: Is it all in WAI area?
SZ: Mostly, but also QA.
JK: there's a Web Compliance Manager which has a standalone version and as a CMS, it's not only an AT tool, but a rule engine supporting lots of things.
SZ: Another thing for Libby, foaf:Agent is there anything about software?
KD: There is DOAP which is closely related.
SZ: all we require on software is a title
KC: A version would really be useful.
KD: I think you should try and push it outside, perhaps to Edd inside in DOAP.
LB: Edd is quite serious about DOAP so referencing it would be good.
SZ: So we've talked about DOAP before.
<scribe> ACTION: JL to talk to DOAP about taking our software outside of us. [recorded in http://www.w3.org/2006/02/28-er-minutes.html#action01]
JL: When will QA tools produce EARL?
OT: I asked Nick to have a look at the EARL of
the output, and he said yes.
... So the problem with the mark-up validator is infrequent releases, and the
last one was last week.
... The other problem is we're in the middle of a new generation, and most
releases are security or performance.
JL: It should just be a template change I believe so should be quite simple and safe to change.
OT: No perl needed, just template.
... EARL and XML are poorly documented and unofficial.
... I am pushing to get the SOAP output better documented and there's some
resistance.
SZ: Let's see if we can get students to do it.
LB: Can I suggest that you arrange a meeting on #swig on IRC to comment on schemas especially things like the HTTP in RDF.
SZ: Yes I'd like to do that.
... Lunch, and thanks to the QA people for visiting, and to Libby to come
along!
<drooks> resuming after lunch...
<drooks> talking about test resquirement and test case... again...
<drooks> CI: carlos has proposed a model whereby the test case points to the requirement
<drooks> CI: as opposed to the requirement pointing to requirement
<drooks> CI: this way a test case can be associated with multiple requirements
<drooks> alfred gilman joins the wg
<shadi> scribe: drooks
<chaals> [Wow! made it online]
yes there is
<JibberJim> we're discussing tests a bit with Al chaals
shadi: conclusion: we have 1 pointer form the assertion to a test thingy and 0 or more pointers from an assertion to other assertions
jim: leave assertion relationships implied
... we need to be able to extract high level and lower level results
shadi: question; is thingyoucanpassorfail class with a subclassof test requirement and test case an acceptable solution?
<chaals> [works for me]
general concensus is that its a good idea
<JibberJim> chaals, are you still happy to have the action item to write up the thing you can pass or fail and subClass ?
<chaals> yeah.
shadi: question: what are views on test method and test evidence? are we just filling a hole? is it going to be done for us by a test language?
<chaals> [don't know that it will be done by a test language either (although it could) but don't see it as necessary for last call]
thanks chaals
<JibberJim> Discussion of pointing from one Assertion back to another assertion.
<JibberJim> A poor mans evidence
<chaals> [either do a proper one, or leave it alone...]
<chaals> [suggest leaving it alone]
<JibberJim> DR draws picture of WCAG 1.1 A WCAG 1.1 AA and some requirements and tests.
<JibberJim> wants to be able to point to the FAILED test in the assertion to "explain" why it fails
<JibberJim> JL: suggests you can use dc:hasPort etc. if you really want to expand things beyond implied relationships or in the test description language
<JibberJim> JK: The connection could be modelled in the test situation.
<JibberJim> SZ: In the current EARL we've done it with testcases already.
<JibberJim> SZ: Proposal drop EARL:evidence/methodology.
<chaals> [worky for me]
<JibberJim> RESOLUTION: drop EARL:evidence/methodology