29 Mar 2005



Gabriele, Shadi, ChrisR, Chrisoula, Wendy
Myriam, Charles, CarlosI, Johannes, Sandor, Jim


Describing the Location of Test-Subjects

<shadi> http://lists.w3.org/Archives/Public/public-wai-ert/2005Mar/att-0062/errors.html

saz: perhaps this can serve as background to discussion.
... comments?
... the first table, "context-free errors" these don't depend on something else.
... e.g., missing attribute

cr: this is to achieve persistence if the pg has been modified?

saz: wanted to separate persistence and locating, but they seem to keep tying together.

cr: think xpath will be fine if we don't care about persistence.
... assuming that the pg is going to be valid markup.

saz: specifically didn't specify xpath.
... xpath will need a well-formed document.
... can we assume that? if you don't have a valid pg, all we can say is, "you have a problem." if you do have valid pg, then we can locate problem with more granularity.
... the better the page, the better results you'll get w/earl.
... there is a notion of persistence. chrisoula's example had <a..><p>..</p></a> which doesn't validate.
... then another example w/something else in a, such that it changed. not persistent.
... later, have context errors.
... 1st example is the one that chrisoula raised.
... if p inside a, have some persistency. although, it's delicate. we could build a layer on top, e.g., hashing.
... does this build a layer to describe the location? could we build a more sophistacted persistency layer on top of this?

cr: this is a general scheme, and afraid that soemthing general will not work.
... feel that we need to be specific about accessibility errors.
... many of these errors can be caught by a validator. are they accessibiltiy problems?

gabriele: agree w/chris
... should consider persistence. have to think in more generic way that includes persistency. otherwise, have problems later.
... don't think there is a unique way to locate a problem. perhaps use different locators. one is xpath, but have to assume document is well-formed.
... with other mechanisms, can achieve the results of locating problem. other ways: line numbers, character numbers, id attribute.

chrisoula: agree. think xpath in combination with id can provide persistence. was trying to suggeest (in email), that if document is not well-formed, perhaps we don't care about persistency.
... then could have a simple way to identify, such as line numbers.
... if documents are well-formed, then use xpath and that is sufficient for persistency.
... don't think line numbers should be used in general.

saz: agree that we need different mechanisms to locate the problem. let's say we find the problem and the document is well-formed, how do we generate an xpath?
... like we have different strategies for locating the problem, we have different strategies for describing the problem.
... e.g., "contenxt-sensitive error" and can describe the xpath in one way.
... there are more generic patterns we can use to describe the error.
... even if all of the tags have ids (the best case), do we point to id of <a> or <p> element?
... agree that this is a general approach to validation errors, however believe that many accessibility errors will be context-sensitive and will depend on other elements.
... if there is an id, we can use it. but, think we need this fallback: if no id, not well-formed...if we can't do anything, we can give you a line number.

gabriele: i like the idea of patterns.
... if we find other patterns, we can add to a dictionary.

saz: that is what I was trying to get at. context element. have siblings that are causing an error. it's an element w/in an element that is not allowed.
... for that we can use a pattern for how to describe such errors.
... "any p's inside any a's"
... need to take accessibility errors and determine if they can be sorted into such categories of errors and described in more generic ways.
... we might find exceptions.
... believe chris was saying that we need to ground the idea in real accessibility examples.

cr: need to figure out what exactly we'll be storing in earl. will we be recording these thigns

cr: say that you have an img that doesn't have alt attribute.
... that can be detected. do we need to record where this occurs? don't think so. we can find it again.
... "there is an img that is missing an alt attribute" - you can fidn it no matter where it is.
... just need to say that it's there.
... just point to the pg and say "it's an error"
... what we do need to record are those things that need to be found again.
... e.g., "this img needs a long description"

chrisoula: in some cases, need to know the location of the error. e.g., if the report and page go into a repair tool. need for long pages.

saz: I didn't other at the beginning, b/c thought he was saying "for all errors" but he's saying, "only errors that can't be found automatically again"
... although, may want to specifically say, "this image" and then send that to a repair tool.
... are you saying an absolute "no?"

chrisoula: when reporting, you don't just say, "10 errors and here they are" a machine has to be able to use the report to fix errors.
... should be more close to the user scenarios.

gabriele: agree that chris that some errors can be found anyway. important in some cases to locate an error in a more human way.
... should give users more opportunities to find errors.

saz: don't have an example that convinces me that there is a case where you don't need to locate errors. maybe we need two ways of locating or two outputs.
... one is machine-readable (e.g., xpath) the other is line numbers (additional help for humans). unfortunately, line numbers break easily.
... and, do we point to line number for a or p. no persistency. it's different on different tools and systems.
... tools that understand xpath could generate line numbers automatically


saz: then if the page changes, can still find the problem. don't need to hard code line numbers into earl reports.

chrisoula: something more simple than xpath will not be as flexible.


gabriele: if we provide on-line services to give users chance to locate via line number, there is no need to store line number in the earl report if a service is provided that will generate line numbers for people.
... or we state that every tool must provide that info.

saz: what's missing is a tool that shows the full power and potential of earl.
... we have implementations that do partial things (query), but they output rdf which is not for public consumption. w/out robust earl (language), then no tool will be developed on top of that. if no tool provides, then can only talk about such tools that may or not exist one day
... think xpath parsers are common and most programming languages have at least one implementation. should be simple for a tool developer to generate line numbers from xpath expression.

<ChrisR> http://checker.atrc.utoronto.ca uses EARL - reads and writes it

http://xml.apache.org/xalan-c/apiDocs/classKeyDeclaration.html#a8 getLineNumber()

saz: think there are a few earl "dialects" and we're trying to find an approach that works for all of us.
... didn't meant that there are no tools that use earl, but that there is more to do with generating reports and queries and the full potential of earl is not widely deployed. seem to be more tools to generate earl than consume it.

cr: based on test suite. create examples. send tomorrow.

saz: this has been an excellent discussion. feel clarity is on the near horizon.

Conformance to EARL

saz: to demonstrate support and conformance to a W3C specification, need to show two independent, interoperable implementations of every feature of the spec.
... test suites would help us debug our own earl implementations. tool developers could use test cases to determine if they use earl correctly.
... could also test measurability of a feature, e.g., is a feature really implemented.
... could take the wcag test files, although they are not done yet.
... chris, do you think this is feasible or adding dependencies or how best do?

cr: think that's the right way to go. think the reporting has to be based on accessibility tests.
... started out w/200 tests, have gotten through about 100. hope to get through next 100 in few months.

saz: think they are nicely categorized as those that are confirmed, rejected, still to be confirmed, etc.
... think we can begin working through confirmed. we'll need resources to start generating EARL for each test case.

wac: need a tool to run through the test cases and generate earl. isn't that what cr said his tool will do?

cr: yes, exactly. that's what i will begin tomorrow.
... in the program, take all of the test files and the program generates the earl. then read in the earl and determine if the program detected the appropriate error.
... use it as a self-test.

saz: that sounds totally neat.

wac: chris rocks.

gabriele: agree
... you have input files for the test?

cr: yes

gabriele: it's not hard-coded...you can change the test by changing the input.

cr: right. it generates it on the fly. as we generate new tests, it uses them as well.

gabriele: what syntax have you used?

cr: the test files are html.

gabriele: look forward to your posting to the list.

saz: how are ppl doing w/wcag test suite? have ppl looked at them?

gabriele: it's similar to the open accessibility checks that chris and UT produced a few years ago.

cr: it's the same thing.

saz: the limitation is that it is only html.

gabriele: the risk is to build something too large. need to keep in mind generic idea, but try to produce something by using a specific case.
... if go for too big or too generic, risk not producing anything.

saz: we've been focusing so much on html, worry that our model will break when go to other technologies, such as css.

wac: believe css tests are close to happening. good to start with html and evolve out to other techs. if model breaks, fix it, but good to start somewhere.

gabriele: mechanism to translate css into xpath or use fuzzy pointers, could be more generic.

wac: also meant to say that in order to pass many of the css tests, must have already passed some html tests. e.g., visual design in css assumed good structural html.
... for earl, this will push the "heuristic" and "combination" aspects of the language. ie.., how arrived at a decision.

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.119 (CVS log)
$Date: Tuesday 29 March 2005 - 20:44:49