See also: IRC log
simon: let's talk about te web accessibilitymetrics item in the agenda
<sharper> http://www.w3.org/WAI/RD/wiki/Benchmarking_Web_Accessibility_Metrics
ok, thanks
markel: wiki was updated as per actions last
week.
... metrics=way to measure something.
<Joshue> am looking at it now
no i did not.
markel: accessibility wrt conformance is one
thing
... accessibility in use is another one
mv: up to now most metrics are based on results produced by tools
<Joshue> Is SAMBA a tool? have you a URI?
samba is amethod. I have a paper; see from www.dimi.uniud.it/giorgio/publications.html
mv: there are many diverse challenges concerned with metrics.
<Joshue> interesting overview Marco, thanks.
<Joshue> Sorry, Markel.
simon: which open questions are more interesting?
mv: accessibility in use is interesting for me.
<vconway> I am also interested in how we can include the user aspect
<Joshue> I think a very interesting question is "How metrics that measure accessibility in terms of conformance relate to accessibility in use?"
<shadi> +1 to joshue
<sharper> GB: Important part is understanding when a metric is valid and when not
<sharper> GB: need to make it more concrete / operational - 1 idea is to create a test collection
<Joshue> Yes, and how can automated tools do this without explicitly reference some set of conformance criteria such as WCAG?
<sharper> GB: with pages witch known qualities to test these metrics - then expect metrics to perform in specific ways
<vconway> it would be great to develop some type of scoring system
simon: agrees, and refers to NLP and data mining
shadi: carryingout a test is not a job of this
wg
... we should elaborate questions, refine them, ask opinions
<markel> thanks
shadi: the two notions of acc should not be diverging
<Joshue> yes, thanks markel :-)
<markel> :-)
shadi: are these two ways to measure acc? or can we come up with a combined method?
<markel> please could you guys turn the mic off?
<vconway> I am working on trying to set up some kind of scoring system which sounds like what you're asking
<sharper> GB: We should be like a steering committee making suggestions about where to steer this research
<shadi> +1 that our goal is steering researchers
<sharper> GB: we don't do it , but make proposals about the properties it should have
<sharper> +1 that our goal is steering researchers
<sharper> GB: would distinguish the acc in use and acc conf
<sharper> GB: discusses the possibilities of measuring accessibility in use
simon: agrees.
... we should discuss on test collections and also on the 2 notions of acc.
<vconway> I certainly agree that we need both
simon: there are no frequent references in literatutre to these two defs.
<Joshue> sorry, was that to me?
vconway: needs a scoring system to compare monthly but needs to weigh differently different aspects
say again which objects are you analyzing?
vconway: in AU they need to ensure that a site is
conformant monhly
... there are different acc policies and there are different site genres
ok, thanks a lot.
mv: raises another question to vivianne
vivianne: needs a tool for measuring the progress over time
mv: asks vivianne if she used other published metrics
vivianne: has not found a scoring system coupled to wcag2
<markel> thanks vivienne
<markel> Giorgio do you remember the AIR reports?
yes
<vconway> ibm with the adesigner are doing a bit of scoring
<Zakim> shadi, you wanted to talk about "scoring" term
shadi: in use vs conformance
... vivienne case is an interesting one (ie monitoring over time)
<vconway> It would be nice to say 'we have improved from 35% to 80%'
<Joshue> +q
<vconway> I lost the call
<vconway> Just got the call back, so probably missed something
joshue: likes the %approach; acc is not a binary thing
<markel> +q
shadi: questions whether AIR/knowbility is or not conformance
<vconway> I am having a user group also look at my 100 websites to see how the user groups relates to the %score
<Joshue> how about a traffic light system?
<vconway> traffic light?
shadi: if we could come with a scoring system that is objective and that mirors the level of acc it would be fine.
+q
mv: encourages to read the references in thewiki page
<Joshue> yeah, like orange for 'in the middle, not great but not terrible', 'Green for a high level of conformance' and well "red' speaks for itself..
<Joshue> just an idea..
<vconway> good point Markel
<vconway> Josh, the traffic light sounds good
mv: suggsts reading those proposals of metrics and identify weak pointsand AVOID inventing the wheel again (giorgio's emphasis)
<Joshue> thanks vivienne, other abstractions could be expored. I don't know if the driving metaphor is suitable.
mv: what is objectivity? completion time is pretty objective.
<vconway> I think we need both
<Zakim> shadi, you wanted to disagree with "As guidelines conformance does not necessarily entail accessibility in use"
shadi: responds to markel's question of whether
acc in use /acc forconfomance is more important
... if we don't comply with guidelines people with disabilities will have pbms
in accssing the page
<vconway> I have been comparing my results with that of a user group with good success
<Joshue> shadi, it could be a subset of WCAG?
<markel> right
shadi: we might be looking to elicit suggstions of how a perfect scoring system could work
<Joshue> or it we could identify the most important 'must haves' for a11y
<Joshue> Structured content, well formedness etc
shadi: there might be several aspects,
propeorties of metrics to consider; eg cost
... sugegsts a webminar for call of proposals
... maybe setting up a reviewpanel
<shadi> http://www.w3.org/WAI/RD/planning#seminar
<Joshue> Yes, thanks I'll read the supporting materials on the wiki
ok, thank you
sure!
giorgio: volunteers to be a co-editor
mv: will love helping me!
joshue: would like to help in asupporting role
<Joshue> I can do peer review etc
<vconway> I'm happy to help in some way, bt I'm a bit of a baby at this
<shadi> http://www.w3.org/WAI/RD/process#credits
vivienne: would also like (and we'd like to have babies!)
simon: sees editors as program chairs
... we got 2 co-editors MV+GB
shadi: let's not forget rui and yeliz though
vivienne: is available for any help she can
shadi: anybody who would lik to volunteer shouldcontact shadi or simon