<Wilco> https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/Draft_Work_Statement
Wilco: AG reviewed the the work statement and had a survey out
MJ: is the survey done:
Wilco: yes i think it is done and they were going to do a CFC
... no feedback so that is good, they plan to approve
Wilco: plan to have a informal meeting at CSUN, so please fill up the survey
... should we invite Jennifer?
Kasper: yes, she will be going
Wilco: no rules in review, some have been passed to the CG
... i am handling communication to CG and that is a lot of work
... create issues and make updates to the rules or ask the rule author to make the updates and take it bak to CG
... it is causing a bottleneck
... Shadi suggested to divide the work
Shadi: yes, it is a good idea to divide up the work by creating smaller tasks
... create a template for survey and send a survey out. Collect the comments and send it back to CG
... We could get rules from the CG or from outside, so take up all that foot work in turns to shepherd the rules
... folks can pick a rule they are interested in so Wilco is not only person doing all the work
Trevor: SO what is the process?
Wilco: Changes suggested can go into a pull request if not then TF reviews it
Shadi: Collate the info, summarized and put it in the agenda for the TF call and lead it with a conclusion
... once the conclusion is determined, Wilco will raise the issue in CG
... If a person takes all the rules it is lot of work, if divided it makes it easier
Trevor: sounds reasonable and i am willing to help
MJ: i can help too
Charu: i can help too
Shadi: Wilco if you can create a document with steps to follow will be helpful
Wilco: Shadi i will schedule some time for us to do that
<maryjom> https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/Rule_Publication_Tracking
MJ: just started that
... i am putting a document together to track rules status, in review, going to AG WG etc
... i have created certain statuses
... AG WG ready for rules ready to go, AG issues for issues opened. Another state would be a staging state or ready to publish would be another one so we can keep track of all the rule states
... that way we can assign tasks based on various states
Wilco: kasper?
Kasper: yes, we will have a document with all the status?
MJ: yes, is that reasonable?
Kasper: we have a spreadsheet we have used, so if the document is such may work. Another option is Git project
Wilco: ACT has couple
Kasper: OK so we could use that, not sure if that is overdoing that... multiple folks will use and update the document
Wilco: We will have multiple rules ready for survey, updates
... we will review rules every year for accuracy with dates
... we should list a rule author
MJ: ok, i do not want to repeat too much info
... status for rules published and needs maintenance
Trevor: yeah i think that is the way to do it, rule author should be able to edit it
Wilco: we prob should have rules
... a status of ready for survey?
MJ: Yes
Wilco: Status and date will be helpful
... Shadi any thoughts?
Shadi: yes, i think keeping it simple for the handful of rules in review will be good
Wilco: maybe multiple tables will help
... We can finalize on Monday and try it out next week
<Wilco> Implementations Monitoring: Encourage and gather implementation reports from accessibility test tools and methodologies for ACT Rules published by AGWG
<shadi> https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/Draft_Work_Statement
Wilco: So if we were to do this, AG will approve
<Wilco> https://act-rules.github.io/pages/implementations/overview/
Wilco: so the place to start is with the rules WG
<Wilco> https://act-rules.github.io/implementation/axe-core
Wilco: we have list of implementors
and the table with rules implemented
Wilco: and the results of the implemention
... so what are your thoughts on a process like this for published rules
... Shadi any thoughts?
Shadi: i would welcome something like this, now that this is reaching the horizon i would also like to know how it passed? automatic, semi automatic or manual
<shadi> https://act-rules.github.io/pages/implementations/mapping/
Shadi: Maybe if we port that infomation might help answering some questions IBM has
... My understanding is the more rules are published on W3C, the more people will want to implement
... so since we need atleast 3 implementors, this will be important to move forward
Trevor: Since i am not developing the rules, i do not want to cause any problems
... i do like adding qualifiers like automatic and so on
... also add a summary
how the rule is doing generally
Wilco: W3C is not the place to compare the rules
Trevor: Putting the summary may refrain people from looking at the rule details so have to be careful
Shadi: we only have 1 rule published, once we have dozen or so... we can then have SC and implementations available
Trevor: Cocern, people may hardcoding the expected results
Shadi: thats a valid concern
Wilco: there is risk in benchmarking so have to be mindful of
... unless it is open source
... there is some level of trust involved
... are the any others in W3C like this
Shadi: yeah, browser testing
Wilco: those are easy to verify
Shadi: We are not there yet... where folks are gaming the system
MJ: import now is to get implementation