See also: IRC log
Shadi: TPAC meeting on Thurs / Friday for
this group.
... Tech plenary is in SF in the USA.
... Please book now.
... Hope some of you will be able to attend.
Shadi: Reminder - let us know your
availability.
... Please keep it up to date.
<shadi> https://github.com/w3c/wcag-act/pull/94
<shadi> https://github.com/w3c/wcag-act/pull/94/files
Shadi: Changes - versioning.
... It could have been that we more interested in moving to a change
log, rather than semantic versioning.
Moe: Should be use a better adjective than huge.
<MoeKraft> It is recommended that for significant changes, such as those that change the scope of a rule, a new rule is created and the old rule is deprecated.
Shadi: Any thoughts on this.
... Moe that looks simpler.
<cpandhi> + 1 to Moe's change
Shadi: Kathy - changes that Moe suggested
are better.
... Alistair as you have concerns on how versioning is done, please look
at that.
... Any other comments.
Alistair: I think the initial comment was people don't like versions to change to quickly.
Shadi: In the new concept, minor changes would be added to the change list. But major changes would change the test number.
Anne: Would a change to the rule change, by putting in a change to the process in the test, would change the number.
Alistair: But, what would happen if a person tests now, and then tests 3 months from now - and loads of ids have changed.
Wilco: If you break a rule up, then you'd have to give the sub-rules a different id.
Anne: What is an update and what is a
significant change.
... Wilco things only really big changes should change the id; and Shadi
suggests smaller changes might change the id.
... Initially the simplification is OK; but we need to be clear on when
something needs a changed id number.
... Tracking things over time is really important, so I wouldn't want to
change ids too much.
Charu: it is good to be able to have
history; so we have to be clear about what would cause a significant
change.
... What if it was a bug fix, would that force a new id?
Shadi: We need a way of capturing changes to tests; and we need to properly understand "significant" in terms of something which would require a new id.
Wilco: It seems that that section about ids is common sense. If it comes at the cost of loosing your history.
Shadi: Sounds to me like we need more clarity in what to do.
Wilco: take out the rule about create a new id if there are big changes.
Charu: Issue we see in our rules. One of
complaints is, a result is changed through a bug fix.
... People's reports being changed is something they don't like.
... What we do is to have a rule set. Then we release one or two rule
sets per year.
Alistair: Bug fixes in tests are a good thing.
Shadi: If you test a point x and y you
don't know if the content changed; or the rule changed.
... So we need the log.
... You'd need to go to the change log to see if a test has changed.
<MoeKraft> Significant changes, for example those that change the scope of a rule, may require the creation of a new rule and deprecation of the current rule.
Moe: Thinking about to Wilco's suggestion.
Maybe we want to soften the second paragraph.
... The change is "may"
... Its not prescriptive.
Shadi: Thoughts.
Wilco: I like the change.
<Wilco_> +1 to example
<cpandhi> +1 to example
Alistair: With the word "may" you would
need an example.
... Example: if you split a test as you have discovered it is too broad.
The two new tests would then each get a new id, and the old broad test
would be deprecated.
Shadi: Do we need an example of a minor change.
Wilco: Don't think its necessary.
Anne: When developers read through ACT
framework they wanted more examples, and two examples when possible.
... We would like a minor change example.
Alistair: Let the minor change example be a bug fix.
Wilco: I'll put the major and minor examples in.
Moe: Pull request added.
Shadi: Any other comments.
... Will you notify mailing lists of the changes.
<shadi> https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/ACT_Review_Process
Shadi: Take a moment to look through it.
... Have we all read it?
... We want enough incentives so that the community starts to review
their own tests.
... You ask them to get a certain number of implementations.
... Once the contributor can show this validation, then it is easier.
... Basic idea behind this.
... What do people think?
... Is it workable, or too optimistic.
Alistair: How do people show the rule to someone who implements it.
Shadi: They would put the rule in the
GitHub repo with a flag marked untested.
... You keep on updating your rule in GitHub.
... We have the rule, test cases and implementations.
Alistair: What if loads of people place rules which are similar. How do they work together. Then how many rules might we expect.
<Wilco_> On this point, what we discussed previously was that we might publish 'draft' rules that haven't been implemented in enough places yet, as a way to get that ball rolling
Shadi: People would have to try to collaborate.
Charu: In GitHub we can have beta rules.
... We could have loads of tests in there... there needs to be a way for
people to comb through rules.
Shadi: We could weed out certain tests
through formats.
... What if different companies submitted overlapping tests - they each
have their reasons for having their own tests.
Shadi - discussion would have to be peer-to-peer.
Shadi: 4 more minutes.
Shadi - Let's give people more time to work through this - shall we use mailing list, GitHub.
Wilco: Let's use mailing list.
Shadi: You could help move this forward. What would incentivise you to put rules forwards.
Wilco: Worth us checking back with our colleagues.
Alistair: How do we discuss the tests on GitHub.
Shadi: Implementations would be what we'd be interested in.
Kathy: Our testing is different from the tests that aXe can run.
Shadi: Either we have a group that checks rules centrally; or you try and get people to implement your rules in other tools.