Wilco: Anne, you posted a long post in this thread. Please run through the proposed.
Kasper will take it.
Kasper: This is not meant to be included in
the spec word by word.
... But a suggestion and a way of illumating some problems we have with
the current implementation. The current test subject types make too many
assumptions about a given rule. We wanted to make it easier for authors
to pick and choose specifically what is required by the rule they are
writing. In the format we have five examples of test subject types, and
the issue with these are that some of them are mutually exclusive. You
only test on one of them.
... In many cases the rules actually need to mix 'n match between these
different types.
... That would allow you to combine information from the HTTP response
with the DOM tree. So you specify what the rule needs, and not where
should the rule run.
Charu: Like Kasper said, it is essential to know exactly what is required. In IBM we always assume it's run on the rendered page. These aspects are new to me.
Kasper: An example: We have a rule testing if the language attribute on the HTML element matches the language set on the HTTP header. So we would need input from both HTTP response and the constructed DOM tree.
Wilco: It might make sense to do an approach like this. It's a different way of approaching it. Sounds good to me.
<shadi> https://www.w3.org/TR/EARL10-Schema/#TestSubject
Shadi: EARL allows several test subjects.
Here is the spec.
... Look at example #7. Here is a web-page composed of several parts.
Each of these would be test subjects. So test subject can be a compound
of different things.
<rdeltour> sidenote from the peanuts gallery: in digital publishing some of our tests are not based on the single page but the whole publication, the input type would then be the publication (or "page collection" or "set of web pages").
Shadi: EARL does not yet have a type associated with a subject, like "this is a flat file / rendered DOM" or such.
Kasper will take a stab at working a bit more with the proposed changes.
Shadi: There are cases where you would test e.g. consistent navigation on multiple pages, and having multiple test subject types could support this as well.
Shadi: The idea here was to sketch it out
with a diagram of the review process, but haven't had the time yet. The
approach here is currently not very clear.
... The idea is that tests would be contributed through a community
group, as it is easy to set up and does not require W3C membership. And
groups can focus on different aspects. Like the Auto-WCAG group does.
... When a suggestion has enough implementations to show that it has
been accepted by implementors, they can raise a flag and make a pull
request into the W3C rule repository. Then this group can take a look
and hopefully it will be so convincing that it can be approved right
away.
... A lot is still to be worked on, and it needs a better description.
Wilco: So you need two gates: the community group, ensuring the accuracy and having discussions? And the other group wants the rule to be implemented, so that the rule can be a published ACT rule
Maureen: Would that make it too strict for contributions? If they do not have demonstrated implementations, other than code running on their machine
Skotkjerra: How do we deal with implementations for rules with manual testing?
Wilco: "Implementations" means "in use". We should be careful in defining what that means.
Shadi: Agree with Wilco. The review process
document states that implementations can also take other forms. In
regards to how many implementations we need; if someone comes with a
great rule, someone needs to verify the validity of that rule.
... We haven't really seen the dynamic contributions that we have hoped,
due to a slow process. So the idea is to take the vetting outside of the
group, so the group is not the gate-keeper.
Moe: Yeah, let the community drive the acceptance. But when it comes to the working group, who will do the final acceptance? Will we save it for our weekly call?
Shadi: This introduces a third gate-keeper.
1: Community group. 2: Competing vendors / implementors. 3: W3C working
group. And this is why the process has been so slow.
... My hope is that the working group will delegate it to the taskforce,
who will do what needs to be done for it to be accepted. When the
techniques are then updated, the rules should be as well.
... In W3C working groups are, however, not planned to maintain a
specification. But in some cases, like WCAG, the continued work is
needed.
Moe: Since our rules are in a public repository in Github. How can we ensure that it goes through the working group, as gate-keepers. What would be the process for that final vetting on the pull request?
Shadi: The answer is: We will need to discuss this. :-) What happens if someone sends in a comment - will we have to send them to the other group? I would love suggestions for this.
Wilco: Pull requests should only be done be people in community groups. That's easy to track. Other requests are declined with a suggestion to submit to a community group.
Moe: Thinking about how to rewrite the document about contributing. Right now it is all very open to everyone and easy to submit issues and comments for all.
Wilco: This would avoid loading too much onto the taskforce. We want to stay out of developing the rules. Other community groups can do that. Like, a community group for manual rules. We should only accept or declined PRs.
Shadi will finish up the proposal.
Kasper: This process is very different from
open-source, with which I'm more familiar. But of course you need a
balance, so that the repository is not flooded. We need a certain level
of quality. It's difficult.
... The current spec has a lot of things you need to keep in mind. But
Moe's document (contribute.md in the ACT rules repository) is definitely
more light-weight.
... We have three different gate-keepers now, and that has me worried to
some extend.
Wilco: Are you concerned with the speed?
Kasper: Yes. And will people actually follow through? And want to contribute?
Wilco: That's a question for all of us. Are we, ourselves, prepared to go through a process like this?
Skotkjerra: My biggest worry is, if we have enough working power to manage it.
Wilco: We need to engage the group and have
enough discussion to manage it. That's currently very limited. With
multiple contributors, they can have the discussions together and
improve the through-put on Github. But we have to try it to see if it
works, or refine it afterwards.
... I would like to caution it a little bit. Rather publish less, than
publishing things that are wrong.
Shadi will look at changes for his text and we can put it on the survey (maybe after thanksgiving).