Meeting minutes
Intros and Announcements
AC: Is there's anyone who'd like to introduce themselves?
… but normally we do run an onboarding session for new starters but didn't today.
WCAG EM Update
HV: WCAG EM update. Process for evaluating if websites conformance to WCAG.
<hdv> WCAG-EM 2.0 draft: https://
HV: It is 11 years old.
… we thought it would be a great time to kind of try and update it and bring it up to date.
… some of us have started working on a 2.0 version.
… Mostly small maintenance things.
… we've gone through all the links, so a lot of those kind of updates trying to make sure the links go to the right place.
… my colleague Jay and Steve Faulkner are helping to work on this. Reach out if you would like to join us.
<bruce_bailey> EM == Evaluation Methodology
<hdv> GitHub repo w3c/
HV: planning to a face to face 7 and 8 July.
… use email or slack to contact us.
… planning just face to face first.
Introduce Assertions Discussion https://docs.google.com/presentation/d/1A3kRKcBsodmGbW2JRkoU6a_rLSfQCmdUEQWzrgwP2uA/
RM: We have been talking about different things, different solutions to meet requirements, and we want to start making sure we are circling back on those solutions and have answered the questions that are around them that are still outstanding.
… There's a slide deck that went in detail from earlier conversations, there's a folder with other work and Subgroup work we did on assertions.
… The explainer has the results of some of that in the section on assertions. We've had some GitHub discussions about different questions that are outstanding.
(Reads definition)
… we have a number of outstanding questions with assertions.
… The questions for today, so how do we handle assertions that span guidelines, as our subgroups have worked, um, subgroups have come up with assertions that you know, the procedure itself might span multiple guidelines, or have results that apply to multiple guidelines. We need to figure that out.
… Some of these we've kind of touched on and answered in previous conversations, but l'd like to make sure we're all on the A good place with that. And finally, should assertions be dated, expire, or be reviewed on a regular basis?
<julierawe> Have to jump for mandatory team meeting—hope to rejoin this meeting soon, thanks
(Goes over examples in slides)
… So how do we handle some of these questions? If we go to the next two slides, these are a suggested approach, and this is meant to be An initial suggested approach that we can modify, we can throw out. it is trying to answer some of the questions that have come up in a way that ties it to the work that's been done.
(Goes over suggested approaches)
RM: So, this is an approach, there's obviously many other ways we could divvy this up, but we wanted to bring it to the group and get reactions, thoughts, um, how can we make this better, or do we need to make dramatic changes to it?
Gregg: This seems to be saying that an assertion is something that WCAG,, says, here are the assertions you can make, and you pick from those. You don't just make up random ones, which is going to be important if you're going to count them.
… You don't just make up random ones, which is going to be important if you're going to count them.
… It has a short name, and then it says, you say this, and it...
And it needs to include this. In other words, you can't say the assertion is this, and then underneath it say that, oh, it needs to include these other things.
… it has to be actually in the assertion itself. Same thing with... about, uh, you know, any requirements on the assertion should be in the assertion, uh, and... You also mentioned, um, internal documentation, best practices.
… think we need to collapse all the information about the assertion into it, so that It is a statement of what you are asserting, and you're asserting all of the pieces that aren't any additional requirements.
… finally, you use "organization" what if I'm an individual? Perhaps consider a word like "entity" instead of "organization"
<kirkwood> Feel I ned a real world example to talk to it. Assertion: a manual review of alternative text is relevant and valid on a monthly basis etc. Would that be right?
<kirkwood> need an example to talk to this
Wilco: I was really missing the question as... how do you know these assertions are meaningful?
… How do you know they're effective? How do you know they can be trusted?
… Thing where people make all sorts of claims, but... then the effectiveness becomes... pointless.
KF: +1 Gregg and Wilco.
… this does feel like it could be a big malicious compliance vector.
<Zakim> alastairc, you wanted to comment on assertions being a tick-box, and the rest is up to the org to define (or their regulator) and to comment on how we know assertions are useful
<kirkwood> you are “asserting” that you have procedures and that you are following them?
KF: I would want there to be, like, sufficient... There needs to be enough meat to the assertion within the to basically make it enforceable, I guess, is kind of where I'm going with it.
CL: If these are going to be published assertions, and then you talk to internal verification.
That is an internal verification, but can someone publicly look into that, to a degree.
… It seems like it's internal to the group making the assertion and not having the ability for someone to verify that externally.
… To Greg's point, if you're stating something's either like, in an assertion, but then you're making requirements within that assertion that you know, user testing...
<Jon_avila> I agree with Alistair, for many of the things we would use assertions, it would be difficult to make them another type of requirement. So if we don't use assertions then we would not have them at all. We should be careful though to make sure we only use assertions when we can't make them other requirements.
<Zakim> Chuck, you wanted to say I believe the verification is not for testers, its for the authors of the assertion
AC: There are things that are really useful to do for accessibility that aren't covered by the guidelines.
So, a lot of those are things that we know, and this is going to Wilco's point about How do we know they're sort of useful?
… design systems, style guides, usability testing.
… these are... required by our government digital services when they do, um, as part of the, sort of, development process and their gateways through.
<sarahhorton> Are the assertions a way to integrate the Accessibility Maturity Model? https://
Through various things. They are known to be useful for accessibility.
… They are also things that you can't require in a binary guideline in any kind of useful way.
… resounding consensus on you know, if we required organizations to publish publicly this kind of information about what kind of processes they've done in the background.
<kirkwood> We all have a different idea of what an assertion is. We need an example. Are you asserting that you are doing it, asserting that you have procedures, or that you are checking it? We need an example.
<alastairc> sarahhorton - we could draw things from the matuity model to create assertions.
<Zakim> Rachael, you wanted to respond to Gregg
Chuck: One of the things that I think was motivating the addition of assertions is that are aspects of accessibility that we know aren't Very measurable with a degree of consistency across the evaluators, we have a sense through industry
… there's no really clear or clean way of getting a measure.
… That such practices are known to improve accessibility.
… That's typically a good thing, whether or not they've done it well, that's not something we can measure, but the measure is that they are claiming to have done this thing.
<ChrisLoiselle> Not to ask a duplicative question I've asked earlier, but are assertion statements one in the same of an accessibility statement https://
RM: chair hat off, uh, from a personal level, I think because of a lot of the concerns that are raised, my personal preference would be to keep assertions at a supplemental level.
<Zakim> Rachael, you wanted to give example in plain language space
<kirkwood> great
JK: I might be misunderstanding things here, um... But that sounded.... RM was just talking that you're asserting that you're meeting the guideline?
<bruce_bailey> ACAA Air Carrier Accessibility Act
<kirkwood> +1 to Glenda!!
RM: so this isn't meeting a guideline, this is conducting a process
<kirkwood> +2 to Glenda :)
Glenda: So, I'm very pro-assertions. I think this fills an incredibly important gap.
<bruce_bailey> So great to hear from Glenda that the one ACAA sentence has had such noticeable impact!
Glenda: I think that these assertions will be so powerfully positive. I think that we must have some assertions at the... I mean, I'm interpreting a foundational level of WCAG 3.
… 2 ideas for assertions for usability.
<julierawe> Glenda can you add a link?
<LauraTSzivos> +1 on copy-pasta
Glenda will add them.
<Glenda> First example assertion for Usability Testing: Participant-Representation Assertion
<Glenda> * X participants representing at these three functional-need categories
<Glenda> (vision, motor, cognitive/learning) on all critical user journeys of the X App.
<ChrisLoiselle> https://
Wilco: I strongly agree with Glenda on the point that having them complementary level would not address the equity problem.
<Glenda> Continuous-Usability-Testing Assertion: conduct an accessibility-focused usability test for every major product release or at least once every x timeframe (whichever comes first)
Wilco: We need to do better on the minimum baseline.
… I think we really do need to know the quality of these things. We need to make sure that these things work
<Zakim> Rachael, you wanted to ask Wilco for clarification
Wilco: I don't know that assertions work. I do think we need to do better at a foundational level.
<Zakim> Chuck, you wanted to react to Wilco to discuss a point of order
Wilco: I don't know that assertions are a silver bullet solution
<kirkwood> +7
Gregg: The power of having assertions is getting it in front of people.
<Glenda> Yes! Yes! Yes! (to GreggV)
<Jon_avila> I agree with Glenda, the CVAA also required that people with disabilities were consulted during the design of the product or service.
<kirkwood> +1 to Gregg
Gregg: The whole purpose of the assertion is to get things in front of people that are not testable.
<Wilco> -1 Gregg, We've seen how well "in the standard, but not required" works from AAA
Gregg: There shouldn't be any internal verification level.
… be better is to say that you followed a style guide and you point to one.
That is a meaningful style guide.
… I would like to see the assertions. Right there in front of you, mixed in with the requirement...
<kirkwood> examples would be helpful
<Glenda> Glenda wants to share the HECVAT questions that are having really valuable impact in Higher Ed space: https://
KF: appreciative of Glenda's examples and Gregg's remarks around the assertion including specifically what e.g. a style guide needs to cover.
… that clarifies some of what my concerns.
<Zakim> alastairc, you wanted to comment on how to incorporate assertyions
<Wilco> This is the book I mentioned: https://
<kirkwood> +1 to Ken
KF: We just need to be careful how we qualify the assertions.
AC: Not disagreeing with gender's examples. We do need to consider how it relates to different guidelines.
<Chuck> Alastair has closed queue. We will continue this conversation in the future, but we do need to move on to our other agenda items. This conversation will be continued.
<Wilco> Are you absolutely sure you want lawyers to get involved in what can and can't be in a style guide?
AC: The other point I wanted to make around the foundational level. got... we've got requirements at Foundational, we've got requirements at supplemental.
… You have to do the foundational ones, and then we would... use points or percentages on the supplemental ones.
<alastairc> Sorry, queue closed on this topic for today.
AC: that doesn't mean that the minimum regulatory level is just foundational requirements.
… So the minimum regulatory level could be foundational plus certain score or percentage of supplemental and or assertions.
… foundational requirements could be quite minimal.
<bruce_bailey> I don't follow how it is likely that supplemental requirements are better measured by percentage than foundational requirements ?
<Rachael> We can continue this conversation at https://
<Glenda> And here are some recent questions I have had to answer to HigherEd clients wanting to buy my software:
<Glenda> Is your customer support team trained in disability etiquette and effective communication?
AC: But then, at the supplemental and assertions. Some of those could be required.
<GreggVan> Wilco mentioned that we should not rely ONLY on assertions. AGREE We should also think of other things. ONE other thing would be to have Recommendations as well as requirements. Recommendations do not need to be testable
CL: question is just around the end goal of assertions and where they fit in.
<Rachael> Bruce, its measuring the number or percent of supplemental requirements and assertions completed not the percent of pass/fails within a supplemental requirement. Did that answer your question?
<Glenda> How are accessibility issues tracked and remediated?
<Zakim> bruce_bailey, you wanted to say that some assertions COULD be tested -- just not maybe in the way WCAG 2 SC are tested.
CL: You may need to assert some things that are requirements, and then some may not be requirements.
… And where would those fit in if they're not required? Um, and that's where, like, my mind goes to AAA versus AA, and we're... a lot of industry maps back to AA in the current scheme of things.
<alastairc> Wilco - we discussed a lot of things other than pass/fail and assertions, but none were found to be useful, we need a bit more info on the possibilities to know if there is another direction to go.
<Glenda> Do you test with actual users with different disabilities?
Bruce: I just wanted to point out that That assertions could be testable.
Glenda: it's been really fun in the last couple of years is the number of questions I get when clients try to buy our software.
… somebody's always adding a new question outside the HECVAT.
<kirkwood> +1 to Glenda
Glenda: We need to take a step forward here.
<alastairc> The conversation can continue at https://
Glenda: And not wait for perfection.
<kirkwood> ACA is already doing it
Review of Inputs https://docs.google.com/document/d/1cxiE1rfpmYs0fmS1CDyRGq-_-_mTghz7vRlGcAhgWLc/edit?tab=t.88ncy72gp2qe#heading=h.i3dhthm59ysy alastairc]
<bruce_bailey> https://
alastairc: our last item is a review of input requirements
<Chuck> question from Lori that she was unable to ask about assertions: a policy that editors are required to follow the style guide", who enforces this?
alastairc: haven't been through these in a lot of detail… I see Detlev has
alastairc: this is a call for folks to go through this document.
<bruce_bailey> Jenn and Fillipo too !
alastairc: this is a little more generalised version of keyboard requirements, generalised to input methods
alastairc: Gregg could you do a 3 minute runthrough?
GreggVan: we started with the list given to us. We took every one of those on the list page, went through all user needs for input and found several were not covered by those so we added a couple
GreggVan: 2-3 we found out of scope, eg what browsers should do
<bruce_bailey> also HT to Glenda and Giacomo on earlier Keyboard group
GreggVan: the Google Doc has an outline
GreggVan: there are four major outlined
GreggVan: being “input operation”, “keyboard”, ”cognitive/physical effort to use”, “pointer”
GreggVan: under each of those we have foundational requirements
GreggVan: then we have supplemental requirements, we call them supplemental recommendations
GreggVan: good advice, not always things you'd want to do. I recommend we have a category called 'recommendation', sometimes a more extreme version of the one above it
GreggVan: we tried to write these in a final, final form
<Laura_Carlson> s/within the to basically /to basically /
GreggVan: we have page/view in a lot of places here; in the old model it was pages, in the new model we tried to get to view. But we couldn't seem to find a definition that works
GreggVan: that helped us read the sentence with the word page and the word view
GreggVan: we also have techniques that say don't do what is said
GreggVan: we're done with the entire section if people want to review it
<Zakim> Rachael, you wanted to talk about regarding recommendations
Rachael: to add a qualifier, we have foundational reqs and @@@ reqs, both are testable statements
Rachael: recommendations was a loaded word that we stepped away from
GreggVan: in our doc we found most stuff was foundational
GreggVan: some things are more AAA-ish in nature. They may be testable but only work for certain types of content not others
GreggVan: if you get into supplemental, be careful
<Zakim> bruce_bailey, you wanted to follow Gregg
bruce_bailey: re supplemental reqs / best practices / recommendations… a great example of that is that we want web pages to have a level of usage of keyboard proportionate to someone using a mouse, but not sure how to test that
bruce_bailey: I got on queue to half-apologise to someone coming to the doc for the first time… the URL is the right one, it is the main content we are working from, some collapsible accordions etc
bruce_bailey: we found that what we have here should make sense for a new person to make comments on
GreggVan: everything useful is in the section called 'master' and sub pages
bruce_bailey: link kind of puts you in middle of the document, there is a TOC above that
alastairc: a good point to start reviewing
alastairc: if anyone from the sub groups has any general questions you'd like to raise, eg have you gotten stuck on something?
GreggVan: we have some things under 'supplemental', should we throw them away as they can't be recommendations? or should we create a third category 'best practices' in addition to foundational and supplemental\?
<Zakim> Rachael, you wanted to ask about accessibility
Rachael: i'd add another category called best practices. But when you split them up be thoughtful about foundational vs supplemental
<bruce_bailey> Might we also mark them as exploratory -- hoping to come up with testable metric?
Rachael: the things not really testable fall into best practices. Haven't been focused on best practices,but if you capture them do add them into that category and we can work on those
bruce_bailey: keyboard accessibility makes sense to be as a supplemental requirement… maybe add as exploratory as we're looking for metric?
<Zakim> CarrieH, you wanted to ask whether the different types of requirements documented somewhere?
<bruce_bailey> i do like "keyboard-use-fair" maybe as an assertion
CarrieH: do we define somewhere what the different types of requirements are?
alastairc: we have, will look
GreggVan: can we have the ability to add questions? for instance Bruce's comment… if we really want input, people may not comment
<CarrieH> because I feel that for folks that aren't on every call, and also people with auditory processing delays, it's good to have this written down somewhere..
alastairc: yes can add that explicitly
GreggVan: having something be supplemental that can't always be applied but you get credit for doing it… meaning if you have a complex site you'll have a lower rating
<alastairc> Foundational requirement - Must be done or fail. Supplemental - Could be done, improves your score, may not apply to all interfaces. Best practice - may not be testable, or not applicable to a particular interface.
<Zakim> Rachael, you wanted to ask about accessibility of solutions and to
Rachael: chair-hat off… are you qualifying solutions themselves must be accessible or do we assume every req meets the other reqs?
Chuck: I've been presuming in my subgroup that anything we write is presumed to need to meet the other requirements. So yes, that they must be accessible
<bruce_bailey> +1 to chuck, as inputs has done as well
alastairc: good point might want to make a note of this
Subgroup work
alastairc: ok… no further questions, let's head to our breakout rooms