04:39:11 <RRSAgent> RRSAgent has joined #wcag-conformance
04:39:15 <RRSAgent> logging to https://www.w3.org/2025/11/12-wcag-conformance-irc
04:39:15 <breakout-bot> RRSAgent, do not leave
04:39:16 <breakout-bot> RRSAgent, this meeting spans midnight
04:39:16 <breakout-bot> RRSAgent, make logs public
04:39:18 <breakout-bot> Meeting: Effective and trustworthy conformance evaluation for WCAG and beyond
04:39:18 <breakout-bot> Chair: Hidde de Vries, Jeroen Hulscher
04:39:18 <breakout-bot> Agenda: https://github.com/w3c/tpac2025-breakouts/issues/28
04:39:18 <Zakim> Zakim has joined #wcag-conformance
04:39:19 <breakout-bot> Zakim, clear agenda
04:39:19 <Zakim> agenda cleared
04:39:19 <breakout-bot> Zakim, agenda+ Pick a scribe
04:39:20 <Zakim> agendum 1 added
04:39:20 <breakout-bot> Zakim, agenda+ Reminders: code of conduct, health policies, recorded session policy
04:39:20 <Zakim> agendum 2 added
04:39:20 <breakout-bot> Zakim, agenda+ Goal of this session
04:39:22 <Zakim> agendum 3 added
04:39:22 <breakout-bot> Zakim, agenda+ Discussion
04:39:22 <Zakim> agendum 4 added
04:39:22 <breakout-bot> Zakim, agenda+ Next steps / where discussion continues
04:39:23 <Zakim> agendum 5 added
04:39:23 <breakout-bot> Zakim, agenda+ Adjourn / Use IRC command: Zakim, end meeting
04:39:23 <Zakim> agendum 6 added
04:39:23 <breakout-bot> breakout-bot has left #wcag-conformance
05:14:03 <breakout-bot> breakout-bot has joined #wcag-conformance
05:14:06 <breakout-bot> RRSAgent, do not leave
05:14:06 <breakout-bot> RRSAgent, this meeting spans midnight
05:14:07 <breakout-bot> RRSAgent, make logs public
05:14:09 <breakout-bot> Meeting: Effective and trustworthy conformance evaluation for WCAG and beyond
05:14:09 <breakout-bot> Chair: Hidde de Vries, Jeroen Hulscher
05:14:09 <breakout-bot> Agenda: https://github.com/w3c/tpac2025-breakouts/issues/28
05:14:09 <breakout-bot> Zakim, clear agenda
05:14:09 <Zakim> agenda cleared
05:14:09 <breakout-bot> Zakim, agenda+ Pick a scribe
05:14:10 <Zakim> agendum 1 added
05:14:11 <breakout-bot> Zakim, agenda+ Reminders: code of conduct, health policies, recorded session policy
05:14:11 <Zakim> agendum 2 added
05:14:11 <breakout-bot> Zakim, agenda+ Goal of this session
05:14:12 <Zakim> agendum 3 added
05:14:12 <breakout-bot> Zakim, agenda+ Discussion
05:14:12 <Zakim> agendum 4 added
05:14:12 <breakout-bot> Zakim, agenda+ Next steps / where discussion continues
05:14:13 <Zakim> agendum 5 added
05:14:13 <breakout-bot> Zakim, agenda+ Adjourn / Use IRC command: Zakim, end meeting
05:14:13 <Zakim> agendum 6 added
05:14:14 <breakout-bot> breakout-bot has left #wcag-conformance
05:33:31 <jeroen> jeroen has joined #wcag-conformance
05:37:38 <jeroen> scribe+
05:43:33 <JJ> JJ has joined #wcag-conformance
05:43:44 <JJ> present+
05:43:48 <Ben_Tillyer> Ben_Tillyer has joined #wcag-conformance
05:43:52 <Ben_Tillyer> present+
05:43:59 <LenB> LenB has joined #wcag-conformance
05:44:05 <tantek-projector> tantek-projector has joined #wcag-conformance
05:44:09 <jeroen> present+
05:44:09 <hdv> hdv has joined #wcag-conformance
05:45:12 <kenneth> kenneth has joined #wcag-conformance
05:47:22 <hdv> Zakim, start meeting
05:47:22 <Zakim> RRSAgent, make logs Public
05:47:24 <Zakim> please title this meeting ("meeting: ..."), hdv
05:47:39 <hdv> meeting: Effective and trustworthy conformance evaluation for WCAG and beyond
05:47:43 <jeroen> meeting: Effective and trustworthy conformance evaluation for WCAG and beyond
05:48:25 <tamsin1> tamsin1 has joined #wcag-conformance
05:48:51 <kenneth> scribe+
05:49:30 <JJ> present+
05:49:33 <kenneth> present+
05:50:18 <JJ> meeting: Effective and trustworthy conformance evaluation for WCAG and beyond
05:50:49 <kenneth> hdv: For those who don't know, WCAG is Web Content Accessibility Guidelines, describing how you can make web content accessible
05:50:57 <Daniel> Daniel has joined #wcag-conformance
05:51:04 <kenneth> ... used by legislation all around the world, to establish or help people establish what is needed to be accessible
05:51:07 <tamsin1> present+
05:51:09 <Daniel> present+
05:51:20 <kenneth> ... need to evaluate whether things need WCAG, and that's where WCAG-EM comes in
05:51:44 <kenneth> ... I'll briefly describe what's in WCAG-EM. Current version is 1.0, came out in 2014, over 10 years ago now
05:51:51 <Ben_Tillyer> Link to WCAG-EM: https://www.w3.org/TR/WCAG-EM/
05:51:55 <kenneth> ... provides a procedure to evaluate how well websites conform to WCAG
05:52:14 <kenneth> ... it's a procedure. Doesn't say much about accessibility; says how to measure it. How to establish scope, how to select a sample
05:52:30 <kenneth> ... e.g. how to select a representative portion of a website if you have a site with a thousand pages
05:52:47 <kenneth> ... also guidance to report on what you found
05:53:00 <kenneth> ... meant for use on existing websites. You've built a website and want to know how well it conforms.
05:53:05 <kenneth> ... can also be used during development.
05:53:17 <kenneth> ... meant to be technology-agnostic, but quite specific to the web
05:53:41 <kenneth> ... important to note it is a process. It's not anything that adds any new requirements
05:53:56 <Remi> Remi has joined #wcag-conformance
05:53:59 <kenneth> ... it's a document that makes suggestions on how to structure your evaluations, it's not a document that requires anything
05:54:22 <kenneth> ... You start with defining evaluation scope, then explore target website, select a representative sample, then audit the selected sample.
05:54:32 <kenneth> ... Diagram shows arrows pointing both forwards and backwards between steps
05:55:07 <kenneth> ... WCAG-EM gives us a way to establish if a website conforms to WCAG, especially if it's too big to look at everything.
05:55:25 <kenneth> ... it's also a way to ensure consistency between reports, and make them comparable
05:55:43 <kenneth> ... this has been extremely important at Logius working with the Dutch government
05:56:08 <kenneth> ... we would need to look at almost 9,000 websites. Every member state of the EU has to do something like this
05:56:23 <kenneth> s/would need/need/
05:56:31 <kenneth> ... and each of those websites could have over a thousand pages
05:56:43 <jeroen> Link to Dutch dashboard of the ~9,000 websites: https://dashboard.digitoegankelijk.nl/
05:56:44 <kenneth> ... because WCAG-EM results in the same structure across reports, it allows to build dashboards
05:56:58 <kenneth> ... and WCAG-EM is not something just for us, it's something that anyone can use
05:57:08 <kenneth> ... procedure is intended to be helpful to more people.
05:57:21 <kenneth> ... We think it's time to work on the next version, WCAG-EM 2.0
05:57:33 <kenneth> ... a few of us in AGWG have been working on this for about the last 7 months.
05:57:46 <kenneth> ... including Hidde, Jeroen, and Steve Faulkner
05:57:50 <jeroen> Link to WCAG-EM 2.0 Editor's draft: https://w3c.github.io/wai-wcag-em/
05:58:18 <kenneth> ... lots of small bits (editorial), fixes to URIs and spelling, improved wording
05:58:34 <kenneth> ... trying to align with the style guide for WCAG 3 that Tamsin is working on
05:58:46 <kenneth> ... these are not exciting changes but I'm glad we've been doing them
05:58:55 <kenneth> ... the more exciting change is we've been trying to broaden the scope of WCAG-EM
05:59:03 <kenneth> ... try to make it more abstract to be applicable to more than just websites
05:59:22 <kenneth> ... we've replaced 3 terms, to make it compatible with not just websites, but also apps, kiosks, and other things that you want to know how accessible they are
05:59:44 <kenneth> ... replace "sample" with "sample set", then replace "web page" with "view" or "sample"
06:00:20 <kenneth> ... and replace "website" with "digital product", to align with European legislation and other places
06:00:34 <kenneth> ... WCAG 3 may be aligning with the latter as well
06:01:24 <JJ> And the WC part was Web Content and is now W3C, like WCAG3
06:01:32 <kenneth> ... finding this useful for design systems as well
06:01:53 <kenneth> ... Currently getting ready for Draft Note status
06:01:58 <tamsin8> tamsin8 has joined #wcag-conformance
06:02:01 <kenneth> ... which is the step before an actual Note
06:02:13 <kenneth> ... will work on updating report tool
06:02:24 <kenneth> ... Looking to open discussion for questions and ideas
06:02:42 <kenneth> ... what would you expect in a modern methodology for evaluating WCAG? Are there ways we can make it more effective? More trustworthy?
06:03:49 <JJ> q+
06:04:01 <hdv> ack JJ
06:04:32 <kenneth> JJ: I'd like to say I'm really happy about this new development. Also something we've been trying to do with our foundation; we've made our own version of EM previously, APT-EM
06:04:55 <kenneth> ... I'll need to check the latest changes in the draft, no questions at the moment
06:04:59 <LenB> q+
06:05:07 <hdv> ack Len
06:05:16 <JJ> s/APT-EM/Appt-EM
06:05:22 <kenneth> LenB: Something for us that'd be helpful is an evaluation method for design systems
06:05:32 <kenneth> ... before a page or a view exists, is there a way to evaluate a design system meaningfully?
06:05:33 <Ben_Tillyer> q+
06:05:38 <kenneth> s/meaningfully/with a meaningful report
06:05:51 <kenneth> ... With the way that Figma and other tooling works now, there may be something interesting in how those things come together.
06:05:59 <kenneth> ... could take a snapshot and somehow apply it
06:06:30 <kenneth> ... Another thing I was thinking the other day is, plain language is not on my designers, it's on the content strategy team, marketing teams, etc. on the client side who consume our design system
06:06:49 <kenneth> ... is there something we can pass on to them to ensure they are using our guidelines?
06:07:04 <kenneth> ... if I can prevent the issues in creation, then when we get to evaluate the final thing, we'll have already knocked out a lot
06:07:31 <kenneth> hdv: We're talking a lot about how to deal with conformance claims in AGWG as well
06:07:43 <kenneth> ... something in the design system is too small to make a conformance claim about
06:08:01 <kenneth> ... maybe WCAG-EM can work with that
06:08:19 <kenneth> ... with a design system, you want to know whether the building blocks are good from an accessibility perspective
06:08:39 <kenneth> LenB: There's a sister component as well, we run checks on our storybook repos before engineering picks it up and brings it into the UI.
06:08:50 <kenneth> ... It's not that great in the sense that it's still in a wrapper, but it can at least check that component
06:09:07 <kenneth> ... e.g. there could be 14 ways to screw up a text input component
06:09:20 <kenneth> hdv: There could be things to verify or check inside a component
06:09:43 <kenneth> LenB: Wondering if there's a way to adopt this to bring to a design system
06:09:56 <kenneth> hdv: Maybe this is something we can bring up with AGWG; it's a great question
06:10:15 <kenneth> @@1: If you can catch issues in the design phase, it can prevent a lot of issues in development
06:10:31 <hdv> s/@@1/JJ
06:10:39 <kenneth> ... I wonder how it would be implemented, e.g. in Figma. Maybe not a full conformance claim
06:10:55 <kenneth> hdv: Some of the accessibility problems you might expect, might not exist yet within the design phase
06:11:12 <kenneth> ... RE authoring tools, that's another angle you mentioned, that's also a really good way to do things
06:11:30 <kenneth> ... ATAG is looking to revamp
06:11:41 <kenneth> LenB: I do think this will be easier to do with 3.0 than 2.2, e.g. around plain language assertions
06:12:03 <kenneth> ... we specifically avoided passive voice guidance, and said it has to be an assertion. It's a thing we can check, but we can't say that all legal sites have to be active voice
06:12:12 <jeroen> q?
06:12:16 <kenneth> Tamsin: Passive voice is needed in some cases; it's contextual
06:12:25 <hdv> ack Ben
06:13:01 <kenneth> Ben_Tillyer: Whenever I found a problem with something in Figma, I identify the node ID, right click and copy the URL and it's in the URL, which makes it easy for someone to jump to it
06:13:31 <kenneth> ... We have millions of web pages across many subdomains. Within different platforms we have different templates and themes. Issues could come from a content author, a template that's in use, or something else.
06:13:46 <kenneth> ... What I've found useful when collecting other people's thoughts and accessibility findings, is for them to say where they think it's coming from
06:14:08 <kenneth> ... There's one accessibility company I know that is working on reporting whether a shared component has caused an issue and sharing that information
06:14:40 <kenneth> ... it'd be really good if we could have a source/cause; then could make use of filtering
06:14:59 <kenneth> hdv: Sometimes auditors don't know, sometimes they do, e.g. if they're sitting next to the developer
06:15:27 <kenneth> Ben_Tillyer: Auditors may use xpath or some way to select an element, to attempt to help the company work out the issue
06:15:29 <hdv> q?
06:15:54 <kenneth> ... another source of defects can be a quirk of some AT
06:16:02 <hdv> q+ JJ
06:16:03 <hdv> ack JJ
06:16:34 <tamsin3> tamsin3 has joined #wcag-conformance
06:16:34 <kenneth> JJ: Something missing for apps is you don't really have a URL there. Apps may allow you to include a screenshot. Can be hard to get back to a specific state to reproduce.
06:16:50 <kenneth> ... if there are no further questions, maybe spend some time to go through the document?
06:18:48 <kenneth> @@1: Can components individually conform to WCAG, but then fail to conform when combined in a certain way?
06:19:51 <kenneth> hdv: May be worth evaluating in terms of smaller pieces; I know design system teams tend to test accessibility as well. It's not really a WCAG-EM kind of thing
06:19:51 <hdv> s/@@1/Sayaka
06:20:31 <kenneth> Ben_Tillyer: I know this won't help EM, but I've found that one of the key things you can have in a design system is to have really clear documentation regarding how a component can be used
06:20:49 <kenneth> ... one of the common sources of accessibility defects I've seen is when developers use design system components in a way they were not designed for
06:21:19 <hdv> q+ LenB
06:21:20 <hdv> ack LenB
06:21:46 <kenneth> LenB: Something we do is we will identify in our documentation anything that must be included, e.g. alt text
06:21:54 <kenneth> ... this will kick off a set of checks
06:22:09 <nico> nico has joined #wcag-conformance
06:22:18 <kenneth> ... the other thing we do is double-check things like interactions.
06:22:42 <kenneth> ... e.g. with a modal, what happens when the user returns from the modal? They should return to trigger, which means you should not remove the trigger before the user returns from the modal
06:22:53 <kenneth> ... the design system needs to mature well past the component to determine how those kinds of things come together
06:23:03 <kenneth> ... I have been trying to figure out how to determine missing instructions. One way is to examine our list of defects
06:23:28 <kenneth> ... We could totally do this in Figma; there are other tools that are harder to check because we don't have the documentation tools for it
06:23:52 <kenneth> ... onboarding process becomes longer
06:24:15 <kenneth> hdv: I've seen in a lot of design systems that they're used as an opportunity to explain to people how to build accessible stuff, which can be really cool if it's done well, e.g. with annotations
06:24:27 <kenneth> ... starting to think a group note on this subject could be interesting
06:24:45 <kenneth> ... a lot of people are starting to evaluate their design systems as well as their websites
06:25:04 <kenneth> Sayaka: We tend to put everything in the design system in Figma
06:25:11 <kenneth> LenB: which is not accessible, so half my team can't use it
06:25:22 <kenneth> ... users end up having to pair up to get into it
06:25:40 <kenneth> q+
06:26:27 <kenneth> Ben_Tillyer: I know there's some things going on with ARRM; if components can be responsible for an issue, then a role could be responsible for an issue as well
06:26:41 <kenneth> ... if ARRM finally gets mature and published, maybe it would be useful for organizations to put them in different buckets
06:26:52 <kenneth> hdv: Great suggestion to see if we could embed or cross-link ARRM
06:26:58 <Ben_Tillyer> Link to Accessibility Roles and Responsibilities Mapping https://www.w3.org/WAI/planning/arrm/
06:27:04 <kenneth> ... we don't describe accessibility problems, we say you have to write down the results of your evaluation
06:27:14 <kenneth> ... but that's something on our backlog, to say what is a good way to describe what's wrong
06:27:31 <jeroen> scribe+
06:27:32 <kenneth> ... could say to write down whose responsibility it is, can help resolve the problem quicker, if you know what department to assign it to
06:27:47 <LenB> q+
06:28:26 <jeroen> Kenneth: Curious if can of worms was found after chosing for view instead on page
06:28:47 <kenneth> hdv: We've gone for view everywhere, trying to align with what we agreed on in WCAG 3, but I think we might need to correct to page/view in some cases
06:29:10 <kenneth> ... I think how we've done it is used "sample" to refer to both of those as a more abstract way of talking about it, but that may be making it more complicated
06:29:23 <kenneth> ... a lot of evaluators we talked to said they use sample or sample set instead of web page
06:29:41 <kenneth> ... if you're doing a lot of app audits, you might not be talking about pages ("screens")
06:29:52 <Daniel> q+ to ask about the WCAG-EM report tool
06:30:00 <kenneth> ... if we turn this into JSON or whatever, it's going to propagate elsewhere, doesn't matter what the category is called
06:30:13 <jeroen> ack kenneth
06:30:15 <kenneth> ... what's important is we use the more abstract term, then people can use whatever term they want as long as they're following the same process
06:30:26 <hdv> ack LenB
06:30:59 <kenneth> LenB: Has anything been written on what to do with it? Or is that outside our scope?
06:31:23 <kenneth> hdv: In some cases regulators ask for it. e.g. countries in Europe
06:31:36 <kenneth> ... people can upload it within their own domain, use it within their own system
06:31:38 <Ben_Tillyer> Link to WCAG-EM report tool https://www.w3.org/WAI/eval/report-tool/
06:31:50 <kenneth> ... We have the WCAG-EM report tool that people can use as well
06:32:00 <kenneth> ... I guess that's the closest we say to what you can do with it
06:32:11 <kenneth> ... we've also added a link to this report in the newest version (not in the current version)
06:32:19 <kenneth> ... the tool was built after the spec was written
06:32:20 <hdv> ack Daniel
06:32:20 <Zakim> Daniel, you wanted to ask about the WCAG-EM report tool
06:33:00 <kenneth> Daniel: So inside W3C we produce standards, we don't tend to get into regulation, but that's not to say we can't publish documents helping to explain how regulations can be framed around the specs
06:33:39 <kenneth> ... @@ wondering if you could share this as well
06:33:50 <kenneth> hdv: Yes, it's not in the current charter, we're currently updating the process
06:34:30 <kenneth> ... I've worked with the current tool; we don't need to touch the dangerous moving parts much, but would love to update the report tool in tandem
06:34:50 <kenneth> Daniel: I think this was under EO back in the day, so that's probably why it's not in the AGWG charter
06:35:05 <Ben_Tillyer> EU model accessibility statement which mentions "The [non-compliance(s)] [and/or] [the exemptions] are listed below" link: https://eur-lex.europa.eu/eli/dec_impl/2018/1523/oj    UK model accessibility statement which mentions "[List the non-compliance(s) of the website(s)/mobile application(s), and/or, describe which section(s)/content/function(s) are not yet compliant]."
06:35:08 <Ben_Tillyer> https://www.gov.uk/guidance/model-accessibility-statement
06:35:09 <kenneth> hdv: I think it would be great to do, either people from W3C or from our little group
06:35:20 <kenneth> ... we've had some other people reach out that would be happy to work on it
06:35:33 <hdv> q?
06:35:38 <kenneth> ... also need to think about transition between WCAG-EM 1 and 2. There will still be people who want to use the current tool
06:35:39 <Sayaka> Sayaka has joined #wcag-conformance
06:36:18 <kenneth> Ben_Tillyer: For @@2 we're expected to put the findings we would ostensibly get from WCAG-EM
06:36:37 <kenneth> hdv: There is nothing similar; people build their own tools
06:36:53 <kenneth> [mention of German and French tools / procedures]
06:37:05 <kenneth> hdv: any last thoughts?
06:37:41 <jeroen> Link to BITV (German evaluation method): https://bitvtest.de/en/
06:37:55 <kenneth> ... this session is not the only thing. This work will continue. If you think of things in the coming days/weeks, can file issues on GitHub or join the slack channel #agwg-wcag-em, or email any of us
06:38:15 <kenneth> ... very much open to ideas of how to make evaluation better
06:38:26 <kenneth> ... thank you all for attending this session
06:38:31 <kenneth> RRSAgent, draft minutes
06:38:33 <RRSAgent> I have made the request to generate https://www.w3.org/2025/11/12-wcag-conformance-minutes.html kenneth
06:39:43 <Ben_Tillyer> s/@@2/the Public Sector Body Accessibility Regulations (UK)
06:42:28 <kenneth> s/@@/regarding any update to the EM Report Tool,/
06:42:28 <kenneth> RRSAgent, draft minutes
06:42:29 <RRSAgent> I have made the request to generate https://www.w3.org/2025/11/12-wcag-conformance-minutes.html kenneth
06:42:43 <Ben_Tillyer> Ben_Tillyer has left #wcag-conformance
06:43:25 <kenneth> present+ Remi, Sayaka, LenB
06:43:29 <kenneth> RRSAgent, draft minutes
06:43:30 <RRSAgent> I have made the request to generate https://www.w3.org/2025/11/12-wcag-conformance-minutes.html kenneth
13:38:27 <tidoust> tidoust has joined #wcag-conformance
13:38:30 <tidoust> RRSAgent, bye
13:38:30 <RRSAgent> I see no action items