Meeting minutes
<Chuck> agenda+ WCAG 3: Use Cases https://
I could scribe for the first half
Chuck: Do we have intros or new to group or new role?
Chuck: Any new topics? or suggestions? We do keep a list - we review and try to add to future conversations.
<Chuck> https://
chuck: Announcements - TPAC registration is open
Chuck: Also sent out email with unofficial guidance on travel to Canada - some suggestions because of pandemic precautions
<Chuck> https://
Chuck: Inclusion fund link pasted.
Chuck: want to reflect diversity of whole work - which leads to better representation and inclusive design - this fund aims to help to remove barriers to help people attend and contribute.
Charter: Decision policy https://www.w3.org/2002/09/wbs/35422/charter_July_22/
Chuck: You can make an application or encourage others to.
subgroup interrelations
Chuck: Bonus agenda item - subgroup interrelations
Wilco: Questions from mailing list - conversation and time for thoughts and comments on how subgroups relate. We are starting subgroups - run for 8 weeks. Focused at generated prepatory work for TPAC.
Wilco: Will have that conversation at TPAC - to build up more understanding of terminology and generate ideas. Questions around overlap for those groups.
Wilco: currently 4 subgroups - test types and terms - proposed terminology on how to talk about tests. What is subjective and objective - some issues on proposed types.
Wilco: Equity group - focus on how to make sure different disability types are balanced out.
Wilco: Accessibility support - will focus on user agents and assistive technology fits into conformance model.
Wilco: Issue severity will figure out what options we have with severity and impact. It's ok if there is overlaps. Only 2 months away from TPAC.
Wilco: 2 or 3 subgroups have already started.
Wilco: Each group does have a chair or editor on it. Any questions?
Wilco: reach out if you are interested.
Charter: Decision policy https://www.w3.org/2002/09/wbs/35422/charter_July_22/
<kirkwood> could you repeat where to reach out to, wilco?
Chuck: First item is moving the charter forward. Had reviewed but not had approval for decision policy. Put out survey.
Chuck: Will go down list on commentary. Gundala - 2 days is to short - maturity is defined by consensus - first states exist as well.
<AWK> +AWK
<Ben_Tillyer> +1 to Gregg's comment
Chuck: Gregg - suggest step before CFC is called - chairs issues last call to see if there are any major objections. Language changes continually - members need to take final list and it's hard to keep reading the same thing over and over and to know if final comment made it in. a final call should be issued.
Chuck: Gina had said no changes needed.
Chuck: MaryJo - a documented working group resolution should be stated - CFC should take into account amount of content and holidays - 48 may be reasonable if small - a week might be better for larger changes.
<Zakim> alastairc, you wanted to respond to Gundula, 2 days and levels of maturity.
Alastair: In terms of 2 days - that is a stated minimum - generally, we try to give more than that. If we put in a week - that takes you from Wednesday to the next Thursday - that might be suitable. We try to take into account holiday- we can add ifs and buts - but prefer not to add those into decision policy.
Alastair: If we are deeming it as not mature content - like placeholder - we have 3 levels - those are the ones that we are saying in the policy - in the paragraph in bold - what we are saying we aren't doing CFC - getting broad agreement in meetings in whatever tool we use - we record issues and concerns. It's when we get to maturity that's when we need the CFC process.
Alastair: We generally send out email telling people we will be sending out a CFC go review this - but it depends on what has been discussed - the policy is meant to be a minimum.
Chuck: our current policy has the same language regarding time. Has the group been comfortable with the time we have given? We have had extra communications when appropriate. Since this is similar language are you comfortable with how we handle it.
<shadi> https://
Shadi: Not comfortable with the binary are you ok with it - but the time may be too short - but what I wanted to share - experience from EOWG - it may look more complicated - but I found this to be very useful with different stages with the review - with different durations and a process for minor changes.
Shadi: Does look more complicated the way it is written (wiki) then in practice.
<Ryladog> +1 to Last Call in email subject
Gregg: Put last call in subject line - when I get 300-400 emails a day - it's hard to sort out from something at middle or end things like that. Something like that in the subject header.
Gregg: Sometimes abstain as something could be changed but don't want to disrupt at the late point. Don't require folks to open every email and figure out what level it's at.
<kirkwood> +1 to LAST CALL in subject, I’d suggest in ALL CAPS might help as well
Chuck: Do you (Gregg) feel that it should be articulated in decision policy. Could we implement that would putting it in writing?
<jeanne> -1 to add more complications
<alastairc> I suspect this will be more relevant when we have more mature content, we won't be doing CFCs on mature WCAG 3 content for a while.
Gregg: If it's not in policy it may not be followed for very long - it makes me - listening to Shadi - someone is fixing a typo to have a mechanism for that -don't know - if we don't have it there it won't happen. Like some flexibility to the chairs.
Katie: 100% agree with Gregg - people come and go - things come and go - so it's important to have the text in there - we don't need a million other things - that has come up in past.
<Zakim> jeanne, you wanted to object to making things more complicated
Jeanne: I want to strongly object to add more complications to decision policy - we already have heavy weight policy - adding another complication - strong object. Every time we come up with streamline - people add things to it. Can we just approve it and not tinker more.
<JF> +1 to MaryJo
Jeanne: MaryJo - my only concern - and I understand where you are coming form - sometimes the decision policy is also adopted by subgroups - issue with a lot of content come through with only 2 days with CFCs- despite saying something. That's the only thing I am worried about. People have other jobs as wel.
<Wilco> +1, 2 days is not enough
<AWK> +1 to Gregg
Gregg: I am sympathetic - this is the reason for the last call - as we are tired of talking about - we call for a CFC - and people want to talk about it - people didn't know we were calling for consensus - purpose is to not dealy but to speed it up.
Gregg: People sit back and let two people debate it out and now we get a call for CFC - that's the sort of reason to add the last - facilitate so we don't have objections during CFC - so we don't need to have multiple CFCs. So we can avoid future CFCs.
<JF> +1 to Gregg
Alastair: Taking a step back - I don't think we actually have a problem here - this applies to mature content we are bringing to CFC - we have already gotten through 3 levels that we didn't need a CFC.
<GreggVan> +1
<Chuck> +1
Alastair: Notify folks that something is coming up - so people have more time to review - with all of the subgroups as no one will know all of the details.
MichaelG: assuming that holidays fall into 3 day week - just make a CFC 4 days - that way you would not have an issue with holidays. Not sure of impact - but offer to raise bare minimum.
<Zakim> Chuck, you wanted to ask Alastair a question
Chuck: Were you thinking that it's not really an issue and that we didn't need to update - or are you saying this would be a minor update and it's ok?
Alastair: It would be a minor optional step. In terms of subgroups using it - we need to be clear to subgroups - subgroups don't have to stick to heavy policy and wouldn't take on whole policy.
<janina> FYI: We have a maintained calendar of hollidays that can help us avoid scheduling expectations during hollidays. Link is:
<janina> https://
Alastair: have made an amendment or suggestion and then we can do a final call.
Chuck: Will be people who want to see the text.
<GreggVan> +1 to 4 days
<iankersey> +1 to 4 days
<Chuck> Poll: Extend CfC minimum to 4 days?
Alastair: We could do quick poll for 4 days for a CFC - generally we do aim for 3 or 4 days. It just means it would be hard to a fast turnaround for an emergency CFC. e.g. before publication.
<Francis_Storr> +1 to JF
<MelanieP> +1 to 4 business days
<maryjom> +1 to 4 days
<Chuck> Poll: Extend CfC minimum to 4 business days.
<Ben_Tillyer> +1
<Ryladog> +1 to 4 business days
<GreggVan> +1
<Francis_Storr> +1 to 4 business days
<jweismantel> +1
<GN015> +1 to 4 working dates
<jeanne> -1 to extending the time. Emergencies happen, but are rare
Alastair: Are people have with 4 business days? That would be minor update - and then add in an optional final call - I will send around and then review and then CFC.
<JF> +1 to 4 business days
<Makoto_> +1 to 4 days
Awk: When we do CFC - then questions come up - then we reset to resolve and then CFC - might be worth clarifying what can be resolved in CFC period - as it gets longer.
awk: If it's addressed during a CFC - does it go back to day 0 or go back to inline.
Chuck: Echo some good points that Jeanne makes - if we add to many branches and conditions then we make it overly complicated - just give chairs flexibility to make the calls in the moment that are appropriate.
Alastair: on Andrew's point - we have been taking - such as a minor thing on the CFC - we updated and send to list and then ask people if that impact previous +1 then let us know - we are happy to do minor things in that period. If it was more major - we would need to reset the process from survey to re-CFC.
<JF> s/Echo some god points that Jeanne makes/Echo some good points that Jeanne makes
<Zakim> GreggVan, you wanted to say "leave restarting the CFC up to chairs based on impact of change. Yes to what Alastair just said
Alastair: We should have a shorter time for a CFC if it's the 2nd time around.
Gregg: Don't define it to rigidly in case it's done over - the last call period would help the CFC to be shorter.
<Chuck> ach mich
<JF> +1 to that IF we accept the 4 business days proposal (that dreww many +1's)
MichaelG: One suggestions to bumping it to 3 days - 50% more time - could still take care of extended multi-day holidays - gives more flexibility to chairs.
Janina: We require minimum 2 business days at APA - we say we prefer a full 5 business. There is a holiday calendar that we can track that yearly.
<alastairc> Hmm, who's business days? That will vary by country!
JF: If the group does accept 4 day policy - that way if we are in mid-flight - that's the point where we need flexibility - so I could support a standard CFC has 4 day minimum - important that we give everyone time to review. Stuff is done in small subgroups - and we need to build into more review time before it gets to CFC - I know this about mature -but it applies to all transitions.
<Zakim> Chuck, you wanted to say there was support for 4 biz days.
Alastair: 2 business days will vary by country. There was support for 4 business days.
<Chuck> proposed RESOLUTION: Accept the amended draft decision policy, and move forward to CfC.
<Chuck> +1
<ShawnT> +1
<michael> +1
<alastairc> Suggested text: The Call must remain open for a minimum of four working days. If a CFC is re-started, it can be closed after 2 business days.
<iankersey> +1
Alastair: Should I put in text to show amendments.
<alastairc> When the Chairs believe that the group is ready to come to a decision they will propose a resolution for a vote using the Call for Consensus process. Optionally, a 'final call' email can be used to prepare the group for the upcoming CFC and capture any outstanding comments. The decision is recorded as a resolution.
JF: could that lose a day?
Gregg: doesn't mean it would be short.
Alastair: not aware of when we closed a CFC early.
<Ryladog> +1
Alastair: going to amend a little bit.
<alastairc> After the CFC, the decision is recorded as a resolution.
<GreggVan> +1
<Ben_Tillyer> +1
<jeanne> 0
Chuck: yes, we are plusing both amendments.
<JF> +.75
<laura> +1
<MelanieP> +1
<ShawnT> +1
<Makoto_> +1
<Ryladog> +1
+1
RESOLUTION: Accept the amended draft decision policy, and move forward to CfC.
WCAG 3: Natural Language How To https://www.w3.org/2002/09/wbs/35422/2022-07-14-WCAG3-Natural-Language-Howto/
Chuck: move on to next questionnaire - to add changes to natural languages as WCAG how to.
Chuck: Test reliability subgroup has updated - please review - do you agree to publish the how to - with changes, or disagree with publishing the how to.
Chuck: From Jeanne - This complete the examples of test reliability subgroup - not related to test reliability - the how to is rough equivalent to understanding document.
Chuck: another responder - overall like text - but had some question about support.
ShawnT: Wanted to keep some things from prior WCAG content regarding exceptions.
Chuck: From Gregg - the whole thing about changes in languages - in the body of the text - same thing raised - words that have been adopted, proper names - the whole things about changes - but then it says that you need to make the language of whole page - as it's not on topic - this is about changes in the body and not about the whole page.
Chuck: From Wilco - The how it has to work text doesn't belong there
Chuck: From Makoto - we should make description more technology neutral - e.g. lang attribute as WCAG 3 may cover other technologies.
<JF> +1 to makoto
Makoto: want to make sure we must be technology neutral.
MaryJo: wasn't sure if this how to is exploratory or what? What level is it at? At each tab I had some things that struck me - different aspects that weren't clear to me - things that wouldn't stop - but I would enter them as issues.
JF: disagree to publishing to the how to - MaryJo and Wilco covered some of my concerns - with you ask to publish - what category? In terms of looking at this - it says it's complete - but it's not complete - it hasn't been filled out. Disagree that it is complete. Find it has nothing related to testing as it's coming from test reliability group. Agree we need to have something for language. No cross reference to WCAG 2.x as we migrate we s[CUT]
<Zakim> janina, you wanted to discuss lang pronunciation
<Zakim> jeanne, you wanted to explain exploratory
Janina: inline lang for pronunciation - I except we will have a specification - you would expect orthographically correct behaviors - that is increasingly a problem with speech based voice assistants - so we can normatively reference by 3.0
Jeanne: This is exploratory content - the reason it exists - the test reliability subgroup is tasked with finding a way to write more specific tests and more measurable outcomes - we did that work and it was accepted and then we were asked to write an example. This is an example of the process we created for the outcome and method. The only thing that was missing was the how to - the understanding equivalent.
Jeanne: We wanted the example complete - it's the explanation and not a testing document. When I said the work is complete - let's put some text in the how to - some day in the future - a group that is specialized in natural language - they will rework this - this is example so there is not a broken link when it's connected. I don't think the test reliability group is the right people. We should address the placeholder. This isn't the time[CUT]
<Zakim> AWK, you wanted to ask a couple of questions as I wasn't able to get to the survey.
Wilco: There are methods written that describe all of the testing that we wrote - there is more cross linking to the methods that is needed.
<AWK> oine sec
Chuck: in 5 minutes we need a scribe change.
<JF> +1 to AWK - if we have that kind of research, it should be linked/provided somehow
awk: Didn't get to this in advance - trying to find methods that relate to this - first question is where are those? Don't see anything related to changes in natural languages in editor's draft. In light of Jeanne's comment - very surprised to hear this single words or short lanuages don't need a change and that frequent. That makes this less testable as it applies more human judgement. Different from WCAG in terms of expectations. If tha[CUT]
<SuzanneTaylor> +1 on the unexplained differences - I think I made this comment the last time we saw this text
Wilco: The how to is what we have already said in outcome - we will need to have those further discussions later - file an issue.
<Chuck> proposed RESOLUTION: Accept moving Change of natural language how-to to exploratory
Chuck: don't see anyone in queue and Jeanne has given context - and no amendments - will propose a resolution.
<AWK> Still interested in the location of the related methods
<alastairc> Do Wilco / jeanne have a link, or isn't that published yet?
Chuck: Yes, still interested in location of related methods.
Wilco: not sure of a public link to methods.
<alastairc> If it doesn't move up in 6 months, it gets removed.
JF: Of the 3 points - 1 is being qualified in proposed resolution - move this to exploratory. We make micro decision and we will talk about the rest later - when is the later? is there a proposal that once it gets into exploratory does it get worked on? In some cases we haven't actually done exploration.
<Wilco> AWK, here's the method text; https://
alastairc: Exploratory content need review and agreement, or updates from a sub-group. Otherwise there's an exit mechanism if it doesn't continue to be explored.
<Zakim> Chuck, you wanted to call for scribe change
alastairc: We will keep returning to things.
<Chuck> proposed RESOLUTION: Accept moving Change of natural language how-to to exploratory
JF: Wilco provided a link to the content that is missing. Will it be integrated into this? The current document isn't complete.
<sarahhorton> +1
<Chuck> +1
<Ben_Tillyer> +1
<jweismantel> +1
<jeanne> +1
+1
<laura> +1
<michael> +1
<Ryladog> +1
<JF> 0
<alastairc> +1 (with the placeholder is updated/removed)
<Makoto_> +1
<Wilco> +1
<MelanieP> +1
<ToddL> +1
<AWK> +1
<Wilco> Here's the method in WCAG 3 design btw: https://
<ShawnT> +1
<Lauriat> +1
+1 if the link to method will be in it.
RESOLUTION: Accept moving Change of natural language how-to to exploratory
<AWK> @wilco, should the how-to be named to match the outcome name ("Language of Text")
WCAG 3: Use Cases https://raw.githack.com/w3c/silver/use-cases-jul-22/use-cases/index.html
<iankersey> +1
<alastairc> Also, I take JF's point about the survey saying what level the content is for. For the near term everything will be exploratory though.
<Wilco> @AWK, that's the method name, the outcome name is used in the how-to
janina: We took into consideration the feedback from a few weeks ago and help from edits from Gregg. These are the tricky use cases.
<Wilco> https://
janina: These are the use cases that should be taken into account for WCAG 3.
GreggVan: We say they should be addressed "somehow" so are not necessarily to be addressed in the standard, but instead in policy.
<AWK> @wilco, the phrase "changes of natural language" doesn't appear in the doc you sent me the link to
GreggVan: We'll have to work with Judy to ensure we don't go beyond W3C's ability to make any statements about policy, as we aren't a lobbying group.
Chuck: You have made changes based on input. There is no survey, so what are next steps.
janina: We have no next steps right now other than discussion at TPAC. Sub-groups starting are taking precedence over this work.
<Zakim> alastairc, you wanted to suggest we (in the sub-groups) use them
MGower: Concern about avoiding compliance in the language.
alastairc: One of the goals is to use it as a measuring stick for other work. Message to facilitators to take a look at these situations and run through some examples and see the impact on things like issue severity (for that group).
GreggVan: In too many places we're repeating the organization "is doing best efforts". We're trying to get to the point where there are indeed these situations to keep in mind.
<Ryladog> Maybe a note about helping the users understand about unintended consequences
<michael> Yep, I understand. Will do PR. thanks!
janina: Pull requests are welcome, but we won't get to them until after TPAC. We wanted this document available for TPAC discussions.
SuzanneTaylor: Should the title be more qualified rather than just "Use Cases" and will you consider adding other use cases?
WCAG 3: Use Cases https://raw.githack.com/w3c/silver/use-cases-jul-22/use-cases/index.html
janina: Feel free to open issues or pull requests. We won't process till later.
Shadi: How do we handle if the sub-group is closed?
WCAG 3: Conformance architecture testing https://docs.google.com/document/d/1dingDd116FVx0QuxCemgHbReJfNxMZRSF1q3dJ9Uj5U/edit#
alastairc: We will have to queue them up till later.
<Chuck> sure, yes please
jspellman: We are recapping for many new people to AGWG since 2020 and want to raise awareness of what work has been done already.
<jeanne> slide deck
jspellman: We did conformance evaluation of the WCAG 3 FPWD. We did a research paper and a report on web accessibility metrics based on a symposium that the W3C did in 2011
jspellman: This was done by a group of researchers and others and they wrote a report based on the work done in the symposium.
jspellman: We came up with this document for testing/validating Conformance as a model. They recommended 5 measures.
jspellman: Going through slides 4-5. We haven't yet decided on the actual scoring but can check it against the 5 measures. We can analyze if formulas for scoring can stand up against the various metrics.
jspellman: 5 metrics are: Validity, reliability, sensitivity, adequacy, and complexity.
<JF> +1 to defer queue to end of proposal
jspellman: Validity is most important - does score reflect the score the digital product SHOULD get?
jspellman: We want to have web pages, external experts individual feedback, and test the scoring to ensure it holds up. Then open to outside testers to try.
jspellman: Reliability - reproducible results. Details are in slide deck, read aloud by Jeanne.
jspellman: Read through slides 6, 7. Adequacy - this is important and actually pointed out some issues in the FPWD.
jspellman: Read through slide 8. Complexity - we can ask people for data on how long it took them to test and other thoughts on test complexity.
Francis_Storr: We have made an accessible escape room site in GitHub. We have Deque's website that has content in multiple languages and we got permission to use that site.
Francis_Storr: There's a less-structured set of pages to cover things like login considerations. We found examples to test a few of the requirements against.
Francis_Storr: Gov UK created a series of test pages to audit automated testing tools. 142 test cases and tested headings, images and color contrast
Francis_Storr: Also checked a medicare page which was pretty good.
JF: I think this will be really useful. Is there any test based on disability type?
JF: When we ask about validity, valid to whom? E.g. Captioning - depending on the audience, are verbatim captions or edited captions preferred?
JF: There are different answers depending on the audience.
<JF> Caption Key notes the following, which will need to be decided upon: "A re-occurring question about captioning is whether captions should be verbatim or edited. Among the advocates for verbatim are organizations of deaf and hard of hearing persons who do not believe that their right for equal access to information and dialogue is served by any deletion or change of words. Supporters of edited captions include parents and teachers who call for the editing of [CUT]
GreggVan: Validity is ONE of the most important. Reliability - if it isn't there can undermine everything.
<kirkwood> +1 to JF point
GreggVan: Inter-relater reliability should be more specific. Suggest using "testers" instead of "people" to differentiate between those who test and those who use the content being tested.
shadi: What are the recommendations we can make now that we have these criteria. Granularity makes a difference to the structure of the tests. Less granular, more impact it has on reliability if there's a mistake made.
<JF> +1 to shadi
<Zakim> jeanne, you wanted to say that this is not about individual guidelines and to ask how they recommended incorporating granularity? Does it need to be a new metric?
shadi: How you design a requirement (e.g. someone cannot use a mouse) vs. how you test it and equating to functional needs can vary based on test granularity.
jspellman: We are looking at this model to gauge overall scoring, not scoring of individual requirements.
JF: we have conflicting requirements based on disability type. So when we talk about validity, functional needs based on disability type is in legislation today but this document doesn't seem to address this aspect.
<jeanne> +1 Chuck
<Zakim> Chuck, you wanted to echo my understanding at a meta level, while specific guidelines may have been used, this exercise was to evaluate at a higher level
<Zakim> shadi, you wanted to respond to JF
Chuck: Less about specific criteria, and more about checking validity of the overall model.
<jeanne> +1 Shadi. Different groups worked on those issues. We could present on those on another day
shadi: Designing the requirement based on user need will be important.
<Zakim> jeanne, you wanted to ask about presenting that work?
JF: There may be conflicting user needs or ways to solve the user needs. There isn't any reference to disability type as an overall metric which is the concern.
jeanne: We had a different group working on that. Analyzing the different aspects of conflicting requirements and the equity group is going to look at that. We could wait till the equity group presents their work.
<Zakim> Chuck, you wanted to ask about next steps
jeanne: Several years ago there was work done on this with models developed.
<mbgower> Great work! :)
WCAG 3: Categorization exercise
jeanne: We are focusing on having this as a resource for the sub-groups to utilize.
<Chuck> https://
<jeanne> Francis did the vast majority of that work. Kudos to him
Chuck: We have 30 criteria that look like they aren't complete yet. There are a number of breakout rooms to work to complete some of them.
jspellman: We are really close to getting done. Majority are done, several are in progress. We have a template to work from.
jspellman: If you have a 'Partial" with your name on it, please finish it. Any "partial" that have a group on it, please help us finish them.
jspellman: Would like to have them all move to a "Yes" by the end of the meeting.
<jeanne> Folder of all the Google Docs for this exercise
<Chuck> scribing concluded to facilitate the categorization exercise.