Meeting minutes
https://
updates and actions https://docs.google.com/document/d/15HtPkkYx1CIl6bAwP2nsSZKhqTVbqcuMDRz5RmtmvXg/edit#heading=h.1bvszq5s0esc
Lisa: We will cover functional needs later today.
… EO update?
Julie: I had an interaction with Kris Anne. She helped me understand the best way to give feedback.
… I haven't finished that Google Doc yet, but that is the next step.
… This includes how they revised the User Stories. This will help us state how we specifically would like things reworded.
Lisa: Guardianship draft?
Michael: It is on my to-do list to schedule a meeting.
Lisa: Images?
https://
https://
Jennie: We have added the meeting minutes to our subgroup wiki page
… I would like to update the KPIs with suggestions - is that ok?
Lisa: yes
Rain: The group is on the agenda for June 16
Rain: I think Rachael was interested in meeting with us regarding testing.
… for Clear Language
… Rachael had written to us. The Silver group would like to work with us on a test
… This may be a good time to schedule that.
<krisanne> John K and I seem to be in a different meeting space. Is there a new zoom link?
Rachael: I think you are well on your way after reviewing the document - nice job!
… In the next 2-3 weeks, if we could meet. If we could develop a few examples of what a contextual or conditional test would look like.
… This would be a different test in different languages. This would be a good example.
… When can I meet with you, and how can I help?
… Also, I am trying to put a group together for Visible Controls. I think this is also a good example of a conditional or contextual test.
Lisa: Timeline for group on visual controls?
Rachael: It is not a timeline for 2.2 - no urgency. But it is a good example of something for 3.0 to explore test types that may address COGA related success criteria.
… I defer to you on schedule.
Lisa: You would like something in a month?
Rachael: Correct.
Lisa: Our submeeting after this is a subgroup on testing.
… We normally will have it at the same time as plain language. Sometimes we will be together, but this week is just the testing.
… I think that would be a good time for you to attend, even if it is just to get the scope defined.
… Beyond that, the input of "these are the kinds of tests we need"
… I think that will be helpful for us. Using the right terminology, and the right concepts.
… Can you add the different tests you want? The terminology you want for visual controls?
Julie: I have 2 questions for Rachael:
… 1: should we follow the template that has been used that other groups have sent around for CFC?
… The template for methods and testing seems to have been updated.
Rachael: Yes, there is an updated template for methods
Lisa: To help reference them after the meeting to post them are the subgroup page for clear language
<Rachael> Exploratory test section: https://
https://
Lisa: and the subgroups page for testing.
https://
Lisa: I need the links to the different forms of tests
<kirkwood> sorry was in wrong room
Lisa: That would be useful in both places
Julie: Rachael - I believe the group worked last week on avoiding double negatives
… We thought that would be easier for developing tests.
… It is not easy, but may be less challenging. Do you agree with this approach? Starting with something less subjective, and dependent on user testing.
… These could potentially be automated.
Rachael: I think that is a great approach.
… We get push back about Clear Language. I think having a conditional test, with a pre-requesite (language)
… then yes, creating objective check points.
… It would move us forward in a productive way.
… It does fit in with trying to get this kind of test visible to the Accessibility Guidelines Working Group
… It is in line with a protocol test.
… Having a couple of different conditional tests in terms of Clear Language would help address something we have struggled with
… and if it demonstrates we need to try something else, that would be helpful too.
<Rachael> That is fine.
Lisa: I would like to make this an agenda item next week or the week after.
Rain: On June 30, we had planned to have the subgroup meeting be combined Clear Language and Testing.
Lisa: I think it needs to be in the main group. It reaches everything.
Rain: ok
Lisa: reminder to mark yourself present in IRC
<Rachael> I was on queue to say that I have a conflict directly after the meeting so we'll have to find a different approach.
Lisa: Le - are you ok with accessing the research paper?
Le: I hadn't seen that, but thank you.
Lisa: John K?
John K: yes, all has been reported already
subgroups https://www.w3.org/WAI/GL/task-forces/coga/wiki/Subgroups also See timeline at https://docs.google.com/document/d/1JXa94s2lbzJ0v9FHasxxws3CsOcljHHBdlQ2VOxYqAQ/edit
Lisa: Reminder - we have new subgroups.
… To help us meet our timelines
https://
Lisa: People have signed up for the new subgroups
… We don't want more subgroups until we close some of the existing ones
… If you would like to be on more, please let us know.
… We had a subgroup meeting that clashed with Language, but we sent the meeting minutes to the list.
https://
reserch plan https://
Lisa: hopefully more people can attend.
Lisa: The most interesting bit: the research questions we would like to add to Rain's work.
… The issue papers, the research modules, and the gap analysis
… We know we want to update the mental health
… Deciding to have things on the wiki, vs W3C publications
… How important is it that they get updated?
… I know people missed that meeting, but if anyone feels there is another question to be added, please comment on this list or now
… Any other subgroup updates?
We will cancel the July 14 meeting because of the FTF on July 12 - https://www.w3.org/WAI/GL/task-forces/coga/wiki/Meetings/June-2022#Coga_TF_F2F_Meeting
Lisa: July 14 - we have a hybrid face to face.
… The meeting will officially be on July 12, so we won't be meeting on July 14
Lisa: what topics should we discuss at the face to face?
close item 3
What meetings do we want at FTF? Clear Language And :) mental health Note: mental health has its own meeting on Friday - https://www.w3.org/WAI/GL/task-forces/coga/wiki/Meetings/June-2022#Coga_TF_F2F_Meeting
Lisa: We already had a request from Rain to discuss Clear Language and the tests
… We are starting the analysis on mental health, so if there are proposals for new patterns
… The research plan might be good to do face to face
<Zakim> Rain, you wanted to add data analysis for structure
Rain: If we are able to succeed in getting the survey out in the next week or so, then it might be good to have the structure group to do an analysis based on the feedback
Lisa: yes
… We also asked Michael if there is any analytical data that would be available to us.
stakeholder feedback survey questions . See https://docs.google.com/document/d/1lW-ds-UJkQU_SR0RXA9dnR7tjWlLkzPGQC_RBA0S5Vc/edit#heading=h.k2uu05tz7cki and survey at https://docs.google.com/document/d/1n_pV-lbgorW2oihIB4EdONOFN0EcRmBAkmUEEPdTAEw/edit#heading=h.jsq2b4dfhi5h and
Rain: The structure subgroup came up with a data collection plan to try to understand how well Content Usable is working, and the barriers to use
… A quantitative survey
… The COGA community group has been great coming up with what that survey will contain
… I was advised to keep it as concise as we can
… Attrition falls intensely when it is long
… Ask a question at the end, asking if they would be willing to follow up in a more qualitative form
<kirkwood> Great feedback from the community group!
Survey draft: https://
Rain: The survey in its current state - with huge input from the community group
… This is a draft of what will be in the survey
… I have also asked a quantitative research to review it with me next week
… To make sure we are not leading people
… That we are not violating ethical rules, etc.
<kirkwood> How would you like feedback?
Rain: Lisa has also suggested questions about research.
… Very curious to get input from this group.
Lisa: We need the same kind of questions for the research documents.
… Maybe we could have one question, such as "do you use other COGA documents?"
… Then if they say they are available for follow up, we can connect with them about the other documents.
Rain: Great idea
… I'm adding that right now.
Lisa: "And which ones?" Then we know what to ask when we follow up.
Rain: I am adding that right now
Lisa: I think we need a 2nd survey then, possibly to other groups. Example: developers
… so we can focus our efforts where they are most useful
Julie: Rain - I am looking at this - how would you like us to give you feedback?
Rain: Comments in the Google doc is probably easiest - it would be in context.
… Then I can convey that to the subgroup
… You can also add suggestions in suggestion mode.
… This will identify you, then we can follow up if we have questions.
Julie: There are a lot of demographic questions.
Rain: Most were added today on the community group call - they felt it would be helpful.
… The roles, and how often do you use Content Usable were there before the Community Group call.
… we added them for now, and can pare them down later.
Julie: I see there are a lot of yes or no about people's experience with Content Usable.
… I wonder if "how easy" or choices with a 1 through 5 on how easy it was to find specific things would be helpful
Rain: that's a good idea.
Lisa: I don't think we need the age.
… I think the demographic is not as important as the research questions. If we can add 2 questions about research papers, this may mean we don't need to do another survey.
<kirkwood> can we move demographic to the end?
Lisa: Some people may find the age questions offensive.
Le: I agree. Sometimes demographic questions can be overdone
Rain: I want to represent the Community Group on why they suggested the demographic questions
… I am in favor of removing some of these questions
… One member felt that if we are seeing responses under roles that say "yes I am an individual with life experience" and they are indicating they are over 65
… We now have some understanding of where there could be barriers within the document itself
<julierawe> +1 to Jennie's comment!
<kirkwood> well said
<kirkwood> +1
Jennie: I vote for 2 surveys to keep the research question get a separate focus
… And I think that asking the question about the demographics + lived experience can be asked a different way, and will address those that may not self identify as having lived experience
Lisa: Ok
<julierawe> +1 to Kris Anne's comment
<kirkwood> +1 to moving to end
+1
Kris Anne: I recently have seen some surveys where they have you answer their main questions first, and then they ask the other questions later.
… Example: we would be interested in asking questions about who you are, would you be interested in continuing yes or no.
+1 that's a great idea!
Lisa: On the same lines we could do the same with the research questions
… Then we haven't lost so many people
… With surveys, they should be able to end at any time, or save their responses to finish later.
(Jennie is +1 to being able to end at any time and submit, but finishing later too if possible)
Thank you, everyone, for great notes!
functional needs review - see https://docs.google.com/document/d/1ndRziXRfnyAgDaL8ctByQagDdM36H8QxV44lO3u8zgc/edit#
Lisa: this is being made by APA. We weren't very happy with their functional needs.
… Julie's idea was to start with a table so people don't have to look at multiple documents
… FAST is the group making these user needs.
<kirkwood> FAST?
Lisa: the first column is the big group, then what they have included, then what we feel is missing
… If there are items we haven't seen at all, we can add a new row
Lisa: I have added the information and link for FAST
… I will read out what is in each row.
… Before we leave we need to decide if we are comfortable, or need more time.
(Lisa reads from the document)
Julie: I would strongly recommend that if they split language and communication in 2 buckets, and I think we should explain why to prevent a lengthy back and forth
… Lisa - is it because you think it gets so long that people cannot focus on the information?
Lisa: I think when you say language and communication, people just see language
John: I think communication is problematic.
Julie: I think having a separate non language communication would be helpful
<kirkwood> +1 non language communication
Julie: Having it as a separate category
Jennie: Receptive vs productive communication is important
… what we do to communicate back, compared to our understanding of information
Lisa: yes. I think we need more time, and to have members to add information in
… But we also need more discussion
… We have a subgroup meeting in 5 minutes.
<kirkwood> break time!!
<kirkwood> ;)
Testing sub group
<ShawnT> Rain and lisa did you see my email about translating Making Content Usable in French?
Lisa: We have the KPIs of making an initial exploratory draft.
… And feedback from stakeholders.
… In this context: places like Silver and the rules working group
… I also think testers
… What's our plan? schedule?
… This works a lot with what Silver is doing
… Scope can be Content Usable, Silver, or both
… We have our own subgroup page
… I copied (on the Wiki page) the KPI/measurable goals
… We need a task leader
… We need to think about the scope, timeline, what questions we would want to ask about stakeholders.
… We don't have to do the same stakeholders as other groups
… We can make an example, and get feedback on that example.
… My first question: comments or thoughts?
Jennie: I have access to testers we could get feedback from
Lisa: Aaron said that clarity is important
… Having the wording from Silver about the different types of tests
https://
(the link Rachael shared during the COGA main meeting today)
Lisa: there are different types of tests.
… See section 4 - testing
… I will start building a matrix.
… (Lisa drafts in a Google Doc)
https://
(above is the document that Lisa is drafting)
Lisa: We have a section on testing in Content Usable
… It is in the "How to use this document"
… There is a section on testing
… This is a rough testing plan
… "Use" and "avoid" is testable
… Sometimes an explanation would be helpful
… We also have the testable statements that we made for each pattern
… (reads from the Testing Each pattern section of Content Usable)
<ShawnT> I'm getting 404 on those links...
https://
Julie: I am wondering if one group could focus on procedural testing, and the other focuses on conditional testing
* I was looking for it too ShawnT!
Julie: I think building out the procedure for testing something out - I am wondering how many will be similar for testing different COGA pieces
… It may need to be adapted for different methods
… When I asked Rachael earlier about the double negatives would be more similar to the previous 2x series
… This group focusing on those procedural methods may be helpful while the other groups focus on the more familiar style of tests
Lisa: Going back to our KPIs - we want an initial draft of what we are doing, get stakeholder feedback, and make a plan
… You are suggesting that there are the Silver group
<kirkwood> +1 to Julie… i think its valid to essentially break up policy and procedure to make managable
Lisa: doing the work per pattern
… and we focus on conventional testing
John K: I thought that was an extremely good point there. And I view this in that umbrella of policy and procedure.
… Policy is one thing, but I think people are looking to us to get procedures, I think focusing on that would be good.
Lisa: I have an additional option
… do we want per pattern? Or procedures?
… We have a whole section on user testing
… This should be part of our background reading (added link into the Google Doc)
… Julie is suggesting procedural tests in Content Usable
… The conditional tests are more in Silver, but may be linked
Julie: I think that everything might eventually be in Content Usable as a whole
… It might end up being that for different patterns, or methods in Silver
… We would have conditional and conventional tests in both Content Usable and in Silver
… They would be specific
… For a specific method
… If we focus a bit on the procedural testing - how do you do testing with a diverse group, enough of which have cognitive and learning disabilities
… You may not need a different procedural test for each one
… It may be the same kind of advice
… On the whole, when looking at user testing, it may apply to everything
Lisa: It kind of makes sense
… I think we should look at what we already have
… we made a developer resource page
… We even provided a sample consent form people can adapt
<julierawe> +1 to Jennie as good first step in our plan
Jennie: I propose ensuring we example what can be done (even pieces of a test) as more traditional testing, and document when this cannot be addressed any other way
Lisa: I see that subgroups working with Silver can work on conditional and conventional more than the procedural. We would work on the procedural. But Content usable has both - is that correct Julie?
Julie: I think seeing if we can follow some of Silver's procedural information for Content Usable. I think that will be a big chunk of work.
… I like Jennie's idea - either while starting this work, not sure how to map this out
… Figuring out what we can break off as conditional testing, and what needs to be procedural testing, is a good start
… I think it will be a lot of work to getting it into the Silver process
Lisa: their formats are not finished.
… Are their formats completed?
… We made several extensions.
… And there have been changes
Jennie: I heard Rachael say earlier that if the procedural pieces are not working for some of these components, that will be important to share
Lisa: I don't think that plain language is a good example to be working on for that. I think familiar things will be better.
… I think languageis much easier than Clearly Identify Controls and Their Use
… We can also test language, but this may be a good one to start.
Julie: I think the one you identified - that is a great one for talking about procedural test.
… The diverse group of testers - how do you test with people with cognitive disabilities, with people with dementia.
… More detail to build out there
… Some of this in Content Usable, and maybe expanding it.
… We did actually have things you can use.
Lisa: Yes, there are examples in there.
… We have got part of it.
… Then we have section 5
… Do I understand what things are, and then we have a procedure
Julie: That procedure there - does the user know what the page is about? What do we consider informs what is diverse? How many?
… We have some good questions?
Lisa: It is complicated.
… The answer depends a lot.
… While this is informative, we may be expecting people to put it into a procedure.
… It is important that big companies that pay for testing, that they include this.
… For smaller app developers, then can be other ways to complete this
… We should grapple with them
Julie: Looking at this page, at the end of the second paragraph, it says "try to build a group of users who..."
… A procedure could include flexibility
… If this is about user testing: are you doing user testing? As part of your recruiting are you asking about - then including questions about functional issues
Lisa: yes, like "do you give up when a website is getting too difficult for you"
Julie: From a procedural perspective - it is putting guardrails.
… At least getting them to the procedure of testing this, reaching out to different testers
Lisa: We are going to want to have a general procedure, a procedure by objective or user story, or outcome if that is the term
… (drafts in the document)
https://
+1 to that, thank you for calling that out
<julierawe> +1 to Jennie
Jennie: I propose that we consider talking about this at the planning and design phases, not just the coding stage
Lisa: I have added that to the document
Lisa: I have added it in the life cycle - because once it is live, that may have other components
Lisa: What do we think are the next steps to making this happen?
Lisa: Is anyone interested in chairing this subgroup?
Jennie: could we begin with the identify controls one
… And pull apart which pieces could be tested either with automation or manual, vs procedural
Julie: I agree, that would be a good way to start
Lisa: I have added the link in the document
… and I think we need to gather questions - and headings
… we have headings in Content Usable but it doesn't answer what Julie and Jennie are saying enough
… Like - how to work with users
… with end users
… Another is: How big of a first group do you need?
… Maybe you can
Julie: It may also be that each organization sets up their own number
… Maybe the procedural test is to follow what the organization already uses
Lisa: But what if their number is too small?
Julie: I think "try to build a diverse group" is soft. I think having those conversations is helpful.
Lisa: I see 2 things happening: experimenting with how they work with identify controls.
… And putting together headings of what a process may look like.
<Zakim> ShawnT, you wanted to talk about working together
ShawnT: Being one of the newer members - I really liked when we were in Silver and we worked in smaller teams.
… Actually worked on doing things together
… I have not read the whole Making Content Usable
… And for what I have read it is hard to retain
… I will benefit more once we get to the point of working together. Right now I am more listening.
… Going through the document, and breaking it up, together, will help me
Lisa: I think that comment is really important, and should also go to the structure group
… We really want people who have put as much time as you have, to feel on top of what they have already read
Jennie: what is the timeline for completing this first portion of work?
Lisa: We have a year to have an initial exploratory draft, and make a plan to get the final draft.
… That's our KPI
… To get some user feedback.
… we would need a month once we have the feedback - does that make sense as a timeline?
Julie: the phrase "initial exploratory draft" - does that mean doing all of the procedural testing before we ask for feedback?
Lisa: no. It means we have filled out 2 of them. Do you agree Rain?
… So people say they like this structure.
Rain: yes, just as an example.
(Lisa updated the timeline in the document)
Jennie: I can start the Google doc to identify which automated test pieces might be possible, then can share with the group.
Lisa: our next meeting is June 30th.
… I might put down some headings if I get to it - mainly so people like Julie can say how it should be adjusted
<kirkwood> Jennie, when you create the doc for ‘teasing out what can be tested’ I’d be interested in potentially reviewing/contributing
<julierawe> +1 to Lisa's plan!
Jennie: kirkwood - my plan is to send by everyone in this group!
<julierawe> Always helps to have something to react to