Meeting minutes
"Evaluation Tools" vs. "Test Tools" terminology
krisanne: There are pros and cons to both or either terminolgy
sharron: What is the goal of the discussion?
shawn: The goal is to open up the discussion, realise that we have this difference and explore whether we need to change the terminology
sharron: This may need to go to survey as well. Might be worth exploring the rationale for the positions and then open it to the larger group.
… As Wilco indicates, most people in the field to refer to testing tools but internally we talk about evaluation tools.
… Would be interested in the academic view on this as well.
Vera: Not sure on academia. Testing probably makes more sense to most users but there are tools that will simulate impairments that couldn't really be called 'testing.
shawn: Just to clarify is to ask for different perspectives. There is no need to add qualifiers
Wilco: I did a bit of an informal walkthrough of industry tools. I found that it is a bit of a mix between 'checking' and 'testing'.
… 'Evaluation' and 'validation' is sometimes used. Rarely 'auditing' is used.
… In the ACT project I am trying to use the most familiar language. 'Testing' makes most sense in that space.
… 'Evaluation' seems more nuance and is more difficult for people who don't have English as a first language.
Vera: In terms of what the words mean, I would like to go back on what I said earlier. 'Evaluation' seems more broad but it does actually cover aspects of the simulation tools as they are testing aspects of things too.
<Zakim> shawn, you wanted to say process vs tools
shawn: Is the process 'evaluation' and the tools 'test' tools?
krisanne: Is there one way to describe the process? I also wonder if WCAG3's move away from pass/fail approach mean that we have to find a broader term.
dmontalvo: I think there are different context that will make one phrase easy to find to cover it all.
… The 'test tools' do tend to talk about 'testing' things but the broader evaluation might include other tools that aren't necessarily 'testing' something.
… Maybe in the ACT space 'testing' is appropriate but 'evaluation' can be used more broadly.
Wilco: 'Test tool' doesn't usually work without being prefaced by 'accessibility'. 'Evaluation' needs this qualifier slightly less to set the tools appart.
Kevin: The context on that side of things is really improtant. Evaluation refers to the process - I would not say we test things, we talk to testing teams, we try to get them to include this but when we are acting, we are evaluation a product.
<Zakim> shawn, you wanted to ask +1 Daniel, leave as is ?
Kevin: do I talk about testing tools? yeh maybe. Do I talk about evaluation tools? yeh maybe. They re interchangable and that is the problem here.
Shawn: Maybe we go back to the original questions and end up that since we have used both, they both have hsiotroy, maybe we just leave as is.
<Sharron> Daniel: So leave in the ACT as testing and where it is now as Evaluation?
<Sharron> Laura: As a term evaluation seems more inclusiive of all the various ways in which we look at products. In a QA context it makes more sense to be evaluation.
eoncins: I agree that both terms can be use interchangable. For me 'evaluation' is a procedure but 'testing' refers more to the tool.
… I would be looking to understand the distincion within the context rather than having just one.
<Zakim> eoncins, you wanted to ask if "evaluation" is not more related to a procedure and a "test" is more related to a product?
Howard: Looking over some of the resources and my own coause I use both terms. EIther one would works. My preference is to use 'evaluation tools' but they are interchangable.
krisanne: Feeling tends to be that they both work but it is imporant to consider the context. For example being asked to look at a product to be selected would be an 'evaluation' not necessarily a 'test'.
<BrianE> +1 to krisanne
<eoncins> +1 to Sharron
Sharron: Agree with the context being important; we evaluate products before we buy and systems before we use, but we will test systems that we have built.
krisanne: Can we live with them co-existing?
Sharron: Yes, we don't necessarily want to limit the language unless there is a need to.
krisanne: There is such a wide variety of different tools that it seems difficult to constrain our anguage.
Wilco: I think that is reasonable. 'Testing' works in ACT, 'evaluation' can be used elsewhere.
Evaluation Tools List
Vera: Main thing we have been working on is the tool submission form.
… ALso, the process behind that to send the form to ithub and list maintainer for review. Thanks to Shawn for info on how the process currently works.
<Vera> https://
Vera: A big change that has been implemented is that the top menu bar has changed.
… Is there anyone that can explain what has prompted that change?
shawn: The WAI website has the minimal header which is used for resources where people are likely to be working in that resource for a reasonable length of time.
… For example this is used in COGA guidance, Understanding and Techniques, WCAG EM Report Tool.
… We don't have to use this though. This group can choose what approach to actually take.
… So the question would be, how important is the screen real estate for the Tools list and submission form.
… Are people going to be using this list in a focused way and looking to avoid the distraction.
… Or is this a high traffic entry point that we could take advantage of by including a fuller navigation?
Vera: That explains a lot on the reasoning on it. I was a bit surprised because earlier user testing we saw people were confused as they didn't knw what this was a part of or where they could find more information.
… I would really advise, based on this testing, not to go for the minimal toolbar.
shawn: Quick check if people want to agree to Vera's recommendation
<Howard> +1 on having navigation
<Laura> +1 on having navigation
<Sharron> +1 to Vera
shawn: Need to check whether we want consistency on other tools as well though
+1 to having navigation
Vera: We have been working with the course list team to take consistent approaches.
… We are still looking at how people can update their tool entries. Possibly a link from GitHub to a pre-filled form
Vera: On the form we haven't changed many of the fields. We were considering making it multi-step but this was not possible with the set up.
… We have grouped the fields into related elements
… Have changed the features input to encourage more bullet point lists of features rather than just marketing text.
shawn: Why are the date formats different?
Vera: This may be because I am using a Dutch browser set up.
krisanne: My browser shows dd/mm/yyyy which supports this.
Vera: It might be better to remove this from the label is this could cause confusion.
<Zakim> shawn, you wanted to ask about date format
Laura: If we remove the format from the label, then there are concerns that the placeholder may disappear when the field is tabbed into.
shawn: I agree. The best thing would be if the label had the system format as well although I am not sure how possible that is.
krisanne: This doesn't appear to be a placeholder since I can tab through each of the date components.
Laura: I just want to flag that having placeholder only is not somethign that we would want
BrianE: I think these are a text field and will pick up system format. This will be in conflict with any fixed label which would be difficult to overcome.
… Also, looking into date inputs as they have not been great from accessibility perspective.
Howard: I like having both, even with the inconsistency. I think people will figure it out and the label is more prominent and I wouldn't want to lose it.
krisanne: Yes, this may be that the need of some would outweight the inconveinience of others.
shawn: But having the conflict would be wrong from a user perspective
krisanne: And going back to my browser it is showing mm-dd-yyyy rather than what is in the label, dd-mm-yyyy.
Vera: Is there a need for a decision on this now? Or can we look more into this? Also, if anyone has other resources, that would be a help if we can bring this to a survey.
<dmontalvo> +1 to removing label if we are using date input type. That's confusing.
<CarlosD> +1 to remove label if it can't be made consistent with the input field
Vera: One other difference to bring to the group today. Lanugage has a similar input field to that of Features.
… Language has a large list. You can add multiple languages by adding them one at a time.
… Only the last added language can be removed. That is a slight limitation but it does make the form that bit cleaner.
dmontalvo: If you have three languages and you want to remove the first of them, then you have to remove all three and then add in the last two?
Vera: Yes, and that is the same for all users. We haven't looked at the accessibility but my view is that the user experience is not great.
dmontalvo: Yes, that does seem a bit cumbersome!
Vera: Following approval we were planning to contact the tool vendors to update their details. We think that we could get approval to publish before we do this and then publish a populated list.
krisanne: Thanks so much
Sharron: It will be good to give an incentive to vendors update their information.
Vera: Yes, there is a lot of outdated information
Selecting Evaluation Tools
krisanne: Just looking at the opening page for the list of evaluation tools. There has been some discussion on the requirements of this in thepast.
… Before we go through the update we just wanted to check that there was nothing additional to add to the requirements.
https://
<krisanne> https://
shawn: Sorry, I didn't send this out until late last night. Briefly though, there was a longer verson of this page some time ago. Then when the site was redesigned in 2017 we worked on tersifying the content (more succinct and focussed).
… The 2017 has a much smaller scope. In 2021 there was a suggestion to broaden it again to include more user stories and to expand it a bit.
… This was an individual perspective which discussion in EO decided that our default still is succinct and focussed but there are resources where there are options to be more narrative.
… The Selecting Tools resource falls into the first group of resources.
Vera: Looking forward to other perspectives on this and understanding everyones views on the requirements. I am particularly interested in the user experience for someone landing on the tools list page and then coming here to understand how to select tools and use the full list.
krisanne: I like the idea of something being a friendly welcome space when it comes to tools like this. We want to encourage people to use them and user stories can help in that.
… Having something that is informative and not overwhelming can be a big help in this.
CarlosD: I think we are missing a reference to WCAG test rules in this page. I don't know if we need to call this ACT or WCAG test rules.
… I think it needs to be referenced here as something to consider when people are selecting evalution tools.
shawn: It would be most useful if you could add a GitHub issue to the repo
<dmontalvo> +1 to Carlos
Vera: I am aware of the ACT rules as Wilco contact the team. We are looking at how ths can be woven into the tools list and the selecting tools page.
… We are also looking at how tools support the ACT rules and whether we can highlight this in the listings.
krisanne: If anyone has comments, please do put them into the Github so we don't lose them but we will have more opportunities to feedback on the rewrite.
Easy Checks Next Gen
<krisanne> https://
Sharron: There has been a huge amount of work on this brilliant idea to help support people getting a feeling on the accessibility of a site.
… For a few years it was one of the most highly commented on and appreciated resources.
… The comments have become less positive as it has become quite outdated.
… That we have a talented team to research and update this resource is really exciting.
Vera: We haven't made our plan just yet but we are all excited to work on this. Two main challenges:
… - Updating the information; and
… - The findability, structure and information architecture to ensure peopel can find what they need easily
krisanne: The idea of being able to filter the checks is really helpful and should help to cut down on the complexity of the resource and the amount of text that needs to be presented to users.
+1 to excitement!
Vera: I was wondering in terms of following a user journey starting with these checks there is something after the checks to learn more.
… How do we help users go on to fix or address the issues that they have found.
… Also perhaps, what does it mean and who does it impact
shawn: Sounds wonderful but need to be really careful on scope creep
Vera: Agreed, although if we can link to other existing resources that answer these questions
Sharron: Yes, this is something that we have done more and more. As we have more resources available we can build that bigger picture of support and related resources.
krisanne: Again, there will be opportunities as this progresses to feedback on the shape of the resource.
shawn: I am beyond thrilled that this work is happening and that Vera and the team are taking this work on.
Work for this Week
krisanne: The Curricula module review is closing today
Sharron: Something I learned at AccessU is a thing called Blooms taxonomy that is language for learning objectives.
dmontalvo: Yes, this is something that we are trying to align to as much as possible.
… There are a revisions to this taxonomy which can make it difficult to use but it is good to see that people are recognising that we are doing this
krisanne: Please do get comments and feedback on that survey today if you haven't had a chance to do so.
<Sharron> trackbot, make minutes v1
<trackbot> Sorry, Sharron, I don't understand 'trackbot, make minutes v1'. Please refer to <http://