<Lauriat> trackbot, start meeting

<trackbot> Meeting: Silver Community Group Teleconference

<trackbot> Date: 17 July 2018

User testing strategy

<Lauriat> Charles: I suggested previously that prior to formalizing a plan, we should have a strategy or guidance that says what the goal of testing is and what our methodology is.

<Lauriat> …For example, since multiple types of tests can produce different types of insights, we should have some sort of guidance that says we're going to conduct this type of testing with this type of audience to get this sort of insight.

<Lauriat> …There's a couple of ways we can get a winner, but we need to compare things against something.

<Lauriat> …Then we set up who's in charge of which tests, etc. as we get more granular.

<Lauriat> MikeCrabb: And the prototypes will change rapidly. The one I made has changed quite a bit from a week ago.

<Lauriat> …So one of the things we need to think about: what aspect of a particular prototype do we want to test?

<Lauriat> …We'll need a prototype for how people monitor conformance, we have a lot of opportunity here and may have four or five winners, depending.

<Lauriat> Charles: A perfect example of needing a strategy beforehand. The test strategy wouldn't be to set something in front of someone and ask whether they like A or B, but a flow test that looks at how someone gets from one point to another.

<Lauriat> MikeCrabb: Right, I've gotten comments on styling where I expected to get more comments around structure and how people navigate it.

<Lauriat> Jeanne: One of the things coming up right now: we need to start having policy makers and lawyers looking at the conformance prototype to test whether it would hold up in a court of law.

<Lauriat> Charles: The other part of the conversation we had: where does this information go. I think it premature to put it into github, but we can put our strategy and plan into a Google Doc (public to the group) and go through rounds of reviews before moving it to github to make it public to the world.

<Lauriat> Jeanne: Yep.

<Lauriat> Jennison: Will we just have prototyping done with people looking at things and commenting on things, or more structured? Some things won't really lend themselves to people just looking at them.

<Lauriat> Jeanne: We do have someone who has offered an accessibility lab to do testing for us.

<Lauriat> …At least another possibility that we have to do testing, and I think I know another resource for where we could do testing as well.

<Lauriat> MikeCrabb: I have a lab in my neck of the woods as well where we can do testing, and have talked with some other groups who also have interest in testing things out and have the resources to support us.

<Lauriat> Charles: We need to also consider how we can recruit for and run these tests.

<Lauriat> …considering what types of stakeholders we get for running through each of these tests.

<Lauriat> Jeanne: We have a multi-pronged strategy that we'll need to follow. Who do we have who knows about this who we could tap for creating strategy?

<Lauriat> [group talks through some members of the community group with this kind of area of expertise]

<Lauriat> Jeanne: I think I'll just send a message out to the community group to ask about interest.

<Lauriat> Charles & Shawn: Yep, definitely.

<jeanne> ACTION: jeanne to send out an email to the CG looking for UX people

<trackbot> Created ACTION-196 - Send out an email to the cg looking for ux people [on Jeanne F Spellman - due 2018-07-24].

<Lauriat> Cybele: Should we look into the gaps in previous rounds of user testing done in this space?

<Lauriat> Jeanne: Nobody has done usability testing on guidelines before, so nothing really to look at there.

<kirkwood> User Experience UX Testing we often use. If helpful

<kirkwood> Charles is bring up very good points there that we should capture, I think.

<Lauriat> [group discussing feasibility testing (ensuring we have something ready for usability testing) and usability testing process around prototypes]

<Lauriat> MikeCrabb: We'll also need to define what success looks like for each of these. I don't know whether that means coming up with two different prototypes for each thing that we want to test?

Lauriat: It would make sense to have this conversation with the UX experts. I think they'd have a lot more experience than us on the call.

<Lauriat> Cybele: How might we ensure that, with no previous usability testing before, how do we ensure that Silver meets the needs of people with disability and make a difference? How might we meet the needs of a range of people who need to use it, with the different stakeholder types we have?

<Lauriat> …One of the bigger problems we have: use at the last minute and under duress, and how do we change that to early and often use?

<Lauriat> Charles: All excellent points that I think everyone in the group considers, but I don't know if necessarily relevant for the context of testing the prototypes.

<Lauriat> …First up, we need to have a format for guidance, and then extend that to actual guidance, and then out to people using that guidance.

<kirkwood> Agree with Charles statement that we need to have a format for the guidance.

<Lauriat> Jeanne: I have an action item to reach out for usability testing people, so I'll put the notes together that we can give them, but let's move on.

Requirements comments

<jeanne> Lauriat: We have received comments on the Requirements.

Lauriat: Received comments about the requirements. Mainly around two areas.

ah, go ahead

k

Conformance and Testability of Conformance

Lauriat: Seems that there's a lot of confusion of peope thinking of measurability as exclusion of testability
... thinking it requiring bringing in external users to do the measurements
... comments around us trying to force a conformance model from the start
... even though we don't have one yet
... last comment was more around "rather than having requirement of measurability, having a comment that we consider measurability"

<jeanne> https://www.w3.org/2002/09/wbs/35422/silverReqFeedback/results

<jeanne> I found the Flexibility section of 1.2.3 Maintenance to be a bit unclear and/or redundant. "Flexibility" would appear to be a broad goal (almost an intro), but I think it seems to be covered in the other 3 subheadings: scaling, governance and evolving technology. However, some of the points listed under flexibility need to be inserted into these other 3 headings.

<jeanne> I'm a bit concerned that 3.1 "Multiple ways to measure" is phrased in such a way that it could be read as a _requirement_ that every goal have many ways of measuring achievement, whereas I suspect this is intended to be that while some goals may be framed this way, others may have a single way of qualifying success, where appropriate. "Flexible ways to measure" may be more accurate?

<jeanne> How do '3.2 Flexible structure' and '3.3 Multiple ways' differ?

Jeanne: Here are the results of the current questionnaire.
... people don't seem to understand what we're doing, we need to explain it better

MikeCrabb: Another place where we can show them how this is supposed to work.

Jeanne: Table 5 did very concrete testing. Would you take that and turn it into a document, prototype, or something to point it to?

Lauriat: I don't want to point people to it and "this is how it would work" I'd rather say "this is how it could work"
... we had lawyers and policy makers involved in it, but it was more the seed of an idea we should explore more
... want to take the time to play with it and see what we could make of it
... maybe make some iterations of it, shorter, longer, more measurability...and see how each of them work or which might be the best one to use

MikeCrabb: Are the results from table 5 online?

Lauriat: In the Google Drive folder...looking for it now. We have a write-up as well as photos

<mikeCrabb> https://docs.google.com/document/d/1XDOmjQkMRqQ0XKiKYA-mDAuA40oHtaXNl1H0NXLcV80/edit

Lauriat: The prototype has a lot of ideas in here that merit revisiting, expanding upon, replacing, etc. so that we can do something around testing how it would work.
... there's a couple of things we prototypes in here: method of how you document conformance, communicating conformance

MikeCrabb: I like the idea of it. It's going to have to be more granular than a single page or task. Maybe look at individual elements.

Lauriat: We purposefully went the other direction. We wanted to see usability of specific task or flow within a site or application from perspective of type of user.
... instead of "this control fails" we say "can the user do the task?"

MikeCrabb: So we need a way of changing a design prototype into a way for developers, etc. to make their site more accessible

Lauriat: Something that target more user impact than "you're failing contrast requirements, you need to meet contrast ratio"; framing in terms of user interactions. "users with low vision having problems distinguishing things"

MikeCrabb: Maybe look at flavorizing the conformance as well. Give the same information across different domains, in terms of dev, designer, tester, etc. but have same message come out at the end

Jeanne: We need to get the IA going. We're in mid-July. I don't know how to get this moving. What do we do next?

Jennison: So we need a subject matter expert in Information Architecture to drive this?

Jeanne: We can work with the people we have

Charles: There's a lot of work in our roadmap, but going back to the feedback do you think the Google Doc from Table 5 conformance model helps answer that question of "here's one of the things we thought of envisioning a conformance model"

Jeanne: Need more than just linking to that document. They're not going to plow through all of that.

<Charles> the doc I was referring to: https://docs.google.com/document/d/1iOut3_i1JBQu5_16plZ8u7xCd66T4TbW2zPR1KmWWF4/edit?usp=sharing

Lauriat: I'm very interested in working on a conformance prototype. In about a week I'd have time to start.

LuisG: I can help with it in a week.

MikeCrab: I can also help.

Lauriat: As a random side note, I'm really happy we're getting the comments we're getting.
... this will help make sure we get agreement that Silver does what we'll want it to do

Invitation to contribute prototypes

<jeanne> https://docs.google.com/document/d/1enO_9nflCbudzMpu1nKvFA2QVebVtxWkezeIbI9LP80/edit#heading=h.etorfar350s4

Jeanne: I've updated the invitation that we wrote a few weeks ago and would like to send it out
... gonna throw it into the Silver blog and want everyone to tweet it, email, etc.
... maybe I should add a bullet for UX testing?
... please make comments, I'll look at it before I send it out. We need people to start working on this.

Jennison: Want to quickly go back to comment about architecture. You have a list of people ready to go, is it just getting people into a meeting and getting stuff starting to happen there?

Lauriat: yeah, I think so. it'd be a good start
... not sure if there are other barriers

Jennison: If there's that list of people, I can coordinate that meeting.

Jeanne: Sure, there's a volunteer list in the project plan folder in google drive. If you can get that started into a meeting that would be great.
... I know a lot about the project, I don't know a lot about information architecture

This is scribe.perl Revision: 1.152  of Date: 2017/02/06 11:04:15  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Present: Charles shari Lauriat AngelaAccessForAll LuisG kirkwood jeanne Jennison KimD
No ScribeNick specified.  Guessing ScribeNick: LuisG
Inferring Scribes: LuisG

WARNING: No meeting chair found!
You should specify the meeting chair like this:
<dbooth> Chair: dbooth

Found Date: 17 Jul 2018
People with action items: jeanne

WARNING: IRC log location not specified!  (You can ignore this 
warning if you do not want the generated minutes to contain 
a link to the original IRC log.)