<Chuck> scribe: Chuck
<CharlesHall> Cognitive walkthrough An inspection method for evaluating the design of a user interface, with special attention to how well the interface supports exploratory learning, i.e., first-time use without formal training. The evaluation is done by having a group of evaluators go step-by-step through commonly used tasks. It can be performed by evaluators in the early stages of design, before performance testing is possible.
Jeanne: <reviewing
participants>
... Getting started.
Jeanne: Most important thing
that's come up in the last week is that we want to change the
project plan that we've been working on.
... roughly what we were planning is to get the 6 success
criteria written, use that to form an editors draft
... which would include conformance section but with all the
notes of things that had not been worked on.
... There was much pushback. Enough people wanted to work on
conformance now that we...
... We were talking about splitting up the work, having people
continue to work on content (first 6 sc)...
... then have another group that would work on conformance. I
received an email from Alastair (chair of agwg)
... Alastair made some recommendations on how this could work
out.
... I don't have a new firm plan to give today, mostly because
there are a lot of people weighing in on the plan.
... I want to give members of this group who have been doing
the work, an opportunity to speak up and discuss what we should
do and ...
... what order. I want to give people a chance to discuss what
and how.
Jennison: it makes sense to devide up the work. have people focused on 2 different streams. Will help with time and effort.
Jeanne: The problem with it is that the people who have been doing the most work have been doing both.
Jennison: Right. I think the
alternative though is to continue having everybody doing
everything.
... We are trying to speed things up to get things back on
track with plan. We'll have to accomodate.
... As difficult as that might be, that's the only way to get
it to work.
Charles: I would agree, but I
would add that we have a chicken and egg scenario. One of the
biggest road blocks is proving the conformance model.
... In order to test we have to have guidance to use against
the conformance model.
Jennison: You refer to a couple of the guidelines?
Charles: Yes.
Jennison: Once it's written yes. But that doesn't stop the work or the structuring for the conformance model.
Charles: We still have a lot of
divergent thought (argument) on the proposed conformance model
within this group and larger group.
... The best way to solve the argument is to prove it.
Michael Cooper: Alastair's approach to solve chicken and egg is a parallel approach.
Michael Cooper: Work would go back and forth. Alastair's proposal accommodates this.
Jeanne: The issue is the methods. We need content people writing methods. I want to talk to Alastair on monday.
Michael Cooper: I don't recall that, but ok.
Jeanne: It's a good proposal, I'm
grateful for it. Where he assigns things doesn't make sense
with the people who are working on it.
... I have a different perspective. I don't think we can write
methods independent of content.
... If we had people write guidelines right now, that would
invalidate our process.
... We are building up where the guideline is the last step of
the process. But we can talk about it in detail on Monday.
Michael Cooper: Opening up Alastair's spreadsheet. I see the assignments.
Michael Cooper: I think he's assuming that not all of these are final steps.
Michael Cooper: ...then the content people would be able to take that as guidance as method writing.
Michael Cooper: That's what I think he was thinking.
Charles: Leapfrogging gant chart.
Michael Cooper: That's a good description of it.
Jennison: Where does that leave us?
Jeanne: One of the other
possibilities of what we could do is that we could meet as two
separate meetings on Tuesday and on Friday meet together.
... We then put together lessons learned from separate
projects, and bringing things back together again. I'm worried
about divergence.
... that was one of the concerns. Cybele was worried about if
we move too far away from eachother.
... We could lose important perspectives.
Charles: As long as we are coming together that would be mitigated. The bigger problem is scheduling these calls to accommodate everyone.
Jeanne: Including Micoto cuts out
the west coast.
... Unless we stuck with a call at 9:30. Equally inconvenient
for west coast and Japan. I was thinking about moving it a bit
early....
... That would be 5:30 in the morning.
Michael Cooper: The talk about who can participate is based on teleconference participation. We shouldn't let timezones dominate that.
Michael Cooper: It looks to me that a call too late in the evening west coast wouldn't be too early in Japan. We should expect flexibility from all parties.
Michael Cooper: 7pm for us, 4pm for us, 8am for Japan. That's not horrible times. Not perfect but not bad.
Jennison: There will never be a perfect time.
Michael Cooper: If we are including Europe, East Cost, West Coast and Japan.
Michael Cooper: We want to allow for participation outside of teleconferences.
Jennison: Do we know which piece Micoto is most interested in working on?
Jeanne: Conformance.
Jennison: 4pm Tuesday would work. For you Luis?
Luis: Yes.
Jeanne: Let's tentatively say
that for a time. Thanks Michael.
... Do we want to do it on Tuesday, or Monday evening? Any
thoughts?
Jennison: If content ... might
make more sents for content folks to have their meeting earlier
to inform evening meeting on the model.
... And all of us come together on Friday.
Jeanne: That makes sense.
Jennison: Sign me up for conformance.
Cybele: Who's on what team?
Jeanne: We are picking sides at
the moment. We just figured out content writing team would have
morning call Tuesday as is.
... Conformance team would meet that evening 7pm eastern.
Possibly have Friday's be joint call.
Cybele: I love that. In the sense
that I'm happy to do content and would like to be updated on
conformance and keep us together.
... Q on Friday call. Does it alternate or... how do we do
focus of Friday meeting?
... Have a guideline about how it works.
Jeanne: We'll have a call on Monday and discuss details. I don't want to make a commitment without having conversation.
<CharlesHall> it would be ideal to have a shared / visible tracking method – especially for dependencies
Jennison: Point is well taken.
Bruce: I think I was recruited to
talk about conformance model, but not sure how that time will
work for me.
... The major q in my mind is hearing Shawn say that points are
being assigned by the method, prior to that I was sure
that
... points were assigned at guideline level. More methods more
points is a permutation. I'm feeling uneasy about normative
part (guidelines) and methods (points).
... Will follow that keenly.
Jeanne: Shawn always said points should be associated with tests.
Bruce: Which are associated with methods.
Jeanne: It would be a clear whether or not it was pass/fail or what the status was, even though not normative it would show clearly that you met normative guidance.
Bruce: I think that makes sense... but again point assignment.
Jeanne: And a lot of these
details about how to do these advance tests, we need those
tests to say how this would work.
... When call started we were saying that we need an example of
a cognative walkthrough to see how it fits in the model.
... We need something to look at to decide if it makes sense to
put in method or guideline.
... I know a lot are interested in conformance, but we really
need content or we'll be right back where we were in
November.
Jennison: Re cognative walkthrough did we loop in Charles?
<CharlesHall> cognitive
Jeanne: He didn't volunteer... :-)
<CharlesHall> https://www.usabilitybok.org/cognitive-walkthrough
Jennison: Do we have... we have
some folk. It makes sense to send out email to list once you've
had conversations and confirmed new approach.
... And let them know the impact about not having enough people
in conformance (or too many). maybe cap the participation.
Jeanne: The other thing I was
thinking, reach out to people that you know in either ...
friends with or in org that have usability speciality
... And know how to do these tests, so we could get help
writing up these tests from people with experince in writing
them up.
... Charles has been trying hard to educate me on the correct
names, but the would include cognative walkthroughs, user
research...
... We need to write up these anyways. Good to get something
sketched out.
Jennison: I'm happy to get some
help over the summer from my peers. We have usability, user
research people.
... If I know what we are looking for I can find some people to
help.
Charles: Sure. I'm on vac next week, will try to put together an invite like we did for earlier research work.
<KimD> I'd like that info too, Charles. Please share broadly.
Charles: I see 2 distinct types
of usability testing. A lower level and middle level. Lower
level is sort of cognative walkthrough.
... That has to have a meta-level test to validate whether or
not the methodogy and result of methodology aligns to the
intent.
<jeanne> scribe: jeanne
<CharlesHall> Charles accepted action item to draft an invitation to recruit usability professionals
Jennison: I solicit inside LinkedIn and A11yBay to find people to write tests and finding people to validate the tests.
<Chuck_> scribe: Chuck
<Chuck_> Cybele: If you do have time I'd like to go through a scenario and looking at that second kind of test.
<Chuck_> Cybel: I would love to capture that in the process and try to understand and help.
Cybele: I would like time with Charles to talk about meta-level tests
<Chuck_> I'm back and scribing.
<Chuck_> Charles: I'll be publishing office hours soon which will give the group my availability.
<Chuck_> Cybele: I have this worry... that the first test you describe ends up carrying the day (it's probably easier), and the second test makes a bigger difference for users.
<Chuck_> Jeanne: My worry is that we get busy thinking about the process of writing the tests and not writing the tests. We need to write the tests.
<Chuck_> Charles: We need to test a method and we need to test the conformance model. In order to do either we need a conformance model.
<Chuck_> JF: <silence>
<Chuck_> Jeanne: Let's take a look...
<Chuck_> Jeanne: I'd like to talk about working on the 6 sc. It doesn't have to be 6 at this point. I'd be happy with 2.
<Chuck_> Jeanne: We have a list of sc that we picked out in April. We started working on all of them at Access U, but we haven't done a lot on it since.
<Chuck_> Jeanne: Chuck and Cybele worked on them the most, on color contrast and identified a lot of issues involved in level squishing of sc.
<Chuck_> Jeanne: We would like to take a look at the other 5. There are some other ones that are proposals as new guidelines
<Chuck_> Jeanne: for right now I'd like to focus on migration sc. They are...
<Chuck_> Jeanne: In this folder...
https://drive.google.com/drive/folders/1uU3_rH7hwHm01OM8GTluz4tn0ed1V2Gm
<Chuck_> Jeanne: Jennison do you want me to drop this in webex or email? Oh, it's in the agenda.
<Chuck_> Jeanne: The folder in google drive. We have alternative text.
<Chuck_> Jeanne: That would be a strong example.
<Chuck_> Jeanne: Personally I work better with one other person, preferably 2. We have headings, we have pause stop hide.
<Chuck_> Jeanne: Sensory characteristics, timing, visual presentation. We just created that to store some ideas.
<Chuck_> Jeanne: Let's not consider that one.
<Chuck_> Jeanne: Did I say audio description? Bruce has been working on that. Bruce do you want to continue? Do you want help?
<Chuck_> Bruce: Clone me! I'm happy to keep working on it. I'd like to close it out.
<CharlesHall> RE: invitation for usability participation. we already have this as a draft which can be edited: https://docs.google.com/document/d/1BYGQytrg_PDh02h6lYUPE7Tz_LRm4dY2Kzyt7zCG248/edit?usp=sharing
<Chuck_> Chuck: Can we add one?
<Chuck_> Jeanne: I'd prefer to limit it.
<Chuck_> Chuck: I'd like to add language of page.
<Chuck_> Chuck: Because it's so simple.
<Chuck_> Jeanne: We have it in methods prototype. That would be one that would be appropriate to keep filling in because we have parts.
<Chuck_> JF: If you have language of page, do you have language of content?
<Chuck_> JF: Language of document and content, there's a direct relationship. That's what you are thining about?
<Chuck_> Chuck: yes.
<Chuck_> Chuck: To see how it works in the intersection.
<Chuck_> Jeanne: I don't want to overstate how much is done. We used it as a prototype for method. The different types of methods that would work.
<Chuck_> Jeanne: We are moving beyond page orientation. So we had things like what would be a method for vr, for communicating with AT, etc.
<Chuck_> Jeanne: A number of different methods sketched out. There's not a lot of the other things written. That would be a good example to fill out more.
<Chuck_> Jeanne: That's also in the folder.
<Chuck_> Jeanne: Very sketchy, but it was trying to test the possibilities of starting to write a test.
<Chuck_> Jeanne: We did this last november, and things have evolved since then.
<Chuck_> Jeanne: But it's a starting point.
<Chuck_> Chuck: Not sure yet, just like it as a test.
<Chuck_> JF: I can take a poke at it.
<Chuck_> JF: You did mention in the email that you are spliting the team. I'm catching up. My interest lies in conformance.
<Chuck_> JF: I'm happy to use this rough draft here to use it as a springboard for conformance.
<Chuck_> JF: I'm not interested in taking on task of migrating into new template for the reasons I have previously stated.
<Chuck_> Jeanne: I intended to ask Chuck. If he was willing to work on it.
<Chuck_> JF: Chuck did originally add it, and I expanded on it.
<Chuck_> Jeanne: We did have language of page and language of content associated.
<Chuck_> Cybele: Was that migration plan completed?
<Chuck_> Jeanne: We went through all of the sc. I can find a link to it. We did not go through and fill in the user needs. Shawn wanted that to happen next.
<Chuck_> Cybele: i would agree that it doesn't make sense to do the user needs in the migration plan, better in migration.
<Chuck_> Cybele: I think it would be better to work on some issues we have encountered, where there are those splits and mergers.
<Chuck_> Cybele: Have a first go at which ones will probably merge and split.
<Chuck_> Cybele: If we completed going through that.
<Chuck_> Jeanne: Yes we have, I'm getting a link to that now.
Migration Plan by Success criteria - https://docs.google.com/document/d/1aCRXrtmnSSTso-6S_IO9GQ3AKTB4FYt9k92eT_1PWX4/edit
<Cyborg_> Presen+
<Chuck_> Jeanne: For anyone who's not in IRC, you can get the link from the silver wiki home page under silver content.
<Cyborg_> just got kicked out, can someone please repaste that link?
<Cyborg_> thanks
<Chuck_> https://docs.google.com/document/d/1aCRXrtmnSSTso-6S_IO9GQ3AKTB4FYt9k92eT_1PWX4/edit
<Chuck_> Chuck: Not sure yet.
<Chuck_> Chuck: I'd like to try and finish color contrast.
<Chuck_> Chuck: Anybody reading these?
<Chuck_> I can't type fast and unmute fast.
<Chuck_> JF: Type fast!
<Chuck_> Jeanne: Bruce will work on... Chuck will continue color contrast.
<Chuck_> Jennison: This is for the content, correct?
<Chuck_> Jeanne: Yes.
<Chuck_> Jennison: You'll probably get more volunteers for some of these laters.
<Chuck_> Jeanne: Not going to hold my breath waiting for volunteers. Been trying to advance a long time. Need people to step up.
<Chuck_> Jeanne: I've been working on pause stop hide, would like to partner.
<Chuck_> Chuck: I prefer partnering with Cybele on Color Contrast.
<Jan> Jeanne - I can work with you on that.
<Chuck_> Jeanne: Anybody want to work with me on pause stop hide?
<Chuck_> Cybele: I can be a 3rd wheel with you Jeanne.
<Chuck_> Jan: Having a hard time with mute. Yes I'll help.
<Chuck_> Jan: I'm also interested in alt text.
<Chuck_> Jeanne: I'd like to see that happen. You know a lot about testing. To have some tests written for alt text would be very helpful.
<Chuck_> Jeanne: Particularly new tests.
<Chuck_> Jeanne: To assess the quality, that would be great!
<Chuck_> Jeanne: Shari also brings that testing validity experience.
<Chuck_> Shari: I thought I'd be mentioned :-)
<Chuck_> Shari: I'd be happy to help Jan.
<Chuck_> Jeanne: We really need help writing tests that are in the rough category of usability tests. We have a group that wants to work on conformance.
<Chuck_> Jeanne: The problem we ran into last november is we needed examples of tests.
<Chuck_> Jeanne: I don't want to start working on conformance and not have any tests to prove out what we are trying to do.
<Chuck_> Jeanne: Maybe we can talk after this call.
<Chuck_> Jeanne: Give me some advice on how to do this Shari.
<Chuck_> Shari: Ok.
<Chuck_> Jeanne: Methods and tests are what need focus right now. User needs, tests, and new methods. That's what we need the most.
<Chuck_> JF: Q about the tests... Is the intention that all of the test boil down to pass/fail? Or will some tests allow for exceptions? 80% successful for example?
<Chuck_> JF: Or binary it meeds or doesn't meet the need?
<Chuck_> Jeanne: At accessU we sketched out some tests where we set out these are the parameters of the bands that it would meet.
<Chuck_> Jeanne: It would be excellent, good, fair, poor. We weren't worried about scoring, but how would we set up the test to be reproducible and consistent.
<Chuck_> Jeanne: We set those up. That's an example of a test that would not be binary.
<Chuck_> JF: Do you have any metrics?....
<Chuck_> Jeanne: We could still have "this would pass, this would fail", but wouldn't have just 2 answers.
<Chuck_> JF: Do we have any metrics on where those lines are?
<Chuck_> Jeanne: The point of writing up the method was to determine what those were. jan which one was that?
<Chuck_> Jeanne: Must have been headings.
<Chuck_> Jan: Confirmed.
<Chuck_> Jeanne: Here's an example of a new test. We said... <reads>
<Chuck_> Jeanne: We had a list of 4 questions we created. <reads 4 questions>
<Chuck_> Jeanne: Then we set up a rating for those questions. If none of the above answers are true that would be poor.
<Chuck_> Jeanne: As experiment, we said that would be 0 points.
<Chuck_> Jeanne: If first was true and rest were false, then if some were true, then if all were true. Various increasing points to each example.
<Chuck_> Jeanne: Here's a way to measure quality and assign points.
<Chuck_> Jeanne: Whether or not those were the right questions we didn't get into that. And the points we were just coming up with.
<Chuck_> Jeanne: That's an example of a new type of test.
<Chuck_> Jeanne: That's all I'm asking for those writing content. We aren't looking for bullet proof content, just need some examples of how this could work.
<Chuck_> Jeanne: Will provide a link....
<Chuck_> Chuck: Time check.
#4 is new
<Chuck_> Jeanne: John, what were you asking....?
<Chuck_> Jeanne: Or was that your answer?
<Chuck_> Jeanne: It's 3, I'll let everybody go.
This is scribe.perl Revision: 1.154 of Date: 2018/09/25 16:35:56 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00) Present: Chuck CharlesHall bruce_bailey MichaelC LuisG Jennison Cyborg KimD Shari JF Chuck_ Regrets: Chris Shawn Found Scribe: Chuck Inferring ScribeNick: Chuck Found Scribe: jeanne Inferring ScribeNick: jeanne Found Scribe: Chuck Scribes: Chuck, jeanne ScribeNicks: Chuck, jeanne Found Date: 21 Jun 2019 People with action items: WARNING: Input appears to use implicit continuation lines. You may need the "-implicitContinuations" option. WARNING: IRC log location not specified! (You can ignore this warning if you do not want the generated minutes to contain a link to the original IRC log.)[End of scribe.perl diagnostic output]