See also: IRC log
<trackbot> Date: 09 October 2014
<Greg> Survey results at https://www.w3.org/2002/09/wbs/36791/20141007/results
<Greg> Wiki with the draft tests: https://www.w3.org/WAI/UA/work/wiki/Tests_for_CR
<Greg> We start with Jim's draft test for 2.3.1.
<Greg> https://www.w3.org/WAI/UA/work/wiki/2.3.1
<jeanne> GL: I want to talk about the format and general template we are going to use.
<jeanne> JR: last week we agreed to follow the WCAG style of tests
<jeanne> ... the larger issue, is the emotionally satisfying issue of only testing one thing
<jeanne> GL: We need a test document that has samples of different types of elements.
<jeanne> GL: We could provide test documents for a lot of formats, SVG, etc.
<jeanne> JR: HTML will be good enough.
<jeanne> GL: Our samples can serve a template for testers to create documents in other formats their user agents support
Greg: do test documents that W3C already has exist for HTML?
Jeanne: we should how we do our own
Jan: one big document, also in the process of breaking it into smaller documents
Greg: first step go through the
SC and make a list of test documents we need
... we need an ATS document that has one of each type of
enabled element, that would be for testing this set of SC's
Jeane: some of what ATAG did is we put a dummy link to enable elements so we knew when we were finished what we needed
Jeanne: if one person could focus
on getting these documents together that would be useful
... we want to have enough of these done so we can actually do
testing at TPAC
... potentially we could have a lot of observers there, and we
can start going to the test criteria with everyone having a
different user agent and get a lot of data put upon the screen
that says whether or not this browser meets this success
criteria
... that where we are doing important work that needs to get
done in terms of getting to CR, but we are also promoting to
everyone who attends – whether or not this browser meets UAAG,
which puts pressure on all the browsers to meet UAAG. We need
to start letting people see that people are implementing
UAAG
... it's also getting work done but also positioning ourselves
publicly as a force to be reckoned with
... Jan and I are going to sit down on the 21st and 22nd and
blast through any tests that aren't written yet so we have them
in time for TPAC.
Jan: Test documents will take longer
Greg: we could start with the
easy ones. Where is a list of all the enabled elements in HTML
5?
... of these test files supposed to be minimal to demonstrate
the thing being tested or are they supposed to also be
demonstrations of good best practices including best
accessibility practices?
Jeanne: no, just the small files
just for testing
... do we need is not accessible file?
Jan: yeah, just for a couple of things
<Jan> http://www.w3.org/WAI/AU/CR20/TestPrep.html
Jan: UAAG has a lot of stuff about control, but it's mostly things like pause and skip around
Jeanne: videos from NASA
Jan: UAAG is going to have more need of accessible test files than ATAG
<jeanne> http://www.w3.org/WAI/AU/CR20/WCAG2_HTML_Problem_File.html
Jeanne: inaccessible file
<jeanne> http://www.w3.org/WAI/AU/CR20/WCAG2_HTML_Problem_File_Fixed.html
Greg: why a tiny image that's difficult to hover over very narrow vertical bar
Jeanne: media is not there yet
Greg: going back over other comments to see if there's decisions we need to make on 2.3.1
Jan: needs to be reworded to only use words we have predefined
<jeanne> All steps must be true or false
Jan: we have a definition for direct navigation
<jeanne> We cannot use words that aren't defined in our Glossary
Jan: we should stick as closely
as we can to the SC wording
... any rewording is a departure from the normative
Greg: use document instead of
page
... rewrite to say the test document has each type of, and
we'll use some phrase meaning verify for each type
Jan: selection of types – very difficult to pull out every type
Greg: embedded objects
Jan: that should be done
... I agree that we get rid of assertion, procedure instead of
assertion
... cleaning up the format, getting rid of the line breaks
Jeanne: important to write these in the wiki instead of email – can send email with links to wiki
Greg: I was reluctant to edit the wiki and change Jim's wording, because we could have a survey potentially with answers responding to Jim's wording and then have to mine after I changed it
Jan: I don't think we should be linking in surveys to the wiki – it changes too much
Jeanne: anything that we want to change, make changes
Greg: I agree that we should keep a record – Jim writes one, Greg changes, Kim makes comments on Greg's, hasn't seen Jim's
Jeanne: is that the way we work?
Kim: do a copy paste if you do changes so there's an original etc.
Jeanne: can we do the last one on top?
Greg: I did change is right in line so you could compare with what I included for people to decide
Kim: outdated section at the bottom – copy the current, put it at the top of outdated, then make your changes to the current?
Jeanne: I want to make sure not
to have to do a lot of cleanup on these files before I move
them into the SQL database
... I suspect it's going to be a brutal copy paste job
... give people a lot of ways to filter things if you don't
want to see video etc., test page, allows you to check whether
it's pass or fail, records that in the database and gives a
report that tells you – we've got two implementations of this
success criteria but only one or none for this success criteria
– and you can put comments in.
... all the success criteria and tests have to be preloaded
into the database before – that's the copy paste job
... it's a really slick thing once it's working. That's
eventually where this goes, and then we can start testing
... before we start the formal testing we needed informal
report where we know what our success criteria that we may not
get two limitations of – the at risk success criteria. We need
to have a list of those. If we declare something is at risk we
can drop it from the final stack without having to go back to
another CR and go through the whole process again. it's
important to identify the...
... at risk ones, and that's what we will be doing at TPAC
Jeanne: Greg's comments very good, these are things we want to talk about – how we are doing it, this is very useful
Greg: look at the draft wording – consult the manual if necessary
Jan: consult the documentation
Greg: we need a big decision on
how we want to handle this whole extensions thing
... the one about showing direct navigation – let's say a
browser needs the mouseless browsing add-on. Are we expecting
the tester to iterate – I consult the documentation and I can't
see any way to do this, then I go to the extension to provide
this functionality?
Jan: I don't think so – they need
to know ahead of time the package that they need to put
together
... it depends on if they are taking more of an adversarial
role or not – this failed, so let's see if there any extensions
– it's really up to them
Jeanne: keep in mind it's really up to us – we're the ones that will do the testing. But Jan is right we have to assume that the tester will put together the package.
Greg: so it's the user agent and all the extensions that are installed with it
<jeanne> We should put a statement early on that by "user agent" we mean the user agent and all extensions that are all being used to meet it.
Greg: we should make that
explicit somewhere in the startup of the testing
procedure
... as a general thing when everyone is writing these tests
there are certain things to keep in mind – helpful to make a
list of them, such as keep your wording technology neutral
Jan: I agree, and your comments are half of that style guide
Jeanne: we could make it a wiki page, and link it
Greg: I like the method of saying it in a technology neutral fashion and then in parentheses giving an example
Jan: agreed
Greg: for example say shortcut, and then in parentheses a modifier, access key whatever
Jeanne: writing up thoughts for the style guide
<jeanne> https://www.w3.org/WAI/UA/work/wiki/Tests_for_CR
<jeanne> https://www.w3.org/WAI/UA/work/wiki/Test_Writing_Style_Guide
<Greg> How will we handle it when the tests for several SC can and should be combined into a single test pass?
Greg: I'll edit the test template, and then Jeanne can correct later to make sure it's good for cut-and-paste
<jeanne> JS: I think we need to talk about the combo or chained SC tests when we find them.
<Greg> E.g. use heading levels appropriate for the final SQL database, rather than the Wiki pages.
This is scribe.perl Revision: 1.138 of Date: 2013-04-25 13:59:11 Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/ Guessing input format: RRSAgent_Text_Format (score 1.00) Succeeded: s/ rrsgent, make minutes// No ScribeNick specified. Guessing ScribeNick: KimPatch Inferring Scribes: KimPatch Default Present: Kim_Patch, Greg_Lowney, [IPcaller], Jeanne Present: Greg_Lowney Jan Jeanne Kim_Patch Regrets: Jim Kelly Found Date: 09 Oct 2014 Guessing minutes URL: http://www.w3.org/2014/10/09-ua-minutes.html People with action items:[End of scribe.perl diagnostic output]