<richardschwerdtfeger> Meeting: W3C ARIA Working Group - Test Coordination call

<jongund> what's the pass word

<jongund> its not letting me in

<scribe> scribe: JF

<jongund> I tried the "aria" password

<richardschwerdtfeger> https://lists.w3.org/Archives/Public/public-aria-test/2016Feb/0002.html

RS: Goals is to review what is happening due to new members coming into this effort

Need to discuss exit Criteria

all we test is what has changed since ARIA 1.0

<richardschwerdtfeger> https://rawgit.com/w3c/aria/CR-pub/aria/aria.html#sotd

Exit Criteria

MC: Exit CRiteria is what we propose so that we can move forward

requires 2 implementations of each to prove interop

chose to take a slightly different path with 1.1, to avoid having to test all of 1.0

also, success is proven via the mappings to Accessibility API, so often only one

so, we need to list what is new, and what needs to be tested

RS: we also need to talk about the core mapping spec

MC: what we've done is to test to see if the feature can be exposed in the AAPI mapping guide

The wording in the exit criteria is inteneded to be flexible

MC: mappings themselves may be more of a challenge to get past CR

CS: questions about the mappings - they have changed a fair bit since 1.0 - will we be testing those as well?

MC: for ARIA we need to test changes to the mappings

CS: needs to test as part of the products as well
... also, there is a request to keep UIExpress - that may or may not have an impact

RS: two issues: only FF is still using UIExpress, even MS has moved on

other issues is the request has com e quite late, after WG removed previously

RS: other issue for those who are new, need to explain the importance of mappingts

Browsers are a software application in terms of a11y

just like other native a11y tools, browsers support the AAPIs

the reason we test for the mappings, so that we can be sure that what we put in our content is "talking" to the AAPI as well

so, we need to edetermine skillsets

if you cannot test the mappings, you will need to learn to use the tools

else you will need to test against the samples, or wrtie the samples

<richardschwerdtfeger> https://www.w3.org/WAI/PF/testharness/

RS: other important thing is the test harness

need to differentiate between 1.0 and 1.1

<richardschwerdtfeger> https://www.w3.org/WAI/PF/testharness/testcases?testsuite_id=4

ARIA 1.1 is pointing to core AAPI mappings

test cases are testable statements

for each testable statement, we need a simple test case

review test suite, and pick a test, click edit

<MichaelC> https://github.com/w3c/aria/tree/master/testfiles

you will see the anatomy of a test

requires access to GitHub, as that is where everything is stored - see Michael Cooper for access (etc.)

<richardschwerdtfeger> http://rawgit.com/w3c/aria/master/accname-aam/accname-aam.html

<MichaelC> To get push access to the repository, send me your github username (and description of your role if I don´t know you)

<richardschwerdtfeger> http://rawgit.com/w3c/aria/master/core-aam/core-aam.html

<clown> http://rawgit.com/w3c/aria/master/core-aam/core-aam.html#mapping_role_table

RS: in the mapping spec, scrolling down you will see a key part - role and state mapping tables

<richardschwerdtfeger> http://rawgit.com/w3c/aria/master/core-aam/core-aam.html#mapping_role_table

click on "View as a single table" you will see the mappings for each platform

RS: subtle differences between 1.0 and 1.1

big change in 1.1 is that if you are using a native host language... say in 1.0 if you were going to build a grid, you had to declare all of the semantics because HTML4.1 assumed no native semantics

but we are testing agaisnt HTML5, which has native host semantics

we also have a full automation test

CS: Edge is using UIAutomation instead of UIExpress

RS: this is part of automated testing efforts of the softwre

<clown> For comparison: https://www.w3.org/TR/wai-aria-implementation/#mapping_role_table

RS: This time we will be focusing on testing Edge (and not IE)
... we also need to determine what can be automated

FE: name and description calculations are fairly easy to automated

but roles will be more complex

FE: in SVG, automating everything, but it is ugly

RS: ARIA applies to both HTML and SVG, and both have integrated ARIA

so one question is how we divvy up the work between the HTML effort and SVG

this is not an exit criteria for ARIA, but it will be for SVG

JJ: will be automating testing in Edge

likely build a new UI client and use web driver to run the test

will inject between the browser and a regular accessibility tree using UIA

tools can also point at a git repo to get the platform tests, then navigate through them using web driver to navigate against the cases

RS: one goal for ARIA 2.0 is to merge towards a common APIs

so that it doesn't matter if it is native or web, common infrastructure

this is what we'd like to see, instead of platform specific testing

[discussion on automated versus manual tests for ORCA [sic])

mc: A TEST CASE HAS MULTIPLE COMPONENTS

looking at first test case in repository

<Zakim> clown, you wanted to note we didn't change UIA last time

there is meta data, then testable statements (minimal and generated, focused on a very narror issue)

then test file, is the 'executable' code that you can run the test on, in the case of the test harness it is manual, but it can be automated

then there is the expected results, which is done by platform -often the same, but also possible to be different by platform

each test, or groups of tests, can have different requirements

when running the tests in the test harness, if there is an expected result for a specific platform it will show that, else the general requirement

RS: one question, where do folks feel they have gaps (need education)?

do you want to limit your testss to a certain area?

we likely need to write the testable statements first, then write the test case(s)

does everyone know how to use the accessiblity test tools on the different platforms

JG: have recruited some folks, it would be good to have a training session on using the tools

also, how many folks do you need, and where?

+1 to training sessions on the tools

RS: we will need some help with testing SVG as well

?me + 1 to joanies thinking ;)

<fesch> https://www.w3.org/wiki/SVG_Accessibility/Testing/Test_Assertions

s/?me + 1 to joanies thinking ;)//me + 1 to joanies thinking ;)

[discussion around reverse mappings]

<clown> http://rawgit.com/w3c/aria/master/core-aam/core-aam.html#mapping_additional_relations_reverse_relations

JF: Deque folks could use some tools traingin as well. Skills in writing testable statements, and code samples, etc.

<richardschwerdtfeger> acl fesch

<Zakim> fesch, you wanted to ask how reverse relationships are tested

<Zakim> joanie, you wanted to ask if it makes sense to divide according to platform

Should we be dividing things up by platform?

assume everyone gets a piece to check, and then we run automated tests, etc.

so perhaps we need to divide things up based on expertese

Joanie: but writing HTML is not a strong suite

so perhaps divide based on skill sets

RS: looking for people to coordinate this effort

perhaps Jon G and Michael?

Jon, Jemma and Michael to coordinate

CS: I can assist with the more esoteric test cases

JG: wonders if the test harness is fully functional today

<jongund> https://www.w3.org/WAI/PF/testharness/testcases/edit?testsuite_id=4&testcase_id=74

JG: seems that some of the URLs etc are not hooked up correctly

MC: you found a bug, this should be easy to fix

<jemmaku> what is the time frame to finish the testing?

RS: Send a note to Jon Jemma and Michael on team s to write testable statements, etc.
... heard a need for a refresher on test tools

JF: Deque team would probably need

<fesch> +1 on getting update on test tools

<jemmaku> +q

[discussion on that training - use a webex or similar]

JG we can record this for future use

JG can also use Blackboard to coolaborate

CS: if you are at CSUN, we can meet up then for some in-person

next meeting scheduled for March 28 (post CSUN)

CS: next call would like to discuss more on automation

trackbot, end meeting

This is scribe.perl Revision: 1.144  of Date: 2015/11/17 08:39:34  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: RRSAgent_Text_Format (score 1.00)

Succeeded: s/1.-/1.0/
Succeeded: s/platvorm/platform/
Succeeded: s/WVG/SVG/
WARNING: Bad s/// command: s/?me + 1 to joanies thinking ;)//me + 1 to joanies thinking ;)
Found Scribe: JF
Inferring ScribeNick: JF
Default Present: JF, Joanmarie_Diggs, Rich_Schwerdtfeger, fesch, Jossph)Scheuhammer, Joseph_Scheuhammer, AmeliaBR, JaeunJemmaku, MichaelC, Cynthia
Present: AmeliaBR Cynthia JF JaeunJemmaku Joanmarie_Diggs Joseph_Scheuhammer MichaelC Rich_Schwerdtfeger fesch

WARNING: No meeting chair found!
You should specify the meeting chair like this:
<dbooth> Chair: dbooth

Found Date: 29 Feb 2016
Guessing minutes URL: http://www.w3.org/2016/02/29-aria-minutes.html
People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.