Accessible Rich Internet Applications Working Group Teleconference

<joanie> clear agenda

<joanie> agenda: this

<joanie> agenda: be done

<joanie> scribe: joanie

Instructions on how to get started with manual testing

Link to documentation: https://www.w3.org/wiki/ARIA_1.1_Manual_Testing

https://www.w3.org/wiki/ARIA_1.1_Manual_Testing

ARIA 1.1 implementations in report: https://w3c.github.io/test-results/wai-aria/#index-of-implementations-in-reports

<tzviya> https://w3c.github.io/test-results/dpub-aam/#index-of-implementations-in-reports

<MichaelC> scribe: MichaelC

jd: above applies to ARIA and DPub

swapping dpub-aam for wai-aria in URIs

includes contact info of people working on platforms

I´ve done on some non-Windows systems

Jon working on IA2 testing, hopes to be ready by next week

but we still don´t have a plan for UIA testing

so at the moment, manual testing needed on that platform

MS has said they plan to do it; I´ve sent the instructions

if you get to the point you don´t have implementation data, may need to do manual testing, since the automated is happening

<joanie> https://w3c.github.io/test-results/wai-aria/all.html

I´m running automated tests for ATK and AXAPI, 2 UAs each

<joanie> https://w3c.github.io/test-results/dpub-aam/all.html

above links are test reports for ARIA and DPub

The headers in the table are explained in the index of implementations page

the two WebKit implementations show mostly passing results

the other two mostly failures

that doesn´t indicate manual testing needed - it´s valid data about not passing at the moment

it´s lack of data that needs manual testing

<joanie> https://w3c.github.io/test-results/wai-aria/all.html

in above link, fields with orangey-yellow lack data

there are a few reasons this can happen

we added tests but didn´t update report, or the test isn´t applicable to that platform, or we simply lack a test

mc: can we distinguish don´t need data from don´t have but need?

jd: documentation page gives pointers to figure that out

you can usually figure it out

often, when there´s an additional test file under a feature, it´s to test additional platform-specific requirements

<Zakim> MichaelC, you wanted to talk about what we tell Director, and if we can get additional info here

mc: Director will want to easily see difference

also would be good to add more clear ¨key¨ information

jd: requires test runner upgrade, I can approach Shane about how to get that made
... for ARIA 1.1 we added some things to existing roles that weren´t requirements in 1.1

for new roles we just test the whole role

for changes, we kinda need to test the new and old behavior

not sure if we´ll be able to do that automatically, so that could add a small set of new manual tests needed

though my goal is still to automated it

there are only 10 anyways

<Zakim> tzviya, you wanted to ask why I get permission denied messages when I click through

ts: when I follow links, i get ¨permission denied¨ errors

<joanie> https://www.w3.org/wiki/ARIA_1.1_Manual_Testing

jd: when you´re doing manual testing you shouldn´t encounter this

but it is something I plan to fix

<joanie> https://w3c-test.org/tools/runner/index.html

this is the test runner

put the shortname of the spec you want to test in the field (wai-aria or dpub-aam), with a starting slash

can take a while to load after that, then shows test case with a table

each test case is pretty minimalist

the table essentially says what you´re looking for, in the various APIs

you need to use an AAPI inspector for the platform

AccProbe for IA2?

look through the a11y tree to find the element called out in the test, and check its properties

see if it´s got the thing, or property has the value, the table says it should have

then go back to main window and choose ¨pass¨ or ¨fail¨

then it will show the next test

so you rinse and repeat

be sure not to close the test pop-up window, the test runner really wants it

<Zakim> MichaelC, you wanted to ask how you pause and resume a session

mc: can I pause a session and pick up where I left off later? e.g., after my browser has restarted so I didn´t just leave it open

jd: you can walk away from tests and come back, the harness will just sit there waiting on you

you can also select a subset of the tests at the start of the process

you can use regex, but more simply use prefix

you can get a JSON file and send it to me to incorporate into the test results repo

ts: for DPub-AAM, what´s still to test?

jd: Windows

ts: so there´s not a column for it yet?

jd: right, only the UAs listed are known to the system yet

so Windows isn´t included yet

mc: can we have placeholder columns?

jd: yes, after a code review

I haven´t because I don´t like fully empty info

mc: think an empty column is more clear for non-testing reviewers

jd: we´re still not fully sure of MS plans

tz: have access to various systems

jd: Firefox on Windows might be most useful place for you to start

suggest you test a nightly build, so you have the latest implementation

since features are landing as we speak

we are hoping MS will take care of Edge

we can more easily get nightlies of FF

also think you´ll get similar results to FF on Linux

Linux is the redheaded stepchild of the accessibility world

that´ll help you sanity check your process

tz: will look through this and come back with questions

jd: I don´t have Windows, except on a virtual machine in another country, so will appreciate additional guidance in the wiki about that platform if you have it

ts: will look at this, and ask dpub people to as well

jd: people can come to me with questions if needed
... other comments, questions, concerns, volunteers?

jva: also mid-stream on figuring this all out

<joanie> https://www.w3.org/wiki/ARIA_1.1_Manual_Testing

jd: in https://lists.w3.org/Archives/Public/public-aria/2017May/0041.html JG says he hopes to have IA2 ATTA next week

in which case all the magic happens

otherwise manual testing

ARIA 1.1 "actions / status" check in

jd: progress on IA2 ATTA

note an ATTA is an ATTA, it works for any version of specs

jd: I´m focusing on the 2 right columns in the ARIA test report, turning any failures there to passes

mc: who´s doing the other two?

jd: bugs are filed with Mozilla

This is scribe.perl Revision: 1.152  of Date: 2017/02/06 11:04:15  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: Irssi_ISO8601_Log_Text_Format (score 1.00)

Found embedded ScribeOptions:  -final

*** RESTARTING DUE TO EMBEDDED OPTIONS ***

Present: Joanmarie_Diggs tzviya MichaelC matt_king Job
Regrets: Michiel_Bijl Rich_Schwerdtfeger Jon_Gunderson
Found Scribe: joanie
Inferring ScribeNick: joanie
Found Scribe: MichaelC
Inferring ScribeNick: MichaelC
Scribes: joanie, MichaelC
ScribeNicks: joanie, MichaelC
Found Date: 18 May 2017
Guessing minutes URL: http://www.w3.org/2017/05/18-aria-minutes.html
People with action items: 

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.