16:48:26 <RRSAgent> RRSAgent has joined #aria
16:48:26 <RRSAgent> logging to http://www.w3.org/2017/05/18-aria-irc
16:48:28 <trackbot> RRSAgent, make logs world
16:48:31 <trackbot> Zakim, this will be
16:48:31 <Zakim> I don't understand 'this will be', trackbot
16:48:31 <trackbot> Meeting: Accessible Rich Internet Applications Working Group Teleconference
16:48:31 <trackbot> Date: 18 May 2017
16:48:35 <joanie> chair: Joanmarie_Diggs
16:48:35 <joanie> RRSAgent, make log public
16:48:37 <joanie> agenda?
16:48:48 <joanie> clear agenda
16:48:53 <joanie> agenda: this
16:49:07 <joanie> agenda+ Instructions on how to get started with manual testing
16:49:07 <joanie> agenda+ ARIA 1.1 "actions / status" check in
16:49:10 <joanie> agenda: be done
16:49:16 <joanie> agenda?
16:49:29 <joanie> Regrets: Michiel_Bijl, Rich_Schwerdtfeger, Jon_Gunderson
16:49:41 <joanie> scribeOptions: -final
16:57:16 <JasonDuan> JasonDuan has joined #aria
16:57:51 <joanie> present+ Joanmarie_Diggs
17:00:04 <tzviya> present+
17:01:38 <joanie> scribe: joanie
17:01:58 <joanie> Zakim, take up item 1
17:01:58 <Zakim> agendum 1. "Instructions on how to get started with manual testing" taken up [from joanie]
17:02:26 <joanie> Link to documentation: https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
17:02:41 <MichaelC> present+
17:03:03 <JasonDuan> JasonDuan has joined #aria
17:03:41 <Stefan> Stefan has joined #aria
17:04:27 <jcraig> jcraig has joined #aria
17:05:01 <joanie> https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
17:05:24 <joanie> ARIA 1.1 implementations in report: https://w3c.github.io/test-results/wai-aria/#index-of-implementations-in-reports
17:05:33 <tzviya> https://w3c.github.io/test-results/dpub-aam/#index-of-implementations-in-reports
17:05:46 <MichaelC> scribe: MichaelC
17:06:01 <mck> mck has joined #aria
17:06:10 <MichaelC> jd: above applies to ARIA and DPub
17:06:23 <MichaelC> swapping dpub-aam for wai-aria in URIs
17:06:43 <MichaelC> includes contact info of people working on platforms
17:06:55 <MichaelC> I´ve done on some non-Windows systems
17:07:14 <MichaelC> Jon working on IA2 testing, hopes to be ready by next week
17:07:21 <MichaelC> but we still don´t have a plan for UIA testing
17:07:28 <MichaelC> so at the moment, manual testing needed on that platform
17:07:36 <mck> present+ matt_king
17:07:39 <MichaelC> MS has said they plan to do it; I´ve sent the instructions
17:08:19 <MichaelC> if you get to the point you don´t have implementation data, may need to do manual testing, since the automated is happening
17:08:39 <joanie> https://w3c.github.io/test-results/wai-aria/all.html
17:08:48 <MichaelC> I´m running automated tests for ATK and AXAPI, 2 UAs each
17:08:55 <joanie> https://w3c.github.io/test-results/dpub-aam/all.html
17:09:03 <MichaelC> above links are test reports for ARIA and DPub
17:09:48 <MichaelC> The headers in the table are explained in the index of implementations page
17:10:20 <MichaelC> the two WebKit implementations show mostly passing results
17:10:29 <MichaelC> the other two mostly failures
17:10:45 <MichaelC> that doesn´t indicate manual testing needed - it´s valid data about not passing at the moment
17:10:53 <MichaelC> it´s lack of data that needs manual testing
17:10:56 <joanie> https://w3c.github.io/test-results/wai-aria/all.html
17:11:22 <MichaelC> in above link, fields with orangey-yellow lack data
17:11:36 <MichaelC> there are a few reasons this can happen
17:12:34 <MichaelC> we added tests but didn´t update report, or the test isn´t applicable to that platform, or we simply lack a test
17:14:03 <MichaelC> mc: can we distinguish don´t need data from don´t have but need?
17:14:23 <MichaelC> jd: documentation page gives pointers to figure that out
17:14:31 <MichaelC> you can usually figure it out
17:14:32 <MichaelC> q+
17:15:11 <MichaelC> q+ to talk about what we tell Director, and if we can get additional info here
17:15:48 <MichaelC> often, when there´s an additional test file under a feature, it´s to test additional platform-specific requirements
17:16:12 <tzviya> q+ to ask why I get permission denied messages when I click through
17:16:50 <MichaelC> ack me
17:16:51 <Zakim> MichaelC, you wanted to talk about what we tell Director, and if we can get additional info here
17:17:12 <MichaelC> mc: Director will want to easily see difference
17:17:30 <MichaelC> also would be good to add more clear ¨key¨ information
17:17:47 <MichaelC> jd: requires test runner upgrade, I can approach Shane about how to get that made
17:18:44 <MichaelC> jd: for ARIA 1.1 we added some things to existing roles that weren´t requirements in 1.1
17:19:24 <MichaelC> for new roles we just test the whole role
17:19:35 <MichaelC> for changes, we kinda need to test the new and old behavior
17:19:52 <MichaelC> not sure if we´ll be able to do that automatically, so that could add a small set of new manual tests needed
17:20:02 <MichaelC> though my goal is still to automated it
17:20:13 <MichaelC> there are only 10 anyways
17:20:15 <MichaelC> q?
17:20:24 <MichaelC> ack t
17:20:24 <Zakim> tzviya, you wanted to ask why I get permission denied messages when I click through
17:21:00 <MichaelC> ts: when I follow links, i get ¨permission denied¨ errors
17:21:30 <joanie> https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
17:21:58 <MichaelC> jd: when you´re doing manual testing you shouldn´t encounter this
17:22:03 <MichaelC> but it is something I plan to fix
17:22:03 <joanie> https://w3c-test.org/tools/runner/index.html
17:22:21 <MichaelC> this is the test runner
17:22:43 <MichaelC> put the shortname of the spec you want to test in the field (wai-aria or dpub-aam), with a starting slash
17:23:46 <MichaelC> can take a while to load after that, then shows test case with a table
17:24:33 <MichaelC> each test case is pretty minimalist
17:24:49 <MichaelC> the table essentially says what you´re looking for, in the various APIs
17:25:11 <MichaelC> you need to use an AAPI inspector for the platform
17:25:19 <MichaelC> AccProbe for IA2?
17:25:46 <MichaelC> look through the a11y tree to find the element called out in the test, and check its properties
17:26:15 <MichaelC> see if it´s got the thing, or property has the value, the table says it should have
17:26:27 <MichaelC> then go back to main window and choose ¨pass¨ or ¨fail¨
17:27:01 <MichaelC> then it will show the next test
17:27:11 <MichaelC> so you rinse and repeat
17:27:11 <MichaelC> q+
17:27:23 <MichaelC> q+ to ask how you pause and resume a session
17:27:45 <MichaelC> be sure not to close the test pop-up window, the test runner really wants it
17:28:44 <MichaelC> ack me
17:28:44 <Zakim> MichaelC, you wanted to ask how you pause and resume a session
17:29:10 <MichaelC> mc: can I pause a session and pick up where I left off later? e.g., after my browser has restarted so I didn´t just leave it open
17:29:25 <MichaelC> jd: you can walk away from tests and come back, the harness will just sit there waiting on you
17:30:21 <MichaelC> you can also select a subset of the tests at the start of the process
17:31:27 <MichaelC> you can use regex, but more simply use prefix
17:33:10 <MichaelC> you can get a JSON file and send it to me to incorporate into the test results repo
17:33:26 <MichaelC> ts: for DPub-AAM, what´s still to test?
17:33:29 <MichaelC> jd: Windows
17:33:36 <MichaelC> ts: so there´s not a column for it yet?
17:34:57 <MichaelC> jd: right, only the UAs listed are known to the system yet
17:35:03 <MichaelC> so Windows isn´t included yet
17:35:19 <MichaelC> mc: can we have placeholder columns?
17:35:29 <MichaelC> jd: yes, after a code review
17:35:54 <MichaelC> I haven´t because I don´t like fully empty info
17:36:22 <MichaelC> mc: think an empty column is more clear for non-testing reviewers
17:37:43 <MichaelC> jd: we´re still not fully sure of MS plans
17:37:51 <MichaelC> tz: have access to various systems
17:38:10 <MichaelC> jd: Firefox on Windows might be most useful place for you to start
17:39:13 <MichaelC> suggest you test a nightly build, so you have the latest implementation
17:39:35 <MichaelC> since features are landing as we speak
17:39:48 <MichaelC> we are hoping MS will take care of Edge
17:39:56 <MichaelC> we can more easily get nightlies of FF
17:41:10 <MichaelC> also think you´ll get similar results to FF on Linux
17:41:14 <MichaelC> Linux is the redheaded stepchild of the accessibility world
17:41:56 <MichaelC> that´ll help you sanity check your process
17:43:13 <MichaelC> tz: will look through this and come back with questions
17:44:19 <MichaelC> jd: I don´t have Windows, except on a virtual machine in another country, so will appreciate additional guidance in the wiki about that platform if you have it
17:51:00 <MichaelC> ts: will look at this, and ask dpub people to as well
17:51:12 <MichaelC> jd: people can come to me with questions if needed
17:52:23 <MichaelC> jd: other comments, questions, concerns, volunteers?
17:53:03 <MichaelC> jva: also mid-stream on figuring this all out
17:54:13 <joanie> https://www.w3.org/wiki/ARIA_1.1_Manual_Testing
17:54:16 <MichaelC> jd: in https://lists.w3.org/Archives/Public/public-aria/2017May/0041.html JG says he hopes to have IA2 ATTA next week
17:54:27 <MichaelC> in which case all the magic happens
17:54:40 <MichaelC> otherwise manual testing
17:54:49 <MichaelC> zakim, next item
17:54:49 <Zakim> agendum 2. "ARIA 1.1 "actions / status" check in" taken up [from joanie]
17:55:17 <MichaelC> jd: progress on IA2 ATTA
17:56:01 <MichaelC> note an ATTA is an ATTA, it works for any version of specs
17:57:05 <MichaelC> jd: I´m focusing on the 2 right columns in the ARIA test report, turning any failures there to passes
17:57:37 <MichaelC> mc: who´s doing the other two?
17:58:10 <MichaelC> jd: bugs are filed with Mozilla
18:02:27 <MichaelC> rrsagent, make minutes
18:02:27 <RRSAgent> I have made the request to generate http://www.w3.org/2017/05/18-aria-minutes.html MichaelC
18:02:40 <jcraig> jcraig has joined #aria
18:03:13 <MichaelC> present+ Job
18:03:16 <MichaelC> rrsagent, make minutes
18:03:16 <RRSAgent> I have made the request to generate http://www.w3.org/2017/05/18-aria-minutes.html MichaelC
18:03:19 <MichaelC> rrsagent, bye
18:03:19 <RRSAgent> I see no action items