Accessible Rich Internet Applications Working Group Teleconference -- 16 Feb 2017

<joanie> agenda: this

<joanie> agenda: be done

<scribe> scribe: MichielBijl

<joanie> Testable statements assessment: https://lists.w3.org/Archives/Public/public-aria/2017Feb/0034.html

<joanie> https://lists.w3.org/Archives/Public/public-aria/2017Feb/0027.html

JD: that’s a link where he replied to my request

list of items literally doing a diff from the spec

<joanie> https://www.w3.org/WAI/ARIA/wiki/ARIA_1.1_Test_Case_Coverage

This is the list that Michael Cooper compiled

JD: Start putting placeholders for all of the items

<joanie> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements

Rather than another wiki page

I’ve redone the Testable Statements page

I think we had 5 areas left

I think Rich asked Stefan to do them

At no point did Rich say that there would be a ±170

<joanie> https://lists.w3.org/Archives/Public/public-aria/2017Feb/0034.html

That has some assumptions

<joanie> 1. For new, *non-global* properties: We need a test for each role that

<joanie> supports that new property.

<joanie> Rationale: We cannot assume that user agents use our taxonomy

<joanie> programmatically in their implementations. Thus for non-global

<joanie> properties, there's an excellent chance that we won't get the

<joanie> implementation "for free"; instead, user agents will likely have

<joanie> to add new support on a per-role basis.

<clown> +1

Joseph: Only see ?? for button

Do we only test button?

JD: That’s the next point we’ll get to.

Have only talked about non-global properties

I’m beyond curious and hope that people can prove me wrong on point one?

<joanie> 2. For new, *global* properties: We just need to test one representative

<joanie> role, plus any role for which there is special/custom behavior

<joanie> related to that property.

<joanie> Rationale: Global properties without role-specific special/custom

<joanie> behavior is something we *can* assume user agents will not be

<joanie> implementing on a per-role basis. Thus because the implementation

<joanie> itself is likely to be global, exhaustive testing is not needed.

<joanie> Hurray!

JD: Example - on my platform, GTK, is like a widget toolkit, we have placeholders, we have a mapping for it

Gecko never implemented that

It exposes an object attribute

<clown> https://rawgit.com/w3c/aria/master/core-aam/core-aam.html#ariaPlaceholder

In role description what the spec says it’s supported on everything

It can’t be whitespace only, can’t be an empty string, and can only be applied to an explicit or implicit role

Joseph: Shouldn’t we have examples where this is not as clear

Like role=application

JD: As long as it’s a role and the role description is valid

We could test like very last role

But I think that wouldn’t be worth all the trouble

Joseph: Is there a number of things that we’re missing?

<joanie> 3. For new roles: We just need to test non-global properties supported

<joanie> on those roles:

<joanie> Rationale: See rationales 1 and 2 above.

JD: We just need to test non-global properties supported on those roles

If it’s a global property on a role we get it for free

If it’s a non-global property we need to test it

<joanie> https://www.w3.org/wiki/ARIA_1.1_Testable_Statements

Link to testable statements ⤴

JD: Priority number 1 is we need volunteers

Please, please, volunteer for as many test as you can

And fill in your name at the test

So we don’t get duplications

Some tests have “has questions”

Sometimes it boils down to “do we need to test this”

<joanie> In ARIA 1.1, aria-controls went from a supported to a required attribute for combobox.

<joanie> QUESTION: Is this really testable? There is no implicit value. The property was supported on combobox in 1.0, so we do not need to test that. There is nothing in the ARIA spec regarding what user agents need to do if the author fails to provide a value.

JD: Joseph, can I ask you to go through the has questions?

And delete if it cannot be test?

Or update it if it can be tested

It’s really only 24

(if you don’t count the duplications)

MC: If “non-testable” is related to something in the spec wouldn’t that be a spec issue

JD: *points to question earlier*

The difference is between “this is supported” and “author must”

You can’t test that

<joanie> figure (HAS QUESTION)

<joanie> The figure role was added in ARIA 1.1

<joanie> QUESTION: The figure role has no non-global ARIA attributes. It is Joanie's assumption that global attributes do NOT also need to be tested for figure because presumably the user agent implementation of global attributes is not done on a per-role basis. If this is correct, only the role-mapping test below is needed.

JD: We have a 170 tests, but that is based of the three assumptions that we’re going through in this call

<ShaneM> we definitely do not need to test every last global attribute on every element...

JD: Are you suggesting that we leave the questions?

Can you do one so I know what you mean?

Are you saying we need to test all global attr?

Stefan: no

JD: Are you suggesting we document it elsewhere

Stefan: yes, that would be better

JD: Do you agree that given the consensus that it’s okay for Joseph to remove the question

Stefan: he can do what he want

JD: if the group consensus is that we need to test all the things

We should

JS: If it had global attributes we just tested the role mapping

JD: Are you willing to eliminate the has questions?

JS: Yes

MC: I do not believe we ever wrote ?? down

If we did it’s buried in a wiki somewhere

JS: We can put it at the top of this new page

<joanie> https://lists.w3.org/Archives/Public/public-aria/2017Feb/0034.html

Reminder: volunteers to please sign-up at the wiki

<Stefan> :)

<jongund1> can you hear me?

JD: Stefan can I ask you to edit the wiki, and add your name to everything that you’re going to test for

Stefan: yeah, but not today, will do later

JD: A lot of my writing had you in mind
... Anyone else who can volunteer

JG: I don’t have anymore bandwidth

Working on ATTA for MSAA and IA2

If we do start manual testing

I do have two additional people that can help with that

One is a mac user

JD: Any given testable statement singular has multiple assertions

Testable statements - More tests in need of writing than we realized!

ATTA progress updates

JD: Microsoft had indicated that they could provided manual testers

But won’t be doing an ATTA

But the test cases were hardcoded in

It does not in any way reflect the shared API

MS had indicated that they could convert their new testing to ATTA

They kept not doing it

And a couple weeks ago they said they wouldn’t do it

What I wound up doing is I replied to ?? and ??

Boiled down to “thank you, we’ll take you up on your generous offer of the manual testers”

But would be awesome if you could provide an ATTA for your platform

I’m starting to figure out how to drill down into the a11y tree on mac

JG: One of my students has ?? working

We can now respond to PRs and get PRs

JD: Appreciated!

Getting started on manual testing

JD: Started writing documentation for manual testing, not done yet

Might also need a version for people supporting volunteers

If you have people that can test, manually, whose attention to detail is high, we want them!

JD: Any questions

*crickets*

TPAC

CSUN

It’s not next week but the week after right?

<jongund1> I will be at CSUN

JS: Right

<jongund1> I will probably not be available while I am at CSUN

JD: We’re meeting next week, and plan to have a meeting on March 2nd

If we don’t have enough people on the call we’ll cancel

Please please please please please please please start writing testable statements

- DRAFT -

Accessible Rich Internet Applications Working Group Teleconference

16 Feb 2017

Attendees

Contents

Testable statements - More tests in need of writing than we realized!

ATTA progress updates

Getting started on manual testing

TPAC

CSUN

Summary of Action Items

Summary of Resolutions

Scribe.perl diagnostic output