Meeting minutes
s3ththompson: we're kicking off our 2021 work on ARIA-AT project in our standard way at Bocoup, but this year we're going to be running the project in public.
introductions
s3ththompson: we have myself Seth Thompson (he/him), working for Bocoup as Product Owner, Jes Daigle (she/her), eng manager at Bocoup, and Boaz Sender (he/him) supporting the project from Bocoup.
s3ththompson: we also have Mat King who is the project sponsor
s3ththompson reads the Bocoup Code of Conduct summary, and references the w3c CEPC
Scope
s3ththompson: (Reading out the goal from the agenda) our goal is to design and implementation of a hybrid manual and automated testing architecture in the aria-at.w3.org app and github.com/w3c/aria-at test suite. Improvements to the existing manual system. Strategic changes to test authoring tools that help test-authors scale up output. Support for new strategic test suite development.
Matt_King: I want to clarify that this is the app and test runner, but that Sina is running a different project writing tests, and Bocoup is also running another separate project to run automation.
s3ththompson: good point
s3ththompson: this project scope is the app running at https://
Success Criteria
s3ththompson: like we said this is currently a manual test running app
s3ththompson: and we also want to automate, which we'll get to in our success criteria
s3ththompson: our first criteria is about supporting material outside of APG: "Integrate automated tests alongside manual tests and extend tests beyond APG"
s3ththompson: our second criteria is to scale our design to support a lot more tests and test results: "Prove that our app scales as test corpus and results matrix increases in size"
s3ththompson: our third success criteria is about making a more unified experience across all our connected projects in this area, and deal with our acronym soup better: " Unify identity across WAI, ARIA-AT, and APG"
s3ththompson: our fourth success criteria is to make sure that the data from this app are available to web devs, browser teams, and AT makers: "Bring ARIA-AT testing insights to web developers, browser implementers, and AT vendors"
s3ththompson: and our last criteria is to "Involve screen readers and marginalized AT users, manual testers, and others as co-designers"
s3ththompson: anyone else have anything to add to the success criteria?
michael_fairchild: I think these are great, and I don't have others to add.
Jemma: as someone working on APG, I'm really happy to see the integragtion.
Matt_King: I think we might want to make some of these more specific, once we are ready to discuss our sucess metrics
Matt_King: it might be good for us to add key success metrics to the aria-at app
s3ththompson: I like that idea
<Jemma> One of measurements can be the percentage of integration rate with exisiting APG examples
s3ththompson: the next section is priorities, let me add a measurment dashboard to that priorities list
s3ththompson: there was also sina's idea about number of key strokes
Matt_King: yes
Matt_King: I think it would be great if 2-3 times this year we could stop, take a breath and share with the world where we are
s3ththompson: yes, great, lets make a priority for that, potentiall
Priorities
s3ththompson: let me read through the prios
I went through all the work we had in our backlog at the end of last year, and the above criteria to write these
s3ththompson: first, we'd like to focus on the API: "Build a better API foundation and data model that encompasses alternate test sources and automated tests"
s3ththompson: right now we assume all tests are manual and from APG, and we might change that
s3ththompson: next we might do something that to "Improve smart updates for evergreen tests", and this really speaks to the ability to make small changes to test results with minimal overhead
s3ththompson: next we'd liket to "Evolve ARIA-AT working mode and improve alignment with App publication workflow." this goes back to the bar for merging tests.
s3ththompson: the working group might be able improve workflow based on the state of a test
s3ththompson: the next priority is to " Develop rendering engine for test runner as external library (remove iFrames from App)." this is in order to improve accessibility for AT users by getting rid of iframes.
s3ththompson: this could also be for other consumers, not just the app
s3ththompson: next priority is "Add filtering and advanced analysis features to accessible Reports page."
<Jemma> yay for integrating with APG site!
s3ththompson: next we'd like to "Build embeddable version of Reports page for MDN and APG."
s3ththompson: next we'd like to "Upgrade App to support hybrid manual and automated test results." This will be complex.
s3ththompson: next priority is to "Upgrade App to support hybrid manual and automated test results." so again this about allowing us to test non-apg patterns.
s3ththompson: finally, wed like to "Add accessible UX affordances to handle increased scale of data, including pagination and search"
<Jemma> It sounds like a good idea to start with backlog from the previous project.
s3ththompson: do folks have additional priorities in mind?
Matt_King: I think this is the most important part of the meeting.
<Jemma> The flexibility of AT app embed is great.
Matt_King: we really want folks input on this.
<Jemma> but I do see the challenge for builiding API itself.
Matt_King: I really want alignment on these pieces.
s3ththompson: this is about the big pieces of the work for us to align on.
s3ththompson: I see a comment in IRC from Jemma about starting with the backlog. Which is good. We did start there, and this comes from that. Thanks!
Matt_King: let me start with something very specific: it would be amazing if we could prioritize the working mode changes, and merging a test to the main branch easliy.
<Jemma> "Parallel velocity" is really a good key word to remember.
s3ththompson: ok, I'll move that up.
Matt_King: my next question is, is there a feature in the app that is slowing down automation?
Matt_King: my 10k foot view is that humans will run this the first time always.
boazsender: I was thinking sometimes we could start with automation
jscholes: I think we need a human tester at the beginning
michael_fairchild: I agree, but I still think we can automate them earlier, like boazsender is saying
s3ththompson: I wonder if we are conflating the tool as an app for test writers and one for test runners, and perhaps there are two different workflows.
Matt_King: I was hoping the app could manage this
Matt_King: manage some of the writing
s3ththompson: I'll clarify that in the priorities
s3ththompson: is there anything else we're missing?
<Jemma> +1 for Boazsender
Matt_King: do we have a bucket of priorities around usability and accessibility?
Matt_King: do we have something to revisit the ux assumptions after research?
s3ththompson: great point, I'll add that.
jscholes: how do we communicate priorities we have for the script/runner changes?
jscholes: does that go here?
s3ththompson: yes, this is the right place. We can add a priority to focus on API changes.
Matt_King: I think this is related to things like reusable tests, and the way we compile key commends
jscholes: yes, but also github actions, and test formats
boazsender: maybe the prio could be "meet with test writers, catch their pain points, work them into the project"
jscholes: we could name all our issues in the next test writing meeting
s3ththompson: I think some of this is also covered by our named priorities, like expanding past APG, to include material from ARIA attrs, etc
Matt_King: agree
jscholes: agree
jscholes: its the data model
s3ththompson: exactly
Matt_King: could we make this the focus of the next test writing meeting?
s3ththompson: sure!
s3ththompson: ok, wrapping up priorities, please let know after the call if you have anything to add
Logistics
s3ththompson: we're starting work at Bocoup on February 16th.
s3ththompson: what time is the test writing meeting? is it the same time?
Matt_King: no, it's different every week.
s3ththompson: ok, can everyone who wants to attend this regularly send me their calendars?
Matt_King: one possibility might be the hour before this meeting. 11-12 PST on Thursdays
s3ththompson: ok, we'll look at that
s3ththompson: other things to share: we're moving our task management to public github.
Seth: that last time we did work together, there was sprint planning and then share with Matt
Secth: Is there a better way to share weekly sprints
<boazsender> rrsagent: generate
James: We are interested in the schedule to know what's happening so we can comment if needed
<boazsender> rrsagent: please create the minutes
Seth: Publish expected sprint plan for comment
Seth: I think we can do that
Matt: Especially for the first couple of priorities this is important, we want James involved the sprint
James; I would be happy to attend
Matt: That would be a stronger alignment between app and test development
Seth: Previously we would make changes to the App and assign someone on the team to review
Seth: Is anybody willing to review
Matt: I don't want external dependencies, but we want to do planning so external people have a chance to review
James: I would be happy to look at accessibility
Jon: I would be interested too, I have accessibility concerns
Matt: I would like to consider the process for the accessibility and usability, that has a longer time line, put in staging and get feedback, deploy after feedback is addressed
Seth: I put some work into the design process
James: It is a lot easier to review the insitu, the actual app
Matt: That speaks James to the idea, the accessibility and usability of the reports page, break up development into 6 small steps that we all agree are great
Matt: I think the community group can handle a1 month pace, people will appreciate the monthly progress
Seth: New things on a monthly schedule
Matt: Mostly the UI stuff, the backend can work on its own schedule
Seth: I like that framing
Seth: it involves our deployment process with staging and data
James: The UI might be at a different pace than the data model, we can do some testing with the UI
Seth: This is a bigger thing than what we were talking about, because we are using a W3C server, its not quite so easy than using commercial deploy
Matt: I don't want that be a major topic, let's do the best we can, humans are going to need to adapt to the infrastructure
Seth: I won't invent new work items, that is what I had for work plan and communication
Seth: Even though we have two separate repositories, but they are all one project
Seth: any other questions be fore action items?
Action: Item: Email James to set up time to create shared priorities with test writing (s3ththompson)
Action: Item: Develop project plan from kickoff priorities / roadmap (Bocoup)
Action: Item: Confirm AT Automation time with Simon and coordinate meeting invitation (s3ththompson)
Seth: One zoom link for all meetings?
Matt: the Deque one
Seth: I will use that link for all ARIA AT meetings
Seth: When are they sent out, Monday?
Action: Item: Seth to send out regular meeting invites for ARIA-AT on Wednesday, with tentative sprint plan (s3ththompson)
Matt: We usually do Wednesday, but they can be earlier
Seth: Probably Wednesday will have best informatino
Matt: Seth you need to become a cohost
Action: Item: Confirm that Seth can be co-host for meetings (s3ththompson)
Action: Item: Seth confirm with Michael you can be a cohost
Seth: Any other topics?
Action: Item: Confirm time for weekly ARIA-AT workstream, tentatively 2pm ET Thursdays (s3ththompson)
Matt: Next week there will be three meetings, automation on Wednesday,
Matt: How long before a project plan?
Seth: This will probably start in about 3 weeks, look for the invitation
Matt: There will only be two meetings next week