Meeting minutes
Introduction.
jasonjgw: we have a new participant. They're not on the call yet. [We'll move on since we're having connection issues - Shawn can give background, if needed]
Accessibility of Machine Learning and Generative AI draft.
jasonjgw: there were some issues with the bibliography. Scott clarified, and very few issues, they've been fixed. All in order, as far as I'm aware. When build system on Git tries to generate clean HTML version on server, it runs into a time out and it's failing for some reason. Not sure why, locally it's fine.
Janina: where do you see the time out?
jasonjgw: get an email. Go to website and get logs and shows the time out
Janina: from Git? Suggest loop in Roy. We have new tech that streamlines biblio js, something might be going on with it, shouldn't be surprised about bugs.
jasonjgw: it reports remaining time in each command in sequence, but no warning message to let you know what actually happened. On server so i can't debug locally. Might be able to push generated version manually...
Janina: strongly suggest email to Roy and cc APA chair to archive in case it needs to go to development people.
Jasonjgw: ok, maybe later today or tomorrow.
Scott: once completed all references and citations pending will be complete
Scott: i think we've got it all sorted outside of tech issue. I think it's all in place, Janina won't have to go back in...
jasonjgw: I think it's all settled now
… Janina - you have a round of changes to propose. When you feel energetic, implement those changes and we'll come back and discuss them.
Janina: yes. And it won't be anything we haven't discussed. I don't think I've run into anything we haven't touched on. Need to go through minutes to make sure I haven't forgotten anything.
jasonjgw: ok, proposed revisions in a branch from Janina. Thank you again for your work on this.
Shawn: I don't think we're going to get Irene today. I can talk more about that now? Full minuting
… related to this topic.
… First, would be helpful for me and Irene to get an overview of where you are and summary of the status of the document.
jasonjgw: discussions we've been having on issues related to ML based systems and discussions with web and AI interest group and discussion s internally in task force, are ahead of where the draft document is. Janina making substansive revisions we've discussed but not yet incorpoirated. then we'll disuccs what to do to get ready for first public
working draft. Get up to date with discussions going on. Work taking place standardization efforts in Canaada and work (Jason) is involved in, not directly related here. Authoring tools guidelines group starting as well. All going on in the context of what we're discussing and I think as far as thi draft, going to take janina's proposed reivions
and see where we are and get the draft up for review.
Scott: add to that - we have a lot of literature that's cited. And that's part of the tech issue related earlier in the meeting. This wasn't visible before and that will pop up soon.
Janina: that's what not's building
… my summary, of what I'll be doing is reorganizing TOC so we will broaden the scope of the perspecitve of which we discuss AI. Right now primarly a client using in a browser. Expand in multiple directions. Don't want to lay that out top of my head right now. But main next step.
Jasonjgw: use of agent based system becoming significant and not represented in teh draft right now. Web technologies being used extensively in those environments...agent capabilities built into web browsers. Going to have to cover that and an issue that hasn't been addressed elsewhere yet.
Janina: and yet some basic computer expectations not met yet...can you just write results in a file instead of making me clipboard?
Shawn: first, thank you for that overview. Follow up on ATAG and AI and a11y standsrds Canada, have been coordiatning with them there, revive and ATAG working group and formally with with a11y standards Canada and they're particulalry inerested in AI, we're coordinating with them. And with the scope of the paper - very interested in all the
different aspects of AI and accessibility. What issues address and what issues it's not (in the paper - sceop)
janina: cover all of them at least at a high level. They deserve a response. We've minuted what those would be, but won't note that from memory. Not jsut as a user but also as a programmer or content creator, using a tool
Shawn: it is about using tools - tools themselves, issues with bias, etc?
janina: i expect we will have something to say about that, jason more than I. Main concern is TOC that makes sense to cover all of those different perspectives and isseus in a way we can fill in.
jasonjgw: The field has moved even in the last 6 months. Will have to revisit the conversation.
Irene's work
Shawn: we have Irene, a PHD student at MIT. Have some of her time, specifically funded to help with WAI work. Given her interest in AI, we thought helping this group with literature review might be a good fit. Opportuntiy for Irene to help. Limited term project Feb - May is opportunity. this group - think about how Irene could help, and if you
recall I had opened an issue suggesting that we have a literature review that goes beyond bibliography, but a comprehensive review that we might provide in a different format. all of you have done a lot of work on that already, but I don't know the status. You define what you think will be most useful, what's been done, and how Irene can
integrate/contribute into what you already have or additional information, in the format for this task force.
… jason - do you want to talk more about Irene's work now or introductions?
jasonjgw: we should mention briefly who we are. And move into the discussion
… [intros happen]
<scott_h> wiki on AI work https://
Shawn: how can we get Irene going - 1. Make sure Irene knows what's already been done (anythign besides wiki?) and 2. Directions for Irene's work right away, and 3> Best way for sharing information going forward. Wiki? Something more robust? Irene is familiar with Git. So might be other options.
Janina: and we have Irene through May?
Irene: mid-May, but open to collaborate after that
Janina: my suggestion - let them know about biblio.js more than just the wiki. So a task that can be done in the next two months. And become a basis we can build from and hopefilly Irene can be with us after. My thought - review literature what we've looked at. What's relevant, what we missed. Others? jason? Scott?
Scott: Can you drop the link in IRC so Irene can look at that?
Janina: Jason, do you remember the repo links? (usually do command line)
jasonjgw: TOC will give you a framework, but as Janina indicated, reviewing wthat's there and what's relevant would be a good starting point. When we have a better organized TOC for the draft, good way to set up parameters of what you'd be looking for.
… as one small example, accuracy issues associated with various applications of machine learning with a11y - like speech or image recognition and lastest findings on accuracy issues. Anythign written about. One area to think about, and i know various metrics they use in designing the ML systems, and not sure relevant to users to judging how
relevant the system is to the user. Just one area.
… w3c.githhub.io/ai-accessibility
<Zakim> shawn, you wanted to ask scope and to note ATAG and AI and to note existing doc, abstracts, commentary and to say broad definition of "literature", analysis
Shawn: a few things, this is great. Irene will be able to document not jsut this list orf resources, but the points in the resources. I think at one point there are abstracts? What has been done with documenting or analyzing what these resources say?
Scott: There is a comprehensive doc available in that GitHub, including abstract, TOC< examples of AI. Been a comprehensive development. there is a structure of what's been written, once bibliography - i think 40 references at the moment - still needs work and content Janina will be working through. Irene - look through and contribute will be
really exciting.
Jasonjgw: thanks Scott. There is a draft that follows w3c and scholarly practice. Cited as relevant and used in the text. We make sure we're familiar with the issues and literature and make references as appropriate in the draft.
Shawn: Irene has looked at it in the past. Will be good to review after Janina's revisions are made.
… For Irene's work - have a broad definition of "literature" - not jsut research papers. so broader...reddit threads, websites, other things to gather more user experiences not just formal literature.
Janina: should we think of anything citable?
Shawn: Anythign that can inform the information, inform what this group is documenting. That what Irene finds, analyzes, and documents might not be citable but still informative of thinking of the basis of the paper.
Janina: it can influence, biut leery about saying "here's this thing, but we can't point you to it" - we cite it when we use it
Shawn: 2 different things. Formally citing as a references (standard protocol) and seprate literature review that has all the information of all the info we found and reviewed but not all referenced in the paper.
Scott: wiki set up for that purposed. Putting things that could be useful. I don't think there's an issue to expand that. but to janina's point - what actually is pointable in the document.
Janina: URI citable, not academic. As long as it isn't going to go away is how I'm using citable.
Scott: wiki, we can put things there that we need to consider.
Janina: if we point to a reddit thread how does that go in a biblio...
Jasonjgw: high quality research is more valuable than anecdotal info that can be misleading. Rigirous methods applied and alternate hypotheses and all those advantages of research. Prioritize high quality.