Meeting minutes
Accessibility of Machine Learning and Generative AI.
Jasonjgw: first part. Coordination on ongoing draft, second, expand topic to discuss Canadian AI standards. Germain to current topic. General coordination...
Janina: Nothing to read yet, need to do the edits from Scott.
… Due to outside work/contract conflicts.
Jasonjgw: wnat to report on progress next week?
Janina: Yes.
Jasonjgw: Check in next week, no pressure intended.
… second matter Canadian standards doc. Link in the agenda
Miscellaneous topics.
Accessibility of Machine Learning and Generative AI.
Canadian standards
jasonjgw: Brief overview - essentially this is a standard that's been developed under Inclusive design research center in Toronto. Under Accessible Canada Act, which is a fairly recently established act. Not sure what roles it plays under the standard as of yet, such as with anti discrimination cases. It covers a range of issues, relate to the
title, in particular it addresses the accessbility of AI systems including the software and the output in one clause. That section focuses on a range of a11y requirements cited, the En 301 549 standard, which in my view is the most comprehensive standard at the moment.
Janina: this is about teh a11y of the software interface?
jasonjgw: Yes, the output and the tools use...Whole lifecycle. That's one component of this. Second component is it addresses the role of AI systems in making decisions. So any where rights of persons with disabilities would be affected by automated decision-making, it would address issues...statistical outliers, potential misrepresentation, and
role of human intervention in that process. That's a whole section that's separate from the a11y component. There are also provisions related to the governance of AI systems, that includes role of people with disabilities for deployment, usage, and monitoring. Entire section on that. The harms that can potentially be created by those systems,
takses into account reputational harm, like sterotypes on a particular type of disability created by the AI system. So also deals with misrepresentations of stereotypes that will cause harm to people with disabilities. Each of these dimensions, it strikes me as a relatively comprehensive standard. Some discussion on AI systems as an accommodation
by people with disabilities and the need for human alternatives for that. I think in general it covers most of what we would want to address in the draft we're working on if even briefly. Vitally important for evolution in AI systems and possible consequences for people with disabilities. In favor of the intent and the substance. But not clear if
they're accepting comments for future version.
Janina: is it legally adopted?
jasonggw: Yes. This is the formal publication of the adoption.
… there were public comments and it took time to address those. It can be cited.
Janina: yes, I think we might cite. Minimize repetition., though, for things already discussing.
jasonjgw: suggestion, for those who haven't read it yet, good idea to read it and circle back to it next week.
jasonjgw: will be more elaboration in the future. Not supposed to be the last word.
Janina: will be an interesting year for AI and accessibility.
Jasonjgw: come back to it next week (or the week after), hopefully Scott will be back, and Janina will have a chance to check this out
… Janina - complete the work on the draft. That can get merged. Then we look at Candian work and see how it can emphasize our work...cite...separate work from this one. Will take it up next week?
Add a subtopic on how we personally use AI for?
Personal AI use
Janina: Eager to hear what other people are doing... Primary use has been chat GPT, i bought an account. It knows me. Use if mostly from iPhone. Sometimes type sometimes talk. Most useful was go through personal profile and save a profile. So i told it that I'm Blind, tools I used, understand colors and shapes, etc. and told it a few more things. I
am pleased when I ask it to do tasks for me, it does reference that and is cognizant of that profile. Looking for a way to talk to multiple computers from one keyboard, and I bought a keyboard...i didn't know it had a manual switch. I unboxed and felt around, i found multiple position switches and ports...and thought what are these? Googled and
didnt' get far, so took it to Chat gpt and told it the make and model and asked about swtiches and ports and the bottom row of keys about which key is which (function keys to swtich). It gave me the steps it was doing. Repetitive so hard to listen to (re-read entire thing instead of from where it stopped). But got a simple report on what those
switches and ports are and what they're for. Impressed.
… and it did get it right when I checked
Jasonjgw: I think one application I use in the last few years is testing capabilities. When I know the answer to a question and want to see how LLM respond to query and prompts and evaluate the output. Like generating code or answering technical question, or to generate information about topics in philosophy I have an interest in. Providing prompts
for purpose of testing output...quality varies, but it can be good under favorable circumstances. If you're precise you'll ge better respones. Technical quesetions instead of performing a web search. Sometimes it will give good info in a summary. Use Gemini and Chat GPT and Microsfot version that runs Chat GPT, results are fairly similar. Vary in
quality depending on the query. Difficult to say for the queries I've been performing. If I ask a technical question and try and verify accuracy, then it can be useful and faster than web searches and find online and extract information. Manual can be tedious and slow. An improvement to some extent. And fianl tasks, to generate image descriptions
from books and articles I'm reading. Problem is I don't know how accurate the description is. Output is plausable in context, but hard to know if the details are right. Sceptical in taht regard. No easy way to verify without calling s ervice and a human agent to verify the output. Some services offer a human backup. Generate a description with LLM
and submit for a human review. Seems like a useful service, and someone who is interested in scholarship and research I'd be relying on the image description to be accurate for scholarly work. So can't just generate and hope for the best, so depend on a degree of accruacy. Played with Machine learning based text to speech systems, and found they
tend to ffer fairly good text to speech but performance is known for being problematic, which is why not used in screen readers. expericen in those including experimental and proprietary. One of them running locally on one of my systems.
stacey: Company asks us to use Gemini -- To develop descriptive options
stacey: Designer has this problem; How can we help
<janina> s/gemini/Co-Pilot/
stacey: for personal using Gemini
stacey: Asks it for review and suggestions in writing
stacey: Has been great at taking my data, even though sometimes comes up with wrong guidance
stacey: It has given me alternative viewpoints to consider and that's been very helpful
stacey: I consider it my 10-year old assistant!
stacey: I need to be very clear in instructions and what I expect as output
stacey: I have a friend who wrote a boot on how to use AI; had dev'd a questionairre to develop one's AI profile style
stacey: to resonate with how you think
stacey: Think this profile approach very useful
stacey: Don't know whether he took that up ...
stacey: I don't use it write my content, but I do use it as a check on what I write
stacey: Somewhat like an Editor; I don't always agree, of course!
stacey: Trying to be aware of what it's good for, and what not
Jpaton: used Ai quite a bit. First thing, reworking text, Make this read better, But have to give examples of own work, But particular sentence structure I know that they took it rhough Ai. So trying to get a fingerprint of my style. Enhanced search, links for resources. Trying to find agenda for a meeting and it says sure. I can't find it, did you
make it up? And it came back cheerfully and said yes I did, you caught me. I prefer Claude. I think it'd designed to check for accuracy. Free trial on that. Claude is from anthropic. I've used it for work for quite a few things. We've written a white paper on gaming accessibiltiy legislation. and something about the quotes and how it regulates
epgs, guides...a bit roundabout because of the legal scope they have. Asked the AI, gave a link to the legislation, is it fair to say this. It came back and said it'd be better to say something else. So I had a check back and agreed. Also an a11y kick off report generation. Feed in the aspects, FPS, other teammates, etc. Use a backedn database for
the report...if you have this in the game you need to think about this. Asked Claude to help populate the database, so every time something is visual have to think about size, contrast, etc. So populate and do that work and i check it. Quite good for code simulation, works a lot with markdown files. Stick it all in headings and get the
architecture, and it understands it better. Need to put in everything. Tested it and it didn't work, asekd it to be WCAG compliant and then it did it. As long as you have that profile, I think that should help. I do recipe creation, I wanted to try cooking paella in the oven to bake, and i asked Gemini do you have instructions for this and it came
back and gave ideas on how to do that. Ratio, temp, etc. Good for that. Anyhting to come back and look at rules and patterns especially with cooking times, it's quite good at suggestions. I agree - i don't like copilit. Gemini is good for creative. Claude good for accuracy.
Jasonjgw: should we visit misc?
Miscellaneous topics.
jasconjgw: call for consensus for COGA. for first public working draft
Janina: due tonight. Fair number of +1s and one +0, not a negative vote but a good suggestion, put a Git issue in and they should work on that but maybe not block
Jasonjgw: anything else going on?
Janina: i think at the moment, no
jasongjgw: I've been reading recent articles about the ways in which free and open source software communities respond to LLM, and if they'll accept code created by the systems and in what conditions. Seems to be a growing interest in LLM to carry out code reviews, patches to linux kernal where a shortage of human reviewers. If they can use LLM
first before a human review. Might reduce amount of review required. Not using to write as much to evaluate it.
Janina: there are common patterns and eager to talk about this.
jasonjgw: let's take it up next week