Meeting minutes
Accessibility of Machine Learning and Generative AI.
jason: Janina still working on document revisions. Are there any questions or issues that affect ongoing work? Once revisions in a branch we'll come back...Anything to discuss here?
Scott: move it along to another week?
Jason: proposal - we check in on it next week, can also do two weeks. Then, we have a conversation about what needs to be done to be ready for first public working draft.
Scott: agreed
Jason: I have hardware problems. I might not be good for document review until those are sorted out.
Miscellaneous topics.
Jason: nothing on my end that's new
Janina: me either
Scott: nothing from my end
Jason: Been following developments, research on the use of LLM for programming tasks and they can slow down development rather than more productive. But indications in the article that for experienced developers can be faster to correct the output of the LLM so maybe more experienced developers get more out of it than inexperienced. Not sure if
anyone has really looked extensively at producing accessible code with that. None of the articles pertain to accessibility issues specifically. Maybe someone will come forward with that type of research.
Janina: Bring back issue from email, what we might name or words we want to apply to epistemic condiditon - the person with the diability who is the most knowledgeable about most effective accommodation is the least able to judge whether the quality of a particular instance. Example: A Deaf person least able to judge if captions are accurate.
Raja: about how to provide effective feedback. Challenge lies in need background information, if using a microphone, how far away, qualoity of background, etc. and challenging to understand which feedback is valid and allow it to return to the user, like automatic speech recognition. where you can see the Deaf person and the hearing person (both
users) if both looking at the screen.
Janina: looking at how to introduce this contradiction in W3C. Need a name for this. Situational accessibility concerns...(situation disabilities vs temporary vs permanent). Deaf person can't tell if the captions are good or awful Are they the right one. Are they reasonably good. There's no way to judge that.
Jason: It's possible to use contextual cues to make a judgment, but there's a zone of plausible but false information. Can be indistinguishable of incorrect from correct information.
Janina: there's a paradox here. Validation paradox. Names the situation, if we can socialize a good label for this, we can say that we nee to recognize a validation paradox...Need a good name so success criteria creates a path to resolve these.
Scott: something like calling it paradox is quite appropriate way to refer to it
Jason: another example, generating a summary or someone writing a summary of a doc using grammar and vocab more easily understood by certain user groups, they can obtain misunderstandings by reading the summary. They can't tell...applies to cognitive just as much as sensory.
Janina: whether human or machine generated it happens in various contexts. Need a good name and practices to manage it.
Scott: mention paradox, the very thing that helps people, they can't tell if it's really helpful...
Janina: when there's an instance we don't want to call it a paradox...it's a digital accessibility ....contradiction?
Jason: illusion?
Jason: Seems good from users perspective but they won't know that it might not be — problemmatic kind of case. Multiply examples fairly easily.
Jason: probably need something written about it that explains it, might not be obvious. Might not be obvious to most people.
Scott: concept is clear, but articulating into a name. Feel like we're close.
Jason: We'll have to write about this in Machine Learning and Ai doc. But it's a broader question than that.
Janina: should write this up in a seprate note, maybe a two-pager. A couple of names, describe the situation and refer to it from the paper
jason: sounds good. Connect on that next week
Jason: anything else happening in the world in connection with this task force that we should be aware of?
Janina: RQTF has come up in a few APA meetings...Success in AR docs and specifications and improve whole review cases. And that's what APA does, horizontal reviews. Users impacted and spec mods to support better. So likely to have a visit from Matthew to improve pipeline in other direction as well.
Jason: Sounds encouraging
Jason: Daisy consortium work coming to a close. They are not accepting new requirements, trying to publish and resolve open issues. That will include having a list of potential requirements that may go in a future version if needed.