Meeting minutes
Accessibility of Machine Learning and Generative AI draft.
jason: Janina has work going on, good discussions last week and on mailing list (list of issues extracted from minutes/for the draft). Lots of complex issues that will be a challenge to classify and get important ones in. Anything else we should coordinate on?
Janina: Question - working with my favorite Ai helper to take those 43 issues and create a TOC as a framework. It's done a B- lovely job, tweaks to do. Should I keep (when rearrange and create TOC) the issue numbers and comments intact and keep those in place? Useful or should I strip those out?
Jason: Wondering, you wouldn't want that in the headings - at least a paragraph under each section heading explaining what it's going to cover...
Janina: yes, it's keeping those comments by default. Clean up a bit, concise summary...
… it can do that
Jason: would like to review it as a person who is not familiar with the history and look as it's a standalone with comment history
Janina: Can do. It's almost done. Nice to see it in place. Is an email sufficient? Long email...or do markdown?
John: I'm happy with an email
Stacey: same
Janina: I will get that up
Jason: Anything else on that that people want to ask? Can use this as a baseline?
Janina: and will then look for feedback
Jason: should take this as something to discuss by next week's meeting
Game accessibility.
Jason: Thank you John for a draft. Scott sent comments to the list, detailed and thoughtful. Can take those into account. I read draft and seems to be moving in right direction. I think the issues associated with desirable and undesirable difficulties will be central to any draft we produce. Focus of attention, seems to hold up in light of your
review of publications in this area (in reading your gap analysis)
john: I think there is so much prior work on a11y issues, reticent to go over those again. Big thing missing is intentional vs unintentional friction
Jason: in an intro imagine would want to make those connections, may go beyond game related audience you might expect
Janina: powerful observation
Jason: in edu research environment, a colleague wrote extensively of these contexts of education...assess a person's knowledge and skill and other things you don't and need to be clear about what those are and don't inadvertantly introduce difficulties that may be affectd by a person's disability that would influence the outcome of the measurements
John: did a white paper and see legislation - can't fundamentally alter the product. Gaming it's retain that friction, in education and testing someone is test this bit of knowledge, so make sure it doesn't fundamentally alter that process
Jason: it's a learning process as well. Very strong connections of important issues that extend beyond the context. How do you want to address those in a document and make clear to audience that they should be paying attention to
John: I would say, if address in terms of gaming, leave open to reader to go, oh, there's a parallel in other spheres like education. Trying to explore how to avoid fundamentally altering the games. Explore how/at what point do you say Yes this is a design decision and justified in retainging the challenge vs retain the challenge and making it more
accessible...how far you can go without fundamentally altering that challenge
janina: looking forward to this. Concepts in terms we're not using enough in a11y standardization, need to add them to vocabulary. This proposal had legs. We're going to have fun cleaning up how we're talking about this
John: refine the understanding of them as well
Jason: interested in this, the general issue is important
… part of the challenge I keep coming back to, if designing something new you have a choice about essential elements. Adapating what already exists, it's a different situation. Creating you get to decide, what's fundamental and what isn't and what challenges you really want it to have and what's incedental. Can't eliminate all challenges or it
wouldn't have that attribute. We don't have good guidance.
Jason: John - does this give you some material to keep developing this in light of Scott's comments, etc.?
John: Yes, Scott's comments...look at audio game, very few audio games have visuals. I fhtey do they are decorative. So people who do see don't think it's broken. At what point do we say it's OK to not be accessible to someone who is Deaf, as it would alter the fundamentals of the game...but if we can't say it's accessible to everyone, it's not
accessible? If an audio game in itself will never be accessible..same as hidden image games, you can't translate that without fundamentally altering the game. Interesting - do we allow that? Do we say it's a funamental alternation.
Janina: attacking the principle for saying everything has to be accessible. Pilots - we don't want to be piloted with low vision
Jason: if you had all of the accessibility tools in place - that's a research project
Janina: everything is accessible is a societal goal. Fly the plane in a test environment - go for it.
… have redundant systems (2 pilots, etc.)
Jason: audio gamers and the challenge it creates, could add a visual interface could make it accessible, some things would include tonal qualities tied to audio domain. But seems in many cases could do it. Involves fair amount of work and get a more accessible game and arguably could get the same game.
Janina: I don't know, I'd like to see an example of that.
Jason: the game i remember might be an example - a reaction time game. How quickly can you press a key or button when you get a tone, challenge to minimize that
Janina: that you can do with many types of games. Challenge is reaction time.
… but that's not a fundamental characteristic
Jason: adding a visual interface would be possible and other examples...
John: thinking the game kimsgame is a memory game. Pure memory, if you have someone with cognitive issue, like memory loss, you can't make that accessible to them without fundamentally altering the game. There will always be an example like that. You can take agame an interpret it and use it by someone, but we're talking about mights and maybes.
Are we going to allow making a game that can't be universally accessible say it's as accessible as it can be.
Jason: memory game - limits...might be cases in which you could make it easier by giving it fewer objects and still challenge them and maybe have hints in some cases? Becomes what is appropriate given the memory limitations and what you think is reasonable but don't want to elimante the memory challenge altogether. Strike the decision here
John: what i would say is, there's a declarataion about what the game is. If a memoery game, can make it easier, graduated difficulty. you can find that type of threshold, in which it comes a challenge for you. But if you, things that might come out of this - a declaration of This is the fundamental challenge, and a declaration of it won't be
accessilbe to this market without changing the game.
Janina: WCAG 3 might have something useful for this. Simplify or prompt better - all types of things, range of human abilities, you can build challenges that are relatively easy and gradations of difficulty is valuable and training ourselves to be smarter and more effective humans. Memory is learned. You remember more the more you challenge
yourself. If faculties are present you will remember more over time.
… same for vision and hearing
Stacey: gave some info on a college project, example of specific game challenge created as a project with difficulty levels/declaration for teenagers with severe cognitive, physical, and developmental disabilities.
Jason: at what point you're building a different game based on the accessibiltiy. Type of thing that needs to be nailed down
note - last comment was John, not Jason
Jason: memory challenge isn't eliminated, but provided withs scaffolds to help with the difficulty.
Janina: btw want to reserve some time to talk about process
Jason: Example i gave would still be in perameters in preserving the challenge but more accessible to different populations. As a first step - get some items filled in and move on to actual writing
Janina: subtopic - need an APA sign off. I think we already have enough questions surfaced. Difference between explaining the work between doing the actual work. APA needs us to name what we're working on, they don't need the actual work. Wondering John - do you think we can give that to APA in Sept before TPAC?
John: that would be the statement of this is what is missing, separate of gap anal?
Janina: could be part of the intro. part of gap analysis.
John: need to focus a bit more of fundamental alteration in there.
Janina: yes, surface those terms. valuable
Jason: reasonable, write out remaining sections of analysis and then the abstract
janina: we have opportunities because of money, if that's the issue...it's past the deadline, but might be able to still work something out
Jason: check on how it's going next week?
John: yes
jason: can do that next week and keep it moving along.
Miscellaneous topics.
Jason: anything going on we should be aware of?
Janina: will be on the road next couple of weeks, but expect to be on the calls.
Jason: will be traveling too, but not sure when yet.
… plenty happening in ML world at the moment. But nothing comes to mind for what we're doing right now that need to discuss. did find some work done on LLM and identify a11y issues in source code, but might have said last time, given speed at which developing anything published even a year ago would be out of date by now.
Janina: expect training at TPAC