14:33:58 <RRSAgent> RRSAgent has joined #maturity
14:34:02 <RRSAgent> logging to https://www.w3.org/2025/08/13-maturity-irc
14:34:02 <Zakim> RRSAgent, make logs Public
14:34:03 <Zakim> please title this meeting ("meeting: ..."), Fazio
14:34:43 <Fazio> meeting: Maturity Model Task Force
14:35:04 <Fazio> chair: David Fazio
14:36:05 <Fazio> Agenda+ W3C Accessibility Maturity Model Publication Update
14:36:05 <Fazio> Agenda+ AI W3C Accessibility Maturity Model Assessment Demonstration
14:36:23 <Fazio> present+
14:40:12 <Neha> Neha has joined #maturity
14:54:36 <gautierchomel> gautierchomel has joined #maturity
14:56:59 <CharlesL> CharlesL has joined #maturity
14:58:40 <Dr_Keith> Dr_Keith has joined #maturity
14:58:48 <Dr_Keith> present+
15:00:13 <Mark_Miller> Mark_Miller has joined #Maturity
15:00:15 <jtoles> jtoles has joined #maturity
15:00:24 <CharlesL> present+
15:00:28 <jtoles> present+
15:00:28 <Mark_Miller> present+
15:00:36 <gautierchomel> present+
15:01:12 <Roy_Ruoxi> Roy_Ruoxi has joined #maturity
15:01:27 <Roy_Ruoxi> present+
15:03:23 <janina> janina has joined #maturity
15:03:29 <janina> present+
15:03:56 <Sheri_B-H> Sheri_B-H has joined #maturity
15:04:06 <Sheri_B-H> present+
15:04:11 <Sheri_B-H> scribe+
15:05:01 <Sheri_B-H> zakim, next item
15:05:01 <Zakim> agendum 1 -- W3C Accessibility Maturity Model Publication Update -- taken up [from Fazio]
15:06:09 <Sheri_B-H> fazio: welcome to larger group, introduction to demonstration
15:06:24 <tsaulter> tsaulter has joined #maturity
15:07:36 <Sheri_B-H> Fazio: Introduction to demos from Benetech and Neha
15:08:51 <Sheri_B-H> CharlesL Introduction to @John Higgins
15:08:51 <gb> https://github.com/John -> @John
15:10:09 <SusiPallero> SusiPallero has joined #maturity
15:10:42 <Sheri_B-H> CharlesL: expained how Benetech used the maturity model and took the spreadsheet and created a tool to gather the data
15:10:55 <Sheri_B-H> CharlesL: also customized it for Title II
15:12:31 <SusiPallero> Present+
15:12:39 <Sheri_B-H> CharlesL: tested with 4 schools, they found it valuable
15:12:54 <Sheri_B-H> CharlesL: They were able to create some templates to streamline things like Procurement
15:13:12 <Fazio> q?
15:13:51 <Sheri_B-H> CharlesL: added a glossary, made it more like a turbotax interface, cross checked for inconsistencies
15:15:32 <Sheri_B-H> janina: there will be a discussion after the presentation (not recorded)
15:17:50 <Sheri_B-H> John: trying to create a tool that is more self-explanatory to not have to hold the schools hands quite so much
15:18:26 <Sheri_B-H> John: What can we borrow from other people who have done the same thing, like healthcare and SOX
15:19:49 <Sheri_B-H> John: demoed tool for a different maturity model with categories, proof points, progress bar.  Borrowed from this for the AMM
15:20:43 <Sheri_B-H> John: This tool was generated by AI in a day, with 5 hours of clock time
15:21:09 <Sheri_B-H> John: will allow for rapid feedback, remove barriers, get it past a wireframe
15:21:50 <Sheri_B-H> John: Each "Subject Area" (dimension) has a list of what is reviewed, what is the score, and the score for the dimension
15:22:04 <Sheri_B-H> john: top of the page has all subject areas/dimensions rolled up
15:23:03 <Sheri_B-H> John: each subject area/dimension has a details page, there are filters so you can look at stuff that hasn't been scored (for example)
15:24:16 <Sheri_B-H> John: API calls can go out and look at document inventories for example
15:24:44 <Sheri_B-H> John: some proof points are multiple choice questions, there are notes fields, and a place to attach documentation
15:29:42 <Sheri_B-H> John: continued demo
15:29:48 <Sheri_B-H> q+
15:31:19 <stacey> stacey has joined #maturity
15:31:23 <stacey> present+
15:31:54 <Sheri_B-H> John: added admin interface for maintaining organizations, users, proofpoints, and assessments
15:33:07 <janina> q+
15:33:08 <Neha> q+
15:33:27 <Mark_Miller> q+
15:33:44 <Sheri_B-H> ack Sheri_B-H
15:33:44 <Jon_avila> Jon_avila has joined #maturity
15:34:02 <Sheri_B-H> How do you override false positives?
15:34:26 <Ann> Ann has joined #maturity
15:34:27 <Sheri_B-H> John: it will suggest a score based on the API run, but the user can override the suggestions
15:34:31 <Sheri_B-H> q
15:34:38 <Sheri_B-H> ack janina
15:35:52 <Sheri_B-H> Janina: You should propose this for a TPAC breakout.  Can you say a little about the process of how the tool was built?  How did you choose the AI that was chosen? How did the narrative and spreadsheet get fed into the prompts
15:38:09 <Sheri_B-H> John: I wasn't sure what was possible.  I looked for the lowest barrier to entry to get a good result.  Stuff is happening really quickly; what I tell you today might be obsolete in a month.  This was created using Replit.  We didn't have an existing code base, and Replit worked well for that, and it handled the deployment.
15:38:41 <Sheri_B-H> John:  You have to make no assumptions in the logic, tell it what NOT to do in addition to telling it what to do
15:39:31 <Sheri_B-H> John: prompts need to be verbose
15:40:02 <Sheri_B-H> John: could feed the entire spreadsheet into the prompts
15:40:19 <Sheri_B-H> q
15:40:22 <Fazio> q?
15:40:24 <Sheri_B-H> ack neha
15:41:27 <JeffAdams97> JeffAdams97 has joined #maturity
15:41:55 <Sheri_B-H> Neha: Where did the questionaire come from?  At what stages did you use AI
15:42:24 <Sheri_B-H> John: The questionnaire was generated by AI, but it is fairly naive.
15:42:50 <Sheri_B-H> john: aggregated questions that were being asked by schools
15:43:24 <Sheri_B-H> John: think the best path forward is to have it reviewed by SMEs
15:43:36 <Sheri_B-H> John: AI was basically the designer and the engineer
15:44:11 <gpellegrino> gpellegrino has joined #maturity
15:44:17 <Sheri_B-H> CharlesL: has not yet been reviewed for accessibility
15:44:52 <Sheri_B-H> John: instructed the AI to run aXe on all changes, and not commit change until aXe was happy
15:45:16 <Sheri_B-H> q?
15:46:11 <janina> q?
15:46:22 <Sheri_B-H> John: for privacy reasons, the school data was not fed back into the AI
15:46:44 <Sheri_B-H> Neha: can the questionnaire be simplified
15:47:02 <Fazio> q?
15:47:02 <Sheri_B-H> John: yes, we need to get the questionnaire in front of more people and see what they are struggling with
15:47:19 <Sheri_B-H> John: push it out, see what comes back, use that to drive changes
15:47:30 <Sheri_B-H> John: AI might not be able to fully represent users
15:47:46 <Sheri_B-H> ack Mark_Miller
15:48:10 <Fazio> q?
15:48:11 <Sheri_B-H> Mark_Miller's questions were answered by previous discussions
15:48:23 <janina> q+
15:49:36 <Fazio> q?
15:49:36 <Sheri_B-H> ack janina
15:51:39 <Sheri_B-H> John: There is nuance between are you not doing the thing, or can you not prove that you are doing the thing
15:52:17 <Sheri_B-H> q
15:53:06 <Sheri_B-H> Fazio: AMM is not normative, its about measuring progress
15:53:24 <Jon_avila> Thank you for the presentation
15:53:56 <Fazio> q?
15:54:14 <Sheri_B-H> Janina: would be useful to have a resolution for wide review
15:55:19 <Sheri_B-H> RESOLUTION: Maturity Model group is ready for wide review of the editors draft prior to note status
15:56:03 <Sheri_B-H> no objections
15:56:25 <janina> rrsagent, make minutes
15:56:27 <RRSAgent> I have made the request to generate https://www.w3.org/2025/08/13-maturity-minutes.html janina
15:57:01 <Sheri_B-H> zakim, end meeting
15:57:01 <Zakim> As of this point the attendees have been Fazio, Dr_Keith, CharlesL, jtoles, Mark_Miller, gautierchomel, Roy_Ruoxi, janina, Sheri_B-H, SusiPallero, stacey
15:57:04 <Zakim> RRSAgent, please draft minutes
15:57:05 <RRSAgent> I have made the request to generate https://www.w3.org/2025/08/13-maturity-minutes.html Zakim
15:57:10 <Zakim> I am happy to have been of service, Sheri_B-H; please remember to excuse RRSAgent.  Goodbye
15:57:11 <Zakim> Zakim has left #maturity