Beyond Text: WAI-Adapt AAC Symbol Specification, for Visual Markup

13 September 2023


CharlesL, fred_edora_, Janina, Jennie_Delisi, matatk, Maud, Wilco
Lionel Wolberger
CharlesL, matatk

Meeting minutes

Slideset: https://raw.githack.com/w3c/adapt/tpac-2023-breakout/TPAC/2023/


[Slide 4]

Lionel_Wolberger_: *presents deck*

Lionel_Wolberger_: We will prsent the context of our work, and then more details on the current focus which is symbols.

[Slide 6]

Lionel_Wolberger_: We take the view that adaptations help users. Users may be disabled by the environment, which needs to be adapted to support them.

Lionel_Wolberger_: Today, more services than ever are delivered digitally and/or through devices.
… Standards exist to make these accessible but...

[Slide 7]

Lionel_Wolberger_: ...User needs differ. Slide shows examples of how to convey the concept of "save" to people, and how different people have different needs and expectations for interpreting symbols.
… We need semantics to help with this.

[Slide 9]

Lionel_Wolberger_: *shows an example of how a page, how to make a good cup of tea, is adapted to a user's needs by simplifying the content, changing the font size, and adding symbols _in the user's preferred symbol set_*

[Slide 11]

Lionel_Wolberger_: COGA TF created a lot of guidance on this, which we are drawing from.

[Slide 12]

Lionel_Wolberger_: Sir Tim Berners-Lee had the critical insight that the semantics of the document, words, structure are importnat. But the web ended up becoming a very graphical place. Symbols remain semantically unadorned and neglected.
… The first part of our work is to provide a method for providing the semantics of a symbol.
… We created a registry document as the first part, with Blisssymbolics ("Bliss") that is used as a lookup.
… This list of concepts is curated by a decades-old international organization, Blissymbolics.
… Bliss has symbols for almost everything.
… *Looks up sad, happy*
… Now that we have a primary key for all the symbols, you can associate this with any symbol set
… *Shows spec, with markup examples*

[Slide 14]

Lionel_Wolberger_: Global Symbols (with whom we're also working) provide a means for users to discover a range of different symbol sets.

[Slide 15]

Lionel_Wolberger_: *reads short slide*

[Slide 16]

*example markup for our flyer that we have distrubted at the conference*

[Slide 17]

*Bliss home page*

[Slide 18]

*Registry spec document*

[Slide 19]

*Demo of tool*

Wilco: Who would benefit from this?

Lionel_Wolberger_: We've prepared a handout that shows an implementation (in paper form) that we feel answers that question. This is about "My favourite recipe" video. Our core focus is on people with cognitive disabilties.
… We found that it is popular for a lot of people to get information from videos, rather than reading detailed texts.
… We've shown it to experts in the field and they are happy with this layout.

Wilco: Would it help people with literacy challenges?

Janina: We do have some sense of that; we need to spend a bit more time getting the statistis on this; we are resarching the available data now and are pulling it together. There's a variety of skills involved with this. Would Jenni like to comment? We do know that symbols assist in various communitites in building literacy skills.
… There is a wide spectrum of literacy skills in areas where symbols are helpful.

Jennie_Delisi: One of the groups that would benefit from symbosl that maybe aren't the primary target group is someone who had word literacy skills, but then has a declining ability to process text.

Jennie_Delisi: E.g. a person who acquires a brain injury, or has a decline in reading abiltiy for a variety of reasons.

Jennie_Delisi: Another group is of peole who use symbols for speech too, and having a familiar symbol set would help them.

Maud: We discussed already with WAI-Adapt [Accessibility for Children]. We have some more examples...
… Emergency situations in which children or adults need very quick coding of important information.
… Our new co-chair thought we could ask Asha speech therapy organization in the US. They gave us a small list of symbols used by the children they work with.
… Re markup fatigue... children we work with use Macaton and symbols.

<Jennie_Delisi> ASHA: https://www.asha.org/

matatk: prototype of an authoring tool might look like made by myself
… complete with documentation. html markup in the flyer markup. will demo how to get the markup in the flyer
… there will be a digital version of this.
… normal unordered list.: ingredients, prepping, cooking plate and taste
… autogenerated, did a half decent job, but didn't get prepping. did ok with plate & taste, and removed symbols which was not needed.
… can edit the code directly of the HTML markup with adapt-symbol attributes added on the words we want symbols attached to.
… we only need to markup the key items. open up access to something that was not available previously. could be part of CMS's etc. you can review the symbols as you go.
… Machine Learning could be an option here in the future to do this automatically or improved accuracy etc.
… ML at authoring time can make it easier. at run time maybe not so much.
… end of tool demo.
… . we need input from authors. main element works well. some machine readable metadata could be of help.
… simplification of content we are working on.
… purpose of form fields
… actions and destinations is another area we are working on.
… helping people get to a part of a site. login / search / products / accessibility statement page for example.
… well known URL could solve our problem
… could be of benefit relies on existing frameworks.
… another is distractions TBD.
… we have an explainer, and the symbols module.
… get involved, provide feedback, sites, content markup. web extensions, implementations, follow our github repo file issues. join our group.

Jennie_Delisi: Thank you. I'm a big proponent of symbols. Excited to see the work progress. Can you speak to applying mark-up word-by-word and using something like a browser's dictionary that's built in, like spell-check? Could a user have a symbol exposed like they would a definition of a word.

matatk: we are not the experts on symbols but COGA does and can help us here.
… with which words for symbols, that does take some skill, to determine which ones will help. symbols on everything may be too distracting.
… just get definition of a word would be great to get as a symbol. sometime the meaning may be challenging to determine. ideally yes this would be a great idea but knowing what symbol for a word only the author may be able to do this to some degree.

Janinia: Bliss Symbolics participates with the Adapt TF, we are getting support from the AAC community.
… with auto lookup something like auto captions it may be similar experience with any word look up for symbol. so this could be a win. to be configurable. reliability may be a little weak.

Jennie_Delisi: variety of use cases for symbol consumption, identify user needs and multiple solutions may be required.
… expose symbols as they are consuming a document I can discuss this further.

jcraig: what do you mean symbol lookup vs automatic speech lookup

<Zakim> jcraig, you wanted to ask about the polyfill, and about the bit you Matt mentioned yesterday about incorporation into unicode

Janina: text lookup to symbol may be similar experience to speech lookup to text.

jcraig: Please expand on your comment yesterday that Bliss Symbolics is moving to Unicode.
… markup and the polyfill, is there some polyfill added to do some rendering, and if it goes to unicode would you replace that the unicode character?

matatk: unicode for bliss is in the process to convert to unicode.


matatk: the Bliss expert would not be appropriate to use bliss unicode characters lookup for some technical reasons we don't fully understand.
… rendering at the moment, the editor uses custom elements but we wouldn't have to. the web extention doesn't use. it will use CSS to add the symbol. we don't think it will break layout, we will test this in an automated way and run it on many pages to see how the rendering looks on various webpages. but we are looking to do these test to see it if feasible.
… it seems so far that if you have a site that is modern and cleanly coded, but are looking at getting more data.

jcraig: like a pseudo element that gets replaced? Experimented with Ruby?

matatk: that is worth looking at, but the technique might be useful.
… AAC community prefers the symbol above the word.

jcraig: ruby would help here to respect the word box boundries. ruby may complicate things although for screen reader users.

Wilco: Two questions (1) what level of adoption does this have? (2) have you considered automated evaluation tools for the wider Adapt work?

Janina: Adoption of symbols in web contnet is low/zero because we lacked the Rosetta stone to make it possible.

matatk: We are at CR but got agreements from orgs to markup their websites. right now it is very low, but we are hoping to change this

Janina: the ADA request form we will mark this up.

Wilco: Warning about typos in the markup would be great.

matatk: like linting yes that would be useful.

Wilco: put into some tools, we can look ACT can help write some rules. and help move this forward.

<Jennie_Delisi> *Great presentation. Thank you.

Minutes manually created (not a transcript), formatted by scribe.perl version 221 (Fri Jul 21 14:01:30 2023 UTC).


Succeeded: s/]slide 14]/[slide 14]/

Succeeded: s/ess test/ess text/

Succeeded: s/@@/Asha/

Succeeded: s/tatse/taste

Succeeded: s/Bliss Symbolics moving to Unicode./Please exapnd on your comment yesterday that Bliss Symbolics is moving to Unicode./

Succeeded: s/exapnd/expand/

No scribenick or scribe found. Guessed: matatk

Maybe present: Janinia, jcraig, Lionel_Wolberger_

All speakers: Janina, Janinia, jcraig, Jennie_Delisi, Lionel_Wolberger_, matatk, Maud, Wilco

Active on IRC: CharlesL, fred_edora_, Ian, jcraig, Jennie_Delisi, matatk, Maud, Roy, Wilco