W3C

– DRAFT –
Accessible Platform Architectures Working Group, WHATWG Joint Meeting

26 September 2024

Attendees

Present
Adam_Page, Angela, annevk, AvneeshSingh, chaals, CharlesL, Daihei, Dr_Keith, duga, Fazio, Fredrik, fserb, gautierchomel, George, gpellegrino, Hadrien, hiroki_endo, iali, ikkwong, janina, jaunita_george, JenStrickland, jkamata, jkline, joncohn, julierawe, jyasskin, Lionel_Wolberger, MasakazuKitahara, matatk, mgarrish, mike_beganyi, past, PaulG, present, Roy, Russell, Sheri_B-H, shiestyle, toshiakikoike, tzviya, wendyreid, zcorpan
Regrets
-
Chair
-
Scribe
hdv, matatk

Meeting minutes

ARIA attribute reflection

ARIA attribute reflection

annevk: there is this mismatch between ARIA and HTML because none of them are defined as enumerated attributes

annevk: tthere was a discussion on Monday, people involved probably already know next steps, can skip it here maybe

matatk: I thought quesiton would be about cross root referencing

<matatk> w3ctag/gaps#2

matatk: general problem with that issue, that Lea and Jeffrey raised, about referencing to elements in general

matatk: with mix of people in the room you may have thoughts

jyasskin: solution that was raised my solve it for ARIA too

matatk: ok we'll look at the minutes

Symbols

<jyasskin> The session I mentioned: w3c/tpac2024-breakouts#30

matatk: problem we're trying to solve: some people want symbols to better understand what's on the page

janina: these people have used symbols for ages, before the internet

<Fazio> watch the Theory of everything. Stephen Hawking used them to communicate

janina: that are not intereropable… we're talking 100s of 1000s of people

janina: we probably can't markup everything on the web but probably can do procedures and signposts for general things

Fazio: to understand how important this is… imagine Stephen Hawking not being able to communicate… when he got ill he learned how to communicate without speaking through physical symbols

janina: Cambridge University hired somebody to voice him because he was important

Fazio: and Intel sponsored him

<Fazio> Also known as Alternative Augmentative Communication (AAC)

matatk: the card [we just handed out] has symbols on both sides that signify the same concepts but they are different and people's preferences differ

matatk: the kind of killer app we're targeting to start with… people who need symbols to help them understand things will read text, but symbols give them context

matatk: they come to the web for video content as it is easier to take in in general

matatk: so the sorts of things we're looking to add symbols to is @@@

<hdv> s/.@@@/chapters

<past> https://github.com/w3c/adapt/blob/add-explainers/explainers/symbols.md

<Adam_Page> https://www.w3.org/TR/adapt/

matatk: our idea is to have an attribute that somehow authors use, they set the attribute to indicate the concept they're trying to show

matatk: the concept is how we'll map to different symbol sets

matatk: eg symbol for the concept of cooking is diffferent in different symbol sets

matatk: the number of symbols is 0 if target set doesn't have a symbol for it

janina: kind of like how a word is made up of letters

<Fazio> Most widely used AAC symbols: https://omazingkidsllc.com/2021/04/11/most-widely-used-aac-symbols/

matatk: we're not experts on AAC but we work with an organisation whose dictionary of concepts we use

matatk: because they maintain this dictionary of concepts, which is more comprehensive than other symbol sets, we went to them… we're not using it in a language or translation sense

matatk: in the future we want it to be the symbol set that the user wants

matatk: we don't know the number of the symbol, only once we know which symbol set they use

matatk: we've had a number of approaches suggested

matatk: when we started this, we had an attribute… the thing is how to we key in to the dictionary that bliss gives us

matatk: originally we had their concept ID, which is just an integer

matatk: each concept has an id

matatk: we had a long running issue where people were suggesting that as a subset of bliss is going into Unicode we should use Unicode

matatk: bliss is also a well known organisation but not globally

matatk: bliss has about 1400 chars going to Unicode, but many more that aren't

matatk: if you want to represent some concept, you can't just look up the Unicode code point and look that up directly

matatk: at that point you're asking the author to know about bliss

annevk: versions are an established thing in the Unicode world; eg if you have like a wavy hand, you can give it a skin tone

matatk: taking that idea; if it was rendered as a character it could look like that character

annevk: I wonder if that can be addressed through fonts

matatk: lots of symbol sets are pictures

<Fazio> Keep in mind, to be useful users wouldn't use individual symbols they would combine symbols to illustrate a though or sentence or action

annevk: fonts these days are pictures too

matatk: but sometimes a symbol is one thing to the user but @@@

jyasskin: can do with fonts but depends

Adam_Page: yes with ligatures

jyasskin: so they can combine multiple things

jyasskin: so is technically possible, not necessarily best way but is possible

Fazio: user would not necessarily press a button with symbol on a website but would sometimes press the button on their AAC device… ideally would use a number of symbols to convey an action

Fazio: I don't know if that's possible with what we're talking about

annevk: that's more like an input method

matatk: good context of how folks communicate, we're doing the other way around

matatk: they want to know what each is about

janina: i think we'll get that when we talk to rtc

annevk: this illustrates to me how important it is to get this into Unicode, given how widespread Unicode is and how core it is to how we communicate

annevk: if we don't take it to Unicode and do it in markup, you get the same problem as @@@ eg with Japanese

annevk: given how vital it is to this community it is important to make sure it is in Unicode

janina: it's on its way into Unicode

annevk: but only a subset

matatk: effort to get bliss into Unicode has been mostly building blocks for making bliss symbols… some are in there because they are simple

matatk: what goes into Unicode is not the whole dictionary

janina: enough to communicate the concepts for rendering

matatk: we looked at this, from authoring perspective is possible to use Unicode

matatk: let me show an indication of what an authoring tool may be like. W3C made a registry of what this could look like

<Roy> https://www.w3.org/TR/aac-registry/

[shares authoring tool on screen]

annevk: this could be solved with an input method, eg just like how you enter Japanese on a phone

zcorpan: Unicode is the correct layering in my opinion, should focus your efforts to get everything into Unicode and not HTML

zcorpan: if Unicode is too slow, use custom elements or a polyfill or something. I think it doesn't make sense to add it into HTML as well as into Unicode

annevk: I agree

<Zakim> jyasskin, you wanted to ask about word names for the concepts

zcorpan: as far as I can tell it supports technically what you want to do

jyasskin: agree Unicode is the right place… feel it's similar to ruby but not exactly… the attribute that says whicih one it is… I think I'd suggest to allow the input, for things that aren't in Unicode, use strings rather than numbers

jyasskin: the registry has strings

matatk: if we did that and we have a way to key into dictionary of concepts that exists… but how do we turn that into the right symbol for the user?

matatk: if we don't want to add that to HTML… are we saying we don't need an attribute to mark up the concepts? where do we put the key?

annevk: if you have one of these characters… what you're asking for… there are two users where one would want to see one symbol and the other multiple… is that what we're trying to cater for?

anenvk: I think it could be as simple as using a diferent font to display specific Unicode code points

matatk: practical thing is… how symbols industry works, they produce PNGs or SVGs… how do we cater to the fact that the format they're using is not fonts… 

annevk: you can take the SVGs and put them into a font files

PaulG: only works if there are no heteronyms… how can a ligature font tell the diff between life and live?

PaulG: eg which word is it

annevk: the font counts on the unicode code points for bliss

annevk: so the font doesn't tell a difference

zcorpan: user could then even use CSS to override

PaulG: ah ok so author would author the code points

annevk: yes and then it's just a setting in the font somewhere

PaulG: we have to hide that from AT as screenreaders or braille display would have garbage

annevk: presumably operating systems could add support for these code points just like they support emoji now

janina: are we suggesting that all these symbol sets should get their symbols into bliss?

annevk: no just get bliss into Unicode and map to that

zcorpan: the symbols don't need to be standardised

<Fazio> Ideally images would be universal (standardized) as much as possible

jyasskin: one thing I think I'm hearing… these symbols are alternate representations for things that are on the page

<Fazio> think internationalization

jyasskin: you need some way to tell screenreaders that they should probably skip reading the symbolic representation

jyasskin: either that's ruby or we're missing an element

Fazio: I know it's not our job to standardise what these look like, but ideally we would try to harmonise

matatk: we think we're covering that by using bliss as the basis for the concept

matatk: and then we can have symbols rendered in appropriate cultural context

matatk: so we've talked about alternative representation

matatk: it would be difficult for symbol producers to put their symbols in a font

matatk: it sounds like we could what we want with a font… want to make a proof of concept

matatk: we can prototype it with a data attribute… but we still need a place to put it for authors

<Fazio> we have an accessible sag expert if that's a prototype route we want to go

<Fazio> ?SVG not sag

matatk: ruby has been suggested and unsuggested by various people… i18n folks have said probably not use ruby

annevk: it sounds like ruby to me… would suggest use ruby

@@@: what's the argument against ruby?

matatk: see the issue… what I remember from it… it was something like that ruby is meant for Asian script and using it for this was like using it for something it was not meant for

janina: appreciate fazio's input as you lived through this

<past> w3c/adapt#240

<matatk> explainer (draft): https://github.com/w3c/adapt/blob/add-explainers/explainers/symbols.md

fazio: yes it can very much slow people down if they cannot communicate through AAC… it's not just single images or icons, it's an active picture of what's going on

Adam_Page: probably implicit in this discussion… can see use cases where this is additional and not just alternative

Adam_Page: we have lots of icons on the HIlton website… but then allow people to swap our branded icons with these bliss icons

CharlesL: another use case is when symbols are all by themselves, not how ruby works as it is attached to specific words

CharlesL: because, like images, there might be an alt text, but in other cases there is already text there so that an alt text is not needed

annevk: right but if it's in Unicode, like with emojis, there would already be a well defined alternative

annevk: when I look at the discussion with Richard Ishida I don't see, from quick look, things that would stop us from using ruby

<Zakim> jyasskin, you wanted to mention alternatives to "fonts"

<JenStrickland> It's important to not make it too complicated, or people won't do it.

jyasskin: I think there are different semantics these symbols can used with… as a primary representation, or as text in a flow, or as an explanation (that might be ruby)… it sounded like there is a third semantic too… I'd encourage you to list the semantis adn that can drive what we'd pick

jyasskin: we have a mechanism in fonts that drive type of font, pre defined font setes like monospace/bold… operating systems have ways to configure those

jyasskin: if it's not fonts maybe it can be something font-like

jyasskin: there is also the author use case… where the website wants to pick the font, eg icons that are part of the branding

<jyasskin> +1 to JenStrickland

matatk: sorry that queue is closed, I'd love us to talk before the next TPAC

matatk: with some prototyping we can look at some things to see how technically feasible they are

matatk: we'd like to keep talking… 

<jyasskin> s/bold.../fantasy/

Well-known destinations

<matatk> Draft explainer: https://github.com/w3c/adapt/blob/add-explainers/explainers/well-known-destinations.md

matatk: problem from the user's perspective… at the AC meeting earlier in the year, we proposed a technical solution for it that we knew at the time

matatk: in terms of implementation, we thought of two ways that are in the explainer

matatk: the presentation is linked from the explainer

matatk: btw this is still under construction

matatk: the Adapt TF proposes a way for a site to define popular pages they offer, this allows UA to discover which popular pages a site provides

matatk: we're talking about things like home page, product page, contact page etc

matatk: one reason we want to do this is that sites vary in what terms they use

matatk: the central thing to this is that we'd standardise a set of tokens that correspond to popular destinations that exist on a site

matatk: what might this look like from a user perspective? I made a browser extension that did this

matatk: it gives you a menu with links to these specific destinations

matatk: the destinations vary per site

matatk: looking for your suggestions on how to implement it, and querstions re the use case

<Zakim> annevk, you wanted to point out this already exists in theory and used to exist in practice (Opera did it first!)

annevk: this already exists in HTML today, the <link> element has a specified list

annevk: Opera used to have an implementation of this

annevk: similar to the UI you shared

fserb: Firefox had it as well, but with shortcuts

annevk: so maybe this is more about getting it adopted again, and maybe not a need for a new solution

annevk: the other problem is adoption

matatk: both advised solutions require rel attrs and link types, we very much try to build on what's there with both approaches we look at

zcorpan: wanted to say the same thing as Anne said

zcorpan: also… if you want to provide this UI and have it work, even if the web developer didn't use the link tag, we could still figure it out with heuristics

annevk: I saw you did consider some existing implementation

fserb: how important is it that you'd have these specific well known places with always the same semantics or that it is the most popular?

matatk: what's important on this page is a really important question, not necessarily what we look at here

matatk: DOM order, landmarks, headings also provide ways to navigate

matatk: what we're trying to provide is sign posts to particular destination, to help people who struggle with that fundamental navigation

matatk: say you go to the contact us page, for those to be highlighted

janina: and everything else greyed

janina: complexity management is a game changer for people with intellectual disabilities

matatk: what I was on queue for… I want to build on existing tech… but what we didn't find is in one HTTP request, to have a way to get that list

matatk: it's knowing that this is the definitive list that is very useful

matatk: we couldn't just do that with link elements

annevk: XML sitemaps is kind of what you're after

zcorpan: with the solution you proposed you still need the web developer to define the destination? eg we could give guidance to provide links to all pages

zcorpan: don't affect rendering because they are not visible

matatk: wouldn't that be a bit bloaty?

annevk: probably not, HTML is least significant part of page load in most cases

matatk: two things we were looking at… a special well known URI is that you can provide the list… that has the drawback that the extra bit is not in the spec… the second drawback is that well known URIs that relate to an origin, you might have sites with subsites where the scope may be tricky

matatk: we also got feedback about linksets

matatk: these allow you to give relationships between pages

matatk: it's like as if you put the link elements on each page… but you can just refer to a single document

matatk: both approaches are in the explainer, we wondered which one is better and think the second probably is

matatk: there is also the approach of putting all the links on all pages

matatk: challenge probably is encouraging people to do it

annevk: I would start out with using the link attr, and then if we want to make it easier you can talk to the web manifest people

matatk: the advantage of how we were proposing to interpret link sets was to make it easier to deal with subsits

annevk: feel like UI you want is the things relevant to that document… can't really convey UI that make sense across documents

matatk: understand where you are coming from, from user perspective, they visit a site and want to know it for that site

annevk: trying to convey that info beforehand user would need to go to sitemap and explore how it is built

matatk: we can ask the COGA TF

annevk: we have link rel accessibility

<PaulG> RDF+json (similar to linksets) seems like a good match for the semantics and extensibility but asking authors to generate/maintain a second sitemap seems problematic. Shoving the metadata into the sitemap seems to break the "single use" doctrine because it's main focus is SEO and not helping users find resources.

zcorpan: I think sitemaps are actually used by sites

zcorpan: don't know if websites typically link to sitemaps from markup or if it is a well known location

matatk: if we do something some way and it only uses standard, we can make it more efficient

hdv: One of the goals of EU regs is for sites to include an accessibility statement
… Helps people know where to go for the info or to file the complaint.
… This is a good use case

<Zakim> matatk, you wanted to ask about highlighting the relevant bit on the page (semantically)

matatk: one of our TF participant did say to include a11y statements

matatk: so far everything we talked about we can actually do

matatk: if I've gone to the login page and I want to highlight that page, some of our users would need that type of direction, would we need an attribute, that we can prototype with `data-*`, is that feasible to prototype?

janina: so highlight the login portion of page and grey out everything else

Adam_Page: so more than a deeplink you highlight the part of the page?

annevk: what if you have single long document?

annevk: would you set it dynamically?

<PaulG> Chrome's text fragment linking seems like the right path forward. I'd push back against "supressing" a portion of the UI

matatk: destination would take avalue

nathan: I understand the use case of an extention as how people interact with it, eg voiceover rotor, would be cool if you could navigate that in the AT you're already using

matatk: love the idea, a lot of people with cognitive barriers don't use an AT, so can't be the only way

<Fazio> Lookup doorway effect and how it effects short term memory

zcorpan: for highlighting the destination… do you envision the website providing that?

matatk: website would provide the landing zone

zcorpan: could the website also be responsible for the highlight thing

zcorpan: if so we could have link targets, and use the target pseudo class in CSS, incl graying out everything else

<Fazio> The doorway effect is a phenomenon that occurs when people forget information after passing through a doorway or when switching between contexts, such as opening a new tab on a computer. The effect is caused by the brain's tendency to discard information when it encounters a change in context, and is thought to be a subconscious

zcorpan: we started with looking at well known links, but with fragments, you're absolutely right we can do that

Fazio: important because of the doorway effect

Fazio: when you forget what you've just odne

Fazio: also happens when you go from one web page to another or between screens, see also the link I posted

matatk: thanks everyone, this is super helpful. How do we keep talking to you?

annevk: variety of ways, always good to start with an issue, generally html repo or if it is very generic our meta repo

annevk: then we have a pointer

annevk: we also have a weekly call that you are welcome to join

annevk: if there are many issues we can also do a dedicated people.

annevk: we can also join your meeting provided timing works out

annevk: can be a bit trickier to get timing to work out for all of the WHATWG folks

annevk: we also have a matrix instance where you can ask questions and have a mostly asynchronous convo, sometimes synchronous

Minutes manually created (not a transcript), formatted by scribe.perl version 229 (Thu Jul 25 08:38:54 2024 UTC).

Diagnostics

Failed: s/.@@@/chapters

Succeeded: s/number/numbers

Succeeded: s/@@@/they should probably skip reading the symbolic representation/

Succeeded: s/like italics/like monospace/

Failed: s/bold.../fantasy/

Succeeded: s/destination/destinations

Succeeded: s/link@@@/linksets

Succeeded: s/ozne?/zone

Succeeded: s/poseted/posted

Maybe present: @@@, anenvk, hdv, nathan

All speakers: @@@, Adam_Page, anenvk, annevk, CharlesL, Fazio, fserb, hdv, janina, jyasskin, matatk, nathan, PaulG, zcorpan

Active on IRC: Adam_Page, annevk, CharlesL, Fazio, fserb, hdv, JenStrickland, jyasskin, matatk, nathan, past, PaulG, Roy, zcorpan