Important note: This Wiki page is edited by participants of the EOWG. It does not necessarily represent consensus and it may have incorrect information or information that is not supported by other Working Group participants, WAI, or W3C. It may also have some very useful information.

UF June 2015/Results

From Education & Outreach
Jump to: navigation, search


  • (1) Both, overview and customization on the, left.
  • (2) Left and right columns for overview and customization, respectively.
  • (3) like (2) but with a fixed bar on the top.
  • (4) Two left side bars.
  • (5) Customize on the top of the main column.
  • (6) Customize as a button on the top of the left sidebar, tags closed when clicking on the button. — We did not test this because too similar to (7) and (7) seemed to be more sensible to test.
  • (7) Customize as a button on the top of the left sidebar, tags open when clicking on the button.


All participants agreed to this Consent & Recording Release.

Session I

Evaluator. Youtube

Prototype (1)

  • Scrolls nicely
  • Understands tags and how to unselect them
  • Understands level filters
  • Not using “only” buttons
  • Inverted text
  • clear button preferred than doing it manually (this was since fixed)

Prototype (3)

  • Nice that it’s possible to hide customizations
  • Likes that the customize bar can be collapsed and that the button stays visible

Session II

Evaluator, very experienced with WCAG2. Youtube

Prototype (7)

  • Only first section expanded (in overview)
  • What’s the checkmark??
  • wonders where to filter
  • doesn't see the customize button
  • sees techniques (sufficient/advisory) as filters
  • find filters eventually
  • customize <=> filters
  • likes the “only” button
  • is confused by losing position in the document
  • wonders about the categories. They seemed generic.
  • once more specific tags, for example custom elements instead of keyboards, …
  • keyboard is a requirement, Not a need
  • wants a clear all filters button
  • again wondering about the checkmark. A signal for okay, not done.
  • expects understanding/techniques to open in the same tab (because W3C) but prefers a new tab
  • emphasize technique categories
  • select multiple tags -> expects to see only items where are all of the selected tags are applicaple
  • would prefer the scroll to top link to be on the left

Prototype (5)

  • not sure if likes the left side bar
  • collapse on zooming -> basically mobile view
  • prefers opinionated standard settings, for example hiding the side bar
  • likes if more space is left
  • would like to see iconography for principles and guidelines

Session III

Senior inspector, carrying out accessibility tests, trainer. Youtube

Prototype (2)

  • sees customize, doesn't know how to use it
  • clicks on on techniques for guideline repeatedly
  • doesn't know how it updates
  • is getting confused when only failures are open (we need to do more for highlighting)

Prototype (4)

  • tags very unclear, confused with tags in HTML
  • prefer the column on the right
  • likes the text height and show in the column headers
  • tags should also hide

Session IV

Evaluator, researcher. Had used the old quickref. Youtube, feedback session was conducted in German

Prototype (5)

  • likes the navigation
  • confuses guideline techniques for 1.1 with techniques for 1.1.1
  • likes to formatting better than old quickref
  • filtering good
  • link to techniques: depends on use case. Techniques long -> a new page is probably an advantage because of space issues.
  • likes the site bar
  • doesn't like that the overview menu collapses and expands while scrolling
  • tagging is a good idea!
  • doesn't like that filtering changes the scroll position
  • expects the navigation to react
  • might be good to have a classic form
  • would prefer filters on the left
  • likes the word hide/show
  • hard to do
  • show full description: probably not have success criteria details collapsed
  • glossary items show up in a dialogue
  • new WCAG reference
  • share function important
  • opened items could be displayed more prominent
  • good to compare different success criteria: usually uses two browser windows to compare
  • the color of the levels look like links
  • likes the borders that indicate nesting the techniques
  • would like to see non-alphabetic order of techniques, General, HTML, ARIA, CSS, …

Prototype (1)

  • likes (1) very much
  • changes are more noticeable
  • position of current filters good
  • would like to have understanding expandable
  • techniques might be expandable as well
  • keep context!
  • light box?
  • techniques navigation horizontally?
  • scared of the interactiveness of the overview navigation
  • contrasts in overview navigation too different, attention grabbing

Session V

Very inexperienced. Youtube

Prototype (4)

  • text too small
  • adaptable
  • doesn't see column 2

Prototype (7)

  • in the Netherlands, additional information goes to the right.
  • contents in the left sidebar seems much more supplementary to the participant
  • the participant thought customize was for customizing the layout of the page

Session VI

Experienced with WCAG, more researcher type. Youtube

Prototype (7)

  • does not see customize right away but after a few seconds
  • likes the filters
  • confused as didn’t know which tags are active and which were not (none were)
  • want to select multiple tags at once
  • clicks guidelines techniques button, thinking it was the button for the success criteria below
  • didn't know the success criteria on guidelines level
  • icons are unclear-ish
  • is confused by the abbreviation in the link “Understanding SC 1.1.1”
  • expects complete text of success criteria visible somehow
  • uses the only button
  • PDF and audio -> expect to see success criteria where both apply at the same time
  • are filters and tags different?
  • clicking a tag again to unselect is unexpected
  • things tags are useful
  • would love to see the tags in the content as well
  • uses the table of contents
  • doesn't use back to top link, also not on other sites
  • would like to see understanding in-line In the document as well (expandable)
  • thinks linking out is okay

Session VII

Researcher for cognitive disabilities, relatively new to accessibility and WCAG. (Unfortunately no video of this session due to technical problems.)

Prototype (4)

  • has read WCAG2 to support research
  • did go into deeper layers and to see what is applicaple to the research
  • doesn’t look into overview navigation and customization
  • immediately starts reading
  • finds “future” links confusing
  • wonders if tags are sufficient
  • not clear how the content changes
  • levels not really clear

Prototype (3)

  • thinks focus is too much on the right
  • reading experience better than in (4)
  • “this might actually make more sense”
  • collapses sidebars, doesn’t find it too useful as reading is easier on shorter line lengths
  • does not like how the navigation changes on scrolling
  • back to top link is not seen, no need to focus on the bottom. doesn’t usually use back to top links.
  • finds hide/show text confusing, wants better icons

Session VIII

Evaluator/developer. Youtube

Prototype (5)

  • immediately explores filters and tags
  • confused by guideline technique
  • would prefer understanding to be shown in the same page
  • technique navigation “nice”
  • would like an open in another tab icon for techniques
  • would not inline techniques
  • liked share button, was confused by the email link text
  • would copy and paste the url instead of using the link anyways
  • expects social share links (like twitter, facebook)
  • doesn’t close customize panel
  • likes overview, but when clicking a link, the content disappears behind the fixed top bar
  • tags are good if you don’t know what is in WCAG and where it is
  • thinks would use tags a lot
  • expects to select multiple tags, items that have one tag or the other should be displayed
  • tries to click 1.1.1 and level A

Session IX

Regulatory background. Youtube

Prototype (3)

  • hides left, shows tags, likes collapsible areas
  • expects that filtering for AA automatically contains A as well as A could be missed when someone requires level AA
  • wants a possibility to select all filters of a group (in addition to clear all)
  • confused by the filtered out notation
  • thinks a delete button for individual filters would be good
  • would like to have levels and techniques separated in the filter status bar
  • thinks color scheme can be improved
  • likes the scrolling of the overview section
  • thinks navigation is good for orientation purposes
  • scrolling on the left might be distracting
  • would prefer to stay on the page
  • thinks a close techniques button on the bottom of the techniques could be helpful
  • also possibility to open all techniques to help searching with ctrl/cmd+f
    • we then briefly talked about search in general, liked the idea of the inline search that displays how many things are invisible, we came up with the idea of intercepting ctrl/cmd+f setting the focus to our own search bar (and notifying users that we have done this and give them the option to opt out)
    • Another idea: write how many search results are in the collapsed sections into the heading of that section
  • misses the tag “forms”, wants us to look at types of content when gathering tags

Themes & Topics

Overview navigation

  • Several participants didn’t like that the navigation changed in their view. It was seen as a major distraction. On the other side, they also thought that having the state shown.


  • Update the navigation only when scrolling stops, instead of continuously.
  • Alternatively: Have this navigation in a horizontal way with drop downs. — Quick brainstorm: Would probably require a bit of additional work, and I’m not sure if it really is a usability improvement. {EricE, 2015-06-29}


  • Either participants noted both columns right away or they ignored one or both of them. There seems to be no clear preferred column order or positioning.
  • Collapsing the sidebars generally was seen as useful.


  • Decide for a column layout, make it visually very obvious which column does what. — I think I prefer a layout that is similar to prototype 3. {EricE, 2015-06-29}

Checkmark icon

  • Two participants noted that the checkmark in the new status indicator is not working for them. Especially the always present checkmark icon was confusing.


  • Rework the status indicator.
  • Also, we need to have a “yellow fade” effect or something similar inside the content.


  • One participant proposed icons for principles and guidelines, so they would be easier to recognize. — I guess that would mean a lot of coordination with WCAG WG. {Eric, 2015-06-19}
  • One participant commented that there were some unclear icons. — That is not too surprising as this is still a prototype. {Eric, 2015-06-19}


  • Add some icons, clarify the use of the icons we are using.

“Back to top” link

  • None of the participants used the back to top link without being pointed at it.


  • I think they just don’t expect a fixed to top link on the bottom of the page.


  • Maybe a fixed to top link in the left sidebar could be more visible and apparent for people as that’s the navigation column.

“Only” button

  • The “only“ button to chose only one technique or level was either ignored or very much liked.


  • Leave as is.


  • Tags unclear if there are some activated when loading the page for the first time.
  • Tag as a word not clear (confused with HTML tags).
  • Participants wanted to be able to select more than one tag, yet there was no consensus on how they should work.
    • AND: When “live” and “video” is selected, they expected only success criteria to show up that has both, “live” and “video”, as tags.
    • OR: When “live” and “video” is selected, they expected success criteria to show up that has either, “live” or “video”, as tags.



Undecided. Needs group input.

Filter groups (Levels & Techniques)

  • One participant wanted a possibility to reset the filter groups individually.


Add a “select all” button to the filter groups.


  • One participant pointed out the that using the in-browser search is not informing them about search results that are on the page but hidden.


  • We then briefly talked about search in general, liked the idea of the inline search that displays how many things are invisible, we came up with the idea of intercepting ctrl/cmd+f setting the focus to our own search bar (and notifying users that we have done this and give them the option to opt out)
  • Another idea: write how many search results are in the collapsed sections into the heading of that section


  • Unsure if we really want to pursue this as it is really hard work (that is mostly already done).
  • At least we need to have “expand all”/“collapse all” buttons.