UF June 2015/Results

From Education & Outreach

Prototypes

  • (1) Both, overview and customization on the, left.
  • (2) Left and right columns for overview and customization, respectively.
  • (3) like (2) but with a fixed bar on the top.
  • (4) Two left side bars.
  • (5) Customize on the top of the main column.
  • (6) Customize as a button on the top of the left sidebar, tags closed when clicking on the button. — We did not test this because too similar to (7) and (7) seemed to be more sensible to test.
  • (7) Customize as a button on the top of the left sidebar, tags open when clicking on the button.

Sessions

All participants agreed to this Consent & Recording Release.

Session I

Evaluator. Youtube

Prototype (1)

  • Scrolls nicely
  • Understands tags and how to unselect them
  • Understands level filters
  • Not using “only” buttons
  • Inverted text
  • clear button preferred than doing it manually (this was since fixed)

Prototype (3)

  • Nice that it’s possible to hide customizations
  • Likes that the customize bar can be collapsed and that the button stays visible

Session II

Evaluator, very experienced with WCAG2. Youtube

Prototype (7)

  • Only first section expanded (in overview)
  • What’s the checkmark??
  • wonders where to filter
  • doesn't see the customize button
  • sees techniques (sufficient/advisory) as filters
  • find filters eventually
  • customize <=> filters
  • likes the “only” button
  • is confused by losing position in the document
  • wonders about the categories. They seemed generic.
  • once more specific tags, for example custom elements instead of keyboards, …
  • keyboard is a requirement, Not a need
  • wants a clear all filters button
  • again wondering about the checkmark. A signal for okay, not done.
  • expects understanding/techniques to open in the same tab (because W3C) but prefers a new tab
  • emphasize technique categories
  • select multiple tags -> expects to see only items where are all of the selected tags are applicaple
  • would prefer the scroll to top link to be on the left

Prototype (5)

  • not sure if likes the left side bar
  • collapse on zooming -> basically mobile view
  • prefers opinionated standard settings, for example hiding the side bar
  • likes if more space is left
  • would like to see iconography for principles and guidelines

Session III

Senior inspector, carrying out accessibility tests, trainer. Youtube

Prototype (2)

  • sees customize, doesn't know how to use it
  • clicks on on techniques for guideline repeatedly
  • doesn't know how it updates
  • is getting confused when only failures are open (we need to do more for highlighting)

Prototype (4)

  • tags very unclear, confused with tags in HTML
  • prefer the column on the right
  • likes the text height and show in the column headers
  • tags should also hide

Session IV

Evaluator, researcher. Had used the old quickref. Youtube, feedback session was conducted in German

Prototype (5)

  • likes the navigation
  • confuses guideline techniques for 1.1 with techniques for 1.1.1
  • likes to formatting better than old quickref
  • filtering good
  • link to techniques: depends on use case. Techniques long -> a new page is probably an advantage because of space issues.
  • likes the site bar
  • doesn't like that the overview menu collapses and expands while scrolling
  • tagging is a good idea!
  • doesn't like that filtering changes the scroll position
  • expects the navigation to react
  • might be good to have a classic form
  • would prefer filters on the left
  • likes the word hide/show
  • hard to do
  • show full description: probably not have success criteria details collapsed
  • glossary items show up in a dialogue
  • new WCAG reference
  • share function important
  • opened items could be displayed more prominent
  • good to compare different success criteria: usually uses two browser windows to compare
  • the color of the levels look like links
  • likes the borders that indicate nesting the techniques
  • would like to see non-alphabetic order of techniques, General, HTML, ARIA, CSS, …

Prototype (1)

  • likes (1) very much
  • changes are more noticeable
  • position of current filters good
  • would like to have understanding expandable
  • techniques might be expandable as well
  • keep context!
  • light box?
  • techniques navigation horizontally?
  • scared of the interactiveness of the overview navigation
  • contrasts in overview navigation too different, attention grabbing

Session V

Very inexperienced. Youtube

Prototype (4)

  • text too small
  • adaptable
  • doesn't see column 2

Prototype (7)

  • in the Netherlands, additional information goes to the right.
  • contents in the left sidebar seems much more supplementary to the participant
  • the participant thought customize was for customizing the layout of the page

Session VI

Experienced with WCAG, more researcher type. Youtube

Prototype (7)

  • does not see customize right away but after a few seconds
  • likes the filters
  • confused as didn’t know which tags are active and which were not (none were)
  • want to select multiple tags at once
  • clicks guidelines techniques button, thinking it was the button for the success criteria below
  • didn't know the success criteria on guidelines level
  • icons are unclear-ish
  • is confused by the abbreviation in the link “Understanding SC 1.1.1”
  • expects complete text of success criteria visible somehow
  • uses the only button
  • PDF and audio -> expect to see success criteria where both apply at the same time
  • are filters and tags different?
  • clicking a tag again to unselect is unexpected
  • things tags are useful
  • would love to see the tags in the content as well
  • uses the table of contents
  • doesn't use back to top link, also not on other sites
  • would like to see understanding in-line In the document as well (expandable)
  • thinks linking out is okay

Session VII

Researcher for cognitive disabilities, relatively new to accessibility and WCAG. (Unfortunately no video of this session due to technical problems.)

Prototype (4)

  • has read WCAG2 to support research
  • did go into deeper layers and to see what is applicaple to the research
  • doesn’t look into overview navigation and customization
  • immediately starts reading
  • finds “future” links confusing
  • wonders if tags are sufficient
  • not clear how the content changes
  • levels not really clear

Prototype (3)

  • thinks focus is too much on the right
  • reading experience better than in (4)
  • “this might actually make more sense”
  • collapses sidebars, doesn’t find it too useful as reading is easier on shorter line lengths
  • does not like how the navigation changes on scrolling
  • back to top link is not seen, no need to focus on the bottom. doesn’t usually use back to top links.
  • finds hide/show text confusing, wants better icons

Session VIII

Evaluator/developer. Youtube

Prototype (5)

  • immediately explores filters and tags
  • confused by guideline technique
  • would prefer understanding to be shown in the same page
  • technique navigation “nice”
  • would like an open in another tab icon for techniques
  • would not inline techniques
  • liked share button, was confused by the email link text
  • would copy and paste the url instead of using the link anyways
  • expects social share links (like twitter, facebook)
  • doesn’t close customize panel
  • likes overview, but when clicking a link, the content disappears behind the fixed top bar
  • tags are good if you don’t know what is in WCAG and where it is
  • thinks would use tags a lot
  • expects to select multiple tags, items that have one tag or the other should be displayed
  • tries to click 1.1.1 and level A

Session IX

Regulatory background. Youtube

Prototype (3)

  • hides left, shows tags, likes collapsible areas
  • expects that filtering for AA automatically contains A as well as A could be missed when someone requires level AA
  • wants a possibility to select all filters of a group (in addition to clear all)
  • confused by the filtered out notation
  • thinks a delete button for individual filters would be good
  • would like to have levels and techniques separated in the filter status bar
  • thinks color scheme can be improved
  • likes the scrolling of the overview section
  • thinks navigation is good for orientation purposes
  • scrolling on the left might be distracting
  • would prefer to stay on the page
  • thinks a close techniques button on the bottom of the techniques could be helpful
  • also possibility to open all techniques to help searching with ctrl/cmd+f
    • we then briefly talked about search in general, liked the idea of the inline search that displays how many things are invisible, we came up with the idea of intercepting ctrl/cmd+f setting the focus to our own search bar (and notifying users that we have done this and give them the option to opt out)
    • Another idea: write how many search results are in the collapsed sections into the heading of that section
  • misses the tag “forms”, wants us to look at types of content when gathering tags

Themes & Topics

Overview navigation

  • Several participants didn’t like that the navigation changed in their view. It was seen as a major distraction. On the other side, they also thought that having the state shown.

Proposal

  • Update the navigation only when scrolling stops, instead of continuously.
  • Alternatively: Have this navigation in a horizontal way with drop downs. — Quick brainstorm: Would probably require a bit of additional work, and I’m not sure if it really is a usability improvement. {EricE, 2015-06-29}

Columns

  • Either participants noted both columns right away or they ignored one or both of them. There seems to be no clear preferred column order or positioning.
  • Collapsing the sidebars generally was seen as useful.

Proposal

  • Decide for a column layout, make it visually very obvious which column does what. — I think I prefer a layout that is similar to prototype 3. {EricE, 2015-06-29}

Checkmark icon

  • Two participants noted that the checkmark in the new status indicator is not working for them. Especially the always present checkmark icon was confusing.

Proposal

  • Rework the status indicator.
  • Also, we need to have a “yellow fade” effect or something similar inside the content.

Iconography

  • One participant proposed icons for principles and guidelines, so they would be easier to recognize. — I guess that would mean a lot of coordination with WCAG WG. {Eric, 2015-06-19}
  • One participant commented that there were some unclear icons. — That is not too surprising as this is still a prototype. {Eric, 2015-06-19}

Proposal

  • Add some icons, clarify the use of the icons we are using.

“Back to top” link

  • None of the participants used the back to top link without being pointed at it.

Comment

  • I think they just don’t expect a fixed to top link on the bottom of the page.

Proposal

  • Maybe a fixed to top link in the left sidebar could be more visible and apparent for people as that’s the navigation column.

“Only” button

  • The “only“ button to chose only one technique or level was either ignored or very much liked.

Proposal

  • Leave as is.

Tags

  • Tags unclear if there are some activated when loading the page for the first time.
  • Tag as a word not clear (confused with HTML tags).
  • Participants wanted to be able to select more than one tag, yet there was no consensus on how they should work.
    • AND: When “live” and “video” is selected, they expected only success criteria to show up that has both, “live” and “video”, as tags.
    • OR: When “live” and “video” is selected, they expected success criteria to show up that has either, “live” or “video”, as tags.

Comment

Proposal

Undecided. Needs group input.

Filter groups (Levels & Techniques)

  • One participant wanted a possibility to reset the filter groups individually.

Proposal

Add a “select all” button to the filter groups.

Search

  • One participant pointed out the that using the in-browser search is not informing them about search results that are on the page but hidden.

Comment

  • We then briefly talked about search in general, liked the idea of the inline search that displays how many things are invisible, we came up with the idea of intercepting ctrl/cmd+f setting the focus to our own search bar (and notifying users that we have done this and give them the option to opt out)
  • Another idea: write how many search results are in the collapsed sections into the heading of that section

Proposal

  • Unsure if we really want to pursue this as it is really hard work (that is mostly already done).
  • At least we need to have “expand all”/“collapse all” buttons.
Retrieved from "https://www.w3.org/WAI/EO/wiki/index.php?title=UF_June_2015/Results&oldid=13274"