This Wiki page is edited by participants of the WCAG Working Group. It does not necessarily represent consensus and it may have incorrect information or information that is not supported by other Working Group participants, WAI, or W3C. It may also have some very useful information.

Results of Bookmarklet Tests for Issue 78

From WCAG WG
Jump to: navigation, search


This page describes an evaluation of HTML web pages to Adapting Text SC metrics. Alastair Campbell's Bookmarklet automatically sets spacing to what is specified in the SC. That is:

  1. line height (line spacing) to at least 1.5 times the font size
  2. spacing underneath paragraphs to at least 2 times the font size
  3. letter spacing (tracking) to at least 0.12 times the font size
  4. word spacing to at least 0.16 times the font size

Between July 25 and August 3, 2017, 67 pages were tested. They included top 50 Alexa pages and some smaller pages. The pages tested may have changed since that time.

Testing Process

First a bookmark of a page was created and the URL was replaced with the following:

javascript:(function()%7Bdocument.body.appendChild(document.createElement(%27script%27)).src%3D%27https://alastairc.uk/tests/layouts/text-adaptation.js%27%3B%7D)()%3B

Then:

  1. Zoom level was set to 100%.
  2. Window size was set to 1280 pixels by 768 pixels. If the viewport not was at 1280 pixels wide due to scroll bars, the window was increased until viewport was 1280 pixels.
  3. The page to be tested was visited.
  4. The new bookmark was clicked (activated).

The following information was recorded:

  • Spacing changes according to spec? (Pass/Fail)
  • Loss of content or functionality? (Pass/Fail)
  • Screenshot of the page was taken and it's link added to the table.

Optionally after running the initial test, some pages were tested at and 400%. That procedure was:

  1. Reload
  2. Zoom to 400%
  3. Test
  4. Document any zoom problems in the table

Results

Please note: cells in the following Bookmarklet Data Table marked N/A means that the bookmarklet script refused to load. A header was sent that prevented the external script from running. This will not be a problem if/when the script is put into a proper extension, but it did prevent the bookmarklet from working for 6 test cases. In addition it will not be a problem for developers with local environments as they rarely have that header.

 
Bookmarklet Data
Web Page Tested Spacing changes according to spec? (Pass/Fail) Loss of content or functionality? (Pass/Fail) Testing Notes Link to Screenshot Comments Date tested Initials of person(s) testing
Google.com Pass Pass Also tested at 400% - Pass. Google screenshot Rated #1 in Alexa top sites 26-Jul-2017 AA
Youtube.com Pass Pass Also tested at 400% - Pass. Youtube screenshot Rated #2 in Alexa top sites. 26-Jul-2017 AA
Facebook.com N/A N/A Refused to load the script. N/A Rated #3 in Alexa top sites. 25-Jul-2017 GLS
Baidu.com Pass Fail AA: Small failure - footer text is lost slightly.
Also tested at 400% - same problem at 100%
Baidu screenshot Rated #4 in Alexa top sites. 26-Jul-2017 AA
Wikipedia.org Pass Pass Actual page tested: Wikipedia Main Page Wikipedia screenshot Rated #5 in Alexa top sites. 25-Jul-2017 GLS
LC
Wikipedia Mobile Pass Pass

Also tested at 400% - pass.
Actual page tested: Wikipedia Mobile View

Wikipedia mobile screenshot AC
Yahoo.com Pass Fail AA: Text is cut off from the 'Featured Links' section. Also tested at 400% - Navigation content is lost.
LC: Truncated text: In the heading "NFL lineman decides to retire for unusual reason," the bottom portion
Yahoo screenshot Rated #6 in Alexa top sites. 26-Jul-2017
3-Aug-2017
AA
LC
Qq.com Pass Fail Text is cut off Qq screenshot Rated #7 in Alexa top sites. 26-Jul-2017 AA
Reddit.com Pass Pass Also tested at 400% - Fails due to zoom Reddit screenshot Rated #8 in Alexa top sites. 26-Jul-2017 AA
Google.co.in Pass Pass Also tested at 400% - Fails due to zoom Google screenshot Rated #9 in Alexa top sites. 26-Jul-2017 AA
Amazon.com Pass Pass Also tested at 400% - Content overlaps Amazon screenshot Rated #10 in Alexa top sites. 26-Jul-2017 AA
Taobao.com Pass Fail Navigation content is jumbled up Taobao screenshot Rated #11 in Alexa top sites. 26-Jul-2017 AA
Twitter.com N/A N/A Refused to load the script. N/A Rated #12 in Alexa top sites. 26-Jul-2017 AA
Google.co.jp Pass Pass Also tested at 400% - Fails due to zoom Google.co.jp screenshot Rated #13 in Alexa top sites. 26-Jul-2017 AA
Tmall.com Pass Pass Also tested at 400% - pass. Tmall screenshot Rated #14 in Alexa top sites. 26-Jul-2017 AA
Vk.com Pass Pass Vk screenshot Rated #15 in Alexa top sites. 26-Jul-2017 AA
Live.com Pass Pass Live screenshot Rated #16 in Alexa top sites. 26-Jul-2017 AA
Sohu.com Pass Pass Sohu screenshot Rated #17 in Alexa top sites. 28-Jul-2017 LC
Instagram.com Pass Pass Also tested at 400% - pass. Instagram screenshot Rated #18 in Alexa top sites. 26-Jul-2017 AA
Sina.com.cn Pass Pass Sina screenshot Rated #19 in Alexa top sites. 26-Jul-2017 AA
Google.de Pass Pass Also tested at 400% - Fails due to zoom Google DE screenshot Rated #23 in Alexa top sites. 26-Jul-2017 AA
Google.co.uk Pass Pass Also tested at 400% - Fails due to zoom Google UK screenshot Rated #24 in Alexa top sites. 26-Jul-2017 AA
Linkedin.com Pass Pass LC: LinkedIn Homepage sign in page passes for me.
AA: However, failure after signing in. The small 'New Connections' section had text that overlapped and overflowed. In order to avoid this the container needs to be able to expand so that the text can flow smoothly and freely within the container.
Linkedin screenshot Rated #25 in Alexa top sites. 26-Jul-2017 AA
Google.com.br Pass Pass Also tested at 400% - Fails due to zoom Google BR screenshot Rated #27 in Alexa top sites. 26-Jul-2017 AA
Google.fr Pass Pass Also tested at 400% - Fails due to zoom Google FR screenshot Rated #28 in Alexa top sites. 26-Jul-2017 AA
Google.ru Pass Pass Also tested at 400% - Fails due to zoom Google RU screenshot Rated #31 in Alexa top sites. 26-Jul-2017 AA

Yandex.ru

N/A N/A Refused to load the script. N/A Rated #30 in Alexa top sites. 28-Jul-2017 LC
Google.com.hk Pass Pass Also tested at 400% - Pass Google HK screenshot Rated #32 in Alexa top sites. 26-Jul-2017 AA
Yahoo.co.jp Pass Pass Yahoo JP screenshot Rated #33 in Alexa top sites. 28-Jul-2017 LC
Google.it Pass Pass Text is cut off from 'Top Stories' - however doesn't affect usability
Actual page tested: Google Search results
Google IT search results screenshot Rated #34 in Alexa top sites. 26-Jul-2017 AA
T.co Pass Pass T screenshot Rated #35 in Alexa top sites. 28-Jul-2017 LC
Ebay.com Pass Pass Ebay screenshot Rated #36 in Alexa top sites. 26-Jul-2017 AA
Google.es Pass Pass Google ES screenshot Rated #37 in Alexa top sites. 26-Jul-2017 AA
Imgur.com Pass Pass Imgur screenshot Rated #39 in Alexa top sites. 26-Jul-2017 AA
Bing.com Pass Fail Text above photos at bottom of page is truncated "..." Bing screenshot Rated #40 in Alexa top sites. 26-Jul-2017 AA
Twitch.tv Pass Fail Loss of content Twitch TV screenshot Rated #41 in Alexa top sites. 26-Jul-2017 AA
Msn.com Pass Fail

AA: Loss of content.
LC: e.g. The text "Name These 1950s" is truncated in a sponsored card (Lucille Ball). Text should read "Only A Few Americans Can Name These 1950s". Also occurs in "Most Viewed" and "Trending Video" sections.

MSN screenshot Rated #42 in Alexa top sites. 26-Jul-2017
3-Aug-2017
AA
LC
Alipay.com Pass Pass Actual page tested: zhuzhan20160818 Alipay screenshot Rated #43 in Alexa top sites. 26-Jul-2017 AA
Google.ca Pass Pass Google CA screenshot Rated #44 in Alexa top sites. 26-Jul-2017 AA
Tumblr.com N/A N/A Refused to load the script. N/A Rated #45 in Alexa top sites. 26-Jul-2017 AA
Google.com.mx Pass Pass Google MX screenshot Rated #46 in Alexa top sites. 26-Jul-2017 AA
Onclkds.com Pass Pass Also tested at 400% - Pass Onclkds screenshot Rated #47 in Alexa top sites. 26-Jul-2017 AA
Mail.ru N/A N/A Refused to load the script. N/A Rated #49 in Alexa top sites. 26-Jul-2017 AA
Ok.ru N/A N/A Refused to load the script. N/A Rated #50 in Alexa top sites. 26-Jul-2017 AA
BBC - Homepage Pass Pass Also tested at 400% - Pass BBC screenshot 26-Jul-2017 AA
Wellness Mama: Water Kefir Recipe Pass Pass Wellness Mama screenshot 26-Jul-2017 AA
Argos Fail Fail AA: Navigation text doesn't change. Loss of content. Also tested at 400% - Same issue at 100%
LC: Text does not changing according to the SC's metrics in the mega menu.
Argos screenshot 27-Jul-2017
3-Aug-2017
AA
LC
Boots Pass Fail Loss of content
Also tested at 400% - Pass
Boots screenshot 27-Jul-2017 AA
National Rail Pass Fail Content overlaps
Also tested at 400% - Same issue at 100%
National Rail screenshot 27-Jul-2017 AA
Bristol Post Pass Pass Also tested at 400% - Pass Bristol Post screenshot 27-Jul-2017 AA
University of Birmingham Pass Pass Also tested at 400% - Pass U Birmingham screenshot 27-Jul-2017 AA
Las Iguanas Pass Pass Also tested at 400% - Pass Iguanas screenshot 27-Jul-2017 AA
Three Pass Pass Also tested at 400% - Pass 3 screenshot 27-Jul-2017 AA
Leeds City Council Pass Pass Also tested at 400% - Pass Leeds gov screenshot 27-Jul-2017 AA
British Gas Pass Fail AA: Loss of content. Tested at 400% - Pass
LC: Overlapping Text. e.g. the word "complaints" overlaps and obscures the text "Want to know what we are doing." The word "offers" also overlays and obscures the text beneath it. It is unreadable.
British Gas screenshot 27-Jul-2017
3-Aug-2017
AA
LC
Confused.com Pass Pass Tested at 400% - Pass Confused screenshot 27-Jul-2017 AA
adobe.com Pass Pass   Adobe screenshot 3-Aug-2017 LC
bestbuy.com Pass Fail Loss of content. Truncated text: "See our recommendations" is unreadable. Only the very top portion is visible. Bestbuy screenshot 3-Aug-2017 LC
cnn.com Pass Pass   CNN screenshot 3-Aug-2017 LC
dnb.de Pass Pass   Deutsche Nationalbibliothek screenshot 3-Aug-2017 LC
duluthgrill.com Pass Pass   Duluth Grill screenshot 3-Aug-2017 LC
ens-lyon.fr Pass Fail Loss of content. Overlapping and truncated text. Ecole Normale Supérieure de Lyon screenshot 3-Aug-2017 LC
exxonmobil.com Pass Pass   Exxon Mobil screenshot 3-Aug-2017 LC
foxnews.com Pass Fail Loss of content. Truncated text. e.g. Only the very top portion of the Nasdaq percentage is visible. Fox News screenshot 3-Aug-2017 LC
northwestoutlet.com Pass Pass   Northwest Outlet screenshot 3-Aug-2017 LC
npr.org Pass Pass   NPR screenshot 3-Aug-2017 LC
usa.gov Pass Pass   USA.gov screenshot 3-Aug-2017 LC

Discussion

Most pages passed the Adapting Text SC. However, some failed due to fixed-height CSS containers that caused text to overlap, truncate, or overflow containers. The following 14 web pages failed the SC.

  1. Argos
  2. Baidu
  3. Best Buy
  4. Boots
  5. Bing
  6. British Gas
  7. Ecole Normale Supérieure de Lyon
  8. Fox News
  9. MSN
  10. National Rail
  11. Qq
  12. Taobao
  13. Twitch
  14. Yahoo

Types of Failures

Types of failures include cut off text, overlapping text, and fixed that is unable to be overridden to the Success Criterion's metrics.

Example 1: Text Cut Off

The bottom portion of the words "Your Needs" is cut off in a heading making that text unreadable. It should read "We Provide a Mobile Application Service to Meet Your Needs."

Figure 1: Text cut off vertically is a failure.


In Figure 2 the last portion of text is cut off in 3 side-by-side headings. The 1st heading should read "A cog in the wheel." But it reads "A cog in the whe". Only half of the second "e" is visible and the letter "l" is completely missing. The 2nd heading should read "A penny for your thoughts". But it reads "A penny for your". The 3rd should read "Back to the drawing board." But it reads "Back to the drawi".

Figure 2: Text cut off horizontally is a failure.

Example 2: Text Overlap

In Figure 3 the last 3 words "Groups and Programs" of the heading "Technologists Seeking Input from Groups and Programs" overlap the following sentence. That sentence should read, "You are invited to share ideas and areas of interest related to the integration of technology from a group or program perspective." But the words "You are invited to share ideas" are obscured and unreadable.

Figure 3: Overlapping text is a failure.

Example 3: Fixed Text

Text fails the SC when it is fixed and not able to be overridden to the Success Criterion's metrics.

Text that allows for overriding to the metrics passes. Line height must be able to adapt to 1.5 times the font size. Letter spacing must be able to adapt to 0.12 times the font size. Word spacing must be able to adapt to 0.16 times the font size. Spacing underneath paragraphs must be able to adapt to 2 times the font size.

Figure 4: Absolutely fixed text is a failure. Adaptable text passes.

Conclusions

Most Sites Adapt Well

Overall most content on most sites adapt well. Based on this evaluation, the metrics specified in the Adapting Text SC should be feasible in WCAG 2.1 for HTML documents. Spacing is able to adapt without loss of content or functionality across a wide range of sites.

Techniques

A technique has been drafted, Allowing for Spacing Override, to show how to test for the metrics set in this SC.

Most or all of the issues found in this study could be solved by a technique (similar to C28: Specifying the size of text containers using em units) to use min-height instead of height. To pass the Adapting Text SC containers must be able to expand. This can be done with technologies that use CSS by ensuring a min-height property is set rather than height. Setting a min-height ensures containers will always expand to 100%. And if needed, they can go past 100%.

Evaluators

  • Alastair Campbell
  • Amani Ali
  • Glenda Sims
  • Laura Carlson

Special thanks to Amani for doing the lion's share of the testing and providing a first draft of the discussion section of this document.