Results of Bookmarklet Tests for Issue 78
This page describes an evaluation of HTML web pages to Adapting Text SC metrics. Alastair Campbell's Bookmarklet automatically sets spacing to what is specified in the SC. That is:
- line height (line spacing) to at least 1.5 times the font size
- spacing underneath paragraphs to at least 2 times the font size
- letter spacing (tracking) to at least 0.12 times the font size
- word spacing to at least 0.16 times the font size
Between July 25 and August 3, 2017, 67 pages were tested. They included top 50 Alexa pages and some smaller pages. The pages tested may have changed since that time.
Testing Process
First a bookmark of a page was created and the URL was replaced with the following:
javascript:(function()%7Bdocument.body.appendChild(document.createElement(%27script%27)).src%3D%27https://alastairc.uk/tests/layouts/text-adaptation.js%27%3B%7D)()%3B
Then:
- Zoom level was set to 100%.
- Window size was set to 1280 pixels by 768 pixels. If the viewport not was at 1280 pixels wide due to scroll bars, the window was increased until viewport was 1280 pixels.
- The page to be tested was visited.
- The new bookmark was clicked (activated).
The following information was recorded:
- Spacing changes according to spec? (Pass/Fail)
- Loss of content or functionality? (Pass/Fail)
- Screenshot of the page was taken and it's link added to the table.
Optionally after running the initial test, some pages were tested at and 400%. That procedure was:
- Reload
- Zoom to 400%
- Test
- Document any zoom problems in the table
Results
Please note: cells in the following Bookmarklet Data Table marked N/A means that the bookmarklet script refused to load. A header was sent that prevented the external script from running. This will not be a problem if/when the script is put into a proper extension, but it did prevent the bookmarklet from working for 6 test cases. In addition it will not be a problem for developers with local environments as they rarely have that header.
| Web Page Tested | Spacing changes according to spec? (Pass/Fail) | Loss of content or functionality? (Pass/Fail) | Testing Notes | Link to Screenshot | Comments | Date tested | Initials of person(s) testing |
|---|---|---|---|---|---|---|---|
| Google.com | Pass | Pass | Also tested at 400% - Pass. | Google screenshot | Rated #1 in Alexa top sites | 26-Jul-2017 | AA |
| Youtube.com | Pass | Pass | Also tested at 400% - Pass. | Youtube screenshot | Rated #2 in Alexa top sites. | 26-Jul-2017 | AA |
| Facebook.com | N/A | N/A | Refused to load the script. | N/A | Rated #3 in Alexa top sites. | 25-Jul-2017 | GLS |
| Baidu.com | Pass | Fail | AA: Small failure - footer text is lost slightly. Also tested at 400% - same problem at 100% |
Baidu screenshot | Rated #4 in Alexa top sites. | 26-Jul-2017 | AA |
| Wikipedia.org | Pass | Pass | Actual page tested: Wikipedia Main Page | Wikipedia screenshot | Rated #5 in Alexa top sites. | 25-Jul-2017 | GLS LC |
| Wikipedia Mobile | Pass | Pass | Also tested at 400% - pass. |
Wikipedia mobile screenshot | AC | ||
| Yahoo.com | Pass | Fail | AA: Text is cut off from the 'Featured Links' section. Also tested at 400% - Navigation content is lost. LC: Truncated text: In the heading "NFL lineman decides to retire for unusual reason," the bottom portion |
Yahoo screenshot | Rated #6 in Alexa top sites. | 26-Jul-2017 3-Aug-2017 |
AA LC |
| Qq.com | Pass | Fail | Text is cut off | Qq screenshot | Rated #7 in Alexa top sites. | 26-Jul-2017 | AA |
| Reddit.com | Pass | Pass | Also tested at 400% - Fails due to zoom | Reddit screenshot | Rated #8 in Alexa top sites. | 26-Jul-2017 | AA |
| Google.co.in | Pass | Pass | Also tested at 400% - Fails due to zoom | Google screenshot | Rated #9 in Alexa top sites. | 26-Jul-2017 | AA |
| Amazon.com | Pass | Pass | Also tested at 400% - Content overlaps | Amazon screenshot | Rated #10 in Alexa top sites. | 26-Jul-2017 | AA |
| Taobao.com | Pass | Fail | Navigation content is jumbled up | Taobao screenshot | Rated #11 in Alexa top sites. | 26-Jul-2017 | AA |
| Twitter.com | N/A | N/A | Refused to load the script. | N/A | Rated #12 in Alexa top sites. | 26-Jul-2017 | AA |
| Google.co.jp | Pass | Pass | Also tested at 400% - Fails due to zoom | Google.co.jp screenshot | Rated #13 in Alexa top sites. | 26-Jul-2017 | AA |
| Tmall.com | Pass | Pass | Also tested at 400% - pass. | Tmall screenshot | Rated #14 in Alexa top sites. | 26-Jul-2017 | AA |
| Vk.com | Pass | Pass | Vk screenshot | Rated #15 in Alexa top sites. | 26-Jul-2017 | AA | |
| Live.com | Pass | Pass | Live screenshot | Rated #16 in Alexa top sites. | 26-Jul-2017 | AA | |
| Sohu.com | Pass | Pass | Sohu screenshot | Rated #17 in Alexa top sites. | 28-Jul-2017 | LC | |
| Instagram.com | Pass | Pass | Also tested at 400% - pass. | Instagram screenshot | Rated #18 in Alexa top sites. | 26-Jul-2017 | AA |
| Sina.com.cn | Pass | Pass | Sina screenshot | Rated #19 in Alexa top sites. | 26-Jul-2017 | AA | |
| Google.de | Pass | Pass | Also tested at 400% - Fails due to zoom | Google DE screenshot | Rated #23 in Alexa top sites. | 26-Jul-2017 | AA |
| Google.co.uk | Pass | Pass | Also tested at 400% - Fails due to zoom | Google UK screenshot | Rated #24 in Alexa top sites. | 26-Jul-2017 | AA |
| Linkedin.com | Pass | Pass | LC: LinkedIn Homepage sign in page passes for me. AA: However, failure after signing in. The small 'New Connections' section had text that overlapped and overflowed. In order to avoid this the container needs to be able to expand so that the text can flow smoothly and freely within the container. |
Linkedin screenshot | Rated #25 in Alexa top sites. | 26-Jul-2017 | AA |
| Google.com.br | Pass | Pass | Also tested at 400% - Fails due to zoom | Google BR screenshot | Rated #27 in Alexa top sites. | 26-Jul-2017 | AA |
| Google.fr | Pass | Pass | Also tested at 400% - Fails due to zoom | Google FR screenshot | Rated #28 in Alexa top sites. | 26-Jul-2017 | AA |
| Google.ru | Pass | Pass | Also tested at 400% - Fails due to zoom | Google RU screenshot | Rated #31 in Alexa top sites. | 26-Jul-2017 | AA |
| N/A | N/A | Refused to load the script. | N/A | Rated #30 in Alexa top sites. | 28-Jul-2017 | LC | |
| Google.com.hk | Pass | Pass | Also tested at 400% - Pass | Google HK screenshot | Rated #32 in Alexa top sites. | 26-Jul-2017 | AA |
| Yahoo.co.jp | Pass | Pass | Yahoo JP screenshot | Rated #33 in Alexa top sites. | 28-Jul-2017 | LC | |
| Google.it | Pass | Pass | Text is cut off from 'Top Stories' - however doesn't affect usability Actual page tested: Google Search results |
Google IT search results screenshot | Rated #34 in Alexa top sites. | 26-Jul-2017 | AA |
| T.co | Pass | Pass | T screenshot | Rated #35 in Alexa top sites. | 28-Jul-2017 | LC | |
| Ebay.com | Pass | Pass | Ebay screenshot | Rated #36 in Alexa top sites. | 26-Jul-2017 | AA | |
| Google.es | Pass | Pass | Google ES screenshot | Rated #37 in Alexa top sites. | 26-Jul-2017 | AA | |
| Imgur.com | Pass | Pass | Imgur screenshot | Rated #39 in Alexa top sites. | 26-Jul-2017 | AA | |
| Bing.com | Pass | Fail | Text above photos at bottom of page is truncated "..." | Bing screenshot | Rated #40 in Alexa top sites. | 26-Jul-2017 | AA |
| Twitch.tv | Pass | Fail | Loss of content | Twitch TV screenshot | Rated #41 in Alexa top sites. | 26-Jul-2017 | AA |
| Msn.com | Pass | Fail | AA: Loss of content. |
MSN screenshot | Rated #42 in Alexa top sites. | 26-Jul-2017 3-Aug-2017 |
AA LC |
| Alipay.com | Pass | Pass | Actual page tested: zhuzhan20160818 | Alipay screenshot | Rated #43 in Alexa top sites. | 26-Jul-2017 | AA |
| Google.ca | Pass | Pass | Google CA screenshot | Rated #44 in Alexa top sites. | 26-Jul-2017 | AA | |
| Tumblr.com | N/A | N/A | Refused to load the script. | N/A | Rated #45 in Alexa top sites. | 26-Jul-2017 | AA |
| Google.com.mx | Pass | Pass | Google MX screenshot | Rated #46 in Alexa top sites. | 26-Jul-2017 | AA | |
| Onclkds.com | Pass | Pass | Also tested at 400% - Pass | Onclkds screenshot | Rated #47 in Alexa top sites. | 26-Jul-2017 | AA |
| Mail.ru | N/A | N/A | Refused to load the script. | N/A | Rated #49 in Alexa top sites. | 26-Jul-2017 | AA |
| Ok.ru | N/A | N/A | Refused to load the script. | N/A | Rated #50 in Alexa top sites. | 26-Jul-2017 | AA |
| BBC - Homepage | Pass | Pass | Also tested at 400% - Pass | BBC screenshot | 26-Jul-2017 | AA | |
| Wellness Mama: Water Kefir Recipe | Pass | Pass | Wellness Mama screenshot | 26-Jul-2017 | AA | ||
| Argos | Fail | Fail | AA: Navigation text doesn't change. Loss of content. Also tested at 400% - Same issue at 100% LC: Text does not changing according to the SC's metrics in the mega menu. |
Argos screenshot | 27-Jul-2017 3-Aug-2017 |
AA LC |
|
| Boots | Pass | Fail | Loss of content Also tested at 400% - Pass |
Boots screenshot | 27-Jul-2017 | AA | |
| National Rail | Pass | Fail | Content overlaps Also tested at 400% - Same issue at 100% |
National Rail screenshot | 27-Jul-2017 | AA | |
| Bristol Post | Pass | Pass | Also tested at 400% - Pass | Bristol Post screenshot | 27-Jul-2017 | AA | |
| University of Birmingham | Pass | Pass | Also tested at 400% - Pass | U Birmingham screenshot | 27-Jul-2017 | AA | |
| Las Iguanas | Pass | Pass | Also tested at 400% - Pass | Iguanas screenshot | 27-Jul-2017 | AA | |
| Three | Pass | Pass | Also tested at 400% - Pass | 3 screenshot | 27-Jul-2017 | AA | |
| Leeds City Council | Pass | Pass | Also tested at 400% - Pass | Leeds gov screenshot | 27-Jul-2017 | AA | |
| British Gas | Pass | Fail | AA: Loss of content. Tested at 400% - Pass LC: Overlapping Text. e.g. the word "complaints" overlaps and obscures the text "Want to know what we are doing." The word "offers" also overlays and obscures the text beneath it. It is unreadable. |
British Gas screenshot | 27-Jul-2017 3-Aug-2017 |
AA LC |
|
| Confused.com | Pass | Pass | Tested at 400% - Pass | Confused screenshot | 27-Jul-2017 | AA | |
| adobe.com | Pass | Pass | Adobe screenshot | 3-Aug-2017 | LC | ||
| bestbuy.com | Pass | Fail | Loss of content. Truncated text: "See our recommendations" is unreadable. Only the very top portion is visible. | Bestbuy screenshot | 3-Aug-2017 | LC | |
| cnn.com | Pass | Pass | CNN screenshot | 3-Aug-2017 | LC | ||
| dnb.de | Pass | Pass | Deutsche Nationalbibliothek screenshot | 3-Aug-2017 | LC | ||
| duluthgrill.com | Pass | Pass | Duluth Grill screenshot | 3-Aug-2017 | LC | ||
| ens-lyon.fr | Pass | Fail | Loss of content. Overlapping and truncated text. | Ecole Normale Supérieure de Lyon screenshot | 3-Aug-2017 | LC | |
| exxonmobil.com | Pass | Pass | Exxon Mobil screenshot | 3-Aug-2017 | LC | ||
| foxnews.com | Pass | Fail | Loss of content. Truncated text. e.g. Only the very top portion of the Nasdaq percentage is visible. | Fox News screenshot | 3-Aug-2017 | LC | |
| northwestoutlet.com | Pass | Pass | Northwest Outlet screenshot | 3-Aug-2017 | LC | ||
| npr.org | Pass | Pass | NPR screenshot | 3-Aug-2017 | LC | ||
| usa.gov | Pass | Pass | USA.gov screenshot | 3-Aug-2017 | LC |
Discussion
Most pages passed the Adapting Text SC. However, some failed due to fixed-height CSS containers that caused text to overlap, truncate, or overflow containers. The following 14 web pages failed the SC.
- Argos
- Baidu
- Best Buy
- Boots
- Bing
- British Gas
- Ecole Normale Supérieure de Lyon
- Fox News
- MSN
- National Rail
- Taobao
- Twitch
- Yahoo
Types of Failures
Types of failures include cut off text, overlapping text, and fixed that is unable to be overridden to the Success Criterion's metrics.
Example 1: Text Cut Off
The bottom portion of the words "Your Needs" is cut off in a heading making that text unreadable. It should read "We Provide a Mobile Application Service to Meet Your Needs."
Figure 1: Text cut off vertically is a failure.
In Figure 2 the last portion of text is cut off in 3 side-by-side headings. The 1st heading should read "A cog in the wheel." But it reads "A cog in the whe". Only half of the second "e" is visible and the letter "l" is completely missing. The 2nd heading should read "A penny for your thoughts". But it reads "A penny for your". The 3rd should read "Back to the drawing board." But it reads "Back to the drawi".
Figure 2: Text cut off horizontally is a failure.
Example 2: Text Overlap
In Figure 3 the last 3 words "Groups and Programs" of the heading "Technologists Seeking Input from Groups and Programs" overlap the following sentence. That sentence should read, "You are invited to share ideas and areas of interest related to the integration of technology from a group or program perspective." But the words "You are invited to share ideas" are obscured and unreadable.
Figure 3: Overlapping text is a failure.
Example 3: Fixed Text
Text fails the SC when it is fixed and not able to be overridden to the Success Criterion's metrics.
Text that allows for overriding to the metrics passes. Line height must be able to adapt to 1.5 times the font size. Letter spacing must be able to adapt to 0.12 times the font size. Word spacing must be able to adapt to 0.16 times the font size. Spacing underneath paragraphs must be able to adapt to 2 times the font size.
Figure 4: Absolutely fixed text is a failure. Adaptable text passes.
Conclusions
Most Sites Adapt Well
Overall most content on most sites adapt well. Based on this evaluation, the metrics specified in the Adapting Text SC should be feasible in WCAG 2.1 for HTML documents. Spacing is able to adapt without loss of content or functionality across a wide range of sites.
Techniques
A technique has been drafted, Allowing for Spacing Override, to show how to test for the metrics set in this SC.
Most or all of the issues found in this study could be solved by a technique (similar to C28: Specifying the size of text containers using em units) to use min-height instead of height. To pass the Adapting Text SC containers must be able to expand. This can be done with technologies that use CSS by ensuring a min-height property is set rather than height. Setting a min-height ensures containers will always expand to 100%. And if needed, they can go past 100%.
Evaluators
- Alastair Campbell
- Amani Ali
- Glenda Sims
- Laura Carlson
Special thanks to Amani for doing the lion's share of the testing and providing a first draft of the discussion section of this document.