WCAG-EM Report Tool Comments

From Education & Outreach

Note: Comments marked [done] have been addressed. Comments marked [pending] have been resolved but not yet implemented.

comment template:

  • @@summary — comment {name, 00-August-2014}
  • [done] summary — comment {name}

Major Changes Since September

Some of the major changes between the previous EO review (in September 2014) and the current version 1.0 include:

  • Improvements to the overall navigation and styling
  • Extensive changes to the text and instructions
  • Moving selection functionality from Step 2 to Step 3
  • Web browser compatibility checking at the start
  • Other checks, such as inconsistency in the results
  • Testing and debugging, and detailed issue recording

More details on the comments received and how they were addressed is listed below. The Github issues list also shows open issues planned to be addressed in future versions.

February 2015 Review Comments

Survey Results - All comments have been addressed, except for one from Brent and another from Shawn, that will be addressed in the next revision (they impact page design and complexity). More specifics about how each comment was addressed is provided below:

  • [done] "there is a strange sign in the explanations that one can use ctrl+s or "strange sign" S to save the report" - added inline explanations, like "use Windows shortcut keys Ctrl+S or Mac shortcut keys ⌘S" to demystify the sign (thanks Shawn)
  • [done] "when choosing the next step button, I have no information that the page has changed unless I press the say title command. the focus remains on the "next step" button, is it intended?" - focus now goes to main page heading to indicate the change
  • [done] "You can save partially-complete reports (and notes) and work on them later" (add "and notes" to the sentence) - added on the save page but not on the main page, as it would be confusing there (people wouldn't know what the notes are)
  • [open] Add simple "collapse all" and "expand all" buttons at the top of the list, to the right of "Success Criteria to Evaluate" - added as issue 204 for future revisions (we expect the entire page may need updating after the usability testing)
  • [done] Howard's comments - all addressed as indicated in response to Howard's comments
  • [done] Shawn's comments - all addressed, except one minor issue, as indicated in response to Shawn's comments

September 2014 Review Comments

misc

  • [done] text - "This web browser is not supported. You will not be able to use all features, which may include loading and saving. Use a web browser for which you have the latest version instead." needs editing.
    • Agreed (issue #122). {Shadi, 5 Oct}
    • Changed to "The web browser you are using does not support some functionality, such as saving and loading reports. You can continue to use this report tool with limited functionality, or you may be able to update or use a different web browser." {Shadi, 6 Feb}
  • [done] CSUN Universal Design Center wants to do a thorough usability test with a comprehensive report of their findings :-) Unfortunately, they're in the middle of several other assessments and the lead time was too short, so they've only looked at it in a very cursory manner and don't have anything to add right now. It will come though. (Paul - 11 September 2014)
    • Thank you for pursuing this. Following-up offline about expected timeline. {Shadi, 12 Sept}
  • [done] User feedback on WCAG-EM Report Tool - 10 Sept e-mail from Kevin
    • Specific text edits (below) set better expectation and context for people new to the tool (and WCAG-EM). {Shadi, 5 Oct}

Overall

[thanks] This tool has some really awesome design & functionality ! ! !

Bugs

  • [done] URIs w/o http no worky - Step 4: Under "Sample to Evaluate", link icons: user input of "http://www.w3.org" works, but user input of "www.w3.org" gives a GitHub 404.
    For now: make URIs starting with www work.
    For enhancement: Consider a custom 404 page? {Shawn, 10-Sept}
    • Added as issue #136 to Github, good catch. {Wilco, 15 Sept}
  • [open] persistent bubbles - Step 4: Under "Sample to Evaluate", when I hover over the link icon, the info that the user typed in for Address (URL) appears in a little bubble. Nice! However, they're getting stuck after I click them. Scenario: Click one (which opens new window), close that window -- all OK. Click the next one (which opens new window), close that window -- now the first bubble is visible and I cannot get rid of it, even by clicking on other fields. As I open other pages, those bubbles also get stuck. The only way I was able to get rid of them was to leave the page and come back. {Shawn, 10-Sept}
    • Added as issue #139 to Github, good catch. {Wilco, 15 Sept}
    • Issue comes from coding library. Contacted developers. Will be fixed in future versions. {Wilco, 16 Dec}
  • [done] Save & Open not working? also: fill-in filename. (I switched to Firefox 31 so as not to find any more Opera 12 issues :-) User action & results:
    • Enter data
    • Click "Save Report"
    • Click Download and save button
    • Save File
    • Click Back link
    • [all looks OK]
    • Close browser tab
    • Get message "This page is asking you to confirm that you want to leave - data you have entered may not be saved. [Leave Page][Stay on Page]
    • [think, hmmm, maybe the Save didn't work. I'll try again]
    • Click "Save Report"
    • Click Download and save button
    • [darn, it comes up with the default filename placeholder again instead of the name that I just typed in. well, that's going to be annoying to retype my filename every time I save it...]
    • Save File
    • Click Back link
    • Close browser tab
    • Get message "This page is asking you to confirm that you want to leave - data you have entered may not be saved. [Leave Page][Stay on Page]
    • [but I didn't make any changes after I saved it. OK, I'll try not using the Back link...]
    • Click Stay on Page
    • Click "Save Report"
    • Click Download and save button
    • [retype my filename again and get a little annoyed at that.]
    • Save File
    • Close browser tab
    • Get message "This page is asking you to confirm that you want to leave - data you have entered may not be saved. [Leave Page][Stay on Page]
    • [urg! well, I guess this is an infinite loop, I'll just close it.]
    • Click Leave Page
    • In browser tab, open http://w3c.github.io/wcag-em-report-tool/dist/#/
    • Click Open Report
    • [get distracted... come back and without thinking,]
    • Click Load data file button
    • Get message "Argument 1 of FileReader.readAsText is not an object."
      [hum, that's not very clear at all -- looks like a default message that someone forgot to customize. but I see that I forgot to select the file,]
    • Click Browse... button
    • find file, click Open
    • Click Load data file button
    • [1. Define Scope page displays]
    • [it's empty, my data is not there :-(
         I tried this a few different times and a few different ways (which is why this testing is taking me so long, also to write it up) and was not able to get my saved data at all. {Shawn, 4-Sept}
    
    • (1) The "you may lose data" warning will not appear after save, unless data is changed (issue #143), (2) there is a warning for browsers that don't support save/load )issue 122), (3) there is a better default filename now (issue #144), (4) there is more error catching and error message customization {Shadi, 5 Oct}
  • [done] Level A, AA, AAA filtering - below 1 is Define Scope page and 4 is Audit Sample page
    User selection in 1: Conformance target Level A. Results in SC shown in 4: Levels A & AA [shouldn't this be just A?]
    User selection in 1: Conformance target Level AA. Results in SC shown in 4: Levels A & AA
    User selection in 1: Conformance target Level AAA. Results in SC shown in 4: Levels A & AA, & AAA
    {Shawn, 4-Sept}
  • [done] Save Report is not working. (Thus I was not able to test what happens when I open a saved report. :-) {Shawn, 3-Sept}
    • Older browsers may also experience problems with this. Unfortunately WeRT requires relatively new technologies. Currently the latest versions of IE, Firefox and Chrome have been tested. Edit: Also seems to work in Opera.(Wilco 4 sept)
    • When I click Save Report, I get the following. This is with Opera 12.16 on Windows Vista. (Is that considered an older and unsupported browser?) {Shawn, 4-Sept}
         {{ setTitle( translate('HD_SAVE') ) }}
         {{ 'INTRO_SAVE' | translate }}
         {{ 'BTN_DOWNLOAD_DATA_FILE' | translate }}
         {{'BTN_BACK_TO_EVAL' | translate }}
      
      • I've added a few checks to prevent WeRT from crashing when certain features aren't supported. Saving still won't work but it shouldn't crash the page. {Wilco, 15 Sept}
      • I think the tool doesn't work at all in the browser that Helle has on her work computer (and she's not allowed to upgrade), and it seems not to work at all in IE 8. I guess we'll need a statement somewhere about accessibility-support. :) {Shawn, 4-Sept}
      • This is less about accessibility-support than about backward compatibility. Compatibility testing will be implemented to inform the user, at the first page, which parts of the tool, if any, may or may not work with their browser. {Shadi, 12 Sept}
      • I've added a test that checks for key features. If the browser can't be supported the user will be shown a warning. (issue #122) {Wilco, 15 Sept}

Text

  • [open] font size — needs to be bigger. see WAI website & Tutorials {Shawn, 15-August-2014}
    +1 to larger font size {Jan, 22-August-2014}
    • Changing font-size from em to px (issue #156) {Shadi, 5 Oct 2014}
    • Did some improvements but changing to em requires changing the code library. Contacted developers but suggest to keep for now {Shadi, 16 Jan 2015}
    • EO agreed to keep current setting for version 1.0 but keep the issue open for future improvements (EO minutes of 23 January) {Shadi, 6 Feb 2015}
  • [done] borders around text boxes — need to be darker. It looks like the Twitter Bootstrap UI Template was used, perhaps? Consider changing the border around the text boxes to Hex #555555. {Jan, 21-August-2014}
    • (I undid the "done" on this one.) I think we talked about having a light gray background on the main page and the text boxes have a white background. In the 3 September version, the text fields themselves have a kinda of 3D rounded-corner border that to me makes them look more like buttons than text input fields. This may be a difference between common Windows UIs vs. iOS UIs, but to a Windows user, it doesn't work well. I'm used to forms with gray backgrounds and white fields, with almost square borders... but maybe I'm also showing my age. :-( {Shawn}
    • Fixed (issue #156) {Shadi, 5 Oct 2014}
  • [done] Levels styling - On Step 4 and later pages, the levels (e.g., "Level A") look like flat buttons, but they're static text — and they really draw my visual focus. I'm not sure that we want them to stand out so much (if at all), and instead make them the same font as the preceding text. 7 August e-mail}
  • [done] "web page states" – I think link the definition the first time it is used. Then, only use it occasionally where it is particularly relevant, and deleted it elsewhere. Yes, it's needed in the technical document, but in this tool, I think it adds complication and is not needed most places. {Shawn, 3-Sept}
    • Agreed. {Shadi, 12 Sept}
  • [done] Add to the introductory paragraphs of each page a link to the relevant WCAG-EM section when there is additional useful info there. (e.g., I noticed this for the top of Select Sample and think it would be good to link to WCAG-EM Step3) {Shawn, 3-Sept}
    • Agreed. {Shadi, 12 Sept}
  • [done] "Start a new evaluation report" -> can be simplified to: "Start a new report" {Shawn, 3-Sept}
    • Agreed to rename this to "Step 1: Define Scope", in consistency with all other forwards/back buttons. See Minutes of 5 Sept. {Shadi, 12 Sept}
  • [done] Start — The first page is labeled "Start" in the navigation bar along the top. The button at the bottom that takes you to the next page (which is "1. Define Scope") is labeled "Start a new eval report"." Potentially confusing — although minor, I think worth changing. Perhaps change the "Start" in the navigation bar to... ? {I'll try to add some brainstorms later}.
    Also, on <http://w3c.github.io/wcag-em-report-tool/dist/#/audit/scope>, the Previous button is labeled "Start" but that doesn't really make sense, 'cause that's not really the Start of a new report, that's the Introduction to the tool... {Shawn, 28-August-2014}
    • Agreed to rename button from "Start ..." to "Step 1: Define Scope", in consistency with all other forwards/back buttons. See Minutes of 5 Sept. {Shadi, 12 Sept}
  • [done] Under Key Resources, do we want to list the proper full name of WCAG-EM, i.e., change "WCAG Evaluation Methodology (WCAG-EM)"." To "Website Accessibility Conformance Evaluation Methodology (WCAG-EM)" ? {Shawn, 3-Sept}
  • [done] Under Key Resources, do we want to further explain and tie into the commonly-used-at-least-among-some "quick ref" by changing "How to Meet WCAG 2.0" to "How to Meet WCAG 2.0: A customizable quick reference" ? {Shawn, 3-Sept}
  • [done] Under Key Resources, the WCAG link goes to the TR doc. Do we want to add a link above that to the WCAG Overview page <http://www.w3.org/WAI/intro/wcag> ?
  • [done] Whatever we decide for the links above, probably should also be changed at the bottom of the Report as well. (although there you have more room so might want to write out even more){Shawn, 10-Sept}
    • Agreed, though want to keep the same naming for consistency and recognition. {Shadi, 12 Sept}
  • [done] For the Previous & Next buttons at the bottom, put the same words as in the nav bar at the top – e.g., change from just" Sample" to "Select Sample" {Shawn, 3-Sept}
    • Agreed. {Shadi, 12 Sept}
  • [done] "homepage" -> "home page"; "sitemap" -> "site map" {Shawn, 3-Sept}
    • Agreed. {Shadi, 12 Sept}

Functionality/UI

  • [open] Save functionality (high priority) — As a user, I want to Save this very often (since I'm entering the results from a lot of work and I don't want to lose it) -- I actually kept doing Ctrl+S as I was going (it's an automatic thing I do when I'm working) -- of course that doesn't work. :-/
    Currently do I have to scroll up to the top to click the Save Report link that's in the middle of a light gray line of links? Can we improve on that? Perhaps maybe just an idea is to have a floating save button that's always visible?
    (A related but different point: Do we need to explain in Help or the Start page that the browser Save functionality does not save the data?) {Shawn, 28-August-2014}
    • Floating save would require considerable amount of re-design, though we will consider for future versions (issue #131). For now back-to-top links have been added (issue #153), and Ctrl+s will be implemented (issue #145) {Shadi, 5 Oct 2014}
    • Ctrl+s has been implemented. EO agreed to keep as-is for version 1.0 but keep the issue open for future improvements to the top menu bar (EO minutes of 23 January) {Shadi, 6 Feb 2015}
  • [open] top bar (medium-low) - I don't think it makes sense to have New Report and Open Report visible on all pages. "New Report" doesn't do anything really -- it just takes you to the Start page -- so why have it at all? I think rarely would users want to Open an Existing Report from anywhere other than the Start page -- so how about just have it on that page.
    That leaves 2 things: Save Report & Key Resources. I wonder how often people will use Key Resources from other than the first page? I'm guessing seldom. If they want info from WCAG-EM they probably already displayed the (i) flyout and will follow the link there that will take them right to the relevant place. For WCAG, I'm *guessing* most either don't need to look at WCAG as they go through the pages (e.g., they already have the review done and are just entering data), or they leave WCAG open all the time in a separate window.
    Soooo, maybe that just leaves the Save? Then I wonder if you can put Save somewhere else that takes up less space -- (per EOWG telecon 12 Sept)?
    [If you decide to keep all those links up there, [high] for me would be to put Save Report either first or last so it's the easiest to click, since some of us will use it often (and will use the others links rarely if ever). {Shawn, 28-August-2014}
    • Not sure that we agree with all assumptions, in particular the use of "key resources" and leading with "save" on the start page. These changes also introduce consistency issues, and they would require considerable amount of re-design. For now, Ctrl+s will be implemented (issue #145) and we plan improvements to the menu bar in future versions (issue #131). {Shadi, 5 Oct 2014}
    • Ctrl+s has been implemented. EO agreed to keep as-is for version 1.0 but keep the issue open for future improvements to the top menu bar (EO minutes of 23 January) {Shadi, 6 Feb 2015}
  • [done] button & link design affordance mixed up [medium] - the style of blue background & white text is used for totally different things: highlight, actions, and heading {Shawn 10 Sept}
    • In nav bar at the top, the current page is indicated with a blue background & white text. It [should] not be clickable. {Shawn 10 Sept}
    • In the first page, the main action (Start a new evaluation report" is a buttony-looking link with a blue background & white text. Also, on most of the pages, the Next action is a buttony-looking link with a blue background & white text. {Shawn 10 Sept}
    • In the Save Evaluation Report page, the main action (Download and save) is a buttony-looking link with a blue background & white text {Shawn 10 Sept}
    • (however, I actually didn't stumble up on this until I got to the next one, where I did get confused and try to click the wrong thing a few times:) {Shawn 10 Sept}
    • In the View Report page, "Download and customize this report" is blue background & white text, so I kept trying to click it. (I processed it as a box with 3 options and the first one was highlighted) {Shawn 10 Sept}
    Fixed (issue #146) {Shadi 5 Oct}
  • [done] nav bar current page no link [low] - In the nav bar at the top, the current page should not be a link and should not have an underline. {Shawn 10 Sept}
  • [done] fields with default text [medium-low] - In some places, these provide info that is not available elsewhere, e.g., 4. "Observations made during evaluation". That's why I wasn't keen on it disappearing when the text box gets focus. Currently, once I start to type then it is deleted. I think that's good. However, I still stumbled with the current implementation. When I got to the field, I pressed Delete but nothing happened. I did Ctrl+A, nothing. I also tried to highlight it (first with keyboard then with mouse) and I couldn't. (All these do work after I enter text into the field.)
    I'm not sure what is the best solution. Possibly to go back to just having it disappear with focus? As Wilco mentioned, that is a common UI -- and it seems that default text is used less so it's not as important in most places (but maybe not?). {Shawn 10 Sept}
    • (1) Will ensure that all placeholder text is adequately reflected in the info fields. (2) Making placeholder text disappear on focus rather than when text is typed will be considered for future versions (issue #170). {Shadi 5 Oct}
  • [done] design proximity headings/sections [low-medium] - On pages with multiple sections, I would find it easier if the section distinctions were clearer, at least with better spacing for proximity relationship (and possibly with lines, although lines may be too cluttering).
    Just one example: see <http://lists.w3.org/Archives/Public/wai-eo-editors/2014Sep/0003.html> -- the first is a screen capture of the current design. If you squint (a common UI design eval technique :), you see there is lots of what space above Short name, which visually separates it from the info above -- and there is little space above "Randomly Selected Sample" so it looks like it belongs with the fields above. In the second image, I've tweaked the spacing so there is more space above "Randomly Selected Sample" and less space under the text, so there is better relationship fro proximity. :-) {Shawn 10 Sept}
  • [done] New Report – When I click New Report in the nav across the top, I get the main page and then I have to still click "Start a new evaluation report" (which is at the opposite corner of the page. I don't think this is optimum. {@@ ideas for how else to do it?} {Shawn, 3-Sept}
    • Each report will have a report title reflected on the first page to distinguish the different tabs/windows (issue #123) {Shadi 5 Oct}
  • [done] When I add a new line – e.g., a new web page – how about putting the focus in the first field? (otherwise keyboard only have to do 3 backwards tabs) {Shawn, 3-Sept}
  • [done] In smaller-width window, the field & labels get messed up, and additional text appears (Web page 1", "Web page 2") screen grabs sent to wai-eo-editors {Shawn, 3-Sept}
    • Resolved this issue (issue #121). {Wilco, 15 Sept}
button placement
  • [done] Steps 2 and on have previous & next links (buttons) at the bottom of the pages and currently positioned at LHS & RHS respectively. UXMovement.com suggests placing them adjacent to each other with good rationale (see section "Place the buttons in the corners, or keep them together?") {Andrew, 16-August-2014}
    • this article talks about dialog boxes, where it seems quite custom to place buttons adjacent to each other. form-based applications with "forwards" and "back" buttons typically have these in the corners. {Shadi, 21-August-2014}
    • Sharron's button research e-mail 21 August
    • Along same lines as Sharron's e-mail, 21 August 2014, Anna Belle
    • Look at slides 6 & 7 for an example of button positioning that failed with users, 21 August 2014, Jan
    • (fyi, I consider Caroline Jarrett to be an expert in Forms design. I'll ask her about these buttons...~Shawn)
    • Anna Belle's 26 August e-mail (follow the "Next in Thread" to see more)
    • WAI IG thread by Andrew -- follow the "Next in Thread" to see replies
    • yeah! Caroline shared perspectives from her years of research & UI experience, particularly with forms design. Since she couldn't edit the wiki, she replied to me via e-mail. {Shawn 28-August-2014}
      • Re Sharron's e-mail: Caroline clarified her position from the UXmatters article:

        "put the primary action button, Next or OK or Send, as close as possible to the left-hand end of the last field on the form"

        (She also pointed out flaws in the UXMovement article Andrew cites and the Mick Couper article, which Caroline addresses in the comments and in her button presentation.)
      • Re progress indicators & summary menus:

        "For many years, I've been trying to explain to people that progress indicators aren't quite as useful as they imagine. I've recently been getting a bit of understanding of this point, and some actual data to support it, from my colleagues at the Government Digital Service. Most people never look at the progress indicator until they have become worried about how far they have progressed in the form.
        IF the form is simple and easy to navigate, then a progress indicator doesn't do any harm. It's a decorative feature on the page which can seem pleasant enough. You can leave it on the page or take it off - won't make any difference. My colleague Ben Holliday reports on this here: https://designnotes.blog.gov.uk/2014/07/07/do-less-problems-as-shared-spaces / It's quite a lot of developer and designer work to create a progress indicator, keep it up to date, and make it work properly - rather too much, in my view, when you could design a nice logo or some other reassuring message in far less time and have approximately the same effect.
        IF the form is complex, then the progress indicator may do some harm and may actually be a black-hat pattern. It's difficult to know what a step-by-step progress indicator is supposed to represent when a form has multiple sections that may have varying lengths depending on what the user has answered. Futhermore, complex forms often require answers to be obtained from all sorts of places and possibly from different users; forcing a step-by-step approach onto the user can be deeply frustrating for them, especially if the form doesn't allow the user to look ahead (very few do) or has poor save-and-resume features.
        I have been trying very hard to get someone to try some experiments on this with forms but so far no success. It has been extensively tested in experiments on long surveys by Mick Couper, who showed that progess indicators *increase* drop outs for all cases apart from when they lie, that is when the progress indicator tells the user that the progress so far has been better than it actually has. As we can't condone a black-hat pattern, that rules out progress indicators for surveys.
        In my book (and in many other forums and presentations), I've explained that a summary menu works far better for a complex form. A summary menu is a page that lists all the sections of the form, and that allows users to choose which of those sections to tackle; it also allows a user to move back out of a section of the form (leaving it incomplete) and tackle another section that may be more convenient. If they choose to do that, then the menu updates itself with a status showing which sections are complete, which ones are partially complete but need more work, and which ones haven't yet been tackled at all. It's easier to implement than a progress indicator and it's not ambiguous; it's also responsive to sections that vary in length according to the user's answers or that may prove not to be relevant based on previous answers.
        The UK tax return works on this basis - not coincidentally, it's a feature that I designed for the paper form and that got implemented in the online version, luckily. I have also seen successful implementations on a complex form to do with financial regulation (based on my advice) and on a job application form (that I had nothing to do with).
        But: the industry seems to be in love with progress indicators. I don't know why. Even people who I've heard repeatedly describing me as deeply expert on forms design just tell me that I'm wrong on this one: no amount of evidence, research, or data appears to be able to shake them in their view that a progress indicator is a good thing. They believe that they use progress indicators and they believe that therefore progress indicators are valuable. They have rarely or never encountered a summary menu so they believe that summary menus aren't very good. I expect that you and your team will be equally sceptical.

        I don't know enough about your form to be able to tell whether it counts as complex or simple: that will require some investigation.

      • Conclusion:

        My recommendation to you:
        - create a left margin on the form and put the 'previous' button in it
        - put the 'next' button where it belongs, immediately under the left-hand end of the last field on each page
        - find out whether your typical users can complete the form in one sitting, without needing to break off to find other information or to consult other people. If so, your progress indicator won't do any harm so don't worry about it. If users sometimes or often can't complete the form in one sitting, then redesign it to have a summary menu.

      • Shawn replied in e-mail: "What we have now is not really a progress indicator (I incorrectly called it a progress bar). Instead, it's *navigation arranged horizontally*. Users can go to any page at any time -- they don't have to go sequentially or finish one page before going to another. Also, at least in the first version, we won't have any indication of "progress"/pages completed (partly because we expect that often users will go back and add to or change data on previous "pages"). While in some cases users might complete the whole thing in one sitting, we expect often users will save and come back. We also expect users to often jump around between the pages.
        Do you have any examples of summary menus that we can see?"
    • Based on all the input and also playing around with it with my user hat on, I now suggest that we have at the bottom a "Next" button left-aligned, no Previous button, and a Back to top link to get them to the overall actions (including Save Report) and the page navigation bar. {Shawn 28-August-2014}
    • Whichever placement is decided, the button's text is equally important. "Next step >" lacks semantic meaning (i.e. - what's next?) and reinforces the idea of a linear progression suggested by the numeric steps in the navigation bar at the top of the page. Instead, button text should read "Next: Select Sample >" The buttons will be different sizes which may offend design sensibilities, but the semantic meaning is more precise and provides a good sense of where you are in the app without the trouble and expense of a progress bar. I'd go with a single left-aligned button beneath the last input field. Much simpler. {Paul 28-August-2014}
    • Comment on 3 September version: The line above the buttons has opposite effect, imho – it separates the buttons from the form fields and — and it does nothing to connect the buttons to each other and signal to screen magnification users that there is another button off screen on the right. I was thinking of something like the pale blue bar on the line with the right-aligned back to page contents under http://www.w3.org/WAI/intro/people-use-web/principles#captions. Another brainstorm (that might or might not work well): Have a line connecting the two buttons? {Shawn 3-September-2014}
    • Adding WAI branding, in particular the broader and brighter horizontal rule before the footer (under the buttons) seems to have reduced the emphasis on the horizontal rule above. {Shadi 5 Oct 2014}

Footer

  • [done] See standard WAI footer, e.g., footer
    Please put on the first line: "Status: …"
    I'm OK if you want to leave out "[WAI Site Map] [Help with WAI Website] [Search] [Contacting WAI]" {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}
  • [done] Add e-mail address: "Please send bug reports and improvement suggestions through the GitHub Issues List or e-mail to wai-eo-editors@w3.org." {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}
  • [done] I think we can use the short copyright: "Copyright © 2014 W3C ® (MIT, ERCIM, Keio, Beihang) Usage policies apply." Like bottom of < http://www.w3.org/> (action shawn: update throughout WAI website) {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}
  • [done] "Development Team: Shadi Abou-Zahra (Project Lead, W3C WAI), Wilco Fiers (Design and development, accessibility.nl)." maybe ->
    "Development Team: Shadi Abou-Zahra, W3C WAI (Project Lead); Wilco Fiers, accessibility.nl (Design and development). Developed with the W3C WAI Education and Outreach Working Group (<http://www.w3.org/WAI/EO/>EOWG</>)." {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}

First page

  • [done] Wonder about restructuring the opening para with bullets, e.g.
    The WCAG-EM Report Tool:
    • helps you generate a report according to the Website Accessibility Conformance Evaluation Methodology (WCAG-EM).
    • does not perform any accessibility evaluation
    • provides options that allow you to record information about the website and the evaluation, including the evaluation results
    • helps you follow the steps suggested by WCAG-EM, and to generate a structured report from the input that you provide.
    None of the input is required and you can go back and forth between the steps in any sequence. Once you have finished recording the results, you can download and further customize the report to your liking.
    {Andrew, 29-August-2014}
    • Revised wording with these and other suggestions. {Shadi, 5 Oct}
  • [done] wording suggestion — Third para does not seem to adequately convey the fact that you can save the 'data' and resume later - just 'save an evaluation report' which might be interpreted as saving when finished rather than save and resume. {Andrew, 29-August-2014}
    • Revised wording with these and other suggestions. {Shadi, 5 Oct}
  • [done] wording suggestion — Suggest last sentence of open para reads "Once you have finished recording the results, you can download and further customize the report to your liking". {Andrew, 29-August-2014}
    • Revised wording with these and other suggestions. {Shadi, 5 Oct}
  • [done] wording suggestion — Perhaps "About this Tool" should say "Saving your Work" instead. The latter more likely to catch the user's attention. {Howard, 27-August-2014}
    • Revised wording with these and other suggestions. {Shadi, 5 Oct}
  • [done] wording [medium-high] – suggest rewrite as follows. (I can explain the reasoning behind the order of the info. see also Kevin's feedback e-mail. (I think we might want to edit and change the order of the info later after the tool is out for a while and fewer newbies are coming to it.)) {Shawn 11 Sept}

    WCAG-EM Report Tool

    Website Accessibility Evaluation Report Generator

    This tool helps you generate a report according to the Website Accessibility Conformance Evaluation Methodology (WCAG-EM). It does not perform any accessibility checks. It helps you follow the steps of WCAG-EM, and it generates a structured report from the input that you provide.

    Background knowledge

    This tool does not teach accessibility or evaluation. If you do not know much about web accessibility or evaluation and you want to get an initial idea of the accessibility of a web page, you can follow Easy Checks - A First Review of Web Accessibility.

    This tool is designed for experienced evaluators who know Web Content Accessibility Guidelines (WCAG) 2.0 and are at least somewhat familiar with WCAG-EM. For an introduction to WCAG-EM, see the WCAG-EM Overview.

    How it works in your browser

    • All functionality provided by this tool is now loaded and running locally in your web browser. You don't need an Internet connection beyond this point. When you close your browser window, any unsaved data would be lost.
    • This tool does not record or automatically save the information that you enter. Use the Save Report link that is at the top of all pages to save your data in a file locally on your computer. You can Save periodically as you work to avoid losing data if your browser closes.
    • You can save partially-complete reports and work on them later. Use the Open Report link at the top and find the file that you previously saved.

    Tips for using the tool

    • You can go back and forth between the steps in any order. None of the fields are required.
    • To get more information about a field and a link to the specific section of WCAG-EM, select the (i) icon next to the field label.
    • You can download the report as an HTML file and customize it.

    alternative idea for first two paragraphs:

    This tool helps you follow the steps of Website Accessibility Conformance Evaluation Methodology (WCAG-EM), and it generates a structured report from the input that you provide. (It doesn't do any evaluation for you.)

    Background knowledge

    This tool is designed for experienced evaluators who know Web Content Accessibility Guidelines (WCAG) 2.0 and are at least somewhat familiar with WCAG-EM. For an introduction to WCAG-EM, see the WCAG-EM Overview.

    If you do not know much about web accessibility or evaluation and you want to get an initial idea of the accessibility of a web page, you can follow Easy Checks - A First Review of Web Accessibility.

    • Revised wording with these and other suggestions. {Shadi, 5 Oct}
  • [done] spacing [very low] – suggest less space between the first & second line of the heading: {Shawn 11 Sept}

Step 1

Text

  • [done] "Define the overall parameters and scope of your evaluation. This step is ideally carried out in consultation with the evaluation commissioner who may or may not be the website owner, to ensure common expectations about the scope of the evaluation."->
    "Define the overall parameters and scope of the evaluation. Ideally, this is done with the person who commissioned the evaluation and/or the website owner, to ensure common expectations about the scope of the evaluation." {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}
  • [done] "Define the scope of the website, so that for each web page it is unambiguous whether it is within the scope of evaluation or not." ->
    "Define the scope of the website, so it is clear which web pages and web content is included in the evaluation."
    Also, this need examples – even reading WCAG-EM, I don't know what might go here… {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}
  • [done] "Define the web browsers, assistive technologies, and other user agents for which features provided on the website are to be accessibility supported." Simplify language. Maybe:
    "Define the web browsers, assistive technologies, and other user agents that will be accessibility supported by the website." {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}
  • [done] "Define any additional evaluation requirements agreed upon with the evaluation commissioner." -> " Define any additional evaluation requirements."
    Also, this needs examples, OR add to the link that there are example there, e.g., "For examples and more information, see WCAG-EM Step 1.d: Define Additional Evaluation Requirements." {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}

Functionality/UI

  • [open] When I select "Conformance target", it should *not* limit the SC to only those levels. (I'm not sure if it does, I just thought of that when I was on this page.) Even though my website is only required to meet Level AA, I still want to check and report on Level AAA SC. And we want to encourage that. It would be good give people the choice on how they are reported, and I guess the choice to hide them in page 4 - but I would want a note saying it's good to check them even if not required for conformance... {Shawn, 3-Sept}
    • Excellent point. WCAG-EM Even suggests evaluating beyond the conformance target. This change would be quite significant to the project though. It would also require adding a filter in step 4, as well as some kind of filter on the final report. And what should you do with untested? I think this is a great idea but I'd suggest leaving it for a later version. (Wilco 4 Sept)
    • I'm fine leaving the expanded functionality for a later version; however, let's think about what to do for this version. For example, maybe we have two fields like: "Conformance target" and "Show Success Criteria" so the user can choose to show Level AAA throughout the Tool & Report, but the Report states that the Conformance target is only AA. ? (and we'd need (i) text explaining this & how people might want to use it){Shawn, 4-Sept}
    • Will explore the possibility of an opt-in system, where people can "activate" SCs from levels beyond the conformance target (issue #126). See Minutes of 5 Sept. {Shadi, 12 Sept}
    • Designing and implementing an opt-in system is quite a challenge. There are work-around that are now documented on the start page of the tool. EO agreed to keep as-is for version 1.0 but keep the issue open for future improvements (EO minutes of 23 January) {Shadi, 6 Feb 2015}

Step 2

Functionality/UI

  • [open] Left align the labels "Short name" and "Address (URL)" with the left edges of the text boxes. – also " Web Technology" lower down & all others like this. {Shawn, 3-Sept}
    • Agreed (issue #172). {Shadi, 12 Sept}
    • re-opened - not done yet in Step 3 {Shawn 6 Feb}
  • [open] I agree that the example text is helpful in these fields. What about having it show up only in the first one, and not in all the others that I add? I think that would improve usability. :) {Shawn, 3-Sept}
    • Why is this better? Note that we plan to have placeholder text disappear when the items are focused. {Shadi, 5 Oct}
    • It would just reduce text overall and thus a little clutter. But I don't think it's a big deal at all. fyi, Currently the placeholder text does not disappear with focus, but is deleted as soon as I start typing. I'm OK with this. And OK to close this comment. {Shawn, 5 Feb}
    • This is standard browser behavior for HTML5 (ie difficult to change). Keeping open for possible future user-testing. {Shadi, 6 Feb 2015}
  • [done] Web Technology Relied Upon — What if you're using a technology not listed, such as jquery (this may expose my ignorance of the ins and outs of scripting) {Howard, 27-August-2014}
    • You can type anything in these fields. What is displayed is a pre-set of suggestions but they are not exclusive. {Shadi, 29-August-2014}
    • But that's not clear -- and it's not common drop-down behaviour. I think it's neat, but it needs to be made clear. (will send ideas with other comments). {Shawn}
    • Fixed. Due to limited spacing it now says "E.g. HTML5, CSS, DOM" {Shadi, 5 Oct}
    • Added corresponding note to the info-box as well {Shadi, 13 Dec}
  • [done] The "Web Technology" and "Specification URI" are drop-downs. Many people will assume that you have to choose and item from the list and cannot enter free-form text – because that is common drop-down behaviour. I think it's great that you have both the suggestions and allow user entry! :-) So, you need a way to indicate that. Just an idea for consideration: make the first line something like "[type a technology not listed]"? {Shawn, 3-Sept}
    • Fixed. Due to limited spacing it now says "E.g. HTML5, CSS, DOM" {Shadi, 5 Oct}
    • Added corresponding note to the info-box as well {Shadi, 13 Dec}
  • [done] Enhancement: When I select something from the Web Technology list, the Specification URI field gets automatically filled in. :-) {Shawn, 3-Sept}
    • This is a browser support issue again. Browser support is now checked at the start (#issue 122). {Shadi, 12 Sept}
  • [done] When I went through this as a user, I noted that "Common web pages" and "Other relevant web pages" have places for entering new pages but "Essential functionality of the website" and "Variety of web page types" are just text boxes without a place to enter new pages. That doesn't make sense.
    Also, the "Web technologies relied upon" seemed out of place. I know that you generally want to follow the flow of WCAG-EM, however, I wonder if this might be a place where it'd be better to have a different flow for better tool usability?
    … Later I figured out that that tool has adding pages related to "Essential functionality of the website" and "Variety of web page types" on the next step. This flow doesn't work well. As I'm thinking about the essential functionality and the different web page types, I'd like to be able to add the pages that are needed for those… (see additional comments under Step 3) {Shawn, 3-Sept}
    • Confirmed issue (#issue 148) but awaiting more real-life testing of the tool before taking action. See Minutes of 5 Sept. {Shadi, 12 Sept}
    • Moved selection widgets from step 2 to step 3, to make step 2 purely exploratory (better distinguish functionality of both pages). EO agreed to go ahead with the for version 1.0 and see if problems continue (EO minutes of 23 January) {Shadi, 6 Feb 2015}
  • [done] Step 3 comments have additional ideas – in any case, we need to make it more clear how the pages for Step 2 & 3 are related – specifically, what to enter here and what on the next page. {Shawn, 3-Sept}
    • Confirmed issue (#issue 148) but awaiting more real-life testing of the tool before taking action. See Minutes of 5 Sept. {Shadi, 12 Sept}
    • Moved selection widgets from step 2 to step 3, to make step 2 purely exploratory (better distinguish functionality of both pages). EO agreed to go ahead with the for version 1.0 and see if problems continue (EO minutes of 23 January) {Shadi, 6 Feb 2015}
  • [done] "Other relevant web pages" -> "Other web pages relevant for accessibility" {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}

Text

  • [done] Intro text: suggestion for revision:
    Explore the website to understand its purpose, functionality, and use. This steps helps you determine which web pages to use for the evaluation. You can come back and add web pages to this list later. Usually it is best to get input from the website owners and developers for this step. {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}
  • [done] (i) for Variety of web page types – I think delete "This list will be used in the following steps to help you select representative web page instances for evaluation." because it also applies to the box above. Maybe put it in always visible text before "Essential functionality of the website", or maybe the suggested revision to the intro text is enough ("This steps helps you determine which web pages to use for the evaluation."). {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}
  • [done] "Identify web pages and web page states that are relevant to people with disabilities and to accessibility of the website, if they have not already been identified as common web pages." ->
    "Identify web pages that are particularly relevant to people with disabilities and to the accessibility of the website, if you haven't already listed them above as common web pages." {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}
  • [done] "Examples of other relevant web pages include web pages explaining the accessibility features of the website, information and help on the use of the website, explanations for settings, preferences, options, and shortcuts, and contact information, directions, and support instructions." ->
    "Examples of other relevant web pages include those that explain the accessibility features of the website; instructions and help for using the website; explanations for settings, preferences, options, and shortcuts; and contact information, directions, and support instructions." OR simplify more:
    "Examples of other relevant web pages include those that explain the accessibility features of the website; help for using the website; explanations for settings, preferences, options, and shortcuts; and contact information." {Shawn, 3-Sept}
    • Fixed. {Shadi, 5 Oct}

Step 3

Functionality/UI

  • [done] See also comments on Step 2 about entering specific pages there. {Shawn, 3-Sept}
    • Confirmed issue (#issue 148) but awaiting more real-life testing of the tool before taking action. See Minutes of 5 Sept. {Shadi, 12 Sept}
    • Moved selection widgets from step 2 to step 3, to make step 2 purely exploratory (better distinguish functionality of both pages). EO agreed to go ahead with the for version 1.0 and see if problems continue (EO minutes of 23 January) {Shadi, 6 Feb 2015}
  • [done] In most cases, when users get to this page, they will have already listed many common pages and maybe a few other relevant pages. As I understand it, the purpose of this tool page is to add more pages to specifically cover the essential functionality, web page types, and web technologies relied upon. However, when I scroll down to get to the Add web page button, I've lost my information on essential functionality and web page types, which is at the top of the page.
    I think it would be better if the UI for entering specific web pages that is currently split between tool page 2. Explore Website and 3. Select Sample were combined. If you want to keep it closely tied to WCAG-EM, maybe Step 2 doesn't have users enter any specific pages, it just guides users through the general things to think about, and then all the specific pages are only entered in the tool Step 3 page?
    Then the tool Step 3 page is redesigned so it shows the user info from the previous tool page and has areas for entering new web pages under each topic (wcag-em sub-step) – i.e., Common web pages section: info previously entered, then Add new page; Essential functionality pages section: info previously entered, then Add new page; etc. {Shawn, 3-Sept}
    • Confirmed issue (#issue 148) but awaiting more real-life testing of the tool before taking action. See Minutes of 5 Sept. {Shadi, 12 Sept}
    • Moved selection widgets from step 2 to step 3, to make step 2 purely exploratory (better distinguish functionality of both pages). EO agreed to go ahead with the for version 1.0 and see if problems continue (EO minutes of 23 January) {Shadi, 6 Feb 2015}
  • [done] Also, if I'm on Step 3 and I want to add another essential functionality or web page type, the current design forces me to go back to the previous page to edit that, which is not good from the user perspective. It would be better if I could edit it from here. (It would be OK if I had to press a separate [edit] button or such to change it from a view-only field to editable.) {Shawn, 3-Sept}
    • Confirmed issue (#issue 148) but awaiting more real-life testing of the tool before taking action. See Minutes of 5 Sept. {Shadi, 12 Sept}
    • Moved selection widgets from step 2 to step 3, to make step 2 purely exploratory (better distinguish functionality of both pages). EO agreed to go ahead with the for version 1.0 and see if problems continue (EO minutes of 23 January) {Shadi, 6 Feb 2015}
  • [done] Also, this page gives me what I entered on the previous page for "Essential functionality of the website" and "Variety of web page types" but not the "Web technologies relied upon". {Shawn, 3-Sept}
    • Confirmed issue (#issue 148) but awaiting more real-life testing of the tool before taking action. See Minutes of 5 Sept. {Shadi, 12 Sept}
    • Moved selection widgets from step 2 to step 3, to make step 2 purely exploratory (better distinguish functionality of both pages). EO agreed to go ahead with the for version 1.0 and see if problems continue (EO minutes of 23 January) {Shadi, 6 Feb 2015}
  • [done] (p.s. It's pretty cool that the tool automatically generates the appropriate number of blank Randomly Selected Sample page fields! :-) {Shawn, 3-Sept}
    • Thanks. {Shadi, 12 Sept}

Text

  • [done] "In some cases, such as for small websites, it may be feasible to evaluate all web pages and web page states, which is highly recommended." ->
    "It is highly recommended to evaluate all web pages when it is feasible, such as for small websites." {Shawn, 3-Sept}
    • Agreed. {Shadi, 12 Sept}
  • [done] "Select a randomly selected sample of web pages and web page states. The number of web pages and web page states to randomly select is 10% of the structured sample selected above." ->
    "Randomly select sample web pages; select 10% of the structured sample selected above." {Shawn, 3-Sept}
    • Agreed. {Shadi, 12 Sept}
  • [done] "A structured sample of X web pages and web page states requires X randomly selected web pages and web page states" ->
    "Based on your structured sample of X web pages, chose at least X randomly selected web pages (to meet the 10% requirement in WCAG-EM)." {Shawn, 3-Sept}
    • Agreed. {Shadi, 12 Sept}

Step 4

Functionality/UI

  • [done] Results for the entire sample [medium?] — I am having problems with "Results for the entire sample:". It just doesn't feel right. For example, several pages could have failed, but the user chooses "Passed" -- maybe on accident or on purpose. Maybe I'm thinking the tool would generate the Results for the entire sample based on what I enter for the individual pages?
    (Scenario: I get through with most of my evaluation and data entry into the tool. For SC 1.1.1, all are Passed so I select Passed for Results for the entire sample... Later I find a Failed for SC 1.1.1 -- I come to this page and I have lots of individual pages selected. I scroll down to find the page and change it, then I go somewhere else... oops, I missed that I needed to also change the Results for the entire sample -- it was scrolled off my visible area...)
    Also, I don't like that it defaults to "Not checked". Take the case where I'm going page-by-page and I've done a lot of pages, but still have more to go. I can't fill select the Results for the entire sample until I've done all the pages. It is a bit disconcerting for it to say "Not checked" all the time.
    Also, I'm not sure what I should have in there in some cases where the results are mixed across pages...
    Maybe this one needs to have a blank option? (obviously it should be completed by at some point)
    If it stays as is, could we add functionality that checks the results for the individual pages and if the Results for the entire sample doesn't make sense, then it gives a warning? (e.g., if any individual are Failed, then entire sample should have Failed; if any have Not checked or Cannot tell, then entire sample should, too.; if all are Passed, the entire sample should be, too.) {Shawn 10 Sept}
    • Recorded as #issue 149 {Shadi, 13 Dec}
    • Tool now warns about inconsistency between results for entire sample and individual pages {Shadi, 6 Feb 2015}
  • [open] misc [medium]
    User task: I want to enter the results for one page.
    Action: Under "Sample to Evaluate", click the first one.
    Look at "Success Criteria to Evaluate".
    Reaction: "Results for the entire sample:" and "Show results for individual web pages" don't make sense.
    Action: Complete the data entry for the first 5 SC.
    User: Now I want to enter some results across multiple pages.
    Action: Under "Sample to Evaluate", click the next 3 pages.
    [user gets pulled away from it... a long distracting time... comes back later...]
    User thinks: OK, where am I? It looks like I've got the first 5 SC data entered for the 4 pages checked under "Sample to Evaluate".
    Problem: That's wrong. They only have data entered for the first page, not the other pages -- but that's not what the UI is showing (without "Show results for individual web pages"). {Shawn 10 Sept}
    • Are there suggestions for how to address this issue? {Shadi, 13 Dec}
    • The edited text helps some. The other is a bigger issue -- that "Results for the entire sample:" is done manually and thus can conflict with the results of the individual pages. I'm fine leaving it for Version 1. Let's leave it as an open issue for usability testing and for users to give us feedback on. {Shawn 5 Feb}
    • Tool now warns about inconsistency between results for entire sample and individual pages. Keeping this open as suggested for potential future usability testing. {Shadi, 6 Feb 2015}
  • [done] spacing/alignment [medium-low] - The large amount of space between "Results for:" and the page shortname disconnects them. My brain processed "Results for:" as the heading for the drop down box, then "Structured page 1" as the heading for the text field. (/me didn't check what is that field's label in the code) I suggest just having a single space between "Results for:" and the page shortname, rather than putting the page shortname over on the right aligned with the text box. (Also, maybe don't have it in italics since text in italics is generally harder to read and there's no need to distinguish this text.) {Shawn 10 Sept}
  • [done] order of pages listed in "Sample to Evaluate"[low-medium] — Currently they seem to be listed in the order that I added the pages, and then the random pages at the bottom. So when I went back and added pages in Step 2, they were at the bottom of the list, instead of up with the other common pages where I expected them. Do we want a more logical order, e.g., maybe to match the overall grouping with WCAG-EM:
    1. Common pages, 2. Essential functionality pages, 3. Variety of types pages, 4. Web technology pages, 5. accessibility relevant pages, 6. Random pages. ?
    or the way they are grouped in the tool:
    1. Common pages, 2. accessibility relevant pages, 3. Essential functionality pages, 4. Variety of types pages, 5. Web technology pages, 6. Random pages. {Shawn 10 Sept}
  • [done] shading [low-medium] — I thought EOWG agreed to have the text in the Results for drop-down be color coded, and not to have the overall shading. I find the shading too distracting. {Shawn 10 Sept}
  • [open] De/Select All [low] — for "Sample to Evaluate" there was a suggestion for Select All and Deselect All. Did that get missed, or decided against, or put of enhancements for later? {Shawn 10 Sept}
    • Recorded as #issue 67 but this not simple, given how complex the page already is. We may need to put this as a feature request for future versions. {Shadi, 12 Sept}
    • Not implementing in version 1.0 due to complexity but keeping the issue open for future improvements {Shadi, 6 Feb 2015}
  • [done] back to top [low] - I scrolled a lot. I wonder if back to top a few places (like to the left of each guideline?) would be useful and worth the clutter? {Shawn 10 Sept}
    • Does the expand/collapse functionality help at all? Would you keep both, the expand/collapse an back-to-top functionality? {Shadi, 12 Sept}
    • Ah, hum, in this case it didn't. :( Even though I know about it, I didn't use it. I guess because I was thinking I wanted to leave all the data there, just jump up to the top and select another page. Anyway, I marked this as low so maybe skip it for now and let's see if we can check in more usability testing later what would be useful for other users? {Shawn, 12 Sept}
    • Added "back to top" throughout {Shadi, 13 Dec}
  • [open] long observations [very low] - I like that the "observations" boxes expand on focus then collapse. It would be nice if when they are collapsed, there is an indication that there is more text than is visible -- e.g., "...". {Shawn 10 Sept}
    • Recorded as #issue 154 and #issue 159 {Shadi, 13 Dec}
    • Not implementing in version 1.0 but keeping it open for future improvements. {Shadi, 6 Feb 2015}
  • [done] Mark completed pages [enhancement for later] - provide a way for the user to indicate (for themselves) that they are done with a page, e.g., an icon toggle in the list under "Sample to Evaluate" {Shawn 10 Sept}
    • Recorded as #issue 63 for future versions. {Shadi, 12 Sept}

Text

@@SLH

Step 5

Functionality/UI

  • [done] date format [high] - The month needs to be written out ('cause 2014-01-02 is January 2nd in some places and February 1st in other places). I would prefer the date first: 00 Month 2000 {Shawn 10 Sept}
  • [done] scroll to bottom [medium-low] - They can't change anything under "Detailed audit results" and maybe won't even review it sometimes, so make it easy to skip it. Possibly have an additional "Next step: View Report" button before "Detailed audit results" -- and maybe brief text explaining there's nothing below to change? {Shawn 10 Sept}
    • Recorded as #issue 184 {Shadi, 13 Dec}
    • Now expand/collapse heading to skip over the section easily {Shadi, 6 Feb 2015}

Text

@@SLH

View Report

Functionality/UI

  • [open] Findings when empty [low-medium] - instead of "- No text provided -", I suggest just leaving it blank to reduce clutter. {Shawn 10 Sept}
    • I had it blank originally, but that didn't look right. It leaves big holes in the report for unexplained reasons. {Wilco 15 Sept}
    • Recorded as #issue 188 {Shadi, 13 Dec}
    • Keeping as-is for version 1.0 but keeping the issue open for possible usability testing {Shadi, 6 Feb 2015}
  • [done] alignment [very low] - Personally I don't like the tables under "Scope of the evaluation" and "Overview of audit results" centered and would prefer them left aligned. {Shawn 11 Sept}
    • It looks like to me they have been left aligned so this is closed? {Shawn 5 Feb}
  • [done - as long as you don't change that :-] I *really* like that the final report lists all SC, including AAA, even if the conformance target is less! {Shawn 10 Sept}
    • This is actually a bug. It shows the AAA requirements if you've selected AAA in Step 1 (even if you change that back again to AA). This relates to the [#displayedSCs issue on displaying SCs]. Only the ones evaluated (including opt-in SCs from a higher conformance level) will be displayed in the final report. {Shadi, 12 Sept}
    • This bug has been fixed (sorry Shawn, but it really was a bug!). We can figure out a way to make the AAA data available, but as discussed, this is something that will take more time to develop properly. {Wilco, 15 Sept}
    • EO agreed to postpone the implementation of an opt-in system till after version 1.0 (EO minutes of 23 January) {Shadi, 6 Feb 2015}

Text

  • [open] page title (medium-low) - The last page is for previewing and saving the report, thus "View Report" doesn't seem quite right. {Shawn 10 Sept}
    • Probably it would have been good to ask others in EOWG what they think of this. However, I don't think it's a show-stopper for Version 1. Please let's keep it as a minor open issue. It's something we could check in usability testing, and have on a list of things that we'd like user feedback on.{Shawn 5 Feb}

@@SLH

Closed comments (archived)

  • [done] examples — The link to WCAG-EM is very helpful but I think more examples would help further. {Howard, 27-August-2014}
  • [done] title — I think we want the title and h1 the same. See Title ideas below. {Shawn, 15-August-2014}
  • [done] links — Put links at the end of the sentences whenever feasible (per EOWG previous discussions on improving readability) e.g.:
    "Refer to WCAG-EM Step 1.a: Define the Scope of the Website for more information." ->
    "For more information, see WCAG-EM Step 1.a: Define the Scope of the Website."
  • [done] punctuation — add comma in "Once you are done providing input, you can download and further customize the report to your liking." {Shawn, 15-August-2014}
  • [done] capitalization — internet -> Internet {Shawn, 15-August-2014}
  • [done] wording fix — evaluating accessibility using WCAG 2.0 (not "using the WCAG") "evaluating accessibility using the Web Content Accessibility Guidelines (WCAG) 2.0" -> "evaluating accessibility using Web Content Accessibility Guidelines (WCAG) 2.0" {Shawn, 15-August-2014}
  • [done] Typo on start page — Change "Also, efore evaluating" to "Also, before evaluating" {Anna Belle, 14-August-2014}
  • [done] In the footer — There needs to be a space between "Member" and "privacy." {Jan, 21-August-2014}
  • [done] Additional Evaluation Requirements Information Box — On the "Define Scope" chevron, in the "Additional Evaluation Requirements Information Box, add "upon" after "...additional evaluation agreed" {Jan, 21-August-2014}
  • [done] Explore Website — The first sentence is a bit difficult to read. Possible edit: "To develop an initial understanding of its purpose, functionality, and use, explore the website to be evaluated." {Jan, 21-August-2014}
  • [done] Common Web Pages Info Box — The second sentence is a very long sentence with too much punctuation. Consider breaking it up as follows: "... This includes the homepage, login page, and other entry pages. Where applicable, it also includes the sitemap, contacts page, site help, legal information, and similar web pages that are typically linked from places such as the header, footer, or navigation menu, located on every page in the site. {Jan, 21-August-2014}
  • [done] Report Findings — In the "Evaluation Commissioner" and "Evaluator" information boxes, consider changing the first word from "State" to "List" ... in the "Executive Summary" information box, consider changing the second sentence to read as follows: "For example, describe the overall accessibility of the website and provide key observations, such as frequently occurring issues and patterns."{Jan, 22-August-2014}
  • [done] wording fix — "The tool does not perform any accessibility evaluation." "Any" implies a plural to follow. Change "any" to "an" or "evaluation" to "evaluations" {Howard, 27-August-2014}
    • Changed to "checking" {Shadi, 29-August-2014}
  • [done] wording suggestion — In "Also, before evaluating an entire website it is usually good to do a preliminary evaluation of different web pages" change "do" to "conduct" {Howard, 27-August-2014}
    • Changed to "carry out" {Shadi, 29-August-2014}
  • [done] visible focus indicator — In Chrome, the visible focus indicator is Hex #66AFE9 which fails contrast requirements. Consider changing the focus indicator to a thicker blue line, Hex #477BA5, with a halo so that it is consistent across browsers and more perceivable. {Jan, 21-August-2014}
  • [done] text color in information boxes — Should the text color in the blue information boxes be #333333 and the links be #31708F, instead of the other way around? With blue text on a blue background, even though it passes contrast, it seems kind of hard to read. Plus, does "blue" kind of mean "clickable" to end users? {Jan, 21-August-2014}
  • [done] Explore Website — True confessions - I have not read the WCAG-EM, which may be the reason I am confused by the words "Handle" and "Web Page." If I choose to "Add a web page or web page state," I don't know what is meant by "Handle" at all and I wondered whether I needed to put a URL or just a web page title in the "Web Page" field. If these are terms from WCAG-EM, it might help to add links to their definitions. Also, the "x" in the button to the right of the "web page field" did not look like an "x" to me at first. It looked like an asterisk or a fan." I had to click on it before I realized that it was an "x" and that it was intended to close or delete my Handle and Web Page that I added. {Jan, 21-August-2014}
  • [done] Explore Website - Web Technology Relied Upon — The drop-down menu button should be shorted to just read "Name of the Web Technology," rather than "Name of the Web Technology that is re" {Jan, 21-August-2014}
  • [done] Select Sample — There appears to be a different style of text box on this page than other pages. The box beneath the phrase, "Essential functionality of the website" appears to be a text box, but I cannot click inside of it to write anything. I clicked on the phrase above it, thinking it was a link to information, but it just closed the drop-down and the box disappeared. The boxes underneath "Essential functionality of the website" and "Variety of web page types" are too faint - I could barely see them at all. As I scrolled down further, I saw that these boxes are probably there for tips. There is a tip listed under "Randomly Selected Sample." Again, the box is too faint - consider darkening the border around the box. {Jan, 22-August-2014}
  • [done] Report Findings — For the "Evaluation Date" field, either consider reducing the width of the field, or bring the date controls closer to the actual date. Also, the information box associated with this field states, "Provide the completion date or duration period for carrying out this evaluation" but it is not clear how to provide a duration period — it appears that you can only provide a single date. You might also consider making the "Generate the sample report" button available at both the top and the bottom of this page. With all of the success criteria listed on this page, there is a lot of scrolling that has to be done to get to that button at the bottom.{Jan, 22-August-2014}

[Done] Title

brainstorms for title:

  • WCAG Reporter {Wilco}
  • WCAG-EM Report Tool {Shawn}

+1 WCAG-EM Report Tool {Jan, 21, August 2014}

+1 {Sharron, 28 August 2014}

  • WCAG Audit Reporter {Kevin White, 19-Jun-2014}
  • ...