Progress Updates
Silver Update for 4th Quarter 2017
Executive Summary
- Participation in Silver Task Force and Silver Community Group has been growing. There are currently 71 members in the Silver Community Group.
- Literature Review project is writing its findings. The first paper will be on Conformance.
- Legacy Interview project and the Conformance Interview project are in process.
- The survey of the usability of the individual WCAG 2.0 success criteria has the data complete and analysis is in process.
- Audience feedback from 3 Silver presentations is in the process of data analysis.
- A Conformance Survey is in process.
- Four Research Partners have concluded their work.
- Job Stories have been written for each of the 31 Silver stakeholder roles. These will be used to evaluate Silver designs.
- Problem Statements have been written for three key areas: Conformance, Usability, and Maintenance. These will be used for creating solutions at the Design Sprint.
- A Design Sprint has been scheduled for March 19 & 20 at San Diego State University. Experts in different areas have been invited. A expert moderator will be leading the Design Sprint. The purpose of the Design Sprint is to develop prototypes of Silver Designs based on the research.
Growth in Participation
Participation in Silver has increased from around 50 participants in September 2017 to 71 participants as of 9 February 2018. Participation in task force meetings continues to increase with the majority of participants coming from the Community Group. We also had an outreach effort to bring in new participation. Participation continues to increase based on the presentations from November 2017.
Research Status
Although this is a report of the 4th Quarter, the research information is up to date as of 9 February 2018.
Literature Review
The Literature Review has been used to inform work on the problem statements. The Literature Review results are being compiled as a paper for the Design Sprint. The Conformance section of the Literature is scheduled to be complete by 1 March 2018.
Interviews
There are two interview projects being run in parallel. Both projects share the same participation consent document. The interviews are being recorded and transcripts created of the recordings. The Silver task force decided to conduct confidential interviews to encourage participants to speak candidly, and to give the participants control of any quotations that would be used publicly. Each interview lasts 30-60 minutes. There are two people from Silver working on each interview: one main interviewer and a note-taker who also can ask follow-up questions. Once the interviews are completed, there will be data analysis based on the transcripts. The data analysis for both projects is scheduled to be completed on 1 March 2018.
The Legacy project seeks to capture some of the experience of the people involved in developing W3C accessibility guidelines including WCAG, ATAG, and UAAG. There are currently 6 participants scheduled for interviews. Each participant is asked the same base set of questions, and the interviewer may ask follow-up questions:
- Tell me about your role in developing [WCAG 2, UAAG, or ATAG]
- Tell me about the “no brainers” and why they were straightforward to resolve.
- Tell me about the most difficult issue you had to deal with.
- Tell me about the solutions to difficult issues that worked well in hindsight.
- Tell me about the solutions to difficult issues that you regret in hindsight.
- Tell me about the resolutions (if any) where you disagreed with the decision of the working group.
- Tell me about any outcomes from the published standard that were unexpected.
- What might we do differently in developing Silver?
The Conformance project is run similarly to the Legacy project, with different questions and different participants. The six participants currently scheduled were selected based on their knowledge of different aspects of WCAG conformance and have either legal/policy expertise or disability advocate expertise. The base set of questions for the Conformance Interview project are:
- How well does the current conformance model for W3C Accessibility Guidelines serve accessibility for people with disabilities short-term and long-term?
- How can we improve on the conformance model?
- Tell me about what is needed (around conformance) to serve people with disabilities?
- Tell me about what is needed to serve policymaker needs?
- Tell me about what is needed to serve WCAG users (developers, designers, evaluators)?
- Where do we set the boundaries of what to cover in accessibility guidelines? (Web content, native, platforms, hardware, where do you stop?)
- What platforms should we include in the accessibility guidelines?
WCAG Usability Survey
The WCAG Usability Survey studies the usability of individual success criteria and compares them to others. For each success criterion (SC), survey respondents were asked if the SC was easy to learn, easy to remember, and easy to teach others. Each success criterion had a optional comment field for open responses. 133 valid survey responses were received. Data analysis will be completed by 1 March 2018.
Audience Feedback
Three presentations on Reimagining Accessibility Guidelines were given to audiences in October-November 2017 with about 30-50 people attending each session: Bay Area Accessibility Meetup (San Francisco); A11yTO Camp (Toronto) and a W3C TPAC conference breakout session. Each group was given a 15 minute presentation on Silver, and then were asked the same questions. The presentation slide deck shows the introductory material and the questions asked of all the audiences. The questions asked were:
- Where do you go when you need a reference on accessibility? Why?
- What works well for you around accessibility guidelines?
- What doesn't work well for you around accessibility guidelines?
- What's on your wishlist? (around accessibility guidelines)
Notes were taken of all the answers. There was no recording or transcript. The Bay Area Accessibility Meetup was a large professional audience of mostly developers and designers. A11yTO Camp was a more diverse audience that also included policy and disability advocates, and novices to accessibility. The W3C TPAC session drew a more technical crowd experienced in standards including user agent and authoring tool developers. Data analysis of the responses is scheduled to be complete by 1 March 2018.
Conformance Survey
The Conformance survey focuses on the 5 conformance requirements of WCAG 2.0. The purpose of this survey is to determine how effectively the structure of W3C accessibility guidelines help meet needs of both people with disabilities and people that use W3C accessibility guidelines. There are additional questions rating the effectiveness of WCAG 2.0 as applied to web sites, dynamic web applications, native mobile apps, electronic documents, and kiosk/public access terminal based resources. There are questions on legal and regulatory impact of WCAG, and on declarations of conformance. The survey has not been sent out, but will be sent to 70 people.
Research Partner updates
Silver has partnered with academic and corporate researchers who have agreed to share their research findings on questions that the Silver Task Force identified.
- Scott Hollier (Curtin University) produced a paper on “Internet of Things (IoT) Education: Implications for Students with Disabilities”. Status: complete
- Tyson McMillen (Tarrant County College) is researching community college teacher perspectives on accessibility standards. Status: ongoing
- Peter McNally (Bentley University) asked UX professionals about the usability of WCAG. Status: complete
- Mike Crabb (Robert Gordon University) asked his undergraduate honors class to select a topic from the Silver research questions list. Nine of the top graded papers have been forwarded to the Silver task force. They are being reviewed for validity before being included in the Silver research. However, caution must be applied to all results due to low sample sizes present within the work. The results are interesting, nonetheless. Status: papers are complete, analysis by Silver members is in process.
- Eleanor T. Loiacono (Worcester Polytechnic Institute) produced a paper on the perceptions of Web Accessibility among student Web and application developers. Status: complete, but not yet released publicly.
Job Stories
In 2016, Silver identified 31 stakeholder roles of people who used accessibility guidelines. This quarter, Charles Hall with support from the Silver Task Force and Community Group created Job Stories for Silver Stakeholders.
A Job Story is a framework for use-centered design that focuses on the task(s) to complete or jobs to be done. A stakeholder for this context is the presumed role of an individual who needs to access and use accessibility guidelines. Each job story reflects one unique and presumed high priority task for the given stakeholder. Due to the nature of some roles, more than one job story may be illustrated.
The Job Stories will be used to evaluate Silver designs. The Silver consensus was that use-centered design would be more applicable to this phase of designing Silver than creating personas.
Problem statements
The Silver design has significant problems to solve to meet the project goals. A clear statement of the problem is useful to inform solution design activities. The format for these problem statements include: the specific problem; the result of the problem; the situation and priority; and the opportunity. The problems identified are the results of the research completed to date. They will be updated as more of the research is completed. For this phase of the Silver design, the problem statements are in three major areas:
- Conformance
- Usability
- Maintenance
The Problem Statements will be used in the Design Sprint.
Design Sprint
The Design Sprint will be a two-day face to face meeting in San Diego, California March 19 & 20 immediately prior to the CSUN AT Conference. The Design Sprint will be led by an outside moderator, Camron Shimy of Google, who is experienced with leading design sprints and training others in leading successful design sprints. A maximum of 30 people will be attending. Attendees have been invited across a variety of roles:
- Accessibility Influencers
- UX
- Developers
- Legal / Law / Policy
- Advocates
- Authoring Tool Developers
- User Agent Developers
- Assistive Technology developers
- Guideline Development Institutional Memory
- Information Architecture
- W3C Process
Silver Update for 3rd Quarter 2017
Executive Summary
- Research data is being shared with the TF
- Public input is planned for Q-4 using the Re-imagining Accessibility Guidelines slide deck
- Design Workshop for Prototyping Silver is tentatively scheduled for 19 & 20 March 2018 in San Diego preceding CSUN AT Conference
- Silver Update to the Accessibility Guidelines Working Group slide deck has additional details. This update was given in person at the AGWG face-to-face meeting at the TPAC Conference.
Silver Update for 2nd Quarter 2017
Executive Summary
- Several research projects are underway
- New research projects are ramping up for a total of 11 active projects.
- The research questions have been amplified and refined.
- There are new contributions to the Literature Review projects.
- The Silver Task Force has standardized on a small set of demographic questions that will be included in each survey for meta analysis.
Research Project Update
Currently active projects are examining the following research questions:
- How flexible are current standards in supporting emerging technologies?
- What are the factors that determine effective (i.e. widespread) adoption of and compliance with a new set of guidelines?
- How well does the current structure of W3C Accessibility Guidelines serve different stakeholder groups?
- How well does the current structure of W3C Accessibility Guidelines support learning and remembering key principles of web accessibility?
- How well do W3C Accessibility Guidelines communicate the benefits of creating web content and applications with accessibility in mind?
- How well do W3C Accessibility Guidelines communicate that accessibility supports business goals by expanding customer reach/growing revenue?
- What are the main reasons why accessibility is not seen as on the critical path for product success?
- What could W3C do to communicate the importance of accessibility to the business community?
- What aspects of WCAG do people drop?
- What aspects of WCAG are most important and why?
- What aspects of WCAG do people think are missing?
- What do people change about WCAG?
- What things are reworded for clarity?
- Investigate how web developers think about accessibility or people with disabilities and what influences their ability & desire to follow established guidelines.
- How is WCAG currently used by web developers? (diary study)
- How usable is WCAG to web developers?
- When organizations choose to produce adaptations of WCAG, how usable are W3C Accessibility Guidelines to different stakeholder groups? -- especially to people new to accessibility.
Literature Review Update
We are currently reviewing the research of the Cognitive Accessibility Task Force for inclusion in the Silver Literature Review. One of the researchers has applied for a grant to help fund a grad student to work on the Literature Review. Another researcher has volunteered to help specifically with the review of COGA material.
Silver Update for 1st Quarter 2017
Executive Summary
The Silver Task Force currently has 21 research partners and 7 research projects identified as of April 4 2017, with additional projects that are in the process of being solidified. Some of these projects will use the Stakeholder Map completed in January. The Literature Review project has begun. We have identified a list of research questions. We gave two presentations at the CSUN Assistive Technology Conference on Silver. There is now a W3C Silver Community Group for the research partners and general public who want to contribute to the project.
Stakeholder Map
We’ve completed mapping roles and actions for our Stakeholder Map, and have 350 Stakeholders identified, with research projects linked up to use our stakeholder contacts.
Research Questions and Projects
On the Research side of things, we have a seed list of research questions, created in collaboration with researchers, around the structure, scope, and actual content of accessibility guidelines. For a few examples:
- What are the needs of people with disabilities who are under-served from a standards perspective?
- How maintainable are W3C Accessibility Guidelines in their current form?
- When organizations choose to produce adaptations of WCAG…what do people change about WCAG?
- How have guideline development initiatives outside the field of accessibility engaged and solicited feedback from all relevant stakeholders? What methods have been particularly effective or ineffective?
We currently have 21 researchers (not counting students, etc. helping out), with seven research projects actively starting up right now. These currently active projects seek to answer the following questions:
- How well does the current content of W3C Accessibility Guidelines meet the accessibility needs of people with disabilities?
- What are the needs of people with disabilities who are under-served from a standards perspective?
- How flexible are current standards in supporting emerging technologies?
- What are the factors that determine effective (i.e. widespread) adoption of and compliance with a new set of guidelines?
- How well does the current structure of W3C Accessibility Guidelines serve different stakeholder groups?
- How well does the current structure of W3C Accessibility Guidelines support learning and remembering key principles of web accessibility?
- How well do W3C Accessibility Guidelines communicate the benefits of creating web content and applications with accessibility in mind?
- How well do W3C Accessibility Guidelines communicate that accessibility supports business goals by expanding customer reach/growing revenue?
- What are the main reasons why accessibility is not seen as on the critical path for product success?
- What could W3C do to communicate the importance of accessibility to the business community?
- What aspects of WCAG do people drop?
- What aspects of WCAG are most important and why?
- What aspects of WCAG do people think are missing?
- What do people change about WCAG?
- What things are reworded for clarity?
- How do W3C Accessibility Guidelines influence attitudes and biases related to accessibility and people with disabilities?
- How does inclusion of a specific suggestion from a disability community affect the accessibility of web content for other disability communities, based on survey response and interview feedback?
- Investigate how web developers think about accessibility or people with disabilities and what influences their ability & desire to follow established guidelines.
- How can national Mobile standards in Korea help WCAG 2.0 after?
Literature Review
We’ve started the process of organizing literature for review, using Zotaro to manage it all. The wiki page on the Literature Review process explains things in more detail, along with how to participate in the Literature Review.
Silver Talks at CSUN
The Silver Task Force presented at CSUN, two talks:
- What Comes After WCAG 2.1?, a formal talk on Silver, where it came from, what we’ve done so far, and what’ll happen from here. We had a packed room, and got very good (and ultimately positive) questions, as well as recruits for our just-then-created-and-announced Community Group.
- Q&A: Accessibility Guidelines, the Next Generation, a brief overview of the project, followed by open Q&A, fielded by Shawn, Jeanne, Michael, and Andrew. We also got quite a lot of good questions in this session.
New Community Group & Research Discussion Forum
We’ve created and launched a Community Group, with 31 participants (and counting), who will help with different stages of Silver development, most immediately research coordination and structure for other participation. The Community Group will enable us to delegate more of the short-term tasks outside of the immediate Task Force in order to make faster progress and include non-Working Group members in the process, as well as provide a platform for feedback and overall discussion. See our initial onboarding posts:
Silver Update for December 2016
The Silver Task Force has been preparing:
- Creating the Stakeholder Map
- Research preparation
Stakeholder Map
A Stakeholder Map is a technique of project management where representative people or groups that will be affected by Silver are identified and categorized. This allows us to identify, weigh, and balance conflicting interests. The Task Force created an online form for people to submit names to be considered for Stakeholders. The response greatly exceeded our expectations. As of 9 January 2017, there are 277 responses, which contain duplicate entries, but there are at least 240 unique submissions. The form will remain open, and new entries will continue be processed. Names submitted are international in scope, representing at least 12 countries (as determined from personal knowledge or email address).
In December, the Silver Task Force held a Face to Face meeting in New York City where we worked on identifying and defining all the roles applicable to Silver.
The Task Force identified the various actions that Silver could be used for, such as: consult, make policy, use policy, make content, teach, research, make standards, make decisions, assist people with disabilities, communicate requirements, measure or test, and is a beneficiary of standards. Based on those actions, we identified the various job tiles and roles and mapped what actions were performed by each job title/role. The Stakeholder Roles and Activities spreadsheet contains a brief explanation of each action. Because the Stakeholder Roles and Activities spreadsheet contains contact information, it is not currently public. A static version of the Stakeholder Roles and Activities will be made public without contact information.
The stakeholder names that have been submitted were mapped to the Roles and Activities so the Task Force could identify areas where we needed more stakeholders. The Task Force members have been doing outreach to request more stakeholders in roles where we determined that more names were needed, for example, from assistive technology vendors, and lawyers from outside the USA.
Next Steps in January/February:
- All the names submitted will be contacted to inform them about the Silver project and ask for their participation.
- As part of the research phase for Silver, the stakeholders who agree to participate will be sent surveys, or select people will be contacted for interviews or to participate in the self-reporting research project.
Research Partners
Since the Silver research phase involves a considerable amount of work, the members of the Task Force have been reaching out to faculty in the accessibility research field asking for assistance. The Task Force created a page of information for faculty, including the expectations for working with the Silver Task Force. Fourteen people have responded with interest.
Next Steps in January/February:
- Survey faculty for specific areas of interest
- Assign a Task Force member to each person as their liaison/partner
- Assist the researcher in starting the project
Other Next Steps for January/February
- Begin Literature Review. JaEun Ku (Jemma) of University of Illinois will be organizing the Literature Review phase of the project.
- Begin WCAG Analysis (leader TBD)
- Start WCAG adaption review (leader TBD)
- Increase recruiting for Silver Task Force, with a focus (but not exclusively) on people with research interest and skills.