User Survey: Browsers and scripts

Abstract

This document summarizes the results of an informal survey of users. The purpose of the survey was to gather information on the usage of browsers that support client-side scripting.

Status of this document

This document is intended to be used by the Web Content Accessibility Guidelines Working Group in discussions about making scripts directly accessible as well as what is needed when a user chooses not to run scripts. This document is not intended to be used by authors to determine when to begin to support client-side scripting, or stop providing text equivalents for client-side scripts.

This document does not represent any information that is statistically significant. It is an informal survey that will most likely be the basis for further questions and tests.

Introduction

The Web Content Accessibility Guidelines Working Group (WCAG WG) has been discussing how to make scripts transform gracefully. At issue is text equivalents for dynamic content and making scripts directly accessible.

This survey has two purposes:

1. to estimate how many people use browsers that support a specific kind of dynamic content (scripts)
2. to determine if people who use assistive technologies, such as screen readers, scanning software, screen magnifiers, etc. have difficulty with dynamic Web pages.

The results of the survey are based on the responses of a small, self-selected group of participants. These responses provide useful information about the use of technology by some users. However, the survey results are most likely skewed towards technically-savvy people as the original questionnaire was written in technical language. A less technical version was released to one listserv and non-technical respondents found it less intimidating and easier to understand. Therefore, this survey could be released again to draw in less technical people for more realistic results.

Survey Distribution

On 1 June 2000 the following survey was sent to the following listservs that discuss Web accessibility issues: basr-l@trace.wisc.edu, webwatch@telelists.com, and w3c-wai-ig@w3.org.

Hello all,

I am cross-posting this to a few lists. I apologize if you receive this more than once.

I am trying to get a feel for how many people use browsers that support scripting of some sort, such as JavaScript. Could you please respond to the following six questions? Send your response to me at wendy@w3.org by Monday, 12 June 2000. Please respond with the subject "User Survey: Browsers and scripts". Thank you!

--wendy

1. Do you use a Web browser that supports scripting (such as JavaScript)? yes/no
2. If your browser supports scripts, do you allow scripts to be run on your browser? Some people do not run scripts for security reasons. If you allow scripts to be run, please respond yes. If not, respond no. yes/no
3. If you do run scripts, do you use an assistive technology such as scanning software, a screen reader, a screen magnifier, etc? yes/no
4. If you do use an assistive technology, do you run into problems with pages that use scripts? yes/no
5. If you do run into problems, could you tell me more about them?
6. Please provide information about which browser(s) you use and with which assistive technologies.

I received comments on the webwatch-l list that the survey was too technical. Thus on that list I released the following survey on 5 June 2000:

These are good questions and comments, so let me take another stab at the survey. Please let me know if it is still too technical. Thank you in advance for your feedback.

Please note: if you have already filled out the survey, no need to fill it out again!! Thank you.

There are two purposes of the survey

1. to determine how many people use browsers that support a specific kind of dynamic content (scripts)
2. to determine if people who use assistive technologies, such as screen readers, scanning software, screen magnifiers, etc. have difficulty with dynamic Web pages.

There are several technologies that may be used to make a Web page dynamic, client-side scripting is one method. Other methods are Flash and Java applets.

The reason that I am focusing on scripts is because of a question that came up last week in the Web Content Accessibility Guidelines Working Group. This is the group of people who developed the Web Content Accessibility Guidelines and who continue to discuss how the guidelines are being applied.

If you use a browser like Lynx, IBM Home Page Reader, PW WebSpeak or Amaya, you have not run into scripts. Also, older browsers such as Mosaic 1.0 or Netscape 2.0 do not support scripts. You may have run into some pages that may not work or may be blank because they need scripts in order to run. You may have also run into some pages that say, "upgrade your browser, you need one that runs scripts to view this page." or some similar message.

Please return your answers to the following six questions to wendy@w3.org by Monday, 12 June 2000. Please respond with the subject "User Survey: Browsers and scripts". Thank you for your time!

1. Do you use one of the following browsers:
   • Netscape 3.0b5 or later
   • Microsoft Internet Explorer 3.0 or later
   • Opera 3.0 or later
   • WebTV 1.4 or later

   or another browser that you know supports client-side scripting? yes/no

2. If you do use one of these browsers, have you explicitly turned off scripts? For example, to turn off scripts in Netscape Communicator 4.7 you have to select "Preferences" from the Edit menu, then select the "Advanced" option, and uncheck, "Enable JavaScript". yes - I have turned off scripts/no, I let scripts run
3. If you have not explicitly turned off scripts in your browser, do you use an assistive technology such as scanning software, a screen reader, a screen magnifier, etc? yes/no
4. If you do use an assistive technology, do you run into problems with Web pages that seem to change as you interact with them? yes/no
5. If you do run into problems with dynamic pages, could you tell me more about them?
6. Please provide information about which browser(s) you use and with which assistive technologies.

I received further suggestions for making the survey easier to understand and other lists to include in the distribution of future surveys.

Participants

Participants were self-selected and voluntarily responded to the survey.

A total of 27 people responded. 25 people responded to the original survey, 2 people responded to the less technical survey.

Summary of responses by question

Do you use a Web browser that supports scripting? yes - 26, no - 1

Do you allow scripts to be run? 24 yes, 2 sometimes, 1 sometimes but mostly no

Do you use an Assistive Technology (AT)? 16 yes, 11 no

With an AT, do you run into problems with scripts? n/a 10, yes 13, no 4

Participants provided the following responses to the question, "What problems do you run into?" Some responses have been summarized, most are verbatim.

• Mouse overs and timelines cause problems for screen readers
• Downloading new plug-ins is annoying if the content is not extravagant
• Keyboard navigation and screen reading is problematic if not impossible
• Page specifies a 4x browser, but they really mean a specific browser's 4x
• Scripts don't work on the Macintosh
• "JavaScript" gets read for each control.
• Dynamic frames that change position seem to confuse my screen reader, sometimes reading them over and over without reading the rest of the page.
• Pop-up windows that do not take system focus are not read.
• Expandable lists of topic links are not read at all.
• DHTML expanding hot spots are not read.
• Frames embedded with IFRAME or OBJECT are not read.
• Scrolling text confuses the screen reader and the page will not be read.
• Sometimes they crash my PC.
• Sometimes the reading of the page is garbled.
• Some scripts work fine, it depends on the tools I use.
• Now that I have the latest version of JAWS for Windows, the biggest problem is that the virtual image does not get updated in response to scripts. If I know that a page uses a collapsible outline or menus activated by web links, I can use JAW's refresh feature to force the screen reader to update its buffer after the menu/collapsible outline script has run.
• The only problem that I have is that IBM Home Page Reader does not support JavaScript.
• Scripts that generate pop up windows cause disorientation due to lack of context making the page essentially incomprehensible. Pages activated by mouseover events are especially frustrating.
• This does not mean that scripts are inherently frustrating, just that their careful use seems not to be widespread.
• These pages often don't provide the same information in an alternative manner.
• Sometimes links don't get read, sometimes there is just an image to click or something that cannot be read. Sometimes you don't know what's happening.

Which browser do you use and with which assistive technology?

• no assistive technology - 27 [note that many people reported using more than one browser and no assistive technology]
• HAL - 3
• Jaws - 14 (with Alva - 1) [note that several people use more than one assistive technology]
• Window Eyes - 3
• Invision - 1
• Outspoken - 1
• Amaya - 1
• Home Page Reader - 2
• IE - 24
• Lynx - 3
• Mozilla - 1
• Navigator/Communicator - 14
• Opera - 6
• WebTV - 1

Note that

• HAL, Jaws, Window Eyes, and Outspoken are screen readers.
• ALVA is a dynamic braille display.
• Home Page Reader, IE, Lynx, Mozilla, Navigator/Communicator, Opera, and WebTV are browsers.

Which combinations are used?

• IE and HAL - 1
• IE and Jaws - 13
• IE and WE - 1
• Lynx and Jaws - 1
• Navigator and HAL - 1
• Navigator and Jaws - 3
• Navigator and WE - 1
• Opera and HAL - 1
• Opera and Jaws - 1
• Opera and WE - 1

Other comments from participants:

• Least problems with Jaws3.2 (or later) and Navigator 4.08
• I use Internet Explorer 5.01 with JAWS for windows 3.5. Other browsers, including Amaya, are almost totally inaccessible, at least by comparison.

Conclusions

The majority of people responding to the survey used browsers that support scripting. However, it seems that those people that responded were technically-savvy enough to understand the concept of scripts. Therefore for a more accurate study, it is recommended that the study be released based on the suggestions received from survey respondents. A wider distribution is also recommended to increase the number of respondents.

This experience raises other questions as well such as:

• Is there a difference between how many people in general use browsers that support scripts and how many people within the community of assistive technology users use browsers that support scripts?
• If many assistive technology users use script-enabled browsers does it make it more or less important that the scripts transform gracefully?
• Can assistive technology users detect problems with scripts? If pages are so unusable because of scripts how will someone who is using an assistive technology know to attribute it to scripts?
• What is the best way to ask non-technical people about technical issues? It was suggested that participants should interact with a series of test pages and answer questions based on those interactions.

Other suggested changes to make to the survey:

• Clarify that "scanning software" refers to software that is used with a switch to select objects on the screen rather than Optical Character Recognition software.
• Distribute a survey to a wider audience. The responses were skewed towards people who use screen readers most likely because the listservs that were chosen have strong representation from that population (basr-l - browsers and screen readers especially).
• Since MSIE was used by most of the respondents, describe how to disable scripting in MSIE as well as Netscape (Opera as well?).
• Eliminate the last two questions by asking "which ones?"

Thanks to Lisa LaNell Mauldin and Kitch Barnicle for their feedback.

$Date: 2000/07/06 18:17:09 $ Wendy Chisholm