WAI Site Redesign Usability Testing 2
DRAFT Initial Findings Notes
First test
Grade: C+
Describing the WAI Web Site
- It has a lot of really good information but it is hard to get to and
decipher.
- if you are gearing towards developers, then I would look for keywords
specific to developing sites. It needs keywords, instead of leafing
through the whole site.
- It is hard to navigate, words are repeated over and over. I feel a
little bit frustrated navigating it. I feel like I went in circles.
- Navigation on this site is often different from 99% of what you see out
there.
- It feels like the page gets developed as information becomes developed.
The overall site is not taken into consideration.
- There's too much information. There's so much of 'this is our news,'
'this is how we put stuff together,' and not 'this is what we have for
you.'
- It would be easier for a screen reader if you didn't have to read
through everything and instead have the meat of the stuff first.
Words or Characteristics
- Confusing
- Exclusive, like a club I don't belong to
- Overwhelming with information
- Content-rich
- Information-rich
- Slightly difficult to navigate
- Technical
- Low budget
- Useful
- Informative
- Insufficient
- Stark
- Challenging
- Technical
Second Test
Grade: A-/B+
Rating of W3C Documents: D+
Describing the WAI Web Site
- Helpful
- resources
- text heavy
- hard to navigate
- full of information
- easy on the eyes
- pretty quick
- some of terminology was a little fuzzy
- user friendly
- very professional
- very informative
- So far it's the best web accessibility website I've seen
- easy to use
- neat interface
- very organized
- frustrating
- necessary
- informative
- up to date
- helpful
- comprehensive
- useful
- very good amount of practicable information
- Clean-cut: a neat design
Comments
W3C Docs: “confusing to read. Has great info, there's a lot of it.
Not organized in a user friendly way. Really hard to navigate. And find the
info I'm looking for. Very text heavy. It's organized like a book with a
table of contents, maybe not that way. Maybe should have separate pages for
separate things instead of jump down the page.” (P1 1:32:24- )
WAI Site: “Pretty impressed with it. Didn't like the left hand
cascade, but the more I used it, and saw it had function, like on long pages
can sae you going half way down pages, didn't see anything glaring that said
"this is bad." (1:19:58 P3)
“Very professional, nicely laid out. I like it a lot. A lot more
organized, easy to navigate. Looks very nice.”
Task Completion Rates
Initial Usability Testing
| Task |
Task Description |
% completed |
% completed w/ assistance |
% failed |
% who rated task as "easy" |
| 3 |
WAI Mission |
100% |
0% |
0% |
88% |
| 4 |
Quick Tips |
63% |
12% |
25% |
63% |
| 5 |
Participation |
63% |
0% |
37% |
63% |
| 6 |
Evaluation |
0% |
50% |
50% |
25% |
| 7 |
Online Forms |
38% |
12% |
50% |
12% |
| 8 |
Training Info |
50% |
25% |
25% |
50% |
Second Usability Testing
| Task |
Task Description |
% completed |
% completed w/ assistance |
% failed |
% who rated task as "easy" |
| 3 |
WAI Mission |
63% |
25% |
13% |
75% |
| 4 |
Quick Tips |
50% |
12% |
38% |
50% |
| 5 |
Participation |
100% |
0% |
0% |
88% |
| 6 |
Evaluation |
88% |
12% |
0% |
50% |
| 7 |
Online Forms |
75% |
25% |
0% |
38% |
| 8 |
Training Info |
50% |
13% |
38% |
75% |
Post-Test Evaluation Survey Results
We asked participants to fill out a survey that asked them a series of
questions about the Web site. They then rated their level of agreement or
disagreement with each statement on a 1-7 scale where 1 was "disagree
strongly" and 7 was "agree strongly." Survey responses are provided below.
Table 1 - Existing WAI Site
| Statement |
P1 |
P2 |
P3 |
P4 |
P5 |
P6 |
P7 |
P8 |
Average |
| The homepage is attractive |
3 |
4 |
5 |
2 |
4 |
5 |
5 |
n/a |
4.0 |
| The overall site is attractive |
3 |
2 |
4 |
2 |
4 |
4 |
5 |
n/a |
3.4 |
| The site's graphics are pleasing |
n/a |
n/a |
3 |
2 |
n/a |
5 |
n/a |
n/a |
3.3 |
| The site has a good balance of graphics versus text |
n/a |
3 |
2 |
2 |
3 |
3 |
6 |
n/a |
3.2 |
| The colors used throughout the site are attractive |
6 |
4 |
6 |
3 |
4 |
6 |
3 |
n/a |
4.6 |
| The typography is attractive |
6 |
5 |
6 |
4 |
6 |
6 |
6 |
n/a |
5.6 |
| |
|
|
|
|
|
|
|
|
|
| The homepage's content makes me want to explore the site
further |
4 |
6 |
6 |
6 |
5 |
6 |
5 |
6 |
5.5 |
| It is easy to find one's way around the site |
3 |
4 |
2 |
6 |
3 |
3 |
6 |
4 |
3.9 |
| You can get information quickly |
2 |
4 |
2 |
7 |
2 |
2 |
5 |
4 |
3.5 |
| It is fun to explore the site |
2 |
3 |
5 |
6 |
3 |
5 |
n/a |
3 |
3.9 |
| It is easy to remember where to find things |
4 |
5 |
2 |
6 |
3 |
4 |
6 |
5 |
4.4 |
| Information is layered effectively on different screens |
4 |
4 |
3 |
6 |
3 |
3 |
n/a |
4 |
3.9 |
| |
|
|
|
|
|
|
|
|
|
| The homepage is attention-getting |
6 |
3 |
1 |
3 |
3 |
3 |
3 |
n/a |
3.1 |
| Information is easy to read |
2 |
3 |
3 |
6 |
3 |
6 |
6 |
3 |
4.0 |
| Information is written in a style that suits me |
2 |
5 |
3 |
6 |
3 |
3 |
6 |
4 |
4.0 |
| Screens have the right amount of information |
5 |
4 |
3 |
5 |
2 |
5 |
6 |
6 |
4.5 |
| The site effectively communicates the company's image |
5 |
6 |
4 |
6 |
2 |
4 |
6 |
7 |
5 |
| Information is relevant |
5 |
7 |
6 |
7 |
6 |
7 |
6 |
6 |
6.3 |
| |
|
|
|
|
|
|
|
|
|
| The site is designed with me in mind |
3 |
6 |
6 |
6 |
4 |
3 |
5 |
4 |
4.6 |
| The site's content interests me |
6 |
7 |
6 |
7 |
7 |
7 |
6 |
6 |
6.5 |
| The site's content would keep me coming back |
6 |
7 |
6 |
7 |
6 |
7 |
6 |
7 |
6.5 |
| The site has characteristis that make it especially appealing |
3 |
5 |
3 |
6 |
4 |
3 |
5 |
6 |
4.4 |
| The site reflects progressive, leading edge design |
3 |
6 |
4 |
2 |
2 |
2 |
3 |
5 |
3.4 |
| The site is exciting |
3 |
2 |
4 |
4 |
2 |
2 |
n/a |
4 |
3.0 |
| |
|
|
|
|
|
|
|
|
|
| The site is well-suited to first-time visitors |
4 |
3 |
2 |
6 |
1 |
2 |
5 |
2 |
3.1 |
| Ths site is well-suited to repeat visitors |
6 |
4 |
5 |
6 |
2 |
5 |
6 |
5 |
4.9 |
| The site has a clear purpose |
6 |
6 |
7 |
7 |
3 |
6 |
7 |
6 |
6.0 |
| It is always clear what to do next |
3 |
4 |
2 |
5 |
1 |
2 |
7 |
4 |
3.5 |
| It is clear how screen elements work |
3 |
6 |
4 |
7 |
5 |
3 |
6 |
7 |
5.1 |
| Mistakes are easy to correct |
4 |
5 |
4 |
7 |
2 |
5 |
6 |
5 |
4.8 |
| Average |
4.0 |
4.6 |
4.0 |
5.2 |
3.4 |
4.2 |
5.5 |
4.9 |
4.5 |
Table 2 - Redesigned WAI Site
| Statement |
P1 |
P2 |
P3 |
P4 |
P5 |
P6 |
P7 |
P8 |
Average |
| The homepage is attractive |
7 |
5 |
6 |
7 |
6 |
4 |
6 |
5 |
5.8 |
| The overall site is attractive |
4 |
5 |
6 |
7 |
6 |
4 |
5 |
5 |
5.3 |
| The site's graphics are pleasing |
7 |
3 |
7 |
5 |
3 |
3 |
3 |
5 |
4.5 |
| The site has a good balance of graphics versus text |
7 |
5 |
7 |
5 |
4 |
4 |
3 |
5 |
4.8 |
| The colors used throughout the site are attractive |
7 |
4 |
6 |
7 |
7 |
4 |
5 |
5 |
5.8 |
| The typography is attractive |
7 |
4 |
6 |
7 |
7 |
5 |
5 |
5 |
5.9 |
|
|
|
|
|
|
|
|
|
|
| The homepage's content makes me want to explore the site
further |
7 |
5 |
5 |
7 |
6 |
6 |
5 |
6 |
5.9 |
| It is easy to find one's way around the site |
4 |
4 |
6 |
6 |
6 |
6 |
6 |
5 |
5.4 |
| You can get information quickly |
4 |
4 |
6 |
6 |
6 |
6 |
6 |
5 |
5.4 |
| It is fun to explore the site |
4 |
4 |
6 |
6 |
6 |
5 |
5 |
6 |
5.3 |
| It is easy to remember where to find things |
2 |
3 |
6 |
4 |
4 |
6 |
5 |
7 |
4.8 |
| Information is layered effectively on different screens |
2 |
4 |
n/a |
6 |
4 |
4 |
6 |
5 |
4.6 |
| |
|
|
|
|
|
|
|
|
|
| The homepage is attention-getting |
6 |
3 |
5 |
7 |
6 |
4 |
5 |
5 |
5.1 |
| Information is easy to read |
7 |
4 |
7 |
7 |
6 |
6 |
5 |
6 |
6.0 |
| Information is written in a style that suits me |
7 |
5 |
7 |
7 |
6 |
4 |
5 |
6 |
5.9 |
| Screens have the right amount of information |
6 |
5 |
7 |
7 |
6 |
4 |
5 |
3 |
5.0 |
| The site effectively communicates the company's image |
3 |
3 |
4 |
7 |
6 |
5 |
6 |
4 |
5.1 |
| Information is relevant |
7 |
6 |
7 |
6 |
7 |
6 |
5 |
7 |
6.4 |
| |
|
|
|
|
|
|
|
|
|
| The site is designed with me in mind |
5 |
5 |
6 |
7 |
7 |
4 |
5 |
6 |
5.5 |
| The site's content interests me |
7 |
4 |
7 |
7 |
7 |
7 |
5 |
6 |
6.4 |
| The site's content would keep me coming back |
7 |
6 |
7 |
7 |
7 |
7 |
5 |
6 |
6.5 |
| The site has characteristis that make it especially appealing |
6 |
4 |
6 |
6 |
7 |
5 |
5 |
5 |
5.5 |
| The site reflects progressive, leading edge design |
3 |
3 |
7 |
6 |
7 |
4 |
6 |
3 |
4.9 |
| The site is exciting |
3 |
3 |
6 |
6 |
7 |
2 |
4 |
5 |
4.5 |
| |
|
|
|
|
|
|
|
|
|
| The site is well-suited to first-time visitors |
3 |
4 |
5 |
6 |
7 |
4 |
6 |
6 |
5.1 |
| Ths site is well-suited to repeat visitors |
5 |
5 |
7 |
7 |
7 |
4 |
6 |
5 |
5.8 |
| The site has a clear purpose |
6 |
6 |
6 |
7 |
7 |
7 |
7 |
6 |
6.5 |
| It is always clear what to do next |
3 |
4 |
6 |
5 |
6 |
5 |
5 |
7 |
5.1 |
| It is clear how screen elements work |
7 |
3 |
7 |
5 |
n/a |
5 |
5 |
6 |
5.4 |
| Mistakes are easy to correct |
5 |
6 |
7 |
6 |
n/a |
6 |
6 |
6 |
6.0 |
| Average |
5.3 |
4.3 |
6.2 |
6.3 |
6.2 |
4.9 |
5.2 |
5.4 |
5.5 |
Copyright 1994-2004 W3C (MIT, ERCIM, Keio), All Rights Reserved. W3C
liability, trademark, document use and software licensing rules apply. Your
interactions with this site are in accordance with our public and Member
privacy statements.