Individual Differences

Introduction: Ranked Choice Voting (RCV) Ballot of Candidates by E-mail

I did a quick little informal accessibility and usability study, and the results are a good example of individual differences. I'm putting the study information here in case others find it useful for:

An illustration of individual differences, particularly related to accessibility
An example of how a quick, informal "study" with real users can lead to substantive improvements
Example wording of a ranked choice voting (RCV) ballot of candidates by e-mail

~Shawn Lawton Henry

Recommendation

This section has the recommendations based on the accessibility and usability input detailed in later sections of this page.

E-mail ballot wording:

Rank candidates in the order of your choice.
- Replace the asterisk (*) before the candidate's name with 1 for your first choice, 2 for your second choice, and so on.
- You do not have to rank all candidates. Leave the asterisk for those you do not want to rank.
- Optionally, you may re-order the list of candidates, keeping the numbers with the names.
- Leave all 13 candidate names in your e-mail ballot.
Please do not modify this ballot in any other way.

Candidates are initially listed in alphabetical order by family name.

* Ananya Barad
* Taahid Bhattacharyya
* Xiaoping Chén
* Marius Keller
* Henri Krämer
* Jaxton Lindsey
* Apolline Masson
* Sébastien Mathieu
* Léopold Melgares
* María Carmen Ramirez
* Kennedy Reyes
* Jürgen Schwarz
* Tomita Yoshio

Ballot validation:

Check that the ballot includes all 13 names. If not, ask the voter to re-submit a new ballot.
If duplicate numbers (e.g., two candidates ranked 3), ask the voter to re-submit a new ballot.

Initial Input

The options were drafted based on input from accessibility and usability specialists, and guidance in Best practices for ranked choice voting ballots and other materials (PDF). Some perspectives in this section.

Accessibility and usability specialist

Hypothesis: Screen reader users will prefer asterisks, and cognitive accessibility specialists will prefer brackets.

For my brain, it would be easier to put the names in order, rather than adding numbers with names out of order.

That is, this is harder:
3. Alice Abrams
1. Barbara Baker
2. Charly Cook

Whereas, re-ordering the names works better for my brain:
1. Barbara Baker
2. Charly Cook
3. Alice Abrams

And brackets adds more clutter visually and for screen readers:
[3] Alice Abrams
[1] Barbara Baker
[2] Charly Cook

Expert in voting usability and accessibility

If the voting is via email, looks like any reordering is manual.
I agree that it's much easier to proof your selections when you can see the candidates in order (our accessible electronic prototype suggests that voters can choose when to reorder during marking, so they don’t get lost if there is a long list of candidates)

"Study" Details

E-mail request with options

I sent the following plain text e-mail:

Subject: Accessibility and usability feedback - short

Hi, [name],

In drafting the e-mail for the upcoming W3C, Inc. Board election, we would like to know which option is more accessible and usable. Below my signature is option 1 listing the candidates with asterisks, and option 2 listing candidates with brackets.

We welcome your feedback asap this week on if either option is somewhat or significantly better. Feel free to comment on the instructions as well.

Please CC: wai@w3.org if you want your feedback visible to W3C staff.

Thanks much!
~Shawn

# Option 1 for listing candidates with asterisks:

Rank candidates in the order of your choice.
* Before the candidate’s name, put 1 for your first choice, 2 for your second choice, and so on.
* You may re-order the list of candidates, keeping the numbers next to each name.
* You do not have to rank all candidates.
* Please do not modify this ballot in any other way.

Candidates are initially listed in alphabetical order by family name.

* Alice Abrams
* Barbara Baker
* Charly Cook

# Option 2 for listing candidates with brackets:

Rank candidates in the order of your choice.
* In the brackets before the candidate’s name, put 1 for your first choice, 2 for your second choice, and so on.
* You may re-order the list of candidates, keeping the numbers next to each name.
* You do not have to rank all candidates.
* Please do not modify this ballot in any other way.

Candidates are initially listed in alphabetical order by family name.

[   ] Alice Abrams
[   ] Barbara Baker
[   ] Charly Cook


<end>

Respondents

I sent the e-mail to people active in W3C, most of whom I know personally.

3 people who are blind and are advanced screen reader users
4 people involved in cognitive accessibility, at least one with cognitive disabilities and one with low vision

Responses:

5 replied
1 vacation auto responder
1 no reply

All Results

Summary: Two preferred the asterisks option, two preferred the brackets option, and one suggested putting the numbers after the names.

Blind screen reader user 1

Mild preference for Option 1 with asterisks

Thanks for asking. In truth I think both approaches are fine. I do prefer the first, but not strongly.

Blind screen reader user 2

Option 1 with asterisks

Asterisks (*) are better for me. People may not be familiar with brackets and may not be clear on where the numbers should go. It's not clear to me based on the explanations below.
If going for brackets, I suggest.
Original:
> * In the brackets before the candidate’s name, put 1 for your first choice, 2 for your second choice, and so on.
Proposed:
> * Inside the brackets before the candidate’s name, put 1 for your first choice, 2 for your second choice, and so on.
Or
> * Between the opening and closing brackets before the candidate’s name, put 1 for your first choice, 2 for your second choice, and so on.

Cognitive TF participant with low vision

No comment on astricks or brackets
Suggests changing instructions to putting number after the name, instead of before

I think the instructions are fine: short, simple, clear.
I suggest changing one word: “Before” to “After.”
Result: “After the candidate’s name, put 1 for your first choice, 2 for your second choice, and so on.”
I think screen reader and magnification users would find it easier to add a number after a name than to see, dodge, or manipulate characters (asterisks and brackets) before names.

Cognitive TF participant 1

Option 2 because visual clue to put something between brackets
In the instructions, the re-ordering list option causes extra cognitive processing

Thank you for reaching out for input. Please know that my response is specific to me and my own interpretations. Others may respond differently.
I think option 2 is more clear, as there is a visual clue that I should put something between the brackets. The first option leaves me guessing about whether or not I'm supposed to replace the * (even if I've read the text).
The instructions seem reasonably clear, with the exception of this one: "* You may re-order the list of candidates, keeping the numbers next to each name." With that second item, I'm left wondering if I have to reorder them, or if this is an option, and if one is better or preferred. If you remove this item entirely, then the directions generally become a lot simpler.

Cognitive TF participant 2

Prefers Option 2 brackets because of clarity for where to put numbers

... thanks so much for reaching out. It's great to have the opportunity to think with you on this!
Two concerns about both options, especially if the ballot has a long list of candidates:
If users can re-order the list of candidates, can they accidentally erase one or more of the candidates? Is there a way to undo an action?
What happens if a user tries to put, say, a "3" next to more than one candidate's name?
Option 1 vs Option 2:
If the only two options are asterisks or brackets, I prefer the brackets for two reasons:
Option 2 is clearer that the numbers should be typed in between the brackets. With Option 1, it's not clear if the number should go in front of the asterisk or between the asterisk and the candidate's name.
Option 2 may look less cluttered once users have entered their numbers and so might be easier for users to review their choices.
Option 3:
I'm assuming it's too late to discuss possible alternatives?
Thanks again for the opportunity to weigh in, cheers!

Changes

The user feedback provided insights for making the instructions more specific for the asterisks option, and for helping to result in more valid ballots.

The following changes were made to the e-mail ballot based on user input:

Changed: "Before the candidate’s name, put 1..."
to: "Replace the asterisk (*) before the candidate's name with 1..."
Added 'optionally' to: "Optionally, you may re-order the list of candidates..."
Added instructions to leave asterisk for no rank: "You do not have to rank all candidates. Leave the asterisk for those you do not want to rank."
Added: "Leave all 13 candidate names in your e-mail ballot."
In the instructions, changed the plain text "bullet" from asterisks to dash.

Also, added for the process (not the ballot itself):

Ballot validation:
Check that the ballot includes all 13 names. If not, ask the voter to re-submit a new ballot.

If duplicate numbers (e.g., two candidates ranked 3), ask the voter to re-submit a new ballot.

Notes

The names in the Recommendation section are randomly generated. If any happen to be real people's names, let me know (shawn@w3.org) and I'll change it.
This was not part of any formal or funded project work.

Acknowledgements

Project Lead / Editor: Shawn Lawton Henry

Thanks to all who contributed to the drafts and the user input, including: Rain Michaels, Whitney Quesenbery, Julie Rawe, Janina Sajka, Wendy Seltzer, Ralph Swick, (others awaiting permission to list: DM, JR).