WAI R&D Symposia » Way-Finding Home » Proceedings » This paper.

This paper is a contribution to the Accessible Way-Finding Using Web Technologies. It was not developed by the W3C Web Accessibility Initiative (WAI) and does not necessarily represent the consensus view of W3C staff, participants, or members.

Extended Abstract for the RDWG Symposium on Accessible Way-Finding Using Web Technologies

Challenges and solutions to crowdsourcing accessibility evaluations

• Christos Kouroupetroglou, Caretta-Net Technologies, Greece, chris.kourou@cnt.gr
• Adamatios Koumpis, University of Passau, Germany, adamantios.koumpis@uni-passau.de

1. Problem Addressed

Crowdsourcing accessibility evaluations of places is an approach that has been followed from many applications and web sites using a variety of different approaches. A key point in the success of gathering such information is the mobilization and motivation of communities and people beyond the scope of people with disabilities (PwD) and their relatives. This paper discusses issues that should be taken under account in motivating and enabling larger groups of people than the PwD communities to provide such data.

2. Relevant background and approaches

As already referred earlier, there are a number of applications in various forms that currently are providing evaluations of the accessibility of specific places. Wheelmap [ 9] for example is such an application that enables rating the accessibility of various places and getting back related information from other users. A strong point of Wheelmap is the technical infrastructure that lies underneath and is based on OpenStreetMap [5] where users of various applications are providing information to create their own maps. This allows for communication and collaboration with other similar applications. However, the accessibility rating is based on a single attribute giving information about wheelchair accessibility and not being able to get or store information for different kinds of needs such as existence of Braille in the elevator, or loud music/noise etc. Jaccede [4] on the other hand enables the rating of a variety of attributes but keeps its own database of geographical information making it more difficult to cooperate with other applications. Rollout [7], is heavily focused on Athens and although it uses Foursquare [3] to get general information about a venues, has also its own scheme of attributes for rating accessibility. Similarly, AXSMap [2] and AbleRoad [1] are also following their own schemes of attributes which in AXSMap is quite short and for AbleRoad is quite extensive and detailed.

A key aspect for all of them is that they are highly connected with the community of PwD. Therefore it is hard for a user beyond that scope to get motivated to use it and rate places. It is even more difficult for them when they do not really know the needs of PwD. Doing the right thing is good motivation point for some users but what happens when such a user cannot really estimate if the step on the entrance is high enough to pose difficulties to a wheelchair user? This might discourage them and eventually lead them away from submitting any evaluation.

3. Challenges and potential solutions

3.1 Reliability of early gathered data

To overcome some of the aforementioned problems some applications ask from users to actually measure distances and physical properties of places, giving them specific expected values for a place to be rated as accessible. Comparing the approach of using 5-stars rating schemes to an approach that uses specific measurable physical properties to identify the accessibility of a place one could argue that such an evaluation is more objective and possibly more correct than a 5-stars rating schema. On the other hand, the usage of such measures might mean that users will have to be very good in judging distances, dimensions, etc. or that they carry appropriate equipment as it happens on the events organized by Jaccede.

Another important factor in enabling wider audience to submit ratings for such attributes as discussed earlier is that in many cases the attributes used are highly connected with the usage of assistive technologies such as wheelchairs. For people not familiar with such technologies this might discourage them from submitting evaluations due to increased uncertainty. For example, the accessibility of an entrance to a place might in most cases be considered as the existence of steps in the entrance. However, when it comes to the width of the entrance people who do not use wheelchair might not think of its importance.

A potential solution to that problem could be the introduction of equivalence of ratings using metaphors easily understandable by most people. For example, the internal space accessibility could be evaluated by saying that for 5 stars you should be able to easily walk inside the place with your arms stretched out without any problems or obstacles in your way. This way, users will be able to understand easier the essence of their evaluations and will probably have a set of rules of thumb to use for their evaluations.

3.2 Breadth vs depth of evaluations

As already discussed the applications use a variety of rating schemas from small to extensive list of attributes. Human Access for example uses 6 attributes that are as generic and widely applicable as possible so that we elicit as much information as possible from users. However, compared with other approaches PwD will not get detailed information about the place. If too much information is requested in an evaluation then people might be discouraged to complete them. If on the other hand evaluations are based on generic attributes there might be not detailed enough information for PwD. Applications in the domain must therefore find a balance between easiness in gathering information and providing detailed information.

A solution to this problem could be the grouping of specific attributes under more generic ones so that users can select the depth of evaluation they will conduct. Taxonomies of attributes that go into more details could lead to solving both problems especially if such taxonomies are also providing rules for calculating and generating more in depth values from generic attributes and vice versa. Moreover, this progressive disclosure feature [8, 6] in evaluations could also work in retrospective, triggering more detailed analysis of an evaluation after the generic one is done. For example, a user can evaluates 5 generic attributes for a place the first time he visits it but go into more details every other time he visits it.

3.3 Contextualization

Another important aspect of the evaluations gathered from Human Access is that in some cases the attributes used for evaluation are highly dependent on the context of evaluation. Contextualization could be linked with the type of venue being evaluated. Connecting specific attributes to specific types of venues could lead to a better more contextualized user experience. Therefore, a user who visits a hotel could evaluate if there are rooms for PwD or not while when visiting a restaurant there could be an attribute about the menus being available in Braille. This kind of contextualization could help in guiding the user to provide the most important information for a place rather than leading him to provide information for generic attributes.

Another important aspect of contextual information is the time of the day when the evaluation happens. Currently, Human Access keeps the time and date when each evaluation happens but does not use it in any way. Some attributes like lighting level and noise level could vary depending on the time of the day. For example, a café with wide windows in Greece might be well lit during the day and bad lit during night when it might be using mainly candle lights for its lighting during nights. Therefore, evaluations of lighting level could vary depending on the time of the day. This could lead to a lot conflicting evaluations rendering such attributes useless for PwD. If such attributes are connected with timing information they could possibly provide the appropriate contextual information when a user searches using the specific attribute.

4. Conclusions

Having seen the different approaches followed in applications that use crowdsourcing for gathering accessibility related data it is necessary to stress one crucial points that covers all three challenges and solutions presented earlier. If such applications want to widen their user base beyond the scope of PwD and their communities what needs to be done is to make such applications relevant and potentially useful for all. Knowing if a place is well lit or not might affect everyone’s decision to go into a place or not based on the mood they are. Similarly, other attributes used in such evaluations need to become more relevant to wider audiences and easier for them to evaluate too.

References

1. AbleRoadTM - Disability Access. (n.d.). Retrieved October 20, 2014, from http://ableroad.com
2. AXS Map. (n.d.). Retrieved October 20, 2014, from http://www.axsmap.com/
3. Foursquare. (n.d.). Retrieved October 20, 2014, from https://foursquare.com/
4. Jaccede.com - le guide collaboratif de vos bonnes adresses accessibles ! (n.d.). Retrieved October 20, 2014, from http://www.jaccede.com/fr/
5. OpenStreetMap. (n.d.). Retrieved October 20, 2014, from http://www.openstreetmap.org/#map=5/51.500/-0.100
6. Porter, J. (n.d.). Principles of User Interface Design.
7. RollOut Οδηγός Πόλης Για Εμποδιζόμενα Άτομα. (n.d.). Retrieved October 20, 2014, from http://www.rollout.gr/
8. Tidwell, J. (2010). Designing Interfaces. O’Reilly Media, Inc.
9. Wheelmap.org - Rollstuhlgerechte Orte suchen und finden. (n.d.). Retrieved October 20, 2014, from http://wheelmap.org/