Talk:Test template

Suggested addition: applicability

I would like to add:

• Non Applicability

To describe when the success criterion is applicable or not. This is related to the selector, but a further refinement. When a success criterion is not applicable, the outcome should be EARL:inApplicable (http://www.w3.org/TR/EARL10/#OutcomeValue)

For example: The success criterion caption for prerecorded video:

• the selector would be 'video'

But this success criterion is only applicable for

• Prerecorded audio content in synchronized media

And not applicable for

• Live video content
• Video without audio
• When the media is a media alternative for text and is clearly labeled as such

Also note that a selector is great if the website is semantic, but if it fails on semantic we need advanced selectors, like a pattern to detect a list of items that should be a list.

--Hanno (talk) 13:20, 21 May 2014‎ (UTC)

Suggestion for test specification addition

One improvement for the test specification is to determine the type of test according to the way it is tested. We can distinguish between at least two approaches for users to carry out the tests. One where the user will need access to the actual perceived page (In page), and a second where the content can be extracted from the original context and presented to the user on a new generated page (Out of page).

In page approach

Test the page in a way that it is perceived by the user. In this case, the test is done on the actual page. Therefore, it is necessary to have direct access to the page for these tests. For example, the test that checks if the title of the page is descriptive is an in page test because the whole page should be evaluated and compared with the title.

Out of page approach

It refers to tests which check specific elements of the page without the need to access the whole page. For example, the test that checks if the alt tag of an associated image is descriptive can be done out of page. A list of images and their alt tags can be presented to the user for evaluation, without having to be on the original page. This may be easier to use and probably easier to implement.

--Kamyar Rasta (talk) 10:47, 26 May 2014 (UTC)

The "in page approach" corresponds to test subject 4. rendered page. The "out of page approach" could be added as new item in the test subject list.

--Annika Nietzio (talk) 14:45, 27 May 2014 (UTC)

Result tables

--Wilco Fiers (talk) 15:18, 27 May 2014 (UTC) The existing test descriptions used a format that should probably be part of the template as well.

--Wilco Fiers (talk) 15:18, 27 May 2014 (UTC) I think we may need a more formal structure for these result tables. This still seems a bit vague and open ended to me. Maybe take a look at what features are part of EARL. I think a straight forward mapping to EARL may be useful to consider.

--Annika Nietzio (talk) 12:09, 3 June 2014 (UTC) As decided in the telecon on 28 May 2014, we will try to map the result structure to EARL and specify extensions where necessary. The suggested approach is descirbed in the Result template.