W3C

W3C mobileOK Pro Tests Version 1

Group Working Draft 28 February 2008

This version:
http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/mobileOKPro/drafts/ED-mobileOK-pro10-tests-20080228
Latest version:
http://www.w3.org/2005/MWI/BPWG/Group/TaskForces/mobileOKPro/drafts/latest
Previous version:
http://www.w3.org/2005/MWI/BPWG/Group/Drafts/mobileOK-Pro-1.0-Tests/20070716 [member-only]
Editors:
Kai Scheppe, Deutsche Telekom AG, Products & Innovation
Phil Archer, Family Online Safety Institute
Dave Rooks, Segala

Abstract

This document outlines a set of tests requiring human evaluation, based on the Mobile Web Best Practices 1.0 document and the mobileOK Basic Tests document, provided by the Mobile Web Initiative Best Practices Working Group

Status of this Document

This document is an editors' copy that has no official standing.

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

Publication as a Group Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document has been produced by the mobileOK Pro Tests Taskforce of the Mobile Web Best Practices Working Group as part of the Mobile Web Initiative . Please send comments on this document to the Working Group's public email list public-bpwg-pro@w3.org, a publicly archived mailing list .

This document was produced under the 5 February 2004 W3C Patent Policy . W3C maintains a public list of patent disclosures made in connection with this document; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) with respect to this specification must disclose the information in accordance with section 6 of the W3C Patent Policy.

Revision Description

First Editor's Draft

See [todo] for a list of todos.

Table of Contents

1Introduction
    1.1 Purpose
    1.2 Scope
    1.3 Audience

2 Test format

3 Sampling scheme

4 Tests
    4.1 Access Keys
    4.2 Auto Refresh
    4.3 Avoid Free Text
    4.4 Background Image Readability
    4.5 Balance
    4.6 Device Capabilities
    4.7 Central Meaning and Limited
    4.8 Clarity
    4.9 Color Contrast
    4.10 Content Format Preferred
    4.11 Control Labeling
    4.12 Control Position
    4.13 Cookies
    4.14 Deficiencies
    4.15 Error Messages
    4.16 Fonts
    4.17 Graphics for Spacing
    4.18 Link Target ID
    4.19 Minimize Keystrokes
    4.20 Navbar
    4.21 Navigation
    4.22 Non-text Alternatives
    4.23 Objects or Scripts
    4.24 Page Size Usable
    4.25 Page Title
    4.26 Provide Defaults
    4.27 Scrolling
    4.28 Structure
    4.29 Style Sheets Size
    4.30 Style Sheet Support
    4.31 Suitable
    4.32 Tab Order
    4.33 Tables Layout
    4.34 Tables Support
    4.35 Testing
    4.36 URIs
    4.37 Use of Color

Appendices

A To Do (Non-Normative)
References (Non-Normative)


1 Introduction

1.1 Purpose

The purpose of this document, despite its subjective nature, is to provide guidelines to content producers on how to go beyond mobileOK Basic and make the claim of having achieved mobileOK Pro, by following the guidelines laid out in this document.
mobileOK Best Practices is a set of guidelines which are partially testable via the mobileOK Basic Tests Checker, a tool for automated tests of a subset of Best Practices.

mobileOK Pro then provides a set of guidelines that fill the gaps left by the limitations of automated tests and thus completes the set of Best Practices.

1.2 Scope

The scope of this document is limited to the mobileOK Best Practices, the mobileOK Basic Tests resulting from those best practices and the difference between the two.

1.3 Audience

This document is intended for creators, maintainers and operators of Web sites. Readers of this document are expected to be familiar with the creation of Web sites, and to have a general familiarity with the technologies involved, such as Web servers and HTTP. Readers are not expected to have a background in mobile-specific technologies.

2 Test format

2.1 Test template

This section will be removed prior to publication

x.x Name

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

None

[Pass]

[Warn]

[Fail]

Examples:

None

Open issues:

None

3 Sampling

This still needs reworking by Alan

Say something about the sampling scheme.
For example, use the UWEM scheme (http://www.wabcluster.org/uwem1_2/) which is applicable. Some tests assume a defined sample scope to carryout the test (for example, NAVIGATION).

4 Tests

4.1 Access Keys

Note to BPWG:

Limitations of this test:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Where there are elements, particularly navigation links and form controls, that would benefit from access keys:

Examples:

Open issues:

One might argue what benefits and what doesn’t?

To be more explicit - if you have navigation options and/or form controls, there should be access keys.

4.2 Auto Refresh

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

The Basic test detects the presence of an automatic refresh, the Pro test looks for a user option to prevent automatic refresh.

Pseudo code:

If the Basic Test results in a warning about auto refresh and there is no link provided to another instance of the content which does not refresh [FAIL]

Examples:

None

Open issues:

None

4.3 Avoid Free text

Note to BPWG:

None

Limitations of this test:

There are many different ways to aid data input, with no one 'right' way. However, leaving it entirely up to the user to enter the data freehand without any attempt to offer pre-selection or pre-filling of the input field by automated means, where avilable, is readily detectable as 'the wrong way.'

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

If there are one or more free text input fields:

Could they be converted into any of:

If these techniques are avilable, but not implemented: [FAIL].

If the techniques above are not avilable and if data has been entered previously, if the previously-entered data is not used to pre-fill the input field: [FAIL].

Examples:

Open issues:

None

4.4 Background Image readability

Note to BPWG:

None

Limitations of this test:

This test does extend to users with visual impairment. The Web Content Accessibility Quick Reference offers more advice on this topic.

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests, but it is worth noting that the DDC does support background images which are part of CSS 1.

Pseudo code:

Where there is a background image, if perceiving content in the foreground is not easy under normal daylight conditions: [FAIL]

Examples:

Open issues:

Should we include a reference to the WCAG 2.0 techniques? If so, did Alan choose the right one?

4.5 Balance

Note to BPWG:

None

Limitations of this test:

This test is highly dependent on context. Some content, such as site maps, online encyclopedias and other reference materials may warrant a high link density. As the scrolling action on many mobile devices selects each link in turn, however, including a lot of links on a single page can easily create an unsatisfactory user experience.

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Unless the primary purpose of the content is navigation, such as a site map, or the user could readily want to persue links to further resources in an unpredictable manner:

If the page contains more than 30 links: [FAIL]

Examples:

None

Open issues:

None

4.6 Device Capabilities

Note to BPWG:

None

Limitations of this test:

This test cannot cover all the possible capabilities that might be found in devices. However, it is usually possible to detect presentation that is artificially constrained by the limited and largely hypothetical Default Delivery Context. Unlike the DDC, many real devices support scripting, XML HTTP requests, DOM capabilities, cookies and CSS including media queries. Furthermore most current devices have screens well in excess of 120 pixels wide

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

If the presentation of the content is artificially limited by the capabilities of the DDC and if the presentation is similarly limited on more capable devices: [FAIL]

Examples:

Open issues:

This is a can of worms, but it is important to counteract the “working down to the DDC” issue

4.7 Central Meaning and Limited

Note to BPWG:

None

Limitations of this test:

There are subtle differences between these two Best Practices. Central Meaning is concerned with the physical layout of the page, ensuring that desktop-style navigation does not dominate whilst Limited is concerned with presenting information in a manner that reflects the likely real world situation of the audience - on the move with limited time and attention span, perhaps in an environment where there are many forces competing for the user's attention. Despite these differences, it is convenient to test conformance with these two Best Practices together.

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examples:

None

Open issues:

None

4.8 Clarity

Note to BPWG:

None

Limitations of this test:

This test is subjective and cannot be conclusive. It must therefore remain at a warning level to be informative.

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

If the text is, for the context, considered to be unnecessarily complex or verbose [WARN]

Examples:

Open issues:

None

4.9 Color Contrast

Note to BPWG:

None

Limitations of this test:

This test is actually defined in http://www.w3.org/TR/2007/WD-WCAG20-TECHS-20070517/Overview.html#G18, to which we refer here

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

If the contrast ratio is less than 5:1, according to http://www.w3.org/TR/2007/WD-WCAG20-TECHS-20070517/Overview.html#G18 [FAIL]

Examples:

None

Open issues:

None

4.10 Content Format Preferred

Note to BPWG:

This test is machine testable and we are not sure why is it not covered in the basic tests.

Limitations of this test:

This test cannot be failed, because alternate formats may not be available.

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

The follow is not very explanatory and needs some wordsmithing.
How would one determine a "change" in the quality headers, which denote preference of a content format?
Or did we mean if they are different, as in 0.1 - 0.9 over a variety of formats?

If the quality factors of the accept headers are changed and the content which is delivered does not reflect that change [WARN]

Examples:

A change in preference between GIF and JPEG, character encoding or markup for a given device, should be reflected accordingly.

Open issues:

None

4.11 Control Labeling

Note to BPWG:

This test is machine testable

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examples:

None

Open issues:

4.12 Control position

Note to BPWG:

None

Limitations of this test:

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examples:

None

Open issues:

None

4.13 Cookies

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

If the content or function cannot be accessed without cookies [FAIL]

Examples:

A site that requires a user to login might store that login in a cookie to save the user typing in their credentials each time they visit. If cookies are unavailable, an acceptable degradation would mean that the user was prompted for a login each time they visited that page, but would browse the site without further logins from then on. An poor (and unacceptable) cookieless degradation would render a site useless by always checking for a non-existent cookie and so not letting the user past the login page.

Open issues:

None

4.14 Deficiencies

Note to BPWG:

None

Limitations of this test:

This test cannot test all existing deficiencies of devices. Rather the test limited to the author/provider being aware of common deficiencies and do something about those. This test is not intended to test pixel perfect rendering across devices, but rather focuses on usability across devices.

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

If there are common deficiencies of devices, which impinge significantly on usability of the content being offered, measures must be taken to work around those deficiencies. If not [FAIL]

Examples:

Some devices which render tables badly will wrap columns by default, if the meaning of the page is dependent on layout which does not render well on these sorts of devices then an alternative rendering should be used when that user-agent is detected.

Open issues:

None

4.15 Error Messages

Note to BPWG:

An external tester can not know what errors generate error messages, or what languages are available.

That is not quite true. At least a 404 is easy to produce and available languages can be inquired.

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Provoke a server-side error.

2. Examine the content of the error page. All the following must be true, if not [FAIL].

It must:

Examples:

None

Open issues:

The mechanism by which a context specific error message can be displayed in not clear. For example, how to display a French error message if the user was in the French section of the site.

4.16 Fonts

Note to BPWG:

The intent of the BP stated in 5.4.16.1 is that meaning should be conveyed without relying on font-related styling. However, the "Machine Test" under 5.4.16.2 fails on any presence of any font-related styling (whether it conveys meaniong or not).

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

  1. Check for presence of font-related styling, by means of the font element, or the bold, italic or underline elements, or CSS.
  2. Examine the content with CSS enabled and disabled. If the meaning is significantly different [FAIL]
  3. Examine any font-related elements present and the effect they have on the meaning of the content. If they are used to convey meaning significantly, [FAIL]

Examples:

Open issues:

None

4.17 Graphics for Spacing

Note to BPWG:

None

Limitations of this test:

This test does not rely on a particular graphics format that are being evaluated.

Differences to mobileOK Basic Tests:

This BP is partially covered by the GRAPHICS_FOR_SPACING MobileOK basic Test.

Pseudo code:

Does the content comply with GRAPHICS_FOR_SPACING MobileOK Basic Test? If not [FAIL].

View all images in a page, for example in a separate list of images, or by outlining them. For XHTML these will be included using the <img> element.
Determine visually whether any of the images do not convey information and are used for spacing. If they are [FAIL].

Note: Spacer images do not convey useful information. They are normally very small.

Examples:

Open issues:

Define unambiguously what is and is not a spacer image.

4.18 Link Target ID

Note to BPWG:

The abbreviation "ID" included in the handle of the BP, is confusing and suggests it involves the ID attribute. The test covers the use of hreflang, type and charset attributes which are not mentioned in the BP.

Limitations of this test:

This test only deals with the descriptive portion of a link

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

  1. Each link in the text is described by attributes as follows. If not, [FAIL].
    • If the target content is in a language different to that of the tested content, it is correctly marked with hreflang attribute;
    • If it is in a format other than that of the tested content, it is correctly described by the type attribute;
    • If it uses a character encoding different to that of the tested content, it is described by the charset attribute.
  2. Check that the link text (including alternative text for any non-text elements) clearly describes the effect of activating the element. If not, [FAIL]
  3. Select elements pairwise. Check that two links with same link text (including alternative text for non-text elements) and same title attribute (if provided) point to the same resource. If not, [FAIL].

(These liberally adapted from UWEM 1.0 13.1).

Examples:

Open issues:

Can contextual information can provide enough explanation for a “click here” link?

What is meant here is that there are instances were enough information is provided in the preceeding or surrounding text that a more detailed link description is not needed

4.19 Minimize Keystrokes

This test is covered by tests for AVOID_FREE_TEXT, URIS, CENTRAL_MEANING and PROVIDE_DEFAULTS.

4.20 Navbar

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examples:

Navigation bar consisting of only: Home; Up; Site map; Search...

Open issues:

4.21 Navigation

Note to BPWG:

There seems to be confusion in the MWBP document. The title refers to "consistent navigation". However, the second paragraph discusses "Intelligent grouping" and the "How to do it" section suggests providing a drill-down list based on section headings, rather than about being consistent (it doesn't say use drill-down lists consistently). The latter is more relevant to [STRUCTURE] than to this BP. What was the intent, consistent navigation across a site or just good navigation in a page? This test assumes that consistency is what must be tested.

Limitations of this test:

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

The word "sample" in the pseudo code was fitted with links which were non-sensical. Are there samples missing or was some other point being made?

Examples:

Open issues:

More examples needed.

4.22 Non-text alternatives

Note to BPWG:

It is not made explicit in the BPs whether all non-text content is covered or only images included with the <img> element.

Limitations of this test:

This test deals with images only. Other non-text content is not mentioned explicitly in the Best Practices and is beyond the scope of this test. Such content includes horizontal rules, form controls, audio, video, multimedia elements, embedded objects and frames.

Differences to mobileOK Basic Tests:

This test covers the meaning of textual alternatives. The mobileOK Basic test only covers the presence or absence of the alt attribute, not its value.

Pseudo code:

If content does not meet Basic Test NON_TEXT_ALTERNATIVES, [FAIL]

For XHTML content if any of the following UWEM 1.0 tests are not met, [FAIL].

Examples:

Open issues:

None

4.23 Objects or scripts

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test covers the functionality of documents without objects or scripts, in addition to the mobileOK Basic Tests.

Pseudo code:

If the document cannot be viewed properly without objects or scripts active or present [FAIL]

Examples:

None

Open issues:

None

4.24 Page Size Usable

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examples:

None

Open issues:

None

4.25 Page Title

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test covers the meaning of the title, in addition to the existing mobileOK Basic Tests

Pseudo code:

Examples:

None

Open issues:

None

4.26 Provide Defaults

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test covers the usefulness of the preselection, in addition to the existing mobileOK Basic Tests.

Pseudo code:

Examples:

None

Open issues:

None

4.27 Scrolling

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Is the usage of horizontal scrolling justified? If not [FAIL]

Examples:

None

Open issues:

None

4.28 Structure

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examples:

None

Open issues:

It is not clear what structural markup is and which elements are included in this definition.

4.29 Style Sheets Size

Note to BPWG:

None

Limitations of this test:

There is no guidance on this parameter and a value was picked

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examples:

None

Open issues:

4.30 Style Sheet Support

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

If content is unreadable or unintelligible with stylesheets disabled [FAIL]

Examples:

None

Open issues:

None

4.31 Suitable

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examine content to determine if, given the subject matter, it is appropriate in a mobile context. If not [FAIL]

Examples:

Open issues:

None

4.32 Tab Order

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Is the tab order of links, form controls and objects disorganized, illogical or confusing? If not [FAIL]

Examples:

None

Open issues:

None

4.33 Tables Layout

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

In addition to the mobileOK Basic Tests, this test checks usage for layout.

Pseudo code:

If tables are used in a fashion that could be achieved using CSS [FAIL]

Examples:

An image or text which are spaced and positioned with the aid of a table

Open issues:

CSS currently is itself deficient in that it does not support grid layout that auto adjusts vertically

4.34 Tables Support

Note to BPWG:

We had originally asked why this test is not included in basic tests, but the Open Issue below spells it out. How can it be determined if a device supports tables? Just look it up in UAProf or can the Checker do something here?

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

If the device does not support tables and the table element is found within the source code [FAIL]

Examples:

None

Open issues:

Is there a way to programatically determine if a device supports tables?

4.35 Testing

Note to BPWG:

This BP gives no information about what is being tested, which calls this BP into question as being useless.

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

I would also offer: Has this content been validated? If not [FAIL] and Has this content been tested with the Checker? If not [FAIL]

Examples:

None

Open issues:

None

4.36 URIs

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examples:

Entry URIs should not include information such as author, date, title. See “Cool URIs don’t change

Open issues:

It seems that at least the file extension and the subdomain ought to be machine testable?

4.37 Use of color

Note to BPWG:

None

Limitations of this test:

None

Differences to mobileOK Basic Tests:

This test does not exist in mobileOK Basic Tests.

Pseudo code:

Examples:

Open issues:

None