Web Testing Activity Proposal (July 2011)

This Activity Proposal is Public.

The structure of this Activity Proposal follows Section 5.6 of the W3C Process Document.

Executive Summary

For years, W3C has been testing technologies in various Working Groups. Each specification follows its own methods for testing the underlying technology and no coordination between Working Groups is happening for testing methods. However technologies like HTML, CSS, or APIs need to be tested in the same user agents, combining some of them sometimes. Efforts have been successful in the past in the Mobile Web Initiative but the prevalence of the Web platform across various devices is increasing the needs to ensure better coordination and interoperability between Web technologies. The work of the Quality Assurance activity produced guidelines and checklist for specifications. The role of the Web Testing activity is to develop the testing mechanisms, outreach activities, and collateral materials for testing Web technologies.

The Web Testing Activity would be part of the Interaction Domain.

Current Status and Position

What has been done already?

For the past 10 years, W3C has been developing test suites for the purpose of demonstrating interoperable implementations of each specification when requesting the Director to approve a Proposed Recommendation transition. For example, one of the most recent specifications, CSS 2.1, has a test suite of around 9000 tests. Each Working Group has been developing test suites and generating test results in their own ways. Using ad-hoc processes and testing methods, most of them did not attempt to reuse existing methods or approaches. In the past, the Mobile Web Initiative did produce a series of integrated test suites for the Mobile platform and this kind of effort needs to be generalized across W3C Working Groups.

Why is a new Activity being started?

With the new wave of developments in the Open Web Platform, the increase diversity in devices, and the increase in demands for real interoperability between the technologies, it is important for W3C to step up its efforts and coordinate the energies in testing new technologies like HTML5, CSS3, ARIA, etc. There is a strong need for Web technologies to work out "out of the box" on any devices (Desktop, TV, mobile, tablet) and reliably in more consumer oriented use cases, including accessibility.

The intent of the acitivty is to enable testing of the various facets of Web agents: rendering, scripting, animation, performance, user interaction, or integration with platform APIs, such as Accessiblity APIs.

What is the market within the area of the proposal? Who or what group wants this (providers, users, etc.)?

Several groups have been pressing for this:

Developers are always demanding to increase the interoperability between user agents.
Content Providers
Content Providers would like to significantly reduce the cost of content development by an increase in interoperability of Web software.
Software Providers
Providers will get access to test suites to ensure that their products correctly implement the W3C technologies required for Web access. Using the test suites, they can communicate their level of standards-compatibility effectively to the consumers/users of their software. They will be able to harmonize and share the cost of test case development with others.
Web Users
Users will get a more interoperable and stable experience when accessing the Web.
Operators will benefit by a lower the cost of deployment and a reduced costs for customer support.
Device Manufacturers
Device manufacturers, such as CE/TV manufacturers, are interested in reducing production costs and supports for Web softwares and applications.

Are members of this community part of W3C now?

Many of them are. Many implementors are already Members.

Will they join the effort?

The testing project, which started earlier this year, already gather several implementers and organizations interested in the effort, especially in the web browser, mobile and accessibility areas.

Who or what currently exists in the market?

Each user agent implementation has already its own way of testing their platform. Several Working Groups have been developing test suites and producing test results. The CSS Working Group is probably the most advanced group in terms of managing tests for example. Several test methods have been developed in the past few years, such as Mozilla reftests or Selenium.

Are there external groups dealing with Web Testing?

Several efforts have been ongoing in this area but no coordination has been done.

Is the market mature/growing/developing a niche?

The interest in new technologies like HTML5 or CSS3 has increased the demand to ensure proper interoperability.

Are there existing implementations?

As mentioned earlier, several W3C Working Groups have been developing test suites in the past.

What is the scope of the work

See the respective charters.

What are initial timetables?

See the respective charters.

Is there a window of opportunity that cannot be missed?

Several specifications are scheduled to be completed in the next 12 months or 2 years. The HTML5 specification is scheduled to become a W3C Recommendation by 2014, with the first initial release of the test suite by early 2012.

How might a potential Recommendation interact and overlap with existing international standards and Recommendations?

No overlap.

What organizations are likely to be affected by potential overlap?


Should new groups be created?

Yes, see the respective charters.

How should this area be coordinated with related W3C Activities?

The proposed Groups would be coordinated via the chairs and team contacts participation in the Hypertext Coordination group.

Proposal for Membership Consideration

This proposal is for consideration by the Membership according to section 5.6 of the Process Document.

Charter Overview

Resource Statement

The resource requirements for this Activity include:

The proposed total staff FTE for this Activity is 1.4 FTE, including ressources from the Testing project.

Intellectual Property

See the section on Patent Disclosures in the WG charter.


The Activity will run for two years, until 30 June 2013.


The Lead for the Testing Activity is Michael Smith (W3C) <mike@w3.org>

$Id: activity-proposal.html,v 1.7 2011/07/22 14:09:40 plehegar Exp $