Testing

From HTML WG Wiki

Revision as of 09:43, 28 October 2012

This page explains how to participate in the W3C HTML5 test suite effort. This effort is managed by the HTML5 test suite task force, part of the HTML Working Group.

There are a number of ways to participate. We welcome contributions of new test cases. The more tests we have, the faster we can improve interoperability among Web browsers.

If you wish to join the test suite task force, contact Philippe Le Hégaret <plh@w3.org>.

The tests live in a Mercurial repository which you can check out by running

hg clone https://dvcs.w3.org/hg/html

Status

Kris presented the latest status at the May 2012 HTML f2f meeting.

As of October 3rd 2012 the HTML5 test suite contains 11,898 Approved tests! A text file also exists that contains links to all of the tests in the test suite.

HTML5 Features Included In Test Suite

• audio
• atob/btoa
• canvas
• data-*
• document.write
• editing
• events
• MathML parsing
• named character references
• SVG in HTML (aka foreign content)
• parsing
• video
• XHTML5

Discussion

• The task force mailing list is public-html-testsuite@w3.org (archive)
• #Conference Calls (held every two weeks)

Running the test suite

• HTML5 Test Suite Conformance Results. Results not yet significant. Test reports are only accepted from user agent (e.g. Web browsers) implementors at this time. Given that this is work in progress, it is not our priority to update those results.
• Run the tests

Approved tests

• Approved HTML tests.
• These have been approved by this task force (not the full HTML WG) according to the task force's HTML5 Test Case Review and Approval Process.

Writing Tests

See the Guidelines for authoring tests.

New tests

• Existing HTML5 tests. The have been submitted in proper format but have not yet been approved by the task force.
• Ideas for new tests. These are ideas but no tests have been submitted yet.
• Submit a test case and learn about the repository.

Bug Reports

• Report a test suite bug
• We track bugs in BugZilla

Areas for improvement

Beyond the question of the number of tests:

• the coverage of the HTML5 features
• the classification per feature of those tests
• the testing harness, to produce as many automatic tests as possible

Conference Calls

• The group meets every two weeks, mainly on irc.w3.org:6665, #htmlt
• You may find some of us on the phone at the same time
  • Time 16:00-17:00 UTC (11:00am-12:00pm Boston local)
  • Zakim Bridge +1.617.761.6200, conference 48658

Upcoming meetings

July 17th 31st (no meeting on 3rd Holiday)
August 14th 28th
September 11th 25th
October 9th 23rd
November 6th 20th
December 4th 18th