HTML5 Test Case Submission Process

We welcome all tests that help validate the HTML5 specification and lead to browser interop!

The tests in the HTML5 test suite are intended to be released under both the W3C Test Suite license and the 3-clause BSD license, so unless you are a participant in the HTML Working Group, you must give your explicit permission for us to use your contributions under these licenses.

How to License Your Contribution

The test suite is licensed under both the W3C Test Suite License and the 3-clause BSD License.

To give us permission to distribute your contribution under these two licenses, you need to fill out W3C's license grant form (unless you or your company is a member of the HTML Working Group, in which case you're covered already).

How to Submit a New Test

You can submit a new test(s) by

1. Preferably to the html5 test repository, however, if you are not a W3C member and do not have access then any public location is ok.

1. Send email to the public-html-testsuite mailing list describing the test cases you are submitting as well as giving permission for their use as described in the licensing section above, using the grant of license.

Please make sure submissions at this time target a specific feature of the current HTML5 specification. New tests are subject to the review process. We may ask you to make updates to your submitted test cases. For example you should treat all browsers the same. Your tests should not use any special logic for a specific browser.

The group has agreed that all tests need to be written in the following manner.

1. You should write the test using the javascript harness, here is an example.
   
2. If the is a visual test (not api specific) then you should create a 'ref' test, here is an example test and the ref. Note the use of -ref added to the testcase name to identify the 'reference' page.
   
3. The final option is a self-describing test, this should be used only when a javascript or a ref test is not appropriate. For example a video tests that displays video running or an audio test. Here is an example of a self-describing video test video.

HTML5 Test Repository

We use Hg (Mecurial) for version control of the tests. If you are not familiar with Hg email this list for help or search the web for how to use 'Hg'. To submit a test you normally will just have to do a 'hg add', 'hg commit', hg push https://[YOUR-W3C-ACCOUNT]@dvcs.w3.org/hg/html/.

At this time we are creating tests based on HTML5 'Features'.

Submitted tests should be 'pushed' to /html5/tests/submissions/[SUBMITTER]/[FEATURE]/.
The [SUBMITTER] is your name or company.
The [FEATURE] should be the name of the feature and not the HTML5 specification chapter.

1. Pull The Respository
   1. Install Mercurial.
   2. hg clone https://dvcs.w3.org/hg/html/
   3. see also Setting up Mercurial for W3C by Anne van Kesternen
2. Prepare a Directory in the Staging Area
   1. Go to the tests/submission directory in the staging area.
   2. If you already have a directory for yourself or your group, use it, else create one.
3. Write Your Test Case
   1. Write a test case - one example you can copy from is in resources/apisample.htm in the repository.
4. Commit your test case
   1. Set your username in your global mercurial configuration.
   2. In the [paths] section of the .hg/hgrc file inside the top level of your clone of the repository, add the line: default-push = https://[USERNAME]@dvcs.w3.org/hg/html/
   3. Use hg add to add the new files. (See hg help add. Or, if you want to be fancy, you can use hg addremove; see hg help addremove.)
   4. Run hg diff to see what you're going to commit and then use hg commit (with a good commit message, the -m option). (For more information, see hg help commit and hg help diff.)
   5. Run hg outgoing to see what you're about to push and then hg push to push to the repository.
      1. If you're told that the push creates new remote heads, you need to merge with them. At this point, you have two options:
         1. hg pull, hg merge, and hg commit -m"Merge.", and then try pushing again.
         2. rebase your changesets either using the rebase extension or using mq

See Also

• Test Approval Process