Difference between revisions of "Webapps/Submitting tests"

From W3C Wiki
Jump to: navigation, search
(Initial version of GitHub testing doc)
Line 1: Line 1:
= Submitting tests =
==Test Licensing==
==Test Licensing==

Revision as of 17:53, 24 January 2013

Test Licensing

All of the tests in WebApps' test suites are intended to be released under both the W3C Test Suite license and the 3-clause BSD license.

If a Contributor is a member of WebApps, there is no need to complete any license forms. However, if a Contributor is not a member of WebApps, the Contributor must grant permission to distribute their contribution under these two licenses, and this requires completing the W3C's license grant form.

Test Repository Structure

The WG uses the W3C's html-testsuite repository on GitHub for its tests.

The structure of the repository, is:

  • .../[specification]/
  • .../[specification]/[section]/test.htm


  • specification is a short name of a specification e.g. XMLHttpRequest, WebWorkers
  • section is up to three level deep sections of the spec where the test belongs. This is not used in some specifications.

If the test folder for the specification you're writing a test for uses the sections, you should too.

Test Case Requirements

The group has agreed that all tests need to be written in the following manner:

  1. Tests should use the JavaScript harness described in Harness
  2. If a test is a visual test (not API specific) a Reference Test (aka RefTest) must be created. Note that -ref must be added to the testcase name to identify the reference page. For some information on RefTests, see:
    1. RefTests by the CSS WG
    2. Creating reftest-based unit tests by Mozilla
  3. The final option is a self-describing test and should be used only when a JavaScript or a RefTest is not appropriate.

How to Submit a New Test - The Short Version

The way to contribute is just the usual GitHub way:

  1. Fork the GitHub repository (and make sure you're still relatively in sync with it if you forked a while ago)
  2. Create a branch for your changes, git checkout -b submission/your-name
  3. Make your changes
  4. Push that to your GitHub repository
  5. Send in a pull request based on the above

You might get asked about completing W3C's license grant form if your employer is not already a member and you have not done so before. Someone will review your test and you might have to do some changes based on that review.

You are also welcome to ping the public-webapps-testsuite mailing list for any reason. Such as explicitly asking for review, discuss or announce your submission.

How to Submit New Tests - The Long Version

GitHub itself has a nice long help document explaining how to contribute to projects.

If you are brand new to Git and GitHub, you should probably read the getting started guide from the start. It's nice and friendly. The GitHub help pages has lots of other great information and guides. And for everything else you are welcome to send an email to the WG's public-webapps-testsuite e-mail list (archive).