Testing the Open Web Platform

Tobie Langel
Testing Lead • W3C
@tobie

Who am I?

Some (hopefully) relevant background about me

  • Originally a self taught front-end programmer.
  • Co-maintainer of the Prototype JavaScript Framework.
  • W3C AC Rep for Facebook.
  • Coremob CG instigator, initial chair & spec editor.
  • Tech lead and ad interim PM for Ringmark (rng.io).
  • W3C Testing lead & Fellow (on behalf of Facebook).

Current State of browser testing

  1. Grass roots efforts
  2. WG test suites
  3. Browser vendor test suites

Grass roots efforts

Most testing efforts so far (Can I use..., Ringmark, HTML5 test, etc.) focus on feature detection.

W3C WG test suites

  • Focus on proving that the spec can be implemented, not on testing the implementations.
  • Many specs still don't have one.

Browser vendor test suites

  • Great but incomplete contributions from vendors.
  • Little sharing of tests, if any.
  • No common test framework until recently, so lots of legacy content.
  • Not organized per specs, hard to know test coverage.
  • Not always open-source.

Leads to "meh" interoperability.

Death by a thousand paper cuts.

New W3C Testing Effort

Focuses on thorough testing of user agents (aka browsers) to improve interoperability.

Scope

Depth: correct implementation of specs by user agents.

Breadth: union of the Coremob and Web+TV profiles.

Quite a big challenge in itself.

Who already wrote W3C tests?

Plan

  1. Infrastructure
  2. Test development

Infrastructure

  1. Move all test suites to a single repository on GitHub:
    github.com/w3c/web-platform-tests
  2. Streamline test contribution and review processes.
  3. Build tools to simplify test review.
  4. Consolidate and expand documentation.
  5. Overhaul test framework so it's easy to run tests and gather results.
  6. Collect test results and make them easily accessible to developers.
  7. Make it easy for browser vendors to run the test suite as part of their build system

Test development

  • Browser vendor contributions
  • Crowd-sourcing (notably through Test the Web Forward events)
  • Out-sourcing
  • W3C fellowships

How you can help

  • Learn how to write and review tests on testthewebforward.org.
  • Contribute and review tests.
  • Build tools on top of the data we're going to expose that help developers.
  • Improve existing tools or tools we're building. Hey, they're open-source!
  • Get your employer excited about the opportunity and help fund the effort or contribute fellows.

Thanks. Questions?