Test the Web Forward
Next Steps
Cambridge, MA
Advisory Committee Meeting
Plan 2013: A Value Based Proposition
- Goal: Increasing the interoperability and Reliability of the
Open Web Platform
- W3C seeks sponsors (financially and through resources) to
test the standards and build a solid Testing Infrastructure
-
Objectives:
- Comprehensive, automated, and freely available system for
testing Web products
-
Database of test results of tested products
-
Collaborative process for determining requirements and
priorities
-
Minimize industry costs using a coordinated
testing initiative
Outcome
- Almost no interest in financial support
- However, was educational and helped changed attitudes
- Got great contributions/minds to leverage: Adobe, Opera, Facebook, Mozilla, Intel
By end of 2013
- Vendor contributions: Mozilla started work on infrastructure
- Intel was ramping up its test contributions
- Tobie Langel (Facebook fellow) has revamped our approach to testing to lower the entry bar
- Adobe initiative launched Test the Web Forward and gave the brand to W3C
Past six months
- Vendor and crowd sourcing effort revived by changed of attitudes
- Contributions have doubled
- Repository (WPT) now contains 60 different test suites, but coverage varies (DOM has ~47K tests, HTML has ~142K tests)
- James Graham, Mozilla, is doing tremendous work on the infrastructure
- 3 Test the Web Forward events (Seattle, Seoul, Tokyo)
- Web&TV test task force is reevaluating its goals and priorities
Community Contributions
Infrastructure
- Goal: Modular and simple way for everyone to submit and run tests
- Single repository for tests and tools
- testhtewebforward.org is used for events and documentation
- Easier deployment (wptserve server)
- More complete tool for test execution (wptrunner)
- Automatic contribution checks (travis integration)
What's next?
- Foster the crowd sourcing effort
- It's a community effort: working groups, web developers at large
- Encourage individuals to contribute and review tests
Ideas
We don't lack ideas…
- More tests, more cleanup
- Improve wptrunner
- Improve automatic test checker
- Getting more browser implementors writing directly for WPT
- Nightly test results from browser vendors
- Reintroduce support for HTTPS
- WPT and CSS repo cooperation
- Saucelabs results?
- Implicit meta information, including monitoring coverage?
Take aways
- Back in early 2013, we faced two problems:
- We didn't know how to scale up testing due to its complexity
- Industry wanted us to do more on testing
- Testing proposition didn't get support, but the focus changed attitudes
- Improving infrastructure facilitated test contributions
- W3C Members can help by contributing and reviewing tests
testthewebforward.org