Building QA Experience in the W3C

Speed of the W3C

• First formed as a reaction to vendors getting out in front of the specs
• Very successful: W3C is seen as the authority on HTML, and not just HTML
• Now specs have gotten out in front of implementations
• CSS 3 Working group chugging along...fully CSS 1 conformant implementations are still relatively small percentage of deployed browsers

Well-resourced W3C QA Activity Needed

• Create economic incentive for companies to build conformant products
• Actively lobby for conformance requirements in contracts
• W3C Recommendations need to be authored in such a way that "the letter of the law" is clear; QA Activity can police that
• Working groups need to take seriously every new feature - they could be held to implement them (may get quiet vendors to speak up)
• Give recommendations "teeth"
• Of course, vendors will need tools to get conformant....

Recommendations - QA Review Group

• Reviews specifications in the same spirit as the WAI and i18n groups, but focusing on testability and completeness
• Submit guidelines for specification writers that could become part of the mandatory process for W3C Recommendations.
• Design watertight definition of the exit criteria for Candidate Recommendation
• Different definitions are needed for modules intended for incorporation in many specifications, and profiles which are intended to define document types.

Recommendations - Certification program

• Potential certification authorities apply to the W3C for approval to certify implementations of W3C Recommendations (scoped to itemized Recs)
• Appeals process to ensure that the W3C has final appellate authority
• W3C would need ability to revoke specific certification stamps as well as certification authorities
• Certification stamps should never be permanent, only for fixed durations (e.g. one year).

Recommendation - Certified Recommendation Phase

• Higher step for specs with long-term experience and proven robustness and applicability
• No sooner than 18 months after the W3C Recommendation is approved
• CertRecs could pull features from the preceding W3C Recs that don't have two certified implementations
• Participating vendors hoping to get certified would be more careful about adding features
• More vendors would be motivated to get involved to ensure that poorly designed features don't make it into the specifications
• Vendors motivated to get certified so that pet features have higher odds of making CertRec

Testcase/testplan markup language

• Would be great to standardize
• W3C may be the right place to do it
• Being a heavy customer makes the organization qualified
• Can be considered a higher level activity (though could be at same level as XSLT)
• Cautionary note: this is a tool toward a goal - shouldn't get so busy working on markup language that other QA needs don't get addressed

(more dead horse beating) More formal issue tracking on suites and specifications

• Otherwise, those that submit issues have to have deep knowledge of the process (know their rights)
• Auditability

Vendors must support backwards compatibility with their old products

• "The perfect is the enemy of the good"
• Invalid documents may be allowed by otherwise good user agents

• Often authored using plain text editor and non-validating user agent
• Declaration Of Independence should still be viewable even though there are grammatical errors by today's standards.

...but there are tools that can be used

• Specification features can help with this process - strong document versioning very handy tool

• QA Working Group should develop W3C Recommendation which recommends bad document handling behavior, using these points and Common User Agent Problems note (http://www.w3.org/TR/cuap) as starting point