Proposal to Revise the QAF Document Family

QA Chairs have looked at recent implementation feedback about the QA Framework (QAF), and believe that some changes could make it more user-friendly, more helpful, more welcomed by the Working Groups, and perceived as less burdensome. A wealth of good material has been developed, and these benefits can probably be achieved mostly with reorientation and reorganization.

The QA Chairs have taken a look at the status, progress, and W3C response to the QA Framework (QAF) document family. Based on

• recent W3C input during the CR period of Operational Guidelines (OpsGL) and Specification Guidelines (SpecGL),
• resources available to the QA Working Group (QAWG) for development of the QAF,

it becomes clear that some changes are needed to the content, overall organization, and future development plan for the QAF.

Specifically, we in QA must adjust to two main problems:

• while some QA-friendly groups have received the current QAF positively, there is also (probably more) indifference and even hostility to the QAF in its current form;
• and, the resources that QAWG has available to apply to the QAF are well short of what would be needed to push ahead with the current plan for the QAF.

This proposal is driven by these two big issues, and some smaller ones as well. There is a general consensus (amongst qa-chairs) about needed high-level changes to the QAF family. Lots of specific details are still undefined, especially about the size and organization of the individual pieces of the QAF puzzle, and how they are developed.

The current QAF family

The current QAF family consists of an introduction and three normative components, each of which have associated informative components:

• QA Framework: Introduction (status: WG Note)
• QA Framework: Operational Guidelines (status: at CR)
  • Plus: OpsET & associated templates, checklist, ICS-checklist
• QA Framework: Specification Guidelines (status: at CR)
  • Plus: SpecET, checklist, ICS-checklist
• QA Framework: Test Guidelines (status: at WD)
  • Plus: checklist, ICS-checklist

(Note. OpsET = QA Framework: Operational Examples & Techniques; SpecET = QA Framework: Operational Examples & Techniques)

Major QAF issues

The still-growing QAF CR issues list is at:

• http://www.w3.org/QA/WG/2004/02/cr-issues

The specific conceptual issues that potentially affect the scope & goals, content, architecture, look & feel, and future processing of the QAF are: CR-1 through CR-6; CR35, CR-37 through CR-40.

Changes to the QAF components

Proposal for new-QAF:

1. Discontinue the OpsGL components (OpsGL, OpsET, and associated templates) as a normative part of the QAF family.
2. Combine the best parts of QAF-intro and the OpsGL/ET components (including helpful templates) into a user-friendly, non-normative "QA Handbook".
   • Consider changing the language/guideline/document model of surviving OpsGL guidance along the lines of WCAG20 (see also current WCAG20 internal draft), or Web Architecture, or ...
   • Advocate for some minimal Charter-commitments requirements about test materials to be added to W3C Process document.
3. Continue to progress a much-simplified SpecGL/ET as a set of normative Guidelines, with potential changes such as:
   • reduce the number of CPs and/or GLs (e.g., perhaps remove P2 and P3 and publish them separately, for now, as a companion "Spec. Additional Recommended Practices").
   • Keep (probably) the DoV concept as an organizing model, but clarify it and simplify its implementation-related details.
   • Concurrently, fully develop the appropriate SpecET parts, including templates, tools, implementation aids, etc.
   • Solve all remaining relevant open SpecGL/ET issues.
4. TestGL/ET: progress in background for near future, commensurate with resources (people) who are interested in and willing to work on it. Guture coordinated publication with new-QAF as a goal, or not? Evaluate future coordination with other bits after some trial period with this approach.

Open questions

1. The above proposal defines "what?". I.e., what will be in the final assemblage that we call QAF. It doesn't define "how?", and doesn't define any low-level or detailed structure. There are at least two interesting possibilities:
   • new-QAF is a collection of traditional documents, similar to the current ones; but they are simplified, streamlined, better inter-related, and completed.
   • Karl has suggested an intriguing alternate model under the title "QAF to FAQ", with several interesting features:
     • don't try to maintain our traditional 3-component partition, because the inter-dependencies end up complicating things too much;
     • instead develop a larger number of smaller pieces... each one is like a Q in a FAQ, and addresses a single topic. The sum total of all of 'em spans all of our current most important QAF topics. Each topic is developed completely with guidelines and checkpoints (if appropriate), tutorial, techniques, tools, examples, possible markup and validators, etc.
     • some example topics: (spec) scope definition; classes of products; extensions; test assertions; test case; charter; conformance section; etc...
     • this develop model could be tackled progressively, one Q/topic at a time.
2. In either of those development models, there is the question of how much coordinated publication we need, versus how much we might be able to sequence the pieces.
3. There are questions of language/style of th epieces. RFC2119-based commandments ("blah MUST blah")? Or imperative-voice language? Or test assertions? The answer may be different for different pieces, e.g., the QA Handbook might use gentler language, and SpecGL might use RFC2119 language.
4. The above-proposed lack of coordination between TestGL and other bits -- does this run afoul of the big issue of JC (all parts must advance synchronously)? Or does the issue substantially go away with the proposed changes above (esp. #1 - #3).
5. ...there are more, but out of time for now...

Schedule considerations

Without any changes to QAF structure/content/organization, we were guess-timating that a next coordinated publication -- a 2nd Last Call -- could happen in Oct-Nov 2004 time frame. This guess was based on working on SpecGL/ET first, then OpsGL/ET, and concurrently bringing TestGL to a publishable 1st Last Call.

For the proposed QAF revision, there are too many open questions to estimate schedule. However, it certainly (we believe) would not require more time and effort. Almost certainly, it should be achievable with less time and effort. Therefore Oct-Nov might be a generous upper limit for new-QAF: 2nd Last Call of remaining normative SpecGL/ET bits; the QAF Handbook; ??TestGL 1st LC??.