I18N Charter review against "QA Framework: Operational Guidelines"

I have used this version: http://www.w3.org/TR/2003/CR-qaframe-ops-20030912/qaframe-ops-ics

The review has been made by Karl Dubost.

Last version of this document : $Date: 2005/06/23 13:03:02 $

Checklist table

Degree A Conformance

To be A-conformant with the guidelines, the following Priority 1 checkpoints must be fulfilled:

Nbr	Checkpoint	No
Guideline 1. Commit to Quality Assurance in Working Group activities.
1.1	Define QA commitment levels for operations, specifications, and test materials.	You haven't defined your level of QA Commitment. i.e. if you intend to be QA compliant Level A, Level AA, or level AAA with regards to Ops. For each specification, you have to define also the level of commitment against QA Spec GL. This can be done during a rechartering or in a recorded consensus like minutes of a meeting. See the Examples and Techniques for this CP.
1.4	Enumerate QA deliverables and expected milestones.	For each of your deliverables, you haven't defined the QA deliverables. Though you have defined Some use cases for the Web services which are the QA materials per se for Web Services. Just a line or two to add for each of your documents in your charter.
Guideline 2. Commit to resource level for Working Group QA activities.
2.1	Address where and how conformance test materials will be produced.	You do not say who and how the test materials will be produced.
2.2	Address QA staffing commitments.	You do not make explicit the level of commitment you need to create the QA materials
2.3	Request allocation of QA resources to the Working Group.	You do not explicit what are your needs to create QA Materials, which will be helpful if you need external help or people interested by committing time to your WG.
Guideline 3. Synchronize QA activities with the specification milestones.
3.2	Support specification versioning/errata in QA deliverables.	As you don't define your test materials, you failed this CP which is about synchronizing Test materials with publication of deliverables.
Guideline 4. Define the QA process.
4.1	Appoint a QA moderator.	You can do it right away. Nominate a QA moderator, a person in charge of all the aspects of QA. It's not about doing the job, but coordinating it.
4.3	Produce the QA Process Document.	You don't have a QA process document, see the examples and techniques.
Guideline 5. Plan test materials development.
5.2	Ensure test materials are documented and usable for their intended purposes.	We didn't find documentation for your Test Materials
5.4	Address license terms for submitted test materials.	We didn't find license terms for your Test Materials
Guideline 6. Plan test materials publication.
6.1	Ensure a suitable repository location for test materials.	It seems not clear if there's a common repository for your test materials. You could create for example something like www.w3.org/International/Test/ where your test materials belong organized by specifications, deliverables etc.
6.2	Define the licenses applicable to published test materials.	No defined license
6.4	Provide a conformance verification disclaimer with the test materials.	We haven't found a disclaimer.
Guideline 7. Plan the transfer of test materials to W3C if needed.
7.2	Identify sufficient staff resources to meet the needs of any transferred test materials.	We haven't found a document explaining how people could contribute and under which terms.
7.3	For any transferred test materials, resolve all IPR issues with the external party that produced the test materials.	Not given
Guideline 8. Plan for test materials maintenance.
8.2	Specify a test materials update procedure to track new specification versions/errata.	No documentation explaining the maintenance of the materials

We decided to stop the review here, to not add burden on your work. We will try to go through the level A, and as soon it's done, we might push together for the next level.

Degree AA Conformance

To be AA-conformant with the guidelines, the following Priority 2 checkpoints must be fulfilled (in addition to the above Priority 1 checkpoints):

Nbr	Checkpoint	Yes	No	N/A
Guideline 1. Commit to Quality Assurance in Working Group activities.
1.2	Commit to test materials.
1.5	Define QA criteria for Recommendation-track advancement.
Guideline 3. Synchronize QA activities with the specification milestones.
3.1	Synchronize the publication of QA deliverables and the specification's drafts.
Guideline 4. Define the QA process.
4.2	Appoint a QA task force.
4.4	Specify means for QA-related communication.
Guideline 5. Plan test materials development.
5.1	Define a framework for test materials development.
5.3	Define a contribution process.
5.5	Define review procedures for submitted test materials.
Guideline 6. Plan test materials publication.
6.3	Describe how and where the test materials will be published.
6.5	Promote testing and the publication of test results.
Guideline 7. Plan the transfer of test materials to W3C if needed.
7.1	Perform a quality assessment of any test materials that are candidates for transfer.
Guideline 8. Plan for test materials maintenance.
8.3	Identify a procedure for test validity appeals.

Degree AAA Conformance

To be AAA-conformant with the guidelines, the following Priority 3 checkpoints must be fulfilled (in addition to the above Priority 1 and Priority 2 checkpoints):

Nbr	Checkpoint	Yes	No	N/A
Guideline 1. Commit to Quality Assurance in Working Group activities.
1.3	Commit to complete test materials.
Guideline 4. Define the QA process.
4.5	Define branding policy details.
Guideline 8. Plan for test materials maintenance.
8.1	Provide for the long-term maintenance of the contribution and review procedures.

References

QAF-OPS: QA Framework: Operational Guidelines, L. Henderson, D. Hazaël-Massieux, L. Rosenthal, Eds., W3C Candidate Recommendation, September 2003, available at http://www.w3.org/TR/2003/CR-qaframe-ops-20030912/.