Issues regarding the QA Framework (QAF) Candidate Recommendation (CR) documents and associated Notes produced by the QA Working Group (QAWG) should be reported to the QAWG per the instructions in the "Status..." sections of those documents: send mail to www-qa@w3.org (public archives).
Comments on this issues list should be sent to the www-qa@w3.org mailing list (Archives).
In this document,
num | Status | Spec | Topic | Class | Date | Title |
---|---|---|---|---|---|---|
CR-14 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | Operational Examples & Techniques contains no examples, only case studies. |
CR-15 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | Clarify what is meant by "New Working Groups" in Guideline 1. |
CR-16 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | Checklists should include conformance requirements. |
CR-17 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | A, AA, AAA should be defined before their first use. |
CR-18 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | The single HTML file should be the normative version of OpsGL. |
CR-19 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | CP1.4 does not define "QA Deliverables & Milestones". |
CR-20 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | OpsGL should not require so much planning and assignment of resources prior to chartering a WG. |
CR-21 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | The Rationale of CP2.1 is inconsistent with the statement of the CP and its Conformance Requirements. |
CR-22 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | The phrase "where-and-how-plan" is used in ConfReqs of CP2.2 but not defined anywhere. |
CR-23 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | The scope of QA Moderator position is ill-defined and not distinguisable from "test development manager". |
CR-24 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | CP5.1 is hard to distinguish from CP2.1 & CP2.2, and contains inconsistencies. |
CR-25 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | QAF-OPS and OPS-EXTECH docs out of synch. |
CR-26 | Active | OpsGuide | OpsGuide | Substantive | 2004-02-09 | CP6.2 and CP5.4 should be merged. |
CR-13 | Active | OpsGuide | OpsGuide | Editorial | 2004-02-09 | The normative version of OpsGL should be a single HTML file. |
CR-1 | Active | QAF | QAF | Substantive | 2004-02-09 | QAF is too big and too expensive. |
CR-2 | Active | QAF | QAF | Substantive | 2004-02-09 | Comprehensive conformance test materials versus usable and useful test materials. |
CR-3 | Active | QAF | QAF | Substantive | 2004-02-09 | Rec-track documents should not constrain the WGs or their specifications. |
CR-4 | Active | QAF | QAF | Substantive | 2004-02-09 | Conformance with QAF is not the only way to meet reasonable quality goals. |
CR-5 | Active | QAF | QAF | Substantive | 2004-02-09 | QAF should be a flexible, user-friendly set of resources, referenced but not mandated by W3C Process. |
CR-6 | Active | QAF | QAF | Substantive | 2004-02-09 | QAF-Intro needs more prominence as QAF starting point, and should maybe absorb OpsGL. |
CR-7 | Active | QAF | QAF | Substantive | 2004-02-09 | The use of MUST in conformance requirements is too strong. |
CR-8 | Active | QAF | QAF | Substantive | 2004-02-09 | Put all QA-related terminology into QA Glossary. |
CR-9 | Active | QAF | QAF | Substantive | 2004-02-09 | All common boiler-plate sections in the QAF specs should be moved into Intro. |
CR-11 | Active | QAF | QAF | Substantive | 2004-02-09 | Add conformance requirements to checklists. |
CR-12 | Active | QAF | QAF | Substantive | 2004-02-09 | Confirm consistency of overlapping information in QAF document families. |
CR-32 | Active | QAF | QAF | Substantive | 2004-02-11 | QAF contravenes W3M resolution that Rec-track specs should not coerce the WGs. |
CR-33 | Active | QAF | QAF | Substantive | 2004-02-11 | QAWG failed to formally address Last Call comment before requesting CR transition. |
CR-34 | Active | QAF | QAF | Substantive | 2004-02-11 | QAWG failed to list open issues when advancing to CR. |
CR-35 | Active | QAF | QAF | Substantive | 2004-02-11 | QAF is only a "test development framework", therefore it is misnamed. |
CR-36 | Active | QAF | QAF | Substantive | 2004-02-11 | QAWG has failed to meet the quality commitments in its charter. |
CR-37 | Active | QAF | QAF | Substantive | 2004-02-11 | Make QAF informative, and represent its key normative content with small changes to W3C Process Document. |
CR-38 | Active | QAF | QAF | Substantive | 2004-02-11 | QAF violates RFC2119. |
CR-39 | Active | QAF | QAF | Substantive | 2004-02-11 | All parts of QAF, including supporting informative & resource references, must advance synchronously. |
CR-40 | Active | QAF | QAF | Substantive | 2004-02-11 | 2nd Last Call needed, with stricter advancement criteria. |
CR-41 | Active | QAF | QAF | Substantive | 2004-02-11 | QAWG & QAF have many breaches of W3C Process, pubrules, etc. |
CR-10 | Active | QAF | QAF | Editorial | 2004-02-09 | Use consistent document abbreviations throughout the framework. |
CR-29 | Active | SpecGL | SpecGL | Substantive | 2004-02-11 | Detectable errors for instance data CoP, and error reporting requirements for associated processor (software) CoP. |
CR-42 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Test Assertions -- definitions |
CR-43 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Test Assertions -- how and where? |
CR-44 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Test Assertions -- should SpecGL require them? |
CR-45 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | SpecGL & SpecET inconsistencies about CoP and scope requirements. |
CR-46 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | CP2.2 is hard to implement. |
CR-47 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Rationale of CP2.3 (spec. category) is unconvincing, and it is unclear how to satisfy the CP. |
CR-48 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Error in Conformance Requirements of CP3.2 (profile min. rqts.)? |
CR-49 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | The statement of CP5.1 (discretionary items) is unrelated to the given Conformance Requirements. |
CR-50 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Redundant requirement in CP5.3 (number of disc. items/choices). |
CR-51 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Inconsistencies in CP5.4 (consistent handling of discretionary choices). |
CR-52 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Problems in CP5.5 (relationship of discretionary items with other DoV). |
CR-53 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Error in Conformance Requirements of CP6.5 (mitigate extensions)? |
CR-54 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | CP8.1 (universal reqts for min functionality) is difficult to implement. |
CR-55 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Define "conformance concepts" in CP8.2. |
CR-56 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | How do "conformance designations" (CP8.5) relate to "conformance concepts" (CP8.2)? |
CR-57 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | The meaning and purpose of "conformance disclaimer" are unclear in CP9.2. |
CR-58 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | Clarify what comprises "list" of Test Assertions. |
CR-59 | Active | SpecGL | SpecGL | Substantive | 2004-02-15 | ...~20 substantive SpecGL issues from JC will start here... |
CR-27 | Active | SpecGL | SpecGL | Editorial | 2004-02-11 | Use of abbreviations in ConfReqs makes them much harder to read. |
CR-28 | Active | SpecGL | SpecGL | Editorial | 2004-02-11 | The styling of checkpoints makes it difficult to recognize checkpoint headings. |
CR-30 | Active | SpecGL | SpecGL | Editorial | 2004-02-11 | SpecGL does not define "testable" or "measurable" but uses them in ConfReqs. |
CR-31 | Active | SpecGL | SpecGL | Editorial | 2004-02-11 | SpecGL does not define what is an "implementation". |
num | Spec | Date | Topic | Class | Status | Raised By | Owner | |
---|---|---|---|---|---|---|---|---|
CR-1 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | Lofton Henderson | |
Title: QAF is too big and too expensive. | ||||||||
Description: Three interrelated sub-issues can be identified
here:
Related CR issues: 20. |
||||||||
Proposal: [Originator] No specific proposals are associated directly to
these high-level issues by originator.
[LH] The proposed revisions to the QAF would mitigate some or all of these concerns. |
||||||||
Resolution: ...tbd... | ||||||||
CR-2 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | Lofton Henderson | |
Title: Comprehensive conformance test materials versus usable and useful test materials. | ||||||||
Description:
WebOnt opted for the first, to good effect; and, believes that the thoroughness and extensive procedures implied by following QAF for the second would cost a lot, without commensurate return of value. The scope of this issue might appear to pertain narrowly to TestGL. However, the specificity of "conformance test materials" appears in Intro and in OpsGL requirements as well (although that specificity is inconsistent -- there are some references just to "test materials".) Therefore the issue needs to be resolved consistently throughout synchronized QAF components. |
||||||||
Proposal: [Originator] (Implicit) QAF should recognize WebOnt
"usable and useful" approach as satisfying TM needs of the WG.
[LH] While the proposed revisions to the QAF would not directly address these concerns, nevertheless this is an issue that would have to be worked out during development of the details of the proposal (if it were accepted). |
||||||||
Resolution: ...tbd... | ||||||||
CR-3 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | Dominique Hazaël-Massieux | |
Title: Rec-track documents should not constrain the WGs or their specifications. | ||||||||
Description:
It is inappropriate for QAWG, in the QAF documents, to mandate conformance of WGs to procedures (OpsGL) and conformance of WG specifications (SpecGL). This is effectively the same issue as CR-32, which contains somewhat more detail. |
||||||||
Proposal: [Originator] Don't mandate conformance of
WG procedures and specifications within QAF.
[LH] Originator misunderstands what QAF actually says. QAF gives requirements that must be met for the conformance target -- a WG (OpsGL) or a specification (SpecGL) -- to conform to the stated guidelines. But QAF does not say that the objects must conform, which would effectively be defining W3C policy. This could be clarified and emphasized more in QAF. |
||||||||
Resolution: ...tbd... | ||||||||
CR-4 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | Lofton Henderson | |
Title: Conformance with QAF is not the only way to meet reasonable quality goals. | ||||||||
Description:
Specifically, WebOnt believes that it accomplished its stated quality goals without conforming to the procedural requirements of OpsGL, and believes that its success is due to its freedom to choose the most appropriate approaches. |
||||||||
Proposal: [Originator] No specific associated proposal.
[LH] The proposed revisions to the QAF would address this concern. |
||||||||
Resolution: ...tbd... | ||||||||
CR-5 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | Lofton Henderson | |
Title: QAF should be a flexible, user-friendly set of resources, referenced but not mandated by W3C Process. | ||||||||
Description:
Instead of a monolithic set of terse, conformance-driven documents, WebOnt would like to see QAF become a collection of useful and helpful tools, templates and guidelines. This issue concerns the general philosopy or nature of QAF. Specific details maybe found in related issues CR-6 through CR-14. |
||||||||
Proposal: [Originator] Specific associated proposal are found in
issues CR-6 through CR-14.
[LH] The proposed revisions to the QAF would address this concern. |
||||||||
Resolution: ...tbd... | ||||||||
CR-6 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | Lofton Henderson | |
Title: QAF-Intro needs more prominence as QAF starting point, and should maybe absorb OpsGL. | ||||||||
Description:
Emphasize QAF-Intro as starting point into QAF. Consider removing OpsGL as separate component of QAF, and using Intro "Primer" to serve same role. (This issue is a proposed implementation detail of CR-5.) |
||||||||
Proposal: [Originator] As proposed in comment.
[LH] The proposed revisions to the QAF would address this concern. |
||||||||
Resolution: ...tbd... | ||||||||
CR-7 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | unassigned | |
Title: The use of MUST in conformance requirements is too strong. | ||||||||
Description:
MUST ought to be weakened to SHOULD (at least), or MAY. Flexibility is needed by the WGs, especially in situations where timeliness is critical. (This issue is a proposed implementation detail of CR-5.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-8 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | unassigned | |
Title: Put all QA-related terminology into QA Glossary. | ||||||||
Description:
All QA-specific terms such as Commitment Levels, Priority Levels, etc should be in the QAF Glossary. (This issue is a proposed implementation detail of CR-5.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-9 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | unassigned | |
Title: All common boiler-plate sections in the QAF specs should be moved into Intro. | ||||||||
Description:
Consider moving boiler plate sections to Intro. In any case move term definitions to a place prior to their use in base documents. (This issue is a proposed implementation detail of CR-5.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-10 | QAF | 2004-02-09 | QAF | Editorial | Active | Jim Hendler | unassigned | |
Title: Use consistent document abbreviations throughout the framework. | ||||||||
Description:
Use consistent document abbreviations throughout the framework. [Ed note. Ask Originator for examples where this is not done.] (This issue is a proposed implementation detail of CR-5.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-11 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | unassigned | |
Title: Add conformance requirements to checklists. | ||||||||
Description:
Add conformance requirements to checklists. (This issue is a proposed implementation detail of CR-5.) This CR issue group: 5 (parent), 6 through 14 (details). LH comment: this would almost turn the checklists into test materials, instead of (currently) results summaries. See CR-16 -- this issue essentially duplicates that one. |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-12 | QAF | 2004-02-09 | QAF | Substantive | Active | Jim Hendler | unassigned | |
Title: Confirm consistency of overlapping information in QAF document families. | ||||||||
Description:
While this should obviously be done for *all* QAF components, the specific instance pointed out by Originator is: Priority of checkpoints between OpsGL and OpsET (note problems with this in Guideline 6). (This issue is a proposed implementation detail of CR-5.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-13 | OpsGuide | 2004-02-09 | OpsGuide | Editorial | Active | Jim Hendler | unassigned | |
Title: The normative version of OpsGL should be a single HTML file. | ||||||||
Description:
Make the single HTML file version of QAF-OPS normative (if QAF-OPS stays an independent document). (This issue is a proposed implementation detail of CR-5.) This CR issue group: 5 (parent), 6 through 14 (details). See CR-18 -- this issue essentially duplicates that one. |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-14 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: Operational Examples & Techniques contains no examples, only case studies. | ||||||||
Description:
Either change the Operational Examples and Techniques to contain reusable examples or change its name to reflect its true content. Suggest "Operational Case Studies and Techniques." (This issue is a proposed implementation detail of CR-5.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-15 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: Clarify what is meant by "New Working Groups" in Guideline 1. | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: [Originator] These guidelines should only apply to new work items begun after the QAF becomes a recommendation. | ||||||||
Resolution: ...tbd... | ||||||||
CR-16 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: Checklists should include conformance requirements. | ||||||||
Description:
(Only description is at "Comment reference" for now.) See CR-11 -- this issue essentially duplicates that one. |
||||||||
Proposal: [Originator] Include conformance requirements in checklists. | ||||||||
Resolution: ...tbd... | ||||||||
CR-17 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: A, AA, AAA should be defined before their first use. | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: [Originator] Include explanations of A, AA, AAA before their first use. | ||||||||
Resolution: ...tbd... | ||||||||
CR-18 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: The single HTML file should be the normative version of OpsGL. | ||||||||
Description:
(Only description is at "Comment reference" for now.) See CR-13 -- this issue essentially duplicates that one. |
||||||||
Proposal: [Originator] Make the single HTML file the normative version of OpsGL. | ||||||||
Resolution: ...tbd... | ||||||||
CR-19 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: CP1.4 does not define "QA Deliverables & Milestones". | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-20 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: OpsGL should not require so much planning and assignment of resources prior to chartering a WG. | ||||||||
Description:
(Only description is at "Comment reference" for now.) Related issues: CR-1 (3rd part). |
||||||||
Proposal: ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-21 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: The Rationale of CP2.1 is inconsistent with the statement of the CP and its Conformance Requirements. | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-22 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: The phrase "where-and-how-plan" is used in ConfReqs of CP2.2 but not defined anywhere. | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-23 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: The scope of QA Moderator position is ill-defined and not distinguisable from "test development manager". | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-24 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: CP5.1 is hard to distinguish from CP2.1 & CP2.2, and contains inconsistencies. | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-25 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: QAF-OPS and OPS-EXTECH docs out of synch. | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-26 | OpsGuide | 2004-02-09 | OpsGuide | Substantive | Active | Jim Hendler | unassigned | |
Title: CP6.2 and CP5.4 should be merged. | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: [Originator] As proposed in comment. | ||||||||
Resolution: ...tbd... | ||||||||
CR-27 | SpecGL | 2004-02-11 | SpecGL | Editorial | Active | Bjoern Hoehrmann | unassigned | |
Title: Use of abbreviations in ConfReqs makes them much harder to read. | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: [Originator] Spell out all abbreviations in ConfReqs. | ||||||||
Resolution: ...tbd... | ||||||||
CR-28 | SpecGL | 2004-02-11 | SpecGL | Editorial | Active | Bjoern Hoehrmann | unassigned | |
Title: The styling of checkpoints makes it difficult to recognize checkpoint headings. | ||||||||
Description:
(Only description is at "Comment reference" for now.) |
||||||||
Proposal: [Originator] Highlight checkpoint headings (see comment reference). | ||||||||
Resolution: ...tbd... | ||||||||
CR-29 | SpecGL | 2004-02-11 | SpecGL | Substantive | Active | Bjoern Hoehrmann | unassigned | |
Title: Detectable errors for instance data CoP, and error reporting requirements for associated processor (software) CoP. | ||||||||
Description:
(Only description is at "Comment reference" for now.) [@@Ed note. A concise synopsis of the issue is needed.@@] |
||||||||
Proposal: [Originator] "all W3C specifications defining a notion of instance data should be explicitly required to identify all programmatically reportable errors, make reporting these a requirement for a specific class of product, define how to identify such software and define how to identify instances which do not have reportable errors." | ||||||||
Resolution: ...tbd... | ||||||||
CR-30 | SpecGL | 2004-02-11 | SpecGL | Editorial | Active | Bjoern Hoehrmann | unassigned | |
Title: SpecGL does not define "testable" or "measurable" but uses them in ConfReqs. | ||||||||
Description:
(Only description is at "Comment reference" for now. Significant discussion thread starts there.) |
||||||||
Proposal: ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-31 | SpecGL | 2004-02-11 | SpecGL | Editorial | Active | Bjoern Hoehrmann | unassigned | |
Title: SpecGL does not define what is an "implementation". | ||||||||
Description:
(Only description is at "Comment reference" for now. Significant discussion thread starts there.) |
||||||||
Proposal: ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-32 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Dominique Hazaël-Massieux | |
Title: QAF contravenes W3M resolution that Rec-track specs should not coerce the WGs. | ||||||||
Description:
"Comment reference" states the nature of Originator's objection: he asserts that QAF violates a W3M resolution, which basically says that Rec-track specifications should address technology requirements, and may define conformance for implementations of the technologies, but should not place requirements on Working Groups themselves, that they must conform to some specified requirements. In dialog on qa-chairs list, Karl Dubost and Dominique Hazaël-Massieux present counter-arguments and assert that in fact W3M pointed to QAF as a *correct* approach. As they point out, the MUST (and other RFC2119) usage in OpsGL describe what a WG must do to conform to OpsGL. Nowhere does it say that a WG must conform to OpsGL. The latter (according to previous QAWG discussions and resolutions) is a W3C policy issue. Commenter refers to QAWG Issue 16 and Issue 71. While these issues express QAWG sentiment (from 2002) that some conformance to QAF should be required in the future, such (potential) policy is not embedded in the QAF specs themselves (or anywhere else, now). Summary (and possible proposed resolution): the comment misinterprets what QAF actually addresses. XyzGL (Xyz = Ops or Spec or Test) defines what an object (WG, spec, TM) must do to conform to XyzGL. It does not require that the object conform. Queue an editorial issue to see if this could be made even clearer in QAF. See also related detail and argumentation by author. Note. In the author's collected list of formal issues, this one is specifically identified as one requiring a response. |
||||||||
Proposal: ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-33 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Dominique Hazaël-Massieux | |
Title: QAWG failed to formally address Last Call comment before requesting CR transition. | ||||||||
Description:
Issue synopsis. "Comment reference" states the nature of Originator's objection: he objects that QAWG did not formally address Last Call comments of his before advancing to CR. He points to a long email message of July 2003, about the May 2003 TestGL Working Draft (WD), in which he had many criticisms. The main thrust of his message was about the waterfall model, and we did in fact make substantial revisions to TestGL in response. "Comment reference" specifically highlights the QAWG failure to address his July 2003 comment about "QAF contravenes W3M policy" (newly restated in January 2004 as CR-32). QAWG observations:
See also issue CR-34. |
||||||||
Proposal: ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-34 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Dominique Hazaël-Massieux | |
Title: QAWG failed to list open issues when advancing to CR. | ||||||||
Description:
Issue synopsis. "Comment reference" states the nature of Originator's objection: he asserts that two comments threads of his about (early-WD) TestGL were not listed as open issues when requesting CR advancement for OpsGL and SpecGL. The stated rationale for the relevance of his TestGL WD comments to LC/CR SpecGL and OpsGL is:
This issue is closely related to CR-33. |
||||||||
Proposal: [Originator] Return all documents to Last Call when the Test Guidelines are ready, after all comments on all documents have been formally addressed, and then advance them all in synchronization. | ||||||||
Resolution: ...tbd... | ||||||||
CR-35 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Lofton Henderson | |
Title: QAF is only a "test development framework", therefore it is misnamed. | ||||||||
Description:
(Only description is at "Comment reference" for now.) Note. In the author's collected list of formal issues, this one is specifically identified as one requiring a response. Related issues: the author's (non-critical) preceding comment is similar and supportive to this one. |
||||||||
Proposal: [Originator] Rename QAF. | ||||||||
Resolution: ...tbd... | ||||||||
CR-36 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Dominique Hazaël-Massieux | |
Title: QAWG has failed to meet the quality commitments in its charter. | ||||||||
Description:
(Only description is at "Comment reference" for now.) Comment. The referenced QAWG charter is the new one, approved by AC in fall 2003 (@@exact date?@@). Note. In the author's collected list of formal issues, this one is specifically identified as one requiring a response. |
||||||||
Proposal: [Originator] See comment reference. | ||||||||
Resolution: ...tbd... | ||||||||
CR-37 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Lofton Henderson | |
Title: Make QAF informative, and represent its key normative content with small changes to W3C Process Document. | ||||||||
Description:
(Only description is at "Comment reference" for now.) Note. In the author's collected list of formal issues, this one is specifically identified as one requiring a response. |
||||||||
Proposal: [Originator] Make QAF informative, and represent its key normative
content with small changes to W3C Process Document.
[LH] The proposed revisions to the QAF would address this concern. |
||||||||
Resolution: ...tbd... | ||||||||
CR-38 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Dominique Hazaël-Massieux | |
Title: QAF violates RFC2119. | ||||||||
Description:
The "Comment reference" is the starting place for the substantive complaint of Originator. The bullet list of references at "Comment reference" are examples of where the alleged violation of RFC2119 (Sec.6) occurs. History: extensively discussed and resolved during Last Call, in LC-67. Note. In the author's collected list of formal issues, this one is specifically identified as one requiring a response. |
||||||||
Proposal: [Originator] See "Comment reference". | ||||||||
Resolution: ...tbd... | ||||||||
CR-39 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Lofton Henderson | |
Title: All parts of QAF, including supporting informative & resource references, must advance synchronously. | ||||||||
Description:
This issue incorporates by reference a list of sub-issues or comments, which appear to warrant individual treatment (QAWG may agree with some, disagree with others):
Note. In these sub-issues, numerous references to "inappropriate" or "circumvent pubrules & W3C Process". But specific references to violated rules are @@missing@@. Some research is needed here. Note. Many of the comments in this list are repeated in issue CR-41, which alleges numerous specific breaches of W3C Process, Pubrules, etc. Note. In the author's collected list of formal issues, this one is specifically identified as one requiring a response. |
||||||||
Proposal: [Originator] ...tbd...
[LH] The development plan which would necessarily be associated with the proposed revisions to the QAF would address many of these individual points -- either they would become moot, or would be approved, or QAWG would disapprove the need for synchronization. Point-by-point details TBD. |
||||||||
Resolution: ...tbd... | ||||||||
CR-40 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Lofton Henderson | |
Title: 2nd Last Call needed, with stricter advancement criteria. | ||||||||
Description:
Note. In the author's collected list of formal issues, the "2nd Last Call" part of this one is specifically identified as one requiring a response. |
||||||||
Proposal: [Originator] ...tbd...
[LH] The proposed revisions would necessitate synchronization of the components at another Last Call. |
||||||||
Resolution: ...tbd... | ||||||||
CR-41 | QAF | 2004-02-11 | QAF | Substantive | Active | Jeremy Carroll | Dominique Hazaël-Massieux | |
Title: QAWG & QAF have many breaches of W3C Process, pubrules, etc. | ||||||||
Description:
Submitter alleges numerous specific violations of W3C Process, Pubrules, etc. He enumerates these: 4.8, 4.11, 5.8, 5.9, 5.11, 6.1, 6.2, 6.3, 6.4, 8.1, 8.2, 8.3, 8.4, 8.5, 11.2, 11.4, 13.1. Originator identifies his comment 8.5, about QAWG's handling of Dan Connolly's late LC comment about RFC2119, as alone sufficient to support the present issue. There is @@some history@@ [links tbd] about the RFC2119 issue. The present objection is apparently not about the substance of the issue itself (LC- 67), but a procedural objection that QAWG didn't handle DC's comment properly (note that it was handled under old Process, pre-July-2003.) The enumerated alleged specific violations are (mostly) given without a specific reference to a rule being violated. To respond to this issue, QAWG must either: research each point and prove "no discernable violation"; or, request Originator to provide a specific Process or Pubrules reference for each one. Author has supplied these clarifications and references.Note. Many of the comments in this list are repeated in issue CR-39, which is about the need to advance all QAF- related at equal levels of maturity. Note. In the author's collected list of formal issues, this one is specifically identified as one requiring a response. |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-42 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: Test Assertions -- definitions | ||||||||
Description:
This is one of a handful of related issues about Test Assertions that have been generated out of the mail thread starting at "Comment reference." This concerns definition of terms used in the discussion. In the middle of the thread, it becomes clear that different people are using the terms Conformance Requirements and Test Assertion slightly differently.
Precise definitions of the terms should be agreed -- either: 1.) reaffirm the existing definitions (and historical interpretation), and enhance them (e.g., with examples) to further clarify; or 2.) change them to agreed consensus definitions. |
||||||||
Proposal: [Originator] Option 1, reaffirm and clarify. | ||||||||
Resolution: ...tbd... | ||||||||
CR-43 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Mark Skall | unassigned | |
Title: Test Assertions -- how and where? | ||||||||
Description:
This is one of a handful of related issues about Test Assertions (TAs) that have been generated out of the mail thread starting at "Comment reference." This concerns how QAWG should handle the derivation of Test Assertions for the GL specifications. Two options are suggested by Originator [MS]:
Note. This issue interacts with SpecGL Checkpoint 7.1, which creates strong pressure towards the form of testable statements that uses RFC2119 keywords -- the Test Assertion form of expression would require justification in a specification. |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-44 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lynne Rosenthal | unassigned | |
Title: Test Assertions -- should SpecGL require them? | ||||||||
Description:
This is one of a handful of related issues about Test Assertions that have been generated out of the mail thread starting at "Comment reference." This issue concerns whether or not SpecGL should require that specifications provide test assertions, i.e., should Guideline 10 and its two checkpoints be kept in SpecGL? Two questions are posed:
|
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-45 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: SpecGL & SpecET inconsistencies about CoP and scope requirements. | ||||||||
Description:
In CP1.1, SpecGL says that Class of Product (CoP) help define scope. SpecET, for CP1.2, says list-of-CoP help illustrate scope. These should be reconciled and made consistent. This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-46 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: CP2.2 is hard to implement. | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." It is unclear what is wanted for the implementation of CP2.2. On the one hand, there are the many detailed Conformance Requirements associated with all of the technical details of a specification. On the other hand, there are high-level conformance criteria or "schemes" that typically are applied to various CoP. I think we mean the relatively few criteria of the latter, rather than the (scores, hundred, thousands, ..) detailed conformance requirements. The examples of SpecET help some, but this should be disambiguated in the Techniques of SpecET (at least). |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-47 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: Rationale of CP2.3 (spec. category) is unconvincing, and it is unclear how to satisfy the CP. | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." This seems like a "weak" checkpoint:
|
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-48 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: Error in Conformance Requirements of CP3.2 (profile min. rqts.)? | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." In the Conformance Requirements of CP3.2, is "for each profile" wrong? It seems to make more sense that minimum requirements would apply across a group of things, i.e., across all profiles. I.e., this would be the intersection of the sets of requirements for all profiles. If "for each profile" is indeed intended, then how do "minimum requirements for Profile XYZ" differ from "the requirements for Profile XYZ"? |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-49 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: The statement of CP5.1 (discretionary items) is unrelated to the given Conformance Requirements. | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." The statement of CP5.1 is about circumstances under which discretionary items are allowed. The Conformance Requirements are about explaining the Rationale and naming each item. Furthermore, in Rationale the "link target for each one" -- perhaps more useful than a name in automated environments -- is not required in the ConfReqs. |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-50 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: Redundant requirement in CP5.3 (number of disc. items/choices). | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." The 2nd Conformance Requirement -- indicating any dependency of the number upon other DoV -- is apparently redundant with CP5.5 (relationship to other DoV). |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-51 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: Inconsistencies in CP5.4 (consistent handling of discretionary choices). | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." There are several problems in the checkpoint:
|
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-52 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: Problems in CP5.5 (relationship of discretionary items with other DoV). | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." There are several problems in this checkpoint:
|
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-53 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: Error in Conformance Requirements of CP6.5 (mitigate extensions)? | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." Is last sentence of Conformance Requirements true? Why wouldn't this CP apply, e.g., to extensions to an API standard? |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-54 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: CP8.1 (universal reqts for min functionality) is difficult to implement. | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." There is ambiguity which make this checkpoint (IMO) unimplementable, namely: How do "universal requirements for minimum functionality" differ from CP2.2's "per-CoP conf. reqs."? I.e., what is the difference between the Conformance Requirements for a CoP and the "universal requirements for minimum functionality for [that] CoP"? |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-55 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: Define "conformance concepts" in CP8.2. | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." To be implementable, CP8.2 needs to define "conformance concepts", or give lots of guidance and/or examples to illustrate what the phrase means. |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-56 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: How do "conformance designations" (CP8.5) relate to "conformance concepts" (CP8.2)? | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." How does CP8.5, "define all conformance designations", relate to CP8.2, "define (special) conformance concepts"? Is one an aspect of the other? |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-57 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: The meaning and purpose of "conformance disclaimer" are unclear in CP9.2. | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." OpsGL requires a conformance disclaimer for test suites -- passing the test suite does not imply conformance to the spec. But this is SpecGL. What is the disclaimer required by this CP? The intent and purpose of the required disclaimer is very mysterious. |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-58 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Lofton Henderson | unassigned | |
Title: Clarify what comprises "list" of Test Assertions. | ||||||||
Description:
This is one of several issues that have been uncovered during a close analysis of SpecGL, at "Comment reference." CP10.1 Conformance Requirements are not specific enough to decide whether or not they are satisfied by some cases. For example, if TAs were marked-up & styled in-line in text, does that comprise a "list" and satisfy CP10.1? |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... | ||||||||
CR-59 | SpecGL | 2004-02-15 | SpecGL | Substantive | Active | Jeremy Carroll | ...tbd... | unassigned |
Title: ...~20 substantive SpecGL issues from JC will start here... | ||||||||
Description:
...tbd... |
||||||||
Proposal: [Originator] ...tbd... | ||||||||
Resolution: ...tbd... |