QAWG Last Call Issues

Last update: $Date: 2003/10/15 23:09:32 $

Issues regarding the Last Call documents produced by the QA Working Group (QAWG) should be reported to the QAWG using the forms that are linked from the status sections of those documents. As a last resort, if you cannot use the forms for some reason, you can send mail to www-qa@w3.org (public archives). (The forms will put a copy on this email list and in the archives.)

Comments on this issues list should be sent to the www-qa@w3.org mailing list (Archives).

In this document,

Summary List of Outstanding Last Call Issues

num Status Spec Topic Class Date Title

Detailed List of Last Call Issues

num Spec Date Topic Class Status Raised By Owner
LC-1 SpecGuide 2003-02-28 SpecGuide Substantive Closed Alex Rousskov Lynne Rosenthal
Title: require a "Security Considerations" section
Description:

Specification reference.

comment about "Overall" :

Any spec SHOULD have a Security Consideration section. Protocol or behavioral specs MUST have a Security Consideration section.

Security sections make spec authors think about potential vulnerabilities and address at least some of them before the bad guys can exploit them. These sections are also a great place to warn implementors and users about most security-sensitive areas of the spec and, perhaps, common exploits.

IETF's Internet Architecture Board has published the following Internet Draft that may be of use to SpecGL authors: http://www.ietf.org/internet-drafts/draft-iab-sec-cons-03.txt

Discussed at 20030331 telecon. QAWG believes that the proposal has merit, but that specifying such requirements is outside of the scope of the QA Framework. Will be referred to appropriate W3C group or team.

Proposal: Require "Security Considerations" sections just like we already require conformance sections.
Resolution: The proposal has merit, and should be addressed by W3C, but the QAWG considers security requirements to be outside of the scope of QA Framework. Will be referred to an appropriate W3C group or team.
LC-2 SpecGuide 2003-02-25 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: Grammatical error in Scope
Description:

Specification reference.

comment about "Abstract, Status,..." : "Section 3 defines explains how to make conformance claims that W3C TRs satisfy the requirements of section 2. It defines conformance for this document"

It seems as if either 'defines' or 'explains' is meant, not both. The first sentence is grammatically incorrect, thus the intended meaning is unclear. The second sentence might seem superfluous, depending on the intended meaning of the first.

Proposal:
Resolution: This has been fixed, by deleting "defines". See draft revised text.
LC-3 OpsGuide 2003-02-23 OpsGuide Substantive Closed lynne rosenthal Lofton Henderson
Title: Committment Table and its CPs
Description:

Specification reference.

comment about "Guideline 1: Integrate Quality Assurance into Working Group activities." :

Remove the Committment Table and CP1.1, 1.2, 1.3. These items do not add information and are redundant. To satisfy Level 3 (5 or 7), you must satisfy other checkpoints in the OpsGL. In fact, if we did our jobs right, Level 3 should equal Conformance Level A (satifying all P1), Level 5 should equal Level Double A, and Level 7 = Triple A. Isn't it a goal to get WGs to conform to the OpsGL? That would mean that they must satisfy all the P1 checkpoints. Thus, they must have a committment level to P1. So, why do we use a new term - Level 3.

The Table introduces a DOV - called committment level. The OpsGL does not conform to the SpecGL's CPs that require DoVs to address the relationship to conformance, to other DoVs, etc.

Email discussion , including proposed resolution.

Discussed and resolved at 20030501 telecon and subsequent email . Table will be eliminated, and CP1.1 - 1.3 will be replaced. CP1.1 will require choice of A-AAA for OpsGL, choice of A-AAA for SpecGL, and choice of A-AAA for TestGL. Plus 2 additional CPs preserving the commitment to have some test materials.

Related LC issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal: [Originator] Remove the Table and CP1.1, 1.2, 1.3. An alternative to removing the Table is to amend it and move it to either the Introduction section (prior to Guideline section) or an Appendix. It should be amended by adding an additional column called, 'checkpoint'. For each row, indicate the checkpoint that applies.

[LH] See email.

Resolution: The table is eliminated, and CP1.1 - 1.3 are replaced. CP1.1 will require choice of A, AA, or AAA for OpsGL, choice of A, AA, or AAA for SpecGL, and choice of A, AA, or AAA for TestGL. New CP1.2 and CP1.3 will require commitment to (respectively) "some" or "complete" test materials, preserving that normative aspect of the previous table. All previous normative requirements are preserved, while the redundant conformance scheme is eliminated. See draft revised text of CP1.1 (as well as CP1.2, CP1.3).
LC-4 SpecGuide 2003-02-19 SpecGuide Editorial Closed Olivier Thereaux Lynne Rosenthal
Title: no definition for unconditional conformance
Description:

Specification reference.

comment about "Overall" : In the QAframe-spec glossary (http://www.w3.org/TR/2003/WD-qaframe-spec-20030210/#definitions) there is no definition for the unconditional conformance term.

See subsequent email proposal, to remove empty (unused) definition.

Related issues: 4, 32.

Proposal: [Originator] add one? :)
Resolution: This has been fixed by removing the empty definition from SpecGL. (It has been referred to the "QA Glossary" editors, to add a definition there.) See draft revised text.
LC-5 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: Ck 2.2 list of classes
Description:

Specification reference.

comment about "2.1 Identify all classes of product. " : second conformance requirement refers to list of classes but its not clear which list it is referring to. If it's the list under guideline 2 then that list is non-exhaustive so requiring people to use that list is somewhat limiting.

Discussed at 20030331 telecon, classified as editorial.

Proposal: [Proposed wording to be provided by SpecGL editors.]
Resolution: This is fixed by qualifying the reference to read, "...define the conformance requirements for each class of product identified in checkpoint 2.1" See draft revised text.
LC-6 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: Guideline 2, typo
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " : thrid para - typo 'as either or producers' remove 'or'
Proposal:
Resolution: This has been fixed as suggested. See draft revised text.
LC-7 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: Checkpoint 1.2
Description:

Specification reference.

comment about "1.2 Illustrate what is in scope" : Can use cases and examples be in a separate document from the main spec?

Discussed at 20030407 telecon. Resolved "yes", they can be in a separate document, referenced from the main document.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal:
Resolution: Yes, examples and use cases can be in a separate document, that is referenced from the main document. This has been clarified. See draft revised text.
LC-8 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: checkpoint 1.1
Description:

Specification reference.

comment about "1.1 Include the scope of the specification" : rather wooly conformance requirements

Previous email: ISO 'scope' definition; QAWG discussion thread.

Discussed at 20030331 telecon, classified as editorial. Action. (Someone) request clarification from originator.

Proposal: [Proposed wording to be provided by SpecGL editors.]
Resolution: Without specific suggestions for rewording the conformance requirements, or identification of specific problems in the existing wording, we are unsure what's wrong with them and how to change them. To help remove any ambiguity in the conformance requirements, we have improved their context -- better verbiage in the statement of the checkpoint itself, and supporting explanatory materials. See draft revised text.
LC-9 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: INTRODUCTION section 1.1, second bullet
Description:

Specification reference.

comment about "Abstract, Status,..." : Second sentence implies that all checkpoints must be satisfied to comply with the guidelines where as only priority 1 checkpoints are mandatory.

Discussed at 20030331 telecon, classified as editorial.

Proposal: [Proposed wording to be provided by SpecGL editors.]
Resolution: This has been fixed (2nd bullet of Scope section) to clarify and remove the confusion. See draft revised text.
LC-10 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: Section 7
Description:

Specification reference.

comment about "Overall" : date for LC WD is in the future (or it was when the doc was published).
Proposal: Reviewer used the WG version of the document and not the LC version of the SpecGL. Moot point.
Resolution: Reviewer used the QA WG version of SpecGL that preceded the Last Call version, and not the LC version. Moot point.
LC-11 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: CK 2.3 Category of object: clarification
Description:

Specification reference.

comment about "2.3 Identify which of the categories of object..." : What is a category of object? The same as a class of product

Discussed at 20030331 telecon, classified as editorial.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email. Proposed wording to be provided by SpecGL editors.
Resolution: The verbiage on classes and categories is revised consistent with the 4-point proposal in email comments. The solution involves: define the terms and add them to "Definitions", introduce and separate the concepts better, move the main part of the verbiage of Guideline 2 (GL2) into a new subsection in new chapter "2. Concepts", and reference that verbiage from GL2 and its checkpoints. See draft revised text.
LC-12 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: section 3.4 Conformance definition
Description:

Specification reference.

comment about "Overall" : Example - not true, see comment on 3.3

(LH clarification.) The example does not correspond to the requirements that are listed immediately before it -- apparently the latter were updated without updating the example.

Proposal:
Resolution: The example has been updated to match the requirements. See draft revised text.
LC-13 SpecGuide 2003-03-11 SpecGuide Substantive Closed Marc Hadley Lynne Rosenthal
Title: Section 3.3 Conformance and TAs
Description:

Specification reference.

comment about "Overall" : amusing that this section doesn't meet checkpoint 14.1 and therefore renders the document as only A-conforming to itself. Would be better if the document were AAA-conforming to itself IMO.

Discussed at 20030331 telecon. QAWG agrees that SpecGL should be AAA-conformant to SpecGL, and commits to AAA conformance by Candidate Recommendation.

Discussed and resolved at June QAWG face-to-face. QAWG (MS) will develop a set of test assertions for SpecGL.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal: Make SpecGL AAA-conformant to SpecGL by Candidate Recommendation.
Resolution: QAWG has developed a set of test assertions for SpecGL. These are linked from the "Normative References" section of SpecGL (CR and later versions).
LC-14 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: CP 14.1 - clarify conformance requirement
Description:

Specification reference.

comment about "14.1 Provide test assertions" : conformance requirements - is a separate document OK or does this have to be in the same doc as the rest of the spec?

Related discussion about whether testability is an inherent part of "conformance requirement" in this email sub-thread (of a thread about some OpsGL issues).

See mail thread (20030606), which includes suggested resolutions.

See also these directly related QAWG issues: issue #98, issue #99, issue #110.AR-001. These were marked as closed at Tokyo and after, however later email and telecon decisions, including new agreed definitions and some samples, suggest reviewing the earlier closures.

Discussed and resolution re-affirmed at 6/2003 QAWG face-to-face.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal:
Resolution: Yes, a separate document is allowed. This has been clarified, especially in the explanatory material of CP14.1 (new CP10.1). See draft revised text.
LC-15 SpecGuide 2003-03-11 SpecGuide Substantive Closed Marc Hadley Lynne Rosenthal
Title: extensions
Description:

Specification reference.

comment about "Guideline 9 Allow extensions or NOT!" : A very well thought out section IMO.

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal: Thanks to commentor, this is useful feedback on a controversial topic.
Resolution: Closed (no issue).
LC-16 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: CP 8.4 clarify conformance requirement
Description:

Specification reference.

comment about "8.4 Promote consistent handling of discretionary choices." : conformance requirements not clear, what does 'document the identified policies for handling discretionary choices' mean?

Discussed at 20030331 telecon, classified as editorial.

Discussed at 20030410 telecon. AI given to DM (LH & DH assisting) to draft clarification.

New proposal circulated. Discussed and re-resolved at 20030505 telecon. 8.4 will be kept, broadened to "items" instead of "choices", and final text (derived from the new proposal) will be revised and clarified by SpecGL editors.

Discussed again at 20030908 telecon.

Related LC issues: 16, 27, 39.

Proposal: [Proposed wording to be provided by SpecGL editors.] Draft simple clarification, or remove if no QAWG consensus on the clarified text.
Resolution: 8.4 (new 5.4) is kept, conformance requirements are broadened to "items" instead of "choices", and final text is revised and clarified by SpecGL editors. See draft revised text.
LC-17 SpecGuide 2003-03-11 SpecGuide Substantive Closed Marc Hadley Lynne Rosenthal
Title: CP 7.1 deprecated features
Description:

Specification reference.

comment about "7.1 Identify each deprecated feature. " : conformance requirements imply a single section for deprecated features - is it not OK to include deprecations where they occur without a summary section?

(See subsequent email discussion.)

20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: there must be a single starting point to find all deprecated features. This could for example be a subsection of the Conformance Clause. It could consist of a table or list of links to individual parts of the specification.

Proposal:
Resolution: There must be a single point of entry to find all deprecated features, as implied by the conformance requirement, "..in a normative section..". For example, this could be a subsection of the Conformance Clause. This could consist of a table or list of links to individual parts of the specification, where the deprecated features are actually defined in detail. This has been clarified. See draft revised text.
LC-18 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: GL 5 non-hierarchiacal modules
Description:

Specification reference.

comment about "Guideline 5 Address the use of modules to divide the technology." : Modules are non-hierarchiacal - can modules have dependencies on other modules? If so, isn't this a hierarchy?

(See subsequent email discussion.)

Discussed at 20030331 telecon, classified as editorial.

Proposal: [Proposed wording to be provided by SpecGL editors.]
Resolution: What is meant is that hierarchy is not a necessary or even typical attribute of modules. This has been clarified in the new "Concepts" chapter, subsection on profiles, modules, and levels. See draft revised text.
LC-19 SpecGuide 2003-03-11 SpecGuide Substantive Closed Marc Hadley Lynne Rosenthal
Title: CP 4.4 explanation clarification
Description:

Specification reference.

comment about "4.4 If profiles are chosen, address rules for profiles." : 'experience shows ... meets all the pertinent checkpoints of this document' - what experience? As this is not yet a recommendation this seems like a rather strong statement.

20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: The wording has been changed to eliminate this confusion.

Proposal: The reference is to previous experience in both W3C and other venues, and this previous experience has formed much of the profile-related checkpoint content. Improve the wording to remove the possibility of confusion.
Resolution: The reference is to previous experience in both W3C and other venues. This previous experience has formed much of the profile-related checkpoint content. The wording is improved to eliminate the point of confusion. See draft revised text.
LC-20 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: CP 3.1 Conformance Requirements - clarify
Description:

Specification reference.

comment about "3.1 any universal requirements for minimum functionality." : conformance requirements - only one section for this or are multiple O.K?

See related issue LC-96.

20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: one section. As CP3.1 has now been clarified in other issue discussion, its nature looks somewhat different. It is a summation of information that could be deduced from other individual requirements (although the process could be difficult and error prone). Therefore only "single section" makes sense.

Proposal:
Resolution: One section. As CP3.1 has been clarified in other issue discussion, its nature looks somewhat different. It is a summation of information that could be deduced from other individual requirements that are distributed throughout the specification (although the process could be difficult and error prone). Therefore only "single section" makes sense. This has been clarified. See draft revised text.
LC-21 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: CP 2.4 - relationships of DOV - clarify
Description:

Specification reference.

comment about "2.4 If there are several classes of products, define their relationships..." : 'define their relationships and interaction with other dimensions of variability' this is a confusing checkpoint that is repeated in each successive guideline. It's really not clear exactly what is intended.

Email comments, including proposal (20030418).

Discussed and resolved at 20030421 telecon.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: See email.
Resolution: This has been addressed and further explained in a "DoV" subsection of the new chapter "2. Concepts", which is linked from the appropriate guidelines and checkpoints. See draft revised text.
LC-22 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: CP 2.3 placement of Checkpoint
Description:

Specification reference.

comment about "2.3 Identify which of the categories of object..." : This checkpoint includes a requirement for "where in your specification," even though the intro says those requirements are limited to GL1 and GL10-14.

20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: [provisional resolution -- revisit] Yes, a part of CP2.3 deals with where the information needs to be placed. Due to many changes, this intro text may need to be rewritten. For now, we have deleted the mention of which Guidelines fall into which general type. It may be that a guideline falls into both categories.

Proposal:
Resolution: This contradiction has been removed, as a part of revision of the Introduction chapter, the addition of a "Concepts" DoV chapter, and the reorganization and consolidation of the Guidelines. See draft revised text.
LC-23 SpecGuide 2003-03-11 SpecGuide Substantive Closed Ian Jacobs Lynne Rosenthal
Title: CP 1.4 vague
Description:

Specification reference.

comment about "1.4 Provide examples. " :

I think this checkpoint is too vague. Is it possible to break it down into a few more concrete requirements that are more verifiable? For instance:

  • For markup specifications, provide at least one example of each markup construct.
  • For protocol specifications, provide at least one example of ...
  • For transformation specifications, illustrate each transformation capability with an example showing input and output.
  • For UI specifications, provide an example of each construct, and illustrate the desired output using at least one mechanism other than the specification itself (e.g., the SVG specification should not rely on SVG rendering alone to explain what something should look like).

While you may miss some cases, I think spec editors will find this more helpful than the general goal to "provide examples."

Email discussion and discussed in 20030307 telecon. Alternatives:

  • Alt.1: Implement the proposal: make more specific requirements and tie it to specification categories.
  • Alt.2: Implement the proposal in the ExTech document, but leave CP as is.
  • Alt.3: Implement the proposal in the ExTech document, and make CP more specific without mentioning categories. (I'm not sure how we do this).

Resolved in favor of a variation on Alt.2 (which might be what was meant by Alt.3?) -- the discussion and breakdown by category will be done in ExTech. But the Discussion of CP1.4 will mention the concept raised by comment originator (that different kinds of specification categories require different particular associated kinds of examples), and maybe even give a single brief "for example."

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal: Originator proposed Alt #1.
Resolution: Resolved in favor of modified Alternative 2. The discussion and breakdown by category will be done in Spec Examples & Techniques (SpecET). But the Discussion of CP1.4 (now CP1.3) will mention the concept raised by comment originator (that different kinds of specification categories require different particular associated kinds of examples), and gives a brief "for example." This change is effective in SpecGL CR and later versions.
LC-24 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: Introduction, Sect 1.4
Description:

Specification reference.

comment about "Overall" : last para: Pubrules and the MoS aren't really specifications. What about "resources?"
Proposal:
Resolution: This has been changed as suggested. See draft revised text.
LC-25 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: Introduction: scope and goals
Description:

Specification reference.

comment about "Overall" : Section 1.1 Last paragraph -- I think the previous paragraph doesn't belong here; it could be deleted or moved to the status section (after some editorial fixes).

20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: Agreed, the text has been deleted as suggested.

Proposal:
Resolution: Agreed, it has been deleted. See draft revised text.
LC-26 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: Multiple CPs - It is not applicable
Description:

Specification reference.

comment about "Overall" : CP 2.4, 4.1, 4.2, 4.3, 4.4, 5.1, 5.2, 6.1, 7.1, 7.2, 7.4, 7.5, 8.1, 8.3, 9.2, 9.4, 9.5, 9.6,

Adopt the UAAG 1.0 approach of separating requirements from applicability exclusions (called "normative inclusions/exclusions" in UAAG 1.0). Suggest that the statment "It is not applicable if..." be labeled as "Normative inclusion/exclusion" as in UAAG 1.0

Discussed at 20030331 telecon, classified as editorial.

Email discussion, including proposal.

Discussed and closed at 20030602 telecon (DRAFT). QAWG appreciates the utility of the "normative exclusion" concept in the complex conformance landscape of UAAG 1.0. For the few and simple occurrences of the "not applicable" usages in SpecGL, QAWG considers that it is cleaner and clearer not to introduce a new concept and terminology such as "normative exclusion".

Related LC applicability/normative-exclusion issues: 26, 73.2, 80.

Proposal:
Resolution: QAWG appreciates the utility of the "normative exclusion" concept in the complex conformance landscape of UAAG 1.0. For the few and simple occurrences of the "not applicable" usages in SpecGL, QAWG considers that it is cleaner and clearer not to introduce a new concept and terminology such as "normative exclusion". Resolved: no change.
LC-27 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: Typos, grammar, etc.
Description:

Specification reference.

comment about "Overall" : GL 3, 2md para, Remove: "Overall, the intent of the WG should be clear" This doesn't add anything

CP 7.1, 7.4, 7.5 ConfReq, last sentence: change 'is' to 'are' in "...if there is no deprecated feature"

CP8.4 Rationale: change 'identifying' to 'identify'

CP9.1, last para: "This is strict conformance" is a repeat of text in intro of GL9

CP 11.4, last para: modify "proper use of the conformance icons" to "proper use of any conformance icons"

GL13, last para: suggest switching the order of Manual of Style and PubRules

Related LC issues: 16, 27, 39.

Proposal:
Resolution: These have all been fixed as suggested.
LC-28 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: document organization suggestions
Description:

Specification reference.

comment about "Overall" : Use style sheets to hide links (e.g., to examples and techniques) for printed version.

20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: Agreed, the stylesheets have been modified as suggested.

Proposal:
Resolution: Agreed, this has been done.
LC-29 SpecGuide 2003-03-11 SpecGuide Substantive Closed Ian Jacobs Lynne Rosenthal
Title: questions and suggestions
Description:

Specification reference.

comment about "Overall" :
  1. In general, specifications make functional requirements. In some cases, it may be necessary to make a performance requirement. How should that be handled?

    20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: if performance requirements are within the scope of the standard, and they presented the same as any other conformance requirement -- e.g., using RFC2119 keywords -- then there would not seem to be any issue. For conformance purposes, they would not need any special handling. (No changes to SpecGL.)

  2. In UAAG 1.0, we realized that for some of our configuration requirements, it didn't matter whether the user agent offered the desired configuration or didn't implement the functionality in the first place. E.g., we decided that it was ok for a user agent to not support blinking at all, or, if blinking is implemented, to offer a configuration to turn it off. However, in other cases, the configurability was "just as important" as the functionality to be configured. Authors should consider these when they specify configuration options.

    20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: Since SpecGL 1.0 has not gone to such a level of detail as addressing configuration options, we are unsure how this comment would affect SpecGL. Note that we (QAWG) have decided that the scope of SpecGL 1.0 should be limited to the most critical and basic topics. This could be added to requirements for SpecGL 2.0. It could also be mentioned, possibly, in SpecET (Examples & Techniques). If Originator thinks that this topic fits under one of SpecGL's existing checkpoints, then perhaps he would be interested to make a contribution, to be put into SpecET. The SpecGL scope has been clarified to indicate the "basic and critical" scope of SpecGL 1.0. See draft revised text.

  3. It might be valuable for a specification to explain how to include its requirements in another specification. This is not the same as explaining how to reference the spec, or how to claim conformance to it.

    Discussed at 20030707 telecon. We (QAWG) have decided that the scope of SpecGL 1.0 should be limited to the most critical and basic topics. We think that this topic has merit, but is more advanced and detailed than we are dealing with in SpecGL 1.0. It could be added to requirements for SpecGL 2.0. Alternately or in addition, it could also be mentioned in SpecET (Examples & Techniques). If Originator thinks that this topic fits under one of SpecGL's existing checkpoints, then perhaps he would be interested to make a contribution, to be put into SpecET. The SpecGL scope has been clarified to indicate the "basic and critical" scope of SpecGL 1.0. See draft revised text.

  4. It might be valuable to explain some desirable characteristics of a specified technical requirement:
    • Mutual independence from other requirements
    • Expresses a minimal requirement
    • Distinguish and label: requirements, exceptions to those requirements, necessary and/or sufficient techniques for satisfying those requirements.

    Related discussion about whether testability is an inherent part of "conformance requirement" in this email sub-thread (of a thread about some OpsGL issues).

    See mail thread (20030606), which includes suggested resolutions.

    For test-assertion topics, see also these TA-related QAWG issues: issue #98, issue #99, issue #110.AR-001.

    Discussed at June QAWG face-to-face. Further clarification needed from originator, sought and discussed in email thread.

    Discussed and resolved at 20030623 telecon. Agreed that it is useful to discuss some attributes of well-written conformance requirements. Some verbiage has been added to the introductory text of GL7 (new number). More extensive material may be added to SpecET (contributions would be welcome!). See draft revised text.

  5. I think that more could be stated about useful ways of allowing extensibility. See, for example, how CSS (forward-compatible parsing), XML, and HTTP handle this. What should be avoided? Is the general practice of "ignore what you don't know" a good idea or a big mistake?

    Extension-group processing plan, including proposals, and subsequent email discussion.

    Resolved at 20030505 telecon, as part of the extensibility issue group resolution. This should not be covered in SpecGL, but it ought to be covered in the SpecET material on extensions. SpecET should illustrate numerous kinds of extensions -- some that have serious interoperability impacts, some that mitigate the impacts, some approaches that might be considered "useful", etc. Also, QAWG discussion sees some problems with examples such as CSS "forward compatible parsing" -- it opens up the ill-defined topic of what is an unknown extension versus what is corrupted or erroneous content. This has been added to the SpecET queue.

  6. A spec should clearly indicate which illustrations (e.g., images) and examples are normative, if any.

    20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: agreed. The wording of CP13.2 has been adjusted to include all sorts of normative content. See draft revised text.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal:
Resolution:
LC-30 SpecGuide 2003-03-11 SpecGuide Substantive Closed Ian Jacobs Lynne Rosenthal
Title: Profile, module, level
Description:

Specification reference.

comment about "Overall" : I am struggling to understand a sharp distinction between profile, module, and level. They are all mechanisms for defining and labeling a set of technical requirements. I have the feeling guidelines 4, 5, and 6 could be combined, and the requirements rephrased "Whatever subsetting mechanism you use..."

Discussed at 20030410 telecon. Disagreement about keeping or merging the concepts. AIs given to: draft a proposal merge the concepts; survey the usage of the terms and concepts in W3C specs.

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. QAWG agrees that clarification is needed. The solution for the whole profile/module/level issue group is:

  • the concepts of profiles, modules, and levels are kept separate, each a separate DoV -- although the concepts are sometimes misused, and although other ways to subdivide the specification could be postulated, the QAWG believes that these three are heavily used in current practice, and that SpecGL can help to improve consistent;
  • a subsection (sec 2.3?) to clarify the definitions and concepts of profiles, modules, and levels will be put into the new Ch.2, "Concepts";
  • the three guidelines GL4,5,6 will be combined into a single guideline, with reduced verbiage, mostly pointing back to section 2.3;
  • however the checkpoints will be kept substantially as they are, each dealing individually with one of the concepts of profiles or modules or levels -- while there is interest in combining checkpoints, on the other hand it was tried in the 1st PWD of SpecGL (May 2002) and caused too many problems;
  • the exception is that the three separate "relationship to other DoV" checkpoints will be combined into one.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal: Email proposal.
Resolution: See the comprehensive solution for the profiles/modules/levels issue group. Agreed to clarify concepts with better definitions and discussion, and to merge the three separate guidelines into a single guideline. However the 3 individual concepts will be kept as separate DoVs within the merged guideline, and will be dealt with independently in the appropriate checkpoints. See draft revised text.
LC-31 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: general
Description:

Specification reference.

comment about "Overall" :

General comments/conformance model I think the document is well-organized, clearly written, and very helpful. Having spent a lot of time thinking about conformance issues (notably for UAAG 1.0), I thought it covered a lot of ground and did so well.

I like the distinction between requirements that relate to the conformance model and those that involve implementing the model in the spec.

Proposal: Thanks to commentor. This is useful feedback.
Resolution: Thanks for the useful feedback. (No issue.)
LC-32 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: 4. Definitions
Description:

Specification reference.

comment about "Overall" : unconditional conformance -- This is used in HTTP. It means "all requirements met."

See subsequent email proposal, to remove empty (unused) definition.

Related issues: 4, 32.

Proposal: [LH] Remove empty (unused) definition.
Resolution: Since the term is not used anywhere in SpecGL, the empty definition is removed from SpecGL. (And a definition will be added to "QA Glossary".) See draft revised text.
LC-33 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: 4. Definitions
Description:

Specification reference.

comment about "Overall" : "deprecated -- An existing feature that has become outdated by a newer construct or is no longer viable. Deprecated features should not be used"

Hmm, "used" may not be a specific enough term. A spec may encourage a UA to support a feature, but discourage an author from producing it, and may be removed in some future version.

See email discussion.

Discussed at 20030410 telecon. Resolved: agree with the suggested clarifications, will make appropriate changes.

Related LC deprecation issues: 33, 40, 99.

Proposal:
Resolution: The language has been clarified and made more specific. See draft revised text.
LC-34 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: Section 3.4 Conformance definition
Description:

Specification reference.

comment about "Overall" : " A checkpoint is satisfied by satisfying all of the individual @@conformance requirements@@. Failing one individual mandatory requirement means that the checkpoint is not satisfied." Is previous sentence required?
Proposal: "Mandatory" should be in the first sentence (which clarifies that recommended or optional requirements need not be satisfied to pass the checkpoint). Then the 2nd sentence can probably be eliminated.

20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: agreed (and "mandatory" is added to the first sentence).

Resolution: The second sentence is dropped as suggested, and the wording of the first sentence is tuned. See draft revised text.
LC-35 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: Section 3.2 Extensibility
Description:

Specification reference.

comment about "Overall" : "...to this specification MUST not contradict nor negate the requirements of this specification." Delete previous statement per comment in checkpoint 9.3.

See extension-group processing plan, including proposals, and subsequent email discussion.

Resolved at 20030505 telecon, as part of the extensibility issue group resolution. Although CP9.3 may be redundant, the redundancy doesn't hurt here. Not every reader is going to logically connect the dots, as has the commentor. Therefore CP9.3 (new CP6.2) should remain as is.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal:
Resolution: Although CP9.3 may be redundant, the redundancy doesn't hurt here. Not every reader is going to logically connect the dots, as has the commentor. Therefore QAWG believes that CP9.3 should remain as is.
LC-36 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: Section 3.1 Normative sections
Description:

Specification reference.

comment about "Overall" : Is the glossary normative?

Discussed at 20030410 telecon. See also subsequent email thread.

SpecGL uses a focused definition of normative -- normative stuff is directly connected to conformance requirements -- that is narrowly applied. By this definition and its application, QAWG has considered that things like Glossary definitions ("5. Definitions") are not normative. SpecGL WD text (20030912) clarified what's normative in the subsection "Normative parts" in "4. Conformance". Also, in "1.6 Usage of terminology", it clarified that defined terms, when used in SpecGL, have the meanings given in the definitions. Because of the latter, it is actually irrelevant from a SpecGL conformance standpoint whether definitions are labelled "normative" or "informative" -- the conformance requirements for SpecGL are clear and unambiguous. This solution was illustrated in WG-only draft revised text.

Discussed again at 20030908 telecon, at which QAWG decided that the most expeditious (and harmless) way to close the issue pre-CR was to call definitions normative.

Related LC whats-normative issues: 36, 65, 106, 108.

Proposal: SpecGL uses a focused definition of normative -- normative stuff is directly connected to conformance requirements -- that is narrowly applied. See proposals in email, to help clarify the definition and its usage in SpecGL. Per 20030414 telecon and subsequent email discussion, we will clarify "what's normative" in Section 3 (Conformance).
Resolution: The "5. Definitions" chapter of SpecGL is labelled "normative" (in SpecGL CR and later versions), is mentioned in the "Normative parts" subsection of "4. Conformance", and "QA Glossary" is moved from "Informative References" to "Normative References".
LC-37 SpecGuide 2003-03-11 SpecGuide Substantive Closed Ian Jacobs Lynne Rosenthal
Title: CP 9.3
Description:

Specification reference.

comment about "9.3 Prevent extensions from contradicting the specification." : I think checkpoint 9.3 should be deleted. I think that it's straightforward that if the spec says "A" and someone else says "not A," then anyone that does "not A" doesn't conform.

See extension-group processing plan, including proposals, and subsequent email discussion.

Resolved at 20030505 telecon, as part of the extensibility issue group resolution. Although CP9.3 may be redundant, the redundancy doesn't hurt here. Not every reader is going to logically connect the dots, as has the commentor. Therefore CP9.3 should remain as is.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal:
Resolution: Although CP9.3 may be redundant, the redundancy doesn't hurt here. Not every reader is going to logically connect the dots, as has the commentor. Therefore has resolved to keep CP9.3 as is.
LC-38 SpecGuide 2003-03-11 SpecGuide Substantive Closed Ian Jacobs Lynne Rosenthal
Title: CP 9.1 and 9.2 combine
Description:

Specification reference.

comment about "9.1 Indicate if the specification is extensible." : I think checkpoints 9.1 and 9.2 should be combined into one.

Discussed at 20030407 telecon. Agreed that they should be combined. Details tbd by SpecGL editors.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal: Combine CP 9.1 and 9.2.
Resolution: Agreed, CP9.1 and 9.2 have been combined (into new CP6.1). See draft revised text.
LC-39 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: CP 8.4 policies for discretionary choices
Description:

Specification reference.

comment about "8.4 Promote consistent handling of discretionary choices." : Can "document the identified policies" be simplified?

Discussed at 20030331 telecon, classified as editorial.

Discussed at 20030410 telecon. AI given to DM (LH & DH assisting) to draft clarification.

New proposal circulated. Discussed and re-resolved at 20030505 telecon. 8.4 will be kept, broadened to "items" instead of "choices", and final text (derived from the new proposal) will be revised and clarified by SpecGL editors.

Discussed again at 20030908 telecon.

Related LC issues: 16, 27, 39.

Proposal: [Proposed wording to be provided by SpecGL editors.] Draft simple clarification, or remove if no QAWG consensus on the clarified text.
Resolution: 8.4 (new 5.4) is kept, conformance requirements are broadened to "items" instead of "choices", and final text is revised and clarified by SpecGL editors. See draft revised text.
LC-40 SpecGuide 2003-03-11 SpecGuide Substantive Closed Ian Jacobs Lynne Rosenthal
Title: GL7 add obsolete features
Description:

Specification reference.

comment about "Guideline 7 Identify the relation between deprecated features and conformance." : I think it's also important to identify obsolete features and provide althernatives to them. E.g., HTML 4.0 obsoleted a few elements.

See email thread.

Discussed at 20030410 telecon. No general consensus yet. There were differing views on what obsolete means (is it closer to deprecation or to removal)? The nature of the HTML example needs to be looked at. LR takes action to discuss with originator, look at HTML, draft proposal for QAWG discussion.

Related LC deprecation issues: 33, 40, 99.

Proposal: [LR Proposed resolution.]
Resolution: Per LR proposed resolution: add a definition of obsolete, add a new checkpoint about obsolete features, add language to GL7 (new GL4) verbiage. See draft revised text.
LC-41 SpecGuide 2003-03-11 SpecGuide Substantive Closed Ian Jacobs Lynne Rosenthal
Title: CP 4.4 derived profile
Description:

Specification reference.

comment about "4.4 If profiles are chosen, address rules for profiles." : What is the definition of a derived profile?

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. See the description of the solution for the profile/module/level issue group, in LC issue 30.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: "Derived profile" is now defined in the new chapter "2. Concepts", and has been added to "Definitions", and that definition is linked from appropriate places in the guidelines. See draft revised text.
LC-42 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: GL3: contradiction? regarding examples
Description:

Specification reference.

comment about "Guideline 3 Specify conformance policy. " : last para, last sentence: Does "possibly provide examples" conflict with MUST provide examples of 1.4?

Discussed and resolved at 20030602 telecon (DRAFT). While the verbiage in GL3 is informative, versus the normative requirements of CP1.4, nevertheless the SpecGL editors will look at the text in question and fix it if MUST does indeed apply in the scope and context of the text in question.

Related LC GL3-policy issues: 42, 96.

Proposal:
Resolution: While the verbiage in GL3 is informative, versus the normative requirements of CP1.4, nevertheless the text in question has been revised, removing "possibly", to avoid any apparent conflict with "MUST provide examples". See draft revised text.
LC-43 OpsGuide 2003-03-12 OpsGuide Substantive Closed Peter Fawcett Lofton Henderson
Title: OpsGL Appendix 1 - Process Document Template
Description:

Specification reference.

comment about "Overall" :

The following comments about the QA Process Document (QAPD), which is linked from the Ops Extech, are addressed in draft revised QAPD text. At 20030512 telecon, proposed and resolved that the new text resolves the issue.

  1. - "test material" is referred to in multiple ways in the document including but not limited to "test material" and "test material type". We could not determine a difference in concept from context nor could we come up with an example that needed both.
  2. - The language in Rec 1 seems odd. Especially the second sentence. "The Checklist should be developed in a public framework. The development of this framework itself is a central area of interest. The QAWG welcomes the participation of interested parties in developing the Checklist."
  3. - The third sentence in Rec 3 also seems a bit odd
  4. - We found a number of uses of "test materials" and "test suite" that did not use the "fill-in" markup.
  5. - DocumentFramework should be Document Framework
  6. - In Test materials contrabuition section in the a/b choice rather than saying tests or series.. say contributions.
  7. - In Test Materials contribution section there is a missing 'span' element around address. above]'/span' mailing list: [address].'p'
  8. - We found the first item in the list under Test Materials Contribution to be some what redundant. At least in our case as any submissions would already go to the list as it is. The test or series of tests, get submitted to the framework. Submitter should also send a notification to the mailing list that will be set up for the Checklist to indicate that he/she has submitted a test to the class="fill-this-in">[test material name] framework.
  9. - There is a typo in section Receipt and review of test contributions "according tot he" should be "according to the"
  10. - In Receipt and Review section list item 1 it currently says: "since it tests a feature that is not in the specification" should it say something more like: "since it tests a recommendation that is not in the specification" as 'feature' is not a universally applicable term.
  11. - In Receipt and Review section list item 3: This item makes the assumption that the test cases will be written in some xml based language. How ever can we make this assumption? What about languages like rdf that don't use an xml syntax?
  12. - this same section (item 3) also has a hanging ']'/span'' on it.
  13. - In Receipt and Review section list item 4: "'li''em'Scenarios'/em' that underlay the particular test layout and its intended scope.'/li'" We found the use of "test layout" to be out of place in the document. it is the only place that this term is used. We believe that it could just as easily use language that's more standard to the document.
  14. - In Receipt and Review section list item 5: The point of this checkpoint is valid, in that if the submitted test doesn't follow the Development Guide it should be rejected, how ever the notion of the Development Guide is introduced here (in this document). We feel that it should be listed earlier in the document as well. perhaps in the requirements, that the test materials be developed in accordance with the Development Guide.
  15. - Typo in last sentence of Receipt and Review section: is "publication arereurned to" should be: "publication are returned to"
  16. - In Test status and review procedures Section: In the second paragraph it reads: "the status of the test is changed to reflect the fact that its validity has been disputed" Earlier we say that "status is changed to "accepted" we should use the same verbiage here like: "state changed to "disputed""
  17. - In Test status and review procedures Section: The last sentence of the second paragraph has very odd wording.
  18. - In Test status and review procedures Section: second paragraph uses term "Task Force" in terms of the maintainers of the test materials. This is another concept that is first introduced in this context. This too would work better if the concept of appointing a "Test Task Force" was done at an earlier step.
  19. - In Test status and review procedures Section in the first item of the list. The label "stable" may not be the best word for case 1 as it doesn't really reflect what is meant. "Error Free" or "Valid" would work better.
  20. - In Test status and review procedures Section in the third item of the list. It reads: "questioned the correctness, consistency, clarity, etc" we think that correctness does not belong.
  21. - In Test status and review procedures Section in the second and third item of the list. "A consensus exists in the community" What community? The w3c, the 'whole internet' the WG (as this is specifically talking about the task force.) We are also fairly uncomfortable with the notion that the task force can be over ridden by the (not clearly defined) community. Either the taskforce agrees or disagrees and the case accepted or not. Unless "community" means WG in which case this might make sense...
  22. - In "Section Status of the test suite according to the above" We feel that the wording of this section heading could be improved quite a lot. We assume this means that if you conform to all of the above. or you've done/are doing all of the above.
  23. - In "Section Status of the test suite according to the above" second paragraph It reads: "It is proposed that the W3C WG representative act as moderator and controller for incoming tests. The moderator is chosen by the [WG name] WG. All tests should be kept for archive purposes, whether they get published or not." Much of this is repeated from above. The new information is that a moderator should be chosen. This is another case where it may be beneficial to introduce the concept of a Moderator earlier in the document. This is already covered above to some degree. But with moderator.
  24. - Both the Documentation and "See above" sections do not have references back to a GL document.
Proposal: Per draft revised QAPD text.
Resolution: Per updated (20030706) draft revised QAPD text.
LC-44 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: Grammatical errors
Description:

Specification reference.

comment about "1.1 Include the scope of the specification" : "

This document is applied and conformance (to this document) achieved as new TRs are being written." (grammatically incorrect, intended meaning unclear) This document applies to new TRs and conformance (to this document) is achieved as they are being written.

As for legacy specification, they may indirectly comply with the spirit or intent of some checkpoints, without actually satisfying all requirements in those checkpoints. (grammatical/spelling error) "legacy specifications"

Within this Specification Guidelines document, the term "specifications' is specifically limited to W3C Technical Reports, even though these guidelines could be applied to other documents. (unbalanced quote marks)

Proposal:
Resolution: These editorial problems have been fixed, by removal of some text and correction of the rest.
LC-45 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: Design goal of guidelines
Description:

Specification reference.

comment about "1.2 Illustrate what is in scope" : 1.2 Class of Product and Audience -- "It is a design goal of these guidelines the WGs can apply them in a common-sense and workable manner."
Proposal: These guidelines are designed so that the WGs can apply ....
Resolution: This has been fixed as suggested. See draft revised text.
LC-46 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: Ambiguity
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " :

I'm finding it difficult to follow which terms are reserved for 'classes of products'. The word 'consumer' is used to describe the classes of products, and itself is listed within the classes of products. For example, I find the following sentence semantically confusing: "For a processor-type specification, the processor is the consumer of an XML vocabulary defined in the specification." "For content-type specifications, there may be one or more consumers that take the content and 'play' it in some way." "Play" refers to a media player, or play refers to "process" ?

Divide this (enumerated) list into processor, consumer, or content? Make the terminology in this area unique, so that there will be no ambiguity? (It could be that the terminology is already unique, but in its current format, I can't be sure.)

Email discussion.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon. Rewrite the GL verbiage for clarification. The detail about sub-dividing the list of classes is to be decided by SpecGL editors during rewrite of the verbiage.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: The verbiage on classes and categories has been revised consistent with the 4-point proposal in email comments. The verbiage has all been moved to a new subsection of the new chapter "2. Concepts", with individual sub- subsections for "specification category" and "class of product". The definitions are in "5. Definitions", and all of these bits are linked from the appropriate places in the guidelines and checkpoints. Some of the text has been rewritten to improve clarity. See draft revised text.
LC-47 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: Spelling error in Example and Techniques
Description:

Specification reference.

comment about "2.2 For each class of product, define the conformance requirements. " : Spelling error in corresponding example and techniques: http://www.w3.org/QA/WG/2003/02/qaframe-spec-extech-20030203#Ck-define-scope "XHTML", not "XHML" :)
Proposal:
Resolution: This has been referred to the SpecET editor to fix.
LC-48 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: Categories of object not previously clearly defined
Description:

Specification reference.

comment about "2.3 Identify which of the categories of object..." :

"Checkpoint 2.3. Identify which of the categories of object are specified in the document as a whole. [Priority 3]"

Reader should be able to understand what 'categories of object' are upon reference - this is the first instance that this phrase is used in this document. Even though a URI is provided to the applicable definition of 'categories of object', the definition itself should introduce the list with this phrase. Eg: "Most specifications can be classified into one of the following categories of object ..."

Discussed at 20030418 telecon.

Discussed and resolved at 20030418 telecon. Rewrite the GL verbiage for clarification. "Categories of object" will be replaced by the standard terminology, "specification category".

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: The verbiage on classes and categories has been rewritten consistent with the 4-point proposal in email comments. There is a new subsection dealing with "class of product" and "specification category" in the new chapter "2. Concepts". The concepts are defined there, added to the "Definitions" chapter, and the discussion is generally improved. The concepts and definitions are linked from their occurrences in appropriate guidelines and checkpoints. See draft revised text.
LC-49 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: Complexity in explanation
Description:

Specification reference.

comment about "Guideline 4 Address the use of profiles to divide the technology." : I can't understand what 'profile' means by this explanation; this should perhaps be simplified?

Discussed at 20030410 telecon.

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. See the description of the solution for the profile/module/level issue group, in LC issue 30.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: The solution for the profile/module/level issue group should clarify the definition and better illustrate the "profile" concept. The given definition has long been in use in a number of venues. The verbiage has been re-organized and put into a separate "Concepts" section, which should improve clarity. When the companion material (and examples) in SpecET are completed, that should further help clarify the concepts. See draft revised text.
LC-50 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: Ambiguity or error?
Description:

Specification reference.

comment about "4.1 Indicate whether or not the use of profiles is mandatory..." : "For example, is content required to conform to one of the profiles, or is there a concept of conformance of content independent of conformance to one of the profiles?"

Ambiguity or error? (I count four 'of's in the second clause! :D)

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. See the description of the solution for the profile/module/level issue group, in LC issue 30.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: The text has been revised to improve readability and clarity. See draft revised text.
LC-51 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: GLs 4, 5 and 6
Description:

Specification reference.

comment about "Overall" : Explanations for profiles, modules and functional levels are vague. Use diagrams for examples ?

Discussed at 20030410 telecon.

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. See the description of the solution for the profile/module/level issue group, in LC issue 30. Note that the specific suggestion to use diagrams was not addressed (SpecGL editors?).

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: The definitions and illustrations of the three concepts -- profiles, modules, and levels -- have been consolidated and improved in one subsection of the new chapter "2. Concepts", as part of the implementation of the solution for the profile/module/level issue group. Some simple diagrams have been included as well. See draft revised text.
LC-52 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: NOT ?
Description:

Specification reference.

comment about "Guideline 9 Allow extensions or NOT!" : 'NOT!' seems rather uncharacteristic and out of line with regards to the remaining document. What about simply "Not" ?

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal:
Resolution: Agreed, "NOT!" has been changed to "not." See draft revised text.
LC-53 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: Ambiguity
Description:

Specification reference.

comment about "Overall" : 1.3 Motivation and Expected Benefits (Introduction) -- "Providing requirements and definitions about conformance topics, as well as guidance in the structure and anatomy of specifications, will foster a mutual understanding among developers of specifications, implementations, and conformance test materials." (Comment: meaning is ambiguous) "foster a mutual understanding among developers about ..."
Proposal:
Resolution: The wording has been clarified to remove confusion. See draft revised text.
LC-54 SpecGuide 2003-03-13 SpecGuide Editorial Closed Stephanie Troeth Lynne Rosenthal
Title: GLs 3, 10, 11, 12 and 13
Description:

Specification reference.

comment about "Overall" : These are all guidelines which refer specifically to conformance. Would it make more sense to number these sequentially (in order to group them together), rather than having the big gap between 3 and 10 ?

Discussed and resolved at 6/2003 QAWG face-to-face. Taking the whole group of related guideline (GL) consolidation and reordering issues together, it was resolved to: merge GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12.

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal:
Resolution: Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9. It was decided to keep GL1 (about scope) as the first guideline, as it corresponds with the first thing that spec authors must address. See draft revised text.
LC-55 SpecGuide 2003-03-13 SpecGuide Substantive Closed Jon Gunderson Lynne Rosenthal
Title: Accessibility
Description:

Specification reference.

comment about "Overall" : Specifications should have a section or the ability to highligh the features of the specification that benefit people with disabilities.

Discussed at 20030331 telecon. QAWG believes that the proposal has merit, but that specifying accessibility requirements is outside of the scope of the QA Framework. It more appropriately belongs in pubrules. Will be referred to Comm team.

Discussed with commenter in a couple of QAWG email threads.

Proposal: Include a requirement that a specification have a section summarizing the accessibility features of the specification
Resolution: QAWG considers that the suggested accessibility requirements and topics are outside of the scope of the QA Framework 1.0. Following email discussion and clarification about aspects of the issue, QAWG has discussed and endorse a fuller issue summary and resolution. A requirement for an informative accessibility summary section would seem fit well in Pubrules (e.g., close to the "1.6.4 Accessibility requirements" sub-section). This possibility will be referred to Comm team for consideration. The resolution of this issue is related to and overlaps the resolution of Last Call issue #55.
LC-56 OpsGuide 2003-03-13 OpsGuide Substantive Closed Jon Gunderson Lynne Rosenthal
Title: Accessibility
Description:

Specification reference.

comment about "Overall" : QA test suites should also include tests that test the accessibility features of a specification based on the accessibility requirements found in other W3C documents. This may require having a specific person in charge of defining and monitoring the inclusion of accessibility features.

Discussion. See previous QAWG issue 12.

Email discussion, including proposed resolution.

Resolved at 20030501 telecon, per prior email proposal.
Proposal: [Originator] Include a requirement in the Operation Guidelines for a person to be responsible for accessibility tests of a specification.

[LH] See email.

Resolution: To the extent that accessibility requirements are written into the target specification as conformance requirements of that specification, then they are adequately covered by the QA Framework guidelines family, specifically by SpecGL and TestGL. No additional QA Framework requirements are needed. In this case, tests of accessibility requirements are within the domain of the (OpsGL-required) Test Moderator's job, and therefore a special accessibility test coordinator is not needed.

If, on the other hand, accessibility requirements are not integrated into the target specification as conformance requirements, then the problem is beyond the scope of the QA Framework as currently defined (for more about this scope point, see email discussion.) In this case, the issue is larger than OpsGL (or TestGL or SpecGL). QAWG issue #12 explored this -- the relationship amongst the QA, WAI, and I18N horizontals.

LC-57 OpsGuide 2003-03-12 OpsGuide Substantive Closed Dominique Hazael-Massieux Lofton Henderson
Title: GL and timeline of a document
Description:

Specification reference.

comment about "Overall" : most guidelines are only applicable at some point in the WG's life, but the GL don't identify this aspect: this is something that absolutely needs to be stressed, and could even be used as a strategy to organize the GL as a whole, e.g. what you need to do before starting a WG, what needs to be done when you start developing a new spec, what needs to be done when you envision building a test suite, etc.

Email discussion, including proposals.

Discussed at 20030512 telecon, and resolved similar to email proposal. There is a strong connection between some guidelines/checkpoints (GL/CP) and the stages in a WG's life. However, the link is ideal, not absolute, and the GL/CP are written in recognition of this. There is already a close association of GL/CP order with WG life cycle. For this reason, and to avoid disruptions to the OpsGL structure and text that don't fix serious problems, QAWG resolved not to attempt a major re-organization of GL/CP based on chronology. However, an informative section will be added to chapter 1 (Introduction), explaining the chronological connections with a graphics or a table.

See also related issue LC-60.1.

Proposal: [Originator] at least, provide a section (an image?) linking the GL or the CP to the milestones of a WG life

[LH] See email (essentially same as Originator's).

Resolution: There is already an approximate association of GL/CP order with WG life cycle (although the linkage cannot always pertain). For this reason, and to minimize disruptions to the OpsGL structure and text, OpsGL will not further re-organize based on chronology. However, per originator's (alternate) proposal, an informative section will be added to chapter 1 (Introduction), explaining the chronological connections with a graphic or a table. See draft revised text.
LC-58 OpsGuide 2003-03-12 OpsGuide Substantive Closed Dominique Hazael-Massieux Lofton Henderson
Title: Process Document requirement is too specific
Description:

Specification reference.

comment about "Overall" : a WG should still be able to comply to a CP without having a QA Process Document.
Proposal: [Originator] replace QA process document references by a documented WG decision?

[LH] See email closure proposal.

Resolution: Thirteen other checkpoints specificy the minimal content of a QAPD. The OpsGL text will be changed to indicate that the QAPD can be a separate standalone document, or a TOC to distributed bits of required QAPD information, or a hybrid. See draft revised text.
LC-59 OpsGuide 2003-03-12 OpsGuide Editorial Closed Dominique Hazael-Massieux Lofton Henderson
Title: Testability concerns
Description:

Specification reference.

comment about "Overall" : The following expressions seemed either hard to apply or to test:

The following seven items discussed in email thread, including proposals, and at 20030512 telecon, and resolved similar to email proposals. Resolutions are illustrated in draft revised text.

  1. "QA deliverables" is very broad and not defined (GL3)

    Per email proposal, the ConfReq is made more precise, the "For example..." in Discussion is replaced with minimal content of "QA deliverables", and broader examples are also presented. See draft revised text.

  2. "ensure" is not testable in a conformance requirement (at least CP 3.2, 6.1)

    Per email proposal, the ConfReq is reworded (in both CP3.2 & 6.1) to replace "ensure" with more testable language. See draft revised text of CP3.2 and CP6.1.

  3. "commensurate" isn't either (CP 4.2)

    Per email proposal, the ConfReq is reworded to eliminate "commensurate" and make it more testable, and a new clarifying sentence is added to Discussion. See draft revised text of CP4.2 and CP2.2.

  4. using the future makes a CP untestable (CP 6.3)

    Per email proposal, the ConfReq is reworded to eliminate the problem. See draft revised text of CP6.3.

  5. "a quality assessment" is not well defined (CP 7.1)

    Per email proposal, a handful of examples are added to Discussion, to illustrate what goes into a TM quality assessment, and a more exhaustive list/template will be put into OpsET. See draft revised text of CP7.1.

  6. "sufficient" is not testable (CP 7.2)

    The comment is moot for the occurrence of "sufficient", as it is in the statement of the CP, which is merely its "title" (not normative). However "commensurate" in the ConfReq is fixed, per email proposal, and a clarifying sentence is added to Discussion. See draft revised text of CP7.2.

  7. 'all' is not testable (CP 7.3)

    Per email proposal, delete the occurrence of "all" in ConfReq. See draft revised text of CP7.3.

Proposal:
Resolution:
LC-60 OpsGuide 2003-03-14 OpsGuide Substantive Closed Patrick Curran Lofton Henderson
Title: Structure/Organization of Guidelines
Description:

Specification reference.

comment about "Overall" : Comments on the structure and organization of the guidelines and checkpoints of SpecGL.
  1. Overall, I think we have the right checkpoints, but I think they would be easier to understand if they were grouped chronologically. Start with what needs to be done when the WG is formed, and proceed through the spec and test development cycle to the maintenance phase, as we have tried to do with the restructuring of TestGL.

    Email discussion, including proposals.

    Discussed at 20030512 telecon, and resolved similar to email proposal. QAWG resolved to minimize large changes to the OpsGL structure and text that don't fix serious problems. Therefore, OpsGL will not be reorganized based on chronology. Further, there is already an approximate association of guidelines and checkpoints (GL/CP) with the ideal point of applicability in a WG life, and this is partially reflected in the GL/CP ordering. An informative section is added to chapter 1 (Introduction), explaining these chronological connections with a graphic or a table. See also related issue LC-57.

  2. Guideline 1: I find checkpoints 1.1 - 1.3 very confusing. These are "compound checkpoints" that incorporate multiple other checkpoints. We don't use this structure anywhere else in our docs - why here? The document states "This seven-point enumeration is derived from the proposal to the QA mail list, after the 4/2001 QA Workshop", but how we got here is not really interesting to the reader.

    The 'sub-checkpoints' on the 'left hand side' of the table are all spec-related. Either these duplicate checkpoints from SpecGL, or they should be incorporated into that document. Those on the 'right hand side' are operations-related, but they seem to overlap with other checkpoints specified in this document.

    The remaining checkpoints under guideline 1 are different kinds of beasts (not compound) and as such, the transition to them seems somewhat abrupt.

    Recommendation: drop the compound structure, make sure that all the spec-related sub-checkpoints are covered by SpecGL, and move the 'test materials' sub-checkpoints into the body of this doc.

    Email discussion , including proposed resolution.

    Discussed and resolved at 20030501 telecon. See LC-3 for description and resolution. See draft revised text of CP1.1 (as well as CP1.2, CP1.3).

  3. Guideline 2: I'd say "Allocate resources..." rather than "Define resources..."

    See email proposal. Resolved per this proposal at 20030516 telecon. The wording of GL1 and GL2 is changed consistently, to emphasize that these are about commitment, and GL2 now starts with "Commit to resource level..." See draft revised text.

  4. Guideline 3 begins with some text ("The benefits of....") that seems to be a rationale. Other Guidelines jump straight into the Conformance requirements - this one should too.

    Discussed in email. Withdrawn by originator while discussing clarification of the request.

  5. How is checkpoint 3.1 different from 1.5?

    See email proposal. Resolved per this proposal at 20030516 telecon. Add "Related checkpoints" to each of CP1.5 and CP3.1, pointing to the other and explaining the distinction. See draft revised text.

  6. Checkpoint 3.2 doesn't seem to be directly related to the Guideline under which it's classified. The Note for 3.2 points out that checkpoint 8.2 is related - doesn't this suggest that the overall structure should be re-worked (if they're related, why are they so far apart numerically?)

    See email proposal. Resolved per this proposal at 20030516 telecon. This and several other aspects of CP3.2 and 8.2 are clarified and resolved all at once in the proposal. See draft revised text.

  7. Guideline 4: Probably should be #1 (it's the first chronologically). When we summarized this document in our outreach presentation we made checkpoint 4.1 the first bullet item...

    See email proposal. Resolved per this proposal at 20030516 telecon. GL1/2 are about early *commitment* to QA and QA deliverables. Possibly even in the Charter, before the WG is rolling. GL4 is about process and operations within the functioning WG. It is equally arguable that this is the best order. Per the 20030512 resolution of LC-60.1, we will avoid any re-ordering or re-grouping of GL/CP except to fix something that is really broken. Accordingly, it is resolved to keep the current order here.

  8. Checkpoints 4.1 and 4.2 would seem to belong in Guideline 2 (define/allocate resources) rather than here?

    See email proposal. Resolved per this proposal at 20030516 telecon. GL1 and 2 are about early commitment (ideally Charter) to QA levels, staffing levels, etc. GL4 is about performance -- specific staff assignments -- going forward. To each of CP2.2, CP4.1 and CP4.2, added reciprocal clarifications about the others, and disambiguate the similarity between them. See draft revised text.

  9. The Discussion for checkpoint 4.3 says "To summarize...". This implies that somewhere there's a more detailed description of what the QAPD must address, but I don't think there is. This checkpoint really seems to amount to "document how you meet these other checkpoints", yet the list of "other checkpoints" that must be implemented is incomplete. Why doesn't this simply require that *all* checkpoints be documented?

    See email proposal. Resolved per this proposal at 20030516 telecon. This is clarified, in conjunction with solution to LC-58, by adding a new paragraph at the beginning of CP4.3 "Discussion". The discussion is also expanded to specifically mention the other QAPD-related checkpoints, and a "Related checkpoints" is added. See draft revised text.

  10. Checkpoint 4.5 uses the ambiguous term "framework" (we tried to avoid this in our re-write of TestGL).

    Discussed in 20030516 telecon. See subsequent email proposal. Resolved per email proposal by email (default). LC-72.7 also raises this issue. Other comments indicate confusion why CP4.5 is here at all, instead of TestGL. All of these are addressed in the proposal, for complete replacement text. The text clarifies why the topic is dealt with in OpsGL (instead of or in addition to TestGL), and changes the wording "QA framework" to "framework for test materials development". ("Framework" is still used, in lieu of specific proposals for a better phrasing.) The checkpoint is also moved, to become the first checkpoint of Guideline 5. See draft revised text.

  11. Checkpoint 4.6 addresses branding - another argument for a chronological rather than a 'logical' grouping (this should be at the end).

    See email proposal. Resolved per this proposal at 20030516 telecon. Because branding policy decision has potential knock-on effects on other aspects of the WGs QA life, process, TM, and operations, it is equally arguable that its early consideration is beneficial. Given the resolution that we are not going to do a major chronological re-org (LC-60.1) except to fix really broken things, it is resolved to keep it as is.

  12. Guideline 5: Checkpoint 5.2 - Define a contribution process. Why only priority 2? Without a contribution process you have nothing, surely?

    See email proposal. Resolved per this proposal at 20030516 telecon. There are test suites that have been built entirely without any formal or defined contribution process -- mostly by a single person, with ad-hoc acceptance procedures for internal contributions. Certainly this is not optimal, but also not critical in all cases (perhaps in some). If possible, we want to avoid increasing the number of P1 checkpoints unless we feel they are really critical/essential. Therefore, it is resolved to keep it as is.

  13. Guideline 6: Checkpoint 6.1 contains a mixture of stuff. The guideline addresses "publication" but much of this checkpoint addresses "management". Moreover, the bullet items in the Discussion section don't seem to relate to repositories at all.

    See email proposal. Resolved per this proposal by email (default). Take the bullet list out of CP6.1, fine tune the wording, and put it in the verbiage of GL6. Add comment to the GL6 verbiage explaining the preprequisite relationship of some management aspects to publishing, and therefore their appropriate treatment of those here in GL6. See draft revised text.

  14. Checkpoint 6.2 (defining the license for published test materials) is closely related to 5.3 (defining the license for submissions). Should these be grouped together (under a guideline that addresses submission processes)?

    See email proposal. Resolved per this proposal at 20030516 telecon. Consistent with the LC-60.1 resolution of "minimize improvements that don't fix something really broken", it was resolved that we should clarify and cross link them, instead of trying to combine them. Part of the clarification includes wording changes approved by W3C Legal, and other interested parties in the TM license issue. See draft revised text.

  15. Guideline 7: This guideline is labelled "plan the transfer of test materials to W3C if needed", and explicitly states "all of the checkpoints... are not applicable if the WG does not transfer..." (should be "none of the checkpoints are applicable if..."). However, all the checkpoints seem to apply whether or not the materials are "transferred". It's obviously important to review the quality of submitted tests, to ensure that we have the staffing to deal with submissions, and to resolve IPR issues.

    See email proposal. Resolved per this proposal by email (default). All of the submission stuff does get addressed in GL5. GL7 is about the wholesale transfer of an entire externally written test suite. While this could be considered as a "submission" (therefore redundant with some of GL5), on the other hand the sense of GL5 is more about piecemeal submission during the building of test materials. QAWG has had experience moderating such a transfer, which is why GL7 exists separately -- it's useful to have it in one place, even if there are parallels elsewhere in OpsGL for piecemeal planning, development, staffing, submission, etc. Resolved to fine tune the wording of the GL7 verbiage and add some more rationale/explanation. See draft revised text.

For LC-60.3 through LC-60.15, see email, including some proposals.

Related LC QA-commit issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal: [Originator] I think we have approximately the right checkpoints, but I think they would be easier to understand if they were grouped chronologically. A possible set of guidelines might be:
  • Charter
  • Allocation of resources
  • Planning & synchronization
  • Test submission/development
  • Test management
  • Test publication
  • Conformance testing (test usage)
  • Maintenance
[LH alternate proposal] See email.
Resolution: See the resolutions for each of the individual sub-issues.
LC-61 SpecGuide 2003-03-14 SpecGuide Substantive Closed Susan Lesch Lynne Rosenthal
Title: document product class
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " : Guideline 2 could give "document" and "resource" either as product classes or as examples of the "content" class. (Checkpoint 2.1 does finally mention "document.")

Email comments (20030417).

Discussed at 20030418 telecon.

Discussed and resolved at 20030418 telecon. We don't see documents and resources as examples of 'content' class (perhaps some better definition of 'content' class is needed?). We agreed that a 'specification' class should be added.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: A 'specification' class has been added. See draft revised text.
LC-62 SpecGuide 2003-03-14 SpecGuide Editorial Closed Susan Lesch Lynne Rosenthal
Title: typos
Description:

Specification reference.

comment about "Overall" : In section 3.3, s/This Operational Guidelines document/This Specification Guidelines document/

In the guideline title and table of contents entry for Guideline 12 and in 3.4 last par., "pro forma" is two words.

The table of contents link to section 3.1 is broken.

(These probably have been reported by now but just in case.)

Proposal:
Resolution: These have been fixed.
LC-63 SpecGuide 2003-03-14 SpecGuide Editorial Closed Susan Lesch Lynne Rosenthal
Title: Table of Contents
Description:

Specification reference.

comment about "Overall" : Would the table of contents in QA Framework specs that have guidelines be easier to follow if non-Guideline sections all appeared by number in the table of contents? E.g. "3. Conformance" could be followed by "3.1 Normative sections" rather than by "1. Normative sections"? The difference between for example "checkpoint 1.1" and "section 1.1" (both about scope) then would be distinct and easier to talk about.

Discussed at 20030407 telecon. Agreed to implement suggested change -- subsections in TOC will be numbered N.M (M=1,2...), instead of just M.

Related LC Section/GL/CP numbering issues: 63, 77.ET-3, 91.

Proposal: Subsections in TOC will be numbered N.M (M=1,2...), instead of just M.
Resolution: Changed as suggested. Subsections in TOC are now numbered N.M (M=1,2...), instead of just M. See draft revised text.
LC-64 SpecGuide 2003-03-14 SpecGuide Editorial Closed Susan Lesch Lynne Rosenthal
Title: conformance terms
Description:

Specification reference.

comment about "3.2 If special conformance terms are used, include a definition..." : Checkpoint 3.2 says: "the specification MUST be defined, either by reference or by including the definition in the text." Did you mean to say that the whole spec is considered to be "defined" through references and definitions? In that case, please ignore this comment.

Email discussion, including proposal for closure.

Proposal: Could read something like: "terms used to describe conformance MUST be defined, either by reference or by including the definition in the text." (I'm afraid "conformance terms..." might mean you'd have to define "terms.")
Resolution: Editorial glitch, fixed by inserting "any conformance concepts used in" at the beginning of the conformance requirements statement. See draft revised text.
LC-65 SpecGuide 2003-03-14 SpecGuide Editorial Closed Susan Lesch Lynne Rosenthal
Title: sentences and paragraphs (section 3.1)
Description:

Specification reference.

comment about "Overall" : In section 3.1, "sentences" becomes "typically, one paragraph...." Is a sentence containing an RFC 2119 key word a unit of being normative, or do you mean that a paragraph containing such a sentence is or can be the unit? I'm not sure if it is important to draw a line. Section 4's definition of normative says "text in a specification which is prescriptive or contains conformance requirements" which seems to mean any text with no boundaries (for example, like section, chapter, slice).

Discussed at 20030410 telecon. See also subsequent email thread. Agreed that section 3.1 has some problems that will be fixed: it is not comprehensive (omits some normative stuff); and has misleading characterization of the normative units of text.

See mail thread (20030606), which includes suggested resolutions.

See also these directly related QAWG issues: issue #98, issue #99, issue #110.AR-001. These were marked as closed at Tokyo and after, however later email and telecon decisions, including new agreed definitions and some samples, suggest reviewing the earlier closures.

Related LC whats-normative issues: 36, 65, 106, 108.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal:
Resolution: Agreed that there were problems here. The language in section 3.1 (now 4.1) has been improved for accuracy, and the "what's normative" list has been edited for completeness. The terms normative and informative have been linked to their (sec.5) definitions. See draft revised text.
LC-66 SpecGuide 2003-03-14 SpecGuide Substantive Closed Susan Lesch Lynne Rosenthal
Title: edition and version DoVs
Description:

Specification reference.

comment about "Overall" : Guidelines 4-6 and the section 4 definitions are great descriptions of profiles, modules, and levels, thanks. Are "editions" and "versions" DoVs? If for example, a requirement changed between Version 1.0 and Version 1.1 of some specification, so that a 1.0 processor could not read 1.1, that might be a "variability."

Email discussion (20030331).

Email discussion (20030417).

Email comments, including proposal (20030418).

Discussed at 20030421 telecon, resolved per email proposal.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: [Originator] If you think edition and version do matter, they could be addressed in section 1.8, or in a separate Guideline. [LR] See email proposal. [LH] See email.
Resolution: Per email proposal, version and edition are not DoV. DoV are concerned with conformance to a single, specific version/edition of a specification at a time. This detail has been clarified in the new "2. Concepts" chapter, sub-section 2.1. See draft revised text.
LC-67 SpecGuide 2003-03-14 SpecGuide Substantive Closed Susan Lesch Lynne Rosenthal
Title: limits of RFC 2119 key words
Description:

Specification reference.

comment about "13.1 Use conformance key words." :
  1. [Part 1]: "the specification MUST use RFC 2119 keywords to denote whether or not requirements are mandatory, optional, or suggested" is Priority 1. Must all testable statements and or test assertions (sorry I'm not clear on the difference between them and maybe that needs to be clarified, too) contain RFC 2119 key words?

    Discussion. Note counter example in UAAG 1.0: "This document demands substantially more conformance flexibility than can be achieved using the terms "must", "should", and "may" alone, as defined in RFC 2119 [RFC2119]. Where "must", "should", "required", and "may" appear in this document, they are used consistently with RFC 2119 for a chosen conformance profile. The imperative voice (e.g., "Allow configuration ...") used in the checkpoint provisions implies "must", but a user agent is only obligated to satisfy the requirements of a chosen conformance profile."

    Related discussion about whether testability is an inherent part of "conformance requirement" in this email sub-thread (of a thread about some OpsGL issues).

    See mail thread (20030606), which includes suggested resolutions.

    See also these directly related QAWG issues: issue #98, issue #99, issue #110.AR-001. These were marked as closed at Tokyo and after, however later email and telecon decisions, including new agreed definitions and some samples, suggest reviewing the earlier closures.

    Discussed at 6/2003 QAWG face-to-face -- clarification solicited about UAAG10 approach, in this email thread. Discussion resumed on email list, with copious comments generated.

    Discussed again and resolved at 20030707 telecon. Resolution is: SpecGL (old CP13.1) will strongly encourage RFC2119 keywords as the most widely applicable and usually the best method, but exemptions are allowed, provided that: the spec unambiguously defines how its conformance requirements are identified (even if RFC2119 keywords are used); and, the rationale for not using RFC2119 keywords is explained in the spec. See draft revised text.

  2. [Part 2]: RFC 2119 [1] section "6. Guidance in the use of these Imperatives" puts limits [on] their use. (I mentioned this to the QAWG about a year ago.) They musn't be used only to ask implementers to do something a Working Group would like to see.

    Email discussion (20030418).

    Discussed and resolved 2nd part -- guidance about RFC keyword usage -- at 20030421 telecon. Resolved: SpecGL won't try to legislate correct use of RFC2119 keywords. However, SpecET (Examples & Techniques) will mention that correct-use guidance is given in Section 6 of RFC2119, and will point to it. QAWG is concerned that quoting the particular wording of RFC2119 -- with reflects a particular perspective (communications & protocols) and orientation -- creates confusion rather than clarity, in some conformance contexts. There is also considerable disagreement (see the email dialogs about this issue) within W3C about the "correct use" assertions, and there is a significant body of current practice that is contrary to them. QAWG thinks that it would be useful to consider a joint Comm-QAWG project to draft a W3C guidance Note on this topic.

Subsequent to resolution of 2nd part, additional comments & discussion occurred on QAIG list.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal: [Originator] One way to solve this is to quote or paraphrase and link to the RFC.

"Imperatives of the type defined in this memo must be used with care and sparingly. In particular, they MUST only be used where it is actually required for interoperation or to limit behavior which has potential for causing harm (e.g., limiting retransmisssions) For example, they must not be used to try to impose a particular method on implementors where the method is not required for interoperability."

[LH] See email proposal: don't legislate it in SpecGL; develop a Note about it and reference that from SpecET.

[1] http://www.rfc-editor.org/rfc/rfc2119.txt

Resolution: 2nd part: SpecGL won't try to legislate correct use of RFC2119 keywords. SpecET (Examples & Techniques) will mention that correct-use guidance is given in Section 6 of RFC2119, and point to it. There is concern that quoting the particular wording of RFC2119 -- which reflects a particular perspective (communications & protocols) and orientation -- "muddies the water" in some other contexts. QAWG thinks that it would be useful to investigate a joint Comm-QAWG project to draft a guidance Note on this topic.
LC-68 IntroGuide 2003-03-14 IntroGuide Editorial Closed Susan Lesch Lofton Henderson
Title: Intro draft
Description:

Specification reference.

comment about "General: miscellaneous & other" : I found Ian Jacob's SpecGL edits [1] in your archive and would like to send ideas for the QA Framework Introduction. I expect they will be ready at [2] by 14 March 24:00 Pacific.
Proposal: Use or not, as you see fit. As this is my last Last Call comment, I just want to say that it must have been a monumental task to put the framework together. You've made it look elegant, and easy to use. Best wishes for your project.
Resolution: Accepted. Thank you for the excellent contribution. It has been reviewed and adopted as submitted. See draft revised text.
LC-69 IntroGuide 2003-03-15 IntroGuide Editorial Closed Colleen Evans Lofton Henderson
Title: Several comments on various parts of Introduction
Description:

Specification reference.

comment about "General: miscellaneous & other" : [Entered into form by LH. Some are significant editorial, but all are classified as "Editorial" because there are no conformance implications or suggestions for major document reorganization.]

Review comments:

  1. In general it provides useful guidance on how to use the Framework, defining audience and WG activity applicability for each document.

    [No action required.] Thank you for the positive feedback -- it is useful to know that it is fulfilling its purpose.

  2. Is this a Working Draft or Last Call Working Draft? Title indicates the former.

    W3C process and pubrules do not distinguish between WD and Last Call WD in the dated subtitle. The distinction is required in the "Status of this document" section.

  3. Interspersed usage of the terms 'document', 'guideline', and 'specification' to describe the QA Framework documents could be confusing. E.g., paragraph 4 of Status: "... It is anticipated that this specification will eventually progress, along with its Operational Guidelines and Specification Guidelines companions, to Candidate Recommendation (CR) and beyond. The timing of progression of this specification will be determined by the progression of the companion guidelines documents." Similarly, 'TR', 'standard', 'specification', and 'recommendation' are used interchangeably to describe the output of WGs.

    This has been fixed as suggested -- document is used uniformly when "Introduction" refers to itself.

  4. Section 1.3, first sentence: "The last underscores a key reality of improved quality practices associated with W3C technical reports". Not clear what 'the last' is (previous section?).

    This has been fixed as a result of the substantial rewrite of "QA Framework: Introduction" from issue #68 -- as a result of consolidation, that text is gone.

  5. Section 1.4, paragraph 3, first sentence incomplete? "While some might perceive QA projects as a regrettable drain on WG resources, there is ample experience, both within W3C as well as other standards venues, that shows significant improvement to the products of the WGs."

    This has been fixed as a result the substantial rewrite of "QA Framework: Introduction" from issue #68 -- as a result of consolidation, that text is gone.

  6. Sections 1.3 (paragraph 1) and 1.4 (paragraphs 2 and 3) contain general justification arguments for QA efforts in WGs - may be more appropriate content for Section 1.2.

    This has changed as a result of the substantial rewrite of "QA Framework: Introduction" from issue #68.

  7. Section 3.1 Application Domain - does this belong in Section 3 (Structure and content of Framework documents)? Seems more like Section 1 (Overview) content where target audience is covered.

    This has changed as a result of the substantial rewrite of "QA Framework: Introduction" from issue #68.

  8. Sections 3.5.2 - 3.5.5 describe each document - information on content, audience, and objective. It may help readability to use a consistent order for presenting this information across sections.

    It is our intent that the order be uniform where the framework family is discussed, unless driven by another criterion such as roles or scenarios. We are not aware of any non-uniformities in the revised text.

  9. Section 3.5.4 Single item bullet list?

    This has changed as a result of the substantial rewrite of "QA Framework: Introduction" from issue #68.

  10. Section 4.1.3 Useful breakout of document relevance by role within a WG.

    [No action required.] Thank you for the positive feedback.

  11. Section 4.2 Provides a good life cycle view of the relationship between Framework documents and WG activities. A table summary might be useful as well.

    We will consider this for a future version of "QA Framework: Introduction" (which is now being published as a "Note"). We have put a "chronology diagram" into "QA Framework: Specification Guidelines." One lesson from there is that the chronology is only approximate, and there can be wide variation without substantially diluting the benefits. A submission of a draft table would be welcomed and considered for inclusion.

  12. Section 4.1.3 "WG-TS moderator" - Section 4.2.2 "test materials (QA) moderator". Same role?

    Yes. The 4.2.2 wording has been changed to simply, "QA moderator".

  13. Section 4.2.3, paragraph 4, second sentence is unclear: "Normally, this should not be considered as a good time to bring a specification for 'Specification Guidelines' conformance, as the latter could significantly disrupt and restructure the specification.". Is 'the latter' referring to bringing a spec to specification guidelines conformance, or something in a previous sentence?

    The first choice (bringing spec to conformance). The sentence has been revised to remove the ambiguity.

  14. Section 4.2.5. Intra-WG build of test materials calls for an acceptance procedure for the individual bits. Import and assemble only call for quality assessment and assessment criteria - is an acceptance procedure required / implied?

    This has been revised to clarify.

  15. (Editorial) Usage of Working Group vs. WG inconsistent throughout

    As a general rule, we have used "WG" a lot for brevity. We have tried to have a full "Working Group" spelled out nearby, e.g., somewhere early in the same section. We have used markup extensively to define the WG acronym.

  16. Inconsistent bullet list punctuation (';' vs. ',' vs. nothing at line end, etc.)

    This has been made consistent (";").

  17. Section 1.2, paragraph 1 "...." at end of first sentence

    Fixed.

  18. Section 4.2.5, paragraph 2 "? -- as " in middle of second sentence

    Fixed.

Proposal:
Resolution:
LC-70 OpsGuide 2003-03-15 OpsGuide Substantive Closed Phill Jenkins Lofton Henderson
Title: Checklist format issue
Description:

Specification reference.

comment about "Overall" : [Entered into form by LH. Comment applies to Checklists of SpecGL and TestGL as well OpsGL.]

I have noticed a number of "checklists" being proliferated on the W3C pages. I have a strong recommendation for improving the adaptability of the checklist format - namely the number of column in the layout. For example, today most have a column for the number of the checkpoint, the description of the checkpoint, and then a number of columns for YES, NO, N/A. Please consider adapting the following format:

  • Top matter: Version, date, owner, etc
  • Column 1 10% Checkpoint number
  • Column 2 40% Description
  • Column 3 10% YES, NO, N/A, Planned
  • Column 4 40% Comments
  • Bottom matter: Footnotes, exceptions, references, etc.

an example we find useful is at http://www-3.ibm.com/able/accesssoftware.html .

[[Following is (unarchived) response comment from Ian Jacobs: We redesigned the UAAG 1.0 checklist [1] based on earlier comments from you [1]. We don't specify fixed widths for table columns.

Discussed and resolved at 20030707 telecon. We need to clarify the purpose of the checklists. They are intended to be implementation conformance reporting forms. Accordingly, the category of "planned" is not considered appropriate, nor is the provision of a "Comments" field. Without the 40% "comments" field, the remainder of the reformatting proposal would need re-adjustment. QAWG has identified and intends to produce another sort of form where a comments field would indeed be appropriate -- a "test material" for the Guidelines documents, in the form of a questionnaire, which has a question/item per-conformance-requirement (as opposed to per-checkpoint).

Proposal:
Resolution: The QA Guidelines checklists are intended to be implementation conformance reporting forms. Accordingly, the category of "planned" is not considered appropriate, nor is the provision of a "Comments" field. Without the 40% "comments" field, the remainder of the reformatting proposal would need significant re-adjustment. QAWG has identified and intends to produce another sort of form where a comments field would indeed be appropriate -- a "test material" for the Guidelines documents, in the form of a questionnaire, which has a question/item per-conformance-requirement (as opposed to per-checkpoint). See clarification of checklist purpose in draft revised text.
LC-71 IntroGuide 2003-03-15 IntroGuide Substantive Closed Leonid Arbouzov Lofton Henderson
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "General: miscellaneous & other" : [Entered into form by LH.]

Six substantive and editorial issues,

  1. [IN-1]: Define "conformance" and "interoperability".

    Agreed, the terms should be defined. Note that "conformance" is defined in the "QA Glossary", but "interoperability" is not. An action item has been given to the glossary owners to add this. A note has been added to "Terminology", to refer to "QA Glossary" for definitions.

  2. [IN-2]: Add "coverage measurement tools" to list in "Technical Assets".

    Agreed, this has been added.

  3. [IN-3]: [editorial] Define the "WAI" acronym in section 2.

    Agreed, the first occurrence of WAI is defined (in the text substantially revised per LC-68.)

  4. [IN-4]: [editorial] Fix last bullet ("assets") in the resource list in section 2.

    Done, "assets" is now "technical assets" (per the 2nd following subsection).

  5. [IN-5]: Since Intro is entirely non-normative, consistently refer to it as "document" instead of "specification".

    Agreed, "Introduction" is uniformly referred to as "document" now.

  6. [IN-6]: Clarify what are the items in the bullet list of 4.2.3.

    This has been clarified by identifying them as SpecGL topics, and adding a Guideline # reference to each list item.)

on "QA Framework: Introduction" are included in the document at

http://www.w3.org/QA/WG/2003/03/qareview20030314.html ,

which was submitted to QA by XML Schema on 14 Mar 2003:

http://lists.w3.org/Archives/Public/www-qa/2003Mar/0079.html .

Proposal:
Resolution:
LC-72 OpsGuide 2003-03-15 OpsGuide Substantive Closed Leonid Arbouzov Lofton Henderson
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "Overall" : [Entered into form by LH.]

The following thirteen substantive and editorial issues on "QA Framework: Operational Guidelines" are found in the (linked) document:

which was submitted to QA by XML Schema on 14 Mar 2003:

  1. [OG-1]: Editorial -- unclear wording in section 1.1.

    Proposal. Accept Originator's proposed rewording. Resolved, proposal endorsed at 20030512 telecon. See draft revised text (4th bullet).

  2. [OG-2]: 7-level table creates confusing conformance requirements that contradict the priorities/degrees system.

    Email discussion , including proposed resolution.

    Discussed and resolved at 20030501 telecon. See LC-3 for description and resolution. See draft revised text of CP1.1 (as well as CP1.2, CP1.3).

  3. [OG-3]: CP1.1 is priority 1, but requires checkpoints of other priorities.

    Email discussion , including proposed resolution.

    Discussed at 20030501 telecon. Becomes moot as a result of the resolution of LC-3 (and previous sub-issue.) See draft revised text of CP1.1 (as well as CP1.2, CP1.3).

  4. [OG-4]: GL3 wording emphasizes interoperable implementations, should instead emphasize conformant implementations.

    Proposal. Accept Originator's proposed rewording. Resolved, proposal endorsed at 20030512 telecon. See draft revised text (3rd bullet).

  5. [OG-5]: CP3.2, "support specification versioning/errata" should be priority 1 instead of 3.

    Proposal. Accept originators request -- change p3 to p1. Discussed at 20030512 telecon and 20030516 telecon. Resolved to change priority to P1, and also to rephrase requirements in terms of capability to unambiguosly associate spec and TM versions (for which TM infrastructure support is one method of achieving.) Relationship to CP8.2 also documented. See draft revised text.

  6. [OG-6]: Editorial -- poor wording in CP4.3.

    Proposal. Agree that it's poor wording, change "minimally addresses all of the topics" to "addresses at least all of the topics". Resolved, proposal endorsed at 20030512 telecon. See draft revised text.

  7. [OG-7]: Editorial -- CP4.5 uses "QA Framework" in a different (but undefined) sense that the OpsGL title.

    Proposal. Agree, the same phrase is used with a different meaning. Change the CP wording from "Define the QA framework for test materials development" to "Define a framework for test materials development." Resolved, proposal endorsed at 20030512 telecon. (The checkpoint is also moved, to become the first checkpoint of Guideline 5.) See draft revised text.

  8. [OG-8]: CP4.5 should be priority 2 instead of priority 1.

    Email proposal [LH]: Accept originator comment -- defining the framework for TM development is not critical enough to be Priority 1. Lower to priority 2. Discussed and resolved as proposed at 20030512 telecon. (The checkpoint is also moved, to become the first checkpoint of Guideline 5.) See draft revised text.

  9. [OG-9]: Editorial -- ambiguous wording in CP5.4.

    Proposal. Agree with the intent of the comment. Change "..the procedure MUST minimally address criteria for.." to "..the procedure MUST address at least criteria for.." Resolved, proposal endorsed at 20030512 telecon. See draft revised text.

  10. [OG-10]: CP6.2 -- "address TM licenses" -- should be substantively modified.

    Email discussion & proposal, and discussion at 20030501 telecon, which provided an interim resolution. Interim Resolution -- describe the attributes of the licenses, and include a caveat about open issues involving the Document License. See interim draft revised text. It was also resolved that a task team of interested parties will be convened to close the remaining disagreements and possibly design a Test Materials license. The task team will either be formed and operate in a broader venue than QAWG, or will operate under QAWG if a' priori commitments of adequate resources are made by the interested parties. These interim results will be updated with the task team results.

    License task team results. Extensive discussion by TM-license task team at a 20030611 telecon, involving all identified principal stakeholders (member companies, QAWG, W3C Legal, etc): see public summary and detailed minutes (for now, member-only). Resolution: W3C will continue to pursue the Document and Software licenses as the mechanisms for governing test suites. A new TM license is not seen as needed, and there are no major defects that compromise the usability and applicability the Software License or the Document License for W3C Test Materials, or of both together applied to different components of a complete Test Materials. W3C Legal will continue to investigate a number of possible (relatively minor) improvements. Guidance in the licenses FAQ will be investigated. QAWG will continue to look at both adjustments to its Guidelines documents (e.g., TestGL guidance on partitioning the TM components suitably for different license application), and possibly some further FAQ-like guidance in "Operational Examples & Techniques" (specifically for Checkpoint 6.2). See draft revised text.

  11. [OG-11]: CP6.3 arguments against publishing TM in the TR space are not convincing, should be improved (proposal included).

    Proposal. Accept the proposed rewording. Resolved, proposal endorsed at 20030512 telecon. See draft revised text.

  12. [OG-12]: Problems with CP6.5 ("publish test results"), discussion of test harnesses, should be fixed.

    Email proposal [LH]: Agree with intent of Originator. In Discussion, change the sentence, "Such publication should include or describe a test harness that would allow anyone to reproduce the results" to "Such publication should include or describe a test harness that allows reproduction of the results." Resolved, proposal endorsed at 20030512 telecon. See draft revised text.

  13. [OG-13]: CP8.2 (versioning/errata in maintenance) should be priority 1 instead of 2.

    Proposal to accept (change priority 2 to priority 1 in CP8.2 [versioning/errata in maintenance]) was discussed at 20030512 telecon and 20030516 telecon. Resolved to change priority to P1, and also to clarify scope of "versions" (minimally, "editions"). Relationship to CP3.2 also documented. See draft revised text.

Related LC QA-commit issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal:
Resolution: The 13 individual issues are resolved as individually documented above.
LC-73 SpecGuide 2003-03-15 SpecGuide Substantive Closed Leonid Arbouzov Lynne Rosenthal
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "Overall" : [Entered into form by LH.]

The following nine substantive and editorial issues on "QA Framework: Specification Guidelines" are found in the (linked) document:

which was submitted to QA by XML Schema on 14 Mar 2003:

  1. [SG-1]: Editorial (SoTD).

    Resolved, it has been fixed as suggested. See draft revised text.

  2. [SG-2]: Remove all "not applicable if..." statements.

    Email discussion, including proposal.

    Discussed and resolved at 20030602 telecon (DRAFT). QAWG agrees that there are some editorial problems in these clauses. As pointed out, "It" is undefined (should be "This checkpoint"). There is also inconsistent usage, e.g., some checkpoints in Guideline 9 have applicability exclusions, and some do not. In general, however, it is thought useful that there be an explicit applicability exclusion, rather than leaving the reader to interpret this (*only* if there is an applicability exclusion can the n/a box in the ICS be checked). SpecGL editors will improve the language, clarify the ambiguous statements, and apply consistent structuring, for all checkpoints that have applicability exclusions. See draft revised text.

  3. [SG-3]: "Enumerate all CoP" is unreasonable requirement.

    Email comments (20030417), including proposals.

    Discussed and resolved at 20030418 telecon. We think that perhaps originator is confusing "all classes of product" with "all products". Nevertheless, the wording will be improved by: remove "all" from the checkpoint statement; and, change 1st bullet from "MUST list the classes of product it addresses" to "MUST list the classes of product for which it defines conformance requirements". See draft revised text.

  4. [SG-4]: Editorial (8.5, ConfReq, change "profiles").

    Resolved, this has been fixed as suggested. See draft revised text.

  5. [SG-5]: Editorial (Questions "NOT!" in GL9.)

    Resolved, changed "NOT!" to "not." See draft revised text.

  6. [SG-6]: Editorial (Typo in CP9.1)

    Resolved. The typo has been fixed during the merger of 9.1 and 9.2 (which was the resolution of another, substantive issue.) See draft revised text.

  7. [SG-7]: "Normative refs" requirement should be priority 1.

    20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: agreed, the priority of CP10.2 (now CP7.5) has been changed to 1. See draft revised text.

  8. [SG-8]: Add Rationale and Discussion to CP13.3 ("Use consistent terminology").

    20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: Agreed, a rationale has been added. See draft revised text.

  9. [SG-9]: Improve definition of "test assertions" (GL14).

    Related discussion about whether testability is an inherent part of "conformance requirement" in this email sub-thread (of a thread about some OpsGL issues).

    See mail thread (20030606), which includes suggested resolutions.

    Discussed and resolved at 6/2003 QAWG face-to-face. QAWG will make the various definitions consistent across the GL documents and the QA Glossary, but otherwise will leave the definition more or less as it was in the initial verbiage of (Last Call) SpecGL GL14.

See also these related closed QAWG issues: issue #98, issue #99, issue #110.AR-001.

Extension-group processing plan and subsequent email discussion.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Related LC applicability/normative-exclusion issues: 26, 73.2, 80.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal:
Resolution:
LC-74 SpecGuide 2003-03-15 SpecGuide Substantive Closed Lofton Henderson Lynne Rosenthal
Title: Simplify & consolidate the guidelines of SpecGL
Description:

Specification reference.

comment about "Overall" :

If the 14 specification guidelines can be clearly summarized in 6 bullets, as we claim is done in the QA Outreach Kit, http://www.w3.org/2003/Talks/qa-outreach/slide5-0.html , then perhaps the guidelines themselves ought to be consolidated and reduced in number. I believe that most of the checkpoints are appropriate, but perhaps should be repackaged.

Six bullets:

  • Scope - define the scope, identifying what needs to conform and how
  • Conformance Clause - specify the conformance policy and requirements
  • If you must subset - use profiles, modules, functional levels
  • Extensions - specify whether and how extensions are allowed
  • Tag for later use - identify testable assertions, discretionary items
  • Claims - specify how conformance claims and statements are made

I think the first 4 are almost right for guidelines. The later ones don't seem to me to be quite right.

  • new GL1: define and illustrate scope (includes old GL1, GL2)
  • new GL2: define conformance policy and requirements (old GL3, GL10)
  • new GL3: subset as needed (old GL4, GL5, GL6)
  • new GL4: extensions (old GL9)

    (It gets a little rougher from here on...)

  • new GL5: get a handle on other conformance variability
  • new GL6: is there a nice umbrella statement that would accomodate GL13 and GL14?
  • new GL7: claims -- specify how and provide an ICS [old GL11, GL12]

So at best, we would have 7 guidelines. At worst, 9 (if the new GL5 wouldn't stick together, and we couldn't find a new GL6 that was satisfactory).

Email comments (20030417), including proposals.

Discussed and resolved at 6/2003 QAWG face-to-face. Taking the whole group of related guideline (GL) consolidation and reordering issues together, it was resolved to: merge GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12.

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal: See email.
Resolution: Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9. See draft revised text.
LC-75 SpecGuide 2003-03-16 SpecGuide Substantive Closed Patrick Curran Lynne Rosenthal
Title: Comments on SpecGL Guidelines
Description:

Specification reference.

comment about "Overall" :
  1. Guideline 1: Checkpoint 1.4 is priority 2, yet it's listed after 1.3 which is priority 3. I suggest listing checkpoints in priority order.

    Discussed at 20030407 telecon. Agreed to switch the order in this case. However, this is NOT a general endorsement of "order by priority". Generally agreed that logical flow should have precedence, and then priority order if there is no determinant based on logical flow.

    As part of the resolution of the larger issue group, CP1.3 has been merged into CP1.2 and the single checkpoint has been reworded as "Illustrate scope", so the issue becomes moot. CP1.4 (now CP1.3) is reworded as "Illustrate technical details". See draft revised text.

  2. Guideline 3: The Conformance Policy is likely to appear towards the end of a Spec. I suggest ordering the guidelines as they are likely to appear in the spec. (Besides, it belongs with, and maybe could even be combined with, guidelines 10, 11, 12, and 13.)

    Discussed and resolved at 6/2003 QAWG face-to-face. Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. SeeSee new GL7, GL8, GL9. See draft revised text.

  3. Guidelines 4-9: This is a large number of guidelines and checkpoints to deal with something that's important, but that we wish to discourage (DOVs). Could we combine some or all of these guidelines?

    Email comments, including proposal (20030418).

    Discussed and resolved at 20030421 telecon. It was decided to consolidate GL4-6 (profiles/modules/levels), into one guideline, while preserving the identities of profiles, modules, and levels as DoV. But otherwise, the other DoV checkpoints don't seem amenable to consolidation. See resolution summary of this and related DoV issues. See draft revised text.

  4. The "or NOT!" language in guideline 9 seems a little too informal,

    Resolved at 20030505 telecon. Agreed, "NOT!" is replaced with "not." See draft revised text.

  5. but more substantively, why do we wait until this guideline [GL9] to express our opinion that DOVs are undesirable.

    Email discussion.

    Email comments, including proposal (20030418).

    Discussed and resolved at 20030421 telecon. A modified version of the previous per-GL DoV "excessive variability" caveat has been restored. See also resolution summary of this and related DoV issues. See draft revised text.

  6. Guideline 10: Checkpoint 10.2 doesn't seem to be directly related to the guideline.

    Discussed and resolved at 6/2003 QAWG face-to-face. Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9. See draft revised text.

  7. Guidelines 10 - 13: As suggested above, perhaps these could be combined.

    Discussed and resolved at 6/2003 QAWG face-to-face. Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9. See draft revised text.

  8. Guideline 14: This is a key guideline - I'd like to see it earlier in the list.

    Discussed and resolved at 6/2003 QAWG face-to-face. Taking the whole group of related guideline (GL) consolidation and reordering issues together, it was resolved to: merge GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12. While test assertions are pivotal and early in the process in *Test* Guidelines, it is resolved that this is an appropriate placement in the *SpecGL* context.

  9. Section 3.3: "This... document does not enumerate a list of test assertions". Haven't we agreed that our checklist is (the closest thing to) a list of assertions?

    Related discussion about whether testability is an inherent part of "conformance requirement" in this email sub-thread (of a thread about some OpsGL issues).

    See mail thread (20030606), which includes suggested resolutions.

    See also these related QAWG issues: issue #98, issue #99, issue #110.AR-001.

    Discussed and closed at June QAWG face-to-face. The checklists are a test results reporting form. Recognizing that there are possibly multiple conformance requirements (CR) per checkpoint, a test material for the Guidelines documents would be something like a per-CR questionnaire, "How is this CR satisfied?".

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal: Restructure guidelines, combining several resulting in a smaller number.
Resolution:
LC-76 IntroGuide 2003-03-19 IntroGuide Substantive Closed Roger Gimson Lofton Henderson
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "General: miscellaneous & other" : [Entered into form by LH.]

Four substantive and editorial issues (IN-1, .., IN-4) on "QA Framework: Introduction" are included in the document at

http://www.w3.org/QA/WG/2003/03/DIWGcomments.html ,

which was submitted to QA on 14 Mar 2003:

http://lists.w3.org/Archives/Member/w3c-di-wg/2003Mar/0074.html .

  1. [IN-1]: Rephrase "ultimate mission of W3C".

    Accepted, "ultimate" is replaced with "ongoing".

  2. [IN-2]: Meaningless phrase, "unimplementable language".

    Agreed, the phrase has been changed to read, "early detection and correction of unintentionally vague or contradictory language".

  3. [IN-3]: More detail in 3.5.2 audience list than in "Audience" section.

    This has been fixed during the substantial rewrite of QA Framework: Introduction that resulted from issue #68. As a result of consolidation of similar parts, the 3.5.2 list is gone.

  4. [IN-4]: Combine target audience sections -- 4.1 and 1.3.

    Agreed, the disparate sections been consolidated during the substantial rewrite of QA Framework: Introduction that resulted from issue #68.

Also there are three issues applicable to all Framework, (FR-1, ..), and six general comments to QA (GC-1, .., GC-6).

Proposal:
Resolution:
LC-77 SpecGuide 2003-03-19 SpecGuide Substantive Closed Roger Gimson Lynne Rosenthal
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "Overall" : Substantive and editorial issues on several QA Framework and other QA topics are found in the (linked) document:

which was submitted by DI WG to QA on 14 Mar 2003:

Two substantive and editorial issues on "QA Framework: Specification Guidelines", plus four related comments about the Specification Examples & Techniques documents,

  1. [SG-1] More discussion of several DoV and their relationships.

    Email comments, including proposal (20030418).

    Discussed and resolved at 20030421 telecon. Subsection 2.1 of the new chapter "2. Concepts" provides additional discussion about relationships amongst DoV. See draft revised text.

  2. [SG-2] Standard headings for (Framework GL?) documents.

    20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: Agree that this is a good principle and we will adhere to it, as best we can, recognizing that each Framework document may have some unique chapters. All documents will have the same core headings and in the same order: Intro, Guidelines, Conformance, Definitions (if applicable), Acknowledgements, References, Change History. No changes to SpecGL needed. See draft revised text.

  3. [ET-1] (Affects Intro, and all GL-ET pairs) Explain GL-ET pairing better.

    Proposal. See similar OpsGL text.

    Discussed at 20030714 telecon. Agreed, this is now handled by additional clarifying text in the GL document, and/or references to "QA Framework: Introduction". See draft revised text.

  4. [ET-2] (Affects SpecET and the other ETs). More E&T, in order to conform to SpecGL (GL1)!

    Resolved: it is the QAWG intention that SpecET (Examples & Techniques) will have at least one example per checkpoint. This is queued for SpecET editors, but may lag slightly behind SpecGL publication (after the structure and checkpoint collection of SpecGL stabilizes.)

  5. [ET-3] (Affects SpecGL/ET, and the other GL/ET) Change GL, CP, and section numbering scheme.

    Discussed at 20030407 telecon. Closed with resolution to leave numbering scheme as is. Reasons: given that the context is usually clear (or easily made clear), the shorter GL/CP numbers are considered both more convenient and sufficient; and, the numbering scheme is familiar from WAI specifications and has been successfully used there.

  6. [ET-4] (Affects SpecET) When will SpecET be released? (!?)

    The QAWG policy is that each new publication of SpecGL will be accompanied by a new version of SpecET (possibly lagging slightly behind the SpecGL publication). In this sense, it is already "released". SpecET may be updated and improved more frequently than SpecGL publications. It is for this reason that QAWG resolved to keep SpecET (as well as OpsET and TestET) in /QA/WG/ space as a Note. When SpecGL 1.0 has finally stabilized and stopped changing in /TR/, and when active work on SpecET is substantially over, then SpecET will be published in /TR/ (as a Note, since it is purely informative.)

An additional three comments/issues applicable to all Framework -- [FR-1]..[FR-3] -- can now be found at Last Call issue #111.

An additional six general comments/issues to the QA activity -- [GC-1]..[GC-3] -- can now be found at Last Call issue #112.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Related LC Section/GL/CP numbering issues: 63, 77.ET-3, 91.

Proposal:
Resolution:
LC-78 SpecGuide 2003-03-19 SpecGuide Substantive Closed David Marston Lynne Rosenthal
Title: Should provide a disclaimer template
Description:

Specification reference.

comment about "11.3 Provide a conformance disclaimer." :

When I visited the joint XSL/XQ WG session (March 6 in Cambridge), there was quite a bit of discussion about the claims and disclaimers suggested in Guideline 11 of SpecGL. WG members attending wanted to have something more concrete to start with, like a "boilerplate" paragraph or two. In particular, we talked about number of test cases passed as a bad metric for conformance, because it implies that each case has equal weight. The current checkpoints don't call out this practice as warranting discouragement.

20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: good idea. Volunteers are sought, to help make the templates.

Proposal: [originator] Include in Ck 11.3, and probably 11.2 as well, a template that editors can paste into their specs. The template should include sentences addressing particular good (11.2) or bad (11.3) practices that might occur, allowing editors to remove irrelevant sentences.

I think that for 11.2, you already know that you intend a template resembling: This [class of product] was tested for conformance to [name of spec] version [N.nn], [Nth] Edition, dated [date], and all errata issued through [date], ... more specs cited same way .... The testing occurred on [date] using [test suite identifier] and [name of product and version] was found to conform at [level] level except for [enumeration of failing tests]. A full report of the parameter settings for the test harness and all results is posted at [URL] for open, public review.

For 11.3, the template could look like this: While the test suite provides [hundreds, thousands] of test cases, not all cases should be considered to carry equal weight. A product that passes all the tests may still not conform in some untested area. The W3C hereby states that claims of passage of a certain number of test cases or a certain percentage of the test cases, but not all, are invalid as relative measurements of conformance or worthiness, and that the only valid data that can be derived from such a result is that the product being tested does not pass all the tests. [Optional: More tests may be added to the suite in the future, and existing tests may be changed when errata are discovered. Failing some test cases cannot be interpreted as failing to conform without corroboration.] [Optional: The test suite can be tailored to suit permissible variability in product behavior. The W3C encourages implementers to provide information in their Implementation Conformance Statement that will lead to accurate configuration of the test suite, but holds the test lab responsible for obtaining the information and tailoring the suite accordingly, or else reporting which pieces of information were undetermined and indicating that some test failures may in fact be due to configuration problems.]

Resolution: The suggested templates are a good idea. They would belong in Examples & Techniques (SpecET), as was done for OpsGL templates. The editors will work with Orginator to make acceptable templates, and integrate them into SpecET.
LC-79 SpecGuide 2003-03-19 SpecGuide Substantive Closed Lofton Henderson Lynne Rosenthal
Title: SpecGL fails checkpoints 1.2 and 1.3
Description:

Specification reference.

comment about "Guideline 1 Define Scope." : [This is really a comment about the "Introduction" section.] "Specification Guidelines" (SpecGL) fails its own Checkpoints 1.2 (priority 2) and 1.3 (priority 3), by not illustrating its scope with examples and/or use cases, and not providing usage scenarios.

Alternatives:

  • Alt.1: Add examples and/or use cases, and provide usage scenarios;
  • Alt.2: No need to, these are "not applicable" to SpecGL;
  • Alt.3: No need to, SpecGL only needs to conform to itself at level "A-Conforming" (only priority 1 checkpoints).

While I believe that alternative #1 is the correct alternative, it might be helpful for SpecGL to explain its intentions about self-conformance (p1 or p2 or p3?). I.e., should SpecGL contain a SpecGL conformance claim?

Discussed at 20030407 telecon. Resolved (and action item assigned) to add use cases and/or usage scenarios and/or examples to illustrate scope.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal: Alternative #1.
Resolution: Use cases have been added to illustrate scope. The text already had (and still has) numerous examples to illustrate technical details. See draft revised text.
LC-80 SpecGuide 2003-03-19 SpecGuide Substantive Closed Lofton Henderson Lynne Rosenthal
Title: SpecGL should address the topic of CP applicability.
Description:

Specification reference.

comment about "Overall" : [This comment probably applies to the "Conformance" section.]

It is arguable that some SpecGL checkpoints -- applied to SpecGL itself, or to any other specification for that matter -- might be "not applicable", because of the nature of the specification. For some checkpoints (CP), SpecGL states conditions under which the CP is not applicable. It is unlikely that SpecGL has anticipated all of the circumstances under which a CP might be "not applicable".

Email discussion, including proposal.

Discussed and closed at 20030602 telecon (DRAFT). The suggested change -- instructions in "3. Conformance" about what one should do if one considers a checkpoint to be not applicable -- are considered to be unnecessary and possibly more confusing than useful. Confusing because they might be interpreted as relieving of the obligation to conform to the checkpoint.

Related LC applicability/normative-exclusion issues: 26, 73.2, 80.

Proposal: In "Conformance" section, address what a specification tester, who is applying SpecGL's ICS, should do if he/she believes that a checkpoint is "not applicable" (n/a), but SpecGL has not indicated that the CP might n/a.
Resolution: No change. The proposed change is considered to be unnecessary, and possibly more confusing than useful. Confusing because it might be interpreted as relieving spec writers of the obligation to conform to the checkpoint.
LC-81 SpecGuide 2003-03-19 SpecGuide Substantive Closed Lofton Henderson Lynne Rosenthal
Title: CP9.6 conformance requirements and rationale may be too narrow.
Description:

Specification reference.

comment about "9.6 Require that implementations ... alternatives to extensions" :

Checkpoint 9.6 conformance requirements talk about an operating mode under which only strict-conforming content may be produced. I think that this may be too narrow a requirement. Would we consider the intent of the checkpoint to be satisfied if an implementation generated strict-conforming 'alt' content (to use the HTML analogy) to a private extension? In my opinion, a no-extensions mode is the best way to satisfy this CP. But is it the only way?

Note. There is some question why someone would include the extension at all, if the 'alt' content is "equivalent". Perhaps it is a way of round-tripping private functions of an implementation, while providing an alternative formulation that other implementations can use.

Extension-group processing plan and subsequent email discussion.

Discussed at 20030505 telecon, without resolution.

Discussed and resolved at 20030602 telecon (DRAFT). CP9.6 will be kept, but the requirements on target specifications will be revised. Rather than requiring that target specifications place the current specific conformance requirement on extension-generating implementations (for a no-extensions mode), CP9.6 will require that target specifications explicitly define a policy about implementation requirements for mitigation of the interoperability impacts of extensions. Such policy could, to cite three examples, include a requirement that implementations have a no-extensions mode; or could include a requirement that implementations include equivalent alternative (standard) content with any extensions; or could explicitly state that there are in fact no implementation requirements for mitigation of interoperability impacts of extensions. [PC volunteered to draft text.]

Further discussion at 20030623 telecon, resolved to clarify that this applies only to producers of content.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal: 'Alt' mechanisms that contain only strict-conforming content, and achieve equivalent effect, should qualify.
Resolution: CP9.6 requirements on target specifications are revised. Rather than requiring that target specifications place the current specific conformance requirement on extension-generating implementations (for a no-extensions mode), CP9.6 requires that target specifications explicitly define a policy about implementation requirements for mitigation of the interoperability impacts of extensions. Such policy could, to cite three examples, include a requirement that implementations have a no-extensions mode; or could include a requirement that implementations include equivalent alternative (standard) content with any extensions; or could explicitly state that there are in fact no implementation requirements for mitigation of interoperability impacts of extensions. In addition, clarify that this applies only to producers of content. See draft revised text.
LC-82 OpsGuide 2003-03-19 OpsGuide Substantive Closed Lofton Henderson Lofton Henderson
Title: OpsGL fails SpecGL checkpoints 1.2 and 1.3
Description:

Specification reference.

comment about "Overall" : [This is really a comment about the "Introduction" section.] "Operational Guidelines" (OpsGL) fails Checkpoints 1.2 (priority 2) and 1.3 (priority 3) of "Specification Guidelines", by not illustrating its scope with examples and/or use cases, and not providing usage scenarios.

Alternatives:

  • Alt.1: Add examples and/or use cases, and provide usage scenarios;
  • Alt.2: No need to, these are "not applicable" to OpsGL;
  • Alt.3: No need to, OpsGL only needs to conform to itself at level "A-Conforming" (only priority 1 checkpoints).

While I believe that alternative #1 is the correct alternative, it might be helpful for OpsGL to explain its intentions about conformance (A or AA or AAA?) to SpecGL. I.e., should OpsGL contain a SpecGL conformance claim?

Resolved at 20030501 telecon, per prior email proposal.

Proposal: Alternative #1.
Resolution: Use cases are added to OpsGL to illustrate its scope. See draft revised text.
LC-83 OpsGuide 2003-03-31 OpsGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Seven levels vs. Three
Description:

Specification reference.

comment about "Overall" : All specs include seven "levels" of conformance. However, only three levels are actually public; these are called "priorities".

Ed note. Submitted as SpecGL issue, reclassified as OpsGL, as QAWG believes that the problem originates with the table of CP1.1 of OpsGL.

Email discussion , including proposed resolution.

Discussed and resolved at 20030428 telecon. See LC-3 for description and resolution. See draft revised text of CP1.1 (as well as CP1.2, CP1.3).

Related LC issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal: [Originator] Collapse the seven levels into the three real ones (now called priorities), since these are the basis of conformance measurement.

[LH] See email.

Resolution: The 7-level commitment table has been eliminated, and the commitment requirements have been written into new CP1.1 - CP1.3 that are consistent with the 3-level conformance model of the QA Framework family. See LC-3 for description and resolution. See draft revised text of CP1.1 (as well as CP1.2, CP1.3).
LC-84 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Group dimensions of variability
Description:

Specification reference.

comment about "Overall" : Guidelines two through nine are grouped as "dimensions of variability," and referred to as such by themselves and by other guidelines. If the concept of dimensions of variability is of this much importance, it should be reflected in the structure of the document.

Email comments (20030417), including proposals.

Email comments, including proposal (20030418).

Discussed and resolved at 20030421 telecon.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: [Originator.] Guidelines two through nine should be grouped, structurally, as "dimensions of variability". [LH] See email.
Resolution: This has been addressed with the new chapter -- "2. Concepts" -- and with an expanded DoV discussion in one of its subsections. With these improvements, restructuring to segregate the DoV guidelines into their own document sections is thought to be unnecessary. See draft revised text.
LC-85 SpecGuide 2003-03-31 SpecGuide Editorial Closed Jonathan Marsh Lynne Rosenthal
Title: Self-sufficient guidelines and checkpoints
Description:

Specification reference.

comment about "Overall" : In general, the language is insufficiently rigorous/precise for a common use case, and the model may be explained in text rather than structurally. The problem is that, while many WG members may read the entire spec, others may be tasked with judging conformance to a particular guideline, or even a particular checkpoint.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [Originator] Insofar as is possible guidelines and checkpoints should contain sufficient definition for local understanding, and pointers to all related items in the document. That is, a reader should be able to enter via any checkpoint, and gather all and only the information needed to understand the checkpoint starting from that entry point. [LH] See email.
Resolution: Resolved per proposal in email. Agreed that these are good principles. While it is beyond our resources to completely rewrite the document with these in mind, we have tried to apply them going forward with the editing and revision of SpecGL.
LC-86 SpecGuide 2003-03-31 SpecGuide Editorial Closed Jonathan Marsh Lynne Rosenthal
Title: Spelling, grammar, and style
Description:

Specification reference.

comment about "Overall" : Spell and grammar check. There are a number of problems with both, which probably do not deserve direct mention. Also, the style of the prose occasionally changes radically (from a formal, romance-language-influenced european english to a very colloquial american style). This can be jarring.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [LH] See email.
Resolution: In the final WG-only SpecGL draft preceding the published version, we will make a review for grammar and spelling [MS & LR]. With our mix of authors, there is not much that we can do about different styles, other than to ensure that everything is grammatically correct.
LC-87 SpecGuide 2003-03-31 SpecGuide Editorial Closed Jonathan Marsh Lynne Rosenthal
Title: Abbreviations
Description:

Specification reference.

comment about "Overall" : If abbreviated forms are used to refer to other documents, these abbreviations ought to also appear in the bibliography. In general, reference to a document should always be a <bibloc>.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [LH] See email.
Resolution: We believe that we are doing references correctly (per "Manual of Style"), but we see that we have omitted the brackets "[]" on the abbreviations in the References section. That has been fixed. We [LR/MS] will also check style of references during grammar/spelling review (for issue 86.) (About 'bibloc', note also that we are not using xmlspec, but rather XHTML customized with some classes.) See draft revised text.
LC-88 SpecGuide 2003-03-31 SpecGuide Editorial Closed Jonathan Marsh Lynne Rosenthal
Title: Reformat bullet list
Description:

Specification reference.

comment about "1.3 Provide Usage Scenarios. " : Bullet lists are there to call out important items. An eight-item list should either be shortened or reformatted; it fails as bullet points.

Note. We assume that the reference to 1.3 is wrong, and this actually refers to the 8-item list in the verbiage of GL2.

20030714 telecon directed that it be resolved as indicated in ungrouped-issue proposals: it should be left to the editors' discretion to research any W3C style requirements, and chose a style that they consider appropriate, while conforming to any existing style guidelines.

Proposal:
Resolution: There are no applicable guidelines on this in the W3C "Manual of Style", or elsewhere as far as the editors can determine. Being a matter of preference and taste, it was resolved to keep the current style.
LC-89 SpecGuide 2003-03-31 SpecGuide Editorial Closed Jonathan Marsh Lynne Rosenthal
Title: Consolidate glossary and terminology
Description:

Specification reference.

comment about "Introduction" : Why isn't the glossary in section 1.6, instead of at the end?

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [Originator] Move the definitions appendix to section 1.6. [LH] See email.
Resolution: Resolved per email proposal. It is done both ways in W3C specifications -- definitions up front versus definitions at the end -- and QAWG prefers this way for SpecGL. Also these two changes have been made: change the title of 1.6 from "Terminology" to "Usage of terminology in this document" (removing possibly confusion that 1.6 is the glossary); and, this sentence is added to 2nd paragraph, "When used in this specification, terms have the meanings assigned in 'Definitions' and 'QA Glossary' [QA-GLOSSARY]." (Making moot the question of whether definitions are normative or informative.) See draft revised text.
LC-90 SpecGuide 2003-03-31 SpecGuide Editorial Closed Jonathan Marsh Lynne Rosenthal
Title: Restructure DoV
Description:

Specification reference.

comment about "Introduction" : If DoV is important enough to occupy a full page in 1.8, and be referred to without further explanation elsewhere, it's important enough to restructure so as to identify clearly the membership of DoV. Wherever dimensions of variability are mentioned in the document, it should be possible to hyperlink to the appropriate place (which means a briefer discussion, as an introduction to the collection of guidelines 2-9).

Email comments (20030417), including proposals.

Email comments, including proposal (20030418).

Discussed and resolved at 20030421 telecon.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: [Originator] Drop section 1.8, and restructure. [LH] See email, and new chapter proposal.
Resolution: This has been resolved consistently with the new chapter proposal, and addressed in a new DoV sub-section in the new chapter "2. Concepts". Each DoV guideline (now GL2-GL6) links back to the subsection. See draft revised text.
LC-91 SpecGuide 2003-03-31 SpecGuide Editorial Closed Jonathan Marsh Lynne Rosenthal
Title: Complexity of numbering
Description:

Specification reference.

comment about "Overall" : Guidelines escape the section/technical numbering of the remainder of the document. Why isn't guideline 1 section 2.1? And checkpoint 1.2 section 2.1.2? This is going to cause confusion, when others make reference to the document: if I say 3.2, do I mean a checkpoint or a section? Section 2 is by far the bulk of the document; perhaps each guideline ought to be given a section number instead (guidelines 2 through 15, then, instead of 1 through 14).

Discussed at 20030407 telecon. Closed with resolution to leave numbering scheme as is. Reasons: given that the context is usually clear (or easily made clear by the usual prefix of "guideline" or "checkpoint" [x.y]), the current GL/CP numbers are considered both more convenient and sufficient; the numbering scheme is familiar from WAI specifications and has been successfully used there; and, the proposed numbering scheme would have the undersireable characteristic that every guideline would start with "2." (e.g., 2.8) and every checkpoint would start with "2." (e.g., 2.8.4). The QAWG believes that the GL number should be the most significant component, and believes it is easier to quickly recognize "checkpoint 8.4" than "checkpoint 2.8.4".

Related LC Section/GL/CP numbering issues: 63, 77.ET-3, 91.

Proposal: Find a non-conflicting numbering scheme.
Resolution: Given that the context is usually clear (or easily made clear by the usual prefix of "guideline" or "checkpoint" [x.y]), the current GL/CP numbers are considered both more convenient and sufficient; the numbering scheme is familiar from WAI specifications and has been successfully used there; and, the proposed numbering scheme would have the undersireable characteristic that every guideline would start with "2." (e.g., 2.8) and every checkpoint would start with "2." (e.g., 2.8.4). No change.
LC-92 SpecGuide 2003-03-31 SpecGuide Editorial Closed Jonathan Marsh Lynne Rosenthal
Title: Examples vs. Illustrations
Description:

Specification reference.

comment about "Guideline 1 Define Scope." : What's the difference between examples and illustrations? (Checkpoint 1.2 versus checkpoint 1.4)

Discussed at 20030407 telecon: CP1.2 requires examples of what is in scope; CP 1.4 requires examples of specific functionality, concepts, behavior. Need to make this clearer, perhaps remove 'concepts' from CP 1.4 will help. Alternatives:

  • Alt.1: Clarify the difference that 1.2 is broader examples, whereas 1.4 is very specific to a function or behavior.
  • Alt.2: Remove CP 1.4
  • Alt.3: Remove CP 1.2

Proposed resolution: re-word 1.2 as "Illustrate scope.", reword 1.4 as "Illustrate technical details", and clarify difference further in the ConfReqs, Rationale, and Discussion. Also, examples (use cases) need not be in the specification itself, they can be in companion document(s). Language should change to use "provide" rather than "include", to avoid implication of "within specification".

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal: Re-word 1.2 as "Illustrate scope.", reword 1.4 as "Illustrate technical details", and clarify difference further in the ConfReqs, Rationale, and Discussion.
Resolution: CP1.3 has been merged into CP1.2 and the single checkpoint has been reworded as "Illustrate scope" (with examples and/or use cases); CP1.4 (now CP1.3) has been reworded as "Illustrate technical details"; and further clarifications are made in the Conformance Requirements, Rationale, and Discussion sections. See draft revised text.
LC-93 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Classes vs. categories
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " : The use of categories versus classes of products is altogether unclear. If categories and classes of products are to be called out, normatively, then these should have status in the TOC (the list of categories and the list of classes should both have ids, be targets for hyperlinks, and should have subheadings to identify them visually).

Email comments (20030417), including proposals.

Discussed at 20030418 telecon.

Discussed and resolved at 20030418 telecon. Rewrite the GL verbiage for clarification. Implementation of the TOC suggestion depends on whether a sub-section on 'class' and 'category' is set up in a new "Ch.2 Concepts".

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: The verbiage on classes and categories has been revised consistent with the 4-point proposal in email comments. The verbiage has all been moved to a new subsection of the new chapter "2. Concepts", with individual sub- subsections for "specification category" and "class of product". The definitions of the two concepts are in "5. Definitions", and all of these bits are linked from the appropriate places in the guidelines and checkpoints. Some of the text has been rewritten to improve clarity. See draft revised text.
LC-94 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Loophole in Classes and Categories
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " : The checkpoints based on classes of product and categories are awkward, because the use of the existing enumeration is REQUIRED but only if applicable. That is, the use of the existing enumerations is COMPLETELY OPTIONAL. The language should reflect that it leaves an enormous loophole for spec authors to ignore the existing lists.

Email comments (20030417), including proposals.

Discussed at 20030418 telecon.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: Resolved consistently with proposal in email comments. The lists and verbiage have been revised and clarified, and have been moved into a new subsection of the new chapter "2. Concepts", which are referenced from GL2 and its checkpoints. We have clarified that the enumeration covers the most common cases seen in W3C technologies, that an enumerated item is to be used if it matches the specification category (SC) or class of product (CoP), and that specs should define their own item only if it is NOT matched by one of the enumerated items. We think that the intent is now very clear. It might still be possible to circumvent the requirements with devious intent, but we don't consider it productive to take further extraordinary steps in SpecGL to close all possible loopholes, in order to to prevent such intentional circumvention. See draft revised text.
LC-95 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Why is conformance policy a DoV?
Description:

Specification reference.

comment about "Guideline 3 Specify conformance policy. " : Why should conformance policy be considered a dimension of variability? It is potentially partitioned by those dimensions, but does not introduce its own dimensions.

Email comments, including proposal (20030418).

Discussed at 20030421 telecon, and in email thread.

Discussed at 20030428 telecon, and further analyzed in another email thread. Resolution postponed -- there are related issues in the "re-org" group, for consolidating various amongst GL3/10, GL11/12, etc.

Related previous QAWG issue 95 (which appears more "Postponed" than "Closed"), additional email proposal (20030429), email suggestion (20030429) of additional checkpoint.

Discussed at 6/2003 QAWG face-to-face, during the discussion of the "major-restructure issues" group. All of the bits of GL3 which would have qualified it as a DoV have been changed or moved elsewhere, and GL3 is to be merged into GL10.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: See email.
Resolution: GL3 is no longer a DoV (reducing the number of DoV from eight to seven.) All of the bits of GL3 which would have qualified it as a DoV have been changed or moved elsewhere, and GL3 is to be merged into GL10. See draft revised text.
LC-96 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Priorities confusing
Description:

Specification reference.

comment about "Guideline 3 Specify conformance policy. " : The priorities here are very strange. Why is justifying the use of a dimension priority one, while establishing a minimum requirement is priority two?

Discussed and resolved at 20030602 telecon (DRAFT). CP3.1 is priority 2 because it is implicit (but not explicitly spelled out now in SpecGL) that the requirements that comprise the universal- minimum set (across CoP) all occur elsewhere, in the definition of the conformance requirements for each individual Classes of Product (CoP). Thus, the enumeration of CP3.1 could be derived. Because the enumeration could be difficult or obscure, it is considered important/desirable (P2), but not critical/essential (P1). SpecGL editors will ensure that this is clarified.

Related LC GL3-policy issues: 42, 96.

Proposal:
Resolution: CP3.1 is priority 2 because it is implicit (but not explicitly spelled out now in the checkpoint) that the requirements that comprise the universal-minimum set (across CoP) all occur elsewhere, in the definition of the conformance requirements for each individual Class of Product (CoP). Thus, the enumeration of CP3.1 could be derived. Because the enumeration could be difficult or obscure, it is considered important/desirable (P2), but not critical/essential (P1). This has been clarified in CP3.1. See draft revised text.
LC-97 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Modules as extension points
Description:

Specification reference.

comment about "Guideline 5 Address the use of modules to divide the technology." : Unlike G4, which notes that profiles may be a point of extension, G5 does not consider modules to be a point of extension. In the web services world, "modules" certainly are a point of extension, and so have rules for defining new modules (just as, in G4, there are assertions associated with rules for defining new profiles). The document should recognize this.

Email comments (20030417), including proposal to seek clarification.

Discussed (for clarification) at 20030418 telecon, and in subsequent email thread (20030418).

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. During discussion, a key distinction between extension modules and from Rules for Profiles was established -- new profiles are not *extensions* to the specification, but rather different ways to subdivide the (typically) standard functionality of the specification for different conformance targets. Agreed that the problem should be resolved along the lines in LR email proposal -- a combination of 1.) clarification to Originator about how existing checkpoints actually cover this case; and, 2.) some additional clarifying verbiage (to be drafted, for either section 2.3, or the modules GL verbiage, or GL9/extensibility -- tbd). For further context, see also the solution for the profile/module/level issue group.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal:
Resolution: A new CP is not thought to be needed. This case -- involving modules and extensions -- is somewhat different than the Rules for Profiles case (where extensions are typically not a factor.) QAWG thinks that CP9.4, CP9.7, CP5.2 (new CP6.3, CP6.6, CP3.4) cover this adequately. Clarifying explanation has been added, per the clarification proposal. See draft revised text.
LC-98 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Conformance levels
Description:

Specification reference.

comment about "Guideline 6 Address the use of functional levels to divide the technology." : Levels are the primary defining mechanism for the QA framework, but there is only one checkpoint here. There should also be (at least) a checkpoint to establish that levels create a hierarchy of conformance; that the more advanced levels include the earlier levels (thus establishing that there is a minimum level).

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. That "levels create a hierarchy of conformance" is part of the definition of levels. It was thought unnecessary and redundant to try to bind the definition into a Checkpoint and test requirements. For further context, see also the solution for the profile/module/level issue group.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: The suggested checkpoint and associated normative conformance requirements are considered redundant and unnecessary. It (hierarchy) is part of the definition of "levels", and the terms in SpecGL "...have the meanings assigned in 'Definitions' and 'QA Glossary' [QA-GLOSSARY]."
LC-99 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Awkward deprecation requirements
Description:

Specification reference.

comment about "7.3 If deprecation is used, define its relationships..." : This checkpoint basically demands that if some feature has been deprecated, possibly because no one can agree on its proper definition, then it must be properly defined and its interactions fully characterized. It is likely that some things will be deprecated precisely because they cannot be well and unambiguously characterized; this checkpoint ensures that as long as these features are part of the main spec, it can conform at priority two, but as soon as the features are deprecated, it cannot.

See email thread.

Discussed at 20030410 telecon. The analysis of the email thread was basically agreed -- it is not necessary to fully define the deprecated functionality, in order to meet the intent of the checkpoint. That intent is that the impact of deprecating the feature on other DoV needs to be discussed. E.g., if the spec is modularized or profiled, how does a feature's deprecation impact those DoV. DM agree to draft clarifying discussion, including at least one "for example", to prevent the confusion.

Related LC deprecation issues: 33, 40, 99.

Proposal: The way Originator has understood the checkpoint is not how we intended it. We have rewritten the verbiage (incl. "for examples") to clarify and prevent further confusion. [DM draft proposal.]
Resolution: How the commentor has understood the checkpoint is not how QAWG intended it. It is not necessary to fully define the deprecated functionality, in order to meet the intent of the checkpoint. That intent is that the impact of deprecating the feature on other DoV needs to be discussed. E.g., if the spec is modularized or profiled, how does a feature's deprecation impact those DoV? CP7.3 (new CP4.3) has been modified to clarify and to avoid this confusion, and some "for examples" have been added. See draft revised text.
LC-100 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Don't discourage extensibility
Description:

Specification reference.

comment about "Guideline 9 Allow extensions or NOT!" : The position of the QA framework WG, that extensions should not be allowed, is quite clear. This is a political position, and doesn't accomodate those working on specifications that clearly demand public extensibility. In the guidelines, describe conformance. Discourage extensibility elsewhere. We note that these guidelines are, themselves, extensible.

Extension-group processing plan and subsequent email discussion.

Resolved at 20030505 telecon.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Subsequent email discussion, including proposed resolution.

Proposal: [LH proposed resolution.]
Resolution: "Extensions should not be allowed" is not in fact what the verbiage of GL9 and its checkpoints actually says. It actually says to exercise caution, and weigh the perceived benefits against negative interoperability impacts. That notwithstanding, some editorial changes have been made to further neutralize the language, without changing the substantive content. The QAWG also disagrees with the assertion that this is a "political" position. It is based on extensive experience, especially where unconstrained or careless use of extensibility has had disasterous interoperability impacts (some of these, as well as some more positive examples, will be documented in SpecET). See draft revised text.
LC-101 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Remove Checkpoint 9.6
Description:

Specification reference.

comment about "9.6 Require that implementations ... alternatives to extensions" : Extensions may be allowed in order to permit new functionality to be introduced and tested prior to standardization. There may not be any alternatives (interoperable or otherwise) to the use of a particular extension, and in particular, it is completely impossible for any specification that permits extensions to supply a workaround to the use of every uninvented extension imaginable. In other words, no specification that allows extensions can conform at priority three, ever.

Extension-group processing plan and subsequent email discussion.

Discussed at 20030505 telecon, without resolution.

Discussed and resolved at 20030602 telecon (DRAFT) -- the Originator has misunderstood what Checkpoint 9.6 actually says. Nevertheless, some changes will be made to CP9.6, according to the resolution of LC-81.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Subsequent email discussion, including proposed resolution. Extension-group processing plan and subsequent email discussion.

Proposal: Remove this Checkpoint.
Resolution: Originator has misinterpreted what (old) CP9.6 actually required. CP9.6 said that specifications that allow extensions must contain conformance requirements that require extension-supporting *implementations* to provide a no-extensions mode. It does not say, as Originator asserts, that the specification must spell out workarounds for all possible yet-to-be-invented extensions. Based as it is on this misunderstanding, the follow-on assertion is also incorrect, "In other words, no specification that allows extensions can conform at priority three, ever." A target specification could conform (to this P3 checkpoint) if it required a no-extensions mode of conforming implementations. Finally, note the significant changes to the CP9.6 (new CP6.5) requirements on target specifications in the resolution of related issue LC-81 -- see draft revised text.
LC-102 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Consolidate G3 and G10.
Description:

Specification reference.

comment about "Guideline 10 Provide a conformance clause." : This is G3. Or it should be. Or G3 belongs in Ops, and G10 here. If a policy has been established, then it has been documented as well; the two are inextricable, when considering a written specification.

Discussed and resolved at 6/2003 QAWG face-to-face. Taking the whole group of related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9.

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal: [originator] Consolidate G3 and G10, or move G3 to Ops.
Resolution: Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9. See draft revised text.
LC-103 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Distributed conformance section OK
Description:

Specification reference.

comment about "10.1 Include a conformance section." : The QA specification does not conform to this checkpoint, which is priority one. It must be acceptable to place the conformance requirements in each section of the document. For instance, it must be acceptable to place the conformance requirements for each checkpoint in a QA document inside the checkpoint.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [Originator] Remove this checkpoint. [LH] See email.
Resolution: Agreed, it is indeed the intent (per Originator's suggestion, "distributed conformance section OK"), that most of the specific conformance requirements ("technical requirements") may be distributed throughout the specification. It is GL3 sorts of things (overall conformance policy) that should be in a conformance section. To remove this confusion, the words, "and specific conformance requirements" are deleted from the ConfReqs statement. See draft revised text.
LC-104 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Consolidate G11 with G3/G10
Description:

Specification reference.

comment about "Guideline 11 Specify how to make conformance claims." : This is also still part of G3/G10. This section should also discuss how one describes "module conformance" versus "profile conformance" versus "level conformance" versus "conforming extension", or how these might be combined. Perhaps this information is in the examples (I haven't looked).

Discussed and resolved at 6/2003 QAWG face-to-face. Taking the whole group of related guideline (GL) consolidation and reordering issues together, it was resolved to: merge GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12.

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal: Consolidate G11 with G3/G10. Clarify various conformance terminiology.
Resolution: Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9. See draft revised text.
LC-105 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: G3, G10, G11, G13
Description:

Specification reference.

comment about "Guideline 13 Clearly identify conformance requirements." : The division of the various conformance-related guidelines into three, ten, eleven, and thirteen is not entirely clear, nor is it entirely clear that these separate sections do not contradict one another.

Email comments (20030417), including proposals.

Email discussion (20030418).

Discussed and resolved at 6/2003 QAWG face-to-face. Taking the whole group of related guideline (GL) consolidation and reordering issues together, it was resolved to: merge GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12.

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal: [Originator] Rationalize these sections. [LH] See email.
Resolution: Considering all related guideline (GL) consolidation and reordering issues together, it was resolved to: merge (old numbers) GL3 into GL10; move current CP10.2 into GL13; move CP11.1 into GL10; merge GL11 and GL12; and reorder as GL13, GL10+GL3, GL11+GL12. See new GL7, GL8, GL9. See draft revised text.
LC-106 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Section 3.1 - Poor Defn of Normative
Description:

Specification reference.

comment about "Overall" : "All sentences using a capitalized keyword from RFC 2119" ... this is a totally strange definition of how to identify normative text. Since the definitions of terms are not normative, I am free to redefine them however I wish, and claim whatever conformance I care to. Note that the stated priorities for each checkpoint are not normative.

See email thread, with proposal: SpecGL's definition of normative is okay in glossary, but is more focused than originator assumes (so that it includes only direct conformance statements such as conformance requirements and test assertions). Per 20030410 telecon section 3.1 should be clarified and improved.

Discussed again and resolved at 20030414 telecon.

Related LC whats-normative issues: 36, 65, 106, 108.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal: [Originator] Develop a rational definition of Normative.
Resolution: QAWG considers that SpecGL's focused definition of normative ("5. Definitions") and its narrow application are appropriate for SpecGL. We note that it is more focused and narrowly applied than definitions in some other specifications. Agreed that are some problems in wording of section 3.1 (now 4.1), including the characterization of normative units of text. This has been revised, the lists in conformance section 3.1 (now 4.1) have been made more comprehensive, and the priorities have been added. Regarding the possibility of someone redefining the terms used in SpecGL, it has been clarified in "1.6 Usage of terminology" that defined terms, when used in SpecGL, have the meanings given in glossary. See draft revised text.
LC-107 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Section 3.4 - AAA-terminology useless
Description:

Specification reference.

comment about "Overall" : The introduction of A[A][A]-conforming as a synonym for the three priorities is rather absurd, as is the creation of four levels that will never be used. A-conforming = priority one = level three; AA-conforming = priority two = level five; AAA-conforming = priority three = level seven.

Email comments (20030417), including proposals.

Discussed and closed (as SpecGL issue) in email.

Email discussion , including proposed resolution (of OpsGL issue).

Discussed and resolved at 20030501 telecon. See LC-3 for description and resolution.

See draft revised text of CP1.1 (as well as CP1.2, CP1.3).

Related LC issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal: [Originator] Pick one set of terms and toss the remainder. (N.B.: the association of priorities with levels is a consequence of Ops guideline 1). [LH] See email.
Resolution: This is dealt with by fixing OpsGL, in the resolution of OpsGL issues 3, 60.2, 72.2, 72.3, 83. See OpsGL draft revised text of CP1.1 (as well as CP1.2, CP1.3).
LC-108 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Definitions
Description:

Specification reference.

comment about "Overall" : To avoid the problems introduced in section 3.1, change it to include, as normative, all of sections 1 and 4, which define terms. Or create a better definition.

See email thread.

Discussed and resolved at 20030414 telecon.

Related LC whats-normative issues: 36, 65, 106, 108.

Proposal: Disagree, our definition of normative is different, focused on direct connection to conformance requirements. See email.
Resolution: The issue has been addressed by fixing the text and "what's normative" lists in section 3.1 (now 4.1), and stipulating (section 1.6) that terms used in SpecGL have the meanings given in "Definitions" and "QA Glossary". The problem with blanket designation of sections 1 - 4 as normative is that a lot of the information is clearly informative. We think that with this resolution, SpecGL now has clear and unambiguous identification of all of its conformance-related information. See draft revised text.
LC-109 SpecGuide 2003-04-07 SpecGuide Substantive Closed Lofton Henderson Lynne Rosenthal
Title: Is CP1.3 needed?
Description:

Specification reference.

Originated during 20030407 telecon, during discussion of issue #92. Discussion: It is unclear what is the difference between CP 1.2 and CP 1.3, according to their respective stated rationales. Is CP 1.3 a superset of CP1.2? Can they be combined? Do we need both? If we need both, then what purposes are the "usage scenarios" of CP1.3 serving, that are distinct from the purposes of CP1.2?

Discussed at 20030410 telecon. Resolved that CP1.2 and 1.3 should be combined, and that use case and/or usage scenario should be emphasized as valuable technique in discussion (so that it is somewhat distinguished from "examples").

See previous QAWG (non-LC) issues #72, #84, and #86.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal:
Resolution: CP1.3 has been merged into CP1.2, which is reworded as "Illustrate scope", and it clarified that use cases and/or examples can be used to satisfy the checkpoint. See draft revised text.
LC-110 OpsGuide 2003-04-16 OpsGuide Substantive Closed Dominique Hazael-Massieux Lynne Rosenthal
Title: Collected OpsGL comments from Team.
Description:

Specification reference.

comment about "Overall" :

[email]

Ed Note -- DH and KD organized a project review of the QA OpsGL for the W3C Team. This is an informal summary of the comments they got. Note that the presentation was about OpsGL, but most of the comments apply equally to SpecGL.

The following seven items discussed in email thread, including proposals, and resolved at 20030512 telecon.

  1. QA is very important, and the QA WG has the right goals. [No issue.]

    Thanks for the comment -- the positive affirmation of the QAWG goals is useful to us.

  2. There were a lot of discussions regarding the writing style we adopted for the framework, namely the fact that we use RFC keywords in conformance requirements. Some people thought it was too "aggressive", other felt it was the right thing to do.

    QAWG reaffirms that it considers the RFC keywords appropriate for OpsGL and the other Framework documents. See also Last Call issue 67.

  3. [OpsGL only] The table in OpsGL GL 1 caused much confusion and was deemed as not-understandable, which I think we already more or less agree with.

    Discussed and resolved at 20030501 telecon. See LC-3 for description and resolution. See draft revised text of CP1.1 (as well as CP1.2, CP1.3).

  4. The intents of the priorities/degrees is not always clear. Proposal [DH]: we should probably emphasize somewhere that the minimal recommended degree is to be AA conformant (or that is the intention of the WG to request it to be the lowest level for work in W3C).

    Discussed and resolved at 20030512 telecon. First, it is our intent that Level A be the minimum. This will somehow be worked into the description of the semantics of the levels, in the Conformance clause ("Conformance definition"). Second, in the re-write of the introduction, we will make generic reference to checkpoint priorities, and move the details into the Conformance clause. See draft revised text in the Introduction, and the "Conformance definition".

  5. Generally speaking, the distinction between GL and ExTech was not always clear. Proposal [DH]: we probably need to rework the introductory sections to clarify that.

    Resolved per email proposals -- it will be clarified and emphasized. In order to comply with the "better readability" request of point #7 in this issue, the detail will be put into QA Framework: Introduction, and linked from the brief treatment in the OpsGL introduction. See draft revised text.

  6. The summarized view (ICS/Checklists) are not easy enough to find, and are not explicitly recommended enough. Proposal [DH]: again, that means some re-working of the introduction.

    Resolved per email proposals -- they will be given more exposure and emphasis. See draft revised text.

  7. The introduction needs to be much more efficient to read. Proposal [DH?]: some kind of an executive summary rather than the long prose we currently have.

    Discussed and resolved at 20030512 telecon. It is not so much "executive summary", as writing style and organization that needs attention: more concise, more use of emphasis, use bullets instead of long prose, etc. See the clarifications (and useful Web-writing-style reference) in the email thread. See extensively rewritten introduction in the draft revised text.

Proposal:
Resolution:
LC-111 IntroGuide 2003-03-19 IntroGuide Substantive Closed Roger Gimson Lofton Henderson
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "Overall" : A number of substantive and editorial issues on several QA Framework and other QA topics are found in the (linked) document:

which was submitted by DI WG to QA on 14 Mar 2003:

Three issues applicable to all-Framework (which have been classified as comments against "Introduction", for processing purposes):

  1. [FR-1] (Affects SpecGL and other GLs) Can ISO9000 be applied?

    After preliminary discussion, QAWG does not think that ISO9000 could replace QA Framework. However an action item to generate a comparison has been assigned and is in progress. It will be circulated to QA distribution when finished. A future version of the "Introduction" part of the QA Framework (now a W3C Working Group Note) could mention or point to the results.

  2. [FR-2] (Affects Intro, and maybe all GLs). Give a cost/benefit analysis for conforming to the GLs.

    An action item has been assigned and is in progress, to generate a some cost-benefit data. It will be circulated to QA distribution when finished. A future version of the "Introduction" part of the QA Framework (now a W3C Working Group Note) could mention or point to the results. Some preliminary results have been incorporated into the new (for renewal) QA Activity statements and charters.

  3. [FR-3] (Affects Intro, and maybe all GLs). Explain/highlight the relationship of QA Framework to W3C processes.

    Related issues: QAWG issue 16, QAWG issue 71.

    QA Framework: Introduction has been substantially revised, and in the process the motivational material and discussion of QA within W3C has been substantially condensed and focused. Regarding any formal relationship of QA to W3C processes as described in "W3C Process" document itself -- there is no defined relationship now. As the status of any such relationship is likely to be changeable and volatile, QAWG considers it inadvisable to address that in any of the Framework documents.

(Other comments/issues from DIWG: LC-77, LC-112.)

Proposal:
Resolution:
LC-112 IntroGuide 2003-03-19 IntroGuide Substantive Closed Roger Gimson Lofton Henderson
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "Overall" : A number of substantive and editorial issues on several QA Framework and other QA topics are found in the (linked) document:

which was submitted by DI WG to QA on 14 Mar 2003:

Six general comments to the QA activity (which have been classified as comments against "Introduction", for processing purposes):

  1. [GC-1]: QA work appreciated, but please "author good tools" to minimize cost to WGs.

    Resolution. It is QAWG's intention to shift its resources to tools now, as well as finishing Test Guidelines. How much we can achieve is ultimately limited QAWG resources (number of active participants). Tool suggestions are welcome.

  2. [GC-2]: A DIFF application for subsequent (WG) spec versions would be useful.

    Resolution. A tool like this might be considered outside of the scope of QAWG/IG charters (more in-scope for Comm, for example). That notwithstanding, a number of such tools exist. A summary of diff tools was a topic on the W3C spec-prod list.

  3. [GC-3]: A set of standard universal templates to help WGs write their specs.

    Resolution. Templates to facilitate the QA-related aspects of operations, specifications, and test materials is definitely within the QAWG/IG scope and future agenda. The suggested "universal template" might actually fall in the domain of the Comm team, or possible joint Comm-QAWG project. In recent Chairs email, a new homepage for editors was announced. This suggestion will be forwarded to Comm for further discussion.

  4. [GC-4]: Can QA suggest review procedures/guidelines that will ensure early involvement of non-WG reviewers?

    QAWG recognizes the value of early involvement of non-WG members in QA deliverables. It will be added to QAWG's agenda, to consider where and how this might best be addressed. It is unclear whether it should be addressed in one of the QA Framework documents (which one?), in a QA Note, or some other way.

  5. [GC-5]: [This is a comment with no apparent action needed -- it is a forward (now-current) reference to their CC/PP work.]

    Resolution. Subsequent to this comment, QAWG did a review of CC/PP according to Last Call WD of Specification Guidelines, and a discussion followed. Thanks for your consideration of QAWG's review comments in your CC/PP progression.

  6. [GC-6]: Questions about how to manage multiple glossaries and what's happening with the Glossary Project (3/2003 Technical Plenary).

    The glossary project have been through 5 months of development and is being tested at the moment, It can scrape, manipulate and index glossaries from various sources (including multiple, dated versions of a document), as well as translations. We are planning to send some news to the public list public-glossary@w3.org (to which the commenter could subscribe to get updates on the project, by the way) very soon now.

(Other comments/issues from DIWG: LC-77, LC-111.)

Proposal:
Resolution:

Table Legend

num
Last Call issue number
Title
Short title/name of the issue
Spec
Document referred to in Last Call issue (Intro = QA Framework: Introduction; OpsGuide & OpsGL = QA Framework: Operational Guidelines; SpecGuide & SpecGL = Framework: Specification Guidelines)
Description
Short description of issue, possibly including link to origin of issue
Date
The date at which the issue was raised or initially logged.
Topic
Rough topic categorization, one of: ...tbd... [replace following XMLP stuff... env(elope), rpc, enc(oding), meta(issue), bind(ing), fault]
Class
Substantive or Editorial
Status
One of: Unassigned, Active, Closed, Postponed
Proposal
Current proposal for resolution of issue, possibly including link to further text
Resolution
Short description of resolution, possibly including link to a more elaborate description
Raised by
Person who raised the issue
Owner
QA WG Member responsible for the issue

Maintained by Lofton Henderson.