QAWG Last Call Issues

Last update: $Date: 2003/05/07 17:08:31 $

Issues regarding the Last Call documents produced by the QA Working Group (QAWG) should be reported to the QAWG using the forms that are linked from the status sections of those documents. As a last resort, if you cannot use the forms for some reason, you can send mail to www-qa@w3.org (public archives). (The forms will put a copy on this email list and in the archives.)

Comments on this issues list should be sent to the www-qa@w3.org mailing list (Archives).

In this document,

Summary List of Outstanding Last Call Issues

num Status Spec Topic Class Date Title
LC-3 Resolved OpsGuide OpsGuide Substantive 2003-02-23 Committment Table and its CPs
LC-83 Resolved OpsGuide OpsGuide Substantive 2003-03-31 Seven levels vs. Three
LC-23 Resolved SpecGuide SpecGuide Substantive 2003-03-11 CP 1.4 vague
LC-30 Resolved SpecGuide SpecGuide Substantive 2003-03-11 Profile, module, level
LC-38 Resolved SpecGuide SpecGuide Substantive 2003-03-11 CP 9.1 and 9.2 combine
LC-41 Resolved SpecGuide SpecGuide Substantive 2003-03-11 CP 4.4 derived profile
LC-61 Resolved SpecGuide SpecGuide Substantive 2003-03-14 document product class
LC-66 Resolved SpecGuide SpecGuide Substantive 2003-03-14 edition and version DoVs
LC-79 Resolved SpecGuide SpecGuide Substantive 2003-03-19 SpecGL fails checkpoints 1.2 and 1.3
LC-84 Resolved SpecGuide SpecGuide Substantive 2003-03-31 Group dimensions of variability
LC-93 Resolved SpecGuide SpecGuide Substantive 2003-03-31 Classes vs. categories
LC-94 Resolved SpecGuide SpecGuide Substantive 2003-03-31 Loophole in Classes and Categories
LC-97 Resolved SpecGuide SpecGuide Substantive 2003-03-31 Modules as extension points
LC-98 Resolved SpecGuide SpecGuide Substantive 2003-03-31 Conformance levels
LC-99 Resolved SpecGuide SpecGuide Substantive 2003-03-31 Awkward deprecation requirements
LC-103 Resolved SpecGuide SpecGuide Substantive 2003-03-31 Distributed conformance section OK
LC-106 Resolved SpecGuide SpecGuide Substantive 2003-03-31 Section 3.1 - Poor Defn of Normative
LC-108 Resolved SpecGuide SpecGuide Substantive 2003-03-31 Definitions
LC-109 Resolved SpecGuide SpecGuide Substantive 2003-04-07 Is CP1.3 needed?
LC-4 Resolved SpecGuide SpecGuide Editorial 2003-02-19 no definition for unconditional conformance
LC-2 Resolved SpecGuide SpecGuide Editorial 2003-02-25 Grammatical error in Scope
LC-5 Resolved SpecGuide SpecGuide Editorial 2003-03-11 Ck 2.2 list of classes
LC-6 Resolved SpecGuide SpecGuide Editorial 2003-03-11 Guideline 2, typo
LC-7 Resolved SpecGuide SpecGuide Editorial 2003-03-11 Checkpoint 1.2
LC-8 Resolved SpecGuide SpecGuide Editorial 2003-03-11 checkpoint 1.1
LC-9 Resolved SpecGuide SpecGuide Editorial 2003-03-11 INTRODUCTION section 1.1, second bullet
LC-11 Resolved SpecGuide SpecGuide Editorial 2003-03-11 CK 2.3 Category of object: clarification
LC-12 Resolved SpecGuide SpecGuide Editorial 2003-03-11 section 3.4 Conformance definition
LC-16 Resolved SpecGuide SpecGuide Editorial 2003-03-11 CP 8.4 clarify conformance requirement
LC-18 Resolved SpecGuide SpecGuide Editorial 2003-03-11 GL 5 non-hierarchiacal modules
LC-21 Resolved SpecGuide SpecGuide Editorial 2003-03-11 CP 2.4 - relationships of DOV - clarify
LC-24 Resolved SpecGuide SpecGuide Editorial 2003-03-11 Introduction, Sect 1.4
LC-27 Resolved SpecGuide SpecGuide Editorial 2003-03-11 Typos, grammar, etc.
LC-32 Resolved SpecGuide SpecGuide Editorial 2003-03-11 4. Definitions
LC-33 Resolved SpecGuide SpecGuide Editorial 2003-03-11 4. Definitions
LC-36 Resolved SpecGuide SpecGuide Editorial 2003-03-11 Section 3.1 Normative sections
LC-44 Resolved SpecGuide SpecGuide Editorial 2003-03-13 Grammatical errors
LC-45 Resolved SpecGuide SpecGuide Editorial 2003-03-13 Design goal of guidelines
LC-46 Resolved SpecGuide SpecGuide Editorial 2003-03-13 Ambiguity
LC-47 Resolved SpecGuide SpecGuide Editorial 2003-03-13 Spelling error in Example and Techniques
LC-48 Resolved SpecGuide SpecGuide Editorial 2003-03-13 Categories of object not previously clearly defined
LC-49 Resolved SpecGuide SpecGuide Editorial 2003-03-13 Complexity in explanation
LC-50 Resolved SpecGuide SpecGuide Editorial 2003-03-13 Ambiguity or error?
LC-51 Resolved SpecGuide SpecGuide Editorial 2003-03-13 GLs 4, 5 and 6
LC-52 Resolved SpecGuide SpecGuide Editorial 2003-03-13 NOT ?
LC-53 Resolved SpecGuide SpecGuide Editorial 2003-03-13 Ambiguity
LC-62 Resolved SpecGuide SpecGuide Editorial 2003-03-14 typos
LC-63 Resolved SpecGuide SpecGuide Editorial 2003-03-14 Table of Contents
LC-65 Resolved SpecGuide SpecGuide Editorial 2003-03-14 sentences and paragraphs (section 3.1)
LC-85 Resolved SpecGuide SpecGuide Editorial 2003-03-31 Self-sufficient guidelines and checkpoints
LC-86 Resolved SpecGuide SpecGuide Editorial 2003-03-31 Spelling, grammar, and style
LC-87 Resolved SpecGuide SpecGuide Editorial 2003-03-31 Abbreviations
LC-89 Resolved SpecGuide SpecGuide Editorial 2003-03-31 Consolidate glossary and terminology
LC-90 Resolved SpecGuide SpecGuide Editorial 2003-03-31 Restructure DoV
LC-92 Resolved SpecGuide SpecGuide Editorial 2003-03-31 Examples vs. Illustrations
LC-71 Active IntroGuide IntroGuide Substantive 2003-03-15 Collected substantive & editorial comments
LC-76 Active IntroGuide IntroGuide Substantive 2003-03-19 Collected substantive & editorial comments
LC-68 Active IntroGuide IntroGuide Editorial 2003-03-14 Intro draft
LC-69 Active IntroGuide IntroGuide Editorial 2003-03-15 Several comments on various parts of Introduction
LC-43 Active OpsGuide OpsGuide Substantive 2003-03-12 OpsGL Appendix 1 - Process Document Template
LC-57 Active OpsGuide OpsGuide Substantive 2003-03-12 GL and timeline of a document
LC-58 Active OpsGuide OpsGuide Substantive 2003-03-12 Process Document requirement is too specific
LC-56 Active OpsGuide OpsGuide Substantive 2003-03-13 Accessibility
LC-60 Active OpsGuide OpsGuide Substantive 2003-03-14 Structure/Organization of Guidelines
LC-70 Active OpsGuide OpsGuide Substantive 2003-03-15 Checklist format issue
LC-72 Active OpsGuide OpsGuide Substantive 2003-03-15 Collected substantive & editorial comments
LC-82 Active OpsGuide OpsGuide Substantive 2003-03-19 OpsGL fails SpecGL checkpoints 1.2 and 1.3
LC-110 Active OpsGuide OpsGuide Substantive 2003-04-16 Collected OpsGL comments from Team.
LC-59 Active OpsGuide OpsGuide Editorial 2003-03-12 Testability concerns
LC-17 Active SpecGuide SpecGuide Substantive 2003-03-11 CP 7.1 deprecated features
LC-19 Active SpecGuide SpecGuide Substantive 2003-03-11 CP 4.4 explanation clarification
LC-29 Active SpecGuide SpecGuide Substantive 2003-03-11 questions and suggestions
LC-37 Active SpecGuide SpecGuide Substantive 2003-03-11 CP 9.3
LC-40 Active SpecGuide SpecGuide Substantive 2003-03-11 GL7 add obsolete features
LC-67 Active SpecGuide SpecGuide Substantive 2003-03-14 limits of RFC 2119 key words
LC-73 Active SpecGuide SpecGuide Substantive 2003-03-15 Collected substantive & editorial comments
LC-74 Active SpecGuide SpecGuide Substantive 2003-03-15 Simplify & consolidate the guidelines of SpecGL
LC-75 Active SpecGuide SpecGuide Substantive 2003-03-16 Comments on SpecGL Guidelines
LC-77 Active SpecGuide SpecGuide Substantive 2003-03-19 Collected substantive & editorial comments
LC-78 Active SpecGuide SpecGuide Substantive 2003-03-19 Should provide a disclaimer template
LC-80 Active SpecGuide SpecGuide Substantive 2003-03-19 SpecGL should address the topic of CP applicability.
LC-81 Active SpecGuide SpecGuide Substantive 2003-03-19 CP9.6 conformance requirements and rationale may be too narrow.
LC-95 Active SpecGuide SpecGuide Substantive 2003-03-31 Why is conformance policy a DoV?
LC-96 Active SpecGuide SpecGuide Substantive 2003-03-31 Priorities confusing
LC-100 Active SpecGuide SpecGuide Substantive 2003-03-31 Don't discourage extensibility
LC-101 Active SpecGuide SpecGuide Substantive 2003-03-31 Remove Checkpoint 9.6
LC-102 Active SpecGuide SpecGuide Substantive 2003-03-31 Consolidate G3 and G10.
LC-104 Active SpecGuide SpecGuide Substantive 2003-03-31 Consolidate G11 with G3/G10
LC-105 Active SpecGuide SpecGuide Substantive 2003-03-31 G3, G10, G11, G13
LC-14 Active SpecGuide SpecGuide Editorial 2003-03-11 CP 14.1 - clarify conformance requirement
LC-20 Active SpecGuide SpecGuide Editorial 2003-03-11 CP 3.1 Conformance Requirements - clarify
LC-22 Active SpecGuide SpecGuide Editorial 2003-03-11 CP 2.3 placement of Checkpoint
LC-25 Active SpecGuide SpecGuide Editorial 2003-03-11 Introduction: scope and goals
LC-26 Active SpecGuide SpecGuide Editorial 2003-03-11 Multiple CPs - It is not applicable
LC-28 Active SpecGuide SpecGuide Editorial 2003-03-11 document organization suggestions
LC-34 Active SpecGuide SpecGuide Editorial 2003-03-11 Section 3.4 Conformance definition
LC-35 Active SpecGuide SpecGuide Editorial 2003-03-11 Section 3.2 Extensibility
LC-39 Active SpecGuide SpecGuide Editorial 2003-03-11 CP 8.4 policies for discretionary choices
LC-42 Active SpecGuide SpecGuide Editorial 2003-03-11 GL3: contradiction? regarding examples
LC-54 Active SpecGuide SpecGuide Editorial 2003-03-13 GLs 3, 10, 11, 12 and 13
LC-64 Active SpecGuide SpecGuide Editorial 2003-03-14 conformance terms
LC-88 Active SpecGuide SpecGuide Editorial 2003-03-31 Reformat bullet list
LC-91 Active SpecGuide SpecGuide Closed 2003-03-31 Complexity of numbering

Detailed List of Last Call Issues

num Spec Date Topic Class Status Raised By Owner
LC-1 SpecGuide 2003-02-28 SpecGuide Substantive Closed Alex Rousskov Lynne Rosenthal
Title: require a "Security Considerations" section
Description:

Specification reference.

comment about "Overall" :

Any spec SHOULD have a Security Consideration section. Protocol or behavioral specs MUST have a Security Consideration section.

Security sections make spec authors think about potential vulnerabilities and address at least some of them before the bad guys can exploit them. These sections are also a great place to warn implementors and users about most security-sensitive areas of the spec and, perhaps, common exploits.

IETF's Internet Architecture Board has published the following Internet Draft that may be of use to SpecGL authors: http://www.ietf.org/internet-drafts/draft-iab-sec-cons-03.txt

Discussed at 20030331 telecon. QAWG believes that the proposal has merit, but that specifying such requirements is outside of the scope of the QA Framework. Will be referred to appropriate W3C group or team.

Proposal: Require "Security Considerations" sections just like we already require conformance sections.
Resolution: The proposal has merit, and should be addressed by W3C, but security requirements are outside of the scope of QA Framework. Will be referred to appropriate W3C group or team.
LC-2 SpecGuide 2003-02-25 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: Grammatical error in Scope
Description:

Specification reference.

comment about "Abstract, Status,..." : "Section 3 defines explains how to make conformance claims that W3C TRs satisfy the requirements of section 2. It defines conformance for this document"

It seems as if either 'defines' or 'explains' is meant, not both. The first sentence is grammatically incorrect, thus the intended meaning is unclear. The second sentence might seem superfluous, depending on the intended meaning of the first.

Proposal:
Resolution:
LC-3 OpsGuide 2003-02-23 OpsGuide Substantive Resolved lynne rosenthal Lofton Henderson
Title: Committment Table and its CPs
Description:

Specification reference.

comment about "Guideline 1: Integrate Quality Assurance into Working Group activities." :

Remove the Committment Table and CP1.1, 1.2, 1.3. These items do not add information and are redundant. To satisfy Level 3 (5 or 7), you must satisfy other checkpoints in the OpsGL. In fact, if we did our jobs right, Level 3 should equal Conformance Level A (satifying all P1), Level 5 should equal Level Double A, and Level 7 = Triple A. Isn't it a goal to get WGs to conform to the OpsGL? That would mean that they must satisfy all the P1 checkpoints. Thus, they must have a committment level to P1. So, why do we use a new term - Level 3.

The Table introduces a DOV - called committment level. The OpsGL does not conform to the SpecGL's CPs that require DoVs to address the relationship to conformance, to other DoVs, etc.

Email discussion , including proposed resolution.

Discussed and resolved at 20030428 telecon and subsequent email . Table will be eliminated, and CP1.1 - 1.3 will be replaced. CP1.1 will require choice of A-AAA for OpsGL, choice of A-AAA for SpecGL, and choice of A-AAA for TestGL. Plus 1-2 additional CPs preserving the commitment to have some test materials.

Related LC issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal: [Originator] Remove the Table and CP1.1, 1.2, 1.3. An alternative to removing the Table is to amend it and move it to either the Introduction section (prior to Guideline section) or an Appendix. It should be amended by adding an additional column called, 'checkpoint'. For each row, indicate the checkpoint that applies.

[LH] See email.

Resolved per emailed final text.

Resolution: Per above, see emailed final text.
LC-4 SpecGuide 2003-02-19 SpecGuide Editorial Resolved Olivier Thereaux Lynne Rosenthal
Title: no definition for unconditional conformance
Description:

Specification reference.

comment about "Overall" : In the QAframe-spec glossary (http://www.w3.org/TR/2003/WD-qaframe-spec-20030210/#definitions) there is no definition for the unconditional conformance term.

See subsequent email proposal, to remove empty (unused) definition.

Related issues: 4, 32.

Proposal: [Originator] add one? :)
Resolution: Remove empty definition from SpecGL. (Add to "QA Glossary".)
LC-5 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: Ck 2.2 list of classes
Description:

Specification reference.

comment about "2.1 Identify all classes of product. " : second conformance requirement refers to list of classes but its not clear which list it is referring to. If it's the list under guideline 2 then that list is non-exhaustive so requiring people to use that list is somewhat limiting.

Discussed at 20030331 telecon, classified as editorial.

Proposal: [Proposed wording to be provided by SpecGL editors.]
Resolution:
LC-6 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: Guideline 2, typo
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " : thrid para - typo 'as either or produceser' remove 'or'
Proposal:
Resolution:
LC-7 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: Checkpoint 1.2
Description:

Specification reference.

comment about "1.2 Illustrate what is in scope" : Can use cases and examples be in a separate document from the main spec?

Discussed at 20030407 telecon. Resolved "yes", they can be in a separate document.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal:
Resolution: Yes, examples and use cases can be in a separate document.
LC-8 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: checkpoint 1.1
Description:

Specification reference.

comment about "1.1 Include the scope of the specification" : rather wooly conformance requirements

Previous email: ISO 'scope' definition; QAWG discussion thread.

Discussed at 20030331 telecon, classified as editorial. Action. (Someone) request clarification from originator.

Proposal: [Proposed wording to be provided by SpecGL editors.]
Resolution:
LC-9 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: INTRODUCTION section 1.1, second bullet
Description:

Specification reference.

comment about "Abstract, Status,..." : Second sentence implies that all checkpoints must be satisfied to comply with the guidelines wherease only priority 1 checkpoints are mandatory.

Discussed at 20030331 telecon, classified as editorial.

Proposal: [Proposed wording to be provided by SpecGL editors.]
Resolution:
LC-10 SpecGuide 2003-03-11 SpecGuide Editorial Closed Marc Hadley Lynne Rosenthal
Title: Section 7
Description:

Specification reference.

comment about "Overall" : date for LC WD is in the future (or it was when the doc was published).
Proposal: Reviewer used the WG version of the document and not the LC version of the SpecGL. Moot point.
Resolution:
LC-11 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: CK 2.3 Category of object: clarification
Description:

Specification reference.

comment about "2.3 Identify which of the categories of object..." : What is a category of object? The same as a class of product

Discussed at 20030331 telecon, classified as editorial.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email. Proposed wording to be provided by SpecGL editors.
Resolution: The verbiage on classes and categories will be rewritten according to the 4-point proposal in email comments. [LR tentatively has drafting action.]
LC-12 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: section 3.4 Conformance definition
Description:

Specification reference.

comment about "Overall" : Example - not true, see comment on 3.3

(LH clarification.) The example does not correspond to the requirements that are listed immediately before it -- apparently the latter were updated without updating the example.

Proposal:
Resolution: Update the example to match the requirements.
LC-13 SpecGuide 2003-03-11 SpecGuide Substantive Closed Marc Hadley Lynne Rosenthal
Title: Section 3.3 Conformance and TAs
Description:

Specification reference.

comment about "Overall" : amusing that this section doesn't meet checkpoint 14.1 and therefore renders the document as only A-conforming to itself. Would be better if the document were AAA-conforming to itself IMO.

Discussed at 20030331 telecon. QAWG agrees that SpecGL should be AAA-conformant to SpecGL, and commits to AAA conformance by Candidate Recommendation.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal: SpecGL MUST be AAA-conformant to SpecGL by Candidate Recommendation.
Resolution: SpecGL MUST be AAA-conformant to SpecGL by Candidate Recommendation.
LC-14 SpecGuide 2003-03-11 SpecGuide Editorial Active Marc Hadley Lynne Rosenthal
Title: CP 14.1 - clarify conformance requirement
Description:

Specification reference.

comment about "14.1 Provide test assertions" : conformance requirements - is a separate document OK or does this have to be in the same doc a the rest of the spec?

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal:
Resolution:
LC-15 SpecGuide 2003-03-11 SpecGuide Substantive Closed Marc Hadley Lynne Rosenthal
Title: extensions
Description:

Specification reference.

comment about "Guideline 9 Allow extensions or NOT!" : A very well thought out section IMO.

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal: Thanks to commentor, this is useful feedback on a controversial topic.
Resolution: Closed (no issue).
LC-16 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: CP 8.4 clarify conformance requirement
Description:

Specification reference.

comment about "8.4 Promote consistent handling of discretionary choices." : conformance requirements not clear, what does 'document the identified policies for handling discretionary choices' mean?

Discussed at 20030331 telecon, classified as editorial.

Discussed at 20030410 telecon. AI given to DM (LH & DH assisting) to draft clarification.

Related LC issues: 16, 27, 39.

Proposal: [Proposed wording to be provided by SpecGL editors.] Draft simple clarification, or remove if no QAWG consensus on the clarified text.
Resolution:
LC-17 SpecGuide 2003-03-11 SpecGuide Substantive Active Marc Hadley Lynne Rosenthal
Title: CP 7.1 deprecated features
Description:

Specification reference.

comment about "7.1 Identify each deprecated feature. " : conformance requirements imply a single section for deprecated features - is it not OK to include deprecations where they occur without a summary section?

(See subsequent email discussion.)

Proposal:
Resolution:
LC-18 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: GL 5 non-hierarchiacal modules
Description:

Specification reference.

comment about "Guideline 5 Address the use of modules to divide the technology." : Modules are non-hierarchiacal - can modules have dependencies on other modules? If so, isn't this a hierarchy?

(See subsequent email discussion.)

Discussed at 20030331 telecon, classified as editorial.

Proposal: [Proposed wording to be provided by SpecGL editors.]
Resolution:
LC-19 SpecGuide 2003-03-11 SpecGuide Substantive Active Marc Hadley Lynne Rosenthal
Title: CP 4.4 explanation clarification
Description:

Specification reference.

comment about "4.4 If profiles are chosen, address rules for profiles." : 'experience shows ... meets all the pertinent checkpoints of this document' - what experience? As this is not yet a crecommendation this seems like a rather strong statement.
Proposal:
Resolution:
LC-20 SpecGuide 2003-03-11 SpecGuide Editorial Active Marc Hadley Lynne Rosenthal
Title: CP 3.1 Conformance Requirements - clarify
Description:

Specification reference.

comment about "3.1 any universal requirements for minimum functionality." : conformance requirements - only one section for this or are multiple O.K?
Proposal:
Resolution:
LC-21 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Marc Hadley Lynne Rosenthal
Title: CP 2.4 - relationships of DOV - clarify
Description:

Specification reference.

comment about "2.4 If there are several classes of products, define their relationships..." : 'define their relationships and interaction with other dimensions of variability' this is a confusing checkpoint that is repeated in each successive guideline. It's really not clear exactly what is intended.

Email comments, including proposal (20030418).

Discussed and resolved at 20030421 telecon.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: See email.
Resolution: This will be addressed in a subsection of the planned new chapter -- "2. Concepts".
LC-22 SpecGuide 2003-03-11 SpecGuide Editorial Active Ian Jacobs Lynne Rosenthal
Title: CP 2.3 placement of Checkpoint
Description:

Specification reference.

comment about "2.3 Identify which of the categories of object..." : This checkpoint includes a requirement for "where in your specification," even though the intro says those requirements are limited to GL1 and GL10-14.
Proposal:
Resolution:
LC-23 SpecGuide 2003-03-11 SpecGuide Substantive Resolved Ian Jacobs Lynne Rosenthal
Title: CP 1.4 vague
Description:

Specification reference.

comment about "1.4 Provide examples. " :

I think this checkpoint is too vague. Is it possible to break it down into a few more concrete requirements that are more verifiable? For instance:

  1. For markup specifications, provide at least one example of each markup construct.
  2. For protocol specifications, provide at least one example of ...
  3. For transformation specifications, illustrate each transformation capability with an example showing input and output.
  4. For UI specifications, provide an example of each construct, and illustrate the desired output using at least one mechanism other than the specification itself (e.g., the SVG specification should not rely on SVG rendering alone to explain what something should look like).

While you may miss some cases, I think spec editors will find this more helpful than the general goal to "provide examples."

Email discussion and discussed in 20030307 telecon. Alternatives:

  1. Implement the proposal: make more specific requirements and tie it to specification categories.
  2. Implement the proposal in the ExTech document, but leave CP as is.
  3. Implement the proposal in the ExTech document, and make CP more specific without mentioning categories. (I'm not sure how we do this).

Resolved in favor of a variation on Alt.2 (which might be what was meant by Alt.3?) -- the discussion and breakdown by category will be done in ExTech. But the Discussion of CP1.4 will mention the concept raised by comment originator (that different kinds of specification categories require different particular associated kinds of examples), and maybe even give a single brief "for example."

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal: Originator proposed Alt #1.
Resolution: Resolved in favor of modified Alt 2. Details tbd.
LC-24 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Ian Jacobs Lynne Rosenthal
Title: Introduction, Sect 1.4
Description:

Specification reference.

comment about "Overall" : last para: Pubrules and the MoS aren't really specifications. What about "resources?"
Proposal:
Resolution:
LC-25 SpecGuide 2003-03-11 SpecGuide Editorial Active Ian Jacobs Lynne Rosenthal
Title: Introduction: scope and goals
Description:

Specification reference.

comment about "Overall" : Section 1.1 Last paragraph

I think the previous paragraph doesn't belong here; it could be deleted or moved to the status section (after some editorial fixes).

Proposal:
Resolution:
LC-26 SpecGuide 2003-03-11 SpecGuide Editorial Active Ian Jacobs Lynne Rosenthal
Title: Multiple CPs - It is not applicable
Description:

Specification reference.

comment about "Overall" : CP 2.4, 4.1, 4.2, 4.3, 4.4, 5.1, 5.2, 6.1, 7.1, 7.2, 7.4, 7.5, 8.1, 8.3, 9.2, 9.4, 9.5, 9.6,

Suggest that the statment "It is not applicable if..." be labeled as "Normative inclusion/exclusion" as in UAAG 1.0

Email discussion, including proposal.

Related LC applicability/normative-exclusion issues: 26, 28, 73.2, 80.

Proposal:
Resolution:
LC-27 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Ian Jacobs Lynne Rosenthal
Title: Typos, grammar, etc.
Description:

Specification reference.

comment about "Overall" : GL 3, 2md para, Remove: "Overall, the intent of the WG should be clear" This doesn't add anything

CP 7.1, 7.4, 7.5 ConfReq, last sentence: change 'is' to 'are' in "...if there is no deprecated feature"

CP8.4 Rationale: change 'identifying' to 'identify'

CP9.1, last para: "This is strict conformance" is a repeat of text in intro of GL9

CP 11.4, last para: modify "proper use of the conformance icons" to "proper use of any conformance icons"

GL13, last para: suggest switching the order of Manual of Style and PubRules

Related LC issues: 16, 27, 39.

Proposal:
Resolution:
LC-28 SpecGuide 2003-03-11 SpecGuide Editorial Active Ian Jacobs Lynne Rosenthal
Title: document organization suggestions
Description:

Specification reference.

comment about "Overall" : Adopt the UAAG 1.0 approach of separating requirements from applicability exclusions (called "normative inclusions/exclusions" in UAAG 1.0). Use style sheets to hide links (e.g., to examples and techniques) for printed version.

Discussed at 20030331 telecon, classified as editorial.

Email discussion, including proposal.

Related LC applicability/normative-exclusion issues: 26, 28, 73.2, 80.

Proposal: [Proposed wording to be provided by SpecGL editors.]
Resolution:
LC-29 SpecGuide 2003-03-11 SpecGuide Substantive Active Ian Jacobs Lynne Rosenthal
Title: questions and suggestions
Description:

Specification reference.

comment about "Overall" :
  1. In general, specifications make functional requirements. In some cases, it may be necessary to make a performance requirement. How should that be handled?
  2. In UAAG 1.0, we realized that for some of our configuration requirements, it didn't matter whether the user agent offered the desired configuration or didn't implement the functionality in the first place. E.g., we decided that it was ok for a user agent to not support blinking at all, or, if blinking is implemented, to offer a configuration to turn it off. However, in other cases, the configurability was "just as important" as the functionality to be configured. Authors should consider these when they specify configuration options.
  3. It might be valuable for a specification to explain how to include its requirements in another specification. This is not the same as explaining how to reference the spec, or how to claim conformance to it.
  4. It might be valuable to explain some desirable characteristics of a specified technical requirement:
    1. Mutual independence from other requirements
    2. Expresses a minimal requirement
    3. Distinguish and label: requirements, exceptions to those requirements, necessary and/or sufficient techniques for satisfying those requirements.
  5. I think that more could be stated about useful ways of allowing extensibility. See, for example, how CSS (forward-compatible parsing), XML, and HTTP handle this. What should be avoided? Is the general practice of "ignore what you don't know" a good idea or a big mistake?
  6. A spec should clearly indicate which illustrations (e.g., images) and examples are normative, if any.

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal:
Resolution:
LC-30 SpecGuide 2003-03-11 SpecGuide Substantive Resolved Ian Jacobs Lynne Rosenthal
Title: Profile, module, level
Description:

Specification reference.

comment about "Overall" : I am struggling to understand a sharp distinction between profile, module, and level. They are all mechanisms for defining and labeling a set of technical requirements. I have the feeling guidelines 4, 5, and 6 could be combined, and the requirements rephrased "Whatever subsetting mechanism you use..."

Discussed at 20030410 telecon. Disagreement about keeping or merging the concepts. AIs given to: draft a proposal merge the concepts; survey the usage of the terms and concepts in W3C specs.

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. QAWG agrees that clarification is needed. The solution for the whole profile/module/level issue group is:

  1. the concepts of profiles, modules, and levels are kept separate, each a separate DoV -- although the concepts are sometimes misused, and although other ways to subdivide the specification could be postulated, the QAWG believes that these three are heavily used in current practice, and that SpecGL can help to improve consistent;
  2. a subsection (sec 2.3?) to clarify the definitions and concepts of profiles, modules, and levels will be put into the new Ch.2, "Concepts";
  3. the three guidelines GL4,5,6 will be combined into a single guideline, with reduced verbiage, mostly pointing back to section 2.3;
  4. however the checkpoints will be kept substantially as they are, each dealing individually with one of the concepts of profiles or modules or levels -- while there is interest in combining checkpoints, on the other hand it was tried in the 1st PWD of SpecGL (May 2002) and caused too many problems;
  5. the exception is that the three separate "relationship to other DoV" checkpoints will be combined into one.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal: Email proposal.
Resolution: See above solution. Agreed to clarify concepts with better definitions and discussion, and to merge the three separate guidelines into a single guideline. However the 3 individual concepts will be kept separate within the checkpoints.
LC-31 SpecGuide 2003-03-11 SpecGuide Editorial Closed Ian Jacobs Lynne Rosenthal
Title: general
Description:

Specification reference.

comment about "Overall" :

General comments/conformance model I think the document is well-organized, clearly written, and very helpful. Having spent a lot of time thinking about conformance issues (notably for UAAG 1.0), I thought it covered a lot of ground and did so well.

I like the distinction between requirements that relate to the conformance model and those that involve implementing the model in the spec.

Proposal: Thanks to commentor. This is useful feedback.
Resolution: Closed (no issue).
LC-32 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Ian Jacobs Lynne Rosenthal
Title: 4. Definitions
Description:

Specification reference.

comment about "Overall" : unconditional conformance -- This is used in HTTP. It means "all requirements met."

See subsequent email proposal, to remove empty (unused) definition.

Related issues: 4, 32.

Proposal: [LH] Remove empty (unused) definition.
Resolution: Remove empty definition from SpecGL. (Add to "QA Glossary".)
LC-33 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Ian Jacobs Lynne Rosenthal
Title: 4. Definitions
Description:

Specification reference.

comment about "Overall" : "deprecated -- An existing feature that has become outdated by a newer construct or is no longer viable. Deprecated features should not be used"

Hmm, "used" may not be a specific enough term. A spec may encourage a UA to support a feature, but discourage an author from producing it. and may be removed in some future version.

See email discussion.

Discussed at 20030410 telecon. Resolved: agree with the suggested clarifications, will make appropriate changes.

Related LC deprecation issues: 33, 40, 99.

Proposal:
Resolution: Agree to clarify language to resolve originator's issue. [Tbd]
LC-34 SpecGuide 2003-03-11 SpecGuide Editorial Active Ian Jacobs Lynne Rosenthal
Title: Section 3.4 Conformance definition
Description:

Specification reference.

comment about "Overall" : " A checkpoint is satisfied by satisfying all of the individual @@conformance requirements@@. Failing one individual mandatory requirement means that the checkpoint is not satisfied." Is previous sentence required?
Proposal:
Resolution:
LC-35 SpecGuide 2003-03-11 SpecGuide Editorial Active Ian Jacobs Lynne Rosenthal
Title: Section 3.2 Extensibility
Description:

Specification reference.

comment about "Overall" : "...to this specification MUST not contradict nor negate the requirements of this specification." Delete previous statement per comment in checkpoint 9.3.

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal:
Resolution:
LC-36 SpecGuide 2003-03-11 SpecGuide Editorial Resolved Ian Jacobs Lynne Rosenthal
Title: Section 3.1 Normative sections
Description:

Specification reference.

comment about "Overall" : Is the glossary normative?

Discussed at 20030410 telecon. See also subsequent email thread. Generally resolved that our (sec.4) definition of "normative" is okay, but it is different than some others use (e.g., UAAG), involving the notion that normative text is directly connected to conformance requirements. By our definition, things like Glossary and Priorities definitions are NOT normative.

Related LC whats-normative issues: 36, 65, 106, 108.

Proposal: SpecGL uses a more focused definition of normative -- normative stuff is directly connected to conformance requirements. See proposals in email, to help clarify the definition and its usage in SpecGL. Per 20030414 telecon, MS will draft an addition to the definition of normative to clarify this.
Resolution:
LC-37 SpecGuide 2003-03-11 SpecGuide Substantive Active Ian Jacobs Lynne Rosenthal
Title: CP 9.3
Description:

Specification reference.

comment about "9.3 Prevent extensions from contradicting the specification." : I think checkpoint 9.3 should be deleted. I think that it's straightforward that if the spec says "A" and someone else says "not A," then anyone that does "not A" doesn't conform.

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal:
Resolution:
LC-38 SpecGuide 2003-03-11 SpecGuide Substantive Resolved Ian Jacobs Lynne Rosenthal
Title: CP 9.1 and 9.2 combine
Description:

Specification reference.

comment about "9.1 Indicate if the specification is extensible." : I think checkpoints 9.1 and 9.2 should be combined into one.

Discussed at 20030407 telecon. Agreed that they should be combined. Details tbd by SpecGL editors.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal: Combine CP 9.1 and 9.2.
Resolution: Combine CP9.1 and 9.2, details tbd.
LC-39 SpecGuide 2003-03-11 SpecGuide Editorial Active Ian Jacobs Lynne Rosenthal
Title: CP 8.4 policies for discretionary choices
Description:

Specification reference.

comment about "8.4 Promote consistent handling of discretionary choices." : Can "document the identified policies" be simplified?

Discussed at 20030331 telecon, classified as editorial.

Discussed at 20030410 telecon. AI given to DM (LH & DH assisting) to draft clarification.

Related LC issues: 16, 27, 39.

Proposal: [Proposed wording to be provided by SpecGL editors.] Draft simple clarification, or remove if no QAWG consensus on the clarified text.
Resolution:
LC-40 SpecGuide 2003-03-11 SpecGuide Substantive Active Ian Jacobs Lynne Rosenthal
Title: GL7 add obsolete features
Description:

Specification reference.

comment about "Guideline 7 Identify the relation between deprecated features and conformance." : I think it's also important to identify obsolete features and provide althernatives to them. E.g., HTML 4.0 obsoleted a few elements.

See email thread.

Discussed at 20030410 telecon. No general consensus yet. There were differing views on what obsolete means (is it closer to deprecation or to removal)? The nature of the HTML example needs to be looked at. LR takes action to discuss with originator, look at HTML, draft proposal for QAWG discussion.

Related LC deprecation issues: 33, 40, 99.

Proposal:
Resolution:
LC-41 SpecGuide 2003-03-11 SpecGuide Substantive Resolved Ian Jacobs Lynne Rosenthal
Title: CP 4.4 derived profile
Description:

Specification reference.

comment about "4.4 If profiles are chosen, address rules for profiles." : What is the definition of a derived profile?

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. See the description of the solution for the profile/module/level issue group, in LC issue 30.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: "Derived profile" will be defined as part of the solution for the profile/module/level issue group.
LC-42 SpecGuide 2003-03-11 SpecGuide Editorial Active Ian Jacobs Lynne Rosenthal
Title: GL3: contradiction? regarding examples
Description:

Specification reference.

comment about "Guideline 3 Specify conformance policy. " : last para, last sentence: Does "possibly provide examples" conflict with MUST provide examples of 1.4?

Related LC GL3-policy issues: 42, 96.

Proposal:
Resolution:
LC-43 OpsGuide 2003-03-12 OpsGuide Substantive Active Peter Fawcett Lofton Henderson
Title: OpsGL Appendix 1 - Process Document Template
Description:

Specification reference.

comment about "Overall" :
  1. - "test material" is referred to in multiple ways in the document including but not limited to "test material" and "test material type". We could not determine a difference in concept from context nor could we come up with an example that needed both.
  2. - The language in Rec 1 seems odd. Especially the second sentence. "The Checklist should be developed in a public framework. The development of this framework itself is a central area of interest. The QAWG welcomes the participation of interested parties in developing the Checklist."
  3. - The third sentence in Rec 3 also seems a bit odd
  4. - We found a number of uses of "test materials" and "test suite" that did not use the "fill-in" markup.
  5. - DocumentFramework should be Document Framework
  6. - In Test materials contrabuition section in the a/b choice rather than saying tests or series.. say contributions.
  7. - In Test Materials contribution section there is a missing 'span' element around address. above]'/span' mailing list: [address].'p'
  8. - We found the first item in the list under Test Materials Contribution to be some what redundant. At least in our case as any submissions would already go to the list as it is. The test or series of tests, get submitted to the framework. Submitter should also send a notification to the mailing list that will be set up for the Checklist to indicate that he/she has submitted a test to the class="fill-this-in">[test material name] framework.
  9. - There is a typo in section Receipt and review of test contributions "according tot he" should be "according to the"
  10. - In Receipt and Review section list item 1 it currently says: "since it tests a feature that is not in the specification" should it say something more like: "since it tests a recommendation that is not in the specification" as 'feature' is not a universally applicable term.
  11. - In Receipt and Review section list item 3: This item makes the assumption that the test cases will be written in some xml based language. How ever can we make this assumption? What about languages like rdf that don't use an xml syntax?
  12. - this same section (item 3) also has a hanging ']'/span'' on it.
  13. - In Receipt and Review section list item 4: "'li''em'Scenarios'/em' that underlay the particular test layout and its intended scope.'/li'" We found the use of "test layout" to be out of place in the document. it is the only place that this term is used. We believe that it could just as easily use language that's more standard to the document.
  14. - In Receipt and Review section list item 5: The point of this checkpoint is valid, in that if the submitted test doesn't follow the Development Guide it should be rejected, how ever the notion of the Development Guide is introduced here (in this document). We feel that it should be listed earlier in the document as well. perhaps in the requirements, that the test materials be developed in accordance with the Development Guide.
  15. - Typo in last sentence of Receipt and Review section: is "publication arereurned to" should be: "publication are returned to"
  16. - In Test status and review procedures Section: In the second paragraph it reads: "the status of the test is changed to reflect the fact that its validity has been disputed" Earlier we say that "status is changed to "accepted" we should use the same verbiage here like: "state changed to "disputed""
  17. - In Test status and review procedures Section: The last sentence of the second paragraph has very odd wording.
  18. - In Test status and review procedures Section: second paragraph uses term "Task Force" in terms of the maintainers of the test materials. This is another concept that is first introduced in this context. This too would work better if the concept of appointing a "Test Task Force" was done at an earlier step.
  19. - In Test status and review procedures Section in the first item of the list. The label "stable" may not be the best word for case 1 as it doesn't really reflect what is meant. "Error Free" or "Valid" would work better.
  20. - In Test status and review procedures Section in the third item of the list. It reads: "questioned the correctness, consistency, clarity, etc" we think that correctness does not belong.
  21. - In Test status and review procedures Section in the second and third item of the list. "A consensus exists in the community" What community? The w3c, the 'whole internet' the WG (as this is specifically talking about the task force.) We are also fairly uncomfortable with the notion that the task force can be over ridden by the (not clearly defined) community. Either the taskforce agrees or disagrees and the case accepted or not. Unless "community" means WG in which case this might make sense...
  22. - In "Section Status of the test suite according to the above" We feel that the wording of this section heading could be improved quite a lot. We assume this means that if you conform to all of the above. or you've done/are doing all of the above.
  23. - In "Section Status of the test suite according to the above" second paragraph It reads: "It is proposed that the W3C WG representative act as moderator and controller for incoming tests. The moderator is chosen by the [WG name] WG. All tests should be kept for archive purposes, whether they get published or not." Much of this is repeated from above. The new information is that a moderator should be chosen. This is another case where it may be beneficial to introduce the concept of a Moderator earlier in the document. This is already covered above to some degree. But with moderator.
  24. - Both the Documentation and "See above" sections do not have references back to a GL document.
Proposal:
Resolution:
LC-44 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: Grammatical errors
Description:

Specification reference.

comment about "1.1 Include the scope of the specification" : "

This document is applied and conformance (to this document) achieved as new TRs are being written." (grammatically incorrect, intended meaning unclear) This document applies to new TRs and conformance (to this document) is achieved as they are being written.

As for legacy specification, they may indirectly comply with the spirit or intent of some checkpoints, without actually satisfying all requirements in those checkpoints. (grammatical/spelling error) "legacy specifications"

Within this Specification Guidelines document, the term "specifications' is specifically limited to W3C Technical Reports, even though these guidelines could be applied to other documents. (unbalanced quote marks)

Proposal:
Resolution:
LC-45 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: Design goal of guidelines
Description:

Specification reference.

comment about "1.2 Illustrate what is in scope" : 1.2 Class of Product and Audience -- "It is a design goal of these guidelines the WGs can apply them in a common-sense and workable manner."
Proposal: These guidelines are designed so that the WGs can apply ....
Resolution:
LC-46 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: Ambiguity
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " :

I'm finding it difficult to follow which terms are reserved for 'classes of products'. The word 'consumer' is used to describe the classes of products, and itself is listed within the classes of products. For example, I find the following sentence semantically confusing: "For a processor-type specification, the processor is the consumer of an XML vocabulary defined in the specification." "For content-type specifications, there may be one or more consumers that take the content and 'play' it in some way." "Play" refers to a media player, or play refers to "process" ?

Divide this (enumerated) list into processor, consumer, or content? Make the terminology in this area unique, so that there will be no ambiguity? (It could be that the terminology is already unique, but in its current format, I can't be sure.)

Email discussion.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon. Rewrite the GL verbiage for clarification. The detail about sub-dividing the list of classes is to be decided by SpecGL editors during rewrite of the verbiage.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: The verbiage on classes and categories will be rewritten according to the 4-point proposal in email comments. The detail about sub-dividing the list of classes is to be decided by SpecGL editors during rewrite of the verbiage. [LR tentatively has drafting action.]
LC-47 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: Spelling error in Example and Techniques
Description:

Specification reference.

comment about "2.2 For each class of product, define the conformance requirements. " : Spelling error in corresponding example and techniques: http://www.w3.org/QA/WG/2003/02/qaframe-spec-extech-20030203#Ck-define-scope "XHTML", not "XHML" :)
Proposal:
Resolution:
LC-48 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: Categories of object not previously clearly defined
Description:

Specification reference.

comment about "2.3 Identify which of the categories of object..." :

"Checkpoint 2.3. Identify which of the categories of object are specified in the document as a whole. [Priority 3]"

Reader should be able to understand what 'categories of object' are upon reference - this is the first instance that this phrase is used in this document. Even though a URI is provided to the applicable definition of 'categories of object', the definition itself should introduce the list with this phrase. Eg: "Most specifications can be classified into one of the following categories of object ..."

Discussed at 20030418 telecon.

Discussed and resolved at 20030418 telecon. Rewrite the GL verbiage for clarification. "Categories of object" will be replaced by the standard terminology, "specification category".

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: The verbiage on classes and categories will be rewritten according to the 4-point proposal in email comments. [LR tentatively has drafting action.]
LC-49 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: Complexity in explanation
Description:

Specification reference.

comment about "Guideline 4 Address the use of profiles to divide the technology." : I can't understand what 'profile' means by this explanation; this should perhaps be simplified?

Discussed at 20030410 telecon.

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. See the description of the solution for the profile/module/level issue group, in LC issue 30.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: The definition and illustration of the "profile" concept will be improved as part of the solution for the profile/module/level issue group.
LC-50 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: Ambiguity or error?
Description:

Specification reference.

comment about "4.1 Indicate whether or not the use of profiles is mandatory..." : "For example, is content required to conform to one of the profiles, or is there a concept of conformance of content independent of conformance to one of the profiles?"

Ambiguity or error? (I count four 'of's in the second clause! :D)

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. See the description of the solution for the profile/module/level issue group, in LC issue 30.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: The specific editorial problem will be examined and fixed as part of the implementation of the solution for the profile/module/level issue group.
LC-51 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: GLs 4, 5 and 6
Description:

Specification reference.

comment about "Overall" : Explanations for profiles, modules and functional levels are vague. Use diagrams for examples ?

Discussed at 20030410 telecon.

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. See the description of the solution for the profile/module/level issue group, in LC issue 30. Note that the specific suggestion to use diagrams was not addressed (SpecGL editors?).

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: The definitions and illustrations of the three concepts -- profiles, modules, and levels -- are to be improved as part of the implementation of the solution for the profile/module/level issue group. [Note to SpecGL editors: use of diagram(s) is tbd.]
LC-52 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: NOT ?
Description:

Specification reference.

comment about "Guideline 9 Allow extensions or NOT!" : 'NOT!' seems rather uncharacteristic and out of line with regards to the remaining document. What about simply "Not" ?

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal:
Resolution: Resolved, change "NOT!" to "not."
LC-53 SpecGuide 2003-03-13 SpecGuide Editorial Resolved Stephanie Troeth Lynne Rosenthal
Title: Ambiguity
Description:

Specification reference.

comment about "Overall" : 1.3 Motivation and Expected Benefits (Introduction) -- "Providing requirements and definitions about conformance topics, as well as guidance in the structure and anatomy of specifications, will foster a mutual understanding among developers of specifications, implementations, and conformance test materials." (Comment: meaning is ambiguous) "foster a mutual understanding among developers about ..."
Proposal:
Resolution:
LC-54 SpecGuide 2003-03-13 SpecGuide Editorial Active Stephanie Troeth Lynne Rosenthal
Title: GLs 3, 10, 11, 12 and 13
Description:

Specification reference.

comment about "Overall" : These are all guidelines which refer specifically to conformance. Would it make more sense to number these sequentially (in order to group them together), rather than having the big gap between 3 and 10 ?

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal:
Resolution:
LC-55 SpecGuide 2003-03-13 SpecGuide Substantive Closed Jon Gunderson Lynne Rosenthal
Title: Accessibility
Description:

Specification reference.

comment about "Overall" : Specifications should have a section or the ability to highligh the features of the specification that benefit people with disabilities.

Discussed at 20030331 telecon. QAWG believes that the proposal has merit, but that specifying accessibility requirements is outside of the scope of the QA Framework. It more appropriately belongs in pubrules. Will be referred to Comm team.

Proposal: Include a requirement that a specification have a section summarizing the accessibility features of the specification
Resolution: Specifying accessibility requirements is outside of the scope of the QA Framework. It more appropriately belongs in Pubrules. This proposal will be referred to Comm team.
LC-56 OpsGuide 2003-03-13 OpsGuide Substantive Active Jon Gunderson Lynne Rosenthal
Title: Accessibility
Description:

Specification reference.

comment about "Overall" : QA test suites should also include tests that test the accessibility features of a specification based on the accessibility requirements found in other W3C documents. This may require having a specific person in charge of defining and monitoring the inclusion of accessibility features.

See previous QAWG issue 102.

Email discussion, including proposed resolution.

Proposal: [Originator] Include a requirement in the Operation Guidelines for a person to be responsible for accessibility tests of a specification.

[LH] See email.

Resolution:
LC-57 OpsGuide 2003-03-12 OpsGuide Substantive Active Dominique Hazael-Massieux Lofton Henderson
Title: GL and timeline of a document
Description:

Specification reference.

comment about "Overall" : most guidelines are only applicable at some point in the WG's life, but the GL don't identify this aspect: this is something that absolutely needs to be stressed, and could even be used as a strategy to organize the GL as a whole, e.g. what you need to do before starting a WG, what needs to be done when you start developing a new spec, what needs to be done when you envision building a test suite, etc.

Email discussion, including proposals.

Proposal: [Originator] at least, provide a section (an image?) linking the GL or the CP to the milestones of a WG life

[LH] See email (essentially same as Originator's).

Resolution:
LC-58 OpsGuide 2003-03-12 OpsGuide Substantive Active Dominique Hazael-Massieux Lofton Henderson
Title: Process Document requirement is too specific
Description:

Specification reference.

comment about "Overall" : a WG should still be able to comply to a CP without having a QA Process Document.
Proposal: replace QA process document references by a documented WG decision?
Resolution:
LC-59 OpsGuide 2003-03-12 OpsGuide Editorial Active Dominique Hazael-Massieux Lofton Henderson
Title: Testability concerns
Description:

Specification reference.

comment about "Overall" : The following expressions seemed either hard to apply or to test:
  1. - "QA deliverables" is very broad and not defined (GL3)
  2. - "ensure" is not testable in a conformance requirement (at least CP 3.2, 6.1)
  3. - "commensurate" isn't either (CP 4.2)
  4. - using the future makes a CP untestable (CP 6.3)
  5. - "a quality assessment" is not well defined (CP 7.1)
  6. - "sufficient" is not testable (CP 7.2)
  7. - 'all' is not testable (CP 7.3)
Proposal:
Resolution:
LC-60 OpsGuide 2003-03-14 OpsGuide Substantive Active Patrick Curran Lofton Henderson
Title: Structure/Organization of Guidelines
Description:

Specification reference.

comment about "Overall" : Comments on the structure and organization of the guidelines and checkpoints of SpecGL.
  1. Overall, I think we have the right checkpoints, but I think they would be easier to understand if they were grouped chronologically. Start with what needs to be done when the WG is formed, and proceed through the spec and test development cycle to the maintenance phase, as we have tried to do with the restructuring of TestGL.

    Email discussion, including proposals.

  2. Guideline 1: I find checkpoints 1.1 - 1.3 very confusing. These are "compound checkpoints" that incorporate multiple other checkpoints. We don't use this structure anywhere else in our docs - why here? The document states "This seven-point enumeration is derived from the proposal to the QA mail list, after the 4/2001 QA Workshop", but how we got here is not really interesting to the reader.

    The 'sub-checkpoints' on the 'left hand side' of the table are all spec-related. Either these duplicate checkpoints from SpecGL, or they should be incorporated into that document. Those on the 'right hand side' are operations-related, but they seem to overlap with other checkpoints specified in this document.

    The remaining checkpoints under guideline 1 are different kinds of beasts (not compound) and as such, the transition to them seems somewhat abrupt.

    Recommendation: drop the compound structure, make sure that all the spec-related sub-checkpoints are covered by SpecGL, and move the 'test materials' sub-checkpoints into the body of this doc.

    Email discussion , including proposed resolution.

    Discussed and resolved at 20030428 telecon. See LC-3 for description and resolution.

  3. Guideline 2: I'd say "Allocate resources..." rather than "Define resources..."
  4. Guideline 3 begins with some text ("The benefits of....") that seems to be a rationale. Other Guidelines jump straight into the Conformance requirements - this one should too.
  5. How is checkpoint 3.1 different from 1.5?
  6. Checkpoint 3.2 doesn't seem to be directly related to the Guideline under which it's classified. The Note for 3.2 points out that checkpoint 8.2 is related - doesn't this suggest that the overall structure should be re-worked (if they're related, why are they so far apart numerically?)
  7. Guideline 4: Probably should be #1 (it's the first chronologically). When we summarized this document in our outreach presentation we made checkpoint 4.1 the first bullet item...
  8. Checkpoints 4.1 and 4.2 would seem to belong in Guideline 2 (define/allocate resources) rather than here?
  9. The Discussion for checkpoint 4.3 says "To summarize...". This implies that somewhere there's a more detailed description of what the QAPD must address, but I don't think there is. This checkpoint really seems to amount to "document how you meet these other checkpoints", yet the list of "other checkpoints" that must be implemented is incomplete. Why doesn't this simply require that *all* checkpoints be documented?
  10. Checkpoint 4.5 uses the ambiguous term "framework" (we tried to avoid this in our re-write of OpsGL).
  11. Checkpoint 4.6 addresses branding - another argument for a chronological rather than a 'logical' grouping (this should be at the end).
  12. Guideline 5: Checkpoint 5.2 - Define a contribution process. Why only priority 2? Without a contribution process you have nothing, surely?
  13. Guideline 6: Checkpoint 6.1 contains a mixture of stuff. The guideline addresses "publication" but much of this checkpoint addresses "management". Moreover, the bullet items in the Discussion section don't seem to relate to repositories at all.
  14. Checkpoint 6.2 (defining the license for published test materials) is closely related to 5.3 (defining the license for submissions). Should these be grouped together (under a guideline that addresses submission processes)?
  15. Guideline 7: This guideline is labelled "plan the transfer of test materials to W3C if needed", and explicitly states "all of the checkpoints... are not applicable if the WG does not transfer..." (should be "none of the checkpoints are applicable if..."). However, all the checkpoints seem to apply whether or not the materials are "transferred". It's obviously important to review the quality of submitted tests, to ensure that we have the staffing to deal with submissions, and to resolve IPR issues.

For LC-60.3 through LC-60.15, see email, including some proposals.

Related LC issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal: [Originator] I think we have approximately the right checkpoints, but I think they would be easier to understand if they were grouped chronologically. A possible set of guidelines might be:
  • Charter
  • Allocation of resources
  • Planning & synchronization
  • Test submission/development
  • Test management
  • Test publication
  • Conformance testing (test usage)
  • Maintenance
[LH alternate proposal] See email.
Resolution:
LC-61 SpecGuide 2003-03-14 SpecGuide Substantive Resolved Susan Lesch Lynne Rosenthal
Title: document product class
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " : Guideline 2 could give "document" and "resource" either as product classes or as examples of the "content" class. (Checkpoint 2.1 does finally mention "document.")

Email comments (20030417).

Discussed at 20030418 telecon.

Discussed and resolved at 20030418 telecon. We don't see documents and resources as examples of 'content' class (perhaps some better definition of 'content' class is needed?). We agreed that a 'specification' class should be added.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: Add a 'specification' class. [LR tentatively has drafting action -- do this as part of issue 11, 46, 48, 93 solution.]
LC-62 SpecGuide 2003-03-14 SpecGuide Editorial Resolved Susan Lesch Lynne Rosenthal
Title: typos
Description:

Specification reference.

comment about "Overall" : In section 3.3, s/This Operational Guidelines document/This Specification Guidelines document/

In the guideline title and table of contents entry for Guideline 12 and in 3.4 last par., "pro forma" is two words.

The table of contents link to section 3.1 is broken.

(These probably have been reported by now but just in case.)

Proposal:
Resolution:
LC-63 SpecGuide 2003-03-14 SpecGuide Editorial Resolved Susan Lesch Lynne Rosenthal
Title: Table of Contents
Description:

Specification reference.

comment about "Overall" : Would the table of contents in QA Framework specs that have guidelines be easier to follow if non-Guideline sections all appeared by number in the table of contents? E.g. "3. Conformance" could be followed by "3.1 Normative sections" rather than by "1. Normative sections"? The difference between for example "checkpoint 1.1" and "section 1.1" (both about scope) then would be distinct and easier to talk about.

Discussed at 20030407 telecon. Agreed to implement suggested change -- subsections in TOC will be numbered N.M (M=1,2...), instead of just M.

Related LC Section/GL/CP numbering issues: 63, 77.ET-3, 91.

Proposal: Subsections in TOC will be numbered N.M (M=1,2...), instead of just M.
Resolution: Subsections in TOC will be numbered N.M (M=1,2...), instead of just M.
LC-64 SpecGuide 2003-03-14 SpecGuide Editorial Active Susan Lesch Lynne Rosenthal
Title: conformance terms
Description:

Specification reference.

comment about "3.2 If special conformance terms are used, include a definition..." : Checkpoint 3.2 says: "the specification MUST be defined, either by reference or by including the definition in the text." Did you mean to say that the whole spec is considered to be "defined" through references and definitions? In that case, please ignore this comment.

Email discussion, including proposal for closure.

Proposal: Could read something like: "terms used to describe conformance MUST be defined, either by reference or by including the definition in the text." (I'm afraid "conformance terms..." might mean you'd have to define "terms.")
Resolution:
LC-65 SpecGuide 2003-03-14 SpecGuide Editorial Resolved Susan Lesch Lynne Rosenthal
Title: sentences and paragraphs (section 3.1)
Description:

Specification reference.

comment about "Overall" : In section 3.1, "sentences" becomes "typically, one paragraph...." Is a sentence containing an RFC 2119 key word a unit of being normative, or do you mean that a paragraph containing such a sentence is or can be the unit? I'm not sure if it is important to draw a line. Section 4's definition of normative says "text in a specification which is prescriptive or contains conformance requirements" which seems to mean any text with no boundaries (for example, like section, chapter, slice).

Discussed at 20030410 telecon. See also subsequent email thread. Agreed that section 3.1 has some problems that will be fixed: it is not comprehensive (omits some normative stuff); and has misleading characterization of what is normative. We note that SpecGL's definition of normative (sec.4) is more focused than some other use, and it does not qualify a Glossary as normative, for example.

Related LC whats-normative issues: 36, 65, 106, 108.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal:
Resolution: Improve the language in section 3.1, and make sure that its "what's normative" list is complete. Terms normative and informative to be linked to (sec.4) definitions. [Tbd, by LR.] Per 20030414 telecon, MS to draft addition to definition of normative, to help clarify its focus on direct connection to conformance requirements.
LC-66 SpecGuide 2003-03-14 SpecGuide Substantive Resolved Susan Lesch Lynne Rosenthal
Title: edition and version DoVs
Description:

Specification reference.

comment about "Overall" : Guidelines 4-6 and the section 4 definitions are great descriptions of profiles, modules, and levels, thanks. Are "editions" and "versions" DoVs? If for example, a requirement changed between Version 1.0 and Version 1.1 of some specification, so that a 1.0 processor could not read 1.1, that might be a "variability."

Email discussion (20030331).

Email discussion (20030417).

Email comments, including proposal (20030418).

Discussed at 20030421 telecon, resolved per email proposal.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: [Originator] If you think edition and version do matter, they could be addressed in section 1.8, or in a separate Guideline. [LR] See email proposal. [LH] See email.
Resolution: Per email proposal, version and edition are not DoV. DoV are concerned with conformance to a single, specific version/edition of a specification at a time. [Detail TBD -- will SpecGL clarify this about the scope of DoV, in the new chapter, "2. Concepts"?]
LC-67 SpecGuide 2003-03-14 SpecGuide Substantive Active Susan Lesch Lynne Rosenthal
Title: limits of RFC 2119 key words
Description:

Specification reference.

comment about "13.1 Use conformance key words." : "the specification MUST use RFC 2119 keywords to denote whether or not requirements are mandatory, optional, or suggested" is Priority 1. Must all testable statements and or test assertions (sorry I'm not clear on the difference between them and maybe that needs to be clarified, too) contain RFC 2119 key words?

RFC 2119 [1] section "6. Guidance in the use of these Imperatives" puts limits their use. (I mentioned this to the QAWG about a year ago.) They musn't be used only to ask implementers to do something a Working Group would like to see.

Email discussion (20030418).

Discussed and resolved 2nd part -- guidance about RFC keyword usage -- at 20030421 telecon. Resolved: SpecGL won't try to legislate correct use of RFC2199 keywords. SpecET (Examples & Techniques) will mention that correct-use guidance is given in Section 6 of RFC2119, and point to it. There is concern that quoting the particular wording of RFC2119 -- with reflects a particular perspective (communications & protocols) and orientation -- "muddies the water" in some contexts. QAWG thinks that it would be useful to investigate a joint Comm-QAWG project to draft a guidance Note on this topic.

Subsequent to resolution of 2nd part, additional comments & discussion occurred on QAIG list. (While apparently not changing the resolution, it may require more explanation of our rationale, in response to commenters.)

The 1st part -- must every conformance requirement use RFC2119 keywords -- is still tbd. (Note counter example in UAAG 1.0: "This document demands substantially more conformance flexibility than can be achieved using the terms "must", "should", and "may" alone, as defined in RFC 2119 [RFC2119]. Where "must", "should", "required", and "may" appear in this document, they are used consistently with RFC 2119 for a chosen conformance profile. The imperative voice (e.g., "Allow configuration ...") used in the checkpoint provisions implies "must", but a user agent is only obligated to satisfy the requirements of a chosen conformance profile."

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal: [Originator] One way to solve this is to quote or paraphrase and link to the RFC.

"Imperatives of the type defined in this memo must be used with care and sparingly. In particular, they MUST only be used where it is actually required for interoperation or to limit behavior which has potential for causing harm (e.g., limiting retransmisssions) For example, they must not be used to try to impose a particular method on implementors where the method is not required for interoperability."

[LH] See email proposal: don't legislate it in SpecGL; develop a Note about it and reference that from SpecET.

[1] http://www.rfc-editor.org/rfc/rfc2119.txt

Resolution: 2nd part: SpecGL won't try to legislate correct use of RFC2199 keywords. SpecET (Examples & Techniques) will mention that correct-use guidance is given in Section 6 of RFC2119, and point to it. There is concern that quoting the particular wording of RFC2119 -- with reflects a particular perspective (communications & protocols) and orientation -- "muddies the water" in some contexts. QAWG thinks that it would be useful to investigate a joint Comm-QAWG project to draft a guidance Note on this topic.
LC-68 IntroGuide 2003-03-14 IntroGuide Editorial Active Susan Lesch Lofton Henderson
Title: Intro draft
Description:

Specification reference.

comment about "General: miscellaneous & other" : I found Ian Jacob's SpecGL edits [1] in your archive and would like to send ideas for the QA Framework Introduction. I expect they will be ready at [2] by 14 March 24:00 Pacific.
Proposal: Use or not, as you see fit. As this is my last Last Call comment, I just want to say that it must have been a monumental task to put the framework together. You've made it look elegant, and easy to use. Best wishes for your project.
Resolution:
LC-69 IntroGuide 2003-03-15 IntroGuide Editorial Active Colleen Evans Lofton Henderson
Title: Several comments on various parts of Introduction
Description:

Specification reference.

comment about "General: miscellaneous & other" : [Entered into form by LH. Some are significant editorial, but all are classified as "Editorial" because there are no conformance implications or suggestions for major document reorganization.]

Review comments:

  1. In general it provides useful guidance on how to use the Framework, defining audience and WG activity applicability for each document.
  2. Is this a Working Draft or Last Call Working Draft? Title indicates the former.
  3. Interspersed usage of the terms 'document', 'guideline', and 'specification' to describe the QA Framework documents could be confusing. E.g., paragraph 4 of Status: "... It is anticipated that this specification will eventually progress, along with its Operational Guidelines and Specification Guidelines companions, to Candidate Recommendation (CR) and beyond. The timing of progression of this specification will be determined by the progression of the companion guidelines documents." Similarly, 'TR', 'standard', 'specification', and 'recommendation' are used interchangeably to describe the output of WGs.
  4. Section 1.3, first sentence: "The last underscores a key reality of improved quality practices associated with W3C technical reports". Not clear what 'the last' is (previous section?).
  5. Section 1.4, paragraph 3, first sentence incomplete? "While some might perceive QA projects as a regrettable drain on WG resources, there is ample experience, both within W3C as well as other standards venues, that shows significant improvement to the products of the WGs."
  6. Sections 1.3 (paragraph 1) and 1.4 (paragraphs 2 and 3) contain general justification arguments for QA efforts in WGs - may be more appropriate content for Section 1.2.
  7. Section 3.1 Application Domain - does this belong in Section 3 (Structure and content of Framework documents)? Seems more like Section 1 (Overview) content where target audience is covered.
  8. Sections 3.5.2 - 3.5.5 describe each document - information on content, audience, and objective. It may help readability to use a consistent order for presenting this information across sections.
  9. Section 3.5.4 Single item bullet list?
  10. Section 4.1.3 Useful breakout of document relevance by role within a WG.
  11. Section 4.2 Provides a good life cycle view of the relationship between Framework documents and WG activities. A table summary might be useful as well.
  12. Section 4.1.3 "WG-TS moderator" - Section 4.2.2 "test materials (QA) moderator". Same role?
  13. Section 4.2.3, paragraph 4, second sentence is unclear: "Normally, this should not be considered as a good time to bring a specification for 'Specification Guidelines' conformance, as the latter could significantly disrupt and restructure the specification.". Is 'the latter' referring to bringing a spec to specification guidelines conformance, or something in a previous sentence?
  14. Section 4.2.5. Intra-WG build of test materials calls for an acceptance procedure for the individual bits. Import and assemble only call for quality assessment and assessment criteria - is an acceptance procedure required / implied?
  15. (Editorial) Usage of Working Group vs. WG inconsistent throughout
  16. Inconsistent bullet list punctuation (';' vs. ',' vs. nothing at line end, etc.)
  17. Section 1.2, paragraph 1 "...." at end of first sentence
  18. Section 4.2.5, paragraph 2 "? -- as " in middle of second sentence
Proposal:
Resolution:
LC-70 OpsGuide 2003-03-15 OpsGuide Substantive Active Phill Jenkins Lofton Henderson
Title: Checklist format issue
Description:

Specification reference.

comment about "Overall" : [Entered into form by LH. Comment applies to Checklists of SpecGL and TestGL as well OpsGL.]

I have noticed a number of "checklists" being proliferated on the W3C pages. I have a strong recommendation for improving the adaptability of the checklist format - namely the number of column in the layout. For example, today most have a column for the number of the checkpoint, the description of the checkpoint, and then a number of columns for YES, NO, N/A. Please consider adapting the following format:

  • Top matter: Version, date, owner, etc
  • Column 1 10% Checkpoint number
  • Column 2 40% Description
  • Column 3 10% YES, NO, N/A, Planned
  • Column 4 40% Comments
  • Bottom matter: Footnotes, exceptions, references, etc.

an example we find useful is at http://www-3.ibm.com/able/accesssoftware.html .

[[Following is (unarchived) response comment from Ian Jacobs: We redesigned the UAAG 1.0 checklist [1] based on earlier comments from you [1]. We don't specify fixed widths for table columns.

Proposal:
Resolution:
LC-71 IntroGuide 2003-03-15 IntroGuide Substantive Active Leonid Arbouzov Lofton Henderson
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "General: miscellaneous & other" : [Entered into form by LH.]

Six substantive and editorial issues,

  1. [IN-1]:
  2. [IN-2]:
  3. [IN-3]:
  4. [IN-4]:
  5. [IN-5]:
  6. [IN-6]:

on "QA Framework: Introduction" are included in the document at

http://www.w3.org/QA/WG/2003/03/qareview20030314.html ,

which was submitted to QA by XML Schema on 14 Mar 2003:

http://lists.w3.org/Archives/Public/www-qa/2003Mar/0079.html .

Proposal:
Resolution:
LC-72 OpsGuide 2003-03-15 OpsGuide Substantive Active Leonid Arbouzov Lofton Henderson
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "Overall" : [Entered into form by LH.]

Thirteen substantive and editorial issues on "QA Framework: Operational Guidelines",

  1. [OG-1]: Editorial -- unclear wording in CP1.1.

    Proposal. Accept Originator's proposed rewording.

  2. [OG-2]: 7-level table creates confusing conformance requirements that contradict the priorities/degrees system.

    Email discussion , including proposed resolution.

    Discussed and resolved at 20030428 telecon. See LC-3 for description and resolution.

  3. [OG-3]: CP1.1 is priority 1, but requires checkpoints of other priorities.

    Email discussion , including proposed resolution.

    Discussed at 20030428 telecon. Becomes moot as a result of the resolution of LC-3.

  4. [OG-4]: GL3 wording emphasizes interoperable implementations, should instead emphasize conformant implementations.

    Proposal. Accept Originator's proposed rewording.

  5. [OG-5]: CP3.2, "support specification versioning/errata" should be priority 1 instead of 3.

    Proposal. Accept -- change p3 to p1.

  6. [OG-6]: Editorial -- poor wording in CP4.3.

    Proposal. Agree that it's poor wording, change "minimally addresses all of the topics" to "addresses at least all of the topics".

  7. [OG-7]: Editorial -- CP4.5 uses "QA Framework" in a different (but undefined) sense that the OpsGL title.

    Proposal. Agree, the same phrase is used with a different meaning. Change the CP wording from "Define the QA framework for test materials development" to "Define the framework for test materials development.".

  8. [OG-8]: CP4.5 should be priority 2 instead of priority 1.

    Proposal [LH]. Accept -- defining the framework for TM development is not critical enough to be Priority 1. Lower to priority 2.

  9. [OG-9]: Editorial -- ambiguous wording in CP5.4.

    Agree with the intent of the comment. Change "..the procedure MUST minimally address criteria for.." to "..the procedure MUST address at least criteria for.."

  10. [OG-10]: CP6.2 -- "address TM licenses" -- should be substantively modified.

    Email discussion & proposal, and discussion at 20030428 telecon. We believe that the telecon decisions resolve this -- describe the attributes of the licenses, and include a caveat about open issues involving the Document License. See proposed revised text.

  11. [OG-11]: CP6.3 arguments against publishing TM in the TR space are not convincing, should be improved (proposal included).

    Proposal. Accept the proposed rewording.

  12. [OG-12]: Problems with CP6.5 ("publish test results"), discussion of test harnesses, should be fixed.

    Proposal. Agree with intent of Originator. In Discussion, change the sentence, "Such publication should include or describe a test harness that would allow anyone to reproduce the results" to "Such publication should include or describe a test harness that allows reproduction of the results."

  13. [OG-13]: CP8.2 (versioning/errata in maintenance) should be priority 1 instead of 2.

    Proposal. Accept, change priority 2 to priority 1 in CP8.2 (versioning/errata in maintenance).

are included in the document at:

http://www.w3.org/QA/WG/2003/03/qareview20030314.html ,

which was submitted to QA by XML Schema on 14 Mar 2003:

http://lists.w3.org/Archives/Public/www-qa/2003Mar/0079.html

Related LC issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal:
Resolution:
LC-73 SpecGuide 2003-03-15 SpecGuide Substantive Active Leonid Arbouzov Lynne Rosenthal
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "Overall" : [Entered into form by LH.]

Nine substantive and editorial issues on "QA Framework: Specification Guidelines",

  1. [SG-1]: Editorial (SoTD).
  2. [SG-2]: Remove all "not applicable if..." statements.

    Email discussion, including proposal.

  3. [SG-3]: "Enumerate all CoP" is unreasonable requirement.

    Email comments (20030417), including proposals.

    Discussed and resolved at 20030418 telecon. We think that perhaps originator is confusing "all classes of product" with "all products". Nevertheless, the wording will be improved by: remove "all" from the checkpoint statement; and, change 1st bullet from "MUST list the classes of product it addresses" to "MUST list the classes of product for which it defines conformance requirements".

  4. [SG-4]: Editorial (8.5, ConfReq, change "profiles").
  5. [SG-5]: Editorial (Questions "NOT!" in GL9.)

    "Resolved. Change "NOT!" to "not."

  6. [SG-6]: Editorial (Improve wording in CP9.1)
  7. [SG-7]: "Normative refs" requirement should be priority 1.
  8. [SG-8]: Add Rationale and Discussion to CP13.3 ("Use consistent terminology").
  9. [SG-9]: Improve definition of "test assertions" (GL14).

are included in the document at:

http://www.w3.org/QA/WG/2003/03/qareview20030314.html ,

which was submitted to QA by XML Schema on 14 Mar 2003:

http://lists.w3.org/Archives/Public/www-qa/2003Mar/0079.html .

Extension-group processing plan and subsequent email discussion.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Related LC applicability/normative-exclusion issues: 26, 28, 73.2, 80.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal:
Resolution:
LC-74 SpecGuide 2003-03-15 SpecGuide Substantive Active Lofton Henderson Lynne Rosenthal
Title: Simplify & consolidate the guidelines of SpecGL
Description:

Specification reference.

comment about "Overall" :

If the 14 specification guidelines can be clearly summarized in 6 bullets, as we claim is done in the QA Outreach Kit, http://www.w3.org/2003/Talks/qa-outreach/slide5-0.html , then perhaps the guidelines themselves ought to be consolidated and reduced in number. I believe that most of the checkpoints are appropriate, but perhaps should be repackaged.

Six bullets:

  • Scope - define the scope, identifying what needs to conform and how
  • Conformance Clause - specify the conformance policy and requirements
  • If you must subset - use profiles, modules, functional levels
  • Extensions - specify whether and how extensions are allowed
  • Tag for later use - identify testable assertions, discretionary items
  • Claims - specify how conformance claims and statements are made

I think the first 4 are almost right for guidelines. The later ones don't seem to me to be quite right.

  1. new GL1: define and illustrate scope (includes old GL1, GL2)
  2. new GL2: define conformance policy and requirements (old GL3, GL10)
  3. new GL3: subset as needed (old GL4, GL5, GL6)
  4. new GL4: extensions (old GL9)

    (It gets a little rougher from here on...)

  5. new GL5: get a handle on other conformance variability
  6. new GL6: is there a nice umbrella statement that would accomodate GL13 and GL14?
  7. new GL7: claims -- specify how and provide an ICS [old GL11, GL12]

So at best, we would have 7 guidelines. At worst, 9 (if the new GL5 wouldn't stick together, and we couldn't find a new GL6 that was satisfactory).

Email comments (20030417), including proposals.

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal: See email.
Resolution:
LC-75 SpecGuide 2003-03-16 SpecGuide Substantive Active Patrick Curran Lynne Rosenthal
Title: Comments on SpecGL Guidelines
Description:

Specification reference.

comment about "Overall" :
  1. Guideline 1: Checkpoint 1.4 is priority 2, yet it's listed after 1.3 which is priority 3. I suggest listing checkpoints in priority order.

    Discussed at 20030407 telecon. Resolved to switch the order in this case. However, this is NOT a general endorsement of "order by priority". Generally agreed that logical flow should have precedence, and then priority order if there is no determinant based on logical flow.

  2. Guideline 3: The Conformance Policy is likely to appear towards the end of a Spec. I suggest ordering the guidelines as they are likely to appear in the spec. (Besides, it belongs with, and maybe could even be combined with, guidelines 10, 11, 12, and 13.)
  3. Guidelines 4-9: This is a large number of guidelines and checkpoints to deal with something that's important, but that we wish to discourage (DOVs). Could we combine some or all of these guidelines?

    Email comments, including proposal (20030418).

    Discussed and resolved at 20030421 telecon. There may be some potential for consolidation in GL4-6 (profiles/modules/levels) -- TBD. But otherwise, the other DoV checkpoints don't seem amenable to consolidation. See resolution summary of this and related DoV issues.

  4. The "or NOT!" language in guideline 9 seems a little too informal,
  5. but more substantively, why do we wait until this guideline [GL9] to express our opinion that DOVs are undesirable.

    Email discussion.

    Email comments, including proposal (20030418).

    Discussed and resolved at 20030421 telecon. A modified version of the previous per-GL DoV caveat will be restored. See resolution summary of this and related DoV issues.

  6. Guideline 10: Checkpoint 10.2 doesn't seem to be directly related to the guideline.
  7. Guidelines 10 - 13: As suggested above, perhaps these could be combined.
  8. Guideline 14: This is a key guideline - I'd like to see it earlier in the list.
  9. Section 3.3: "This... document does not enumerate a list of test assertions". Haven't we agreed that our checklist is (the closest thing to) a list of assertions?

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal: Restructure guidelines, combining several resulting in a smaller number.
Resolution:
LC-76 IntroGuide 2003-03-19 IntroGuide Substantive Active Roger Gimson Lofton Henderson
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "General: miscellaneous & other" : [Entered into form by LH.]

Four substantive and editorial issues (IN-1, .., IN-4) on "QA Framework: Introduction" are included in the document at

http://www.w3.org/QA/WG/2003/03/DIWGcomments.html ,

which was submitted to QA on 14 Mar 2003:

http://lists.w3.org/Archives/Member/w3c-di-wg/2003Mar/0074.html .

Also there are three issues applicable to all Framework, (FR-1, ..), and six general comments to QA (GC-1, .., GC-6).

Proposal:
Resolution:
LC-77 SpecGuide 2003-03-19 SpecGuide Substantive Active Roger Gimson Lynne Rosenthal
Title: Collected substantive & editorial comments
Description:

Specification reference.

comment about "Overall" :

Two substantive and editorial issues on "QA Framework: Specification Guidelines",

  1. [SG-1] More discussion of several DoV and their relationships.

    Email comments, including proposal (20030418).

    Discussed and resolved at 20030421 telecon. Subsections of the planned new chapter -- "2. Concepts" -- should provide the requested clarification.

  2. [SG-2] Standard headings for (Framework GL?) documents.
plus four related comments about the Specification Examples & Techniques documents,
  • [ET-1] (Affects Intro) Explain GL-ET pairing better.
  • [ET-2] (Affects SpecET and the other ETs). More E&T, in order to conform to SpecGL (GL1)!

    Resolved: it is the QAWG intention that SpecET will have at least one example per checkpoint. Tbd when the structure and checkpoint collection of SpecGL stabilizes.

  • [ET-3] (Affects SpecGL/ET, and the other GL/ET) Change GL, CP, and section numbering scheme.

    Discussed at 20030407 telecon. Closed with resolution to leave numbering scheme as is. Reasons: given that the context is usually clear (or easily made clear), the shorter GL/CP numbers are considered both more convenient and sufficient; and, the numbering scheme is familiar from WAI specifications and has been successfully used there.

  • [ET-4] (Affects SpecET) When will SpecET be released? (!?)
are included in the document at

http://www.w3.org/QA/WG/2003/03/DIWGcomments.html ,

which was submitted by DI WG to QA on 14 Mar 2003:

http://lists.w3.org/Archives/Member/w3c-di-wg/2003Mar/0074.html.

Also there are three issues applicable to all Framework,

  • [FR-1] (Affects SpecGL and other GLs) Can ISO9000 be applied?
  • [FR-2] (Affects Intro, and maybe all GLs). Give a cost/benefit analysis for conforming to the GLs.
  • [FR-3] (Affects Intro, and maybe all GLs). Explain/highlight the relationship of QA Framework to W3C Processes.

and six general comments to QA ( GC-1, .., GC-6).

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Related LC Section/GL/CP numbering issues: 63, 77.ET-3, 91.

Proposal:
Resolution:
LC-78 SpecGuide 2003-03-19 SpecGuide Substantive Active David Marston Lynne Rosenthal
Title: Should provide a disclaimer template
Description:

Specification reference.

comment about "11.3 Provide a conformance disclaimer." :

When I visited the joint XSL/XQ WG session (March 6 in Cambridge), there was quite a bit of discussion about the claims and disclaimers suggested in Guideline 11 of SpecGL. WG members attending wanted to have something more concrete to start with, like a "boilerplate" paragraph or two. In particular, we talked about number of test cases passed as a bad metric for conformance, because it implies that each case has equal weight. The current checkpoints don't call out this practice as warranting discouragement.

Proposal: Include in Ck 11.3, and probably 11.2 as well, a template that editors can paste into their specs. The template should include sentences addressing particular good (11.2) or bad (11.3) practices that might occur, allowing editors to remove irrelevant sentences.

I think that for 11.2, you already know that you intend a template resembling: This [class of product] was tested for conformance to [name of spec] version [N.nn], [Nth] Edition, dated [date], and all errata issued through [date], ... more specs cited same way .... The testing occurred on [date] using [test suite identifier] and [name of product and version] was found to conform at [level] level except for [enumeration of failing tests]. A full report of the parameter settings for the test harness and all results is posted at [URL] for open, public review.

For 11.3, the template could look like this: While the test suite provides [hundreds, thousands] of test cases, not all cases should be considered to carry equal weight. A product that passes all the tests may still not conform in some untested area. The W3C hereby states that claims of passage of a certain number of test cases or a certain percentage of the test cases, but not all, are invalid as relative measurements of conformance or worthiness, and that the only valid data that can be derived from such a result is that the product being tested does not pass all the tests. [Optional: More tests may be added to the suite in the future, and existing tests may be changed when errata are discovered. Failing some test cases cannot be interpreted as failing to conform without corroboration.] [Optional: The test suite can be tailored to suit permissible variability in product behavior. The W3C encourages implementers to provide information in their Implementation Conformance Statement that will lead to accurate configuration of the test suite, but holds the test lab responsible for obtaining the information and tailoring the suite accordingly, or else reporting which pieces of information were undetermined and indicating that some test failures may in fact be due to configuration problems.]

Resolution:
LC-79 SpecGuide 2003-03-19 SpecGuide Substantive Resolved Lofton Henderson Lynne Rosenthal
Title: SpecGL fails checkpoints 1.2 and 1.3
Description:

Specification reference.

comment about "Guideline 1 Define Scope." : [This is really a comment about the "Introduction" section.] "Specification Guidelines" (SpecGL) fails its own Checkpoints 1.2 (priority 2) and 1.3 (priority 3), by not illustrating its scope with examples and/or use cases, and not providing usage scenarios.

Alternatives:

  1. Add examples and/or use cases, and provide usage scenarios;
  2. No need to, these are "not applicable" to SpecGL;
  3. No need to, SpecGL only needs to conform to itself at level "A-Conforming" (only priority 1 checkpoints).

While I believe that alternative #1 is the correct alternative, it might be helpful for SpecGL to explain its intentions about self-conformance (p1 or p2 or p3?). I.e., should SpecGL contain a SpecGL conformance claim?

Discussed at 20030407 telecon. Resolved (and action item assigned) to add use cases and/or usage scenarios and/or examples to illustrate scope.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal: Alternative #1.
Resolution: Alternative #1. Details tbd.
LC-80 SpecGuide 2003-03-19 SpecGuide Substantive Active Lofton Henderson Lynne Rosenthal
Title: SpecGL should address the topic of CP applicability.
Description:

Specification reference.

comment about "Overall" : [This comment probably applies to the "Conformance" section.]

It is arguable that some SpecGL checkpoints -- applied to SpecGL itself, or to any other specification for that matter -- might be "not applicable", because of the nature of the specification. For some checkpoints (CP), SpecGL states conditions under which the CP is not applicable. It is unlikely that SpecGL has anticipated all of the circumstances under which a CP might be "not applicable".

Email discussion, including proposal.

Related LC applicability/normative-exclusion issues: 26, 28, 73.2, 80.

Proposal: In "Conformance" section, address what a specification tester, who is applying SpecGL's ICS, should do if he/she believes that a checkpoint is "not applicable" (n/a), but SpecGL has not indicated that the CP might n/a.
Resolution:
LC-81 SpecGuide 2003-03-19 SpecGuide Substantive Active Lofton Henderson Lynne Rosenthal
Title: CP9.6 conformance requirements and rationale may be too narrow.
Description:

Specification reference.

comment about "9.6 Require that implementations ... alternatives to extensions" :

Checkpoint 9.6 conformance requirements talk about an operating mode under which only strict-conforming content may be produced. I think that this may be too narrow a requirement. Would we consider the intent of the checkpoint to be satisfied if an implementation generated strict-conforming 'alt' content (to use the HTML analogy) to a private extension? In my opinion, a no-extensions mode is the best way to satisfy this CP. But is it the only way?

Note. There is some question why someone would include the extension at all, if the 'alt' content is "equivalent". Perhaps it is a way of round-tripping private functions of an implementation, while providing an alternative formulation that other implementations can use.

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal: 'Alt' mechanisms that contain only strict-conforming content, and achieve equivalent effect, should qualify.
Resolution:
LC-82 OpsGuide 2003-03-19 OpsGuide Substantive Active Lofton Henderson Lofton Henderson
Title: OpsGL fails SpecGL checkpoints 1.2 and 1.3
Description:

Specification reference.

comment about "Overall" : [This is really a comment about the "Introduction" section.] "Operational Guidelines" (OpsGL) fails Checkpoints 1.2 (priority 2) and 1.3 (priority 3) of "Specification Guidelines", by not illustrating its scope with examples and/or use cases, and not providing usage scenarios.

Alternatives:

  1. Add examples and/or use cases, and provide usage scenarios;
  2. No need to, these are "not applicable" to OpsGL;
  3. No need to, OpsGL only needs to conform to itself at level "A-Conforming" (only priority 1 checkpoints).

While I believe that alternative #1 is the correct alternative, it might be helpful for OpsGL to explain its intentions about conformance (A or AA or AAA?) to SpecGL. I.e., should OpsGL contain a SpecGL conformance claim?

Proposal: Alternative #1.
Resolution:
LC-83 OpsGuide 2003-03-31 OpsGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Seven levels vs. Three
Description:

Specification reference.

comment about "Overall" : All specs include seven "levels" of conformance. However, only three levels are actually public; these are called "priorities".

Ed note. Submitted as SpecGL issue, reclassified as OpsGL, as QAWG believes that the problem originates with the table of CP1.1 of OpsGL.

Email discussion , including proposed resolution.

Discussed and resolved at 20030428 telecon. See LC-3 for description and resolution.

Related LC issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal: [Originator] Collapse the seven levels into the three real ones (now called priorities), since these are the basis of conformance measurement.

[LH] See email.

Resolution:
LC-84 SpecGuide 2003-03-31 SpecGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Group dimensions of variability
Description:

Specification reference.

comment about "Overall" : Guidelines two through nine are grouped as "dimensions of variability," and referred to as such by themselves and by other guidelines. If the concept of dimensions of variability is of this much importance, it should be reflected in the structure of the document.

Email comments (20030417), including proposals.

Email comments, including proposal (20030418).

Discussed and resolved at 20030421 telecon.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: [Originator.] Guidelines two through nine should be grouped, structurally, as "dimensions of variability". [LH] See email.
Resolution: This will be addressed with the planned new chapter -- "2. Concepts". With these improvements, restructuring to segregate the DoV guidelines into their own document sections is not thought to be necessary.
LC-85 SpecGuide 2003-03-31 SpecGuide Editorial Resolved Jonathan Marsh Lynne Rosenthal
Title: Self-sufficient guidelines and checkpoints
Description:

Specification reference.

comment about "Overall" : In general, the language is insufficiently rigorous/precise for a common use case, and the model may be explained in text rather than structurally. The problem is that, while many WG members may read the entire spec, others may be tasked with judging conformance to a particular guideline, or even a particular checkpoint.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [Originator] Insofar as is possible guidelines and checkpoints should contain sufficient definition for local understanding, and pointers to all related items in the document. That is, a reader should be able to enter via any checkpoint, and gather all and only the information needed to understand the checkpoint starting from that entry point. [LH] See email.
Resolution: Resolved per proposal in email. Agreed that these are good principles. While it is beyond our resources to completely rewrite the document with these in mind, we will apply them going forward, as we edit and revise SpecGL. (E.g., links to definitions, to concept discussions, etc.)
LC-86 SpecGuide 2003-03-31 SpecGuide Editorial Resolved Jonathan Marsh Lynne Rosenthal
Title: Spelling, grammar, and style
Description:

Specification reference.

comment about "Overall" : Spell and grammar check. There are a number of problems with both, which probably do not deserve direct mention. Also, the style of the prose occasionally changes radically (from a formal, romance-language-influenced european english to a very colloquial american style). This can be jarring.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [LH] See email.
Resolution: MS and LR will make a review for grammar and spelling -- in the final WG-only SpecGL draft preceding the next published version. With our mix of authors, there is not much that we can do about different styles, other than to ensure that everything is grammatically correct.
LC-87 SpecGuide 2003-03-31 SpecGuide Editorial Resolved Jonathan Marsh Lynne Rosenthal
Title: Abbreviations
Description:

Specification reference.

comment about "Overall" : If abbreviated forms are used to refer to other documents, these abbreviations ought to also appear in the bibliography. In general, reference to a document should always be a <bibloc>.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [LH] See email.
Resolution: We believe that we are doing all references correctly, but see that we have omitted the brackets "[]" in the References section. That will be fixed. LR/MS check these also during grammar/spelling review (issue 86.)
LC-88 SpecGuide 2003-03-31 SpecGuide Editorial Active Jonathan Marsh Lynne Rosenthal
Title: Reformat bullet list
Description:

Specification reference.

comment about "1.3 Provide Usage Scenarios. " : Bullet lists are there to call out important items. An eight-item list should either be shortened or reformatted; it fails as bullet points.
Proposal:
Resolution:
LC-89 SpecGuide 2003-03-31 SpecGuide Editorial Resolved Jonathan Marsh Lynne Rosenthal
Title: Consolidate glossary and terminology
Description:

Specification reference.

comment about "Introduction" : Why isn't the glossary in section 1.6, instead of at the end?

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [Originator] Move the definitions appendix to section 1.6. [LH] See email.
Resolution: Resolved per email proposal. It is done both ways in W3C specifications, and QAWG prefers this way for SpecGL. Also make these two changes: change the title of 1.6 from "Terminology" to "Usage of terminology in this document"; and, add this sentence to 2nd paragraph, "When used in this specification, terms have the meanings assigned in 'Definitions' and 'QA Glossary' [QA-GLOSSARY]." (And hyperlink the two references?)
LC-90 SpecGuide 2003-03-31 SpecGuide Editorial Resolved Jonathan Marsh Lynne Rosenthal
Title: Restructure DoV
Description:

Specification reference.

comment about "Introduction" : If DoV is important enough to occupy a full page in 1.8, and be referred to without further explanation elsewhere, it's important enough to restructure so as to identify clearly the membership of DoV. Wherever dimensions of variability are mentioned in the document, it should be possible to hyperlink to the appropriate place (which means a briefer discussion, as an introduction to the collection of guidelines 2-9).

Email comments (20030417), including proposals.

Email comments, including proposal (20030418).

Discussed and resolved at 20030421 telecon.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: [Originator] Drop section 1.8, and restructure. [LH] See email.
Resolution: This will be addressed and resolved by the planned new chapter -- "2. Concepts".
LC-91 SpecGuide 2003-03-31 SpecGuide Closed Active Jonathan Marsh Lynne Rosenthal
Title: Complexity of numbering
Description:

Specification reference.

comment about "Overall" : Guidelines escape the section/technical numbering of the remainder of the document. Why isn't guideline 1 section 2.1? And checkpoint 1.2 section 2.1.2? This is going to cause confusion, when others make reference to the document: if I say 3.2, do I mean a checkpoint or a section? Section 2 is by far the bulk of the document; perhaps each guideline ought to be given a section number instead (guidelines 2 through 15, then, instead of 1 through 14).

Discussed at 20030407 telecon. Closed with resolution to leave numbering scheme as is. Reasons: given that the context is usually clear (or easily made clear by the usual prefix of "guideline" or "checkpoint" [x.y]), the current GL/CP numbers are considered both more convenient and sufficient; the numbering scheme is familiar from WAI specifications and has been successfully used there; and, the proposed numbering scheme would have the undersireable characteristic that every guideline would start with "2." (e.g., 2.8) and every checkpoint would start with "2." (e.g., 2.8.4). The QAWG believes that the GL number should be the most significant component, and believes it is easier to quickly recognize "checkpoint 8.4" than "checkpoint 2.8.4".

Related LC Section/GL/CP numbering issues: 63, 77.ET-3, 91.

Proposal: Find a non-conflicting numbering scheme.
Resolution: No change.
LC-92 SpecGuide 2003-03-31 SpecGuide Editorial Resolved Jonathan Marsh Lynne Rosenthal
Title: Examples vs. Illustrations
Description:

Specification reference.

comment about "Guideline 1 Define Scope." : What's the difference between examples and illustrations? (Checkpoint 1.2 versus checkpoint 1.4)

Discussed at 20030407 telecon: CP1.2 requires examples of what is in scope; CP 1.4 requires examples of specific functionality, concepts, behavior. Need to make this clearer, perhaps remove 'concepts' from CP 1.4 will help. Alternatives:

  1. Clarify the difference that 1.2 is broader examples, whereas 1.4 is very specific to a function or behavior.
  2. Remove CP 1.4
  3. Remove CP 1.2

Proposed resolution: re-word 1.2 as "Illustrate scope.", reword 1.4 as "Illustrate technical details", and clarify difference further in the ConfReqs, Rationale, and Discussion. Also, examples (use cases) need not be in the specification itself, they can be in companion document(s). Language should change to use "provide" rather than "include", to avoid implication of "within specification".

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal: Re-word 1.2 as "Illustrate scope.", reword 1.4 as "Illustrate technical details", and clarify difference further in the ConfReqs, Rationale, and Discussion.
Resolution: As proposed.
LC-93 SpecGuide 2003-03-31 SpecGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Classes vs. categories
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " : The use of categories versus classes of products is altogether unclear. If categories and classes of products are to be called out, normatively, then these should have status in the TOC (the list of categories and the list of classes should both have ids, be targets for hyperlinks, and should have subheadings to identify them visually).

Email comments (20030417), including proposals.

Discussed at 20030418 telecon.

Discussed and resolved at 20030418 telecon. Rewrite the GL verbiage for clarification. Implementation of the TOC suggestion depends on whether a sub-section on 'class' and 'category' is set up in a new "Ch.2 Concepts".

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: The verbiage on classes and categories will be rewritten according to the 4-point proposal in email comments. Implementation of the TOC suggestion depends on whether a sub-section on 'class' and 'category' is set up in a new "Ch.2 Concepts". [LR tentatively has drafting action.]
LC-94 SpecGuide 2003-03-31 SpecGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Loophole in Classes and Categories
Description:

Specification reference.

comment about "Guideline 2 Identify what needs to conform and how. " : The checkpoints based on classes of product and categories are awkward, because the use of the existing enumeration is REQUIRED but only if applicable. That is, the use of the existing enumerations is COMPLETELY OPTIONAL. The language should reflect that it leaves an enormous loophole for spec authors to ignore the existing lists.

Email comments (20030417), including proposals.

Discussed at 20030418 telecon.

Related LC cat-class issues: 11, 46, 48, 61, 73.3, 93, 94.

Proposal: See email.
Resolution: Resolved per proposal in email comments. We can't possibly write SpecGL in such a way as to prevent devious and intentional circumvention. Rewrite lists and verbiage to ensure that our GL2 implementation of the "non-exhaustive" idea is clearly explained -- these represent an enumeration of some of the most common SC and CoP, and you may need to define your own if one of them doesn't appropriately fit your SC or CoP. [LR tentatively has drafting action, as part of the drafting solution to issues 11, 46, 48, 93.]
LC-95 SpecGuide 2003-03-31 SpecGuide Substantive Active Jonathan Marsh Lynne Rosenthal
Title: Why is conformance policy a DoV?
Description:

Specification reference.

comment about "Guideline 3 Specify conformance policy. " : Why should conformance policy be considered a dimension of variability? It is potentially partitioned by those dimensions, but does not introduce its own dimensions.

Email comments, including proposal (20030418).

Discussed at 20030421 telecon, and in email thread.

Discussed at 20030428 telecon, and further analyzed in another email thread. Resolution postponed -- there are related issues in the "re-org" group, for consolidating various amongst GL3/10, GL11/12, etc.

Related previous QAWG issue 95 (which appears more "Postponed" than "Closed"), additional email proposal (20030429), email suggestion (20030429) of additional checkpoint.

Related LC DoV issues: 21, 66, 75.3, 75.5, 84, 90, 95, 77.SG-1.

Proposal: See email.
Resolution:
LC-96 SpecGuide 2003-03-31 SpecGuide Substantive Active Jonathan Marsh Lynne Rosenthal
Title: Priorities confusing
Description:

Specification reference.

comment about "Guideline 3 Specify conformance policy. " : The priorities here are very strange. Why is justifying the use of a dimension priority one, while establishing a minimum requirement is priority two?

Related LC GL3-policy issues: 42, 96.

Proposal:
Resolution:
LC-97 SpecGuide 2003-03-31 SpecGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Modules as extension points
Description:

Specification reference.

comment about "Guideline 5 Address the use of modules to divide the technology." : Unlike G4, which notes that profiles may be a point of extension, G5 does not consider modules to be a point of extension. In the web services world, "modules" certainly are a point of extension, and so have rules for defining new modules (just as, in G4, there are assertions associated with rules for defining new profiles). The document should recognize this.

Email comments (20030417), including proposal to seek clarification.

Discussed (for clarification) at 20030418 telecon, and in subsequent email thread (20030418).

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. During discussion, a key distinction between extension modules and from Rules for Profiles was established -- new profiles are not *extensions* to the specification, but rather different ways to subdivide the (typically) standard functionality of the specification for different conformance targets. Agreed that the problem should be resolved along the lines in LR email proposal -- a combination of 1.) clarification to Originator about how existing checkpoints actually cover this case; and, 2.) some additional clarifying verbiage (to be drafted, for either section 2.3, or the modules GL verbiage, or GL9/extensibility -- tbd). For further context, see also the solution for the profile/module/level issue group.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Proposal:
Resolution: A new CP is not thought to be needed. This case -- involving modules and extensions -- is somewhat different than the Rules for Profiles case (where extensions are typically not a factor.) QAWG thinks that CP9.4, CP9.7, CP5.2 cover this adequately, as described in email proposal. Some clarifying explanation will be added.
LC-98 SpecGuide 2003-03-31 SpecGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Conformance levels
Description:

Specification reference.

comment about "Guideline 6 Address the use of functional levels to divide the technology." : Levels are the primary defining mechanism for the QA framework, but there is only one checkpoint here. There should also be (at least) a checkpoint to establish that levels create a hierarchy of conformance; that the more advanced levels include the earlier levels (thus establishing that there is a minimum level).

There is email discussion of the issue group here, and see also numerous related messages nearby on the same list.

Discussed and resolved at 20030428 telecon. That "levels create a hierarchy of conformance" is part of the definition of levels. It was thought unnecessary and redundant to try to bind the definition into a Checkpoint and test requirements. For further context, see also the solution for the profile/module/level issue group.

Previous profile/module/level QAWG issues: 73, 74, 75, 76.

Related LC profile/module/level issues: 30, 41, 49, 50, 51, 97, 98.

Proposal:
Resolution: The suggested checkpoint is considered redundant and unnecessary, because it is part of the definition of "levels", and the terms in SpecGL "...have the meanings assigned in 'Definitions' and 'QA Glossary' [QA-GLOSSARY]."
LC-99 SpecGuide 2003-03-31 SpecGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Awkward deprecation requirements
Description:

Specification reference.

comment about "7.3 If deprecation is used, define its relationships..." : This checkpoint basically demands that if some feature has been deprecated, possibly because no one can agree on its proper definition, then it must be properly defined and its interactions fully characterized. It is likely that some things will be deprecated precisely because they cannot be well and unambiguously characterized; this checkpoint ensures that as long as these features are part of the main spec, it can conform at priority two, but as soon as the features are deprecated, it cannot.

See email thread.

Discussed at 20030410 telecon. The analysis of the email thread was basically agreed -- it is not necessary to fully define the deprecated functionality, in order to meet the intent of the checkpoint. That intent is that the impact of deprecating the feature on other DoV needs to be discussed. E.g., if the spec is modularized or profiled, how does a feature's deprecation impact those DoV. DM agree to draft clarifying discussion, including at least one "for example", to prevent the confusion.

Related LC deprecation issues: 33, 40, 99.

Proposal:
Resolution: The way Originator has understood the checkpoint is not how we intended it. We will redraft the verbiage (incl. "for examples") to clarify and prevent further confusion. [DM draft proposal.]
LC-100 SpecGuide 2003-03-31 SpecGuide Substantive Active Jonathan Marsh Lynne Rosenthal
Title: Don't discourage extensibility
Description:

Specification reference.

comment about "Guideline 9 Allow extensions or NOT!" : The position of the QA framework WG, that extensions should not be allowed, is quite clear. This is a political position, and doesn't accomodate those working on specifications that clearly demand public extensibility. In the guidelines, describe conformance. Discourage extensibility elsewhere. We note that these guidelines are, themselves, extensible.

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Subsequent email discussion, including proposed resolution.

Proposal:
Resolution:
LC-101 SpecGuide 2003-03-31 SpecGuide Substantive Active Jonathan Marsh Lynne Rosenthal
Title: Remove Checkpoint 9.6
Description:

Specification reference.

comment about "9.6 Require that implementations ... alternatives to extensions" : Extensions may be allowed in order to permit new functionality to be introduced and tested prior to standardization. There may not be any alternatives (interoperable or otherwise) to the use of a particular extension, and in particular, it is completely impossible for any specification that permits extensions to supply a workaround to the use of every uninvented extension imaginable. In other words, no specification that allows extensions can conform at priority three, ever.

Extension-group processing plan and subsequent email discussion.

Related LC extensibility issues: 15, 29.5, 35, 37, 38, 52, 73.5, 73.6, 75.4, 81, [97?, ] 100, 101.

Subsequent email discussion, including proposed resolution. Extension-group processing plan and subsequent email discussion.

Proposal: Remove this Checkpoint.
Resolution:
LC-102 SpecGuide 2003-03-31 SpecGuide Substantive Active Jonathan Marsh Lynne Rosenthal
Title: Consolidate G3 and G10.
Description:

Specification reference.

comment about "Guideline 10 Provide a conformance clause." : This is G3. Or it should be. Or G3 belongs in Ops, and G10 here. If a policy has been established, then it has been documented as well; the two are inextricable, when considering a written specification.

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal: Consolidate G3 and G10, or move G3 to Ops.
Resolution:
LC-103 SpecGuide 2003-03-31 SpecGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Distributed conformance section OK
Description:

Specification reference.

comment about "10.1 Include a conformance section." : The QA specification does not conform to this checkpoint, which is priority one. It must be acceptable to place the conformance requirements in each section of the document. For instance, it must be acceptable to place the conformance requirements for each checkpoint in a QA document inside the checkpoint.

Email comments (20030417), including proposals.

Discussed and resolved at 20030418 telecon.

Proposal: [Originator] Remove this checkpoint. [LH] See email.
Resolution: Originator's recommended "distributed conformance section OK", for the bulk of the conformance requirements, is indeed the intent, but it is obscured by confusing wording. GL3 sorts of things should be in Conformance section. Delete from the ConfReqs statement, "and specific conformance requirements". (Consider further clarification of intent during editing.)
LC-104 SpecGuide 2003-03-31 SpecGuide Substantive Active Jonathan Marsh Lynne Rosenthal
Title: Consolidate G11 with G3/G10
Description:

Specification reference.

comment about "Guideline 11 Specify how to make conformance claims." : This is also still part of G3/G10. This section should also discuss how one describes "module conformance" versus "profile conformance" versus "level conformance" versus "conforming extension", or how these might be combined. Perhaps this information is in the examples (I haven't looked).

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal: Consolidate G11 with G3/G10. Clarify various conformance terminiology.
Resolution:
LC-105 SpecGuide 2003-03-31 SpecGuide Substantive Active Jonathan Marsh Lynne Rosenthal
Title: G3, G10, G11, G13
Description:

Specification reference.

comment about "Guideline 13 Clearly identify conformance requirements." : The division of the various conformance-related guidelines into three, ten, eleven, and thirteen is not entirely clear, nor is it entirely clear that these separate sections do not contradict one another.

Email comments (20030417), including proposals.

Email discussion (20030418).

Related LC major-restructure issues: 54, 74, 75.2, 75.3, 75.7, 75.8, 102, 104, 105.

Proposal: [Originator] Rationalize these sections. [LH] See email.
Resolution:
LC-106 SpecGuide 2003-03-31 SpecGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Section 3.1 - Poor Defn of Normative
Description:

Specification reference.

comment about "Overall" : "All sentences using a capitalized keyword from RFC 2119" ... this is a totally strange definition of how to identify normative text. Since the definitions of terms are not normative, I am free to redefine them however I wish, and claim whatever conformance I care to. Note that the stated priorities for each checkpoint are not normative.

See email thread, with proposal: SpecGL's definition of normative is okay in glossary, but is more focused than originator assumes (so that it includes only direct conformance statements such as conformance requirements and test assertions). Per 20030410 telecon section 3.1 should be clarified and improved.

Discussed again and resolved at 20030414 telecon.

Related LC whats-normative issues: 36, 65, 106, 108.

Related LC TA-requirements issues: 13, 14, 29.4, 65, 67, 73.9, 75.9, [106?].

Proposal: [Originator] Develop a rational definition of Normative.
Resolution: QAWG confirms that our more focused definition of normative is the one we want in SpecGL. There are some problems in wording of section 3.1, and these will be improved, per 20030410 telecon and 20030414 telecon. The lists in 3.1 will be made comprehensive, and it will include the priorities. Regarding redefining terms with your own definitions, it will be clarified that terms, when used in SpecGL, have the meanings given in glossary.
LC-107 SpecGuide 2003-03-31 SpecGuide Substantive Closed Jonathan Marsh Lynne Rosenthal
Title: Section 3.4 - AAA-terminology useless
Description:

Specification reference.

comment about "Overall" : The introduction of A[A][A]-conforming as a synonym for the three priorities is rather absurd, as is the creation of four levels that will never be used. A-conforming = priority one = level three; AA-conforming = priority two = level five; AAA-conforming = priority three = level seven.

Email comments (20030417), including proposals.

Discussed and closed (as SpecGL issue) in email.

Email discussion , including proposed resolution (of OpsGL issue).

Discussed and resolved at 20030428 telecon. See LC-3 for description and resolution.

Related LC issues: 3, 60.2, 72.2, 72.3, 83, 107.

Proposal: [Originator] Pick one set of terms and toss the remainder. (N.B.: the association of priorities with levels is a consequence of Ops guideline 1). [LH] See email.
Resolution: This will be dealt with by fixing OpsGL, in the resolution of OpsGL issues 3, 60.2, 72.2, 72.3, 83.
LC-108 SpecGuide 2003-03-31 SpecGuide Substantive Resolved Jonathan Marsh Lynne Rosenthal
Title: Definitions
Description:

Specification reference.

comment about "Overall" : To avoid the problems introduced in section 3.1, change it to include, as normative, all of sections 1 and 4, which define terms. Or create a better definition.

See email thread.

Discussed and resolved at 20030414 telecon.

Related LC whats-normative issues: 36, 65, 106, 108.

Proposal: Disagree, our definition of normative is different, focused on direct connection to conformance requirements. See email.
Resolution: As proposed. See also issue resolution.
LC-109 SpecGuide 2003-04-07 SpecGuide Substantive Resolved Lofton Henderson Lynne Rosenthal
Title: Is CP1.3 needed?
Description:

Specification reference.

Originated during 20030407 telecon, during discussion of issue #92. Discussion: It is unclear what is the difference between CP 1.2 and CP 1.3, according to their respective stated rationales. Is CP 1.3 a superset of CP1.2? Can they be combined? Do we need both? If we need both, then what purposes are the "usage scenarios" of CP1.3 serving, that are distinct from the purposes of CP1.2?

Discussed at 20030410 telecon. Resolved that CP1.2 and 1.3 should be combined, and that use case and/or usage scenario should be emphasized as valuable technique in discussion (so that it is somewhat distinguished from "examples").

See previous QAWG (non-LC) issues #72, #84, and #86.

Related LC example/use-case issues: 7, 23, 75.1, 77.ET-2, 79, 92, 109.

Proposal:
Resolution: Combine CP1.2 and 1.3, and emphasize in verbiage that use case and/or usage scenario are valuable (recommended) techniques. [Tbd...LR to draft.]
LC-110 OpsGuide 2003-04-16 OpsGuide Substantive Active Dominique Hazael-Massieux Lynne Rosenthal
Title: Collected OpsGL comments from Team.
Description:

Specification reference.

comment about "Overall" :

[email]

Ed Note -- DH and KD organized a project review of the QA OpsGL for the W3C Team. This is an informal summary of the comments they got. Note that the presentation was about OpsGL, but most of the comments apply equally to SpecGL.

  1. QA is very important, and the QA WG has the right goals. [No issue.]
  2. There were a lot of discussions regarding the writing style we adopted for the framework, namely the fact that we use RFC keywords in conformance requirements. Some people thought it was too "aggressive", other felt it was the right thing to do.
  3. [OpsGL only] The table in OpsGL GL 1 caused much confusion and was deemed as not-understandable, which I think we already more or less agree with.
  4. The intents of the priorities/degrees is not always clear. Proposal [DH]: we should probably emphasize somewhere that the minimal recommended degree is to be AA conformant (or that is the intention of the WG to request it to be the lowest level for work in W3C)
  5. Generally speaking, the distinction between GL and ExTech was not always clear. Proposal [DH]: we probably need to rework the introductory sections to clarify that.
  6. The summarized view (ICS/Checklists) are not easy enough to find, and are not explicitly recommended enough. Proposal [DH]: again, that means some re-working of the introduction.
  7. The introduction needs to be much more efficient to read. Proposal [DH?]: some kind of an executive summary rather than the long prose we currently have.
Proposal:
Resolution:

Table Legend

num
Last Call issue number
Title
Short title/name of the issue
Spec
Document referred to in Last Call issue (Intro = QA Framework: Introduction; OpsGuide & OpsGL = QA Framework: Operational Guidelines; SpecGuide & SpecGL = Framework: Specification Guidelines)
Description
Short description of issue, possibly including link to origin of issue
Date
The date at which the issue was raised or initially logged.
Topic
Rough topic categorization, one of: ...tbd... [replace following XMLP stuff... env(elope), rpc, enc(oding), meta(issue), bind(ing), fault]
Class
Substantive or Editorial
Status
One of: Unassigned, Active, Closed, Postponed
Proposal
Current proposal for resolution of issue, possibly including link to further text
Resolution
Short description of resolution, possibly including link to a more elaborate description
Raised by
Person who raised the issue
Owner
QA WG Member responsible for the issue

Maintained by Lofton Henderson.