This document records the QA Working Group's response to comments on the November 2004 Last Call Working Draft of QA Framework: Specification Guidelines.
The full statement of each issue and its full processing history can be found in QAWG's second Last Call Issues List. The issue number in the first column of the table links to that. The Resolution column has links to specific text in the April 2005 Working Draft, whose purpose is to illustrate the Disposition.
The "Status" column indicates QAWG's disposition of the issue:
accepted: originator's suggestion (if any) was accepted substantially as proposed.
accepted with modifications: QAWG accepts that originator's issue is valid, but we implemented something else than what (if anything) was suggested - originators, please check that it does solve your issue.
not accepted: originator's comments and/or suggestions were not accepted — originators, please check that you accept QAWG's explanation and rationale.
The deadline for replies to this disposition of comments was May 18, 2005. When we did not hear from the originator by the deadline, the default reply is accept QA Working Group's disposition of comments
..
Last update: $Date: 2005/06/29 14:32:05 $
Issue # | Originator | Description | Resolution | Status | Originator's reply |
---|---|---|---|---|---|
985 What's a core module | Dave Marston | David Marston makes a difference between usual modules and core module. Karl Dubost thinks it's just a specialization or naming not a functionnal difference. Should it be a topic for Spec GL or Variability in Specifications? |
As a response to your comment, the QA Working Group has accepted your comment. It's not necessary a topic which belongs to QA Specification Guidelines and then it has been decided that the topic could be developed if necessary in Variability in Specification. | accepted | accepted |
986 Conformance section for a technology or for a specification (005) | Dave Marston | Description From monolithic specification, WG are producing more and more multi-documents specifications. Each document being having its own life with dependencies with others. Though the division in many documents is not necessary a functionnal division (module, profile, level), but it could be a topic division. Let's imagine for example that the conformance section of a technology is one document. How to make clear where and when the conformance section or sections should go? Comment David Marston: "when a document is separate from another with which it was originally thought to be associated, it takes on a life of its own. " Proposal David Marston: 3. Each Recommendation addresses conformance. Those that don't specify behavior of a Class of Product may simply say that they are informative, but beware: even defining terms or stating principles can be normative if some other document could cite the terms or principles normatively. If your WG issues several Recommendations and some refer normatively to others in the set, try to isolate a Class of Product in each Rec and anticipate that other Recs may cite any individual Rec normatively. Resolution Solved with the notion of Umbrella Specification. |
As a response to your comment, the QA Working Group has not accepted your comment. An additional decision of the QA WG has removed the "Umbrella Specification" topic from Specification Guidelines to move it to Variability in Specifications. One technique of Requirement 01 addresses the notion of Conformance when the technology is divided in multiple documents. | not accepted | accepted |
984 B3 list of normative and non-normative references | Bjoern Hoehrmann | ISSUE: Bjorn Hohrmann has raised an important issue about the way normative references are given and their implications on the normativity of the specification itself. An short abstract of the issue can be foreseen as: When making a normative reference, you need to see how future versions of the said specification may affect your own document Address the way you "use" the conformance model of the referred specification The QA WG is still discussing the issue. You are welcome to participate on the QA IG Mailing list (www- qa@w3.org publicly archived), but read first this two mail threads: B3 list of normative and non-normative references Bjorn's Issue |
RESOLVED: The WG has added a good practice on normative references, insisting on the aspects that need consideration when making such a decision. |
accepted | (considered accepted unless originator responds otherwise) |
1148 Text in example of section 2.3 needs fixing | TAG/Chris Lilley | In section 2.3, the text "Using the concept of URI-reference as a normative reference and then in the prose of the specification, define that it is an error to use it." ... is ungrammatical and sort of tails off. Please reword. The third and fifth example seem primed for a discussion that does not happen. |
As a response to your comment, the QA Working Group has accepted your comment and has decided to drop the specific example which was indeed not clear without more context. | accepted | accepted |
1149 Suggestions of examples for section 2.3 | TAG/Chris Lilley | In section 2.3, the TAG would like to see a reference to 'referencing Unicode' (a pointer to charmod would suffice) and 'referencing XML (1.0, 1.1, or infoset). These are tow useful, common examples that it would be beneficial to discuss. |
As a response to your comment, the QA Working Group has accepted your comment and the following examples have been added to SpecGL:
|
accepted | accepted |
1151 Positive statements for non applicability of depreaction | TAG/Chris Lilley | The specification requires "Identify deprecated features." Even in version 1 of a specification? (Yes, it can happen that version 1 formalizes industry practice, so defines and immediately deprecates something). Conversely, is it required to state explicitly "there are no deprecated features" to meet this requirement? Otherwise, its hard to distinguish between a pass (there are no deprecated features listed) and a fail (deprecated features are there, but were not identified as such). We suggest that a bullet "It has no deprecated features" be added to the bulleted list under "Conformance criteria". |
The QA Working Group agrees that for deprecated, obsolete, optional features and subdivisions, it was not clear how a specification could match the requirements if none of these were present in the specification. To clarify this, the Good Practices and Requirements have been amended. For instance, the Identify deprecated features now reads in What does it mean?:
Similarly for subdivisions, optional features and obsolete features |
accepted | accepted |
1143 positive statements for non applicability | TAG/Chris Lilley | Several places where a specification might have some option (deprecated, extended, subsetted, profiled, etc) there is no requirement to make a positive statement that the spec has no (deprecated elements, extensions, subsets, profiles, etc). It is therefore hard to tell specs that conform (because they don't have these things) from ones that do not conform (because they have these things and don't clearly identify them). The QA Specification Guidelines is an example of a spec that does not clearly state that it does not have several items in this category. Thus, its not easy to tell if it conforms to itself or not. |
|||
1144 specification and workflow mixed up | TAG/Chris Lilley | There seem to be two components being mixed up in this specification. One is a specification produced by a working group, the other is the workflow of a working group. Its not clear that these are fully distinguished, or should be evaluated on the same proforma. |
The QA Working Group has accepted the comment and decided to create a new section which is not mandatory : Beyond Conformance. This section contains previous Good Practices which were workflow related and that a WG might want to explore during the specification editing. Some of the good practices have been rewritten as well to remove their workflow orientation when it was not necessary. See also the correspondence table |
accepted | accepted |
1145 Conformance clause optionality is not reflected in proforma conformance claim | TAG/Chris Lilley | The specification does in fact have a large optional part. "the conformance clause may be an explanation of why there is no "conformance to this document" and may be presented in another section rather than in a separate conformance section." This large optional part (either make this claim that there is no conformance, OR meet all of the Requirements) is not reflected in the proforma. It would probably be better to provide language for a claim that specifically addresses this case. For example
|
The QA Working Group has accepted the comment and has used the example provided:
|
accepted | accepted |
1146 Table of Contents should be more detailed | TAG/Chris Lilley | Please put third-level items in the TOC (because these are the good practices and requirements). Alternatively, make a separate table of good practices and requirements. |
the QA Working Group has accepted the comment and has created a detailed Table of contents giving access to the Requirements and Good Practices. | accepted | |
1147 Unclear legend for illustration in section 4.1 | TAG/Chris Lilley | In section 4.1, the text: "Profile X defines an implementation of Level 2 (then Level 1 too) for a specific class of products for example." is unclear. Does it define a separate implementation of level 1? Does it say that all Level 1 content conforms with profile X? Perhaps "(and thus, implicitly, Level 1 also)" would be better. Its not clear which of several possible points the parenthetical clause is seeking to make. Please clarify it. |
the QA Working Group has accepted the
comment and has accepted the rewording for the legend of the
illustration: |
accepted | accepted |
1150 Extensions and conformance interferences are based on too limited assumptions | TAG/Chris Lilley | The text "Warn implementers to create extensions that do not interfere with conformance." is unclear. It would perhaps be better worded as "Warn implementers not to create extensions that interfere with conformance." There are also some assumptions here about who is doing the extending. Its assumed to always be "implementors". There are several possible classes of extender - implementors, the original spec writers, writers of another spec that uses the first spec, users ..... it is perfectly possible to define an extension that does affect conformance, and indeed this case is discussed elsewhere in the QA Specification Guidelines document. This case is where the extension (spec version n+1) removes something that existed in spec version n (perhaps spec version n had deprecated it already). Thus, content conforming to spec version n and using deprecated features is no longer conformant to spec version n+1 and this is expected and encouraged. |
The QA Working Group agrees that the wording was too limitative, and has now reworded Good Practice 19 as follows:
and complemented by the following text in the "what does it mean":
|
accepted | (considered accepted unless originator responds otherwise) |
1152 Wording of "define ... deprecated feature..." | TAG/Chris Lilley | The text: "Define how deprecated feature is handled by each class of product." should perhaps be "Define how each deprecated feature is handled by each class of product." |
The text now reads: Define how each each class of product handles each depreacted feature. |
accepted | yes |
1153 Wording of "use formal languages..." | TAG/Chris Lilley | "Use formal languages and define which from prose and formal languages has priority." should that be "which of "? |
The quoted text has been reworded and doesn't include the reported error any more. | accepted | |
1154 Error processing for specs that are no language nor protocols | TAG/Chris Lilley | The spec says "For a language, address what effect an error (be it syntactic or semantic) in the input has to a processor of this language. For a protocol, address how should behave a party to this protocol when a bogus message is received." For things that are not languages and not protocols, what is required? |
The QA Working Group agrees the former text was narrowing the scope of the good practice without good reason and made the new text broader, while illustrating it with the former text. Good practice 23 now reads:
|
accepted | yes |
1155 Use your own example explicitly | TAG/Chris Lilley | Noticing the text: "One Working Group's document entered Candidate Recommendation prior to a through quality review, resulting in a huge number of issues to resolve and the eventual retreat back to Working Draft for major revisions. " Aww c'mon, mention that you are talking about the CR version of SpecGL about (the current specification is a huge improvement, by the way). |
The QA Working Group agrees that being more specific about the story can only helps understand the point made and is now making an explicit reference to itself in the text:
(while the story doesn't exactly match the reality of what happened, the QA Working Group believes it can still be used as a fair illustration of the point made) |
accepted | accepted |
1156 Wording in tips for getting the work done | TAG/Chris Lilley | "For the edition of each Good Practice" should be "For editing each Good Practice". (Edition is not a verb in English). |
The QA Working Group has accepted the comment and has reworded as follows:
|
accepted | accepted |
1157 ICS needs more than y/n/n-a | TAG/Chris Lilley | Comments on SpecGL ICS 1) The Implementation Conformance Statement uses URIs of the form ./#foo which will only work correctly if the implementation conformance statement for each specification lives under the TR directory. For a template, these should be converted to absolute links. 2) "The table includes spaces for scoring each checkpoint, "yes" (satisfied), "no" (not satisfied), "n/a" (not applicable)." It should be encouraged to link to the part of the spec that demonstrates each claim of conformance, as well as just marking yes or no or n/a. |
The QA Working Group has accepted the comment and has added verbiage to the new version of the ICS with:
|
accepted | accepted |
1158 SpecGL conformance to itself | TAG/Chris Lilley | The TAG has filled in the conformance proforma with respect to the spec the QA Framework specification guidelines itself, to determine whether the specification conforms to itself. This mainly highlights the points made in other comments. Please ensure that the specification fully conforms to itself. |
The QA Working Group has completed a SpecGL ICS for SpecGL itself. We believe SpecGL now conforms to itself, since it implements all the Requirements; it also implements all but one Good Practice. Follow belows details answers to the comments made in the footnotes of the ICS the TAG filled for the previous version of SpecGL.
|
mostly accepted | yes |
1159 Warn against untested hooks | Dan Connolly | My experience is that untested hooks are bad; really bad. Please share this experience with your readers. This bit of SpecGL seems relevant, though not exactly the "untested hooks are bad" bumper-sticker I'd like to see: Make sure there is a need for the optional feature. 4.2 Good Practice A: in 4. Managing Variability of SpecGL The 1999 RDF spec had a couple hooks that were not well tested: [[ When an RDF processor encounters an XML element or attribute name that is declared to be from a namespace whose name begins with the string "http://www.w3.org/TR/REC-rdf-syntax" and the processor does not recognize the semantics of that name then the processor is required to skip (i.e., generate no tuples for) the entire XML element, including its content, whose name is unrecognized or that has an attribute whose name is unrecognized. ]] The WG observed that implementations were all over the map, so we got rid of that one. This one was sorta contradictory: [[ Other values of parseType are reserved for future specification by RDF. With RDF 1.0 other values must be treated as identical to 'Literal'. ]] I think the WG observed that implementations follow the latter sentence and standardized it, though I can't find the decision. There's a relevant test, which seems to be associated with this issue... but I don't see a pointer in the other direction. I know we decided that using this as a hook for parseType="Quote" is not allowed: I think SMIL has some system-foo hooks that are more well tested. There are plenty of other well-tested, well-used hooks in the web: HTTP mime types, XSLT extensions, CSS fallback rules (though those were, at one point, insufficiently tested) etc. The issue of untested hooks came up recently in a WG meeting I chaired; in editing the minutes, I was curious if SpecGL had an "untested hooks are bad" bumper sticker; almost... [[ SPARQL Protocol Spec We noted SPARQL Protocol for RDF W3C Working Draft 14 January 2005 ... Regarding the Abstract protocol, DanC noted that it comes with a testing obligation; if we're only going to test one concrete protocol, we should fold the essential material from the abstract protocol into it. He later clarified that it would be OK if a second concrete protocol (SOAP, SMTP, ...) were not normatively specified, but only specified in a Note or even only in the test materials; as long as the abstract protocol "hook" is tested, we've met our obligations. postscript: this bit of SpecGL seems relevant, though not exactly the "untested hooks are bad" bumper-sticker I'd like to see: Make sure there is a need for the optional feature. 4.2 Good Practice A: in 4. Managing Variability of SpecGL ]] -- |
As a response to the proposal, the QA Working Group decided to include in its Good Practice regarding extensibility mechanisms a technique on this topic:
|
accepted | (considered accepted unless originator responds otherwise) |
1057 "Specifying conformance" wording | Gary Feldman | The first paragraph is worded awkwardly, and really needs to be clearer. If for no other reason, this should be done to set a good example, since clarity is one of the points of the paragraph. The subject of the first sentence has three prepositional phrases, which is two too many for a subject. Perhaps "A clear presentation of conformance is crucial to the ...." The reader should surely know at this point that conformance, in this document, is shorthand for conformance to specification. If not, then perhaps the second paragraph defining conformance should be moved to go first. The second sentence, while not technically run-on, is too verbose. "The conformance model, its description" is redundant in this context. In a sentence this long, the examples need to be split off into a second sentence. Here's a possible rewording: The conformance model and the language used for normative information determine the testability of a specification. They also influence the overall implementation landscape, ranging from a narrow conformance with few allowable variations in implementations to multiple conformance types, resulting in numerous conforming implementations. The model must be chosen carefully, to produce the intended implementation range. In the second paragraph, third sentence, I think "states all the criteria" should be replaced with "identifies all the criteria." A reasonable reader should not misinterpret this as implying that nothing outside the clause is necessary for conformance. Nevertheless, even reasonable readers may stumble over this, to figure out what it really intends. Even with the suggested change, I don't think this fully conveys the idea, but I'm also not sure that's needed. |
QA Working Group has accepted the comment and reworded the affected sections as recommended and they now read:
|
accepted | (considered accepted unless originator responds otherwise) |
1058 Structure and numbering confusing | Gary Feldman | I find the inconsistent section numbering to be both surprising at this stage in the process, and confusing. I'm referring to the numbers used for the subsections of the Guidelines section, but no other sections or subsections are numbered. The net effect for me is to make it seem that only the numbered sections are part of the document, and the surrounding text is annotation (just like the status section at the beginning). It also makes it look that not only was this designed by committee (not bad in itself), but that it has not yet been edited into a cohesive piece. |
SpecGL has been re-organized so that now every section is numbered, with a consistant style across the document. |
accepted | (considered accepted unless originator responds otherwise) |
1059 Parsing the definition of specification | Gary Feldman | I found it difficult to parse and understand the definition of specification. It says "...interface to accomplish a task." But interfaces are static, passive entities. They don't accomplish anything. Perhaps "for accomplishing a task" would read better, but even that feels uncomfortable. What task does XML accomplish? (It is used for many more things besides machine-independent data interchange.) I'm not sure that this phrase is necessary at all. Saying that "A specification is a set of technical requirements which define a reliable interface." seems adequate. If really necessary to qualify "interface", then "interface between actors" is what I'd suggest. On the same sentence, though less important, the phrase "which aim at ..." is one of those unnecessary phrases (like "in order to") that are best removed. |
The QA Working Group has accepted the comment. Rather than attempting to create our own definition of "specification" it has decided to use the definition provided by ISO Guide 2-4:
|
accepted | (considered accepted unless originator responds otherwise) |
1060 Umbrella specification concept defined but not used | Gary Feldman | Is the entire discussion of umbrella specifications really necessary? The word "umbrella" isn't used anywhere else in the document, and I'm not sure that the concept is mentioned elsewhere. I think it's enough to have just the first two sentences "Specifications can be defined in one or several ....in a well defined manner," but delete the clause "denoted below as umbrella specifications" along with the figure and the paragraph explaining the figure. The idea of a composite document isn't rocket science, and doesn't deserve this much space - at least not unless it played a more prominent role later in the document. |
The QA Working Group has accepted the comment. The notion of umbrella spec has been removed from SpecGL, moved to the Variability in Specifications document, and alluded to in the Scope and Goals section of the Specification Guidelines. The relevant section now reads:
|
accepted | (considered accepted unless originator responds otherwise) |
1061 Non-specification specifications (in About this document) | Gary Feldman | The "About this document" section, first paragraph, has a sentence "Note that for some specifications ... separate conformance section." The items listed, QA Handbook and Architecture of the World Wide Web simply aren't specifications. The earlier definition of specification said, correctly, that a specification is a set of technical requirements. If there are no requirements, then it's not a specification. The entire sentence can be deleted. If it's necessary to comment on such documents, then it should be something like "Some of the documents produced by the W3C process aren't specifications (in the sense used here), and hence this document does not apply" (or they need no conformance clause). I don't agree with a blanket statement that documents for which conformance is not an issue should have a conformance clause that explains why it doesn't need a conformance clause. Not only is there the obvious circular contradiction, but the only justification for such a statement would be if there might be confusion about it. For example, since the QA Handbook (for example) begins by saying that it's non-normative, there's no need for that document to belabor the point. Perhaps it might say "Documents for which conformance is not an issue may choose to include a statement explaining the lack of a conformance clause, if there might be confusion around that point." though personally, I think even that's saying too much. Minor error: the previous sentence, "A conformance clause template ..." has an error at "...to assist editors satisfy requirements...." The second "to," between "editors" and "satisfy" is missing. |
The QA Working Group agreed that the terminology was imprecise, and substituted the term "technical reports" to describe all the documents that go through the W3C process, normative or not. |
accepted | (considered accepted unless originator responds otherwise) |
1142 Do not require conformance clauses for non-normative technical reports | Gary Feldman | (Derived from originally issue 1061) I don't agree with a blanket statement that documents for which conformance is not an issue should have a conformance clause that explains why it doesn't need a conformance clause. Not only is there the obvious circular contradiction, but the only justification for such a statement would be if there might be confusion about it. For example, since the QA Handbook (for example) begins by saying that it's non-normative, there's no need for that document to belabor the point. Perhaps it might say "Documents for which conformance is not an issue may choose to include a statement explaining the lack of a conformance clause, if there might be confusion around that point." though personally, I think even that's saying too much. |
The QA WG disagrees with the assessment that such an explanation is often unnecessary. We believe that based on past experience there is often a significant amount of confusion about whether or not a technical report is normative and consequently whether or not one could conform to it. We therefore feel that the simplest approach is always to require an explanation of whether or not conformance is an issue. We did agree with you that it is not necessary to require an actual conformance section in such documents. Our revised text reads:
|
disagreed | (considered accepted unless originator responds otherwise) |
1040 Structure/Numbering confusing | Ian Hickson | Sent by Ian Hickson: Some of the sections are empty. For example, take this markup: <h4 id="what-conform">2.2 What needs to conform</h4> <div id="implement"> <div class="principle"> <h4 id="implement-principle"><span class="principle-label">2.2 Requirement A:</span> Identify who or what will implement the specification.</h4> </div> ... It is semantically equivalent to (removing the semantically-neutral <div>s, <span>s, and id=""s for clarity): <h4>2.2 What needs to conform</h4> <h4>2.2 Requirement A: Identify who or what will implement the specification.</h4> ... ...which means there are two sections labelled 2.2, with the first one being empty of content. There are also examples where you have two sections with the same number, for example there are two sections numbered 2.3. Maybe the second set each time was supposed to be an <h5> instead of an <h4>? |
The QA Working Group has accepted the comment and has restructured and renumbered the document by improving the markup as well as changing the numbering system for requirements and good practices. This new structure and numbering system should make the document clearer and easier to read and understand. | accepted | accepted |
1041 Conformance is not a yes/no proposition (wrt filling an ICS) | Ian Hickson | "1.2 Good Practice B" suggests that an ICS form be provided with yes/no questions: "1. Create a list, table or form listing all features (capabilities) and indicating if it is mandatory or not. 2. Provide space for the implementer to check: Yes, No, Not Applicable". However, this is unrealistic. For example, take CSS user agents. How is an implementor to determine if he has implemented margin collapsing correctly? All that can really be said is that the user agent passes a certain set of tests. For any even mildly complicated specification it will always be possible to show that a user agent is in some way non-compliant, it's just a matter of finding a suitable test. Therefore I would suggest changing this section to instead suggest leaving space for the implementor to list the URIs to (publically available) tests that the implementor has used to verify interoperability and compliance, listing which tests the implementor determined passed in the user agent and which failed (if any). Specification authors may wish to provide a list of URIs to the tests that form part of the specification's formal test suite (as used to check for interoperability as per the CR exit criteria), although naturally such a test suite can never be complete enough to really be used to claim conformance so implementors would be expected to also provide links to other tests that they used. (The existing suggestions could be kept for the rare specs in which a test suite is inappropriate, such as the two examples the spec currently gives: the QA spec guidelines and the WCAG. However, this applies to very few specifications and so should not IMHO be the primary suggestion in the document.) |
The QA Working Group has accepted the comment and reworded the affected sections to clarify the meaning and use of the Implementation Conformance Statement (ICS), including explaining the meaning of checking Yes/No. The affected sections now read:
Changes were also made to related Good Practice 05
|
accepted | accepted |
1042 Scope also helps reviewers determine when the specification is over-stepping its mandate | Ian Hickson | "2.1 Requirement A: Define the scope" -- another good reason to define the scope is it helps reviewers determine when the specification is over-stepping its mandate. Sometimes a specification veers away from its original goals and scope; on occasion this is due to evolving goals and thus the scope needs to be updated, but sometimes it is because the working group has strayed outside the boundaries of what the specification should be addressing, and the spec itself needs revising instead. |
The QA Working Group has accepted the comment and has reworded the affected section as recommended and it now reads
|
accepted | accepted |
1043 Comment about HTML spec examples needs refinement | Ian Hickson | "2.1 Good Practice B: Provide examples, use cases, and graphics. [...] HTML 4.01 [HTML401]: The HTML 4.01 specification has been designed in a very educative way. By its design, the specification has very good examples." -- actually, there are several examples in the HTML4 specification that are non-conformant HTML in various ways, for example showing very poor use of the "alt" attribute. |
The QA Working Group has accepted the comment and has included the example you recommended and it now reads
|
accepted | unsatisfied, but not objecting |
1044 Case of RFC2119 terms | Ian Hickson | "3.2 Requirement A: Use a consistent style for conformance requirements and explain how to distinguish them." mentions that RFC2119 terms are uppercase, but it should be noted that nothing in RFC2119 (other than consistent usage as such) requires them to be used in uppercase, despite specifications frequently explicitly mentioning that they use lowercase variants instead. |
The QA Working Group has accepted the comment and has reworded the affected section as recommended and it now reads:
|
accepted | accepted |
1045 Avoiding device-dependent profiles | Ian Hickson | "4.1 Good Practice A: Create subdivisions of the technology when warranted" -- One additional "technique" I would recommend here is to avoid profiling device-independent technologies in device-dependent ways. For example, having mobile profiles for markup languages. Such profiles will lead to a fragmented Web, as content written for the full profile will only work on full UAs, and content for smaller profiles will be written with the assumption that they will only be used from small devices. Instead, I would recommend having either just a lowest common denominator, implementable on all devices, or having optional features with well-defined fallback behaviour, so that they can be used in all content with the technology usably interoperable even in implementations that implement only the required parts. For example, CSS requires all UAs to implement full RGB color, but on black-and-white devices the specified colours are then down-sampled to fit the available hardware. |
The QA Working Group has accepted your comment in principle but believe that the current wording discourages variability and that it isn't appropriate to get into technical details as to what is good or bad variability, but rather should be carefully evaluated by each Working Groups (as expressed by when warranted).
|
no change brought to the text | accepted |
1046 Correction to example about CSS extensibility | Ian Hickson | In 4.2, there is an example that reads "CSS3 module: Syntax [CSS3-SYNTAX] defines an extension mechanism which uses a not valid start character for identifiers in CSS, so it is guaranteed never to be used by any current or future level of CSS. CSS-conforming parsers will skip rules that contain identifiers with such a character. Therefore it is not authorized to redefine the properties defined in the specification." -- this has been changed in CSS2.1. The properties are not valid, but the working group guarentees to never start a standard identifier with a hyphen, which has the same result. |
The QA Working Group has accepted the comment and has reworded the affected section as recommended and it now reads
|
accepted | accepted |
1047 Additions to error mechanism section | Ian Hickson | "4.5 Good Practice A: Define an error handling mechanism" -- another useful technique that could be mentioned here is designing the spec so that there are few ways to get into an error case (for example, defining the meaning of every combination of language construct). Also, this section should spend more time on the concepts of mustIgnore and mustUnderstand (and why the camelCase?). Both have their places: mustUnderstand is important when data corruption could end up having critical consequences (e.g. a corrupted credit card transaction could be costly), whereas mustIgnore is important when the worst effect data corruption could have is a slightly degraded rendering (e.g. a corrupted stylesheet). |
The QA Working Group has accepted the comment and has reworded the affected section as recommended and it now reads:
|
accepted | accepted |
1048 Additions to "write tests" | Ian Hickson | "5 Good Practice C: Write sample code or tests" -- another technique is to go back and create new tests for old sections once the sections are better understood and more mature. It also helps to write tests that check the interactions of different sections. Also, especially when using test assertions, the temptation is to have one (or more) tests per assertion. However, one must absolutely check the _interactions_ of assertions, as that is where most bugs are likely to be found. For example, the two assertions "when X happens, A must happen" and "when Y happens, B must happen" can be tested individually, but they must also be checked together, to ensure that if X and Y both happen, first A happens then B happens, etc. |
The QA Working Group has accepted the comment with modification. The comment is a good one, but a bit out of scope since it is about how to write good tests. However, we have reworded the affected section to clarify and it now reads:
|
accepted with modification | accepted |
1049 Formal vs prose language normativity | Ian Hickson | "5 Good Practice E [...] make it clear when the English prose and the formal language overlaps which one should be held for true in case of discrepancy" -- I would recommend making them _both_ normative and not have one override the other. If prose and formal language are inconsistent, then an error has crept into the specification, and there is no guarentee that the error will be in the one that has been marked non-normative. Whenever there is an inconsistency, the working group should IMHO issue a (normative) errata to address the problem. |
The QA Working Group has accepted the comment and has reworded the affected section as recommended and it now reads:
|
accepted, with modifications | can live with it |
1050 Modesty requirement | Ian Hickson | Finally I would add one other good practice: specifications should not claim to be simple, easy, device-independent, conformant to WAAA or QAG, or make other claims about their quality or conformance to other specifications. While it is fine to indicate that one of the requirements of the specification may have been to be easy / device- independent / whatever, it should IMHO be up to the reader to make the determination of whether the working group was successful or not. (This is the last of the eleven comment mails I had on the QASG document. I hope they were of use.) |
The QA Working Group disagrees with the comment. The Quality Assurance Working Group believes that such a good practice would be inappropriate for the following reasons:
|
disagreed | accepted |
1051 "Identify depreacted features" too strong | Jeremy Carroll | Identify deprecated features. is slightly too strong, since the informative text below starts " When a technology evolves,". What to do with a first version of a technology is usually "N/A", and admittedly the simple step of "This is a first version, so we have no deprecated featues, and don't need to document it at all" does fulfil this requirement. Although, for example OWL, in its first version, does provide: "Changes from DAML+OIL" which does identify features deprecated from the member submission DAML+OIL from which OWL evolved. I think this comment could be addressed by a sentence early in the section, maybe the very first sentence, like "Not normally applicable to first versions of a specification." |
The WG agreed with the comment, and will incorporate the proposed wording or a slight modification of it. |
accepted | (considered accepted unless originator responds otherwise) |
1052 "Classes of product" unclear and dangerous | XML Core/Paul Grosso | The XML Core WG reviewed: QA Framework: Specification Guidelines and has a concern about "classes of products." Specifically, the WG has a problem with: 2.2 Requirement A: Identify who or what will implement the specification. and 4.4 Requirement B: Define how deprecated feature is handled by each class of product. Our problem with the latter is just the "by each class of product" part which reduces to our issue with the former. We find 2.2 Requirement A to be unclear and potentially dangerous. It is not clear how to define a class of product, and it is not clear what the full set of classes of products might be. We don't believe there are (or could be) clear definitions of distinct classes, and we are concerned that any attempt to list classes affected by a spec might end up excluding some products for which the spec should apply. Rather, any product should be evaluated against the spec to determine if the spec applies to it. For example, it isn't clear if xml:id is applicable to an XHTML browser UA. It depends on whether the UA relies on a parser of other xml processor that has implemented xml:id (in which case xml:id doesn't apply directly to the UA) or whether the UA does its own id recognition. In summary, we object to 2.2 Requirement A being mandatory in these Specification Guidelines. We object to 4.4 Requirement B being mandatory as long as it includes the words "by each class of product", but we remove our objection if those words are removed. |
The Working Group thinks that clarifying the extent to which a specification is expected to list its classes of products - namely, for those for which it defines conformance requirements - should address the XML Core Working Group concerns, while not actually implementing their requested changes. |
accepted | accepted |
1080 Why Care of section 4.4 | XML Schema/Tony Cincotta | [[[In section 4.4 Deprecation, the subheading entitled "Why care?" states "It helps implementers know which features are expected to be obsoleted in the next version of the technology." The XML Schema WG suggests modifying the text of this sentence by: a) adding the phrase "as well as users". b) synchronizing it with the indefinite status of features under deprecation. Suggest: "It helps implementers as well as users know which features may become obsolete in future versions of the technology."]]] |
The Working Group agreed with the suggested change. | accepted | (considered accepted unless originator responds otherwise) |
1081 Typos in SpecGL - comment from XML Schema WG | XML Schema/Tony Cincotta | [[[ Aside from this comment, typos identified include: - "implementations that needs" in 4.4 Requirement B, Who cares? - numerous instances of "table of content" vs. "table of contents". - "Deprecating a features" in 5.5 Good Practice C, What does it mean. ]]] |
The Working Group has made the appropriate corrections, and thanks the reviewer for his careful reading of the document. |
accepted | (considered accepted unless originator responds otherwise) |
1082 Extensibility in the case of WCAG 2.0 | WAI CG/Wendy Chisholm | [[[ About section: 4.3 Requirement A It seems that "extensibility" of WCAG 2.0 is related to policy makers adopting or extending the criteria of WCAG 2.0. For example, JIS creating additional success criteria specific to Japanese accessibility issues. Is this correct or does it not apply? ]]] |
The QA Working Group indicates that the WAI CG's interpretation as stated in Original Comment is correct and that WCAG2.0 guidelines/success criteria can be extended, because the Guidelines are deliberately generic; thus they can be applied and adapted (extended) by different policy makers, and extensibility is appropriate. The QA Working Group suggests that WCAG2.0 documentation indicate that it is allowable to add additional guidelines/criteria to those already in WCAG2.0 for the previously-stated purpose, but that such additions should be in the style of existing WCAG2.0 success criteria/guidelines if possible. Furthermore, the QA Working Group suggests that the mechanism for such extensibility should be defined formally in the WCAG2.0 documentation. The QA Working Group believes that such extensibility considerations as mentioned previously are consistent with 2.4.3 Extensibility and Extensions in the revised SpecGL draft. | accepted | (considered accepted unless originator responds otherwise) |
1083 Deprecation in case of WCAG 1.0/WCAG 2.0 | WAI CG/Wendy Chisholm | [[[ About section: 4.4 Requirement A - "deprecated features" Does "Mapping Between WCAG 1.0 and the WCAG 2.0 Working Draft" satisfy this requirement? Some of the WCAG 1.0 Checkpoints have evolved into WCAG 2.0 Techniques while others are likely to be deprecated. However, because the style is so different between WCAG 1.0 and WCAG 2.0, we are not able to include WCAG 1.0 Checkpoints directly in WCAG 2.0 in a way that makes sense and would allow us to mark them as deprecated (in the WCAG 2.0 spec).]]] |
The QA Working Group acknowledges that the WCAG1.0 and WCAG2.0 documents are very different in style, and that mapping between WCAG1.0 Checkpoints and WCAG2.0 in such a way that would allow such Checkpoints to be marked as decprecated in WCAG2.0 would be indirect and difficult. However, the QA Working Group believes that the creation of such a mapping is important, because specification users need to know the evolution of particular features from WCAG1.0 to WCAG2.0. The QA Working Group suggests that an appendix be created in WCAG2.0 with the mapping desired by the QA Working Group (indicated previously), showing and explaining the evolution of particular features from WCAG1.0 to WCAG2.0; in such an appendix, the QA Working Group suggests to use whatever format is most appropriate in meeting the objectives of the appendix. The QA Working Group believes that such an approach mentioned previously would be consistent with 2.4.4 Deprecation in the revised SpecGL draft. | accepted | (considered accepted unless originator responds otherwise) |
1085 Detailed Table of contents | WAI CG/Wendy Chisholm | [[[ About: General The printed document is 75 pages long, and it is easy to get lost while trying to find a specific requirement or good practice. Request: increase the levels of the Table of Contents for Guidelines section. ]]] |
The QA Working Group agrees with the WAI CG that the table of contents should go into deeper levels to to be able to find requirements and good practices more easily and quickly. Note that in the revised SpecGL draft, the List of Requirements and List of Good Practices are linked directly after the Table of Contents, and are also numbered consecutively for easier reference. | accepted | (considered accepted unless originator responds otherwise) |
1087 Address Accessibility in Requirements | WAI CG/Wendy Chisholm | [[[ About: General SpecGL does not mention the need to consider accessibility while writing a spec, particularly the need for XML-based vocabularies to conform to something like the XML Accessibility Guidelines. Request: an additional requirement. For example, consider adding a Requirement to 1.1: [proposal]Address Accessibility What does it mean? Accessibility must be encouraged by the Working Group. The benefit of addressing accessibility is the increased likelihood that both user agents and authoring tools will implement the accessibility features of the specification from the beginning. Otherwise, it make take several revisions before software addresses accessibility features, leaving people with disabilities behind. Formalizing the position of the Working Group by a clear defined section and prose removes ambiguities for the specification users about the possibility of addressing accessibility. Refer to the XML Accessibility Guidelines. [/proposal] ]]] |
The QA Working Group agrees with the request of the WAI CG that SpecGL should mention the need to consider accessibility while writing a specification, but disagree with making this a normative requirement in SpecGL, since its scope in this version of the document is mainly focused on conformance related items and that QA WG participants don't have enough background and experience to add further requirements on this topic at this stage of development. In addition to accessibility, the QA Working Group has decided that SpecGL should mention the need to additionally consider internationalization and device independence while writing a specification. Accordingly, a new Section 3.3 (Accessibility, Internationalization, and Device Independence Considerations) has been created in the revised SpecGL draft. Note that there is a reference to the XML Accessibility Guidelines within Section 3.3. | partially accepted | accepted [Member only] Text of the resolution available. |
1089 Need clarifications for 2.2 Requirement A | WAI CG/Wendy Chisholm | [[[ About: 2.2 Requirement A: Identify who or what will implement the specification. Instead of "Who or what?" maybe it should be "who and/or what?" For WCAG 2.0, not only will developers claim conformance dependent on the use of technologies (like HTML, CSS, etc.) [i.e., "what"], but organizations may want to make a conformance claim (about their content) [i.e., "who"]. Perhaps this requirement needs clarification. ]]] |
The QA Working Group agrees with the request of the WAI CG to change the Requirement "Identify who or what will implement the specification" to "Identify who and/or what will implement the specification". Accordingly, Requirement 03 (old 2.2A) in the revised SpecGL Draft has been changed as indicated. | accepted | (considered accepted unless originator responds otherwise) |
1090 Managing variability in WCAG 2.0 | WAI CG/Wendy Chisholm | [[[ About: Section 4 Managing Variability. Since we are not writing a language, we were not sure how we would apply this to a recommendation like WCAG 2.0. We seem to manage variability through conformance to three levels of Success Criteria: in some instances a success criterion does not apply either because it is at a higher level of conformance (than the author is aiming for) or the technology being addressed by the success criterion is not being used in the Web content for which the conformance claim is made. Is this the correct interpretation? By using success criteria do we address the requirements in this section? ]]] |
The QA Working Group agrees with the WAI WG that the approach taken by the WAI CG in managing variability through conformance to three levels of success criteria (as indicated in Original Comment) is correct in interpretation. The QA Working Group believes that this approach is consistent with 2.4 Managing Variability of revised SpecGL draft. | accepted | (considered accepted unless originator responds otherwise) |
1091 Formal Language vs Prose: Ambiguity | WAI CG/Wendy Chisholm | [[[ 5 Good Practice e: "Use formal languages and define which from prose and formal languages has priority." This sentence is ambiguous. Propose using: "Use formal languages and if also using prose, clarify if prose or formal language has priority." ]]] |
The QA Working Group has accepted the comment. See Good Practice 11 in the revised SpecGL draft. | accepted | (considered accepted unless originator responds otherwise) |