Issues regarding the documents produced by the QA Working Group should be reported to www-qa@w3.org (public archives).
Comments on this list should be sent to the www-qa@w3.org mailing list (Archives).
num | Status | Spec | Topic | Class | Date | Title |
---|---|---|---|---|---|---|
79 | Resolved | TestGuide | TestGuide | Substantive | 2002-09-03 | Should Test Guidelines address how to write a good test suite? |
45 | Resolved | WG-process | WG-process | Substantive | 2002-01-14 | How should the QA WG interact with TAG? |
18 | Active | All-Framework | All-Framework | Editorial | 2002-01-02 | Should the Framework documents be published as Note or as Working Draft? |
23 | Active | TestMaterials | TestMaterials | Substantive | 2001-12-28 | Tests for MAY/SHOULD assertions. |
num | Spec | Date | Topic | Class | Status | Raised By | Owner |
---|---|---|---|---|---|---|---|
1 | IssueList | 2001-12-20 | IssueList | Editorial | Closed | Mark Skall | |
Title: How should QA Issues be managed and tracked? | |||||||
Description: [email] This was raised and discussed at Brussels f2f, 11/2001, without clear resolution. Bugzilla was discussed at that meeting. Another proposal (Henderson & Gavrylyuk): Maintain them in a simple XML grammar that can be XSLT-transformed into tabular or other document format (SOAP and DOM do this). Alternative (Henderson): SVG has a simple method, that seems to be based on a simple plain text master document, http://www.w3.org/Graphics/SVG/Group/issues.txt, which has very little prose text and relies mainly on references to the mail archive. Other alternative, to use SourceForge (per DOM-TS) was proposed at the 2001-12-06 QAWG teleconference. Discussed and finally resolved at 2001-12-20 QAWG teleconference. | |||||||
Proposal: | |||||||
Resolution: Use simple XML/XSLT scheme, similar to SOAP and DOM. | |||||||
2 | IssueList | 2001-12-20 | WG-process | Editorial | Closed | Lofton Henderson | |
Title: How should issues be identified or numbered? | |||||||
Description: What do we want for an issue number scheme? Alternatives: no number, use a mnemonic name, such as "issue-number-scheme"; or, simple ordinal numbering 1, 2, 3, ...(SVG does this); or, year and number 2001-1, 2001-2, ... | |||||||
Proposal: | |||||||
Resolution: Have simple ordinal 'issue-num' plus meaningful 'id'. | |||||||
3 | IssueList | 2001-12-20 | IssueList | Editorial | Closed | Lofton Henderson | |
Title: Should editorial and substantive issues be in same Issues List? | |||||||
Description: Some discussion at Brussels face-to-face that we should separate them. Given that much of our initial work will take place in the context of producing the Framework documents, it might not always be easy to separate issues ("doc" versus "substance"). Classifying or categorizing will increase management complexity. | |||||||
Proposal: | |||||||
Resolution: Editorial and substantive issues in single Issues List. | |||||||
4 | Minutes | 2001-12-20 | ActionItems | Editorial | Closed | Lofton Henderson | |
Title: How should Action Items (AI) be managed and tracked? | |||||||
Description: How should Action Items (AI) be managed and tracked? We need to keep the open AI visible, and also to document the when and how of closing AIs. No firm proposals, but some options. Option: Karl Dubost is working on tracking open items in an in-design "Agenda" addition to a re-organized QA Web site (unclear whether this tracks all AI, including competion and closure). Option: closure and completion is tracked in minutes of f2f and teleconferences (original de-facto method). Option: both the open-AI list and documentation of individual closures and completions is tracked only in minutes. Option: something like the method of SVG (see the issue about "Issue tracking"). | |||||||
Proposal: | |||||||
Resolution: Action Items document (HTML table) will be kept in the WG web space. Updated by email solicitation and response before telcons, only discussed in telcons if any AIs are problematic. Updated with additions after telcons and f2f meetings. | |||||||
5 | All-Framework | 2001-12-20 | All-Framework | Editorial | Closed | Lofton Henderson | |
Title: How should the QA Framework documents be partitioned? | |||||||
Description: How should the QA Framework documents be partitioned? Input to Brussels face-to-face was single document. It was partitioned at Brussels into: Intro & Process Guidelines; Operational Guidelines; Spec Guidelines; Test Materials Guidelines. Working on the first two pieces, Henderson & Gavrylyuk propose to move the Process Guidelines from "Intro" to "Operational" document. See qaframe-intro-1202.html. | |||||||
Proposal: | |||||||
Resolution: Introduction; Process&Operational Guidelines; Specification Guidelines; Test Materials Guidelines | |||||||
6 | All-Framework | 2001-12-20 | All-Framework | Editorial | Closed | Lofton Henderson | |
Title: what should be the schedule for FPWD of the first two Framework parts | |||||||
Description: What should be the schedule for First Public Working Draft (FPWD) of the first two parts? At Brussels face-to-face, the decision was: mid-December and early-January. Since the substantive Process guidelines have moved into the "Operational" part (now "Process & Operational Guidelines"), should we still put out FPWD "Intro" by itself? Is it substantial and interesting enough, and can it stand alone without at least one other part (notice proposed linkage between Intro and the other parts). | |||||||
Proposal: | |||||||
Resolution: Publish Introduction and "Process & Operational Guidelines" at the same time (target: Mid-January 2002). | |||||||
7 | All-Framework | 2001-12-20 | All-Framework | Editorial | Closed | Lofton Henderson | |
Title: What should be the names of the four Framework parts | |||||||
Description: Names for the (current four) QA Framework documents? See proposed names in qaframe-intro-1202.html, section 1.1. | |||||||
Proposal: | |||||||
Resolution: "QA Framework:" followed by one of the four "Introduction", "Process & Operational Guidelines", "Specification Guidelines", "Test Materials Guidelines" | |||||||
8 | All-Framework | 2001-12-20 | All-Framework | Editorial | Closed | Lofton Henderson | |
Title: What format for the Framework documents? | |||||||
Description: Agreement at Brussels face-to-face, Guidelines and verifiable checkpoints, similar to the WAI standards. "Examples & Techniques" documents associated with the guidelines documents, to handle per-WG and taxonomy-specific details, examples, and implementations (similar to WAI "Techniques"). | |||||||
Proposal: | |||||||
Resolution: per Brussels agreement. | |||||||
9 | OpsGuide | 2001-12-20 | OpsGuide | Substantive | Closed | Daniel Dardailler | |
Title: Where should test suites reside? | |||||||
Description: Where should test suites reside? The draft Framework input to Brussels face-to-face has a Guideline that says they should ultimately live in the responsible W3C WG. In the 2-jan-2002 draft Proc&Ops document, see section 2.2.2.2. This has since been questioned, email from Lynne Rosenthal. It was discussed at 2001-01-17 telcon, with apparent consensus around some basic principles: W3C/WG should have a stake in the work and be an active participant/partner; it must be open and freely available; must have stable, secure repository; must have dependable maintenance and update plan; etc. The scope of this issue is limited to: free availability and ensured stability of at least one high quality test suite for a WGs work. Revised in Guideline 2 of Process & Operational Guidelines (01-feb- 2002, first published version). See related issue #44. Closed at 14-feb-2002 telcon. | |||||||
Proposal: Daniel Dardailler proposed an interpretation of the checkpoint which seems agreeable to all. | |||||||
Resolution: Per proposal. | |||||||
10 | SpecGuide | 2001-12-20 | SpecGuide | Substantive | Closed | Daniel Dardailler | |
Title: How will QA WG perform spec reviews? | |||||||
Description: When, how, and for what Working Group will QA perform a "spec
review"? QA reviews of WG specs is one of our documented QAWG
deliverables. The issue was mentioned at
Brussels face-to-face. No
proposals resulted. See also issue #36 Discussed and closed at Montreal face-to-face (6/2002): QAWG, in general, will not do spec reviews for the specs of the Working Groups (no resources); however, each QAWG member will do one spec review according to the Spec Guidelines. We will keep a matrix on /QA/Group/, with the assignments, and the results will be kept and linked from there. These reviews are principally for the benefit of QAWG, to help in refining and advancing the Spec Guidelines document. QAWG will encourage the WGs (through the Team contact and/or chair) to apply the Spec Guidelines and do their own reviews. QAWG believes that WG spec reviews should be required in the future (see issue #16). |
|||||||
Proposal: | |||||||
Resolution: As described above: QAWG will do some reviews for SpecGuide development purposes; otherwise it is the responsibility of the WGs. | |||||||
11 | SpecGuide | 2001-12-20 | SpecGuide | Substantive | Closed | Daniel Dardailler | |
Title: How should QA spec guidelines relate to pubrules? | |||||||
Description:
How will QA guidelines, especially the "Specification
Guidelines", relate to publication and style rules (Comm team)? Raised
at
Brussels face-to-face,
needs to be pursued and defined.
See issue #25. This is redundant with part of the scope of that issue. |
|||||||
Proposal: | |||||||
Resolution: See issue #25. | |||||||
12 | All-Framework | 2001-12-20 | WG-process | Substantive | Closed | David Marston | Daniel Dardailler |
Title: How will QAWG interact with WAI and I18N WGs? | |||||||
Description:
How will QAWG interact with WAI and I18N WGs, and how will QA
guidelines interact with WAI and I18N guidelines? Raised in
email
and at at
Brussels face-to-face.
The Framework document input to
Brussels face-to-face
stipulates certain
levels of WAI conformance in a WGs test materials. But the issue hasn't
been carefully discussed yet. A further dimension, after defining
the relationship, is: where and how will it be documented in the QA
Framework and/or QA Web site resources? (Related issues:
issue #25,
issue #45.)
See proposed way forward in
DD email:
current separate ad-hoc review systems are
okay for now; and, we (QA) need to work out clear problem statement, what we want to
do and why wrt horizontal technical reviews.
Discussed again at Montreal face-to-face (6/2002). For lack of any other proposals, closed as suggested, the WGs will use their "separate ad-hoc review systems." Interaction of QA guidelines with those of WAI and I18N are not addressed. |
|||||||
Proposal: Per DD email, as just summarized above. | |||||||
Resolution: Separate ad-hoc review systems of the WGs. No specific defined relationship between respective guidelines. | |||||||
13 | TestGuide | 2001-12-20 | TestGuide | Substantive | Closed | David Marston | Peter Fawcett |
Title: Is inter-standard or multi-standard interoperability within the QA scope? | |||||||
Description:
Is inter-standard or multi-standard interoperability within the
QA scope? Issue raised in
email thread
(DM, 2001-11-07, 2nd pgph).
Discussed at Tokyo face-to-face, DD given assignment to: draft wording (informative text) about not introducing inconsistencies when using/referencing other specs. Discussed at Seattle face-to-face, and reclassified from OpsGL issue to TestGL. It doesn't apply to OpsGL. It applies to SpecGL, and is adequately addressed by the normative-references checkpoint (CP10.2). In addition to addressing multi-standard test suites, TestGL needs to consider whether this is a new point in the test materials taxonomy. Discussed at Boston face-to-face. Clarification, the originator's original issue description was: "Also say something to position the idea of testing across multiple Recommendations, possibly from multiple WGs. This is needed for certain kinds of software but may be wishful at this time, so I'd say that the notion should be acknowledged but positioned as a long-range objective. Some of the goals in 1.1 around common test tools would be more compelling if they could be used to test the interoperation of multiple Recs." Resolved: in order to keep the first version of TestGL simple and implementable, the first version of TestGL will not address the issue. |
|||||||
Proposal: Originator proposes that multi-standard test suites are in-scope, but are lower priority and/or should be long-term goals. | |||||||
Resolution: In order to keep TestGL simple and implementable, the first version of TestGL will not address the issue. | |||||||
14 | n/a | 2001-12-20 | WG-process | Substantive | Closed | Daniel Dardailler | |
Title: How should Education & Outreach be carried out? | |||||||
Description:
How should Education & Outreach be carried out,
specifically what sort of presence should QA have at conferences? Issue
raised at
Brussels face-to-face.
The desireability is recognized, but the realization
is not decided.
Discussed at 30030317 telecon. Resolved that QAWG won't attempt to define a blueprint or master plan, but rather will continue its current ad hoc way of addressing E&O. That involves time for IG topics and E&O planning at the start of each face-to-face meeting, and teleconference time as needed. |
|||||||
Proposal: | |||||||
Resolution: To be reviewed and planned at each face-to-face meeting, and at telecons as needed, and executed accordingly. | |||||||
15 | All-Framework | 2001-12-20 | All-Framework | Substantive | Closed | Lofton Henderson | |
Title: Should QA Framework address the topic of valid conformance claims about W3C standards? | |||||||
Description: [email]
Should QA (e.g., Framework) address the topic of valid
conformance claims, especially when W3C test materials are involved? It
is fairly well accepted that W3C does not intend to get into the
certification business. There is email thread and
Brussels face-to-face
discussion
about related issues: accreditation of third party certifiers; use (or
abuse) and claims of conformance related to W3C test materials; logos and
branding, such as XHTML, WCAG1.0, etc (see for example
issue #32). There are probably several
closely related issues to be separated here.
Discussed at Tokyo face-to-face, and changed from SpecGuide to All-Framework classification (topic). Within SpecGL, the related question has been affirmatively answered, "Should SpecGL have requirements that specs detail how to make a conformance claim?" Discussed at Seattle face-to-face. OpsGL checkpoint 6.4 addresses "open disclosure", by requiring WG to associate proper conformance disclaimers with any test materials, and checkpoint 4.6 requires WGs to consider and address branding (icon) policy. Several checkpoints of a conformance-claims guideline (GL11) adequately address requirements for conformance claims guidance in specifications, including recommended content and required minimal content. Beyond this -- e.g., policing or enforcing claims -- QAWG believes it is not the business of QA Framework (it is the business of certifying bodies). |
|||||||
Proposal: | |||||||
Resolution: As just described (Seattle resolution). | |||||||
16 | OpsGuide | 2001-12-20 | OpsGuide | Substantive | Closed | Lofton Henderson | |
Title: Should the W3C Process Document be modified to require QA deliverables and schedule of WGs? | |||||||
Description:
Should there be modifications to the W3C Process Document, to
require that Activity statements and/or WG Charters address QA
deliverables? There are at least two aspects: require that Activity
statements and charters document QA deliverables for Activity/WG; and,
require some QA deliverables synchronized with Rec-track phases (e.g., test
suites/tools before CR-exit.)
Discussed at
Brussels face-to-face.
General feeling at Brussels seemed to be that
it was unnecessary. The QA Framework will require it. The Activity
Statements and Charters get AC review and approval. This will suffice to
ensure that QA deliverables are properly addressed.
Discussed again at Montreal face-to-face, and subsequent email proposal suggested that Ops Guidelines probably ought to be something like an addendum to Process Document. Subsequent email contribution fully explored the options for "how" to make GL conformance mandatory. See related issue #71, on whether Guidelines (GL) conformance should be mandatory or not. |
|||||||
Proposal: | |||||||
Resolution: For now, QAWG will not propose any changes. | |||||||
17 | OpsGuide | 2001-12-20 | OpsGuide | Substantive | Closed | Lofton Henderson | |
Title: Where should conformance test materials reside after a WG disbands? | |||||||
Description: Where should conformance test materials reside "long term"? After a WG disbands, who is responsible for managing test materials? Issue was mentioned at Brussels face-to-face. No clear resolution. Closed at 4-apr-2002 QAWG teleconference. | |||||||
Proposal: | |||||||
Resolution: See 2002-04-05 Operational Guidelines checkpoint 6.1 | |||||||
18 | All-Framework | 2002-01-02 | All-Framework | Editorial | Active | Lofton Henderson | Daniel Dardailler |
Title: Should the Framework documents be published as Note or as Working Draft? | |||||||
Description: [email]
Should the Framework documents be posted as Note
or as Working Draft? Currently, the document-template icon says "Note" but
the Status says "Working Draft.
Ian Jacobs proposed, "as a Working Draft", using both "Working Draft" titles
and css style, see
his email proposal. Revisited at the Montreal face-to-face, and extended per subsequent email proposal. Initial publication as a Working Draft is reaffirmed. Thereafter,
After Montreal f2f, issue was closed with resolution: "As WD, using WD titles and stylesheets. Then progress through LC, CR, and finally to Note". At Team Project Review (member-only) of QAWG, June 2002, some team members argued strongly that Framework docs should complete processing to Recommendation (see around 13:42:12 UTC). Therefore the issue is reopened, in this detail only: should Framework progress finally to "Note" or to "Recommendation"? See related issue #71. |
|||||||
Proposal: Proposal-1: As WD, using WD titles and stylesheets. Then progress through LC, CR, and finally to Note; proposal-2: "...finally to Rec". | |||||||
Resolution: | |||||||
19 | All-Framework | 2002-01-02 | All-Framework | Editorial | Closed | Lofton Henderson | Lofton Henderson |
Title: Should each Framework document have a "Glossary" appendix? | |||||||
Description:
Should each Framework document have a "Glossary" appendix?.
Currently, the documents say in their terminologies section that unusual QA
terms will be defined when first used, will be collected into a per-document
glossary appendix, and will ultimately migrate to the central QA Glossary.
Resolved at
4-apr-2002 QAWG teleconference ("no").
The resolution was changed to "yes" (or more accurately, "maybe" -- case-by-case per GL document) at an editors meeting in July, where the editors of Intro, OpsGL/Extech, SpecGL, and TestGL agreed that a "Definitions" section that develops with and tracks the document is essential to the quality and clarity of at least some of the evolving GL documents (SpecGL and TestGL). These definitions sections can also have more elaborate functional or contextual definitions, as compared to the terse QA Glossary. So it is possible that the definitions will *not* migrate to the QA Glossary (although the terms should migrate with an appropriate generic definition). Discussed at Tokyo face-to-face, and re-opened with request to: reverse the resolution that Framework documents may have definitions that differ from those in QA Glossary (by expanding on and supplementing them with functional or contextual information). Subsequent email discussion. Discussed again at 2002-10-16 telecon. Agreement about the process and coordination issues -- yes, GL documents need a definition section as they progress. Definitions not already in QA Glossary may migrate to QA Glossary at some point when GL document has stabilized (LC? CR? ...). No unanimity yet about the QA Glossary "identicalness" issue: should the definitions remain in GL document when/if they migrate to the QA Glossary. Options: no; yes, but only a pointer; yes, extended (optionally). Details of "yes, extended":
Agreed to start new QAIG email discussion thread. Discussed at Seattle face-to-face. The basic scheme above was re-affirmed (and a glossary project of W3C-wide scope, using a similar model, was discussed). |
|||||||
Proposal: | |||||||
Resolution: Yes, each Framework part may have a definitions section, to be decided as needed on a case-by-case basis, to supplement and not contradict any existing QA Glossary definition. | |||||||
20 | Introduction | 2002-01-02 | WG-process | Editorial | Closed | Lofton Henderson | |
Title: Should the QA WG web page have a central bibliography index to notes and other documents? | |||||||
Description: Should the QA WG web page have a central bibliography index to Notes and other documents? This is described in the 2001-01-02 draft of the "Framework: Introduction", but it is not on the WG action items list. Resolved at 4-apr-2002 QAWG teleconference. | |||||||
Proposal: | |||||||
Resolution: Yes, it is an action item to develop one. | |||||||
21 | All-Framework | 2002-01-02 | All-Framework | Editorial | Closed | Lofton Henderson | |
Title: Where should WG-only drafts of the Framework documents reside? | |||||||
Description: Where should WG-only drafts of the Framework documents reside? Currently they are being put into ../QA/WG, but their "This version" indicates locations like ../QA/2002/01/. | |||||||
Proposal: | |||||||
Resolution: latest : /QA/WG/qaframe-part(.html) redirects to dated : /QA/WG/YYYY/framework-YYYYMMDD/qaframe-part(.html) | |||||||
22 | Introduction | 2002-01-02 | Introduction | Substantive | Closed | Dimitris Dimitriadis | |
Title: Should the scope of the Framework Introduction be expanded? | |||||||
Description: [email]
Should the scope of the Framework Introduction be expanded?
Currently, the Intro scope covers: overview of QA activity and goals; survey
of QA resources; description and roadmap of QA Framework document family; and,
guide and primer to using the framework documents. It has been proposed that
the scope be expanded to include:
|
|||||||
Proposal: | |||||||
Resolution: The "Introduction" should introduce the Framework documents, not the QA Activity. No to #1 and #3. Remove "Resource Survey" to Web site. (Uncertain Re: #2) | |||||||
23 | TestMaterials | 2001-12-28 | TestMaterials | Substantive | Active | Dimitris Dimitriadis | Active |
Title: Tests for MAY/SHOULD assertions. | |||||||
Description:
Should tests of MAY/SHOULD assertions be part of a conformance suite?
They are useful, so we need to discuss these tests in the Framework documents,
probably in Test Materials Guidelines and/or Spec Guidelines. Note that this
issue applies to test suites related to W3C recommendations, not to the Framework
documents themselves.
issue #39
deals with normative language in the Framework documents themselves.
The issue was discussed at the
2002-03-01 March QAWG face-to-face meeting, although the documentation
of the discussion seems to refer more to issue #39. The sense of the
discussion was "yes", test for SHOULD and MAY assertions in technical
standards (Recommendations). The form of the tests and how they enter
into the "conform/non-conform" verdict will depend on how the
specification is written, specifically how it defines conformance
related to SHOULD and MAY assertions.
Discussed at Boston face-to-face. The minutes are unclear of the exact resolution, but indicate that MS and DM have action items to (respectively) raise the corresponding issue in SpecGL and draft a resolution. |
|||||||
Proposal: Yes, method and verdict TBD by conformance statements of Recommendation. | |||||||
Resolution: | |||||||
24 | WG-process | 2001-12-28 | WG-process | Substantive | Closed | Kirill Gavrylyuk | |
Title: Appeal process when QA WG rejects a request from WG | |||||||
Description:
Do we need any appeal process? It is currently implied in
section
2.6.1 of 2-jan-2002 Proc&Ops draft. See also Issue #52
and Issue #53.
At
4-apr-2002 QAWG teleconference, it was decided to postpone
this until discussion resumes on Issue #53.
Discussed at 30030317 telecon. Resolved that there won't be any formal appeal process. Reasons. Requests that are within the scope of external relations requests (see draft QA Process document) will be handled if resources are available. If there are no resources, that is essentially not subject to appeal. |
|||||||
Proposal: | |||||||
Resolution: No formal appeals process will be defined. | |||||||
25 | All-Framework | 2001-01-14 | WG-process | Substantive | Closed | David Marston | Daniel Dardailler |
Title: How should QA interact with W3C teams, notably the Comm team. | |||||||
Description: [email]
The QA Framework documents will include a part on
"Specification Guidelines".
The Comm team has normative
pubrules,
and (currently)
non-normative "Style Manual".
How will QA interact with Comm, and how will the respective
specifications relate? A further dimension is: where in the QA
Framework documents or other resources should this be
documented? Related issues:
issue #12;
issue #45.
See proposed way forward in DD email In particular, note that (aside from documents), there is a second aspect of "how relate" -- how will QA use Comm to promote and publicize its work and products? This has not been addressed. Discussed again at Montreal face-to-face (6/2002). Liaison re-affirmed as adequate solution. "Spec Guidelines" mentions pubrules and Style Manual, and their role for editors. Issue declared closed. (Note. One aspect of the issue has been discussed on "Chairs", without conclusion. The proposal -- advocated by DD for example -- that pubrules, Style Manual, Spec Guidelines ought eventually to be combined into one document with normative and informative parts.) |
|||||||
Proposal: For now, establish liaison/point-of-contact with Comm (Dominique Hazaël-Massieux, most likely) | |||||||
Resolution: Liaison to communicate significant events, such as publication of new Spec Guidelines versions, and request feedback. | |||||||
26 | OpsGuide | 2001-12-28 | OpsGuide | Substantive | Closed | Lofton Henderson | Kirill Gavrylyuk |
Title: Conformance section | |||||||
Description: This document should have a conformance clause. Uncertain what it should say now, as it is unclear what is the normative force of this document. At least it should probably address what is involved in (possibly voluntary) conformance to this document. Precisely what it might say is an issue that perhaps we can address later? Discussed in 2002-01-03 telcon, and resolved: similar to WCAG 1.0 clause. | |||||||
Proposal: | |||||||
Resolution: Similar to WCAG1.0, including three levels of conformance based on prioritized checkpoints (p1, p2, p3). | |||||||
27 | OpsGuide | 2001-12-28 | OpsGuide | Editorial | Closed | Lofton Henderson | Kirill Gavrylyuk |
Title: Need test maintenance guidelines/checkpoints | |||||||
Description: Need to distill guidelines/checkpoints from maintenence section. See 2.5.1-4 of 2-jan-2002 Proc&Ops draft. | |||||||
Proposal: | |||||||
Resolution: See Guideline 8 of Proc&Ops, first published version. | |||||||
28 | OpsGuide | 2001-12-28 | OpsGuide | Substantive | Closed | Kirill Gavrylyuk | Active |
Title: Test maintenance task force | |||||||
Description: Is it obligatory to have a task force for test materials maintenance? Should we use SHOULD or SHALL in the text? Working Group SHOULD/SHALL designate a task force or another entity to maintain the test suite, as well as maintenance procedures. See 2.5.1 of 2-jan-2002 Proc&Ops draft, and Guideline 8 of Proc&Ops, FPWD (1-feb-2002 first published WD). Closed at 4-apr-2002 QAWG teleconference. | |||||||
Proposal: | |||||||
Resolution: See resolution in text of 2002-04-05 Operational Guidelines, guideline 8. | |||||||
29 | OpsGuide | 2001-12-28 | OpsGuide | Substantive | Closed | Lofton Henderson | |
Title: Should there be a checkpoint about publication of testing results? | |||||||
Description: Do we need guidelines/checkpoints for publications of testing results? See 2.4.6 of 2-jan-2002 Proc&Ops, and checkpoint 6.7 of Proc&Ops FPWD (1-feb-2002, first published WD). Closed at 4-apr-2002 QAWG teleconference. | |||||||
Proposal: | |||||||
Resolution: Yes, see 2002-04-05 Operational Guidelines checkpoint 6.5 | |||||||
30 | OpsGuide | 2001-12-28 | OpsGuide | Editorial | Closed | Lofton Henderson | |
Title: Is test contribution review process a checkpoint or example? | |||||||
Description: It is not clear whether we want review process to be a guideline or just an example. See 2.4.5.4 of 2-jan-2002 Proc&Ops draft. | |||||||
Proposal: Both - a checkpoint and example(s), the latter to be in in "Ex&Tech" when it is written. | |||||||
Resolution: Both. See checkpoint 6.5 of Proc&Ops FPWD (1-feb-2002 first published WD). | |||||||
31 | OpsGuide | 2001-12-28 | OpsGuide | Editorial | Closed | Kirill Gavrylyuk | |
Title: Transfer of test material implies Rec phase is reached | |||||||
Description: Text for transfer of test materials from outside implies that spec in question has already Recommendation status. Need to add wording/bifurcation for the other case. See 2.4.4 of 2-jan-2002 Proc&Ops draft. | |||||||
Proposal: | |||||||
Resolution: Moot. Issue disappeared in Proc&Ops FPWD (1-feb-2002 first published WD), see guideline 7. | |||||||
32 | OpsGuide | 2001-12-28 | OpsGuide | Substantive | Closed | Lofton Henderson | |
Title: Guidelines/checkpoints for Branding | |||||||
Description:
Is Branding in scope of QA WG? What guidelines/checkpoints do we offer
for Branding?
Shall it be separate section or just an item under subsection "Scope"?
This is contentious issue; also borders on "conformance claims",
which was a big, hot www-qa argument.)
- WAI does this for content; CSS also and basic HTML validation. - Controversial topic, as email thread and UAAG CR comments show. - Taxonomy dependent: content (WCAG, CSS, etc) is less difficult/controversial; UAs will be contentious. Discussed at Seattle face-to-face. Beyond the OpsGL checkpoint that requires WGs to address the topic of branding, it is considered out of scope for QAWG to try to specify further details or requirements. |
|||||||
Proposal: | |||||||
Resolution: Detailed branding policy requirements are out-of-scope. Requiring that WGs address the issue is in-scope. | |||||||
33 | OpsGuide | 2001-12-28 | OpsGuide | Substantive | Closed | Lofton Henderson | Kirill Gavrylyuk |
Title: Guideline for minimum level of WG commitment to QA | |||||||
Description: Do we need a guideline/recommendation for minimum level of WG commitment to QA? See 2.2.1 of 2-jan-2002 draft. First published version of the Process & Operational Guidelines has de-facto proposal, "level-3". 2002-01-28 telcon decided to leave issue open till after first publication (2002-02-01). Resolved at 4-apr-2002 QAWG teleconference, resolution is implemented in 2002-04-05 Operational Guidelines document. | |||||||
Proposal: As just stated, "level-3". | |||||||
Resolution: Table wording revised, at-least-level-3 is P1, higher levels (subsuming level-3) are P2/P3. | |||||||
34 | OpsGuide | 2001-12-27 | OpsGuide | Substantive | Closed | Daniel Dardailler | |
Title: Should QA bind entry/exit of spec milestones to QA deliverables? | |||||||
Description: Should we provide any guidelines for exit criteria bound to spec milestones in the document? See 2.1 of 2-jan-2002 Proc&Ops draft. | |||||||
Proposal: Significant test materials by CR-exit. | |||||||
Resolution: See checkpoint 1.2 of Proc&Ops FPWD (1-feb-2002, first published WD). | |||||||
35 | WG-process | 2001-12-27 | WG-process | Substantive | Closed | Daniel Dardailler | |
Title: Resources supplement to other WGs | |||||||
Description:
Issue raised at
Brussels face-to-face,
whether QA WG should provide
resources to some WGs for building test materials.
Lynne suggested involve universities, setup internship programs.
See
2.6, 2.6.1, 2.6.4 of 2-jan-2002 Proc&Ops draft.
Discussed at 30030317 telecon. Resolved it is neither affordable nor desirable -- QAWG has neither the resources nor the domain expertise. QAWG does, however, intend to provide generally useful tools, toolkits, and frameworks that should be helpful to individual WGs that are building their own test materials. |
|||||||
Proposal: | |||||||
Resolution: QAWG will not provide staff resources to WGs for direct participation in building test materials. QAWG does intend to provide generally useful tools, toolkits, and frameworks. | |||||||
36 | OpsGuide | 2001-12-27 | OpsGuide | Substantive | Closed | Lofton Henderson | |
Title: Should QAWG initiate proactive reviews? | |||||||
Description:
It is still an open policy question, whether the QA Activity will
pro-actively initiate reviews, and whether it has adequate resources to do so.
The given wording in the Procs&Ops document may be soften a bit. If we want to do
this proactive, should we also define some preliminary steps where by QAWG suggests,
requests permission, informs WG? See
2.6.2 of 2-jan-2002 Proc&Ops draft.
See related issue #10.
Discussed and closed at Montreal face-to-face (6/2002): QAWG, in general, will not do reviews for the QA aspects -- Operations, Specifications, Test Materials -- of the Working Groups (no resources); however, each QAWG member will do one review of each of the three types, according to the relevant Guidelines document. We will keep a matrix on /QA/Group/, with the assignments, and the results will be kept and linked from there. These reviews are principally for the benefit of QAWG, to help in refining and advancing the three Guidelines documents. QAWG will encourage the WGs (through the Team contact and/or chair) to apply the Guidelines documents and do their own reviews. QAWG believes that WG QA-guidelines reviews should be required in the future (see issue #16). |
|||||||
Proposal: | |||||||
Resolution: As described above: QAWG will do some reviews for Guidelines documents development purposes; otherwise it is the responsibility of the WGs. | |||||||
37 | OpsGuide | 2001-12-27 | OpsGuide | Substantive | Closed | Kirill Gavrylyuk | Active |
Title: Coordination of QA work with external entities | |||||||
Description:
Can an arbitrary WG request QAWG help in coordination between the
WG and external entities? Examples where coordination might be desired:
|
|||||||
Proposal: Yes. | |||||||
Resolution: Yes, see 2002-04-05 Operational Guidelines, Chapter 3. | |||||||
38 | OpsGuide | 2001-12-27 | OpsGuide | Substantive | Closed | Kirill Gavrylyuk | |
Title: Resolutions of external QA requests | |||||||
Description: Should QAWG formally assist in resolution of external QA related questions that involves one or more W3C standards. See 2.6.5 of 2-jan-2002 Proc&Ops draft. Closed at 4-apr-2002 QAWG teleconference. | |||||||
Proposal: Yes, QAWG should redirect them to the appropriate WG. | |||||||
Resolution: Yes, see 2002-04-05 Operational Guidelines, Chapter 3. | |||||||
39 | OpsGuide | 2001-12-27 | OpsGuide | Substantive | Closed | Lofton Henderson | Lofton Henderson |
Title: Use of Keywords SHOULD, SHALL, etc in OpsGuide. | |||||||
Description: [email]
Need to determine SHOULD, SHALL, MAY, etc
keywords use throughout Proc&Ops document. Note. As of the
Proc&Ops FPWD (1-feb-2002, first published WD), most of the normative
language is replaced by Priority 1, Priority 2, Priority 3 checkpoints. See
issue #55,
regarding the latter. Note that this
issue applies the Framework
documents themselves, not to test suites related to W3C recommendations.
issue #23
deals with test suites related to W3C recommendations.
Inter-related set of issues: #101, #80, #57, #39, #64 Discussed at Seattle face-to-face. All SHOULD and MAY usages were considered, and endorsed or changed. |
|||||||
Proposal: Editors to make initial choice, QAWG to do per-keyword review (prerequisite to marking issue "Closed"). | |||||||
Resolution: As reflected in OpsGL Last Call text (20030210). | |||||||
40 | OpsGuide | 2001-12-27 | OpsGuide | Editorial | Closed | Lofton Henderson | Kirill Gavrylyuk |
Title: Test Materials home | |||||||
Description: Currently (2-jan-02 draft) there is an overlap between 2.2.2, 2.5 and 2.4.5.5 (materials publication). Need to add at least pointers to the these 3 sections. (Note, there is a substantive issue on this topic as well, issue #9.) Closed at 4-apr-2002 QAWG teleconference. | |||||||
Proposal: | |||||||
Resolution: Moot, by virtue of text cleanup in subsequent versions. | |||||||
41 | OpsGuide | 2001-12-27 | OpsGuide | Editorial | Closed | Lofton Henderson | Kirill Gavrylyuk |
Title: Guidelines/checkpoints are not numbered | |||||||
Description: Guidelines and checkpoints are not yet numbered. Need to do before final version. (By adding to editors' XSLT stylesheet for transforming unformatted, unnumbered sections and guidelines/checkpoints.) | |||||||
Proposal: | |||||||
Resolution: "G 1. Statement of gdln..." "C 1.1. Statement of ckpt..." | |||||||
42 | Introduction | 2002-01-08 | n/a | Editorial | Closed | Daniel Dardailler | |
Title: Where should the Taxonomy document be kept? | |||||||
Description: Raised in
QAWG teleconference.
The Taxonomy document currently sits in the QA
Web space, linked from several locations on QA web pages. It is currently a short
(1-page) outline document with no descriptive text. Alternatives:
|
|||||||
Proposal: | |||||||
Resolution: Leave it as a separate document. | |||||||
43 | All-Framework | 2002-01-08 | certification | Editorial | Closed | Daniel Dardailler | |
Title: How should the Certification Note relate to the Framework(s)? | |||||||
Description:
NIST (Lynne Rosenthal) submitted a document about Certification to the QA Activity. We need to work out what is the relationship between this note and the Framework's. Discussed at Seattle face- to-face, and resolved that it does not relate to the QA Framework. It is a reference document to be kept in The W3C QA Library |
|||||||
Proposal: | |||||||
Resolution: No special relationship to the QA Framework, but a valuable reference document for The W3C QA Library. | |||||||
44 | OpsGuide | 2002-01-26 | OpsGuide | Substantive | Closed | Mark Skall | |
Title: Should W3C endorse externally produced test suites? | |||||||
Description: Split off from issue #9 in 2001-01-17 teleconference. If W3C assumes control of a test suite, then in that case there is a quality assessment and (implicit) endorsement. If a W3C WG actively participates in a co-developed or externally developed test suite, there is also (implicit) endorsement in this case. These cases are covered by issue #9. In other cases, where W3C has no active participation, should W3C give endorsement to one or more externally developed test suites? Discussed and closed at 2002-03-01 March QAWG face-to-face meeting. See related issue #58 | |||||||
Proposal: | |||||||
Resolution: No, do not endorse suites. But QA will link them from TheMatrix, provided that owner allows QA to publish the results of applying any appropriate QA Checklist (assessment) documents (e.g., future "Test Guidelines"). | |||||||
45 | WG-process | 2002-01-14 | WG-process | Substantive | Resolved | David Marston | Daniel Dardailler |
Title: How should the QA WG interact with TAG? | |||||||
Description: [email]
How will QAWG interact with TAG.
Where and how will it be documented in the QA
Framework and/or QA Web site resources? Similar issue
(for the "horizontal groups" WAI and I18N) raised in
email
and at at
Brussels face-to-face.
Related issues:
issue #25,
issue #12.
See proposed way forward in
DD email.
Discussed again at Montreal face-to-face (6/2002). Issue put into hibernation ("postpone") until someone proposes some specific, actionable resolution. Discussed at 30030317 telecon. Resolved that QAWG will designate a liaison (initially DH) to communicate QAWG concerns to TAG, and to be first point of contact within QAWG for TAG concerns. This will role will be documented on our Web site (as will as other roles like Lead Editors, owners of other documents and Web pages, etc). [KD takes action to make such documentation.] |
|||||||
Proposal: Establish liaison/point-of-contact, most likely with Ian Jacobs (TAG lead editor). | |||||||
Resolution: QAWG liaison to TAG established. [Documentation on QAWG pages tbd.] | |||||||
46 | OpsGuide | 2002-01-23 | OpsGuide | Substantive | Closed | Kirill Gavrylyuk | |
Title: Should "QAWG relationship to WGs" be normatively described with prioritized checkpoints? | |||||||
Description: How the QA WG relates to other WGs -- expertise, requests, resources, reviews, etc -- is described in Chapter 3 of the 18-jan-2002 draft of "QA Framework: Process & Operational Guidelines". There are no guidelines or checkpoints. Is this how it should be? Raised originally in email by KG on editors list, discussed by DD/KG in email thread, apparent consensus. Note. Whether it is normative or informative is a separate issue -- currently there is normative language usage, e.g., "SHOULD" requirements. | |||||||
Proposal: Leave it without checkpoints and explain that at beginning. | |||||||
Resolution: Leave it without checkpoints and explain that at beginning. | |||||||
47 | OpsGuide | 2002-01-23 | OpsGuide | Substantive | Closed | Kirill Gavrylyuk | |
Title: Should test materials be published in TR space? | |||||||
Description: [email]
Should test materials -- i.e., test suites and test tools --
be published in /TR/ space? Question has been raised as a
reasonable option. WG discussed and resolved "no", for these reasons:
|
|||||||
Proposal: No, test materials should not be published in TR space. | |||||||
Resolution: No. | |||||||
48 | TestMaterials | 2002-01-23 | TestMaterials | Substantive | Closed | Daniel Dardailler | |
Title: Should the planned Technical Guidelines be a guideline-checkpoint part of the Framework doc family? | |||||||
Description: [email]
The initial plan for the Framework document family specified a
fourth part, consisting of a document pair, "Technical Guidelines" and "Technical
Examples and Techniques". Three issues were raised:
|
|||||||
Proposal: 1.) Change "technical" to "Test materials"; 2.)...; 3.)...; | |||||||
Resolution: Change name, keep guideline-checkpoints plus example-techniques parts of Framework family. | |||||||
49 | OpsGuide | 2002-01-28 | OpsGuide | Substantive | Closed | Andrew Thackrah | Lofton Henderson |
Title: Should there be a global (W3C-standard) license for use and distribution of test materials? | |||||||
Description: [email]
There were originally two aspects to the issue:
See survey of current practice within W3C, generated subsequent to closure of the issue. Reopened at Montreal face-to-face, with presentation of a proposed test materials license (KG, to be circulated). Further suggestions regarding accommodation of commercial test suites raised in email thread.
Discussed at 2002-10-21 telecon, proposal-2 agreed by QAWG. KG to send electronic copy to LH immediately, LH to review with Joseph Reagle before publishing. Discussions with Joseph Reagle have caused the issue to be reopened again. He is not convinced that a new license is necessary. The remaining point of contention is whether the lack of scope-of-use restrictions in the W3C TM publication policy and license will prevent some companies from contributing test materials to W3C. (Pending) Discussed at Seattle face-to-face. A compromise was adopted, to try to address objections to introducing and maintaining a new W3C license. Use doc license if possible, software or W3C approved custom license if necessary; propose TM license as an example of license with scope of use restrictions. (There is still ongoing discussion between Kirill and Joseph.) Note. Subsequent to completion of OpsGL Last Call text (20030210), new comments were received. The issue will likely be re-opened as a Last Call issue. Issue summary presented and briefly discussed at Boston face-to-face (20030306). Reopened during Last Call issues resolution, discussed and closed at 20030501 telecon. |
|||||||
Proposal: proposal-2, KG contribution. (First aspect is Issue #59.) | |||||||
Resolution: If the Document License will work, it has attractive characteristics; if it won't work (non-modification and/or usability reasons), use the Software License; may use different ones for test documentation, test cases, and test software; consult with W3C Legal if neither Software nor Document works for you | |||||||
50 | OpsGuide | 2002-01-28 | OpsGuide | Substantive | Closed | Andrew Thackrah | Lofton Henderson |
Title: How should W3C deal with potential legal liability arising from third-party use of test materials? | |||||||
Description: [email]
If third parties use W3C-supplied test materials to do certification or
any sort of third-party testing of others' products, there is potential
for W3C liability for improper statements or claims by the third party.
Discussed at
2002-01-28 teleconference.
Discussion whether
checkpoint 3.3 of Process & Operational Guidelines
(1-feb-2002, first published
version) provides adequate liability protection. Discussion focussed on
the accuracy of the statements in checkpoint 3.3. Some people
thought that the statements need to include the context dependencies --
i.e., the conditionality (levels, profiles, etc) of conformance statements from
subject Recommendations.
Further elaboration in
email from originator
indicates two sub-issues in addition to the checkpoint 3.3 question:
Discussed at Seattle face-to-face. Resolved that it is adequately handled by the existing W3C distribution licenses. |
|||||||
Proposal: | |||||||
Resolution: It is adequately handled by the existing W3C distribution licenses. | |||||||
51 | SpecGuide | 2002-01-29 | SpecGuide | Substantive | Closed | Karl Dubost | Lynne Rosenthal |
Title: How should Spec Guidelines deal with deprecated features and withdrawn features? | |||||||
Description:
[email] Resolution of this issue needs to start with agreement on the definitions of "deprecate" and "withdraw". The original question was "What does conformance mean in the presence of deprecated features?" There seems to be some agreement that the conformance claim necessarily depends on how the spec is written to address deprecated and withdrawn features. That in turn depends on type of Rec/spec and the test materials taxonomy position. Peripheral questions to be answered include, what to do if the spec does not address the issue, i.e., what should be a default conformance interpretation. Some agreement that QAWG needs to define a menu/option approach to guide the WGs in writing specs. It is addressed in Guideline 9 of FPWD Specification Guidelines (2002-05-15, first public draft). Discussed at 2002-10-28 QAWG telecon (@@DRAFT@@), declared closed (de facto) with resolution: handle with a deprecation-policy guideline (now GL9) and checkpoints, add definitions of deprecate and withdraw to SpecGL. |
|||||||
Proposal: See David Marston email proposal. | |||||||
Resolution: Handle with a deprecation-policy guideline (now GL9) and checkpoints, add definitions of deprecate and withdraw to SpecGL. | |||||||
52 | OpsGuide | 2002-01-29 | OpsGuide | Substantive | Closed | Lynne Rosenthal | |
Title: Does the QA Working Group need a "Process Document"? | |||||||
Description: [email]
In draft rewrite of
Chapter 3 of the Process & Operational Guidelines
(1-feb-2002, first published version), Lynne Rosenthal
excluded much of the process-oriented information and
recommended that this information be put in a separate document that she
called, "Request of Assistance Process". Just as other WGs should have a
TS-Process document (guideline 6), the QAWG should have a Process
document. This document would describe the process for formal
consultation, active reviews, establishing teams of reviewers, etc.
Additionally, it is important that QAWG discuss what our process actually is
- the Framework:P&O editors provided a starting place for discussion and
the WG should discuss and decide how it will actually provide
assistance. There are two Issues.....
1. Should we have a Process Document? (this issue) 2. What is the process for handling requests from the WGs? (Issue #53) . How does someone formally request assistance, who is the point-person responsible to ensure that requests are processed, is there a task team appointed ad-hoc for each request, what is the timeframe for answering, etc. The issue was discussed and closed at the 2002-03-01 March QAWG face-to-face meeting. |
|||||||
Proposal: | |||||||
Resolution: Yes, QAWG needs a process document. | |||||||
53 | WG-process | 2002-01-29 | WG-process | Substantive | Closed | Lynne Rosenthal | |
Title: What is the process for handling requests from the WGs? | |||||||
Description: [email]
In draft rewrite of
Chapter 3 of the Process & Operational Guidelines
(1-feb-2002, first published version), Lynne Rosenthal
recommendeded that information concerning intra-QAWG process for handling
WG requests belonged not in Frm:P&O, but in a new, separate "Request of Assistance Process".
See related related
Issue #52 for more description.
Specific request-response aspects to be
decided include: how does
someone formally request assistance? Who is the point-person responsible to
ensure that requests are processed? Is there a task team appointed ad-hoc
for each request? What is the timeframe for answering? Etc.
The issue was discssed at the
2002-03-01 March QAWG face-to-face meeting;
decided to postpone further discussion until strawman draft of QAWG process
document is written and circulated.
Discussed at ???. Agreed that it should be reclassified from OpsGL issue, to WG-process issue (QAWG Process Document). Discussed at 30030317 telecon. This is exactly the purpose of Section 2 of the QAWG Process Document (see draft QA Process document). |
|||||||
Proposal: | |||||||
Resolution: Defined in Section 2 of the QAWG Process Document. | |||||||
54 | OpsGuide | 2002-02-12 | OpsGuide | Editorial | Closed | Daniel Dardailler | |
Title: Several structure and content issues about Proc&Ops Guideline 6. | |||||||
Description: [email]
In the email thread, a several sub-issues are raised with
Guideline 6 of Process & Operational Guidelines FPWD
(1-feb-2002, first published WD):
|
|||||||
Proposal: | |||||||
Resolution: New Frm:Ops draft breaks it up, 6.1 becomes "Note", other bits are clarified. | |||||||
55 | OpsGuide | 2002-02-14 | OpsGuide | Substantive | Closed | Lofton Henderson | Kirill Gavrylyuk |
Title: All of the checkpoint priorities in Proc&Ops should be confirmed. | |||||||
Description: The checkpoint priorities in the Proc&Ops FPWD (1-feb-2002, first published WD) have not been explicitly discussed and endorsed by the QAWG. At the 14-feb-2002 telcon, QAWG split this out of issue #39, and agreed to discuss and confirm all priorities. Discussion and further processing at 2002-03-08 QAWG teleconference, covered checkpoints through 4.1, in the 2002-02-25 Operational Guidelines, editors draft. Processing finished and minuted at: | |||||||
Proposal: | |||||||
Resolution: Checkpoints added/modified/removed as documented in above-referenced telcon minutes. | |||||||
56 | All-Framework | 2002-03-01 | All-Framework | Substantive | Closed | Mark Skall | |
Title: All guideline/checkpoint documents must have associated objective test criteria. | |||||||
Description: The issue was raised at the 2002-03-01 March QAWG face-to-face meeting. Since QA guideline/checkpoint documents are standards to be applied to the process and content of W3C test building activities, they should have test suites just like any other W3C recommendation. As stated by the originator, this issue applies to all W3C guideline/checkpoint standards, not just those of QA activity. There was discussion about what constitutes a "test suite" for a Gd/Ck specification, without any conclusion or consensus. One alternative that was offered: a checklist, together with a "Techniques" document that defines suitable ways to satisfy each checkpoint. | |||||||
Proposal: | |||||||
Resolution: Agreed in principle, details TBD. | |||||||
57 | OpsGuide | 2002-03-11 | OpsGuide | Substantive | Closed | Kirill Gavrylyuk | Kirill Gavrylyuk |
Title: Use of RFC2119 in Operational Guidelines | |||||||
Description: [email]
We aren't using words "MUST", "SHOULD", "MAY" as RFC2119
normative keywords. (Is this correct? See for example
the usage of "must", "should", etc, in
2002-02-25 Operational Guidelines, editors draft.)
How can we reflect this in
the the Terminology section? Should we change the document back to use them?
Alternatives: We can go either way:
|
|||||||
Proposal: | |||||||
Resolution: Alt b: RFC2119 keywords in checkpoint text and description text have the normative meanings for determining conformance within the checkpoint. | |||||||
58 | All-Framework | 2002-03-01 | All-Framework | Substantive | Closed | Karl Dubost | |
Title: Should W3C rate test suites? | |||||||
Description:
After some discussion in
[email]
of what icon to use in The Matrix, for indicating
the existence of test suites,
another thread continued and broaded the discussion, which raised
more fundamental issues: what is the value in the proposed QA
guideline/checkpoint ratings of test suites? And, should QAWG pursue the
rating of test materials with such rating tools?
The issue was discussed and resolved at the
2002-03-01 March QAWG face-to-face meeting
(see in those minutes, the discussion of issue #44).
The resolution acknowledges:
See related issue #44 . |
|||||||
Proposal: | |||||||
Resolution: Yes, see above. | |||||||
59 | OpsGuide | 2002-04-4 | OpsGuide | Substantive | Closed | Andrew Thackrah | Lofton Henderson |
Title: Should there be a global (W3C-standard) license for submitted test materials? | |||||||
Description: [email]
This was originally part of Issue #49,
which (initially) dealt with the two topics:
Discussed and resolved at the 20021216 telecon, more or less as suggested in email proposal. Related to this resolution, it was agreed to revise OpsGL checkpoint 5.3 ("WG must publish an acceptable TM submission license") to be more consistent. New proposed CP5.3 text requires a submission license *policy*, not a draft license itself. 2003-01-03, reopened following email from Joseph Reagle, about new W3C-wide contribution license and policy. This does not necessarily change this issue's resolution, nor the related CP5.3. However, the existence of these materials and policies has not been considered previously in this issue's discussions, and the materials need to be looked at before final closure. Discussed at Seattle face-to-face. Resolved to add a link in the discussion to W3C-wide contribution terms and form, which in turn has reference to W3C licenses. |
|||||||
Proposal: No. (See email proposal.) | |||||||
Resolution: No. | |||||||
60 | OpsGuide | 2002-04-15 | OpsGuide | Substantive | Closed | Lofton Henderson | Lofton Henderson |
Title: What are the QA commitments and responsibilities of existing Working Groups? | |||||||
Description: [email]
In the
2002-04-05 Operational Guidelines
document, the checkpoints don't clearly address existing Working Groups.
The introductory section 1.3, "Navigating..", says: "This document is applicable to all Working Groups, including those that are being rechartered or already exist. Working Groups may already be doing some of these activities and should review the document and in so far as possible incorporate principles and guidelines into their work" The first couple of guidelines -- QA responsibility, QA commitment, resource allocation, etc -- are all written for new groups. There is no mention of how an existing group should make its commitment, the TS responsibilities of a group that has published a Rec and has rechartered or is rechartering. For example:
In the first couple of Guidelines/Checkpoints, it is unclear what such a WG is recommended (required?) to do. Modify charter? Produce another sort of commitment document? Discuss and minute the commitment and resource allocation? Other? Proposal to fix the problem -- none yet, but one or more of the following options might be appropriate in Ops Guidelines:
Currently (2002-10-21), the resolution is closest to #2 (e.g., in 2002-05-15 OpsGL draft). "Responsible" proposes option #4. Discussed at 2002-10-21 telecon, option #4 agreed by QAWG. Resolution of this and resolution of issue #67 are inter-dependent. |
|||||||
Proposal: Option #4. | |||||||
Resolution: Remove "In the charter" from statement of GDs and CKs, add treatment of new/old to conditional test requirements within CKs (and discuss in Extech). | |||||||
61 | SpecGuide | 2002-04-22 | SpecGuide | Substantive | Closed | Mark Skall | |
Title: Should standard terminology be required? | |||||||
Description: [email]Guideline 4
of the Specification Guidelines (20020515 publication) discusses what needs
to conform by explicitly addressing an enumerated list of "classes of
products". This issue addresses whether or not we should require
specifications to categorize their classes of products as being one of
the items within this enumerated list or whether specification developers
are free to describe their classes of products in their own terms. In a
larger sense, the issue also addresses whether to constrain specification
developers to standard terminology to describe other things in the specification.
(Additional reference: DM paper at QAWG Workshop, April 2002.) Discussed at Tokyo face-to-face, closed with resolution: per test assertion(s) in Checkpoint 2.1, use the SpecGL terminology for product classes, when your class is one of those defined in SpecGL GL2. Else use and define your own -- list is not exhaustive. |
|||||||
Proposal: Yes, require standard terminology by stating the requirement in a lower-priority checkpoint. | |||||||
Resolution: Use the SpecGL terminology for product classes, when your class is one of those defined in SpecGL. Else use and define your own -- list is not exhaustive. | |||||||
62 | All-Framework | 2002-05-24 | All-Framework | Editorial | Closed | Karl Dubost | |
Title: What convention should QA Guidelines documents use for conformance labels? | |||||||
Description: [email]
The current method of designating levels of conformance in the
Guidelines documents of the QAWG Framework is: A, AA, AAA (good,
better, best). This is similar to the WAI method of A, double-A,
triple-A. It is thought that this might create confusion, and some
other methods are suggested in the mail thread of this issue.
Discussed and closed at 2002-05-30 telcon. |
|||||||
Proposal: | |||||||
Resolution: Leave as is -- A, AA, AAA. | |||||||
63 | SpecGuide | 2002-05-28 | SpecGuide | Substantive | Closed | Lofton Henderson | Lofton Henderson |
Title: All of the checkpoint priorities in Spec Guidelines must be confirmed. | |||||||
Description: The checkpoint priorities in the SpecGuide FPWD (15-may-2002, first published WD) have not been explicitly discussed and endorsed by the QAWG. Similar to issue #55 (which applies to OpsGuide), all checkpoint priorities must be discussed and confirmed. See related issue #64, about the use of normative keywords. Background -- the guidelines and checkpoints themselves were discussed and reviewed at QAWG teleconferences, see: | |||||||
Proposal: | |||||||
Resolution: Closed at 2002-08-14 telecon. | |||||||
64 | SpecGuide | 2002-05-28 | SpecGuide | Editorial | Closed | Lofton Henderson | Lynne Rosenthal |
Title: Use of Keywords SHOULD, SHALL, etc in SpecGuide. | |||||||
Description: [email] The use of keywords SHOULD, SHALL, MAY, etc throughout the SpecGuide document must be confirmed. Document reference: SpecGuide FPWD (15-may-2002, first published WD). Most of the normative content in that specification version is represented by imperative language in Priority 1, Priority 2, Priority 3 checkpoints. However if the resolution of OpsGL issue #57 is applied to SpecGL as well, then these keywords have the RFC2119 normative meanings when used within the text and description of checkpoints. The scope of this present issue is limited to whether the correct normative keyword (MUST or SHOULD or ...) is being used in each place. See related issue #63, regarding the whether checkpoints have the right priorities. Inter-related set of issues: #101, #80, #57, #39, #64 Discussed at Tokyo face-to-face. Conclusion: editors to make initial choice, QAWG to do per-keyword review. Although it was labelled as closed, the per-keyword review was *not* done. Subsequently decided the issue is not closed until per-keyword review is done. Another checkpoint criterion established: Every ckpt should have at least 1 normative aspect - each should have at least one MUST. At 2002-10-16 telecon, it was discussed and re-affirmed that the per-keyword review is needed (in particular, only to confirm all SHOULD occurrences?). Discussed at Seattle face-to-face. The resolution for SpecGL effectively eliminates any normative keywords except for MUST, from the conformance requirements. (This may have been an explicit resolution, but it is not minuted.) |
|||||||
Proposal: Editors to make initial choice, QAWG to do per-keyword review (prerequisite to marking issue "Closed"). | |||||||
Resolution: As reflected in SpecGL Last Call text (20030210). | |||||||
65 | Glossary | 2001-05-28 | Glossary | Editorial | Closed | Lofton Henderson | Mark Skall |
Title: Level of detail in QA Glossary definitions. | |||||||
Description: [email]
The issue was first raised in an email thread
about new glossary items. The definitions in the current
QA Glossary (2002-05-28) are fairly terse and dictionary-
like. It was suggested in the thread that definitions of complex terms and concepts
should be much richer,
similar to the level of information for example in WAI's
WCAG 1.0 Recommendation. Discussed at the
23-may-2002
telcon, without resolution.
Discussed at Montreal face-to-face (6/2002). No general agreement on level of detail. However, agreed that commentors not satisfied with detail should submit proposed wording for replacement definition, which will be accepted or treated as a QAWG issue for discussion. |
|||||||
Proposal: | |||||||
Resolution: | |||||||
66 | Glossary | 2001-05-28 | Glossary | Editorial | Closed | Karl Dubost | Karl Dubost |
Title: Should the QA Glossary be maintained as dated versions? | |||||||
Description:
The issue was raised in the
23-may-2002
telcon. Previously we decided (issue #19) that we won't have
glossaries in individual Framework documents. Therefore the Framework
documents (including those in /TR/) must refer to the central "QA
Glossary". This suggests that dated Framework documents should have a
stable reference.
Discussed and closed at 2002-05-30 telcon. |
|||||||
Proposal: Maintain dated versions in /QA/WG/YYYY/MM/ space, with dated names, with redirect from /QA/glossary.html. | |||||||
Resolution: Yes (KD and MS to propose details). | |||||||
67 | All-Framework | 2001-05-29 | All-Framework | Substantive | Closed | Lofton Henderson | Lofton Henderson |
Title: What should be the scope of the Framework "ExTech" parts? | |||||||
Description:
As originally envisioned, the Framework parts,
have a scope that includes not only real-world examples and case studies, but also specific enumeration of techniques that satisfy the checkpoints. In the latter sense, they contain something like alternative "verdict criteria" for each Checkpoint -- if the target under consideration does one of the techniques, then it satisfies the checkpoint. Presently, the Ops-ExTech is a collection of case studies that illustrate how several cases relate to the operational checkpoints. The issue is: should the scope be limited to examples and case studies, or should the ExTech parts be developed per the original scope? Options could include:
Techniques will raise a number of additional issues: must (can) the enumeration be exhaustive? how precise (i.e., verifiable) can these be in diverse operational, specification, and test environments? how long would it take to develop exhaustive, verifiable techniques and do we have the resources? etc? Background/Reference. See the Abstract of "HTML Techniques for Web Content Accessibility Guidelines 1.0". In the WAI model, the techniques are intended to "help authors of Web content who wish to claim conformance". It further disclaims that "while the techniques in this document should help people author HTML that conforms [to WCAG 1.0], these techniques are neither guarantees of conformance nor the only way an author might produce conforming content." Discussed and closed at Montreal face-to-face (6/2002). Basically option #2 above. QAWG members who review OpsGL and SpecGL will make contributions. The Extech documents will include both per-checkpoint case study material, and per-checkpoint "virtual example" (techniques) material. See related issue #56 |
|||||||
Proposal: | |||||||
Resolution: The Extech documents will include both per-checkpoint case study material, and per-checkpoint "virtual example" (techniques) material. | |||||||
68 | All-Framework | 2001-05-29 | All-Framework | Substantive | Closed | Daniel Dardailler | Lofton Henderson |
Title: What should be the status of the "ExTech" parts, and how should they relate to their "Guidelines" parts? | |||||||
Description:
The issue was raised in the
23-may-2002
telcon. It relates to issue #67, and probably
depends on that resolution. As originally planned, the three ExTech parts would
be separate standlone Technical Reports (WD at least),
in separate /TR/ directories from their
Guidelines parts. It has been suggested that these could alternatively be:
In all configurations (including current /TR/), the two parts are closely cross-linked. The "informal" option probably saves staff resources. (Related issue not yet recorded -- "how to maintain" -- can an open-edit or open-contribute scheme be used in /QA/WG/?) Discussed at Montreal face-to-face (6/2002), without resolution. Discussed again at 20020918 telecon. Followup mail presented proposal for closure, which most closely resembles #2 above. A key point is that the ET parts have no normative content -- they are completely informative, showing how the checkpoints of the GL part "might be satisfied." By keeping them in /QA/WG/ for now, they can be changed and improved more frequently than the GL parts in /TR/. The GL parts in /TR/ would be linked to redirects in /QA/WG/ which are updated to point to the best current ET version. The ultimate disposition -- /TR/ or /QA/WG/ -- can be revisited when the GL parts (and the ET parts) stabilize more. Discussed at Tokyo face-to-face, closed with resolution: as described in the previous paragraph, with some refinements.
|
|||||||
Proposal: As described above and in email proposal | |||||||
Resolution: Extech remain in /QA/WG/ space, but linked from /TR/, and eventually (when "stabilized") be moved to /TR/ as W3C Note. Details above. | |||||||
69 | SpecGuide | 2002-05-29 | SpecGuide | Substantive | Closed | Dan Connolly | Lofton Henderson |
Title: Should Spec Guidelines discourage "flavors of conformance"? | |||||||
Description: [email]
In the FPWD (2002-05-15) of Specification Guidelines, there is no
language in
"Guideline 3. Specify flavors of conformance." indicating that
options/levels/flavors are bad. Originator proposes that there should be.
It was discussed some
in the email thread, without resolution.
Discussed at 2002-05-30 telcon. Discussed at Montreal face-to-face (6/2002), without definitive resolution. However, there is general agreement that excessive variability hurts interoperability, and this will at least be reflected with warnings in new versions of SpecGuide (see for example August 2002 published SpecGuide WD). The remaining problem is: is there a quantified or objectively verifiable variability constraint that should be placed on specifications? It was agreed to move forward by reorganizing SpecGuide according to DM email proposal. This in turn will be the basis for further input from WGs and continued discussion on the IG list. A specific "quantified" proposal for a checkpoint to limit to 3 dimensions was discussed but not accepted -- too inflexible to fit all the sorts of W3C specs. See related issue #70 Discussed at Tokyo face-to-face. Summary follows.
|
|||||||
Proposal: Originator proposes that minimizing (esp. zero) flavors should be a design goal for all W3C specs. | |||||||
Resolution: Continue to warn about excessive variability, require justification of DoV used, don't attempt (in SpecGL) to further quantify it. | |||||||
70 | SpecGuide | 2001-05-29 | SpecGuide | Substantive | Closed | Lofton Henderson | Lofton Henderson |
Title: Spec Guidelines needs a specific enumeration of "flavors of conformance". | |||||||
Description: [email]
The email thread indicates that there are widely differing
understandings of what is meant by "flavors of conformance" in
"Guideline 3. Specify flavors of conformance." of the FPWD
(2002-05-15) version of Specification Guidelines. Before it can
be argued whether flavors are good or bad, the statement of the
guideline needs to be clarified. It needs a specific enumeration of all of the
sorts of flavors to which it applies. So far, only "Strict
Conformance", "Unconditional Conformance", and "Conditional Conformance"
are mentioned.
Discussed at 2002-05-30 telcon. See DM email proposal. This proposal was discussed Montreal face-to-face (6/2002). It was agreed to reorganize the SpecGuide document accordingly, in which case the "flavors" go away, a complete enumeration of "dimensions of variability" affecting conformance in specs is implemented, and "flavors of confomance" GL becomes a "conformance policy" GL. See related issue #69 |
|||||||
Proposal: | |||||||
Resolution: As decided at Montreal (see above). | |||||||
71 | All-Framework | 2001-06-13 | All-Framework | Substantive | Closed | Dominique Hazaël-Massieux | |
Title: Should WG conformance with QA Guidelines documents be mandatory? | |||||||
Description: [email]
Opened and discussed again at
Montreal face-to-face, closed
according to subsequent
email proposal.
QAWG consensus is that it
should be mandatory for WGs to:
Whether or not these mandatory requirements are documented in the Process Document was not specifically addressed. The mechanism is still unresolved. Another email contribution fully explores options about "how" to make it mandatory. The post-Montreal email proposal suggested simply: an addendum to Process Document requiring Ops GL conformance; and amendment to pubrules requiring Spec GL conformance. See related issue #16. |
|||||||
Proposal: See above. | |||||||
Resolution: Yes, mandatory use and mandatory conformance, according to the above proposal. | |||||||
72 | SpecGuide | 2002-07-10 | SpecGuide | Substantive | Closed | Lofton Henderson | |
Title: What is the definition of "normative use case"? | |||||||
Description: [email]
Opened at
2002-07-10 telecon.
Two framework documents mention "normative use case":
There are two problems: "use case" means widely differing things to different people, ranging from careful prose description to rigorous coded scenario (SpecGL says "formally defined terms"); and "normative" implies that some testable requirements are placed on some target. The first problem (definition) begs for clarification and examples (see checkpoint 1.3!). And the second problem (normativeness) raises several questions. What is the target of such a requirement: an instance or implemenation of some class of product? or, the specification itself in which the use case is found? And finally what is the nature of the testable requirement? |
|||||||
Proposal: Proposal above. Plus proposals from LRAT | |||||||
Resolution: Definitions of use case and user scenario adapted from UML. GL1 and CK1.1 reworded, made informative by default. See 2002-08-26 SpecGuide. | |||||||
73 | SpecGuide | 2002-08-22 | SpecGuide | Substantive | Closed | Dominique Hazaël-Massieux | |
Title: Should SpecGL require atomicity of modules? | |||||||
Description: [email] A number of W3C specifications (e.g., SMIL 2.0, XHTML Modularization) impose constraints that modules be atomic (indivisible) in order to qualify for some conformance designations. It was proposed that this should be made into a checkpoint of Guideline 4 (GL4) of SpecGuide. It was counter-proposed that this was premature, and also that the current "clean design" wording should be removed. Pending discussion and resolution of the issue, the 2002-08-26 SpecGuide reflects a middle option: GL4 describes the attributes of atomicity, present examples, and requires (in a checkpoint) that specifications that chose modularization must address the atomicity question. Discussed at Tokyo face-to-face, closed with resolution: leave it as it is in checkpoint 4.3 (2002-08-26 SpecGL version) - do not require atomicity, but if modules are used, then the spec must address atomicity. Move atomicity definition from GL verbiage into checkpoint, move other discussion (recommendations) into ExTech. |
|||||||
Proposal: | |||||||
Resolution: Don't require atomicity, but require that modularized specs address atomiticy. | |||||||
74 | SpecGuide | 2002-08-22 | SpecGuide | Substantive | Closed | Lofton Henderson | |
Title: What is the scope of the minimality of level 1 in a leveled technology? | |||||||
Description: [email] In email threads such as the above, there is discussion that level 1 in a leveled technology is the core or minimum. Within the levels dimension of variability, that is agreed to be true. It has been suggested (see the email, see @@minutes? spoken in telecon but apparently not recorded or minuted@@) that level 1 should be minimal across all dimensions. I.e., profiles would not subset level 1, modules would not subdivide level 1. There are counter-examples in W3C practice, and no examples have been put forward yet. Still there is the issue: in principle, is this a spec (or technology, more precisely) architectural principle that is agreed; and if so, should SpecGuide encourage or require it? Related issues: issue #75 and issue #76. Discussed at Tokyo face-to-face, closed with resolution: Do not try to define/mandate a "correct" relationships amongst DoV in SpecGL, discuss the possibilities in Spec Extech. |
|||||||
Proposal: | |||||||
Resolution: Do not try to define/mandate a "correct" relationships amongst DoV in SpecGL, discuss the possibilities in Spec Extech. | |||||||
75 | SpecGuide | 2002-08-22 | SpecGuide | Substantive | Closed | David Marston | |
Title: What is the relationship of modules and levels? | |||||||
Description: [email]
Also discussed in the
2002-07-24 telecon. Modules and levels are two ways to divide a technology. When both are present in a specification, there is the question of whether one subdivides the other. I.e., levels subdivide modules, or do modules aggregate together to build levels? There are no examples of the former, although a clean design case can be postulated. It is unclear whether the latter is found in W3C specifications (a careful look is needed at the modularized specifications), although it can be postulated how such an architecture might evolve with "historical levels". It has been observed about the latter, that profiles would be a better choice than levels. The issue that is raised is: should SpecGuide take a position on the goodness or badness of the two design possibilities. I.e., is one relationship better than the other and to be encouraged or required? (Other alternatives: no, neither is preferable; or, these two dimensions should not be used together.) Related issues: issue #74 and issue #76. Discussed at Tokyo face-to-face, closed with resolution: Do not try to define/mandate a "correct" relationships amongst DoV in SpecGL, discuss the possibilities in Spec Extech. |
|||||||
Proposal: | |||||||
Resolution: Do not try to define/mandate a "correct" relationships amongst DoV in SpecGL, discuss the possibilities in Spec Extech. | |||||||
76 | SpecGuide | 2002-08-22 | SpecGuide | Substantive | Closed | Lofton Henderson | |
Title: What is the relationship of levels and profiles? | |||||||
Description: [email] (none) This issue is raised, but not yet resolved, whether SpecGuide should promote requirements about how profiles and functional levels relate, i.e., should particular architectures be encouraged or discouraged, prescribed or proscribed? The (current) 2002-08-26 SpecGuide describes some possibilities, without taking position. From Guideline 3, "A profile can require a particular level in its entirety. A profile can also be defined as a subset of a level or levels. Hierarchy is an essential attribute of levels. While it is not an essential attribute of profiles, nevertheless profiles can be hierarchical in the sense that Profile B can be defined as all of Profile A plus additional capabilities." From Checkpoint 3.5, "levels [...] could depend on profiles." Related issues: issue #74 and issue #75. Discussed at Tokyo face-to-face, closed with resolution: Do not try to define/mandate a "correct" relationships amongst DoV in SpecGL, discuss the possibilities in Spec Extech. |
|||||||
Proposal: | |||||||
Resolution: Do not try to define/mandate a "correct" relationships amongst DoV in SpecGL, discuss the possibilities in Spec Extech. | |||||||
77 | SpecGuide | 2002-08-22 | SpecGuide | Substantive | Closed | Lofton Henderson | Lynne Rosenthal |
Title: Should SpecGL have any different requirements for older specifications? | |||||||
Description: [email] First raised in QAWG telecon of 2002-08-07, the issue concerns whether or not SpecGuide should, under any circumstances, give special consideration to legacy specifications in any checkpoints -- specs that precede the publication of SpecGuide. The QAWG decided that the general answer is "no" -- SpecGuide is being written for new specifications. That notwithstanding, QAWG also believes that the wording of checkpoints should not arbitrarily or unnecessarily exclude legacy specifications which meet the checkpoint in spirit. (See last paragraph of section 1.2 of 2002-08-26 SpecGuide.) The specific case in point here concerns a previous statement that existed in many of the dimension of variability (DoV) checkpoints, that required specifications to document whether or NOT they used a DoV. The telecon discussion shows problems with the negative disclaimer requirement, not only for legacy specifications but going forward if new DoV are added. The interim solution is a new priority-2 checkpoint, 10.5, which collects all negative disclaimers (so legacy specs could still be A-conforming), and the various DoV checkpoints (3.1, 4.1, 7.1, etc) have dropped the NOT requirements. This legacy exemption needs to be discussed and endorsed, or else eliminated. If kept, then the style and structure of 3.1, 4.1, etc need to be looked at. The issue is raised again in email thread about checkpoint 10.5 in the 20020826 published WD of SpecGL. Should documenting DoV for existing specs be in the SpecGL? Although useful, this information should be in the Examples & Techniques or perhaps elsewhere. Is a target specification required to address all dimensions of DoV, even those it doesn't use? There is no need to establish a dimension that is not being used - that violates 'minimal set' requirement. Discussed at Tokyo face-to-face, closed with resolution: do not try to accommodate past work, i.e., do not try to structure the checkpoints so that legacy documents might pass easier. Goal is to make new specifications better. Change front matter -- to indicate that these guidelines target new specs, although existing specs can use the guidelines to see how they do. Remove CK10.5 and related stuff. |
|||||||
Proposal: | |||||||
Resolution: Do not try to accommodate past work, focus is to make new specifications better. | |||||||
78 | SpecGuide | 2002-08-22 | SpecGuide | Substantive | Closed | Lynne Rosenthal | Lynne Rosenthal |
Title: Should Guideline 8 have a checkpoint proscribing implementation dependent features?" | |||||||
Description:
[email] (see sub-thread at the end of message).
The 2002-08-26 SpecGuide discusses (in Guideline 8) discretionary items. There are flavors: discretionary choices (implementation choses from 2 or more well defined behaviors); optional features (impl choses whether or not to implement a well-defined feature); implementation-dependent features (what the implementation does is completely free and undefined). It is suggested that the last one -- implementation-dependent features - is considerably worse for interoperability than the others and deserves a checkpoint of its own (either ban them at some priority level, or at least require that specifications address the interoperability impact of allowing them.) Discussed at Tokyo face-to-face, closed with resolution: add a new checkpoint, or new MUST requirement within CK8.2, "MUST document interoperability impact of any implementation dependent features". A checkpoint to ban implementation dependent features at P2 or P3 was discussed inconclusively -- not accepted at this time. |
|||||||
Proposal: | |||||||
Resolution: Add a new checkpoint, or new MUST requirement within CK8.2, "MUST document interoperability impact of any implementation dependent features". | |||||||
79 | TestGuide | 2002-09-03 | TestGuide | Substantive | Resolved | Alex Rousskov | |
Title: Should Test Guidelines address how to write a good test suite? | |||||||
Description:
[email] The 2002-07-01 TestGuide draft (1st discussion draft for QAWG telecon input) addresses:
Because the guidelines in the second category are often subjective and limit developer options, originator suggests moving them to a separate informational document if not completely eliminating them. Discussed at , without clear resolution. However, it was decided that the next TestGL draft (Oct or Nov 2002) would at least disambiguate in each checkpoint, what are required results or characteristics in a TS (test suite) -- "deliverables" -- and what are process aspects. With that clarification, it will be possible to decide whether to keep the process-oriented bits, remove them, or migrate them to the TestET (Examples & Techniques) document, when the latter is written. Discussed at Boston face-to-face. Agreed that TestGL should describe the results -- i.e., the test materials -- not the process of producing them. TestGL "Scope" should clarify this. |
|||||||
Proposal: Originator proposes removing "how to" guidelines from the TestGuide document. | |||||||
Resolution: Agreed. TestGL will describe the test materials, not the process. TestGL "Scope" will clarify this [tbd]. | |||||||
80 | SpecGuide | 2002-09-24 | SpecGuide | Editorial | Closed | Alex Rousskov | Lynne Rosenthal |
Title: RFC 2119 Keywords are used inconsistently in the SpecGL. | |||||||
Description:
[email],
[origin] Reference: 20020826 Spec Guidelines draft (WD) The document does not use MUST, SHOULD or MAY keywords as defined and recommended in RFC2119 -- upper case. This is how the words are presented in section 1.4, "Terminology", however there are numerous usages in SpecGL which are lower case, capitalized, etc. Alternatives:
The related issue #64 and (OpsGL) issue #57 deal with the substantive aspects of RFC2119 usage. Inter-related set of issues: #101, #80, #57, #39, #64 Discussed at Tokyo face-to-face, closed with resolution: use uppercase in all occurrences that have normative meaning. Note this in "Terminology" subsections. Editors Action Item: apply this to all Framework Guideline documents. |
|||||||
Proposal: Upper-case all occurrences. | |||||||
Resolution: Upper-case all occurrences, where they are used with RFC2119 normative meaning. Note this in "Terminology" subsections. | |||||||
81 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Definition of specification testability | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) There is confusion and disagreement as to the meaning of specification testability. The current definition states that there needs to be a finite cost-effective process that checks that "the requirement is met". Discussion regarding finite and testing behavioral specifications; what does 'check that requirements are met' mean. As pointed out in email thread, current definition was taken from an IEEE definition (@@reference?@@). Discussed at Tokyo face-to-face, closed with resolution: Change "the requirement" to "all requirements", take out "finite cost effective", and split into two sentences: one defines single testable requirement, and the other uses that to defines testable specification. |
|||||||
Proposal: (Numerous conflicting proposals in email thread.) | |||||||
Resolution: Change "the requirement" to "all requirements", take out "finite cost effective", and split into two sentences: one defines single testable requirement, and the other uses that to define testable specification. | |||||||
82 | SpecGuide | 2002-09-24 | SpecGuide | Editorial | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Re-title Guideline 1, replacing "use cases" with "scope". | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) "Scope" would better define the guideline objective and can be normative, whereas use cases are complementary and should not be normative. Does the current wording reflect the entire guideline and convey the appropriate message or will it be misconstrued? Scope is covered by checkpoint 1.1 Discussed at Tokyo face-to-face, closed with resolution: change Guideline 1 to "Define Scope." |
|||||||
Proposal: Originator suggests to rename GL1, "Define scope." | |||||||
Resolution: Change Guideline 1 to "Define Scope." | |||||||
83 | SpecGuide | 2002-09-24 | SpecGuide | Editorial | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Clean up subjective and imprecise wording. | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) Subjective qualifiers such as 'clearly' need to be removed. These are not testable terms. Note that there is a lurking issue here -- the phrasing of the checkpoint is merely a "title" for the checkpoint, and the normative requirements are carried in (RFC2119) language within the checkpoint description. This is the WAI model, and implied by the resolution of issue #57. Additional email thread discussion. Discussed at Tokyo face-to-face, closed with resolution: agree that subjective imprecise language should not be in the checkpoint titles, and must not be in any test assertion statements (requirements) within the checkpoints. |
|||||||
Proposal: Remove subjective, imprecise qualifiers from checkpoint titles, as well as from any normative language within the checkpoint. | |||||||
Resolution: Remove subjective, imprecise qualifiers from checkpoint titles, as well as from any normative language within the checkpoint. | |||||||
84 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Should examples and user scenarios be lower priorities? | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) It was suggested that user scenarios and examples don't belong in a specification since the more non-normative scenarios/examples the more chance for conflict with a normative statement and eventual implementation bugs. Specifications should be precise and succinct and illustrations don't belong unless they are normative. Suggested lowering priorities to 3. Discussed at Tokyo face-to-face, closed with resolution: disagree with originators proposal. Some rationale follows. Examples and user scenarios add to the understanding of a spec. Those reading the spec will not necessarily misinterpret. SpecGL shouldn't say that examples have to be embedded in the spec -- they can be provided outside the spec, but published concurrently and linked from Spec. Examples are understood to be informative. That can be emphasized under the ckpt related to make sure it is clear what is normative/informative (CK13.2). |
|||||||
Proposal: Originator suggests priority 3 (if the checkpoints 1.2 & 1.3 are kept). | |||||||
Resolution: Keep checkpoints 1.2 and 1.3 at current priorities, investigate enhancing checkpoint 13.2. | |||||||
85 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Should every test assertion be covered by a example? | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) Why is an example needed for each test assertion? Satisfying this requirement will make some specifications (e.g., protocol specifications) huge without added benefit. Not every test assertion needs an example, nor would it be helpful. How many examples are enough? Counter argument pointed out that one example could cover many test assertions. Discussed at Tokyo face-to-face, closed with resolution: agree with originators proposal, remove checkpoint 1.4. Further discussion. An example covering every TA is neither needed nor desirable in SpecGL. What are positive benefits of an example for every TA? (Possible answer - clarifies the meaning.) But copious examples do not need to be in spec. Every feature should have an example, not every TA (need to define "feature"). Remove 1.4. Can SpecGL or Spec Extech provide informative guidance of how to determine how many examples? Remove "The more numerous the examples, the better." May want to 'save' some of the words in 1.4 for ExTech. |
|||||||
Proposal: Originator suggests eliminating the checkpoint (1.4). | |||||||
Resolution: Remove checkpoint 1.4. Investigate possibility of some informative guidance (esp. in Spec Extech) for the "right number" of examples. | |||||||
86 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Checkpoint 1.5 should be lowered to priority 3. | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) Checkpoint 1.5: "Include an interpretation section (priority 2)". If somebody doesn't understand the formal descriptions, they should not rely on the specification to help them. A short informal introduction section is (@@ed error, remainder of statement lost@@). Discussed at Tokyo face-to-face, closed with resolution: disagree with originators proposal. Additional issue discussion summary follows. (@@Ed note. Fix editorial truncation above.@@) Discussion wrestled with, "What is meant by interpretation section"? Resolution: re-title the checkpoint as, "Include examples for formal descriptions." Add a "not applicable" exemption for those specs that don't have formal descriptions (i.e.., if you don't have formal description, this ckpt does not apply). Leave as P2. Explain or enumerate what is a formal description - syntactic descriptions, eg. IDL, BNF, ... SpecGL lead editor (DHM) to work on improvement of statement of checkpoint. |
|||||||
Proposal: Originator proposed to lower ckpt 1.5 priority to 3. | |||||||
Resolution: Re-title as "Include examples for formal descriptions", leave at P2, make other improvements as indicated. | |||||||
87 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Should the SpecGL elaborate on DoV definition and discussion. | |||||||
Description:
[email],
[origin] Reference: 20020826 Spec Guidelines draft (WD) Is the discussion on DoV excessive? Is there a benefit from talking about all the well-known forms of variability? It was proposed that the DoV discussion is not needed to define a good specification. Advising that less variability is good, may be enough. Prior WG discussions resulted in agreement to include a discussion on DoV, which includes more than just degrees of conformance, but other sorts of variations. The SpecGL should not mention DoV model in any normative way and should point to a DoV tutorial or white paper. Numerous alternatives seem implied in the email thread. Some alternatives:
Discussed at Tokyo face-to-face, closed with resolution: similar to #2 above, and
|
|||||||
Proposal: Originator proposal is closest to alt. 1 above. | |||||||
Resolution: Keep concept of DoV as organizing principle for SpecGL, simplify and clean up as indicated. | |||||||
88 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: SpecGL objective and value. | |||||||
Description:
[email],
[origin] Reference: 20020826 Spec Guidelines draft (WD) Should the SpecGL be a guide that applies to a given spec and not the process of writing specs. Is the SpecGL's purpose to help editors (and WGs) focus on key issues and make explicit choices? Is it an "instructions how to write a good spec"? Everything in the form of "the authors should consider or be aware of this or that" is an instruction. Suggested that 'useful and helpful instructions to spec editors should be outside the SpecGL scope -- apparently regardless of whether it is stated as process guidance or requirements on result (i.e., the specification). In counter-argument, it is asserted that current SpecGL does not in fact address process (despite possible misleading implications of "imperative" voice in checkpoint statements). See additional comments from DM. Discussed at Tokyo face-to-face, closed with resolution: it is our intention that SpecGL should be about the characteristics of good specifications and not how to write specs. SpecGL is about result of writing the spec, not the process. |
|||||||
Proposal: Originator suggests to remove material whose motivation is "useful and helpful guidance" to spec authors. | |||||||
Resolution: SpecGL address the characteristics of good specifications, not how to write them. | |||||||
89 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Should SpecGL conform to SpecGL? | |||||||
Description:
[email],
[origin] Reference: 20020826 Spec Guidelines draft (WD) If the SpecGL is a spec, it should be about other specs (the subject is a spec), not about ways of writing other specs (the subject is the author of a spec or the writing process). The SpecGL should conform to itself. Discussed at Tokyo face-to-face, closed with resolution:
Higher degree of self-conformance (AA or AAA) will be discussed and decided in the future. |
|||||||
Proposal: (Unclear, for this extracted issue.) | |||||||
Resolution: SpecGL must be self-conformant, A-conforming by Last Call. | |||||||
90 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Remove guideline on deprecation (GL6). | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) It is suggested that Guideline 6 is unnecessary, since it is already covered by other guidelines, specifically, the spec must define conformance. Disagreement with that assertion, in the email thread -- deprecated features have been inconsistently dealt with within the WGs. Was included as in the SpecGL due to a previous email thread. Discussed at Tokyo face-to-face, closed with resolution: disagree with originator proposal. This was previously discussed via an email thread (@@reference?@@). GL6 was included as a result of consensus by the QAWG. No compelling new arguments to reverse that consensus. |
|||||||
Proposal: Originator proposes removing GL6. | |||||||
Resolution: GL6 (about deprecation) will be kept. | |||||||
91 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Why should discretionary items be handled consistently within an implementation (CK8.4)? | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) Originator: This shouldn't be required. Is browser X that is user-configured to do FOO the same implementation as browser X that is user-configured to do BAR? Can browser display cached documents differently from first-hand responses? Counter-argument. The idea is to lower the bad effect of discretionary items. Probably need to rewrite Checkpoint. Additional clarifying email thread from the author indicates that the checkpoint as stated does not capture the intent, and that "items" in the checkpoint statement should be "choices" (one of three kinds of "items"). Discussed at Tokyo face-to-face, closed with resolution: replace "items" with "choices" in statement of checkpoint; and, implementations must do the same things in the same conditions. Further details of Tokyo discussion: Checkpoint addresses specifically discretionary choices - e.g., choice one of a,b,c - not discretionary items in general. Users should be able to count on getting the same thing under same conditions. If conditions have changed, then maybe you're not going to get same behavior - may not always be possible to have the same conditions - e.g., cache changes, network changes (recognize that requirement might be useless in some environments.) There is a possible inter-related issue which is not addressed by this resolution: consistent treatment of groups of discretionary choices. If "A or B" is a discretionary choice applicable to several different circumstances within the specification -- e.g., two possible responses to a collection of several different error conditions -- then should specifications treat these consistently as a group. |
|||||||
Proposal: Originator proposes that checkpoint 8.4 be removed. | |||||||
Resolution: Keep CK8.4; replace "items" with "choices" in statement of checkpoint; and, specifications must state that given identical conditions, the effect of a discretionary choice is consistent within a single implementation. | |||||||
92 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Remove guideline on Extensions (GL9). | |||||||
Description:
[email],
[email2] Reference: 20020826 Spec Guidelines draft (WD) It was suggested that Guideline 9 be removed since if a spec mentions an extension, it MUST document compliant behavior just like any non-extension. Alternative suggested: a) define compliant behavior for every conformance object, b) and, define compliant behavior for every non-conforming object. May need to define what is meant by 'extension'. Extension is being addressed since it isn't always obvious to all spec writers what the consequences to interoperability are, nor some of the various ways to allow extensions. Discussed at (2002-10-21 telecon. Resolution -- keep the guideline -- extensions are sufficiently harmful to interoperability that SpecGL should specifically address them. (There will be reworking of the wording of the GL and some of checkpoints.) |
|||||||
Proposal: Originator proposes that GL9 be removed. | |||||||
Resolution: Keep GL9, with reworking of the wording of the GL and some of checkpoints. | |||||||
93 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Why register extensions? | |||||||
Description:
[email],
[email2] Reference: 20020826 Spec Guidelines draft (WD) Originator suggested that ckpt 9.5 is unnecessary and introduced too much overhead. Responders gave counter-examples: XSLT, CGM. Plus suggestions: Publication/registration of extensions important for some specs and less critical for others as long as there are mechanisms that can mitigate the extensions. The SpecGL should distinguish between published and registered. Currently SpecGL instructs authors how to make some extension work: "register your extension and you are covered". It should require that extension mechanisms maintain interoperability among parties that may not be aware of each extension definition. It should address the core of the problem and not require a specific solution. Further discussed at 2002-10-21 telecon. Tentative resolution -- keep the checkpoint, but LR and MS to redraft the checkpoint for further discussion (possibly incorporating refinements discussed above and in telecon). Discussed at Seattle face-to-face. The requirement to "register" is dropped, and a requirement to "publish" remains. |
|||||||
Proposal: Originator proposes that checkpoint 9.5 be removed. | |||||||
Resolution: Drop "register", in favor of "publish". | |||||||
94 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Why identify unused DoV (ckpt 10.5)? | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) This is overkill. All that is necessary is to define what is in scope, everything else is out of scope. Why would you want authors to define non-scope and non-conformance. The conformance section is becoming too verbose. Discussed at Tokyo face-to-face, closed with resolution: agree with proposal, remove checkpoint 10.5. Additional notes. There will also be impacts on checkpoints X.1 which have some verbiage about negative disclaimers and reference to CK10.5. Some additional discussion about a possible new checkpoint to require that all used DOV are documented, in one place in the conformance clause. |
|||||||
Proposal: Originator proposes that CK10.5 be removed. | |||||||
Resolution: Remove checkpoint 10.5. | |||||||
95 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Is it practical to "provide specific wording of conformance claims"? | |||||||
Description:
[email],
[origin] Reference: 20020826 Spec Guidelines draft (WD) Ckpt 11.2 -- provide specific wording of conformance claim -- is questioned as to its practicality and verbosity. Counter-argument: What is required by this ckpt is routine and sensible for software components. Should this be required for content or other target objects? Discussed at Tokyo face-to-face, closed with resolution: In checkpoint, list the minimum things that need to be included in a claim, and require that conformance claims contain this minimal list. Checkpoint can't provide and require all possible wording. Additional treatment and recommendations will be in ExTech. The other aspect/question of the issue: What about conformance icons? If you put a conformance icon on a page does that equal all the verbage of a conformance claim, either implicitly or explicitly (e.g., by link)? Resolution about icons: If there exists an icon to indicate conformance, then the explanation of what that icon represents (i.e., the conformance claim) must be available somewhere. The conformance claim would be written such that it would apply to the content (or whatever) that is using the icon. For example: The use of this icon X, indicates conformance of the content upon which this icon resides to Specification XYX, version 000, DDMMYYYY at conformance level P1." (Note. The later implies a new MUST or SHOULD assertion in the checkpoint.) Action item given to LR and DHM to rewrite checkpoint. |
|||||||
Proposal: None yet. | |||||||
Resolution: In checkpoint, list the minimum things that need to be included in a claim, and require that conformance claims contain this minimal list; and, there must be an association (linkage) between any icon and corresponding conformance claim(s). | |||||||
96 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Why require Implementation Conformance Statement? | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) Although nice to have, an ICS has nothing to do with the spec. Remove guideline (GL12). Counter-argument: the SpecGL are not just about the specification understood as a document but about the abstract specification as described by Al Gilman. Discussed at Tokyo face-to-face, closed with resolution: disagree with originator proposal. Priority 3 (P3) equates roughly to "nice to have" (useful/beneficial), and the checkpoints of GL12 are P3. Some specs do have ICS. Examples: checklists of WAI and QA guidelines are ICS. Further discussed at (2002-10-21 telecon, in email, and (2002-10-28 telecon (@@DRAFT@@), with the result that the two checkpoints (in GL12, 2002-10-25 SpecGL editors draft) will be refined somewhat: specs must have ICS *if applicable*; and specs must require ICS as part of conformance claim (either the spec's ICS, or another that contains at least as much information -- i.e., a superset of the spec's model ICS is permissible.) |
|||||||
Proposal: Originator proposes that GL12 be removed. | |||||||
Resolution: Keep GL12 and keep its checkpoints at priority 3. | |||||||
97 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Distinguish normative and informative text. | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) This (checkpoint 13.2) should be priority 1. A spec is no good if normative and informative statements are indistinguishable. Discussed at Tokyo face-to-face, closed with resolution: agree with originator proposal. Change CK13.2 to priority 1. |
|||||||
Proposal: Raise checkpoint 13.2 to priority 1. | |||||||
Resolution: Raise checkpoint 13.2 to priority 1. | |||||||
98 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Should test assertions be included in a spec? | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) Test assertions should not be included since there should be a single normative definition and nothing else. Why not publish test assertions outside the spec? Counter-argument(s): By including test assertions in a spec, they are normative. Question(s): May need to clarify the rationale, as well as the meaning of "include". Can "include" encompass identification of test assertions in-line, by markup or other reference, so that they are not duplicated? Discussed at Tokyo face-to-face, closed with resolution: it is is mandatory to provide test assertions (TA), and the provided TA are normative except in case of conflict with the text of the specification which shall take precedence. (TestGL must be rewritten to reflect the resolution). The resolution does not limit some inter-related details: are the test assertions internal to the specification (either separate list, or in-line markup); or are they a referenced external (normative) document. How the TA map to the specification -- i.e., the relationship between listed test assertions and bits of normative text in the specification -- was also discussed but is not addressed or limited by this resolution. |
|||||||
Proposal: Originator suggests that GL15 be removed or modified. | |||||||
Resolution: It is is mandatory to provide test assertions (TA), and the provided TA are normative except in case of conflict with the text of the specification which shall take precedence. | |||||||
99 | SpecGuide | 2002-09-24 | SpecGuide | Substantive | Closed | Alex Rousskov | Lynne Rosenthal |
Title: Scope of definition of test assertion. | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) Does the definition preclude expressing a test assertion in a formal language without words and phrases rather than in sentences of a human language? It was suggested in the email thread to match the definition with the one used in the glossary. Counter-suggestion: see if the SpecGL definition can be fixed to cover the question/objection, because issue #19 (initially) resolved that in-document "Definitions" sections of the Framework documents could contain definitions that differ from but are compatible with the central QA Glossary.) Discussed at Tokyo face-to-face (DRAFT!), tentatively closed with resolution: remove TA definition from SpecGL. Reopened per email request. Discussion in 20021216 teleconference indicates that there is significant disagreement and/or misunderstanding about the definition of test assertion, or what comprises test assertions. Followup email enumerated, for example, the submitter's view of the test assertions that correspond to the SpecGL "to fulfill" criteria. Discussed at Seattle face-to-face. Resolved that the definition should be in the central QA Glossary, not in the SpecGL Definitions section, and the definition in the QA Glossary should be the one that is in the verbiage of SpecGL Guideline 14. |
|||||||
Proposal: Several competing. | |||||||
Resolution: As just described (Seattle resolution). | |||||||
100 | SpecGuide | 2002-10-01 | SpecGuide | Editorial | Closed | Lynne Rosenthal | Lynne Rosenthal |
Title: "Non-normative" is undefined, and used in place of the defined "informative". | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) Within the SpecGL, Section 6 References has the additional qualifier, "(non-normative)". Although logic tells us that non-normative is the same as informative, is there a reason that we did not label this section "informative". The reason for this question, is that we don't define non-normative but define informative. Additionally, Checkpoint 13.2 states: Distinguish normative and informative text and Checkpoint 13.4 states: use consistent terminology. Discussed at Tokyo face-to-face, closed with resolution: agree with originator proposal. Change label to "Informative". |
|||||||
Proposal: Replace non-normative with informative. | |||||||
Resolution: Replace non-normative with informative. | |||||||
101 | All-Framework | 2002-10-01 | All-Framework | Substantive | Closed | Lynne Rosenthal | Lynne Rosenthal |
Title: Should the GL documents use conformance levels instead of prioritized checkpoints? | |||||||
Description:
[email] Reference: 20020826 Spec Guidelines draft (WD) The WCAG WG, in their recently published Working Draft of Web Content Accessibility Guidelines 2.0 [WCAG20] has changed their conformance criteria - from using prioritized checkpoints to using Levels of satisfaction of checkpoints. (Minimum, Level 2, Level 3). The QAWG model for its Guidelines documents was adapted from the original WAI models (WCAG, ATAG, and UAAG). As summarized in a recent email message the QAWG model is:
Inter-related set of issues: #101, #80, #57, #39, #64 Discussed at Tokyo face-to-face, without closure (too premature to decide). OT to investigate. Discussed at Seattle face-to-face. The current QAWG model is considered to be a good match for QA Guidelines document requirements. Resolved that there doesn't seem to be an obvious reason to move to this new scheme. (Additional reference: Requirements for WCAG 2.0 Checklists and Techniques |
|||||||
Proposal: (Investigate WCAG's rationale, including talk to WCAG.) | |||||||
Resolution: No, it does not appear to be as suitable as the present model, for QA Framework application. | |||||||
102 | SpecGuide | 2002-11-04 | SpecGuide | Substantive | Closed | Lofton Henderson | Lynne Rosenthal |
Title: Scope and content of the "conformance policy" guideline (GL3). | |||||||
Description:
[email],
[email2],
[email3] Reference: Spec Guidelines 20021108 (WD) This issue is comprised of a couple of sub-issues:
The issue was discussed at the 20021104 telecon, with no resolution other than to formulate it as an issue and deal with it after the 20021108 SpecGL publication. Discussed at Seattle face-to-face. The issue was withdrawn (therefore closed for now) -- the currently queued changes to Guideline 3 are so extensive, that no one is sure whether the issue is still relevant or moot. |
|||||||
Proposal: Document GL scope as catch-all and/or global; add new CK3.5 similar to proposed. | |||||||
Resolution: As just described (Seattle resolution). | |||||||
103 | SpecGuide | 2002-11-04 | SpecGuide | Substantive | Closed | David Marston | Lynne Rosenthal |
Title: What are the fulfillment criteria for navigation requirements of CK13.4? | |||||||
Description:
[email] Reference: Spec Guidelines 20021108 (WD) Several previous checkpoints requiring TOC entries (to locate DoV and other conformance information) were combined into the single new CK13.4. As pointed out in email thread, the fulfillment criteria are not developed enough to prevent trivial satisfaction without meeting the intent of the checkpoint. It was suggested that there should be a MUST requirement, that the mechanism work in hardcopy versions of the specification. The issue was discussed at the 20021104 telecon, without clear resolution other than to formulate it as an issue and deal with it after the 20021108 SpecGL publication. Discussed and closed at the 20021216 telecon, the resolution being:
It was not resolved, how to handle the suggested easy-in-hardcopy requirement. A new issue, issue 108, was spun off to deal with that aspect. |
|||||||
Proposal: Add MUST requirement, that navigation mechanism work in hardcopy. | |||||||
Resolution: As detailed above, with (#108) for the hardcopy question. | |||||||
104 | SpecGuide | 2002-11-04 | SpecGuide | Substantive | Closed | Dominique Hazaël-Massieux | Lynne Rosenthal |
Title: Should CK7.4 (deprecation examples) be rewritten to cover both producers and consumers? | |||||||
Description:
[email] Reference: Spec Guidelines 20021108 (WD) The issue was posed during review of draft text for the 20021108 SpecGL publication, and was discussed at the 20021104 telecon. The question was raised: does this checkpoint apply just to producers and not consumers (the wording implies "producers")? Some think it should also apply to consumers. If that is accepted, then there is a choice of adding a parallel checkpoint for consumers, or rewording CK7.4 so that it applies to both producers and consumers. Discussed at Seattle face-to-face. Resolved that it applies only to producers. |
|||||||
Proposal: Reword CK7.4 so that it applies to both producers and consumers. | |||||||
Resolution: No, producers only. | |||||||
105 | TestGuide | 2002-12-11 | TestGuide | Substantive | Closed | Mark Skall | Lynne Rosenthal |
Title: Should the "fail" part of the conformance disclaimer be eliminated? | |||||||
Description:
[email] Reference: Note. Although indicated for Test Guidelines, the issue pertains to the other guidelines documents as well. The second part of the conformance disclaimer states "Failing to achieve conformance degree of at least A-conforming does not mean that the subject specification is necessarily deficient to its intended purposes, nor does it mean that it is an unacceptable basis for the development of quality test materials." Originator: "This undermines our attempt to get WGs, recs and test suites to adhere to at least Level A. In fact, it may be in direct contradiction to this goal." Options:
Discussed at Seattle face-to-face. Resolved to remove the fail part of the Conformance Disclaimer. |
|||||||
Proposal: Originator recommends option #3. | |||||||
Resolution: Option #3, remove the fail part of the disclaimer. | |||||||
106 | SpecGuide | 2002-12-11 | SpecGuide | Substantive | Closed | Lofton Henderson | Lynne Rosenthal |
Title: Should the "extensions" checkpoints 9.1 and 9.2 be repackaged? | |||||||
Description:
[email] Reference: Spec Guidelines 20021108 (WD) CP9.1 says, "Indicate if extensions are disallowed", and requires definition of the conditions under which they are disallowed. CP9.2 says, "Indicate if extensions are allowed," and enumerates details that must be supplied, including the conditions under which they are allowed. There are a couple problems: the packaging of the two is awkward; and, there is little distinction between CP9.1 and CP3.2 ("Identify strict conformance requirements"). A equivalent and cleaner packaging would be:
Another equivalent rewording of 9.1 would be, "Indicate the conditions under which conformance requirements are strict and not strict". This wording emphasizes that these checkpoints have a lot of redundancy with 3.2. Discussed and closed at the 20021216 telecon, with resolution as proposed. |
|||||||
Proposal: Originator recommends repackaging 9.1, 9.2 as indicated. (No proposal about 3.2). | |||||||
Resolution: As proposed. | |||||||
107 | TestGuide | 2002-12-11 | TestGuide | Substantive | Closed | Lynne Rosenthal | Kirill Gavrylyuk |
Title: Differentiate between test materials for CR and those for Recs. | |||||||
Description:
[email] Reference: Test Guidelines 20021205 (WD) There is a topic that is not addressed in TestGL, and ought to be: in reality, test suites developed at the CR stage may be different than a test suite at the Rec stage. The objectives for these test suites are often very different - for example: for CR the objective may be to 'test the specification', whereas a test suite for a Rec targets interoperability of the implementations. Since Operational Guidelines advocates having tests for CR exit, this aspect must be considered. There should be discussion in the Introduction, and/or treatment in the checkpoints, to capture the idea that test suites for CR may be different, and all of the CPs in TestGL may not be applicable (e.g., CPs about maintenance of tests). Discussed at Boston face-to-face. Resolved that TestGL needs a new checkpoint (LR to draft), that test materials clearly document their scope and intended purpose. |
|||||||
Proposal: | |||||||
Resolution: Add a new checkpoint to TestGL, that test materials clearly document their scope and intended purpose. | |||||||
108 | SpecGuide | 2002-12-26 | SpecGuide | Substantive | Closed | David Marston | Lynne Rosenthal |
Title: Should the fulfillment criteria for CP13.4's navigation requirements specifically include usability in hardcopy? | |||||||
Description:
[email] Reference: Spec Guidelines 20021220 (WG discussion draft) In closing issue 103, several changes were made to SpecGL to improve the easy findability of critical conformance information by the required navigation mechanisms. However, one aspect of the issue was not addressed: it is suggested that there ought to be a MUST requirement -- either a new MUST requirement in CP13.4 or a new checkpoint -- that the mechanism work in hardcopy versions of the specification. Others believe that this is a detail that should not be in SpecGL's fulfillment criteria (or test assertions), but rather is a detail that should be dealt with in Spec Examples & Techniques (SpecET). Discussed at Seattle face-to-face. Resolved that it will not be in the requirements to satisfy the checkpoint, but will be in the discussion for the checkpoint. |
|||||||
Proposal: Add a MUST requirement, that navigation mechanisms work in hardcopy. | |||||||
Resolution: No, not a requirement, but document it in the checkpoint's discussion as a useful and recommended attribute of the navigation mechanism. | |||||||
109 | SpecGuide | 2002-12-26 | SpecGuide | Substantive | Closed | Lofton Henderson | Lynne Rosenthal |
Title: How do the "extensions" checkpoints (9.1 & 9.2) relate to strict conformance checkpoint (3.2)? | |||||||
Description: Reference: Spec Guidelines 20021220 (WG discussion draft) Issue raised during 20021216 telecon. CP9.1 and CP9.2 are phrased in terms of extensions, but given the definition of strict conformance in GL3, an equivalent wording of CP9.1 would be: "Indicate the conditions under which conformance requirements are strict and not strict". Worded that way, it is almost completely redundant with CP3.2. There are two sub-issues:
Discussed at Seattle face-to-face. Multi-faceted resolution: remove checkpoint 3.2 and transfer it's strict-by-class concept to guideline 9 (or checkpoint 9.1?), migrate the definition of "strict conformance" to guideline 9, as well as some of the other verbiage of guideline 3 (2nd pgph). Related issue: issue 106. |
|||||||
Proposal: | |||||||
Resolution: As just described (Seattle resolution). | |||||||
110 | SpecGuide | 2002-12-26 | SpecGuide | Substantive | Closed | Dominique Hazaël-Massieux | Lynne Rosenthal |
Title: It is unclear how to apply CP3.3, "distinguish requirements from product-specific extra features." | |||||||
Description:
[email] Reference: Spec Guidelines 20021220 (WG discussion draft) Issue raised during 20021216 telecon. The fulfillment criteria state, "To full this checkpoint, a specification MUST state in its conformance section all facets of the requirements where the required features represent the maximum allowable capabilities." While applying the specGL to the opsGL, originator couldn't determine what the checkpoint and fulfillment criteria means. QAWG agreed on the call that this CP definitely needs some clarification. Discussed at Seattle face-to-face. Resolved to remove checkpoint 3.3. |
|||||||
Proposal: | |||||||
Resolution: Remove checkpoint 3.3. | |||||||
111 | OpsGuide | 2003-01-03 | OpsGuide | Substantive | Closed | Lofton Henderson | |
Title: Should OpsGL require that WGs have a general process document? | |||||||
Description: Reference: Ops Guidelines 20021220 (WG discussion draft) At the reference above, checkpoint 4.3 (CP4.3) requires, "Produce the QA Process Document" (QAPD). The QAPD addresses multiple facets of the quality practices of the WG. In QAWG teleconference, the issue was raised: should OpsGL require, in addition to producing a QAPD, that the WGs also produce a general process document that addresses the non-QA facets of WG life and process.
Discussed at Seattle face- to-face. Option #2. It is not QAWG's business, and even giving (informative) advice on the topic should be considered out-of-scope for QA. |
|||||||
Proposal: Option #3. | |||||||
Resolution: Option #2. | |||||||
112 | OpsGuide | 2003-01-12 | OpsGuide | Substantive | Closed | \1 | Lofton Henderson |
Title: There are structural and testability problems in the QA-commitment table in OpsGL. | |||||||
Description: [email] Reference: Ops Guidelines 20021220 (WG discussion draft) The problems are three-fold:
Discussed and resolved, per proposal, at 20030113 teleconference. |
|||||||
Proposal: Structural issues per email proposal, and testability issues per table re-write. | |||||||
Resolution: As proposed. | |||||||
113 | SpecGuide | 2003-01-02 | SpecGuide | Substantive | Closed | Alex Rousskov | Lofton Henderson |
Title: Collected editorial and substantive issues on SpecGL. | |||||||
Description: Reference: Spec Guidelines 20021220 (WG discussion draft) Originator has submitted collected comments containing 28 editorial and substantive issues. Discussed and resolved at Seattle face-to-face, 20030113 telecon, and 20030122 telecon. The resolutions are summarized in a collected disposition of comments. |
|||||||
Proposal: | |||||||
Resolution: See disposition of comments. |