QA WG/IG F2F Minutes in Boston, March 3rd-4th 2005 (Technical Plenary week)

This document tries to summarize the bulk of the discussions during this two-days meeting. The full minutes are also linked for reference.

Participants
Discussions summary
Summary of action items

Participants

Below is the list of participants to the meeting, along with their initials (used at some points in the minutes).

(TB) Tim Bolland (NIST)
(PC) Patrick Curran (Sun Microsystems)
(KD) Karl Dubost (W3C, WG Chair) - on the phone
(DH) Dominique Hazaël-Massieux (W3C)
(LH) Lofton Henderson (CGMO) - Thursday only
(RK) Richard Kennedy (Boeing)
(DM) Dave Marston (IG, guest to WG discussions)
(LR) Lynne Rosenthal (NIST - IG co-chair)
(MS) Mark Skall (NIST)
(OT) Olivier Théreaux (W3C - IG co-chair)

Discussions summary

Thursday, March 3rd, AM

raw minutes (scribe: OT).

Future of QAWG

Work of QAWG has been appreciated, good feedback on LC. Management decided WG can continue work until August (+a few month extension if SpecGL goes to REC around this date). Karl has been sole chair and requests more activity on the list. Participants renew commitment to more activity after busy times.

QAIG report

Olivier summarizes role of IG, reports recent work. Idea that the IG could be made more of a common ground for working groups to share info and experience on testing (see next topic).

Report on "join test suite" lunch table

Discussing the "join test suite" lunch table session the previous day. Surprisingly good participation for such a topic at such a time. There is a need (for us?) to provide a common repository of experience on testing, implementation reports (we already have the QA Matrix), this could be done by inviting people to participate in www-qa.

As a result of this discussion, Dom took an action item to advertize our existing resources to the Chairs mailing list.

QA WG Document Status

Varibility in Specifications: still some work necessary to achieve WG Note status. Dom in charge, with target publication for end of April. Karl notes that theoretical issues with variability should be left for TAG to address, QAWG should focus on practicalities.
Test FAQ: Need more examples, but could be ready for publication by end of March. Bugzilla will be used to track its issues. Goal is to make it small, practical, fast.
TCDL: Renewed interest. It would be good to clarify status on oct 2003 contribution.
QA Handbook: mid-june seems to be a possible target
Spec GL: discussing whether to skip CR phase

Thursday, March 3rd, PM

raw minutes (scribe: MS)

SpecGL LC Issues

LC Issues List

Issue 1041: Conformance is not a yes/no proposition: Resolved: Should limit ICS to just declarative, but an ICS linked to tests that are passed can strengthen a conformance claim.
Issue 955: positive statements about absence of obsolete features (and deprecated, ...): Resolved: We agree there should be a positive statement about features referred to in every Good Practice. If they don't exist, it should be stated in the conformance clause.
Issue 1042: Scope helps determine whether spec is overstepping its mandate.: Resolved: agreed with comment, Karl will incorporate suggestion in SpecGL
Issue 1045 Avoiding device-dependent profiles: Resolved: the Working Group disagrees that the current text encourages usage of profiles, and doesn't think it should get into more precise details on how to decide how to subdivide
Issue 1050: Modesty Requirement: Resolved: the WG disagrees it should discourage conformance claims to SpecGL and other specs, and doesn't think it needs to put such a strong focus on other type of qualities claim
Issue 1051 "Identify deprecated features" too strong: This issue says this is too strong because it only applies to a second version of a specification. Will incorporate this into SpecGL.
Issue 1052 "Classes of product" unclear and dangerous: Resolved: There will be a link from Section 2.2 A to the Variability in Specification document (CoP section). There will also be a link to the defintion.; Resolved: Change "To which the specification applies" to "which requirements are imposed upon" the CoP in the "what does it mean?" section.; AI: Dom to draft response to the XML Core WG.
Issue 1058: Structure and numbering confusing. Only some parts of the document are numbered and numbering scheme not consistent.: Waiting for a proposal from Karl; it was suggested that least we should better explain scheme and make sure Good Practices are linked form Table of Contents.
Issue 1061: argues that if there are no requirements it is not a specification.: Resolved: We will change "specification" to "technical report" in the "About This Document" Section.; Resolved: There should still be a conformance clause even for specifications that are not normative.

Next F2F Meeting

The WG discussed where to held its next face to face meeting, and resolved to targeting Dublin with back-ups of Montreal and Sophia, France: June 21-23.

Friday, March 4th, AM

raw minutes (scribe: LR)

Discussion of SpecGL comments: Issues 1144-1159

#1144: specification and workflow mixup

Proposal: move the related GP of section 5, so it isn't in the main body of the specification. This would be consistent with not mixing normative and informative information. Rename good practice in something else.

2.3GP B systematic review of normative references → rework so that it remains in the body of SpecGL (AI DM).
GP Write test assertions should also be part of the spec.
5GP A, B is workflow.
5GP D, E could fit in chapter 3, if rewritten.
4.2GPA need for optional feature reword as use optional features as warranted.
Link 2.1B to 5C.

ACTION: Move to appendix. Reformat perhaps more textual, rename so there are no good practices. Label as informative. Karl
ACTION: rework GP2.3 B to be less workflow, but keep it in the main body of the document. Dave 15 March
ACTION: Move 5GP D test assertions into chapter 3 and reword. Patrick. 15 March
ACTION: Move 5GP E formal languages into chapter 3 and reword. Dom 15 March
ACTION: change title of 4.2GPA to Use optional features as warranted.

RESOLUTION: agree with the TAG, will separate the workflow aspects from the specifications aspects of the document.

Issue 1145 conformance clause optionally is not reflected in proforma conformance claim

RESOLUTION: agree with the TAG and will adopt the example provided.

Issue 1150 extensions and conformance interferences assumptions are limited: Implementers are not the only one creating extensions.

ACTION: reword Warn implementers not to create extensions... to Warn extension creators not to create extensions...
ACTION: add paragraph in 4.3 introduction about various types of extension creators: Dom 15 March

RESOLUTION: agree with the TAG and will make appropriate changes to the text.

Issue 1154 error processing for non language or protocol specs.

ACTION: Reword text and include examples Dom 15 March

RESOLUTION: agree with the TAG and will make appropriate changes to the text

Issue 1155 Use your own example: Section 5 Story: identify the group, QAWG.

RESOLUTION: agree with the TAG. We won't be so modest and will come out of the closet.

Issue 1157 ICS needs more than y/n/na

RESOLUTION: agree with the TAG, making appropriate changes to the text

Issue 1158 ICS for SpecGL

1) Not clear that SpecGL requires a completed ICS in order to claim conformance to SpecGL. In Conformance Claim section, add bullet to 'include a completed ICS; Add to the example reference to the ICS: e.g., An ICS proforma is at <give URI>. Clarify the 'you can claim conformance' that this is one example of what the claim can look like it. Need to move SpecGL's ICS from informative to normative. Reword, in Conformance Criteria section last sentence to: If all the Requirements are checked as being satisfied, then conformance can be claimed as below. Converse is handled by statement that, To conform to this SpecGL, all Requirements must be implemented.
2) Keep definition of specification in the Scope. Add definition to Glossary. What is SpecGL's COP? Technical Reports with strong focus on specifications.
3) Not different.
4) Changed Conformance Clause is simple to Conformance Model of the SpecGL is simple.
5) Discussed earlier, under workflow mix.
6) Make more explicit - State that no subdivision is warranted. Add positive statement regarding explaining why not. We want to encourage people not to subdivide unless necessary, this seems to counter that. Status quo is that you don't subdivide. In 4.1 GP A add 'only when warranted. Talk about cost of subdividing in the What does this mean.
7) No extensibility mechanism is presented, although we allow it. Fine for others to add new requirements (functions) REJECT this.
8) Error handling: not applicable for SpecGL. Need to reflect that this is not applicable to WGAC, SpecGL, etc.
9) No obsolete features. Need to mention that SpecGL has no deprecation, obsolete, etc. in its conformance clause.
10) Internal process for review: addressed earlier by earlier TAG comment
11) Addressed by earlier TAG comment

RESOLUTION agree with the TAG, except as Noted (7). Have reviewed ICS and taken appropriate actions (e.g., modified SpecGL) to ensure that all items result in Yes.

Issue 1159 Warn against untested hooks

Seems to be a process/workflow issue. Under reviewing and testing, can mention putting in untested hooks. Add RDF story provided.

ACTION: Add as technique in 5 GP B

RESOLUTION: Agreed.

Tim report on CSS WG discussions on QA

CSS3 is a collection of many functions. These functions are grouped into modules. Modules can stand on their own. A module should be able to conform to SpecGL, as should a Profile.
COP for CSS3 triggered discussion as to whether authoring tools and validators were within scope and to be considered a COP -no decision made.
Rules to define profiles in CSS3: none have been contemplated, since they don't think there are rules or haven't thought about this before. Informally there are rules, since the dependencies need to be identified and consistent with the dependency graphs.
How does CSS validator know that the stylesheet has been written for a particular profile: user needs to tell the validator which profile. Discussion, but no resolution.

Questions provided by Karl stimulated good discussion in CSS WG

WCAG Issues: quick review

Dom presents overview of WCAG

Issue 1082 Extensibility of WCAG: Question as to whether a guideline can be extended: can WCAG success criteria be extended. The guidelines are deliberately generic so that they can be applied and adapted (extended) by different policy makers. Maybe want to indicate that it is O.K. to add additional guidelines/criteria, but do it in the style of the ones here. Yes, extensibility is appropriate and define the mechanism formally.
Issue 1083 Deprecation WCAG 1.0/2.0: Documents are very different and mapping between the two is neither easy nor direct. One possibility is to declare 2.0 as a new document and not a revision of 1.0. This avoids the issue. Think this mapping may be difficult but important that people know the evolution of a feature. Suggest there be an appendix with the mapping, showing and explaining the evolution. Use whatever format works.
Issue 1090 Managing variability: Agree with their discussion.

Friday, March 4th, PM

raw minutes (scribe: PC)

Joint meeting with WAI CG

Present from WAI CG:

Al Gilman [AG]: Protocols & Formats WG
Wendy Chisholm [WC]: WCAG staff contact

Observer from WCAG WG: Michael Cooper [MC], chair of WCAG Techniques Task Force.

1) WAI CG would like the QAWG to address accessibility in SpecGL

Summary: The QA WG and the WAI CG could not get on consensus on how to best represent the accessibility requirements in SpecGL; the QA WG proposal to put a notice in the scope and gives details and links in a non-normative part of SpecGL only partially addressed the WAI CG needs for a greater coordination of horizontal activities that the QA Activity was supposed to stand for. The lack of resources in the QA WG and the likely non-renewal of the WG was acknowledged.

Resolved: add a new item in the new "workflow" part of SpecGL about accessibility and other "horizontal" aspects of qualities for a Specification.

2) Issues/questions raised by WCAG in response to their attempts to to conform to Spec GL

QAWG will respond by email. [WC] will respond if any additional concerns...

Future meetings

Next teleconference has been scheduled for Monday, March 14th at the usual time.

Next F2F is scheduled for June 21 to 23rd, in Dublin (to be confirmed)

Summary of action items

Here is a list of all action items from the raw minutes. If no due date is indicated, it defaults to March 14 2005.

ACTION: dom to add pointer to primer in QAWG homepage: [done]
ACTION: dom to create a "Test FAQ" component in the QA bugzilla [done]
ACTION: dom to create a standard set of slides to present the primer to new groups - by ~ march 10
ACTION: dom to send a mail to chairs@ , pointing to matrix, inviting to discussion in www-qa
ACTION: dom to send link to slideset used for CDF presentation [done]
ACTION: karl to send ideas for extensibility section of Variability in Spec
ACTION: Patrick to give list of examples we are looking for (Test FAQ) by March 18th [done]
ACTION: patrick to put the existing Test FAQ issues in bugzilla
AI-20050303-1 LR to provide a 1 sentence disclaimer in the"What does it mean" section that the ICS should positively emphasize that it's not about conformance [done]
AI-20050303-2 KD to update SpecGL to incorporate issue 1042 into SpecGL, concluding that the Scope helps determine whether the spec is overstepping its mandate. [done]
AI-20050303-3 KD to draft response to Issue 1045: Avoiding device-dependent profiles.
AI-20050303-4 DH to draft response to the XML Core WG to respond to issue 1052.
ACTION Karl: Move content of Section 5 to separate Main section in SpecGL. Reformat perhaps more textual, rename so there are no good practices. Label as informative. [done]
ACTION Dave: rework GP2.3 B to be less workflow, but keep it in the main body of the document. 15 March [done]
ACTION Patrick: Move 5GP D test assertions into chapter 3 and reword.. 15 March
ACTION Dom: Move 5GP E formal languages into chapter 3 and reword. 15 March [done]
ACTION Dom: change title of 4.2GPA to Use optional features as warranted. [done]
ACTION Dom: reword "Warn implementers not to create extensions" to "Warn extension creators not to create extensions" [done]
ACTION Dom: add paragraph in 4.3 introduction about various types of extension creators. 15 March [done]
ACTION Dom: Reword text regarding " error processing for non language or protocol specs" (bug Issue 1154) ; 15 March [done]
ACTION Dom: Encourage WG to link to the appropriate part of the specification that demonstrates each claim of conformance in the ICS. Add a Comments column.
ACTION Dom, Karl: implement resolutions of TAG notes regarding SpecGL ICS of SpecGL.
ACTION Dom: Add "warn against untested hooks" as technique in 5 GP B [done]