QA WG/IG F2F Minutes in Boston, March 3rd-4th 2005 (Technical Plenary week)
This document tries to summarize the bulk of the discussions during this two-days meeting. The full minutes are also linked for reference.
Table of Contents
Participants
Below is the list of participants to the meeting, along with their initials (used at some points in the minutes).
- (TB) Tim Bolland (NIST)
- (PC) Patrick Curran (Sun Microsystems)
- (KD) Karl Dubost (W3C, WG Chair) - on the phone
- (DH) Dominique Hazaël-Massieux (W3C)
- (LH) Lofton Henderson (CGMO) - Thursday only
- (RK) Richard Kennedy (Boeing)
- (DM) Dave Marston (IG, guest to WG discussions)
- (LR) Lynne Rosenthal (NIST - IG co-chair)
- (MS) Mark Skall (NIST)
- (OT) Olivier Théreaux (W3C - IG co-chair)
Discussions summary
Thursday, March 3rd, AM
raw minutes (scribe: OT).
Future of QAWG
Work of QAWG has been appreciated, good feedback on LC. Management
decided WG can continue work until August (+a few month extension if
SpecGL goes to REC around this date). Karl has been sole chair and
requests more activity on the list. Participants renew commitment to
more activity after busy times.
QAIG report
Olivier summarizes role of IG, reports recent work. Idea that the IG
could be made more of a common ground for working groups to share info
and experience on testing (see next topic).
Report on "join test suite" lunch table
Discussing the "join test suite" lunch table session the previous
day. Surprisingly good participation for such a topic at such a time.
There is a need (for us?) to provide a common repository of experience
on testing, implementation reports (we already have the QA Matrix),
this could be done by inviting people to participate in www-qa.
As a result of this discussion, Dom took an action item to advertize our existing resources to the Chairs mailing list.
QA WG Document Status
- Varibility in Specifications: still some work necessary to achieve WG Note status. Dom in
charge, with target publication for end of April. Karl notes that
theoretical issues with variability should be left for TAG to address,
QAWG should focus on practicalities.
- Test FAQ: Need more examples, but could be ready for publication by end
of March. Bugzilla will be used to track its issues. Goal is to make it
small, practical, fast.
- TCDL: Renewed interest. It would be good to clarify status on oct 2003
contribution.
- QA Handbook: mid-june seems to be a possible target
- Spec GL: discussing whether to skip CR phase
Thursday, March 3rd, PM
raw minutes (scribe: MS)
SpecGL LC Issues
LC Issues List
- Issue 1041: Conformance is not a yes/no proposition
- Resolved: Should limit ICS to just declarative, but an ICS linked to
tests that are passed can strengthen a conformance claim.
- Issue 955: positive statements about absence of obsolete features (and deprecated, ...)
- Resolved: We agree there should be a positive statement about features
referred to in every Good Practice. If they don't exist, it should be
stated in the conformance clause.
- Issue 1042: Scope helps determine whether spec is overstepping its mandate.
- Resolved: agreed with comment, Karl will incorporate suggestion in SpecGL
- Issue 1045 Avoiding device-dependent profiles
- Resolved: the Working Group disagrees that the current text encourages usage of profiles, and doesn't think it should get into more precise details on how to decide how to subdivide
- Issue 1050: Modesty Requirement
- Resolved: the WG disagrees it should discourage conformance claims to SpecGL and other specs, and doesn't think it needs to put such a strong focus on other type of qualities claim
- Issue 1051 "Identify deprecated features" too strong
- This issue says this is too strong because it only applies to a second
version of a specification. Will incorporate this into SpecGL.
- Issue 1052 "Classes of product" unclear and dangerous
- Resolved: There will be a link from Section 2.2 A to the Variability in
Specification document (CoP section). There will also be a link to the
defintion.
- Resolved: Change "To which the specification applies" to "which
requirements are imposed upon" the CoP in the "what does it mean?" section.
- AI: Dom to draft response to the XML Core WG.
- Issue 1058: Structure and numbering confusing. Only some parts of the
document are numbered and numbering scheme not consistent.
- Waiting for a proposal from Karl; it was suggested that least we should better explain scheme and make sure Good Practices are linked
form Table of Contents.
- Issue 1061: argues that if there are no requirements it is not a
specification.
- Resolved: We will change "specification" to "technical report" in the
"About This Document" Section.
- Resolved: There should still be a conformance clause even for
specifications that are not normative.
Next F2F Meeting
The WG discussed where to held its next face to face meeting, and resolved to targeting Dublin with back-ups of Montreal and Sophia, France: June 21-23.
Friday, March 4th, AM
raw minutes (scribe: LR)
Discussion of SpecGL comments: Issues 1144-1159
- #1144: specification and workflow mixup
- Proposal: move the related GP of section 5, so it isn't
in the main body of the specification. This would be consistent with not
mixing normative and informative information. Rename good practice in something else.
- 2.3GP B systematic review of normative references → rework so that it remains in the body of SpecGL (AI DM).
- GP Write test
assertions should also be part of the spec.
- 5GP A, B is workflow.
- 5GP D, E
could fit in chapter 3, if rewritten.
- 4.2GPA need for optional feature
reword as use optional features as warranted.
- Link 2.1B to 5C.
- ACTION: Move to appendix. Reformat perhaps more textual, rename so there
are no good practices. Label as informative. Karl
- ACTION: rework GP2.3 B to be less workflow, but keep it in the main body of the document. Dave 15 March
- ACTION: Move 5GP D test assertions into chapter 3 and reword. Patrick. 15
March
- ACTION: Move 5GP E formal languages into chapter 3 and reword. Dom 15 March
- ACTION: change title of 4.2GPA to Use optional features as warranted.
- RESOLUTION: agree with the TAG, will separate the workflow aspects from the specifications aspects of the document.
- Issue 1145 conformance clause optionally is not reflected in proforma
conformance claim
- RESOLUTION: agree with the TAG and will adopt the example provided.
- Issue 1150 extensions and conformance interferences assumptions are limited:
Implementers are not the only one creating extensions.
- ACTION: reword Warn implementers not to create extensions... to Warn
extension creators not to create extensions...
- ACTION: add paragraph in 4.3 introduction about various types of extension
creators: Dom 15 March
- RESOLUTION: agree with the TAG and will make appropriate changes to the text.
- Issue 1154 error processing for non language or protocol specs.
- ACTION: Reword text and include examples Dom 15 March
- RESOLUTION: agree with the TAG and will make appropriate changes to the text
- Issue 1155 Use your own example: Section 5 Story: identify the group, QAWG.
- RESOLUTION: agree with the TAG. We won't be so modest and will come out
of the closet.
- Issue 1157 ICS needs more than y/n/na
- RESOLUTION: agree with the TAG, making appropriate changes to the text
- Issue 1158 ICS for SpecGL
-
- 1) Not clear that SpecGL requires a completed ICS in order to claim
conformance to SpecGL. In Conformance Claim section, add bullet to
'include a completed ICS; Add to the example reference to the ICS: e.g.,
An ICS proforma is at <give URI>. Clarify the 'you can claim conformance'
that this is one example of what the claim can look like it. Need to move
SpecGL's ICS from informative to normative. Reword, in Conformance
Criteria section last sentence to: If all the Requirements are checked as
being satisfied, then conformance can be claimed as below. Converse is
handled by statement that, To conform to this SpecGL, all Requirements
must be implemented.
- 2) Keep definition of specification in the Scope. Add definition to
Glossary.
What is SpecGL's COP? Technical Reports with strong focus on specifications.
- 3) Not different.
- 4) Changed Conformance Clause is simple to Conformance Model of the
SpecGL is simple.
- 5) Discussed earlier, under workflow mix.
- 6) Make more explicit - State that no subdivision is warranted. Add
positive statement regarding explaining why not. We want to encourage
people not to subdivide unless necessary, this seems to counter
that. Status quo is that you don't subdivide. In 4.1 GP A add 'only when
warranted. Talk about cost of subdividing in the What does this mean.
- 7) No extensibility mechanism is presented, although we allow it. Fine
for others to add new requirements (functions) REJECT this.
- 8) Error handling: not applicable for SpecGL. Need to reflect that
this is not applicable to WGAC, SpecGL, etc.
- 9) No obsolete features. Need to mention that SpecGL has no
deprecation, obsolete, etc. in its conformance clause.
- 10) Internal process for review: addressed earlier by earlier TAG comment
- 11) Addressed by earlier TAG comment
- RESOLUTION agree with the TAG, except as Noted (7). Have reviewed ICS and
taken appropriate actions (e.g., modified SpecGL) to ensure that all items
result in Yes.
- Issue 1159 Warn against untested hooks
- Seems to be a process/workflow issue. Under reviewing and testing, can
mention putting in untested hooks. Add RDF story provided.
- ACTION: Add as technique in 5 GP B
- RESOLUTION: Agreed.
Tim report on CSS WG discussions on QA
- CSS3 is a collection of many functions. These functions are grouped
into modules. Modules can stand on their own. A module should be able to
conform to SpecGL, as should a Profile.
- COP for CSS3 triggered discussion as to whether authoring tools and
validators were within scope and to be considered a COP -no decision made.
- Rules to define profiles in CSS3: none have been contemplated, since
they don't think there are rules or haven't thought about this
before. Informally there are rules, since the dependencies need to be
identified and consistent with the dependency graphs.
- How does CSS validator know that the stylesheet has been written for a
particular profile: user needs to tell the validator which
profile. Discussion, but no resolution.
Questions provided by Karl stimulated good discussion in CSS WG
WCAG Issues: quick review
Dom presents overview of WCAG
- Issue 1082 Extensibility of WCAG
- Question as to whether a guideline can be extended: can WCAG success
criteria be extended. The guidelines are deliberately generic so that they
can be applied and adapted (extended) by different policy makers. Maybe
want to indicate that it is O.K. to add additional guidelines/criteria, but
do it in the style of the ones here. Yes, extensibility is appropriate
and define the mechanism formally.
- Issue 1083 Deprecation WCAG 1.0/2.0
- Documents are very different and mapping between the two is neither easy
nor direct. One possibility is to declare 2.0 as a new document and not a
revision of 1.0. This avoids the issue. Think this mapping may be
difficult but important that people know the evolution of a feature.
Suggest there be an appendix with the mapping, showing and explaining the
evolution. Use whatever format works.
- Issue 1090 Managing variability
- Agree with their discussion.
Friday, March 4th, PM
raw minutes (scribe: PC)
Joint meeting with WAI CG
Present from WAI CG:
- Al Gilman [AG]: Protocols & Formats WG
- Wendy Chisholm [WC]: WCAG staff contact
Observer from WCAG WG: Michael Cooper [MC], chair of WCAG Techniques Task Force.
1) WAI CG would like the QAWG to address accessibility in SpecGL
Summary: The QA WG and the WAI CG could not get on consensus on how to best represent the accessibility requirements in SpecGL; the QA WG proposal to put a notice in the scope and gives details and links in a non-normative part of SpecGL only partially addressed the WAI CG needs for a greater coordination of horizontal activities that the QA Activity was supposed to stand for. The lack of resources in the QA WG and the likely non-renewal of the WG was acknowledged.
Resolved: add a new item in the new "workflow" part of SpecGL about accessibility and other "horizontal" aspects of qualities for a Specification.
2) Issues/questions raised by WCAG in response to their attempts to to
conform to Spec GL
QAWG will respond by email. [WC] will respond if any additional
concerns...
Future meetings
Next teleconference has been scheduled for Monday, March 14th at the usual time.
Next F2F is scheduled for June 21 to 23rd, in Dublin (to be confirmed)
Summary of action items
Here is a list of all action items from the raw minutes. If no due date is indicated, it defaults to March 14 2005.
- ACTION: dom to add pointer to primer in QAWG homepage: [done]
- ACTION: dom to create a "Test FAQ" component in the QA bugzilla [done]
-
ACTION: dom to create a standard set of slides to present the primer to
new groups - by ~ march 10
-
ACTION: dom to send a mail to chairs@ , pointing to matrix, inviting to
discussion in www-qa
-
ACTION: dom to send link to slideset used for CDF presentation [done]
-
ACTION: karl to send ideas for extensibility section of Variability in Spec
-
ACTION: Patrick to give list of examples we are looking for (Test FAQ)
by March 18th [done]
-
ACTION: patrick to put the existing Test FAQ issues in bugzilla
- AI-20050303-1 LR to provide a 1 sentence disclaimer in the"What does it
mean" section that the ICS should positively emphasize that it's not about
conformance [done]
- AI-20050303-2 KD to update SpecGL to incorporate issue 1042 into SpecGL,
concluding that the Scope helps determine whether the spec is overstepping
its mandate. [done]
- AI-20050303-3 KD to draft response to Issue 1045: Avoiding
device-dependent profiles.
- AI-20050303-4 DH to draft response to the XML Core WG to respond to issue
1052.
- ACTION Karl: Move content of Section 5 to separate Main section in SpecGL. Reformat perhaps more textual, rename so there
are no good practices. Label as informative. [done]
- ACTION Dave: rework GP2.3 B to be less workflow, but keep it in the main body of the document. 15 March [done]
- ACTION Patrick: Move 5GP D test assertions into chapter 3 and reword.. 15
March
-
ACTION Dom: Move 5GP E formal languages into chapter 3 and reword. 15 March [done]
-
ACTION Dom: change title of 4.2GPA to Use optional features as warranted. [done]
-
ACTION Dom: reword "Warn implementers not to create extensions" to "Warn
extension creators not to create extensions" [done]
-
ACTION Dom: add paragraph in 4.3 introduction about various types of extension
creators. 15 March [done]
-
ACTION Dom: Reword text regarding " error processing for non language or protocol specs" (bug Issue 1154) ; 15 March [done]
-
ACTION Dom: Encourage WG to link to the appropriate part of the specification that
demonstrates each claim of conformance in the ICS. Add a Comments column.
-
ACTION Dom, Karl: implement resolutions of TAG notes regarding SpecGL ICS of SpecGL.
-
ACTION Dom: Add "warn against untested hooks" as technique in 5 GP B [done]