Action Items· Minutes, AM· Minutes, PM·
QA Homepage· Latest News· QA Resources· QA IG· QA WG· QA Calendar·
See the Agenda and the Montreal logistics page.
Scribe: Olivier Théreaux
Roll Call
Small note about logistics (lunch, etc)
We are talking about attendance, rules of good standing. The question whether we should enforce rules of good standing. We should approach members of the WG who've missed meetings, then AC reps if necessary.
AI - KD, to deal with issue of good standing in the WG
In the first discussion about it, Lofton proposed to switch schedule quarterly (at equinox/solstice) between Europe/Asia friendly and North-america friendly
olivier and dom already authored one. Next would be karl, or jack.
AI : olivier to update IG page to point to weekinQA plus add link to latest one
Mark : is it working well?
KD: It's too early,I didn't have time to ask people if they're reading it, etc at least people in the room liked the last issue summarizing testable assertions, maybe should pubicize it
AI : Karl to contact comm / advertize this service review/valuate the service?
The WG has decided to evaluate before next f2f and decide the future of this service during October f2f meeting
Karl doing summary of the WoW BoF (meeting with web E&O people) at www2002, need for EO, etc as a result : Web E&O proposal
There's a discussion about it... not really related to the WG itself. Some ideas about speaking about evangelists, wasp, etc. Mark think it's a good idea, needs to be done.
IG is useful, people not from the WG (not members) can participate and discuss. Mark (playing devil's advocate) asks about dilution of resources / loss of focus.
Conclusion: whoever is responsible for a technology/resource should get sure comments and issues are properly disposed of
There's a need for coordination within the WG about test. Should we have a different group, chair, etc, or not? We don't know, but seems to make sense and group would work horizontally, coordinationg with chairs of al WGs. We need both the skill AND the liaison with WGs.
How to/ can we make the latter happen? Different solutions are possible: Should be one "QA contact" in each group, should be in the charters.
Karl talks about the re-chartering of HTML as an example. The problem is that nothing is mandatory now. The problem would disappear if QA process guidelines were added to the process.
Dimitris discussing pros/cons of having one or two working groups.
Karl gives idea that once the QAFW is finished the WG could be modified to do such consulting work. Some concerns are raised about creating a new WG that will suck resources (too many?) out of the QA WG. The task force withing the WG seems to be the way to go. We are discussing the participation from other working groups. We are hoping the tech review will help, communication towards team members (staff contacts).
The project review is a way to present a technology, an activty, etc to the rest of the Team.
Lofton is asking about the format? the rules? What are the questions we want to ask the W3C team? We want the team to ask? The question of mandatoriness is raised. Can it be made mandatory now, since documents are not mature yet? "it" being some level of conformance to our guidelines. It won't be mandatory to conform until documents are "finished" anyway (says Mark)
There are some discussions (missed a lot in the minutes) about testable assertions in the specs in general.. and ours.
AI : dom to draft position of the WG about enforcing guidelines
Scribe: Dominique Hazaël-Massieux
LH: We probably don't have any resources to do a proactive review on a generic basis
kd: We should first educate WG to raise their awareness about these GL
lh: if they simply reject your comments, you can send comments as LC comments and they have to list them in their list of comments
kd: During the past 2 years, I have had bad experiences trying to enforce the presence of a conformance section people didn't seem to care some times we need to find strategies to convince people people get many reviews (WAI, I18N, pubrules) in I18N, one people is commited to a review
lh: if the team contact could advocate for us, that would already help
kd: convincing team contacts might be sometimes difficult preserving harmony in the WG is sometimes more important for them team contact is not always a very easy position
lh: there is 2 aspects in an ideal world, we would have enough people to do these reviews but it's hard to get a full involvement from WG members one idea: each person in the WG should do one review with each of our docs 2 reasons: it's hard to appreciate all the subtilities without having actually applied them (benefits for our WG) 2nd, we need the input to make these documents progress we did get only one real comment, and it was from somebody outside the WG
Discussions to know wether such an assignement should be optional or mandatory
dhm: should be mandatory, with possible excuses to the chairs
lh: so that would be one review possible now [ops GL] one in one month or so for spec GL and later for the Tests GL Should the choice of WG/spec be free?
dhm: I think it should be since we enforce the assignment
lh: most people have an interest in another activity they'll probably choose it accordinly
kd: to which status of spec should we apply our GL?
lh: for a REC, we still benefit of it
mark: I'm not sure it really help us to see how it could have been done
lh: the benefit to us is that we get to know what our GL are about it allows us to refine our GL
kd: each time I read the HTML spec, I find a new bug it can help us to find broken mechanism
lh: re status of the spec, I'd say it doesn't matter it's helpful for our own purposes any way as for benefits for an external WG, the best is to see with them Last Call doc?
lh: probably better, yes the first review that'll do is against Ops GL We can coordinate this by creating a table on our WG space AI: dom to create such a table
[what's vonlunteering mean?] means forgetting to shut up
should close 10 and 36: we don't review as a general policy, but each WG member will do it per GL we do encourage through team and staff that each WG applies them
lh: if 3 people began to use the spec GL right now, that could help us create the extechs for spec
- relationship with the comm team: pubrules, manual of style and spec GL?
dhm: I don't think that we need to bother about it pubrules and spec GL are really orthogononal pubrules is about style, spec GL is about content
lh: how to close 11 and 25?
kd: we should keep an eye on them to be sure they don't clash nor overlap too much
lh: the resolution would be to have a designated liaison person that would ask Comm to review new versions of our spec and to be informed by Comm when there is a new version of their docs
dhm: I think there shouldn't be any overlap between spec GL and manual of style if there is any, I think it means that we have gone too far in spec GL maybe we need to clarify the goal of the spec GL?
kd: the introductions speaks about clarity
dhm: is there any CP about it?
kd: no
dd: clarity is about non ambiguity
lh: so, just a fuzzy liaison concept and maybe more later closed
Other relationships with horizontal activities, TAG, ...
+DMarston
lh: maybe should we wait more about this?
dhm, ot and kd present the role of the QA Activity in the maintainance of these
DM presents his proposal about reorganizing the spec GL
DM: I think in the case of UAAG, it's really a question of modules, not conditional conformance
SM and mark explaining what's confusing in the current CP "the other" conformance style is really confusing
dm: "choose a conformance policy" is really only a headline we need to specify it in more details we need to say that the spec must give his conformance philosophy other GL will details modules/levels/flavors/ ...
dm presents http://lists.w3.org/Archives/Public/www-qa-wg/2002Jun/0003.html
discussions about the maximum of dimensions of variability
re DanC's mail, http://lists.w3.org/Archives/Public/www-qa/2002May/0035.html
no ojections to re-organize the spec GL alongs DM's email
lh: I'll start the re-arragement DM, would you want to draft text for the new GL 5
DM: I think it came from Lynn I can try but I'm not sure it'll come up as she wanted
lh: Mark and Sandra could serve as a proxy for Lynn we'll discuss it in teleconf do you think I should start changing the organization first or can you do it in parallel?
DM: we can do it in //
lh: let's do that
AI: lh to reorganize spec GL, dm to redraft the conformance policy GL within 2 weeks [next teleconf] 24 June as a deadline for merging them about priority, we could make various CPs to get a better rating
mark objects to the existence of the CP
resolution: issue 70 closed next steps to be taken issue 69: re-titled: how to discourage variability in a guideline? make it clear that's it within a spec
adjourned for today
back tomorrow at 9am
Scribe: Sandra I. Martinez
Lofton reviewed yesterdayís meeting.
Karl will contact Peter Fawcett to recruit him to help with the Week in QA.
Dimitris will propose how to develop a testing group activity due August 1st.
On Thursday June 20, Lofton will present a project review to a W3C team where he will discuss accomplishments, activities, issues, etc. One of the issues he will present is whether the QA guidelines are to become mandatory or not. The WG agreed that some degree of being mandatory should be required when the documents reach a certain level. It was also agreed that the issue about being mandatory should be brought up at the next chairís meeting in the fall.
Each WG member will apply each of the three wg guidelines to a working group, specification or test suite. The purpose is for each working group member to gain experience in applying our guidelines to real example and to improve the guidelines. While this is being done, each wg member, as a courtesy, should communicate this effort to each WG chair.
An extended discussion concerning issue 67 (What should be the scope of the framework ex/tech parts?) took place. The consensus was that we need case studies, as well as specific enumeration of techniques that satisfy the checkpoints (a virtual working group). The discussion proceeded to look at how to do this in a timely manner. It was agreed that the best way to get this accomplished is for each wg member do a case study, then an enumeration of techniques that satisfy the checkpoints, for one of our documents (rather than case studies for three documents).
Issue 68 was discussed. This issue pertains to whether the three ex/tech parts should be separate standalone reports or the second part of each guideline. No consensus was reached.
Dimitris presented his proposal for Taggable Test Assertion. This proposal discusses writing testable assertions in the spec itself using a detail markup language (XMLspec.). This will allow the specification to be machine processable. Advantages of this approach include:
If this proposal is adopted, another question concerns which version would be normative the XML or the HTML version. Right now the HTML version is normative but if can show that the two versions are equivalent it should not matter. If the XML version becomes normative, the test suite becomes normative. It was recommended that the proposal should be presented in small increments rather than as one monolithic proposal.
Scribe: Mark Skall
The wg continued the discussion about the Testable Assertion proposal. It was agreed upon that a cost benefit analysis should be developed. This will be helpful in convincing the wg chairs that the proposal will result in real benefits to overall test and specification development effort. A simple questionnaire will be developed to survey the working group chairs to determine what they are doing, why they are doing it and how well that works.
Kirill presented a proposed outlined for the Test Materials Guideline document. The document consisted of seven guidelines:
A general Working Group comment, not related to any specific guideline had to do with adding checkpoint/guidelines on automatic generation of tests. Specific comments/clarification for each guideline follow:
Guideline 1
Checkpoint 1.2: Clarification testable assertion are identified for all requirements plus non-required statements (i.e. shalls, shoulds, and mays)
Checkpoint 1.3: The test assertions that are part of the conformance criteria (i.e. the shalls) are determined in this checkpoint.
Checkpoint 1.5: Change vague to intentional and unintentional ambiguities. We also want to add a separate checkpoint for contradictory behavior.
Guideline 2:
Checkpoint 2.2: Change from priority 2 to priority 1.
Add another checkpoint for combination of test (molecular test).
A discussion ensued about when to stop testing. It was decided that priorities could be used to control when to stop.
The special telcom on June 20 will continue discussion on the test guideline document.
The next face to face in Tokyo will be a three-day meeting, October 8-10, 2002.
Issue 65 (Level of detail in the QA glossary) was discussed. It was agreed that some definitions should be short, but that other items might need to be expanded. The glossary editors will modify some of the definitions. It was also agreed that any comments on the glossary must be accompanied by specific suggested wording for any modification of the definitions to be considered. A statement to this effect should be added at the beginning of the glossary document.
Issue 49 (Global license for use on distribution of test materials) was discussed and Kirill presented a sample license to be reviewed. Working Group members will review the sample license for content.
