Venue and Logistics· Agenda· Minutes·
QA Homepage· Latest News· QA Resources· QA IG· QA WG· QA Calendar·
Read the logistic page and detailed agenda for this meeting.
This joint QA WG/IG meeting was held at FORTH in Heraklion, Crete. The QA WG expresses its many thanks to FORTH, Prof Markatos and Anna Doxastaki for hosting the meeting. Also thanks to Prof. Markatos and Anna for hosting a wonderful feast. Many thanks to Anna for handling the local arrangements and for making sure that we had everything we needed to conduct a successful meeting.
Below is an excerpt from the QAWG Action Items Table withaction items given during the meeting (and still opened after the meeting). The actual Action Items Table has up-to-date information about current AIs.
| Id | Owner | Action Item | Deadline |
|---|---|---|---|
| AI-20031016-1 | DD | answer question in last call about relationship of QA specs and guideline to ISO 9000 | 2003-07-16 |
| AI-20031016-2 | Dom & Lynne | estimate the dates for Spec GL | before we leave Crete |
| AI-20031016-3 | Daniel | create outline of document presenting arguments for value of "QA now" | 2003-07-16 |
| AI-20031016-4 | Daniel | draft new WG and IG charters | 2003-07-16 |
| AI-20030617-1 | MS | to develop test assertions for Test GL | 2003-09-01 |
| AI-20030617-2 | LH | to ask Ian Jacobs for clarification regarding LC29.4 by email | 2003-06-20 |
| AI-20030617-3 | DH | ask Ian Jacobs why he doesn't want to use RFC keywords and in case there are good reasons go for the mapping | 2003-06-24 |
| AI-20030617-4 | Mark | rewrite SpecGL 8.1 | 2003-06-30 |
| AI-20031018-1 | Mark, Lynne | look for availability in their Boulder Office for a F2F | 2003-07-11 |
| AI-20031018-2 | Patrick | look for availability in Boulder and California locations for a F2F | 2003-08-01 |
| AI-20031018-3 | Karl | to create the markup and the style for a complete CP | 2003-06-25 |
| AI-20031018-4 | Lofton | integrate the new markup/styling | 2003-07-15 |
| AI-20031018-5 | Dom | to integrate the new markup/styling | 2003-08-01 |
| AI-20031018-6 | Dom | to reply to Pat Hayes' email | 2003-06-26 |
| AI-20031018-7 | Lofton | to reply to David that the QA WG is interested in TCDL and will have reviews on the proposed TOC | 2003-06-256 |
| AI-20031018-8 | Patrick | to find a reviewer at Sun on the TCDL Proposal | 2003-08-01 |
| AI-20031018-9 | Dimitris | review the TCDL Proposal | 2003-07-01 |
| AI-20031018-10 | Dimitris | make a review of the Dom level 3 Core draft against the latest SpecGL | 2003-07-24 |
The Raw minutes for this part of the meeting are available in the QAWG mailing-list archives. This document summarizes them in a more readable form.
Daniel reported about the latest Advisory Commitee Meeting (meeting with representatives of W3C membership). The topic of certification was addressed there.
The WG tried to identify a potential candidate for a certification experiment and thougt of SVG tiny as a good candidate.
The discussion moved to the general question of the interest of certification versus education, since they share the goal of better implementations. Better communication, more advertising, easier-to use, free tests may achieve more usage and passage of the tests, but does not preclude certification later.
Some expressed their impression that neither W3C specifications nor the test suites are mature enough to ensure a proper certification program.
The group reviewed the present scope of the IG in order to prepare its rechartering.
Analysis as follows:
The Raw minutes for this part of the meeting are available in the QAWG mailing-list archives. This document summarizes them in a more readable form.
Need to convince people that QA now is cheaper than later. The "cost" later is lack of interoperability, and the saving is in test development (share the burden).
We want to write a document on this topic, addressed to both monopolists (e.g "we only need to develop for this product") and non-monopolists. Daniel will draft an outline.
Discussing policies (notably anti-spam mechanisms) for our lists, and the scope of our lists in general. No specific decision taken.
The group discussed the status and plan for completion of the QA framework.
And then remains the question of what the documents will be in the end... Recommendation or Note. (No decision reached this time)
The group reviewed the present scope of the WG in order to prepare its rechartering.
Analysis as follows:
The Raw minutes for this part of the meeting are available in the QAWG mailing-list archives. This document summarizes them in a more readable form.
The QAWG is in the process of rechartering and we want to conform to the QA OPs Guidelines. We will comply to the triple-A conformance. Daniel will write a draft charter and we will fill the holes in it.
Later in the discussion the group tried to define what "Test materials" mean for the Spec and Ops Guidelines. Our Examples and Techniques? Do we have Test Materials? Is the Check Point on giving Test Material applicable?
The discussion moved to the question of whether the Techniques were sufficient Test Material.
If the techniques do not cover the Test needed for a particllar CheckPoint, we clause which says that you can explain the techniques you have used to test it. But what's happening when the technique used is wrong. Is the test bad? Is the CP bad? The discussion is trying to nail all details of this.
Question: What is TM? Resolution: a questionnaire asking how the individual requirements are met.
Processing the Test Assertion spin-off issues..
Any specifications which want to be AA compliant have to include Test Assertions but at the same time it's a big burden. Test Assertions have a value for good specifications.
It allows to improve the quality of the spec itself by making people writing a better specification.
Should the CPs must be P2 or P3? What is the exact value of this CP and how much does it require to do it?
SpecGL is not to make better Test Suites but better Spec. Are the testable assertions useful for the clarity of the language used in the spec?
P2: No disagreement
Warning wording: No disagreement
Auto-generation: Agreed on the wording done by Lynne.
Exemption from TA: This is not applicable to specification written in a formal language that allows for automatic generation of test materials
The Raw minutes for this part of the meeting are available in the QAWG mailing-list archives. This document summarizes them in a more readable form.
Issue closed by the following action item : Mark to develop test assertions fo Test GL, completed by September 1
dom: providing means to have a link from the specification to a separate document
lynne: need to include in the explanation of the checkpoint that they can be provided either in the document or by reference.
Need to Clarify this in the terminology. The Editors will add something to this effect.
pc: identifying and labelling is necessary
lh: 6 terms in use, the only one defined is "test assertion"
lr: someone needs to draft the definition of "conformance requirement"
"conformance requirement" = necessary or recommended optional conditions for conformance to the specification
Lofton will to ask Ian for clarification regarding LC29.4
(done)
lh: conformance requirements do not have to use 2119 keywords. testable statements is undefined.
dom: if there are good reasons, provide mapping between RFC keywords and the conformance requirements
Dom will ask Ian why he doesn't want to use RFC keywords and in case there are good reasons, will go for the mapping
top definition goes into the main glossary.
We'll make our definitions consistent.
(closed)
(resolved)
Reorder the guidelines with the current numbers to be GL13, GL10+GL3, GL11+GL12
agreement to add wording explaining that prioritites 1-3 are included to progressively go from good to better to best as far as qualitative assessment is concerned
The Raw minutes for this part of the meeting are available in the QAWG mailing-list archives. This document summarizes them in a more readable form.
Disposition of Comments: Aug 1st
Next Publication to WG : Sept 1st
The group went through the current TestGL draft, and reviewed the document structure and content. The Test Guildelines deal with analysis, structure, and management of the test suite.
The group discussed different wordings, changes of priorities, etc. see the raw minutes for the details.
These comments apply to a previous TestGL draft
DM-1.1 Agreed
DM1.2 Need to make sure we cover versioning. Should metadata tie back to SpecGL DoV? We talk about filtering, but should filtering on the DoV allowed by the specification be called out? Yes. Want to include or exclude tests based on DoVs. Need to include a test purpose or description.
GL4: talking about test case review. Can say something about where get test data from put this is anaylsis guideline.
CP5.4: should there be mention of storage of results? Covered in test results management. We don"t mention anything about coverage. Need to add something about coverage.
GL4: concepts have been clarified.
CP4.3 Agreed. Add a sentence to clarify.
CP5.1 Need to clarify and define terms. Use some of the suggested words.
1. Addressed
2. Agreed
3. need to look at intro
4. Relates to coverage and strategy. Will say something about coverage.
5, 6, 7. Agreed
WG draft by late September
Need ExTech document prior to first call
First Last Call: ??
Disagreement on whether a scope can be a set of requrements. Class of products also included interoperability. Intended audience, delete conformance. Remove or move last paragraph of 1.4. Sections 1.5, 1.6, 1.7 make sure that wording is consistent with other documents. Need to create use cases for TestGL.
The group agreed that the definitions needed work, and the Conformance Section needed to be reviewed.
The Raw minutes for this part of the meeting are available in the QAWG mailing-list archives. This document summarizes them in a more readable form.
The group studied these possible locations :
NIST people will check the availability of in their Boulder Office for a F2F, and Patrick in Boulder and California locations, for a meeting around mid-October (for last call planning of TestGL)
General agreement that karl's proposal for the style of our Guidelines is good. The proposal is thus adopted, and other categories ("examples", "related CP") will be added to Karl's final work.
Dom will reply to Pat Hayes' email by June 26
This work is something considered as part of what the WG should work on, and the QA WG would welcome it as a submission.
It was decided that Lofton will send this reply to David, while Patrick will find reviewer at SUN for it, and Dimitris will review it too.
Dimitris will make a review of the Dom level 3 Core draft against the latest SpecGL.
It was decided that the chairs should check before each telecon which action items are overdue so that the AI list doesn't fill up indefinitely
As a preparation for the charter of the Test Task Force (TTF), there was a discussion on the prioritarization of the possible deliverables, based on easyness vs usefulness.
Results of a quick poll as follows, for each participant:
Note regarding Test assertion spec markup: there are two different projects: authoring spec with marking up of test assertions vs identifying test assertions in an existing spec with specific metadata (annotating).
Meeting adjourned.
Created Date: 2003-03-25 by Olivier Thereaux