Draft minutes: F2F, 17-june, AM

Minutes:  F2F Santa Clara QAWG Meeting
17 June 2004, AM

Scribe: Lynne Rosenthal
Summary of new action items:
AI-20040617-1 PC to extract from minutes useful info on metadata classes 
into the Wiki, by 2004-06-30
AI-20040617-2 LH to respond to Jeremy's QAH comments by 2004-06-30

Thunderous applause and thanks to Patrick and Sun Microsystems for hosting 
the meeting, with excellent facilities and hotel and for ensuring we never 
went hungry or thirsty by providing refreshments and lunch arrangements.

*Test Case Model
http://lists.w3.org/Archives/Public/www-qa-wg/2004Jun/0025.html

There are 2 levels of metadata:  metadata for test case and metadata for 
test suite.
Graph represents the metadata for the test case.
Is a 'created date' important?  Yes, it should be optional.

**What's the difference between Description and Purpose?

**What is TypeOf?
Encourage people to declare their test suite model and metadata.  This 
would be useful to people using the test suite.

**What's the content of Expected Results?
Dependent on the test suite.  Sometimes coded into test, sometimes external 
file or description.  In XSLT, to enable automatic comparison, they 
invented a binary infoset to put the results that are derived from a tree 
of all possible results.  A cannalization of the results.
Another view of expected results is what is being done with the SGML 
Validator.  Since test materials include validators, this needs to be 
addressed.  With validator, is valid/not valid equal to P/F.  In SVG, it is 
the Reference Result that is compared.  It's a good idea to document the 
model.  But we should not impose a test case model, but useful to document 
one to show its relevance and useage, illistrating through examples the 
possible approaches.  This is part of the documentation.  We can give 
guidance, although it would be great if we could reference/provide tools

Interesting to focus on the classes of information we think is important 
and why useful.  Four possible classes of metadata are to enable:
1. test harness - automate the test execution.
2. sorting/filtering of tests based on profiles, subsets, conformance 
designations, or other areas of interest.
3. versioning  select specific version or errata related tests
4. coverage  i.e., ability to tie to assertions and derive coverage 
numbers.  Continue discussion on Wiki to discuss different classes, 
examples, etc.

** What are the different values of Status
Submitted, reviewed, rejected.  There was a list of status in an earlier 
version.

**Why there's no property for modified ? or should a date be attached.
There is software to track changes, so don't need to reinvent this.  Could 
suggest using CVS.

In the model, there is no reference to the spec.  One of the most critical 
and often overlooked references is to the errata.  The metadata for the 
reference to the spec and errata is in the Wiki.  It would be useful to 
have id's in the specification to link to and/or to have assertions in the 
spec.

**Conformance requirements metadata.
Not sure what this is, but may be related to the different levels of 
conformance (A, AA, AAA).  If there are conformance designations, metadata 
should be able to handle this.  This is handled in the 'classes of 
metadata  filtering/sorting.

Don't reinvent the wheel  where it makes sense, use the Dublin Core 
metadata categories. Not requiring the use of RDF as your technique for 
encoding the metadata.

**What is the Priority of a test?
Labeling areas of a spec for what is more important than other areas and 
then use these labels to determine where to focus development efforts.


Issues List and Responses.
In Issues list for the documents, should we capture the results of the 
response to the issues responses?  We need to record somewhere, in one 
place, all the decisions we have made and whether the resolution has been 
accepted.  Propose a central index document with the links to the email to 
the commenter and their response.  Lofton to initiate for QAH and Lynne to 
continue for SpecGL.

Recruitment.
Lynne/Mark to contact Jacques Durant at Fujitsu.
Mark to contact Don Deutsch at Oracle.
Contact Jim Bell (HP), GMD AC member
Dom to pursue patent policy declaration from IBM

Meeting adjourned