<ginny> scribe: Virginia Smith

<ginny> scribenick: ginny

<scribe> meeting: W3C SML Teleconference of 2008-09-18

<MSM> is anyone speaking?

Approval of minutes from Sep 11

RESOLUTION: minutes approved

<MSM> scribe: ginny_

<MSM> scribenick: ginny_

<MSM> scribenick: ginny

http://www.w3.org/Bugs/Public/show_bug.cgi?id=6056

RESOLUTION: The working group agrees to endorse this bug.

<johnarwe_> http://lists.w3.org/Archives/Public/public-sml/2008Apr/att-0069/sml-interop-test-plan.pdf

Continue with test case discussion

<ginny_> scribe: ginny_

<scribe> scribenick: ginny_

<ginny> current location: page 3 section 5

<ginny> MSM: does not believe that test results should be only boolean valid/invalid

<ginny> ... e.g. should include errors/messages

<ginny> Ginny: you're talking about manual comparison in this case

<ginny> Pratul: in previous interop workshop, the implementations provided diagnostics and were manually compared

<ginny> Ginny: agree that this information would be useful even if not automate-able

<ginny> Sandy: 2 kinds of validity results - model and SML-IF document

<ginny> Kumar: where does SML-IF validation come in?

<scribe> scribe: ginny_

<scribe> scribenick: ginny_

Kumar: 2 kinds of test cases - sml and sml-if

MSM: suggest discussing this in the future: what the testable assertions are in sml and sml-if

<johnarwe_> we revised 3-4 last week with much discussion

<johnarwe_> MSM text pasted from process doc

<MSM> Entrance criteria: The Director calls for review when satisfied that the Working Group has:

<MSM> Fulfilled the general requirements for advancement;

<MSM> Shown that each feature of the technical report has been implemented. Preferably, the Working Group should be able to demonstrate two interoperable implementations of each feature. If the Director believes that immediate Advisory Committee review is critical to the success of a technical report, the Director may accept to Call for Review of a Proposed Recommendation even without adequate implementation experience;

<MSM> Satisfied any other announced entrance criteria (e.g., any included in the request to advance to Candidate Recommendation, or announced at Last Call if the Working Group does not intend to issue a Call for Implementations).

<ginny> discussion of # of implementations for optional features

<ginny> moving to section 6

<johnarwe_> perhaps add "...in order to be a complete impl"?

<scribe> scribe: ginny_

<scribe> scribenick: ginny_

<ginny> MSM: can have a partial implementation that implements a feature as contributing toward the "2 implementations" test.

<ginny> The sentence under discussion is "Each participating implementation must pass all tests in this section."

<MSM> Possible new wording:

<MSM> 1. Tests for required features:

<MSM> These features are "required" in the sense that conforming processors

<MSM> must support them. If any partial implementations participate in the

<MSM> testing effort, those partial implementations may not pass all of

<MSM> these tests.

<scribe> scribe: ginny_

<scribe> scribenick: ginny_

<ginny> MSM changes above are accepted by the group

<ginny> discussing section 6, line 14

<ginny> MSM: should think about requiring specific behavior when an implementation does not support an optional feature. ignore? error?

<ginny> ... this is probably a spec change.

<johnarwe_> "spec" change is just words... the interesting question is, would such changes be -substantive-

<scribe> scribe: ginny_

<scribe> scribenick: ginny_

<ginny> Ginny: preference is to use IF format for tests

<ginny> MSM: also consider embedding metadata if desired

<ginny> Pratul: anyone disagree with using IF format?

<ginny> RESOLUTION: agree to use IF format for test cases rather than defining another test case format

<ginny> discussion of Kirk's comments around interoperability - interoperabilty of models or implementations

<ginny> MSM: suggests defining a notion of interoperability as a triple of model, producer, and consumer

<scribe> scribe: ginny_

<scribe> scribenick: ginny_

Pratul: in previous interop, did round-trip testing between 2 implementations

<ginny> John: need to define equivalence of the models

<ginny> scribe: ginny

<scribe> scribenick: ginny

MSM: just a heads up that W3C is looking at definition of interoperability but we should proceed to define this for our purposes

<johnarwe_> just for context in minutes: this discussion is based on http://lists.w3.org/Archives/Public/public-sml/2008Apr/0087.html

<scribe> ACTION: MSM to define "interoperability" for SML and SML-IF testing purposes [recorded in http://www.w3.org/2008/09/18-sml-minutes.html#action01]

<trackbot> Created ACTION-201 - Define \"interoperability\" for SML and SML-IF testing purposes [on Michael Sperberg-McQueen - due 2008-09-25].

moving to John's comments in second draft http://lists.w3.org/Archives/Public/public-sml/2008May/att-0017/sml-interop-test-plan20080504.doc

<ginny_> scribe: ginny_

<scribe> scribenick: ginny_

<johnarwe_> ok pdf is in the pipe

<johnarwe_> to public-sml and direct cc to msm

<ginny> scribe: ginny

<scribe> scribenick: ginny

Discussion of John's comments in the document

Section 1

Ginny and MSM agree with comments

Section 2

Kumar has some reservations on option 3 (and also option 1 but we've agreed not to use option 1)

<ginny_> scribe: ginny_

<scribe> scribenick: ginny_

<ginny> scribe: ginny

<scribe> scribenick: ginny

Kumar: option 3 requires more work when most test cases are embedded files anyway

John: option 3 does not require that we use locator but gives the option to do the work required if needed

MSM: will need both embedded and locator in the case where we test the locator element

<pratul> I will be representing Microsoft for the rest of the call since Kumar has left the call

Section 3

John: implementation characteristics would be, e.g., features implemented

MSM: xquery does this by saying one possible values of test result is "does not apply"

Ginny: need to consider effort versus value derived

<ginny_> scribe: ginny_

<scribe> scribenick: ginny_

<ginny> John: not discussing final output format at this time

<ginny> Section 6

<ginny> John: in previous interop workshop, they manually reviewed the error output and decided if it was the 'same' or not

<scribe> scribe: ginny_

<scribe> scribenick: ginny_

Ginny: is comparison of errors in scope for this testing?

<pratul> Manual/human comparison is OK

<ginny> Ginny: Is manual checking part of 'passing the test'?

<ginny> Pratul: this is not a compliance test suite.

<johnarwe_> http://lists.w3.org/Archives/Public/public-sml/2008May/att-0017/sml-interop-test-plan20080504.doc

<johnarwe_> hang on

<johnarwe_> http://lists.w3.org/Archives/Public/public-sml/2008May/0023.html

<johnarwe_> last email in the list :-)

<scribe> scribe: ginny_

<scribe> scribenick: ginny_

<ginny> ACTION: Kumar to update test case document to incorporate any specific decitions made over the past 2 weeks and to add a list a list of decisions still to be made by the working group [recorded in http://www.w3.org/2008/09/18-sml-minutes.html#action02]

<trackbot> Created ACTION-202 - Update test case document to incorporate any specific decitions made over the past 2 weeks and to add a list a list of decisions still to be made by the working group [on Kumar Pandit - due 2008-09-25].

This is scribe.perl Revision: 1.133  of Date: 2008/01/18 18:48:51  
Check for newer version at http://dev.w3.org/cvsweb/~checkout~/2002/scribe/

Guessing input format: RRSAgent_Text_Format (score 1.00)

Succeeded: s/implentations/implementations/
Succeeded: s/defiing/defining/
Succeeded: s/cy/ce/
Found Scribe: Virginia Smith
Found ScribeNick: ginny
Found Scribe: ginny_
Found ScribeNick: ginny_
WARNING: No scribe lines found matching ScribeNick pattern: <ginny_> ...
Found ScribeNick: ginny
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny
Inferring ScribeNick: ginny
Found ScribeNick: ginny
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny
Inferring ScribeNick: ginny
WARNING: No scribe lines found matching previous ScribeNick pattern: <ginny_> ...
Found ScribeNick: ginny
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny
Inferring ScribeNick: ginny
WARNING: No scribe lines found matching previous ScribeNick pattern: <ginny_> ...
Found ScribeNick: ginny
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
Found Scribe: ginny_
Inferring ScribeNick: ginny_
Found ScribeNick: ginny_
WARNING: No scribe lines found matching ScribeNick pattern: <ginny_> ...
Scribes: Virginia Smith, ginny_, ginny
ScribeNicks: ginny, ginny_
Default Present: MSM, pratul, +1.845.433.aaaa, +1.603.823.aabb, Kirk, Sandy, johnarwe_, Ginny_Smith, +1.425.836.aacc, Kumar
Present: MSM pratul +1.845.433.aaaa +1.603.823.aabb Kirk Sandy johnarwe_ Ginny_Smith +1.425.836.aacc Kumar

WARNING: Replacing previous Regrets list. (Old list: Jim, Julia, Kumar)
Use 'Regrets+ ... ' if you meant to add people without replacing the list,
such as: <dbooth> Regrets+ Jim, Julia

Regrets: Jim Julia
Agenda: http://lists.w3.org/Archives/Public/public-sml/2008Sep/0014.html
Got date from IRC log name: 18 Sep 2008
Guessing minutes URL: http://www.w3.org/2008/09/18-sml-minutes.html
People with action items: kumar msm

WARNING: Input appears to use implicit continuation lines.
You may need the "-implicitContinuations" option.