I think this change should wait for the SML-IF spec to stabilize with regard to consumer and producer definitions and requirements.

The SML-IF spec does not appear to require the sml:uri scheme. It only requires a scheme based on xs:anyURI which I don't think is the same thing. SML references are not mentioned in the SML-IF spec - it talks about inter-document references. We should reconcile this terminology.

From the SML-IF spec:
[Definition: In the context of SML-IF, an inter-document reference is any content in a document in the interchange set whose type is xs:anyURI or a type derived from xs:anyURI and whose context in the document implies that the URI can (given the right permissions and connectivity) be dereferenced using the default retrieval action for the URI's scheme.]

...

SML [SML 1.1] defines two reference schemes, the sml:uri scheme and the EPR scheme. It also permits new schemes to be created without limit. Schemes that do not use URIs or whose use of URIs does not imply that the URIs may be dereferenced for retrieval using the default action (e.g., for the HTTP scheme, the GET method) are not inter-document references in the context of SML-IF. Three consequences flow from this.

First, to successfully interchange documents using such schemes, the sml:ref elements containing them MUST also contain an sml:ref scheme that is an inter-document reference in the SML-IF context. For example, an sml:ref that contains an EPR scheme reference (which is not an inter-document reference in SML-IF) could also contain an sml:uri scheme reference (which is).

Second, the producer of the SML-IF document and the consumer of must agree on the scheme(s) being used since SML-IF only requires consumers to understand the sml:uri scheme.

Third, when creating a new sml:ref scheme, authors MUST be explicit about whether the scheme is an SML-IF inter-document reference.

SML-IF (submission: 3.3.3, latest draft: 3.4.3) requires all consumers to implement the sml:uri ref scheme when it says:
Second, the producer of the SML-IF document and the consumer of must agree on the scheme(s) being used since SML-IF only requires consumers to understand the sml:uri scheme.

Agree this is "challenging" to find in its current state, hence this bug.  Probably belongs in a conformance section.

Agree that inter-doc refs need to be "reconciled" with sml refs, however I think that is a separate bug that needs to be opened.  Sandy was going to take a crack at a proposal for idrs ala his sml refs proposal, as of last week.

Resolution: both consumers and producers are required to support sml uri scheme; a producer should be able to produce IF using sml uri scheme; define 2 levels of conformance for the IF documents; mark the defect editorial

5.3.3 SML reference schemes that are not SML-IF interdocument references

5.3.3 SML reference schemes that are not SML-IF interdocument references
now reads 

. . . Second, the producer of the SML-IF document and the consumer of must agree on the scheme(s) being used. SML-IF defines two levels of conformance regarding the sml:uri scheme. Level 1 conformant producers support all of the SML-IF scheme with the exception of the sml:uri scheme. Level 2 conformant producers support all of the SML-IF scheme including the sml:uri scheme. All consumers MUST support the sml:uri scheme. . . .

what is "SML-IF scheme" ?

Regarding John's question "What is an SML-IF scheme?" I think what was in my head was that a Level 1 producer conforms in every other way but does not support the sml_uri scheme, although in rereading this it this sentence is not clear. For background from the face2face see:
http://www.w3.org/2007/10/16-sml-minutes.html#item13
I'll rework this - all comments and suggestions welcome.

"Level 1 conformant producers support all of the SML-IF scheme with the exception of the sml:uri scheme."

Does this mean that a level 1 producer must support *all* SML ref schemes except sml:uri?

Reading this comment makes it clear just how bad the wording is on this sentence. I now believe we or not clear on what the minimum is for Level 1 conformance. The sentence I am trying to complet is:
"Level 1 conformant producers support ..., not the sml:uri scheme.

How about:

Level 1 conformant producers MAY support any SML reference schemes. Level 2 conformant producers MUST support at least the URI reference scheme.

I thought that the conclusion we came to was that Level 1 conformance meant conformance to the SML-IF schema (with an "a") and Level 2 conformance meant support for the sml:uri scheme by BOTH consumers and producers.  (We back-tracked on the earlier decision that only producers would have to support the sml:uri scheme once we adopted the 2 level conformance standard.)

Final Wording:
SML-IF defines two levels of conformance regarding the sml:uri scheme:

All consumers MUST be able to process IF documents using the sml:uri scheme.

To be Level 1 conformant a producer MUST be able to produce IF documents using the sml:uri scheme.

To be Level 2 conformant a producer MUST produce SML-IF documents which use the sml:uri scheme for all SML references.

I think this still needs a review.

1 - the resolution states that we agreed on 2 levels of conformance on the IF document not on the producer. The IF document using a uri scheme for all refs is a level 2 conforming document. An IF doc that does not provide a uri scheme for at least 1 SML ref is a level 1 conformant document. (assuming it conforms to all other conformance criteria, of course) 

2 - A conformant producer must be -able- to produce an IF document using uri scheme but a producer that produces an IF document not using uri scheme (if requested to do so), in my mind is still a conforming producer. It is just not producing a level 2 conformant document at that moment. 

3 - This needs to be reconciled with the following statement in the same section which does not require the uri scheme, per se:
First, to successfully interchange documents using such schemes, the sml:ref elements containing them MUST also contain an sml:ref scheme that is an interdocument reference in the SML-IF context. 

4 - This bug fix interferes with bug 5121 which removed this section. Need to get this straightened out.

Agree with 1 and 2 in Comment #14

Resolution: See 4 points in: http://lists.w3.org/Archives/Public/public-sml/2007Oct/0261.html
but amended as follows:
- in bullet 3 and 4, should say "non-null SML references"

Conformance Criteria section now reads:

5.1 Conformance Criteria

SML-IF defines two levels of conformance for SML-IF Documents:

   1. A level 2 conforming SML-IF Document MUST adhere to all requirements in this specification as described in the normative sections. In particular, all non-null SML references in the document MUST contain a URI scheme representation [SML 1.1].
   2. A level 1 conforming SML-IF Document MUST adhere to all requirements in this specification as described in the normative sections with the exception that a URI scheme representation [SML 1.1] is not required for SML references. An SML-IF Document that contains at least one non-null SML reference which has no URI scheme representation but satisfies all other SML-IF requirements is a level 1 conforming SML-IF Document. 

A conforming SML-IF Producer MUST be able to generate a level 2 conforming SML-IF Document from an SML model.

A conforming SML-IF Consumer MUST process a conforming SML-IF Document using, in whole or part, semantics defined by this specification. It is OPTIONAL that a conforming SML-IF Consumer process all elements defined in this specification, but any element that is processed MUST be processed in a manner that is consistent with the semantics defined here. 

I don't think we should have 2 levels of compliance for SML-IF documents. This does not help with regard to interoperability which is SML-IF's primary goal. Rather, this dilutes the interoperability that SML-IF brings to the table.

I propose the following 2 changes:

=================
5.1 Conformance Criteria

A conforming SML-IF Document MUST adhere to this specification as described in the normative sections. 

A conforming SML-IF Producer MUST be able to generate a conforming SML-IF Document from an SML model.

A conforming SML-IF Consumer MUST process a conforming SML-IF Document using,
in whole or part, semantics defined by this specification. It is OPTIONAL that
a conforming SML-IF Consumer process all elements defined in this
specification, but any element that is processed MUST be processed in a manner
that is consistent with the semantics defined here. 



================ 
Add new section - insert after section 5.2

5.3 SML References

An SML reference can contain multiple representations using different reference schemes. SML-IF requires that all non-null SML references in the interchange set MUST contain an SML URI scheme [SML 1.1] representation. Any SML reference MAY also include other scheme representations as well.

There are a few consequences we need to be aware of, if we require that all non-null references in IF documents MUST use the URI scheme.

1. It may not be easy for producers to satisfy this requirement (Kirk's comment "That seems to be calling on the SML-IF Producer to do the impossible." [1])

If the producer understand scheme(s) used in the SML model, then it's possible to perform this using John's algorithm [2]. It may be a big performance burden though.

But in case when the producer is presented with a model using schemes it doesn't understand, then it's impossible for it to add the correct <sml:uri> elements.

2. Requiring modification to the embedded document could be dangerous.

This could affect signatures on the embedded documents. This could also invalidate the embedded documents. (Consider the case where the schema says my reference element must only contain an instance of <my:uri>.)

3. It's also very difficult to achieve if documents are included by reference.

If the producer is handed an URL for the document to be referenced, does it have to modify the document that that URL, or make a copy of that document and expose the new URL? (How to determine whether the new URL is accessible by the consumer?)

4. It discourages usage of other useful schemes

Currently the URI scheme uses a subset of XPath 1.0. What if people want to experiment with a bigger (or smaller) subset, or start using XPath 2.0? Different vendors could agree on a new scheme and start using it before it's adopted by the next version of the spec. Do we want to clearly disallow that kind of behavior?


Base on these, I think I'm not convinced that specifying only 1 level of conformance is the right decision.

On the other hand, I do see the value of inter-op and the desire to have inter-op for conforming documents. Maybe a compromise could be to still define 2 levels, but instead of calling them level 1 vs. 2, they can be given names that reflect the degree of inter-op implied by these levels. (Again, very bad at coming up with names.)

Latest Proposal (also changed bug to needsAgreement to more accurately reflect the bug status:

-----------conformance section ------------
5.1 Conformance Criteria

SML-IF defines two levels of conformance for SML-IF Documents:
1. Full Conformance: A full conforming SML-IF Document MUST adhere to all requirements in this specification as described in the normative sections. In particular, all non-null SML references in the document MUST contain a URI scheme representation [SML 1.1]. 
2. Minimal Conformance: A minimally conforming SML-IF Document MUST adhere to all requirements in this specification as described in the normative sections with the exception that a URI scheme representation [SML 1.1] is not required for SML references. An SML-IF Document that contains at least one non-null SML reference which has no URI scheme representation but satisfies all other SML-IF requirements is a minimally conforming SML-IF Document. 

A conforming SML-IF Producer MUST be able to generate fully conforming SML-IF Document from an SML model.

A conforming SML-IF Consumer MUST process a fully or minimally conforming SML-IF Document using, in whole or part, semantics defined by this specification. It is OPTIONAL that a conforming SML-IF Consumer process all elements defined in this specification, but any element that is processed MUST be processed in a manner that is consistent with the semantics defined here.
================

---------non-normative text -----------
The goal of SML-IF is to enable the exchange of SML models. However, this interoperability goal is affected by several aspects of SML models:

• The use of reference schemes for SML references. Use of the URI Reference Scheme as defined in the SML specification is the only guaranteed way of achieving interoperability for all SML references in the model. Use of any other reference scheme requires that the consumer know about its use in the document and understand how to dereference it.

• SML documents can be included by reference using the locator element and, therefore, are not directly embedded in the SML-IF document. This can be very useful, especially when the SML-IF document is large or when the documents are readily accessible to the consumer. However, it is unclear what the SML-IF producer should do with the SML references in this referenced document. It may not always be possible to add a URI reference scheme representation to all SML references that do not use this scheme.

• The SML-IF document may be schema-incomplete [see section ???]. An SML model represented by a schema-incomplete SML-IF document is not necessarily invalid. However, SML-IF cannot guarantee interoperability for a schema-incomplete SML-IF document.

Comments from Valentina and Sandy:

http://lists.w3.org/Archives/Public/public-sml/2007Dec/0089.html

I'm glad Ginny marked this needsAgreement.

I think the text below is subtly confusing two distinct ideas, conformance and interop.  Conformance is something we define, and we should not be changing it at this late stage.  Interop is something we can describe conditions for, conditions which either help or hinder.  1, 2, and 2i are conformance definitions, essentially re-statements of text we've had for months.  The full set of 2, 2i, 2ii, and 2iii describe conditions under which a document will be widely interoperable, also text we've had or discussed for some time.  It happens to be true that widely interoperable documents will be "fully conforming" (i.e. old level 2) documents, since 2i is part of both definitions, but the reverse does not automatically follow.  I see no reason to assert that a document exploiting <sml:locator> for example is not fully conformant; it is true that (because locator support is optional) that such a document will not be as widely interoperable as one that embedded everything, but the two are separate and distinct (although related) issues.  As Kirk aptly put it, there is a spectrum of interop.

I propose we keep the definitions of 1, 2, and 2i for conformance, copy 2i to the new interop section, and MOVE 2ii and 2iii to the new interop section.  That  gets us back to the old level 2 definition of conforming, and let's us talk about interop separately.

SML-IF defines two levels of conformance for SML-IF Documents:
1.	Minimal Conformance: A minimally conforming SML-IF Document MUST adhere to all requirements in this specification as described in the normative sections. 
2.	Full Conformance: A fullly conforming SML-IF Document MUST adhere to all requirements in this specification as described in the normative sections. In addition, it MUST satisfy the following conditions:
i.	Each non-null SML reference in the document is an instance of the SML URI reference schema [SML 1.1].
ii.	All documents are included as embedded documents [ref 5.2.1].
iii.	The schemaComplete attribute (/model/definitions/@schemaComplete) has an actual value "true".

Created attachment 502 [details]
Updated conformance level definition proposal

Please see the attached (http://www.w3.org/Bugs/Public/attachment.cgi?id=502) for the updated proposal based on John and Ginny's comments.

Note that only part of the document applies to this bug. The rest applies to bug 5306 and needs to be considered separately.

I believe the wording regarding conformance vs. interoperability in the attachment in comment 24 is now acceptable (though are we sure we like "minimally" conforming?)

I would also add the discussion of the proposed <referenceScheme> Header element (its XSDL type (QName vs. URI / single value vs. list) and the semantics of multiple occurences of the element) to 5306, or maybe it should be a separate issue.

Note that comment #24 is not correct.

See latest proposal from Sandy and my comments at: http://lists.w3.org/Archives/Public/public-sml/2008Jan/0012.html

Latest proposal (extracted from the attachment "Updated conformance level definition proposal"):

5.1 Conformance Criteria

SML-IF defines two levels of conformance for SML-IF Documents:

1. Minimal Conformance: A minimally conforming SML-IF Document MUST adhere to all requirements in this specification as described in the normative sections. 
2. Full Conformance: A fully conforming SML-IF Document MUST adhere to all requirements in this specification as described in the normative sections. In addition, each non-null SML reference in the document MUST be an instance of the SML URI reference scheme [SML 1.1].

A conforming SML-IF Producer MUST be able to generate fully conforming SML-IF Document from an SML model.

A conforming SML-IF Consumer MUST process a fully or minimally conforming SML-IF Document using, in whole or part, semantics defined by this specification. It is OPTIONAL that a conforming SML-IF Consumer process all elements defined in this specification, but any element that is processed MUST be processed in a manner that is consistent with the semantics defined here. 


Interoperability of SML Models (this is non-normative text):

The goal of SML-IF is to enable the exchange of SML models. However, this interoperability goal is affected by several aspects of SML models:
 The use of reference schemes for SML references. Use of the URI Reference Scheme as defined in the SML specification is the only guaranteed way of achieving interoperability for all SML references in the model. Use of any other reference scheme requires that the consumer know about its use in the document and understand how to dereference it.
 SML documents can be included by reference using the locator element and, therefore, are not directly embedded in the SML-IF document. This can be very useful, especially when the SML-IF document is large or when the documents are readily accessible to the consumer. However, the locator element may be ignored by the consumer or may not resolve or may resolve to different resources in different contexts. Because of these uncertainties, interoperability is not guaranteed when documents are included by reference.
 The SML-IF document may be schema-incomplete [see section ???]. An SML model represented by a schema-incomplete SML-IF document is not necessarily invalid. However, SML-IF cannot guarantee interoperability for a schema-incomplete SML-IF document.

The names "Minimal Conformance"/"Full Conformance" seem to be unnecessarily evaluative and, in the case of the former, may convey undesirable connotations ("I just did enough to get by to call myself conformant.").  More objective names should be selected that describe what is going on in these types of conformance.   The current names might also imply a scale of conformance from minimal to full with possible inbetween levels.

For "Minimal Conformance" I would suggest "Specification Conformance" or just "Conformance".  For "Full Conformance", we might be use "Reference Conformance" or "URI Conformance."  The difference between just "Conformance" and "Reference/URI Conformance" would then also convey the idea that "Reference Conformance" is an extension of (standard) conformance related to reference schemes. 

In the third public draft of SML-IF, the conformance definitions read something like the following:

A level X  conforming SML-IF Document MUST adhere to all requirements in this specification as described in the normative sections. . ..

(Just by way of a note: there is no definition of a normative section.)
 
The conformance definition states that a document needs to adhere to "all requirements" in this specification in any and all normative sections.  This statement needs some qualification because there seems to be two types of normative statements: (1) statements directly about the structure and characteristics of an SML-IF document, and (2) statements about the behavior of consumers and producers of SML-IF documents.  Often, such these types of statements are mixed in the same section, for example in sections 5.2.1 and 5.2.2.  Obviously a conforming document cannot adhere to statements of type 2.  Thats obvious, BUT the definition still says "all requirements in this specification".  Even if we assume that it will be clear to the reader what requirements apply to the document itself and which we do not, the definitions needs to make the distinction more explicit.  

 

Resolution: conformance names are "minimal conformance" and "reference conformance". Also, make editorial fix per comment #29.

Added a new non-normative section:
=========
4.5 Interoperability of SML Models

The goal of SML-IF is to enable the exchange of SML models. However, this interoperability goal is affected by several aspects of SML models.

   1. The use of reference schemes for SML references. Use of the SML URI Reference Scheme as defined in the SML specification is the only guaranteed way of achieving interoperability for all SML references in the model. Use of any other reference scheme requires that the consumer know about its use in the document and understand how to dereference it.
   2. SML documents can be included by reference using the locator element and, therefore, are not directly embedded in the SML-IF document. This can be very useful, especially when the SML-IF document is large or when the documents are readily accessible to the consumer. However, the locator element may be ignored by the consumer, may not resolve, or may resolve to different resources in different contexts. Because of these uncertainties, interoperability is not guaranteed when documents are included by reference.
   3. The SML-IF document may be schema-incomplete [4.4 Schema Document Bindings]. An SML model represented by a schema-incomplete SML-IF document is not necessarily invalid. However, SML-IF cannot guarantee interoperability for a schema-incomplete SML-IF document.

=========

Section 5.1 now reads:
==========
5.1 Conformance Criteria

SML-IF defines two levels of conformance for SML-IF Documents:

   1. Minimal Conformance: A minimally conforming SML-IF Document MUST adhere to all SML-IF document requirements as described in the normative sections of this specification.
   2. Reference Conformance: A referentially conforming SML-IF Document MUST adhere to all SML-IF document requirements as described in the normative sections of this specification. In addition, each non-null SML reference in the document MUST be an instance of the SML URI Reference Scheme [SML 1.1]. 

A conforming SML-IF Producer MUST be able to generate a referentially conforming SML-IF Document from an SML model.

A conforming SML-IF Consumer MUST process a conforming SML-IF Document using, in whole or part, semantics defined by this specification. It is OPTIONAL that a conforming SML-IF Consumer process all elements defined in this specification, but any element that is processed MUST be processed according to the requirements stated in the normative sections of this specification.

==========