This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 7031 - XSD 1.1 doesn't support conversion of xs:sequence to xs:all because xs:all can't contain groups references
Summary: XSD 1.1 doesn't support conversion of xs:sequence to xs:all because xs:all ca...
Status: CLOSED FIXED
Alias: None
Product: XML Schema
Classification: Unclassified
Component: Structures: XSD Part 1 (show other bugs)
Version: 1.1 only
Hardware: PC Linux
: P2 normal
Target Milestone: ---
Assignee: David Ezell
QA Contact: XML Schema comments list
URL:
Whiteboard:
Keywords: resolved
: 12238 (view as bug list)
Depends on:
Blocks:
 
Reported: 2009-06-17 11:51 UTC by Anthony B. Coates
Modified: 2011-03-05 00:22 UTC (History)
4 users (show)

See Also:


Attachments
Colored diff showing wording adopted to resolve issue 7031 (157.08 KB, text/html)
2009-08-22 00:05 UTC, C. M. Sperberg-McQueen
Details

Description Anthony B. Coates 2009-06-17 11:51:42 UTC
Apologies that this is late in the day to be logging issues for XML Schema 1.1, but this has a significant impact.  Many of us expect that, when 1.1. is available, perhaps the majority of Schemas will be converted to use xs:all in place of xs:sequence, since the imposition of sequencing into "data-oriented" XML is usually a purely technical device that feels "unnatural" to users.  The sooner we can move to unordered content for such Schemas, the better.

Some work has been done on this for FpML, the Financial Product Markup Language, which is widely used for representing information about financial derivatives.  The problem that has come out of the FpML conversion is that you can't simply change "xsd:sequence" to "xsd:all" if your sequences contain references to model groups, because "xsd:all" can't contain such references.

This is a significant issue for anyone hoping that XML Schema 1.1 will allow them to move from ordered content to unordered content.  I'm concerned it may block such moves, and remove one of the major reasons for migrating from 1.0 to 1.1.

It seems to me that the problem would be removed if "xsd:all" could be allowed to contain references to "xsd:group", perhaps with the caveat that those groups would have to contain "xsd:all" rather than a sequence or choice.

Is this plausible?  It will be a real blow if this stops us from migrating Schemas as I had expected we would be able to.
Comment 1 David Ezell 2009-06-26 16:27:13 UTC
The WG has sympathy for this issue.  There are procedural hurdles, such as republication of the spec, notification of interested parties, etc.
The WG is leaving this bug open for further consideration. 
Comment 2 David Ezell 2009-07-13 14:49:45 UTC
See WG minutes of 2009-07-10 for a proposed resolution:
http://lists.w3.org/Archives/Member/w3c-xml-schema-ig/2009Jul/0002.html
Comment 3 C. M. Sperberg-McQueen 2009-07-13 18:37:28 UTC
It has been pointed out that the minutes of the call referred to in comment 2 are not available to the public; this is to summarize the technical discussion of this issue in those minutes.  

The meaning of a nested all-group depends on whether all-groups are given the 'SGML interpretation' (all the children, in any order) or the 'interleave interpretation' (the interleave of the children).  For the SGML interpretation, (a & (b & c)) != (a & b & c) because (b, a, c) and (c, a, b) are legal in the latter but not the former.  For the interleave interpretation, the two expressions are equal.  

In XSD 1.0, all-groups are compatible with both interpretations, because the differences between them are visible only with maxOccurs > 1 or with nested subgroups.  

Since the WG has already adopted the interleave interpretation, when relaxing the constraint on maxOccurs, there is no problem of interpretation, and allowing references to an all-group from within an all-group does not introduce any new expressive power in content models, or present any particular implementation difficulties, for the case when the group reference has minOccurs = maxOccurs = 1.

Accordingly, the editors have been asked to prepare a wording proposal specifying that:

1 references to named all-groups may be made from within an all-group.

2 minOccurs and maxOccurs must both be = 1 on that group reference.

3 The language identified by the all-group is as described above.

I note in passing that if we define the meaning of nested all-groups as described here, it becomes possible in some future revision of the spec to simplify the treatment of all-group extension:  instead of the existing ad hoc text and hand-waving about adding items to the all-group, the extension of all-groups can follow the same pattern as the extension of sequences:  extending all-group G1 with all-group G2 amounts to defining the extension as (G1 & G2).    This would also suggest a pattern which could be followed for or-groups (allow the extension of an or-group G3 by some group G4 to mean (G3 | G4)); there is some evidence in the comments on XSD 1.0 and 1.1 that some users would like the ability to add choices to an or-group.  Both of these possible follow-on steps are described here only for the record; I do not propose that we make either of these changes to XSD 1.1 at this time (even the purely editorial one).
Comment 4 Noah Mendelsohn 2009-07-13 19:06:34 UTC
To avoid confusion, let me say first that I have no problem at all with the direction adopted by the working group with respect to bug 7031.  This comment is provided just in case it is of interest.

From the original bug report:

> Many of us expect that, when 1.1. is available,
> perhaps the majority of Schemas will be converted
> to use xs:all in place of xs:sequence, since the
> imposition of sequencing into "data-oriented" XML
> is usually a purely technical device that feels
> "unnatural" to users.


Yes, but do note that the order of elements in an XML infoset is significant, at least in principle, regardless of what the schema says.  So, if you write a content model:

<all>
  <element ref="a"/>
  <element ref="b"/>
</all>

That says that both:

   <a/> <b/>

and

   <b/> <a/>

are valid.  It does NOT say that both of those two documents should be treated as meaning the same thing.  The elements in an XML infoset are considered ordered, whether or not validation is performed.

This is among the reasons that certain data-oriented applications of XML force a canonical order in cases where, to the application, the order of the "fields" is truly not significant.

The fact that XSD has limitations in support of <all> is indeed another reason, and I infer that those working on FpML have made the (reasonable) decision that it would be convenient to allow all possible XML orderings in cases where multiple fields are to be specified.  The proposed change to XSD will indeed likely be useful when such a choice is made, but at the XML level, the order of the elements will remain significant.

Some years ago, the XSD working group got a request, from the Query working group I think, to invent an xsd:annotation that could be used to signal ("I know XML considers the order of these elements to be significant, but trust me, it isn't.")  The intention was to signal to database mapping software that, yes, the obvious mapping to a relational database, in which column order is not maintained, is appropriate.  The Schema working group did a bit of work on specifying such an attribute (see: http://www.w3.org/XML/Group/2000/11/lc200), but by the time progress was made the requirement seemed to have gone away, and the work was dropped.  To research the history of this, I suggest searching the schema interest group mailing list with the string "LC-200", which is a reference to the issue name under which this was discussed.  

Again, this comment is intended to fill in some background, not to question the proposed direction of the working group with respect to bug 7031.

Noah








Comment 5 Sandy Gao 2009-07-30 21:37:00 UTC
The following (member-only) link contains a proposal to address this issue, based on the WG decision, as described in comment #3.

http://www.w3.org/XML/Group/2004/06/xmlschema-1/structures.b7031.html
Comment 6 Michael Kay 2009-07-30 21:51:16 UTC
In 3.8.6.2

"It appears only as the value of one or both of the following properties:"

needs to be

"It appears only as the value of one or more of the following properties:"
Comment 7 C. M. Sperberg-McQueen 2009-08-07 16:39:29 UTC
At its call of 7 August, the WG amended the proposal mentioned in comment 5 as follows:

  - in clause 1 of All Group Limited, s/one or both/one or more/
  - in clause 1.3 of All Group Limited, for " and P is is" read ", where P is"
  - add a clause 2 to All Group Limited, reading "For every particle P 
    in its {particles}, if P.{term} is a model group, then P.{term}.{compositor} 
    = all.

The WG then adopted the proposal as amended.  Accordingly I'm marking the bug decided.  

Tony, as the original poster, you'll need to review the proposal.  When time allows, I'll generate a document showing the changes made to resolve this issue and ask you to confirm that you're happy with it, but I'm marking the bug decided, to record the decision made today, in case I don't manage to make the clean proposal until week after next. 
Comment 8 C. M. Sperberg-McQueen 2009-08-22 00:05:35 UTC
Created attachment 744 [details]
Colored diff showing wording adopted to resolve issue 7031

I attach a colored diff showing the wording changes adopted on 7 August
(including the amendments listed in an earlier comment).
Comment 9 C. M. Sperberg-McQueen 2009-08-22 00:09:07 UTC
As promised in comment 7, a document showing the changes adopted on 
7 August has been attached to this Bugzilla record.

Tony, as the originator of the issue, I can now ask you to review the
changes made and determine whether you believe they satisfactorily
resolve the issue you raised.  If you agree that they do and are
satisfied with this resolution of the issue, please indicate this by 
changing the status of this issue from RESOLVED to CLOSED.  If you
are for any reason unhappy with the resolution, please show this
by changing the status to REOPENED and explaining what's the matter.
If we don't hear from you by mid-September, we will assume that 
silence implies consent and that you are happy with the result.
Comment 10 Anthony B. Coates 2009-08-22 13:36:21 UTC
Thanks.  I noticed a slight typo in the diff in the 'Status of this document':

----
Group references are now allowed in <xs:all> model groups; they must Such references must have minOccur=maxOccurs=1 and refer to other <xs:all> groups.
----

I guess it should say "they must ..." or "Such references must ..." but not both.

Otherwise, it all looks great!  Thank you all so very much for your help, and for being so responsive at such a late stage in XML Schema 1.1's development.  I look very forward to XML Schema 1.1 becoming a W3C Recommendation before too long, and seeing it implemented in the XML Schema tools that people use every day, giving them new and improved options for how they can design their Schemas.
Comment 11 C. M. Sperberg-McQueen 2009-08-22 14:07:52 UTC
Thank you; the cut and paste error noted in comment 10 has now been corrected
in the source; new status-quo documents are being generated.
Comment 12 Michael Kay 2010-05-14 22:42:12 UTC
In the published specification dated 3 December 2009, section 3.8.2, the agreed change to the content model of xs:all does not appear to have been implemented. It still reads

Content: (annotation?, (element | any)*)
Comment 13 David Ezell 2010-06-04 15:49:24 UTC
Has been addressed, but is not yet in a published document.
Comment 14 C. M. Sperberg-McQueen 2010-08-16 16:43:23 UTC
The status-quo document at 

  http://www.w3.org/XML/Group/2004/06/xmlschema-1/structures.html#declare-contentModel

now has the correct content model; I'm re-closing this issue.
Comment 15 David Ezell 2010-11-10 17:48:55 UTC
The WG reported this bug as FIXED on 2010-08-16.  We are closing this bug
as requiring no futher work.  If there are issues remaining, you can reopen
this bug and enter a comment to indicate the problem.  Thanks very much for the
feedback.
Comment 16 C. M. Sperberg-McQueen 2011-03-05 00:22:19 UTC
*** Bug 12238 has been marked as a duplicate of this bug. ***