This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 23923 - [QT3TS] XQueryX conversion errors
Summary: [QT3TS] XQueryX conversion errors
Status: RESOLVED FIXED
Alias: None
Product: XPath / XQuery / XSLT
Classification: Unclassified
Component: XQuery 3 & XPath 3 Test Suite (show other bugs)
Version: 2nd Edition Recommendation
Hardware: PC Windows NT
: P2 normal
Target Milestone: ---
Assignee: O'Neil Delpratt
QA Contact: Mailing list for public feedback on specs from XSL and XML Query WGs
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-11-26 11:22 UTC by Tim Mills
Modified: 2014-03-18 15:31 UTC (History)
1 user (show)

See Also:


Attachments

Description Tim Mills 2013-11-26 11:22:23 UTC
A number of tests are affected by the following conversion errors.


1. Incorrect conversion of \uXXXX.  e.g. re00741 where the string literal

'[^a-f-[\x00-\x60\u007B-\uFFFF]]+'

becomes

<xqx:stringConstantExpr>
<xqx:value>[^a-f-[\x00-\x60{-&#65535;]]+</xqx:value>
</xqx:stringConstantExpr>


2.  More XML 1.1 problems.  e.g. line-ending-Q006, where
               <xqx:stringConstantExpr>
                 <xqx:value> &#8232; </xqx:value>
               </xqx:stringConstantExpr>

isn't the same as ' &#x2028; '.

3. Incorrect conversion of element(name, type?) (e.g. fn-nilled-40) 
missing <xqx:nillable />

4. Incorrect conversion of eqname-009.

Q{ http://www.example.com/ ns }b

is converted to

    <xqx:nameTest xqx:URI=" http://www.example.com/ ns ">b</xqx:nameTest>

but should probably be

    <xqx:nameTest xqx:URI="http://www.example.com/ ns">b</xqx:nameTest>
Comment 1 Michael Dyck 2013-12-03 01:21:04 UTC
(In reply to Tim Mills from comment #0)
> 
> 1. Incorrect conversion of \uXXXX.  e.g. re00741 where the string literal
> 
> '[^a-f-[\x00-\x60\u007B-\uFFFF]]+'
> 
> becomes
> 
> <xqx:stringConstantExpr>
> <xqx:value>[^a-f-[\x00-\x60{-&#65535;]]+</xqx:value>
> </xqx:stringConstantExpr>

I think this is the same problem as in Bug 13796.
Comment 2 Michael Dyck 2013-12-03 02:29:04 UTC
(In reply to Tim Mills from comment #0)

> 2.  More XML 1.1 problems.  e.g. line-ending-Q006, where
>                <xqx:stringConstantExpr>
>                  <xqx:value> &#8232; </xqx:value>
>                </xqx:stringConstantExpr>
> 
> isn't the same as ' &#x2028; '.

Looks the same to me. What's the problem?
Comment 3 Michael Dyck 2013-12-03 02:54:45 UTC
> 
> 3. Incorrect conversion of element(name, type?) (e.g. fn-nilled-40) 
> missing <xqx:nillable />

That definitely looks like a bug.
Comment 4 Michael Dyck 2013-12-03 03:08:43 UTC
> 4. Incorrect conversion of eqname-009.
> 
> Q{ http://www.example.com/ ns }b
> 
> is converted to
> 
>     <xqx:nameTest xqx:URI=" http://www.example.com/ ns ">b</xqx:nameTest>

I'm pretty sure that's a valid conversion, because it back-converts to exactly the original URIQualifiedName.


> but should probably be
> 
>     <xqx:nameTest xqx:URI="http://www.example.com/ ns">b</xqx:nameTest>

I believe that too would be a valid conversion, because it back-converts to a URIQualifiedName that, although lexically different from the original, is semantically the same.

Does the Working Group have a preference?
Comment 5 Tim Mills 2013-12-03 05:23:58 UTC
(In reply to Michael Dyck from comment #2)
> (In reply to Tim Mills from comment #0)
> 
> > 2.  More XML 1.1 problems.  e.g. line-ending-Q006, where
> >                <xqx:stringConstantExpr>
> >                  <xqx:value> &#8232; </xqx:value>
> >                </xqx:stringConstantExpr>
> > 
> > isn't the same as ' &#x2028; '.
> 
> Looks the same to me. What's the problem?

&#x2028; is LINE SEPARATOR.

This test is for a processor which follows the XML 1.1 line ending rules.  Such a processor will perform line ending normalization on &#x2028;.

When converted to XQueryX/XML 1.0 the &x2028; is not treated by the XML parser as a line ending character.  It would be if it were XQuery/XML 1.1.
Comment 6 Tim Mills 2013-12-03 05:30:22 UTC
(In reply to Michael Dyck from comment #4)
> > 4. Incorrect conversion of eqname-009.
> > 
> > Q{ http://www.example.com/ ns }b
> > 
> > is converted to
> > 
> >     <xqx:nameTest xqx:URI=" http://www.example.com/ ns ">b</xqx:nameTest>
> 
> I'm pretty sure that's a valid conversion, because it back-converts to
> exactly the original URIQualifiedName.
> 
> 
> > but should probably be
> > 
> >     <xqx:nameTest xqx:URI="http://www.example.com/ ns">b</xqx:nameTest>
> 
> I believe that too would be a valid conversion, because it back-converts to
> a URIQualifiedName that, although lexically different from the original, is
> semantically the same.
> 
> Does the Working Group have a preference?

True.  As an aside, I note that xqx:URI is an xs:string, and wonder why it's not an anyURI.   

 I'll change our XQueryX processor to perform the normalization.
Comment 7 Michael Dyck 2013-12-03 09:07:40 UTC
(In reply to Tim Mills from comment #5)
> (In reply to Michael Dyck from comment #2)
> > (In reply to Tim Mills from comment #0)
> > 
> > > 2.  More XML 1.1 problems.  e.g. line-ending-Q006, where
> > >                <xqx:stringConstantExpr>
> > >                  <xqx:value> &#8232; </xqx:value>
> > >                </xqx:stringConstantExpr>
> > > 
> > > isn't the same as ' &#x2028; '.
> > 
> > Looks the same to me. What's the problem?
> 
> &#x2028; is LINE SEPARATOR.
> 
> This test is for a processor which follows the XML 1.1 line ending rules. 
> Such a processor will perform line ending normalization on &#x2028;.
>
> When converted to XQueryX/XML 1.0 the &x2028; is not treated by the XML
> parser as a line ending character.  It would be if it were XQuery/XML 1.1.

Ah, right, the conversion-to-XQueryX code only ever generates files with
    <?xml version="1.0"?>
I suppose it could generate version="1.1" for test-cases that have
    <dependency type="xml-version" value="1.1"/>
but is that enough? Could there be an XQueryX processor that only accepts 1.1 files, and so would (also) want a "1.1" version of the vast majority of test-cases that don't have an xml-version dependency?

Or is an XQueryX processor obliged to handle input in *all* versions of XML? The XQueryX spec doesn't seem to say explicitly, but it says:
    An XQueryX processor that claims to conform to this
    specification MUST implement the XQueryX syntax as defined in
    4 'An XML Schema for the XQuery XML Syntax' of this specification ...
and an XML document conforms to that syntax regardless of its version decl, right? So if "implementing the syntax" means "recognizing all instances of the syntax", that suggests that a conforming XQueryX processor must accept input in all versions of XML.
Comment 8 Michael Dyck 2013-12-03 18:57:00 UTC
(In reply to Tim Mills from comment #5)
> 
> When converted to XQueryX/XML 1.0 the &x2028; is not treated by the XML
> parser as a line ending character.  It would be if it were XQuery/XML 1.1.

On second thought, I'm not so sure. Semantically, an XQueryX processor evaluates a query by:
(a) transforming it to the human-readable syntax; and
(b) evaluating the human-readable syntax.
And it's not clear that the version of XML used in (a) must match the choice of choice of rules for normalizing line breaks in (b).

Specifically, although the query begins with
    <?xml version="1.0"?>
and so must be processed by an XML 1.0 processor, which will leave the Line Separator as is, that happens at (a), and it's still possible for the XQuery processor at (b) to use the end-of-line rules of XML 1.1 (as indicated in the test-case), which *will* normalize the Line Separator into a Line Feed.

(Or it can skip the test-case if it's unable to use the 1.1 rules at (b).)
Comment 9 Michael Dyck 2014-01-07 19:59:11 UTC
(In reply to Michael Dyck from comment #3)
> > 
> > 3. Incorrect conversion of element(name, type?) (e.g. fn-nilled-40) 
> > missing <xqx:nillable />
> 
> That definitely looks like a bug.

I believe I've now fixed that bug in the converter.
This adds <xqx:nillable/> to its output for the following tests:
    fn-nilled-40
    fn-nilled-42
    fn-nilled-43
    fn-nilled-49
    fn-nilled-53
    hof-039
    hof-053
    K2-ExternalVariablesWith-22
    K2-ExternalVariablesWith-22a
    K2-ExternalVariablesWith-23
Comment 10 O'Neil Delpratt 2014-03-18 15:31:43 UTC
I believe this bug is now fixed.