This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
Several tests (take orderBy20 as a concrete example) will produce errors or results not catalogued as possible <output-file> or <expected-error> if the processor does not support schema import. Firstly the system could raise XQST0009 if it can not handle the import. The test harness is allowed to change the schema import to a namespace declaration in which case this error is avoided, and for some tests everything proceeds as intended, but here the "negative numbers" will be read as xdt:untypedAtomic so sorted as strings, which results in the opposite order to the numeric ordering intended, so currently any system not doing schema import will fail this test. The catalog could catalog these alternative conforming outcomes, but in these cases the test isn't really testing the intended feature (the ascending order modifier) and so it might be better to explictly force the data to be numeric with number() or xs:integer or something rather than using a schema import. Several tests of casting and ordering have the same feature. David
This report is perhaps badly worded. The tests do not depend directly on schema import but they do depend on the ability to generate a data model input from a PSVI. However the end result is the same, it isn't possible for a system that doesn't support schema validation to pass these tests, even though the tests are testing features unrelated to schema support. David
The affected tests are (as far as I can see): Constr-compattr-compname-9 Constr-compelem-compname-8 Constr-compelem-compname-9 casthc42 casthcds1 casthcds2 casthcds3 casthcds4 casthcds5 casthcds6 casthcds7 casthcds8 casthcds9 casthcds10 casthcds11 casthcds12 casthcds13 casthcds14 casthcds15 casthcds16 casthcds17 casthcds18 casthcds19 casthcds20 casthcds21 casthcds22 casthcds23 casthcds24 casthcds25 casthcds26 casthcds27 casthcds28 casthcds29 casthcds30 casthcds31 casthcds32 casthcds33 casthcds34 casthcds35 casthcds36 casthcds37 casthcds38 casthcds39 casthcds40 casthcds41 casthcds42 fn-except-node-args-014 fn-except-node-args-015 fn-except-node-args-016 fn-except-node-args-017 fn-except-node-args-018 fn-except-node-args-019 fn-except-node-args-020 fn-except-node-args-021 fn-intersect-node-args-014 fn-intersect-node-args-015 fn-intersect-node-args-016 fn-intersect-node-args-017 fn-intersect-node-args-018 fn-intersect-node-args-019 fn-intersect-node-args-020 fn-intersect-node-args-021 fn-intersect-node-args-022 fn-lang1args-1 fn-lang1args-2 fn-lang1args-3 fn-union-node-args-014 fn-union-node-args-015 fn-union-node-args-016 fn-union-node-args-017 fn-union-node-args-018 fn-union-node-args-019 fn-union-node-args-020 fn-union-node-args-021 fn-union-node-args-022 orderBy1 orderBy2 orderBy3 orderBy4 orderBy5 orderBy6 orderBy7 orderBy8 orderBy9 orderBy10 orderBy11 orderBy12 orderBy13 orderBy14 orderBy15 orderBy16 orderBy17 orderBy18 orderBy19 orderBy20 orderBy21 orderBy22 orderBy23 orderBy24 orderBy25 orderBy26 orderBy27 orderBy28 orderBy29 orderBy30 orderBy31 orderBy32 orderBy33 orderBy34 orderBy35 orderBy36 orderBy37 orderBy38 orderBy39 orderBy40 orderBy41 orderBy42 orderBy43 orderBy44 orderBy45 orderBy46 orderBy47 orderBy49 orderBy50 orderBy51 orderBy52 orderBy53 orderBy54 orderBy55 orderBy56 orderBy57 orderBy59 predicates-1 predicates-2 predicates-3 predicates-4 predicates-5 predicates-6 predicates-7 predicates-8 predicates-9 predicates-10 predicates-11 predicates-12 predicates-13 predicates-14 predicates-17 predicates-18 predicates-19 predicates-20 predicates-21 predicates-22 predicates-23 predicates-24 predicates-25 predicates-26 predicates-27 predicates-28 It might be nice to tag tests which require optional features (Schema Import, Module Import, Full Axis...), so implementors can easily filter them.
(In reply to comment #2) > The affected tests are (as far as I can see): > > fn-intersect-node-args-015 > Not exactly, for example fn-intersect-node-args-015 _does_ work correctly if you replace the schema import by a namespace declaration, as is already allowed by the test harness. (Which means incidentally that it would be better if the test was already in that form, as then it would not have this unnecessary apparent dependency on schema import. In the cases that really do need the schema typing (many of which do use the atomic.xml input file) I think it would be better to simply build a list of typed values using numeric or string literals within the query, and not use any PSVI input at all outside the part of the test suite that tests schema support. > > It might be nice to tag tests which require optional features (Schema Import, > Module Import, Full Axis...), so implementors can easily filter them. Yes I agree with this, that any remaining cases should be flagged, although in most of these cases the test can be written not to depend on the optional feature. which would be better, as that would mean that systems not implementing the feature can still attempt to pass the test. David
David: Andrew Eisenberg will have an answer to this bug sometime soon. The issue was disccussed and a resolution taken. Carmelo
An implementation that does not support schema import should remove these lines before query execution or change them to namespace declarations. Our guidelines for running the test suite do say, "Test cases can refer to validated documents (which requires namespace declaration) or imported schemas. Such test cases contain schema import declarations located within the same comments as the context item described above." As you say, we could have built a list of typed values using numeric or string literals in order to avoid this dependency. We did not think that relying on validated documens would exclude any XQuery implementations from running the test suite. Were we wrong?
> We did not think that relying on > validated documens would exclude any XQuery implementations from running the > test suite. Were we wrong? Yes, very wrong! My own implementation (xq2xsl) has no means of importing a data model derived from a PSVI if it is using a basic XSLT processor (which the XSLT spec forbids from handling type annotated nodes) saxon8-B XQuery implementation can not import a schema validated data model either, you need the saxon8-SA for that, and I would guess seeing as the X-hive people have commented on this report that they also have problems with these files (but I haven't confirmed this). Even if the currently available procesors could import a PSVI then surely it is wrong to make the test of a core feature like ordering depend on an optional feature. Especially for order by. As you will know, order by is one of the few XQuery features without a direct XSLT analogue, and I sweated blood getting what I believe to be a conformant implementation in xq2xsl. I don't see why I have to fail half the order by tests just because I don't support an explictly optional feature that is unconnected with ordering. Actually more importantly than being able to claim brownie points for running the test, I would just like to be _able_ to run the test. The test suite has been incredibly useful, and shown up several bugs in my code (which I've hopefully fixed) but handling FLWOR is the most complicated part of my code and it's not tested (by this suite) to anything like the amount it should be, because the catalogue claims (erroneously) that the answer I get is in error. I have modified the input document orderBy20 as authorised, replacing the schema import by a namespace declaration,and I ran the test, I believe that the answer I produce is conformant. It may not be the answer the author of the query intended, but surely "ExpectedResults" should mean "results authorised by the spec", not "results the author of the query expected". If they would be accepted I would offer to do the work to change the tests not to use PSVI import myself, but I'll not do it if that wouldn't be used:-) David
>If they would be accepted I would offer to do the work to change the tests not > to use PSVI import myself, but I'll not do it if that wouldn't be used:-) To be explict, both saxon8-B and xq2xsl "fail" orderby20 as currently specified. If modified as below they both obtain the result expected by the catalogue. I would propose that all tests using orderData.xml are modified in the same way so as not to use schema import, and to use the local:data() function to extract the typed value for sorting. A similar local:data() function could be written for the tests that use atomic.xml. David declare default element namespace "http://www.w3.org/XQueryTestOrderBy"; (: name : orderBy20 :) (: description : Evaluation of "order by" clause with the "order by" clause of a FLWR expression set to "$x ", where $x is a set of negative numbers and the ordering mode set to ascending :) (: insert-start :) declare variable $input-context1 external ; (: :=doc('orderData.xml') :) (: insert-end :) declare function local:data ($e as element()) as item() { if (local-name($e/..)= 'Strings') then string($e) else xs:decimal(data($e)) }; <results> { for $x in $input-context1/DataValues/NegativeNumbers/orderData order by local:data($x) ascending return $x } </results>
David: casthc42 seems ok with no schema dependencies. I am missing something? predicates 17-28 import a schema that is not used. I will remove the schema import statement from those tests. For rest of the tests, the task force came ot a resolution yesterday. Andrew will be posting that at a later time. Carmelo
(In reply to comment #8) > > casthc42 seems ok with no schema dependencies. I am missing something? It looks OK to me too. I didn't actually list all the files I had problems with, Martin Probst added a list as an additional Comment, but it's easy to get cut and paste errors in such a list. grep schema import is a more reliable way of getting the exact list of those tests using schema import:-) David
Thanks. Already changed and submitted changes for predicates 17-28. Results should remain the same. Thanks, Carmelo
(In reply to comment #10) > Thanks. Already changed and submitted changes for predicates 17-28. Results > should remain the same. > > Thanks, > Carmelo Is it possible (in general not just here) to say exactly what the change is, so that I (and other testers) can make the change locally and update our reported results accordingly? Or perhaps more simply could the files be made available on a continually updated basis from an "unpacked" area in an addition to the snapshot zip archives that make up each numbered release? (I know, I'm the user from @@##:-) I have W3C CVS access, so I suppose if I knew where they were, I could get them myself, but probably not everyone running the tests has such access. David
David: For predicates 17-23, removed the shmea import statement. The task force will make this tests vaialble with weekly updates. This should be on line soon. I will mail you a zip file with the new tests (17-28) in it. Carmelo
We have considered your feedback and decided not to rely upon validated data. We have modified our guidelines for test case creation by adding the following items: 1) For tests cases that are sensitive to typed values, submittors will provide a result for both the typed and untyped data. 2) For test cases where typed data is imperative, submittors will need to use constructor functions instead of relying on typed data. We will review the test cases that you and Martin have identified and modify them where appropriate. On your comment about seeing these changes, we are planning to release new versions of the test suite monthly. We are also investigating making our CVS repository visible to the public. Please close the bug report if you agree with our resolution.
Thanks. As I said in an earlier comment I think that in most cases using constructor functions (or literals of an appropriate type) will produce a better result than cataloging results with and without typed input, as doing the latter for the orderby tests leaves the order modifier essentially untested, as the two sets of expected results typically have the resulting sequence in oposite orders, and the test catalogue (presumably) won't distinguish which type of processor should make which result. So a processor that doesn't implement "ascending/descending" will pass anyway as they will pick up the expected result for the other input form (typed or not typed). However in practice there are enough tests that weakening a few isn't catastophic, and certainly process wise this is a perfectly acceptable outcome, and basically I'm happy for you to fix each test on a case by case basis either by changing the test or by adding extra result files, as you see fit, so I'm closing this. Sorry if this has generated extra work on the testing group... David