This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 3444 - doc tests (in public cvs)
Summary: doc tests (in public cvs)
Status: CLOSED FIXED
Alias: None
Product: XML Query Test Suite
Classification: Unclassified
Component: XML Query Test Suite (show other bugs)
Version: unspecified
Hardware: PC Windows XP
: P2 normal
Target Milestone: ---
Assignee: Andrew Eisenberg
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-07-11 00:37 UTC by David Carlisle
Modified: 2006-08-09 09:56 UTC (History)
1 user (show)

See Also:


Attachments

Description David Carlisle 2006-07-11 00:37:29 UTC
the guidelines for doc tests have changed a bit recently but they still don't
appear to match the tests.

They say

(: Name: FnDoc001 :)
(: Description: Test fn:doc() function :)

(: insert-start :)
declare variable $input-context external;
(: insert-end :)

$input-context/child


ie suggest a format in which doc() doesn't appear in the test explictly at all
The actual tests use the format (eg fn-doc-5)

(: Name: fn-doc-5 :)
(: Description: Evaluation of fn:doc as per example 1 from the functions and Ops. for this function.:)

(: insert-start :)
declare variable $input-context external;
(: insert-end :)

fn:doc($input-context) is fn:doc($input-context)`

which is fine so long as the catalogue specifies that $input-context should be set to the URI of the test input file.
but the catalogue has
                     <input-file role="principal-data" variable="input-context">works-mod</input-file>
which means that there is no indication that $input-file should not be set to the document node reulting from parsing the file, which is what my test harness currently does which means that I fail fn-doc-5 ... fn-doc-22

should there be an attribute on input-context (or preferably a different element instead of input-context) that can be used to specify that the variable should be set to a URI rather than a document node?

David
Comment 1 David Carlisle 2006-07-11 09:18:04 UTC
I wrote

  should there be an attribute on input-context (or preferably a different
  element instead of input-context) 

Sorry, I meant input-file (the XQTS Catalogue element) not input-context
Comment 2 Andrew Eisenberg 2006-07-12 17:22:37 UTC
This does need to be fixed. The context attribute may be used with input-file elements. Values "implicit", "external", "doc", "collection", and "default collection" may be specified. A test like fn-doc-1 that wants to test a bad URI is written as:

fn:doc("http:\\invalid&gt;URI\someURI")

The context attribute wil not effect this test.

fn-doc-5 should be written as follows:

(: Name: fn-doc-5 :)
(: Description: Evaluation of fn:doc as per example 1 from the functions and Ops. for this function.:)

(: insert-start :)
declare variable $input-context external;
(: insert-end :)

$input-context is $input-context

Putting context="doc" into the catalog entry will force this to be transformed into the following:

fn:doc(some URI) is fn:doc(some URI)

Comment 3 Andrew Eisenberg 2006-07-12 21:17:21 UTC
I've added context="doc" to the catalog and modified the query as necessary for the following test cases:

fn-doc-5
fn-doc-6
fn-doc-7
fn-doc-15
fn-doc-16
fn-doc-18
fn-doc-19
fn-doc-20
fn-doc-21
fn-doc-22



I've removed tes test cases fn-doc-8 (to 14) altogether. These test cases were trying to construct URIs for fn:doc dynamically.

Please close this bug report if you are satisfied with this resolution.
Comment 4 David Carlisle 2006-07-13 09:18:13 UTC
why??? have such a convoluted mechanism where the test file does not have a call to the function that it is testing?

Why not have the much simpler system that the test file stays as it is

with doc($variable) is doc($variable)

and the catalogue just specify that $variable is to be set using the URI of the file rather than the file itself. (The URI is already specified in the catalogue, all that's needed is a hook to tell the test harness to use the uri not the document located at that uri)
Comment 5 Andrew Eisenberg 2006-07-13 17:48:47 UTC
I suppose that convoluted is in the eye of the beholder. We decided long ago to write our queries using external variables, and then to allow the writers of test harnesses to transform the query in a number of ways. $input-context//hours can be executed as is, or transformed into one of the following:

.//hours
fn:doc('some URI')//hours
fn:collection('some URI')//hours
fn:collection()//hours

We defined the context attribute in the test metadata to allow the test writer to force one of these transformations.
Comment 6 David Carlisle 2006-07-14 02:07:06 UTC
(In reply to comment #5)

> .//hours
> fn:doc('some URI')//hours
> fn:collection('some URI')//hours
> fn:collection()//hours
> 
> We defined the context attribute in the test metadata to allow the test writer
> to force one of these transformations.
> 

Yes sure, that's how all of the test suite works, but that is rather different
from the case for the doc tests. You have to allow that flexibility in general for setting up the initial context as xquery doesn't specify that a system has to be able to set the implicit initial context, so it might need doc() or collection() or an external variable or whatever. But in all these cases a system isn't _forced_ to edit the
query unless they can't handle the default input-context external variable mechanism. 

Having a system where the distributed query just has the wrong text
together with instructions that _every_ implementer must edit the query to say the right thing seems odd to me. That said, I know you want to get this thing done, so I'm going to close this report, after adding this moan...

David
Comment 7 David Carlisle 2006-07-26 10:09:20 UTC
When I allowed this report to close, I thought that it was sufficient for the doc tests to change
declare variable $input-context external;
to 
declare variable $input-context := doc('some path to test file');


however 
http://www.w3.org/Bugs/Public/show_bug.cgi?id=3532#c5
suggests that that is not sufficient and instead the test harness has to delete
the line
declare variable $input-context external;
and textually replace every occurrence of $input-context by doc('...')

In particular, currently when testing fn-doc-19 I evaluate

declare variable $input-context :=doc('works-mod.xml');
($input-context is $input-context) and fn:true()

so doc() is only called once and this isn't really testing the stability of the function.

If the intention of the guidelines are that I should edit the entire test file, not just the part flagged with special comments, and delete the variable declaration and change the text to

(doc('works-mod.xml') is doc('works-mod.xml')) and fn:true()

Then I don't think that is at all reasonable.

If instead the test file was distributed as


(: insert-start :)
declare variable $input-uri external;
(: insert-end :)
(doc($input-uri) is doc($input-uri)) and fn:true()

with a catalog entry that specified $input-uri should be set to the uri of the works-mod file, then implementers would not have to re-implement their entire test harness mechanisms just to cope with these few tests which really ought to be very simple, but which have had a stream of bugs just relating to the way the document uri is specified in the test file and the catalog.

This has been requested several times before, by (at least)
me
http://www.w3.org/Bugs/Public/show_bug.cgi?id=2900
Martin Probst
http://www.w3.org/Bugs/Public/show_bug.cgi?id=2900#c3
Michael Kay
http://www.w3.org/Bugs/Public/show_bug.cgi?id=3532#c9
and probably others, they were just ones that I remembered.

David
Comment 8 David Carlisle 2006-07-26 12:41:20 UTC
I note that bug #2900 (which is marked as fixed but not closed) ends with
http://www.w3.org/Bugs/Public/show_bug.cgi?id=2900#c18
which indicates that the doc tests were modified as suggested here (and they were)
however they have since been changed to the current format, so 2900 is unfixed presently (although I won't re-open it again as it would be a duplicate of this)

Comment 9 Andrew Eisenberg 2006-07-26 19:21:34 UTC
(In reply to comment #7)

> If the intention of the guidelines are that I should edit the entire test file,
> not just the part flagged with special comments, and delete the variable
> declaration and change the text to
> 
> (doc('works-mod.xml') is doc('works-mod.xml')) and fn:true()
> 
> Then I don't think that is at all reasonable.


This is the transformation that we require for this test case.


I hear what you are saying, but must decline to change the test cases and documentation in the way that you suggest.

I believe this approach to testing fn:doc and fn:collection, while ineligant in your eyes, is workable.

We will improve our documentation in this area and I will call people's attention to it when I announce XQTS 1.0.

Please mark this closed if you agree with this resolution. If you would like me to bring this to the WGs for consideration, then please reopen it.
Comment 10 David Carlisle 2006-07-27 08:50:15 UTC
Sorry the doc tests were fixed last month as commented in

http://www.w3.org/Bugs/Public/show_bug.cgi?id=2900#c18

But they have since been unfixed. The current mechanism has proved impossible to document, is very hard to implement as the entire test query needs to be edited,
and is completely different from every other test in the test suite.

I don't accept that it's too late to change this as you are close to 1.0 as 
a) it was (for the doc tests) correct for a while last month. It's a relatively recent change that broke them again, and 
b) testers have been asking for this to be fixed for months.

David
Comment 11 Andrew Eisenberg 2006-08-03 03:56:35 UTC
The XSL and XQuery WGs discussed this on Tuesday and asked me to change our tests for fn:doc and fn:collection in the way that you have been suggesting.

I have made these changes (reflected in XQTS_current.zip v1.28). Rather than explain the changes here, I suggest that you look at the new "Test Cases with fn:doc and fn:collection" section in our Guideline for Running the XML Query Test Suite.

Please mark this bug report as closed if you agree with our resolution.
Comment 12 David Carlisle 2006-08-09 09:56:23 UTC
thanks