3540 2006-07-28 10:46:25 +0000 comments on guidelines (editorial) 2006-08-09 09:53:08 +0000 1 1 1 Unclassified XML Query Test Suite XML Query Test Suite unspecified PC Windows XP CLOSED FIXED P2 normal --- 1 davidc carmelo public-qt-comments oldest_to_newest 10866 0 davidc 2006-07-28 10:46:25 +0000 I started to make some comments in bug #3532 but that's probably the wrong place for editorial comments, so I opened another report. These comments are on the guidelines cvs version 1.22 I think that the whole section "Customizing Namespaces" should be removed. All tests using schema import have now been removed from the minimal conformance section, and the tests that do use schema import, changing it to a namespace declaration will not produce the expected result. the section on module location hints says The "at" keyword specifies an optional location hint. Location Hints can be interpreted or disregarded in an implementation-dependent way. An implementation can choose to use any of the location hints, or none at all. which is more or less what it has to say, given the specification, but as noted in bug #3003, some of the tests assume that all the location hints are used (as they test for duplicated declarations) I think the guidelines need to allow implementors testing a system that would only use one location hint to rewrite the query to use a URI to a system specific module which has done the merge "by hand". Customizing XQueryX Tests The example XQueryX is not valid xqx:external may not take content. (first reported in bug #2400) in Comparing Results you say Error: The expected result of the test case is and error, identified as an eight-character error code (e.g., XPST0003). The result of a test is true, if the implementation raises an error. apart from the typo "and error" which should read "an error" Am I correct to read this as saying that it is _not_ necessary to check that exactly the correct error code is produced. My current test harness checks the codes but classes the test as a pass anyway, adding a comment if the codes are different. I thought I was doing that incorrectly and was planning to classify these as failures but if it is OK to classify these as pass, that is good (for me). 10897 1 andrew.eisenberg 2006-07-31 19:20:27 +0000 (In reply to comment #0) > I started to make some comments in bug #3532 but that's probably the wrong > place for editorial comments, so I opened another report. > > These comments are on the guidelines cvs version 1.22 > > I think that the whole section "Customizing Namespaces" should be removed. > All tests using schema import have now been removed from the minimal > conformance section, and the tests that do use schema import, changing it to a > namespace declaration will not produce the expected result. Our thinking when we wrote this was that some XQuery implementations would choose to augment their static context with the necessary schema definitions and statically known namespaces, making the import schema clauses unnecessary. > Customizing XQueryX Tests The example XQueryX is not valid xqx:external may > not take content. (first reported in bug #2400) I believe that you are suggesting that the example be rewritten as: <xqx:varDecl> <xqx:varName>input-context</xqx:varName> <xqx:external/> </xqx:varDecl> <xqx:varRef> <xqx:name>input-context</xqx:name> </xqx:varRef> > in Comparing Results > you say > > Error: The expected result of the test case is and error, identified as an > eight-character error code (e.g., XPST0003). The result of a test is true, if > the implementation raises an error. > > apart from the typo "and error" which should read "an error" Am I correct to > read this as saying that it is _not_ necessary to check that exactly the > correct > error code is produced. My current test harness checks the codes but classes > the test as a pass anyway, adding a comment if the codes are different. I > thought I was doing that incorrectly and was planning to classify these as > failures but if it is OK to classify these as pass, that is good (for me). You are doing this correctly. In section 2.3.2, Identifying and Reporting Errors, XQuery says: "The method by which an [XPath/XQuery] processor reports error information to the external environment is implementation-defined." We didn't feel that we could require implementations to return specific URIs to identify specific errors. 10929 2 davidc 2006-08-01 09:32:14 +0000 > <xqx:external/> yes, so it's schema valid:-) > You are doing this correctly. Ah OK thanks for the confirmation. I hadn't noticed that previously. Actually each time I've released a new version of my test results I've always tried to take care to flag this as something I'm _not_ doing correctly. So I can stop doing that now. I will however keep my current practice of commenting the tests where I get "incorrect" codes. The development version of xq2xsl is actually a lot better at getting the codes right but there are still a couple of thousand or so cases where I get the wrong code (usually because the W3C XQuery parser doesn't distinguish errors with codes, or because I can't trap underlying XSLT errors, so XSLT rather than XQuery codes are reported). Probably other implementors will have similar issues with codes so I agree classing "wrong error code" as "fail" would be a bit severe. However it may be worth having a different classification other than pass such as "pass-up-to-error-code" (probably with a shorter name) so that you can more easily see which systems are using the W3C specified error codes and which are not, even if you treat pass and pass-up-to-error-code as equivalent for the purpose of the CR implementability testing. 11056 3 andrew.eisenberg 2006-08-08 21:22:57 +0000 (In reply to comment #0) > I think that the whole section "Customizing Namespaces" should be removed. > All tests using schema import have now been removed from the minimal > conformance section, and the tests that do use schema import, changing it to a > namespace declaration will not produce the expected result. Yes, this has been changed. > the section on module location hints says > > The "at" keyword specifies an optional location hint. Location Hints can be > interpreted or disregarded in an implementation-dependent way. An > implementation can choose to use any of the location hints, or none at all. > > which is more or less what it has to say, given the specification, but as noted > in bug #3003, some of the tests assume that all the location hints are used (as > they test for duplicated declarations) I think the guidelines need to allow > implementors testing a system that would only use one location hint to > rewrite the query to use a URI to a system specific module which has done the > merge "by hand". I believe that Carmelo made ths change. > Customizing XQueryX Tests The example XQueryX is not valid xqx:external may > not take content. (first reported in bug #2400) This has been changed. > in Comparing Results > you say > > Error: The expected result of the test case is and error, identified as an > eight-character error code (e.g., XPST0003). The result of a test is true, if > the implementation raises an error. > > apart from the typo "and error" which should read "an error" Am I correct to > read this as saying that it is _not_ necessary to check that exactly the > correct > error code is produced. My current test harness checks the codes but classes > the test as a pass anyway, adding a comment if the codes are different. I > thought I was doing that incorrectly and was planning to classify these as > failures but if it is OK to classify these as pass, that is good (for me). Yes, it is not necessary to have the correct error code to pass the test (see comment #1). Please mark this report closed if you agree with our resolutions. 11062 4 davidc 2006-08-09 09:53:08 +0000 thanks