This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
The test uses as supposingly simple input avg((1,2,4)) and expects 2.333333333333333333 or 2.333333333333. But it is evaluated as 7 div 3, which becomes an xs:decimal which has implementation-defined precision (For xs:decimal values the number of digits of precision returned by the numeric operators is ·implementation-defined·.). My implementation returns 2.33333333333333333
The WG discussed this issue at today's telcon. The resolution is to use the assert-eq which allows us compare the actual result with '7 div 3'.
The resolution of '7 div 3' adds the potential risk of allowing any answer. What would be better is to give a range of something between 2.332 and 2.334, or similar. Maybe there is a minimum preision that we can use here.
Or a regular expression 2\.3+
I would like to push back on the proposed solutions in comment#2 and comment#3. fn:avg((1,2,4)) is fn:sum((1,2,4)) div 3, which is xs:integer(7) div xs:integer(3) resulting in xs:decimal with value of 7/3. The spec mandates that minimally conforming processors support at least 18 total digits (XSD 1.0) or 16 total digits (XSD 1.1, see also Bug 29552) . That means that: 2.33333333333333333 (Benito's processor, is correct: 18 digits) 2.333333333333333 (XSD 1.1 minimum, is correct: 16 digits) 2.333333333333 (alternative in current test is incorrect: 13 digits) 2.333333333333333333 (alternative in current test is allowed, but not required: 19 digits) The solution in comment#2 allows wrong answers regardless of the precision, and the solution in comment#3 allows too little precision. I suggest something like any of the following three alternatives: exact: 2.33333333333333333 exact: 2.333333333333333 regex: 2.333333333333333333+ Since rounding is not an issue here, any precision will always end with a sequence of "3", so this will work as well and catches all mandatory and optional precisions (preventing an implementation to allow, for instance, to have 2.333300 or 2.33334 as valid answer): regex: 2.333333333333333(33+)?
> regex: 2.333333333333333(33+)? that should probably be ^2.333333333333333(33+)?$
F+O says in section 4.2: For xs:decimal values the number of digits of precision returned by the numeric operators is ·implementation-defined·. This is probably far more liberal than we really intended: for example it allows 1.01 + 1.04 to return 2. However, it doesn't say that the result of division must have any particular minimum precision, and I don't think we should read it as if it did. As a further complication, there is no requirement to implement avg((1,2,4)) as sum((1,2,4)) div count((1,2,4)): we say that "the implementation may use an otherwise equivalent algorithm that avoids arithmetic overflow". Avoiding overflow may well involve loss of precision, and the loss of precision can be justified by the fact that the precision of "div" is implementation defined. So a final result like 2.33333333334 is not at all out of the question. Pragmatically, the spec is loosely worded in this area and it therefore makes sense for the corresponding tests to have loosely-defined pass criteria.
> may use an otherwise equivalent algorithm that avoids arithmetic overflow I would understand "equivalent" as not allowing a different outcome... > For xs:decimal values the number of digits of precision returned by the > numeric operators is ·implementation-defined·. I understood that as the implementation-defined limit we have on xs:decimal anyway, with a minimum of 18 digits. I think (but not sure) that in IEEE-754 there's a section on decimals that requires additional digits if necessary to prevent precision-issues, but I don't think XPath leans on IEEE-754 for decimal arithmetic (only used for double/float). If what you are say is the de facto rule, then I think there are many tests that should be more lenient in what answers they expect. I remember fixing quite a few on my end to get the proper amount of digits in the result to fit the tests....
>I remember fixing quite a few on my end to get the proper amount of digits in the result to fit the tests.... I, for one, even had to write an entire arbitrary precision arithmetic library for that. That some other xs:float test wanted 2.33333... and did not accept 2.33...34 helped to find a bug there
At the meeting on 2016-04-12, the WG decided: Add text along these lines to the F&O specification: "implementation_defined" for results of calculations with xs:decimal or derived types *SHOULD* be at least the minimal requirement for the number of digits to be supported for xs:decimal Tests will reflect the above semantics. Action A-639-06 was raised to track this change