This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
Our implementation returns the following results which differ from the expected results by a single bit in their representation. math-exp10-004: 3.162277660168379 math-tan-006: 1.6331778728383844E16 math-tan-007: -1.6331778728383844E16
Yes, we observed a similar effect with Saxon - the .NET library methods give slightly different results to the Java library methods. I haven't attempted to work out whether both are conformant to the IEEE spec, which is what we reference.
We need discussion on whether to accept these results.
The first test is exp10(0.5), which means the same as sqrt(10), which Wolfram Alpha gives as 3.162277660168379331998893544432718533719555139325216826857504852792594438639238221344248108379300295187... This lies between the two candidate results of 3.162277660168379 and 3.1622776601683795. Note that for math-tan-006 and math-tan-007 the mathematical answer is infinity (and this is the value that Wolfram Alpha gives). So the test is close to a singularity in the function, which means that expecting high precision in the answer is unreasonable. If we accept pi = 3.141592653589793, we have two possible value for pi/2 depending on the rounding mode: 1.5707963267948966 and 1.5707963267948967. According to WA, tan(1.5707963267948966) is 5.1998506188720270660194741661226868475811544986... × 10^16, while tan (1.5707963267948966) is -1.238103706714353220651983005784184862681851980... × 10^16. So if we allow pi/2 to be rounded either way, the range of permissible values here is vast. In the spec we say "the preferred quantum is implementation-defined". I think this may be a misunderstanding of what the 'quantum' is. I suspect that by specifying that the operations are done in xs:double floating point, we have effectively defined the quantum. More pertinently to this bug, th IEEE spec says (in 4.3) there is a choice of "rounding direction", with options such as towards zero, towards positive, towards negative. We haven't said anything about this. We should probably say "The choice of rounding direction, and any mechanisms for influencing the choice of rounding direction, are implementation-defined." This means that the two results for sqrt(10) given above differ only in the choice of rounding direction, and both are therefore acceptable. Recommendation: (a) for math-tan-006 and math-tan-007, change the tests to avoid evaluating the function so close to a singularity, where there is a high sensitivity to legitimate implementation differences. (b) for math-exp10-004, accept both results (c) in the spec, change what we say about precision of results, to include the statement about rounding direction given above.
The recommendations in comment 3 were accepted by the WG and have been implemented.
For math-tan-006: math:tan(math:pi() div 4) I get the result 0.9999999999999999 For math-tan-007: math:tan(math:pi() div 4) I get the result -0.9999999999999999
Those look identical to the results expected - I'm not sure what the problem is.
The problem turned out to be an error in casting from decimal to double in the test harness. Note that the expected result is expressed as a decimal, whereas the actual result is a double.
*** Bug 18478 has been marked as a duplicate of this bug. ***