This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 3346 - K-NumericAdd-5, K-NumericAdd-6: floating point assumption wrong
Summary: K-NumericAdd-5, K-NumericAdd-6: floating point assumption wrong
Status: CLOSED FIXED
Alias: None
Product: XML Query Test Suite
Classification: Unclassified
Component: XML Query Test Suite (show other bugs)
Version: 0.9.4
Hardware: PC Windows XP
: P2 normal
Target Milestone: ---
Assignee: Frans Englich
QA Contact:
URL:
Whiteboard:
Keywords:
: 3382 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-06-18 20:14 UTC by Michael Kay
Modified: 2006-06-27 15:48 UTC (History)
1 user (show)

See Also:


Attachments

Description Michael Kay 2006-06-18 20:14:08 UTC
The test is:

xs:double(1.1) + xs:double(2.2) eq 3.3

I believe the correct answer is false().

The internal representation of double 1.1 is x3ff199999999999a

The exact decimal value of this is 
1.100000000000000088817841970012523233890533447265625

This is the correct representation because the value is closer to 1.1 than the next lower value in the value space of double, whose exact decimal value is
1.0999999999999998667732370449812151491641998291015625

The internal representation of double 2.2 is x400199999999999a

The exact decimal value of this is 
2.20000000000000017763568394002504646778106689453125

This is the correct representation because the value is closer to 2.2 than the next lower value in the value space of double, whose exact decimal value is
2.199999999999999733546474089962430298328399658203125

The internal representation of double 3.3 is x400a666666666666

The exact decimal value of this is 
3.29999999999999982236431605997495353221893310546875

This is the correct representation because the value is closer to 3.3 than the next higher value in the value space of double, whose exact decimal value is
3.300000000000000266453525910037569701671600341796875

The result of adding the doubles 1.1 and 2.2 is the double whose internal representation is x400a666666666667, whose exact decimal value is
3.300000000000000266453525910037569701671600341796875 

This is not the same as the internal representation of the double 3.3.

So 1.1 + 2.2 != 3.3. 

QED.

I believe the same reasoning applies to K-NumericAdd-6, which uses float arithmetic, though I have not done the detailed analysis for this case.
Comment 1 Frans Englich 2006-06-27 15:25:21 UTC
*** Bug 3382 has been marked as a duplicate of this bug. ***
Comment 2 Frans Englich 2006-06-27 15:47:34 UTC
K-NumericAdd-5 was changed to:
xs:double(1.1) + xs:double(2.2) ne 3.3

K-NumericAdd-6 was changed to:
xs:float(1.1) + xs:float(2.2) ne 3.3

Nice analysis. Might very well come in handy in other situations.

Since this erport was fixed as suggested, I will change status to CLOSED.



Frans