This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 3529 - [F&O] digits of precision returned by the numeric operators
Summary: [F&O] digits of precision returned by the numeric operators
Alias: None
Product: XPath / XQuery / XSLT
Classification: Unclassified
Component: Functions and Operators 1.0 (show other bugs)
Version: Candidate Recommendation
Hardware: PC Windows XP
: P2 normal
Target Milestone: ---
Assignee: Ashok Malhotra
QA Contact: Mailing list for public feedback on specs from XSL and XML Query WGs
Depends on:
Reported: 2006-07-24 21:04 UTC by Andrew Eisenberg
Modified: 2007-02-25 23:33 UTC (History)
1 user (show)

See Also:


Description Andrew Eisenberg 2006-07-24 21:04:48 UTC
Section 6.2, Operators on Numeric Values, states in part:

"For xs:decimal values the number of digits of precision returned by the numeric operators is ·implementation-defined·. If the number of digits in the result exceeds the number of digits that the implementation supports, the result is truncated or rounded in an ·implementation-defined· manner."

Consider the query "1 div 3". Certainly a result of 0.333333333333333333 (18 digits) would be permitted. Would a value of 0.333333333333333 (15 digits) be permitted as well?

Michael Kay has argued:

"The first sentence must be read in the context of the second, which creates a
relationship between the number of digits in the result of decimal division and
the number of digits supported by the xs:decimal data type. The latter must be
at least 18, and I believe that the result of division should therefore not
truncate or round unless it requires more than 18 digits."

This question is being raised based on several test cases, as reported on the XML Query Test Suite in [1] and [2].

Comment 1 Carmelo Montanez 2006-08-01 15:50:53 UTC

This bug was closed with no further action as per the Working Group's decision on August 1, 2006.  15 digist will be acceptable for this type of this type
of operations.

Comment 2 Jim Melton 2007-02-25 23:33:18 UTC
Closing bug because commenter has not objected to the resolution posted and more than two weeks have passed.