This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 9009 - oops! in recommended minimum partial implementation of precisionDecimal
Summary: oops! in recommended minimum partial implementation of precisionDecimal
Status: CLOSED FIXED
Alias: None
Product: XML Schema
Classification: Unclassified
Component: Datatypes: XSD Part 2 (show other bugs)
Version: 1.1 only
Hardware: Macintosh Mac System 9.x
: P2 normal
Target Milestone: ---
Assignee: David Ezell
QA Contact: XML Schema comments list
URL:
Whiteboard:
Keywords: decided
Depends on:
Blocks:
 
Reported: 2010-02-16 04:18 UTC by Dave Peterson
Modified: 2010-04-26 14:58 UTC (History)
1 user (show)

See Also:


Attachments

Description Dave Peterson 2010-02-16 04:18:20 UTC
In section 5.4 we require:

"All ·minimally conforming· processors must support all precisionDecimal values in the ·value space· of the otherwise unconstrained ·derived· datatype for which totalDigits is set to sixteen, maxScale to 369, and minScale to −398."

By comparison, we recommend:

"Note: The conformance limits given in the text correspond to those of the decimal64 type defined in [IEEE 754-2008], which can be stored in a 64-bit field. The XML Schema Working Group recommends that implementors support limits corresponding to those of the decimal128 type. This entails supporting the values in the value space of the otherwise unconstrained datatype for which totalDigits is set to 34, maxScale to 6176, and minScale to −6111."

The IEEE spec prescribes a maximum floating-point precision (total significant digits) and a maximum exponent for each of their datatypes (which they call "formats").  The total digits limit the significand, a number of absolute value less than 10; i.e., one that can be represented by a decimal numeral with one digit preceding the decimal point.  It also points out that if that significand is to be interpreted as an integer (no decimal point), the corresponding exponent must be offset by one less than the floating-point precision.  We interpret maxScale and minScale as limiting with an integer significand, so we must offset the exponent limits.

The IEEE specifies decimal64 to have a maximum allowed exponent of 384 and floating-point precision of 16.  Therefore our corresponding maxScale should be (and is) 384 - 15 = 369.  OK so far.  But for decimal128, the max exponent is specified as 6144 and the max f-p precision as 34, so our corresponding  maxScale should be 6144 - 33 = 6111 .  Correspondingly, minScale should be -6143 - 33 = -6176.  We got the signs right but the digit strings switched in the Note quoted above.