2989 2006-03-07 04:18:08 +0000 Datatypes 2006-02-17 WD: floatingPointRound function 2007-09-18 12:46:12 +0000 1 1 1 Unclassified XML Schema Datatypes: XSD Part 2 1.1 only Macintosh All CLOSED FIXED thimble, easy P2 normal --- 1 xan.gregg cmsmcq www-xml-schema-comments oldest_to_newest 8613 0 xan.gregg 2006-03-07 04:18:08 +0000 The function floatingPointRound() in APpendix E.1 appears to have incorrect and/or redundant information. Below is an excerpt, converted to ASCII. 2. So select e that 2^cWidth * 2^(e ? 1) < |nV| <= 2^cWidth * 2^e . 3. So select c that (c ? 1) * 2^e <= |nV| <c × 2^e and 2^(cWidth?1) < c <= 2^cWidth . It's the second clause of step #3 (3b) that is at issue (the part after "and"). "2^(cWidth?1) < c" seems redundant since is c <= 2^(cWidth-1), then by the (3a), |nV| < 2^(cWidth -1 + e), but by (2), we already have 2^(cWidth + e - 1) < |nV|. "c <= 2^cWidth" indicates something is wrong. Suppose |nV| == 2^cWidth * 2^e, as permitted by (2). Then, |nV| < c * 2^e (from (3a)) implies 2^cWidth * 2^e < c * 2^e which reduces to 2^cWidth < c, so c can't be less than or equal to 2^cWidth as (3b) requires. Perhaps the ... <= |nV| < ... in (3a) should be ... < |nV| <= ... and (3b) is entirely redundant. 8643 1 davep 2006-03-09 03:31:13 +0000 (In reply to comment #0) > Perhaps the ... <= |nV| < ... in (3a) should be ... < |nV| <= ... and (3b) is > entirely redundant. Actually, it's 2 that is in error. The '<' and '<=' should be switched. That puts |nV| in the proper top-open interval of normaiized values corresponding to e. Then 3 correctly finds adjacent values (c-1)*2^e and c*2^e in the *closed* interval that bracket |nV|. If |nV| *is* one of the floating point values, the '<= will insure that we ultimately get the one that is in the interval. (Note that in 3b. the '<' and '<=' should *not* be switched. The '<' insures that (c-1)*2^e is in the e-segment and the '<=' allows the approximation to round correctly for exact values higher than the last floating-point value in the partition.) It's entirely possible that an exact value |nV| very close to the top of the e-segment will round to the floating-point value that is the top of the segment and is not part of the interval, but at that point the algorithm no longer cares what segment the rounded-to value is in. Of course, I'm fallible; check me on this! 12961 2 davep 2006-11-18 02:03:40 +0000 Approved by the WG; awaiting incorporation into the status quo document. 16638 3 cmsmcq 2007-09-18 00:40:02 +0000 The change proposed above was approved by the WG in its call of 17 November 2006. It is now reflected in the status quo version of the Datatypes spec. Accordingly, I am setting the disposition of this issue to RESOLVED / FIXED. If the originator of the issue would examine the change and let us know whether it satisfactorily resolves the problem or not, we'd be grateful. To signal that the resolution is acceptable, change the status of the issue to CLOSED. Otherwise, to signal that it's NOT acceptable, change the status to REOPENED (and tell us what's wrong). If we don't hear from you in the next three weeks, we'll assume that silence betokens consent, and close the issue ourselves. 16666 4 xan.gregg 2007-09-18 12:46:12 +0000 Marking closed.