This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
The function floatingPointRound() in APpendix E.1 appears to have incorrect and/or redundant information. Below is an excerpt, converted to ASCII. 2. So select e that 2^cWidth * 2^(e ? 1) < |nV| <= 2^cWidth * 2^e . 3. So select c that (c ? 1) * 2^e <= |nV| <c × 2^e and 2^(cWidth?1) < c <= 2^cWidth . It's the second clause of step #3 (3b) that is at issue (the part after "and"). "2^(cWidth?1) < c" seems redundant since is c <= 2^(cWidth-1), then by the (3a), |nV| < 2^(cWidth -1 + e), but by (2), we already have 2^(cWidth + e - 1) < |nV|. "c <= 2^cWidth" indicates something is wrong. Suppose |nV| == 2^cWidth * 2^e, as permitted by (2). Then, |nV| < c * 2^e (from (3a)) implies 2^cWidth * 2^e < c * 2^e which reduces to 2^cWidth < c, so c can't be less than or equal to 2^cWidth as (3b) requires. Perhaps the ... <= |nV| < ... in (3a) should be ... < |nV| <= ... and (3b) is entirely redundant.
(In reply to comment #0) > Perhaps the ... <= |nV| < ... in (3a) should be ... < |nV| <= ... and (3b) is > entirely redundant. Actually, it's 2 that is in error. The '<' and '<=' should be switched. That puts |nV| in the proper top-open interval of normaiized values corresponding to e. Then 3 correctly finds adjacent values (c-1)*2^e and c*2^e in the *closed* interval that bracket |nV|. If |nV| *is* one of the floating point values, the '<= will insure that we ultimately get the one that is in the interval. (Note that in 3b. the '<' and '<=' should *not* be switched. The '<' insures that (c-1)*2^e is in the e-segment and the '<=' allows the approximation to round correctly for exact values higher than the last floating-point value in the partition.) It's entirely possible that an exact value |nV| very close to the top of the e-segment will round to the floating-point value that is the top of the segment and is not part of the interval, but at that point the algorithm no longer cares what segment the rounded-to value is in. Of course, I'm fallible; check me on this!
Approved by the WG; awaiting incorporation into the status quo document.
The change proposed above was approved by the WG in its call of 17 November 2006. It is now reflected in the status quo version of the Datatypes spec. Accordingly, I am setting the disposition of this issue to RESOLVED / FIXED. If the originator of the issue would examine the change and let us know whether it satisfactorily resolves the problem or not, we'd be grateful. To signal that the resolution is acceptable, change the status of the issue to CLOSED. Otherwise, to signal that it's NOT acceptable, change the status to REOPENED (and tell us what's wrong). If we don't hear from you in the next three weeks, we'll assume that silence betokens consent, and close the issue ourselves.
Marking closed.