This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 3615 - [F&O] Precision in fn:avg()
Summary: [F&O] Precision in fn:avg()
Status: CLOSED FIXED
Alias: None
Product: XPath / XQuery / XSLT
Classification: Unclassified
Component: Functions and Operators 1.0 (show other bugs)
Version: Candidate Recommendation
Hardware: PC Linux
: P2 normal
Target Milestone: ---
Assignee: Ashok Malhotra
QA Contact: Mailing list for public feedback on specs from XSL and XML Query WGs
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-08-22 17:41 UTC by John Snelson
Modified: 2007-01-30 20:33 UTC (History)
0 users

See Also:


Attachments

Description John Snelson 2006-08-22 17:41:54 UTC
The specification for fn:avg() includes the line:

    Otherwise, returns the average of the values computed as sum($arg) div
    count($arg).

Whilst this is a useful description of the results required from the function, it could also be seen as being perscriptive of how the function must be implemented. One consequence of this can be seen in the XQTS test fn-avgflt2args-1, with a discussion about it in bug #3610.

To summerise, the test query is:

fn:avg((xs:float("-3.4028235E38"),xs:float("-3.4028235E38")))

and the test expects a result of "-INF", since "-3.4028235E38" is the lower bound for an xs:float and the fn:sum() operation will underflow. However, I think it is entirely reasonable to implement fn:avg() in such a way as to return the answer "-3.4028235E38" for this query, which is both a valid xs:float and the correct mathematical result.

I propose that the sentance above from the F&O specification be changed to the following:

    Otherwise returns the average of the values, computed as sum($arg) div
    count($arg) or otherwise.
Comment 1 Michael Kay 2006-08-22 18:09:37 UTC
"or otherwise" is far too loose: it means that anything goes.

Suggestion: 

"Otherwise, returns the average of the values computed as sum($arg) div count($arg); but if this expression causes an arithmetic overflow, the implementation may use an otherwise equivalent algorithm that avoids the overflow.
Comment 2 Ashok Malhotra 2006-08-22 19:23:27 UTC
I would like to propose a minor change to Mike's suggested text:

"Otherwise, returns the average of the values computed as sum($arg) div count($arg); or an otherwise equivalent algorithm."

This allows not only algorithms that avoid the overflow but also algorithms
that may be preferable for performance or other reasons.
Comment 3 Michael Kay 2006-08-22 19:56:15 UTC
I don't think that change is necessary. We don't mandate the algorithm you use, only that its result must be equivalent to the result of sum($arg) div
count($arg). We're now introducing an exception to that rule for the case where that expression causes an overflow. We don't need any other exceptions, as far as I can see.
Comment 4 Michael Kay 2006-08-25 18:17:37 UTC
The WGs discussed the proposals. There was agreement with the general sentiment of the comment that the implementation was allowed to avoid the overflow here, but a fair bit of discussion about how to achieve it. The meeting converged on the following wording:

Otherwise, returns the average of the values as sum($arg) div count($arg); but the implementation may use an otherwise equivalent algorithm that avoids arithmetic overflow.