This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 24125 - [xslt 3.0] intrinsically unstreamable functions
Summary: [xslt 3.0] intrinsically unstreamable functions
Status: CLOSED FIXED
Alias: None
Product: XPath / XQuery / XSLT
Classification: Unclassified
Component: XSLT 3.0 (show other bugs)
Version: Last Call drafts
Hardware: PC All
: P2 normal
Target Milestone: ---
Assignee: Michael Kay
QA Contact: Mailing list for public feedback on specs from XSL and XML Query WGs
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-12-17 11:04 UTC by Michael Kay
Modified: 2014-05-15 14:00 UTC (History)
1 user (show)

See Also:


Attachments

Description Michael Kay 2013-12-17 11:04:33 UTC
Certain functions including reverse, innermost, and fold-right are marked with an operand usage of "N", which makes the function non-streamable unless the argument is grounded. The same applies to some instructions such as perform-sort.

However, these functions are not truly streamable in the case of an argument that is consuming and grounded, for example //emp/copy-of().

Currently we allow operations such as reverse(//emp/copy-of()) within streamed templates. This might be what we want to do, but it is not truly streamable. In fact we don't distinguish in any way between copy-of(//emp) (a copy of a sequence), and //emp/copy-of() (a sequence of copies). 

So perhaps we should make no change here other than a note to make it clear that using copy-of() in this way does not guarantee "windowed streaming" when such functions are used.
Comment 1 Michael Kay 2013-12-17 12:42:23 UTC
Another problem with using "N" (navigation usage) to classify these functions is that it causes the function to become free-ranging when used with an input having climbing posture.

Consider the very reasonable expression reverse(ancestor::*)/name(.)

Because reverse has operand usage N this is considered free-ranging.
Comment 2 Michael Kay 2014-01-16 18:00:19 UTC
We decided this needed discussion at the F2F meeting.

Regarding comment 0, there was a feeling that when you use copy-of(), all bets are off on the amount of memory needed, so this is not a real problem. However, perhaps we need to say something more specific about the expectations of memory usage with "windowed streaming"?
Comment 3 C. M. Sperberg-McQueen 2014-02-10 09:38:53 UTC
The WG discussed this during the ftf meeting in Prague.

Among other things, we considered the two examples

  <xsl:for-each select="copy-of(.//emp)">...</xsl:for-each>

and

  <xsl:for-each select=".//emp/copy-of()">...</xsl:for-each>

At first glance, the first of these seems to invite materialization of the entire sequence and the second not.  But on further examination, we realized (with MK's help) that in fact a really naive processor would materialize both sequences completely, and a processor that does pipelined or lazy evaluation would materialize neither.  Adding reverse() around either of them makes it necessary to materialize the entire sequence in either case.

ABr suggested perhaps adding a pipeline=yes|no attribute to xsl:variable, to signal that at most one member of the sequence needs to be materialized at a time.  (There was some confusion over whether it would then be illegal to say reverse($v) or not.  In the end, we didn't reach a firm consensus either way on labeling the variables.)

We did seem to have consensus that the spec should have a note warning users that memory requirements for user-controlled storage will be higher when entire sequences must be materialized (as when reverse() and xsl:sort are used) than when they can be pipelined.  We may also want a note saying explicitly that implementations of the streaming feature are (informally) expected to use pipelined or lazy evaluation.
Comment 4 Michael Kay 2014-02-10 09:47:03 UTC
Concerning comment #1, we felt that it was reasonable to require the circumlocution

reverse(ancestor::*/name())
Comment 5 Michael Kay 2014-02-10 09:49:42 UTC
Left this with an action on the editor to draft a Note.
Comment 6 Michael Kay 2014-04-24 14:08:54 UTC
As directed, I have drafted non-normative notes to explain the status quo for the reverse, fold-right, and innermost functions.