This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 26585 - Proposal: fn:apply
Summary: Proposal: fn:apply
Alias: None
Product: XPath / XQuery / XSLT
Classification: Unclassified
Component: Functions and Operators 3.1 (show other bugs)
Version: Working drafts
Hardware: PC All
: P2 normal
Target Milestone: ---
Assignee: Michael Kay
QA Contact: Mailing list for public feedback on specs from XSL and XML Query WGs
Depends on:
Reported: 2014-08-14 19:32 UTC by Marc
Modified: 2015-02-10 17:01 UTC (History)
6 users (show)

See Also:

demo for apply (limited) functionality built in xquery (455 bytes, text/plain)
2014-08-20 13:44 UTC, Rob
apply module built in xquery (limited functionality) (1.88 KB, text/plain)
2014-08-20 13:52 UTC, Rob
apply with array semantics (2.33 KB, text/plain)
2014-08-26 07:31 UTC, Marc
unit tests for apply with array semantics (1.57 KB, text/plain)
2014-08-26 07:32 UTC, Marc

Description Marc 2014-08-14 19:32:38 UTC
XQuery 3.0 introduced functions that are derived from functional programming.
Function items can be passed as arguments to functions like fn:fold-left().
Together they allow a more functional style of coding. Most of these functions
can be used to perform map, filter, reduce on sequences.

However, one important function that is missing and for which I do not know a
reasonable workaround for is "apply" [1]. The wikipedia page shows that many
languages have ways to achieve this and Clojure has a function with the same
name [2].

In XQuery we can bind a variable to a function item. But when we want to use
it we are limited to one argument.

  $f = function my-add($a,$b,$c) { $a + $b + $c }
  $args = (1,2,3)
  $f($args) == $f((1,2,3))

If we had an apply function this would do it.
  fn:apply($f, $args) == $f(1,2,3)

Of course we could rewrite the function my-add to something like:

  $f = function my-add($seq) { sum($seq) }
  $f($args) == 6
But this is only practical for a limited number of cases and the function body
would need to unpack the sequence, and we also lose the type checking.

This limitation also shows with the proposed arrow operator (=>).

Someone coming from a language like Clojure, Lisp or Scheme would interpret the

  (1,2,3) => $f

to mean 


but in XQuery it would mean


The arrow operator is similar to Clojure's threading operator (->) [3].


  (-> (1 2 3) f1 f2) == (f2 (f1 1 2 3))

Note that in Clojure -> is a macro which re-writes the form before the
compiler will see it. But I digress.

  (1,2,3) => $f1 => $f2 

Maybe this is a consequence of the fact that sequences cannot be nested, and
are effectively flattened.

I cannot oversee all the consequences but like a map may contain other maps
arrays can be nested so they could be used for "apply" semantics.

Currently we have:

  (1,2,3) => $f


With array apply semantics:
  [1,2,3] => $f1 => $f2
  == $f2($f1(1,2,3))
  [(1,2),(3,4)] => $f1
  == $f1((1,2),(3,4))
  == fn:apply($f1, [(1,2),(3,4)])

On the surface this looks like an elegant solution to me and is closer to
the apply semantics in other languages.

A particular use case for fn:apply arose when I tried implementing Clojure Ring
and Compojure libraries in XQuery. These are comparable to WSGI in Python and
Rack in Ruby.

Below I simplified things but I hope it illustrates a concrete use case.

I have a function for defining a routing handler which maps an HTTP GET request to
a handler function. The def-route() signature is

    $method as xs:string,
    $url-template as xs:string,
    $params as array(*),
    $handler as function(*)
  ) as function(map(*)) as map(*)
This means that a route handler function is something that takes a request
map as input and produces a response map as output.

  $handler = 
      function($name, $age) { 
        'Person: ' || $name || ' (age ' || $age ')' 
  $route = def-route('GET', '/person/{name}/{age}', ['name', 'age'], $handler)
  $route(request('GET', '/person/Joe/10'))

When a GET request comes in on '/person/Joe/10'. The route-handler $route
matches and the 'name' and 'age' parameters are parsed from the URL and added to
the request map. The third argument is an array that specifies which keys need
to be taken from the request map and builds the $params array: ['Joe', '10'].
Finally it can invoke the $handler using

  fn:apply($handler, $params)


  $params => $handler

  == ['Joe', '10'] => $handler
This results in a response('Person: Joe (age 10)') which is then rendered to the

Note that the handler does not need to know where to find the parameters because
this is set up by the $route handler thus achieving a clean separation of
concerns and the possibility of using $handler in other situations because
it is not tied to the plumbing/routing logic.

As I said earlier, I may not be able to oversee all the consequences as I'm not
super intimate with all the specs involved. Or, my pseudo code is confusing. Mea


Comment 1 Michael Kay 2014-08-14 20:03:02 UTC
In the past it hasn't been possible to treat the list of arguments of a function as a single value because sequences can't be nested, as you suggest. That difficulty is partly solved by the introduction of arrays. But I think there are considerable difficulties that remain, notably the fact that the arity of a function in XQuery is so much part of its nature. Doing this would be powerful, but a lot of work to do well.
Comment 2 Rob 2014-08-20 13:44:37 UTC
Created attachment 1503 [details]
demo for apply (limited) functionality built in xquery
Comment 3 Rob 2014-08-20 13:52:31 UTC
Created attachment 1504 [details]
apply module built in xquery (limited functionality)

The 'APPLY:argument.constructor' function gives problems in xquery. But since the module should be built in java these problems probably can be overcome.
Comment 4 Rob 2014-08-20 14:03:06 UTC
for inspiration a apply-function built in xquery. The challenge in xquery is the (re-)creation of the argumentlist for the evaluationstring. But if the apply-function would be a part of the xquery-language itself it probably will be built in java. I'm not a java-expert but I wander if the problem that exists on xquery-level also exists on java-level.
Comment 5 Marc 2014-08-26 07:31:14 UTC
Created attachment 1505 [details]
apply with array semantics

Rob's example almost worked for me. I have added some extra argument serialization and handling of arrays as described in my proposal. Here's the adapted code. Note that 1) it may now be BaseX specific due to use of map and array modules and 2) I heavily reformatted the code so it matches the other code in my little library and 3) I have done little testing on this so there are probably many potholes.
Next attachment will have some unit tests.
Comment 6 Marc 2014-08-26 07:32:17 UTC
Created attachment 1506 [details]
unit tests for apply with array semantics

Here are the BaseX specific unit tests.
Comment 7 Leo Wörteler 2014-09-11 15:22:04 UTC
The current implementation of the arrow operator in the Working Draft [1] defines it as a syntactic macro:

> If `$i` is an item and `f()` is a function, then `$i=>f()` is equivalent
> to `f($i)`, and `$i=>f($j)` is equivalent to `f($i, $j)`.

I think this is not very elegant and possibly confusing. Since e.g. `local:foo#0` and `local:foo#1` can be completely different functions in XQuery, it is potentially dangerous that in

  1 => local:bar() => local:foo()

it is not immediately obvious which of them is called.

I would propose that the second argument of `=>` should instead be a function item taking one argument. Then `$arg => $f` can be translated into `$f($arg)` directly and the Spec can define it simply as equivalent to:

  function op:arrow-apply(
    $arg as item()*,
    $func as function(item()*) as item()*
  ) as item()* {

As a nice bonus this also makes the feature more flexible because the argument to be inserted does not have to be the first one in the function:

  $file-extension => csv:get-separator() => (tokenize($line, ?))()

could be written as

  $file-extension => csv:get-separator#1 => tokenize($line, ?)

Everything that was possible before should still work when adding a "?" at the start of the ArgumentList of each right-hand side of `=>`. The example from the Spec becomes

  $string => upper-case(?) => normalize-unicode(?) => tokenize(?, "\s+")

or (shorter and more elegant):

  $string => upper-case#1 => normalize-unicode#1 => tokenize(?, "\s+")

In conclusion, using function items is more flexible and less confusing, and the syntactic translation scheme makes for only marginally less verbose tyntax.

Comment 8 Christian Gruen 2014-09-23 11:29:24 UTC
I copied Leo Wörteler's proposal in an extra issue:
Comment 9 Liam R E Quin 2014-10-28 22:02:29 UTC
The WG is considering comment 7 separately (thank you, Christian, for copying it).

On the main subsance of this issue, it's possible (I think) that a function like fn:apply($f as function, $args as array) could be defined by F & O in the future, but the Working Groups have not made such a decision now.
Comment 10 C. M. Sperberg-McQueen 2014-11-29 02:27:53 UTC
On the joint call of 25 November I took an action to try to work out a simple use case for the fn:apply() function requested by the OP, as sketched by Michael Kay in [1] and as commented on by John Snelson in [2].


I have posted the results separately to the public QT comments list [3], to avoid overburdening a Bugzilla comment.


In that mail, I provide examples of the use of fn:apply() as sketched in [1] for currying functions, for composing functions, and for use in meta-circular interpreters.

The bottom line is that while I agree with Michael Kay and John Snelson that variable-arity functions would be handy, and would make fn:apply() more useful and convenient, I believe that it's useful enough to be included in the spec even without variable-arity functions.
Comment 11 Michael Kay 2014-12-02 22:40:41 UTC
The proposal to add an fn:apply() function was accepted.

The semantics of fn:apply($f, [$a, $b, $c, ...]) are the same as $f($a, $b, $c, ...) but of course the arity does not need to be known statically.