This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 3618 - [FT] let score clause
Summary: [FT] let score clause
Status: CLOSED WONTFIX
Alias: None
Product: XPath / XQuery / XSLT
Classification: Unclassified
Component: Full Text 1.0 (show other bugs)
Version: Working drafts
Hardware: PC Windows XP
: P2 normal
Target Milestone: ---
Assignee: Jochen Doerre
QA Contact: Mailing list for public feedback on specs from XSL and XML Query WGs
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-08-23 10:10 UTC by David Carlisle
Modified: 2006-11-20 21:48 UTC (History)
0 users

See Also:


Attachments

Description David Carlisle 2006-08-23 10:10:10 UTC
section 2.3 says
The let variable may be dropped from the let clause, if the score variable is present.

and this is more or less the full description of the meaning of this expression.

I assumed from this that it meant that if the "normal" let variable is not used
in the return clause, it could be dropped, and that this was just a syntactic
simplification, so that

let $x score $s := $input[title ftcontains "foo"]
return
$s

could be equivalently written without the $x as

let score $s := $input[title ftcontains "foo"]
return
$s

however all actual examples of let score in the document so not use a node-valued expression but rather the full text boolean expression itself.

let score $s := title ftcontains "foo"
return
$s


Could the specification clarify for the let score clause whether the ExprSingle to the right of the := must be an ftselection and if it may either be an ftselection or a path expression, what exactly is being scored in either case.

The semantics of the expresion are further expanded in 4.3.2 but again that is essentially silent on whether fts:score expects a path expression or an ftselection as its argument.

This question came about trying to use scoring in xpath, where I need to return the score as a value. To return the score of testing the current node against "foo" I thought from reading the text that I had to do

let score $s :=.[. ftcontains "foo"] return $s

but the examples suggest that I could do

let score $s := . ftcontains "foo" return $s

but it isn't clear if these are equivalent. Of course as currently defined neither can be used in XPath so I have to use

for $x score $s in .[. ftcontains "foo"] return $s

which is fairly horrible, and strongly suggests that the currently proposed syntax for scoring is unsuitable for use in XPath.

It would be much better for XPath (and XQuery) use if there was a function syntax such as score() which returned the score of the current item (from wherever it was selected) The spec has some arguments why this is not possible in as the value would depend on the context but those arguments would apply equally to position() or xslt's current(), The context would have to hold the current nodes score as well as it's value. It doesn't mean that score() has to be a constant as it is a 0-ary function.

That would allow

<xsl:for-each select="/a/b[c ftcontains'x']">
  <xsl:sort select="score()"/>

(I'm using xslt here but any non-xquery binding of xpath would have the same issues).

I suspect that the WG would have some resistance to revisiting the score syntax
but if the current syntax is kept then at the very least I think that the let clause should be added back into xpath as extended by full text. Then at least
Xpath would have

<xsl:for-each select="/a/b[c ftcontains 'x']">
  <xsl:sort select="let score $s := c ftcontains 'x' return $s"/>

which is rather verbose and repetitive but much better than the current

<xsl:for-each select="/a/b[c ftcontains 'x']">
  <xsl:sort select="for $x score $s in .[c ftcontains 'x'] return $s"/>

See also 

David
see also the related bug #3596 and the earlier thread on qt-comments starting
http://lists.w3.org/Archives/Public/public-qt-comments/2006Aug/0081.html
Comment 1 Michael Rys 2006-11-01 03:59:29 UTC
Hi David

The Fulltext TF discussed this issue at its face-to-face meeting and took the following actions based on your feedback:

1. We change the syntax of the let expression to disallow let $v score $s and simplify it to:

(("let" "$" VarName TypeDeclaration?) | ("let" "score" "$" VarName))
":=" ExprSingle
    ("," (("$" VarName TypeDeclaration?) | FTScoreVar) ":=" ExprSingle)*

We are also going to clarify that the expression provided to a score variable will be interpreted as a boolean expression. That means that the normalization will add a call to fn:boolean() to the expression tree that is being passed to the internal score calculation engine.

We decided that allow any expression and allow minimal conforming implementations to restrict themselves to and and or combinations of ftselections.

As to the XPath proposal: We decided that we would consider this once the let clause is being added to XPath. Using a score() context functions is somewhat problematic since a score value is not intrinsic to the node like its position but it depends on the score expression. So you could have an arbitrary number of them associated with a node for every concurrent query.

The taskforce thinks that we have addressed your comments and we will resolve the bug as FIXED. Please close it if you agree with the resolution or reopen it if you are dissatisfied.
Comment 2 David Carlisle 2006-11-01 09:55:32 UTC
Thanks for your comments, however I'm re-opening the report while seeking some further clarifications.


> As to the XPath proposal: We decided that we would consider this once the let
> clause is being added to XPath.

Do you mean that there is an intention to add let to Xpath 2+n? Is there any
(public or W3C member) list of proposed additions where this is documented?

> Using a score() context functions is somewhat
> problematic since a score value is not intrinsic to the node like its position
> but it depends on the score expression. 

I'm sorry but I do not understand this comment at all. position() is not intrinsic to a node as it does not relate to a position in a node tree, but to its position in the sequence currently selected, if I select a node with parent::* it will always have position 1. I can select the same node with child::* and get position() 1001 or any other number. In what way would this be different from score() ?

David


Comment 3 Jim Melton 2006-11-01 17:18:49 UTC
David,

You asked:
   Do you mean that there is an intention to add let to Xpath 2+n? Is there any
(public or W3C member) list of proposed additions where this is documented?

I think it's safer to say that the XML Query and XSL WGs did nothing to XPath 2.0 that would rule out that possibility.  As it happens, the WGs are currently starting their planning cycle for their future work and I'm sure that the possibility of adding the let clause to XPath will be suggested by at least one party.  Because this reply gets copied to public-qt-comments, there is nothing else that I can add. 

With respect to scoring, you also asked:
   In what way would this be different from score() ?

If all you are talking only about syntactic differences, then of course the difference significantly affects where and how a score may be requested.  Semantically, it would be no different (as far as I can tell).  I won't claim to speak for each individual in the task force, but the resistance against going the route of providing scoring via a function may be based on the hope that it can be put into XPath in some future version.   Sorry I can't help more with the reasoning behind this decision. 
Comment 4 David Carlisle 2006-11-01 17:41:20 UTC
Jim,

I appreciate that a bug report on full text isn't the place to be committing the
joint working groups to any future xpath extensions, so I understand your reluctance to expand on the initial reply, but then in that case I think it's more accurate to say that there are currently no plans to add let to xpath, so the initial reply which was used to mark this report as "fixed" was at best misleading "once the let clause is being added" might better have been phrased as "if in some future release xpath syntax is extended". As such (if the specification isn't changed) I think this report would have to be marked as "WONTFIX" rather than "FIXED".


> If all you are talking only about syntactic differences, then of course the
> difference significantly affects where and how a score may be requested.

I was asking for clarification on the reasoning given for the WG not using a syntax more similar to position() which was;
> "a score value is not intrinsic to the node like its position"
whatever the merits of the chosen score syntax and any proposed alternatives
the reasoning given here appears to be simply false: the position of a node as returned by position() is not an intrinsic property of a node. 

David


Comment 5 Michael Rys 2006-11-01 19:13:04 UTC
Re score(). 

Dear David. Let me clarify my statement below. Note that I said position and not the result of position(). the position of a node inside a document is a fixed property of the node. You refer to the context item position. That property is determined by the query, but only step expressions and XPath predicates set the context item and as such can have an impact on the context item properties. Score gets calculated in expressions that are not setting the context item and extending the language to incorporate this seems to be more costly than the benefits it can provide, given that there is a simpler syntactic extension of XPath2.0.

Michael (speaking for himself)
Comment 6 David Carlisle 2006-11-02 17:05:25 UTC
Michael,
Thanks for the clarification. I pushed back on the word "position" as it was only originally introduced with respect to comparing a possible score() function with the existing position() function, but anyway I think we all understand each other now, so I'll let that drop.

Since I re-opened the report I suppose I should state how I think it could be resolved.

Possibly unwisely my original report raised two interconnected issues.

1) asking for an editorial clarification on the description of the differences between the two variants of let score clause (with or without the main let variable)

The WG resolved that by removing one of the forms, which wasn't the resolution I expected, but that's fine by me.

2) Asking for a score syntax (possibly but not necessarily a score function) usable in XPath (including the XPath subset of XQuery, but especially usable in non-xquery contexts).

My (currently) preferred solution to this second point would have been to add a score() function, but as I indicated in the original comment, I didn't expect that the WGs would really want to make such a large change at this point, so a fallback position was to add the let score clause to XPath (as extended by full text) once the definition of let score had been clarified as requested in the first part of the comment.

My understanding of the WG position is that they have resolved point 1, but decided to take no action on point 2. As such I think that FIXED is an incorrect state for the report and it should be WONTFIX or (preferably) LATER.

David

Comment 7 Jim Melton 2006-11-20 19:34:09 UTC
The Full-Text Task Force decided to mark this as WONTFIX.
Comment 8 David Carlisle 2006-11-20 21:48:19 UTC
Jim,

> The Full-Text Task Force decided to mark this as WONTFIX.

As you may guess, this isn't the final outcome that I'd most wish to see, however
it's not an unexpected outcome, and I think it leaves the report in a state that more correctly reflects the situation than the originally proposed state of FIXED
so no more complaints here, I'm marking this as closed,

David