This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 5414 - Remove consecutive from explanation of tokenization
Summary: Remove consecutive from explanation of tokenization
Status: CLOSED FIXED
Alias: None
Product: XPath / XQuery / XSLT
Classification: Unclassified
Component: Full Text 1.0 (show other bugs)
Version: Last Call drafts
Hardware: PC Windows XP
: P2 normal
Target Milestone: ---
Assignee: Pat Case
QA Contact: Mailing list for public feedback on specs from XSL and XML Query WGs
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-01-24 15:31 UTC by Pat Case
Modified: 2008-01-24 16:55 UTC (History)
0 users

See Also:


Attachments

Description Pat Case 2008-01-24 15:31:49 UTC
In 4.1 Tokenization bullet 1
Each token MUST consist of one or more consecutive characters.

Changing to:
Each token MUST consist of one or more characters.

Removing consecutive to allow implementations to return "vorstellen" in the following sentence "Er stellte sic vor."
Comment 1 Pat Case 2008-01-24 15:36:01 UTC
The correct sentence is "Er stellte sie vor"

Fixed as stated in the bug.
Comment 2 Pat Case 2008-01-24 15:36:45 UTC
Closed
Comment 3 Michael Kay 2008-01-24 16:11:42 UTC
There are many words that could appear as the third word of that sentence: mich, dich, sich, jemand, euch, uns, Ihnen. But not sic or sie. I assumed "sich" was intended.

Comment 4 Michael Kay 2008-01-24 16:55:03 UTC
I talk nonsense. Of course it can be "sie" - "He introduced her". I just had "Er stellte sich vor" - "He imagined" in my head.