2892 2006-02-18 13:01:41 +0000 QName listed twice in state ITEMTYPE 2006-04-04 16:22:55 +0000 1 1 1 Unclassified XPath / XQuery / XSLT XQuery/XPath Tokenizer Candidate Recommendation Other Linux CLOSED WONTFIX P2 normal --- 1 frans.englich scott_boag public-qt-comments oldest_to_newest 8317 0 frans.englich 2006-02-18 13:01:42 +0000 Hi, I'm definitely no scanner expert so I can very well be wrong. Nevertheless: In the table for the ITEMTYPE state, QName is listed twice. Once in the second row after void(), and once on the sixth row next to item(). In the former case the action is a transition to OPERATOR while the latter is OCCURRENCEINDICATOR. I think the first occurrence of the QName should be removed, since the latter is sufficient and it would also avoid ambiguity. Perhaps the first QName was at somepoint added to handle when no occurrence indicator is specified(cardinality exactly-one), but the OCCURRENCEINDICATOR state handles that, from what I can tell. (Will file separately on void()/empty-sequence()) Regards, Frans 8952 1 scott_boag 2006-03-29 22:06:14 +0000 The "Building a Tokenizer for XPath or XQuery" is obsolete. If you do want to know what states we use to build our current test parser, the states are specified in the XML definition of the joint grammar, http://www.w3.org/XML/Group/xsl-query-specs/grammar/parser/applets/xpath-grammar.xml. Note that the current test parser uses selective LL(2) or LL(3) lookahead, and the number of tokenizer states has been greatly decreased.