This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.
In the tokenizer, if there's a current tag token, but an EOF occurs, is the current tag token abandoned (never emitted)? That's how it looks, but it might be worth clarifying that that's the intent.
Where would you like to see the clarification? (That is, where did you look and not find one?)
Well, you could insert "Abandon the current token." wherever appropriate, but that'd be a lot of places, and more trouble than it's worth. Instead, you could add one paragraph (maybe just a note) to cover all cases. E.g., in the "Tokenization" section, between the paragraphs beginning "The output of the tokenization step" and "When a token is emitted", you could insert something like: Creating a token is distinct from emitting it. If the instructions say to create a token, but not to emit it, then it becomes the tokenizer's "current token", allowing later states to fill in its details. Usually, some state will eventually emit the token, but it may also happen (e.g., on EOF) that it is simply abandoned. Note that this might also allow you to delete occurrences of: (Don't emit the token yet; further details will be filled in before it is emitted.)
I added: <p class="note">Creating a token and emitting it are distinct actions. It is possible for a token to be implicitly abandoned, e.g. if the file ends unexpectedly while parsing the characters that are being parsed into a start tag token.</p> Is that enough? I didn't want to add the "current token" definition, lest it add more confusion than clarity (plus I'd have to hyperlink all the uses of the term, to be consistent with the rest of the spec...).
Checked in as WHATWG revision r8167. Check-in comment: Clarify that creating and emitting a token isn't the same thing. http://html5.org/tools/web-apps-tracker?from=8166&to=8167
(In reply to Ian 'Hixie' Hickson from comment #3) > > Is that enough? It's enough to address the original question, thanks. However, the placement of the new note (between two paragraphs about parse errors for end tag tokens) seems like a mistake. Immediately before or after the paragraph beginning "When a token is emitted" would make more sense to me.
Checked in as WHATWG revision r8175. Check-in comment: Rejigger the create vs emit token note http://html5.org/tools/web-apps-tracker?from=8174&to=8175
Roger. Moved it around.
And improved the wording somewhat. Thanks.