<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://www.w3.org/Bugs/Public/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.4"
          urlbase="https://www.w3.org/Bugs/Public/"
          
          maintainer="sysbot+bugzilla@w3.org"
>

    <bug>
          <bug_id>19484</bug_id>
          
          <creation_ts>2012-10-11 21:12:07 +0000</creation_ts>
          <short_desc>Inconsistency in regards to EOF (and EOF token)</short_desc>
          <delta_ts>2012-10-11 21:12:07 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>1</classification_id>
          <classification>Unclassified</classification>
          <product>CSS</product>
          <component>Syntax</component>
          <version>unspecified</version>
          <rep_platform>PC</rep_platform>
          <op_sys>Linux</op_sys>
          <bug_status>NEW</bug_status>
          <resolution></resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>P2</priority>
          <bug_severity>normal</bug_severity>
          <target_milestone>---</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Brandon Frohs">bfrohs</reporter>
          <assigned_to name="Tab Atkins Jr.">jackalmage</assigned_to>
          
          
          <qa_contact>public-css-bugzilla</qa_contact>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>76034</commentid>
    <comment_count>0</comment_count>
    <who name="Brandon Frohs">bfrohs</who>
    <bug_when>2012-10-11 21:12:07 +0000</bug_when>
    <thetext>In the &quot;Data state&quot;, an &quot;end-of-file token&quot; is emitted:

&gt; EOF
&gt;     Emit an end-of-file token. 


In the Tree Construction Definitions, an &quot;EOF token&quot; is mentioned:

&gt; next input token
&gt;     The token following the current input token in the list of tokens produced by the tokenizer. If there isn&apos;t a token following the current input token, the next input token is an *EOF token*.


In the &quot;Top-level mode&quot;, &quot;At-rule-prelude mode&quot;, &quot;Rule-block mode&quot;, and several other modes, an &quot;EOF token&quot; is expected. E.g.:

&gt; EOF token
&gt;     Finish parsing. 


However, neither &quot;EOF token&quot; or &quot;end-of-file token&quot; is mentioned in the list of possible tokens in the Tokenization overview:

&gt; The output of the tokenization step is a series of zero or more of the following tokens: identifier, function, at-keyword, hash, string, bad-string, url, bad-url, delim, number, percentage, dimension, unicode-range, whitespace, comment, cdo, cdc, colon, semicolon, [, ], (, ), {, }.</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>