9351 2010-03-27 22:41:00 +0000 Do not interpret & followed by an entity name followed by = as an entity reference in attribute values (maybe in text content too) 2010-10-04 13:55:41 +0000 1 1 1 Unclassified HTML WG pre-LC1 HTML5 spec (editor: Ian Hickson) unspecified PC All CLOSED DUPLICATE 9207 P2 normal --- 1 mjs ian ayg hsivonen ian mike public-html-admin public-html-wg-issue-tracking rubys public-html-bugzilla oldest_to_newest 33971 0 mjs 2010-03-27 22:41:00 +0000 It's been suggested that an unterminated entity (one not followed by a semicolon) that is followed by an equal sign in an attribute value should not be treated as an entity reference. It seems that rather few pages overall would be affected by changing this, one study found 50 occurrences out of approximately 425k pages: http://lists.w3.org/Archives/Public/public-html/2009Jun/0463.html It was also reported that most of these occurrences appeared to be cases where the author did not expect their text to be interpreted as an entity reference, and review of these 50 instances seems to confirm that impression. It seems like there is at least some content that would be broken by changing the interpretation: http://lists.w3.org/Archives/Public/public-html/2009Jul/0421.html Specifically, it seems that &amp= may occasionally be intended as "&" rather than as "&amp=" or "&=". On the whole, it still seems like the proposed change would still fix more content than it breaks. One possible variable is to exclude &amp= from this change. However, it seems that if authors write that when they mean "&", then their content will not work as intended even under either the existing parsing rule or the proposed new one. However, content that writes "&amp=" when it means "&amp=" would be fixed. There were instances of both in the data set. In addition to the direct benefits of fixing content, this change would also make it safer to change parsing rules to allow unescaped & in attributes. 34071 1 hsivonen 2010-03-29 07:55:26 +0000 What's the rationale of restricting this change request to attribute values? URLs also get copied and pasted to element content occasionally and making them tokenize differently there would violate the principle of least surprise. 34072 2 mjs 2010-03-29 08:17:30 +0000 (In reply to comment #1) > What's the rationale of restricting this change request to attribute values? > URLs also get copied and pasted to element content occasionally and making them > tokenize differently there would violate the principle of least surprise. > After some consideration, I think this change should probably be applied, though I would also be limited by a version limited only to attribute values. Updating title accordingly. The reasons I originally asked only for attribute values (which I no longer think are valid): 1) I thought the study evaluating occurrence of this pattern was limited to attribute values, so there was no information about the risk of the greater change. But that's not right - it included cases in text content and in script as well. 2) I did not think there was any benefit in text content, but I failed to think about the case of a URL in text content, or copying between attributes in text content. It's likely that the benefit is lower in text content, but the pattern seems much less likely to occur in the first place. Title amended. 34073 3 mjs 2010-03-29 08:35:42 +0000 One interesting side note: processing of entities (aka "named character references" apparently) is just about the only thing that is already different between quoted attribute values and text content, as far as the tokenizer is concerned: http://dev.w3.org/html5/spec/Overview.html#attribute-value-double-quoted-state http://dev.w3.org/html5/spec/Overview.html#data-state Making = get treated as an extra "additional allowed character" would in fact have the desired effect suggested by this bug. That being said, I think it might still be a good idea to do the change everywhere. 34074 4 mjs 2010-03-29 08:47:35 +0000 Further correction: the "additional allowed character" is a pretty minor difference, because it applies only right after the ampersand. The actual key difference is: "If the character reference is being consumed as part of an attribute, and the last character matched is not a U+003B SEMICOLON character (;), and the next character is in the range U+0030 DIGIT ZERO (0) to U+0039 DIGIT NINE (9), U+0041 LATIN CAPITAL LETTER A to U+005A LATIN CAPITAL LETTER Z, or U+0061 LATIN SMALL LETTER A to U+007A LATIN SMALL LETTER Z, then, for historical reasons, all the characters that were matched after the U+0026 AMPERSAND character (&) must be unconsumed, and nothing is returned." Adding = to that list would effect the change proposed here, but only for attribute values, not for text. However, since processing of entities is already quite a bit different between attribute values and text content, I am no longer so sure that making this change for text content too is warranted. Updating title to bias it a bit the other way. 34441 5 ian 2010-04-02 21:44:14 +0000 see also bug 9207 34444 6 ian 2010-04-02 22:39:54 +0000 *** This bug has been marked as a duplicate of bug 9207 *** 34446 7 mjs 2010-04-02 22:40:27 +0000 Agree that this is a duplicate given the resolution of 9207