11234 2010-11-05 11:30:31 +0000 Invalidate documents whose text content contains improperly balanced bidi formatting characters 2011-09-04 17:38:48 +0000 1 1 1 Unclassified HTML WG LC1 HTML5 spec unspecified All All RESOLVED FIXED P3 normal --- 1 aharon.lists.lanin ian addison aharon.lists.lanin ayg fantasai.bugs ian mike public-html-admin public-html-wg-issue-tracking public-i18n-bidi shachar public-html-bugzilla oldest_to_newest 42177 0 aharon.lists.lanin 2010-11-05 11:30:31 +0000 As has surfaced in the discussion of bug 10809, it would be helpful to declare invalid documents where any element's text node children (*not* descendants generally) contain improperly balanced LRE, RLE, LRO, RLO, or PDF characters. In other words, for the purposes of validation, treat every LRE, RLE, LRO, or RLO character as the opening tag of an imaginary element, something like <bidi-formatting>, and PDF as that imaginary element's closing tag. This applies to these character's entities, as well, of course. Examples of invalid usage: 1. <div>‪</div> 2. <div>‬</div> 3. <div>‬‪</div> 4. <div>‪‪‬</div> 5. <div>‪<br>‪‬</div> 6. <div>‪<span>‬</span></div> 7. <div><span>‪</span>‬</div> An example of valid (but not recommended!) usage: <div>‪<span>...</span>‬</div> 42233 1 ian 2010-11-08 08:01:28 +0000 This shouldn't be too hard to add to the spec. 42275 2 zcorpan 2010-11-08 15:17:36 +0000 What about attribute values? 42280 3 aharon.lists.lanin 2010-11-08 17:11:24 +0000 (In reply to comment #2) > What about attribute values? Not sure what you mean. 42286 4 zcorpan 2010-11-08 21:07:45 +0000 I mean, should the following be invalid? <p title="‪"> 42303 5 ian 2010-11-09 02:17:12 +0000 Yes. 42305 6 aharon.lists.lanin 2010-11-09 07:17:12 +0000 (In reply to comment #5) > Yes. Yeah, it makes sense. They should be balanced within an attribute value. 43981 7 ian 2011-01-10 09:31:46 +0000 EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document: http://dev.w3.org/html5/decision-policy/decision-policy.html Status: Accepted Change Description: see diff given below Rationale: Concurred with reporter's comments. 43982 8 contributor 2011-01-10 09:55:28 +0000 Checked in as WHATWG revision r5754. Check-in comment: Define conformance criteria around bidi formatting characters http://html5.org/tools/web-apps-tracker?from=5753&to=5754 53920 9 mike 2011-08-04 05:34:29 +0000 mass-move component to LC1 54378 10 aharon.lists.lanin 2011-08-08 13:42:59 +0000 The checked-in change seems to say that the use of the formatting characters (when restricted as specified) is perfectly fine: "[Text content] may contain characters in the range U+202A to U+202E (the bidirectional-algorithm formatting characters)." "Note: *For convenience*, where possible authors will likely prefer to use the dir attribute, the bdo element, and the bdi element, rather than maintaining the bidirectional-algorithm formatting characters manually." (emphasis mine) The use of the formatting characters, even when they obey the given rules, should still be discouraged. It is *not* equivalent to the use of the dir attribute and the bdo element, for two reasons. (BTW, the bdi element should not be mentioned at all. There is no way to faithfully emulate its behavior using the formatting characters.) 1. The dir attribute sets the element's directionality. The formatting characters don't. That means that they do no affect the proposed CSS4 :dir(ltr|rtl} pseudo-class. 2. When used around an element that introduces bidi paragraph break, e.g. "LRE <br> PDF" or "LRE <div></div> PDF", the formatting characters go completely haywire, since the paragraph break resets the bidirectional state, so that the effect of the opening character is lost after the paragraph break, and the closing formatting character is unmatched. The effects of the dir attribute, on the other hand, are carefully defined in CSS (via its effect on unicode-bidi) to be reopened after the paragraph break. Neither of these can be fixed. Thus, the use of the formatting characters, even when they obey the given rules, should be discouraged wherever mark-up can be used instead. The bug as opened suggested ruling certain uses of formatting characters completely invalid. It did not suggest pronouncing the remaining use perfectly fine. Certainly the use of the dir attribute etc. is more than a matter of convenience. It is *the only recommended way* of declaring text direction in HTML (except for those places where mark-up can not be used, e.g. inside <option> and <title>). The use of both CSS and formatting characters for this purpose is discouraged (for different reasons). 54379 11 shachar 2011-08-08 13:48:46 +0000 (In reply to comment #10) > Certainly the use of the dir attribute etc. is more than a matter of > convenience. It is *the only recommended way* of declaring text direction in > HTML While I do not disagree with you on this point (which is to say, I agree), I think we should not go as far as recommending against ("should not"). The BiDi control characters can come in handy when different sources product the HTML entities and the content, and are sometimes the only practical option available. Shachar 54390 12 fantasai.bugs 2011-08-08 19:34:35 +0000 (In reply to comment #10) > (BTW, the bdi element should > not be mentioned at all. There is no way to faithfully emulate its behavior > using the formatting characters.) I disagree on this point; you can't faithfully emulate <bdi> with formatting characters as it's not equivalent to any one of them, but some of the problems that can are solved with formatting characters (like &rlm;) are better solved with <bdi>, so this cross-reference should be given. 54451 13 aharon.lists.lanin 2011-08-09 11:38:56 +0000 (In reply to comment #12) > (In reply to comment #10) > > (BTW, the bdi element should > > not be mentioned at all. There is no way to faithfully emulate its behavior > > using the formatting characters.) > > I disagree on this point; you can't faithfully emulate <bdi> with formatting > characters as it's not equivalent to any one of them, but some of the problems > that can are solved with formatting characters (like &rlm;) are better solved > with <bdi>, so this cross-reference should be given. Currently, the sentence says that the mark-up is just a convenience that translates to formatting characters, which is not really true for dir= and <bdo>, and completely untrue for <bdi>. If the sentence is changed to encourage people to use dir=, <bdo>, and <bdi> instead of formatting characters, then I fully agree with fantasai. 54681 14 aharon.lists.lanin 2011-08-11 07:40:21 +0000 (In reply to comment #11) > (In reply to comment #10) > > Certainly the use of the dir attribute etc. is more than a matter of > > convenience. It is *the only recommended way* of declaring text direction in > > HTML > > While I do not disagree with you on this point (which is to say, I agree), I > think we should not go as far as recommending against ("should not"). The BiDi > control characters can come in handy when different sources product the HTML > entities and the content, and are sometimes the only practical option > available. > > Shachar The spec could recommend using directional mark-up instead of directional formatting characters whenever feasible. It could also have a note warning that placing an element between an LRE, RLE, LRO, or RLO and its matching PDF does not work well with various HTML and CSS features, and has effects that vary radically depending on the element's style. 55327 15 ian 2011-08-17 19:23:52 +0000 EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document: http://dev.w3.org/html5/decision-policy/decision-policy.html Status: Accepted Change Description: see diff given below Rationale: Concurred with reporter's comments. Specifically, I changed the spec to encourage authors to use the elements instead, and made the conformance rules not allow "LRE <div></div> PDF". 55328 16 contributor 2011-08-17 19:24:27 +0000 Checked in as WHATWG revision r6487. Check-in comment: More useful conformance rules and advice for bidi formatting characters http://html5.org/tools/web-apps-tracker?from=6486&to=6487 55547 17 aharon.lists.lanin 2011-08-21 15:39:59 +0000 The change looks great, with two small flaws: 1. The treatment given to an element that is flow content but is not also phrasing content should be extended to <br>, which also serves as a bidi paragraph break, and thus (by design) terminates the effects of the bidi formatting characters. 2. The comment that the formatting characters interact poorly with CSS is too narrow - they also interact poorly with some HTML features (even when used as currently spec'ed). An example: <div dir=rtl>‪If this works I will eat my <input />.‬</div> The <input> will have RTL directionality despite being between an LRE and its matching PDF. I am not suggesting adding this example or changing the validity spec - just expanding the note to include some unspecified HTML features (as opposed to just CSS). 55661 18 ian 2011-08-23 05:25:01 +0000 I'll add something about <br>. > <div dir=rtl>‪If this works I will eat my <input />.‬</div> That's not a poor interaction IMHO. 56080 19 addison 2011-08-31 05:18:11 +0000 BTW> The I18N WG supported re-opening this bug and Aharon's comments generally (I18N-ACTION-66). In looking at the changes, I note that there may be a very minor typo where is says: -- The strings resulting from the applying the following algorithm... -- It should say "The string", since "output" is a single string? 56291 20 ian 2011-09-04 17:35:32 +0000 (In reply to comment #19) > It should say "The string", since "output" is a single string? "output" is a list of strings. EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document: http://dev.w3.org/html5/decision-policy/decision-policy.html Status: Partially Accepted Change Description: see diff given below Rationale: Addressed the <br> issue. 56292 21 contributor 2011-09-04 17:38:48 +0000 Checked in as WHATWG revision r6533. Check-in comment: Make sure <br> is handled right in the requirements regarding bidi formatting characters. http://html5.org/tools/web-apps-tracker?from=6532&to=6533