26614 2014-08-20 15:36:32 +0000 change "violation of Unicode" notes to something less scary (per Unicode) 2014-09-01 17:15:22 +0000 1 1 1 Unclassified WHATWG Encoding unspecified PC Windows NT RESOLVED FIXED P2 normal Unsorted 1 addison annevk ishida mike www-international sideshowbarker+encodingspec oldest_to_newest 110356 0 addison 2014-08-20 15:36:32 +0000 Ken Whistler advised us: == Re the note in Section 4.2, I do not understand at all why you word this as “In violation of section 1.4 of UTS #22…” How is this a “violation” of anything? The wording in UTS #22 is: “… For best results, names should be compared after applying the following transformations: …” That is simply a recommendation for how to minimize non-recognition of variations in spelling of charset names in labels. It doesn’t really have anything to do with the actual conformance clause of UTS #22. So I don’t see how anybody could actually be in “violation” of it. The W3C “Encodings” document just makes a much more detailed and prescriptive mapping of charset labels to the specified encodings it enumerates. Why don’t you just say *that*, instead: ============================================================= Note: This specification provides a more detailed and prescriptive mapping of charset labels to encodings than the loose matching for charset aliases recommended by UTS #22 … etc., etc. ============================================================= See? No violation anywhere. I have a similar reaction to your notes in 14.2 and 14.4. I also do not see those as “violations” of the Unicode Standard (which, by the way, I would spell with a capitalized “Standard”). Start with 14.4 utf-16le. The Unicode Standard does not specify “labels” for charsets, so I don’t see how you’d be in violation of the standard by defining how you interpret charset labels. Essentially, you are saying: ========================================================= Note: For [insert reason here] the label “utf-16” is treated as synonymous with the label “utf-16le”, and also identifies the utf-16le encoding. =========================================================== And for your note in 14.2, I think the statement is just wrong. This is not a violation of the Unicode Standard. It is very much in the spirit of the definition of the UTF-16 encoding scheme to treat the BOM as signature and use it to identity the actual byte order of a stream. And if that is used to override an explicit (but erroneous) charset labeling, so be it. See Asmus’ comment, which just crossed mine. In any case, I would advise rewording all three of these notes in your document. Rather than having a rhetorical stance that says, “We violate the Unicode Standard, but that’s o.k., because this item is uncontroversial, and …”, why would you need to state any violations here at all? Just put in clarifying notes to forestall people from *claiming* that these practices violate the Unicode Standard (or UTS #22). 110357 1 annevk 2014-08-20 15:41:12 +0000 Interesting. Are these comments archived somewhere? As well as Asmus' comment referred to above? 110358 2 www-international 2014-08-20 15:43:43 +0000 http://lists.w3.org/Archives/Public/public-i18n-core/2014JulSep/0060.html 110395 3 annevk 2014-08-21 10:41:27 +0000 So for this change I should acknowledge Ken Whistler? 110396 4 www-international 2014-08-21 11:15:06 +0000 yes 110481 5 annevk 2014-08-22 16:03:01 +0000 https://github.com/whatwg/encoding/commit/a0429a6b2b043d9b7e130554529d66636c73133f 110543 6 addison 2014-08-25 16:15:41 +0000 For reference, here is Asmus's note, which he gave permission to include here: -- The first note refers to a SHOULD specification in UTS#22. It would be overstating to call it a "violation" to deviate from it. If I understand the issue correctly is that you need the BOM to be able to override conflicting external designations. Hence, an encoding is only "known" to be correctly labeled when it doesn't contradict an internal BOM. Otherwise, you implicitly treat the declared encoding as erroneous. Seems a fine approach by me, given the realities. Those are my 2 cents. A./ -- 110771 7 ishida 2014-08-29 15:23:14 +0000 Martin Duerst provided some additional feedback on wording: -- > [Section 14.2] > -- > Checking for and using a byte order mark happens before an encoding to decode a byte stream is chosen, as seen in the decode algorithm, as is deemed more accurate than any label. > -- I really had problems parsing this sentence. One problem is that two clauses start with "as". I suggest changing the connective for the second clause to "because", and maybe moving that clause to the start of the sentence. Other improvements might work too. Also, unless this is clear from e.g. a link that's missing in the text version in this email, it would also be useful to be specific about whether the "decode algorithm" is something in the Unicode spec or in the encoding spec. In addition, "before an encoding ... is chosen" looks problematic to me because 1) "checking and using a BOM" also actually may choose an encoding, and 2) because the "to encode a byte stream" makes the structure difficult to parse (my first (and second and third) parse was "checking ... happens before an encoding"). I suggest something along the lines of: "A byte order mark has priority over an encoding label..." -- And Asmus Freytag also said: -- I had difficulties as well with that sentence, but couldn't put my finger on it; thanks for pointing out the reason. There's nothing objectionable to the content that it intends to express, but it would be improved if reworded along the lines you suggest. -- 110799 8 annevk 2014-08-31 12:15:20 +0000 Reopening per comment 7. 110823 9 annevk 2014-09-01 17:15:22 +0000 https://github.com/whatwg/encoding/commit/31b5075d63115f2f07ee970224653611a7dbcea5