This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 26614 - change "violation of Unicode" notes to something less scary (per Unicode)
Summary: change "violation of Unicode" notes to something less scary (per Unicode)
Status: RESOLVED FIXED
Alias: None
Product: WHATWG
Classification: Unclassified
Component: Encoding (show other bugs)
Version: unspecified
Hardware: PC Windows NT
: P2 normal
Target Milestone: Unsorted
Assignee: Anne
QA Contact: sideshowbarker+encodingspec
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-08-20 15:36 UTC by Addison Phillips
Modified: 2014-09-01 17:15 UTC (History)
3 users (show)

See Also:


Attachments

Description Addison Phillips 2014-08-20 15:36:32 UTC
Ken Whistler advised us:
==
Re the note in Section 4.2, I do not understand at all why you word this
as “In violation of section 1.4 of UTS #22…” How is this a “violation” of
anything? The wording in UTS #22 is:

“… For best results, names should be compared after applying the
following transformations: …”

That is simply a recommendation for how to minimize non-recognition
of variations in spelling of charset names in labels. It doesn’t really have
anything to do with the actual conformance clause of UTS #22. So I
don’t see how anybody could actually be in “violation” of it.

The W3C “Encodings” document just makes a much more detailed
and prescriptive mapping of charset labels to the specified encodings
it enumerates. Why don’t you just say *that*, instead:

=============================================================

Note: This specification provides a more detailed and prescriptive
mapping of charset labels to encodings than the loose matching
for charset aliases recommended by UTS #22 … etc., etc.

=============================================================

See? No violation anywhere.

I have a similar reaction to your notes in 14.2 and 14.4. I also do not see
those as “violations” of the Unicode Standard (which, by the way, I would
spell with a capitalized “Standard”).

Start with 14.4 utf-16le. The Unicode Standard does not specify “labels” for
charsets, so I don’t see how you’d be in violation of the standard by
defining how you interpret charset labels. Essentially, you are saying:

=========================================================

Note: For [insert reason here] the label “utf-16” is treated as synonymous
with the label “utf-16le”, and also identifies the utf-16le encoding.

===========================================================

And for your note in 14.2, I think the statement is just wrong. This is
not a violation of the Unicode Standard. It is very much in the spirit
of the definition of the UTF-16 encoding scheme to treat the BOM
as signature and use it to identity the actual byte order of a stream.
And if that is used to override an explicit (but erroneous) charset
labeling, so be it. See Asmus’ comment, which just crossed mine.

In any case, I would advise rewording all three of these notes in
your document. Rather than having a rhetorical stance that
says, “We violate the Unicode Standard, but that’s o.k., because
this item is uncontroversial, and …”, why would you need to state
any violations here at all? Just put in clarifying notes to forestall
people from *claiming* that these practices violate the Unicode
Standard (or UTS #22).
Comment 1 Anne 2014-08-20 15:41:12 UTC
Interesting. Are these comments archived somewhere? As well as Asmus' comment referred to above?
Comment 3 Anne 2014-08-21 10:41:27 UTC
So for this change I should acknowledge Ken Whistler?
Comment 4 i18n IG 2014-08-21 11:15:06 UTC
yes
Comment 6 Addison Phillips 2014-08-25 16:15:41 UTC
For reference, here is Asmus's note, which he gave permission to include here:

--
The first note refers to a SHOULD specification in UTS#22. It would be overstating to call it a "violation" to deviate from it.

If I understand the issue correctly is that you need the BOM to be able to override conflicting external designations.
Hence, an encoding is only "known" to be correctly labeled when it doesn't contradict an internal BOM. Otherwise, you implicitly treat the declared encoding as erroneous.

Seems a fine approach by me, given the realities.

Those are my 2 cents.

A./
--
Comment 7 Richard Ishida 2014-08-29 15:23:14 UTC
Martin Duerst provided some additional feedback on wording:

--
> [Section 14.2]
> -- 
> Checking for and using a byte order mark happens before an encoding to decode a byte stream is chosen, as seen in the decode algorithm, as is deemed more accurate than any label.
> -- 

I really had problems parsing this sentence. One problem is that two clauses start with "as". I suggest changing the connective for the second clause to "because", and maybe moving that clause to the start of the sentence. Other improvements might work too.

Also, unless this is clear from e.g. a link that's missing in the text version in this email, it would also be useful to be specific about whether the "decode algorithm" is something in the Unicode spec or in the encoding spec.

In addition, "before an encoding ... is chosen" looks problematic to me because 1) "checking and using a BOM" also actually may choose an encoding, and 2) because the "to encode a byte stream" makes the structure difficult to parse (my first (and second and third) parse was "checking ... happens before an encoding").

I suggest something along the lines of:
"A byte order mark has priority over an encoding label..."
--

And Asmus Freytag also said: 

--
I had difficulties as well with that sentence, but couldn't put my finger on it; thanks for pointing out the reason. There's nothing objectionable to the content that it intends to express, but it would be improved if reworded along the lines you suggest.
--
Comment 8 Anne 2014-08-31 12:15:20 UTC
Reopening per comment 7.