This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 13396 - i18n-ISSUE-77: HTTP and defaulting to UTF-16LE
Summary: i18n-ISSUE-77: HTTP and defaulting to UTF-16LE
Status: RESOLVED FIXED
Alias: None
Product: HTML WG
Classification: Unclassified
Component: LC1 HTML5 spec (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal
Target Milestone: ---
Assignee: Ian 'Hixie' Hickson
QA Contact: HTML WG Bugzilla archive list
URL:
Whiteboard:
Keywords: a11y, a11ytf
Depends on:
Blocks:
 
Reported: 2011-07-27 18:20 UTC by I18n Core WG
Modified: 2011-11-18 15:58 UTC (History)
7 users (show)

See Also:


Attachments

Description I18n Core WG 2011-07-27 18:20:45 UTC
8.2.2.2 Character encodings
http://www.w3.org/TR/html5/parsing.html#character-encodings-0

Supported by the i18n WG.

"When a user agent is to use the UTF-16 encoding but no BOM has been found, user agents must default to UTF-16LE."

If the HTTP header declares the file to be UTF-16BE, which I believe it can, and in which case a BOM should *not* be used, then I think that this would not be true. If the HTTP header declares the file to be UTF-16, then there must be a BOM, so I assume that this is a recovery mechanism if someone does declare UTF-16 in HTTP but omits the BOM. I'd think that some kind of clarification and perhaps error message would be in order though.
Comment 1 Henri Sivonen 2011-07-28 08:03:42 UTC
(In reply to comment #0)
> 8.2.2.2 Character encodings
> http://www.w3.org/TR/html5/parsing.html#character-encodings-0
> 
> Supported by the i18n WG.
> 
> "When a user agent is to use the UTF-16 encoding but no BOM has been found,
> user agents must default to UTF-16LE."
> 
> If the HTTP header declares the file to be UTF-16BE, which I believe it can,
> and in which case a BOM should *not* be used, then I think that this would not
> be true.

Then the user agent isn't to use the UTF-16 encoding but the UTF-16BE encoding. The quoted sentence shouldn't say "UTF-16LE". It should say "little-endian UTF-16", unless the spec intends the reported encoding for the document to change and I'm pretty sure that's not the intention.

> If the HTTP header declares the file to be UTF-16, then there must be
> a BOM, so I assume that this is a recovery mechanism if someone does declare
> UTF-16 in HTTP but omits the BOM.

Yes.
Comment 2 Martin Dürst 2011-07-29 02:02:56 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > 8.2.2.2 Character encodings
> > http://www.w3.org/TR/html5/parsing.html#character-encodings-0
> > 
> > Supported by the i18n WG.
> > 
> > "When a user agent is to use the UTF-16 encoding but no BOM has been found,
> > user agents must default to UTF-16LE."
> > 
> > If the HTTP header declares the file to be UTF-16BE, which I believe it can,
> > and in which case a BOM should *not* be used, then I think that this would not
> > be true.
> 
> Then the user agent isn't to use the UTF-16 encoding but the UTF-16BE encoding.
> The quoted sentence shouldn't say "UTF-16LE". It should say "little-endian
> UTF-16", unless the spec intends the reported encoding for the document to
> change and I'm pretty sure that's not the intention.

This would definitely make things clearer. I'd also suggest to change "is to use the UTF-16 encoding" at the start of the sentence to something that makes it clearer that this is stuff *labeled* with a label of "UTF-16" (using explicit quotes).

> > If the HTTP header declares the file to be UTF-16, then there must be
> > a BOM, so I assume that this is a recovery mechanism if someone does declare
> > UTF-16 in HTTP but omits the BOM.
> 
> Yes.

It may help to make this clear in the text.

Regards,   Martin.
Comment 3 Michael[tm] Smith 2011-08-04 05:34:59 UTC
mass-move component to LC1
Comment 4 Ian 'Hixie' Hickson 2011-08-17 22:27:57 UTC
EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Accepted
Change Description: see diff given below
Rationale: Concurred with reporter's comments.
Comment 5 contributor 2011-08-17 22:28:43 UTC
Checked in as WHATWG revision r6498.
Check-in comment: Clean up how we refer to UTF-16.
http://html5.org/tools/web-apps-tracker?from=6497&to=6498