This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 9989 - Is the number of replacement characters supposed to be well-defined? If not this should be explicitly noted. If it is then more detail is required.
Summary: Is the number of replacement characters supposed to be well-defined? If not t...
Status: RESOLVED FIXED
Alias: None
Product: WebAppsWG
Classification: Unclassified
Component: WebSocket API (editor: Ian Hickson) (show other bugs)
Version: unspecified
Hardware: Other other
: P3 normal
Target Milestone: ---
Assignee: Ian 'Hixie' Hickson
QA Contact: public-webapps-bugzilla
URL: http://www.whatwg.org/specs/web-apps/...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-06-23 09:51 UTC by contributor
Modified: 2010-09-30 08:28 UTC (History)
4 users (show)

See Also:


Attachments

Description contributor 2010-06-23 09:51:30 UTC
Section: http://www.whatwg.org/specs/web-apps/current-work/complete.html#handling-errors-in-utf-8-from-the-server

Comment:
Is the number of replacement characters supposed to be well-defined? If not
this should be explicitly noted. If it is then more detail is required.

Posted from: 88.131.66.80
Comment 1 Ian 'Hixie' Hickson 2010-07-22 05:27:48 UTC
I don't understand what isn't well-defined.
Comment 2 Simon Pieters 2010-07-22 13:25:19 UTC
The spec says to replace bytes *or* sequences of bytes that are not valid utf-8 with U+FFFD. It is thus not well-defined how many U+FFFD are expected for any given sequence of bytes that are not valid utf-8. It could be one or the same amount of bytes that are not valid, or anything in between.

(The same applies to text/html parsing.)
Comment 3 Anne 2010-07-26 06:29:11 UTC
This really ought to be fixed in the UTF-8 specification or in some encoding layer specification as a whole bunch of specifications are affected by this.
Comment 4 Anne 2010-07-26 06:29:37 UTC
Scrap that bit about UTF-8, misread something.
Comment 5 Ian 'Hixie' Hickson 2010-08-13 07:27:56 UTC
What would you consider an acceptable replacement for the current text? I intentionally use the same prose throughout the Web Apps 1.0 spec, because I thought it was what we'd agreed was clear, but I'm happy to change it to something else if you have a concrete proposal for what to change it to.
Comment 6 Ian 'Hixie' Hickson 2010-09-24 14:56:48 UTC
EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Did Not Understand Request
Change Description: no spec change
Rationale: see comment 5
Comment 7 Simon Pieters 2010-09-27 11:22:36 UTC
It seems the bugzilla monster ate my comment. Trying again:


First try, probably isn't quite right:
 
Numbers are bytes in hex. "Anything but ..." includes EOF.
 
Stray 80-BF:
FE-FF:
replace with one U+FFFD.
 
C0-C1 followed by 80-BF:
replace the 2-byte sequence with one U+FFFD.
 
C0-FD followed by anything but 80-BF:
replace the first byte with one U+FFFD and reprocess the second byte.
 
E0-FD followed by 80-BF followed by anything but 80-BF:
replace the first two bytes with one U+FFFD and reprocess the third byte.
 
F0-FD followed by two 80-BF followed by anything but 80-BF:
replace the first three bytes with one U+FFFD and reprocess the forth byte.
 
F0-F4 followed by three 80-BF that represent a code point above U+10FFFF:
replace all four bytes with one U+FFFD.
 
F5-FD followed by three 80-BF followed by anything but 80-BF:
replace the first four bytes with one U+FFFD and reprocess the fifth byte.
 
FC-FD followed by four 80-BF followed by anything but 80-BF:
replace the first five bytes with one U+FFFD and reprocess the sixth byte.
 
Overlong forms (e.g. F0 80 80 A0):
replace the whole byte sequence with one U+FFFD.
Comment 8 Ian 'Hixie' Hickson 2010-09-28 07:29:37 UTC
Any volunteers for a Web UTF-8 spec?

I guess I'll put this in the HTML spec's infrastructure section and then refer to it from all the other specs of relevance.
Comment 9 Ian 'Hixie' Hickson 2010-09-30 08:28:03 UTC
something weird is happening with this bug
Comment 10 Ian 'Hixie' Hickson 2010-09-30 08:28:25 UTC
anyway this bug if fixed now