9989 2010-06-23 09:51:30 +0000 Is the number of replacement characters supposed to be well-defined? If not this should be explicitly noted. If it is then more detail is required. 2010-09-30 08:28:25 +0000 1 1 1 Unclassified WebAppsWG WebSocket API (editor: Ian Hickson) unspecified Other other RESOLVED FIXED http://www.whatwg.org/specs/web-apps/current-work/#handling-errors-in-utf-8-from-the-server P3 normal --- 1 contributor ian annevk ian mike public-webapps public-webapps-bugzilla oldest_to_newest 36368 0 contributor 2010-06-23 09:51:30 +0000 Section: http://www.whatwg.org/specs/web-apps/current-work/complete.html#handling-errors-in-utf-8-from-the-server Comment: Is the number of replacement characters supposed to be well-defined? If not this should be explicitly noted. If it is then more detail is required. Posted from: 88.131.66.80 37058 1 ian 2010-07-22 05:27:48 +0000 I don't understand what isn't well-defined. 37064 2 zcorpan 2010-07-22 13:25:19 +0000 The spec says to replace bytes *or* sequences of bytes that are not valid utf-8 with U+FFFD. It is thus not well-defined how many U+FFFD are expected for any given sequence of bytes that are not valid utf-8. It could be one or the same amount of bytes that are not valid, or anything in between. (The same applies to text/html parsing.) 37094 3 annevk 2010-07-26 06:29:11 +0000 This really ought to be fixed in the UTF-8 specification or in some encoding layer specification as a whole bunch of specifications are affected by this. 37095 4 annevk 2010-07-26 06:29:37 +0000 Scrap that bit about UTF-8, misread something. 37389 5 ian 2010-08-13 07:27:56 +0000 What would you consider an acceptable replacement for the current text? I intentionally use the same prose throughout the Web Apps 1.0 spec, because I thought it was what we'd agreed was clear, but I'm happy to change it to something else if you have a concrete proposal for what to change it to. 39291 6 ian 2010-09-24 14:56:48 +0000 EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document: http://dev.w3.org/html5/decision-policy/decision-policy.html Status: Did Not Understand Request Change Description: no spec change Rationale: see comment 5 39565 7 zcorpan 2010-09-27 11:22:36 +0000 It seems the bugzilla monster ate my comment. Trying again: First try, probably isn't quite right: Numbers are bytes in hex. "Anything but ..." includes EOF. Stray 80-BF: FE-FF: replace with one U+FFFD. C0-C1 followed by 80-BF: replace the 2-byte sequence with one U+FFFD. C0-FD followed by anything but 80-BF: replace the first byte with one U+FFFD and reprocess the second byte. E0-FD followed by 80-BF followed by anything but 80-BF: replace the first two bytes with one U+FFFD and reprocess the third byte. F0-FD followed by two 80-BF followed by anything but 80-BF: replace the first three bytes with one U+FFFD and reprocess the forth byte. F0-F4 followed by three 80-BF that represent a code point above U+10FFFF: replace all four bytes with one U+FFFD. F5-FD followed by three 80-BF followed by anything but 80-BF: replace the first four bytes with one U+FFFD and reprocess the fifth byte. FC-FD followed by four 80-BF followed by anything but 80-BF: replace the first five bytes with one U+FFFD and reprocess the sixth byte. Overlong forms (e.g. F0 80 80 A0): replace the whole byte sequence with one U+FFFD. 39698 8 ian 2010-09-28 07:29:37 +0000 Any volunteers for a Web UTF-8 spec? I guess I'll put this in the HTML spec's infrastructure section and then refer to it from all the other specs of relevance. 40087 9 ian 2010-09-30 08:28:03 +0000 something weird is happening with this bug 40088 10 ian 2010-09-30 08:28:25 +0000 anyway this bug if fixed now