17151 2012-05-22 17:15:25 +0000 How should UTF-16BE "\xD8\x00" be decoded? This is an ill-formed UTF-16 code unit sequence, but it can be converted to a Unicode code point. Firefox/Opera currently convert it to U+FFFD, which seems like the preferred behaviour. 2012-07-18 18:40:46 +0000 1 1 1 Unclassified WHATWG HTML unspecified Other other RESOLVED DUPLICATE 16768 http://www.whatwg.org/specs/web-apps/current-work/#the-input-byte-stream P3 normal Unsorted 1 contributor ian annevk geoffers+w3cbugs mike contributor oldest_to_newest 68020 0 contributor 2012-05-22 17:15:25 +0000 Specification: http://www.whatwg.org/specs/web-apps/current-work/ Multipage: http://www.whatwg.org/C#the-input-byte-stream Complete: http://www.whatwg.org/c#the-input-byte-stream Comment: How should UTF-16BE "\xD8\x00" be decoded? This is an ill-formed UTF-16 code unit sequence, but it can be converted to a Unicode code point. Firefox/Opera currently convert it to U+FFFD, which seems like the preferred behaviour. Posted from: 81.110.227.73 by geoffers@gmail.com User agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1130.1 Safari/536.11 68021 1 geoffers+w3cbugs 2012-05-22 17:23:15 +0000 This pertains to the following: > Bytes or sequences of bytes in the original byte stream that could not be converted to Unicode code points must be converted to U+FFFD REPLACEMENT CHARACTERs. Specifically, if the encoding is UTF-8, the bytes must be decoded with the error handling defined in this specification. > Note: Bytes or sequences of bytes in the original byte stream that did not conform to the encoding specification (e.g. invalid UTF-8 byte sequences in a UTF-8 input byte stream) are errors that conformance checkers are expected to report. "\xD8\x00" can obviously be decoded as if it were UTF-16BE to HTML5's definition of a "Unicode code point" (which include lone surrogates), but according to the Unicode specification it is an invalid UTF-16 code unit sequence. It would seem preferable that lone surrogates get converted to U+FFFD as they currently are in Opera/Firefox. 68029 2 zcorpan 2012-05-23 06:40:30 +0000 http://dvcs.w3.org/hg/encoding/raw-file/tip/Overview.html#utf-16-decoder already handles lone surrogates, AFAICT. HTML just needs to reference that spec. 68030 3 annevk 2012-05-23 07:52:49 +0000 *** This bug has been marked as a duplicate of bug 16768 ***