16157 2012-02-29 01:15:34 +0000 WebSocket shouldn't throw SyntaxError on unpaired surrogates 2012-05-02 20:06:47 +0000 1 1 1 Unclassified WebAppsWG WebSocket API (editor: Ian Hickson) unspecified All All RESOLVED FIXED P2 normal --- 1 glenn ian brian.raymor ian jonas mike public-webapps zcorpan public-webapps-bugzilla oldest_to_newest 64718 0 glenn 2012-02-29 01:15:34 +0000 > If the method's second argument has any unpaired surrogates, then throw a SyntaxError exception and abort these steps. and > If the data argument has any unpaired surrogates, then throw a SyntaxError exception. Don't throw exceptions on unpaired surrogates. Instead, use the WebIDL "convert a DOMString to a sequence of Unicode characters" [1] algorithm, which converts unpaired surrogates to U+FFFD, as well as defining the conversion itself. http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/1589.html [1] http://dev.w3.org/2006/webapi/WebIDL/#dfn-obtain-unicode 64941 1 ian 2012-03-02 23:28:49 +0000 Silently scrambling data seems like a bad idea. Why would we do this? 64942 2 glenn 2012-03-02 23:30:17 +0000 Please see the thread at http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/1589.html, so we don't start the discussion from scratch. 64943 3 ian 2012-03-02 23:46:14 +0000 I read that thread before and didn't see any reason to do this. The only argument I've seen so far is about what happens if a user types in a message with astral characters and the script truncates it naïvely half-way through a surrogate and then sends it through the socket. That does seem like a potentially rare case (wouldn't be caught in the design). Not clear that replacing the half-surrogate with U+FFFD is especially nice either but it seems better than crashing. 65655 4 jonas 2012-03-16 07:33:16 +0000 How is this different from the "draconian" error handling the XML parsers are required to do and which many people, you included, has argued strongly against. The problem with throwing for unpaired surrogates is that easy-to-make data-dependent mistakes produces very fatal results. I.e. if for example you want to send string data in smaller chunks a very easy "mistake" to make would be to simply chop up the JS-string into 10k sized chunks and send each separately. This will generally work great, however in languages which produces a lot of surrogates this will fail 50%-67% of the time. If we could make it throw consistently then I agree it would have been a more reasonable strategy. But I can't think of a way to not make this very data dependent which means that it's likely to not fail on developers machines, but fail in the real world. And yes, putting in a replacement character also results in destroyed data. However in the example stated above, having one destroyed character every 10k of data should be a low enough error rate that the message is still understandable to a human. Just like the layout errors produced by a missing end tag likely produces a page understandable to humans. 67256 5 contributor 2012-05-02 20:06:47 +0000 Checked in as WHATWG revision r7084. Check-in comment: Make WebSocket silently convert isolated surrogated to U+FFFD rather than throwing an exception. This will result in data corruption when a user types in astral-plane characters that get truncated by naiive script half-way through, rather than crashing the application. http://html5.org/tools/web-apps-tracker?from=7083&to=7084