This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 19646 - WebIDL definitions of TextDecoder and TextEncoder are invalid per WebIDL spec
Summary: WebIDL definitions of TextDecoder and TextEncoder are invalid per WebIDL spec
Status: RESOLVED FIXED
Alias: None
Product: WHATWG
Classification: Unclassified
Component: Encoding (show other bugs)
Version: unspecified
Hardware: PC Windows 3.1
: P2 normal
Target Milestone: Unsorted
Assignee: Anne
QA Contact: sideshowbarker+encodingspec
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-10-20 17:10 UTC by Masatoshi Kimura
Modified: 2012-12-28 10:42 UTC (History)
4 users (show)

See Also:


Attachments

Description Masatoshi Kimura 2012-10-20 17:10:51 UTC
See https://bugzilla.mozilla.org/show_bug.cgi?id=764234#c85 for details.
Gecko is using the following signature to avoid the syntax error.
  DOMString decode(optional ArrayBufferView? view = null,
                   optional TextDecodeOptions options);
  Uint8Array encode(DOMString? string, optional TextEncodeOptions options);
Comment 1 Masatoshi Kimura 2012-10-20 17:31:02 UTC
(In reply to comment #0)
>   Uint8Array encode(DOMString? string, optional TextEncodeOptions options);
This should be
  Uint8Array encode(optional DOMString? string = null, optional TextEncodeOptions options);
to make the first argument optional.
Comment 2 Masatoshi Kimura 2012-10-20 17:32:16 UTC
(In reply to comment #1)
> (In reply to comment #0)
> >   Uint8Array encode(DOMString? string, optional TextEncodeOptions options);
> This should be
>   Uint8Array encode(optional DOMString? string = null, optional
> TextEncodeOptions options);
> to make the first argument optional.
Or
  Uint8Array encode(optional DOMString string = "", optional TextEncodeOptions options);
?
Comment 3 Masatoshi Kimura 2012-10-21 08:33:57 UTC
I prefer
  Uint8Array encode(optional DOMString? string = null, optional TextEncodeOptions options);
so that functions such as
function encodeWrapper(encoding, input) {
  return new TextEncoder(encoding).encode(input);
}
will work as expected even if the input argument is not passed.
Comment 4 Anne 2012-10-21 11:10:24 UTC
I think that's a bug in how Web IDL defines implicit dictionaries.
Comment 5 Anne 2012-10-21 11:11:58 UTC
heycam, what do you think?
Comment 6 Cameron McCormack 2012-12-07 00:30:34 UTC
I don't know that I explicitly thought of the case of dictionary arguments having an implicit default value and how that triggers the rule of all preceding optional arguments needing a default value, but I think it is consistent with that.

The way optional arguments are handled currently is by having the effective overload set include all the valid invocations of the function.  So if you had

  void f(long a, optional long b = 1, optional long c);

the effective overload set is:

  f(long)
  f(long, long)
  f(long, long, long)

At the end of the overload resolution algorithm, default values are filled in.  If you called f(0), the overload resolution algorithm will return <f, (0, 1)>, and that way you never in prose need to handle the case where you have a single value passed to f.

If we allowed:

  void f(long a, optional long b, optional long c = 2);

then maybe we could return something like <f, (0, unspecified, 2)>, which prose could then detect.  You could write, in the definition of f:

  * Do something with a.  [You can assume a always has a value.]
  * If b was specified, do something with b.  [You can't assume b has a value.]
  * Do something with c.  [You can assume c always has a value.]

Default values are all about making it easier for the specification author, so it might well be worth trying to make this work.
Comment 7 Cameron McCormack 2012-12-07 00:41:46 UTC
Actually I am not sure why decode() called with no arguments is useful.  Is it?
Comment 8 Cameron McCormack 2012-12-07 00:43:10 UTC
bz, can you sanity check comment 6 for whether it has an impact on how difficult it is to implement overload/argument processing?
Comment 9 Joshua Bell 2012-12-07 00:44:29 UTC
(In reply to comment #7)
> Actually I am not sure why decode() called with no arguments is useful.  Is
> it?

If you're doing a streaming decode it can be used to terminate the stream without providing any more bytes since. Ideally it's a shortcut for decode(new Uint8Array(), {stream: false}).
Comment 10 Cameron McCormack 2012-12-07 01:00:17 UTC
I see.  Do you explicitly want to not support a call like

  decode(null)

?  If you're happy with decode(null) meaning the same as decode(), then we can just do what comment #0 suggests.
Comment 11 Boris Zbarsky 2012-12-07 01:17:02 UTC
> bz, can you sanity check comment 6 for whether it has an impact on how
> difficult it is to implement overload/argument processing?

I don't think it makes overload processing any less of a "complicated, slow, large codesize, pick two or three" mess than it already is.  ;)

For normal argument processing.  Gecko represents optional arguments without a default value by a pair of (boolean flag, value), with the value being safe to touch only if the flag says so.  So it's no problem for us to loosen this restriction, I think.

The real reason the restriction was there, as I recall, is that JS itself has no way to represent arguments not being passed before other arguments that _are_ passed, short of allowing "undefined" to mean that, whether it was passed explicitly or not.  Which is a bit weird when you have optional strings, of course...  But I seem to recall at least Allen voting for these semantics for undefined.

So here's a question.  Is ES allowing default values for arguments that come after other values with default arguments?  If they are, WebIDL should too.  If not, we should find out why not, yes?
Comment 12 Anne 2012-12-07 10:47:14 UTC
(In reply to comment #10)
> ?  If you're happy with decode(null) meaning the same as decode(), then we
> can just do what comment #0 suggests.

I try not to allow null if there's no compelling reason from an API perspective. And I think here there is not (other than not doing it that way breaking WebIDL).
Comment 13 Masatoshi Kimura 2012-12-27 23:15:23 UTC
Adding an overloaded method will work without changing the WebIDL spec:
  DOMString decode(); // shorthand for decode(Uint8Array());
  DOMString decode(ArrayBufferView input, optional TextDecodeOptions options);

TextEncoder will not require an overloaded method:
  Uint8Array encode(optional DOMString input = "", optional TextEncodeOptions options);
(But it will not throw for null because null will be stringified to the string "null".)
Comment 14 Boris Zbarsky 2012-12-28 06:56:03 UTC
> (But it will not throw for null because null will be stringified to the string
> "null".)

If you wanted to, you could do:

  Uint8Array encode(optional DOMString? input = "", 
                    optional TextEncodeOptions options);
Comment 15 Masatoshi Kimura 2012-12-28 07:16:18 UTC
(In reply to comment #14)
> > (But it will not throw for null because null will be stringified to the string
> > "null".)
> 
> If you wanted to, you could do:
> 
>   Uint8Array encode(optional DOMString? input = "", 
>                     optional TextEncodeOptions options);

  encode(null) should throw, but encode() shouldn't (for the same reason as comment #9).
Comment 16 Boris Zbarsky 2012-12-28 07:27:22 UTC
>  encode(null) should throw, but encode() shouldn't 

Right. Making the first argument |optional DOMString? input = ""| and saying in prose to throw if null is passed would give you that behavior, no?  Though it would also throw if undefined is passed, of course.  Not sure whether you want that.
Comment 17 Masatoshi Kimura 2012-12-28 07:31:30 UTC
(In reply to comment #16)
> >  encode(null) should throw, but encode() shouldn't 
> 
> Right. Making the first argument |optional DOMString? input = ""| and saying
> in prose to throw if null is passed would give you that behavior, no? 
> Though it would also throw if undefined is passed, of course.  Not sure
> whether you want that.

I thought the binding would complement the empty string if null is passed, so encode() would not see the null value. Is it wrong?
Comment 18 Masatoshi Kimura 2012-12-28 07:32:45 UTC
(In reply to comment #17)
> I thought the binding would complement the empty string if null is passed,
> so encode() would not see the null value. Is it wrong?

Ah, I'm confused optional with nullable. Got it.
Comment 19 Anne 2012-12-28 10:42:12 UTC
Null becoming "null" is the desired behavior for strings, so I think that's comment 13 is great. Thanks!

https://github.com/whatwg/encoding/commit/07f5b2e4fc382290717913fb9ae7589a1e3df7af