This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 11836 - Don't specify the transport, just specify API and protocol
Summary: Don't specify the transport, just specify API and protocol
Status: RESOLVED WONTFIX
Alias: None
Product: WebAppsWG
Classification: Unclassified
Component: HISTORICAL - Server-Sent Events (editor: Ian Hickson) (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal
Target Milestone: ---
Assignee: Ian 'Hixie' Hickson
QA Contact: public-webapps-bugzilla
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-01-21 23:35 UTC by Robert Kieffer
Modified: 2011-06-21 05:23 UTC (History)
6 users (show)

See Also:


Attachments

Description Robert Kieffer 2011-01-21 23:35:21 UTC
EventSource shouldn't concern itself with transport issues that aren't directly related to what's required to drive the EventSource API.

Conceptually an EventSource is simply a client-side API that is driven by an EventSource data stream.  Whether that stream is an XHR request, or a WebSocket, or a raw TCP stream is not important to the core purpose of an EventSource.  Trying to specify the details and interface required to implement a specific transport (e.g. what domain restrictions should be imposed) will be confusing when there are already multiple specifications in place for exactly this (i.e. XHR and WebSockets).

Instead, the EventSource interface should abstract all this out by adding a "transport" property that points to "the implementation-specific object used to connect to the server".  This will allow developers to access the internals of the transport as needed, without having to mirror the API in the EventSource interface.  (For example, the Mozilla XHR object has a "withCredentials" property that developers will want to access when doing cross-domain requests.)

You might also want to clarify that the "readyState" property and the open/message/error events are not expected to correspond directly to any similar interface in the transport object, even though similarities are expected.
Comment 1 Anne 2011-01-22 10:32:24 UTC
How exactly do you envision EventSource to work then? (Maybe you could give some example code.)
Comment 2 Robert Kieffer 2011-01-23 16:23:54 UTC
(In reply to comment #1)
> How exactly do you envision EventSource to work then? (Maybe you could give
> some example code.)

So, here's the current sample from the EventSource spec:

    var source = new EventSource('updates.cgi');
    source.onmessage = function (event) {
      alert(event.data);
    };

So how would I issue a CORS request with the 'withCredentials' property set (XHR - level 2)?  After all, it's only natural to assume there's an XHR API under the covers here.  One approach would be to mirror the XHR api in the EventSource interface and add a withCredentials property:

    source.withCredentials = true;

But that ties EventSource to XHR, which I don't believe is appropriate.  There are cases where the transport won't be XHR.  For example, Node.js doesn't have an XHR object, but it does have the http.ClientRequest object.  Hence my proposal for the 'transport' property as a way of encapsulating and exposing the transport-specific API:

    var source = new EventSource('updates.cgi');

    source.transport.withCredentials = true;

    source.onmessage = function (event) {
      alert(event.data);
    };

It's important to note that I'm assuming the transport doesn't issue the request until after the current event loop finishes.  I.e. there's an implicit "setTimeout(source.transport.send, 0)" built ito the EventSource constructor.  I'm not sure if that's the right approach, but it's either that or add a "send()" method that users will have to call send the request.

Does this help?  I have to confess I wasn't quite sure what you wanted clarification on.
Comment 3 Anne 2011-01-23 23:17:46 UTC
So you want certain functionality of XMLHttpRequest exposed on EventSource but in a generic way, with a common object? The way I read your first comment was that you would have to first create an XMLHttpRequest object somehow and tie it to EventSource... 

I think we will just specify EventSource to always include credentials and not offer that option.
Comment 4 Robert Kieffer 2011-01-24 00:43:12 UTC
I wasn't thinking that developers would manually instantiate an XHR transport, but it's an idea worth considering.  I would expect that for convenience EventSource implementations would create the XHR object as part of the constructor and do the necessary initialization.  But I wouldn't be surprised if there were cases people wanted to wrap an EventSource around an already-created transport.

Always sending credentials is no wiser than never sending them.  Clients should be able to make cross-domain requests without having to unnecessarily expose session cookies or other potentially sensitive auth information to 3rd party sites.  Making assumptions like this is exactly the kind of thing I would like to discourage.
Comment 5 Anne 2011-01-24 11:15:19 UTC
I do not really follow. First of all nothing would be exposed, since the credentials belong to the third-party site. And nobody is going to implement EventSource on top of XMLHttpRequest. Much like <img> is not implemented on top of XMLHttpRequest either.
Comment 6 Robert Kieffer 2011-01-24 16:08:04 UTC
(In reply to comment #5)
> I do not really follow. First of all nothing would be exposed, since the
> credentials belong to the third-party site.

Sorry, I brain farted on that.  I'm was rushing to come up with a reason why 'withCredentials' is useful.  Regardless, given that the default is 'false',  I would expect having EventSource default that to true to raise some eyebrows.  I'll let more XHR-savvy people than I explain the 'why' behind that.   The point being that transports

> And nobody is going to implement
> EventSource on top of XMLHttpRequest. Much like <img> is not implemented on top
> of XMLHttpRequest either.

To the contrary, there's already a popular jquery plugin for exactly that: https://github.com/rwldrn/jquery.eventsource 

EventSource on top of XHR is an obvious solution - so obvious that there's no need to build this into native browsers - the problem is already solved (well, aside from that code being jquery-specific, but that's a minor detail.)

Given that, where is the value in having EventSource specify behavior that is contradictory or redundant with this reality?
Comment 7 Anne 2011-01-24 16:14:00 UTC
The plugin falls back to polling. That is far from the same. And by default it tries to use the EventSource implementation of the browser. EventSource together with other protocols also opens up the possibility for push notifications which is something you cannot get otherwise.

And the reason XMLHttpRequest has withCredentials is because it allows much more complicated requests. EventSource does not get more complicated than <script> or <img> so can use the same cross-origin model. (I am the editor of the XMLHttpRequest specification by the way.)
Comment 8 Robert Kieffer 2011-01-24 19:29:49 UTC
(In reply to comment #7)
> The plugin falls back to polling. That is far from the same.

And yet, there's nothing to prevent such a plugin from using interactive-XHR (available in both webkit and FF) to process a streamed response... which would be the same for all intents.

> And by default it tries to use the EventSource implementation of the browser.

... which (currently) has the least desirable behavior due to x-domain/connection limit issues.

> EventSource
> together with other protocols also opens up the possibility for push
> notifications which is something you cannot get otherwise.

Are you referring to the verbiage about "connectionless push and other features" that's currently in the spec?  I don't understand how that relates here.  Sure, EventSource is a server-push protocol.  And as long as the transport layer supports server-push (which interactive XHR does) why should the specification care beyond that.

I apologize if I'm being dumb here.  I'm just not sure how "other protocols" bear on a discussion about transports.

> And the reason XMLHttpRequest has withCredentials is because it allows much
> more complicated requests. EventSource does not get more complicated than
> <script> or <img> so can use the same cross-origin model. (I am the editor of
> the XMLHttpRequest specification by the way.)

EventSource is more complicated in the sense that it exposes the content of the requested URL to client scripts - something that browsers are careful to avoid with script and image tags.  E.g. you can create an image or script element using javascript, but you don't get access to the actual script or image data.

Because of that, I think it's better to treat EventSource as a specialized form of XHR rather than as script or image tags.
Comment 9 Ian 'Hixie' Hickson 2011-02-14 10:21:34 UTC
I don't understand this request. Can you succinctly describe what user-facing or author-facing problem you are trying to solve with your proposal?
Comment 10 Robert Kieffer 2011-02-14 14:51:13 UTC
(In reply to comment #9)
> I don't understand this request. Can you succinctly describe what user-facing
> or author-facing problem you are trying to solve with your proposal?

The problem is that there are existing HTTP transport APIs that can be used to implement EventSources, and there are issues in doing HTTP transport that are important to users, but that aren't directly germane to providing the core EventSource feature: server driven messaging.  So either this spec provides an abstraction for allowing users to access these transport APIs directly, or it must expose all current and future functionality that may be required at the transport layer.

Currently we seem to be headed down the latter path.  I see this as a rather tragic waste of time and energy.  Why does the EventSource spec need to define how "withCredentials" property works, or what the "Access-Control-Allow-Origin" header is for?  This is already covered in XHR and CORS, and should not be duplicated here.

To my mind, a much simpler, more maintainable, more durable approach, is to simply acknowledge that there are existing transport APIs on which EventSources are going to be built, and that users will need to interact with them directly.  We should just define a "transport" property that is "implementation specific", and get out of the way.
Comment 11 Robert Kieffer 2011-05-24 15:54:42 UTC
Ian, any further thoughts on this?

Given the lack of response to my last comment, let me try again to explain the issues I see.

 It's a given that someone will use XHR+CORS to create a JS library that papers over the x-browser differences in SSE implementations.  Sure, it may use long-polling, but the XHR spec allows for progress notifications, which means it could very easily be a real event stream.  And since XHR allows CORS for cross-domain requests, developers won't be hobbled by the rather pointless requirement this spec currently has about imposing a same-domain restriction.  That's the main issue I have here.

I hope you'll agree that SSE on top of XHR+CORS is an obvious x-browser solution, because it demonstrates how the SSE protocol can and should be written with as few assumptions as possible about how the transport layer on which it's implemented works.  Because while XHR may be the obvious choice today, tomorrow it could very easily be WebSockets.

And even though this spec is targeted at browsers, there are a lot of other HTTP agents out there that aren't browsers.  Hell, just pick a language - C, C#, PHP, Python, Ruby... the list goes on, even JS, ala Node.js.  All of these have supports for agents not only is same-origin unnecessary, it doesn't even make sense!   What's the "origin" for server A, when it's issuing an isolated request for SSE events from Server B?  Dunno.

This kind of server-side application of SSE may not be your target audience, but it's a perfectly reasonable application of SSE, and as such has the potential to provide significant value in that space.
Comment 12 Robert Kieffer 2011-05-24 15:58:24 UTC
(In reply to comment #11)
> have supports for agents not only is same-origin unnecessary, it doesn't even

"...has support for agents where not only is a same-origin restriction unnecessary, it doesn't even..."
Comment 13 Ian 'Hixie' Hickson 2011-06-17 19:58:04 UTC
Sorry about the delay, I deal with bugs in the order that they were last changed, so every time a bug is touched it goes to the back the queue. :-)

(In reply to comment #10)
> 
> The problem is that there are existing HTTP transport APIs that can be used to
> implement EventSources, and there are issues in doing HTTP transport that are
> important to users, but that aren't directly germane to providing the core
> EventSource feature: server driven messaging.  So either this spec provides an
> abstraction for allowing users to access these transport APIs directly, or it
> must expose all current and future functionality that may be required at the
> transport layer.

That doesn't seem like a problem to me. Maybe a matter of design aesthetic...


> Currently we seem to be headed down the latter path.  I see this as a rather
> tragic waste of time and energy.

I would agree, if there was much to expose. But what is there to expose?

The only thing I can think of is CORS, and we can do that without touching the API (and I'm about to do so).


> Why does the EventSource spec need to define
> how "withCredentials" property works, or what the "Access-Control-Allow-Origin"
> header is for?  This is already covered in XHR and CORS, and should not be
> duplicated here.

XHR seems irrelevant here, but yes, CORS defines this. So the spec just defers to CORS.


> To my mind, a much simpler, more maintainable, more durable approach, is to
> simply acknowledge that there are existing transport APIs on which EventSources
> are going to be built, and that users will need to interact with them directly.

What other transports are there?


> We should just define a "transport" property that is "implementation
> specific", and get out of the way.

What do you mean by "implementation specific"?


(In reply to comment #11)
> 
> It's a given that someone will use XHR+CORS to create a JS library that papers
> over the x-browser differences in SSE implementations.  Sure, it may use
> long-polling, but the XHR spec allows for progress notifications, which means
> it could very easily be a real event stream.  And since XHR allows CORS for
> cross-domain requests, developers won't be hobbled by the rather pointless
> requirement this spec currently has about imposing a same-domain restriction. 
> That's the main issue I have here.

If you just want cross-site SSE, then we can solve that problem by adding CORS to SSE. No reason to have abstractions, transports, etc.


> And even though this spec is targeted at browsers, there are a lot of other
> HTTP agents out there that aren't browsers.  Hell, just pick a language - C,
> C#, PHP, Python, Ruby... the list goes on, even JS, ala Node.js.  All of these
> have supports for agents not only is same-origin unnecessary, it doesn't even
> make sense!   What's the "origin" for server A, when it's issuing an isolated
> request for SSE events from Server B?  Dunno.

Such systems wouldn't use this API. That would make no sense. They'd design their own APIs that make sense for them.


> This kind of server-side application of SSE may not be your target audience,
> but it's a perfectly reasonable application of SSE, and as such has the
> potential to provide significant value in that space.

My goal is to make the Web better, not to write generic APIs that apply everywhere and work well nowhere. Better IMHO to have dedicated APIs for each environment.

We don't need to have common APIs when they couldn't possibly interoperate anyway (e.g. you can't just run Ruby code in your PHP interpreter).
Comment 14 Robert Kieffer 2011-06-20 20:27:13 UTC
Hey Ian, thanks for your attention on this.

I guess we have a bit of a philosophical disconnect.  Fortunately with the recent spec changes I don't think it manifests in significant ways, but I do think the "why" behind it may be important in keeping the spec on track, so let me elaborate for a moment.

First, I see two aspects to this.  There's the client API - the EventSource methods and events - and then there's the actual over-the-wire protocol.  I agree the main audience for the EventSource API is  browser clients (although I suspect there will be some server-side JS implementations modeled on it, as I've mentioned).  Of more interest is the over-the-wire protocol, which it sounds like you don't expect non-browser systems to find particularly interesting.  I'm pretty sure that's incorrect.

Specifically, any service that has a public streaming API is going to be looking at SSE as the "standard" solution.  The Twitter streaming API (http://dev.twitter.com/pages/streaming_api_methods)  is a particularly good example of this.  It's currently an ad-hoc protocol but there's no reason it couldn't/shouldn't be SSE.  And, in fact, as SSE support in browsers becomes more prevalent, there will be increasing pressure for them to adopt it.  But the Twitter API is very clearly designed for more than just browser clients. In fact, they talk about using curl to test in development, and there are countless examples of 3rd-party services that do server-to-server integration using the public API. 

Anyway, my point is that this API most assuredly is of interest to a broader audience than just browser clients, because it specifies both a protocol and a client API.  I think you're right to be focused on the browser clients, and to make sure that first-and-foremost we solve the problems there, but I think it would be a mistake to rule out it's use in other contexts unnecessarily.
Comment 15 Ian 'Hixie' Hickson 2011-06-21 05:23:38 UTC
EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Rejected
Change Description: no spec change
Rationale: If people want to use this protocol in other situations for which it is not well suited and for which it was not designed, that's their business, but that doesn't mean we should change the API so that it makes handling those cases easier. If anything, it's better that the API be less suitable for those cases, as it would discourage it.

Reductio ad absurdum: if someone were to use this protocol to communicate between two chips on a  cat food dispenser microcontroller, should we change the API and protocol to have a field for listing the number of cats? No, we should tell them that they're making a mistake.