This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 27378 - Apparently DNT started using "effective script origin" for no reason. Perhaps we can rename it "legacy origin" to make sure people do not use it for new things?
Summary: Apparently DNT started using "effective script origin" for no reason. Perhaps...
Status: RESOLVED MOVED
Alias: None
Product: WHATWG
Classification: Unclassified
Component: HTML (show other bugs)
Version: unspecified
Hardware: Other other
: P3 normal
Target Milestone: Unsorted
Assignee: Ian 'Hixie' Hickson
QA Contact: contributor
URL: https://html.spec.whatwg.org/#effecti...
Whiteboard:
Keywords:
: 27205 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-11-20 10:11 UTC by contributor
Modified: 2016-03-23 12:42 UTC (History)
8 users (show)

See Also:


Attachments

Description contributor 2014-11-20 10:11:39 UTC
Specification: https://html.spec.whatwg.org/multipage/browsers.html
Multipage: https://html.spec.whatwg.org/multipage/#effective-script-origin
Complete: https://html.spec.whatwg.org/#effective-script-origin
Referrer: https://html.spec.whatwg.org/multipage/

Comment:
Apparently DNT started using "effective script origin" for no reason. Perhaps
we can rename it "legacy origin" to make sure people do not use it for new
things?

Posted from: 80.218.216.100 by annevk@annevk.nl
User agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:36.0) Gecko/20100101 Firefox/36.0
Comment 1 Ian 'Hixie' Hickson 2014-11-20 20:43:57 UTC
Well you want to use it for most APIs, otherwise, document.domain actually opens up new security holes (where you can get to content from both the document.domain origin and the original origin, and it depends which side of the shared document.domain boundary you are on whether it works or not). How did DNT use it? Aren't they non-scripted?
Comment 2 Anne 2014-11-21 06:57:49 UTC
http://www.w3.org/2011/tracking-protection/drafts/tracking-dnt.html has a pretty big API these days.

By most APIs, do you mean that e.g. the checks in https://notifications.spec.whatwg.org/ are wrong? I thought origin was always more restrictive.
Comment 3 Ian 'Hixie' Hickson 2014-11-21 23:59:47 UTC
Well the bigger problem with the way notifications.spec.whatwg.org does origin checks is that it doesn't use "same origin" checks, but equality, and it's not obvious whether two tuples are equal in general.

But as far as document.domain goes, I don't know if that spec does it right. What is the security invariant you're trying to maintain? What's the attack vector you're trying to protect again? As it's specced, if pages A and B use document.domain to match effective origins, then if B calls a script in A that displays a notification, it'll do so, in A's origin. Is that what you want? Personally I would be tempted to use the effective script origin for the tests, and only allow the user to grant permission to origins that are {scheme,host,port} tuples with real ports, so that pages with opaque origins, and pages that use document.domain, just can't post notifications at all.
Comment 4 Anne 2014-11-22 11:00:50 UTC
I see, that would actually make sense. However, that requires anyone that designs a new API to know about document.domain :-( I don't really see a good way around that though.
Comment 5 Boris Zbarsky 2014-11-22 21:45:34 UTC
If you're designing an API that does origin checks, you better know about document.domain.  The right answer is to not design APIs that need origin checks if you can avoid them.
Comment 6 Anne 2014-11-23 08:28:59 UTC
That is nice in theory, but I doubt e.g. this came up in the design of service workers. Or the design of getUserMedia(). The design of the Push API still talks about an undefined term "webapp" rather than anything to do with origins... :-(
Comment 7 Boris Zbarsky 2014-11-23 23:33:00 UTC
Well, then we need to rely on the TAG to catch this, or something.  Fundamentally, there's only so much we can do about incompetent people designing APIs.  :(
Comment 8 Anne 2014-11-24 12:46:36 UTC
Incompetent, sure, though until fairly recently nobody brought up concerns with the way the Notifications API did this.

But now I know there's at least two issues:

1) Whether to use origin or effective script origin.

2) Whether to use entry setting object's origin, incumbent settings object's origin, or associated Realm/global object's origin (does this last one work for statics?).

And although thanks to Ian I have now a reasonable clue as how to address 1) in a way that breaks down when developers use document.domain (seems good) it's still unclear how to address 2).

I guess that's a separate bug and we can close this one, but I don't believe for a minute that the TAG has a better grasp of this subject matter than I do.
Comment 9 Boris Zbarsky 2014-11-24 14:39:21 UTC
Sorry, my use of "incompetent" was probably not warranted.  Security is hard.  :(  The legacy insanity we have to deal with on the web in the form of document.domain makes it extra hard.  It doesn't help that the security model for document.domain in the spec isn't actually implemented by all UAs, and won't be, because some UAs consider it insecure.

So the main issue here is people not knowing what they don't know.  :(

> or associated Realm/global object's origin (does this last one work for
> statics?)

Work in what sense?

> but I don't believe for a minute that the TAG has a better grasp of this
> subject matter than I do

The only value the TAG can provide here is to serve as a chokepoint which (1) looks at all APIs and (2) red-flags APIs that use origins for anything and asks browser security experts for feedback specifically on the origin usage (with an explicit request to consider document.domain interactions).

For example, I really doubt Bobby Holley has looked at the Notifications API.  Has Adam Barth?  Equivalents from Microsoft and Apple?

Note that there are other interesting issues resulting from the irreconcilable divergence between spec and implementations.  A concrete example is that the "manual override" port thing doesn't exist in at least Gecko.  The way origins are compared is slightly different from the spec, and there is an object representing a pair of origin and effective script origin that is what comparisons are really performed on.  So there using "effective script origin" and comparing to a URI origin can actually return equality, which it can never do in the spec as far as I can tell.  I can't speak for other UAs here.

Also, the lack of testcases around this stuff means implementors are likely not even aware of what the spec says here (e.g. this is the first I've heard of the manual override bit and its implication for comparing effective script origins to URI origins, and I'd be surprised if other implementors were aware of it).
Comment 10 Boris Zbarsky 2014-11-24 14:40:57 UTC
> which it can never do in the spec as far as I can tell

And comment 3's analysis relies on this property, I believe.
Comment 11 Anne 2014-11-24 15:08:53 UTC
For statics, e.g. a question is what origin Notification.requestPermission() needs to look at and what the correct wording would be to refer to that origin.

But it sounds like there are some larger issues around the current concepts of origin.
Comment 12 Boris Zbarsky 2014-11-24 15:20:11 UTC
> For statics, e.g. a question is what origin Notification.requestPermission()
> needs to look at

OK, so this is only maybe different from non-statics in that there are 4 possible origins to consider: the entry settings origin, the incumbent settings origin, the origin of the global of the Realm of the requestPermission function and the origin of the global of the Realm of the Notification function.  Except for non-statics we can still usefully talk about this last if we really want to, since Web IDL objects have associated Realms.  We've just decided we never want to talk about it, because nothing uses that origin so far and there's no reason to start.

So I don't think this is really any different from the non-static case.
Comment 13 Anne 2014-11-24 15:40:17 UTC
In the IDL -> API -> IDL setup I think it would be useful for IDL to also pass "origin" and "effective script origin" directly to API so API doesn't have to concern itself with what object to pull them from (or pass the correct object, I suppose).

Other bugs around this:

* bug 26603 (merging the concept of incumbent global with the current Realm)
* bug 27204 (guidance on entry vs incumbent settings objects)
* bug 27205 (similar to this one)
* bug 27301 (define context variables, such as "context object")
Comment 14 Bobby Holley (:bholley) 2014-11-24 17:07:21 UTC
(In reply to Ian 'Hixie' Hickson from comment #1)
> Well you want to use it for most APIs, otherwise, document.domain actually
> opens up new security holes (where you can get to content from both the
> document.domain origin and the original origin, and it depends which side of
> the shared document.domain boundary you are on whether it works or not). How
> did DNT use it? Aren't they non-scripted?

Wait, what? When we spoke about this last December, you said that you were explicitly trying to kill document.domain by not supporting it in new features. This caused me to go and do [1] in Gecko. I would be pretty unhappy if we backpedaled from that.

It seems to me that the best way to accomplish the above would be to make any of these new APIs throw if document.domain is set. Maybe we should try that?

In terms of the security risk of ignoring document.domain, I'm not really sold. Pages can always perform operations with their old origin if they really want to, by setting up an iframe before setting document.domain and postMessaging commands to it.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=956382#c0
Comment 15 Anne 2014-11-24 17:15:54 UTC
Ignoring what object holds origin (O) and effective script origin (ESO) this was my idea based on Hixie's input for the Notifications API:

* Notification.requestPermission throws if O is not same origin with ESO; stores permission for O
* every point that checks against the permission store would use ESO (due to "manual override" this could never match a stored permission if not same origin with O)
Comment 16 Boris Zbarsky 2014-11-24 17:31:46 UTC
> throws if O is not same origin with ESO

Just to be clear, this is only the case in the "document.domain explicitly set" case?

> every point that checks against the permission store would use ESO

And this is equivalent to checking "document.domain not explicitly set" and then using O, right?
Comment 17 Anne 2014-11-24 17:38:53 UTC
As far as I can tell, yes.
Comment 18 Ian 'Hixie' Hickson 2014-11-24 19:26:17 UTC
There's many ways to spec this, sure. My point is just that you have to take it into account. You can't just ignore the effective script origin; you have to handle the case where document.domain has been set, even if it's just to throw. Or you can not throw, and ignore document.domain (that's how localStorage is implemented in browsers, despite the implications this has on the storage mutex). But you have to know what you're doing. Yes, writing Web specs requires understanding the Web's security model; that's a prerequisite of developing technology for any platform.
Comment 19 Boris Zbarsky 2014-11-24 19:29:12 UTC
"the Web's security model" is a bit of a misnomer given the UA and spec differences in handling of document.domain.  :(

We can maybe talk about there being a single security model if no one ever touches document.domain (though even there there are some differences between UAs and the spec is just not defined at all pending bug 20701 being resolved)...

All in all, a mess.
Comment 20 Bobby Holley (:bholley) 2014-11-24 20:33:40 UTC
(In reply to Ian 'Hixie' Hickson from comment #18)
> There's many ways to spec this, sure. My point is just that you have to take
> it into account.You can't just ignore the effective script origin;

Why? Can't we just ignore it and use the regular origin? You cited a security risk but I'm not convinced - see the last paragraph of comment 14.
Comment 21 Ian 'Hixie' Hickson 2014-11-25 07:18:40 UTC
Because if you design an API while being ignorant of a fundamental part of a platform's security model, you're asking to shoot yourself in the foot.

For example, Web Storage used to have a clear cut-out for document.domain; setting document.domain essentially neutered localStorage. Now implementations don't do this, and the result is that nobody who implements localStorage on a per-domain basis can implement the storage mutex without risking deadlock. Thus we have either forced the Web to have an API with an inherent data corruption race, or forced the Web to have a cross-process jank-inducing mutex, both of which are terrible situations to have painted ourselves into. Spec writers that mess with origins _have_ to be aware of these things so that they don't make things even worse.

IMHO renaming this concept "legacy origin" would encourage spec authors to dismiss it as some old thing they don't need to think about.
Comment 22 Anne 2014-11-25 16:41:16 UTC
Hixie, why would we succeed with a different model for other APIs? If implementers did not care about this for localStorage, why would they care elsewhere?

It seems that Bobby argues that new APIs should just base everything around "origin" and not use the "effective script origin" concept.

I would be happy with something where all new APIs check a "document.domain setter ran flag" and ceased to function if we can get agreement on that though. Crippling document.domain further seems good.
Comment 23 Bobby Holley (:bholley) 2014-11-25 18:13:24 UTC
(In reply to Anne from comment #22)
> I would be happy with something where all new APIs check a "document.domain
> setter ran flag" and ceased to function if we can get agreement on that
> though. Crippling document.domain further seems good.

This is my ideal scenario.

(In reply to Ian 'Hixie' Hickson from comment #21)
> For example, Web Storage used to have a clear cut-out for document.domain;
> setting document.domain essentially neutered localStorage. Now
> implementations don't do this, and the result is that nobody who implements
> localStorage on a per-domain basis can implement the storage mutex without
> risking deadlock.

I don't understand this part. If localStorage just blindly ignores document.domain, what new implementation constraints does that create? The spec already effectively requires that there is no process separation between any two browsing contexts in a "unit of related similar-origin browsing contexts", so strict process-per-domain is a non-starter. And even if it _were_ possible, ignoring document.domain for localStorage is exactly what we'd want to avoid cross-process considerations in a process-per-domain world.

Or am I misunderstanding something?
Comment 24 Ian 'Hixie' Hickson 2014-11-26 19:31:01 UTC
The storage mutex (up to the point where this was broken by localStorage implementations ignoring the document.domain restriction) was carefully designed so that you could actually have a separate storage mutex per origin.

Anyway.

I don't think we're particularly disagreeing with each other here. My point is just that spec authors have to be careful, and to be careful they have to know about this kind of thing, we shouldn't try to bury this kind of landmine.
Comment 25 Bobby Holley (:bholley) 2014-11-26 20:03:37 UTC
(In reply to Ian 'Hixie' Hickson from comment #24)
> I don't think we're particularly disagreeing with each other here. My point
> is just that spec authors have to be careful, and to be careful they have to
> know about this kind of thing, we shouldn't try to bury this kind of
> landmine.

Well, there are basically 3 courses of action:

(A) Get all new features that do anything with origins to fail in the presence of document.domain.
(B) Discourage the use of effective script origin.
(C) Encourage the use of effective script origin.

We both agree that (A) is ideal. But you're proposing that (C) is preferable to (B), and I'm saying the opposite.

I would obviously prefer spec authors to understand security as much as possible. But on balance, I think the original script origin is the Right Thing more often than the effective script origin.
Comment 26 Ian 'Hixie' Hickson 2014-11-27 00:17:04 UTC
*** Bug 27205 has been marked as a duplicate of this bug. ***
Comment 27 Ian 'Hixie' Hickson 2014-11-28 04:17:36 UTC
I didn't mean to propose any of A-C, sorry if that was unclear.

My point is just that we shouldn't change the spec to make the effective script origin be something other spec authors ignore.
Comment 28 Bobby Holley (:bholley) 2014-11-28 07:15:24 UTC
(In reply to Ian 'Hixie' Hickson from comment #27)
> I didn't mean to propose any of A-C, sorry if that was unclear.

OK - This is how I interpreted the first sentence in comment 1.
 
> My point is just that we shouldn't change the spec to make the effective
> script origin be something other spec authors ignore.

The problem is that right now, spec authors don't think about it, and use "effective script origin" because "effective" sounds like the right thing.

If we have some way to get them to think about it, I think we want to explain push for a combination of (A) and (B). Do you have a concrete way to make them think about it?

If we don't actually have a good way to get them to think about it, then we can pick one of the following:
(I) Let them pick at random (status quo)
(II) Nudge them away from effective script origin (B)
(III) Nudge them towards effective script origin (C)

(II) seems best.
Comment 29 Simon Pieters 2014-11-28 08:30:21 UTC
I think it would be more obvious what "effective script origin" means if it's called something like "document.domain-affected origin".
Comment 30 Ian 'Hixie' Hickson 2014-12-01 19:02:17 UTC
Ah, my apologies. I did indeed overstate my position in comment 1. Please disregard that comment.

Right now I don't think all spec authors use effective script origin. They sometimes use one and sometimes use the other and it's not clear to me that they always understand why.

I have no idea how to make anyone think, whether they're spec authors or implementors or authors or users or anyone else. :-(

Renaming "effective script origin" to something like "document.domain-affected origin" seems reasonable.
Comment 31 Bobby Holley (:bholley) 2014-12-02 06:06:05 UTC
(In reply to Ian 'Hixie' Hickson from comment #30)
> Renaming "effective script origin" to something like
> "document.domain-affected origin" seems reasonable.

That sounds great.

Do we want to encourage people designing new features to blackball document.domain? If so, is there somewhere we could put that encouragement in prose?
Comment 32 Ian 'Hixie' Hickson 2014-12-03 19:53:21 UTC
Spec writing advice probably belongs on the wiki.
Comment 33 Anne 2014-12-03 22:47:56 UTC
I suggest we rename as a first step. Then we should probably close this bug. 

I will try to write up a plan how we can further cripple document.domain in both existing and new APIs that browser vendors can sign off on and then implement.

And I guess I should talk with bz at some point how specification origins are a poor match for implementations.
Comment 34 Anne 2016-03-16 18:14:32 UTC
Plan for phasing out document.domain is now:

  https://github.com/whatwg/html/issues/829

That leaves renaming for this bug.

This is also worth exploring further:

(In reply to Boris Zbarsky from comment #9)
> Note that there are other interesting issues resulting from the
> irreconcilable divergence between spec and implementations.  A concrete
> example is that the "manual override" port thing doesn't exist in at least
> Gecko.  The way origins are compared is slightly different from the spec,
> and there is an object representing a pair of origin and effective script
> origin that is what comparisons are really performed on.  So there using
> "effective script origin" and comparing to a URI origin can actually return
> equality, which it can never do in the spec as far as I can tell.  I can't
> speak for other UAs here.

Could you maybe expand on the check that Gecko does?
Comment 35 Boris Zbarsky 2016-03-16 19:06:16 UTC
I'm not sure what there is to expand on.  I thought I explained the setup Gecko has in comment 9...  What part is unclear?

It's only an issue if you compare the effective script origin of a document to the origin of a URI.  Which is something best not done anyway...
Comment 36 Anne 2016-03-17 05:55:29 UTC
(In reply to Boris Zbarsky from comment #35)
> It's only an issue if you compare the effective script origin of a document
> to the origin of a URI.  Which is something best not done anyway...

Right, and this does not happen in the specification currently, as far as I can tell.

However, it does mean that Gecko also has a different algorithm for "same origin" and potentially does not allow cross-port communication whereas the specification does? (Since Gecko does not have "manual override" for ports.)


FYI, I will do the renaming to "document.domain-affected origin" after

  https://github.com/whatwg/html/pull/868
  https://github.com/whatwg/html/pull/870

land. 

We could perhaps even simplify things by having similar to Gecko. Where origin is typically a single thing, but sometimes it might have an additional field stored on it that is used by certain origin checks. The current duplication is rather weird since only documents and environment settings objects seem to really need the effective script origin concept.
Comment 37 Boris Zbarsky 2016-03-17 05:59:51 UTC
> However, it does mean that Gecko also has a different algorithm for
> "same origin"

Does it, for cases when not talking about effective script origins?  I don't think it does, but maybe I'm missing something.

> and potentially does not allow cross-port communication whereas the
> specification does?

I think I need more context here.  What situation are you thinking of?
Comment 38 Anne 2016-03-17 06:28:36 UTC
You're saying Gecko does not set port to "manual override". Say I have documents at https://example.org:8000/ and https://example.org:9001/. Both set document.domain to "example.org". It's hard to believe that does not work, so maybe you don't set the port to "manual override", but instead use the default? Otherwise the "same origin" check would have to be different, to ignore the port. I don't see how those two could compare, or how you could compare those to the origin of a URL, otherwise.
Comment 39 Boris Zbarsky 2016-03-17 12:58:29 UTC
What Gecko has is the following setup:

There is an "origin object" attached to each Document.  This origin object has an origin, and can have an effective script origin override.

When comparing one origin object to another, they are considered "same origin" if their origins match.  They are considered "same effective script origin" if either they both have effective script origin overrides and their overrides match or neither one has an override and the origins match.

Various other things (e.g. stylesheets and images) can also have "origin objects" attached to them.  These never have an effective script origin override.

There is also a way to compare an origin object to a URI for things like CORS; this always ignores the effective script origin override.

Going back to your example, you would have two documents, both with effective script origin override set to "https://example.org".
Comment 40 Mike West 2016-03-21 09:31:20 UTC
> There is an "origin object" attached to each Document.  This origin object has an origin, and can have an effective script origin override.

This is more or less what Blink does as well. We have a "SecurityOrigin" object which stores both the (scheme,host,port) tuple, as well as a "domain". That object has a "canAccess" method hanging off of it that determines whether to do an origin comparison or a domain comparison: https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/Source/platform/weborigin/SecurityOrigin.cpp&sq=package:chromium&l=226&rcl=1458530990
Comment 41 Anne 2016-03-23 12:42:50 UTC
Wrote a patch for HTML to do away with effective script origin and instead have a mutable domain field on origin coupled with same origin-domain comparison algorithm:

https://github.com/whatwg/html/pull/925