This is an archived snapshot of W3C's public bugzilla bug tracker, decommissioned in April 2019. Please see the home page for more details.

Bug 26365 - [Shadow]: Need an equivalent definition of 'in a Document' for shadow trees
Summary: [Shadow]: Need an equivalent definition of 'in a Document' for shadow trees
Status: RESOLVED MOVED
Alias: None
Product: WebAppsWG
Classification: Unclassified
Component: HISTORICAL - Component Model (show other bugs)
Version: unspecified
Hardware: PC All
: P1 normal
Target Milestone: ---
Assignee: Hayato Ito
QA Contact: public-webapps-bugzilla
URL:
Whiteboard:
Keywords:
: 22361 25562 (view as bug list)
Depends on: 22443 27406
Blocks: 25563 26845 26934 26943 27325 27379 27380 27757 28552
  Show dependency treegraph
 
Reported: 2014-07-17 04:13 UTC by Hayato Ito
Modified: 2015-05-27 03:25 UTC (History)
10 users (show)

See Also:


Attachments

Description Hayato Ito 2014-07-17 04:13:53 UTC
The context is:

https://code.google.com/p/chromium/issues/detail?id=393350
https://code.google.com/p/chromium/issues/detail?id=394327

According to the current definition of a 'in a Document' [1], some important features, such as scripting, should not work in shadow trees as per the spec.

[1]: http://dev.w3.org/html5/spec-preview/infrastructure.html#in-a-document


Because I am afraid that it's not a good idea to change the definition of 'in a Document', we need a better terminology for alternative of 'in a Document', such as 'in a Document or in a shadow tree', and should fix the relevant specs by using that.
Comment 1 Boris Zbarsky 2014-07-17 05:25:53 UTC
> such as 'in a Document or in a shadow tree'

That's wrong if the shadow host in the light DOM is not in a document.

What are the cases in which you'd want different behavior between "in a Document" and whatever your new concept is?
Comment 2 Hayato Ito 2014-07-17 05:56:31 UTC
(In reply to Boris Zbarsky from comment #1)
> > such as 'in a Document or in a shadow tree'
> 
> That's wrong if the shadow host in the light DOM is not in a document.
> 
> What are the cases in which you'd want different behavior between "in a
> Document" and whatever your new concept is?

I think that should be 'in a composed tree whose root element is a Document'.
That's still a tentative idea, though.
Comment 3 Boris Zbarsky 2014-07-17 06:01:05 UTC
Yes, but what will break if the definition of "in a Document" is changed to mean that (with s/element/node/, I assume).
Comment 4 Hayato Ito 2014-07-17 06:10:44 UTC
(In reply to Boris Zbarsky from comment #3)
> Yes, but what will break if the definition of "in a Document" is changed to
> mean that (with s/element/node/, I assume).

Ops. An aggressive idea! I can't predict the impact of this change, however, I like it. :) That should be *no impact*, I hope.

I just thought that there are some cases where we want to distinguish 'in a Document' and 'in a composed tree whose root is a Document'.
Comment 5 Olli Pettay 2014-07-17 09:41:05 UTC
I would start with shadow DOM being not-in-document by default and then case by
case verify that if the host is in doc how various features should work.
Otherwise we may easily accidentally leak internals of shadow dom to the
outside worlds, like in case of https://code.google.com/p/chromium/issues/detail?id=394332
Comment 6 Boris Zbarsky 2014-07-17 12:35:11 UTC
> That should be *no impact*, I hope.

This isn't something to hope; this is something to audit in the specs; comes with changing the fundamental conceptual model...

That said, one of the very first hits on "in a Document" in HTML is in "reset the form owner", and in that case it's not clear to me that things inside a shadow DOM should be considered "in a document".

So it does sound like shadow DOM should not be "in a document" by default but we need a new term X for "light DOM nodes are X if in a document, shadow DOM nodes are X if shadow host is X" that other specs then need to be adjusted to use.  That means filing spec bugs on those specs.

Normally this would happen either during specification or implementation; it's a bit disturbing to me that neither one happened here so far until we started implementing, since it suggests Chrome wasn't actually implementing what the spec says.  :(
Comment 7 Hayato Ito 2014-07-18 03:38:29 UTC
Yet another related issue:
https://code.google.com/p/chromium/issues/detail?id=394295
Comment 8 Hayato Ito 2014-07-18 03:53:31 UTC
One more:
https://code.google.com/p/chromium/issues/detail?id=394307

Once we can resolve the spec bug, blink guys should triage these relevant bugs.
Comment 9 Hayato Ito 2014-07-18 04:40:32 UTC
(In reply to Boris Zbarsky from comment #6)
> > That should be *no impact*, I hope.
> 
> This isn't something to hope; this is something to audit in the specs; comes
> with changing the fundamental conceptual model...
> 
> That said, one of the very first hits on "in a Document" in HTML is in
> "reset the form owner", and in that case it's not clear to me that things
> inside a shadow DOM should be considered "in a document".
> 
> So it does sound like shadow DOM should not be "in a document" by default
> but we need a new term X for "light DOM nodes are X if in a document, shadow
> DOM nodes are X if shadow host is X" that other specs then need to be
> adjusted to use.  That means filing spec bugs on those specs.


Agreed on that we should proceed in a careful way. We must audit the usage of 'in a Document' in specs, right?
That might not be an easy task. However, I am sure we can agree that we should be careful here.


I think we have to distinguish the following cases: 

1. A node is *In a Document*.

2. A node is *in a composed tree whose root is a Document.

3. A node is *In a Document*, but not in a composed tree whose root is a Document.
   (That means there is a state where node is in *1*, but not in *2*, such as a node which is a child of a shadow host, but it's not distributed.)


We might want to have a new TERM x to represents the state of 2 (and 3?).
Comment 10 Hayato Ito 2014-07-18 07:18:49 UTC
I think the previous classification is not good. The following might be better:

(1): A node is *in a Document*. The meaning of this doesn't change.

(2): A node is *in a shadow tree which participates in a tree of trees whose root tree is a document tree*.

(3): (1) or (2). We might want to call it, "A node is in a tree of trees rooted by a Document".

(4). A node is *in a composed tree whose root is a Document*.
     Note that a node can be ((3), but not in (4)) when a node is a child of a shadow host, but the node isn't distributed to anywhere.
Comment 11 Olli Pettay 2014-09-19 11:49:10 UTC
I don't understand the difference between 3 and 4.
"A node is in a tree of trees rooted by a Document" vs
"A node is *in a composed tree whose root is a Document*"
Comment 12 Ian 'Hixie' Hickson 2014-09-20 01:19:07 UTC
First step here seems to me to be to enumerate all the cases we care about.
Comment 13 Hayato Ito 2014-09-22 01:27:00 UTC
(In reply to Olli Pettay from comment #11)
> I don't understand the difference between 3 and 4.
> "A node is in a tree of trees rooted by a Document" vs
> "A node is *in a composed tree whose root is a Document*"

For example, the following codes could be in (3), but not in (4):

- A node is a child node of a shadow host, but the node is not distributed to any insertion points.
- A node is in the older shadow tree, but the younger shadow tree doesn't have <shadow>  element.
- A node is a child of content insertion points.

These nodes couldn't participate in the composed tree.
Comment 14 Hayato Ito 2014-09-22 01:27:58 UTC
> For example, the following codes could be in (3), but not in (4):

s/codes/nodes/
Comment 15 Hayato Ito 2014-09-22 01:34:54 UTC
(In reply to Ian 'Hixie' Hickson from comment #12)
> First step here seems to me to be to enumerate all the cases we care about.

Yeah, I agree.

We have to enumerate all cases where "in a Document" is being used in relevant specs and have to make a decision whether we can leave the usage of "in a Document" as is or replace the usage with better terminology, which we haven't had yet. :(
Comment 16 Olli Pettay 2014-09-22 09:41:16 UTC
(In reply to Hayato Ito from comment #13)
> For example, the following codes could be in (3), but not in (4):
> 
> - A node is a child node of a shadow host, but the node is not distributed
> to any insertion points.
> - A node is in the older shadow tree, but the younger shadow tree doesn't
> have <shadow>  element.
> - A node is a child of content insertion points.
> 
> These nodes couldn't participate in the composed tree.

I see. In Gecko we're going to make, at least for now until the spec
is clarified, nodes in a shadow host, but not distributed, to behave
like it was in a disconnected subtree.
Comment 17 Ian 'Hixie' Hickson 2014-09-26 18:29:23 UTC
So who is going to go through and file the bugs for the relevant cases? I don't know Web Components well enough to do this myself.
Comment 18 Ian 'Hixie' Hickson 2014-09-26 18:36:11 UTC
Seems like this should include cases like how radio buttons pick their radio button group, how scripts decide if they should execute (and if they do, whether document.currentScript is the right API to update when they execute), how labels decide what controls they are labelling, how getElementById() and getElementsByTagName() work, whether iframes should load in shadow trees, what gets included in APIs like document.links, how image maps are found, how target=""s are found, etc.

This seems pretty critical.
Comment 19 Hayato Ito 2014-09-29 06:38:51 UTC
(In reply to Ian 'Hixie' Hickson from comment #17)
> So who is going to go through and file the bugs for the relevant cases? I
> don't know Web Components well enough to do this myself.

I thought I had to do. I guess I'd need help from experienced spec editors.
This issue has been in my TODO list, however, I couldn't spend my time on this at all so far.

I am aware that this is an important issue, however, to be honest, it's likely that I can't start this task in a few weeks.
Comment 20 Hayato Ito 2014-09-30 07:12:10 UTC
Yeah, this is super critical. How should we proceed? I appreciate advice from experienced spec editors.

Here is very rough current plan how to proceed:

1. Define good terminologies, as I described in comment 10, so that we can use these terms to update specs.

   This might be important because well defined terminologies would make it easy to update the spec.
   To define a good terminology set, we might want to pick up some concrete examples from currently filed bugs so that the terminology set is really helpful.

2. Go though specs and file bugs.

  - Which specs should we take care of? DOM and HTML and... maybe all?
  - Should we manage the list of relevant bugs in W3c bugzilla? 
    e.g. One master (meta) bug and other relevant bugs which depends on the meta-bug.

3. Use our best judgement and update the spec in each filed bug.


(In reply to Ian 'Hixie' Hickson from comment #18)
> Seems like this should include cases like how radio buttons pick their radio
> button group, how scripts decide if they should execute (and if they do,
> whether document.currentScript is the right API to update when they
> execute), how labels decide what controls they are labelling, how
> getElementById() and getElementsByTagName() work, whether iframes should
> load in shadow trees, what gets included in APIs like document.links, how
> image maps are found, how target=""s are found, etc.
> 
> This seems pretty critical.
Comment 21 Hayato Ito 2014-10-01 11:37:57 UTC
Let me share my very rough memo as a basis for discussion.

Let's use the following trees as an example:

# Closing tags are being omitted.

HTML:
<body>
  <div id=a class='shadow-host'>
    <div id=b class='distributed-host-child'>
    <div id=c class='distributed-but-not-used-host-child'>
    <div id=d class='not-distributed-host-child'>
  <content id=e class='content-insertion-point-behaves-as-an-unknown-element'>


The node #A hosts three shadow trees as follows, from older to younger:

1. #oldest-shadow-root
  <div id=f class='.not-distributed-to-shadow-insertion-point'>  // #older-shadow-root doesn't have a shadow insertion point.
  <content id=g select='.distributed-but-not-used-host-child'>

2. #older-shadow-root
  <div id=h class='distributed-to-shadow-insertion-point'>

3. #youngest-shadow-root
  <div>
    <shadow id=i>
       <div id=j class='not-used-shadow-child'>
    <content id=k select='.distributed-host-child'>
       <div id=l class='not-used-fallback-element'>
    <content id=m select='#nothing'>
       <div id=n class='fallback-element'>


In addition, suppose that we have *disconnected* nodes as follows:

var o = document.createElement('div');
var p = o.createShadowRoot();


The composed tree will be:

document:
 <body>
    <div id=a class='shadow-host'>
      <div id=h class='distributed-to-shadow-insertion-point'>
      <div id=b class='distributed-host-child'>
      <div id=n class='fallback-element'>
    <content id=e class='content-insertion-point-behaves-as-an-unknown-element'>




We can classify the nodes as follows:

                                              a b c d e f g h i j k l m n o p
A). In a document                             x x x x x
B). In a shadow tree                                    x x x x x x x x x   X
C). In a composed tree (root is a document)   x x     x     x           x
D). In a tree of trees (root is a document)   x x x x x x x x x x x x x x
E). Disconnected (from D)                                                 x x
Comment 22 William Chen 2014-10-02 01:25:14 UTC
It may also be useful to define a new type of parent/child relation for nodes in composed trees and tree of trees. The terms "parent", "child", "ancestor" and "descendant" may not make sense if we change things from "in document" to "in composed tree rooted at document" or "in tree of trees rooted at document".

Just as an example, one of the steps in the "reset the form owner" algorithm [1] involves finding "the nearest such ancestor form element". If we wanted to allow form-associated-element to associate with forms outside the shadow tree, then the current algorithm isn't sufficient because the parent chain terminates at the shadow root and "nearest ancestor" may not be the node you want.

For "in document" and "in shadow tree", using the current terminology is probably fine.

For "in composed tree" we probably need new terms that describe a parent/child relation between shadow host/shadow root and parent of insertion points/nodes distributed into insertion points. Nodes that are distributed but don't match an insertion point should probably have no parent in this relation.

For "in tree of trees" we would probably also need terms to describe parent/child relation between shadow host/shadow root, but ignore insertion points and distribution.

[1] https://html.spec.whatwg.org/multipage/forms.html#reset-the-form-owner
Comment 23 Hayato Ito 2014-10-02 10:49:28 UTC
Good point. Unless otherwise noted, I think parent/child relations for nodes should mean the relation in a document tree or a shadow tree. The meaning shouldn't change.

In regard to 'parent/child relation in composed trees',  I've being using 'a parent node in a composed tree', for example, *unofficially*, in code reviews in chrome. We might want to have a better term.

In regard to 'parent/child relation in a tree of trees',  the situation is not so simple. Although I used a term of 'In a tree of trees (root is a document)' in the previous comment, this is a kind of abbreviation for 'In a node tree which participates in a tree of trees whose root tree is a document tree'.

The participant of 'tree of trees' is not a node. That's a node tree.

You might want to know that the spec already have terms, 'child or hosted shadow root' [1] and 'deep descendant' [2].
I've introduced that when I tried to introduce 'deepContains' [3].
I am aware that this definition doesn't define the tree order between sibling nodes in a *tree of trees* yet.

[1]: http://w3c.github.io/webcomponents/spec/shadow/#dfn-child-or-hosted-shadow-root
[2]: http://w3c.github.io/webcomponents/spec/shadow/#dfn-deep-descendant
[3]: That's reverted in https://github.com/w3c/webcomponents/commit/463916e5873e99e43136ef4fe7e75b4da4c722aa
Comment 24 Hayato Ito 2014-10-28 02:08:58 UTC
Yet another relevant issue in chromium bug tracker: https://code.google.com/p/chromium/issues/detail?id=426050
Comment 25 Boris Zbarsky 2014-10-28 02:36:23 UTC
That's covered by bug 26943, which is blocked on this bug...  that was clearly stated in the chromium bug you reference.
Comment 26 Hayato Ito 2014-10-31 04:55:43 UTC
It seems there are a lot of things to todo, however, I think

> D). In a tree of trees (root is a document)   x x x x x x x x x x x x x x

defining D (with a good naming) clearly in the spec and replace 'In a document' with D where we need to update is a low-hanging fruit and good starting point for us.
Comment 27 Olli Pettay 2014-10-31 11:00:36 UTC
Do f and g actually belong to D. I'd expect f and g to behave as if they were
in a disconnected subtree which has ShadowRoot as the root.
f and g after all don't affect to layout nor participate event propagation path.
Comment 28 Hayato Ito 2014-11-06 22:06:18 UTC
I am okay that we should exclude f and g from *D*.

That made me think about yet another case. Suppose that node #c (or #d) has a shadow tree, do we consider the shadow tree hosted by #c (or #d) *disconnected*?

That means the shadow host, #c, itself is not disconnected, but the nodes in the shadow tree hosted by the shadow host, #c, are disconnected. Is it weird?


If you notice any other missing case, please let me know that.


BTW, if someone has a good naming candidate for *D*, it's highly welcome.
Comment 29 Olli Pettay 2014-11-07 10:17:14 UTC
(In reply to Hayato Ito from comment #28)
> That made me think about yet another case. Suppose that node #c (or #d) has
> a shadow tree, do we consider the shadow tree hosted by #c (or #d)
> *disconnected*?

The nodes in the shadow tree of #c or #d would be in a composed tree (with document as root), since the host is such tree. 
So the shadow dom would be "in composed document" (D).
Comment 30 Hayato Ito 2014-11-10 01:46:23 UTC
No, the actual is:

- #c (or #d) is *not* in the composed tree (with document as root).

  These nodes are child nodes of the shadow host (#a), however, they are not distributed at all.

- However, #c (or #d) is "in a document".


A "composed document" (D) might be a good name for D, but I'm afraid that it is confusing name for D because #c (or #d) must belong to D, but #c (or #d) is not in the composed tree (with document as root).

I think "composed document" might be a good name for "C", "composed tree (with document as root), rather.
Comment 31 Hayato Ito 2014-11-10 01:49:24 UTC
A small correction. #c is actually distributed, but it is not in a composed tree (with document as root) because the destination insertion point of #c is in the oldest shadow tree, which is *disconnected*.

(In reply to Hayato Ito from comment #30)
> No, the actual is:
> 
> - #c (or #d) is *not* in the composed tree (with document as root).
> 
>   These nodes are child nodes of the shadow host (#a), however, they are not
> distributed at all.
> 
> - However, #c (or #d) is "in a document".
> 
> 
> A "composed document" (D) might be a good name for D, but I'm afraid that it
> is confusing name for D because #c (or #d) must belong to D, but #c (or #d)
> is not in the composed tree (with document as root).
> 
> I think "composed document" might be a good name for "C", "composed tree
> (with document as root), rather.
Comment 32 Hayato Ito 2014-11-10 02:10:52 UTC
I forgot to express my opinion. The shadow trees hosted by #c (or #d) should be in category D. However, I might change my mind. That's hard for me to categorize it.

This might be a minor issue. What do you think?

Let me make a formal definition for D. That might be helpful for the discussion. I hope a formal definition would help me to understand the problem.
Comment 33 Hayato Ito 2014-11-10 04:23:13 UTC
How about the following definition for category D?


A node, |A|, is 'in a document deeply' (a tentative name until we have a better one) if |A| meets at least one of the following conditions:

- |A| is in a document
- |A| is in the youngest shadow tree, |T|, and the shadow host which hosts |T| is 'in a document deeply'
- |A| is in an older shadow tree, |T|, and the younger shadow tree relative to |T| has a shadow insertion point, |I|, and |I| is 'in a document deeply'
Comment 34 Hayato Ito 2014-11-10 06:08:11 UTC
Yet another idea:

  A node, |A| is 'in a document deeply' if |A| is *inclusive deep descendant* [1] of a document.

That would make things simpler, doesn't that?


[1]: http://w3c.github.io/webcomponents/spec/shadow/#dfn-inclusive-deep-descendant
Comment 35 Olli Pettay 2014-11-10 13:25:26 UTC
(In reply to Hayato Ito from comment #32)
> I forgot to express my opinion. The shadow trees hosted by #c (or #d) should
> be in category D.
So we agree here. I said (D) in comment 29.
Comment 36 Hayato Ito 2014-11-11 06:32:07 UTC
Thanks. If we agree that we should include shadow trees hosted by #c in 'D' (aka 'in a document deeply), I am wondering what is the difference between:

- the shadow tree hosted by #c
and
-  the oldest shadow tree in the example.

Any node in either tree isn't used in rendering at all (aka they are not in the composed tree (with document as root)), however,
- Nodes in the shadow tree hosted by #c *are* 'in a document deeply',
- Nodes in the oldest shadow trees, such as #f and #g, *are not* 'in a document deeply'.

That looks inconsistent to me.

Now I am thinking that the proposal in comment #34 is more consistent and even *better* than the proposal in comment #33 in most cases.
You can see the difference between the two proposals bellow (in the summary). Node #f and #g, which are in the oldest shadow tree, are the difference.


Here is the rationale why I thought proposal 2 (comment #34) is helpful than proposal 1 (comment #33):

I think 'nodes aren't contributing rendering' doesn't matter here because it's like a node with *display: none* in a document.
My design principle of shadow trees so far is that shadow trees are mainly used to construct *one dom tree* so that we can separate different concerns to each shadow trees.
Each node in shadow trees are 'in a document', conceptually. Although some nodes don't participate in the composed tree after the composition algorithm, I think we should consider such nodes like a node marked with 'display: none'. That's still *in the document* (conceptually, again).


Note that if we try to consider the nodes which doesn't participate in the composed tree are not 'in a document deeply', that will fail because node #c (or #d) is in a document, but it doesn't participate in the composed tree (with document as root). We can't adapt this idea because it would break backward-compatibility.



The summary is:


Proposal 1 (in comment #33):

A node, |A|, is 'in a document deeply' if |A| meets at least one of the following conditions:

- |A| is in a document
- |A| is in the youngest shadow tree, |T|, and the shadow host which hosts |T| is 'in a document deeply'
- |A| is in an older shadow tree, |T|, and the younger shadow tree relative to |T| has a shadow insertion point, |I|, and |I| is 'in a document deeply'

The result would be:
(We use the example tree of trees in comment #21 again here. In addition to that, suppose that #c hosts a shadow tree, which contains the node |q|)

                                              a b c d e f g h i j k l m n o p q
A). In a document                             x x x x x
B). In a shadow tree                                    x x x x x x x x x   X X
C). In a composed tree (root is a document)   x x     x     x           x
D). In a document deeply                      x x x x x     x x x x x x x     X
E). Disconnected (from D)                               x x               x x



Proposal 2 (in comment #34):

A node, |A| is 'in a document deeply' if |A| is *inclusive deep descendant* [1] of a document.

[1]: http://w3c.github.io/webcomponents/spec/shadow/#dfn-inclusive-deep-descendant


The result would be:

                                              a b c d e f g h i j k l m n o p q
A). In a document                             x x x x x
B). In a shadow tree                                    x x x x x x x x x   X X
C). In a composed tree (root is a document)   x x     x     x           x
D). In a document deeply                      x x x x x x x x x x x x x x     X
E). Disconnected (from D)                                                 x x


If you find any mistake on this summary or any other ideas, please let me know that.
I'd like to make a decision and add the definition of 'in a document deeply' (a better name is still wanted) in the spec.
Comment 37 Olli Pettay 2014-11-11 12:22:18 UTC
(In reply to Hayato Ito from comment #36)
> Thanks. If we agree that we should include shadow trees hosted by #c in 'D'
> (aka 'in a document deeply), I am wondering what is the difference between:
> 
> - the shadow tree hosted by #c
> and
> -  the oldest shadow tree in the example.
> 
> Any node in either tree isn't used in rendering at all (aka they are not in
> the composed tree (with document as root)), however,
> - Nodes in the shadow tree hosted by #c *are* 'in a document deeply',
> - Nodes in the oldest shadow trees, such as #f and #g, *are not* 'in a
> document deeply'.
> 
> That looks inconsistent to me.
Not to me.
In my mind only the youngest shadow tree is in the documented rooted composed tree if the host is in document rooted composed tree.
Older shadow trees are in document rooted composed tree only if they are attached to a shadow insertion point which is in
a document rooted composed tree. 
I think this would be a quite simple way to spec this

> I think 'nodes aren't contributing rendering' doesn't matter here because
> it's like a node with *display: none* in a document.
true


> Each node in shadow trees are 'in a document', conceptually.
Not quite, since the host might not be in a document.
Comment 38 Hayato Ito 2014-11-12 04:46:36 UTC
(In reply to Olli Pettay from comment #37)
> (In reply to Hayato Ito from comment #36)
> > Thanks. If we agree that we should include shadow trees hosted by #c in 'D'
> > (aka 'in a document deeply), I am wondering what is the difference between:
> > 
> > - the shadow tree hosted by #c
> > and
> > -  the oldest shadow tree in the example.
> > 
> > Any node in either tree isn't used in rendering at all (aka they are not in
> > the composed tree (with document as root)), however,
> > - Nodes in the shadow tree hosted by #c *are* 'in a document deeply',
> > - Nodes in the oldest shadow trees, such as #f and #g, *are not* 'in a
> > document deeply'.
> > 
> > That looks inconsistent to me.
> Not to me.
> In my mind only the youngest shadow tree is in the documented rooted
> composed tree if the host is in document rooted composed tree.
> Older shadow trees are in document rooted composed tree only if they are
> attached to a shadow insertion point which is in
> a document rooted composed tree. 

This is exactly what I thought when I saw your comment in #27. 
After that, I've changed my mind because I found the inconsistency, as I explained in comment #36. 
Could you have a closer look at comment #36? I'd like to understand why you don't think this is inconsistent.

It's not reasonable if we exclude only such an older shadow root, while including the shadow tree hosted by #c only for the reason that one is an older shadow tree and one is the youngest shadow tree. 
Both shadow trees are in the same category in a sense that neither contribute to the document-rooted composed tree at all.

Basically, I think the distribution result shouldn't have any effect to 'in a document'-ness.
'In a document'-ness should be purely determined by the structure of the tree of trees.

In other words, adding/removing a <shadow> element somewhere in the tree of trees shouldn't have any effect of 'in a document'-ness for other nodes in the tree of trees.

I guess there are two different kind of views for 'in a document'-ness in the shadow dom world.

A). Focusing a *static* structure of a tree of trees.
B). Focusing a dynamic structure. That's the document-rooted composed tree, which is the result of the distribution algorithm and the composition algorithm.

I think you have been focusing on B, but I'd like to focus on A here.
B will be likely to cause an inconsistency and make things complex. Unless we can find a serious issue in A, I'd like to go for A here.


> > Each node in shadow trees are 'in a document', conceptually.
> Not quite, since the host might not be in a document.

Yeah, I shouldn't have omitted 'as long as the host is in a document deeply' here.
Comment 39 Boris Zbarsky 2014-11-12 10:05:10 UTC
This comes down to questions like whether a non-distributed <iframe> should load its document, whether a non-distributed <style> should affect the styles of nodes, whether a non-distributed <link rel="stylesheet"> should load a stylesheet, whether a non-distibuted <embed> that was at one point distributed should continue running the plugin it loaded, whether a media element that stops being distributed should pause, whether non-distributed form controls should associate to forms, what should happen with showModal on a non-distributed <dialog>, whether a <script> inserted under a non-distributed node should run, etc.

Note that it might turn out that the answers are different in those various different situation, in that some of them want non-distributed elements to act as if they were in a document and others don't...
Comment 40 Olli Pettay 2014-11-12 15:13:49 UTC
(In reply to Hayato Ito from comment #38) 
> It's not reasonable if we exclude only such an older shadow root, while
> including the shadow tree hosted by #c only for the reason that one is an
> older shadow tree and one is the youngest shadow tree. 
Why it is not reasonable?

> Both shadow trees are in the same category in a sense that neither
> contribute to the document-rooted composed tree at all.
Well, youngest tree does always contribute to the composed tree of rooted by its host.
Older, not distributed to any shadow insertion points are different.

> 
> Basically, I think the distribution result shouldn't have any effect to 'in
> a document'-ness.
> 'In a document'-ness should be purely determined by the structure of the
> tree of trees.

'in a document' -ness doesn't affect to the distribution.
Perhaps I should rephrase a bit. It is not only about the in-a-document, but about the composed tree rooted to a host
(and if that host is in document, that composed tree is in a composed tree which has document as root.).


> A). Focusing a *static* structure of a tree of trees.
> B). Focusing a dynamic structure. That's the document-rooted composed tree, which is the result of the distribution algorithm 
>     and the composition algorithm.

> I think you have been focusing on B, but I'd like to focus on A here.
> B will be likely to cause an inconsistency and make things complex. Unless we can find a serious issue in A, I'd like to go for 
> A here.
Yeah, I'm thinking about B) and how event handling and such behave in the whole setup.


However, I think I could live with A too. I'm certainly not strongly against it :)
But let me think this a bit, like this evening.
Comment 41 Olli Pettay 2014-11-12 17:45:22 UTC
So I think if one overrides existing shadow dom of an element without
providing shadow insertion point for the old shadow tree, I would expect the 
elements in that older shadow tree to stop behaving as if they where
still in document. Say, you have an <iframe> there, and the younger 
shadow tree just wants to override all that with something more lightweight
(and not provide <shadow>). I think the <iframe> should be unloaded as if it was
removed from document.
If one wants to keep the old stuff working, then just explicitly provide shadow insertion point.

And still, I could live with (A), but it feels a bit odd to keep the older
shadow working while one probably wants to override it all with something else.

#c case is different. There you have an element #c which is in document already.
(so if #c is an iframe, it does load a new document). So if you know create a shadow root for #c and insert <iframe> to it, why wouldn't it load.
Comment 42 Olli Pettay 2014-11-12 18:21:42 UTC
Though, there is then the issue that if
the old shadow dom has <iframe> and a new one is created, before the
new one has any content, the <iframe> from the older one would be unloaded.

To make that work properly there should be a way to create a shadow dom with one function call.
Something like
element.createShadowRoot("<iframe></iframe><span>I'm in shadow dom</span>");
or
element.createShadowRoot(someOtherElementWhichWillBeAppendedToTheShadowRoot);
Comment 43 Hayato Ito 2014-11-13 07:27:32 UTC
(In reply to Olli Pettay from comment #41)
> So I think if one overrides existing shadow dom of an element without
> providing shadow insertion point for the old shadow tree, I would expect the 
> elements in that older shadow tree to stop behaving as if they where
> still in document. Say, you have an <iframe> there, and the younger 
> shadow tree just wants to override all that with something more lightweight
> (and not provide <shadow>). I think the <iframe> should be unloaded as if it
> was
> removed from document.

Yeah, that's a good point and that's one of the potential issue in idea A.

I think the concern comes from the fact that there is no way to get rid of the existing old shadow trees completely from the shadow host in the current API set of Shadow DOM.
The reason we don't have such an API is that I've not heard the requirement for the removal, AFAIK.
I guess much attention has not been paid to such a wasteful resource, which would be consumed by *overridden* shadow trees, because that's a rare use case.

I'm okay to add such an API if this would be a non-ignorable use case and developers would start to complain about this kind of wastefulness.


(In reply to Boris Zbarsky from comment #39)
> This comes down to questions like whether a non-distributed <iframe> should
> load its document, whether a non-distributed <style> should affect the
> styles of nodes, whether a non-distributed <link rel="stylesheet"> should
> load a stylesheet, whether a non-distibuted <embed> that was at one point
> distributed should continue running the plugin it loaded, whether a media
> element that stops being distributed should pause, whether non-distributed
> form controls should associate to forms, what should happen with showModal
> on a non-distributed <dialog>, whether a <script> inserted under a
> non-distributed node should run, etc.
> 
> Note that it might turn out that the answers are different in those various
> different situation, in that some of them want non-distributed elements to
> act as if they were in a document and others don't...

Yeah, that's a good point. That should be always on my radar, however, it seems I forgot to mention that in my recent posts.

The basic idea in my head is:
  - We should treat such elements so that we got the equivalent result to "They were in a document (in a traditional meaning), but it is marked 'display: none'.

For example:

- As for <script> element, we have to run the <script> 'in a document deeply, but not in the document-rooted composed tree', as long as <script> in a document runs even if it is marked with 'display: none'.
- As for <iframe> element, we have to load the resources from <iframe> 'in a document deeply, but not in the document-rooted composed tree' as long as <iframe> 'in a document' loads resources even if this is marked with 'display: none'.

Of course, there should be a exception. For example, the current spec disables the capability of the <link> and <base> elements in shadow trees as if they were in a document fragment. The answers should be different in those various different situation as you mentioned.

I'd expect the my basic idea covers the most cases, hopefully.
Comment 44 Olli Pettay 2014-11-13 11:27:34 UTC
(In reply to Hayato Ito from comment #43) 
> I'm okay to add such an API if this would be a non-ignorable use case and
> developers would start to complain about this kind of wastefulness.

Yeah, ok, perhaps (A) for this case, and possibly add
createAndReplaceExistingShadowRoot() in the future.
Comment 45 Boris Zbarsky 2014-11-13 16:32:48 UTC
>  - We should treat such elements so that we got the equivalent result to "They
> were in a document (in a traditional meaning), but it is marked 'display: none'.

This is the part that's non-obvious to me, frankly.  I mean, obviously if we treat them as being in a document they should be treated the same way as display:none.  But the premise that they should be treated as in a document is not a given.

> Of course, there should be a exception.

What's needed is a concrete list of those exceptions (in either direction).  Starting with the list Chrome implements would be a good start, say, because then we at least have something to discuss.
Comment 46 Hayato Ito 2014-11-14 08:00:45 UTC
Yeah, I think it's time to take a closer look at each case.

For example, in the HTML Living Standard, #in-a-document is referenced in:

2.1.3 DOM trees
3.1.3 DOM tree accessors
4.2.6 The style element
4.2.7 Interactions of styling and scripting
4.6.5.14 Link type "stylesheet" (2) (3) (4) (5)
4.8.5 The img element
4.8.7 The embed element (2)
4.8.8 The object element
4.8.14.8 Playing the media resource (2)
4.8.14.11.3 Assigning a media controller declaratively
4.10.18.3 Association of controls and forms (2)
4.10.19.8.3 User interface for bulk autofill
4.11.6.1 Facets
4.11.7 The dialog element
4.12.1 The script element (2) (3)
6.2 Inert subtrees
6.5.3 Processing model
7.1.1 Nested browsing contexts (2)
7.2.3 Accessing other browsing contexts (2)
15.3.1 The applet element
15.3.3 Frames


I've taken a quick look at each usage:

4.2.6 The style element
> If element is not in a Document, then abort these steps.

4.2.7 Interactions of styling and scripting
> ... last time the event loop reached step 1, the element was in that Document, and the user agent hasn't given up on that particular style sheet yet.

4.6.5.14 Link type "styleseet"
> When the external resource link is created on a link element that is already in a Document.

4.8.5 The img element
> While a user agent is running the above algorithm for an element x, there must be a strong reference from the element's node document to the element x, even if that element is not in its Document.

4.8.6 The iframe element
> For example if the user agent supports listing all the links in a document, links in "seamlessly" nested documents would be included in that list without being significantly distinguished from links in the document itself.

4.8.7 The embed element
> ... The element is in a Document or was in a Document the last time the event loop reached step 1.

4.8.8 The object element
> ... If the element has an ancestor media element, or has an ancestor object element that is not showing its fallback content, or if the element is not in a Document with a browsing context, o

4.8.14.8 Playing the media resource
> .. Media elements that are potentially playing while not in a Document must not play any video, but should play any audio component.

4.8.14.11.3 Assigning a media controller declaratively
> If there is another media element whose Document is the same as m's node document (even if one or both of these elements are not actually in the Document), and which also has a mediagroup attribute, and whose mediagroup attribute has the same value as the new value of m's mediagroup attribute, then let controller be that media element's current media controller.

4.10.18.3 Association of controls and forms
> ... If the element is reassociateable, has a form content attribute, and is itself in a Document, then run these substeps:

4.10.19.8.3 User interface for bulk autofill
> ... form is no longer in a Document

4.11.6.1 Facets
> The element is in a Document that has an associated browsing context.

4.11.7 The dialog element
> If subject is not in a Document, then throw an InvalidStateError exception and abort these steps.

4.12.1 The script element
> The script element is in a Document and a node or document fragment is inserted into the script element, after any script elements inserted at that time.
> The script element is in a Document and has a src attribute set where previously the element had no such attribute.

6.2 Inert subtrees
> While a browsing context container is marked as inert, its nested browsing context's active document, and all nodes in that Document, must be marked as inert.

6.5.2 The accesskey attribute
6.5.3 Processing model
> When the user presses the key combination corresponding to the assigned access key for an element, if the element defines a command, the command's Hidden State facet is false (visible), the command's Disabled State facet is also false (enabled), the element is in a Document

7.1.1 Nested browsing contexts
> Certain elements (for example, iframe elements) can instantiate further browsing contexts. These are called nested browsing contexts. If a browsing context P has a Document D with an element E that nests another browsing context C inside it, then C is said to be nested through D, and E is said to be the browsing context container of C. If the browsing context container element E is in the Document D

7.2.3 Accessing other browsing contexts
> The length IDL attribute on the Window interface must return the number of child browsing contexts that are nested through elements that are in the Document that is the active document of that Window object,

8.1.5.4 Events and the Window object
> When an event is dispatched at a DOM node in a Document in a browsing context, if the event is not a load event, the user agent must act as if, for the purposes of event dispatching, the Window object is the parent of the Document object. [DOM]

15.3.1 The applet element
> The element is not in a Document.

15.3.3 Frames
> A frame element is said to be an active frame element when it is in a Document.



In addition to 'in a Document', I think we need to have better words in replace of 'inserted into a document' and 'removed from a document' in some places.

'inserted into a document' is referenced in:

4.2.5.3 Pragma directives (2)
4.2.6 The style element
4.6.5.14 Link type "stylesheet"
4.8.6 The iframe element
4.8.8 The object element
4.10.18.3 Association of controls and forms (2)
4.10.19.6.1 Autofocusing a form control: the autofocus attribute (2)
4.12.1 The script element (2)
7.6.3 Page load processing model for XML files (2)
8.1.4.1 Definitions
8.1.4.3 Generic task sources


'removed from a document' is referenced in:
4.2.6 The style element
4.8.6 The iframe element
4.8.8 The object element
4.8.14.8 Playing the media resource
4.8.14.19 Best practices for implementors of media elements
4.10.18.3 Association of controls and forms
4.11.7 The dialog element


Note that this is just a quick check. I have to do take another closer look in the next iteration.
Comment 47 Hayato Ito 2014-11-14 08:18:10 UTC
Here is the filed chrome bugs:

- iframe: https://code.google.com/p/chromium/issues/detail?id=393350
- script: https://code.google.com/p/chromium/issues/detail?id=394327
- accesskey: https://code.google.com/p/chromium/issues/detail?id=394295
- label: https://code.google.com/p/chromium/issues/detail?id=394307
- doucment.currentscript: https://code.google.com/p/chromium/issues/detail?id=394332
- attachedCallback (custom elements): https://code.google.com/p/chromium/issues/detail?id=426050



I think the intentional exception in the current blink's implementation is <base> and <link> elements. See 7.1 Inert HTML Elements: http://w3c.github.io/webcomponents/spec/shadow/#inert-html-elements

I think the paragraph in the Shadow DOM spec: 

> 7.1 Inert HTML Elements
> All other HTML elements in the shadow trees must behave as if they were part of the document tree.

must be updated. Further more, we can remove this 7.1 section from the Shadow DOM spec if we finish all tasks here, hopefully.
Comment 48 Hayato Ito 2014-11-19 05:09:26 UTC
*** Bug 25562 has been marked as a duplicate of this bug. ***
Comment 49 Hayato Ito 2014-11-19 07:47:31 UTC
I've defined 'in a document deeply' (as a tentative name) in the Shadow DOM spec.
https://github.com/w3c/webcomponents/commit/7b4bdc8d0d2d3cce0edaf1f1209b8805624e50b8

Now it's time to take a closer look at each case. I think <style> [1] is a good starter case for us.

Should we file a bug for HTML living standard [2] for each case and continue discussion there for each case?

If there is a need to discuss something in general further, we could use this thread anytime.

[1] https://html.spec.whatwg.org/multipage/semantics.html#the-style-element
[2] https://www.w3.org/Bugs/Public/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=HTML&list_id=47544&product=WHATWG
Comment 50 Boris Zbarsky 2014-11-19 14:34:31 UTC
> Should we file a bug for HTML living standard [2] for each case

That seems like a good idea, yes.
Comment 51 Dimitri Glazkov 2014-11-19 15:46:46 UTC
(In reply to Boris Zbarsky from comment #50)
> > Should we file a bug for HTML living standard [2] for each case
> 
> That seems like a good idea, yes.

Another thing that I ran by Hixie is just creating a patch against the HTML spec mirror (https://github.com/whatwg/html-mirror) -- if that's more lightweight.
Comment 52 Hayato Ito 2014-11-20 07:22:10 UTC
(In reply to Dimitri Glazkov from comment #51)
> (In reply to Boris Zbarsky from comment #50)
> > > Should we file a bug for HTML living standard [2] for each case
> > 
> > That seems like a good idea, yes.
> 
> Another thing that I ran by Hixie is just creating a patch against the HTML
> spec mirror (https://github.com/whatwg/html-mirror) -- if that's more
> lightweight.

Thanks. I like it. It's a convenient way for me to create a patch, on which we could discuss.
Comment 53 Ian 'Hixie' Hickson 2014-11-21 23:25:51 UTC
Thanks for the list above, Hayato.

When considering these, one should also consider:
- a node A that's a child of an element that has a shadow tree that doesn't have an insertion point for A.
- a node A that's a grandchild of an element that has a shadow tree that doesn't have an insertion point for A.
- a node that's in a shadow tree of an element B that itself is a child of an element with a shadow tree that doesn't have an insertion point for B.
- a node that is display:none.
- an orphan node.
- a node in a shadow tree of an orphan node.
- a node that is being rendered
- a node that is not being rendered but that is in the composed tree

In general, I think we want to have as few categories of elements here as possible. In particular, I think "being rendered vs not being rendered", "in the document deeply vs not", "in the document directly vs not", and "scoped to the current subtree" are reasonable categories.

So it seems to me that for the above we should go as follows:

"In the document": 
  <link> (e.g. you don't want rel=import working from inside a shadow tree)

"Being rendered":
  <embed>
  <object>
  focusable
  <applet>

"Scoped to the current subtree":
  media controllers
  form control associations

"In the document deeply":
  <script>
  <command> exposure
  accesskey
  <dialog>
  autofocus
  <iframe>, <frame>
  window.length
  inertness of browsing context container
  <style>
  <video> autopause
  whether autofill is canceled

How does that sound?
Comment 54 Ian 'Hixie' Hickson 2014-11-21 23:30:10 UTC
I filed bug 26365 to track the HTML spec side of this.
Comment 55 Boris Zbarsky 2014-11-22 02:21:08 UTC
>"Being rendered":
>  <embed>
>  <object>
>  focusable
>  <applet>

Sadly, no.  These want different behavior in the "removed from document" and "display becomes none" cases, iirc.  I think they may also want different behavior in the "display has been none all along" and "display becomes none" cases...

The rest sounds pretty reasonable at first glance except perhaps window.length and named/indexed access on the window; that probably wants to be scoped to the current subtree.
Comment 56 Hayato Ito 2014-12-03 11:58:48 UTC
Sorry for the late replay.
We've raised the priority of this issue P1. Let me take a closer look in a few days.
Comment 57 Hayato Ito 2014-12-11 06:43:38 UTC
Thank you, Hixie. I appreciate your proposal.

I've taken a closer look at each case. Let me add inline comments.

(In reply to Ian 'Hixie' Hickson from comment #53)

> In general, I think we want to have as few categories of elements here as
> possible. In particular, I think "being rendered vs not being rendered", "in
> the document deeply vs not", "in the document directly vs not", and "scoped
> to the current subtree" are reasonable categories.

Totally agreed. I'd like to minimize the number of 'primitives' to explain the Web platform.
That's one of the motivations I tried to introduce categories.

> 
> So it seems to me that for the above we should go as follows:
> 
> "In the document": 

Yeah, "In the document" matches the "category A" in the comment #21.

>   <link> (e.g. you don't want rel=import working from inside a shadow tree)

+1. <link> is also mentioned explicitly in the Shadow DOM spec (http://w3c.github.io/webcomponents/spec/shadow/#inert-html-elements). We should upstream it to the HTML standard.


> "Being rendered":

Oh, I am glad to notice that 'Being rendered' is clearly defined in the HTML standard. I should have used this term in this discussion, instead of 'display: none'.

Category C, 'An element is in the composed tree (with document as root)', is a necessary condition for 'Being rendered'. I'd like to mention that "An Element which is not in Category C doesn't have any associated CSS layout boxes" in the Shadow DOM spec, though the css-scoping spec implies that (http://drafts.csswg.org/css-scoping/#inheritance), I think. We might want to clarify that.


The current Shadow DOM spec, 2.4 Composed Trees (http://w3c.github.io/webcomponents/spec/shadow/#composed-trees), also says:

> In rendering a document tree, or presenting it visually, the composed tree must be used instead of the document tree.
> The composed tree must be updated before the rendering occurs.

Now I am feeling this is redundant. To be honest, I don't like this paragraph because it is not well defined (I should be blamed).
Instead of that, I think what we have to say is:

1. CSS inheritance should be resolved based on relationship in the composed tree, as the current css-scoping spec says.
2. If an element is not in the composed tree (with document as root), the element doesn't have any associated CSS layout boxes.

This looks enough to me, doesn't that?



>   <embed>
>   <object>
>   focusable
>   <applet>

I don't have strong opinion for these except for *focusable*.

'focusable' is also mentioned in the Shadow DOM spec:

http://w3c.github.io/webcomponents/spec/shadow/#focus-navigation
>  If a node doesn’t participate in the composed tree, the node must be skipped from the navigation order [CSS3UI] sequence.

We can also rewrite this definition by using 'Being Rendered'. We don't have to use the composed tree here.


> 
> "Scoped to the current subtree":
>   media controllers

+1.
I am not confident about what is desired behavior for media controllers association. However, limiting the association to the current scope sounds a good starting point.

>   form control associations

+1.
I think "Scoped to the current subtree" are implicit here unless otherwise mentioned.
In the past, the Shadow DOM spec mentioned explicitly <form> control association doesn't cross Shadow DOM boundaries, but it was removed from the spec because we thought it was implicit because each node tree should be considered 'separated', unless otherwise mentioned, basically.

> 
> "In the document deeply":
>   <script>
>   <command> exposure
>   accesskey

+1 with a nit.

As for <script>:
We don't have an agreement on what document.currentScript should return. In other words, currenetScript should be scoped or not.
If we prefer scoped here, we might want to let ShadowRoot to have ShadowRoot.currentScript.


As for accesskey:
In the current implementation of blink, accesskey is not *scoped*. There is one global accesskey map which is shared among every node trees which participate in the same tree of trees.
Is there any spec for the conflict case? I thought that we should define how to resolve the case where two elements use the same access key. In other words, we might want to define a *document* order between nodes in the same tree of trees.


>   <dialog>
>   autofocus
>   <iframe>, <frame>
>   window.length
>   inertness of browsing context container
>   <style>
>   <video> autopause
>   whether autofill is canceled

+1 basically.

Note that we should also mention that {Document,ShadowRoot}.styleSheets property is *scoped*.
Comment 58 Hayato Ito 2014-12-12 05:49:14 UTC
As an experiment, may I write a patch as a pull-request to https://github.com/whatwg/html-mirror?

A pull-request is not intended to be merged. I just would like to know how things go well if we are actually to write a patch for each case.

A patch for <style> element would be a good starting point for me.
Comment 59 Anne 2015-04-23 19:01:49 UTC
*** Bug 22361 has been marked as a duplicate of this bug. ***
Comment 60 Hayato Ito 2015-05-27 03:25:17 UTC
Moved to https://github.com/w3c/webcomponents/issues/57