(In reply to comment #0)
> When a script element node is created, if it is being flagged as
> parser-inserted, set its force-async flag to false. Otherwise, set its
> force-async flag to true. (Note that createContextualFragment, innerHTML and
> XSLTProcessor::transformToFragment-created scripts are not flagged as
> parser-inserted.) This flag setting happens before any attributes (even
> parser-set ones) are set on the node (so a fragment parser-set async attribute
> may modify the flag shortly after).

That doesn't really make sense. The elements created by the parser are created with their attributes already set. But I guess we can get the same effect in a different way.


Overall this proposal seems to add a lot of complexity without a really good argument. Are there really that many sites that depend on this? What are other UAs intending to do here?

For what it's worth, my preference is to make .async default to false for elements created through the DOM. And make .async equals false mean that elements execute in order.

This would mean that <script> elements created through the DOM would by default execute in order.

@Ian-

> Are there really that many sites that depend on this?

The question you are asking if fair, but it's actually a little bit the wrong question. Let me explain:

It is undeniable that the vast majority of existing web sites (~95% or greater?) rely on ordering behavior of markup script tags to ensure that if they include something like "jquery.js" before "jquery-ui.js", that even if the browser is smart and downloads them in parallel (which all modern browsers do), it will execute them in insertion-order, so that dependencies are taken care of.

This is so common across all browsers and most sites, the "async" attribute was added specifically to allow markup script tag authors the ability to say that a certain script DOESN'T have any execution-order dependencies and should therefore be executed "as soon as possible". While we've seen a fair uptick on the web of authors using the "async" attribute on some script tags, there remains a large chunk of web scripts that are still being loaded in such a way that they execute in insertion-order.

The fact is this: the use-cases for "insertion order execution" AND "as soon as possible execution" are both valid and quite prevalent on the web.

While we may argue that scripts should never have been designed to rely on execution order, the fact is that millions of scripts and 10's of millions of sites DO that, and rely on it. That isn't going to change any time soon. 

**** So to answer the spirit of your original question, there are LOTS of sites that rely on insertion-order execution. But most of them right now do so with markup script tags, and NOT by using script loaders. ****

HOWEVER, in the last year or two especially, but even well before that in certain frameworks, the use of "script loaders" has begun to pop up more and more frequently. There are several huge noteable sites that are beginning to employ script loaders, including Twitter, Zappos, Vimeo, Mefeedia, Hotmail, and many others. They are doing so because they realize several advantages over regular script markup tags.

Web Performance Optimization experts note a variety of different reasons why dynamic script loaders offer performance optimization benefits over normal script tags (even markup script tags where the browser loads in parallel).

Some advantages to dynamic script loaders:

1. Markup script tags *block* many other page-load activities, like downloading/rendering of images/css/etc, DOMContentLoaded/DOM-ready events, etc. Even though they may load in parallel to each other, because of document.write(), the browser must be pessimistic and must force everything else on the page to wait in case a document.write() is found in one of the scripts.

2. There are potentially HUGE performance advantages to on-demand or lazy-loading techniques, where a page-load can be optimized by reducing to the bare minimum (or none at all!) the scripts that are loaded during the initial page-load, and instead putting off until later the loading (or execution) of such scripts. By getting content in front of users quicker, the perception of page-load speed is greatly improved, which improves user satisfaction on sites.

"defer" helps a little, but there's lots of use-cases that are beyond the capability of "defer", and thereofre require a script-loader.

Even if you ignore the performance benefits that dynamic script loaders give sites, there are other use cases why they are important, including module dependency management, etc.

----------------
So, here's the problem: currently, as the spec reads, a script loader CANNOT load scripts in parallel (like the browser can with markup script tags) and *also* execute them in insertion order. 

To TRY do so, many script loaders, including mine (LABjs), have resorted to ugly and brittle hacks, like cache-preloading. These hacks are obviously unreliable as browsers continue to change and evolve. If a script loader doesn't use these hacks (for fear of browser breakage), they are left with ONLY one choice. They *must* load scripts one-at-a-time (serially) to force the insertion order execution.

This creates a HUGE frustrating catch-22 for web page authors. They want to improve the performance of their sites, so they use a script loader. But the script loader now can't load in parallel if it needs to preserve execution order. So it has to go back to the really old (IE6/7, FF3) days of serial one-at-a-time loading, which kills all their performance optimization. 

In fact, they quickly realize that in that situation, they'd be better off ditching the script loader and just using regular markup script tags. So we're right back to where we started. 

OR, I am forced to re-organize all of my script content. I'm forced to rewrite scripts so they don't have execution-order dependencies (like jQuery and jQuery-UI). Or, I have to self-host all my script code and combine it all into one big file. Not only do I lose *all* parallel loading benefits with that approach, but I also lose the shared cache CDN effect across multiple sites.

As a web page author, I can't get the true web performance that I ought to with existing content. I have to make really unfortunate trade-off choices.

Shouldn't I as a web page author be able to say: "I want to load script resources in parallel from my own server and/or from a remote CDN location, at any time in the lifetime of my page, and be able to opt-in to whether I need those scripts to execute in a certain order or I can declare that they are independent and should execute 'as soon as possible' instead."

----------------
This WHOLE discussion fundamentally comes down to this: I should be able to, **if I need it**, get all the benefits of insertion-order execution preservation that markup script tags enjoy, but ALSO all the benefits of dynamic script loading that markup script tags can't get (because they have to be pessimistic about document.write()). 

As a web performance optimization expert, I should not have to choose. And I definitely shouldn't have to use brittle browser hacks to do it.

To explain further the motivations behind this proposal (and Mozilla's implementation of it):


With respect to the underlying functionality being requested:

Scripts that are added as script tags in the HTML markup have the ability to choose if the author wants them to execute in insertion-order or in "as soon as possible" order. This is of course the `async` attribute.

However, it is strange and frustrating (as described above) that script elements which are dynamically added to the DOM do *not* have the ability to choose between these two ordering modes.

The functionality request is to make the "async" feature symmetrical between parser-inserted scripts and script-inserted scripts. That is, an author should be able to set `true` or `false` on the `async` property on a script-inserted script element just like they can add or withhold the `async` attribute in markup for parser-inserted script elements.

--------
With respect to the `async` property default value being `true` instead of `false`:

According to the way the spec currently reads, a spec-conforming browser should treat a script-inserted script element in such a way that it executes "as soon as possible". This means that a script-inserted script element currently behaves in most ways very similar to a parser-inserted script element that the `async` attribute set on it. 

It therefore makes some sense (again, consistency and symmetry wise) to default the script-inserted script element's `async` property to `true` to reflect this behavioral relationship.

Moreover, it's EXTREMELY important that the addition in browsers and the spec be done in a way that this augmented behavior (selecting ordering behavior on script-inserted script elements by setting the `async` property) be **feature-testable**.

Since currently all browsers that implement the "async" functionality default the `async` property on script-inserted script elements to `false`, I suggested that changing the default to be `true` would kill two birds with one stone: it would bring the `async` property more into semantic line with the current spec'd behavior for script-inserted script elements, but it would also provide a reasonable feature-test for the new behavior.

By creating a dynamic script element and examining the default value of the `async` property, an author can know if they are in a browser implementing this augmented behavior or not.

var async_ordering = (document.createElement("script").async === true);

In fact, now that Mozilla did this with FF4b8, and LABjs updated to do that feature-test, it proves the viability of that approach because LABjs now works just fine in both FF4b8 with the proposal implementation, and in older FF's (and other browsers) without the proposal implemented.


@Jonas-
If we switched the spec AND preserved the default value and behavior as `false` (aka, insertion-ordered), how could we feature-test for this?

https://bugs.webkit.org/show_bug.cgi?id=50115

> For what it's worth, my preference is to make .async default to false for
> elements created through the DOM. And make .async equals false mean that
> elements execute in order.

That would be a performance regression for WebKit, which is why we ended up with this goofy property-not-reflecting-attribute scheme.

It would be helpful in figuring out what to do here if we had a list of sites that don't work with the spec's current behavior.  I believe WebKit currently matches the spec precisely in this regard, so those sites should presumably break in the WebKit nightly.

(In reply to comment #6)
> > For what it's worth, my preference is to make .async default to false for
> > elements created through the DOM. And make .async equals false mean that
> > elements execute in order.
> 
> That would be a performance regression for WebKit, which is why we ended up
> with this goofy property-not-reflecting-attribute scheme.
> 
> It would be helpful in figuring out what to do here if we had a list of sites
> that don't work with the spec's current behavior.  I believe WebKit currently
> matches the spec precisely in this regard, so those sites should presumably
> break in the WebKit nightly.

WebKit historically hasn't had an order guarantee for DOM-inserted scripts, and sites would UA sniff and use a different technique.

> It would be helpful in figuring out what to do here if we had a list of sites
> that don't work with the spec's current behavior.

I addressed that request here:

https://bugs.webkit.org/show_bug.cgi?id=50115#c43

For those of you who don't want to wade through the WebKit bug, the sites aren't broken today because they're not using this feature of LABjs.  Not using the feature makes them slower than they would be if we resolved this issue and provided a way for sites to load scripts in parallel and execute them sequentially from JavaScript.

(in reply to comment #9)

Actually, the REAL reason those pages are not yet "breaking" in Webkit nightlies is described in this comment:

https://bugs.webkit.org/show_bug.cgi?id=50115#c46

I've now discovered that Webkit's change to stop fetching scripts with unrecognized types was *only* applied to markup script tags, whereas LABjs and other script loaders use dynamic script elements.

As you (Adam) awknowledged there, the change probably *should* have been applied to both parser-inserted scripts AND script-inserted scripts, so it's a probably a bug that Webkit will likely address quickly to bring behavior in line with intentions.

I'm quite certain that when Webkit *does* apply the change also to script-inserted script elements, we'll have a better example of how a site can break by not being able to "preload".

Those sites are avoiding "preloading" (to the detriment of performance) because they don't want to be blind-sided by LABjs breaking if Webkit does in fact release a breaking change, which is obviously going to happen sooner than later. 

Just because they're making that performance tradeoff for more stability doesn't mean it's not a perfectly valid use-case, that needs to be addressed.

(In reply to comment #1)
> (In reply to comment #0)
> > When a script element node is created, if it is being flagged as
> > parser-inserted, set its force-async flag to false. Otherwise, set its
> > force-async flag to true. (Note that createContextualFragment, innerHTML and
> > XSLTProcessor::transformToFragment-created scripts are not flagged as
> > parser-inserted.) This flag setting happens before any attributes (even
> > parser-set ones) are set on the node (so a fragment parser-set async attribute
> > may modify the flag shortly after).
> 
> That doesn't really make sense. The elements created by the parser are created
> with their attributes already set. But I guess we can get the same effect in a
> different way.

If you want to make the spec treat element creation and attribute setting as an atomic operation when performed by the parser, you can rewrite the above as the force-async flag getting initialized to false when the parser creates a parser-inserted node that has the async attribute.

> Overall this proposal seems to add a lot of complexity without a really good
> argument. Are there really that many sites that depend on this?

There are individual sites (Wikia, Hotmail, beta.edmunds.com) and libraries (LABjs, 'order' plug-in for RequireJS and OpenLayers) that exploit the way Firefox < 4 deviates from WebKit/IE/spec to get improved performance over WebKit/IE/spec. The rationale for this feature in Firefox 4 is to give these sites/libraries a way not to regress performance relative to Firefox 3.6 while making it possible for Firefox 4 to default to the WebKit/IE/spec behavior (if you don't touch the async DOM property). Fortunately, the sites and libraries that have been relying on the old spec-incompliant and WebKit/IE-inconsistent behavior of Firefox have been few enough and extraordinarily responsive to evangelism to make it possible to proceed what is described in comment 0.

Putting what's suggested in comment 0 into the spec and having other browsers implement it would allow other browsers to benefit from the performance optimizations now available to Firefox.

Currently, in cases other than LABjs and RequireJS, WebKit away by not breaking due to UA sniffing.

> What are other UAs intending to do here?

As far as I can tell, WebKit seems to be pretty far along implementing this if only you made the spec change first:
https://bugs.webkit.org/show_bug.cgi?id=50115#c38

If WebKit waits for you and you wait for WebKit, there's a deadlock. AFAICT, WebKit waiting for what you do meets your standard of "whatever browsers would implement".

(In reply to comment #11)
> Currently, in cases other than LABjs and RequireJS, WebKit away by not breaking
> due to UA sniffing.

...WebKit *gets* away not breaking...

I plan to spec what's proposed in comment 0 (modulo editorial issues).

Kyle mentioned t previously, but I wanted to also note:  The spec all ready suggests (and IE implements) a solution to this problem -- begin loading the url upon src assignment, wait until insertion to execute.  It's buried somewhere in Step 12 IIRC. This provides a mechanism by which deferred execution can be achieved.  Since Ordered execution is just a specific type of dererral, it's handled as well.  We've had almost identical functionality using images forever.

While it may seem to be, IEs readystate is *not* needed.  Notifcation in-flight scripts have finished download, while nice, is not required.  Simply create the scripts, assign the src, and use run-of-the-mill onload chaining.  Since order will always be preserved, it degrades well in browsers that do not implement.

I dont think it should be feature-testable.  The browser vendor is presumably aware of the spec's suggestion and chose not to heed.  Making it testable encourages attempts to thwart the intended behavior. 

Wlile the symmetry is naturally appealing, I dont see how this async method can be preferable.  A global queue is a natural bottleneck and it doesnt seem right that my options are: 1) serially download or 2) introduce artificial dependencies and risk significant delays.  Once "trusted" third-parties start using this "feature," it's value erodes quickly.  I have struggled with FF's current behavior and know others do to.  Would hate to end up back there when, conceptually, there's an alternative.

(in reply to comment #14)

I still strongly feel the proposal for the symmetry of "async=false" has value in most of the use-cases surrounding script loading. I am very glad and thankful that Ian has considered spec'ing the simple proposal.

The majority of sites do NOT have such a complex dependency graph of multiple independent trees of dependencies, in which the single "global queue" you refer to would become any kind of a bottleneck or hindrance.

That's not to say that type of use-case is invalid. Sites like that certainly exist, and there's value to a mechanism that could address that use-case, very well perhaps using the "preloading" mechanism you referred to, which is currently specified as a "may" option for browser vendors (in step 12 of the script loading algorithm). 

In fact, if combined with something like `readyState` monitoring (like IE's implementation does), there's a very valuable set of advanced use-cases for loading scripts but deferring (some or all of) their execution until much later. This is functionality we should definitely ask browser vendors to implement (as it's already suggested in the spec). 

Granted, the "preloading" mechanism IS functionality that would not only fully serve, but also exceed, the capabilities of the currently discussed "async=false" proposal.

HOWEVER, I think it's a mistake to conflate these two different sets of functionality into one discussion, or to suggest that one precludes the utility of the other. I think they're separate, and should be kept that way, and that each of them has independent merit.

The value to the currently discussed "async=false", on top of the consistency/symmetry argument (which is strong in and of itself), is that it serves a pretty sizeable chunk of the web's current needs for script loading, and does so in a way that is very simple and straightforward for script loaders to utilize. If say 80% of all sites' script loading needs can be well-served by "async=false" (a relatively simple change for most browsers), then I think it has value (instead of a more complex system that might also work).

In a separate (but certainly useful) effort, I think the more advanced 20% use-cases (more complex dependency trees) can and should be pushed via the other "preloading" mechanism (aka, load-but-don't-execute). However, *that* discussion doesn't really belong here (since the spec already mentions both its method and value), but rather with browser vendors directly.

In turn, I see uses of rigorous execution order beyond network-fetching like sub-module loading.  AFAIK, is only available to parser-inserted scripts using document.write().  An eager-fetching feature-test would seem to provide the necessary flexibility.  Perhaps this can be considered.

EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are satisfied with this response, please change the state of this bug to CLOSED. If you have additional information and would like the editor to reconsider, please reopen this bug. If you would like to escalate the issue to the full HTML Working Group, please add the TrackerRequest keyword to this bug, and suggest title and text for the tracker issue; or you may create a tracker issue yourself, if you are able to do so. For more details, see this document:
   http://dev.w3.org/html5/decision-policy/decision-policy.html

Status: Accepted
Change Description: see diff given below
Rationale: Implementations are converging on reporter's comments.

Checked in as WHATWG revision r5817.
Check-in comment: Make script-inserted external scripts that have .async=false execute in the insertion order, default to true
http://html5.org/tools/web-apps-tracker?from=5816&to=5817

Thank you.

mass-move component to LC1