In particular, the reason *why* they do "diffs" and minimal edits is to preserve the "invisible state" of elements like form inputs - there's no way to know or restore the input state of a text input if it's in the middle of doing IME, for example.

In a previous EWS, we discussed having a way to extract this state in an opaque-to-JS object, and be able to spam it into another element of the same type, so there wasn't as much need for tree-diffing.  They could instead just crawl the before-change tree, extract the states, and then crawl the after-change tree and restore them.

This avoids us having to define a specific tree-diffing algo, which is non-trivial and has a bunch of fiddly bits that can be adjusted.

There's more to diffing than just input states. For example, trashing the whole DOM via innerHTML destroys application state such as an element having an event handler, or a variable pointing to a node in the DOM tree.

The overarching reason libraries are using a virtual dom diff strategy is to be able to apply multiple changes to a DOM tree atomically, while keeping unchanged nodes intact for performance reasons (as opposed to recreating them and restoring their states). Other js frameworks' data binding algorithms (dirty checking, Object.observe + rAF, KVO + rAF, etc) also have this same fundamental goal.

Non-diff strategies currently require a library to recursively visit every node in a DOM tree (since there's no way to efficiently compile a list of js-value-to-DOM-placeholder mappings), plus implement DOM versions of Array::splice, etc.

A `.diffHTML` whose usage is similar to `.innerHTML`, but with diff semantics, would be an immediately obvious way for a naive developer to apply js data to the DOM.

Some resources from React you may want to look at while investigating this.

http://facebook.github.io/react/docs/reconciliation.html#problematic-case
https://speakerdeck.com/vjeux/oscon-react-architecture (starting slide 16)

The two big challenges that I haven't seen evoked yet in this thread are:

 - How do you implement the concept of identity when doing the diff. We opted in for comparing elements pair-by-pair and assume they are identical if they have the same tag. Or using the `key` attribute to map two elements within a list.

 - How do you deal with higher order components. One of the critical aspect of React is that you can nest components. It implies two things: you can do a diff on partially resolved components and alter the flow of the diff algorithm while it's running via shouldComponentUpdate.

Also, I would be pretty sad if the API would involve serializing that virtual DOM into a string to pass it to the browser.

One common use case within React to integrate with 3rd party code that manipulates the DOM is to have a wrapper component that always renders a <div /> and use lifecycle methods to manipulate that DOM node.

How would it look like with .diffHTML, it would see that the <div /> is now empty and would remove everything that's inside.

Also, note that React doesn't need any support from the browser to work. There's the possibility that it's best left off to be implemented by libraries rather than inside of the browser.

We (React) currently use innerHTML as a way to automatically create all elements at once. That's because the JS/DOM bridge seems to high latency characteristics. That's not inherent to a browser architecture.

If it was fast enough, we could just use document.createElement instead, which could contain new or existing event handlers too.

If performance of the outer DOM APIs was faster, this wouldn't be an issue. replaceChild could be fast. Which adresses Leo's concerns.

As vjeux points out, the strategy used for what exactly preserves internal state is still an unresolved issue. For example, in React, we still don't allow children to move to a different parent. We also have different semantics for stateful high-level components. That's still patterns that are still evolving.

The big issue is, as Tab points out, that the hidden state can't be transferred to another element somehow.

Diffing is inherently only needed when we're targeting an imperative API with retained state. When we're targeting an API like WebGL or Canvas drawing, we don't actually use diffing anywhere at all. That is an indication that it's not actually needed.

We actually just use diffing to hack around the current browser limitations. I think it would be a big mistake to standardize it. It's actually very inefficient compared to the ideal solution.

What we really need is a way to either be in full control of internal state. This can be done, for example, by a callback that provides a new object with the internal state. We can intercept any publicly accessible state and replace it with our own. Private state can be transferred from one node to another.

So you want a way to extract and replace private state of any element? Or just a set of elements for which it is hard to do? Asynchronous or synchronous? Opaque or serializable?

Might be good to track that separately eventually and WONTFIX this bug, but I guess we should first figure out what the requirements are so we know whose problem this is going to be.

After a short conversation on Twitter Marcos asked me to write down my ideas here, and I'm doing that. I don't know if I'm qualified to have an opinion / participate in the discussion, but I try my best.

I come primarily from a canvas2d / webgl background, and am really interested in web components and their potential to make the DOM usable for applications that are traditionally inaccessible. I spent the last ~ 6 months working on what started out as an attempt to wrap all of WebGL into Web Components, which was actually successful but just incredibly inefficient and difficult to use, despite a many attempts to make it work somehow. Eventually I decided to focus on the js part and not add web component bindings until I reach a much higher level of abstraction.

In my experience many people don't bother with the DOM at all or at most use it to easily create a GUI, but people already successfully built even that completely in WebGL => http://www.reddit.com/r/gamedev/comments/1a0yv5/a_webgl_terrain_engine_and_gui/   So it's not just canvas2d that has been for that. The main selling points of the DOM, in my opinion, aren't the rendering or layout, both of which don't really satisfy current and future needs, and are, if you believe the rumors, especially alienating for programmers coming from desktop development. 

The main reason for using the DOM is providing an accessible, and descriptive structure of an application not just for users, but also for the majority, mostly hobbyist web developers to build their sites and applications from.

I think those are two worlds...not colliding...but rather coexisting with little exchange. One world, traditional web development uses the DOM way too much and is usually mocked for how extremely slow and inefficient their applications are. In this world everything, from structure, to layout to rendering is achieved with the DOM, which is huge and slow. And that's obviously not good but has been the only way to do things for ages. Not only is the performance spectacularly bad, the accessibility suffers because all the rendering and layout elements obscure the structure with noise and cryptic class names. Far away from being a descriptive representation!

The other world are people who loathe all the specific ( mostly legacy ) knowledge, and hacks and polyfills required to achieve anything, because apparently it's so far off from what people are used to. Learning all that is widely considered a waste of time, as it doesn't really transfer to any other areas. So those people often don't bother with the DOM at all, and just keep everything in the canvas, or develop their own solutions, because hey even if they fail, at least they got a huge boost in personal experience out of it they wouldn't get out of spending years learning all the specific legacy bullshit.

But that's not good either, because it invalidates not only the work to make the web accessible for all users, but also to developers with non-expert knowledge. Based on how few people actually picked up technologies like WebGL or even just canvas in the recent years I assume that's just way beyond the scope of the majority of content of authors on the web and will probably forever be.

I think web components could become a middle ground between those worlds. It would be possible to encapsulate those concepts in web components, and not only make those accessible to the web developer community, but to users who suddenly have a perfectly fine DOM focused purely on the basic elements of an application they can interact with, with zero layout / rendering noise in between.

So that's where I believe the web is headed, and we already have almost all APIs needed to accomplish most of it already, both in the standards and implemented in browsers, we just need to wrap our heads around it. But of course there's also room for improvement, even though it's not strictly needed, it would be greatly appreciated:

* a compile / transpile to js standard for vendors to build compile / transpile tools into the browsers, optimally just allowing people to supply their code in any version of ecma or any programming language and have the browser do the compiling / transpiling, allowing the vendor to apply any optimization and ensure compatibility ( and give feedback about it ) for their own implementation and deliver optimized builds ( or web components ) with one common interface but different code depending on user agent.
* extensible CSS, similar to custom Elements. There's no reason to hardcode for example material properties in JS, when CSS is ideal, and specifically made for this purpose. It's currently tightly coupled with the DOM, and close to useless for anything that doesn't use the DOM for layout and rendering. Optimally separate layout and rendering properties in CSS entirely. There's no reason for example to have the overhead of 2d ( text ) layout properties on the material of a 3d mesh.
* also a slimmer Element / HTMLElement if it should be the base class for all web components. Currently you have to *delete* 90% of the properties on the HTMLElement prototype, to prepare it to be extended for anything you want to use for non-rendering non-layout purely structural purposes. 
* Completely sealed shadow DOM, alternatives for styling. There's no point in having a shadow DOM if you can and *have to* penetrate it ( that part isn't about webgl, but for the rendering / layout use case of components ). It goes hand in hand with the custom CSS. You don't want to ever penetrate into shadow dom, but instead apply custom, designed style rules, or, if it doesn't help - use another custom element or fork / improve the source code.
* Redesign / extend the DOM and CSSOM to work with its own Data Types or at least enums rather than DOMStrings in 99% of the cases. Not only for performance but just for sanity and usability. There's no reason why you can just construct, for example, a CSSColor, and just using it in itself. Being able to actually set the color of an element from that should go without saying. And all those cases where you have to go back to the documentation to see the exact string expected by an attribute, rather than just looking up the flag and using it. That's what hobbyist developers struggle with and waste a lot of time on and it's mostly avoidable.


I'm not entirely sure how on topic all that is, or if this is the proper place to address those issues / ideas.

Thank you for bringing this up Marcos and others for participating in the discussion. I'm going to close this since there's no clear idea of how this would affect the DOM Standard.

Once you have a clear idea of a concrete thing we could introduce I'd be happy to discuss that in a GitHub issue, the new way to report issues against the DOM Standard.