First stab:

http://aryeh.name/gitweb.cgi?p=editing;a=commitdiff;h=eeaedb93
http://aryeh.name/gitweb.cgi?p=editing;a=commitdiff;h=69d2d303

You can run the tests here:

http://aryeh.name/spec/editing/conformancetest/runtest.html

The expected results are generated here:

http://aryeh.name/spec/editing/conformancetest/gentest.html

Documentation is in the spec:

http://aryeh.name/tmp/editing/editing.html#tests


Issues to work out still:

* Tests don't really work at all in IE or Opera (whoops).
* They don't test queryCommand*() yet.
* Test that DOMs match, not just innerHTML.  (But this might require fixups in the algorithms to merge text nodes or such, maybe not worth it.)
* Try to produce less confusing failure messages when style normalization kicks in.  I have to normalize style="" attributes to work around the fact that the serialization format isn't actually standardized meaningfully, but this shouldn't be reflected in the failure messages.
* Make sure expected results start getting generated as they should for the unserializable cases where it's just browser bugs (like <xmp> in Gecko, or <pre> followed by newline in Gecko/WebKit).
* Review browser test failures to unearth further bugs.

But it's a start.

Also should remove the dependency in runtest.html on correct HTML5 serialization.  It causes lots of noisy failures in Opera (presumably IE too) because it omits closing tags when serializing the same DOM.

(In reply to comment #1)
> * Tests don't really work at all in IE or Opera (whoops).

Filed bug 14063.

> * They don't test queryCommand*() yet.

Filed bug 14065.

> * Test that DOMs match, not just innerHTML.  (But this might require fixups in
> the algorithms to merge text nodes or such, maybe not worth it.)

I'm not going to try this for now.  We'd need some way of serializing a DOM perfectly, or else make sure every DOM really really serializes correctly (like by throwing normalize() into execCommand() someplace).

> * Try to produce less confusing failure messages when style normalization kicks
> in.  I have to normalize style="" attributes to work around the fact that the
> serialization format isn't actually standardized meaningfully, but this
> shouldn't be reflected in the failure messages.

http://aryeh.name/gitweb.cgi?p=editing;a=commitdiff;h=2b19002f

> * Make sure expected results start getting generated as they should for the
> unserializable cases where it's just browser bugs (like <xmp> in Gecko, or
> <pre> followed by newline in Gecko/WebKit).

Filed bug 14061.

> * Review browser test failures to unearth further bugs.

This will be an ongoing effort.

Marking fixed.  Will work on followups.