As you might have noticed, I haven't put a lot of time into that algorithm. Improvements welcome; otherwise I'll get to it at some point.

FWIW, currently DOM Core mandates that CDATA doesn't exist, and at least Gecko is interested in trying to get rid of it:

https://bugzilla.mozilla.org/show_bug.cgi?id=660660

mass-move component to LC1

Isn't this just XML's problem? Why would we need to redefine the XML spec here? I don't understand the problem. What is the interoperability risk here?

I have heard of cases where sites serialize some XML and then break because we included an XML declaration. And when specifications fail to deliver, we address the gaps. At least, we have been doing that to some extent.

(In reply to comment #4)
> Isn't this just XML's problem? Why would we need to redefine the XML spec here?
> I don't understand the problem. What is the interoperability risk here?


XML doesn't define an algorithm for serialising a DOM tree.
You could just say that the "xml fragment serialisation" as referred to by the html spec meant "any string which would parse to the same tree given an XML parser" but then subsequent processing needs to be defined to be tolerant to the implementation-specific differences that that generates. As Anne just mentioned there are choices about whether to use xml declarations, whether to use foo/> or foo></foo> whether to use &gt; or > etc etc. If it woul dbe sufficient to define the xml serialisation in this way one would expect that it would be sufficient to define the html serialisation the same way: any string which parses to the same dom using the html parse algorithm. However that is not the way the html serialisation is defined, one particular serialisation is defined (which seems like a good thing to me from an interop point of view)

I agree that it needs to be defined, but how to serialise XML interoperably seems like an issue for the XML spec, not the HTML spec.

(In reply to comment #7)
> I agree that it needs to be defined, but how to serialise XML interoperably
> seems like an issue for the XML spec, not the HTML spec.

Not really. The XML spec doesn't define any object model that the xml syntax parses to, so there is nothing for it to define the serialisation of. The place in the "XML family of specs" where an object model for which serialisation could be defined was/is the DOM specs, but as you know, most of the parts of that relevant to the web browser context are either incorporated into the html spec or being refreshed as part of the same suite. In this case since one of that family of specs is explicitly about serialising the DOM, that seems like the natural place for it.

If you think how to serialise XML should be defined by the DOM specs and not the XML specs then I guess we can see what the DOM spec people think. But personally I think that's absurd.

In either case, having the _HTML_ spec define XML's serialisation is simply inappropriate. The HTML spec defines HTML's syntax, parsing rules (including error handling), and serialisation. The CSS specs define CSS's syntax, parsing rules (including error handling), and serialisation. The XML spec should define XML's syntax, parsing rules, and serialisation.

DOM Parsing and Serialization has a start. Someone should finish it.

(In reply to comment #9)
> If you think how to serialise XML should be defined by the DOM specs and not
> the XML specs then I guess we can see what the DOM spec people think. But
> personally I think that's absurd.

As I say, you can only specify the serialisation of an in-memory structure, you can't specify the serialisation of a syntax. So in XML's case it s the serialisation of the DOM that needs to be specified.

> 
> In either case, having the _HTML_ spec define XML's serialisation is simply
> inappropriate. 

I agree. As Anne commented in comment 10, the relevant part has already been split off into DOM parsing and serialisation, so this entire bug is a comment on that (and I see someone has changed the component of this report appropriately)

It made sense for XML serialization to be in the DOM specs when DOM applied to both XML and HTML, if the current DOM group is only concerned with HTML, then it may make sense to have a new XML DOM group separate from HTML DOM which will concern itself with matters like this.

If there is no interest within the W3C on further building standard DOM apis for XML, then perhaps its time for the W3C to seek other stewards for those specs. (Of course an argument could be made that DOM Level 2 is already enough for the needs of non-browsers, I'm not going to make that argument, but I'd be interested in hearing from someone who wants to)


(In reply to comment #9)
> If you think how to serialise XML should be defined by the DOM specs and not
> the XML specs then I guess we can see what the DOM spec people think. But
> personally I think that's absurd.
> 
> In either case, having the _HTML_ spec define XML's serialisation is simply
> inappropriate. The HTML spec defines HTML's syntax, parsing rules (including
> error handling), and serialisation. The CSS specs define CSS's syntax, parsing
> rules (including error handling), and serialisation. The XML spec should define
> XML's syntax, parsing rules, and serialisation.

As a concrete use case for a well-defined XML serialization, I'll note that there is no way currently specified to generate an HTML serialization of a document once it is parsed as XML.  Because the XML serialization is ill-defined, browsers can (and do) emit constructs like "<span/>" which can't be parsed by the HTML parser.

I'd suggest that the XML serialization be defined to conform to the recommendations in http://dev.w3.org/html5/html-polyglot/html-polyglot.html such that (in the absence of unsupported XML features such as namespaces) it is also a valid HTML document.

That is, as far as possible, doc1 and doc2 should be identical in:

var parser = new DOMParser();
var doc1 = parser.parseFromString(stringContainingXMLSource, "application/xml");
var out = (new XMLSerializer()).serializeToString(doc);
var doc2 = parser.parseFromString(out, "text/html")

(In reply to C. Scott Ananian from comment #13)
> As a concrete use case for a well-defined XML serialization, I'll note that
> there is no way currently specified to generate an HTML serialization of a
> document once it is parsed as XML.  Because the XML serialization is
> ill-defined, browsers can (and do) emit constructs like "<span/>" which
> can't be parsed by the HTML parser.

Don't parse XML as HTML.

> I'd suggest that the XML serialization be defined to conform to the
> recommendations in http://dev.w3.org/html5/html-polyglot/html-polyglot.html
> such that (in the absence of unsupported XML features such as namespaces) it
> is also a valid HTML document.

Please no, that introduces unnecessary complexity to the XML serializer. The polyglot spec is supposed to only affect authors that want to waste their time following it, not UAs.

> That is, as far as possible, doc1 and doc2 should be identical in:
> 
> var parser = new DOMParser();
> var doc1 = parser.parseFromString(stringContainingXMLSource,
> "application/xml");
> var out = (new XMLSerializer()).serializeToString(doc);
> var doc2 = parser.parseFromString(out, "text/html")

Why would you want to do this? It doesn't make sense. Either you serialize and parse as HTML, or you serialize and parse as XML. You wouldn't serialize an image as GIF and then parse it as JPEG, either.

The user who was doing this explained they were passing/processing data as XML "so that IE won't screw it up".  But then they wanted to pass it to my service, which only accepts HTML (and uses domino, a spec-compliant HTML parser).  Running document.outerHTML on their XHTML document gave them XML (which is spec-compliant, although surprising), and they complained that I "didn't parse <span/> correctly".

I think it's best not to try to cast aspersions on the user here, but instead try to figure out what the best way to handle XML serialization and conversion.  I see no good reason for the XML serializer to emit "<span/>" instead of "<span></span>", for instance.

OK.

I still think the best solution to your problem is to use the same format in the serializer and the parser, whether that means you gain an XML parser or they use an HTML serializer. It's possible that the Web platform doesn't provide a way to serialize an XML document as HTML currently, that seems like a bug.

The formats are not compatible. <span/> is just the tip of the iceberg. Even if the XML serializer somehow implemented things from the Polyglot spec, it still wouldn't work for arbitrary DOMs. The DOM also needs to follow the Polyglot spec, and the UA can't enforce that. So it's a futile exercise. It would also lead people into thinking that it's a good idea to serialize in one format and parse as another, which it isn't.

In this bug I'm just politely asking for the XML serialization spec, if/when it is defined, to at least try to avoid gratuitous HTML incompatibilities like shorttags.  I understand that there are XML/HTML incompatibilities, but if the document author has made the effort to create a polyglot document it is unfortunate if the XML serialization doesn't preserve it.

(Bug 25225 asks for an HTML serialization mechanism for an XML document.)

I think you need to be more specific about what you want exactly. How do you want a "br" element to be serialized, for instance?

@Simon: consistent with http://dev.w3.org/html5/html-polyglot/html-polyglot.html#empty-elements
That is,
"All elements listed as void in the HTML specification or in an extension spec, MUST in polyglot markup have the syntactic form of an XML empty-element tag (<foo/>)."

OK so the XML serializer would keep a list of the HTML void elements. If that's all there is I guess that's acceptable, but it seems like a slipperly slope to me. In any case, if you want this, please file a new bug.

I'm not sure if David is monitoring this bug anymore. In any case, it looks like C. Scott Ananian is still interested in seeing this bug through.

C. Scott, please take a look at:
http://www.w3.org/TR/DOM-Parsing/#dfn-concept-xml-serialization-algorithm, step 15. I think it keeps things as close to HTML serializing interop as practically possible. For other html-namespaced element is preserves begin/end tag serialization.

Let me know if there's more that you think can be done here. Thanks.

monitoring is probably putting it too strongly, but I'm here.
I'm happy with the way the document's progressed, thanks.

David

It looks to me like steps 15 and 16 solve the problem which originally led me here -- that is, they ensure that we only use self-closing tags for HTML void elements.  I'll check with the others here at Wikimedia (in particular we have HTML/XML interop issues with the the Parsoid and Visual Editor components) and see if this is sufficient.  (I'll probably have to implement the new spec by hand and transition Visual Editor to use it, and/or audit Visual Editor's existing serializer against the new spec.)