Copyright © 2000 W3C® (MIT, INRIA, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply.
This specification describes a method for generating a physical representation, the canonical form, of an input XML document, that does not vary under syntactic variations of the input that are defined to be logically equivalent by the XML 1.0 Recommendation [XML]. If an XML document is changed by an application, but its Canonical-XML form has not changed, then the changed document and the original document are considered equivalent for the purposes of many applications. This document does not establish a method such that two XML documents are equivalent if and only if their canonical forms are identical.
This is presently a WG and editor's copy that demonstrates a new approach to a Canonical XML specification. Prior versions of this document were published by the XML Core Working Group, which delegated the completion of this specification to the IETF/W3C XML Signature Working Group.
The W3C Membership and other interested parties are invited to comment on this direction, review the specification and report implementation experience. While we welcome implementation experience reports, the XML Signature Working Group will not allow early implementation to constrain its ability to make changes to this specification prior to final release.
Please send comments to the the list <w3c-ietf-xmldsig@w3.org>. Publication as a Working Draft does not imply endorsement by the W3C membership or IESG. It is inappropriate to cite W3C Drafts as other than "work in progress." A list of current W3C working drafts can be found at http://www.w3.org/TR/. Current IETF drafts can be found at http://www.ietf.org/1id-abstracts.html.
There have been no solicitations or declecrations regarding patents related to this specification.
1 Introduction
2 Canonical XML Data Model
3 Document Order for Canonical XML
4 Generation of Canonical XML
5 XML Document Subsets
The XML 1.0 Recommendation [XML] specifies the syntax of a class of resources called XML documents. It is possible for XML documents which are equivalent for the purposes of many applications to differ in physical representation. In particular, they may differ in their entity structure, attribute ordering, and character encoding.
It is not a goal of this work to establish a method such that two XML
documents are equivalent if and only if their canonical forms are
identical. Such a method is unachievable, in part due to application-specific
rules such as those governing unimportant whitespace and equivalent data (e.g.
<color>black</color> versus
<color>rgb(0,0,0)</color>). There are also equivalencies
established by other W3C Recommendations and Working Drafts. Accounting for
these additional equivalence rules is beyond the scope of this work. They can
be applied by the application or become the subject of future
specifications.
The XPath 1.0 Recommendation [XPath] specifies a data model for representing an input XML document as well as an expression syntax for describing portions of the document (as well as arbitrary strings, booleans and numbers). When an XPath expression is used to describe portions of an XML document, the result is called a document subset.
This specification describes a method for generating a physical representation of an input XML document or document subset that does not vary under syntactic variations of the input XML document that are defined to be logically equivalent by the XML 1.0 Recommendation. The input must be a well-formed XML document with an optional XPath expression and evaluation context. The output physical representation is called a canonical form or simply Canonical XML.
The Canonical XML generated for an entire XML document is well-formed. The canonical form of an XML document subset may not be well-formed XML. However, since the canonical form will often be subjected to further XML processing, most XPath expressions provided for canonicalization will be designed to produce a document subset that is a well-formed XML document or external general parsed entity.
Canonical XML is designed to be used by applications that require the ability to test whether a document or document subset has been changed in a way that is not defined to be logically equivalent by the XML 1.0 Recommendation. For example, a digital signature over the canonical form of an XML document or document subset would allow the signature digest calculations to be oblivious to changes in the document's physical representation provided that the changes are defined to be logically equivalent by the XML 1.0 Recommendation.
The data model used to create Canonical XML is equivalent to the data model defined in the XPath 1.0 Recommendation [XPath]. Although an implementation of this specification need not be based on an XPath implementation, this specification discusses the canonicalization method based on the XPath definition of a node-set.
Under the XPath data model, an XML processor is used to perform the following tasks in order:
Canonical XML requires that the input document be well-formed XML, but the input need not be validated. However, Canonical XML requires that attribute value normalization and entity reference resolution be performed in accordance with the behaviors of a validating XML processor. Thus, the declarations in the document type declaration are used to help create the canonical form, but the document type declaration is not retained in the canonical form (in part because it is omitted from the XPath data model and in part because it is not needed by the canonical form).
In the XPath data model, there exist the following node types: root, element, comment, processing instruction, text, attribute and namespace. There exists a single root node whose children are text nodes, processing instruction nodes, and comment nodes to represent information outside of the top-level element. The root node also has a single element node representing this top-level element. Each element node can have child nodes of type element, text, processing instruction, and comment. The attributes and namespaces associated with an element are not considered to be child nodes of the element, but they are associated with the element by inclusion in the element's attribute and namespace axes. Note that attribute and namespace axes may not directly correspond to the text appearing in the element's start tag in the original document.
Although the XML 1.0 Recommendation states that an XML processor need not provide the text of comments, the XPath data model and hence Canonical XML requires comments.
An element has attribute nodes to represent the non-namespace attribute
declarations appearing in its start tag as well as nodes to represent default
attributes that were not specified and not declared as
#implied.
By virtue of the XPath data model, Canonical XML is namespace-aware [Names], but it cannot and therefore does not account
for namespace equivalencies via namespace rewriting (see below). In the XPath data model, each element
and attribute has a name returned by the function name() which
can, at the discretion of the application, be the QName appearing in the
original document. Canonical XML requires that the XML processor retain the
sufficient information such that the QName of the element as it appeared in
the original document can be provided.
An element E has namespace nodes that represent its namespace
declarations, any namespace declarations made by its ancestor that have not
been overridden in E's declaration, the default namespace if it
is non-empty, and the declaration of the prefix xml.
Character content is represented in the XPath data model with text nodes. All consecutive characters are placed into a single text node. Furthermore, the text node's characters are represented in the UCS character domain. Canonical XML does not perform character model normalization (see below).
The XPath node-set required by the Canonical XML generator is defined to be the result of setting an initial evaluation context of:
then evaluating the expression (//. | //@* | //namespace::*).
This expression generates a node-set containing every node of the XML
document.
Although an XPath node-set is defined to be unordered, the XPath 1.0 Recommendation [XPath] defines the term document order to be the order in which the first character of the XML representation of each node occurs in the XML representation of the document after expansion of general entities, except for namespace and attribute nodes whose document order is application-dependent.
During XPath expression evaluation, Canonical XML imposes no order on the namespace and attribute axes of elements. After evaluating the expression, a node-set is processed by imposing the following additional document order rules on the namespace and attribute nodes of an element:
Lexicographic comparison is based on the UCS codepoint values, which is equivalent to lexical ordering based on UTF-8.
The XPath node-set is converted into a UTF-8 string by generating the representative text for each node in the node-set in ascending document order with a UTF-8 encoding. No node is processed more than once. Note that processing an element node E includes the processing of all members of the node-set for which E is an ancestor. Therefore, directly after the representative text for E is generated, E and all nodes for which E is an ancestor are removed from the node-set (or some logically equivalent operation occurs such that the node-set's next node in document order has not been processed).
The method of text generation is dependent on the node type and given in the following list:
xmlns="". Then, generate the
representative text for each namespace node that is in the element's
namespace axis and in the node-set, except omit the namespace node
with prefix xml if its string value is
http://www.w3.org/XML/1998/namespace. &, all double quote
characters with ", and all whitespace characters
(#x9, #xA, #xD, and #x20) with character references, except for #x20
characters with no preceding #x20. When whitespace characters are
replaced, the character references are written in uppercase hexadecimal
with no leading zeroes (for example, #xD is represented by the character
reference
).&, all open angle brackets (<) are replaced by
<, and all #xD characters are replaced by

.
. If the string
value is empty, then the leading space is not added.<!--),
the string value of the node, and the close comment sequence
(-->).The QName of a node is either the local name if the namespace prefix string is empty or the namespace prefix, a colon, then the local name of the element. The namespace prefix used in the QName MUST be the same one which appeared in the input document.
Some applications require the ability to create a physical representation for an XML document subset. Canonical XML implementations based on XPath can provide this functionality with little additional overhead. The following additional steps must be taken:
The node-set passed to the canonical form generator is calculated by setting the initial evaluation context as described in the section Canonical XML Data Model, except replacing the variable bindings and namespace declarations with those provided above, then evaluating the following XPath expression:
(X) | (//. | //@)[self::xml:lang or self::xml:space or
@xml:lang or @xml:space]
The result of the given expression X is combined with the set
containing all declarations of xml:lang andxml:space
as well as the parent elements containing them. Note that if the result of
X is not a node-set, then an XPath error results.
Combined with the propogation of namespace nodes in the XPath data model,
this measure to preserve xml:lang andxml:space
declarations ensures XML-specific information is not lost when an element's
ancestors are removed from the node-set passed to the canonical form
generator. XML entities can derive application-specific meaning from anywhere
in the XML markup as well as by rules not expressed in XML. Clearly, these
rules cannot be specified in this document, so the author of the expression
X must be responsible for creating an expression that preserves
the information necessary to capture the full semantics of the members of the
resulting node-set.
Although this specification now defines Canonical XML in terms of the XPath data model rather than XML InfoSet, the canonical form described in this document is quite similar in most respects to the canonical form described in prior versions of the Canonical XML specification. However, there are some differences. This section discusses the differences and provides a rational for changes.
Canonical XML now includes comments. It is conceivable that comments may carry critical information in certain scenarios. For example, JavaScript is often embedded in HTML using comments, so the canonical form of an HTML must include the comments. Furthermore, even if comments are solely for the benefit of XML document authors, comments must be preserved if XML document authoring tools are to adopt Canonical XML. Finally, any application that requires the canonical form to omit comments can do so by specifying an appropriate XPath expression to eliminate them. For example, to create the canonical form of an entire input document less the comments, use the follow XPath expression:
(//. | //@* | //namespace::*)[not(self::comment())]
The idea of also eliminating processing instructions has been discussed, but it was rejected because in a number of scenarios they carry critical information value. However, if the application must eliminate processing instructions as part of its equivalence testing, an appropriate XPath expression can be defined to eliminate them.
Prior drafts of the Canonical XML specification eliminated all whitespace outside of the top-most element except for a single linefeed after each processing instruction. This specification is based on the XPath data model, so the whitespace is preserved. Applications that do not want any whitespace outside the topmost element to affect the canonical form can specify an appropriate XPath expression to eliminate the text nodes. For example, to keep all document nodes except whitespace outside of the topmost element, use the following expression:
(//. | //@* | //namespace::*)[parent::* or
not(self::text())]
It is not possible in XPath 1.0 to directly detect the root node, but the
parent axis has a principal node type of element, so parent::* returns an
empty set, which corresponds to a boolean false, for nodes whose parent is not
an element. Every node has an element parent except for the children of the
root node. The non-text children of the root are accepted by the subexpression
not(self::text()).
Prior drafts of the Canonical XML specification replaced all occurences of
> with > when they appeared in character
content (text nodes, attribute values, and so forth). There did not appear to
be a reason for this.
The Unicode standard [Unicode] allows multiple different representations of certain "precomposed characters" (a simple example is "ç"). Thus two XML documents with content that is equivalent for the purposes of most applications may contain differing character sequences. The W3C has recommended a normalized representation [CharModel]. Prior drafts of Canonical XML used this normalized form. However, character model normalization handles equivalencies of characters with differing UCS code points, whereas XML 1.0 and the XPath data model are based on UCS character equivalence. Furthermore, applications that must solve this problem are likely to perform character model normalization as character content is created, which would obviate the need for character model normalization during canonicalization. Therefore, character model normalization has been moved out of scope for Canonical XML.
Prior drafts of the Canonical XML specification described a method for rewriting namespace prefixes such that two documents having logically equivalent namespace declarations would also have identical namespace prefixes. However, the statement in Namespaces in XML that "the prefix functions only as a placeholder for a namespace name" is incorrect. Namespace prefixes can impart information value in an XML document if they are referenced in an attribute value or element content (for example, and element or attribute containing an XPath expression). Thus, rewriting the namespace prefixes would damage such a document by changing its meaning (and it cannot be logically equivalent if its meaning has changed). The theorems below state the results more formally.
Theorem 1: With namespace rewriting, there exist two XML documents D1 and D2 that are logically equivalent yet their canonical forms are not equal.
Proof:Let D1 be a document containing an XPath in an attribute value or element content that refers to namespace prefixes used in D1. Further assume that the namespace prefixes in D1 will all be rewritten by the canonicalization method. Let D2 = D1, then modify the namespace prefixes in D2 and modify the XPath expression's references to namespace prefixes such that D2 and D1 remain logically equivalent. Since namespace rewriting does not include occurences of namespace references in attribute values and element content, the canonical form of D1 does not equal the canonical form of D2 because the XPath will be different. []
Remark:The same condition exists if we remove namespace rewriting. The purpose of this theorem is simply to show that namespace rewriting does not accomplish the goal for which it is intended.
Theorem 2:With namespace rewriting, there exist two XML documents D1 and D2 that have equivalent canonical forms and yet are not logically equivalent.
Proof:Let D1 be a document containing an XPath in an attribute value or element content that refers to namespace prefixes used in D1. Further assume that the namespace prefixes in D1 will all be rewritten by the canonicalization method. Now let D2 = the canonical form of D1. Clearly, the canonical forms of D1 and D2 are equivalent (since D2 is the canonical form of the canonical form of D1), yet D1 and D2 are not logically equivalent because the aforementioned XPath works in D1 and doesn't work in D2. []
Remark:Since D1 and D2 are not logically equivalent, and D2 is the canonical form of D1, we can conclude that namespace rewriting is harmful rather than simply ineffective.
The conclusion to be draw from these theorems is that namespace prefixes should not be altered by XML canonicalization. Applications that need to test for logical equivalence will need to perform more sophisticated tests than mere octet stream comparison. However, this is quite likely to be necessary in any case in order to test for logical equivalencies based on application rules as well as rules from other XML-related recommendations, working drafts, and future works.
Prior drafts of the Canonical XML specification stated that the default
namespace is not used. In the XPath data model, a non-empty default namespace
is indicated by a namespace node with an empty local name. An empty namespace
is indicated by the absence of such a node. In keeping with the policy of not
rewriting namespace prefixes, which includes not adding prefixes that were not
in the source document, the default namespace system has been added to
Canonical XML. When there is no default namespace node, the canonicalization
method indicates this with xmlns="" even if the source document
did not contain this declaration explicitly (because there is no way to find
out whether it did or not). The result is logically equivalent but, like the
addition of default attribute nodes, implies that XPath expression authors
should be wary of creating expressions that test for the position of attribute
or namespace nodes (they are bound to fail in most cases because the sorting
of namespace and attribute axes occurs only on output, not during the XPath
expression evaluation).