Copyright © 2000 W3C® (MIT, INRIA, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply.
This specification describes a method for generating a physical representation, the canonical form, of an input XML document, that does not vary under syntactic variations of the input that are defined to be logically equivalent by the XML 1.0 Recommendation [XML]. If an XML document is changed by an application, but its Canonical-XML form has not changed, then the changed document and the original document are considered equivalent for the purposes of many applications. This document does not establish a method such that two XML documents are equivalent if and only if their canonical forms are identical.
This is the second Last Call of the Canonical XML specification and the third draft of an alternative approach to the first (20000119) Last Call. The Last Call ends on July 28, 2000. (See proposal and resolved thread to go to last call.) This specification differs from the first Last Call in that it (1) uses the XPath [XPath] data model, and (2) includes a few substantive changes that affect the canonical serialization of an XML document. It is not necessary for implementations to use XPath to generate the canonical form of an XML document. XPath simply provides a data model that is simplified compared to InfoSet, yet sufficient for the purpose of canonicalization. XPath also provides an expression syntax for describing the desired portion of a whole document. Any variances between that result from this specification's use of the XPath [XPath] data model and the XML Information Set [InfoSet] will be reported to the XML Information Set's comments list.
Prior versions of this document were published by the XML Core Working Group (the last of which was the 20000119 Last Call), which delegated the completion of this specification to the IETF/W3C XML Signature Working Group.
The XML Signature and XML WGs and other interested parties are invited to comment on this proposed direction, review the specification and report implementation experience. While we welcome implementation experience reports, the XML Signature Working Group will not allow early implementation to constrain its ability to make changes to this specification.
Please send comments to the editors and cc: the list <w3c-ietf-xmldsig@w3.org>. Publication as a Working Draft does not imply endorsement by the W3C membership or IESG. It is inappropriate to cite W3C Drafts as other than "work in progress." A list of current W3C working drafts can be found at http://www.w3.org/TR/. Current IETF drafts can be found at http://www.ietf.org/1id-abstracts.html.
There have been no declarations regarding patents related to this specification within the Signature WG.
1 Introduction
2 Canonical XML Data Model
3 Document Order for Canonical
XML
4 Generation of Canonical
XML
5 XML Document Subsets
A Resolutions
B References
C Acknowledgements
The XML 1.0 Recommendation [XML] specifies the syntax of a class of resources called XML documents. It is possible for XML documents which are equivalent for the purposes of many applications to differ in physical representation. In particular, they may differ in their entity structure, attribute ordering, and character encoding.
It is not a goal of this work to establish a method such that
two XML documents are equivalent if and only if their
canonical forms are identical. Such a method is unachievable, in
part due to application-specific rules such as those governing
unimportant whitespace and equivalent data (e.g.
<color>black</color> versus
<color>rgb(0,0,0)</color>). There are also
equivalencies established by other W3C Recommendations and Working
Drafts. Accounting for these additional equivalence rules is beyond
the scope of this work. They can be applied by the application or
become the subject of future specifications.
The XPath 1.0 Recommendation [XPath] specifies a data model for representing an input XML document as well as an expression syntax for describing portions of the document (as well as arbitrary strings, booleans and numbers). When an XPath expression is used to describe portions of an XML document, the result is called a document subset.
This specification describes a method for generating a physical representation of an input XML document or document subset that does not vary under syntactic variations of the input XML document that are defined to be logically equivalent by the XML 1.0 Recommendation. The input must be a well-formed XML document with an optional XPath expression and evaluation context. The output physical representation is called a canonical form or simply Canonical XML.
The Canonical XML generated for an entire XML document is well-formed. The canonical form of an XML document subset may not be well-formed XML. However, since the canonical form will often be subjected to further XML processing, most XPath expressions provided for canonicalization will be designed to produce a document subset that is a well-formed XML document or external general parsed entity.
Canonical XML is designed to be used by applications that require the ability to test whether a document or document subset has been changed in a way that is not defined to be logically equivalent by the XML 1.0 Recommendation. For example, a digital signature over the canonical form of an XML document or document subset would allow the signature digest calculations to be oblivious to changes in the document's physical representation provided that the changes are defined to be logically equivalent by the XML 1.0 Recommendation.
The data model used to create Canonical XML is equivalent to the data model defined in the XPath 1.0 Recommendation [XPath]. Although an implementation of this specification need not be based on an XPath implementation, this specification discusses the canonicalization method based on the XPath definition of a node-set.
Under the XPath data model, an XML processor is used to perform the following tasks in order:
Canonical XML requires that the input document be well-formed XML, but the input need not be validated. However, Canonical XML requires that attribute value normalization and entity reference resolution be performed in accordance with the behaviors of a validating XML processor. Thus, the declarations in the document type declaration are used to help create the canonical form, but the document type declaration is not retained in the canonical form (in part because it is omitted from the XPath data model and in part because it is not needed by the canonical form).
In the XPath data model, there exist the following node types: root, element, comment, processing instruction, text, attribute and namespace. There exists a single root node whose children are processing instruction nodes and comment nodes to represent information outside of the document element (and outside of the document type declaration). The root node also has a single element node representing this top-level element. Each element node can have child nodes of type element, text, processing instruction, and comment. The attributes and namespaces associated with an element are not considered to be child nodes of the element, but they are associated with the element by inclusion in the element's attribute and namespace axes. Note that attribute and namespace axes may not directly correspond to the text appearing in the element's start tag in the original document.
Although the XML 1.0 Recommendation states that an XML processor need not provide the text of comments, the XPath data model supports comments, so Canonical XML may include comments. However, since XML 1.0 did not require comments to be provided, comment nodes are excluded by default.
An element has attribute nodes to represent the non-namespace
attribute declarations appearing in its start tag as well as nodes
to represent default attributes that were not specified and not
declared as #implied.
By virtue of the XPath data model, Canonical XML is
namespace-aware [Names], but it cannot
and therefore does not account for namespace equivalencies via
namespace prefix rewriting (see
explanation in appendix). In the XPath data model, each element
and attribute has a name returned by the function
name() which can, at the discretion of the application, be
the QName appearing in the original document. Canonical XML
requires that the XML processor retain sufficient information such
that the QName of the element as it appeared in the original
document can be provided.
An element E has namespace nodes that represent
its namespace declarations, any namespace declarations made by its
ancestor that have not been overridden in E's
declaration, the default namespace if it is non-empty, and the
declaration of the prefix xml. The XPath data model
expects the XML processor to convert relative URIs to absolute
URIs.
Character content is represented in the XPath data model with text nodes. All consecutive characters are placed into a single text node. Furthermore, the text node's characters are represented in the UCS character domain. Canonical XML does not perform character model normalization (see explanation in appendix). However, the XML processor used to prepare the XPath data model input is REQUIRED to use Normalized Form C [NFC] when converting an XML document in a non-Unicode encoding to the UCS character domain.
The Canonical XML generator is specified in terms of producing a canonical form by processing an XPath node-set. The node-set is defined to be the result of setting an initial evaluation context of:
then evaluating the following default expression:
| Default XPath Expression | |
|
which generates a node-set containing every node of the XML document except the comments.
Although an XPath node-set is defined to be unordered, the XPath 1.0 Recommendation [XPath] defines the term document order to be the order in which the first character of the XML representation of each node occurs in the XML representation of the document after expansion of general entities, except for namespace and attribute nodes whose document order is application-dependent.
During XPath expression evaluation, Canonical XML imposes no order on the namespace and attribute axes of elements. After evaluating the expression, a node-set is processed by imposing the following additional document order rules on the namespace and attribute nodes of an element:
Lexicographic comparison is based on the UCS codepoint values, which is equivalent to lexical ordering based on UTF-8.
The XPath node-set is converted into a UTF-8 string by generating the representative text for each node in the node-set in ascending document order with a UTF-8 encoding. No node is processed more than once. Note that processing an element node E includes the processing of all members of the node-set for which E is an ancestor. Therefore, directly after the representative text for E is generated, E and all nodes for which E is an ancestor are removed from the node-set (or some logically equivalent operation occurs such that the node-set's next node in document order has not been processed).
The result of processing a node depends on its type and on whether or not it is in the node-set. If a node is not in the node-set, then no text is generated for the node except for the result of processing its namespace and attribute axes (elements only) and its children (elements and the root node). If the node is in the node-set, then text is generated to represent the node in the canonical form in addition to the text generated by processing the node's namespace and attribute axes and child nodes.
NOTE: The node-set is treated as a set of nodes, not a list of subtrees. To canonicalize an element including its namespaces, attributes, and content, the node-set must actually contain all of the nodes corresponding to these parts of the document, not just the element node.
The text generated for a node is dependent on the node type and given in the following list:
xmlns="". Then, process each
namespace node that is in the element's namespace axis and in the
node-set, except omit the namespace node with local name
xml, which defines the xml prefix, if its
string value is
http://www.w3.org/XML/1998/namespace.
&, all open angle brackets (<) with
<, all double quote characters with
", and the whitespace characters #x9, #xA, and #xD,
with character references. The character references are written in
uppercase hexadecimal with no leading zeroes (for example, #xD is
represented by the character reference
).&, all open angle brackets (<)
are replaced by <, all closing angle brackets
(>) are replaced by >, and all #xD
characters are replaced by 
.<?, the PI target name of the node, a leading
space and the modified string value if the string value is not
empty, and the closing PI symbol (?>). The modified
string value is the string value except all #xD characters are
replaced by 
. If the string value is empty,
then the leading space is not added. Also, a trailing #xA is
rendered after the closing PI symbol for PI children of the root
node with a lesser document order than the document element, and a
leading #xA is rendered before the opening PI symbol of PI children
of the root node with a greater document order than the document
element.<!--), the string value of the node, and the
closing comment symbol (-->). Also, a trailing #xA
is rendered after the closing comment symbol for comment children
of the root node with a lesser document order than the document
element, and a leading #xA is rendered before the opening comment
symbol of comment children of the root node with a greater document
order than the document element.The QName of a node is either the local name if the namespace prefix string is empty or the namespace prefix, a colon, then the local name of the element. The namespace prefix used in the QName MUST be the same one which appeared in the input document.
Some applications require the ability to create a physical representation for an XML document subset (other than the one generated by default, which is technically a document subset because the comments are omitted). Canonical XML implementations based on XPath can provide this functionality with little additional overhead. The following additional steps must be taken:
The result of evaluating X is computed by setting the initial evaluation context as described in the section Canonical XML Data Model, except replacing the variable bindings and namespace declarations with those provided above. If the result of X is not a node-set, then the output is simply the XPath string-value of the result encoded in UTF-8. If the result of X is a node-set, then the output is computed by passing the node-set to the canonical form generator.
The resultant node-set MUST contain a comment node for each comment of the input document, except those comments excluded by the expression X.
The processing of the element node is also modified slightly
when the XPath expression is not the default given in the Canonical XML Data Model. The method for
processing the attribute axis of an element Ein the
node-set is enhanced if the element's parent is not in the
node-set. All element nodes along E's
ancestor axis are examined for nearest occurences of
attributes in the xml namespace, such as
xml:lang and xml:space (whether or not they are
in the node-set). From this list of attributes, remove any that are
in E's attribute axis (whether or not they are in the
node-set). Then, lexicographically merge this attribute list with
the nodes of E's attribute axis that are in the
node-set. The result of visiting the attribute axis is computed by
processing the attribute nodes in this merged attribute list.
NOTE:XML entities can derive application-specific meaning from anywhere in the XML markup as well as by rules not expressed in XML 1.0. Clearly, these rules cannot be specified in this document, so the author of the expression X must be responsible for creating an expression that preserves the information necessary to capture the full semantics of the members of the resulting node-set.
Although this specification now defines Canonical XML in terms of the XPath data model rather than XML InfoSet, the canonical form described in this document is quite similar in most respects to the canonical form described in prior versions of the Canonical XML specification. However, there are some differences. This section discusses the differences and provides a rational for changes.
The Unicode standard [Unicode] allows multiple different representations of certain "precomposed characters" (a simple example is "ç"). Thus two XML documents with content that is equivalent for the purposes of most applications may contain differing character sequences. The W3C has recommended a normalized representation [CharModel]. Prior drafts of Canonical XML used this normalized form. However, most XML 1.0 processors do not perform this normalization. Furthermore, applications that must solve this problem typically perform the character model normalization as character content is created, which would obviate the need for character model normalization during canonicalization. Therefore, character model normalization has been moved out of scope for Canonical XML. However, the XML processor used to prepare the XPath data model input is REQUIRED to use Normalized Form C [NFC] when converting an XML document in a non-Unicode encoding to the UCS character domain.
Prior drafts of the Canonical XML specification described a method for rewriting namespace prefixes such that two documents having logically equivalent namespace declarations would also have identical namespace prefixes. However, the statement in Namespaces in XML that "the prefix functions only as a placeholder for a namespace name" is incorrect. Namespace prefixes can impart information value in an XML document if they are referenced in an attribute value or element content (for example, and element or attribute containing an XPath expression). Thus, rewriting the namespace prefixes would damage such a document by changing its meaning (and it cannot be logically equivalent if its meaning has changed). The theorems below state the results more formally.
Theorem 1: With namespace rewriting, there exist two XML documents D1 and D2 that are logically equivalent yet their canonical forms are not equal.
Proof:Let D1 be a document containing an XPath in an attribute value or element content that refers to namespace prefixes used in D1. Further assume that the namespace prefixes in D1 will all be rewritten by the canonicalization method. Let D2 = D1, then modify the namespace prefixes in D2 and modify the XPath expression's references to namespace prefixes such that D2 and D1 remain logically equivalent. Since namespace rewriting does not include occurences of namespace references in attribute values and element content, the canonical form of D1 does not equal the canonical form of D2 because the XPath will be different. []
Remark:The same condition exists if we remove namespace rewriting. The purpose of this theorem is simply to show that namespace rewriting does not accomplish the goal for which it is intended.
Theorem 2:With namespace rewriting, there exist two XML documents D1 and D2 that have equivalent canonical forms and yet are not logically equivalent.
Proof:Let D1 be a document containing an XPath in an attribute value or element content that refers to namespace prefixes used in D1. Further assume that the namespace prefixes in D1 will all be rewritten by the canonicalization method. Now let D2 = the canonical form of D1. Clearly, the canonical forms of D1 and D2 are equivalent (since D2 is the canonical form of the canonical form of D1), yet D1 and D2 are not logically equivalent because the aforementioned XPath works in D1 and doesn't work in D2. []
Remark:Since D1 and D2 are not logically equivalent, and D2 is the canonical form of D1, we can conclude that namespace rewriting is harmful rather than simply ineffective.
The conclusion to be draw from these theorems is that namespace prefixes should not be altered by XML canonicalization. Applications that need to test for logical equivalence will need to perform more sophisticated tests than mere octet stream comparison. However, this is quite likely to be necessary in any case in order to test for logical equivalencies based on application rules as well as rules from other XML-related recommendations, working drafts, and future works.
Prior drafts of the Canonical XML specification stated that the
default namespace is not used. In the XPath data model, a non-empty
default namespace is indicated by a namespace node with an empty
local name. An empty namespace is indicated by the absence of such
a node. In keeping with the policy of not rewriting namespace
prefixes, which includes not adding prefixes that were not in the
source document, the default namespace system has been added to
Canonical XML. When there is no default namespace node, the
canonicalization method indicates this with xmlns=""
even if the source document did not contain this declaration
explicitly (because there is no way to find out whether it did or
not). The result is logically equivalent but, like the addition of
default attribute nodes, implies that XPath expression authors
should be wary of creating expressions that test for the position
of attribute or namespace nodes (they are bound to fail in most
cases because the sorting of namespace and attribute axes occurs
only on output, not during the XPath expression evaluation).
Prior drafts of the Canonical XML specification alternated between namespace declarations and attribute declarations. This is part of the namespace prefix rewriting scheme, which this specification eliminates. This specification follows the XPath data model of putting all namespace nodes before all attribute nodes.
Prior drafts of the Canonical XML specification placed within a start tag only those namespace declarations used within the start tag. This specification follows the XPath data model of including namespace nodes inherited from ancestor elements. Most XML processors provide the namespace context of each element without indication of the namespaces inherited from ancestors. Though a larger output results, it is simpler to output the namespace context than to develop elaborate rules for omitting portions of the namespace context. The resulting canonical forms achieve the same end.
Moreover, this preservation of the full namespace context is particularly important for document subset canonicalization, especially when the application intends to compute a digital signature digest over the canonical form.
Prior drafts of the Canonical XML specification placed a #xA after each PI outside of the document element as well as a #xA after the end tag of the document element. The method in this specification performs the same function except for omitting the final #xA after the last PI (or comment or end tag of the document element). This technique ensures that PI (and comment) children of the root are separated from markup by a linefeed even if root node or the document element are omitted from the output node-set.
The following people provided valuable feedback that improved the quality of this specification: