RFC 3076

Canonical XML Version 1.0

📅 published March 2001, from ietf

Canonical XML Version 1.0 🎀

Network Working Group 💖 Request for Comments: 3076, so this is RFC 3076, and yes it's giving formal standards-body energy.

Author: J. Boyer of PureEdge Solutions Inc. ✨

Category: Informational 🦋 (it's here to inform, not to be an actual Internet standard, so no pressure lol)

Date: March 2001 🌸

Obsoletes/Updates: the title block doesn't list anything this one obsoletes or updates, so it stands on its own 💅

Part 1 of 7 🧚 more parts are coming, so stay tuned bestie.

Status of This Memo ✨

okay so the vibe here is that this memo is just providing information for the Internet community 🌈 It does not specify an Internet standard of any kind, and the distribution is unlimited, so share it with whoever you want 💜

Copyright (C) The Internet Society (2001). All Rights Reserved. 💅

Abstract ✨

okay so here's the big idea 🦋 Any XML (Extensible Markup Language) document is part of a whole family of XML documents that are logically equivalent inside a given application context, even though they look different on the page because XML 1.0 and Namespaces in XML let you make certain syntactic changes. This spec describes a way to generate a physical representation of an XML document, called the canonical form, that takes those allowed changes into account 🎀 Except for limitations around a few unusual cases, if two documents have the same canonical form, then they're logically equivalent within the given application context. Note that two documents can have different canonical forms and still be equivalent in some context, based on application-specific equivalence rules that no generalized XML spec could ever account for 😭 that's just life, bestie.

Table of Contents 🧚

1. Introduction...............................................  2
1.1 Terminology...............................................  3
1.2 Applications..............................................  4
1.3 Limitations...............................................  4
2. XML Canonicalization.......................................  6
2.1 Data Model................................................  6
2.2 Document Order............................................ 10
2.3 Processing Model.......................................... 10
2.4 Document Subsets.......................................... 13
3. Examples of XML Canonicalization........................... 14
3.1 PIs, Comments, and Outside of Document Element............ 14
3.2 Whitespace in Document Content............................ 15
3.3 Start and End Tags........................................ 16
3.4 Character Modifications and Character References.......... 17
3.5 Entity References......................................... 19
3.6 UTF-8 Encoding............................................ 19
3.7 Document Subsets.......................................... 20
4. Resolutions................................................ 21
4.1 No XML Declaration........................................ 21
4.2 No Character Model Normalization.......................... 21
4.3 Handling of Whitespace Outside Document Element........... 22
4.4 No Namespace Prefix Rewriting............................. 22
4.5 Order of Namespace Declarations and Attributes............ 23
4.6 Superfluous Namespace Declarations........................ 23
4.7 Propagation of Default Namespace Declaration in Document
    Subsets................................................... 24
4.8 Sorting Attributes by Namespace URI....................... 24
Security Considerations....................................... 24
References.................................................... 25
Author's Address.............................................. 26
Acknowledgements.............................................. 27
Full Copyright Statement...................................... 28

1. Introduction ✨

okay so the XML 1.0 Recommendation [XML] is the one that specifies the syntax for a whole class of resources called XML documents 💖 Then the Namespaces in XML Recommendation [Names] adds extra syntax and semantics on top of that 🌸 Here's the tea: XML documents can be equivalent for the purposes of lots of applications and still differ in physical representation. Like, they might have different entity structures, attribute ordering, or character encoding, and it's all still the same vibe 🦋 So the goal of this specification is to come up with a method for figuring out whether two documents are identical, or whether an application has left a document unchanged, apart from the transformations that XML 1.0 and Namespaces allow 🔥

1.1 Terminology 💅

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [Keywords]. 🎀

For the definition of QName, see [Names] 🌈

okay so a document subset is a portion of an XML document that's picked out by a node-set, and that node-set doesn't have to include every node in the document 🧚

The canonical form of an XML document is the physical representation of that document produced by the method in this specification 💜 The changes are summarized in the following list ✨

  • The document is encoded in UTF-8
  • Line breaks normalized to #xA on input, before parsing
  • Attribute values are normalized, as if by a validating processor
  • Character and parsed entity references are replaced
  • CDATA sections are replaced with their character content
  • The XML declaration and document type declaration (DTD) are removed
  • Empty elements are converted to start-end tag pairs
  • Whitespace outside of the document element and within start and end tags is normalized
  • All whitespace in character content is retained (excluding characters removed during line feed normalization)
  • Attribute value delimiters are set to quotation marks (double quotes)
  • Special characters in attribute values and character content are replaced by character references
  • Superfluous namespace declarations are removed from each element
  • Default attributes are added to each element
  • Lexicographic order is imposed on the namespace declarations and attributes of each element

canonical XML is the term for XML that's in canonical form 🦋 The XML canonicalization method is the algorithm defined by this spec that generates the canonical form of a given XML document or document subset 🌸 And the term XML canonicalization refers to the process of applying that method to an XML document or document subset 💖

The XPath 1.0 Recommendation [XPath] defines the term node-set and specifies a data model for representing an input XML document as a set of nodes of various types (element, attribute, namespace, text, comment, processing instruction, and root) 🔥 Nodes get included in or excluded from a node-set based on the evaluation of an expression ✨ In this spec, a node-set is used to directly indicate whether or not each node should be rendered in the canonical form. In this sense it's used as a formal mathematical set 🧚 A node that's excluded from the set doesn't get rendered in the canonical form being generated, even if its parent node is included in the node-set 💅 But here's the twist: an omitted node can still impact the rendering of its descendants (for example, by augmenting the namespace context of the descendants) 😭 main character plot twist.

1.2 Applications 🌸

okay so since the XML 1.0 Recommendation [XML] and the Namespaces in XML Recommendation [Names] define multiple syntactic methods for expressing the same information, XML applications tend to take liberties with changes that have no impact on the information content of the document 💜 XML canonicalization is designed to be useful for applications that need to test whether the information content of a document or document subset has been changed 🎀 It does this by comparing the canonical form of the original document, from before application processing, with the canonical form of the document that comes out of application processing ✨

For example, a digital signature over the canonical form of an XML document or document subset would let the signature digest calculations ignore changes to the original document's physical representation 🦋 That works as long as the changes are defined to be logically equivalent by XML 1.0 or Namespaces in XML 🔥 During signature generation, the digest is computed over the canonical form of the document. Then the document gets transferred to the relying party, which validates the signature by reading the document and computing a digest of the canonical form of the received document 🧚 If the digests computed by the signing and relying parties are equivalent (and so the canonical forms they were computed over are equivalent too), then the information content of the document hasn't been altered since it was signed 💖 iykyk, and it's honestly so iconic.

Next up is section 1.3 Limitations, plus everything from section 2 through section 4 🌈 we'll get there in part 2 bestie 💅

1.3 Limitations 🦋

okay so bestie, two XML documents can have different information content and still be logically equivalent in a given application context 💅 Two XML documents are equivalent (aside from the limitations in this section) if their canonical forms are identical. But it is not a goal of this work to create a method where two XML documents are equivalent if and only if their canonical forms are identical 😭 That method is unachievable, partly because of application-specific rules, like what counts as unimportant whitespace and what counts as equivalent data. For example, <color>black</color> and <color>rgb(0,0,0)</color> mean the same thing but look totally different 🎨 There are also equivalencies established by other W3C Recommendations and Working Drafts, and accounting for those is beyond the scope of this work. The application can apply those rules itself, or they can become the subject of future specifications ✨

The canonical form of an XML document may not be completely operational within the application context, though the circumstances where this happens are unusual 🙈 This could matter in certain applications, since the canonical form of a document and the canonical form of the canonical form of the document are equivalent. For example, in a digital signature application, the canonical form can be swapped in for the original document without changing the digest calculation 🔥 The security risk only shows up in the unusual circumstances described below, and all of those can be resolved, or at least detected, before digital signature generation happens 🫶

The difficulties come from the loss of the following information, which isn't available in the data model:

  1. base URI, especially in content derived from the replacement text of external general parsed entity references
  2. notations and external unparsed entity references
  3. attribute types in the document type declaration

Let's start with the first case 💖 A document containing a relative URI [URI] is only operational when it's accessed from a specific URI that provides the proper base URI. And if the document contains external general parsed entity references to content with relative URIs, those relative URIs won't be operational in the canonical form. That's because the canonical form replaces the entity reference with internal content, which implicitly changes the default base URI of that content 😭 Both of these problems can typically be solved by adding support for the xml:base attribute [XBase] to the application, then adding appropriate xml:base attributes to the document element and all top-level elements in external entities ✨ Also, applications often get the chance to resolve relative URIs before they need a canonical form. For example, in a digital signature application, a document is often retrieved and processed before signature generation. The processing SHOULD create a new document in which relative URIs have been converted to absolute URIs, which mitigates any security risk for the new document 🎀

In the second case, losing external unparsed entity references, along with the notations that bind them to applications, means canonical forms cannot properly tell apart XML documents that bring in unparsed data this way 🌸 This case is unusual precisely because most XML processors currently discard the document type declaration. That discards the notation, the entity's binding to a URI, and the attribute type that binds the attribute value to an entity name. For documents that have to go through more than one XML processor, the XML design typically points to unparsed data using a URI in the attribute value 💜

Now the third case, where losing attribute types can affect the canonical form in different ways depending on the type 💅 Attributes of type ID stop being ID attributes. So any XPath expressions that refer to the canonical form using the id() function stop working 💀 The attribute types ENTITY and ENTITIES aren't part of this case, since they're covered in the second case above. Attributes of enumerated type and of types ID, IDREF, IDREFS, NMTOKEN, NMTOKENS, and NOTATION fail to be appropriately constrained during future attempts to change the attribute value, if the canonical form replaces the original document during application processing 🧚 Applications can avoid the difficulties in this case by making sure an appropriate document type declaration is prepended before the canonical form is used in further XML processing. This is likely to be an easy task, since attribute lists are usually acquired from a standard external DTD subset, and any entity and notation declarations that aren't in the external DTD subset are typically built from application configuration information and added to the internal DTD subset 🍓

While these limitations are not severe, it would be possible to resolve them in a future version of XML canonicalization if, for example, a new version of XPath were created based on the XML Information Set [Infoset] currently under development at the W3C 👑

2. XML Canonicalization ✨

2.1 Data Model ✨

okay so the data model from the XPath 1.0 Recommendation [XPath] is what we use to represent the input XML document or document subset 🌸 Implementations SHOULD but don't have to be built on an XPath implementation 💅 XML canonicalization is defined in terms of the XPath idea of a node-set, so implementations MUST produce equivalent results 🔥

the first input parameter is either an XPath node-set or an octet stream holding a well-formed XML document 🎀 Implementations MUST support the octet stream input and SHOULD also support the document subset feature via node-set input 💖 Since everything gets described in terms of a node-set, this section explains how an octet stream turns into one 🧚

the second input parameter is a boolean flag for whether comments should be included in the output 🦋 If the canonical form contains comments that match the comment nodes in the input node-set, it's called canonical XML with comments ✨ Note that the XPath data model doesn't create comment nodes for comments inside the document type declaration (DTD) 🙈 Implementations are REQUIRED to be capable of producing canonical XML excluding all comments that may have appeared in the input document or document subset 😭 Support for canonical XML with comments is RECOMMENDED 💜

if an XML document has to become a node-set, XPath REQUIRES an XML processor to build the nodes of its data model so they fully represent the document 🫶 The XML processor does these tasks in order:

  1. normalize line feeds
  2. normalize attribute values
  3. replace CDATA sections with their character content
  4. resolve character and parsed entity references

The input octet stream MUST contain a well-formed XML document, but the input doesn't need to be validated 👑 However, attribute value normalization and entity reference resolution MUST be performed in accordance with the behaviors of a validating XML processor 💅 Also, nodes for default attributes (declared in the ATTLIST with an AttValue but not specified) get created in each element 🍓 So the declarations in the document type declaration help build the canonical form, even though the document type declaration itself isn't kept in it 🤯

the XPath data model uses UCS characters, so implementations MUST use XML processors that support UTF-8 and UTF-16 and translate into the UCS character domain 🌈 For UTF-16, the leading byte order mark is treated as an artifact of encoding and gets stripped from the UCS character data (but later zero width non-breaking spaces in the UTF-16 data are left alone) [UTF-16, Section 3.2] ✨ Support for ISO-8859-1 encoding is RECOMMENDED, and all other character encodings are OPTIONAL 🎀

All whitespace within the root document element MUST be preserved (except for any #xD characters deleted by line delimiter normalization) 🧁 This includes all whitespace in external entities. Whitespace outside of the root document element MUST be discarded 🧹

in the XPath data model there are these node types: root, element, comment, processing instruction, text, attribute and namespace 🦋 There's a single root node whose children are processing instruction nodes and comment nodes, which hold information outside the document element (and outside the document type declaration) 💫 The root node also has a single element node for the top-level document element. Each element node can have child nodes of type element, text, processing instruction, and comment. Attributes and namespaces aren't child nodes of the element, but they're tied to it through its attribute and namespace axes 💅 Those axes might not line up exactly with the text in the element's start tag in the original document 👀

Note: An element has attribute nodes to represent the non-namespace attribute declarations appearing in its start tag as well as nodes to represent the default attributes.

by virtue of the XPath data model, XML canonicalization is namespace-aware [Names] 🌸 However, it cannot and therefore does not account for namespace equivalencies using namespace prefix rewriting (see the explanation in Section 4) 🙅 In the XPath data model, each element and attribute has a name returned by the function name(), which can, at the discretion of the application, be the QName that appeared in the original document. XML canonicalization REQUIRES that the XML processor retain enough information that the QName of the element as it appeared in the original document can be provided 🧠

Note: An element E has namespace nodes that represent its namespace declarations as well as any namespace declarations made by its ancestors that have not been overridden in E's declarations, the default namespace if it is non-empty, and the declaration of the prefix xml.
Note: This specification supports the recent XML plenary decision to deprecate relative namespace URIs as follows: implementations of XML canonicalization MUST report an operation failure on documents containing relative namespace URIs. XML canonicalization MUST NOT be implemented with an XML parser that converts relative URIs to absolute URIs. 🚨

character content is represented in the XPath data model with text nodes 💕 All consecutive characters go into a single text node, and the text node's characters are represented in the UCS character domain. XML canonicalization does not perform character model normalization (see the explanation in Section 4). However, the XML processor used to prepare the XPath data model input is REQUIRED to use Normalization Form C [NFC, NFC-Corrigendum] when converting an XML document to the UCS character domain from any encoding that is not UCS-based (currently, UCS-based encodings include UTF-8, UTF-16, UTF-16BE, and UTF-16LE, UCS-2, and UCS-4) ✨

Since XML canonicalization turns an XPath node-set into a canonical form, the first parameter MUST either be an XPath node-set or it must be converted from an octet stream to a node-set by doing the XML processing that creates the XPath nodes described above. After that, you set an initial XPath evaluation context of:

  • A context node, initialized to the root node of the input XML document.
  • A context position, initialized to 1.
  • A context size, initialized to 1.
  • Any library of functions conforming to the XPath Recommendation.
  • An empty set of variable bindings.
  • An empty set of namespace declarations.

and then evaluate the following default expression 🧚‍♀️

     Comment Parameter Value    Default XPath Expression
     -----------------------    ------------------------

     Without (false):
                      (//. | //@* |//namespace::*)[not(self::comment())]

     With (true):
                      (//. | //@* | //namespace::*)

The expressions in this table generate a node-set containing every node of the XML document (except the comments if the comment parameter value is false) 🌷

If the input is an XPath node-set, then the node-set must explicitly contain every node to be rendered to the canonical form 🎯 For example, the result of the XPath expression id("E") is a node-set containing only the node for the element with an ID attribute value of "E". Since none of its descendant nodes, attribute nodes and namespace nodes are in the set, the canonical form would consist solely of the element's start and end tags (minus the attribute and namespace declarations), with no internal content 🙈 Section 3.7 shows how to serialize an identified element along with its internal content, attributes and namespace declarations. 🫧

2.2 Document Order 🌷

Although an XPath node-set is defined to be unordered, the XPath 1.0 Recommendation [XPath] defines document order as the order in which the first character of the XML representation of each node occurs in the XML representation of the document after expansion of general entities. The exception is namespace and attribute nodes, whose document order is application-dependent 🦋

The XML canonicalization method processes a node-set by imposing these extra document order rules on the namespace and attribute nodes of each element:

  • An element's namespace and attribute nodes have a document order position greater than the element but less than any child node of the element.
  • Namespace nodes have a lesser document order position than attribute nodes.
  • An element's namespace nodes are sorted lexicographically by local name (the default namespace node, if one exists, has no local name and is therefore lexicographically least).
  • An element's attribute nodes are sorted lexicographically with namespace URI as the primary key and local name as the secondary key (an empty namespace URI is lexicographically least).

Lexicographic comparison orders strings from least to greatest alphabetically, and it's based on the UCS codepoint values. That's equivalent to lexicographic ordering based on UTF-8 💜

2.3. Processing Model

okay so the XPath node-set gets turned into an octet stream (the canonical form 💖) by making the representative UCS characters for each node in the set, going in ascending document order, and then encoding everything as UTF-8 (without a leading byte order mark) ✨. No node gets processed more than once, ever 💅.

Processing an element node E also covers every node-set member that E is an ancestor of 🫶. So right after E's text is generated, E and all of its descendants come out of the node-set. You can also do something logically equivalent, as long as the node-set's next node in document order hasn't been processed yet. The one twist is that an element doesn't leave the node-set until after its children are processed 🧚.

What a node produces depends on its type and on whether it's in the node-set 🦋. If a node isn't in the node-set, it generates no text of its own. It only passes along what comes out of its namespace and attribute axes (elements only) and its children (elements and the root node) 🌸. If it is in the node-set, it gets its own canonical text too, on top of whatever its namespace axis, attribute axis, and child nodes produce 💜.

Note: The node-set is treated as a set of nodes, not a list of subtrees. If you want to canonicalize an element with its namespaces, attributes, and content, the node-set has to actually contain every node for those parts of the document, not just the element node 🙈.

Here's how each node type generates its text 👑:

  • Root Node: The root node is the parent of the top-level document element. It produces the result of processing each of its child nodes that's in the node-set, in document order. It doesn't generate a byte order mark, an XML declaration, or anything from the document type declaration 🎀.
  • Element Nodes: If the element isn't in the node-set, the result is the namespace axis processed first, then the attribute axis, then the element's child nodes that are in the node-set (in document order) 🍓. If the element is in the node-set, the result is built like this, in order: an open angle bracket (<), the element QName, the result of the namespace axis, the result of the attribute axis, a close angle bracket (>), the result of the child nodes in the node-set (in document order), then an open angle bracket, a forward slash (/), the element QName, and a close angle bracket (>) 🔥.
    • Namespace Axis: Take a list L that holds only the namespace nodes on the axis that are also in the node-set, sorted in lexicographic order (ascending) 🌈. To start processing L, if the first node isn't the default namespace node (a node with no namespace URI and no local name), generate a space followed by xmlns="", but only when both of these are true 💅:
      • the element E that owns the axis is in the node-set
      • the nearest ancestor element of E that's in the node-set has a default namespace node in the node-set (default namespace nodes always have non-empty values in XPath)

That second condition is what stops pointless xmlns="" pieces from showing up in the canonical form. An element only gets xmlns="" if its default namespace is empty and its immediate parent in the canonical form has a non-empty default namespace 🫧. To finish processing L, just process every namespace node in it, except the one with local name xml (the one that defines the xml prefix) when its string value is http://www.w3.org/XML/1998/namespace 🧁.

  • Attribute Axis: Process each node that's on the element's attribute axis and in the node-set, in lexicographic order (ascending) 🍬.
  • Namespace Nodes: A namespace node N gets ignored if the nearest ancestor element of N's parent element that's in the node-set already has a namespace node in the node-set with the same local name and value as N 🙅. Otherwise, process N the same way as an attribute node. The one exception is that the default namespace node (if it exists) gets the local name xmlns, since in XPath it has an empty URI and local name 💖.
  • Attribute Nodes: Output a space, the node's QName, an equals sign, an open double quote, the modified string value, and a close double quote ✨. The string value gets changed like so: every ampersand (&) becomes &amp;, every open angle bracket (<) becomes &lt;, every double quote becomes &quot;, and the whitespace characters #x9, #xA, and #xD become character references. Those character references are written in uppercase hexadecimal with no leading zeroes, so #xD turns into &#xD; 🦄.
  • Text Nodes: Output the string value, but with ampersands turned into &amp;, open angle brackets into &lt;, closing angle brackets (>) into &gt;, and #xD characters into &#xD; 🌷.
  • Processing Instruction (PI) Nodes: Output the opening PI symbol (<?), the PI target name, then a leading space and the string value if it isn't empty, then the closing PI symbol (?>) 🎠. If the string value is empty, the leading space gets skipped. Also, PI children of the root node that come before the document element in document order get a trailing #xA after the closing PI symbol, and PI children of the root node that come after the document element get a leading #xA before the opening PI symbol 🌙.
  • Comment Nodes: If you're generating canonical XML without comments, nothing gets output here. For canonical XML with comments, output the opening comment symbol (<!--), the string value of the node, and the closing comment symbol (-->) 💬. Comment children of the root node follow the same newline rule as PIs: a trailing #xA after the closing comment symbol if they come before the document element, and a leading #xA before the opening comment symbol if they come after it. (Comment children of the root node are the comments that sit outside the top-level document element and outside the document type declaration 🫶.)

The QName of a node is either the local name (when the namespace prefix string is empty), or the namespace prefix, then a colon, then the local name of the element 🧚. The namespace prefix used in the QName MUST be the same one that appeared in the input document 🎀.

2.4. Document Subsets

Some apps need a physical representation of an XML document subset, not just the default one 🦋. (The default one can be a proper subset of the document if the comments get left out.) Canonicalization implementations that are based on XPath can offer this with very little extra overhead, by taking a node-set as input instead of an octet stream 💅.

When a node-set is the input and an element's parent is left out of that node-set, processing an element node E MUST be changed a little 🙈. The attribute axis handling for an element E in the node-set gets an upgrade too. You look at all the element nodes along E's ancestor axis and find the nearest occurrences of attributes in the xml namespace, like xml:lang and xml:space (whether or not those are in the node-set) 🌸. Then you remove any of those attributes that are on E's attribute axis (again, whether or not they're in the node-set). After that, you lexicographically merge this attribute list with the nodes on E's attribute axis that are in the node-set. Visiting the attribute axis means processing the attribute nodes in this merged list 🍓.

Note: XML entities can carry application-specific meaning from anywhere in the XML markup, plus from rules that aren't written down in XML 1.0 or the Namespaces Recommendations. Obviously this document can't spell those rules out, so whoever creates the input node-set has to make sure the information needed to capture the full semantics of the node-set's members gets preserved 🔥.

The canonical XML for a whole XML document is well-formed 👑. The canonical form of a document subset might not be well-formed XML, though 💀. Since the canonical form can get fed into more XML processing, most XPath node-sets that get passed in for canonicalization will be built to produce something that's a well-formed XML document or external general parsed entity 🌈. And whether it comes from a full document or a subset, if the canonical form is well-formed XML, running the same canonicalization method on it again changes nothing ✨.

3. Examples of XML Canonicalization

The examples in this section assume a non-validating processor 🧁. That's mainly so a document type declaration can declare entities, plus default attributes and attributes of various types (like ID and enumerated), without you having to declare every attribute for every element in the document 💖. Also, one example has an element that deliberately breaks a validity constraint (but it's still well-formed, so it's fine) 🎀.

3.1. PIs, Comments, and Outside of Document Element

Here's the input document 🦋:

Input Document
--------------
<?xml version="1.0"?>

<?xml-stylesheet   href="doc.xsl"
   type="text/xsl"   ?>

<!DOCTYPE doc SYSTEM "doc.dtd">

<doc>Hello, world!<!-- Comment 1 --></doc>

<?pi-without-data     ?>

<!-- Comment 2 -->

<!-- Comment 3 -->

And here's the canonical form without comments ✨:

Canonical Form (uncommented)
----------------------------
<?xml-stylesheet href="doc.xsl"
   type="text/xsl"   ?>
<doc>Hello, world!</doc>
<?pi-without-data?>

And the canonical form with comments 💜:

Canonical Form (commented)
--------------------------
<?xml-stylesheet href="doc.xsl"
   type="text/xsl"   ?>
<doc>Hello, world!<!-- Comment 1 --></doc>
<?pi-without-data?>
<!-- Comment 2 -->
<!-- Comment 3 -->

This example demonstrates 🍓:

  • Loss of the XML declaration 🌙
  • Loss of the DTD 🫧
  • Normalization of whitespace outside the document element (the first character of both canonical forms is <, and single line breaks separate PIs and comments outside the document element) 🧚
  • Loss of whitespace between the PITarget and its data 🎀
  • Retention of whitespace inside PI data 🌸
  • Comment removal from the uncommented canonical form, including the delimiter for comments outside the document element (the last character in both canonical forms is >) 👑

3.2 Whitespace in Document Content 🫧

okay so this one is all about whitespace being sacred 🫶 the canonical form keeps every single space, tab and newline exactly where it was. no cap, you do not touch the whitespace 💅

Input Document
--------------
<doc>
   <clean>   </clean>
   <dirty>   A   B   </dirty>
   <mixed>
      A
      <clean>   </clean>
      B
      <dirty>   A   B   </dirty>
      C
   </mixed>
</doc>

Canonical Form
--------------
<doc>
   <clean>   </clean>
   <dirty>   A   B   </dirty>
   <mixed>
      A
      <clean>   </clean>
      B
      <dirty>   A   B   </dirty>
      C
   </mixed>
</doc>

Demonstrates:

  • retain all whitespace between consecutive start tags, clean or dirty ✨
  • retain all whitespace between consecutive end tags, clean or dirty 🌸
  • retain all whitespace between an end tag/start tag pair, clean or dirty 🎀
  • retain all whitespace in character content, clean or dirty 💖

Note: in this example, the input document and canonical form are identical 🙈 both end with the > character, so that's the vibe check passed.

3.3 Start and End Tags 🎀

Input Document
--------------
<!DOCTYPE doc [<!ATTLIST e9 attr CDATA "default">]>
<doc>
   <e1   />
   <e2   ></e2>
   <e3    name = "elem3"   id="elem3"    />
   <e4    name="elem4"   id="elem4"    ></e4>
   <e5 a:attr="out" b:attr="sorted" attr2="all" attr="I'm"
       xmlns:b="http://www.ietf.org"
       xmlns:a="http://www.w3.org"
       xmlns="http://example.org"/>
   <e6 xmlns="" xmlns:a="http://www.w3.org">
       <e7 xmlns="http://www.ietf.org">
           <e8 xmlns="" xmlns:a="http://www.w3.org">
               <e9 xmlns="" xmlns:a="http://www.ietf.org"/>
           </e8>
       </e7>
   </e6>
</doc>

Canonical Form
--------------
<doc>
   <e1></e1>
   <e2></e2>
   <e3 id="elem3" name="elem3"></e3>
   <e4 id="elem4" name="elem4"></e4>
   <e5 xmlns="http://example.org" xmlns:a="http://www.w3.org"

xmlns:b="http://www.ietf.org" attr="I'm" attr2="all"
b:attr="sorted" a:attr="out"></e5>
   <e6 xmlns:a="http://www.w3.org">
       <e7 xmlns="http://www.ietf.org">
           <e8 xmlns="">
               <e9 xmlns:a="http://www.ietf.org" attr="default"></e9>
           </e8>
       </e7>
   </e6>
</doc>

Demonstrates:

  • empty element conversion to a start-end tag pair 🧚
  • normalization of whitespace in start and end tags 🫧
  • relative order of namespace and attribute axes 💅
  • lexicographic ordering of namespace and attribute axes 👑
  • retention of namespace prefixes from the original document 🌈
  • elimination of superfluous namespace declarations 🍓
  • addition of a default attribute ⭐

Note: some of the start tags in the canonical form are super long, but every start tag in this example sits on one single line 📏

Note: in e5, b:attr comes before a:attr because the primary key is the namespace URI, not the namespace prefix 🦋 and attr2 comes before b:attr because the default namespace is not applied to unqualified attributes, so the namespace URI for attr2 is empty 🙈

3.4 Character Modifications and Character References 🦋

Input Document
--------------
<!DOCTYPE doc [
<!ATTLIST normId id ID #IMPLIED>
<!ATTLIST normNames attr NMTOKENS #IMPLIED>
]>
<doc>
   <text>First line&#x0d;&#10;Second line</text>
   <value>&#x32;</value>
   <compute><![CDATA[value>"0" && value<"10" ?"valid":"error"]]>
   </compute>
   <compute expr='value>"0" &amp;&amp; value&lt;"10"
?"valid":"error"'>valid</compute>
   <norm attr=' &apos;   &#x20;&#13;&#xa;&#9;   &apos; '/>
   <normNames attr='   A   &#x20;&#13;&#xa;&#9;   B   '/>
   <normId id=' &apos;   &#x20;&#13;&#xa;&#9;   &apos; '/>
</doc>

Canonical Form
--------------
<doc>
   <text>First line&#xD;
Second line</text>
   <value>2</value>
   <compute>value&gt;"0" &amp;&amp; value&lt;"10" ?"valid":"error"
   </compute>
   <compute expr="value>&quot;0&quot; &amp;&amp; value&lt;&quot;10&quot;
?&quot;
valid&quot;:&quot;error&quot;">valid</compute>
   <norm attr=" '    &#xD;&#xA;&#x9;   ' "></norm>
   <normNames attr="A &#xD;&#xA;&#x9; B"></normNames>
   <normId id="' &#xD;&#xA;&#x9; '"></normId>
</doc>

Demonstrates:

  • character reference replacement 💖
  • attribute value delimiters set to quotation marks (double quotes) 🎀
  • attribute value normalization 🫶
  • CDATA section replacement 🌸
  • encoding of special characters as character references in attribute values (&amp;, &lt;, &quot;, &#xD;, &#xA;, &#x9;) 🔥
  • encoding of special characters as character references in text (&amp;, &lt;, &gt;, &#xD;) 💅

Note: the last element, normId, is well-formed but violates a validity constraint for attributes of type ID 💀 for testing canonical XML implementations based on validating processors, remove the line containing this element from the input and canonical form. in general, XML consumers should be discouraged from using this feature of XML.

Note: whitespace character references other than &#x20; are not affected by attribute value normalization [XML] 🧚

Note: in the canonical form, the value of the attribute named attr in the element norm begins with a space, a single quote, then four spaces before the first character reference 🦋

Note: the expr attribute of the second compute element contains no line breaks ✨

3.5 Entity References 🌈

Input Document
--------------
<!DOCTYPE doc [
<!ATTLIST doc attrExtEnt ENTITY #IMPLIED>
<!ENTITY ent1 "Hello">
<!ENTITY ent2 SYSTEM "world.txt">
<!ENTITY entExt SYSTEM "earth.gif" NDATA gif>
<!NOTATION gif SYSTEM "viewgif.exe">
]>
<doc attrExtEnt="entExt">
   &ent1;, &ent2;!
</doc>

<!-- Let world.txt contain "world" (excluding the quotes) -->

Canonical Form (uncommented)
----------------------------
<doc attrExtEnt="entExt">
   Hello, world!
</doc>

Demonstrates:

  • internal parsed entity reference replacement 🍓
  • external parsed entity reference replacement (including whitespace outside elements and PIs) 🌸
  • external unparsed entity reference 🧚

3.6 UTF-8 Encoding 🍓

Input Document
--------------
<?xml version="1.0" encoding="ISO-8859-1"?>
<doc>&#169;</doc>

Canonical Form
--------------
<doc>#xC2#xA9</doc>

Demonstrates:

  • the effect of transcoding from a sample encoding to UTF-8 💜

Note: the content of the doc element is NOT the string #xC2#xA9 🙈 it's actually the two octets whose hexadecimal values are C2 and A9, which is the UTF-8 encoding of the UCS codepoint for the copyright symbol (c) ©️

3.7 Document Subsets 🧚

Input Document
--------------
<!DOCTYPE doc [
<!ATTLIST e2 xml:space (default|preserve) 'preserve'>
<!ATTLIST e3 id ID #IMPLIED>
]>
<doc xmlns="http://www.ietf.org" xmlns:w3c="http://www.w3.org">
   <e1>
      <e2 xmlns="">
         <e3 id="E3"/>
      </e2>
   </e1>
</doc>

Document Subset Expression
--------------------------
(//. | //@* | //namespace::*)
[ <br/>
   self::ietf:e1 or (parent::ietf:e1 and not(self::text() or self::e2))
   or
   count(id("E3")|ancestor-or-self::node()) =
count(ancestor-or-self::node())
]

Canonical Form
--------------
<e1 xmlns="http://www.ietf.org" xmlns:w3c="http://www.w3.org"><e3
xmlns="" id="E3" xml:space="preserve"></e3></e1>

Demonstrates:

  • empty default namespace propagation from an omitted parent element 🫧
  • propagation of attributes in the xml namespace in document subsets 👑
  • persistence of omitted namespace declarations in descendants 💅

Note: in the document subset expression, the subexpression (//. | //@* | //namespace::*) selects all nodes in the input document, subjecting each to the predicate expression in square brackets 🦋 the expression is true for e1 and its implicit namespace nodes, and it is true if the element identified by E3 is in the ancestor-or-self path of the context node (such that ancestor-or-self stays the same size under union with the element identified by E3) 🔥

Note: the canonical form contains no line delimiters 🫶

4. Resolutions 💜

this section is the tea ☕ it walks through the key decision points and gives a rationale for each one. even though this spec now defines XML canonicalization in terms of the XPath data model instead of the XML Infoset, the canonical form here is pretty similar to the one in the January 2000 Canonical XML draft [C14N-20000119] 👯 there are some differences though, and a bunch of the subsections below spill the details.

4.1 No XML Declaration 🌸

the XML declaration, including the version number and character encoding, is left out of the canonical form ✂️ the encoding isn't needed because the canonical form is always UTF-8 💖 and the version isn't needed either, since having no version number unambiguously means XML 1.0 🧚

future versions of XML will be required to include an XML declaration to indicate the version number 👀 however, the canonicalization method in this spec may not be applicable to future versions of XML without some modifications. when canonicalization of a new version of XML is needed, this spec could be updated to include the XML declaration, since the missing declaration from the XPath data model can presumably be fixed by then (e.g., by reissuing a new XPath based on the Infoset data model) 🌈

4.2 No Character Model Normalization 🫧

okay this one is a little nerdy but stay with me 🤓 the Unicode standard [Unicode] lets you write certain "precomposed characters" in more than one way (a simple example is +U00E7, "LATIN SMALL LETTER C WITH CEDILLA"). so two XML documents can have content that's equivalent for most apps but still contain different character sequences 😭 the W3C is preparing a normalized representation [CharModel], and the C14N-20000119 Canonical XML draft used that normalized form. but tbh lots of XML 1.0 processors don't do this normalization. apps that really need to solve this usually enforce character model normalization from the moment character content gets created, so they don't hit processing failures later (e.g., see the example from Cowan) 🙈 which is why character model normalization has been moved out of scope for XML canonicalization.

the XML processor used to prepare the XPath data model input is required (by the Data Model) to use Normalization Form C [NFC, NFC-Corrigendum] when converting an XML document to the UCS character domain from any encoding that is not UCS-based 🔥 (currently, the UCS-based encodings are UTF-8, UTF-16, UTF-16BE, and UTF-16LE, UCS-2, and UCS-4).

4.3 Handling of Whitespace Outside Document Element 🎀

the C14N-20000119 Canonical XML draft put a #xA after each PI outside the document element, plus a #xA after the end tag of the document element 💌 this spec does the same job, except it leaves off the final #xA after the last PI (or comment, or the end tag of the document element). this trick makes sure that PI (and comment) children of the root are still separated from markup by a line feed, even if the root node or the document element gets left out of the output node-set 🦋

4.4 No Namespace Prefix Rewriting 💅

okay so the C14N-20000119 Canonical XML draft had this idea where it would rewrite namespace prefixes so that two documents with logically equivalent namespace declarations would also end up with identical prefixes 🎀. the goal was to stop prefixes from mattering when checking logical equivalence. but plot twist: there are now a bunch of places where a prefix actually carries information in an XML document 👀. an XPath expression sitting in an attribute value or element content can reference a namespace prefix, so rewriting prefixes would wreck that document by changing what it means. and something whose meaning changed can't be logically equivalent anymore 💀.

here's the formal version of that argument, but make it cute 🦋. let D1 be a document containing an XPath in an attribute value or element content that refers to namespace prefixes used in D1. assume the namespace prefixes in D1 will all get rewritten by the canonicalization method. then let D23D D1, and modify the namespace prefixes in D2 plus the XPath's references to those prefixes so that D2 and D1 stay logically equivalent. namespace rewriting doesn't touch namespace references in attribute values and element content, so the canonical form of D1 does not equal the canonical form of D2, because the XPath is now different 😭. so even though rewriting normalizes the namespace declarations, the goal of getting rid of dependence on particular prefixes is still not achieved 🙃.

it gets worse 💅. it's possible to prove that namespace rewriting is harmful, not just ineffective. take the same D1 (an XPath in an attribute value or element content that refers to namespace prefixes used in D1, with all those prefixes rewritten by the canonicalization method). now let D2 be the canonical form of D1. the canonical forms of D1 and D2 are equivalent, because D2 is the canonical form of the canonical form of D1, yet D1 and D2 are not logically equivalent, since the XPath works in D1 and doesn't work in D2 🔥. yikes.

an argument similar to this can be thrown at the XML canonicalization method based on any of the cases in the Limitations 🙈. the difference is that those problems can't easily be fixed in those cases, whereas here we get a chance to avoid purposefully adding a limitation like this in the first place ✨.

apps that need to test for logical equivalence have to do more sophisticated checks than a plain octet stream comparison 🧚. honestly that's probably needed anyway, since logical equivalence also depends on application rules and on rules from other XML-related recommendations, working drafts, and future works 💜.

4.5 Order of Namespace Declarations and Attributes 🌸

the C14N-20000119 Canonical XML draft alternated between namespace declarations and attribute declarations 😵‍💫. that alternating was part of the namespace prefix rewriting scheme, and this specification gets rid of that scheme. instead, this specification follows the XPath data model, which puts all namespace nodes before all attribute nodes 🫶.

4.6 Superfluous Namespace Declarations 🧹

unnecessary namespace declarations do not show up in the canonical form 🙌. this goes for an empty default namespace, a non-empty default namespace, or a namespace prefix binding. the XML canonicalization method leaves out a declaration if it figures out that the immediate parent element in the canonical form already has an equivalent declaration in scope. the root document element gets special treatment since it has no parent element. every namespace declaration on it is kept, except that a declaration of an empty default namespace is automatically left out 👑.

compared with the method of just rendering the entire namespace context of each element, implementations aren't slowed down by more than a constant factor in processing time and memory use 🚀. the perks are:

  • No more xmlns="" overrun in canonical forms from apps that might not even use namespaces, or that only support them minimally 🙅‍♀️
  • No more namespace declarations on elements where they don't belong according to the app's content model, which makes it way easier to reattach a document type declaration to a canonical form 🎀

one thing to know: in document subsets, an element with omissions in its ancestral element chain gets rendered to the canonical form with namespace declarations that might have been made in those omitted ancestors. that's how it keeps the element's meaning intact 🦋.

4.7 Propagation of Default Namespace Declaration in Document Subsets 🍓

the XPath data model shows an empty default namespace by the absence of a node, not by a default namespace node with an empty value 🤯. so for element e3 in the examples below, which isn't namespace qualified, we can't tell the two versions apart: <e1 xmlns="a:b"><e2 xmlns=""><e3/></e2></e1> versus <e1 xmlns="a:b"><e2><e3 xmlns=""/></e2></e1>. all we know is that e3 wasn't namespace qualified on input. so we keep that info on output when e2 gets omitted, which stops e3 from picking up the default namespace qualification of e1 💅.

4.8 Sorting Attributes by Namespace URI 💜

since the requirement is to preserve the namespace prefixes declared in a document, sorting attributes by prefix (instead of by namespace URI) as the primary key is doable and easier to implement 🌈.

the namespace URI got picked as the primary key anyway, because that's closer to what the XML Names specification intends. that spec identifies namespaces by URI and local name, not by a prefix and local name 🧚. the result of the sort is that all attributes in the same namespace get grouped together ✨.

Security Considerations 🔐

security issues are covered in section 1.3 🫶.

References 📚

Author's Address 🖊️

John Boyer PureEdge Solutions Inc.

Phone: 1-888-517-2675 EMail: jboyer@PureEdge.com

Acknowledgements 🙏

The following people provided valuable feedback that improved the quality of this specification:

  • Doug Bunting, Ariba
  • John Cowan, Reuters
  • Martin J. Durst, W3C
  • Donald Eastlake 3rd, Motorola
  • Merlin Hughes, Baltimore
  • Gregor Karlinger, IAIK TU Graz
  • Susan Lesch, W3C
  • Jonathan Marsh, Microsoft
  • Joseph Reagle, W3C
  • Petteri Stenius, Done360
  • Kent Tamura, IBM

Copyright (C) The Internet Society (2001). All Rights Reserved. 🌈

Okay bestie, the legal bit matters, so let's get it right ✨

Anyone can copy this document and share it with others, in whole or in part, with no permission slip needed 🫶 Translations are welcome too 🌸 You can also write derivative works that comment on or explain the document, or help people implement it, and publish and distribute those however you like. The only condition is that the copyright notice and this paragraph stay on every copy and every derivative work 🎀

The document itself is a different story 🙈 You can't modify it in any way. That includes removing the copyright notice or the references to the Internet Society or other Internet organizations. There are two exceptions: you can change it when you're developing Internet standards, but then you have to follow the copyright procedures in the Internet Standards process, or you can change it when translating it into a language other than English 🧚

These permissions are perpetual, so they never expire, and the Internet Society or its successors or assigns can't take them back 👑

Last but not least, this document and the information in it are provided on an "AS IS" basis, and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 💀

Acknowledgement 🍓

Funding for the RFC Editor function is currently provided by the Internet Society 💖 Thank you, ISOC, you're literally keeping the whole RFC Editor function going 🫶

📦 where this page comes from