RFC 3741

Exclusive XML Canonicalization, Version 1.0

📅 published March 2004, from ietf

RFC 3741: Exclusive XML Canonicalization, Version 1.0 🎀

  • Who wrote it: J. Boyer (PureEdge Solutions), D. Eastlake 3rd (Motorola), and J. Reagle (W3C), all from the Network Working Group 💜
  • Category: Informational 🧚
  • Date: March 2004 📅
  • Obsoletes and updates: the title block doesn't list anything here, so there's nothing it replaces or changes 🌈

Status of This Memo 📋

This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited.

Copyright (C) The Internet Society (2004). All Rights Reserved.

Abstract 💖

Okay so, Canonical XML is a standard way to serialize XML. When you apply it to a subdocument, it drags along the subdocument's ancestor context, meaning all of the namespace declarations and the attributes in the "xml:" namespace. 🙈 Sometimes that's not what you want! Some apps need a method that, as much as is practical, leaves ancestor context out of the canonicalized subdocument.

Here's a real-life example: you want a digital signature over an XML payload (a subdocument) inside an XML message. That signature needs to keep working even when the subdocument gets pulled out of its original message and/or dropped into a totally different context. ✨ That's exactly what Exclusive XML Canonicalization is built for. It slays this requirement 👑

Table of Contents 🗂️

1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  2
    1.1.  Terminology. . . . . . . . . . . . . . . . . . . . . . .  2
    1.2.  Applications . . . . . . . . . . . . . . . . . . . . . .  4
    1.3.  Limitations. . . . . . . . . . . . . . . . . . . . . . .  5
2.  The Need for Exclusive XML Canonicalization. . . . . . . . . .  5
    2.1.  A Simple Example . . . . . . . . . . . . . . . . . . . .  6
    2.2.  General Problems with re-Enveloping. . . . . . . . . . .  7
3.  Specification of Exclusive XML Canonicalization. . . . . . . .  8
    3.1.  Constrained Implementation (non-normative) . . . . . . .  9
4.  Use in XML Security. . . . . . . . . . . . . . . . . . . . . . 10
5.  Security Considerations. . . . . . . . . . . . . . . . . . . . 12
    5.1.  Target Context . . . . . . . . . . . . . . . . . . . . . 12
    5.2.  'Esoteric' Node-sets . . . . . . . . . . . . . . . . . . 13
6.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 13
    6.1.  Normative References . . . . . . . . . . . . . . . . . . 13
    6.2.  Informative References . . . . . . . . . . . . . . . . . 14
7.  Acknowledgements (Informative) . . . . . . . . . . . . . . . . 14
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 15
Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 16

1. Introduction 🌸

Okay, so the XML Recommendation [XML] spells out the syntax for a class of objects called XML documents. The Namespaces in XML Recommendation [XML-NS] adds extra syntax and semantics on top of that. 💅 Here's the thing: XML documents and subdocuments can be equivalent for lots of apps and still look different physically. They might differ in entity structure, attribute ordering, or character encoding. 🦋

So the goal of this spec is to set up a way of serializing the XPath node-set representation of an XML document (or a subset of one) so that:

  1. The node-set is minimally affected by any XML context that has been left out.
  2. The canonicalization of a node-set representing well-balanced XML [XML-Fragment] stays unchanged when exclusive canonicalization gets applied again. 🔁
  3. You can tell whether two node-sets are identical, apart from transformations this specification counts as insignificant under [XML, XML-NS].

Bestie, you need to understand the Canonical XML Recommendation [XML-C14N] first. It's required reading 📚

The World Wide Web Consortium Recommendation that matches this RFC lives at http://www.w3.org/TR/xml-exc-c14n. The errata are at http://www.w3.org/2002/07/xml-exc-c14n-errata. 🌈

1.1. Terminology 💅

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [Keywords]. 🫶

Next up, the XPath 1.0 Recommendation [XPath] defines the term node-set. It also sets up a data model that represents an input XML document as a set of nodes of various types: element, attribute, namespace, text, comment, processing instruction, and root. ✨ Nodes get in or out of a node-set based on whether an expression evaluates to true for them. Inside this spec and [XML-C14N], a node-set directly says whether each node gets rendered in the canonical form. It's used like a formal mathematical set, no vibes involved 🧮 A node that's excluded from the set doesn't get rendered in the canonical form being generated, even if its parent node is included. BUT, an omitted node can still affect how its descendants render, for example by changing the namespace context they see. 👀

A document subset is a portion of an XML document, picked out by an XPath node-set, that might not include every node in the document. As [XPath] defines it, every node (element, attribute, namespace, and so on) has exactly one parent, and that parent is either an element node or the root node. 🌳 An apex node is an element node in a document subset that has no element node ancestor in the document subset. An orphan node is an element node whose parent element node is not in the document subset. The output parent of an orphan node that is not an apex node is the nearest ancestor element of that orphan node that's in the document subset. An apex node has no output parent. The output parent of a non-orphan node is just the node's parent. And an output ancestor is any ancestor element node in the document subset. 🧠

Let's look at an example. Imagine a document tree with three generations under the root node A, where capital letters mean the node is in the document subset. So the subset here is (A,E,G).

Pictorial Representation:

[diagram of nodes,
 http://www.w3.org/TR/xml-exc-c14n/exc-c14n.png]

Textual Representation:

A-+-b
  `-c-+-d
      `-E-+-f
          `-G

Here's what's going on in that tree:

  • A is an apex node, the output parent of E, and an output ancestor of (E,G). 👑
  • E is an orphan node and the output parent of G. 🥰

Now, an element E in a document subset visibly utilizes a namespace declaration (a namespace prefix P and a bound value V) if E, or an attribute node in the document subset whose parent is E, has a qualified name where P is the namespace prefix. A similar definition covers an element E in a document subset that visibly utilizes the default namespace declaration. That happens when E has no namespace prefix. 🎀

The namespace axis of an element holds nodes for all non-default namespace declarations made within that element, plus non-default namespace declarations inherited from its ancestors. The namespace axis also includes a node for the default namespace, as long as it isn't the empty string, whether the default namespace was declared in the element itself or by one of its ancestors. ✨ You can include any subset of the nodes in a namespace axis in a document subset. 🧚

This method of canonicalization takes an InclusiveNamespaces PrefixList parameter. That parameter lists the namespace prefixes that get handled the way the Canonical XML Recommendation [XML-C14N] describes. 📝

The exclusive canonical form of a document subset is a physical representation of the XPath node-set, as an octet sequence, made by the method in this specification. It follows the Canonical XML Recommendation [XML-C14N] except for these changes:

  • attributes in the XML namespace, such as xml:lang and xml:space, are not imported into orphan nodes of the document subset, and
  • namespace nodes that are not on the InclusiveNamespaces PrefixList are expressed only in start tags where they are visible, and only if they are not already in effect from an output ancestor of that tag. 💜

The term exclusive canonical XML refers to XML that's in exclusive canonical form. The exclusive XML canonicalization method is the algorithm this spec defines to generate the exclusive canonical form of a given XML document subset. And exclusive XML canonicalization is the process of applying that method to an XML document subset. 🦋

1.2. Applications 🍓

Exclusive XML Canonicalization has applications that are very similar to the ones for Canonical XML [XML-C14N]. The big difference is that exclusive canonicalization, or an equivalent way of leaving out most XML context, is needed for signature applications where the XML context of the signed XML is going to change. 🔄 This kind of change happens all the time in protocol applications.

Also, for the SignedInfo element of [XML-DSig], picking an appropriate canonicalization method is the only technique available to protect the signature from insignificant changes in physical form and from changes in XML context. 🔐

1.3. Limitations 🙈

Exclusive XML Canonicalization has the limitations of Canonical XML [XML-C14N], plus two more:

  1. The XML being canonicalized may depend on the effect of XML namespace attributes, such as xml:lang, xml:space, and xml:base, that appear in ancestor nodes. To avoid problems from those attributes not being imported into an enveloped document subset, either they MUST be explicitly given in a node of the XML document subset being canonicalized where their effect is needed (or in an ancestor of the node where their effect is needed), or they MUST always be declared with an equivalent value in every context where the XML document subset will be interpreted. 😬
  2. Applications that use the XML being canonicalized may depend on the effect of XML namespace declarations where the namespace prefix being bound is not visibly utilized. An example would be an attribute whose value is an XPath expression, so its evaluation depends on namespace prefixes referenced in that expression. Or an attribute value might be treated as a QName [XML-NS] by some applications, while XPath only sees it as a string-value:
<number xsi:type="xsd:decimal">10.09</number>

To avoid problems with those namespace declarations:

  • the XML MUST be modified so that use of the namespace prefix involved is visible, or
  • the namespace declarations MUST appear and be bound to the same values in every context where the XML will be interpreted, or
  • the prefixes for those namespaces MUST appear in the InclusiveNamespaces PrefixList. ✨

2. The Need for Exclusive XML Canonicalization ✨

okay so this is the whole reason this RFC exists 💖 In some cases, especially for signed XML in protocol apps, you need to canonicalize a subdocument so it's basically independent of whatever XML surrounds it. Why? Because in protocol land it's super common to wrap XML in layers of message or transport elements, strip those wrappers off later, and build brand-new messages out of bits pulled from different messages you got earlier 🧩 If those pieces are signed, the canonicalization has to keep those operations from breaking the signature, while the signature still gives as much security as you can practically get 🔐

2.1. A Simple Example

okay let's look at a tiny example of how changing the XML context can wreck a signature 💀 Here's the first document:

<n1:elem1 xmlns:n1="http://b.example">
    content
</n1:elem1>

and then it gets enveloped in another document:

<n0:pdu xmlns:n0="http://a.example">
   <n1:elem1 xmlns:n1="http://b.example">
       content
   </n1:elem1>
</n0:pdu>

the first one is already in canonical form ✨ but once it's enveloped like the second one, you can pull out the subdocument with elem1 as its apex node using an XPath expression like this:

(//. | //@* | //namespace::*)[ancestor-or-self::n1:elem1]

then if you run Canonical XML on that XPath node-set, you get this (line wrapping added so it fits here):

<n1:elem1 xmlns:n0="http://a.example"
          xmlns:n1="http://b.example">
    content
</n1:elem1>

see the n0 namespace sneaking in? 👀 That's because Canonical XML includes namespace context, so the outer n0 declaration tags along. And that extra little declaration is exactly the kind of change that would break a signature over elem1 from the first version 😭

2.2. General Problems with re-Enveloping

this one is a more complete example of how the canonical form can shift when you change the enveloping context of a document subset 🎀 Here's the first document:

<n0:local xmlns:n0="foo:bar"
          xmlns:n3="ftp://example.org">
   <n1:elem2 xmlns:n1="http://example.net"
             xml:lang="en">
       <n3:stuff xmlns:n3="ftp://example.org"/>
   </n1:elem2>
</n0:local>

and here's the second one, where the enveloping around elem2 has been changed:

<n2:pdu xmlns:n1="http://example.com"
        xmlns:n2="http://foo.example"
        xml:lang="fr"
        xml:space="retain">

   <n1:elem2 xmlns:n1="http://example.net"
             xml:lang="en">
       <n3:stuff xmlns:n3="ftp://example.org"/>
   </n1:elem2>
</n2:pdu>

say we make an XPath node-set from each case using this expression:

(//. | //@* | //namespace::*)[ancestor-or-self::n1:elem2]

running Canonical XML on the node-set from the first document gives this (line wrapping added so it fits):

<n1:elem2 xmlns:n0="foo:bar"
          xmlns:n1="http://example.net"
          xmlns:n3="ftp://example.org"
          xml:lang="en">
    <n3:stuff></n3:stuff>
</n1:elem2>

now here's the plot twist 😱 elem2 is represented by the exact same octets in both pieces of external XML, but the Canonical XML version from the second case comes out like this (again, line wrapping added to fit):

<n1:elem2 xmlns:n1="http://example.net"
          xmlns:n2="http://foo.example"
          xml:lang="en"
          xml:space="retain">
    <n3:stuff xmlns:n3="ftp://example.org"></n3:stuff>
</n1:elem2>

so the context change caused a bunch of changes in the subdocument when it's serialized by the inclusive Canonical XML [XML-C14N] 🫠 In the first example, n0 came in from the context, and the identical n3 declaration in the context got bumped up to the apex of the canonicalized form. In the second example, n0 is gone, n2 showed up, n3 isn't bumped up anymore, and an xml:space declaration appeared, all because of the context changes. But not every context change matters 🙄 In the second example, the xml:lang and n1 prefix namespace declarations at the ancestor nodes do nothing, because elem2 already has its own declarations for them.

now compare that to Exclusive XML Canonicalization as specified here. Using it, the physical form of elem2 extracted by the XPath expression above is this (line wrapping added so it fits):

<n1:elem2 xmlns:n1="http://example.net"
          xml:lang="en">
    <n3:stuff xmlns:n3="ftp://example.org"></n3:stuff>
</n1:elem2>

and that's the same in both cases 🥹 that's the whole magic trick ✨

3. Specification of Exclusive XML Canonicalization 💅

okay the data model, the processing, the input parameters, and the output data for Exclusive XML Canonicalization are all the same as for Canonical XML [XML-C14N], with these exceptions:

  1. Canonical XML applied to a document subset needs to search the ancestor nodes of each orphan element node for attributes in the XML namespace, like xml:lang and xml:space. Those get copied into the element node, unless a declaration of the same attribute is already on the element's attribute axis (whether or not that's in the document subset). Exclusive XML Canonicalization skips this search and copying entirely 🙅‍♀️
  2. The Exclusive XML Canonicalization method may get an extra parameter, InclusiveNamespaces PrefixList, which can be null. It's a list of namespace prefixes and/or a token that says whether the default namespace is present. Every namespace node on this list gets handled the way Canonical XML [XML-C14N] handles it 🎀
  3. A namespace node N whose prefix is not in the InclusiveNamespaces PrefixList gets rendered only when all of these conditions are met:
    1. its parent element is in the node-set, and
    2. it is visibly utilized by its parent element, and
    3. the prefix has not yet been rendered by any output ancestor, or the nearest output ancestor of its parent element that visibly utilizes the namespace prefix does not have a namespace node in the node-set with the same namespace prefix and value as N.
  4. If the token for the default namespace is not in the InclusiveNamespaces PrefixList, the rules for rendering xmlns="" change like this. When you canonicalize the namespace axis of an element E that is in the node-set, output xmlns="" if and only if all of these conditions are met:
    1. E visibly utilizes the default namespace (meaning it has no namespace prefix), and
    2. it has no default namespace node in the node-set, and
    3. the nearest output ancestor of E that visibly utilizes the default namespace has a default namespace node in the node-set. (This step for xmlns="" is needed because it isn't represented in the XPath data model as a namespace node. It's represented as the absence of a namespace node. See Section 4.7 Propagation of Default Namespace Declaration in Document Subsets [XML-C14N].)

3.1. Constrained Implementation (non-normative)

this part is non-normative, so it's a suggested way to implement Exclusive XML Canonicalization for lots of straightforward cases 🧁 It assumes the subset is well-formed, and that if an element is in the node-set then its whole namespace axis is too. If the element isn't in the subset, neither is its namespace axis.

  1. Recursively process the entire tree (the one the XPath node-set was selected from) in document order, starting at the root. (Copying ancestor xml: namespace attributes into output apex element nodes is not done here 🙈)
  2. If the node is not in the XPath subset, keep processing its child element nodes recursively.
  3. If the element node is in the XPath subset, output the node according to Canonical XML, except that namespace nodes get rendered like this:
    1. ns_rendered is a copy of a dictionary, taken from the top of the state stack, of prefixes and their values that an output ancestor of the namespace node's parent element has already rendered.
    2. Render each namespace node if and only if all of these conditions are met:
      1. it is visibly utilized by the immediate parent element or one of its attributes, or it is present in the InclusiveNamespaces PrefixList, and
      2. its prefix and value do not appear in ns_rendered.
    3. Render xmlns="" if and only if all of these conditions are met:
      1. the default namespace is visibly utilized by the immediate parent element node, or the default prefix token is present in the InclusiveNamespaces PrefixList, and
      2. the element does not have a namespace node in the node-set that declares a value for the default namespace, and
      3. the default namespace prefix is present in the ns_rendered dictionary.
    4. Insert all the rendered namespace nodes (including xmlns="") into the ns_rendered dictionary, replacing any existing entries. Then push ns_rendered onto the state stack and recurse.
    5. After the recursion returns, pop the state stack.

4. Use in XML Security ✨

Okay bestie, Exclusive Canonicalization can slide into two spots: as a Transform, or as a CanonicalizationMethod algorithm. Both live in XML Digital Signature [XML-DSig] and XML Encryption [XML-Enc] 💖

Identifier:

http://www.w3.org/2001/10/xml-exc-c14n#

http://www.w3.org/2001/10/xml-exc-c14n#WithComments

Same as with [XML-C14N], you can add the "#WithComments" parameter if you want XML comments serialized too 💬 This algorithm also takes an optional explicit parameter: an empty InclusiveNamespaces element with a PrefixList attribute. The PrefixList is a whitespace-delimited list of namespace prefixes, and #default stands for the default namespace. Those get handled per [XML-C14N]. The list uses NMTOKENS format, which is just a whitespace-separated list. Here's an example 🎀

<ds:Transform
   Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
   <ec:InclusiveNamespaces PrefixList="dsig soap #default"
       xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#"/>
</ds:Transform>

This tells you it's the exclusive canonicalization transform, but the namespaces with the "dsig" or "soap" prefixes, plus the default namespace, get processed according to [XML-C14N] 🧚

Schema Definition (expressed in [XML-schema]):

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE schema
  PUBLIC "-//W3C//DTD XMLSchema 200102//EN"
         "http://www.w3.org/2001/XMLSchema.dtd"
 [
   <!ATTLIST schema xmlns:ec CDATA
             #FIXED 'http://www.w3.org/2001/10/xml-exc-c14n#'>
   <!ENTITY ec 'http://www.w3.org/2001/10/xml-exc-c14n#'>
   <!ENTITY % p ''>
   <!ENTITY % s ''>
  ]>

<schema xmlns="http://www.w3.org/2001/XMLSchema"
        xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#"
        targetNamespace="http://www.w3.org/2001/10/xml-exc-c14n#"
        version="0.1" elementFormDefault="qualified">

  <element name="InclusiveNamespaces"
           type="ec:InclusiveNamespaces"/>
  <complexType name="InclusiveNamespaces">
     <attribute name="PrefixList" type="NMTOKENS"/>
  </complexType>
</schema>

DTD:

<!ELEMENT InclusiveNamespaces    EMPTY >
<!ATTLIST InclusiveNamespaces
   PrefixList    NMTOKENS    #REQUIRED >

5. Security Considerations 🔥

Okay, this part is serious, so let's go 👀 This specification is used to serialize an XPath node-set, and it does that under certain assumptions spelled out in [XML-C14N] and in this spec itself. Three examples of those assumptions:

  1. implementations of [XML-C14N] and this specification do not render an XML declaration;
  2. implementations of this specification only render attributes from the "XML" namespace (e.g., xml:lang, xml:space, and xml:base) when they are in the subset being serialized;
  3. implementations of this specification do not consider the appearance of a namespace prefix within an attribute value to be visibly utilized.

These choices match other XML specifications and meet the Working Group's application requirements, but 🚨 an XML application has to build its transforms very carefully so the result is meaningful and unambiguous in its application context. Besides this section, you also need to pay attention to the Limitations of this specification, the Resolutions of [XML-C14N], and the Security Considerations of [XML-DSig] 💅

5.1. Target Context

The goal of this specification is to satisfy applications that "require a method which, to the extent practical, excludes ancestor context from a canonicalized subdocument." 🦋 So when a fragment gets pulled out of its source instance, this spec leaves out any ancestor context that isn't used. Because of that, a signature [XML-DSig] over the fragment stays valid in its source context, when it's removed from the source context, and even in a brand-new target context. BUT (and this is the tea ☕) this specification does not protect the fragment from being confused in a target context.

Example time: say the <Foo/> element gets signed inside <Bar/><Foo/></Bar>, then gets removed and dropped into <Baz xmlns="http://example.org/bar"/><Foo/></Baz>. The signature should still be valid, but it won't be if <Foo/> ends up being read as part of the http://example.org/bar namespace. Whether that happens depends on how the nodes are processed 😬

This specification doesn't define how to remove, insert, or "fix up" a node-set. (If you want an example of that kind of spec, check out the processing required of Creating the Result Infoset (section 4.5) when an [XInclude] is performed.) Instead, applications have to carefully specify the XML (meaning the source, fragment, and target), or define how the node-set gets processed (removal, replacement, and insertion), with respect to default namespace declarations (e.g., xmlns="") and XML attributes (e.g., xml:lang, xml:space, and xml:base) 🌸

5.2. 'Esoteric' Node-sets

Picture an application that uses this specification or [XML-C14N] to serialize a single attribute node 🤔 Neither spec will emit a namespace declaration for that lone attribute node. So a "carefully constructed" transform should build a node-set that includes both the attribute and its relevant namespace declaration, so serialization comes out right.

This example is a warning: once you move past well-formed [XML] into well-balanced XML [XML-Fragment], it gets harder and harder to produce a result that "is meaningful and unambiguous in its application context." 💀

6. References 📚

6.1. Normative References

6.2. Informative References

7. Acknowledgements (Informative) 🫶

Big thanks to these people, whose feedback made this specification way better:

  • Merlin Hughes, Baltimore
  • Thomas Maslen, DSTC
  • Paul Denning, MITRE
  • Christian Geuer-Pollmann, University Siegen
  • Bob Atkinson, Microsoft

Authors' Addresses 💌

John Boyer

PureEdge Solutions Inc.
4396 West Saanich Rd.
Victoria, BC, Canada V8Z 3E9

Phone: +1-888-517-2675
EMail: jboyer@PureEdge.com

Donald E. Eastlake 3rd

Motorola
155 Beaver Street
Milford, MA 01757 USA

Phone: +1-508-634-2066 (h)
       +1-508-786-7554 (w)
EMail: Donald.Eastlake@motorola.com

Joseph M. Reagle Jr., W3C

Massachusetts Institute of Technology
Laboratory for Computer Science
NE43-350, 545 Technology Square
Cambridge, MA 02139

Phone: +1-617-258-7621
EMail: reagle@mit.edu

Copyright (C) The Internet Society (2004). This document is subject to the rights, licenses and restrictions in BCP 78, and apart from what's set out there, the authors keep all their rights.

This document and the information in it are provided on an "AS IS" basis, and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property 🦋

The IETF doesn't take a position on whether any Intellectual Property Rights or other rights that someone might claim over the technology in this document are valid or how far they reach. It also doesn't say whether any license under those rights is available, and it hasn't made any independent effort to find such rights. Info on the procedures for rights in RFC documents is in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat, any assurances of licenses that will be made available, or the results of any attempt to get a general license or permission for implementers or users to use those proprietary rights can be found through the IETF's online IPR repository at http://www.ietf.org/ipr.

The IETF invites anyone to bring to its attention any copyrights, patents, patent applications, or other proprietary rights that might cover technology needed to implement this standard. Send that info to the IETF at ietf-ipr@ietf.org ✨

Acknowledgement 🌈

Funding for the RFC Editor function is currently provided by the Internet Society. 💜

📦 where this page comes from