RFC 3470

Guidelines for the Use of Extensible Markup Language (XML) within IETF Protocols

📅 published January 2003, from ietf

RFC 3470: Guidelines for the Use of Extensible Markup Language (XML) within IETF Protocols 🎀

Okay bestie, so this one came out in January 2003 and it's a Best Current Practice (that's BCP 70 for the nerds 🤓). It's from the Network Working Group, and the three authors are S. Hollenbeck (VeriSign, Inc.), M. Rose (Dover Beach Consulting, Inc.), and L. Masinter (Adobe Systems Incorporated). The header doesn't list anything it obsoletes or updates, so it's standing on its own 💅

Status of This Memo ✨

This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Distribution of this memo is unlimited.

Copyright (C) The Internet Society (2003). All Rights Reserved.

Abstract 🌸

The Extensible Markup Language (XML) is a framework for structuring data. While it evolved from Standard Generalized Markup Language (SGML) -- a markup language primarily focused on structuring documents -- XML has evolved to be a widely-used mechanism for representing structured data.

There are a wide variety of Internet protocols being developed; many have need for a representation for structured data relevant to their application. There has been much interest in the use of XML as a representation method. This document describes basic XML concepts, analyzes various alternatives in the use of XML, and provides guidelines for the use of XML within IETF standards-track protocols.

Table of Contents 🧚

   Conventions Used In This Document  . . . . . . . . . . . . . . . .  2
   1.    Introduction and Overview  . . . . . . . . . . . . . . . . .  2
         1.1   Intended Audience. . . . . . . . . . . . . . . . . . .  3
         1.2   Scope  . . . . . . . . . . . . . . . . . . . . . . . .  3
         1.3   XML Evolution  . . . . . . . . . . . . . . . . . . . .  3
         1.4   XML Users, Support Groups, and Additional
               Information. . . . . . . . . . . . . . . . . . . . . .  4
   2.    XML Selection Considerations . . . . . . . . . . . . . . . .  4
   3.    XML Alternatives . . . . . . . . . . . . . . . . . . . . . .  5
   4.    XML Use Considerations and Recommendations . . . . . . . . .  7
         4.1   XML Syntax and Well-Formedness . . . . . . . . . . . .  7
         4.2   XML Information Set  . . . . . . . . . . . . . . . . .  7
         4.3   Syntactic Restrictions . . . . . . . . . . . . . . . .  8
         4.4   XML Declarations . . . . . . . . . . . . . . . . . . .  9
         4.5   XML Processing Instructions  . . . . . . . . . . . . .  9
         4.6   XML Comments . . . . . . . . . . . . . . . . . . . . . 10
         4.7   Validity and Extensibility . . . . . . . . . . . . . . 10
         4.8   Semantics as Well as Syntax. . . . . . . . . . . . . . 12
         4.9   Namespaces . . . . . . . . . . . . . . . . . . . . . . 12
               4.9.1 Namespaces and Attributes. . . . . . . . . . . . 13
         4.10  Element and Attribute Design Considerations. . . . . . 14
         4.11  Binary Data and Text with Control Characters . . . . . 16
         4.12  Incremental Processing . . . . . . . . . . . . . . . . 16
         4.13  Entity Declarations and Entity References  . . . . . . 16
         4.14  External References  . . . . . . . . . . . . . . . . . 17
         4.15  URI Processing . . . . . . . . . . . . . . . . . . . . 17
         4.16  White Space  . . . . . . . . . . . . . . . . . . . . . 18
         4.17  Interaction with the IANA  . . . . . . . . . . . . . . 19
   5.    Internationalization Considerations  . . . . . . . . . . . . 19
         5.1   Character Sets and Encodings . . . . . . . . . . . . . 19
         5.2   Language Declaration . . . . . . . . . . . . . . . . . 20
         5.3   Other Internationalization Considerations  . . . . . . 20
   6.    IANA Considerations  . . . . . . . . . . . . . . . . . . . . 21
   7.    Security Considerations  . . . . . . . . . . . . . . . . . . 21
   8.    Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22
   9.    Normative References . . . . . . . . . . . . . . . . . . . . 22
   10.   Informative References . . . . . . . . . . . . . . . . . . . 23
   11.   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 27
   12.   Full Copyright Statement . . . . . . . . . . . . . . . . . . 28

Conventions Used In This Document 📝

This document recommends, as policy, what specifications for Internet protocols -- and, in particular, IETF standards track protocol documents -- should include as normative language within them. The capitalized keywords "SHOULD", "MUST", "REQUIRED", etc. are used in the sense of how they would be used within other documents with the meanings as specified in BCP 14, RFC 2119 [1].

1. Introduction and Overview 🦋

The Extensible Markup Language (XML, [8]) is a framework for structuring data. While it evolved from the Standard Generalized Markup Language (SGML, [30]) -- a markup language primarily focused on structuring documents -- XML has evolved to be a widely-used mechanism for representing structured data in protocol exchanges. See "XML in 10 points" [47] for an introduction to XML.

1.1 Intended Audience 👀

Many Internet protocol designers are considering using XML and XML fragments within the context of existing and new Internet protocols. This document is intended as a guide to XML usage and as IETF policy for standards track documents. Experienced XML practitioners will likely already be familiar with the background material here, but the guidelines are intended to be appropriate for those readers as well.

1.2 Scope 🎯

This document is intended to give guidelines for the use of XML content within a larger protocol. The goal is not to suggest that XML is the "best" or "preferred" way to represent data; rather, the goal is to lay out the context for the use of XML within a protocol once other factors point to XML as a possible data representation solution. The Common Name Resolution Protocol (CNRP, [24]) is an example of a protocol that would be addressed by these guidelines if it were being newly defined. This document does not address the use of protocols like SMTP or HTTP to send XML documents as ordinary email or web content.

There are a number of protocol frameworks already in use or under development which focus entirely on "XML protocol" -- the exclusive use of XML as the data representation in the protocol. For example, the World Wide Web Consortium (W3C) is developing an XML Protocol framework based on SOAP ([45] and [46]). The applicability of such protocols is not part of the scope of this document.

In addition, there are higher-level representation frameworks, based on XML, that have been designed as carriers of certain classes of information; for example, the Resource Description Framework (RDF, [38]) is an XML-based representation for logical assertions. This document does not provide guidelines for the use of such frameworks.

1.3 XML Evolution 📈

XML 1.0 was originally published as a W3C recommendation in February 1998 [35], and was revised in a 2nd edition [8] in October 2000. Several additional facilities have also been defined that layer on the base specification. Although these additions are designed to be consistent with XML 1.0, they have varying levels of stability, consensus, and implementation. Accordingly, this document identifies the major evolutionary features of XML and makes suggestions as to the circumstances in which each feature should be used.

1.4 XML Users, Support Groups, and Additional Information 🫶

There are many XML support groups, with some devoted to the entire XML industry [51], some devoted to developers [52], some devoted to the business applications of XML [53], and many, many groups devoted to the use of XML in a particular context.

It is beyond the scope of this document to provide a comprehensive list of referrals. Interested readers are directed to the three references above as starting points, as well as their favorite Internet search engine.

2. XML Selection Considerations 🔍

XML is a tool that provides a means towards an end. Choosing the right tool for a given task is an essential part of ensuring that the task can be completed in a satisfactory manner. This section describes factors to be aware of when considering XML as a tool for use in IETF protocols:

  1. XML is a meta-markup language that can be used to define markup languages for specific domains and problem spaces.
  2. XML provides both logical structure and physical structure to describe data. Data framing is built-in.
  3. XML instances can be validated against the formal definition of a protocol specification.
  4. XML supports internationalization.
  5. XML is extensible. Unlike some other markup languages (such as HTML), new tags (and thus new protocol elements) can be defined without requiring changes to XML itself.
  6. XML is still evolving. The formal specifications are still being influenced and updated as use experience is gained and applied.
  7. XML does not provide native mechanisms to support detailed data typing. Additional mechanisms (such as those described in Section 4.7) are required to specify abstract protocol data types.
  8. XML is text-based, so XML fragments are easily created, edited, and managed using common utilities. Further, being text-based means it more readily supports incremental development, debugging, and logging. A simple "canned" XML fragment can be embedded within a program as a string constant, rather than having to be constructed.
  9. Binary data has to be encoded into a text-based form to be represented in XML.
  10. XML is verbose when compared with many other structured data representation languages. A representation with element extensibility and human readability typically requires more bits when compared to one optimized for efficient machine processing.
  11. XML implementations are still relatively new. As designers and implementers gain experience, it is not uncommon to find defects in early and current products.
  12. XML support is available in a large number of software development utilities, available in both open source and proprietary products.
  13. XML processing speed can be an issue in some environments. XML processing can be slower because XML data streams may be larger than other representations, and the use of general purpose XML parsers will add a software layer with its own performance costs (though these costs can be reduced through consistent use of an optimized parser). In some situations, processing XML requires examining every byte of the entire XML data stream, with higher overhead than with representations where uninteresting segments can be skipped.

Part 1 of 6 is done, bestie 💖 more parts are coming, so stay tuned ✨

3. XML Alternatives 🦋

okay so this section is basically asking "why XML at all, bestie?" 🤔 like, what else could you even use? this section looks at other popular ways to represent data and sees how XML stacks up against them 💅

This document focuses on guidelines for the use of XML. It is useful to consider why one might use XML as opposed to some other mechanism. This section considers some other commonly used representation mechanisms and compares XML to those alternatives.

For many fundamental protocols, the extensibility requirements are modest and the performance requirements are high enough that fixed binary data blocks are the right representation. Mechanisms such as XML just add bloat 🫠. RFC 3252 [23] describes a humorous example of XML as protocol bloat 💀.

Other representation and extensibility frameworks have also been used successfully within communication protocols ✨. For example, Abstract Syntax Notation 1 (ASN.1) [28] along with the corresponding Basic Encoding Rules (BER, [29]) are part of the OSI communication protocol suite, and have been used in many later communications standards (e.g., the ANSI Information Retrieval protocol [27] and the Simple Network Management Protocol (SNMP, [13])). The External Data Representation (XDR, [14]) and variations of it have been used in many other distributed network applications (e.g., the Network File System (NFS) protocol [22]). With some ASN.1 encoding types, data types are explicit in the representation. With XDR, the data types of components are described externally as part of an interface specification 🧚.

Many other protocols use data structures directly (without data encapsulation) by describing the data structure with Backus Normal Form (BNF, [25]). Lots of IETF protocols use Augmented Backus-Naur Form (ABNF, [16]) 🌸. The Simple Mail Transfer Protocol (SMTP, [21]) is an example of a protocol specified using ABNF 📬.

ASN.1, XDR, and BNF are described here as examples of alternatives to XML for use in IETF protocols. There are other alternatives too, but a complete list of every possible one is beyond the scope of this document 🙈.

Other representation methods may differ from XML in several important ways:

  • Text Encoding and character sets: the character encoding used to represent a formal specification. XML defines a consistent character model based on the Universal Character Set (UCS, [31] and [33]). It requires that XML parsers accept at least UTF-8 [4] and UTF-16 [20], and it allows for other encodings 🌈. While ASN.1 and XDR may carry strings in any encoding, there is no common mechanism for defining character encodings within them. Typically, ABNF definitions are defined in terms of octets or characters in ASCII.
  • Data Encoding: XML is defined as a sequence of characters, rather than a sequence of bytes. XML Schema [42] includes mechanisms for representing some data types (integer, date, array, etc.), but many binary data types are encoded in Base64 [15] or hexadecimal. ASN.1 and XDR have rich mechanisms for encoding a wide variety of data types 🔥.
  • Extensibility: XML has a rich extensibility model, so XML specifications can frequently be versioned independently 👑. Specifications can be extended by adding new element names and attributes (if done compatibly). Other extensions can be added by defining new XML namespaces [9], though there is no standard mechanism in XML to indicate whether or not new extensions are mandatory to recognize 🤷‍♀️. Similarly, there are several techniques available to extend ASN.1 specifications. XDR specifications tend not to be independently extensible by different parties, because the framing and data types are implicit and not self-describing. The extensibility of BNF-based protocol elements needs to be explicitly planned 📋.
  • Legibility of protocol elements: as noted above, XML is text-based, and so it carries the advantages (and disadvantages) of text-based protocol elements. This is typically shared with (A)BNF-defined protocol elements. ASN.1 and XDR use binary encodings, which are not easily human readable 👀.

4. XML Use Considerations and Recommendations 💖

this section is the "okay, here's how to actually use XML without messing it up" part 🫶. it points out a bunch of XML quirks and gives some recommendations 🎀.

This section notes several aspects of XML and makes recommendations for use. Since the 1998 publication of XML version 1 [35], an editorial second edition [8] was published in 2000. This section refers to the second edition 📖.

4.1 XML Syntax and Well-Formedness

XML [8] is defined in terms of a concrete syntax: a sequence of characters, using characters like "<", "=", "&", etc. as delimiters. An instance is XML if and only if it is well-formed, i.e., all character and markup data conforms to the structural rules defined in section 2.1 of [8] 🧱.

Character and markup data that is not well-formed is not XML. Well-formedness is the basis for syntactic compatibility with XML 🔑. Without well-formedness, all of the advantages of using XML disappear 💔. For this reason, it is recommended that protocol specifications explicitly require XML well-formedness ("MUST be well-formed") ✅.

The IETF has a long-standing tradition of "be liberal in what you accept," which might seem to be at odds with this recommendation 🤨. Given that XML requires well-formedness, conforming XML parsers are intolerant of well-formedness errors. When specifying the handling of erroneous XML protocol elements, a protocol design must never recommend attempting to partially interpret non-well-formed instances of an element which is required to be XML 🚫. Reasonable behaviors in such a scenario could include attempting retransmission or aborting an in-progress session 🔄.

4.2 XML Information Set

In addition to the concrete syntax of XML, there is an abstract model of XML content known as the "Information Set" (infoset) [37]. You can think of an XML parser as consuming the concrete syntax and producing an XML Information Set for further processing 🧠.

In typical use of XML, the definition of allowable XML documents is often written in terms of the Information Set of the XML, not the concrete syntax. The idea is that any syntactic representation which yielded the same information set would be treated equivalently ✨.

In some cases, protocols have been defined solely in terms of the XML Information Set, or by allowing other concrete syntax representations. However, since the context of XML embedded within other Internet protocols requires an unambiguous definition of the concrete syntax, defining an XML protocol element in terms of its XML Information Set alone, and allowing other concrete syntax representations, is out of scope for this document 🙅‍♀️.

4.3 Syntactic Restrictions

In some circumstances a protocol designer may be tempted to define an XML-based protocol element as "XML," but at the same time impose additional restrictions beyond those imposed by the XML recommendation itself 😬. For example, they might restrict the document character encoding, or avoid CDATA sections and character entity references, or impose additional restrictions on the use of white space, etc. The general category of restrictions covered by this section are ones that would allow some, but not all, of the syntactic representations which have the same canonical representation according to canonical XML, described in RFC 3076 [6] 🪞.

Making these kinds of restrictions in a protocol definition may mean an implementer of the protocol can't use an otherwise conforming XML processor to parse the XML-based protocol elements 😭. In some cases, the motivation for subsetting XML is to let implementers build special-purpose processors that are lighter weight than a full-scale conforming XML processor. There are a number of good, conforming XML parsers that are small, fast, and free 🆓. Special-purpose processors, on the other hand, have frequently been known to fail to handle some cases of legal XML syntax 💀.

In general, such syntactic restrictions should be avoided. In circumstances where restrictions on the variability of the syntactic representation of XML are necessary for one reason or another, designers should consider using "Canonical XML" [6] as the definition of the protocol element, since all such variability has been removed 🧼. Some specific issues are discussed in Section 4.4, Section 4.13, and Section 5.1 below 👀.

4.4 XML Declarations

An XML declaration (defined in section 2.8 of [8]) is a small header at the beginning of an XML data stream that indicates the XML version and the character encoding used. For example:

<?xml version="1.0" encoding="UTF-8"?>

This specifies the use of XML version 1 and UTF-8 character encoding 🌸.

In some uses of XML as an embedded protocol element, the XML is a small fragment inside a larger context, where the XML version is fixed at "1.0" and the character encoding is known to be "UTF-8." In those cases, an XML declaration might add extra overhead 📏. In cases where the XML is a larger component that may find its way alone as an external entity body (transported as a MIME message, for example), the XML declaration is an important marker and is useful for reliability and extensibility 🛡️. The XML declaration is also an important marker for character set/encoding (see Section 5.1), if any encoding other than UTF-8 or UTF-16 is used 🔡. Note that in the case of UTF-16, XML requires that the entity starts with a Byte Order Mark (BOM), which is not part of the character data 🔍. Also note that the XML Declaration itself is not part of the XML document's Information Set 🤓.

Protocol specifications must be clear about the use of XML declarations 📣. XML [8] notes that "XML documents should begin with an XML declaration which specifies the version of XML being used." In general, an XML declaration should be encouraged ("SHOULD be present") and must always be allowed ("MAY be sent") 🙌. An XML declaration should be required in cases where, if allowed, the character encoding is anything other than UTF-8 or UTF-16 🔐.

4.5 XML Processing Instructions

An XML processing instruction (defined in section 2.6 of [8]) is a component of an XML document that signals extra "out of band" information to the receiver 📨. A common use of XML processing instructions is for document applications. For example, the XML2RFC application, used to generate this document and described in RFC 2629 [19], supports a "table of contents" processing instruction:

<?rfc toc="yes"?>

As described in section 2.6 of [8], processing instructions are not part of the document's character data, but must be passed through to the application 🚚. As a consequence, it is recommended that processing instructions be ignored when encountered in normal protocol processing. It is also recommended that processing instructions not be used to define normative protocol data structures or extensions, for the following reasons 🙃:

  • Processing instructions are not namespace aware. There is no way to qualify a processing instruction target with a namespace 🏷️.
  • Processing instruction use cannot be constrained by most schema languages,
  • Character references are not recognized within a processing instruction 🙅.
  • Processing instructions don't have any XML-defined structure beyond the division between the target and everything else. This means that applications typically have to parse the content of the processing instruction in a system-dependent way. If the content were provided within an element instead, the structure could be expressed in the XML and the parsing could be done by the XML parser 🧩.

4.6 XML Comments ✨

okay so XML comments 💬 are a part of the document that's basically just notes for humans. they're not part of the actual character data, and they're very much like the comments you'd write in code. here's a cute example:

<!-- This is a example comment.  -->

the big thing to know: conformant processors are allowed to ignore comments 🙈 so it is strongly recommended that comments not be used to define normative protocol data structures or extensions. it's also strongly recommended that comments be ignored if you run into them during normal protocol processing. don't put your actual rules in a comment bestie, that's giving "trust me bro" and nobody is obligated to read it 💀

4.7 Validity and Extensibility 🦋

okay this section is my favorite because XML has actual formal ways to say "this element has to look like THIS" 🤓 these constrain which elements or attributes are allowed to exist and what values they can hold. there's more than one formalism, and they're all kinda different:

  • a "Document Type Definition" (DTD) is defined in section 2.8 of [8]. it came from a similar mechanism in SGML, and there's significant experience using DTDs, including in IETF protocols 🎀
  • XML Schema (defined in [41] and [42]) gives you extra features for a tighter and more precise description of allowed protocol syntax and data types 💅
  • there are also other ways to describe XML instance validity, like Schematron [49] and RELAX NG [48]. Part 2 of the ISO/IEC Document Schema Definition Language (DSDL, [32]) standard is based on RELAX NG 🌸

ngl there's ongoing drama (and controversy) in the XML community about which validity mechanisms to use and where they fit. the right choice depends on how much extensibility you need versus how much you want a formal language that constrains values and checks that they're followed 👑

here's a spicy one: there are cases where protocols defined validity with one of these mechanisms but never insisted that every protocol element actually be "valid" 😭 the decision depends partly on the design for extensibility. every formalism handles future extensions differently, and a protocol might also have its own versioning, its own way of updating the schema, or its own way of pointing to a new one. for example, using XML namespaces (Section 4.9) with XML Schema lets you add other kinds of extensibility without breaking schema validity ✨

no matter which formalism you pick, there are usually extra syntactic constraints, and inevitably extra semantic constraints, on whether an XML element is valid. those can't be expressed in the formalism itself 🥲

so this document has some recommendations for protocols using XML, and they're giving clear-rules energy:

  • protocols should use an appropriate formalism for defining validity of XML protocol elements 🫶
  • protocols may or may not insist that all corresponding protocol elements be valid, according to the validity mechanism chosen. either way, the extensibility design should be clear. what happens if the data is not valid? 👀
  • as described in Section 3, there is no standard mechanism in XML for indicating whether new extensions are mandatory to recognize. so XML-based protocol specs should explicitly describe their extension mechanisms and the requirements to recognize or ignore extensions 🔥

an idealized model for XML processing might first check for well-formedness. if that's OK, it applies the primary formalism and, if the instance "passes", applies the other constraints, so the whole set (or as much as is machine processable) gets checked at the same time 🧚

however, it's totally reasonable for conforming implementations to skip validation at run-time and use ad-hoc code instead. that avoids the higher expense of things like schema validation, especially since there will probably be extra hand-crafted semantic validation anyway 💖

4.8 Semantics as Well as Syntax 🧠

ok this one is important bestie. defining a protocol element with a validity formalism is useful, but it's not enough 🙅‍♀️ XML by itself doesn't supply semantics, so meaning has to come from somewhere else.

Any document defining a protocol element with XML MUST also have sufficient prose in the document describing the semantics of whatever XML the document has elected to define. ⚠️

so if you define XML, you MUST write the prose that explains what it means. no vibes-only protocols 💅

4.9 Namespaces 🌈

XML namespaces, defined in [9], are how you assign markup to a specific vocabulary 📚 if two elements or attributes from different vocabularies have the same name, they can still be told apart unambiguously as long as they belong to different namespaces. namespaces also give protocol extensibility a big boost, because they can be defined, reused, and processed dynamically 🦋

markup vocabulary collisions are very possible when namespaces aren't used to separate and uniquely identify vocabularies 😬 so protocol definitions should use existing XML namespaces where it makes sense. when a new namespace is needed, its "namespace name" is a URI that identifies it, and it's also handy for that URI to point to a description of the namespace 🎀 the typical practice (and the recommended practice in W3C) is to assign namespace names using persistent http URIs.

for namespaces in IETF standards-track documents, it would be nice to have a permanent part of the IETF's own web space for this 🏠 until then, other permanent URIs can be used, e.g., URNs in the IETF URN namespace (see [11] and [12]). there are instances of IETF specs creating brand new URI schemes to define XML namespaces, but that practice is strongly discouraged 🙅‍♀️

4.9.1 Namespaces and Attributes 🫧

this part gets misunderstood ALL the time 😭 people assume an unprefixed attribute is qualified by the default namespace. it's not! an unprefixed attribute belongs to no namespace at all. check out this example:

<ns1:fox a="xxx" ns1:b="qqq"
 xmlns="http://example.org"/>
<fox a="xxx" ns1:b="qqq"
 xmlns="http://example.org" xmlns:ns1="http://example.org"/>

so the attribute "a" is in no namespace, while "ns1:b" is in the same namespace as its containing element 🧚 a specific description of how default namespaces and attributes relate is in section 5.2 of [9]. the practical takeaway: be careful that no element has multiple attributes with identical names, or with qualified names that have the same local part and prefixes bound to identical namespace names ⚠️

in XML apps, choosing between prefixed and non-prefixed attributes usually comes down to this: do they always show up inside elements from the same namespace (then non-prefixed, and so non-namespaced, names are used)? or do they need to work on elements from other arbitrary namespaces (then a prefixed name is used)? both situations happen in the XSLT [43] language 🌸 attributes are unprefixed inside elements in the XSLT namespace, like this:

<xsl:value-of select="."/>

but they're prefixed when they show up in non-XSLT elements, like the "xsl:version" attribute in "literal result element stylesheets":

<html xsl:version="1.0"
 xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
 xmlns="http://www.w3.org/TR/xhtml1/strict">
   <head>
     <title>Expense Report Summary</title>
   </head>
   <body>
     <p>Total: <xsl:value-of select="exp-rep/total"/></p>
   </body>
</html>

4.10 Element and Attribute Design Considerations 🎨

XML gives designers a LOT of freedom 🤯 you can carry data in elements, attributes, or element content. this section just gives a flavor of the design considerations (there's a whole pile of XML literature on this). the big thing is that consistent use of elements, attributes, and values is a key trait of a sound design 💜

attributes are generally meant to hold meta-data that describes the element, so they come with these restrictions:

  • attributes are unordered,
  • there can be no more than one instance of a given attribute within a given element, though an attribute may contain several values separated by white space ([8], section 2.3 and 3.3.1),
  • attribute values can have no internal XML markup for providing internal structure, and
  • attribute values are normalized ([8], section 3.3) before processing

let's look at an example describing an IP address with an attribute for the address value:

<address addrType="ipv4">10.1.2.3</address>

you could also encode the same info with an <addrType> element instead of an "addrType" attribute:

<address>
  <addrType>ipv4</addrType>
  <value>10.1.2.3</value>
</address>

or use markup for the "addrType" 🤓:

<address>
  <addrType><ipv4/></addrType>
  <value>10.1.2.3</value>
</address>

picking between these designs is all about tradeoffs 🍓 things like the likely extensibility patterns and how well the formalism can constrain values. in the first example the attribute can be thought of as meta-data about the element it modifies, which gives a kind of "element extensibility". the third example allows a different kind of extensibility: the "ipv4" space can be extended using other namespaces, and the <ipv4> element can include additional markup ✨

many protocols have parameters picked from an enumerated set of values. those can be encoded as elements, attributes, or strings inside element values. any protocol design should think about how that set gets extended: by revising the protocol, by including different values in different XML namespaces, or by establishing an IANA registry (as per RFC 2434 [18]) 📋 also, a common XML practice is to use a URI as an XML attribute value or as content 🔗

validity languages (including XML Schema and DTDs) often let you specify "default" values for an attribute. if an element doesn't specify a value for the attribute, the "default" value gets used. this document discourages using default values for attributes 🙅‍♀️ even though the feature can reduce the size and clutter of XML documents, it hurts software that doesn't know the document's validity constraints (e.g., for packet tracing or digital signature) 😭

4.11 Binary Data and Text with Control Characters 🧚

okay bestie, big reveal: XML is a character stream, not a stream of octets 🤯 There is no way to drop raw binary data straight into an XML data stream, so all binary data has to be encoded as characters 💅 There are a few options. XML Schema [42] defines encodings using decimal digits for integers, Base64 [15], or hexadecimal digits. Binary data can also travel over some other communication channel and get referenced inside the XML data with a URI 🔗

Protocols that want one container for both structural data and large amounts of binary data should think carefully about whether XML is the right tool, because Base64 and hex are inefficient 😭 Otherwise, protocols should use the XML Schema mechanisms for representing binary data. Base64 is best for larger quantities of data 🫶

XML does not allow "control" characters (0x00-0x1F) except for TAB (0x09), CR (0x0A), and LF (0x0D). They can not be specified even using character entity references 🙈 There's no common way to encode them inside otherwise ordinary text yet. So strings that count as "text" in an ABNF-defined protocol element may need to be treated as binary data in an XML representation, or someone might need to invent some other encoding mechanism 🌸

4.12 Incremental Processing 🦋

in some situations, you can process an XML document incrementally, as each tag arrives. It's giving browsers rendering HTML pages bit by bit as they load ✨ Just know that incremental processing gets hard to implement when it's spread across multiple interactions. If a protocol needs incremental processing in both directions of a bidirectional stream, it may put an unusual burden on protocol implementers 😅

4.13 Entity Declarations and Entity References 🎀

on top of being a validity mechanism, an XML DTD gives you "entity declarations" ([8], section 4.2). An entity declaration defines a kind of macro inside the DTD. An "entity reference" can then call it up and pull in its content 🧩

this adds complexity to XML processing, and it feels more at home in document processing than in data representation. So this document recommends avoiding entity declarations in protocol specifications 🙅‍♀️

BUT, XML has five standard entity references built in: &amp;, &lt;, &gt;, &apos;, and &quot;. XML can also write character data using numeric entity references (using the Unicode [33] value for the character). Entity references are normally expanded before the XML Information Set is computed. Restricting them would add an extra syntactic restriction (see Section 4.3) for no good reason, so these entity references should be allowed 💖

4.14 External References 👑

when you're using XML in a stateless protocol, whether that's the protocol itself (e.g., SOAP) or just content carried by an existing protocol (e.g., XML/HTTP), be super careful not to make a message's meaning depend on information outside the message itself 🚨 XML provides external entities (see Section 4.13), which are an easy way to make a message's meaning depend on something external. Using schema languages that can change the Infoset, like XML Schema, is another way to do it 💀

4.15 URI Processing 🌈

the XML Base specification [36] defines an attribute "xml:base" in the XML namespace. It's meant to affect the "base" used for relative URI processing, as described in RFC 2396 [17]. xml:base can be really useful for protocol designers who want to control URI processing. But if xml:base is allowed, its interaction with any other protocol facilities for establishing URI context must be specified clearly ✍️ Also, the W3C has deprecated relative URIs in namespace declarations. Some specific issues with relative URIs in namespace declarations and canonical XML are covered in section 1.3 of RFC 3076 [6] 🍓

and this one's a plot twist: in many cases, "URI" and how URIs are used inside XML allow non-ASCII characters. For example, the XML Schema "anyURI" datatype ([42] section 3.2.17) allows direct encoding of characters outside the US-ASCII range. Most current IETF protocols and specifications don't allow that syntax 🙅‍♀️ So protocol specs should be clear about the range of characters they allow. They can add a restriction to the range of characters allowed in the anyURI schema datatype, or specify that characters outside the US-ASCII range should be escaped when passed to older protocols or APIs 🔐

4.16 White Space 🫧

XML's white space handling can be a total source of confusion between protocol designers and implementers 😵‍💫 In XML instances, all white space is considered significant and is visible to processing applications by default. Check out this example from Section 4.10:

<address>
  <addrType><ipv4/></addrType>
  <value>10.1.2.3</value>
</address>

This fragment has an <address> element with two child elements. It also has white space for pretty-printing purposes:

  • at least three line separators, which the XML processor will convert to newline (U+000A) characters (see section 2.11 of [8]), and
  • one or more white space characters prefixing the <addrType> and <value> elements, which an XML processor will make visible to software reading the instance.

implementers might safely assume they can ignore the white space in the example above. But pretty-printing white space can cause confusion in other situations 🙃 Now consider a small change to the <value> element:

<value>
  10.1.2.3
</value>

Here, white space sits on both sides of the IP address. XML processors treat the white space around "10.1.2.3" as an integral part of the <value> element 😳 Missing this behavior can lead to confusion and errors in both design and implementation.

all white space is significant in XML instances. So it's recommended that protocol designers give specific guidelines for white space handling in any protocol that uses XML 📣

4.17 Interaction with the IANA 🗂️

when XML is used in an IETF protocol, there are a few factors that might need IANA action, including:

  • XML media types. A piece of XML in a protocol element is sometimes tied to the protocol context it appears in. It might even be directly derived from, or input to, protocol state-machine implementations. If the XML content has no meaning outside its original protocol context, there's no reason to register a MIME type. But if that XML content could be interpreted outside its original context (like when it's stored in a file system or tunneled over another protocol), a MIME type can be registered to specify the format and give a hint about how to process it 📁

If MIME labeling is needed, the advice of RFC 3023 [5] applies. In particular, if the XML represents a new language or document type, a new MIME media type should be registered, for the reasons described in RFC 3023 sections 7 and A.1. When XML encodes generic structured data (e.g., a document-oriented application that combines XML with a stylesheet), "application/xml" MAY be used. The "text/xml" media type is not recommended, and it SHOULD NOT be used because of issues with display behavior and default charsets 🙅‍♀️

  • URI registration. There's an ongoing effort ([11], [12]) to create a URN namespace for defining URIs for namespace names and other URI-designated protocol elements, for use within IETF standards track documents. It might also establish IETF policy for that use 🌐

5. Internationalization Considerations 🌍

this section covers internationalization considerations for using XML to represent data in IETF protocols. On top of the recommendations here, IETF policy on character sets and languages in RFC 2277 [3] also applies 💜

5.1 Character Sets and Encodings 🔤

IETF protocols often talk about the "character set" or "charset" of a string. That term covers both the character repertoire and the encoding used to represent sequences of characters as sequences of bytes 🧠

XML does all its character processing in terms of the Universal Character Set (UCS, [31] and [33]). XML requires all XML processors to support both the UTF-8 [4] and UTF-16 [20] encodings of UCS, although other encodings (charsets) compatible with UCS may be allowed. Documents and external parsed entities encoded in UTF-16 are required to begin with a Byte Order Mark ([8] section 4.3.3) ✨

IETF policy [3] requires that the UTF-8 charset be allowed for all text 💖

This document requires that IETF protocols using XML allow the UTF-8 encoding of XML data. Since conforming XML processors are also mandated to accept UTF-16 encoding, allowing UTF-16 encoding (with the mandated Byte Order Mark) is recommended too 🌸 Some XML applications use a Byte Order Mark with UTF-8 encoding, but that use shouldn't be encouraged, and it isn't appropriate for XML embedded in other protocols 🙅‍♀️

Restricting XML data to only UTF-8 is an additional syntactic restriction (see Section 4.3). Depending on the circumstances, it might add implementation complexity 🧩 When encodings other than UTF-8 or UTF-16 are used, the encoding must be specified using an "encoding" attribute in the XML declaration (see Section 4.4), even if other protocol mechanisms exist for designating the encoding 📝

5.2 Language Declaration 🗣️

text wrapped in XML can be written in lots of different human languages, and it's often super useful to explicitly say which language is being used. XML defines a special attribute in the "xml" namespace, xml:lang, that specifies the language used to represent data in an XML document. The xml:lang attribute has to be explicitly declared for use within a DTD or XML Schema. Its possible values are defined in section 2.12 of [8] 🌷

It is strongly recommended that protocols representing data in a human language mandate use of an xml:lang attribute if the XML instance might be interpreted in language-dependent contexts 🫶

5.3 Other Internationalization Considerations ✨

Some human languages have typography tricks that plain XML character string data types just can't handle on their own 💅. Pronunciation clues can be added with Ruby annotation [39], and embedding controls (like the ones in section 3.4 of [34]) or an XHTML [40] dir attribute can flag the right display direction for bidirectional text 🔥.

There are also a bunch of tricky issues that pop up when you use extended character sets with XML document formats, ngl 💀. For example:

  • There are different ways of representing characters made of combining characters, and
  • There has been some debate about whether URIs should be represented using a restricted US-ASCII subset or arbitrary Unicode (e.g., "URI character sequence" vs "original character sequence" in RFC 2396 [17]).

Some of these issues come with recommendations in the W3C's "Character Model for the World Wide Web" document [44] 🌸.

Okay so, it is strongly recommended that protocols representing data in a human language reuse existing mechanisms as needed to ensure proper display of human-legible text 🫶.

6. IANA Considerations 🦋

This memo, per se, has no impact on the IANA. Section 4.17 notes some factors that might require IANA action when protocols using XML are defined 💜.

7. Security Considerations 🚨

Network protocols face a LOT of threats, including unintended disclosure, modification, and replay 😭. Passive attacks, like packet sniffing, let an attacker capture and view information meant for someone else 👀. Captured data can be modified and replayed to the original intended recipient, and the recipient has no way to know the information has been compromised, detect modifications, be assured of the sender's identity, or confirm which protocol instance is legitimate.

Several security service options for XML are available to help mitigate these risks 🔐. XML doesn't have any built-in security services, but other protocols and protocol layers provide services that can protect XML protocols. XML encryption [10] provides privacy services to prevent unintended disclosure. Canonical XML [6] and XML digital signatures [7] provide integrity services to detect modification, plus authentication services to confirm the identity of the data source. Other IETF security protocols (e.g., the Transport Layer Security (TLS) protocol [2]) are also available to protect data and service endpoints as appropriate 🛡️.

Given the lack of security services in XML, it is imperative that protocol specifications mandate additional security services to counter common threats and attacks. The specific required services will depend on the protocol's threat model 💀.

Experience has shown that code that parses network traffic is often a "soft target" for blackhats 😈. So implementers MUST take great care to ensure that their XML handling code is robust with respect to malformed XML, buffer overruns, misuse of entity declarations, and so on. No skipping this one, bestie 🚫.

XML mechanisms that follow external references (Section 4.14) may also expose an implementation to various threats by causing the implementation to access external resources automatically. It is important to disallow arbitrary access to such external references within XML data from untrusted sources. Many XML grammars define constructs using URIs for external references; in such cases, the same precautions must be taken. This is a MAJOR danger zone, periodt 🚨

8. Acknowledgements 💖

The authors would like to thank the following people, who have provided significant contributions to the development of this document 🙌:

Mark Baker, Tim Berners-Lee, Tim Bray, James Clark, Josh Cohen, John Cowan, Alan Crouch, Martin Duerst, Jun Fujisawa, Christian Geuer-Pollmann, Yaron Goland, Graham Klyne, Dan Kohn, Rick Jeliffe, Chris Lilley, Murata Makoto, Michael Mealling, Jean-Jacques Moreau, Andrew Newton, Julian Reschke, Jonathan Rosenberg, Miles Sabin, Rich Salz, Peter Saint-Andre, Simon St Laurent, Margaret Wasserman, and Daniel Veillard ✨👑.

9. Normative References 📚

  • [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [2] Dierks, T. and C. Allen, "The TLS Protocol Version 1.0", RFC 2246, January 1999.
  • [3] Alvestrand, H., "IETF Policy on Character Sets and Languages", BCP 18, RFC 2277, January 1998.
  • [4] Yergeau, F., "UTF-8, a transformation format of ISO 10646", RFC 2279, January 1998.
  • [5] Murata, M., St. Laurent, S. and D. Kohn, "XML Media Types", RFC 3023, January 2001.
  • [6] Boyer, J., "Canonical XML Version 1.0", RFC 3076, March 2001.
  • [7] Eastlake, D., Reagle, J. and D. Solo, "(Extensible Markup Language) XML-Signature Syntax and Processing", RFC 3275, March 2002.
  • [8] Bray, T., Paoli, J., Sperberg-McQueen, C. and E. Maler, "Extensible Markup Language (XML) 1.0 (2nd ed)", W3C REC-xml, October 2000, <http://www.w3.org/TR/REC-xml>.
  • [9] Bray, T., Hollander, D. and A. Layman, "Namespaces in XML", W3C REC-xml-names, January 1999, <http://www.w3.org/TR/REC-xml-names>.
  • [10] Imamura, T., Dillaway, B., Schaad, J. and E. Simon, "XML Encryption Syntax and Processing", W3C REC-xmlenc-core, October 2001, <http://www.w3.org/TR/xmlenc-core/>.

10. Informative References 🌈

  • [11] Masinter, L., Mealling, M., Klyne, G. and T. Hardie, "An IETF URN Sub-namespace for Registered Protocol Parameters", Work in Progress.
  • [12] Mealling, M., "The IETF XML Registry", Work in Progress.
  • [13] Case, J., Fedor, M., Schoffstall, M. and C. Davin, "Simple Network Management Protocol (SNMP)", STD 15, RFC 1157, May 1990.
  • [14] Srinivasan, R., "XDR: External Data Representation Standard", RFC 1832, August 1995.
  • [15] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996.
  • [16] Crocker, D. (Ed.) and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, November 1997.
  • [17] Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform Resource Identifiers (URI): Generic Syntax", RFC 2396, August 1998.
  • [18] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 2434, October 1998.
  • [19] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629, June 1999.
  • [20] Hoffman, P. and F. Yergeau, "UTF-16, an encoding of ISO 10646", RFC 2781, February 2000.
  • [21] Klensin, J. (Ed.), "Simple Mail Transfer Protocol", RFC 2821, April 2001.
  • [22] Shepler, S., Callaghan, B., Robinson, D., Thurlow, R., Beame, C., Eisler, M. and D. Noveck, "NFS version 4 Protocol", RFC 3010, December 2000.
  • [23] Kennedy, H., "Binary Lexical Octet Ad-hoc Transport", RFC 3252, April 2002.
  • [24] Popp, N., Mealling, M. and M. Moseley, "Common Name Resolution Protocol (CNRP)", RFC 3367, August 2002.
  • [25] Backus, J., "The syntax and semantics of the proposed international algebraic language of the Zurich ACM-GAMM conference", June 1959.
  • [26] American National Standards Institute, "Code Extension Techniques for Use with the 7-bit Coded Character Set of American National Standard Code (ASCII) for Information Interchange", ANSI X3.41, FIPS PUB 35, 1974.
  • [27] American National Standards Institute, "Information Retrieval: Application Service Definition and Protocol Specification", ANSI Z39.50, ISO Standard 23950, 1995.
  • [28] International Organization for Standardization, "Information Processing Systems - Open Systems Interconnection - Specification of Abstract Syntax Notation One (ASN.1)", ISO Standard 8824, December 1990.
  • [29] International Organization for Standardization, "Information Processing Systems - Open Systems Interconnection - Specification of Basic Encoding Rules for Abstract Syntax Notation One (ASN.1)", ISO Standard 8825, December 1990.
  • [30] International Organization for Standardization, "Information processing - Text and office systems - Standard Generalized Markup Language (SGML)", ISO Standard 8879, 1988.
  • [31] International Organization for Standardization, "Information Technology - Universal Multiple-octet coded Character Set (UCS) - Part 1: Architecture and Basic Multilingual Plane", ISO Standard 10646-1, May 1993.
  • [32] International Organization for Standardization, "DSDL Part 0 - Overview", December 2001, <http://www.jtc1.org/FTP/Public/SC34/DOCREG/0275.htm>.
  • [33] Unicode Consortium, "The Unicode Standard, as it may from time to time be revised or amended", March 2002, <http://www.unicode.org/unicode/standard/standard.html>.
  • [34] Duerst, M. and A. Freytag, "Unicode in XML and other Markup Languages", February 2002, <http://www.w3.org/TR/unicode-xml/>.
  • [35] Bray, T., Paoli, J. and C. Sperberg-McQueen, "Extensible Markup Language (XML) 1.0", W3C REC-xml-1998, February 1998, <http://www.w3.org/TR/1998/REC-xml-19980210/>.
  • [36] Marsh, J., "XML Base", W3C REC-xmlbase, June 2001, <http://www.w3.org/TR/xmlbase/>.
  • [37] Cowan, J. and R. Tobin, "XML Information Set", W3C REC-infoset, October 2001, <http://www.w3.org/TR/xml-infoset/>.
  • [38] Lassila, O. and R. Swick, "Resource Description Framework (RDF) Model and Syntax Specification", W3C REC-rdf-syntax, February 1999, <http://www.w3.org/TR/REC-rdf-syntax>.
  • [39] Suignard, M., Ishikawa, M., Duerst, M. and T. Texin, "Ruby Annotation", W3C REC-RUBY, May 2001, <http://www.w3.org/TR/ruby/>.
  • [40] Pemberton, S., "XHTML 1.0: The Extensible HyperText Markup Language", W3C REC-XHTML, January 2000, <http://www.w3.org/TR/xhtml1/>.
  • [41] Thompson, H., Beech, D., Maloney, M. and N. Mendelsohn, "XML Schema Part 1: Structures", W3C REC-xmlschema-1, May 2001, <http://www.w3.org/TR/xmlschema-1/>.
  • [42] Biron, P. and A. Malhotra, "XML Schema Part 2: Datatypes", W3C REC-xmlschema-2, May 2001, <http://www.w3.org/TR/xmlschema-2/>.
  • [43] Clark, J., "XSL Transformations (XSLT) Version 1.0", W3C REC-xslt, November 1999, <http://www.w3.org/TR/xslt>.
  • [44] Duerst, M., Yergeau, F., Ishida, R., Wolf, M., Freytag, A. and T. Texin, "Character Model for the World Wide Web 1.0", April 2002, <http://www.w3.org/TR/charmod/>.
  • [45] Gudgin, M., Hadley, M., Moreau, JJ. and H. Nielsen, "SOAP Version 1.2 Part 1: Messaging Framework", June 2002, <http://www.w3.org/TR/soap12-part1/>.
  • [46] Gudgin, M., Hadley, M., Moreau, JJ. and H. Nielsen, "SOAP Version 1.2 Part 2: Adjuncts", June 2002, <http://www.w3.org/TR/soap12-part2/>.
  • [47] W3C Communications Team, "XML in 10 points", November 2001, <http://www.w3.org/XML/1999/XML-in-10-points>.
  • [48] OASIS Technical Committee: RELAX NG, "RELAX NG Specification", December 2001, <http://www.oasis-open.org/committees/relax-ng/spec-20011203.html>.
  • [49] Jelliffe, R., "The Schematron", November 2001, <http://www.ascc.net/xml/schematron/>.

URIs 🔗

11. Authors' Addresses ✨

okay bestie, this is the squad roster 🫶 here's who wrote it and how to reach each of them. the names and details are shown exactly as the original has them 🎀

Scott Hollenbeck 💖

Scott Hollenbeck
VeriSign, Inc.
21345 Ridgetop Circle
Dulles, VA  20166-6503
US

Phone: +1 703 948 3257
EMail: shollenbeck@verisign.com

Marshall T. Rose 💜

Marshall T. Rose
Dover Beach Consulting, Inc.
POB 255268
Sacramento, CA  95865-5268
US

Phone: +1 916 483 8878
EMail: mrose@dbc.mtview.ca.us

Larry Masinter 🦋

Larry Masinter
Adobe Systems Incorporated
Mail Stop W14
345 Park Ave.
San Jose, CA  95110
US

Phone: +1 408 536 3024
EMail: LMM@acm.org
URI:   http://larry.masinter.net
ngl this part is legal text, so I'm keeping every word exactly as it is so nothing gets changed by accident 🙈

Copyright (C) The Internet Society (2003). All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement 🌈

Funding for the RFC Editor function is currently provided by the Internet Society. 🫶✨

📦 where this page comes from