RFC 8153

Digital Preservation Considerations for the RFC Series

📅 published April 2017, from ietf

Digital Preservation Considerations for the RFC Series 🎀

Okay bestie, this is RFC 8153 💅 It's an Informational doc (so it's "published for informational purposes," not a standard) from the Internet Architecture Board (IAB). The author is H. Flanagan from the RFC Editor, it's dated April 2017, and its ISSN is 2070-1721. The header doesn't list it obsoleting or updating any other RFC, so it's a standalone vibe 🧚

Abstract ✨

The RFC Editor is both the publisher and the archivist for the RFC Series. This doc is all about the archivist side 📚 It gives guidance on when and how to preserve RFCs, and it describes the tools you need to view or re-create them. It also calls out gaps in the current process and suggests compromises that balance cost with best practice 💜

Status of This Memo 🌸

This doc is not an Internet Standards Track specification. It's published for informational purposes only 🌷

It's a product of the Internet Architecture Board (IAB) and shares information the IAB thinks is valuable enough to keep on permanent record. It represents the consensus of the IAB. Docs the IAB approves for publication are not candidates for any level of Internet Standard, see Section 2 of RFC 7841 🙈

For the current status, any errata, and how to give feedback, check http://www.rfc-editor.org/info/rfc8153 🔗

Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved.

This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the publication date. Please read these carefully, because they spell out your rights and restrictions for this doc 📜

Table of Contents 🗺️

1. Introduction ....................................................2
   1.1. Terminology ................................................4
   1.2. Life Cycle of Digital Preservation .........................4
2. Updating Policy and Procedure ...................................5
   2.1. Acquisition of Documents ...................................6
   2.2. Ingestion of Documents .....................................6
   2.3. Metadata and Document Registration .........................7
   2.4. Normalization and Standardization of Canonical File
        Structure and Format .......................................9
        2.4.1. 'Best Effort' Data Retention .......................10
        2.4.2. Single Format for Archival Purposes ................11
        2.4.3. Holistic Archiving of the Computing Environment ....12
   2.5. Transformation/Migration to Current Publication Formats ...12
   2.6. System Parameters .........................................13
   2.7. Financial Impact ..........................................13
3. Recommendations ................................................14
4. Summary ........................................................15
5. IANA Considerations ............................................15
6. Security Considerations ........................................15
7. Informative References .........................................16
IAB Members at the Time of Approval ...............................18
Author's Address ..................................................18

1. Introduction 📖

The RFC Editor is both the publisher and the archivist for the RFC Series. That's a series of technical specs and policy docs that includes foundational Internet standards [RFC6635] [RFC-SERIES] 🧠 The RFC Editor's goal is to produce clear, consistent, and readable docs for the Internet community 💖 Over time, they'll use lots of modern features, like hyperlinks and content markup, wherever it helps get the author's intended info across to the audience ✨

As the archivist, though, the main goal is to preserve both the info being described and the docs themselves for the indefinite future 🦋 So the RFC Editor has to find a balance between what publishing needs today and what archiving needs tomorrow. And they're doing it with a finite set of resources for both sides of the job 😭

Lots of files get created during editing, but this doc only focuses on the archival needs of the Internet-Drafts (I-Ds) that got approved for publication, and the RFCs that came out of them. I-Ds that haven't been approved for publication by the right stream-approving body are out of scope 🙅‍♀️

To sum up, the big tensions between the publisher and archivist roles are:

  • the publisher's desire to meet what authors want, like the latest tech (think vector graphics, live links, and a rich set of metadata) inside their docs; and
  • the archivist's desire to support only the simplest possible format. Right now, the Series holds that to be plain-text, ASCII-only documents. The tools needed to view them are equally simple and resistant to tech changes, so the docs are easier to archive for at least the next several decades, if not centuries 🕰️

For most of the Series' history, RFCs were plain text with an ASCII-only character set 💅 That was the simplest format most likely to stay available to the biggest number of consumers, and the one most likely to survive tech changes over time. But consumers and authors increasingly want extra features that make reading easier on a wider range of devices, while still keeping all the metadata authors intended 🌸 Back in 2013, RFC 6949 ("RFC Series Format Requirements and Future Development") captured the high-level requirements for the Series. The core problem was that plain-text, ASCII-only docs no longer meet the needs of the communities who use and produce RFCs [RFC6949] 😩

That also means the simple archival process the RFC Editor runs is probably no longer enough. More complex tools and file formats need a more complex process so RFCs can still be read and rendered far into the future 🔮 This doc describes the considerations that have to shape any changes in policy and procedure. It also describes a model for the Series to follow once formats beyond plain-text, ASCII-only RFCs are published 🎀 The functional model behind the archival process here was derived from the ISO Open Archival Information System (OAIS) reference model, defined in "Space data and information transfer systems -- Open archival information system (OAIS) -- Reference model" [ISO14721] 🧚

1.1. Terminology 🏷️

  • Acquisition: The point at which the RFC Editor accepts a document for future inclusion in the archive.
  • Ingestion: The point at which a digital object gets all the metadata it needs to describe itself and its contents, and is added to the archive.
  • Bitstream preservation: Storing and maintaining digital objects over time, making sure no bits making up those objects get lost or corrupted 💾
  • Content preservation: Keeping the ability to read, listen to, or watch a digital file in perpetuity. Content preservation isn't about the stored bits themselves, it's about being able to access those bits and present them to the user 👑

1.2. Life Cycle of Digital Preservation 🔁

Lots of organizations have described how to preserve digital info. From the Life cycle Information For E-Literature (LIFE) project [LIFE] in the United Kingdom, to the ongoing digital preservation work at the U.S. Library of Congress [USLOC], the basic process is pretty straightforward 🙌 Documents get acquired and processed, metadata gets recorded, physical media gets refreshed, and content gets checked regularly to see if it's still accessible to people who care about it. It gets complicated when you need to preserve both the bits of the digital objects and the tools needed to express those bits, in an environment where tech changes rapidly 🌪️

For most of the Series' existence, digital preservation was pretty simple. It focused on bitstream preservation and relied on paper copies of the digital files 📄

The current archival process for the RFC Series goes like this:

  1. Acquisition: The RFC Editor database is updated to show an I-D has been approved for publication. At this point, the document goes through the editorial process on its way to publication [RFC-PUB] ✍️
  2. Ingestion: The RFC gets added to the archive at the time of publication 📥
  3. Metadata creation: The details about an RFC, including RFC number, author, title, abstract, and so on, are created at time of publication. Extra metadata, like status and errata, can be added or changed at any time, following the process of the originating document stream 🗂️
  4. Bitstream preservation: This part is handled as part of IT system administration. All servers, disks, and backup tech get refreshed on a regular cycle 🔄
  5. Content preservation: Every RFC since January 2010 has been printed out on standard office paper at time of publication, and the electronic files have been kept on disk and in backups. There's no particular focus on preserving the entire computing environment used to create the electronic docs 🖨️ Most RFCs from before January 2010 are also available on paper, but there are gaps in the record, and there are ownership issues around those paper copies from that period 😬

When the RFC format moves from plain-text, ASCII-only files to an XML format with multiple outputs, the overall archival process gets more complicated. Extra metadata and some (or possibly all) of the computing environment may need to be added to the archive 💖

2. Updating Policy and Procedure ✨

okay so RFCs are created and published as digital objects, bestie 💾 and unlike paper publications, a digital collection has to focus on keeping the details of the technology as well as the object itself. a digital archive specifically needs to:

  • consider the inherent instability of digital media 🫠
  • plan for a relatively short path to technological obsolescence ⏳
  • schedule regular media updates 🔄
  • apply predefined criteria for technology evaluation, and
  • ensure the continued authenticity and integrity of documents through any changes in technology 🫶

the RFC Editor is the custodian and canonical source of RFCs and their errata, so they must consider how to keep this document series available and intact far into the future 🔮 and figure out whether the focus has to be on bitstream preservation, content preservation, or both 👑

the RFC Editor has several advantages as the digital archivist for the Series, and honestly it's so iconic 💅 since they're the publisher and the archivist, they control the format of the material, the process for adding that material to an archive, and can add any extra metadata they consider necessary. external material is a big deal for more general archives, but the RFC Editor no longer accepts it 🙈 (see "Internet Archaeology: Documents from Early History" [RFC-HISTORY] for the list of non-RFC digital objects held by the RFC Editor 📜)

this document describes several different preservation models that may fit the needs of the Series, and it raises a few points for community discussion 🌈 specifically, it covers information on:

  • Acquisition of documents
  • Ingestion of documents
  • Metadata and document registration
  • Normalization and standardization of canonical file structure and format
  • Transformation/migration to current publication formats
  • Content and computing environment preservation
  • System parameters
  • Financial impact

2.1. Acquisition of Documents

the acquisition process for documents meant for the archive starts when an approved I-D is submitted for publication 📥 during editing, stuff like the document metadata gets finalized before publication ✨ but here's the tea: the initial I-D as submitted and the RFC made from it don't formally enter the archive until publication, and that moment counts as the point of ingestion from an archival perspective 💖

2.2. Ingestion of Documents

once an RFC is published, the canonical format is considered immutable 🔒 at this point, the RFC Production Center (one of the internal roles within the RFC Editor) assigns the document metadata an archivist needs to identify the unique object 🪪

for RFCs, the metadata assigned at publication time includes:

  • the RFC number
  • ISSN
  • publication date
  • Digital Object Identifier (DOI)

additional metadata, like author name, gets assigned earlier in the document creation process, but it can still change up until publication 🎀 more info on metadata is in Section 2.3 ("Metadata and Document Registration") 🧚

now, deciding what to accept into the archive is a major question for most archives, but it's a simple one for the RFC Series 🌸 the RFC Editor accepts documents that have been approved for publication by the approving body of one of the document streams: the IETF, IAB, IRTF, or Independent Submission streams [RFC7841]. each stream has its own defined processes for when and how I-Ds get approved and submitted to the RFC Editor for publication. the RFC Editor does not select documents for publication and archiving. they just edit and publish what the document streams have already approved ✅

the RFC Editor holds no copyright on I-Ds or RFCs 🙌 as per the IETF Trust Legal Provisions [TLP], the copyright for RFCs is held by the authors and the IETF Trust. at any point in time, whoever is providing RFC Editor services must be able to release the archive of RFCs to the IETF Trust 🗝️

Note: The RFC Editor is currently only responsible for RFCs. Any associated datasets or other research data are not part of the RFC Editor's mandate at this time, so this document doesn't cover archival requirements for those datasets. 🙅‍♀️

2.3. Metadata and Document Registration

okay metadata is basically data about data 🤓 in digital archiving, it's the stuff that clearly identifies every aspect of a document, from its identifier (the RFC number and the I-D draft string) to the size and file format of the document, and more 🗂️ metadata lives in a central registry that records exactly what's being preserved and where it's located, info on authenticity and provenance, and details on the hardware and/or software needed to view or create the documents 💜

the RFC Editor maintains this registry as a database that includes all the metadata for documents being edited and for published RFCs 💅 this database feeds the search engine on the RFC Editor website and the info page for every RFC (for example, http://www.rfc-editor.org/info/rfc####) 🦋

the current list of metadata shown on the RFC info pages is:

  • RFC number
  • Canonical URI
  • Title
  • Status
  • Updates (if applicable)
  • Updated by (if applicable)
  • Obsoletes (if applicable)
  • Obsoleted by (if applicable)
  • Authors
  • Stream
  • Abstract
  • Content-Type
  • Character Set
  • ISSN
  • Publication date
  • Digital Object Identifier (DOI)

in the future, this metadata will also get:

  • Publication format URIs

info pages also link to errata, IPR searches, and both plain-text and XML citation files 📎

in terms of best practice, all documents used as normative references within an RFC should also be stored in the archive 📚 when the normative reference is another RFC (the usual case), this happens automatically. keeping a copy of third-party documents, though, is considered out of scope for the RFC Editor 🙈 as the digital archive industry settles down, services like Perma.cc [PERMACC] might be a reasonable compromise 🌈 they give you a permanent URI and an image capture of online documents, so you're buffered against URI and online availability changes 🛡️

2.4. Normalization and Standardization of Canonical File Structure and Format

the normalization process is probably the most technically critical part of digital archiving 🔥 the purpose is content preservation: making sure the data accepted for archiving is in the most stable and easily accessed formats possible for the long-term future, and that it needs the least amount of re-engineering and environment emulation to view later 💖 basically, normalization is about enabling long-term access to the information inside a document ✨

over the history of the RFC Series, documents have been submitted for publication in all kinds of formats, including paper for the earliest RFCs 📜 today, most RFCs are available in both a canonical plain-text format and PDF format. for the exceptions, check the RFC Online Project [RFC-ONLINE] 🔍

right now, all RFCs are printed out to paper and stored at publication time 🖨️ that's been a reasonable backup plan for several decades. plain-text files don't have many of the features you'd expect from a digital document format (like links, metadata inside the document, and line drawings), so they don't lose much, if anything, when printed 🌸 however, as published formats change (see RFC 6949), printing to paper gives less value, since a lot of the metadata that's an intrinsic but invisible part of the rendered document would get lost in the printout 😭 so the focus needs to shift to preserving the new file formats electronically 💻

every RFC today is printed to paper and all electronic versions are stored on multiple hard drives, but no real effort goes into making sure copies of the software used to render or read the canonical plain-text RFC are archived too 🙈 the RFC Editor has a few choices for handling the need to archive a more complex set of data, following best practice as defined by the digital archive community:

  • a simplified bitstream preservation model that focuses on standard "best effort" data-retention practices, which rely on backups, upgrades, and regular equipment change to preserve the data. this model assumes emulators may be built when needed if the formats used go out of common use (a significant part of the model the RFC Editor currently follows) 🛠️
  • a content preservation model that focuses on one publication format as the version most likely to be viewable and to carry all necessary metadata in the future. this is a viable option, since PDF/A-3 [PDF], one of the intended publication formats, was designed for this type of archiving 📄✨
  • a complex bitstream and content preservation model that focuses on archiving the canonical XML and the entire computing environment required to create, view, and render all outputs from that file. this is the "best practice" from an archivist's perspective 👑

those options are listed in order from least to greatest complexity and expense 💸 more detail on each is below 🎀

2.4.1. 'Best Effort' Data Retention

when it comes to very simple data structures like plain-text, ASCII-only files, the experience of the RFC Series shows that for the last few decades, hardware and operating system changes have had minimal impact on the stored document files 🥰 a complete failure of an operating system migration did corrupt the dataset once, but that's a bit of a different problem from the tools themselves changing so plain-text files aren't easily read with existing technology 🤔 since the basic plain-text format and ASCII encoding are still in common use, the standard protections against file corruption and data loss (like disk mirroring, off-site backups, and periodic restoration testing) will keep giving access to the whole RFC Series for the foreseeable future 💜

as this document and broader community discussion have pointed out, that's not enough for complex formats like XML, HTML, PDF, or the proprietary formats offered by today's big IT companies 😬 the risk of technological change deprecating or changing those formats without backwards compatibility is fairly high when you look decades or centuries ahead ⏳

it is recommended that this model of archiving the RFC Series stop being the primary model once plain-text, ASCII-only format is no longer the canonical format 🌸 best effort data retention is a necessary but not sufficient level of effort for preserving a digital archive 🫶 for more guidance on how to define best effort data retention, the section on "Media and Formats, Summary Recommendations" in the 2009 version of the Digital Preservation Handbook [DPC2009] has useful and concrete info 📚🦋

2.4.2. Single Format for Archival Purposes ✨

Okay so, if what you actually want to keep is the information in a document rather than the document itself, then camping out on one single file format is a totally reasonable move 🎀 Some well-supported archival tooling projects do exactly this, like Archivematica [ARCHIVEMATICA]. Picking a feature-rich but fundamentally stable file format means an organization can dodge the super expensive whole-environment rebuild just to be able to view the document 💅 The PDF/A formats were built to be an archival format for electronic documents, and PDF/A-3 is one of the options meant for publication as the RFC Series moves from a plain-text canonical format to an XML canonical format with multiple publication formats 🦋 A PDF/A-3 file can be made that embeds the XML it was generated from, which means you can validate both the original and the rendered document if you've got the right tools to peek at the source of the PDF/A-3 file [RFC7995] 🔍 The XML isn't otherwise visible when you open the PDF/A-3 in typical PDF reader software, so you'd have to know to look for it 🙈

Now, when you're archiving RFCs in a resource-limited environment, a content-preservation-only model has merit, but ngl it's not risk-free 💀 First, PDF/A-3 won't be the canonical format. It's meant to be one of the rendered outputs, so it may contain rendering bugs that weren't intended to be in the document 🐛 Second, even though the various PDF/A formats were designed to be archival, they haven't been put to the test of time yet to see if they actually live up to those design goals ⏳

This is a valid option to consider, but the risks, priorities, and costs must be discussed by the community before anyone decides to go this route 🫶 The best option might be to combine it with one of the other archiving methods described in this document to help minimize both risk and cost 🌈

2.4.3. Holistic Archiving of the Computing Environment 🌸

Preserving everything the RFC Editor publishes so there's a permanent record of information, standards, and best practice is arguably the whole point of being an archival series 👑 You could argue it's not only about the information in an RFC, but also about supporting Intellectual Property Rights (IPR) and keeping the history of the Internet alive 📜 But if you go this route, you have to treat the complexity of the archival environment as matching, and possibly exceeding, the complexity of the file formats being preserved 🤯

Picture a future where XML has been obsolete for half a century, HTML5 was a format used three to four human generations ago, and PDF/A-3 isn't supported by any company's reading software anymore 😭 For RFCs produced with XML as their canonical format, an archive must not only hold the data, it must also hold the entire computing environment that lets the data be rendered and viewed 🖥️ That means operating systems and the hardware they can run on, every major version of every piece of software used or relied on while publishing an RFC, and browsers and readers for HTML, PDF, and any other publication format. All of it has to be preserved somehow 💜 This is considered best practice when archiving digital documents ✨ It's also the most expensive method, and the cost only goes up over time as more and more instances of the computing environment need preserving over the Series' lifetime 💸

This is a valid option to consider, but the sheer scope of resources it needs means the community has to discuss it before deciding anything 🧚 Going this way might require a totally different paradigm for the RFC Editor compared to what's been considered before. Expanding the RFC Editor's scope and resources, finding a third party to take over archiving responsibilities, or some other option may be necessary 🎀

2.5. Transformation/Migration to Current Publication Formats 🦋

Because normalization is a complex subject, it's important to think about how to mitigate the risk of normalization failing 🧩

The RFC Editor is responsible for making RFCs available to the Internet community 🌍 The canonical version of an RFC does not change once it's published. Formats officially rendered from the canonical version, however, may change 🔄 One way to avoid having to preserve the entire computing environment for an RFC, including web browsers and PDF readers, is to take advantage of the non-canonical nature of the publication formats and re-render them from the canonical source at the point where browser or reader technology has changed enough to make RFCs largely unavailable to "modern" tools 🙌

For example, the RFC Editor might make it a practice to review every year the tools needed to view the publication formats the RFC Editor creates. The goal is to figure out whether the current common and popular reader technologies (i.e., web browsers, PDF viewers, e-readers) can still view the existing publication formats 👀 During that review, the RFC Editor would work with the community to decide whether the current publication formats meet the community's needs, and whether any should be retired or added to make information more available to the community at that time 💖

2.6. System Parameters 🛠️

While the industry best practice for backing up and restoring data isn't sufficient as a long-term archival solution, it's still a necessary part of keeping the Series available now and into the future 💅 In the past, nearly 800 RFCs had to be manually transcribed from paper back to electronic format because of a failed server migration and insufficient backups 😱

The underlying servers that host the tools, database, RFCs, and errata are the physical link in the archival environment 🖧 Systems can't and shouldn't stay static and unchanging, but there must be clear documentation about the environment, especially the storage, backups, and recovery processes for all RFC-related material 📋 That documentation must include information on the refresh cycle for physical storage and backup media, and it must describe a regular cycle of data restoration and/or migration testing 🔁

2.7. Financial Impact 💰

Having a policy about digital archiving gives input into the budget process 🧾 The main costs of digital archives come from the complexity and quantity of the material being archived, as described in Section 2.4 on normalization 📚

Estimating potential costs and providing figures are outside the scope of this document, but costs should be noted as a major factor when deciding what level of archival practice an organization will follow 🌟

For more information on potential business plans and cost modeling for digital preservation, see the "Business cases, benefits, costs, and impact" section of the Digital Preservation Handbook [DPC] 📖

  1. Recommendations 🎀

Given the need to balance cost and complexity against keeping information for historic, legal, and informational purposes, preservation efforts should focus on the XML canonical format files, the PDF/A-3 format files, the xml2rfc tool and its documentation, and at least two PDF reader applications capable of extracting the embedded XML 🗂️ Care should be taken that the software included in this archive has a provision for free copies for backup or archival purposes 🔐 All other formats and the overall computing environment should be stored as described in "best effort" data retention (Section 2.4.1), which should in turn be described in the appropriate vendor contract for the RFC Publisher 📝

Particular preservation efforts should be made by:

  • choosing a format designed for archiving RFCs (PDF/A-3 as indicated by [RFC7995]) 🌷
  • embedding the canonical XML format within the PDF/A-3 file for RFCs 🧁
  • retaining a copy of the plain-text or XML file submitted for approved I-Ds 💌
  • retaining all major versions of the tools and their associated documentation used to acquire and ingest an RFC 🛠️
  • retaining the final XML file as well as the PDF/A-3 file with the embedded XML 🎁
  • retaining at least two software reader applications to ensure the PDF/A-3 and XML files can be viewed in the future 👓
  • partnering with other digital archives around the world to mirror copies of the target data 🌍

In order to control costs and focus the archiving effort on the entire content of an RFC, including the metadata and other features embedded within each RFC published in more than just plain text, printing each RFC to paper upon publication is no longer reasonable 🖨️🚫 Proper data storage and mirrored copies of RFCs provide more efficient and effective copies in case of catastrophic failure of the existing archive of material 🛟

Particular focus should be given to finding partners that specialize in digital preservation to ingest RFCs 🤝 Ideally, they will ingest all material associated with an RFC, including all metadata, digital signatures, and the approved I-D that was submitted to the RFC Editor 💎 The possibilities and options should be discussed with each archival partner. At minimum, they must ingest copies of RFCs as they are published, with the basic metadata associated with each document 📦

Preservation efforts should be reviewed and validated through a biennial audit that will verify that the targeted content and all its associated metadata can be read with existing tools 🔍 The full process from acquisition to ingestion should be reviewed to make sure best current practice is being followed from the perspective of the digital archive community 🧐 Since the overall model for the digital archive maintained by the RFC Editor follows the OAIS reference model, the associated audit guidelines should also be followed 🗺️ While the RFC Editor doesn't seek to be recognized as 'OAIS-compliant' at this time, use of the ISO standard "Space data and information transfer systems -- Audit and certification of trustworthy digital repositories" [ISO16363] would provide a solid, accepted method for structuring an audit for this digital archive 🏆

  1. Summary 🌈

The RFC Series is worth archiving 💖 It contains the history of the early Internet, as well as some of the key standards for Internet technology and today's best practice. Who knows what the community will create in the future? 🔮 There are many ways to preserve the Series, from relying on preservation of the bits, to focusing on a single file format, to preserving the entire computing environment. Each possibility, or permutations of them, involves risks and requires varying levels of resources ⚖️ The goal of this document is to describe the possibilities and associated risks so the community can make an informed decision about what it's willing to see supported far into the future 🫶✨

  1. IANA Considerations 🎀

This document does not require any IANA actions 🙅‍♀️

  1. Security Considerations 🔐

This document assumes that the origination of RFCs via the RFC Editor is secure and trusted 🛡️ With that assumption, the activities discussed in this document do not affect the security of the Internet 🌐💜

7. Informative References ✨

okay bestie, here's the reading list 📚💅 every extra doc with all its details so you can go find them yourself 🔍

IAB Members at the Time of Approval

The IAB members at the time this document was approved were (in alphabetical order): 👑

  • Jari Arkko
  • Ralph Droms
  • Ted Hardie
  • Joe Hildebrand
  • Lee Howard
  • Erik Nordmark
  • Robert Sparks
  • Andrew Sullivan
  • Dave Thaler
  • Martin Thomson
  • Brian Trammell
  • Suzanne Woolf

Author's Address 🎀

Heather Flanagan
RFC Editor

Email: rse@rfc-editor.org
URI:   http://orcid.org/0000-0002-2647-2220
📦 where this page comes from