RFC 6778

Requirements for Archiving IETF Email Lists and for Providing Web-Based Browsing and Searching

📅 published October 2012, from ietf

RFC 6778: Requirements for Archiving IETF Email Lists and for Providing Web-Based Browsing and Searching 💌

Okay bestie, first the paperwork 💅 This one is from the Internet Engineering Task Force (IETF) and it's numbered RFC 6778. It was written by R. Sparks at Tekelec, it's dated October 2012, and the ISSN is 2070-1721. It's in the Informational category, so it's not a Standards Track spec, just really useful info 🧠 The header doesn't list anything it obsoletes or updates.

Abstract ✨

The IETF runs on email lists 📧 People keep needing to search and browse those archives and have been begging for better search. The current archive setup could also be more efficient. This memo lays out the requirements for better list archiving and searching systems 💖

Status of This Memo 📋

This doc is not an Internet Standards Track specification. It's published for informational purposes, so no pressure 🫶

It's a product of the Internet Engineering Task Force (IETF) and it represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every doc the IESG approves is a candidate for any level of Internet Standard, so see Section 2 of RFC 5741 for the deets 🔍

Info about the current status of this document, any errata, and how to give feedback on it lives at http://www.rfc-editor.org/info/rfc6778 🦋

Copyright Notice ⚖️

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License 🌈

Table of Contents 🗺️

1. Introduction ....................................................2
2. List Search and Archive Requirements ............................3
   2.1. Search and Browsing ........................................3
   2.2. Archiving Active Lists .....................................5
   2.3. Importing Messages from Other Archives .....................5
   2.4. Exporting Messages from the Archives .......................5
   2.5. Redundancy .................................................6
   2.6. Archive Administration .....................................6
   2.7. Transition Requirements ....................................7
3. Internationalized Address Considerations ........................7
4. IMAP Access .....................................................7
5. Security Considerations .........................................7
6. Acknowledgements ................................................7
7. Informative References ..........................................8

1. Introduction 🎀

Okay so the IETF makes SO much use of email lists to get its work done 📬 People constantly need to search those archives and have asked for better search, especially when the search has to cover a big stretch of time or hop across several lists. Picture document editors, shepherds, working group chairs, and area directors trying to review all the discussion about one particular draft 😭 That chat could be spread across the working group list, one or more directorate lists, and the IETF general list. And sometimes work affects multiple groups, possibly in different areas, so the search has to reach into extra working group lists too. Main character problems 💅

The tools people use for these searches right now need several manually coordinated steps, which is so error prone 💀 If you don't have a local copy of the archive (and it might not even be complete), searching most working group lists takes brute force effort, maybe with some help from web search engines 🙈

More advanced search has been built for a limited subset of the available lists and it's exposed in the "Email Archives Quick Search" section of the main IETF website. Those tools are genuinely helpful, but there's still a lot of room to level up ✨

The current archive mechanism could also be made more efficient 🍓 Right now the practice involves duplicate stores (one for the web interface and one for the ftp interface). That eats storage, messes with replication, and can lead to inconsistency. Not it 🙅‍♀️

This memo captures the requirements for improved email list archiving and searching systems 💜

2. List Search and Archive Requirements 🧚

2.1. Search and Browsing 🔍

The system must provide a web interface for searching and browsing archived messages.
The system must allow browsing the entire archive of a given list by thread or by date.
The system must allow browsing the results of a search by thread or by date.

Both threading based on Message-Id/References/In-Reply-To and threading based on same subject line (modulo short prefixes like re: and fwd:) should be taken into account 🧵

The system must allow searching across any subset of the archived lists (one list, a selection of lists, or all lists).
The system must allow searching of any combination (using AND, OR, and NOT operators) of the following attributes. Richer search capabilities are highly desirable 🤩
- string occurring in sender name or email address
- date range
- string occurring in Subject
- string occurring in message body
- string occurring in message header (in particular, exact match of Message-Id)

For instance, it would be nice to search the entire archive for instances of a message with a given Message-ID with a URL like <http://datatracker.ietf.org/mlarchive/msg?id=4EA6E023.6010603@example.com> 💖

Individual messages must be representable by a long-term stable URI that can be shared between users. That is, the URI must be suitable for reference in an email message.
- It would be preferable for that URI to appear in an Archived-At header field in the message [RFC5064].
Searches should be representable by a URI that can be shared between users.
- Such URIs should be long-term stable.
- The search may be re-executed when the URI is referenced. It is acceptable for the same URI to produce different results if accessed at different times or by different people (for example, by reflecting additional messages that may match the search criteria or reflecting changes in access authorization to lists with restricted archives) 🦋
When the system requires credentials, it must use the datatracker's authentication system 🔐
- While the vast majority of archived lists have an open access policy, some archived lists have restricted archives.
- The system must not require credentials for browsing or searching lists with open archives. (But it is acceptable for a user to browse or search such lists while logged in.)
- The system must make it possible to limit access to a restricted archive based on login credentials.
- Messages from restricted archives must be distinguishable from messages from unrestricted archives in any search results.

2.2. Archiving Active Lists 📥

The archive system must accept messages handled by various mail list manager packages.
- Lists hosted on the IETF systems are served by Mailman [Mailman].
- Lists hosted at other organizations may use other packages.
  - The archive system must accept messages through subscribing to such an external list.
  - The archive system may support other mechanisms for accepting messages into the archive.

2.3. Importing Messages from Other Archives 📦

Lists hosted at other systems are sometimes moved to the IETF servers, and their archive gets moved with them 🚚 The archiving system must be able to import these archives.

At a minimum, the archive system must be able to import mbox formatted archives [RFC4155] [Mbox].
The archive system should be able to import Maildir and Maildir-like (the key characteristic being one-message-per-file) formatted archives [Maildir].
It is acceptable to use a separate utility to convert between these formats before import, as long as the conversion is lossless ✨

2.4. Exporting Messages from the Archives 📤

The archive system must allow both users and administrators to export messages 💅

The archive system must support exporting messages in the mbox format.
The archive system should support exporting messages in the Maildir format.
The archive system must support exporting the entire archive of a given list.
The archive system must support exporting all messages from a given list within a given date range.
The archive system should allow exporting the results of any supported search query.

5. Security Considerations 🔒✨

Okay so, bestie, this section is short but it matters SO much. Creating a new tool for searching and archiving IETF email lists does not affect the security of the Internet in any significant fashion 💅. Lowkey a relief, right? 🫶

Searching can be I/O and CPU intensive 🥵. The implementors of this tool should consider the potential for maliciously crafted searches attempting to consume all available resources 😭. Similarly, the implementors should consider the potential for denial-of-service attacks through making many connections to the browsing system or rapid navigating within it 🙈. Not me clicking 900 links in one second and crashing the whole archive, iykyk 💀.

Preserving the integrity of the archives is important 💜. The implementors should ensure that administrative access is appropriately authenticated, and that message paths into the archive are appropriately configured to avoid unauthorized message insertion 🔐. Nobody should be sneaking fake messages into the archive, periodt 🚫.

6. Acknowledgements 🎀

The Tools Development team provided input into the initial brainstorm 🧚. Text suggestions from Alexey Melnikov, Pete Resnick, S. Moonesamy, Francis Dupont, and Murray Kucherawy have been incorporated 👑. Literally the dream team, no cap 🌈.

7. Informative References 📚

Here are the informative references, all listed in full 💖:

[IMAP-ARCH]  Sparks, R., "IMAP Access to IETF Email List Archives",
             Work in Progress, August 2012.

[IMAP-UTF8]  Resnick, P., Ed., Newman, C., Ed., and S. Shen, Ed.,
             "IMAP Support for UTF-8", Work in Progress,
             October 2012.

[Maildir]    "Maildir", <http://en.wikipedia.org/w/
             index.php?title=Maildir&oldid=503288041>.

[Mailman]    "Mailman", <http://www.list.org/>.

[Mbox]       "mbox", <http://en.wikipedia.org/w/
             index.php?title=Mbox&oldid=515402275>.

[MHonArc]    "MHonArc", <http://www.mhonarc.org/>.

[RFC4155]    Hall, E., "The application/mbox Media Type", RFC 4155,
             September 2005.

[RFC5064]    Duerst, M., "The Archived-At Message Header Field",
             RFC 5064, December 2007.

[RFC6531]    Yao, J. and W. Mao, "SMTP Extension for
             Internationalized Email", RFC 6531, February 2012.

[RFC6532]    Yang, A., Steele, S., and N. Freed, "Internationalized
             Email Headers", RFC 6532, February 2012.

Author's Address 🦋

Robert Sparks
Tekelec
17210 Campbell Road
Suite 250
Dallas, Texas  75254-4203
USA

EMail: RjS@nostrum.com

📦 where this page comes from

ietf: rfc6778 2012-10
💖 girlified by claude-haiku-5-5 on 2026-10-17: a language model's rewrite of the original, under its terms