RFC 4155

The application/mbox Media Type

📅 published September 2005, from ietf

RFC 4155: The application/mbox Media Type 💌

Network Working Group 🌸 Request for Comments: 4155 🎀 Author: E. Hall (Eric A. Hall) 💖 Date: September 2005 ✨ Category: Informational 🦋 Obsoletes or updates: this header doesn't list anything it obsoletes or updates 🍓

Status of This Memo 📜

Okay so this memo is just giving info to the internet community. It does not specify an Internet standard of any kind, and distribution is unlimited 🫶

Copyright (C) The Internet Society (2005). 💜

Abstract 🌙

This memo asks for the application/mbox media type to be authorized for allocation by the IESG, following the terms in RFC 2048. It also defines a default format for the mbox database, which has to be supported by all conformant implementations 💅

1. Background and Overview 🌈

UNIX-like operating systems have been using "mbox" database files for all kinds of local email stuff for ages 🕰️ Usually an mbox file is a straight line of one or more email messages, and local email clients treat the whole thing like a folder of mail 📂 Other tools love mbox too, like mailing list managers, archiving and filtering tools, messaging servers, and a bunch of other apps. Lately mbox databases have also shown up on tons of non-UNIX platforms for the same kinds of jobs. Iconic, honestly 👑

The more places these files show up, the more people want a standard, network-wide way to pass them around as their own database objects. That's why we need a general media type definition for mbox files, and that's the whole point of this memo 🦋

2. About the mbox Database 📬

Here's the tea: the mbox database format doesn't have an authoritative spec 😭 It's more of a well-known output format that's been documented anecdotally, or only documented properly for one specific platform or tool.

mbox databases usually hold a straight-line sequence of email messages. Each message starts with a separator line that says who sent it, plus the date and time it got to the final recipient (either the last-hop system on the way there, or the system that acts as the recipient's mailstore) ⏰ Each message usually ends with an empty line. The end of the database is usually spotted by either no more data showing up, or an explicit end-of-file marker 🏁

The separator lines vary a lot between implementations, but they usually contain the exact sequence From, then a single Space character (0x20), then an email address of some kind, another Space character, some kind of timestamp, and an end-of-line marker 💖 Since there's no authoritative spec, every one of those pieces can vary wildly from tool to tool. The email address can be any addressing syntax that has ever existed on any messaging system in all of history, and that includes address forms that aren't compatible with Internet messages as defined by RFC 2822 [RFC2822] 😳 Timestamps can change depending on what the system spits out, and end-of-line sequences often follow whatever the platform needs. Different data formats can even live in one database, either because a bunch of mbox files got concatenated together, or because one file got accessed by multiple mail clients that each used their own separator line syntax 🙈

Message data inside mbox databases often has site-specific quirks too 🌍 For example, the message body or headers can contain untagged eight-bit character data that quietly assumes a site-specific default language or locale, or local defaults for timestamps and email addresses. None of that is widely portable beyond the local scope 🧳 Message data can also hold unencoded eight-bit binary data, or use encoding formats tied to a specific platform, like BINHEX or UUENCODE sequences 🤖

Lots of implementations escape message body lines that begin with From so that they don't get confused with overly-liberal parsers that don't look for full separator lines 🛡️ Usually a leading Greater-Than symbol (0x3E) does the job, so From turns into >From . Other implementations only escape those lines if they come right after a blank line, or if they also seem to contain an email address and a timestamp. Some implementations do secondary escapes on lines that are already escaped or quoted, and others skip escaping altogether 🙃

There's a comprehensive description of mbox database files on UNIX-like systems at http://qmail.org./man/man5/mbox.html. It should be treated as mostly authoritative for the variations that are otherwise only written down anecdotally. Just know that lots of other platforms and tools use mbox databases too, and there are way more variations out in the wild than that page covers 👀

To help cut down on errors from all these vagaries, this spec defines a "format" parameter for the application/mbox media type declaration. You can use it to say exactly which kind of mbox database is being transferred ✨ This spec also defines a "default" database format that implementations claiming compliance with this spec MUST support. It's also the implicit format for undeclared application/mbox data objects 💅 Additional format types will be defined in later specifications. If a messaging system gets an mbox database with a format parameter value it doesn't know, it SHOULD treat the data as an opaque binary object, as if it had been declared as application/octet-stream

Check out Appendix A for a description of the default mbox format 📖

Also, RFC 2046 [RFC2046] defines the multipart/digest media type for transferring platform-independent message files. That spec sets out neutral, strict formatting rules, so multipart/digest already makes transfer and conversion really predictable 🌸 Implementers are strongly encouraged to support and use that media type wherever they can 🫶

3. Prerequisites and Terminology 🧠

Readers are expected to already know the MIME specification [RFC2045] and MIME-type registrations [RFC2048] 🤓

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] 💜

4. The application/mbox Media Type Registration 📝

This section is the media type registration application (as per [RFC2048]) 🎀

  • MIME media type name: application
  • MIME subtype name: mbox
  • Required parameters: none
  • Optional parameters: The "format" parameter identifies the format of the mbox database and the messages inside it. Its default value is "default", which refers to the formatting rules in Appendix A of this memo. mbox databases without a "format" parameter SHOULD be interpreted as having the implicit "format" value of "default". mbox databases with an unknown value for "format" SHOULD be treated as opaque data objects, as if the media type had been specified as application/octet-stream. Additional values for the format parameter will be defined in later specifications and registered with IANA 🍓
  • Encoding considerations: Here's a big one ⚠️ If an email client gets an mbox database as a message attachment and then saves that attachment inside a local mbox database, the two database files can end up irreversibly intermingled, and both can become unrecognizable 😱 To avoid these collisions, messaging systems that support this spec MUST encode an mbox database (or at minimum, the separator lines) with a non-transparent transfer encoding (such as BASE64 or Quoted-Printable) whenever an application/mbox object is sent through messaging protocols. Other transfer services are generally encouraged to use similar encoding strategies so that any later retransmission works out, but this is not a requirement. Implementers should also be prepared to encode mbox data locally if non-compliant data comes in 🧚
  • Security considerations: mbox data is passive, and it doesn't generally pose a unique or new security threat. Still, sharing any kind of data carries risk, because unintentional information can get exposed, and that absolutely applies to mbox data too 🔐
  • Interoperability considerations: There's no single authoritative spec for mbox databases, so there are tons of variations between formats (see the intro for common examples). It's expected that non-conformant data will get tagged or exchanged by mistake 🙈 The "default" format in this memo doesn't allow for these kinds of vagaries, but sometimes humans still need to negotiate or agree on things beforehand 🤝
  • Published specification: see Appendix A.
  • Applications that use this media type: hundreds of messaging products use the mbox database format in one form or another 📬
  • Magic number(s): mbox database files can be recognized by a leading character sequence of "From", followed by a single Space character (0x20), followed by more printable character data (see Appendix A for the details). However, implementers are warned that not every such file will follow all of the formatting rules, so they should handle these files with a healthy amount of circumspection 🕵️
  • File extension(s): mbox database files sometimes have a .mbox extension, but that's neither required nor expected. As with magic numbers, implementers should avoid reflexive assumptions about what's inside these files 🙃
  • Macintosh File Type Code(s): None are known to be common.
  • Person & email address to contact for further information: Eric A. Hall (ehall@ntrg.com) 💌
  • Intended usage: COMMON 🌟

5. Security Considerations 🔒

Go read section 4, because that's where the security tea is spilled 💅

6. IANA Considerations 🏛️

IANA has registered the application/mbox media type in the MIME registry, using the application provided in section 4 above 🎉

IANA has also set up and will keep a registry of values for the "format" parameter, as described in this memo. The first registration is the "default" value, using the description in Appendix A 💖 Subsequent values for the "format" parameter MUST be accompanied by some form of recognizable, complete, and legitimate specification, such as an IESG-approved specification or some kind of authoritative vendor documentation 🦋

(part 2 next, with Appendix A and the references 🎀)

7. Normative References ✨

okay bestie, these are the receipts 🧾 everything this doc leans on, listed exactly as written so you can go check them yourself:

[RFC2045]   Freed, N. and N. Borenstein, "Multipurpose Internet Mail
            Extensions (MIME) Part One: Format of Internet Message
            Bodies", RFC 2045, November 1996.

[RFC2046]   Freed, N. and N. Borenstein, "Multipurpose Internet Mail
            Extensions (MIME) Part Two: Media Types", RFC 2046,
            November 1996.

[RFC2048]   Freed, N., Klensin, J., and J. Postel, "Multipurpose
            Internet Mail Extensions (MIME) Part Four: Registration
            Procedures", BCP 13, RFC 2048, November 1996.

[RFC2119]   Bradner, S., "Key words for use in RFCs to Indicate
            Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC2822]   Resnick, P., "Internet Message Format", RFC 2822, April
            2001.

Appendix A. The "default" mbox Database Format 🎀

okay this is the part where we get into the nitty gritty 🤓 to make messaging systems play nicely together, this doc defines a "default" mbox database format. and it's not optional vibes, every implementation that claims to be compliant MUST support it 🫶

the "default" format is basically one long linear line-up of Internet messages 🧁. every message gets a separator line sitting right in front of it, and it ends with an empty line. here's the breakdown:

  • Line endings: each message in the database MUST follow the syntax and formatting rules from RFC 2822 [RFC2822] and its related specs. the one twist is that the canonical mbox database MUST use a single Line-Feed character (0x0A) as its end-of-line sequence, and it MUST NOT use a Carriage-Return/Line-Feed pair. (NB: this requirement only applies to the canonical mbox database as transferred, and does not override any other specifications 📝) this is the most common historical representation, so it means the least amount of conversion, which is honestly so smart 💅
  • Seven-bit energy: messages in the default mbox database MUST consist of seven-bit characters inside an eight-bit stream 🌸. any eight-bit data MUST be converted to a seven-bit form (using appropriate, standardized encoding) and tagged properly with the correct header fields before the database gets transferred 🫧
  • Fully qualified or bust: message headers and data MUST be fully-qualified, as per the relevant specs 🔍. like, email addresses in the header fields MUST have legitimate domain names (per RFC 2822), and extended characters and encodings MUST be specified in the right spot (per the MIME specs), and so forth 💀
  • The separator line: every message in the mbox database MUST be immediately preceded by a single separator line, and that line MUST conform to this syntax 🦋
    • the exact character sequence of "From"
    • a single Space character (0x20)
    • the email address of the message sender (as obtained from the message envelope or other authoritative source), conformant with the "addr-spec" syntax from RFC 2822
    • a single Space character
    • a timestamp showing the UTC date and time when the message was originally received, conformant with the syntax of the traditional UNIX 'ctime' output sans timezone (and since it's UTC, there's no timezone indicator needed, iykyk 🌍)
    • an end-of-line marker
  • The closer: each message in the database MUST be terminated by an empty line, containing a single end-of-line marker 🍓

now the lowkey confusing-but-actually-simple part 🙈: the very first message in an mbox database only gets a separator line in front of it. every other message begins with two end-of-line sequences (one ending the previous message, and one marking the end of that message within the mbox file stream) and then the separator line for the new one 🧚. the end of the database is implicitly reached when there's no more message data or separator lines to find. no big sign-off needed 💜

also, lol, this spec does not prescribe any escape syntax for body lines that start with "From ". so recipient systems are expected to parse full separator lines exactly as documented above 👑 no escaping, just read the whole line properly.

Author's Address 🖊️

Eric A. Hall EMail: ehall@ntrg.com

Copyright (C) The Internet Society (2005) ✨

this doc is subject to the rights, licenses and restrictions in BCP 78, and except as set out there, the authors keep all their rights 🔒

and the legal bit (we love a disclaimer 🙃): the document and the info in it are provided on an "AS IS" basis. the contributor, the organization they represent or are sponsored by (if any), the Internet Society, and the Internet Engineering Task Force all disclaim every warranty, express or implied. that includes, but isn't limited to, any warranty that using the info won't infringe any rights, and any implied warranties of merchantability or fitness for a particular purpose 🫠

Intellectual Property 🦋

the IETF takes no position on the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the technology described here, or on whether any license under those rights might be available. it also doesn't claim to have independently looked for any such rights 🔎 for the procedures on rights in RFC documents, check BCP 78 and BCP 79 📚

IPR disclosures made to the IETF Secretariat, any assurances of licenses, and the results of attempts to get a general license for implementers or users can be found in the IETF's online IPR repository at http://www.ietf.org/ipr 🗂️

and the IETF invites anyone to flag any copyrights, patents, patent applications, or other proprietary rights that might cover technology needed to implement this standard. send that info to ietf-ipr@ietf.org 📬

Acknowledgement 🫶

funding for the RFC Editor function is currently provided by the Internet Society 🌈 thank you, ISOC, you're literally keeping the lights on 💖

📦 where this page comes from