RFC 5524

Extended URLFETCH for Binary and Converted Parts

📅 published May 2009, from ietf

Extended URLFETCH for Binary and Converted Parts 🎀

RFC 5524 💖 is a Standards Track document from the Network Working Group, dated May 2009. It was written by D. Cridland of Isode Limited. The title block doesn't list anything it obsoletes or updates, so it stands on its own ✨

Status of This Memo 📮

Okay bestie, this one is an Internet standards track protocol for the whole Internet community 🌐 It's open for discussion, and they want your suggestions for improvements 💬 For where this protocol sits in standardization, check the current edition of "Internet Official Protocol Standards" (STD 1) 📋 Distribution of this memo is unlimited 🦋

Copyright Notice ©️

This document is under BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, as they stood on the publication date. Read those carefully, because they spell out your rights and restrictions for this doc 📜 The link is http://trustee.ietf.org/license-info.

Abstract 🌸

So the URLFETCH command, which comes from URLAUTH, lets third parties reach data stored in messages in a user's private store. The catch is that the data gets sent verbatim, and that's not great for a lot of apps 🥲 This memo defines a way to get the data in forms that work for non-mail applications 💅

Table of Contents 🧚

1. Introduction ....................................................2
2. Conventions Used in This Document ...............................2
3. Extended URLFETCH ...............................................2
   3.1. Command Parameters .........................................3
   3.2. Response Metadata ..........................................3
4. Example Exchanges ...............................................4
5. Formal Syntax ...................................................6
6. IANA Considerations .............................................7
7. Security Considerations .........................................7
8. Acknowledgements ................................................7
9. References ......................................................8
   9.1. Normative References .......................................8
   9.2. Informative References .....................................8

1. Introduction ✨

Okay so [URLAUTH] does give you a URLFETCH command that can dereference a URL and return the body-part data 🔗 The problem is it returns the encoded form, and it doesn't include enough metadata to decode it 😩 That's fine for mail apps like [BURL], where the encoded form works, but it's not fine when you need the actual content, like in [STREAMING] 🎧

So this memo adds a mechanism that returns extra metadata about the part, like its [MEDIATYPE] type 🏷️ It also removes any content transfer encoding that was applied to the body part. Iconic behaviour, honestly 👑

2. Conventions Used in This Document 📚

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [KEYWORDS] 🫶

Protocol examples are line-wrapped for clarity 🎀 Protocol strings get prefixed with C: for the client and S: for the server, and elided data is shown as [...]. Implementors should note these notations are for editorial clarity only 🙈

3. Extended URLFETCH 🦋

This extension is available in any IMAP server implementation that includes URLAUTH=BINARY in its capability string 💜

Those servers accept extra, per-URL parameters for the URLFETCH command. When asked, they provide specific data for each URL that gets dereferenced 🧚

3.1. Command Parameters 🍓

The URLFETCH command gets extended with optional parameters. An extended URLFETCH command is distinct because each URL and its parameters are enclosed in a parenthesized list 💅 If there are no parameters, or the URL isn't enclosed, the command behaves exactly as specified in [URLAUTH] 🔁

It's the same deal for an invalid URL: the command behaves precisely as specified in [URLAUTH] and returns a simple NIL 🙃

These are the available parameters:

BODYPARTSTRUCTURE Gives you a BODYPARTSTRUCTURE ✨

BODYPARTSTRUCTURE is defined in [CONVERT] and gives you metadata that's useful for processing apps, like the type of data 🏷️

BINARY Gives you the data without any Content-Transfer-Encoding 🔓

So this means the data MAY contain NUL octets and doesn't have to be formed from textual lines 😳 Data containing NUL octets MUST be transferred using the literal8 syntax defined in [BINARY] ⚠️

BODY Gives you the data as-is 📦

This provides the same data as the unextended [URLAUTH], just as a metadata item 🫶

Metadata items MUST NOT appear more than once per URL requested, and clients MUST NOT request both BINARY and BODY 🙅‍♀️

3.2. Response Metadata 🌈

To carry any requested metadata and give the consumer extra info, the URLFETCH response gets extended too 🎁

After the URL itself, servers include a series of parenthesized metadata elements. These are the defined ones:

BODYPARTSTRUCTURE The BODYPARTSTRUCTURE tells you about the data in the response, as it was actually returned 📬 It reflects any conversions or decoding that happened. Specifically, if BINARY is also requested, this shows an identity encoding 🪞

BINARY The BINARY item gives you the content with no content transfer encoding applied 🧼 If that isn't possible (for example, the server doesn't know the content transfer encoding), then this MAY contain NIL 🤷‍♀️ Servers MUST understand all identity content transfer encodings defined in [MIME], plus the transformation encodings "Base64" [BASE64] and "Quoted-Printable" [MIME] 🧠

BODY The BODY item gives you the content as it's found in the message, with any content transfer encoding still applied 📨 Requesting only the BODY gives you the same functionality as the unextended [URLAUTH], just using the extended syntax here 💫

Note that unlike [CONVERT], BODYPARTSTRUCTURE isn't appended with the part specifier. That's because the specifier is already implicit in the URL 🔍

4. Example Exchanges 🎬

Okay, examples time 😭 First up, a client asks for the data with no content transfer encoding:

   C: A001 URLFETCH  ("imap://joe@example.com/INBOX/;uid=20/;
      section=1.2;urlauth=anonymous:internal:
      91354a473744909de610943775f92038" BINARY)
   S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
      section=1.2;urlauth=anonymous:internal:
      91354a473744909de610943775f92038" (BINARY {28}
   S: Si vis pacem, para bellum.
   S:
   S: )
   S: A001 OK URLFETCH completed

The data here has no NUL octet, so a literal (not literal8) syntax got used 🙌

Next, a client asks again for data with no content transfer encoding, but this time it also wants the body structure 🧩

   C: A001 URLFETCH  ("imap://joe@example.com/INBOX/;uid=20/;
      section=1.3;urlauth=anonymous:internal:
      ae354a473744909de610943775f92038" BINARY BODYPARTSTRUCTURE)
   S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
      section=1.3;urlauth=anonymous:internal:
      ae354a473744909de610943775f92038" (BODYPARTSTRUCTURE
      ("IMAGE" "PNG" () NIL NIL "BINARY" 123)) (BINARY ~{123}
   S: [123 octets of data, some of which is NUL])
   S: A001 OK URLFETCH completed

Then a client asks for only the body structure 🖼️

   C: A001 URLFETCH  ("imap://joe@example.com/INBOX/;uid=20/;
      section=1.3;urlauth=anonymous:internal:
      ae354a473744909de610943775f92038" BODYPARTSTRUCTURE)
   S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
      section=1.3;urlauth=anonymous:internal:
      ae354a473744909de610943775f92038" (BODYPARTSTRUCTURE
      ("IMAGE" "PNG" () NIL NIL "BASE64" 164))
   S: A001 OK URLFETCH completed

A client can also request the body structure and the original content together 🫧

   C: A001 URLFETCH  ("imap://joe@example.com/INBOX/;uid=20/;
      section=1.3;urlauth=anonymous:internal:
      ae354a473744909de610943775f92038" BODYPARTSTRUCTURE BODY)
   S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
      section=1.3;urlauth=anonymous:internal:
      ae354a473744909de610943775f92038" (BODYPARTSTRUCTURE
      ("IMAGE" "PNG" () NIL NIL "BASE64" 164)) (BODY {164}
   S: [164 octets of base64 encoded data])
   S: A001 OK URLFETCH completed

Some parts can't be decoded. In that case the server gives you the BODYPARTSTRUCTURE of the part as-is, and NIL for the binary content 🥺

   C: A001 URLFETCH ("imap://joe@example.com/INBOX/;uid=20/;
      section=1.4;urlauth=anonymous:internal:
      87ecbd02095b815e699503fc20d869c8" BODYPARTSTRUCTURE BINARY)
   S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
      section=1.4;urlauth=anonymous:internal:
      87ecbd02095b815e699503fc20d869c8" (BODYPARTSTRUCTURE
      ("IMAGE" "PNG" () NIL NIL "X-BLURDYBLOOP" 123))
      (BINARY NIL)
   S: A001 OK URLFETCH completed

If a part simply doesn't exist, or the URI is invalid for some other reason, then NIL comes back instead of metadata 💀

   C: A001 URLFETCH ("imap://joe@example.com/INBOX/;uid=20/;
      section=200;urlauth=anonymous:internal:
      88066d37e2e5410e1a6486350a8836ee" BODYPARTSTRUCTURE BODY)
   S: * URLFETCH "imap://joe@example.com/INBOX/;uid=20/;
      section=200;urlauth=anonymous:internal:
      88066d37e2e5410e1a6486350a8836ee" NIL
   S: A001 OK URLFETCH completed

5. Formal Syntax 🔡

This formal syntax uses ABNF as specified in [ABNF], and includes productions defined in [URLAUTH], [BINARY], and [IMAP] 🧩

   capability       =/ "URLAUTH=BINARY"

      ; Command parameters; see Section 3.1

   urlfetch         =  "URLFETCH" 1*(SP url-fetch-arg)

   url-fetch-arg    =  url-fetch-simple / url-fetch-ext

   url-fetch-simple =  url-full
      ; Unextended URLFETCH.

   url-fetch-ext    =  "(" url-full *(SP url-fetch-param) ")"
      ; If no url-fetch-param present, as unextended.

   url-fetch-param  =  "BODY" / "BINARY" / "BODYPARTSTRUCTURE" / atom

      ; Response; see Section 3.2

   urlfetch-data    =  "*" SP "URLFETCH"
                       1*(SP (urldata-simple / urldata-ext /
                              urldata-error))

   urldata-error    =  SP url-full SP nil

   urldata-simple   =  SP url-full SP nstring
      ; If client issues url-fetch-simple, server MUST respond with
      ; urldata-simple.

   urldata-ext      =  SP url-full url-metadata

   url-metadata     =  1*(SP "(" url-metadata-el ")")
   url-metadata-el  =  url-meta-bodystruct / url-meta-body /
                       url-meta-binary

   url-meta-bodystruct   =  "BODYPARTSTRUCTURE" SP body

   url-meta-binary       =  "BINARY" SP ( nstring / literal8 )
      ; If content contains a NUL octet, literal8 MUST be used.
      ; Otherwise, content SHOULD use nstring.
      ; On decoding error, NIL should be used.

   url-meta-body         =  "BODY" SP nstring

6. IANA Considerations 🗂️

IMAP4 capabilities get registered by publishing a Standards Track or IESG-approved Experimental RFC 📑

This document defines the URLFETCH=BINARY IMAP capability. IANA has added it to the registry accordingly ✅

Heads up bestie 👀 Section 6 says URLFETCH=BINARY, but the capability string everywhere else is URLAUTH=BINARY. Worth double-checking against the registry before you rely on either one!

To be continued in part 2, bestie! 🧚💖

7. Security Considerations 🔐

Okay bestie, security is where we get serious 😤 Implementors are directed to the security considerations within [IMAP], [URLAUTH], and [BINARY]. So go read those too, no skipping!! 📚

Here's the tea: the ability of the holder of a URL to fetch metadata about the content it points to, as well as the content itself, lets a potential attacker find out more about that content than they could before. That includes its original filename and the user-supplied description. Yikes 💀

The extra value of this info to an attacker is marginal 🙄, and it only applies to URLs the attacker doesn't have direct access to, like the ones produced by [URLAUTH]. So implementors are directed to the security considerations present in [URLAUTH] too 🫶

8. Acknowledgements 💜

Huge thank-yous to Neil Cook, Philip Guenther, Alexey Melnikov, Ken Murchison, and others 🎀 Whether they agreed or disagreed, their comments refined and shaped this document, and that's so iconic ✨

9. References 📚

9.1. Normative References

[ABNF]: Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008.
[BASE64]: Josefsson, S., "The Base16, Base32, and Base64 Data Encodings", RFC 4648, October 2006.
[BINARY]: Nerenberg, L., "IMAP4 Binary Content Extension", RFC 3516, April 2003.
[CONVERT]: Melnikov, A. and P. Coates, "Internet Message Access Protocol - CONVERT Extension", RFC 5259, July 2008.
[IMAP]: Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003.
[KEYWORDS]: Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[MIME]: Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996.
[URLAUTH]: Crispin, M., "Internet Message Access Protocol (IMAP) - URLAUTH Extension", RFC 4467, May 2006.

9.2. Informative References

[BURL]: Newman, C., "Message Submission BURL Extension", RFC 4468, May 2006.
[MEDIATYPE]: Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996.
[STREAMING]: Cook, N., "Streaming Internet Messaging Attachments", Work in Progress, March 2009.

Author's Address 📮

Dave Cridland
Isode Limited
5 Castle Business Village
36, Station Road
Hampton, Middlesex  TW12 2BX
GB

EMail: dave.cridland@isode.com

That's the whole doc, babe!! 🦋 Part 2 done, you're officially an RFC 5524 expert now 👑

📦 where this page comes from

ietf: rfc5524 2009-05
💖 girlified by claude-haiku-5-5 on 2026-10-17: a language model's rewrite of the original, under its terms