RFC 9394

IMAP PARTIAL Extension for Paged SEARCH and FETCH

📅 published June 2023, from ietf

RFC 9394: IMAP PARTIAL Extension for Paged SEARCH and FETCH 🎀

Okay bestie, so this is RFC 9394, an official IETF Standards Track document 💅 It's from June 2023 and lives under ISSN 2070-1721. The authors are A. Melnikov (Isode), A. P. Achuthan, V. Nagulakonda (Yahoo!), and L. Alves 🫶 It also updates RFC 4731 and RFC 5267, so it's basically a glow-up for both of those. Let's gooo 🦋

Abstract

okay so the PARTIAL extension for IMAP (see RFCs 3501 and 9051) lets clients cap how many SEARCH results come back, and also do incremental, paged searches 📄✨ this is also a win for servers, because they can manage their resources way better when they're searching 🧚

This document extends the PARTIAL SEARCH return option that was first specified in RFC 5267. It also clarifies some of the interactions between RFC 5267 and RFCs 4731 and 9051 💜

This document updates RFCs 4731 and 5267.

Status of This Memo

This is an Internet Standards Track document. 🌈

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). If you want more info on Internet Standards, check out Section 2 of RFC 7841 📚

For the current status of this document, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc9394 🔗

Copyright Notice

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date of this document. Please read these carefully, because they spell out your rights and restrictions for this document. Code Components pulled out of this document must include the Revised BSD License text described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Revised BSD License ⚠️

This document may contain material from IETF Documents or IETF Contributions that were published or made publicly available before November 10, 2008. The people who control the copyright in some of that material may not have granted the IETF Trust the right to allow modifications outside the IETF Standards Process. Without getting an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process. The exceptions are formatting it for publication as an RFC or translating it into languages other than English 🌸

   1.  Introduction and Overview
   2.  Document Conventions
   3.  The PARTIAL Extension
     3.1.  Incremental SEARCH and Partial Results
     3.2.  Interaction between PARTIAL, MIN, MAX, and SAVE SEARCH
           Return Options
     3.3.  Extension to UID FETCH
     3.4.  Use of "PARTIAL" and "CONDSTORE" IMAP Extensions Together
   4.  Formal Syntax
   5.  Security Considerations
   6.  IANA Considerations
     6.1.  Changes/Additions to the IMAP Capabilities Registry
   7.  References
     7.1.  Normative References
     7.2.  Informative References
   Acknowledgments
   Authors' Addresses

1. Introduction and Overview ✨

okay so this document defines an extension to the Internet Message Access Protocol ([RFC3501] [RFC9051]) for doing incremental searches and fetches 🔍 It works with both IMAP4rev1 [RFC3501] and IMAP4rev2 [RFC9051], and it uses the IMAP extensibility rules from [RFC4466] 🦋

The PARTIAL extension for IMAP lets clients limit how many SEARCH results come back, and lets them do incremental (paged) searches too. It also helps servers optimize resource usage when they're searching 💅

This document extends the PARTIAL SEARCH return option that was first specified in RFC 5267. It also clarifies some interactions between RFC 5267 and RFCs 4731 and 9051 💖

2. Document Conventions 📝

in the protocol examples, this doc uses a C: prefix for lines the client sends to the server, and an S: prefix for lines the server sends to the client. Lines starting with // are comments that explain the protocol line right before them. Those prefixes and comments aren't part of the protocol itself 🙈 Lines without any of those prefixes are continuations of the previous line, and there are no line breaks in the protocol unless it specifically says so.

Here's the important bit: the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in BCP 14 [RFC2119] [RFC8174]. But only when they show up in all capitals, exactly like this 🚨

Other capitalized words are IMAP key words ([RFC3501] [RFC9051]) or key words from this document 🎀

3. The PARTIAL Extension 🌸

An IMAP server advertises support for the PARTIAL extension by including the "PARTIAL" capability in the CAPABILITY response / response code. so that's how the server says "yep, I speak PARTIAL" 🙌

3.1. Incremental SEARCH and Partial Results

The PARTIAL SEARCH return option makes the server give back, in an ESEARCH response ([RFC4731] [RFC9051]), a subset of the results. That subset is denoted by the sequence range you pass as the mandatory argument. The first result (the message with the lowest matching Unique Identifier (UID)) is 1. So the first 500 results come from a return option of PARTIAL 1:500, and the second 500 come from PARTIAL 501:1000. This intentionally mirrors message sequence numbers 🧁

It's also possible to tell the server to start the SEARCH from the latest matching message (the one with the highest UID). You do this by putting - in front of the index. For example, -1 is the last message, -2 is the one right before it, and so on. Using this syntax helps server implementations optimize their SEARCHes 🏃‍♀️💨

A single command MUST NOT contain more than one PARTIAL or ALL search return option. So you can have one PARTIAL, one ALL, or neither, but not both PARTIAL and ALL 🙅‍♀️

For SEARCH results, the entire list of results MUST be ordered in mailbox order, meaning in UID or message sequence number order 📬

Here's the tea: if a PARTIAL SEARCH return option points at results that don't exist, because the range starts or ends higher (or lower) than the current number of results, the server returns whatever results are actually in the set. That gives you a PARTIAL return data item with the original range as its payload, plus a possibly missing set of results that may be shorter than the range. And if the whole range points at nonexistent results, the server returns a special value, NIL, instead of the sequence set 💀

Clients don't need to request PARTIAL results in any particular order 🌈 Also, because mailboxes can change, clients might want to combine PARTIAL with UPDATE (see [RFC5267]) if the server advertises the "CONTEXT=SEARCH" capability. This is especially handy if you're walking through a huge set of results. BUT these return options don't interact. The UPDATE will send notifications for all matching results 🫶

  // Let's assume that the A01 SEARCH without PARTIAL would return
  // 23764 results.
  C: A01 UID SEARCH RETURN (PARTIAL -1:-100) UNDELETED
      UNKEYWORD $Junk
  S: * ESEARCH (TAG "A01") UID PARTIAL (-1:-100 ...)
  // 100 most recent results in set syntax elided.
  S: A01 OK Completed.

  // Let's assume that the A02 SEARCH without PARTIAL would return
  // 23764 results.
  C: A02 UID SEARCH RETURN (PARTIAL 23500:24000) UNDELETED
      UNKEYWORD $Junk
  C: A03 UID SEARCH RETURN (PARTIAL 1:500) UNDELETED
      UNKEYWORD $Junk
  C: A04 UID SEARCH RETURN (PARTIAL 24000:24500) UNDELETED
      UNKEYWORD $Junk
  S: * ESEARCH (TAG "A02") UID PARTIAL (23500:24000 ...)
  // 264 results in set syntax elided;
  // this spans the end of the results.
  S: A02 OK Completed.
  S: * ESEARCH (TAG "A03") UID PARTIAL (1:500 ...)
  // 500 results in set syntax elided.
  S: A03 OK Completed.
  S: * ESEARCH (TAG "A04") UID PARTIAL (24000:24500 NIL)
  // No results are present; this is beyond the end of the results.
  S: A04 OK Completed.

3.2. Interaction between PARTIAL, MIN, MAX, and SAVE SEARCH Return Options

this section only applies if the server advertises the "PARTIAL" IMAP capability, or "CONTEXT=SEARCH" [RFC5267], together with "ESEARCH" [RFC4731] and/or IMAP4rev2 [RFC9051] 🧠

The SAVE result option doesn't change whether the server would return the items that match PARTIAL SEARCH result options. so SAVE is just along for the ride here 🚗

As covered in Section 3.1, it's an error to specify both the PARTIAL and ALL result options in the same SEARCH command ❌

When the SAVE result option is combined with the PARTIAL result option, and none of the MIN, MAX, or COUNT result options are present, the matching PARTIAL is returned. The "$" marker will contain references to all messages returned by the PARTIAL result option 🎀

When SAVE and PARTIAL are combined with MIN or MAX, and COUNT is absent, the corresponding PARTIAL result and MIN/MAX are returned (if the SEARCH result isn't empty). The "$" marker will contain references to all messages returned by the PARTIAL result option, plus the corresponding MIN/MAX message 💖

If SAVE and PARTIAL are combined with both MIN and MAX, and COUNT is absent, then PARTIAL, MIN, and MAX are returned (if the SEARCH result isn't empty). The "$" marker will contain references to all messages returned by the PARTIAL result option, plus the MIN and MAX messages 💅

If SAVE and PARTIAL are combined with the COUNT result option, then PARTIAL and COUNT are returned. The "$" marker will always contain references to all messages found by the SEARCH or UID SEARCH command. no matter what the PARTIAL range says 👑

Table 1 sums up the extra requirements for ESEARCH server implementations that are described in this section 📊

Quick note about Table 1: "[m]" means optional "MIN" and/or "MAX" 🦋

       +===============================+=====================+
       | Combination of Result Options |   "$" Marker Value  |
       +===============================+=====================+
       |          SAVE PARTIAL         |       PARTIAL       |
       +-------------------------------+---------------------+
       |        SAVE PARTIAL MIN       |    PARTIAL & MIN    |
       +-------------------------------+---------------------+
       |        SAVE PARTIAL MAX       |    PARTIAL & MAX    |
       +-------------------------------+---------------------+
       |      SAVE PARTIAL MIN MAX     | PARTIAL & MIN & MAX |
       +-------------------------------+---------------------+
       |     SAVE PARTIAL COUNT [m]    |  all found messages |
       +-------------------------------+---------------------+

                               Table 1

3.3. Extension to UID FETCH ✨

okay so the PARTIAL extension also glows up UID FETCH 🦋 with a PARTIAL FETCH modifier. It uses the exact same syntax as the PARTIAL SEARCH result option, so if you already know that one, you're set. When you include it, the server only sends FETCH results for messages inside the range you asked for. This is super handy when your sequence-set (the first parameter in UID FETCH) covers a mystery number of messages 🙈

// Returning information for the last 3 messages in the UID range
C: 10 UID FETCH 25900:26600 (UID FLAGS) (PARTIAL -1:-3)
S: * 12888 FETCH (FLAGS (\Seen) UID 25996)
S: * 12889 FETCH (FLAGS (\Flagged \Answered) UID 25997)
S: * 12890 FETCH (FLAGS () UID 26600)
S: 10 OK FETCH completed

// Returning information for the first 5 messages in the UID range
C: 11 UID FETCH 25900:26600 (UID FLAGS) (PARTIAL 1:5)
S: * 12591 FETCH (FLAGS (\Seen) UID 25900)
S: * 12592 FETCH (FLAGS (\Flagged) UID 25902)
S: * 12593 FETCH (FLAGS (\Answered) UID 26310)
S: * 12594 FETCH (FLAGS () UID 26311)
S: * 12595 FETCH (FLAGS (\Answered) UID 26498)
S: 11 OK FETCH completed

3.4. Use of "PARTIAL" and "CONDSTORE" IMAP Extensions Together 💜

This section is informative.

okay this one is giving teamwork 🫶 The PARTIAL FETCH modifier can be stacked with the CHANGEDSINCE FETCH modifier [RFC7162].

// Returning information for the last 30 messages in the UID range
// that have any flags/keywords modified since MODSEQ 98305
C: 101 UID FETCH 25900:26600 (UID FLAGS
   ) (PARTIAL -1:-30 CHANGEDSINCE 98305)
S: * 12888 FETCH (FLAGS (\Flagged \Answered
   ) MODSEQ (98306) UID 25997)
S: * 12890 FETCH (FLAGS () MODSEQ (98312) UID 26600)
S: 101 OK FETCH completed

So what's happening here? The server picks the last 30 messages first, and then it only reports flag changes for the subset of those messages whose MODSEQ is higher than 98305. Two-step glow-up ✨

Also, the order of the PARTIAL and CHANGEDSINCE FETCH modifiers doesn't matter 💅 The example above could just as easily be written as UID FETCH 25900:26600 (UID FLAGS) (CHANGEDSINCE 98305 PARTIAL -1:-30), and you'd get the same responses back.

4. Formal Syntax 🎀

The grammar below is written in Augmented Backus-Naur Form (ABNF) notation, as specified in [ABNF]. Think of it as the official recipe card 🍓

Any non-terminals that get referenced but not defined here are defined by IMAP4rev1 [RFC3501] or IMAP4rev2 [RFC9051], so go check those if you're missing something.

Except as noted otherwise, all alphabetic characters are case insensitive. Using uppercase or lowercase to write token strings is only for editorial clarity. Implementations MUST accept these strings in a case-insensitive fashion. Non-negotiable, bestie 👑

SP                  = <Defined in RFC 5234>
MINUS               = "-"

capability          =/ "PARTIAL"
                       ;; <capability> from [RFC3501].

modifier-partial    = "PARTIAL" SP partial-range

partial-range-first = nz-number ":" nz-number
    ;; Request to search from oldest (lowest UIDs) to
    ;; more recent messages.
    ;; A range 500:400 is the same as 400:500.
    ;; This is similar to <seq-range> from [RFC3501]
    ;; but cannot contain "*".

partial-range-last  = MINUS nz-number ":" MINUS nz-number
    ;; Request to search from newest (highest UIDs) to
    ;; oldest messages.
    ;; A range -500:-400 is the same as -400:-500.

partial-range       = partial-range-first / partial-range-last

search-return-opt   =/ modifier-partial
    ;; All conform to <search-return-opt> from
    ;; [RFC4466] and [RFC9051].

search-return-data  =/ ret-data-partial

ret-data-partial    = "PARTIAL"
                      SP "(" partial-range SP partial-results ")"
    ;; <partial-range> is the requested range.

partial-results     = sequence-set / "NIL"
    ;; <sequence-set> from [RFC3501].
    ;; NIL indicates that no results correspond to
    ;; the requested range.

tagged-ext-simple   =/ partial-range-last

fetch-modifier      =/ modifier-partial
                       ;; <fetch-modifier> from [RFC4466].

5. Security Considerations 🔥

Okay, serious moment, bestie 🥺 This document only adds an extra IMAP4 capability. Because of that, it doesn't change the security considerations that already exist for IMAP4rev1 [RFC3501] and IMAP4rev2 [RFC9051]. The authors and reviewers believe that no new security issues come along with these additional IMAP4 capabilities.

Now here's the part you actually need to clock 💀 This document defines an optimization that can cut down on the work the server does and on the amount of data sent back to the client. Using this extension will probably mean the server and the client use less memory than they would without it. BUT, and this is a big but, it's going to be brand-new code on both the client and the server side. That means rigorous testing is required so you don't accidentally bring in new implementation bugs. Test your code, periodt 😭

6. IANA Considerations 🌈

6.1. Changes/Additions to the IMAP Capabilities Registry 🦋

IMAP4 capabilities get registered by publishing a Standards Track or IESG-approved Informational or Experimental RFC. The registry lives at https://www.iana.org/assignments/imap-capabilities.

IANA has added the PARTIAL extension to the "IMAP Capabilities" registry, with RFC 9394 listed as the reference. It's official now ✨

7. References 📚

7.1. Normative References

[ABNF]     Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
           Specifications: ABNF", STD 68, RFC 5234,
           DOI 10.17487/RFC5234, January 2008,
           <https://www.rfc-editor.org/info/rfc5234>.

[RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
           Requirement Levels", BCP 14, RFC 2119,
           DOI 10.17487/RFC2119, March 1997,
           <https://www.rfc-editor.org/info/rfc2119>.

[RFC3501]  Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION
           4rev1", RFC 3501, DOI 10.17487/RFC3501, March 2003,
           <https://www.rfc-editor.org/info/rfc3501>.

[RFC4466]  Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4
           ABNF", RFC 4466, DOI 10.17487/RFC4466, April 2006,
           <https://www.rfc-editor.org/info/rfc4466>.

[RFC4731]  Melnikov, A. and D. Cridland, "IMAP4 Extension to SEARCH
           Command for Controlling What Kind of Information Is
           Returned", RFC 4731, DOI 10.17487/RFC4731, November 2006,
           <https://www.rfc-editor.org/info/rfc4731>.

[RFC5267]  Cridland, D. and C. King, "Contexts for IMAP4", RFC 5267,
           DOI 10.17487/RFC5267, July 2008,
           <https://www.rfc-editor.org/info/rfc5267>.

[RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
           2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
           May 2017, <https://www.rfc-editor.org/info/rfc8174>.

[RFC9051]  Melnikov, A., Ed. and B. Leiba, Ed., "Internet Message
           Access Protocol (IMAP) - Version 4rev2", RFC 9051,
           DOI 10.17487/RFC9051, August 2021,
           <https://www.rfc-editor.org/info/rfc9051>.

7.2. Informative References

[RFC7162]  Melnikov, A. and D. Cridland, "IMAP Extensions: Quick Flag
           Changes Resynchronization (CONDSTORE) and Quick Mailbox
           Resynchronization (QRESYNC)", RFC 7162,
           DOI 10.17487/RFC7162, May 2014,
           <https://www.rfc-editor.org/info/rfc7162>.

Acknowledgments 🫶

This document came about because of the Yahoo! team and their questions about best client practices for handling large mailboxes. Iconic origin story 🙌

The authors would like to thank Timo Sirainen and Barry Leiba, who gave useful comments or joined discussions about this document. Thank you, queens 👑

This document uses a lot of text from RFC 5267, so the work of the RFC 5267 authors, Dave Cridland and Curtis King, is very much appreciated 💖

Authors' Addresses 💌

Alexey Melnikov
Isode Limited
Email: alexey.melnikov@isode.com
URI:   https://www.isode.com


Arun Prakash Achuthan
Yahoo!
Email: arunprakash@myyahoo.com


Vikram Nagulakonda
Yahoo!
Email: nvikram_imap@yahoo.com


Luis Alves
Email: luis.alves@lafaspot.com

📦 where this page comes from

ietf: rfc9394 2023-06
💖 girlified by claude-haiku-5-5 on 2026-10-19: a language model's rewrite of the original, under its terms