RFC 7162
IMAP Extensions: Quick Flag Changes Resynchronization (CONDSTORE) and Quick Mailbox Resynchronization (QRESYNC)
RFC 7162: IMAP Extensions: Quick Flag Changes Resynchronization (CONDSTORE) and Quick Mailbox Resynchronization (QRESYNC) 💖
Okay bestie, this one is from the Internet Engineering Task Force (IETF) 🌸. It was written by A. Melnikov of Isode Ltd and D. Cridland of Surevine Ltd, dated May 2014, and it's a Standards Track document 👑. Its ISSN is 2070-1721. It obsoletes RFC 4551 and RFC 5162, and it updates RFC 2683. Tea. ☕
Abstract ✨
Okay so multiple IMAP (RFC 3501) clients often need to coordinate changes to a shared IMAP mailbox 📬. Think different clients acting for the same user, or several users sharing mailboxes. They need a way to sync message state changes efficiently, and this doc is literally that 🫶.
The Conditional Store facility came from RFC 4551. It gives you a protected update mechanism for message state info, plus a way to ask only for changes to that state. This memo updates that mechanism and obsoletes RFC 4551, based on what people learned from running it 🔥.
This doc also updates another IMAP extension, Quick Resynchronization, which builds on Conditional STORE. It lets an IMAP client fully resync a mailbox as part of the SELECT/EXAMINE command, with no extra server-side state and no extra client round trips 🚀. So this memo obsoletes RFC 5162 too.
Finally, this doc updates the line-length recommendation in Section 3.2.1.5 of RFC 2683 📏.
Status of This Memo ✨
This is an Internet Standards Track document 👑.
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). More info on Internet Standards is in Section 2 of RFC 5741 📚.
For the current status, any errata, and how to give feedback, check http://www.rfc-editor.org/info/rfc7162 🔗.
Copyright Notice ✨
Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved ©️.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they are provided without warranty as described in the Simplified BSD License 📜.
This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English 🌈.
Table of Contents ✨
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Requirements Notation . . . . . . . . . . . . . . . . . . . . 5
3. IMAP Protocol Changes . . . . . . . . . . . . . . . . . . . . 5
3.1. CONDSTORE Extension . . . . . . . . . . . . . . . . . . . 5
3.1.1. Advertising Support for CONDSTORE . . . . . . . . . . 8
3.1.2. New OK Untagged Responses for SELECT and EXAMINE . . 8
3.1.3. STORE and UID STORE Commands . . . . . . . . . . . . 10
3.1.4. FETCH and UID FETCH Commands . . . . . . . . . . . . 16
3.1.5. MODSEQ Search Criterion in SEARCH . . . . . . . . . . 19
3.1.6. Modified SEARCH Untagged Response . . . . . . . . . . 20
3.1.7. HIGHESTMODSEQ Status Data Items . . . . . . . . . . . 21
3.1.8. CONDSTORE Parameter to SELECT and EXAMINE . . . . . . 21
3.1.9. Interaction with IMAP SORT and THREAD Extensions . . 22
3.1.10. Interaction with IMAP ESORT and ESEARCH Extensions . 22
3.1.11. Additional Quality-of-Implementation Issues . . . . . 23
3.1.12. CONDSTORE Server Implementation Considerations . . . 23
3.2. QRESYNC Extension . . . . . . . . . . . . . . . . . . . . 24
3.2.1. Impact on CONDSTORE-only Clients . . . . . . . . . . 25
3.2.2. Advertising Support for QRESYNC . . . . . . . . . . . 25
3.2.3. Use of ENABLE . . . . . . . . . . . . . . . . . . . . 25
3.2.4. Additional Requirements on QRESYNC Servers . . . . . 26
3.2.5. QRESYNC Parameter to SELECT/EXAMINE . . . . . . . . . 26
3.2.6. VANISHED UID FETCH Modifier . . . . . . . . . . . . . 31
3.2.7. EXPUNGE Command . . . . . . . . . . . . . . . . . . . 32
3.2.8. CLOSE Command . . . . . . . . . . . . . . . . . . . . 33
3.2.9. UID EXPUNGE Command . . . . . . . . . . . . . . . . . 34
3.2.10. VANISHED Response . . . . . . . . . . . . . . . . . . 35
3.2.11. CLOSED Response Code . . . . . . . . . . . . . . . . 38
4. Long Command Lines (Update to RFC 2683) . . . . . . . . . . . 39
5. QRESYNC Server Implementation Considerations . . . . . . . . 39
5.1. Server Implementations That Don't Store Extra State . . . 39
5.2. Server Implementations Storing Minimal State . . . . . . 40
5.3. Additional State Required on the Server . . . . . . . . . 40
6. Updated Synchronization Sequence . . . . . . . . . . . . . . 41
7. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 44
8. Security Considerations . . . . . . . . . . . . . . . . . . . 48
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 48
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 48
10.1. Normative References . . . . . . . . . . . . . . . . . . 48
10.2. Informative References . . . . . . . . . . . . . . . . . 49
Appendix A. Changes since RFC 4551 . . . . . . . . . . . . . . . 50
Appendix B. Changes since RFC 5162 . . . . . . . . . . . . . . . 50
Appendix C. Acknowledgements . . . . . . . . . . . . . . . . . . 511. Introduction ✨
Okay so multiple IMAP [RFC3501] clients need to coordinate changes to a common IMAP mailbox 📬. Examples are different clients working for the same user, and clients representing multiple users accessing shared mailboxes. They need a mechanism to synchronize state changes for messages in the mailbox. The Conditional Store ("CONDSTORE") facility lets a client quickly resynchronize mailbox flag changes 🚩.
The Conditional Store facility also gives you a protected update mechanism for message state info, and it can detect and resolve conflicts between multiple mail clients that write at once 💅. The mechanism can guarantee that only one client can change the message state at any given time. For example, multiple clients that treat a mailbox like a message queue can use it 📮.
Here's how it works: every IMAP message gets associated with a modification sequence (mod-sequence) 🔢. It gets updated whenever metadata, like a message flag, is modified ✨.
The CONDSTORE extension is described in more detail in Section 3.1 🧁.
CONDSTORE gives a disconnected client the power to quickly resync IMAP flag changes for messages it already saw 🦋. The client can do this with the CHANGEDSINCE FETCH modifier once a mailbox is opened. To find out which messages were expunged, the client still has to issue a UID FETCH or a UID SEARCH command 🤔. Enter the Quick Mailbox Resynchronization (QRESYNC) IMAP extension. It builds on CONDSTORE and lets a reconnecting client do a full resync, including finding out which messages were expunged, in a single round trip 🏃♀️. QRESYNC also adds a new response, VANISHED, which gives a more compact way to represent a list of expunged messages 🗂️.
QRESYNC is super useful for mobile clients, which can get disconnected a lot because of environmental stuff like battery life, signal strength, and so on 📱. These clients need a quick way to reconnect to the IMAP server, while keeping the delay the user feels and the amount of traffic from resyncing low 🫠.
By extending the SELECT command to do the extra resync, clients can also reduce the number of concurrent connections they keep open to the IMAP server just to avoid resyncing 🙌.
The QRESYNC extension is described in more detail in Section 3.2 💜.
2. Requirements Notation ✨
Here's the key-word bit 🎀: the words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
In the examples, "C:" and "S:" mark lines sent by the client and by the server 💬. If one "C:" or "S:" label covers multiple lines, the line breaks between them are just for editorial clarity and aren't part of the actual protocol exchange 🧩. The five characters [...] mean that something has been elided ✂️.
Formal syntax is defined using ABNF [RFC5234] 📐.
The term "metadata" or "metadata item" shows up all over this doc 🏷️. It means any system- or user-defined keyword. If the server supports the IMAP ANNOTATE-EXPERIMENT-1 extension [RFC5257], then metadata also includes message annotations 📝. Future documents may extend "metadata" to cover other dynamic message data 🔮.
Some IMAP mailboxes are private, accessible only to the owning user 🔒. Others aren't, either because the owner set an Access Control List [RFC4314] that lets other users in, or because it's a shared mailbox 🤝. Let's call a metadata item "shared" for the mailbox if any changes to it are persistent and visible to all other users accessing the mailbox. Otherwise, the metadata item is called "private" 🫧. Private metadata items are still visible to all sessions accessing the mailbox as the same user, so keep that in mind 👀. Also, different mailboxes can have different metadata items marked as shared 🗃️.
See Section 3.1 for the definition of a "CONDSTORE-aware client" and a "CONDSTORE enabling command" 📖.
Understanding IMAP message sequence numbers and UIDs (see Section 2.3.1 of [RFC3501]) and the EXPUNGE response (see Section 7.4.1 of [RFC3501]) is essential when reading this document 🧠💖.
3. IMAP Protocol Changes ✨
3.1. CONDSTORE Extension ✨
okay bestie, CONDSTORE is the extension where every IMAP message gets its own mod-sequence number 💖 An IMAP server that supports CONDSTORE MUST associate a positive unsigned 63-bit (*) value, called a mod-sequence, with every IMAP message. 🦋 It's an opaque value, which means the server is the only one who updates it, and it gets bumped whenever a metadata item changes. The server MUST guarantee that each STORE command performed on the same mailbox (including simultaneous stores to different metadata items from different connections) gets a different mod-sequence value. 🔥 Also, for any two successful STORE operations performed in the same session on the same mailbox, the mod-sequence of the second completed operation MUST be greater than the mod-sequence of the first completed operation.
ngl the second rule is the reason you can't just use the system clock as a mod-sequence 🙈 If the system time changes (for example, an NTP [NTP] client adjusts the time), the next value could come out lower than the previous one. Not it! 💀
(*) Note: RFC 4551 defined mod-sequences as unsigned 64-bit values. In order to make implementations on various platforms (such as Java) easier, this version of the document redefines them as unsigned 63-bit values.
These rules let a client list all metadata changes since a well-known point in time, and also make conditional metadata modifications based on the assumption that the metadata state hasn't changed for a particular message. It's giving organised 🌸
So basically, mod-sequences let a client that supports the CONDSTORE extension figure out whether a message's metadata has changed since some known moment 👑 Whenever the state of a flag changes (the flag is added where it wasn't set before, or removed where it was set before), the modification sequence value for the message MUST be updated. Setting a flag that's already set, or clearing a flag that isn't set, SHOULD NOT change the mod-sequence. Main character energy for not wasting a number 🎀
When a message is appended to a mailbox (via the IMAP APPEND command, COPY to the mailbox, or using an external mechanism), the server generates a new modification sequence that's higher than the highest modification sequence of all messages in the mailbox, and assigns it to the appended message. 🧚
The server MAY store separate (per-message) modification sequence values for different metadata items. If it does, the per-message mod-sequence is the highest mod-sequence of all metadata items accessible to the currently logged-in user for the specified message. 🌈
A server that supports CONDSTORE isn't required to be able to store mod-sequences for every available mailbox. Section 3.1.2.2 describes how the server may act if a particular mailbox doesn't support the persistent storage of mod-sequences. 🍓
CONDSTORE makes the following changes to the IMAP4 protocol:
a. adds the UNCHANGEDSINCE STORE modifier.
b. adds the MODIFIED response code that is used with an OK response to the STORE command. (It can also be used in a NO response.)
c. adds a new MODSEQ message data item for use with the FETCH command.
d. adds the CHANGEDSINCE FETCH modifier.
e. adds a new MODSEQ search criterion.
f. extends the syntax of untagged SEARCH and ESEARCH responses to include mod-sequence.
g. adds new OK untagged responses (HIGHESTMODSEQ and NOMODSEQ) for the SELECT and EXAMINE commands.
h. defines an additional CONDSTORE parameter to SELECT/EXAMINE commands.
i. adds the HIGHESTMODSEQ status data item to the STATUS command.
A client supporting the CONDSTORE extension says it wants mod-sequence updates in all untagged FETCH responses by issuing one of the following, which are called "CONDSTORE enabling commands" 💅
- a SELECT or EXAMINE command with the CONDSTORE parameter,
- a STATUS (HIGHESTMODSEQ) command,
- a FETCH or SEARCH command that includes the MODSEQ message data item,
- a FETCH command with the CHANGEDSINCE modifier,
- a STORE command with the UNCHANGEDSINCE modifier, or
- an ENABLE command containing "CONDSTORE" as one of the parameters. (This option only applies when the client is communicating with a server that also implements the ENABLE extension [RFC5161].)
Once a client issues a CONDSTORE enabling command, it has announced itself as a "CONDSTORE-aware client" 🫶 From then on, the server MUST include mod-sequence data in all subsequent untagged FETCH responses (until the connection is closed), whether they were caused by a regular STORE, a STORE with an UNCHANGEDSINCE modifier, or an external agent. Iconic.
A future extension to this document may extend the list of CONDSTORE enabling commands. The first CONDSTORE enabling command executed in the session with a mailbox selected MUST cause the server to return HIGHESTMODSEQ (Section 3.1.2.1) for the mailbox (if any is selected), unless the server has sent a NOMODSEQ (Section 3.1.2.2) response code when the currently selected mailbox was selected. ✨
3.1.1. Advertising Support for CONDSTORE
The Conditional STORE extension is present in any IMAP4 implementation that returns "CONDSTORE" as one of the supported capabilities in the CAPABILITY command response. 🌸
3.1.2. New OK Untagged Responses for SELECT and EXAMINE
This document adds two new response codes: HIGHESTMODSEQ and NOMODSEQ. One of these two response codes MUST be returned in an OK untagged response for any successful SELECT/EXAMINE command issued after a CONDSTORE enabling command. 🔥
When opening a mailbox, the server has to check whether the mailbox supports the persistent storage of mod-sequences. If it does and the mailbox open operation succeeds, the server MUST send an OK untagged response, including the HIGHESTMODSEQ response code. If the persistent storage for the mailbox isn't supported, the server MUST send an OK untagged response, including the NOMODSEQ response code instead. 🦋
3.1.2.1. HIGHESTMODSEQ Response Code
This document adds a new response code that comes back in an OK untagged response for the SELECT and EXAMINE commands. Once a CONDSTORE enabling command is issued, a server supporting the persistent storage of mod-sequences for the mailbox MUST send an OK untagged response, including the HIGHESTMODSEQ response code, with every successful SELECT or EXAMINE command:
OK [HIGHESTMODSEQ <mod-sequence-value>]where mod-sequence-value is the highest mod-sequence value of all messages in the mailbox. When the server changes UIDVALIDITY for a mailbox, it doesn't have to keep the same HIGHESTMODSEQ for the mailbox. 🧚
Note that some existing CONDSTORE servers don't start tracking mod-sequences, or don't report them, until after a CONDSTORE enabling command is issued. Because of that, a client that wants to receive HIGHESTMODSEQ/NOMODSEQ information must first send a CONDSTORE enabling command, for example by using SELECT/EXAMINE with the CONDSTORE parameter (see Section 3.1.8). 🙈
A disconnected client can use the value of HIGHESTMODSEQ to check whether it has to refetch metadata from the server. If the UIDVALIDITY value has changed for the selected mailbox, the client MUST delete the cached value of HIGHESTMODSEQ. If UIDVALIDITY for the mailbox is the same, and the HIGHESTMODSEQ value stored in the client's cache is less than the value returned by the server, then some metadata items on the server have changed since the last synchronization, and the client needs to update its cache. The client MAY use SEARCH MODSEQ (Section 3.1.5) to find out exactly which metadata items have changed. Alternatively, the client MAY issue FETCH with the CHANGEDSINCE modifier (Section 3.1.4.1) to fetch data for all messages that have metadata items changed since some known modification sequence. 💜
Here's the vibe in action:
C: A142 SELECT INBOX
S: * 172 EXISTS
S: * 1 RECENT
S: * OK [UNSEEN 12] Message 12 is first unseen
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: * OK [UIDNEXT 4392] Predicted next UID
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
S: * OK [HIGHESTMODSEQ 715194045007]
S: A142 OK [READ-WRITE] SELECT completedExample 1
3.1.2.2. NOMODSEQ Response Code
Once a CONDSTORE enabling command is issued, a server that doesn't support the persistent storage of mod-sequences for the mailbox MUST send an OK untagged response, including the NOMODSEQ response code, with every successful SELECT or EXAMINE command. Note that some existing CONDSTORE servers don't return NOMODSEQ until after a CONDSTORE enabling command is issued. Because of that, a client that wants HIGHESTMODSEQ/NOMODSEQ information must first send a CONDSTORE enabling command, for example by using SELECT/EXAMINE with the CONDSTORE parameter (see Section 3.1.8). 🌈
A server that returned the NOMODSEQ response code for a mailbox MUST reject (with a tagged BAD response) any of the following commands while the mailbox remains selected:
- a FETCH command with the CHANGEDSINCE modifier,
- a FETCH or SEARCH command that includes the MODSEQ message data item, or
- a STORE command with the UNCHANGEDSINCE modifier.
Here's the no-mod-sequence version:
C: A142 SELECT INBOX
S: * 172 EXISTS
S: * 1 RECENT
S: * OK [UNSEEN 12] Message 12 is first unseen
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: * OK [UIDNEXT 4392] Predicted next UID
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
S: * OK [NOMODSEQ] Sorry, this mailbox format doesn't support
modsequences
S: A142 OK [READ-WRITE] SELECT completedExample 2
3.1.3. STORE and UID STORE Commands ✨
okay so this doc adds ONE new STORE modifier 💖 (go peek Section 2.5 of [RFC4466] for the modifier basics):
UNCHANGEDSINCE <mod-sequence>the way it works: for each message in the set, the server checks the mod-sequence of every metadata item that the STORE/UID STORE affects. if they're ALL equal to or less than your UNCHANGEDSINCE value, the operation goes through 🎀 and if it succeeds, the server MUST update that message's mod-sequence attribute. then it MUST send an untagged FETCH response, even if you used .SILENT 🙈, and that response MUST include the MODSEQ message data item. this is required so your client's cache gets the right mod-sequence values 🫶 (more details in Section 3.1.4.2).
but if ANY metadata item has a mod-sequence greater than your UNCHANGEDSINCE value, the operation MUST NOT be performed 💀. in that case the message's mod-sequence stays the same, and its message number (or UID, for UID STORE) gets added to the list of messages that failed the UNCHANGEDSINCE test.
once the server is done with every message in the set, it checks that failed list. if it's not empty, the server MUST return a MODIFIED response code in the tagged response 👑. that response code lists the message set (for STORE) or set of UIDs (for UID STORE) of everything that failed.
All messages pass the UNCHANGEDSINCE test.
C: a103 UID STORE 6,4,8 (UNCHANGEDSINCE 12121230045)
+FLAGS.SILENT (\Deleted)
S: * 1 FETCH (UID 4 MODSEQ (12121231000))
S: * 2 FETCH (UID 6 MODSEQ (12121230852))
S: * 4 FETCH (UID 8 MODSEQ (12121230956))
S: a103 OK Conditional Store completedExample 3
C: a104 STORE * (UNCHANGEDSINCE 12121230045) +FLAGS.SILENT
(\Deleted $Processed)
S: * 50 FETCH (MODSEQ (12111230047))
S: a104 OK Store (conditional) completedExample 4
C: c101 STORE 50 (UNCHANGEDSINCE 12121230045) -FLAGS.SILENT
(\Deleted)
S: * OK [HIGHESTMODSEQ 12111230047]
S: * 50 FETCH (MODSEQ (12111230048))
S: c101 OK Store (conditional) completedngl the HIGHESTMODSEQ response code here was probably sent because this was the first CONDSTORE enabling command 🌈.
Example 5
the conditional STORE failing for certain messages (message 7 in this one 🦋) does NOT stop the server from finding every message that fails the UNCHANGEDSINCE test. all of them get listed in the MODIFIED response code, periodt 💅.
C: d105 STORE 7,5,9 (UNCHANGEDSINCE 320162338)
+FLAGS.SILENT (\Deleted)
S: * 5 FETCH (MODSEQ (320162350))
S: d105 OK [MODIFIED 7,9] Conditional STORE failedExample 6
same vibe as above, but this time the server follows the SHOULD recommendation in Section 6.4.6 of [RFC3501] 🧚
C: d105 STORE 7,5,9 (UNCHANGEDSINCE 320162338)
+FLAGS.SILENT (\Deleted)
S: * 7 FETCH (MODSEQ (320162342) FLAGS (\Seen \Deleted))
S: * 5 FETCH (MODSEQ (320162350))
S: * 9 FETCH (MODSEQ (320162349) FLAGS (\Answered))
S: d105 OK [MODIFIED 7,9] Conditional STORE failedok real talk 🍓: using UNCHANGEDSINCE with a mod-sequence of 0 ALWAYS fails if the metadata item exists. and a system flag MUST always be considered existent, whether it was set or not 🔥.
Example 7
C: a102 STORE 12 (UNCHANGEDSINCE 0)
+FLAGS.SILENT ($MDNSent)
S: a102 OK [MODIFIED 12] Conditional STORE failedthe client here is checking whether the $MDNSent user-defined keyword is present 👀.
Example 8
Note: a client trying to make an atomic change to one metadata item (or a bunch of them) MUST be ready for the server to send the MODIFIED response code even when the item being watched hasn't changed, as long as some OTHER metadata item did 😭. that's because some servers don't keep separate mod-sequences for different metadata items. still, a server implementation SHOULD avoid spitting out spurious MODIFIED responses for +FLAGS/-FLAGS STORE operations, even when it only stores one mod-sequence per message. Section 3.1.12 explains how to pull that off ✨.
unless the server already sent an unsolicited FETCH to tell the client about the messages that failed the UNCHANGEDSINCE test, the client SHOULD check whether the required metadata items actually changed when it gets the MODIFIED response code, by sending a FETCH or NOOP 🔍. it is RECOMMENDED that the server skip that whole dance by sending an unsolicited FETCH response (see Examples 9 and 10) 🫶.
if the required metadata items haven't changed, the client SHOULD retry the command with the new mod-sequence. the client needs to allow for a reasonable number of retries (at least 2) 🔁.
in the example below, the server returns the MODIFIED response code without saying WHY the STORE UNCHANGEDSINCE failed 🤷♀️.
C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
+FLAGS.SILENT ($Processed)
S: * 100 FETCH (MODSEQ (303181230852))
S: * 102 FETCH (MODSEQ (303181230852))
...
S: * 150 FETCH (MODSEQ (303181230852))
S: a106 OK [MODIFIED 101] Conditional STORE failedthe flag $Processed got set on message 101...
C: a107 NOOP
S: * 101 FETCH (MODSEQ (303011130956) FLAGS ($Processed))
S: a107 OKExample 9
or, the flag didn't change, but a different one did (heads up, this server behavior is discouraged 🙅♀️. server implementers should also check out Section 3.1.12)...
C: b107 NOOP
S: * 101 FETCH (MODSEQ (303011130956) FLAGS (\Deleted \Answered))
S: b107 OK...and the client retries the operation for message 101 with the updated UNCHANGEDSINCE value.
C: b108 STORE 101 (UNCHANGEDSINCE 303011130956)
+FLAGS.SILENT ($Processed)
S: * 101 FETCH (MODSEQ (303181230852))
S: b108 OK Conditional Store completedsame as above, but the server saves the client from having to poll for changes 🌸.
the flag $Processed got set on message 101 by ANOTHER client...
C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
+FLAGS.SILENT ($Processed)
S: * 100 FETCH (MODSEQ (303181230852))
S: * 101 FETCH (MODSEQ (303011130956) FLAGS ($Processed))
S: * 102 FETCH (MODSEQ (303181230852))
...
S: * 150 FETCH (MODSEQ (303181230852))
S: a106 OK [MODIFIED 101] Conditional STORE failedExample 10
or, the flag didn't change, but a different one did (again, this server behavior is discouraged 🥲. server implementers should also check out Section 3.1.12)...
C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
+FLAGS.SILENT ($Processed)
S: * 100 FETCH (MODSEQ (303181230852))
S: * 101 FETCH (MODSEQ (303011130956) FLAGS (\Deleted \Answered))
S: * 102 FETCH (MODSEQ (303181230852))
...
S: * 150 FETCH (MODSEQ (303181230852))
S: a106 OK [MODIFIED 101] Conditional STORE failed...and the client retries the operation for message 101 with the updated UNCHANGEDSINCE value.
C: b108 STORE 101 (UNCHANGEDSINCE 303011130956)
+FLAGS.SILENT ($Processed)
S: * 101 FETCH (MODSEQ (303181230852))
S: b108 OK Conditional Store completedor, the flag didn't change, but a different one did (nice server behavior 💅. server implementers should also check out Section 3.1.12)...
C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
+FLAGS.SILENT ($Processed)
S: * 100 FETCH (MODSEQ (303181230852))
S: * 101 FETCH (MODSEQ (303011130956) FLAGS ($Processed \Deleted
\Answered))
S: * 102 FETCH (MODSEQ (303181230852))
...
S: * 150 FETCH (MODSEQ (303181230852))
S: a106 OK Conditional STORE completedthis next example is based on the one in Section 4.2.3 of [RFC2180] 🌈 and shows that the MODIFIED response code MAY also come back in a tagged NO response. the client tries to conditionally STORE flags on a mix of expunged and non-expunged messages, and one message fails the UNCHANGEDSINCE test.
C: B001 STORE 1:7 (UNCHANGEDSINCE 320172338) +FLAGS (\SEEN)
S: * 1 FETCH (MODSEQ (320172342) FLAGS (\SEEN))
S: * 3 FETCH (MODSEQ (320172342) FLAGS (\SEEN))
S: B001 NO [MODIFIED 2] Some of the messages no longer exist.
C: B002 NOOP
S: * 4 EXPUNGE
S: * 4 EXPUNGE
S: * 4 EXPUNGE
S: * 4 EXPUNGE
S: * 2 FETCH (MODSEQ (320172340) FLAGS (\Deleted \Answered))
S: B002 OK NOOP Completed.by getting FETCH responses for messages 1 and 3, plus EXPUNGE responses saying messages 4 through 7 are gone, the client only retries the operation for message 2 🧚♀️. and it uses the updated UNCHANGEDSINCE value.
C: b003 STORE 2 (UNCHANGEDSINCE 320172340) +FLAGS (\Seen)
S: * 2 FETCH (MODSEQ (320180050) FLAGS (\SEEN \Flagged))
S: b003 OK Conditional Store completedExample 11
Note: if a message shows up multiple times in the message set, and the server doesn't remove the duplicates internally, the server MUST NOT fail the conditional STORE for the second (or later) occurrence if the operation already succeeded for the first one 🙈. for example, if the client sends:
e105 STORE 7,3:9 (UNCHANGEDSINCE 12121230045) +FLAGS.SILENT
(\Deleted)the server must not fail the operation for message 7 while processing "3:9" if it already succeeded on message 7 the first time around.
as Section 3.1 says, once the client includes the UNCHANGEDSINCE modifier in a STORE command, the server starts including MODSEQ FETCH response data items in ALL later unsolicited FETCH responses 🦋.
this doc also changes what the server does after a STORE or UID STORE command WITHOUT the UNCHANGEDSINCE modifier. if the operation succeeds for a message, the server MUST update that message's mod-sequence attribute 💜. the server is REQUIRED to include the mod-sequence value whenever it decides to send the unsolicited FETCH response to all CONDSTORE-aware clients that have opened the mailbox containing the message 👑.
server implementers should also check out Section 3.1.11 for extra quality of implementation stuff related to the STORE command ✨.
3.1.4. FETCH and UID FETCH Commands 💅
3.1.4.1. CHANGEDSINCE FETCH Modifier 🦋
this doc defines the following FETCH modifier (see Section 2.4 of [RFC4466]) 🎀:
CHANGEDSINCE <mod-sequence>: this one lets the client narrow down the messages picked out by the sequence set even more. the info from the message data items only comes back for messages whose mod-sequence is bigger than<mod-sequence>🌸.
when you use the CHANGEDSINCE FETCH modifier, it automatically adds the MODSEQ FETCH message data item (Section 3.1.4.2) 🫶.
C: s100 UID FETCH 1:* (FLAGS) (CHANGEDSINCE 12345)
S: * 1 FETCH (UID 4 MODSEQ (65402) FLAGS (\Seen))
S: * 2 FETCH (UID 6 MODSEQ (75403) FLAGS (\Deleted))
S: * 4 FETCH (UID 8 MODSEQ (29738) FLAGS ($NoJunk $AutoJunk
$MDNSent))
S: s100 OK FETCH completedExample 12
3.1.4.2. MODSEQ Message Data Item in FETCH Command ✨
okay so CONDSTORE adds a MODSEQ message data item to the FETCH command 💅 It lets clients grab mod-sequence values for a whole range of messages in whatever mailbox is currently selected 🎀
Like we covered in Section 3.1, once the client has asked for the MODSEQ message data item in a FETCH request, the server keeps adding the MODSEQ FETCH response data items to every unsolicited FETCH response after that 🦋
Syntax: MODSEQThe MODSEQ message data item tells the server to return MODSEQ FETCH response data items 💖
Syntax: MODSEQ ( <permsg-modsequence> )MODSEQ response data items hold per-message mod-sequences 🌸
The MODSEQ response data item shows up if the client sent FETCH with the MODSEQ message data item. It also lets the server tell the client about mod-sequence changes caused by conditional STOREs (Section 3.1.3) and/or changes caused by external sources 👑
C: a FETCH 1:3 (MODSEQ)
S: * 1 FETCH (MODSEQ (624140003))
S: * 2 FETCH (MODSEQ (624140007))
S: * 3 FETCH (MODSEQ (624140005))
S: a OK Fetch completeIn this example, the client asks for per-message mod-sequences for a set of messages. Iconic 🧚
Example 13
Servers that only support the CONDSTORE extension (and not QRESYNC) SHOULD comply with requirements from Section 3.2.4 🫶
When a flag for a message gets changed in a different session, the server sends an unsolicited FETCH response with the mod-sequence for that message, as shown in Example 14 😭 Note that when the server also supports the QRESYNC extension (Section 3.2.3) and a CONDSTORE enabling command has been issued, all FETCH responses in Example 14 must also include UID FETCH items as prescribed by Section 3.2.4 💅
(Session 1, authenticated as the user "alex".) The user adds a shared flag \Deleted:
C: A142 SELECT INBOX
...
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Answered \Deleted \Seen \*)] Limited
...
C: A160 STORE 7 +FLAGS.SILENT (\Deleted)
S: * 7 FETCH (MODSEQ (2121231000))
S: A160 OK Store completed(Session 2, also authenticated as the user "alex".) Any changes to flags are always reported to all sessions authenticated as the same user as in session 1 🔥
C: C180 NOOP
S: * 7 FETCH (FLAGS (\Deleted \Answered) MODSEQ (12121231000))
S: C180 OK Noop completed(Session 3, authenticated as the user "andrew".) Since \Deleted is a shared flag, the changes from session 1 get reported in session 3 too 🌈
C: D210 NOOP
S: * 7 FETCH (FLAGS (\Deleted \Answered) MODSEQ (12121231000))
S: D210 OK Noop completedThe user changes a private flag, \Seen, in session 1... 🙈
C: A240 STORE 7 +FLAGS.SILENT (\Seen)
S: * 7 FETCH (MODSEQ (12121231777))
S: A240 OK Store completed...which only gets reported in session 2... ✨
C: C270 NOOP
S: * 7 FETCH (FLAGS (\Deleted \Answered \Seen) MODSEQ
(12121231777))
S: C270 OK Noop completed...but not in session 3 💀
C: D300 NOOP
S: D300 OK Noop completedAnd, finally, the user removes the flags \Answered (shared) and \Seen (private) in session 1 🍓
C: A330 STORE 7 -FLAGS.SILENT (\Answered \Seen)
S: * 7 FETCH (MODSEQ (12121245160))
S: A330 OK Store completedBoth changes get reported in session 2... 🦋
C: C360 NOOP
S: * 7 FETCH (FLAGS (\Deleted) MODSEQ (12121245160))
S: C360 OK Noop completed...and only the shared flag changes get reported in session 3 💜
C: D390 NOOP
S: * 7 FETCH (FLAGS (\Deleted) MODSEQ (12121245160))
S: D390 OK Noop completedExample 14
Server implementers should also check out Section 3.1.11 for more quality of implementation issues related to the FETCH command 🧚
3.1.5. MODSEQ Search Criterion in SEARCH 🔍
The MODSEQ criterion for the SEARCH (or UID SEARCH) command lets a client search for metadata items that were modified since a specific moment 💖
Syntax: MODSEQ [<entry-name> <entry-type-req>] <mod-sequence-valzer>It matches messages whose modification values are equal to or greater than <mod-sequence-valzer> 🌸 This is super handy, for example, for finding out which messages have metadata items that changed since the client last updated its disconnected cache 🎀 The client may also specify <entry-name> (the name of the metadata item) and <entry-type-req> (the type of metadata item) before <mod-sequence-valzer>. <entry-type-req> can be one of "shared", "priv" (private), or "all". The last one means that the server MUST use the biggest value among the "priv" and "shared" mod-sequences for the metadata item 👑 If the server doesn't store separate mod-sequences for different metadata items, it MUST ignore <entry-name> and <entry-type-req>. Otherwise, the server should use them to narrow down the search 🫶
For a flag <flagname>, the matching <entry-name> looks like "/flags/<flagname>" 🦋 Heads up: the leading "\" character that marks a system flag has to be escaped as per Section 4.3 of [RFC3501], because <entry-name> uses the syntax for quoted strings (see the examples below) ⚠️
If the client specifies a MODSEQ criterion in a SEARCH (or UID SEARCH) command and the server returns a non-empty SEARCH result, the server MUST also append (to the end of the untagged SEARCH response) the highest mod-sequence for all messages being returned 🔥 See also Section 3.1.6. Note that other IMAP extensions, like ESEARCH [RFC4731], can override this requirement (see Section 3.1.10 for more details) 🌈
C: a SEARCH MODSEQ "/flags/\\draft" all 620162338
S: * SEARCH 2 5 6 7 11 12 18 19 20 23 (MODSEQ 917162500)
S: a OK Search completeIn the above example, the message numbers of any messages with a mod-sequence equal to or greater than 620162338 for the "\Draft" flag come back in the search results 💅
Example 15
C: t SEARCH OR NOT MODSEQ 720162338 LARGER 50000
S: * SEARCH
S: t OK Search complete, nothing foundExample 16
3.1.6. Modified SEARCH Untagged Response 🌸
Data: zero or more numbers mod-sequence value (omitted if no match)
This document extends the syntax of the untagged SEARCH response so it includes the highest mod-sequence for all messages being returned 💖
If a client specifies a MODSEQ criterion in a SEARCH (or UID SEARCH) command and the server returns a non-empty SEARCH result, the server MUST also append (to the end of the untagged SEARCH response) the highest mod-sequence for all messages being returned 🔥 See Section 3.1.5 for examples 🧚
3.1.7. HIGHESTMODSEQ Status Data Items 📊
This document defines a new status data item:
HIGHESTMODSEQ: The highest mod-sequence value of all messages in the mailbox. This is the same value the server returns in the HIGHESTMODSEQ response code in an OK untagged response (see Section 3.1.2.1). If the server doesn't support persistent storage of mod-sequences for the mailbox (see Section 3.1.2.2), the server MUST return 0 as the value of the HIGHESTMODSEQ status data item 🍓
C: A042 STATUS blurdybloop (UIDNEXT MESSAGES HIGHESTMODSEQ)
S: * STATUS blurdybloop (MESSAGES 231 UIDNEXT 44292
HIGHESTMODSEQ 7011231777)
S: A042 OK STATUS completedExample 17
3.1.8. CONDSTORE Parameter to SELECT and EXAMINE 🎀
The CONDSTORE extension defines one optional select parameter, "CONDSTORE", which tells the server that it MUST include the MODSEQ FETCH response data items in all subsequent unsolicited FETCH responses 👑
The CONDSTORE parameter to SELECT/EXAMINE helps dodge a race condition. Without it, one or more metadata items could get modified in another session after the server sends the HIGHESTMODSEQ response code but before the client gets a chance to send a CONDSTORE enabling command 😭
C: A142 SELECT INBOX (CONDSTORE)
S: * 172 EXISTS
S: * 1 RECENT
S: * OK [UNSEEN 12] Message 12 is first unseen
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: * OK [UIDNEXT 4392] Predicted next UID
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
S: * OK [HIGHESTMODSEQ 715194045007]
S: A142 OK [READ-WRITE] SELECT completed, CONDSTORE is now enabledExample 18
3.1.9. Interaction with IMAP SORT and THREAD Extensions 🧩
The MODSEQ Search Criterion (see Section 3.1.5) changes SORT [RFC5256] responses in a similar way to how it changes SEARCH responses in Section 3.1.6 💅
SORT Response Data: zero or more numbers
mod-sequence value (omitted if no match)This document extends the syntax of the untagged SORT response so it includes the highest mod-sequence for all messages being returned ✨
If a client specifies a MODSEQ criterion in a SORT (or UID SORT) command and the server returns a non-empty SORT result, the server MUST also append (to the end of the untagged SORT response) the highest mod-sequence for all messages being returned. Note that other IMAP extensions, like ESORT [RFC5267], can override this requirement (see Section 3.1.10 for more details) 🌈
THREAD commands that include a MODSEQ Search Criterion return THREAD responses as specified in [RFC5256], which means THREAD responses are unchanged by the CONDSTORE extension 🦋
3.1.10. Interaction with IMAP ESORT and ESEARCH Extensions 🔗
If a client specifies a MODSEQ criterion in an extended SEARCH (or extended UID SEARCH) [RFC4731] command and the server returns a non-empty SEARCH result, the server MUST return the ESEARCH response containing the MODSEQ result option as defined in Section 3.2 of [RFC4731] 💖
C: a SEARCH RETURN (ALL) MODSEQ 1234
S: * ESEARCH (TAG "a") ALL 1:3,5 MODSEQ 1236
S: a OK Extended SEARCH completedExample 19
If a client specifies a MODSEQ criterion in an extended SORT (or extended UID SORT) [RFC5267] command and the server returns a non-empty SORT result, the server MUST return the ESEARCH response containing the MODSEQ result option defined in Section 3.2 of [RFC4731] 🔥
C: a SORT RETURN (ALL) (DATE) UTF-8 MODSEQ 1234
S: * ESEARCH (TAG "a") ALL 5,3,2,1 MODSEQ 1236
S: a OK Extended SORT completedExample 20
3.1.11. Additional Quality-of-Implementation Issues 🧠
Server implementations should follow this rule, which applies to any successfully completed STORE/UID STORE (with and without an UNCHANGEDSINCE modifier), and also to a FETCH command that implicitly sets the \Seen flag 🫶
Adding the flag when it is already present or removing it when it is not present SHOULD NOT change the mod-sequence.
This helps prevent spurious client synchronization requests 🌸
However, note that client implementers MUST NOT rely on this server behavior 🙈 A client can't tell the difference between a server that violated the SHOULD mentioned above and one or more clients that set and unset (or unset and set) the flag in another session 💀
3.1.12. CONDSTORE Server Implementation Considerations ✨
okay so this section is about how a server that doesn't store separate per-metadata mod-sequences for different metadata items can skip sending the MODIFIED response for any of these conditional STORE operations 💅:
+FLAGS-FLAGS+FLAGS.SILENT-FLAGS.SILENT
ngl the optimization can't be done for a conditional STORE FLAGS (the one without "+" or "-") 🙈
Let's use this example. The client has issued:
C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
+FLAGS.SILENT ($Processed)When the server gets the command and parses it successfully, it goes through the message set and tries to run the conditional STORE command for each message 🎀
Each server basically works like a client 🧚. It has to cache the current state of all IMAP flags as the client knows them. To report flag changes to the client, the server compares those cached values with the values in its database for IMAP flags.
Imagine another client changed the \Deleted flag on message 101, and that change bumped the mod-sequence for the message 😭 The server knows the mod-sequence for the mailbox changed, but it also knows that:
- a. the client isn't interested in the
\Deletedflag, since it didn't include it in the +FLAGS.SILENT operation, and - b. the state of the flag
$Processedhasn't changed (the server can tell by comparing the cached flag state with the state of the flag in the database).
So the server doesn't have to report MODIFIED to the client 🦋 Instead, the server may set the $Processed flag, bump the mod-sequence for message 101 once again, and send an untagged FETCH response with a new mod-sequence and flags:
S: * 101 FETCH (MODSEQ (303011130956) FLAGS ($Processed \Deleted
\Answered))See also Section 3.1.11 for additional quality-of-implementation issues 🌸
3.2. QRESYNC Extension 🌈
all the protocol changes and requirements from the CONDSTORE extension also apply to QRESYNC 🫶
QRESYNC puts extra requirements on a server that implements CONDSTORE. Each mailbox that supports persistent storage of mod-sequences (the ones where the server would send a HIGHESTMODSEQ untagged OK response code on a successful SELECT/EXAMINE) MUST increment the per-mailbox mod-sequence when one or more messages are expunged due to EXPUNGE, UID EXPUNGE, CLOSE, or MOVE [RFC6851]. The server MUST associate the incremented mod-sequence with the UIDs of the expunged messages 👑 Also, if the server supports the IMAP METADATA extension [RFC5464], it MUST increment the per-mailbox mod-sequence when SETMETADATA successfully changes an annotation on the corresponding mailbox ✨
A server implementing QRESYNC MUST send untagged events to a client so the client doesn't lose any changes if the connection drops 💖 Specifically, if the server sends MODSEQ FETCH data items while EXPUNGE (or VANISHED) replies with lower mod-sequences are still being delayed, the server MUST send the HIGHESTMODSEQ response code with a lower value than the EXPUNGE's mod-sequence. See the example in Section 6 🔥
3.2.1. Impact on CONDSTORE-only Clients 🍓
A client that supports CONDSTORE but not QRESYNC might resync a mailbox and find that its HIGHESTMODSEQ went up from the value it cached. If the increase is only because messages were expunged since the client last synced, the client is likely to send a FETCH ... CHANGEDSINCE command that returns no data. So a CONDSTORE-only client might take an unneeded round trip when resyncing some mailboxes (the ones that had messages expunged but no flag changes since the last sync) 🙈
This extra round trip only happens for clients that support CONDSTORE but not QRESYNC, and only when a mailbox had messages expunged but no flag changes to the messages that are still there. Since CONDSTORE is a relatively new extension, it is strongly encouraged that clients supporting it also support QRESYNC 💜
3.2.2. Advertising Support for QRESYNC 📣
The quick resync IMAP extension is present if an IMAP4 server returns "QRESYNC" as one of the supported capabilities in response to the CAPABILITY command 🦋
For compatibility with clients that only support the CONDSTORE IMAP extension, servers SHOULD also advertise "CONDSTORE" in the CAPABILITY response 🫶
3.2.3. Use of ENABLE 💅
Servers supporting QRESYNC MUST implement and advertise support for the ENABLE [RFC5161] IMAP extension. Also, having the "QRESYNC" capability implies support for the CONDSTORE IMAP extension, even if "CONDSTORE" isn't advertised. A server compliant with this specification is REQUIRED to support "ENABLE QRESYNC" and "ENABLE QRESYNC CONDSTORE" (these are "CONDSTORE enabling commands", see Section 3.1, and they have identical results) 🎀 Note that the order of parameters doesn't matter, but there's no requirement for a compliant server to support "ENABLE CONDSTORE" by itself 🙃 The "ENABLE QRESYNC"/"ENABLE QRESYNC CONDSTORE" command also tells the server that it MUST start sending VANISHED responses (see Section 3.2.10) instead of EXPUNGE responses for all mailboxes where the server doesn't return the NOMODSEQ response code. This change stays in effect until the connection closes 🔥
A client using QRESYNC MUST issue "ENABLE QRESYNC" once it's authenticated. A server MUST respond with a tagged BAD response if the QRESYNC parameter to the SELECT/EXAMINE command or the VANISHED UID FETCH modifier is specified and the client hasn't issued "ENABLE QRESYNC", or if the server hasn't positively responded (in the current connection) to that command with the untagged ENABLED response containing QRESYNC 💀
3.2.4. Additional Requirements on QRESYNC Servers 👑
Once the client issues a CONDSTORE enabling command, the server MUST automatically include both UID and mod-sequence data in all subsequent untagged FETCH responses (until the connection closes). That covers responses caused by a regular STORE/UID STORE, a STORE/UID STORE with an UNCHANGEDSINCE modifier, a FETCH/UID FETCH that implicitly sets the \Seen flag, or an external agent ✨ Note that this rule doesn't affect untagged FETCH responses caused by a FETCH command that doesn't include UID and/or a MODSEQ FETCH data item (and doesn't implicitly set the \Seen flag), or by a UID FETCH without the MODSEQ FETCH data item 🌸
3.2.5. QRESYNC Parameter to SELECT/EXAMINE 🧚
The Quick Resynchronization parameter to SELECT/EXAMINE commands has four arguments:
- the last known UIDVALIDITY,
- the last known modification sequence,
- the optional set of known UIDs, and
- an optional parenthesized list of known sequence ranges and their corresponding UIDs.
A server MUST respond with a tagged BAD response if the Quick Resynchronization parameter to the SELECT/EXAMINE command is specified and the client hasn't issued "ENABLE QRESYNC" in the current connection, or if the server hasn't positively responded to that command with the untagged ENABLED response containing QRESYNC 🙈
Before opening the mailbox, the server checks all arguments for syntactic validity. If any parameter isn't syntactically valid, the server returns the tagged BAD response, and the mailbox stays unselected 😭 Once that check is done, the server opens the mailbox as if no SELECT/EXAMINE parameters were specified (this is subject to the processing of other parameters as defined in other extensions). Specifically, the server MUST send all untagged responses as specified in Sections 6.3.1 and 6.3.2 of [RFC3501] 💅
After that, the server checks the UIDVALIDITY value the client provided. If the provided UIDVALIDITY doesn't match the UIDVALIDITY of the mailbox being opened, the server MUST ignore the remaining parameters and behave as if no dynamic message data changed. The client can spot this by comparing the UIDVALIDITY value the server returns. This behavior lets the client skip synchronizing the mailbox or pick the best sync strategy without extra fuss 🦋
Example: Attempting to resynchronize INBOX, but the provided UIDVALIDITY parameter doesn't match the current UIDVALIDITY value.
C: A02 SELECT INBOX (QRESYNC (67890007 20050715194045000
41,43:211,214:541))
S: * 464 EXISTS
S: * 3 RECENT
S: * OK [UIDVALIDITY 3857529045] UIDVALIDITY
S: * OK [UIDNEXT 550] Predicted next UID
S: * OK [HIGHESTMODSEQ 90060128194045007] Highest mailbox
mod-sequence
S: * OK [UNSEEN 12] Message 12 is first unseen
S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
S: * OK [PERMANENTFLAGS (\Answered \Flagged \Draft
\Deleted \Seen \*)] Permanent flags
S: A02 OK [READ-WRITE] Sorry, UIDVALIDITY mismatchThe remaining parameters are described in the following subsections 🎀
3.2.5.1. Modification Sequence and UID Parameters
Okay bestie, so if a server doesn't keep mod-sequences stored persistently for the mailbox, it MUST send an OK untagged response including the NOMODSEQ response code on every successful SELECT or EXAMINE command (see Section 3.1.2.2) 💖. Such a server doesn't need to remember mod-sequences for expunged messages in the mailbox. It MUST ignore the remaining parameters and behave as if no dynamic message data changed 😭.
If the provided UIDVALIDITY matches the selected mailbox's, the server then checks the last known modification sequence ✨.
The server sends the client any pending flag changes (using FETCH responses that MUST contain UIDs) and expunges that have happened in this mailbox since the provided modification sequence 🦋.
If the client also provided a list of known UIDs, the server should only report flag changes and expunges for those specified messages. If the client didn't provide the list of UIDs, the server acts as if the client has specified 1:<maxuid>, where <maxuid> is the mailbox's UIDNEXT value minus 1. If the mailbox is empty and never had any messages in it, then the missing list of UIDs is read as an empty set of UIDs 🌸.
So the client can just process these pending events and skip a full resync, no cap 🙌. Without the message sequence number matching info, the result of this step is semantically equivalent to the client issuing:
tag1 UID FETCH "known-uids" (FLAGS) (CHANGEDSINCE "mod-sequence-value" VANISHED)In particular, this means all requirements specified in Section 3.2.6 apply 👑.
Example:
C: A03 SELECT INBOX (QRESYNC (67890007
90060115194045000 41:211,214:541))
S: * OK [CLOSED]
S: * 100 EXISTS
S: * 11 RECENT
S: * OK [UIDVALIDITY 67890007] UIDVALIDITY
S: * OK [UIDNEXT 600] Predicted next UID
S: * OK [HIGHESTMODSEQ 90060115205545359] Highest
mailbox mod-sequence
S: * OK [UNSEEN 7] There are some unseen
messages in the mailbox
S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
S: * OK [PERMANENTFLAGS (\Answered \Flagged \Draft
\Deleted \Seen \*)] Permanent flags
S: * VANISHED (EARLIER) 41,43:116,118,120:211,214:540
S: * 49 FETCH (UID 117 FLAGS (\Seen \Answered) MODSEQ
(90060115194045001))
S: * 50 FETCH (UID 119 FLAGS (\Draft $MDNSent) MODSEQ
(90060115194045308))
S: * 51 FETCH (UID 541 FLAGS (\Seen $Forwarded) MODSEQ
(90060115194045001))
S: A03 OK [READ-WRITE] mailbox selectedIn the example above, flag info for UID 42 isn't returned, presumably because its flags haven't changed since MODSEQ 90060115194045000 🥹.
3.2.5.2. Message Sequence Match Data
A client MAY provide a parenthesized list of a message sequence set and the matching UID sets. Both MUST be provided in ascending order. The server uses this data to narrow the range it gives expunged message info for 🎀.
Conceptually, the client gives a small sample of sequence numbers it knows the UIDs for. The server then compares each sequence number and UID pair with the current state of the mailbox. If a pair matches, the client knows about any expunges up to and including that message, so that range won't be included in the VANISHED response, even if the "mod-sequence-value" the client gave is too old for the server to have data on when those messages got expunged 🧚.
So, if the Nth message number in the first set is 4, and the Nth UID in the second set is 8, and the mailbox's fourth message has UID 8, then no UIDs equal to or less than 8 show up in the VANISHED response. If the (N+1)th message number is 12, and the (N+1)th UID is 24, and the (N+1)th message in the mailbox has UID 25, then the lowest UID included in the VANISHED response would be 9 💅.
In the next two examples, the server can't remember expunges at all, and only UIDs for messages divisible by three are in the mailbox. In the first example the client doesn't use the fourth parameter; in the second, it does. It's a pretty extreme case, but it shows that using sequence match data wisely can save a ton of bandwidth 🍓.
Example:
C: A04 SELECT INBOX (QRESYNC (67890007
90060115194045000 1:29997))
S: * 10003 EXISTS
S: * 4 RECENT
S: * OK [UIDVALIDITY 67890007] UIDVALIDITY
S: * OK [UIDNEXT 30013] Predicted next UID
S: * OK [HIGHESTMODSEQ 90060115205545359] Highest mailbox
mod-sequence
S: * OK [UNSEEN 7] There are some unseen messages in the mailbox
S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
S: * OK [PERMANENTFLAGS (\Answered \Flagged \Draft
\Deleted \Seen \*)] Permanent flags
S: * VANISHED (EARLIER) 1:2,4:5,7:8,10:11,13:14,[...],
29668:29669,29671:29996
S: * 1 FETCH (UID 3 FLAGS (\Seen \Answered $Important) MODSEQ
(90060115194045001))
S: ...
S: * 9889 FETCH (UID 29667 FLAGS (\Seen \Answered) MODSEQ
(90060115194045027))
S: * 9890 FETCH (UID 29670 FLAGS (\Draft $MDNSent) MODSEQ
(90060115194045028))
S: ...
S: * 9999 FETCH (UID 29997 FLAGS (\Seen $Forwarded) MODSEQ
(90060115194045031))
S: A04 OK [READ-WRITE] mailbox selectedExample:
C: B04 SELECT INBOX (QRESYNC (67890007
90060115194045000 1:29997 (5000,7500,9000,9990:9999 15000,
22500,27000,29970,29973,29976,29979,29982,29985,29988,29991,
29994,29997)))
S: * 10003 EXISTS
S: * 4 RECENT
S: * OK [UIDVALIDITY 67890007] UIDVALIDITY
S: * OK [UIDNEXT 30013] Predicted next UID
S: * OK [HIGHESTMODSEQ 90060115205545359] Highest mailbox mod-
sequence
S: * OK [UNSEEN 7] There are some unseen messages in the mailbox
S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen)
S: * OK [PERMANENTFLAGS (\Answered \Flagged \Draft
\Deleted \Seen \*)] Permanent flags
S: * 1 FETCH (UID 3 FLAGS (\Seen \Answered $Important) MODSEQ
(90060115194045001))
S: ...
S: * 9889 FETCH (UID 29667 FLAGS (\Seen \Answered) MODSEQ
(90060115194045027))
S: * 9890 FETCH (UID 29670 FLAGS (\Draft $MDNSent) MODSEQ
(90060115194045028))
S: ...
S: * 9999 FETCH (UID 29997 FLAGS (\Seen $Forwarded) MODSEQ
(90060115194045031))
S: B04 OK [READ-WRITE] mailbox selected3.2.6. VANISHED UID FETCH Modifier
[RFC4466] extended the syntax of the FETCH and UID FETCH commands to include an optional FETCH modifier. This document defines a new UID FETCH modifier: VANISHED ✨.
Heads up, the VANISHED UID FETCH modifier is NOT allowed with a FETCH command 🙈. The server MUST return a tagged BAD response if this response is specified as a modifier to the FETCH command.
A server MUST respond with a tagged BAD response if the VANISHED UID FETCH modifier is specified and the client hasn't issued "ENABLE QRESYNC" in the current connection 💀.
The VANISHED UID FETCH modifier MUST only be specified together with the CHANGEDSINCE UID FETCH modifier. If the VANISHED UID FETCH modifier is used without the CHANGEDSINCE UID FETCH modifier, the server MUST respond with a tagged BAD response 🔥.
The VANISHED UID FETCH modifier tells the server to report those messages from the UID set parameter that have been expunged and whose associated mod-sequence is larger than the specified mod-sequence. In other words, the client asks to be told about messages from the given set that were expunged since the specified mod-sequence. The mod-sequence(s) tied to these messages got updated when the messages were expunged (as described above) 🦋. The expunged messages are reported using the VANISHED (EARLIER) response as described in Section 3.2.10.1. Any VANISHED (EARLIER) responses MUST be returned before any FETCH responses, otherwise the client might get confused about how message numbers map to UIDs 😭.
Note: A server that receives a mod-sequence smaller than <minmodseq>, where <minmodseq> is the value of the smallest expunged mod-sequence it remembers minus one, MUST behave as if it was requested to report all expunged messages from the provided UID set parameter 🧠.
Example 1: Without the VANISHED UID FETCH modifier, a CONDSTORE-aware client needs to issue separate commands to learn about flag changes and expunged messages since the last sync:
C: s100 UID FETCH 300:500 (FLAGS) (CHANGEDSINCE 12345)
S: * 1 FETCH (UID 404 MODSEQ (65402) FLAGS (\Seen))
S: * 2 FETCH (UID 406 MODSEQ (75403) FLAGS (\Deleted))
S: * 4 FETCH (UID 408 MODSEQ (29738) FLAGS ($NoJunk
$AutoJunk $MDNSent))
S: s100 OK FETCH completed
C: s101 UID SEARCH 300:500
S: * SEARCH 404 406 407 408 410 412
S: s101 OK search completedWhere 300 and 500 are the lowest and highest UIDs in the client's cache. The second SEARCH response tells the client that messages with UIDs 407, 410, and 412 are still present, but their flags haven't changed since the specified modification sequence 🌈.
With the VANISHED UID FETCH modifier, you only need one command 🎀:
C: s100 UID FETCH 300:500 (FLAGS) (CHANGEDSINCE 12345
VANISHED)
S: * VANISHED (EARLIER) 300:310,405,411
S: * 1 FETCH (UID 404 MODSEQ (65402) FLAGS (\Seen))
S: * 2 FETCH (UID 406 MODSEQ (75403) FLAGS (\Deleted))
S: * 4 FETCH (UID 408 MODSEQ (29738) FLAGS ($NoJunk
$AutoJunk $MDNSent))
S: s100 OK FETCH completed3.2.7. EXPUNGE Command 🧹✨
Arguments: none
Responses: untagged responses: EXPUNGE or VANISHED
Result: OK - expunge completed
NO - expunge failure: can't expunge (e.g., permission denied)
BAD - command unknown or arguments invalidokay bestie, EXPUNGE is the command that actually nukes stuff 💀 It permanently removes every message with the \Deleted flag set from the currently selected mailbox. Before the server says OK, it reports each removed message using a VANISHED response or EXPUNGE responses.
This section updates the definition of the EXPUNGE command described in Section 6.4.3 of [RFC3501].
Now the serious part 🫶 If the server can store modification sequences for the selected mailbox, it MUST increment the per-mailbox mod-sequence if at least one message was permanently removed by EXPUNGE. For each removed message, the server MUST remember the incremented mod-sequence and the matching UID. And if at least one message got expunged and QRESYNC was enabled, the server MUST send the updated per-mailbox modification sequence using the HIGHESTMODSEQ response code (see Section 3.1.2.1) in the tagged OK response.
Example: C: A202 EXPUNGE
S: * 3 EXPUNGE
S: * 3 EXPUNGE
S: * 5 EXPUNGE
S: * 8 EXPUNGE
S: A202 OK [HIGHESTMODSEQ 20010715194045319] expungedNote: in this example, the client hasn't enabled QRESYNC, so the server is still using untagged EXPUNGE responses 🌸 The HIGHESTMODSEQ response code is optional in this case. If the selected mailbox returned NOMODSEQ, the HIGHESTMODSEQ response code will be absent. Messages 3, 4, 7, and 11 had the \Deleted flag set. The first "* 3 EXPUNGE" reports message #3 as expunged. The second "* 3 EXPUNGE" reports message #4 as expunged, because the message number got decremented after the previous EXPUNGE response. See the description of the EXPUNGE response in [RFC3501] for the full story.
Once the client enables QRESYNC, the server will always send VANISHED responses instead of EXPUNGE responses for mailboxes that support storing modification sequences, so the previous example might look like this 🦋
Example: C: B202 EXPUNGE
S: * VANISHED 405,407,410,425
S: B202 OK [HIGHESTMODSEQ 20010715194045319] expungedHere, messages with message numbers 3, 4, 7, and 11 have respective UIDs 405, 407, 410, and 425. ✨
3.2.8. CLOSE Command 🚪
Arguments: none
Responses: no specific responses for this command
Result: OK - close completed, now in authenticated state
BAD - command unknown or arguments invalidThis section updates the definition of the CLOSE command described in Section 6.4.2 of [RFC3501].
CLOSE permanently removes all messages with the \Deleted flag set from the currently selected mailbox, then goes from the selected state back to the authenticated state 💅 No untagged EXPUNGE (or VANISHED) responses are sent.
If the server can store modification sequences for the selected mailbox, it MUST increment the per-mailbox mod-sequence if at least one message was permanently removed by CLOSE. For each removed message, the server MUST remember the incremented mod-sequence and the matching UID. The server MUST NOT send the updated per-mailbox modification sequence using the HIGHESTMODSEQ response code (see Section 3.1.2.1) in the tagged OK response, because that might cause loss of synchronization on the client 🙈
Example: C: A202 CLOSE
S: A202 OK done3.2.9. UID EXPUNGE Command 🎀
Arguments: message set
Responses: untagged responses: EXPUNGE or VANISHED
Result: OK - expunge completed
NO - expunge failure: can't expunge (e.g., permission denied)
BAD - command unknown or arguments invalidThis section updates the definition of the UID EXPUNGE command described in Section 2.1 of [UIDPLUS], in the presence of QRESYNC. If a server implements both [UIDPLUS] and QRESYNC extensions, it must implement UID EXPUNGE as described in this section 👑
UID EXPUNGE permanently removes from the currently selected mailbox all messages that have BOTH the \Deleted flag set AND a UID included in the specified message set. A message that doesn't have the \Deleted flag set, or whose UID isn't in the specified message set, is not affected.
This command is especially useful for disconnected mode clients 🥹 By using UID EXPUNGE instead of EXPUNGE when resynchronizing with the server, the client can avoid accidentally removing messages that other clients marked as \Deleted between the time it was last connected and the time it resynchronizes.
Before returning an OK to the client, those messages that are removed are reported using a VANISHED response or EXPUNGE responses.
If the server can store modification sequences for the selected mailbox, it MUST increment the per-mailbox mod-sequence if at least one message was permanently removed by UID EXPUNGE. For each removed message, the server MUST remember the incremented mod-sequence and the matching UID. If at least one message got expunged and QRESYNC was enabled, the server MUST send the updated per-mailbox modification sequence using the HIGHESTMODSEQ response code (see Section 3.1.2.1) in the tagged OK response.
Example: C: . UID EXPUNGE 3000:3002
S: * 3 EXPUNGE
S: * 3 EXPUNGE
S: * 3 EXPUNGE
S: . OK [HIGHESTMODSEQ 20010715194045319] OkNote: in this example, the client hasn't enabled QRESYNC, so the server is still using untagged EXPUNGE responses instead of VANISHED responses 🍓 The HIGHESTMODSEQ response code is optional. If the selected mailbox returned NOMODSEQ, the HIGHESTMODSEQ response code will be absent. In this example, at least messages 3, 4, and 5 (UIDs 3000 to 3002) had the \Deleted flag set. The first "* 3 EXPUNGE" reports message #3 as expunged. The second "* 3 EXPUNGE" reports message #4 as expunged, because the message number got decremented after the previous EXPUNGE response. See the description of the EXPUNGE response in [RFC3501] for further explanation.
3.2.10. VANISHED Response 🦋
The VANISHED response reports that the specified UIDs have been permanently removed from the mailbox. It's similar to the EXPUNGE response [RFC3501], but it can cover multiple messages and it returns UIDs instead of message numbers. The first benefit saves bandwidth, and the second is more convenient for clients that only use UIDs to access the IMAP server ✨
The VANISHED response has the same restrictions on when it can be sent as the EXPUNGE response (see below). Once a client has issued "ENABLE QRESYNC" and the server has positively responded with the untagged ENABLED response containing QRESYNC, the server MUST use the VANISHED response without the EARLIER tag instead of the EXPUNGE response for all mailboxes that don't return NOMODSEQ when selected. The server keeps using VANISHED in place of EXPUNGE for the rest of the connection. This affects the EXPUNGE [RFC3501] and UID EXPUNGE [UIDPLUS] commands, plus messages expunged in other connections. Such a VANISHED response MUST NOT contain the EARLIER tag 🙅♀️
The VANISHED response comes in two forms. The first contains the EARLIER tag, which signals that the response was caused by a UID FETCH (VANISHED) or a SELECT/EXAMINE (QRESYNC) command. The second doesn't contain the EARLIER tag and is used to announce message removals within an already selected mailbox.
Clients handle the two forms differently, so servers MUST NOT combine them. Messages are reported in VANISHED responses with or without the EARLIER tag, as appropriate to the cause. If necessary, two VANISHED responses are sent, one with EARLIER and one without. 🌈
3.2.10.1. VANISHED (EARLIER) Response 🧚
Contents: an EARLIER tag
list of UIDsThe VANISHED (EARLIER) response is caused by a UID FETCH (VANISHED) or a SELECT/EXAMINE (QRESYNC) command. It gets sent if the UID set parameter to the UID FETCH (VANISHED) command includes UIDs of messages that are no longer in the mailbox. When the client sees a VANISHED EARLIER response, it MUST NOT decrement message sequence numbers for each successive message in the mailbox. Bestie, don't skip this one 😭💜
3.2.10.2. VANISHED Response without the (EARLIER) Tag ✨
Contents: list of UIDs 💖
Okay bestie, once a client has sent "ENABLE QRESYNC" and the server has said yes with the untagged ENABLED response containing QRESYNC, the server MUST use the VANISHED response without the EARLIER tag instead of the EXPUNGE response. This applies to every mailbox that doesn't return NOMODSEQ when it's selected. 🫶 The server keeps doing this for the whole connection, so it affects the EXPUNGE [RFC3501] and UID EXPUNGE [UIDPLUS] commands, plus messages that get expunged in other connections. Such a VANISHED response MUST NOT contain the EARLIER tag. 🔥
Unlike VANISHED (EARLIER), this one also goes down the message count and shifts the message sequence numbers of the messages left in the mailbox, so the expunged messages get accounted for. 💅 Because of all that housekeeping, the server doesn't need to send an EXISTS response to report the new message count. Peep the example at the end of this section.
A VANISHED response without the EARLIER tag MUST refer only to messages that are visible to the client in the current session at the time the VANISHED response is sent. 👑 That means servers MUST NOT send UIDs for previously expunged messages, or for messages that were never announced to the client via EXISTS. So each UID listed in a VANISHED response makes the client decrement the message count by one. This is required to prevent a possible race condition where new arrivals whose UID the client doesn't know yet get immediately expunged. 😭
A VANISHED response MUST NOT be sent when no command is in progress, and it MUST NOT be sent while responding to a FETCH, STORE, or SEARCH command. 🙈 This rule is necessary to prevent a loss of synchronization of message sequence numbers between the client and server. A command is not "in progress" until the complete command has been received. In particular, a command is not "in progress" during the negotiation of command continuation.
Note: UID FETCH, UID STORE, and UID SEARCH are different commands from FETCH, STORE, and SEARCH. A VANISHED response MAY be sent during a UID command. However, the VANISHED response MUST NOT be sent during a UID SEARCH command that contains message numbers in the search criteria. 🦋
The update from the VANISHED response MUST be recorded by the client. 📝
Example time! 🧚 Let's say the currently selected mailbox maps message numbers to UIDs like this. "D" marks messages with the \Deleted flag set, and "x" stands for UIDs that don't matter for the example:
Message numbers: 1 2 3 4 5 6 7 8 9 10 11
UIDs: x 504 505 507 508 x 510 x x x 625
\Deleted messages: D D D DWith the extension from this document in play, it goes like this:
C: A202 EXPUNGE
S: * VANISHED 505,507,510,625
S: A202 OK EXPUNGE completedWithout the QRESYNC extension, the same example might look like this:
C: A202 EXPUNGE
S: * 3 EXPUNGE
S: * 3 EXPUNGE
S: * 5 EXPUNGE
S: * 8 EXPUNGE
S: A202 OK EXPUNGE completed(Continuing from the previous example.) If messages with UIDs 504 and 508 get marked as \Deleted later on:
C: A210 EXPUNGE
S: * VANISHED 504,508
S: A210 OK EXPUNGE completedFor Example, the last VANISHED response only contains UIDs of messages expunged since the previous VANISHED response. 🍓
Now the difference between VANISHED and VANISHED (EARLIER) is kinda important. 🤓 Say the mailbox contains UIDs 2 and 4. Any of the following responses would be a broken server implementation:
S: * VANISHED 1
S: * VANISHED 3
S: * VANISHED 5However, any of these UIDs can easily be referenced by the VANISHED (EARLIER) response. ✨
3.2.11. CLOSED Response Code
The CLOSED response code has no parameters. 🎀 A server implementing the extension defined in this document MUST return the CLOSED response code when the currently selected mailbox is closed implicitly by a SELECT/EXAMINE command on another mailbox. The CLOSED response code is like a boundary line: all responses before it relate to the mailbox that got closed, and all the responses after it relate to the newly opened mailbox. 🌈
A server that advertises "QRESYNC" or "CONDSTORE" in the capability string must return the CLOSED response code in this case, whether or not a CONDSTORE enabling command was issued. 💜
There is no need to return the CLOSED response code on completion of the CLOSE or the UNSELECT [UNSELECT] command (or similar). Those commands just close the currently selected mailbox without opening a new one, so no boundary is needed. 💅
4. Long Command Lines (Update to RFC 2683) 🌸
While [RFC3501] doesn't specify a line-length limit, several server implementations chose to follow the recommended line-length limit from Section 3.2.1.5 of [RFC2683] to protect against Denial-of-Service attacks. 🛡️ When the limit gets exceeded, those servers return a BAD response (as [RFC3501] requires for a syntactic error), and they might even close the connection. 😬 Clients that support the CONDSTORE/QRESYNC extensions can hit this limit by sending a long UID sequence (one the server returned earlier) in an extended SELECT or FETCH command.
This document updates the recommended line-length limits in Section 3.2.1.5 of [RFC2683]. ✨ The advice in the first paragraph of that section still applies (use compact message/UID set representations). But the 1000-octet limit suggested in the second paragraph turns out to be quite problematic when the CONDSTORE and/or QRESYNC extension is used. 🙃
The updated recommendation goes like this: a client should limit the length of the command lines it generates to approximately 8192 octets (including all quoted strings, but not including literals). 🍰 If the client can't group things into ranges so that the command line fits within that length, it should split the request into multiple commands. The client should also use literals instead of long quoted strings to keep the command length down.
5. QRESYNC Server Implementation Considerations 🧚♀️
This section describes a minimalist implementation, a moderate implementation, and an example of a full implementation. 🦋
5.1. Server Implementations That Don't Store Extra State
Strictly speaking, a server implementation that doesn't remember the mod-sequences associated with expunged messages can still be considered compliant with this specification. 🙌 Such implementations return all expunged messages specified in the UID set of the UID FETCH (VANISHED) command every time, without paying attention to the specified CHANGEDSINCE mod-sequence. Such implementations are discouraged, though, because they can end up returning VANISHED responses that are bigger than the result of a UID SEARCH command for the same UID set. 😵
A client can seriously shrink the size of VANISHED responses by giving the server message sequence match data (see Section 3.2.5.2). 💖 This works especially well in the typical case where no messages have been expunged, or where all the expunges happened toward the end of the mailbox.
5.2. Server Implementations Storing Minimal State
A server that stores the HIGHESTMODSEQ value at the time of the last EXPUNGE can omit the VANISHED response when a client provides a MODSEQ value that is equal to or higher than that HIGHESTMODSEQ value. 🎯 The reason is that no messages were expunged during the time period the client cares about.
A client providing message sequence match data can reduce the scope as above. 🌷 In the case where there have been no expunges, the server can ignore this data.
5.3. Additional State Required on the Server
Compared to the CONDSTORE extension, QRESYNC requires servers to store additional state associated with expunged messages. 📦 Implementations are not required to store this state in persistent storage. However, using persistent storage is advisable. 💾
One possible way to correctly implement QRESYNC is to store a queue of <UID set, mod-sequence> pairs. The <UID set> can be represented as a sequence of <min UID, max UID> pairs. 🧠
When messages get expunged, one or more entries are added to the queue tail. 🗑️
When the server gets a request to return messages expunged since a given mod-sequence, it searches the queue from the tail (going from the highest expunged mod-sequence to the lowest) until it sees the first record with a mod-sequence less than or equal to the given mod-sequence, or until it reaches the head of the queue. 🔍
Storing info about expunged messages forever can cause storage and related problems for an implementation. 😱 In the worst case, this could mean almost 64 GB of storage for each IMAP mailbox. For example, think about an implementation that stores <min UID, max UID, mod-sequence> triples for each range of messages expunged at the same time. Each triple needs 16 octets: 4 octets for each of the two UIDs, and 8 octets for the mod-sequence. Now assume there's a mailbox containing a single message with a UID of 2**32-1 (the maximum possible UID value), where messages previously existed with UIDs starting at 1 and have been expunged one at a time. For this mailbox alone, storage is required for the triples:
<1, 1, modseq1>, <2, 2, modseq2>, ..., <2**32-2, 2**32-2, modseq4294967294>Hence, implementations are encouraged to adopt strategies that protect against such storage problems. 🛟 Examples include limiting the size of the queue used to store mod-sequences for expunged messages, and "expiring" older records when that limit is reached. When the chosen implementation-specific queue limit is reached, the oldest record(s) are deleted from the queue (note that such records sit at the queue head). For all such "expired" records, the server needs to store a single mod-sequence, which is the highest mod-sequence of all the "expired" expunged messages. 🥹
If the client provides the message sequence match data, this can heavily reduce the data cost of sending a complete set of missing UIDs. So it cuts down the problems for clients when a server can't persist much of this queue. ✨ If the queue contains data back to the requested mod-sequence, that data can be ignored.
Also, if the UIDVALIDITY of the mailbox changes, or if the mailbox is deleted, then any state associated with expunged messages doesn't need to be preserved and SHOULD be deleted. 🧹
6. Updated Synchronization Sequence ✨
okay bestie, this section gives the optimized sync stuff from Section 6.1 of [IMAP-DISC] a glow-up for when QRESYNC is in the picture 💅🦋
An advanced disconnected mail client SHOULD use the QRESYNC extension when the server supports it, and SHOULD use CONDSTORE if the server supports that but QRESYNC isn't there 🎀. The client looks at the HIGHESTMODSEQ OK response code it gets when it opens the mailbox to figure out if it needs to resync 🧚. Once the sync is finished, it MUST cache that value (unless the mailbox UIDVALIDITY value has changed, more on that below) 💜. And it MUST update its copy of HIGHESTMODSEQ every time the server sends a later HIGHESTMODSEQ OK response code 🌸.
After a full sync, the client MUST also keep an eye on any unsolicited MODSEQ FETCH data items and HIGHESTMODSEQ response codes the server sends 👀. Whenever the client gets a tagged response to a command, it checks those unsolicited responses to work out the new HIGHESTMODSEQ value 🫶. If a HIGHESTMODSEQ response code shows up, the client MUST use it, even if it has already seen higher mod-sequences 👑. Otherwise, it works out the highest value among all the MODSEQ FETCH data items it received since the last tagged response. If that value is bigger than the client's copy of HIGHESTMODSEQ, the client MUST use it as the new HIGHESTMODSEQ value 🔥.
Here's an example, and it's giving drama 🍓:
C: A150 STORE 1:2 (UNCHANGEDSINCE 96) +FLAGS.SILENT \Seen
S: * 1 FETCH (UID 6 MODSEQ (103))
S: * 2 FETCH (UID 7 MODSEQ (101))
S: * OK [HIGHESTMODSEQ 99] VANISHED reply with MODSEQ 100 is delayed
S: A150 OK [MODIFIED 3] done
C: A151 STORE 3 +FLAGS.SILENT \Seen
S: * 3 FETCH (UID 8 MODSEQ (104))
S: A151 OK [HIGHESTMODSEQ 99] Still delaying VANISHED
C: A152 NOOP
S: * VANISHED 8
S: A153 OK [HIGHESTMODSEQ 104] doneNote: it is NOT safe to update the client's copy of HIGHESTMODSEQ with a MODSEQ FETCH data item value the second it arrives 🙈. Servers are not required to send MODSEQ FETCH data items in increasing mod-sequence order, and some commands can also delay EXPUNGE (or VANISHED) replies that have smaller mod-sequences. If that happens, the client could miss some changes when the connection drops 💀.
Now, when the client opens the mailbox for sync, it uses the QRESYNC parameter on the SELECT/EXAMINE command 🧚✨. That parameter is followed by the UIDVALIDITY and mailbox HIGHESTMODSEQ values the client already knows about. It can optionally be followed by a set of UIDs, which is handy if the client only wants a partial sync of the mailbox 🎀. The client may also send a list with what it knows about message numbers.
If the SELECT/EXAMINE command works, the client compares UIDVALIDITY the way step d-1 in Section 3 of the [IMAP-DISC] describes 🦋. If the cached UIDVALIDITY matches what the server sent back AND the server also returns the HIGHESTMODSEQ response code, the server reports expunged messages and sends back flag changes for every message in the UID set parameter (or for every message in the mailbox, if the client left the UID set parameter out) 💅. At that point the client is synced, except maybe for the new messages 🌈.
If, after a successful SELECT/EXAMINE (QRESYNC) command, the client gets a NOMODSEQ OK untagged response instead of the HIGHESTMODSEQ response code, the client MUST remove its last known HIGHESTMODSEQ value from the cache 🥲. Then it follows the more general instructions in Section 3 of the [IMAP-DISC] 🫶.
At this point the client is in sync with the server about the old messages 🎉. It can now fetch info about the new messages, if the user asked for that ✨.
Step d ("Server-to-client synchronization") in Section 6.1 of [IMAP-DISC] gets amended for when QRESYNC and CONDSTORE are both in play, and here's how it goes 🍓:
- d) "Server-to-client synchronization" -- for each mailbox that needs synchronization, do the following:
- 1a) Check the mailbox UIDVALIDITY (see Section 4.1 of [IMAP-DISC] for more details) after issuing the SELECT/EXAMINE (QRESYNC) command. If the UIDVALIDITY value the server returns is different, the client MUST:
- empty the local cache of that mailbox;
- "forget" the cached HIGHESTMODSEQ value for the mailbox; and
- remove any pending "actions" that refer to UIDs in that mailbox. Note, this doesn't affect actions performed on client-generated fake UIDs (see Section 5 of the [IMAP-DISC]).
- 1b) This step is no longer required. 🙃
- 2) Fetch the current "descriptors".
- I) Discover new messages.
- 3) Fetch the bodies of any "interesting" messages that the client doesn't already have.
- 1a) Check the mailbox UIDVALIDITY (see Section 4.1 of [IMAP-DISC] for more details) after issuing the SELECT/EXAMINE (QRESYNC) command. If the UIDVALIDITY value the server returns is different, the client MUST:
And here's an example where the UIDVALIDITY is the same, but HIGHESTMODSEQ changed on the server while the client was offline 🧚💜:
C: A142 SELECT INBOX (QRESYNC (3857529045 20010715194032001 1:198))
S: * 172 EXISTS
S: * 1 RECENT
S: * OK [UNSEEN 12] Message 12 is first unseen
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: * OK [UIDNEXT 201] Predicted next UID
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
S: * OK [HIGHESTMODSEQ 20010715194045007] Highest
mailbox mod-sequence
S: * VANISHED (EARLIER) 1:5,7:8,10:15
S: * 2 FETCH (UID 6 MODSEQ (20010715205008000)
FLAGS (\Deleted))
S: * 5 FETCH (UID 9 MODSEQ (20010715195517000)
FLAGS ($NoJunk $AutoJunk $MDNSent))
...
S: A142 OK [READ-WRITE] SELECT completed7. Formal Syntax 🎀
okay bestie, this is the grammar section and it's where the rules get locked in 🔒 Everything is written in ABNF (Augmented Backus-Naur Form) as defined in [RFC5234]. Any non-terminal that isn't defined here comes from [RFC5234], [RFC3501], or [RFC4466], so you'll need those open too 📚
Unless noted otherwise, alphabetic characters are case-insensitive. Upper and lower case in the token strings is just there to make it easier to read 👀 Implementations MUST accept these strings in a case-insensitive fashion 💅
capability =/ "CONDSTORE" / "QRESYNC"
status-att =/ "HIGHESTMODSEQ"
;; Extends non-terminal defined in [RFC3501].
status-att-val =/ "HIGHESTMODSEQ" SP mod-sequence-valzer
;; Extends non-terminal defined in [RFC4466].
;; Value 0 denotes that the mailbox doesn't
;; support persistent mod-sequences
;; as described in Section 3.1.2.2.
store-modifier =/ "UNCHANGEDSINCE" SP mod-sequence-valzer
;; Only a single "UNCHANGEDSINCE" may be
;; specified in a STORE operation.
fetch-modifier =/ chgsince-fetch-mod
;; Conforms to the generic "fetch-modifier"
;; syntax defined in [RFC4466].
chgsince-fetch-mod = "CHANGEDSINCE" SP mod-sequence-value
;; CHANGEDSINCE FETCH modifier conforms to
;; the fetch-modifier syntax.
fetch-att =/ fetch-mod-sequence
;; Modifies original IMAP4 fetch-att.
fetch-mod-sequence = "MODSEQ"
fetch-mod-resp = "MODSEQ" SP "(" permsg-modsequence ")"
msg-att-dynamic =/ fetch-mod-resp
search-key =/ search-modsequence
;; Modifies original IMAP4 search-key.
;;
;; This change applies to all commands
;; referencing this non-terminal -- in
;; particular, SEARCH, SORT, and THREAD.
search-modsequence = "MODSEQ" [search-modseq-ext] SP
mod-sequence-valzer
search-modseq-ext = SP entry-name SP entry-type-req
resp-text-code =/ "HIGHESTMODSEQ" SP mod-sequence-value /
"NOMODSEQ" /
"MODIFIED" SP sequence-set
entry-name = entry-flag-name
entry-flag-name = DQUOTE "/flags/" attr-flag DQUOTE
;; Each system or user-defined flag <flag>
;; is mapped to "/flags/<flag>".
;;
;; <entry-flag-name> follows the escape rules
;; used by "quoted" string as described in
;; Section 4.3 of [RFC3501]; e.g., for the
;; flag \Seen, the corresponding <entry-name>
;; is "/flags/\\seen", and for the flag
;; $MDNSent, the corresponding <entry-name>
;; is "/flags/$mdnsent".
entry-type-resp = "priv" / "shared"
;; Metadata item type.
entry-type-req = entry-type-resp / "all"
;; Perform SEARCH operation on a private
;; metadata item, shared metadata item,
;; or both.
permsg-modsequence = mod-sequence-value
;; Per-message mod-sequence.
mod-sequence-value = 1*DIGIT
;; Positive unsigned 63-bit integer
;; (mod-sequence)
;; (1 <= n <= 9,223,372,036,854,775,807).
mod-sequence-valzer = "0" / mod-sequence-value
search-sort-mod-seq = "(" "MODSEQ" SP mod-sequence-value ")"
select-param =/ condstore-param
;; Conforms to the generic "select-param"
;; non-terminal syntax defined in [RFC4466].
condstore-param = "CONDSTORE"
mailbox-data =/ "SEARCH" [1*(SP nz-number) SP
search-sort-mod-seq]
sort-data = "SORT" [1*(SP nz-number) SP
search-sort-mod-seq]
; Updates the SORT response from RFC 5256.
attr-flag = "\\Answered" / "\\Flagged" / "\\Deleted" /
"\\Seen" / "\\Draft" / attr-flag-keyword /
attr-flag-extension
;; Does not include "\\Recent".
attr-flag-extension = "\\" atom
;; Future expansion. Client implementations
;; MUST accept flag-extension flags. Server
;; implementations MUST NOT generate
;; flag-extension flags, except as defined by
;; future standards or Standards Track
;; revisions of [RFC3501].
attr-flag-keyword = atom
select-param =/ "QRESYNC" SP "(" uidvalidity SP
mod-sequence-value [SP known-uids]
[SP seq-match-data] ")"
;; Conforms to the generic select-param
;; syntax defined in [RFC4466].
seq-match-data = "(" known-sequence-set SP known-uid-set ")"
uidvalidity = nz-number
known-uids = sequence-set
;; Sequence of UIDs; "*" is not allowed.
known-sequence-set = sequence-set
;; Set of message numbers corresponding to
;; the UIDs in known-uid-set, in ascending order.
;; * is not allowed.
known-uid-set = sequence-set
;; Set of UIDs corresponding to the messages in
;; known-sequence-set, in ascending order.
;; * is not allowed.
message-data =/ expunged-resp
expunged-resp = "VANISHED" [SP "(EARLIER)"] SP known-uids
rexpunges-fetch-mod = "VANISHED"
;; VANISHED UID FETCH modifier conforms
;; to the fetch-modifier syntax
;; defined in [RFC4466]. It is only
;; allowed in the UID FETCH command.
resp-text-code =/ "CLOSED"Phew, that's the whole grammar 😭 Just remember the big rules here: client implementations MUST accept flag-extension flags, and server implementations MUST NOT generate flag-extension flags unless a future standard or a Standards Track revision of [RFC3501] defines them. That's the main thing to keep in your head 🧠✨
8. Security Considerations 🔐
ngl, this part is serious, so read it slowly 🙈 Thoroughly testing both clients and servers that implement QRESYNC is super important, because it changes how the server reports expunged messages to the client. Don't skip the testing, bestie 💀
It is believed that the CONDSTORE or the QRESYNC extensions don't raise any new security concerns beyond the ones already covered in [RFC3501]. Still, CONDSTORE might make it possible to use IMAP4 for critical applications that couldn't be used before, which means that correct server implementation and operation matter even more 🚨
9. IANA Considerations 🦋
IMAP4 capabilities get registered by publishing a Standards Track or IESG-approved Experimental RFC. The registry lives at http://www.iana.org/assignments/imap-capabilities 🗂️
This document defines the CONDSTORE and QRESYNC IMAP capabilities, and IANA has updated the references for both extensions so they point to this document 🎀
10. References 📚
10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2683] Leiba, B., "IMAP4 Implementation Recommendations", RFC 2683, September 1999.
[RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003.
[RFC4466] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4 ABNF", RFC 4466, April 2006.
[RFC5161] Gulbrandsen, A. and A. Melnikov, "The IMAP ENABLE Extension", RFC 5161, March 2008.
[RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008.
[RFC5256] Crispin, M. and K. Murchison, "Internet Message Access Protocol - SORT and THREAD Extensions", RFC 5256, June 2008.
[RFC5464] Daboo, C., "The IMAP METADATA Extension", RFC 5464, February 2009.
[UIDPLUS] Crispin, M., "Internet Message Access Protocol (IMAP) - UIDPLUS extension", RFC 4315, December 2005.
10.2. Informative References
[IMAP-DISC] Melnikov, A., Ed., "Synchronization Operations For Disconnected Imap4 Clients", RFC 4549, June 2006.
[NTP] Mills, D., Martin, J., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, June 2010.
[RFC2180] Gahrns, M., "IMAP4 Multi-Accessed Mailbox Practice", RFC 2180, July 1997.
[RFC4314] Melnikov, A., "IMAP4 Access Control List (ACL) Extension", RFC 4314, December 2005.
[RFC4731] Melnikov, A. and D. Cridland, "IMAP4 Extension to SEARCH Command for Controlling What Kind of Information Is Returned", RFC 4731, November 2006.
[RFC5257] Daboo, C. and R. Gellens, "Internet Message Access Protocol - ANNOTATE Extension", RFC 5257, June 2008.
[RFC5267] Cridland, D. and C. King, "Contexts for IMAP4", RFC 5267, July 2008.
[RFC6851] Gulbrandsen, A. and N. Freed, "Internet Message Access Protocol (IMAP) - MOVE Extension", RFC 6851, January 2013.
[UNSELECT] Melnikov, A., "Internet Message Access Protocol (IMAP) UNSELECT command", RFC 3691, February 2004.
Appendix A. Changes since RFC 4551 🌸
Mod-sequences are now unsigned 63-bit values instead of unsigned 64-bit values 🔢
The following errata, as posted on <http://www.rfc-editor.org>, got fixed ✅
- Errata ID 3401 ("several typos in UNCHANGEDSINCE spelling")
- Errata ID 3506 ("invalid ABNF for the MODIFIED response code")
- Errata ID 3509 ("correction to an example")
Returning the HIGHESTMODSEQ and NOMODSEQ response codes is only required once a CONDSTORE enabling command has been issued 💜
If several mod-sequences (for different metadata items) are linked to one message, then all of them that affect a particular STORE UNCHANGEDSINCE must be checked ✨
Also: references got updated and there were some editorial fixes 🧚
Appendix B. Changes since RFC 5162 🎀
okay bestie, this is the glow-up log 💅 The big one: mod-sequences are now unsigned 63-bit values instead of unsigned 64-bit values 🔥
The authors also squashed a bunch of errata that were posted on <http://www.rfc-editor.org> 🧚 Here's the list:
- Errata ID 1365 ("clarified that QRESYNC is only enabled when ENABLED QRESYNC is returned")
- Errata ID 1807 ("unsolicited FETCH responses must include UID fetch response item")
- Errata ID 1808 ("HIGHESTMODSEQ response code must not be returned for CLOSE")
- Errata ID 1809 ("clarify how updated mailbox mod-sequence is calculated")
- Errata ID 1810 ("server must send untagged events to client in a way that client doesn't lose any changes in case of connectivity loss")
- Errata ID 3322 ("VANISHED responses must not reference non-existing UIDs")
Clarified that ENABLE QRESYNC CONDSTORE and ENABLE CONDSTORE QRESYNC are equivalent 🦋
This one is a big deal 👑 The requirement to return VANISHED changed from SHOULD to MUST, following the mailing list discussion. The only exception is for mailboxes that return the NOMODSEQ response code when they are selected 🙈
Specified that IMAP SETMETADATA changes update per-mailbox HIGHESTMODSEQ 💜
Clarified that per-message annotations are also considered "metadata" ✨
Fixed some examples so they report data that match the requirements in the document 🍓
Clarified some text and made some requirements normative. Also, corrected a couple of SHOULDs to be MUSTs. Literally the most important thing here is that those requirements got stronger, so read them carefully 😭
Updated references, and made editorial corrections 🌈
Appendix C. Acknowledgements 🌸
Thank you to Steve Hole for co-editing RFC 4551 🫶
In this revision of the document, the authors also acknowledge the feedback provided by Timo Sirainen, Jan Kundrat, Pete Maclean, Barry Leiba, Eliot Lear, Chris Newman, Claudio Allocchio, Michael Slusarz, Bron Gondwana, Arnt Gulbrandsen, David Black, Hoa V. DINH, and Nick Hudson. So many legends 💖
Mark Crispin contributed to RFCs 4551 and 5162 that this document is replacing, and much of his contribution remains in this merged document. 👑
See also the list of people who contributed to RFC 4551, which this document obsoletes 🧚
Authors' Addresses 💅
Alexey Melnikov
Isode Ltd
5 Castle Business Village
36 Station Road
Hampton, Middlesex TW12 2BX
UK
EMail: Alexey.Melnikov@isode.com
Dave Cridland
Surevine Ltd
PO Box 1136
Guildford, Surrey GU1 9ND
UK
EMail: dave.cridland@surevine.com📦 where this page comes from
- ietf: rfc7162 2014-05
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms