RFC 4551

IMAP Extension for Conditional STORE Operation or Quick Flag Changes Resynchronization

๐Ÿ“… published June 2006, from ietf

IMAP Extension for Conditional STORE Operation or Quick Flag Changes Resynchronization ๐ŸŽ€

Okay bestie, let's start with the basics ๐Ÿ’… This is RFC 4551. It's a Standards Track doc from June 2006, written by A. Melnikov (Isode Ltd.) and S. Hole (ACI WorldWide/MessagingDirect), and it updates RFC 3501 (that's IMAP). Network Working Group, obviously ๐ŸŒธ

Status of This Memo โœจ

This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited.

Copyright (C) The Internet Society (2006).

Abstract ๐Ÿ’œ

Okay so IMAP (RFC 3501) clients often need to coordinate changes to one shared mailbox. Think different clients working for the same user, or multiple users poking at the same shared mailbox. They need a way to sync up message state changes, and they must be able to guarantee that only one client can change message state (like message flags) at any given time. Ngl, a super real example is using an IMAP mailbox as a message queue with several clients dequeueing from it ๐Ÿ”ฅ

The Conditional Store facility gives you a protected update mechanism for message state info, and it can detect and resolve conflicts between multiple writing mail clients. It's giving safety โœจ

It also lets a client quickly resynchronize mailbox flag changes. Iconic ๐Ÿ‘‘

This document defines an extension to IMAP (RFC 3501).

Table of Contents ๐Ÿงš

1.  Introduction and Overview ................................. 3
2.  Conventions Used in This Document ......................... 5
3.  IMAP Protocol Changes ..................................... 6
3.1. New OK untagged responses for SELECT and EXAMINE ......... 6
3.1.1. HIGHESTMODSEQ response code ............................ 6
3.1.2. NOMODSEQ response code ................................. 7
3.2. STORE and UID STORE Commands ............................. 7
3.3 FETCH and UID FETCH Commands ..............................13
3.3.1. CHANGEDSINCE FETCH modifier ............................13
3.3.2. MODSEQ message data item in FETCH Command ..............14
3.4. MODSEQ search criterion in SEARCH ........................16
3.5. Modified SEARCH untagged response ........................17
3.6. HIGHESTMODSEQ status data items ..........................17
3.7. CONDSTORE parameter to SELECT and EXAMINE ................18
3.8. Additional quality of implementation issues ..............18
4.  Formal Syntax .............................................19
5.  Server implementation considerations ......................21
6.  Security Considerations ...................................22
7.  IANA Considerations .......................................22
8.  References ................................................23
8.1. Normative References .....................................23
8.2. Informative References ...................................23
9.  Acknowledgements ..........................................23

1. Introduction and Overview โœจ

Okay so the Conditional STORE extension is present in any IMAP4 implementation that returns "CONDSTORE" as one of its supported capabilities in the CAPABILITY command response. If you see that in the list, it's on ๐Ÿ’–

An IMAP server that supports this extension MUST associate a positive unsigned 64-bit value called a modification sequence (mod-sequence) with every IMAP message. This is an opaque value that the server updates whenever a metadata item is modified. The server MUST guarantee that each STORE command performed on the same mailbox (including simultaneous stores to different metadata items from different connections) will get a different mod-sequence value. Also, for any two successful STORE operations performed in the same session on the same mailbox, the mod-sequence of the second completed operation MUST be greater than the mod-sequence of the first completed. Note that the latter rule disallows the use of the system clock as a mod-sequence, because if system time changes (e.g., an NTP [NTP] client adjusting the time), the next generated value might be less than the previous one. The clock is NOT it, bestie ๐Ÿ™ˆ

Mod-sequences let a client that supports the CONDSTORE extension figure out if a message's metadata has changed since some moment it already knows about. Whenever the state of a flag changes (so the flag gets added when it wasn't set before, or removed when it was set), the modification sequence for the message MUST be updated. Adding a flag that's already there, or removing one that isn't, SHOULD NOT change the mod-sequence. Main character energy means not changing things that didn't change ๐Ÿฆ‹

When a message is appended to a mailbox (via the IMAP APPEND command, COPY to the mailbox, or using an external mechanism), the server generates a new modification sequence that's higher than the highest modification sequence of all messages in the mailbox, and assigns it to the appended message. โœจ

The server MAY store separate (per-message) modification sequence values for different metadata items. If it does, the per-message mod-sequence is the highest mod-sequence of all metadata items for that message. Lowkey clever ๐Ÿง 

A server that supports this extension is not required to be able to store mod-sequences for every available mailbox. Section 3.1.2 describes how the server may act if a particular mailbox doesn't support the persistent storage of mod-sequences ๐ŸŒˆ

This extension makes the following changes to the IMAP4 protocol:

  • a) adds UNCHANGEDSINCE STORE modifier.
  • b) adds the MODIFIED response code, which should be used with an OK response to the STORE command. (It can also be used in a NO response.)
  • c) adds a new MODSEQ message data item for use with the FETCH command.
  • d) adds CHANGEDSINCE FETCH modifier.
  • e) adds a new MODSEQ search criterion.
  • f) extends the syntax of untagged SEARCH responses to include mod-sequence.
  • g) adds new OK untagged responses for the SELECT and EXAMINE commands.
  • h) defines an additional parameter to SELECT/EXAMINE commands.
  • i) adds the HIGHESTMODSEQ status data item to the STATUS command.

A client supporting the CONDSTORE extension says "yes please, send me mod-sequence updates" by issuing any of these:

  • a SELECT or EXAMINE command with the CONDSTORE parameter,
  • a STATUS (HIGHESTMODSEQ) command,
  • a FETCH or SEARCH command that includes the MODSEQ message data item,
  • a FETCH command with the CHANGEDSINCE modifier, or
  • a STORE command with the UNCHANGEDSINCE modifier.

The server MUST include mod-sequence data in all subsequent untagged FETCH responses (until the connection is closed), whether they were caused by a regular STORE, a STORE with UNCHANGEDSINCE modifier, or an external agent. Full commitment, no ghosting ๐Ÿ’œ

This document uses the term "CONDSTORE-aware client" for a client that announces its willingness to receive mod-sequence updates as described above. The term "CONDSTORE enabling command" refers to any of the commands listed above. A future extension to this document may extend that list of CONDSTORE enabling commands. A first CONDSTORE enabling command executed in the session MUST cause the server to return HIGHESTMODSEQ (Section 3.1.1), unless the server has already sent the NOMODSEQ (Section 3.1.2) response code when the currently selected mailbox was selected. ๐ŸŽ€

The rest of this document describes the protocol changes more rigorously ๐Ÿซถ


2. Conventions Used in This Document ๐Ÿ’…

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [KEYWORDS].

In examples, lines beginning with "S:" are sent by the IMAP server, and lines beginning with "C:" are sent by the client. Line breaks may appear in example commands solely for editorial clarity; when present in the actual message, they are represented by "CRLF". Little reminder, so you don't get confused in the examples ๐Ÿ“

Formal syntax is defined using ABNF [ABNF]. Grammar nerds, this one's for you ๐Ÿค“

The term "metadata" or "metadata item" is used throughout this document. It refers to any system or user-defined keyword. Future documents may extend "metadata" to include other dynamic message data. โœจ

Some IMAP mailboxes are private, accessible only to the owning user. Other mailboxes are not, either because the owner has set an Access Control List [ACL] that permits access by other users, or because it is a shared mailbox. Let's call a metadata item "shared" for the mailbox if any changes to it are persistent and visible to all other users accessing the mailbox. Otherwise, the metadata item is called "private". Note that private metadata items are still visible to all sessions accessing the mailbox as the same user. Also note that different mailboxes may have different metadata items marked as shared. Different vibes per mailbox, it's a whole thing ๐Ÿฆ‹

See Section 1 for the definition of a "CONDSTORE-aware client" and a "CONDSTORE enabling command". ๐Ÿ’–


3. IMAP Protocol Changes ๐ŸŒธ

3.1. New OK Untagged Responses for SELECT and EXAMINE ๐Ÿงš

This document adds two new response codes, HIGHESTMODSEQ and NOMODSEQ. One of those response codes MUST be returned in the OK untagged response for a successful SELECT/EXAMINE command. Literally one or the other, no exceptions ๐Ÿ’€

When opening a mailbox, the server must check whether the mailbox supports the persistent storage of mod-sequences. If the mailbox supports the persistent storage of mod-sequences and the mailbox open operation succeeds, the server MUST send the OK untagged response including the HIGHESTMODSEQ response code. If the persistent storage for the mailbox is not supported, the server MUST send the OK untagged response including the NOMODSEQ response code instead. That's the whole vibe check for part 1 ๐ŸŒˆโœจ

3.1.1. HIGHESTMODSEQ Response Code โœจ

okay so this section adds a brand new response code ๐Ÿ’… it rides along in the OK untagged response for SELECT and EXAMINE. a server that stores mod-sequences persistently for the mailbox MUST send the OK untagged response with the HIGHESTMODSEQ response code on every single successful SELECT or EXAMINE, no exceptions bestie ๐Ÿซถ

OK [HIGHESTMODSEQ <mod-sequence-value>]

where <mod-sequence-value> is the highest mod-sequence value of all messages in the mailbox. when the server changes UIDVALIDITY for a mailbox, it doesn't have to keep the same HIGHESTMODSEQ for that mailbox ๐Ÿฆ‹

a disconnected client can use HIGHESTMODSEQ to check if it has to refetch metadata from the server ๐Ÿงš if the UIDVALIDITY value has changed for the selected mailbox, the client MUST delete its cached value of HIGHESTMODSEQ ๐Ÿ’€ if UIDVALIDITY is the same, and the HIGHESTMODSEQ stored in the client's cache is less than the value the server returned, then some metadata items on the server have changed since the last sync, so the client needs to update its cache ๐ŸŒธ

the client MAY use SEARCH MODSEQ (Section 3.4) to find out exactly which metadata items changed โœจ or, if it prefers, it MAY issue FETCH with the CHANGEDSINCE modifier (Section 3.3.1) to grab data for all messages with metadata changed since some known modification sequence ๐Ÿ‘‘

Example 1:

C: A142 SELECT INBOX
S: * 172 EXISTS
S: * 1 RECENT
S: * OK [UNSEEN 12] Message 12 is first unseen
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: * OK [UIDNEXT 4392] Predicted next UID
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
S: * OK [HIGHESTMODSEQ 715194045007]
S: A142 OK [READ-WRITE] SELECT completed

3.1.2. NOMODSEQ Response Code ๐Ÿ™ˆ

this one is the mirror image, so pay attention bestie ๐Ÿ‘€ a server that doesn't support persistent storage of mod-sequences for the mailbox MUST send the OK untagged response with the NOMODSEQ response code on every successful SELECT or EXAMINE ๐Ÿ“ a server that returned NOMODSEQ for a mailbox, and then gets one of the following commands while that mailbox is selected:

  • a FETCH command with the CHANGEDSINCE modifier,
  • a FETCH or SEARCH command that includes the MODSEQ message data item, or
  • a STORE command with the UNCHANGEDSINCE modifier

MUST reject any such command with the tagged BAD response ๐Ÿšซ no notes, no excuses, just BAD ๐Ÿ’…

Example 2:

C: A142 SELECT INBOX
S: * 172 EXISTS
S: * 1 RECENT
S: * OK [UNSEEN 12] Message 12 is first unseen
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: * OK [UIDNEXT 4392] Predicted next UID
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
S: * OK [NOMODSEQ] Sorry, this mailbox format doesn't support
       modsequences
S: A142 OK [READ-WRITE] SELECT completed

3.2. STORE and UID STORE Commands โœจ

Okay bestie, this part defines one STORE modifier ๐Ÿ’– (see Section 2.5 of [IMAPABNF]), and it's giving conditional energy ๐Ÿ’…:

UNCHANGEDSINCE <mod-sequence>

For each message in the message set, the server checks the mod-sequence of every metadata item on that message. If none of them is greater than the UNCHANGEDSINCE value (so they're all equal or less), the requested operation (the one described by the message data item) runs ๐ŸŒธ. If that works out, the server MUST update the mod-sequence attribute of the message โœจ. An untagged FETCH response MUST be sent even when the .SILENT suffix is there, and that response MUST include the MODSEQ message data item ๐Ÿซถ. This is needed so the client's cache has the right mod-sequence values. See Section 3.3.2 for more details.

But if any metadata item on the message has a mod-sequence greater than the UNCHANGEDSINCE value, then the requested operation MUST NOT be performed ๐Ÿ’€. In that case the mod-sequence attribute of the message stays as it is, and the message number (or the unique identifier, for UID STORE) gets added to the list of messages that failed the UNCHANGESINCE test ๐Ÿ™ˆ.

When the server finishes the operation on every message in the message set, it looks at the list of messages that failed the UNCHANGESINCE test. If that list isn't empty, the server MUST return a MODIFIED response code in the tagged response ๐Ÿ”ฅ. The MODIFIED response code includes the message set (for STORE) or the set of UIDs (for UID STORE) of every message that failed the UNCHANGESINCE test ๐Ÿ’œ.

Example 3:

All messages pass the UNCHANGESINCE test. โœจ

C: a103 UID STORE 6,4,8 (UNCHANGEDSINCE 12121230045)
    +FLAGS.SILENT (\Deleted)
S: * 1 FETCH (UID 4 MODSEQ (12121231000))
S: * 2 FETCH (UID 6 MODSEQ (12121230852))
S: * 4 FETCH (UID 8 MODSEQ (12121130956))
S: a103 OK Conditional Store completed

Example 4:

C: a104 STORE * (UNCHANGEDSINCE 12121230045) +FLAGS.SILENT
   (\Deleted $Processed)
S: * 50 FETCH (MODSEQ (12111230047))
S: a104 OK Store (conditional) completed

Example 5:

C: c101 STORE 1 (UNCHANGEDSINCE 12121230045) -FLAGS.SILENT
   (\Deleted)
S: * OK [HIGHESTMODSEQ 12111230047]
S: * 50 FETCH (MODSEQ (12111230048))
S: c101 OK Store (conditional) completed

HIGHESTMODSEQ response code was sent by the server presumably because this was the first CONDSTORE enabling command. ๐ŸŒˆ

Example 6:

In spite of the failure of the conditional STORE operation for message 7, the server continues to process the conditional STORE in order to find all messages that fail the test. ๐Ÿงš

C: d105 STORE 7,5,9 (UNCHANGEDSINCE 320162338)
    +FLAGS.SILENT (\Deleted)
S: * 5 FETCH (MODSEQ (320162350))
S: d105 OK [MODIFIED 7,9] Conditional STORE failed

Example 7:

Same as above, but the server follows the SHOULD recommendation in Section 6.4.6 of [IMAP4]. ๐Ÿ’…

C: d105 STORE 7,5,9 (UNCHANGEDSINCE 320162338)
    +FLAGS.SILENT (\Deleted)
S: * 7 FETCH (MODSEQ (320162342) FLAGS (\Seen \Deleted))
S: * 5 FETCH (MODSEQ (320162350))
S: * 9 FETCH (MODSEQ (320162349) FLAGS (\Answered))
S: d105 OK [MODIFIED 7,9] Conditional STORE failed

Use of UNCHANGEDSINCE with a modification sequence of 0 always fails if the metadata item exists. A system flag MUST always be considered existent, whether it was set or not. ๐Ÿฆ‹

Example 8:

C: a102 STORE 12 (UNCHANGEDSINCE 0)
    +FLAGS.SILENT ($MDNSent)
S: a102 OK [MODIFIED 12] Conditional STORE failed

The client has tested the presence of the $MDNSent user-defined keyword. ๐Ÿ“

Note: A client trying to make an atomic change to the state of a particular metadata item (or a set of metadata items) should be prepared to deal with the case when the server returns the MODIFIED response code if the state of the metadata item being watched hasn't changed (but the state of some other metadata item has). This is necessary, because some servers don't store separate mod-sequences for different metadata items. However, a server implementation SHOULD avoid generating spurious MODIFIED responses for +FLAGS/-FLAGS STORE operations, even when the server stores a single mod-sequence per message. Section 5 describes how this can be achieved. โญ

Unless the server has included an unsolicited FETCH to update client's knowledge about messages that have failed the UNCHANGEDSINCE test, upon receipt of the MODIFIED response code, the client SHOULD try to figure out if the required metadata items have indeed changed by issuing FETCH or NOOP command. It is RECOMMENDED that the server avoids the need for the client to do that by sending an unsolicited FETCH response (Examples 9 and 10). ๐Ÿ‘‘

If the required metadata items haven't changed, the client SHOULD retry the command with the new mod-sequence. The client SHOULD allow for a configurable but reasonable number of retries (at least 2). ๐ŸŽ€

Example 9:

In the example below, the server returns the MODIFIED response code without sending information describing why the STORE UNCHANGEDSINCE operation has failed. ๐Ÿ™ˆ

C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
    +FLAGS.SILENT ($Processed)
S: * 100 FETCH (MODSEQ (303181230852))
S: * 102 FETCH (MODSEQ (303181230852))
...
S: * 150 FETCH (MODSEQ (303181230852))
S: a106 OK [MODIFIED 101] Conditional STORE failed

The flag $Processed was set on the message 101...

C: a107 NOOP
S: * 101 FETCH (MODSEQ (303011130956) FLAGS ($Processed))
S: a107 OK

Or the flag hasn't changed, but another has (note that this server behaviour is discouraged. Server implementers should also see Section 5)...

C: b107 NOOP
S: * 101 FETCH (MODSEQ (303011130956) FLAGS (\Deleted \Answered))
S: b107 OK

...and the client retries the operation for the message 101 with the updated UNCHANGEDSINCE value

C: b108 STORE 101 (UNCHANGEDSINCE 303011130956)
    +FLAGS.SILENT ($Processed)
S: * 101 FETCH (MODSEQ (303181230852))
S: b108 OK Conditional Store completed

Example 10:

Same as above, but the server avoids the need for the client to poll for changes. ๐Ÿงš

The flag $Processed was set on the message 101 by another client...

C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
    +FLAGS.SILENT ($Processed)
S: * 100 FETCH (MODSEQ (303181230852))
S: * 101 FETCH (MODSEQ (303011130956) FLAGS ($Processed))
S: * 102 FETCH (MODSEQ (303181230852))
...
S: * 150 FETCH (MODSEQ (303181230852))
S: a106 OK [MODIFIED 101] Conditional STORE failed

Or the flag hasn't changed, but another has (note that this server behaviour is discouraged. Server implementers should also see Section 5)...

C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
    +FLAGS.SILENT ($Processed)
S: * 100 FETCH (MODSEQ (303181230852))
S: * 101 FETCH (MODSEQ (303011130956) FLAGS (\Deleted \Answered))
S: * 102 FETCH (MODSEQ (303181230852))
...
S: * 150 FETCH (MODSEQ (303181230852))
S: a106 OK [MODIFIED 101] Conditional STORE failed

...and the client retries the operation for the message 101 with the updated UNCHANGEDSINCE value

C: b108 STORE 101 (UNCHANGEDSINCE 303011130956)
    +FLAGS.SILENT ($Processed)
S: * 101 FETCH (MODSEQ (303181230852))
S: b108 OK Conditional Store completed

Or the flag hasn't changed, but another has (nice server behaviour. Server implementers should also see Section 5)...

C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
    +FLAGS.SILENT ($Processed)

S: * 100 FETCH (MODSEQ (303181230852))
S: * 101 FETCH (MODSEQ (303011130956) FLAGS ($Processed \Deleted
     \Answered))
S: * 102 FETCH (MODSEQ (303181230852))
...
S: * 150 FETCH (MODSEQ (303181230852))
S: a106 OK Conditional STORE completed

Example 11:

The following example is based on the example from the Section 4.2.3 of [RFC-2180] and demonstrates that the MODIFIED response code may be also returned in the tagged NO response. ๐ŸŒธ

Client tries to conditionally STORE flags on a mixture of expunged and non-expunged messages; one message fails the UNCHANGEDSINCE test. ๐Ÿฆ‹

C: B001 STORE 1:7 (UNCHANGEDSINCE 320172338) +FLAGS (\SEEN)
S: * 1 FETCH (MODSEQ (320172342) FLAGS (\SEEN))
S: * 3 FETCH (MODSEQ (320172342) FLAGS (\SEEN))
S: B001 NO [MODIFIED 2] Some of the messages no longer exist.

C: B002 NOOP
S: * 4 EXPUNGE
S: * 4 EXPUNGE
S: * 4 EXPUNGE
S: * 4 EXPUNGE
S: * 2 FETCH (MODSEQ (320172340) FLAGS (\Deleted \Answered))
S: B002 OK NOOP Completed.

By receiving FETCH responses for messages 1 and 3, and EXPUNGE responses that indicate that messages 4 through 7 have been expunged, the client retries the operation only for the message 2. The updated UNCHANGEDSINCE value is used. ๐Ÿ’–

C: b003 STORE 2 (UNCHANGEDSINCE 320172340) +FLAGS (\Seen)
S: * 2 FETCH (MODSEQ (320180050))
S: b003 OK Conditional Store completed

Note: If a message is specified multiple times in the message set, and the server doesn't internally eliminate duplicates from the message set, it MUST NOT fail the conditional STORE operation for the second (or subsequent) occurrence of the message if the operation completed successfully for the first occurrence. For example, if the client specifies: ๐Ÿ™ˆ

e105 STORE 7,3:9 (UNCHANGEDSINCE 12121230045)
 +FLAGS.SILENT (\Deleted)

the server must not fail the operation for message 7 as part of processing "3:9" if it succeeded when message 7 was processed the first time. โœจ

Once the client specified the UNCHANGEDSINCE modifier in a STORE command, the server MUST include the MODSEQ fetch response data items in all subsequent unsolicited FETCH responses. ๐Ÿ‘‘

This document also changes the behaviour of the server when it has performed a STORE or UID STORE command and the UNCHANGEDSINCE modifier is not specified. If the operation is successful for a message, the server MUST update the mod-sequence attribute of the message. The server is REQUIRED to include the mod-sequence value whenever it decides to send the unsolicited FETCH response to all CONDSTORE-aware clients that have opened the mailbox containing the message. ๐ŸŒˆ

Server implementers should also see Section 3.8 for additional quality of implementation issues related to the STORE command. ๐Ÿงš


3.3. FETCH and UID FETCH Commands ๐ŸŽ€

3.3.1. CHANGEDSINCE FETCH Modifier

This document defines the following FETCH modifier (see Section 2.4 of [IMAPABNF]): ๐Ÿ’…

CHANGEDSINCE <mod-sequence>

The CHANGEDSINCE FETCH modifier lets you make a smaller subset of the list of messages described by the sequence set ๐Ÿฆ‹. The information described by the message data items only comes back for messages whose mod-sequence is bigger than <mod-sequence> โœจ.

When the CHANGEDSINCE FETCH modifier is specified, it implicitly adds the MODSEQ FETCH message data item (Section 3.3.2) ๐Ÿ’–.

Example 12:

C: s100 UID FETCH 1:* (FLAGS) (CHANGEDSINCE 12345)
S: * 1 FETCH (UID 4 MODSEQ (65402) FLAGS (\Seen))
S: * 2 FETCH (UID 6 MODSEQ (75403) FLAGS (\Deleted))
S: * 4 FETCH (UID 8 MODSEQ (29738) FLAGS ($NoJunk $AutoJunk
     $MDNSent))
S: s100 OK FETCH completed

3.3.2. MODSEQ Message Data Item in FETCH Command ๐Ÿฆ‹

okay bestie, this extension adds a MODSEQ message data item to FETCH ๐Ÿ’… It lets clients grab mod-sequence values for a whole range of messages in the mailbox that's currently selected ๐ŸŒธ

Once the client has specified the MODSEQ message data item in a FETCH request, the server MUST include the MODSEQ fetch response data items in all subsequent unsolicited FETCH responses ๐Ÿ‘‘

Syntax:

MODSEQ

The MODSEQ message data item causes the server to return MODSEQ fetch response data items.

Syntax:

MODSEQ ( <permsg-modsequence> )

MODSEQ response data items contain per-message mod-sequences โœจ

The MODSEQ response data item comes back when the client issued FETCH with the MODSEQ message data item. It also lets the server tell the client about mod-sequence changes caused by conditional STOREs (Section 3.2) and/or changes that came from external sources ๐Ÿซถ

Example 13:

C: a FETCH 1:3 (MODSEQ)
S: * 1 FETCH (MODSEQ (624140003))
S: * 2 FETCH (MODSEQ (624140007))
S: * 3 FETCH (MODSEQ (624140005))
S: a OK Fetch complete

In this example, the client asks for per-message mod-sequences for a set of messages ๐Ÿ’–

When a flag for a message gets changed in a different session, the server sends an unsolicited FETCH response containing the mod-sequence for that message ๐Ÿ”ฅ

Example 14:

Okay, so this one is a whole story ๐Ÿ“ Session 1 is signed in as the user "alex", and that user adds a shared flag \Deleted:

C: A142 SELECT INBOX
...
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Answered \Deleted \Seen \*)] Limited
...

C: A160 STORE 7 +FLAGS.SILENT (\Deleted)
S: * 7 FETCH (MODSEQ (2121231000))
S: A160 OK Store completed

Session 2 is also signed in as "alex". Any changes to flags are always reported to every session signed in as the same user as in session 1 ๐ŸŒˆ

C: C180 NOOP
S: * 7 FETCH (FLAGS (\Deleted \Answered) MODSEQ (12121231000))
S: C180 OK Noop completed

Session 3 is signed in as the user "andrew" ๐Ÿงš Since \Deleted is a shared flag, the change from session 1 shows up in session 3 too:

D210 NOOP
S: * 7 FETCH (FLAGS (\Deleted \Answered) MODSEQ (12121231000))
S: D210 OK Noop completed

Now the user changes a private flag, \Seen, in session 1...

C: A240 STORE 7 +FLAGS.SILENT (\Seen)
S: * 7 FETCH (MODSEQ (12121231777))
S: A240 OK Store completed

...which only gets reported in session 2...

C: C270 NOOP
S: * 7 FETCH (FLAGS (\Deleted \Answered \Seen) MODSEQ
     (12121231777))
S: C270 OK Noop completed

...and not in session 3 ๐Ÿ™ˆ

C: D300 NOOP
S: D300 OK Noop completed

And finally, the user removes the flags \Answered (shared) and \Seen (private) in session 1 ๐Ÿ’…

C: A330 STORE 7 -FLAGS.SILENT (\Answered \Seen)
S: * 7 FETCH (MODSEQ (12121245160))
S: A330 OK Store completed

Both changes get reported in session 2...

C: C360 NOOP
S: * 7 FETCH (FLAGS (\Deleted) MODSEQ (12121245160))
S: C360 OK Noop completed

...and only the changes to shared flags get reported in session 3 โœจ

C: D390 NOOP
S: * 7 FETCH (FLAGS (\Deleted) MODSEQ (12121245160))
S: D390 OK Noop completed

Server implementers should also check out Section 3.8 for extra quality-of-implementation stuff related to the FETCH command ๐ŸŽ€


The MODSEQ criterion for the SEARCH command lets a client search for metadata items that were modified since a specified moment ๐Ÿ”

Syntax:

MODSEQ [<entry-name> <entry-type-req>] <mod-sequence-valzer>

This finds messages whose modification values are equal to or greater than <mod-sequence-valzer>. It's super useful, for example, for finding out which messages hold metadata items that changed since the last time a client updated its disconnected cache ๐Ÿ’œ The client may also specify <entry-name> (the name of a metadata item) and <entry-type-req> (the type of metadata item) before <mod-sequence-valzer>. <entry-type-req> can be one of "shared", "priv" (private), or "all". The last one means the server should use the biggest value among the "priv" and "shared" mod-sequences for the metadata item. If the server doesn't store separate mod-sequences for different metadata items internally, it MUST ignore <entry-name> and <entry-type-req>. Otherwise, the server should use them to narrow down the search ๐ŸŒธ

For a flag <flagname>, the matching <entry-name> looks like /flags/<flagname>, as defined in [IMAPABNF]. Heads up: the leading \ character that marks a system flag has to be escaped as described in Section 4.3 of [IMAP4], because the <entry-name> uses quoted-string syntax ๐Ÿ™ˆ

If a client specifies a MODSEQ criterion in a SEARCH command and the server returns a non-empty SEARCH result, the server MUST also append (to the end of the untagged SEARCH response) the highest mod-sequence for all messages being returned. See also Section 3.5 ๐Ÿ‘‘

Example 15:

C: a SEARCH MODSEQ "/flags/\\draft" all 620162338
S: * SEARCH 2 5 6 7 11 12 18 19 20 23 (MODSEQ 917162500)
S: a OK Search complete

In the example above, the search results include the message numbers of any messages that contain the string "IMAP4" in the "value" attribute of the "/comment" entry and that have a mod-sequence equal to or greater than 620162338 for the "\Draft" flag ๐Ÿฆ‹

Example 16:

C: t SEARCH OR NOT MODSEQ 720162338 LARGER 50000
S: * SEARCH
S: t OK Search complete, nothing found

3.5. Modified SEARCH Untagged Response ๐Ÿ’–

Data:

zero or more numbers
mod-sequence value (omitted if no match)

This document extends the syntax of the untagged SEARCH response so it includes the highest mod-sequence for all messages being returned ๐ŸŒˆ

If a client specifies a MODSEQ criterion in a SEARCH (or UID SEARCH) command and the server returns a non-empty SEARCH result, the server MUST also append (to the end of the untagged SEARCH response) the highest mod-sequence for all messages being returned. See Section 3.4 for examples ๐Ÿงš


3.6. HIGHESTMODSEQ Status Data Items ๐Ÿ“

This document defines a new status data item ๐Ÿ’…

HIGHESTMODSEQ

The highest mod-sequence value of all messages in the mailbox. It's the same value the server returns in the HIGHESTMODSEQ response code in an OK untagged response (see Section 3.1.1). If the server doesn't support persistent storage of mod-sequences for the mailbox (see Section 3.1.2), the server MUST return 0 as the value of the HIGHESTMODSEQ status data item ๐Ÿ”ฅ

Example 17:

C: A042 STATUS blurdybloop (UIDNEXT MESSAGES HIGHESTMODSEQ)
S: * STATUS blurdybloop (MESSAGES 231 UIDNEXT 44292
     HIGHESTMODSEQ 7011231777)
S: A042 OK STATUS completed

3.7. CONDSTORE Parameter to SELECT and EXAMINE โœจ

The CONDSTORE extension defines a single optional select parameter, "CONDSTORE", which tells the server that it MUST include the MODSEQ fetch response data items in all subsequent unsolicited FETCH responses ๐Ÿ’œ

The CONDSTORE parameter to SELECT/EXAMINE helps dodge a race condition. Without it, one or more metadata items could be modified in another session after the server has sent the HIGHESTMODSEQ response code, but before the client gets a chance to issue a CONDSTORE enabling command ๐Ÿ˜ญ

Example 18:

C: A142 SELECT INBOX (CONDSTORE)
S: * 172 EXISTS
S: * 1 RECENT
S: * OK [UNSEEN 12] Message 12 is first unseen
S: * OK [UIDVALIDITY 3857529045] UIDs valid
S: * OK [UIDNEXT 4392] Predicted next UID
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
S: * OK [HIGHESTMODSEQ 715194045007]
S: A142 OK [READ-WRITE] SELECT completed, CONDSTORE is now enabled

3.8. Additional Quality-of-Implementation Issues ๐ŸŽ€

Server implementations should follow this rule, which applies to any successfully completed STORE/UID STORE (with and without the UNCHANGEDSINCE modifier), as well as to a FETCH command that implicitly sets the \Seen flag ๐ŸŒธ

Adding the flag when it is already present or removing when it is not present SHOULD NOT change the mod-sequence.

This helps prevent spurious client synchronization requests โœจ

However, note that client implementers MUST NOT rely on this server behavior. A client can't tell apart the case where a server has violated the SHOULD mentioned above from the case where one or more clients set and unset (or unset and set) the flag in another session ๐Ÿ™ˆ

4. Formal Syntax ๐ŸŽ€

Okay so this part is the ABNF zone ๐Ÿ’… It's the formal grammar that spells out exactly what's allowed on the wire. ABNF means Augmented Backus-Naur Form [ABNF], and anything not defined here lives in the ABNF spec [ABNF], the IMAP spec [IMAP4], and the IMAP ABNF extensions spec [IMAPABNF] ๐Ÿ“š

Except where noted, alphabetic characters are case-insensitive โœจ Any upper or lowercase in the token strings is just for editorial clarity ๐Ÿซถ Implementations MUST accept these strings in a case-insensitive fashion. Period.

capability          =/ "CONDSTORE"

status-att          =/ "HIGHESTMODSEQ"
                       ;; extends non-terminal defined in RFC 3501.

status-att-val      =/ "HIGHESTMODSEQ" SP mod-sequence-valzer
                       ;; extends non-terminal defined in [IMAPABNF].
                       ;; Value 0 denotes that the mailbox doesn't
                       ;; support persistent mod-sequences
                       ;; as described in Section 3.1.2

store-modifier      =/ "UNCHANGEDSINCE" SP mod-sequence-valzer
                       ;; Only a single "UNCHANGEDSINCE" may be
                       ;; specified in a STORE operation

fetch-modifier      =/ chgsince-fetch-mod
                       ;; conforms to the generic "fetch-modifier"
                       ;; syntax defined in [IMAPABNF].

chgsince-fetch-mod  = "CHANGEDSINCE" SP mod-sequence-value
                       ;; CHANGEDSINCE FETCH modifier conforms to
                       ;; the fetch-modifier syntax

fetch-att           =/ fetch-mod-sequence
                       ;; modifies original IMAP4 fetch-att

fetch-mod-sequence  = "MODSEQ"

fetch-mod-resp      = "MODSEQ" SP "(" permsg-modsequence ")"

msg-att-dynamic     =/ fetch-mod-resp
search-key          =/ search-modsequence
                       ;; modifies original IMAP4 search-key
                       ;;
                       ;; This change applies to all commands
                       ;; referencing this non-terminal, in
                       ;; particular SEARCH.

search-modsequence  = "MODSEQ" [search-modseq-ext] SP
                      mod-sequence-valzer

search-modseq-ext   = SP entry-name SP entry-type-req

resp-text-code      =/ "HIGHESTMODSEQ" SP mod-sequence-value /
                       "NOMODSEQ" /
                       "MODIFIED" SP set

entry-name          = entry-flag-name

entry-flag-name     = DQUOTE "/flags/" attr-flag DQUOTE
                       ;; each system or user defined flag <flag>
                       ;; is mapped to "/flags/<flag>".
                       ;;
                       ;; <entry-flag-name> follows the escape rules
                       ;; used by "quoted" string as described in
                       ;; Section 4.3 of [IMAP4], e.g., for the flag
                       ;; \Seen the corresponding <entry-name> is
                       ;; "/flags/\\seen", and for the flag
                       ;; $MDNSent, the corresponding <entry-name>
                       ;; is "/flags/$mdnsent".

entry-type-resp     = "priv" / "shared"
                       ;; metadata item type

entry-type-req      = entry-type-resp / "all"
                       ;; perform SEARCH operation on private
                       ;; metadata item, shared metadata item or both

permsg-modsequence  = mod-sequence-value
                       ;; per message mod-sequence

mod-sequence-value  = 1*DIGIT
                       ;; Positive unsigned 64-bit integer
                       ;; (mod-sequence)
                       ;; (1 <= n < 18,446,744,073,709,551,615)

mod-sequence-valzer = "0" / mod-sequence-value

search-sort-mod-seq = "(" "MODSEQ" SP mod-sequence-value ")"

select-param        =/ condstore-param
                       ;; conforms to the generic "select-param"
                       ;; non-terminal syntax defined in [IMAPABNF].

condstore-param     = "CONDSTORE"

mailbox-data        =/ "SEARCH" [1*(SP nz-number) SP
                       search-sort-mod-seq]

attr-flag           = "\\Answered" / "\\Flagged" / "\\Deleted" /
                      "\\Seen" / "\\Draft" / attr-flag-keyword /
                      attr-flag-extension
                       ;; Does not include "\\Recent"

attr-flag-extension = "\\" atom
                       ;; Future expansion.  Client implementations
                       ;; MUST accept flag-extension flags.  Server
                       ;; implementations MUST NOT generate
                       ;; flag-extension flags except as defined by
                       ;; future standard or standards-track
                       ;; revisions of [IMAP4].

attr-flag-keyword   = atom

5. Server Implementation Considerations ๐Ÿง 

This section is all about how a server can be extra clever ๐Ÿฆ‹ Specifically, it covers a server that doesn't keep separate per-metadata mod-sequences for different metadata items. That server can skip sending the MODIFIED response for any of these conditional STORE operations:

+FLAGS
-FLAGS
+FLAGS.SILENT
-FLAGS.SILENT

Heads up though: this optimization can't be done for a conditional STORE FLAGS operation ๐Ÿ™ˆ

Let's use an example, because examples are everything ๐Ÿ“ The client has sent:

C: a106 STORE 100:150 (UNCHANGEDSINCE 212030000000)
   +FLAGS.SILENT ($Processed)

When the server gets the command and parses it successfully, it goes through the message set and tries to run the conditional STORE for each message โœจ

Here's the big idea: each server basically acts like a client ๐Ÿซถ It has to cache the current state of all IMAP flags the way the client knows them. To report flag changes, the server compares those cached values with the values in its database for IMAP flags.

Now imagine another client changed the \Deleted flag on message 101, and that change bumped the mod-sequence for that message. The server knows the mailbox mod-sequence changed, but it also knows two things:

a) the client isn't interested in the \Deleted flag, since it didn't include it in the +FLAGS.SILENT operation; and

b) the state of the $Processed flag hasn't changed. The server can figure this out by comparing its cached flag state with the flag state in the database.

So the server doesn't have to report MODIFIED to the client ๐Ÿ’… Instead, the server may set the $Processed flag, update the mod-sequence for message 101 once again, and send an untagged FETCH response with the new mod-sequence and flags:

S: * 101 FETCH (MODSEQ (303011130956)
   FLAGS ($Processed \Deleted \Answered))

Also see Section 3.8 for additional quality-of-implementation issues ๐Ÿ‘‘

6. Security Considerations ๐Ÿ”

Okay, serious moment ๐Ÿ˜ญ It is believed that the Conditional STORE extension doesn't raise any new security concerns beyond the ones already discussed in [IMAP4]. BUT the availability of this extension may make it possible for IMAP4 to be used in critical applications it couldn't be used for before. So correct IMAP server implementation and operation becomes even more important. No pressure ๐Ÿ’€

7. IANA Considerations ๐Ÿ“‹

IMAP4 capabilities get registered by publishing a standards track or IESG approved experimental RFC. The registry lives at:

http://www.iana.org/assignments/imap4-capabilities

This document defines the CONDSTORE IMAP capability, and IANA has added it to the registry accordingly โœจ

8. References ๐Ÿ“š

8.1. Normative References

[KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

[ABNF] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 4234, October 2005.

[IMAP4] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003.

[IMAPABNF] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4 ABNF", RFC 4466, April 2006.

8.2. Informative References

[ACAP] Newman, C. and J. Myers, "ACAP -- Application Configuration Access Protocol", RFC 2244, November 1997.

[ACL] Melnikov, A., "IMAP4 Access Control List (ACL) Extension", RFC 4314, December 2005.

[ANN] Daboo, C. and R. Gellens, "IMAP ANNOTATE Extension", Work in Progress, March 2006.

[NTP] Mills, D., "Network Time Protocol (Version 3) Specification, Implementation and Analysis", RFC 1305, March 1992.

[RFC-2180] Gahrns, M., "IMAP4 Multi-Accessed Mailbox Practice", RFC 2180, July 1997.

9. Acknowledgements ๐Ÿซถ

Some text was borrowed from "IMAP ANNOTATE Extension" [ANN] by Randall Gellens and Cyrus Daboo, and from "ACAP -- Application Configuration Access Protocol" [ACAP] by Chris Newman and John Myers. Big thanks to them ๐Ÿ’œ

Many thanks to Randall Gellens for his thorough review of the document ๐ŸŒธ

The authors also acknowledge the feedback provided by Cyrus Daboo, Larry Greenfield, Chris Newman, Harrie Hazewinkel, Arnt Gulbrandsen, Timo Sirainen, Mark Crispin, Ned Freed, Ken Murchison, and Dave Cridland. Such a squad ๐Ÿ‘‘

Authors' Addresses ๐Ÿ“ฎ

Alexey Melnikov
Isode Limited
5 Castle Business Village
36 Station Road
Hampton, Middlesex
TW12 2BX,
United Kingdom

EMail: Alexey.Melnikov@isode.com


Steve Hole
ACI WorldWide/MessagingDirect
#1807, 10088 102 Ave
Edmonton, AB
T5J 2Z1
Canada

EMail: Steve.Hole@messagingdirect.com

Copyright (C) The Internet Society (2006).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property ๐Ÿ”

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document, or the extent to which any license under such rights might or might not be available. It also doesn't represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat, any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF online IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

Acknowledgement ๐ŸŒˆ

Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). โœจ

๐Ÿ“ฆ where this page comes from