RFC 5256

Internet Message Access Protocol - SORT and THREAD Extensions

๐Ÿ“… published June 2008, from ietf

Title Block ๐ŸŽ€

This one is RFC 5256, Internet Message Access Protocol - SORT and THREAD Extensions, from June 2008. It's a Standards Track doc, written by M. Crispin (Panda Programming) and K. Murchison (Carnegie Mellon University). The header doesn't list anything it obsoletes or updates, so there's nothing to chase there ๐Ÿ’…

Status of This Memo ๐Ÿ“œ

This document specifies an Internet standards track protocol for the Internet community, and asks for discussion and suggestions for improvements. For the standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited ๐ŸŒธ

Abstract โœจ

Okay so this doc describes the base-level, server-based sorting and threading extensions to IMAP ๐Ÿ’– These extensions give IMAP clients that show sorted and threaded views a seriously big performance boost. Iconic behavior, honestly ๐Ÿ”ฅ

1. Introduction ๐Ÿฆ‹

The SORT and THREAD extensions to the [IMAP] protocol let the server do the sorting and threading of messages. The client doesn't have to download all the data to do it itself, which is why it's especially useful for online clients, as described in [IMAP-MODELS] ๐Ÿซถ

A server that supports the base-level SORT extension says so with a capability name that starts with "SORT". Future upwards-compatible extensions to SORT will also start with "SORT", which shows they're built on top of this base level ๐Ÿงš

A server that supports the THREAD extension says so with one or more capability names made of "THREAD=" followed by a supported threading algorithm name, as described in this document. This leaves room for future upwards-compatible extensions too ๐Ÿ’œ

A server that implements SORT and/or THREAD MUST collate strings in accordance with the requirements of I18NLEVEL=1, as described in [IMAP-I18N]. It SHOULD also implement and advertise the I18NLEVEL=1 extension. Alternatively, a server MAY implement I18NLEVEL=2 (or higher) and follow the rules of that level ๐ŸŒˆ

Discussion: SORT and THREAD are actually several years older than [IMAP-I18N]. When this was written, every known server implementation of SORT and THREAD followed the I18NLEVEL=1 rules, but they don't necessarily advertise it. As discussed in section 4.5 of [IMAP-I18N], all server implementations should eventually be updated to comply with I18NLEVEL=2 ๐Ÿ™ˆ

Historical note: the REFERENCES threading algorithm is based on the [THREADING] algorithm written and used in "Netscape Mail and News" versions 2.0 through 3.0. Throwback alert ๐Ÿ“

2. Terminology ๐Ÿ’…

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [KEYWORDS]. These are the official requirement words, so pay attention to them ๐Ÿ‘‘

The word "can" (not "may") refers to a possible circumstance or situation, as opposed to an optional facility of the protocol. Yes, the difference matters, bestie ๐Ÿ™ˆ

"User" means a human user, whereas "client" means the software the user is running. Not the same thing, periodt โœจ

In examples, "C:" and "S:" mark lines sent by the client and the server, respectively ๐ŸŽ€

2.1. Base Subject ๐Ÿง

Subject sorting and threading use the "base subject", which is the subject with certain subject artifacts stripped off. Those artifacts are complicated enough that the formal syntax for subject extraction rules is ambiguous, so this procedure is the way to find the "base subject". It uses the [ABNF] formal syntax rules described in section 5 ๐Ÿ’–

(1) Convert any RFC 2047 encoded-words in the subject to [UTF-8], as described in "Internationalization Considerations". Convert all tabs and continuations to space. Convert all multiple spaces to a single space.

(2) Remove all trailing text of the subject that matches the subj-trailer ABNF. Repeat until no more matches are possible.

(3) Remove all prefix text of the subject that matches the subj-leader ABNF.

(4) If there is prefix text of the subject that matches the subj-blob ABNF, and removing that prefix leaves a non-empty subj-base, then remove the prefix text.

(5) Repeat (3) and (4) until no matches remain.

Note: you can defer step (2) until step (6), but then step (4) has to check for subj-trailer.

(6) If the resulting text begins with the subj-fwd-hdr ABNF and ends with the subj-fwd-trl ABNF, remove the subj-fwd-hdr and subj-fwd-trl, and repeat from step (2).

(7) The resulting text is the "base subject" used in the SORT.

All servers and disconnected clients (as described in [IMAP-MODELS]) MUST use exactly this algorithm to determine the "base subject". If they didn't, a user could get inconsistent results depending on whether they're connected or disconnected. Nobody wants that drama ๐Ÿ˜ญ

2.2. Sent Date ๐Ÿ“…

In this document, "sent date" means the date and time from the Date: header, adjusted by time zone and normalized to UTC. For example, "31 Dec 2000 16:01:33 -0800" is equivalent to the UTC date and time of "1 Jan 2001 00:01:33 +0000". Time zone math, but make it cute ๐Ÿ•ฐ๏ธ

If the time zone is invalid, the date and time SHOULD be treated as UTC. If the time is also invalid, the time SHOULD be treated as 00:00:00. If there is no valid date or time, the date and time SHOULD be treated as 00:00:00 on the earliest possible date ๐ŸŒ™

This is different from the date-related criteria in the SEARCH command (described in section 6.4.4 of [IMAP]). Those only use the date, not the time, and they aren't adjusted by time zone. So don't mix them up, okay? ๐Ÿ™…โ€โ™€๏ธ

If the sent date can't be determined (the Date: header is missing or can't be parsed), the INTERNALDATE for that message is used as the sent date ๐Ÿงš

When two sent dates match exactly, the order the two messages appear in the mailbox (that is, by sequence number) is the tie-breaker that decides their order ๐ŸŽ€

3. Additional Commands ๐Ÿงธ

These commands are extensions to the [IMAP] base protocol ๐Ÿ’œ

The section headings are meant to line up with where they'd be in the main document if they were part of the base specification ๐Ÿฆ‹

BASE.6.4.SORT. SORT Command ๐Ÿ’

Arguments:  sort program
            charset specification
            searching criteria (one or more)

Data:       untagged responses: SORT

Result:     OK - sort completed
            NO - sort error: can't sort that charset or
                 criteria
            BAD - command unknown or arguments invalid

The SORT command is a variant of SEARCH with sorting semantics for the results. Before the searching criteria argument there are two arguments: a parenthesized list of sort criteria, and the searching charset ๐Ÿ’–

The charset argument is mandatory (unlike SEARCH). It shows the [CHARSET] of the strings in the searching criteria. The US-ASCII and [UTF-8] charsets MUST be implemented. All other charsets are optional ๐ŸŒธ

There's also a UID SORT command that returns unique identifiers instead of message sequence numbers. Heads up: UID SORT and SORT have separate searching criteria for message sequence numbers and UIDs, so the arguments to UID SORT are interpreted the same way as in SORT. This is like UID SEARCH, not like UID COPY, UID FETCH, or UID STORE ๐Ÿ™ˆ

The SORT command first searches the mailbox for messages that match the given searching criteria, using the charset argument to interpret the strings in those criteria. Then it returns the matching messages in an untagged SORT response, sorted by one or more sort criteria โœจ

Sorting is in ascending order. Earlier dates sort before later dates. Smaller sizes sort before larger sizes. Strings are sorted by the ascending values set by their collation algorithm (see "Internationalization Considerations") ๐Ÿ”ฅ

If two or more messages exactly match on the sorting criteria, they're sorted by the order they appear in the mailbox. In other words, there's an implicit sort criterion of "sequence number" ๐Ÿง

When there are multiple sort criteria, the result is sorted in the priority order the criteria appear in. For example, (SUBJECT DATE) sorts messages by their base subject text first, and then, for messages with the same base subject text, by their sent date ๐Ÿ’…

Untagged EXPUNGE responses are NOT permitted while the server is responding to a SORT command, but they ARE permitted during a UID SORT command. Wait, sorry, the original says "not permitted" and "permitted" so I'm keeping it plain: EXPUNGE is not permitted during SORT, but it is permitted during UID SORT ๐Ÿ™ƒ

The defined sort criteria are below. Check the Formal Syntax section for the exact syntax of the arguments. If the associated RFC-822 header for a particular criterion is absent, it's treated as the empty string. The empty string always collates before non-empty strings ๐Ÿซถ

ARRIVAL Internal date and time of the message. This is different from the ON criteria in SEARCH, which only uses the internal date ๐Ÿ“ฌ

CC [IMAP] addr-mailbox of the first "cc" address ๐Ÿ’Œ

DATE Sent date and time, as described in section 2.2 ๐Ÿ—“๏ธ

FROM [IMAP] addr-mailbox of the first "From" address ๐Ÿ“ฎ

REVERSE Followed by another sort criterion, it has the effect of that criterion but in reverse (descending) order ๐Ÿ”„

Note: REVERSE only reverses a single criterion. It does NOT affect the implicit "sequence number" sort criterion if all the other criteria are identical. So REVERSE SUBJECT is not the same as a reverse ordering of a SUBJECT sort. You can avoid this by adding more criteria, e.g., SUBJECT DATE vs. REVERSE SUBJECT REVERSE DATE. In general, though, it's better (and faster, if the client has a "reverse current ordering" command) to reverse the results in the client instead of issuing a new SORT ๐Ÿง 

SIZE Size of the message in octets ๐Ÿ“

SUBJECT Base subject text ๐ŸŽ€

TO [IMAP] addr-mailbox of the first "To" address ๐Ÿ’œ

Example:

C: A282 SORT (SUBJECT) UTF-8 SINCE 1-Feb-1994
S: * SORT 2 84 882
S: A282 OK SORT completed
C: A283 SORT (SUBJECT REVERSE DATE) UTF-8 ALL
S: * SORT 5 3 4 1 2
S: A283 OK SORT completed
C: A284 SORT (SUBJECT) US-ASCII TEXT "not in mailbox"
S: * SORT
S: A284 OK SORT completed

BASE.6.4.THREAD. THREAD Command ๐Ÿงต

Arguments:  threading algorithm
            charset specification
            searching criteria (one or more)

Data:       untagged responses: THREAD

Result:     OK - thread completed
            NO - thread error: can't thread that charset or
                 criteria
            BAD - command unknown or arguments invalid

The THREAD command is a variant of SEARCH with threading semantics for the results. Before the searching criteria argument there are two arguments: a threading algorithm and the searching charset ๐ŸŒˆ

The charset argument is mandatory (unlike SEARCH). It shows the [CHARSET] of the strings in the searching criteria. The US-ASCII and [UTF-8] charsets MUST be implemented. All other charsets are optional ๐Ÿซง

There is also a UID THREAD command that returns unique identifiers instead of message sequence numbers ๐Ÿฆ‹. Heads up, though: searching criteria for message sequence numbers and UIDs are separate things, so the arguments to UID THREAD get interpreted the same way as in THREAD ๐Ÿง . This is giving the same vibe as UID SEARCH, not UID COPY, UID FETCH, or UID STORE ๐Ÿ’….

The THREAD command first searches the mailbox for messages that match the given searching criteria, using the charset argument to interpret strings in those criteria ๐Ÿ”. Then it returns the matching messages in an untagged THREAD response, threaded according to the threading algorithm you picked ๐Ÿงตโœจ.

All collation is in ascending order ๐Ÿ“ˆ. Earlier dates collate before later dates, and strings get collated by ascending values set by their collation algorithm (see "Internationalization Considerations") ๐ŸŒ.

Untagged EXPUNGE responses are NOT permitted while the server is responding to a THREAD command ๐Ÿ™…โ€โ™€๏ธ, but they ARE permitted during a UID THREAD command ๐Ÿ’–.

Okay, the defined threading algorithms are next, and they're all so different bestie ๐ŸŒธ.

ORDEREDSUBJECT

The ORDEREDSUBJECT threading algorithm is also called "poor man's threading" ๐Ÿฅฒ๐Ÿ’€. The searched messages get sorted by base subject and then by sent date. After that they're split into separate threads, where each thread holds messages with the same base subject text ๐Ÿงถ. Last step: the threads get sorted by the sent date of the first message in each thread ๐Ÿ“….

The top level, or "root," in ORDEREDSUBJECT threading holds the first message of every thread ๐Ÿ‘‘. All messages in the root are siblings of each other ๐Ÿซถ. The second message of a thread is the child of the first message, and any later messages in the thread are siblings of the second message, which makes them children of the message at the root ๐ŸŒณ. So there are no grandchildren in ORDEREDSUBJECT threading ๐Ÿ™…โ€โ™€๏ธ.

Children in ORDEREDSUBJECT threading do not have descendents ๐ŸŒฟ. Client implementations SHOULD treat descendents of a child in a server response as being siblings of that child ๐Ÿ’….

REFERENCES

The REFERENCES threading algorithm is the main-character one ๐Ÿ‘‘๐Ÿ”ฅ. It threads the searched messages by grouping them into parent/child relationships based on which messages are replies to others ๐Ÿงต๐Ÿ’œ. Those relationships get built two ways: by reconstructing a message's ancestry from the references inside it, and by checking the original (not base) subject of a message to see if it's a reply to, or a forward of, another message ๐Ÿ•ต๏ธโ€โ™€๏ธ.

Note: "Message ID" in the following description refers to a normalized form of the msg-id in [RFC2822]. The actual text in RFC 2822 may use quoting, resulting in multiple ways of expressing the same Message ID. Implementations of the REFERENCES threading algorithm MUST normalize any msg-id in order to avoid false non-matches due to differences in quoting ๐ŸŽ€.

For example, the msg-id

<"01KF8JCEOCBS0045PS"@xxx.yyy.com>

and the msg-id

<01KF8JCEOCBS0045PS@xxx.yyy.com>

MUST be interpreted as being the same Message ID ๐Ÿคฏ.

The references used for rebuilding a message's ancestry are found using these rules ๐Ÿ”Ž:

  • If a message contains a References header line, then use the Message IDs in the References header line as the references โœจ.
  • If a message does not contain a References header line, or the References header line does not contain any valid Message IDs, then use the first (if any) valid Message ID found in the In-Reply-To header line as the only reference (parent) for this message ๐Ÿ’Œ.
Note: Although [RFC2822] permits multiple Message IDs in the In-Reply-To header, in actual practice this discipline has not been followed ๐Ÿ™ƒ. For example, In-Reply-To headers have been observed with message addresses after the Message ID, and there are no good heuristics for software to determine the difference. This is not a problem with the References header, however ๐Ÿซ .
  • If a message does not contain an In-Reply-To header line, or the In-Reply-To header line does not contain a valid Message ID, then the message does not have any references (NIL) ๐Ÿฅ€.

A message counts as a reply or forward if the base subject extraction rules, applied to the original subject, remove any of the following: a subj-refwd, a "(fwd)" subj-trailer, or a subj-fwd-hdr and subj-fwd-trl ๐Ÿ“จ๐Ÿ’….

The REFERENCES algorithm is WAY more complex than ORDEREDSUBJECT ๐Ÿ˜ญ. It has six main steps, and they're outlined in detail below ๐Ÿ“‹โœจ.

  • (1) For each searched message:
    • (A) Using the Message IDs in the message's references, link the corresponding messages (those whose Message-ID header line contains the given reference Message ID) together as parent/child. Make the first reference the parent of the second (and the second a child of the first), the second the parent of the third (and the third a child of the second), etc. The following rules govern the creation of these links ๐Ÿ”—:
      • If a message does not contain a Message-ID header line, or the Message-ID header line does not contain a valid Message ID, then assign a unique Message ID to this message ๐Ÿ†”.
      • If two or more messages have the same Message ID, then only use that Message ID in the first (lowest sequence number) message, and assign a unique Message ID to each of the subsequent messages with a duplicate of that Message ID ๐Ÿ‘ฏโ€โ™€๏ธ.
      • If no message can be found with a given Message ID, create a dummy message with this ID. Use this dummy message for all subsequent references to this ID ๐Ÿ‘ป.
      • If a message already has a parent, don't change the existing link. This is done because the References header line may have been truncated by a Mail User Agent (MUA). As a result, there is no guarantee that the messages corresponding to adjacent Message IDs in the References header line are parent and child ๐Ÿฅบ.
      • Do not create a parent/child link if creating that link would introduce a loop. For example, before making message A the parent of B, make sure that A is not a descendent of B ๐Ÿ”๐Ÿšซ.
Note: Message ID comparisons are case-sensitive ๐Ÿ” .
  • (B) Create a parent/child link between the last reference (or NIL if there are no references) and the current message. If the current message already has a parent, it is probably the result of a truncated References header line, so break the current parent/child link before creating the new correct one. As in step 1.A, do not create the parent/child link if creating that link would introduce a loop. Note that if this message has no references, it will now have no parent ๐Ÿ’ซ.
Note: The parent/child links created in steps 1.A and 1.B MUST be kept consistent with one another at ALL times ๐Ÿšจ.
  • (2) Gather together all of the messages that have no parents and make them all children (siblings of one another) of a dummy parent (the "root"). These messages constitute the first (head) message of the threads created thus far ๐ŸŒฑ.
  • (3) Prune dummy messages from the thread tree. Traverse each thread under the root, and for each message ๐Ÿช“:
    • If it is a dummy message with NO children, delete it.
    • If it is a dummy message with children, delete it, but promote its children to the current level. In other words, splice them in with the dummy's siblings.
    • Do not promote the children if doing so would make them children of the root, unless there is only one child ๐Ÿ™…โ€โ™€๏ธ.
  • (4) Sort the messages under the root (top-level siblings only) by sent date as described in section 2.2. In the case of a dummy message, sort its children by sent date and then use the first child for the top-level sort ๐Ÿ“…๐Ÿ’–.
  • (5) Gather together messages under the root that have the same base subject text ๐Ÿงบ.
    • (A) Create a table for associating base subjects with messages, called the subject table ๐Ÿ“’.
    • (B) Populate the subject table with one message per each base subject. For each child of the root:
      • (i) Find the subject of this thread, by using the base subject from either the current message or its first child if the current message is a dummy. This is the thread subject ๐ŸŽฏ.
      • (ii) If the thread subject is empty, skip this message ๐Ÿซฅ.
      • (iii) Look up the message associated with the thread subject in the subject table ๐Ÿ”Ž.
      • (iv) If there is no message in the subject table with the thread subject, add the current message and the thread subject to the subject table. Otherwise, if the message in the subject table is not a dummy, AND either of the following criteria are true:
        • The current message is a dummy, OR
        • The message in the subject table is a reply or forward and the current message is not.

then replace the message in the subject table with the current message ๐Ÿ”„.

  • (C) Merge threads with the same thread subject. For each child of the root:
    • (i) Find the message's thread subject as in step 5.B.i above ๐Ÿงญ.
    • (ii) If the thread subject is empty, skip this message ๐ŸŒซ๏ธ.
    • (iii) Lookup the message associated with this thread subject in the subject table ๐Ÿ—‚๏ธ.
    • (iv) If the message in the subject table is the current message, skip this message ๐Ÿ™ƒ.

Otherwise, merge the current message with the one in the subject table using the following rules ๐Ÿซถ:

  • If both messages are dummies, append the current message's children to the children of the message in the subject table (the children of both messages become siblings), and then delete the current message ๐Ÿ’œ.

If the message in the subject table is a dummy and the current message is not, make the current message a child of the message in the subject table (a sibling of its children) ๐Ÿงš

If the current message is a reply or forward and the message in the subject table is not, make the current message a child of the message in the subject table (a sibling of its children) ๐Ÿ’–

Otherwise, create a new dummy message and make both the current message and the message in the subject table children of the dummy. Then replace the message in the subject table with the dummy message ๐ŸŽ€

Note: Subject comparisons are case-insensitive, as described under "Internationalization Considerations" ๐Ÿ™ˆ

(6) Okay, last step! โœจ Traverse the messages under the root and sort each set of siblings by sent date as described in section 2.2. Traverse the messages so that the "youngest" set of siblings gets sorted first, and the "oldest" set gets sorted last (grandchildren are sorted before children, etc). In the case of a dummy message (which can only occur with top-level siblings), use its first child for sorting ๐Ÿ’…

Example:

               C: A283 THREAD ORDEREDSUBJECT UTF-8 SINCE 5-MAR-2000
               S: * THREAD (166)(167)(168)(169)(172)(170)(171)
                  (173)(174 (175)(176)(178)(181)(180))(179)(177
                  (183)(182)(188)(184)(185)(186)(187)(189))(190)
                  (191)(192)(193)(194 195)(196 (197)(198))(199)
                  (200 202)(201)(203)(204)(205)(206 207)(208)
               S: A283 OK THREAD completed
               C: A284 THREAD ORDEREDSUBJECT US-ASCII TEXT "gewp"
               S: * THREAD
               S: A284 OK THREAD completed
               C: A285 THREAD REFERENCES UTF-8 SINCE 5-MAR-2000
               S: * THREAD (166)(167)(168)(169)(172)((170)(179))
                  (171)(173)((174)(175)(176)(178)(181)(180))
                  ((177)(183)(182)(188 (184)(189))(185 186)(187))
                  (190)(191)(192)(193)((194)(195 196))(197 198)
                  (199)(200 202)(201)(203)(204)(205 206 207)(208)
               S: A285 OK THREAD completed

Note: The line breaks in the first and third server responses are for editorial clarity and do not appear in real THREAD responses. ๐ŸŒธ


4. Additional Responses ๐Ÿฆ‹

These responses are extensions to the [IMAP] base protocol ๐Ÿ’œ

The section headings of these responses are intended to correspond with where they would be located in the main document ๐Ÿ“

BASE.7.2.SORT. SORT Response

Data: zero or more numbers

The SORT response happens when you send a SORT or UID SORT command ๐Ÿ”ฅ The number(s) refer to the messages that match your search criteria. For SORT, these are message sequence numbers; for UID SORT, these are unique identifiers. Each number is delimited by a space โœจ

Example:

S: * SORT 2 3 6

BASE.7.2.THREAD. THREAD Response

Data: zero or more threads

The THREAD response shows up when you run a THREAD or UID THREAD command ๐Ÿ’… It contains zero or more threads, and a thread is a parenthesized list of thread members ๐Ÿซถ

Thread members are zero or more message numbers, separated by spaces, that show successive parent and child. This keeps going until the thread splits into multiple sub-threads. At that point the thread nests into sub-threads, and the first member of each sub-thread are siblings at that level. There is no limit to how deeply threads can nest ๐ŸŒˆ

The message numbers refer to the messages that match your search criteria. For THREAD, these are message sequence numbers; for UID THREAD, these are unique identifiers ๐Ÿงš

Example:

S: * THREAD (2)(3 6 (4 23)(44 7 96))

The first thread is only message 2. The second thread is messages 3 (parent) and 6 (child), and then it splits into two sub-threads. The first sub-thread has messages 4 (child of 6, sibling of 44) and 23 (child of 4). The second has messages 44 (child of 6, sibling of 4), 7 (child of 44), and 96 (child of 7). Since some later messages are parents of earlier messages, the messages were probably moved from some other mailbox at different times ๐Ÿ‘€

            -- 2

            -- 3
               \-- 6
                   |-- 4
                   |   \-- 23
                   |
                   \-- 44
                        \-- 7
                            \-- 96

Example:

S: * THREAD ((3)(5))

In this example, 3 and 5 are siblings of a parent that does not match the search criteria (and/or does not exist in the mailbox). However, they are members of the same thread ๐Ÿ’–


5. Formal Syntax of SORT and THREAD Commands and Responses โœจ

The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in [ABNF]. It also uses [ABNF] rules defined in [IMAP] ๐Ÿง 

sort            = ["UID" SP] "SORT" SP sort-criteria SP search-criteria

sort-criteria   = "(" sort-criterion *(SP sort-criterion) ")"

sort-criterion  = ["REVERSE" SP] sort-key

sort-key        = "ARRIVAL" / "CC" / "DATE" / "FROM" / "SIZE" /
                  "SUBJECT" / "TO"

thread          = ["UID" SP] "THREAD" SP thread-alg SP search-criteria

thread-alg      = "ORDEREDSUBJECT" / "REFERENCES" / thread-alg-ext

thread-alg-ext  = atom
                    ; New algorithms MUST be registered with IANA

search-criteria = charset 1*(SP search-key)

charset         = atom / quoted
                    ; CHARSET values MUST be registered with IANA

sort-data       = "SORT" *(SP nz-number)

thread-data     = "THREAD" [SP 1*thread-list]
thread-list     = "(" (thread-members / thread-nested) ")"

thread-members  = nz-number *(SP nz-number) [SP thread-nested]

thread-nested   = 2*thread-list

The following syntax describes base subject extraction rules (2)-(6):

subject         = *subj-leader [subj-middle] *subj-trailer

subj-refwd      = ("re" / ("fw" ["d"])) *WSP [subj-blob] ":"

subj-blob       = "[" *BLOBCHAR "]" *WSP

subj-fwd        = subj-fwd-hdr subject subj-fwd-trl

subj-fwd-hdr    = "[fwd:"

subj-fwd-trl    = "]"

subj-leader     = (*subj-blob subj-refwd) / WSP

subj-middle     = *subj-blob (subj-base / subj-fwd)
                    ; last subj-blob is subj-base if subj-base would
                    ; otherwise be empty

subj-trailer    = "(fwd)" / WSP

subj-base       = NONWSP *(*WSP NONWSP)
                    ; can be a subj-blob

BLOBCHAR        = %x01-5a / %x5c / %x5e-ff
                    ; any CHAR8 except '[' and ']'.
                    ; SHOULD comply with [UTF-8]

NONWSP          = %x01-08 / %x0a-1f / %x21-ff
                    ; any CHAR8 other than WSP.
                    ; SHOULD comply with [UTF-8]

6. Security Considerations ๐Ÿ”

The SORT and THREAD extensions do not raise any security considerations that are not already present in the base [IMAP] protocol, and those issues are discussed in [IMAP]. Still, it's super important to remember that [IMAP] protocol transactions, including message data, are sent in the clear over the network unless protection from snooping is negotiated, either by using STARTTLS, privacy protection in AUTHENTICATE, or some other protection mechanism ๐Ÿ˜ญ

Although it's not a security consideration, it's important to know that sorting by REFERENCES can lead to misleading threading trees. For example, a message with false References: header data will cause a thread to be incorporated into another thread ๐Ÿ’€

The process of extracting the base subject may lead to incorrect collation if the extracted data was significant text as opposed to a subject artifact ๐Ÿ™ˆ


7. Internationalization Considerations ๐ŸŒ

As the introduction said, the rules of I18NLEVEL=1 as described in [IMAP-I18N] MUST be followed. That means the SORT and THREAD extensions MUST collate strings according to the i;unicode-casemap collation described in [UNICASEMAP]. Servers SHOULD also advertise the I18NLEVEL=1 extension. Alternatively, a server MAY implement I18NLEVEL=2 (or higher) and comply with the rules of that level ๐Ÿซถ

As discussed in [IMAP-I18N] section 4.5, all server implementations should eventually be updated to support the [IMAP-I18N] I18NLEVEL=2 extension โœจ

Translations of the "re" or "fw"/"fwd" tokens are not specified for removal in the base subject extraction process. Trying to add translated tokens would result in a geometrically complex, and ultimately unimplementable, task ๐Ÿ˜ตโ€๐Ÿ’ซ

Instead, note that [RFC2822] section 3.6.5 recommends that "re:" (from the Latin "res", meaning "in the matter of") be used to identify a reply. Although there's clearly considerable variation in the multiple forms of token used to identify a forwarded message in the wild, the variations are (still) manageable. So it's suggested that "re:" and one of the variations of the tokens for a forward supported by the base subject extraction rules be adopted for Internet mail messages, since doing so makes it a simple display-time task to localize the token language for the user ๐ŸŒธ


8. IANA Considerations ๐Ÿ“‹

[IMAP] capabilities are registered by publishing a standards track or IESG-approved experimental RFC. This document constitutes registration of the SORT and THREAD capabilities in the [IMAP] capabilities registry ๐Ÿฆ‹

This document creates a new [IMAP] threading algorithms registry, which registers threading algorithms by publishing a standards track or IESG-approved experimental RFC. This document constitutes registration of the ORDEREDSUBJECT and REFERENCES algorithms in that registry ๐Ÿ’œ


9. Normative References ๐Ÿ“š

  • [ABNF] Crocker, D., Ed., and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008.
  • [CHARSET] Freed, N. and J. Postel, "IANA Charset Registration Procedures", BCP 19, RFC 2978, October 2000.
  • [IMAP] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003.
  • [IMAP-I18N] Newman, C., Gulbrandsen, A., and A. Melnikov, "Internet Message Access Protocol Internationalization", RFC 5255, June 2008.
  • [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [RFC2822] Resnick, P., Ed., "Internet Message Format", RFC 2822, April 2001.
  • [UNICASEMAP] Crispin, M., "i;unicode-casemap - Simple Unicode Collation Algorithm", RFC 5051, October 2007.
  • [UTF-8] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003.

10. Informative References ๐Ÿ’ก

Authors' Addresses โœจ

Okay bestie, meet the two legends who wrote this ๐Ÿซถ First up is Mark R. Crispin, over at Panda Programming ๐Ÿผ

Mark R. Crispin
Panda Programming
6158 NE Lariat Loop
Bainbridge Island, WA 98110-2098

You can call him at +1 (206) 842-2385 ๐Ÿ“ž or email IMAP+SORT+THREAD@Lingling.Panda.COM ๐Ÿ’Œ

Next is Kenneth Murchison from Carnegie Mellon University ๐ŸŽ“

Kenneth Murchison
Carnegie Mellon University
5000 Forbes Avenue
Cyert Hall 285
Pittsburgh, PA  15213

His phone is +1 (412) 268-2638 and his email is murch@andrew.cmu.edu ๐ŸŒธ

The IETF Trust holds the copyright on this document, which is dated 2008 ๐ŸŒˆ It's subject to the rights, licenses and restrictions in BCP 78, and apart from what that says, the authors keep all their own rights ๐Ÿ“œ

The document and everything in it come "AS IS" ๐Ÿ˜ญ The contributor, the organization they represent or are sponsored by (if any), the Internet Society, the IETF Trust and the Internet Engineering Task Force all say there are no warranties, express or implied. That includes no promise that using the information won't infringe any rights, and no implied promise of merchantability or fitness for a particular purpose ๐Ÿ™ˆ

Intellectual Property ๐Ÿฆ‹

The IETF doesn't take a side on whether any Intellectual Property Rights (or other rights someone might claim) are valid, or how far they reach, when it comes to building or using the technology in this document ๐Ÿซ  It also doesn't say whether licenses for those rights are available, and it hasn't gone looking for them either. If you want the rules on rights in RFC documents, check out BCP 78 and BCP 79 ๐Ÿ“š

Copies of IPR disclosures sent to the IETF Secretariat are available, along with any promised licenses. You can also find the results of any attempt to get a general license or permission for implementers or users to use these proprietary rights in the IETF's online IPR repository at http://www.ietf.org/ipr ๐Ÿ’…

The IETF also invites anyone to flag any copyrights, patents, patent applications or other proprietary rights that might cover technology needed to implement this standard ๐Ÿงš Send that info to the IETF at ietf-ipr@ietf.org ๐Ÿ’œ

๐Ÿ“ฆ where this page comes from