RFC 1082

Post Office Protocol: Version 3: Extended service offerings

๐Ÿ“… published November 1988, from ietf

RFC 1082 ๐ŸŽ€ Post Office Protocol - Version 3 Extended Service Offerings

Network Working Group โœจ

  • RFC number: 1082
  • Title: Post Office Protocol - Version 3, Extended Service Offerings
  • Author: M. Rose, from TWG ๐Ÿ’…
  • Date: November 1988
  • Category, obsoletes, updates: not listed in this copy of the title block ๐Ÿ™ˆ

Status of This Memo ๐Ÿ“œ

okay so this memo suggests a simple way for workstations to dynamically access mail from a discussion group server ๐Ÿซถ It's basically an add-on to an earlier memo about dynamically accessing mail from a mailbox server using the Post Office Protocol - Version 3 (POP3). This RFC is a proposed protocol for the Internet community, and it's literally asking people to discuss it and send in suggestions for improvements ๐Ÿ’ฌ All of the extensions described in this memo to the POP3 are OPTIONAL. Distribution of this memo is unlimited ๐ŸŒˆ


Introduction and Motivation ๐Ÿงš

bestie, it's assumed you already know RFC 1081, which covers the Post Office Protocol - Version 3 (POP3) [RFC1081] ๐Ÿ“– This memo adds extensions to POP3 that level up the service for clients. The new bit lets a client host reach discussion group mail, which usually sits in its own separate spool area, using the regular POP3 facilities ๐Ÿฆ‹

The next section is the evolution of discussion groups and the tech people use to run them. The quick version:

  • An exploder maps one address to a list of addresses that subscribe to the list, and it redirects any later error reports from delivering each message. This is a big deal for two reasons:
    • Subscribers only need to know one address ๐Ÿ’–
    • The responsible parties get the error reports, not the subscribers ๐Ÿ‘‘
  • Usually each subscription address isn't someone's private maildrop. It's a system-wide maildrop that more than one user can access. This is iconic for several reasons:
    • Only a single copy of each message needs to cross the net for a given site (which might contain several local hosts). That saves bandwidth and cycles โšก
    • Only a single copy of each message needs to sit on each subscribing host. That saves disk space ๐Ÿ’พ
    • Each user's private maildrop stays uncluttered with discussion group mail ๐Ÿงน

Even with all that optimization, sites with more than one host can still save more. Usually those sites do one of two things:

  1. They replicate discussion group mail on each host. This means literally gigabytes of disk space go to storing redundant information ๐Ÿ˜ญ
  2. They keep discussion group mail on one host and give every user a login on that host, on top of whatever other logins they have. This is usually a gross inconvenience for people who work on other hosts, or a burden for people who are stuck working on that host ๐Ÿ’€

As [RFC1081] explains, giving workstations dynamic access to mail from a mailbox server has been explored in serious detail. Originally there was [RFC918]. That prompted the author to write [RFC1081], and independently of that, [RFC918] was upgraded to [RFC937] ๐ŸŒธ A natural fix for the problem above is to keep discussion group mail on a mailbox server at each site, then let the different hosts at that site use POP3 to access it. If it's done properly, this dodges the problems of both strategies above โœจ

ASIDE ๐Ÿ‘€: A good distributed filesystem could also solve this problem. Sadly, "good" distributed filesystems, the ones that don't suffer unacceptable response time for interactive use, are few and far between these days! ๐Ÿ™ƒ

Given all that, we're going to look at discussion groups, both in general and from the point of view of a user agent ๐Ÿ” After that, we'll cover the POP3 extensions defined in [RFC1081]. Finally, we'll talk about some extra policy details and some early experiences ๐ŸŽ€


What's in a Discussion Group ๐Ÿ’ฌ

Ever since mailers and user agents first crawled out of the primordial ARPAnet, people have appreciated discussion groups ๐Ÿฅฐ (even though how they're implemented hasn't always been well understood).

Simply put, a discussion group is a bunch of subscribers who share a common interest. They post mail to a single address, called the distribution address. From there, a copy of each message goes out to every subscriber ๐Ÿ“ฌ Each group has a moderator, the person who administers the group. The moderator can usually be reached at a special address called the request address. Usually the moderator's job is pretty chill, since the mail system handles distribution to subscribers automatically ๐Ÿ˜Œ Sometimes, instead of being sent straight to subscribers, the group's content gets put into a digest format by the moderator first. That's more work for the moderator, but these groups tend to be better organized ๐Ÿ“š

Unfortunately, the setup above has some problems. First, if two users on the same host subscribe to the same interest group, two copies of each message get delivered. That's wasteful of both processor and disk resources ๐Ÿ™…โ€โ™€๏ธ

Second, some of these groups carry a LOT of traffic. Subscribing does show interest, but it's usually not fun to get 50 or so messages a day dumped into your private maildrop, mixed in with personal mail that's likely more important and timely ๐Ÿ˜ฉ

Third, if a subscriber on a group's distribution list goes "bad" somehow, the notification goes to the originator of the message, not the moderator. It's not uncommon for a big list to have 10 or so bogus addresses. That floods the originator with "error messages" from mailers across the Internet saying a given address on the list was bad. Honestly, the originator usually couldn't care less whether the bogus addresses got a copy. They just want to post to the group at large ๐Ÿคทโ€โ™€๏ธ Meanwhile, the moderator actually does care whether there are bogus addresses on the list, but ironically never gets the notification ๐Ÿ™ƒ

There are various ways to solve some or all of these problems. Usually they involve putting an exploder agent at the distribution source of the discussion group. The exploder expands the group's name into the list of subscription addresses for the group. In the process, the exploder also changes the address that receives error notifications so it points to the request address or another responsible party ๐ŸŽฏ

A complementary approach cuts down on resource use of all kinds. It replaces all the subscribers at a single host (or a group of hosts under one administration) with a single address on that host. That address maps to a file on the host, usually in a spool area, which all users can access. (Advanced implementations can also do private discussion groups this way, where a single copy of each message is kept but only a select number of users on the host can access it ๐Ÿ”)

You can combine the two approaches to avoid all of the problems described above ๐Ÿ’…

A third approach helps user agents process mail for the discussion group. To speed up querying the maildrop that holds the local host's copy of the discussion group, two other items are usually associated with the group on a local basis: the maxima and the last-date ๐Ÿงฎ Each time a message is received for the group on the local host, the maxima goes up by at least one. Whenever a new maxima is generated, the current date is determined. That date is called the last date. As the message enters the local maildrop, it's stamped with the current maxima and last-date. This lets the user agent quickly figure out whether new messages are in the maildrop โšก

NOTE ๐Ÿ“: The maxima can be characterized as a monotonically increasing quantity. Although successive values of the maxima need not be consecutive, any maxima assigned is always greater than any previously assigned value. ๐Ÿซถ

Definition of Terms โœจ

okay bestie, before we get into the commands we need to agree on the vocab ๐Ÿซถ this section describes a discussion group from the user agent's side using 7 parameters ๐Ÿ’– (the syntax notes come from [RFC822]):

      NAME            Meaning: the name of the discussion group
                      Syntax:  TOKEN (ALPHA *[ ALPHA / DIGIT / "-" ])
                               (case-insensitive recognition)
                      Example: unix-wizards

      ALIASES         Meaning: alternates names for the group, which
                               are locally meaningful; these are
                               typically used to shorten user typein
                      Syntax:  TOKEN (case-insensitive recognition)
                      Example: uwiz

      ADDRESS         Meaning: the primary source of the group
                      Syntax:  822 address
                      Example: Unix-Wizards@BRL.MIL

      REQUEST         Meaning: the primary moderator of the group
                      Syntax:  822 address
                      Example: Unix-Wizards-Request@BRL.MIL

      FLAGS           Meaning: locally meaningful flags associated
                               with the discussion group; this memo
                               leaves interpretation of this
                               parameter to each POP3 implementation
                      Syntax:  octal number
                      Example: 01

      MAXIMA          Meaning: the magic cookie associated with the
                               last message locally received for the
                               group; it is the property of the magic
                               cookie that it's value NEVER
                               decreases, and increases by at least
                               one each time a message is locally
                               received
                      Syntax:  decimal number
                      Example: 1004

      LASTDATE        Meaning: the date that the last message was
                               locally received
                      Syntax:  822 date
                      Example: Thu, 19 Dec 85 10:26:48 -0800

ok so the last two values (MAXIMA and LASTDATE) are determined locally, for the maildrop attached to the discussion group and for each message in that maildrop ๐Ÿ“ฌ. the last message in the maildrop has a different MAXIMA and LASTDATE than the discussion group does. this often happens when the maildrop has been archived ๐Ÿ—„๏ธ

finally, some local systems automatically archive discussion group mail ๐Ÿงš sometimes there's a two-level setup: current mail stays in the standard maildrop, recent mail lives in an archive maildrop, and older mail is kept off-line. so on top of the standard maildrop for each group, an "archive" maildrop may also be available ๐Ÿ’œ this lets a user agent check the most recent archive using the same mechanisms it uses on current mail โœจ


The XTND Command ๐Ÿ’…

the following commands are valid only in the TRANSACTION state of the POP3. this implies the POP3 server has already opened the user's maildrop (which may be empty) ๐ŸŒธ that maildrop is called the "default maildrop." the phrase "closes the current maildrop" has two meanings, depending on whether the current maildrop is the default maildrop or a maildrop tied to a discussion group ๐Ÿ™ˆ

in the first case, when the current maildrop gets closed, any messages marked as deleted are removed from the maildrop currently in use. then the exclusive-access lock on the maildrop is released, along with any implementation-specific resources (e.g., file-descriptors) ๐Ÿ”ฅ

in the second case, a maildrop tied to a discussion group is considered read-only to the POP3 client ๐Ÿซง so "closes the current maildrop" just means any implementation-specific resources get released. (hence, the POP3 command DELE is a no-op ๐Ÿ’€)

all the new facilities are introduced through a single POP3 command, XTND ๐Ÿฆ‹ all positive responses to the XTND command are multi-line. the most common multi-line response contains a "discussion group listing," which shows the name of the group along with its maxima ๐Ÿ“ to keep parsing simple, all POP3 servers are required to use a certain format for discussion group listings:

NAME SP MAXIMA

this memo makes no requirement on what follows the maxima in the listing. minimal implementations should just end that line of the response with a CRLF pair โœจ more advanced implementations may include other information, as parsed from the message.

NOTE: This memo STRONGLY discourages implementations from supplying additional information in the listing. ๐Ÿšซ

XTND BBOARDS [name]

Arguments: the name of a discussion group (optionally) ๐ŸŽ€ Restrictions: may only be given in the TRANSACTION state. Discussion:

if an argument was given, the POP3 server closes the current maildrop. then it validates the argument as the name of a discussion group. if that works, it opens the maildrop for the group and returns a multi-line response containing the discussion group listing ๐ŸŒˆ if the group name isn't valid, or the associated archive maildrop isn't readable by the user, you get an error response ๐Ÿ™…โ€โ™€๏ธ

if no argument was given, the POP3 server issues a multi-line response. after the initial +OK, the server responds with a line for each known discussion group, containing that group's listing ๐Ÿง only world-readable discussion groups are included in the multi-line response ๐Ÿ‘‘

in order to aid user agents, this memo requires an extension to the scan listing when an "XTND BBOARDS" command has been given ๐Ÿซถ normally, a scan listing (as generated by the LIST command) looks like this:

MSGNO SIZE

where MSGNO is the number of the message being listed, and SIZE is the size of the message in octets ๐Ÿ”ข when reading a maildrop accessed via "XTND BBOARDS," the scan listing looks like this:

MSGNO SIZE MAXIMA

where MAXIMA is the maxima that was assigned to the message when it was placed in the BBoard โœจ

Possible Responses:

+OK XTND
-ERR no such bboard

Examples:

C:    XTND BBOARDS
S:    +OK XTND
S:    system 10
S:    mh-users 100
S:    .
C:    XTND BBOARDS system
S:    + OK XTND
S:    system 10
S:    .

XTND ARCHIVE name

Arguments: the name of a discussion group (required) ๐Ÿ’– Restrictions: may only be given in the TRANSACTION state. Discussion:

the POP3 server closes the current maildrop. then it validates the argument as the name of a discussion group. if that's successful, it opens the archive maildrop associated with the group and returns a multi-line response containing the discussion group listing ๐Ÿ“š if the group name isn't valid, or the associated archive maildrop isn't readable by the user, an error response comes back ๐Ÿ˜ญ

in addition, the scan listing generated by the LIST command is augmented (as described above) ๐ŸŒŸ

Possible Responses:

+OK XTND
-ERR no such bboard Examples:

Examples:

C:    XTND ARCHIVE system
S:    + OK XTND
S:    system 3
S:    .

XTND X-BBOARDS name

Arguments: the name of a discussion group (required) ๐Ÿฆ„ Restrictions: may only be given in the TRANSACTION state. Discussion:

the POP3 server validates the argument as the name of a discussion group. if that's unsuccessful, an error response is returned ๐Ÿ™ƒ otherwise, a multi-line response is returned. the first 14 lines of this response (after the initial +OK) are defined in this memo ๐Ÿ—‚๏ธ minimal implementations need not include other information (and may omit certain information, outputting a bare CRLF pair). more advanced implementations may include other information ๐ŸŒท

here's the line-by-line breakdown ๐Ÿ’…

Line    Information (refer to "Definition of Terms")
----    -----------
  1     NAME
  2     ALIASES, separated by SP
  3     system-specific: maildrop
  4     system-specific: archive maildrop
  5     system-specific: information
  6     system-specific: maildrop map
  7     system-specific: encrypted password
  8     system-specific: local leaders, separated by SP
  9     ADDRESS
 10     REQUEST
 11     system-specific: incoming feed
 12     system-specific: outgoing feeds
 13     FLAGS SP MAXIMA
 14     LASTDATE

most of this info is entirely too specific to the UCI Version of the Rand MH Message Handling System [MRose85] ๐Ÿคทโ€โ™€๏ธ but lines 1, 2, 9, 10, 13, and 14 are of general interest, regardless of the implementation ๐Ÿญ

Possible Responses:

+OK XTND
-ERR no such bboard

Examples:

C:    XTND X-BBOARDS system
S:    + OK XTND
S:    system
S:    local general
S:    /usr/bboards/system.mbox
S:    /usr/bboards/archive/system.mbox
S:    /usr/bboards/.system.cnt
S:    /usr/bboards/.system.map
S:    *
S:    mother
S:    system@nrtc.northrop.com
S:    system-request@nrtc.northrop.com
S:
S:    dist-system@nrtc-gremlin.northrop.com
S:    01 10
S:    Thu, 19 Dec 85 00:08:49 -0800
S:    .

Policy Notes ๐ŸŒˆ

depending on the entity running the POP3 service host, two extra policies might be implemented ๐Ÿงšโ€โ™€๏ธ

1. Private Discussion Groups

in the general case, discussion groups are world-readable ๐ŸŒ any user who's logged in (via a terminal, terminal server, or POP3, etc.) can read the maildrop for each discussion group known to the POP3 service host. still, it's often desirable (usually for privacy reasons) to have private discussion groups too ๐Ÿ”’

supporting this is consistent with the extensions outlined in this memo ๐ŸŽ€ once the AUTHORIZATION state has successfully concluded, the POP3 server grants the user access to exactly those discussion groups the POP3 service host permits the authenticated user to access. as a "security" feature, discussion groups associated with unreadable maildrops should not be listed in a positive response to the XTND BBOARDS command ๐Ÿ™ˆโœจ

2. Anonymous POP3 Users

to minimize the authentication problem, a policy permitting "anonymous" access to the world-readable maildrops for discussion groups on the POP3 server may be implemented ๐Ÿซถ

supporting this is consistent with the extensions outlined in this memo ๐Ÿ’œ the POP3 server can be modified to accept a USER command for a well-known pseudonym (i.e., "anonymous") that's valid with any PASS command. as a "security" feature, it is advisable to limit this kind of access to only hosts at the local site, or to hosts named in an access list ๐Ÿ”๐Ÿฆ‹

Experiences and Conclusions ๐Ÿง

okay bestie, the good stuff!! ๐ŸŽ€ every facility described in this memo and in [RFC1081] has been implemented in MH #6.1, and the first experiences have been, on the whole, very positive ๐Ÿ’–

after that first implementation, some performance tuning was needed ๐Ÿ”ง mostly that meant caching the datastructures which describe discussion groups in the POP3 server. the second optimization was on the client side ๐Ÿ’ป: the program most commonly used to read BBoards in MH got modified so it only retrieves messages when they're actually needed ๐Ÿฆ‹ there are two schemes:

  • if only the headers (and the first few lines of the body) of the message are required (e.g., for a scan listing), then only these are retrieved. the resulting output is then cached, on a per-message basis. ๐Ÿซถ
  • if the entire message is required, then it is retrieved intact, and cached locally. ๐Ÿ“ฆ

with these optimizations, response time is quite adequate when the POP3 server and client are connected via a high-speed local area network ๐Ÿš€ the author even uses this mechanism to access certain private discussion groups over the Internet, and the response is still good ๐Ÿ™Œ but when a 9.6Kbps modem is inserted in the path, response went from good to almost tolerable ๐Ÿ™ˆ (fortunately the author only reads a few discussion groups this way ๐Ÿ˜…)

to conclude: the POP3 is a good thing, not only for personal mail but for discussion group mail as well ๐Ÿ‘‘๐ŸŒˆ


References ๐Ÿ“š

[RFC1081] Rose, M., "Post Office Protocol - Verison 3 (POP3)", RFC
          1081, TWG, November 1988.

[MRose85] Rose, M., and J. Romine, "The Rand MH Message Handling
          System: User's Manual", University of California, Irvine,
          November 1985.

[RFC822]  Crocker, D., "Standard for the Format of ARPA-Internet
          Text Messages", RFC 822, University of Delaware, August
          1982.

[RFC918]  Reynolds, J., "Post Office Protocol", RFC 918,
          USC/Information Sciences Institute, October 1984.

[RFC937]  Butler, M., J. Postel, D. Chase, J. Goldberger, and J.
          Reynolds, "Post Office Protocol - Version 2", RFC 937,
          USC/Information Sciences Institute, February 1985.

Author's Address ๐Ÿ’Œ

Marshall Rose
The Wollongong Group
1129 San Antonio Rd.
Palo Alto, California 94303

Phone: (415) 962-7100

Email: MRose@TWG.COM
๐Ÿ“ฆ where this page comes from