RFC 1849
"Son of 1036": News Article Format and Transmission
RFC 1849: "Son of 1036" 🎀
Part 1 of 24 ✨
This is RFC 1849, titled "Son of 1036": News Article Format and Transmission 💖. H. Spencer of SP Systems wrote it, it's dated March 2010, and it's filed as Historic 📜. It came in through the Independent Submission stream (ISSN 2070-1721), and it's been obsoleted by 5536 and 5537 🫠.
Abstract ✨
Okay bestie, by the early 1990s everyone could see that RFC 1036 (the spec for the Interchange of USENET Messages) needed a serious glow-up 🩹💅. This "Internet-Draft-to-be" never got formally published back then, but it got passed around SO much that it became the de facto standard for implementors of News Servers and User Agents 🌸. It picked up the nickname "Son of 1036" ✨. Honestly, under that name it's probably the best-known Internet Draft (n)ever published 👑, and it became the starting point for the recently adopted Proposed Standards for Netnews 🦋.
It's being published now so we can see the historical background those standards grew out of 📚. Present-day implementors should know that it is NOT NOW APPROPRIATE for use in current implementations 🙈.
Status of This Memo 📜
Heads up: this document is NOT an Internet Standards Track specification. It's published for the historical record 🕰️.
It defines a Historic Document for the Internet community. It's a contribution to the RFC Series, independent of any other RFC stream. The RFC Editor chose to publish it at its discretion and makes no statement about its value for implementation or deployment. Documents approved for publication by the RFC Editor are not a candidate for any level of Internet Standard; see Section 2 of RFC 5741 💅.
For info on its current status, any errata, and how to give feedback, check http://www.rfc-editor.org/info/rfc1849 🔗.
Copyright Notice ⚖️
Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the publication date. Please actually read these, bestie, because they spell out your rights and restrictions for this document 🫶.
No editing this one 🚫✏️. Nobody may modify it or create derivative works of it, except to format it for publication as an RFC or to translate it into languages other than English.
Table of Contents 📚
Preface ............................................................5
Original Abstract ..................................................6
1. Introduction ....................................................6
2. Definitions, Notations, and Conventions .........................8
2.1. Textual Notations ..........................................8
2.2. Syntax Notation ............................................9
2.3. Definitions ...............................................10
2.4. End-of-Line ...............................................13
2.5. Case-Sensitivity ..........................................13
2.6. Language ..................................................13
3. Relation to MAIL (RFC822, etc.) ................................14
4. Basic Format ...................................................15
4.1. Overall Syntax ............................................15
4.2. Headers ...................................................16
4.2.1. Names and Contents .................................16
4.2.2. Undesirable Headers ................................18
4.2.3. White Space and Continuations ......................18
4.3. Body ......................................................19
4.3.1. Body Format Issues .................................19
4.3.2. Body Conventions ...................................20
4.4. Characters and Character Sets .............................23
4.5. Non-ASCII Characters in Headers ...........................26
4.6. Size Limits ...............................................28
4.7. Example ...................................................30
5. Mandatory Headers ..............................................30
5.1. Date ......................................................31
5.2. From ......................................................33
5.3. Message-ID ................................................35
5.4. Subject ...................................................36
5.5. Newsgroups ................................................38
5.6. Path ......................................................42
6. Optional Headers ...............................................45
6.1. Followup-To ...............................................45
6.2. Expires ...................................................46
6.3. Reply-To ..................................................47
6.4. Sender ....................................................47
6.5. References ................................................48
6.6. Control ...................................................50
6.7. Distribution ..............................................51
6.8. Keywords ..................................................52
6.9. Summary ...................................................53
6.10. Approved .................................................53
6.11. Lines ....................................................54
6.12. Xref .....................................................55
6.13. Organization .............................................56
6.14. Supersedes ...............................................57
6.15. Also-Control .............................................57
6.16. See-Also .................................................58
6.17. Article-Names ............................................58
6.18. Article-Updates ..........................................60
7. Control Messages ...............................................60
7.1. cancel ....................................................61
7.2. ihave, sendme .............................................64
7.3. newgroup ..................................................66
7.4. rmgroup ...................................................68
7.5. sendsys, version, whogets .................................68
7.6. checkgroups ...............................................73
8. Transmission Formats ...........................................74
8.1. Batches ...................................................74
8.2. Encoded Batches ...........................................75
8.3. News within Mail ..........................................76
8.4. Partial Batches ...........................................77
9. Propagation and Processing .....................................77
9.1. Relayer General Issues ....................................78
9.2. Article Acceptance and Propagation ........................80
9.3. Administrator Contact .....................................82
10. Gatewaying ....................................................83
10.1. General Gatewaying Issues ................................83
10.2. Header Synthesis .........................................85
10.3. Message ID Mapping .......................................86
10.4. Mail to and from News ....................................88
10.5. Gateway Administration ...................................89
11. Security and Related Issues ...................................90
11.1. Leakage ..................................................90
11.2. Attacks ..................................................91
11.3. Anarchy ..................................................92
11.4. Liability ................................................92
12. References ....................................................93
Appendix A. Archaeological Notes ..................................96
A.1. "A News" Article Format ...................................96
A.2. Early "B News" Article Format .............................96
A.3. Obsolete Headers ..........................................97
A.4. Obsolete Control Messages .................................97
Appendix B. A Quick Tour of MIME ..................................98
Appendix C. Summary of Changes Since RFC 1036 ....................103
Appendix D. Summary of Completely New Features ...................104
Appendix E. Summary of Differences from RFCs 822 and 1123.........105Preface 🌸
Although [RFC1036] was published in 1987, for many years it was basically the only formally published specification for Netnews format and processing 📰. It was widely considered obsolete within a few years, and it has now been superseded by the work of the USEFOR Working Group, which led to the publication of [RFC5536] and [RFC5537]. BUT there was an intermediate step that's of some historical interest 🕰️.
In 1993-4, Henry Spencer wrote and informally circulated a document that became known as "Son of 1036", meant as a first draft of a replacement for [RFC1036]. It went no further at the time (although, more recently, the USEFOR Working Group started from it). It still saw considerable use as a technical reference and even a de facto standard, despite its informal status 💅.
The USEFOR work has eliminated any further relevance of Son of 1036 as a technical reference, but it remains of historical interest 🧚. The USEFOR Working Group asked that it be published as an Historic RFC, to ensure its preservation in an accessible form and to make it easier to reference 📜.
This document is identical to the last distributed version of Son of 1036, dated 2 June 1994, except for reformatting, correction of a few minor factual or formatting errors, completion of the then-empty Appendix D and of the References section, minor editing to match preferred RFC style, and changes to leading and trailing material. Remarks enclosed within {...} indicate explanatory material that wasn't in the original version. References to the current MIME standards (and a few others) have been added, since that was an unresolved issue in 1994 🎀.
The technical content remains unchanged, including the references to the document itself as a Draft rather than an RFC and the presence of unresolved issues. The original section numbering has been preserved, although the original pagination hasn't (among other reasons, it didn't fully follow IETF formatting standards) 🙈.
READERS ARE CAUTIONED THAT THIS DOCUMENT IS OBSOLETE AND SHOULD NOT BE USED AS A TECHNICAL REFERENCE. 🚨
Although Son of 1036 largely documented existing practice, it also proposed some changes, and some of them didn't catch on or are no longer considered good ideas. (Of particular note, the MIME type message/news should not be used.) Consult [RFC5536] and [RFC5537] for modern technical information 💜.
Although a number of people contributed useful comments or criticism during the preparation of this document, its contents are entirely the opinions of the author circa 1994. Not even the author himself agrees with them all now 😭.
The author thanks Charles Lindsey for his assistance in getting this document cleaned up and formally published at last (not least, for supplying some prodding to actually get it done!) 🫶.
The author thanks Luc Rooijakkers for supplying the MIME summary that Appendix B is based on 🦋.
1. Introduction ✨
Okay bestie, so network news articles are basically mail messages that get broadcast to potentially huge crowds 📣. They use a flooding algorithm that sends one copy to each interested host (or group of them) 🦋, usually keeps just one copy per host 💅, and doesn't need any central admin or formal sign-up for the people who are into it 🌸. Network news started as the chat medium for Usenet around 1980 🎀. Since then Usenet has blown up, and loads of Internet sites are in it 😭. The news tech is also everywhere now for other stuff, on the Internet and beyond 🌈.
The earliest news interchange used the so-called "A News" article format. Shortly after, an article format kinda like Internet mail was made and used briefly. Both of those are totally obsolete, so they're only in Appendix A for historical reasons 🧚. With [RFC850] in 1983, news articles started looking a lot like Internet mail messages, with some restrictions and some extra headers. In 1987, [RFC1036] updated [RFC850] without making big changes.
Over those five years in between, the [RFC1036] article format has honestly been pretty solid 💖. Minor extensions still look desirable to keep up with recent developments in areas like multi-media mail 📱.
[RFC1036] itself? Not so much 😬. It's kinda vague and skips some issues entirely, which has caused real interoperability problems at times, and implementations have drifted apart a bit. Worse, even though it was meant to document what people were already doing, it didn't exactly match existing practice even when it came out, and the gaps have only gotten bigger since.
This Draft tries to specify the format of articles, plus the procedures for exchanging and processing them, in enough detail to allow full interoperability 🫶. It also makes some tentative suggestions about future directions, to avoid unnecessary divergence and the interoperability loss that comes with it. Major extensions (e.g., cryptographic authentication) that need significant development effort are left to be undertaken as independent efforts 🔐.
NOTE: You might be wondering why there's no News-Version header, like MIME-Version, to say which version of this spec you're on 🤔. Here's the tea: it doesn't seem useful, because news has to stay backward-compatible. A version number is mainly there to tell apart several INCOMPATIBLE interpretations, and nobody can coordinate a simultaneous switch across news's whole installed base. So incompatible changes (as opposed to extensions) have to be avoided entirely. MIME got a version number mostly because it made incompatible changes to how several "Content-" headers get interpreted. This Draft makes no changes in interpretation, and it looks doubtful that future Drafts will manage to introduce any 🙈.
UNRESOLVED ISSUE: Should this be reconsidered? Honestly, only if the header has SPECIFIC IDENTIFIABLE uses today 🤷. Otherwise it's just useless added bulk 🗑️.
Same as the earlier Drafts, this one doesn't pin down the exact way articles get moved from one host to another 🚚. Network News Transfer Protocol (NNTP) [RFC977] {since replaced by [RFC3977]} is probably the most popular transmission method on the Internet. Plenty of others are still in use, though, like the Unix-To-Unix Copy Protocol [UUCP], which was everywhere in Usenet's early days and is still pretty big on the fringes 🌿.
Ngl, a few of the mechanisms in here might look kinda strange or even bizarre on first read 😭. Just like Internet mail, there's no realistic way to update the entire installed base of news software quickly, so working with old software is crucial and always will be 🫶. Compatibility with existing practice and robustness in an imperfect world necessarily come before elegance, periodt 💅.
2. Definitions, Notations, and Conventions 📚
2.1. Textual Notations
All through this Draft, "MAIL" is shorthand for "[RFC822] as amended by [RFC1123]" 💌. ([RFC1123]'s amendments are mostly pretty small, but they are not insignificant, so don't sleep on them 👀.) Check out the discussion in Section 3 about how this Draft relates to MAIL too. "MIME" is shorthand for "[RFC1341] and [RFC1342]" (or their {since} updated replacements {[RFC2045], [RFC2046], and [RFC2047]}).
UNRESOLVED ISSUE: Update these numbers {now resolved!} ✅
{NOTE: Since this Draft was first published, [RFC822] has gotten updates, first to [RFC2822] and then more recently to [RFC5322]. Still, this Draft is firmly rooted in the original [RFC822] 🌳. Similarly, [RFC821] has also gotten two upgrades in the meantime.}
"ASCII" is shorthand for "the ANSI X3.4 character set" [X3.4] 🔤. People toss the word "ASCII" around for all sorts of character sets that are kinda similar to X3.4. But in this Draft, "ASCII" means [X3.4] and only [X3.4], no exceptions 💀.
NOTE: The name is super traditional (so much that the ANSI standard itself sanctions it 🙃), even though it's no longer an acronym for the name of the standard.
NOTE: ASCII, X3.4, has 128 characters, and not all of them are printable 🖨️. Character sets with more characters aren't ASCII, even though they might include it as a subset.
Certain words that define how much a requirement matters are capitalized 📏. "MUST" means the item is an absolute requirement of the specification 🚨. "SHOULD" means the item is a strong recommendation. There may be valid reasons to ignore it in unusual circumstances, but you should only do that after careful study of the full implications and a firm conclusion that it's necessary, because there are serious disadvantages to doing so 😬. "MAY" means the item is truly optional, and implementors and users are warned that conformance is possible but not something to rely on 🤷.
So "compliant" (said about implementations and so on) means you've met all the relevant "MUST" and "SHOULD" requirements 🎀. "Conditionally compliant" means you've met all the relevant "MUST" requirements but broken at least one relevant "SHOULD" requirement 🥲.
This Draft has explanatory notes formatted like this, and you can totally skip them if you only care about the actual spec 🙈. The notes are there to explain why choices were made, to put things in context, or to suggest possible implementation techniques 🧠.
NOTE: Yeah, explanatory notes might seem extra in theory 🙄, but they really help the less-than-omniscient reader (aka all of us 😭) get the point of the spec and the constraints involved. Natural language is kinda limited for describing stuff, so notes make it more likely that implementors and users get the true intent when the wording isn't totally clear 💜.
All numbers are in decimal unless it says otherwise 🔢. For this purpose, octets are treated as unsigned values 🧮. Big numbers use the North American convention, where "," splits up groups of three digits but doesn't mean anything else 🙌.
2.2. Syntax Notation
Even though everything in this Draft is described in prose, most of it is also described formally in the modified BNF notation from [RFC822] 📐. Implementors need to know this notation to fully get the spec, so go check [RFC822] for the full explanation of modified BNF. Here's a quick example:
sentence = clause *( punct clause ) "."
punct = ":" / ";"
clause = 1*word [ "(" clause ")" / "," 1*word ]
word = <any English word>So this says a sentence is some clauses separated by puncts and ended with a period 📝. A punct is a colon or a semicolon. A clause is at least one <word>, optionally followed by either a parenthesized clause or a comma and at least one more <word>. And a <word> is (informally) any English word 🦋. The characters <> only get used to wrap names when that actually helps tell them apart from the surrounding text 💅.
The full form of the repetition notation is <m>*<n><thing>, meaning <m> through <n> repetitions of <thing>. <m> defaults to zero and <n> to infinity, and the * and <n> can be left out if <m> and <n> are equal. So 1*word is one or more words, 1*5word is one through five words, and 2word is exactly two words ✨.
Fun fact: the character \ isn't special in any way in this notation 🙃.
This Draft is meant to stand on its own 🧍. Every syntax rule it uses is defined right inside it, and a rule with the same name as one in MAIL doesn't have to mean the same thing ⚠️. The lexical layer of MAIL is NOT, repeat NOT, used in this Draft, and its presence must not be assumed 🙅. Basically, this Draft spells out every place where white space is permitted/required, and every place where stuff that looks like MAIL comments can show up 🎀.
NOTE: Real talk, news parsers have historically been way less forgiving than MAIL parsers 😤.
Sooo that's the groundwork 🌸, and the good stuff is coming up next! 💖
2.3. Definitions ✨
Okay, first up: whenever this Draft says "character set," it means a coded character set, in the sense of ISO character set standardization work. It is NOT to be mistaken for "a set of characters" 🙈 iykyk, those are two very different things.
In this Draft, ASCII character 32 is called "blank," because "space" has a more generic meaning 🫶
An "article" is the unit of news, kinda like a MAIL "message" 💌
A "poster" is a human (or software that acts like one) who submits a possibly compliant article so it can be "posted," meaning made available for reading on all the relevant hosts 📮 A "posting agent" is the software that helps posters write articles. It figures out whether the final article is compliant, passes it along to a relayer for posting if it is, and sends it back to the poster with an explanation if it isn't 💅 A "relayer" is software that gets allegedly compliant articles from posting agents and/or other relayers, files copies in a "news database," and possibly passes copies on to other relayers 🔁
NOTE: While the same software may well function both as a relayer and as part of a posting agent, the two functions are distinct and should not be confused. The posting agent's purpose is (in part) to validate an article, supply header information that can or should be supplied automatically, and generally take reasonable actions in an attempt to transform the poster's submission into a compliant article. The relayer's purpose is to move already-compliant articles around efficiently without damaging them.
A "reader" is a human reading news articles 👀 A "reading agent" is software that presents articles to a reader 📖
NOTE: Informal usage often uses "reader" for both these meanings, but this introduces considerable potential for confusion and misunderstanding, so this Draft takes care to make the distinction.
A "newsgroup" is a single news forum, a logical bulletin board, with a name, nominally intended for articles on one specific topic 📌 An article is "posted to" a single newsgroup or several newsgroups. When an article is posted to more than one newsgroup, it's called "cross-posted." Note that this is different from posting the same text as part of each of several separate articles, one per newsgroup 👯 A "hierarchy" is the set of all newsgroups whose names share a first component (see the name syntax in Section 5.5) 🌳
A newsgroup may be "moderated," in which case submissions aren't posted directly. Instead, they're mailed to a "moderator" for consideration and possible posting. Moderators are typically human, but they can be partly or entirely software 🤖
A "followup" is an article containing a response to the contents of an earlier article (the followup's "precursor") 💬 A "followup agent" is a combination of reading agent and posting agent that helps prepare and post a followup ✨
Text comparisons are "case-sensitive" if they treat uppercase letters (like "A") as different from lowercase letters (like "a"), and "case-insensitive" if letters that differ only in case (like "A" and "a") are treated as identical. Categories of text are said to be case-(in)sensitive if comparisons of such texts to others are case-(in)sensitive 🔠
A "cooperating subnet" is a set of news-exchanging hosts that is well-coordinated enough (usually via some kind of central administration) that you can make stronger assumptions about the hosts in the set than about news hosts in general 🤝 This is typically used to relax restrictions that are otherwise required for worst-case interoperability. Members of a cooperating subnet MAY interchange articles that do not conform to this Draft's specifications, provided all members have agreed to this and provided the articles are not permitted to leak out of the subnet. The word "subnet" is there to emphasize that a cooperating subnet is typically not an isolated universe, so you've gotta be careful that traffic leaving the subnet complies with the restrictions of the larger net, not just those of the cooperating subnet 🌍
A "message ID" is a unique identifier for an article, usually supplied by the posting agent that posted it 🆔 It tells the article apart from every other article ever posted anywhere (in theory). Articles with the same message ID are treated as identical copies of the same article, even if they aren't actually identical 👯♀️
A "gateway" is software that receives news articles and converts them into messages of some other kind (for example, mail to a mailing list), or vice versa. It's basically a translating relayer that straddles the boundaries between different methods of message exchange 🌉 The most common type connects newsgroup(s) to mailing list(s), either unidirectionally or bidirectionally, but there are also gateways between news networks using this Draft's news format and those using other formats.
A "control message" is an article that is marked as containing control information. A relayer receiving such an article will (subject to permissions, etc.) take actions beyond just filing and passing on the article 🎛️
NOTE: "Control article" would be more consistent terminology, but "control message" is already well established.
An article's "reply address" is the address that mailed replies should be sent to 💌 This is the address specified in the article's From header (see Section 5.2), unless it also has a Reply-To header (see Section 6.3).
The notation (for example) "(ASCII 17)" following a name means "this name refers to the ASCII character having value 17" 🔢 An "ASCII printable character" is an ASCII character in the range 33-126. An "ASCII control character" is an ASCII character in the range 0-31, or the character DEL (ASCII 127). A "non-ASCII character" is a character having a value exceeding 127 ✨
NOTE: Blank is neither an "ASCII printable character" nor an "ASCII control character".
2.4. End-of-Line 🌈
How the end of a text line is represented depends on the context and the implementation. For Internet transmission via protocols such as SMTP [RFC821], an end-of-line is a CR (ASCII 13) followed by an LF (ASCII 10). ISO C [ISO/IEC9899] and many modern operating systems indicate end-of-line with a single character, typically ASCII LF (aka "newline"), and this is the normal convention when news is transmitted via UUCP. A variety of other methods are in use too, including out-of-band methods where there is no specific character that means end-of-line 🧩
This Draft does not constrain how end-of-line is represented in news, except that characters other than CR and LF MUST NOT be usurped for use in end-of-line representations. Also, obviously, all software dealing with a particular copy of an article must agree on the convention to be used. "EOL" is used to mean "whatever end-of-line representation is appropriate"; it is not necessarily a character or sequence of characters 💖
NOTE: If faced with picking an EOL representation in the absence of other constraints, use of a single character simplifies processing, and the ASCII standard [X3.4] specifies that if one character is to be used for this purpose, it should be LF (ASCII 10).
NOTE: Inside MIME encodings, use of the Internet canonical EOL representation (CR followed by LF) is mandatory. See [RFC2049].
2.5. Case-Sensitivity 🔠
Text in newsgroup names, header parameters, etc. is case-sensitive unless stated otherwise.
NOTE: This is at variance with MAIL, which is case-insensitive unless stated otherwise, but is consistent with news historical practice and existing news software. See the comments on backward compatibility in Section 1.
2.6. Language 🌏
Various constant strings in this Draft, such as header names and month names, are derived from English words. Despite their derivation, these words do NOT change when the poster or reader employing them is interacting in a language other than English 💅
Posting and reading agents SHOULD translate as appropriate in their interaction with the poster or reader, but the forms that actually appear in articles are always the English-derived ones defined in this Draft 🎀
3. Relation to MAIL (RFC822, etc.) ✨
The primary intent of this Draft is to completely describe the news article format as a subset of MAIL's message format (augmented by some new headers). Unless explicitly noted otherwise, the intent throughout is that an article MUST also be a valid MAIL message 📬
NOTE: Despite obvious similarities between news and mail, opinions vary on whether it is possible or desirable to unify them into a single service. However, it is unquestionably both possible and useful to employ some of the same tools for manipulating both mail messages and news articles, so there is specific advantage to be had in defining them compatibly. Furthermore, there is no apparent need to re-invent the wheel when slight extensions to an existing definition will suffice.
Given that this Draft attempts to be self-contained, it inevitably contains considerable repetition of information found in MAIL. This raises the possibility of unintentional conflicts. Unless specifically noted otherwise, any wording in this Draft that permits behavior that is not MAIL-compliant is erroneous and should be followed only to the extent that the result remains compliant with MAIL 🧐
NOTE: [RFC1036] said "where this standard conflicts with the Internet Standard, RFC 822 should be considered correct and this standard in error". Taken literally, this was obviously incorrect, since [RFC1036] imposed a number of restrictions not found in [RFC822]. The intent, however, was reasonable: to indicate that UNINTENTIONAL differences were errors in [RFC1036].
Implementors and users should note that MAIL is deliberately an extensible standard, and most extensions devised for mail are also relevant to (and compatible with) news. Note particularly MIME, summarized briefly in Appendix B, which extends MAIL in a number of useful ways that are definitely relevant to news. Also of note is the work in progress on reconciling Privacy Enhanced Mail (PEM), which defines extensions for authentication and security) with MIME, after which this may also be relevant to news.
UNRESOLVED ISSUE: Update the MIME/PEM information.
Similarly, descriptions here of MIME facilities should be considered correct only to the extent that they do not require or legitimize practices that would violate those RFCs. (Note that this Draft does extend the application of some MIME facilities, but this is an extension rather than an alteration.)
4. Basic Format 🎀
4.1. Overall Syntax 💅
Okay so the overall shape of a news article is literally this 🦋:
article = 1*header separator body
header = start-line *continuation
start-line = header-name ":" space [ nonblank-text ] eol
continuation = space nonblank-text eol
header-name = 1*name-character *( "-" 1*name-character )
name-character = letter / digit
letter = <ASCII letter A-Z or a-z>
digit = <ASCII digit 0-9>
separator = eol
body = *( [ nonblank-text / space ] eol )
eol = <EOL>
nonblank-text = [ space ] text-character *( space-or-text )
text-character = <any ASCII character except NUL (ASCII 0),
HT (ASCII 9), LF (ASCII 10), CR (ASCII 13),
or blank (ASCII 32)>
space = 1*( <HT (ASCII 9)> / <blank (ASCII 32)> )
space-or-text = space / text-characterSo an article is some headers followed by a body, with an empty line separating the two 💖. The headers hold structured info about the article and how it gets transmitted. Each header starts with a header name that identifies it, and it can keep going onto later lines if those continuation lines start with white space. (Heads up: Section 4.2.3 adds some extra restrictions to the header syntax shown here 👀.) The body is mostly unstructured text that only really matters to the poster and the readers ✨.
NOTE: Terminology here follows the current custom in the news community, rather than the MAIL convention of (sometimes) referring to what is here called a "header" as a "header field" or "field". 🌸
Quick thing: the separator line has to be truly empty, not just a line with white space on it 🙈. Any extra empty lines after it are part of the body, and so are empty lines at the end of the article.
NOTE: Some systems make no distinction between empty lines and lines consisting entirely of white space; indeed, some systems cannot represent entirely empty lines. The grammar's requirement that header continuation lines contain some printable text is meant to ensure that the empty/space distinction cannot confuse identification of the separator line.
NOTE: It is tempting to authorize posting agents to strip empty lines at the beginning and end of the body, but such empty lines could possibly be part of a preformatted document.
Implementors, listen up 🍓: trailing white space, whether it's alone on the line or not, MAY be significant in the body, notably in early versions of the "uuencode" encoding for binary data. Trailing white space MUST be preserved unless the article is known to have originated within a cooperating subnet that avoids using significant trailing white space, and SHOULD be preserved regardless. Posters SHOULD avoid using conventions or encodings that make trailing white space significant; for encoding of binary data, MIME's "base64" encoding is recommended. Implementors are also warned that ISO C implementations are not required to preserve trailing white space, and special precautions may be necessary in implementations that do not.
NOTE: Unfortunately, the signature-delimiter convention (described in Section 4.3.2) does use significant trailing white space. It's too late to fix this; there is work underway on defining an organized signature convention as part of MIME, which is a preferable solution in the long run. 🥲
Posters, be warned 🚨: some very old relayer software misbehaves when the first non-empty line of an article body begins with white space.
4.2. Headers ✨
4.2.1. Names and Contents 🎀
Even though the grammar puts restrictions on header-name syntax, relayers and reading agents SHOULD tolerate header names containing any ASCII printable character other than colon (":", ASCII 58).
NOTE: MAIL header names can contain any ASCII printable character (other than colon) in theory, but in practice, arbitrary header names are known to cause trouble for some news software. Section 4.1's restriction to alphanumeric sequences separated by hyphens is believed to permit all widely used header names without causing problems for any widely used software. Software is nevertheless encouraged to cope correctly with the full range of possibilities, since aberrations are known to occur.
Relayers MUST disregard headers not described in this Draft (that is, with header names not mentioned in this Draft) and pass them on unaltered. 🫶
Posters wishing to convey non-standard information in headers SHOULD use header names beginning with "X-". No standard header name will ever be of this form. Reading agents SHOULD ignore "X-" headers, or at least treat them with great care.
The order of headers in an article is not significant. However, posting agents are encouraged to put mandatory headers (see Section 5) first, followed by optional headers (see Section 6), followed by headers not defined in this Draft.
NOTE: While relayers and reading agents must be prepared to handle any order, having the significant headers (the precise definition of "significant" depends on context) first can noticeably improve efficiency, especially in memory-limited environments where it is difficult to buffer up an arbitrary quantity of headers while searching for the few that matter.
Header names are case-insensitive. There is a preferred case convention, which posters and posting agents SHOULD use: each hyphen-separated "word" has its initial letter (if any) in uppercase and the rest in lowercase, except that some abbreviations have all letters uppercase (e.g., "Message-ID" and "MIME-Version"). The forms used in this Draft are the preferred forms for the headers described herein. Relayers and reading agents are warned that articles might not obey this convention.
NOTE: Although software must be prepared for the possibility of random use of case in header names (and other case-independent text), establishing a preferred convention reduces pointless diversity and may permit optimized software that looks for the preferred forms before resorting to less-efficient case-insensitive searches.
In general, a header can consist of several lines, with each continuation line beginning with white space. The EOLs preceding continuation lines are ignored when processing such a header, effectively combining the start-line and the continuations into a single logical line. Take that logical line, minus the header name, the colon, and any white space following the colon, and boom, you've got the "header content" 💜.
4.2.2. Undesirable Headers 🙅♀️
A header whose content is empty is said to be an empty header. Relayers and reading agents SHOULD NOT consider presence or absence of an empty header to alter the semantics of an article (although syntactic rules, such as requirements that certain header names appear at most once in an article, MUST still be satisfied). Posting agents SHOULD delete empty headers from articles before posting them.
Headers that merely state defaults explicitly (e.g., a Followup-To header with the same content as the Newsgroups header, or a MIME Content-Type header with contents "text/plain; charset=us-ascii") or state information that reading agents can typically determine easily themselves (e.g., the length of the body in octets) are redundant, conveying no information whatsoever. Headers that state information that cannot possibly be of use to a significant number of relayers, reading agents, or readers (e.g., the name of the software package used as the posting agent) are useless and pointless. Posters and posting agents SHOULD avoid including redundant or useless headers in articles. 💀
NOTE: Information that someone, somewhere, might someday find useful is best omitted from headers. (There's quite enough of it in article bodies.) Headers should contain information of known utility only. This is not meant to preclude inclusion of information primarily meant for news-software debugging, but such information should be included only if there is real reason, preferably based on experience, to suspect that it may be genuinely useful. Articles passing through gateways are the only obvious case where inclusion of debugging information appears clearly legitimate. (See Section 10.1.)
NOTE: A useful rule of thumb for software implementors is: "if I had to pay a dollar a day for the transmission of this header, would I still think it worthwhile?". 🪙
4.2.3. White Space and Continuations 🫧
The colon following the header name on the start-line MUST be followed by white space, even if the header is empty. If the header is not empty, at least some of the content MUST appear on the start-line. Posting agents MUST enforce these restrictions, but relayers (etc.) SHOULD accept even articles that violate them.
NOTE: MAIL does not require white space after the colon, but it is usual. [RFC1036] required the white space, even in empty headers, and some existing software demands it. In MAIL, and arguably in [RFC1036] (although the wording is vague), it is technically legitimate for the white space to be part of a continuation line rather than the start-line, but not all existing software will accept this. Deleting empty headers and placing some content on the start-line avoids this issue; this is desirable because trailing blanks, easily deleted by accident, are best not made significant in headers.
In general, posters and posting agents SHOULD use blank (ASCII 32), not tab (ASCII 9), where white space is desired in headers. Existing software does not consistently accept tab as synonymous with blank in all contexts. In particular, [RFC1036] appeared to specify that the character immediately following the colon after a header name was required to be a blank, and some news software insists on that, so this character MUST be a blank. Again, posting agents MUST enforce these restrictions but relayers SHOULD be more tolerant. 🎯
Since the white space beginning a continuation line remains a part of the logical line, headers can be "broken" into multiple lines only at white space. Posting agents SHOULD NOT break headers unnecessarily. Relayers SHOULD preserve existing header breaks, and SHOULD NOT introduce new breaks. Breaking headers SHOULD be a last resort; relayers and reading agents SHOULD handle long header lines gracefully. (See the discussion of size limits in Section 4.6.) 🌈
4.3. Body 🍰
Although the article body is unstructured for most of the purposes of this Draft, structure MAY be imposed on it by other means, notably MIME headers (see Appendix B). ✨
4.3.1. Body Format Issues ✨
Okay so the body of an article MAY be empty 🫶, but posting agents SHOULD consider this an error condition (which is worth sending back to the poster for revision) 💅. A posting agent that doesn't reject an article like that SHOULD issue a warning message to the poster and supply a non-empty body. Also, the separator line MUST be present even if the body is empty 🙈.
NOTE: An empty body is probably a poster error except, arguably, for some control messages, and even they really ought to have a body explaining the reason for the control message. Some old reading agents are known to generate empty bodies for "cancel" control messages, so posting agents might opt not to reject bodyless articles in such cases (although it would be better to fix the reading agents to request a body). However, some existing news software is known to react badly to bodyless articles, hence the request for posting agents to insert a body in such cases.
NOTE: A possible posting-agent-supplied body text (already used by one widespread posting agent) is "This article was probably generated by a buggy news reader". (The use of "reader" to refer to the reading agent is traditional, although this Draft uses more precise terminology.)
NOTE: The requirement for the separator line even in a bodyless article is inherited from MAIL and also distinguishes legitimately bodyless articles from articles accidentally truncated in the middle of the headers.
Note that an article body is a sequence of lines terminated by EOLs, not arbitrary binary data, and in particular it MUST end with an EOL. 🦋 However, relayers SHOULD treat the body of an article as an uninterpreted sequence of octets (except as mandated by changes of EOL representation and by control-message processing) and SHOULD avoid imposing constraints on it. See also Section 4.6.
4.3.2. Body Conventions 🎀
Okay, body lines can in principle be super long (see Section 4.6 for some discussion of length limits), but posters SHOULD restrict body line lengths to circa 70-75 characters 📏. On systems where text is usually stored with EOLs only at paragraph breaks and other "hard return" points, with the software doing the line breaking for display or editing, posting agents SHOULD insert EOLs as necessary so that posted articles follow this restriction 🌸.
NOTE: News originated in environments where line breaks in plain text files were supplied by the user, not the software. Love it or hate it, much reading-agent and posting-agent software assumes that news articles follow this convention, so it is often inconvenient to read or respond to articles that violate it. The "70-75" number comes from the widespread use of display devices that are 80 columns wide (with the number reduced to provide a bit of margin for quoting, see below).
Reading agents confronted with body lines much longer than the available output-device width SHOULD break lines as appropriate. Posters are warned that such breaks may not occur exactly where the poster intends 😭.
NOTE: "As appropriate" would typically include breaking lines when supplying the text of an article to be quoted in a reply or followup, something that line-breaking reading agents often neglect to do now.
Styles vary wildly, but for plain text it's usual to use no left margin, leave the right edge ragged, use a single empty line to separate paragraphs, and use normal natural-language conventions for things like upper/lowercase. (In particular, articles SHOULD NOT be written entirely in uppercase. In environments where posters only have access to uppercase, posting agents SHOULD translate it to lowercase. 💀)
NOTE: Most people find substantial bodies of text entirely in uppercase relatively hard to read, while all-lowercase text merely looks slightly odd. The common association of uppercase with strong emphasis adds to this.
Tone of voice doesn't carry well in written text, and misunderstandings are super common when sarcasm, parody, or exaggeration for humorous effect is attempted without explicit warning 😩. It has become conventional to use the sequence :-), which (on most output devices) looks like a rotated smiley face ☺️, as a marker for text not meant to be taken literally, especially when humor is intended. This practice aids communication and averts unintended ill-will, so posters are urged to use it 🫶. A variety of analogous sequences are used with less-standardized meanings [Sanderson].
The order in which news articles arrive at a particular host depends somewhat on transmission paths, and sometimes articles get lost for various reasons 🧚. When responding to a previous article, posters SHOULD NOT assume that all readers understand the exact context. It's common to quote some of the previous article to establish context. This SHOULD be done by putting the character > in front of each quoted line (even if the line is empty). This results in multiple levels of > when the quoted context itself contains quoted context 🔥.
NOTE: It may seem superfluous to put a prefix on empty lines, but it simplifies implementation of functions such as "skip all quoted text" in reading agents.
Readability is boosted when quoted text and new text are separated by an empty line ✨.
Posters SHOULD edit quoted context to trim it down to the minimum necessary. However, posting agents SHOULD NOT attempt to enforce this by imposing overly simplistic rules like "no more than 50% of the lines should be quotes" 🙅♀️.
NOTE: While encouraging trimming is desirable, the 50% rule imposed by some old posting agents is both inadequate and counterproductive. Posters don't respond to it by being more selective about quoting; they respond by padding short responses, or by using different quoting styles to defeat automatic analysis. The former adds unnecessary noise and volume, while the latter also defeats more useful forms of automatic analysis that reading agents might wish to do.
NOTE: At the very least, if a minimum-unquoted quota is being set, article bodies shorter than (say) 20 lines, or perhaps articles that exceed the quota by only a few lines, should be exempt. This avoids the ridiculous situation of complaining about a 5-line response to a 6-line quote 😂.
NOTE: A more subtle posting-agent rule, suggested for experimental use, is to reject articles that appear to contain quoted signatures (see below). This is almost certainly the result of a careless poster not bothering to trim down quoted context. Also, if a posting agent or followup agent presents an article template to the poster for editing, it really should take note of whether the poster actually made any changes, and refrain from posting an unmodified template.
Some followup agents supply "attribution" lines for quoted context, indicating where it first appeared and under whose name 📝. When multiple levels of quoting are present and quoted context is edited for brevity, "inner" attribution lines aren't always kept. The editing process is also somewhat error-prone. Reading agents (and readers) are warned not to assume that attributions are accurate 🌈.
UNRESOLVED ISSUE: Should a standard format for attribution lines be defined? There is already considerable diversity, but automatic news analysis would be substantially aided by a standard convention.
Early difficulties in inferring return addresses from article headers led to "signatures": short closing texts, automatically added to the end of articles by posting agents, identifying the poster and giving his network addresses, etc. 💌 If a poster or posting agent does append a signature to an article, the signature SHOULD be preceded with a delimiter line containing (only) two hyphens (ASCII 45) followed by one blank (ASCII 32), like this:
--Posting agents SHOULD limit the length of signatures, since verbose excess bordering on abuse is common if no restraint is imposed 🙄. 4 lines is a common limit.
NOTE: While signatures are arguably a blemish, they are a well-understood convention, and conveying the same information in headers exposes it to mangling and makes it rather less conspicuous. A standard delimiter line makes it possible for reading agents to handle signatures specially if desired.
(This is unfortunately hampered by extensive misunderstanding of, and misuse of, the delimiter.)
NOTE: The choice of delimiter is somewhat unfortunate, since it relies on preservation of trailing white space, but it is too well-established to change. There is work underway to define a more sophisticated signature scheme as part of MIME, and this will presumably supersede the current convention in due time.
NOTE: Four 75-column lines of signature text is 300 characters, which is ample to convey name and mail-address information in all but the most bizarre situations. 👑
4.4. Characters and Character Sets 🎀
okay bestie, let's talk characters 💅 Header and body lines MAY contain any ASCII characters other than CR (ASCII 13), LF (ASCII 10), and NUL (ASCII 0). So basically all of ASCII is fair game, just not those three troublemakers 🙅♀️
NOTE:CRandLFare excluded because they clash with common EOL conventions.NULis excluded because it clashes with the C end-of-string convention, which is significant to most existing news software. These three characters are unlikely to be transmitted successfully. 💀
Still, posters SHOULD avoid ASCII control characters, except for tab (ASCII 9), formfeed (ASCII 12), and backspace (ASCII 8) 🌸 Tab means "jump over enough horizontal space to land on the next of a set of fixed positions." Heads up though, there's no standard set of those positions, so tabs should be avoided if precise spacing is essential 📏
Formfeed is like a pause button 🛑 It marks a spot where a reading agent SHOULD pause and wait for reader interaction before it displays any more text.
Backspace SHOULD be used only for underlining 🖍️ You do it with a run of underscores (ASCII 95) followed by the same number of backspaces, which means the same number of text characters that come after get underlined. Not every output device can do underlining, so it's not something to count on for essential meaning. Reading agents SHOULD recognize underlining and translate it into the right commands for devices that support it ✨
NOTE: Interpretation of almost all control characters is device-specific to some degree, and devices differ. Tabs and underlining are supported, to some extent, by most modern devices and reading agents, hence the cautious exemptions for them. The underlining method is specified because the inverse method, text and then underscores, is tempting to the naive; however, if sent unaltered to a device that shows only the most recent of several overstruck characters rather than a composite, the result can be utterly unreadable 😭
NOTE: A common interpretation of tab is that it is a request to space forward to the next position whose number is one more than a multiple of 8, with positions numbered sequentially starting at 1. (So tab positions are 9, 17, 25, ...) Reading agents not constrained by existing system conventions might wish to use this interpretation. 🧚
NOTE: It will typically be necessary for a reading agent to catch and interpret formfeed, not just send it to the output device. The actions performed by typical output devices on receiving a formfeed are neither adequate for, nor appropriate to, the pause-for-interaction meaning. 💅
Cooperating subnets that want to use non-ASCII character sets, by using escape sequences (like ESC (ASCII 27), SO (ASCII 14), and SI (ASCII 15)) to change the meaning of characters that look like ASCII, MAY do so 🦋 But they MUST use MIME headers to tell reading agents which character set(s) and escape sequences are in use. A reading agent SHOULD NOT pass such an escape sequence through, unaltered, to the output device unless it confirms that the sequence is one used to affect character sets, and has reason to believe the device can interpret that particular sequence properly.
NOTE: Cooperating-subnet organizers are warned that some very old relayers strip certain control characters out of articles they pass along. ESC is known to be among the affected characters. 🙈
NOTE: There are now standard Internet encodings for Japanese [RFC1345] and Vietnamese [RFC1456] in particular. 🌈
Articles MUST NOT contain any octet with a value above 127, i.e., any octet that is not an ASCII character. That's the big rule 👑
NOTE: This rule, like others, may be relaxed by unanimous consent of the members of a cooperating subnet, provided suitable precautions are taken to ensure that rule-violating articles do not leak out of the subnet. (This has already been done in many areas where ASCII is not adequate for the local language(s).) Beware that articles containing non-ASCII octets in headers are a violation of the MAIL specifications and are not valid MAIL messages. MIME offers a way to encode non-ASCII characters in ASCII for use in headers; see Section 4.5. ⚠️
NOTE: While there is great interest in using 8-bit character sets, not all software can yet handle them correctly, hence the restriction to cooperating subnets. MIME encodings can be used to transmit such characters while remaining within the octet restriction. 🎀
Looking ahead to the day when non-ASCII characters are safe to use anywhere ✨ (and to look after the very real cooperating subnets already using them), transmission paths SHOULD treat news articles as uninterpreted sequences of octets, except perhaps for transformations between EOL representations. Relayers SHOULD treat non-ASCII characters in articles as ordinary characters.
NOTE: 8-bit enthusiasts are warned that not all software conforms to these recommendations yet. In particular, standard NNTP [RFC977] is a 7-bit protocol {but in [RFC3977] it has been upped to 8-bit}, and there may be implementations that enforce this rule. Be warned, also, that it will never be safe to send raw binary data in the body of news articles, because changes of EOL representation may (will!) corrupt it. 💀
Except where cooperating subnets permit more direct approaches, MIME headers and encodings SHOULD be used to send non-ASCII content using ASCII characters. See Section 4.5, Appendix B, and the MIME RFCs for details 📚 If article content can be expressed in ASCII, it SHOULD be. If not, the order of preference for character sets is the one described in MIME.
NOTE: Using the MIME facilities, it is possible to transmit ANY character set, and ANY form of binary data, using only ASCII characters. Equally important, such articles are self-describing, and the reading agent can tell which octet-to-symbol mapping is intended! Designating some preferred character sets is intended to minimize the number of character sets that a reading agent must understand in order to display most articles properly. 🦄
Articles containing non-ASCII characters, articles using ASCII characters (values 0 through 127) to refer to non-ASCII symbols, and articles using escape sequences to shift character sets SHOULD include MIME headers indicating which character set(s) and conventions are being used. They MUST do so, unless such articles are strictly confined to a cooperating subnet that has its own pre-agreed conventions. MIME encodings are preferred over all of these techniques 🫶 If a relayer notices it's being asked to pass an article using such techniques outward, across what it knows to be the boundary of such a cooperating subnet, it MUST report this error to its administrator, and it MAY refuse to pass the article beyond the subnet boundary. If it does pass the article, it MUST re-encode it with MIME encodings to make it conform to this Draft.
NOTE: Such re-encoding is a non-trivial task, due to MIME rules such as the prohibition of nested encodings. It's not just a matter of pouring the body through a simple filter. 😵💫
Reading agents SHOULD note MIME headers and attempt to show the reader the closest possible approximation to the intended content. They SHOULD NOT just send the octets of the article to the output device unaltered, unless there is reason to believe the output device will indeed interpret them correctly. Reading agents MUST NOT pass ASCII control characters or escape sequences, other than as discussed above, unaltered to the output device. Only by chance would the result be the desired one, and there is serious potential for harmful side effects, whether accidental or malicious 🚨
NOTE: Exactly what to do with unwanted control characters/sequences depends on the philosophy of the reading agent, but passing them straight to the output device is almost always wrong. If the reading agent wants to mark the presence of such a character/sequence in circumstances where only ASCII printable characters are available, translating it to"#"might be a suitable method."#"is a conspicuous character seldom used in normal text. 🔍
NOTE: Reading agents should be aware that many old output devices (or the transmission paths to them) zero out the top bit of octets sent to them. This can transform non-ASCII characters into ASCII control characters. 🫠
Followup agents MUST be careful to apply appropriate transformations of representation to the outbound followup, as well as the inbound precursor. A followup to an article containing non-ASCII material is very likely to contain non-ASCII material itself 💜
4.5. Non-ASCII Characters in Headers ✨
Okay bestie, rule number one: every octet in a header MUST be ASCII characters 🎀. But it's still super desirable to have a way to encode non-ASCII characters, especially in "human-readable" headers like Subject 💅. That's where MIME comes in! The full details live in the MIME specs, so here's a quick summary to flag the issues for software authors 🦋.
encoded-word = "=?" charset "?" encoding "?" codes "?="
charset = 1*tag-char
encoding = 1*tag-char
tag-char = < ASCII printable character except
!()<>@,;:\"[]/?= >
codes = 1*code-char
code-char = <ASCII printable character except ?>An encoded word is a sequence of ASCII printable characters that specifies the character set, encoding method, and bits of (potentially) non-ASCII characters 🌸. Encoded words are only allowed in certain positions in certain headers, and specific headers add their own restrictions on what goes inside them, beyond what's in this section. Posting agents MUST ensure that any material resembling an encoded word (complete with all delimiters), in a context where encoded words may appear, really is an encoded word 🙈.
NOTE: The syntax is a bit ugly, but it was designed to minimize chances of confusion with legitimate header contents, and to satisfy difficult constraints on use within existing headers.
An encoded word MUST NOT be more than 75 octets long 💀. Each line of a header containing encoded word(s) MUST be at most 76 octets long, not counting the EOL.
NOTE: These limits are meant to bound the lookahead needed to determine whether text that begins with "=?" is really an encoded word.
The details of charsets and encodings are defined by MIME, and the sequence of preferred character sets is the same as MIME's 🧚. Encoded words SHOULD NOT be used for content that's expressible in ASCII, no cap 🙅.
When an encoded word is used, other than in a newsgroup name (see Section 5.5), it MUST be separated from any adjacent non-space characters (including other encoded words) by white space. Reading agents that display the contents of encoded words (as opposed to their encoded form) should ignore white space adjacent to encoded words ✨.
UNRESOLVED ISSUE: Should this section be deleted entirely, or made much more terse? The material is relevant, but too complex to discuss fully.
NOTE: The deletion of intervening white space permits using multiple encoded words, implicitly concatenated by the deletion, to encode text that will not fit within a single 75-character encoded word.
Reading-agent implementors, listen up 👑: although this Draft completely specifies where encoded words may appear in the headers it defines, there are other headers (e.g., the MIME Content-Description header) that MAY contain them. Be careful with those!
4.6. Size Limits 🍓
Implementations SHOULD avoid fixed constraints on the sizes of lines within an article and on the size of the entire article 💖.
Relayers SHOULD treat the body of an article as an uninterpreted sequence of octets (except as mandated by changes of EOL representation and processing of control messages), not to be altered or constrained in any way 🫶.
If it is absolutely necessary for an implementation to impose a limit on the length of header lines, body lines, or header logical lines, that limit shall be at least 1000 octets, including EOL representations. Relayers and transmission paths confronted with lines beyond their internal limits (if any) MUST NOT simply inject EOLs at random places 😭. They MAY break headers (as described in Section 4.2.3) as a last resort, and otherwise they MUST either pass the long lines through unaltered, or refuse to pass the article at all (see Section 9.1 for further discussion).
NOTE: The limit here is essentially the same minimum as that specified for SMTP mail [RFC821]. Implementors are warned that Path (see Section 5.6) and References (see Section 6.5) headers, in particular, often become several hundred characters long, so 1000 is not an overly generous limit.
All implementations MUST be able to handle an article totalling at least 65,000 octets, including headers and EOL representations, gracefully and efficiently 🔥. All implementations SHOULD be able to handle an article totalling at least 1,000,000 (one million) octets, including headers and EOL representations, gracefully and efficiently. "Gracefully and efficiently" is intended to preclude not only failures, but also major loss of performance, serious problems in error recovery, or resource consumption beyond what is reasonably necessary 🌈.
NOTE: The intent here is to prohibit lowering the existing de facto limit any further, while strongly encouraging movement towards a higher one. Actually, although improvements are desirable in some cases, much news software copes reasonably well with very large articles. The same cannot be said of the communications software and protocols used to transmit news from one host to another, especially when slow communications links are involved. Occasional huge articles that appear now (by accident or through ignorance) typically leave trails of failing software, system problems, and irate administrators in their wake 💀.
NOTE: It is intended that the successor to this Draft will raise the "MUST" limit to 1,000,000 and the "SHOULD" limit still further.
Posters SHOULD limit posted articles to at most 60,000 octets, including headers and EOL representations, unless the articles are being posted only within a cooperating subnet that is known to be capable of handling larger articles gracefully. Posting agents presented with a large article SHOULD warn the poster and request confirmation 🙈.
NOTE: The difference between this and the earlier "MUST" limit is due to margin for header growth, differing EOL representations, and transmission overheads.
NOTE: Disagreeable though these limits are, it is a fact that in current networks, an article larger than 64K (after header growth, etc.) simply is not transmitted reliably. Note also the comments above on the trauma caused by single extremely large articles now; the problems are real and current. These problems arguably should be fixed, but this will not happen network-wide in the immediate future, hence the restriction of larger articles to cooperating subnets, for now.
Posters using non-ASCII characters in their text MUST take into account the overhead involved in MIME encoding, unless the article's propagation will be entirely limited to a cooperating subnet that does not use MIME encodings for non-ASCII characters 🦋. For example, MIME base64 encoding involves growth by a factor of approximately 4/3, so an article that would likely have to use this encoding should be at most about 45,000 octets before encoding.
Posters SHOULD use MIME "message/partial" conventions to facilitate automatic reassembly of a large document split into smaller pieces for posting 🧚. It is recommended that the content identifier used should be a message ID, generated by the same means as article message IDs (see Section 5.3), and that all parts should have a See-Also header (see Section 6.16) giving the message IDs of at least the previous parts and preferably all of the parts.
NOTE: See-Also is more correct for this purpose than References, although References is in common use today (with less-formal reassembly arrangements). MIME reassemblers should probably examine articles suggested by References headers if See-Also headers are not present to indicate the whereabouts of the other parts of "message/partial" articles.
To repeat: implementations SHOULD avoid fixed constraints on the sizes of lines within an article and on the size of the entire article. I'm not kidding, it's that important 👑.
4.7. Example 🎀
Here is a sample article:
From: jerry@eagle.ATT.COM (Jerry Schwarz)
Path: cbosgd!mhuxj!mhuxt!eagle!jerry
Newsgroups: news.announce
Subject: Usenet Etiquette -- Please Read
Message-ID: <642@eagle.ATT.COM>
Date: Mon, 17 Jan 1994 11:14:55 -0500 (EST)
Followup-To: news.misc
Expires: Wed, 19 Jan 1994 00:00:00 -0500
Organization: AT&T Bell Laboratories, Murray Hill
body
body
body5. Mandatory Headers 🌸
An article MUST have one, and only one, of each of the following headers: Date, From, Message-ID, Subject, Newsgroups, Path. Non-negotiable, bestie 💅.
NOTE: MAIL specifies (if read most carefully) that there must be exactly one Date header and exactly one From header, but otherwise does not restrict multiple appearances of headers. (Notably, it permits multiple Message-ID headers!) This appears singularly useless, or even harmful, in the context of news, and much current news software will not tolerate multiple appearances of mandatory headers 😭.
Note also that there are situations, discussed in the relevant parts of Section 6, where References, Sender, or Approved headers are mandatory 🔥.
In the discussions of the individual headers, the content of each is specified using the syntax notation. The convention used is that the content of, for example, the Subject header is defined as <Subject-content> 🧚.
5.1. Date 📅
okay so the Date header is where the article says when it got sent off into the world 💌 here's the exact shape, no cap:
Date-content = [ weekday "," space ] date space time
weekday = "Mon" / "Tue" / "Wed" / "Thu"
/ "Fri" / "Sat" / "Sun"
date = day space month space year
day = 1*2digit
month = "Jan" / "Feb" / "Mar" / "Apr" / "May" / "Jun"
/ "Jul" / "Aug" / "Sep" / "Oct" / "Nov" / "Dec"
year = 4digit / 2digit
time = hh ":" mm [ ":" ss ] space timezone
timezone = "UT" / "GMT"
/ ( "+" / "-" ) hh mm [ space "(" zone-name ")" ]
hh = 2digit
mm = 2digit
ss = 2digit
zone-name = 1*( <ASCII printable character except ()\>
/ space )basically it's a slimmed-down version of the MAIL date format, bestie 🎀
if you include a weekday, it MUST be consistent with the date 📆 (no Tuesday on a Sunday, sis 💀). the modern Gregorian calendar is used, and dates MUST be consistent with its usual conventions. for example, if the month is May, the day must be between 1 and 31 inclusive. the year SHOULD be given as four digits, and posting agents SHOULD enforce this 💅 however, relayers MUST accept the two-digit form, and MUST interpret it as having the implicit prefix "19" 🦋
NOTE: Two-digit year numbers can, should, and must be phased out by 1999.
the time uses the 24-hour clock, so two hours before midnight is "22:00" or "22:00:00" 🌙 the hh must be between 00 and 23 inclusive, the mm between 0 and 59 inclusive, and the ss between 0 and 60 inclusive.
NOTE: Leap seconds very occasionally result in minutes that are 61 seconds long. 🤯
the date and time SHOULD be given in the poster's local time zone, with a numeric offset (which SHOULD include the time zone name, like "EST", tucked into parentheses like a MAIL comment) ✨ if that's not what you're doing, they MUST be given in Universal Time (abbreviated "UT"; "GMT" is a historical synonym for "UT") 🌍 the time zone name in parentheses, if it's there, is just a comment. software MUST ignore it, though reading agents might wish to show it to the reader. time zone names other than "UT" and "GMT" MUST appear only in the comment 🙈
NOTE: Attempts to deal with a full set of time zone names have all foundered on the vast number of such names in use and the duplications (for example, there are at least FIVE different time zones called "EST" by somebody). Even the limited set of North American zone names authorized by MAIL is subject to confusion and misinterpretation, hence the flat ban on non-UT time zone names, except as comments. 💀
NOTE: [RFC1036] specified that use of GMT (aka UT, UTC) was preferred. However, the local time (in the poster's time zone) is arguably information of possible interest to the reader, and this requires some indication of the poster's time zone. Numeric offsets are an unambiguous way of doing this, and their use was indeed sanctioned by [RFC1036] (that is, this is a change of preference only). 🎀
NOTE: There is frequent confusion, including errors in some news software, regarding the sign of numeric time zones. Zones west of Greenwich have negative offsets. For example, North American Eastern Standard Time is zone -0500 and North American Eastern Daylight Time is zone -0400. 🧚
NOTE: Implementors are warned that the hh in a time zone can go up to about 14; it is not limited to 12. This is because the International Date Line does not run exactly along the boundary between zone -1200 and zone +1200. 🌏
NOTE: The comments in Section 2.6 regarding translation to other languages are relevant here. The Date-content format, and the spellings of its components, as found in articles themselves, are always as defined in this Draft, regardless of the language used to interact with readers and posters. Reading and posting agents should translate as appropriate. Actually, even English-language reading and posting agents will probably want to do some degree of translation on dates, if only to abbreviate the lengthy format and (perhaps) translate to and from the reader's time zone. 🫶
5.2. From 💌
the From header is the electronic address of the article's author, and maybe their full name too 🦋 here's the grammar:
From-content = address [ space "(" paren-phrase ")" ]
/ [ plain-phrase space ] "<" address ">"
paren-phrase = 1*( paren-char / space / encoded-word )
paren-char = <ASCII printable character except ()<>\>
plain-phrase = plain-word *( space plain-word )
plain-word = unquoted-word / quoted-word / encoded-word
unquoted-word = 1*unquoted-char
unquoted-char = <ASCII printable character except !()<>@,;:\".[]>
quoted-word = quote 1*( quoted-char / space ) quote
quote = <" (ASCII 34)>
quoted-char = <ASCII printable character except "()<>\>
address = local-part "@" domain
local-part = unquoted-word *( "." unquoted-word )
domain = unquoted-word *( "." unquoted-word )(encoded words are explained in Section 4.5 🌸) so the full name can be told apart from the address in one of two ways: wrap it in parentheses, which looks like a MAIL comment after the address, or wrap the address in angle brackets. the second way is the fave 👑 in the first form, encoded words inside the full name MUST be composed entirely of <paren-char>s. in the second form, encoded words inside the full name can't contain anything except letters (either case), digits, and the characters "!", "*", "+", "-", "/", "=", and "_". the local part is case-sensitive (except that all case counterparts of "postmaster" count as equivalent), the domain is case-insensitive, and every other part of the From content is a comment that MUST be ignored by news software (except where reading agents want to show it to the reader) 💅 posters and posting agents MUST restrict themselves to this subset of the MAIL From syntax. relayers MAY accept a broader subset, but check the discussion in Section 9.1 ⭐
NOTE: The syntax here is a restricted subset of the MAIL From syntax, with quoting particularly restricted, for simple parsing. In particular, the presence of "<" in the From content indicates that the second form is being used; otherwise, the first form is being used. The major restrictions here are those already de facto imposed by existing software.
NOTE: Overly lenient posting agents sometimes permit the second form with a full name containing "(" or ")", but it is extremely rare for a full name to contain "<" or ">", even in mail.
so reading agents that want to know for sure which form they're looking at should check for the presence or absence of "<", not "(" 🧐
the address SHOULD be a valid and complete Internet domain address, one that an Internet host can actually mail to (maybe through an MX (Mail Exchange) record and a forwarder) 📬 the pseudo-domain ".uucp" MAY be used for hosts registered in the UUCP maps (for example, the name "xyz.uucp" for registered site "xyz"), but those hosts SHOULD stop doing this as soon as possible, either by getting a proper Internet address and forwarder, or by using the "% hack" (more on that below) 🏃♀️ bitnet hosts SHOULD use Internet addresses and skip the obsolete ".bitnet" pseudo-domain. other forms of address MUST NOT be used 🚫
NOTE: "Other forms" specifically include UK-style "backward" domains ("uk.oxbridge.cs" is in the Czech Republic, not the UK), pure-UUCP addressing ("knee!shin!foot" instead of "foot%shin@knee.uucp"), and abbreviated domains ("zebra.zoo" instead of "zebra.zoo.toronto.edu"). 🦓
if you need the local part to specify a route relative to the nearest Internet host, this MUST be done using the "% hack", with "%" acting as a secondary "@" 🎀 say you want mail to go to Internet host "foo.bar.edu", then to non-Internet host "ein", then to non-Internet host "deux", and finally get delivered to mailbox "fred". the address would be:
fred%deux%ein@foo.bar.eduanalogous forms that use "!" in the local part MUST NOT be used, because they're ambiguous. they should be written in the "%" form instead 🙅♀️
NOTE: "a!b@c" can be interpreted as either "b%c@a" or "b%a@c", and there is no consistency in which choice is made. Such addresses consequently are unreliable. The "%" form does not suffer from this problem, and although its use is officially discouraged, it is a de facto standard, to the point that MAIL recognizes it. 🥹
relayers MUST NOT, repeat MUST NOT, repeat MUST NOT, rewrite From lines in any way, however minor or innocent it seems 🚨 trying to "fix" a non-conforming address has a super high chance of making things worse. either pass it along unchanged or reject the article. no edits, periodt 💅
NOTE: An additional reason for banning the use of "!" addressing is that it has a much higher probability of being rewritten into mangled unrecognizability by old relayers. 💀
posters and posting agents SHOULD avoid using the characters "!" and "@" in full names, since old, simple-minded news software might rewrite the header because of them 🙈
NOTE: Also, the characters "." and ",", which show up in names all the time (for example, "John W. Campbell, Jr."), are NOT, repeat NOT, allowed in an unquoted word. A From header like the following MUST NOT be written without the quotation marks:
From: "John W. Campbell, Jr." <editor@analog.com>5.3. Message-ID 🦋
okay so the Message-ID header is basically the article's ID card 🪪, a unique identifier that tells it apart from every other article out there 💖
Message-ID-content = message-id
message-id = "<" local-part "@" domain ">"As with From addresses, a message ID's local-part is case-sensitive, and its domain is case-insensitive. The < and > are parts of the message ID itself, not quirks of the Message-ID header ✨
NOTE: News message IDs are a restricted subset of MAIL message IDs. No existing news software copes properly with MAIL quoting conventions inside the local part, so those are forbidden 🙅♀️ It's unfortunate, especially for X.400 gateways that often want characters that aren't legal in unquoted message IDs, but it's impossible to fix net-wide 😩 See the notes on gatewaying in Section 10.
The domain in the message ID SHOULD be the full Internet domain name of the posting agent's host 🏠 Using the .uucp pseudo-domain (for hosts registered in the UUCP maps) or the .bitnet pseudo-domain (for Bitnet hosts) is permissible, but it SHOULD be avoided 🙃
Posters and posting agents MUST generate the local part of a message ID with an algorithm that follows the specified syntax (words separated by ., with certain characters not permitted; see Section 5.2 for details) and that will never repeat itself (ever) 🫠 The algorithm SHOULD NOT generate message IDs that differ only in letter case 🙅♀️ Peep Section 6.5 for the recommended convention for flagging subject changes 👀 Otherwise, the algorithm is up to the implementor 💅
NOTE: The big job of message IDs is telling circulating articles apart from each other, and from articles that went around recently 🔍 They could also be handy as permanent indexing keys (which is why permanent uniqueness matters), but indexers can't fully lean on that, since the earlier RFCs urged it without demanding it 🤷 Every major implementation has always made permanently unique IDs by design, but in some cases that depends on proper administration, so duplicates may have slipped in by accident 😬
NOTE: The most popular way to make local parts is to use the date and time, plus something to tell apart simultaneous postings on the same host (e.g., a process number), all encoded in a suitably restricted alphabet 📅 An older, now less-popular option is a sequence number that goes up each time the host makes a new message ID. That's workable, but it takes careful design to handle simultaneous posting attempts properly, and it isn't as robust when crashes and other malfunctions happen 💀
NOTE: Some buggy news software treats message IDs as completely case-insensitive 🙄, which is why the advice is to avoid relying on case distinctions. The restrictions on the "alphabet" of local parts and domains in Section 5.2 have a handy side effect: you don't have to parse message IDs in complicated ways to split them into case-sensitive and case-insensitive bits ✨
Big rules alert 🚨 The local part of a message ID MUST NOT be postmaster, or any other string that would compare equal to postmaster in a case-insensitive comparison. Message IDs MUST be no longer than 250 octets, including the < and > 💖
NOTE: Postmaster is an irksome exception to case-sensitivity in local parts 🙄, inherited from MAIL. The best way to deal with it is simply avoiding it (not that it's likely, but the issue still needs dealing with). The length limit is undesirable, but it's present in widely used existing software. The limit is actually 255, but a small safety margin is wise 🫶
5.4. Subject 💌
The Subject header's content (the "subject" of the article) is a short phrase describing the topic of the article 📣
Subject-content = [ "Re: " ] nonblank-textEncoded words MAY appear in this header.
If the article is a followup, the subject SHOULD begin with Re: (a "back reference") 🔙 If it's not a followup, the subject MUST NOT begin with a back reference 🚫 Back references are case-insensitive, although Re: is the preferred form. A followup agent helping a poster prepare a followup SHOULD prepend a back reference, unless the subject already begins with one 🫶 If the poster decides the followup's topic differs significantly from what the subject describes, a new, more descriptive subject SHOULD be swapped in (with no back reference) 🔄 An article whose subject starts with a back reference MUST have a References header that points back to the precursor 🧵
NOTE: A back reference is FOUR characters, the fourth being a blank 🔢 (yes, [RFC1036] got this one mixed up 😅). Also, only ONE back reference should be present.
NOTE: There's a semi-standard convention, pretty commonly used, for flagging a subject change. The new Subject-content takes this form:
new topic (was: old topic)Possibly with "old topic" somewhat truncated. Posters who want to do this are urged to use this exact form, so automated analysis stays simple 🤖
For historical reasons, the subject MUST NOT begin with cmsg (note that this sequence ends with a blank 👀).
NOTE: Some old news software treats a subject beginning with cmsg as a sign that the article is a control message (see Sections 6.6 and 7) 🤯 This mechanism is obsolete and undesirable, but accidentally triggering it is still possible 😬
The subject SHOULD be terse ✂️ Posters SHOULD avoid trying to cram their entire article into the headers 🙅♀️ Even the simplest query usually benefits from a sentence or two of elaboration and context, and how headers get displayed varies a lot between reading agents 💅
NOTE: All-in-the-subject articles sometimes happen because of misunderstandings about how a posting agent's interaction works 🤦 Posting agents might want to pay special attention to the possibility that a poster typing a very long subject thought they were typing the body of the article 📝
5.5. Newsgroups 📰
okay so the Newsgroups header is how an article tells everyone which newsgroup(s) it's posted to 💌 and it has its own grammar, which is giving strict but also kind of iconic:
Newsgroups-content = newsgroup-name *( ng-delim newsgroup-name )
newsgroup-name = plain-component *( "." component )
component = plain-component / encoded-word
plain-component = component-start *13component-rest
component-start = lowercase / digit
lowercase = <letter a-z>
component-rest = component-start / "+" / "-" / "_"
ng-delim = ","Encoded words in newsgroup names MUST NOT contain anything except letters, digits, "+", "-", "/", "_", "=", and "?" 🎀 (although they may encode those other characters).
A newsgroup name is one or more components. Each component can be a plain component or, except for the first one, an encoded word ✨ A plain component MUST contain at least one letter, MUST begin with a letter or digit, and MUST NOT be longer than 14 characters. The first component MUST begin with a letter, and subsequent components SHOULD begin with a letter. Newsgroup names MUST NOT contain uppercase letters, except where encodings in encoded words require them. The sequences "all" and "ctl" MUST NOT be used as components 💀
NOTE: The alphabet and syntax cover all the existing names of widespread newsgroups, while avoiding forms that are known to cause problems. Important existing software uses various non-alphanumeric characters as punctuation next to newsgroup names. (It would, in fact, be preferable to ban "+" from newsgroup names, were it not that several widespread newsgroups related to the C++ programming language already use it.) 🙈
NOTE: Lots of existing software turns the newsgroup name into a directory path and stores the articles themselves under numeric filenames, so all-digit components can be troublesome. The "Great Renaming" early in the history of Usenet revised several newsgroup names to get rid of such components 🧚
NOTE: That same storage technique is the reason for the 14-character limit. The limit is now mostly historical, since most modern systems allow much longer directory entry names, but plenty of old systems are still in use. Systems with shorter limits also exist, but news software on those has already had to deal with the problem, since there are several widespread newsgroups with 14-character components in their names. Implementors are warned that the successor to this Draft is intended to increase the 14-character limit, and they are urged to fix their software to handle longer names gracefully (if such fixes are necessary, given the intended domain of application of the particular software). 🦋
NOTE: The requirement that the first character of a name be a letter accommodates existing software that assumes it can tell a newsgroup name apart from other possible syntactic entities by checking the first character. Similar considerations motivate excluding"+","-", and"_"from coming first in a component, and the preference for components that don't begin with digits. The"all"sequence is used as a wildcard symbol in much existing software, and the"ctl"sequence was part of an obsolete historical mechanism for marking control messages, so they're best avoided 🌸
NOTE: Newsgroup names possibly should have been case-insensitive, but all existing software treats them as case-sensitive. ([RFC977] claims that they are case-insensitive in NNTP, but existing implementations are believed to ignore this.) The simplest solution is to just ban uppercase letters, since no widespread newsgroup name uses them anyway. That avoids any possibility of confusion 💅
NOTE: The syntax has the disadvantage of containing no white space, which makes it impossible to continue a Newsgroups header across several lines. Implementors of relayers and reading agents are warned that the successor to this Draft is intended to change the definition of ng-delim to:
ng-delim = "," [ space ]and they are urged to fix their software to handle (i.e., ignore) white space following the commas. Meanwhile, posters must avoid inserting such space (despite the natural-language convention that permits it), and posting agents should strip it out. Lowkey a tiny sacrifice for a big payoff 🫶
NOTE: Encoded words as components are somewhat problematic, but they are clearly desirable for use in non-English-speaking nations 🌈 They are not subject to the 14-character limit, and this (plus the possibility of "/" within them) may require special handling in news software.
Encoded words are allowed in newsgroup names only where non-ASCII characters are necessary to the name, and they must use the "b" encoding [RFC2045] and the first suitable character set in the MIME order of preferred character sets [RFC2047] {ASCII before ISO-8859-* before anything else} ✨
NOTE: Since the newsgroup name is the encoded form, not the underlying non-ASCII form, there's room for terrible confusion here if the choice of encoding for a particular name isn't fully standardized. 😭
Posters SHOULD use only the names of existing newsgroups in the Newsgroups header, because newsgroups are not created simply by being posted to 🙅♀️ However, it's legitimate to cross-post to newsgroup(s) that don't exist on the posting agent's host, as long as at least one of the newsgroups does exist there. Followup agents MUST accept this. Posting agents MAY accept it, but SHOULD at least alert the poster to the situation and request confirmation. Relayers MUST NOT rewrite Newsgroups headers in any way, even if some or all of the newsgroups don't exist on the relayer's host 🔥
NOTE: Early experience with news software that created newsgroups whenever they were mentioned in a Newsgroups header was thoroughly negative. Posters frequently mistype newsgroup names 🙈
NOTE: It's legitimate for some of an article's newsgroups not to exist on the host where it's posted, but this is a rather unusual situation except in followups. (Followups should go to all the newsgroups the precursor was posted to, even if not all of them reach the site where the followup is being posted.) 🍓
NOTE: Rewriting Newsgroups headers to strip out locally unknown newsgroups is superficially attractive. However, early experience with exactly that policy was thoroughly negative. News propagation is more redundant and much less orderly than many people imagine, and in particular it's not unheard of for the (sometimes) fastest path between two (say) University of Toronto sites to pass outside the University of Toronto. In that case, newsgroup stripping can cause incomplete propagation. Having an article's set of newsgroups change as it propagates can also result in followups not achieving the same propagation as the original. It's been tried, it's more trouble than it's worth, so don't do it 🚫
NOTE: In particular, newsgroup stripping superficially looks like a solution to the problem of duplicate regional newsgroup names. For example, both the University of Toronto and the University of Texas have ut.general newsgroups, and material cross-posted to that name and a global newsgroup appears in both universities' local newsgroups. However, the side effects of stripping are sufficiently unacceptable to disqualify it for this purpose. Don't do it. 💀
Cross-posting an article to several relevant newsgroups is far superior to posting separate articles with duplicated content to each newsgroup, because reading agents can detect the situation and show the article to a reader only once 💖 Posters SHOULD cross-post rather than duplicate-post.
NOTE: On the other hand, cross-posting to a large number of newsgroups usually means the poster hasn't thought about their audience. Articles are rarely pertinent to more than (say) half a dozen newsgroups. Posting agents might wish to request confirmation when the number of newsgroups exceeds (say) five in the presence of a Followup-To header, or (say) two in the absence of such a header. ✨
NOTE: One problem with cross-postings is what to do with an article cross-posted to a set of newsgroups that includes both moderated and unmoderated ones. Posters tend to expect such an article to show up immediately in the unmoderated newsgroups, especially if they don't realize that one or more of the newsgroups is moderated. However, since it isn't possible for a moderator to retroactively add an already-posted article to a moderated newsgroup, the only correct action is to mail such an article to one (and only one) of the moderators for action. It's probably best for the posting agent to detect this situation and ask the poster what action they prefer. The acceptable choices are to alter the newsgroup list or to mail to a moderator of the poster's choice. The posting agent SHOULD NOT offer duplicate-posting as an easy-to-request option (if only because many moderators will reject a submission that has already been posted to unmoderated newsgroups). 🙅♀️
NOTE: An article cross-posted to multiple moderated newsgroups really should have approval from all of the moderators involved. In practice, the only straightforward way to do this is to send the article to one of them and have them consult the others. 🫠
A newsgroup SHOULD NOT appear more than once in the Newsgroups header 🎀
Newsgroup names with only one component are reserved for newsgroups whose propagation is restricted to a single host (or the administrative equivalent). It's inadvisable to name a newsgroup "poster", because that word has special meaning in the Followup-To header (see Section 6.1). The names "control" and "junk" are frequently used for pseudo-newsgroups internal to relayer implementations, so they're also best avoided 🦋
NOTE: Beware of the duplicate-regional-newsgroup-names problem mentioned above. There are many, many hosts with a newsgroup named"general", and some surprising things show up in those newsgroups when people cross-post. It's probably better to use multi-component names, which are less likely to be duplicated. Fred's Widget House should use"fwh.general"rather than just"general"as its in-house general-topics newsgroup. 🏠
It's conventional to reserve newsgroup names beginning with "to." for test messages sent on an essentially point-to-point basis (see also the ihave/sendme protocol described in Section 7.2). Newsgroup names beginning with "to." SHOULD NOT be used for any other purpose. The second (and possibly later) components of such a name should, together, make up the relayer name (see Section 5.6) of a relayer. The newsgroup exists only at the named relayer and its neighbors. The neighbors all pass that newsgroup to the named relayer, while the named relayer does not pass it to anyone 🧚
The order of newsgroup names in the Newsgroups header is not significant 💜
5.6. Path 🧚
okay so the Path header is basically the receipts of who already got the article 📜. It keeps track of which relayers have already handled it so nobody sends it back to the people who already saw it 🙅♀️. no redundant re-sends, periodt 💅
Path-content = [ path-list path-delimiter ] local-part
path-list = relayer-name *( path-delimiter relayer-name )
relayer-name = 1*rn-char
rn-char = letter / digit / "." / "-" / "_"
path-delimiter = "!"So the Path content is a list of relayer names separated by path delimiters, and then after one final delimiter comes the local part of a mailing address 🎀. Every relayer MUST stick its own name plus a delimiter onto the front of the Path content in every article it processes 🫶. A relayer MUST NOT pass an article along to a neighboring relayer whose name is already in that article's path list 🙅♀️, unless the neighbor explicitly asks for it in some way. The Path content is case-sensitive 🔠, so capitalization matters, bestie.
NOTE: The Path header a posting agent gives you should normally have only the local part 🍓. The relayer that the posting agent hands the article to for posting adds its relayer name to get the path list started.
NOTE: Pls notice that the trailing local part is not part of the path list 👀. Here's an example Path header:
Path: fee!fie!foe!fumThat header has three relayer names: "fee", "fie", and "foe". A relayer named "fum" is still eligible to be sent this article 💅.
NOTE: This syntax has zero white space, which means you can't continue a Path header across several lines 😭. Implementors of relayers and reading agents, listen up: the successor to this Draft is planned to change the path delimiter definition to:
path-delimiter = "!" [ space ]Please fix your software so it handles (i.e., ignores) white space after the exclamation points. They are urged to hurry, bestie 🏃♀️, because some ill-behaved systems reportedly already feel free to add that white space 💀.
NOTE: [RFC1036] allows way more flexibility in the choice of delimiter, in theory 🤓, but nobody has ever used that flexibility, and most news software doesn't implement it properly. The grammar reflects the current reality. Also, [RFC1036] treats "_" as a delimiter, but it's known to show up in relayer names every now and then 🙈.
Because an article won't propagate to a relayer already in its path list, the path list MUST NOT contain any names other than those of relayers the article has passed through AS NEWS 🗞️. This is trivially obvious for normal news articles, but moderators of moderated newsgroups and the implementors and maintainers of gateways need to pay attention 🫣.
NOTE: For the same reason, a relayer and its neighbors need to agree on the choice of relayer name 🤝, and names shouldn't get changed without telling neighbors.
Relayer names need to be unique among all relayers that will ever see the articles using them ✨. A relayer name is usually either an "official" name for the host the relayer runs on, or some other "official" name controlled by the same organization. Except in cooperating subnets that agree on some other convention and don't let articles using it escape beyond the subnet, a relayer name MUST be either a UUCP name registered in the UUCP maps (without any domain suffix such as ".UUCP") or a complete Internet domain name 🌐. Using a (registered) UUCP name is recommended, where practical, to keep the length of the path list down 📏.
Internet domain names in the path list give us one little problem 🥲: domain names are case-insensitive, but the path list is case-sensitive. Relayers using domain names as their relayer names MUST pick one standard form for the name and use that form consistently to the exclusion of all others 💖. The preferred form, which relayers SHOULD use, is the all-lowercase form 🌸.
NOTE: It's arguably unfortunate that the path list is case-sensitive 😤, but it's way too late to change that. Most Internet sites do, in any event, use one standardized form of their name pretty much everywhere 🙌.
In the ordinary case, where the poster is the author of the article, the local part after the path list SHOULD be the local part of the poster's full Internet domain mailing address 📬.
NOTE: It should be just the local part, not the full address 🙅♀️. The character "@" does not appear in a Path header.
The Path content kinda looks like a mailing address, especially in the UUCP world with its manual routing and "!" address syntax 🧭. Back in the day this resemblance mattered a lot, and the Path content was often used as a reply address. That practice was always a bit unreliable, since news paths aren't always mail paths and news relayer names aren't always recognized by mail handlers, and its reliability has gotten worse lately 📉. The widespread use and recognition of Internet domain addresses, even outside the actual Internet, has mostly fixed the problem ✅. Readers SHOULD NOT use the Path content as a reply address 🚫. On the other hand, relayer administrators are urged not to break this usage without good reason 🫶, and where practical, paths followed by news SHOULD be traversable by mail, and mail handlers SHOULD recognize relayer names as host names 💌.
Gateways and moderators will usually have a hard time supplying a Path content that works as a reply address for the author 😵💫, since the path list they supply will normally be empty. (To reiterate: the path list MUST NOT contain any names other than those of relayers the article has passed through AS NEWS 🗞️.) They SHOULD supply a local part that gets replies to a Path-derived address bounced back to the sender with a brief explanation 💅. Software permitting, the local part "not-for-mail" is recommended 🫧.
NOTE: A moderator or gateway admin who supplies a local part that delivers such mail to an administrative mailbox will quickly figure out why it should get bounced automatically 💀! It's best, though, for the returned message to explain what has probably happened, rather than just a mysterious "undeliverable mail" complaint 🙏, since the sender may not know their software is unwisely using the Path content as a reply address. Reply software might want to question attempts to reply to a Path-derived address ending in "not-for-mail" (which is why a specific name is being recommended here) 🧐.
6. Optional Headers 🎀
Lots of MAIL headers, plus the ones in present and future MAIL extensions, could be useful for news 📨. Headers tied to MAIL's point-to-point transmission paradigm, like To and Cc, SHOULD NOT appear in news articles 🚫. (Gateways that want to keep that info around for debugging probably SHOULD hide it under different names. Prefixing "X-" to the original headers, giving things like "X-To", is the suggestion 🧚.)
The following optional headers are either specific to news or especially notable in news articles 🗞️, and an article MAY contain some or all of them. (Some of them are mandatory in certain circumstances, and those cases are explained under each header 📝.) An article MUST NOT contain two or more headers with any one of these header names 🙅♀️.
NOTE: The ban on duplicate header names doesn't apply to headers not specified in this Draft, like "X-" headers 🌈. Software should not assume that every header name in a given article is unique.
6.1. Followup-To 💬
The Followup-To header contents say where followups should be posted 📍:
Followup-To-content = Newsgroups-content / "poster"The syntax is the same as the Newsgroups content, with one exception: the magic word "poster" means followups should be mailed to the article's reply address instead of posted 💌. If there's no Followup-To, the default newsgroup(s) for a followup are the ones in the Newsgroups header 🫶.
NOTE: If you want followups mailed to a specific address other than the one on the From line, supply "Followup-To: poster" and a Reply-To header ✨. Putting a mailing address in the Followup-To line is incorrect 🙅♀️, and posting agents should reject or rewrite those headers.
NOTE: There's no syntax for "no followups allowed" 🤷♀️, because "Followup-To: poster" does the job without any extra machinery 🧠.
Limiting followups to the smallest reasonable set of newsgroups is generally a good vibe 🌿, especially when the precursor was cross-posted widely. Even so, posting agents SHOULD NOT supply a Followup-To header unless the poster explicitly asks for one 🙅♀️.
NOTE: It's incorrect for the posting agent to assume that followups to a cross-posted article should go to the first newsgroup only 😬. Trimming the list of newsgroups should be the poster's call, not the posting agent's 👑. Still, when an article is going to be cross-posted to a whole bunch of newsgroups, a posting agent might want to suggest to the poster that followups go to a shorter list 💖.
6.2. Expires ⏰
The Expires header content says when the article is no longer useful and should be removed ("expired") 🗑️:
Expires-content = Date-contentThe content syntax is the same as the Date content 📅. If there's no Expires, the default is decided by the administrators of each host the article reaches 🏠, and they MAY also limit how much the Expires header gets honored 🧱.
The Expires header has two main jobs 🎯. One is removing articles whose usefulness ends on a specific date, like event announcements that can go once the event has passed 🎉. The other is keeping articles that'll stay useful for a long time, like info aimed at new readers of a newsgroup 📚. The second one gets abused sometimes 😒. Since individual hosts have their own local expiration policies (depending on things like available disk space, for instance) 💾, posters SHOULD NOT provide Expires headers unless there's a natural expiration date tied to the topic 🙅♀️. Posting agents MUST NOT provide a default Expires header. Just leave it out and let local policies do their thing, unless there's a good reason not to 🫧. Expiry dates are really the decision of individual host admins 👩💻, so posters and moderators SHOULD only set expiry dates that most admins would agree with 🤝.
NOTE: If an article's usefulness ends on a specific day, a poster should typically set the expiry date to the NEXT day 🗓️. A meeting on July 7th is still of interest on the 7th 🎀.
6.3. Reply-To 📩
The Reply-To header content says which reply address to use if it's different from the author's address in the From header 💌:
Reply-To-content = From-contentIf there's no Reply-To, the reply address is just the address in the From header 💅.
Using a Reply-To header is way better than putting a similar request in the article body 🙌, because reply-preparation software can take Reply-To into account automatically 🤖✨.
6.4. Sender 💌
Okay so the Sender header tells everyone who actually hit post 💅, in case that's not the same person named in the From header 🎀. It's shaped just like From:
Sender-content = From-contentIf there's no Sender header, the default poster is the author named in the From header 💖. Easy.
NOTE: The whole point is that Sender should have a pretty high chance of naming the person who really posted the article 👀. Being able to put a From header with someone else's name is handy, but it can totally be abused 💀.
If the poster gives a From header, the posting agent MUST ensure that a Sender header is present, unless it can verify that the mailing address in the From header is a valid mailing address for the poster 🫶. A poster-supplied Sender header MAY be used, if its mailing address is verifiably a valid mailing address for the poster. Otherwise, the posting agent MUST supply a Sender header and delete (or rename, for example, to X-Unverifiable-Sender) any poster-supplied Sender header 🧚.
NOTE: It might be useful to keep a poster-supplied Sender header so the poster can supply the full-name part of the content 🌸. The mailing address, however, has to be right, so the posting agent must generate the Sender header if it can't verify the mailing address of a poster-supplied one.
NOTE: NNTP implementors, in particular, are urged to note this requirement (which would eliminate the need for ad hoc headers like NNTP-Posting-Host) 🙈. There are admittedly some implementation difficulties, though. A user name from an [RFC1413] server and a host name from an inverse mapping of the address, perhaps with a "full name" comment noting where the information came from, would be at least a first approximation:
Sender: fred@zoo.toronto.edu (RFC-1413@reverse-lookup;
not verified)While this doesn't completely meet the specs, it comes a lot closer than not having a Sender header at all. Even just a placeholder for the user name:
Sender: somebody@zoo.toronto.edu (user name unknown)Honestly, that would be better than nothing 🍓.
6.5. References 📚
The References header lists the message IDs of precursors (the posts that came before) ✨:
References-content = message-id *( space message-id )A followup MUST have a References header, and an article that is not a followup MUST NOT have a References header 🌈. The References-content of a followup MUST be the precursor's References-content (if any) followed by the precursor's message ID 💜.
NOTE: Use the See-Also header (Section 6.16) for interconnection of articles that aren't in a followup relationship with each other 🦋.
NOTE: Ngl, RFCs 850 and 1036, and the implementations whose practice they represented, got this one wrong 😭. The proper MAIL header for references to precursors is In-Reply-To, and the References header was meant for the purposes we're now giving See-Also. This incompatibility is far too solidly established to be fixed, unfortunately. The best that can be done is to provide a clear mapping between the two and urge gateways to do the transformation. The news usage is (now) a deliberate violation of the MAIL specifications. Articles containing news References headers are technically not valid MAIL messages, although it's unlikely that much MAIL software will notice, because the incompatibility is at a subtle semantic level that doesn't affect the syntax.
UNRESOLVED ISSUE: Would it be better to just give up and admit that news uses References for both purposes? 🤔
UNRESOLVED ISSUE: Should the syntax be generalized to include URLs as alternatives to message IDs? Perhaps not; too many things know about References already. And non-articles can't be precursors of articles, not really.
Followup agents SHOULD NOT shorten References headers 🙅♀️. If it is absolutely necessary to shorten the header, as a desperate last resort, a followup agent MAY do this by deleting some of the message IDs. However, it MUST NOT delete the first message ID, the last three message IDs (including that of the immediate precursor), or any message ID mentioned in the body of the followup. If it is possible for the followup agent to determine the Subject content of the articles identified in the References header, it MUST NOT delete the message ID of any article where the Subject content changed (other than by prepending of a back reference). The followup agent MUST NOT delete any message ID whose local part ends with "_-_" (underscore (ASCII 95), hyphen (ASCII 45), underscore). Followup agents are urged to use this form to mark subject changes and to avoid using it otherwise 🫧.
NOTE: As software that can exploit References chains has grown more common, the random shortening permitted by [RFC1036] has become increasingly troublesome. ANY shortening is undesirable, and software should do it only in cases of dire necessity. In such cases, these rules try to limit the damage 🛡️.
NOTE: The first message ID is very important as the starting point of the "thread" of discussion, and it absolutely should not be deleted 👑. Keeping the last three message IDs gives thread-following software a fighting chance to rebuild a full thread even if an article or two is missing. Keeping message IDs mentioned in the body is obviously desirable.
NOTE: Subject changes are difficult to determine, but they're significant as possible beginnings of new threads ✨. The "_-_" convention exists so that posting agents (which have more information about subjects) can flag articles containing a subject change in a way that followup agents can detect without access to the articles themselves. The sequence was chosen because it's fairly unlikely to occur by accident.
UNRESOLVED ISSUE: Is "_-_" really worth having? 🥺
When a References header is shortened, at least three blanks SHOULD be left between adjacent message IDs at each point where deletions were made. Software preparing new References headers SHOULD preserve multiple blanks in older References content 🌷.
NOTE: It's desirable to have some marker of where deletions occurred, but the restricted syntax of the header makes this difficult. Extra white space isn't a very good marker, since software that ill-advisedly rewrites headers may delete it, but at least it doesn't break existing software.
To repeat: followup agents SHOULD NOT shorten References headers 📢.
NOTE: Unfortunately, reading agents and other software that analyzes References patterns have to be prepared for the worst anyway 😬. The worst includes random deletions and the possibility of circular References chains (when References is misused in place of See-Also (Section 6.16)).
6.6. Control 🎮
The Control header content marks the article as a control message and specifies the desired actions (other than the usual ones of filing and passing on the article) 🕹️:
Control-content = verb *( space argument )
verb = 1*( letter / digit )
argument = 1*<ASCII printable character>The verb indicates what action should be taken, and the argument(s) (if any) supply details. In some cases, the body of the article may also contain details. Section 7 describes the standard verbs. See also the Also-Control header (Section 6.15) 💫.
NOTE: Control messages are often processed and filed rather differently than normal articles 🗂️.
NOTE: Restricting verbs to letters and digits is new, but it's consistent with existing practice and potentially simplifies implementation by avoiding characters that are significant to command interpreters. Beware that the arguments are under no such restriction in general 🙃.
NOTE: Two other conventions for telling control messages apart from normal articles were formerly in use: a three-component newsgroup name ending in".ctl", or a subject beginning with"cmsg ", was considered to imply that the article was a control message. These conventions are obsolete. Do not use them 🚫.
An article with a Control header MUST NOT have an Also-Control or Supersedes header 🔥.
6.7. Distribution 🌍✨
okay so the Distribution header is basically the "who gets to see this" setting for an article. It puts geographic or organizational limits on where the article can travel 🦋 Here's the syntax:
Distribution-content = distribution *( dist-delim distribution )
dist-delim = ","
distribution = plain-componentA distribution is syntactically identical to a one-component newsgroup name, so it has to follow the same rules and restrictions 💅 If there's no Distribution header at all, the default is "world".
NOTE, bestie: this syntax has zero white space, which means you can't continue aDistributionheader across several lines 😭 Relayer and reading agent implementors, please pay attention: the successor to this Draft is planned to change the delimiter to:dist-delim = "," [ space ]so please go ahead and fix your software now to handle (i.e., ignore) the white space after the commas 🫶
A relayer MUST NOT pass an article to another relayer unless its configuration says it sends to that relayer for BOTH (a) at least one of the article's newsgroup(s), and (b) at least one of the article's distribution(s). Honestly, the only job distributions have is to limit propagation. They stop articles from going places they would've gone if the decision had been made purely on newsgroups 🔥
A posting agent might want to show a menu of possible distributions or suggest a default, but it normally SHOULD NOT supply a default without giving the poster a chance to override it 💖 A followup agent SHOULD initially supply the same Distribution header that the precursor had, although the poster MAY change it if that makes sense.
Even though they look alike and people have historically gotten them mixed up, distributions are NOT newsgroup names 🙈 The whole point of putting a distribution on an article is that it is DIFFERENT from the newsgroup(s). Generally, a good distribution matches some kind of region of propagation: a geographical area, an organization, or a cooperating subnet 🌈
NOTE: distributions have a pretty messy history 😩 The name space is completely uncontrolled, posters get no feedback when propagation is incomplete because of random trash in Distribution headers, and people confuse them with newsgroups (partly because lots of regions and organizations DO have internal newsgroups with names that look like their internal distributions). That's produced a ton of garbage, especially the pointless habit of automatically using the first component of the newsgroup name as a distribution, which is MOST unlikely to restrict propagation! 💀 Lots of sites have chosen to maximize propagation of these ill-formed articles by basically ignoring distributions, which unfortunately gets in the way of legit uses. It's bad enough that distributions must be considered largely useless, except within cooperating subnets that make an organized effort to restrain propagation of their internal distributions.
NOTE: the distributions"world"and"local"have no standard magic meaning (except that the former is the default if none is given). Some software does give them meanings anyway ✨
6.8. Keywords 🎀
The Keywords header is one or more phrases that describe some aspect of the article's content 🧚
Keywords-content = plain-phrase *( "," [ space ] plain-phrase )The keywords are separated by commas, and each one follows the <plain-phrase> syntax defined in Section 5.2. Encoded words in keywords MUST NOT contain characters other than letters (of either case), digits, and the characters "!", "*", "+", "-", "/", "=", and "_" 💜
NOTE, posters and posting agents: keywords are separated by commas, NOT by white space! 🙃 ThisKeywordsheader only has ONE keyword (a rather unlikely and improbable one):Keywords: Thompson Ritchie Multics Linuxit should probably have been written:Keywords: Thompson, Ritchie, Multics, Linuxand this mistake is unfortunately super widespread 😭
NOTE, reading agents and archivers building indexes: user-picked keywords are notoriously bad for indexing unless they come from a predefined set (and in this case, they don't). Also, some followup agents unwisely copy theKeywordsheader from the precursor into the followup by default 🙈 At least one news-based experiment found that the contents ofKeywordsheaders were completely valueless for indexing. Iconic behavior from the keywords, honestly 💀
6.9. Summary 🌸
The Summary header is a short phrase that summarizes the article's content 🦋
Summary-content = nonblank-textJust like the subject, there's no restriction on the content, because it's only meant for humans to read 💖
NOTE: reading agents should know that theSummaryheader is often used as a sort of secondarySubjectheader. If it's there, its contents should maybe be shown whenever the subject is shown 👀
The summary SHOULD be terse. Posters SHOULD avoid cramming their entire article into the headers. Even the simplest query usually benefits from a sentence or two of elaboration and context, and not every reading agent displays every header, so don't rely on that 🫶
6.10. Approved 👑
The Approved header lists the mailing addresses (and possibly the full names) of the people or entities who approved the article for posting ✨
Approved-content = From-content *( "," [ space ] From-content )An Approved header is required in all postings to moderated newsgroups. Its presence or absence is how a posting agent tells apart articles posted by the moderator (normal articles, which get posted normally) and attempted contributions by everyone else (which should be mailed to the moderator for approval) 💅 An Approved header is also required in certain control messages, to reduce the chance of accidentally posting them. Check the relevant parts of Section 7 for those 🔥
NOTE: there's currently no way to authenticate Approved headers, so there's no proof the claimed approval was actually given 😬 There's also no established mechanism for even maintaining a list of legit approvers, because that list would go stale fast if someone had to keep it up by hand. Mechanisms like that, presumably relying on cryptographic authentication, would be a worthwhile extension to this Draft, and experimental work in this area is encouraged. (It's harder than it sounds, because news is used on many systems that don't have real-time access to key servers.) 🔐
NOTE, relayer implementors, please note well: it is the POSTING AGENT that is authorized to tell moderator postings apart from attempted contributions, and to mail the latter to the moderator. As discussed in Section 9.1, relayers MUST NOT, repeat MUST NOT, send such mail. When they get an unApproved article in a moderated newsgroup, they should discard it, NOT turn it into a mail message (except perhaps to a local administrator) 🚫
NOTE: [RFC1036] restrictedApprovedto a single From-content. But multiple moderation isn't rare anymore, and multi-moderatorApprovedheaders are already in use 🌈
6.11. Lines 🦋
The Lines header tells you how many lines are in the article's body 💖
Lines-content = 1*digitThe line count includes all body lines, including the signature (if there is one) and any empty lines (if there are any) at the beginning or end of the body. (The single empty separator line between the headers and the body is not part of the body.) The "body" here means the body as it appears in the posted article, AFTER all transformations such as MIME encodings 🙌
Reading agents SHOULD NOT rely on this header being present, since it's optional and some posting agents don't supply it. They MUST NOT rely on it being precise, because it frequently isn't 🙈
NOTE: the average line length in article bodies is surprisingly consistent at about 40 characters. The line count is usually only used for rough judgements ("is this too long to read quickly?"), so dividing the byte count of the body by 40 gives an estimate that's good enough for normal use ✨ That estimate is NOT adequate if the body has been MIME encoded, but neither is theLinesheader. At least one major relayer will add aLinesheader to an article that doesn't have one, without considering MIME encodings when computing the line count 😭
NOTE: it would be better to have aContent-Sizeheader as part of MIME, so body parts could have their own sizes, and the units could fit the data type. (Line count isn't a useful measure of the size of an encoded image, for example.) Doing that is preferable to trying to fixLines🧚
UNRESOLVED ISSUE: update on Content-Size? 🌸
Relayers SHOULD discard this header if they find it necessary to re-encode the article in a way that would make the original Lines header incorrect 💅
6.12. Xref ✨
okay so the Xref header is basically answering "where did this article actually end up?" 🗺️ it tells you where the article got filed by the last relayer that handled it, bestie 💅
Xref-content = relayer 1*( space location )
relayer = relayer-name
location = newsgroup-name ":" article-locator
article-locator = 1*<ASCII printable character>the relayer's name is in there so software can tell which relayer made the header, and (this is the big one 👀) whether it really was the relayer that filed the copy being checked 🕵️♀️ the locations tell you which newsgroups the article was filed under, which can be different from what's in the Newsgroups header, so don't assume they match 🙈 they also say where it got filed in each one. the exact form of an article locator is up to each implementation, so it's kinda a free-for-all 🎀
📝 Note: reading agents can use this to avoid showing a reader the same article over and over again 🙃 the info is sometimes in system databases too, but having it right there in the article is just so convenient. relayers traditionally only make an Xref header when the article is cross-posted, but that's not mandatory. there's at least one newer use case, "mirroring" (keeping news databases on two hosts identical 🪞), where the header is useful in every article 💖
📝 Note: the traditional locator is a decimal number, with articles in each newsgroup numbered consecutively starting from 1 🔢 NNTP [RFC977] demands that model, and other software might expect it too, but it's good to leave room for unorthodox implementations to do their own thing 🦋
okay now the rules, and these are important 🚨 a relayer that's inserting its own Xref header MUST delete any previous Xref header. a relayer that is not inserting its own Xref header SHOULD delete any previous Xref header. and a relayer MAY delete the Xref header when it passes the article on to another relayer 🤷♀️
📝 Note: [RFC1036] said the Xref header wasn't transmitted when an article got passed to another relayer, but the major news implementations have never obeyed that rule 💀 and apps like mirroring totally depend on that disobedience 😭
a relayer MUST use the same name in Xref headers as it uses in Path headers 💅 and reading agents MUST ignore an Xref header containing a relayer name that differs from the one that begins the path list 🚫
6.13. Organization 🏢
the Organization header is a short phrase that tells everyone who the poster's organization is 🎀 the content is a grammar like this:
Organization-content = nonblank-textthis one is usually filled in by the posting agent 🤖 and the Organization content SHOULD mention geographical location (like city and country) when it's not obvious from the organization's name 🌍
📝 Note: the motive here is that the organization is often hard to guess from the mailing address, isn't always in a signature, and can help identify the poster to the reader 🧚
📝 Note: there is no "s" in "Organization" 🙃
the Organization content is just for identification 🪪 it does not mean the poster speaks for the organization, and it doesn't mean the article is organization policy. posting agents SHOULD permit the poster to override a local default Organization header ✨
6.14. Supersedes 🔁
the Supersedes header says which articles get cancelled when this one shows up 💥 the content looks like this:
Supersedes-content = message-id *( space message-id )Supersedes is equivalent to Also-Control (Section 6.15) with an implicit verb of "cancel" (Section 7.1). it's giving "cancel, but make it cute" 💅
📝 Note: Supersedes is normally used where the article is an updated version of the one(s) being cancelled 🆙
📝 Note: using multiple message IDs in Supersedes is highly desirable (see Section 7.1), BUT posters should be warned that existing implementations often don't handle more than one correctly 😬
📝 Note: there is no "c" in "Supersedes" 💀
an article with a Supersedes header MUST NOT have an Also-Control or Control header 🚫
6.15. Also-Control 🎈
the Also-Control header marks the article as a control message in addition to being a normal news article, and it specifies the actions it wants 🎬
Also-Control-content = Control-contentan article with an Also-Control header gets filed and passed on normally 📬 but the content of the Also-Control header gets processed as if it were found in a Control header. main character energy 👑
📝 Note: sometimes it's desirable to piggyback control actions on a normal article, so the article is filed normally but also gets acted on as a control message 🎒 honestly this header is basically a generalization of Supersedes 🧠
📝 Note: be warned that some old relayers don't implement Also-Control ⚠️
an article with an Also-Control header MUST NOT have a Control or Supersedes header 🙅♀️
6.16. See-Also 🔗
the See-Also header lists message IDs of articles that are related to this one but are not its precursors 🧵
See-Also-content = message-id *( space message-id )See-Also kinda resembles References, but without the restrictions the followup rules put on References 🙌
📝 Note: See-Also is a way to group related articles, like the parts of a single document that had to be split across multiple articles because it was too big 📚 or to cross-reference between parallel threads 🧶
📝 Note: check out the discussion in Section 6.5 about MAIL compatibility issues of References and See-Also 📮
📝 Note: if you want to make another article part of the current one (like for annotating it), MIME's "message/external-body" convention can do that without actually including it 🦋 "news-message-ID" was registered as a standard external-body access method, with a mandatory NAME parameter giving the message ID and an optional SITE parameter suggesting an NNTP site that might have the article if it isn't available locally, by IANA 22 June 1993 📜
Unresolved issue: could the syntax be generalized to include URLs as alternatives to message IDs? 🤔 here it makes way more sense than in References 💭
6.17. Article-Names 🏷️
the Article-Names header says what special significance the article might have in particular newsgroups 🌟 the grammar is:
Article-Names-content = 1*( name-clause space )
name-clause = newsgroup-name ":" article-name
article-name = letter 1*( letter / digit / "-" )each name clause specifies a newsgroup (which SHOULD be among those in the Newsgroups header) and an article name local to that newsgroup 🗂️ article names MAY be used by relayers to file the article in special ways 📂 or they MAY just be noted for possible special attention by reading agents 👀 article names are case-sensitive, so "Intro" and "intro" are NOT the same vibe 🙃
Wait, that last bit uses NOT in caps. Fixing that below.
📝 Note: this header is a way to mark special postings, like introductions or FAQ lists, so reading agents can find them automatically 🔍 the newsgroup name is included for each article name because names can be newsgroup-specific 🧩 for example, lots of FAQ lists get posted to "news.answers" as well as their "home" newsgroup, and they wouldn't be known by the same name(s) in both 🙈
the Article-Names header SHOULD be ignored unless the article also contains an Approved header 🙅♀️
📝 Note: this stipulation is made in anticipation of Approved headers possibly being involved in cryptographic authentication 🔐
the presence of an Article-Names header doesn't necessarily mean the article will be kept unusually long before expiring 🗓️ or that previous articles with similar Article-Names headers will get cancelled when it arrives 💫 posters preparing special postings SHOULD include other appropriate headers, like Expires and Supersedes, to request those actions 🎀
different networks MAY establish different sets of article names for the special postings they think are significant 🌐 it's preferable for usage to be standardized within networks, though individual newsgroups might want different naming conventions in some situations 🫶 article names MUST be 14 characters or less 📏 the following names are suggested but are not mandatory:
intro Introduction to the newsgroup for newcomers.
charter Charter, rules, organization, moderation policies, etc.
background Biographies of special participants, history of the
newsgroup, notes on related newsgroups, etc.
subgroups Descriptions of sub-newsgroups under this newsgroup,
e.g., "sci.space.news" under "sci.space".
facts Information relating to the purpose of the newsgroup,
e.g., an acronym glossary in "sci.space".
references Where to get more information: books, journals, FTP
repositories, etc.
faq Answers to frequently asked questions.
menu If present, a list of all of the other article names
local to this newsgroup, with brief descriptions of their
contents.these articles may be split into subsections using the MIME "multipart/mixed" conventions 🧩 if they're too big and need splitting, names ending in a hyphen and a part number are suggested 🔢 so a three-part FAQ could have article names "faq-1", "faq-2", and "faq-3" 💖
📝 Note: it's kinda early to try standardizing article names, since this is basically a brand-new feature with no experience behind it 🌱 but if reading agents are going to attach special significance to these names, some attempt at standard conventions is absolutely necessary 😤 this is a first attempt at providing some ✨
6.18. Article-Updates 🔄
the Article-Updates header says which previous articles this one is deemed (by the poster) to update, meaning replace 🪄
Article-Updates-content = message-id *( space message-id )each message ID identifies a previous article that this one is deemed to update 📝 this MUST NOT cause the previous article(s) to be cancelled or otherwise altered, unless that's implied by other headers (like Supersedes) 🛡️ Article-Updates is merely an advisory that MAY be noted for special attention by reading agents 👀
📝 Note: this header is a way to mark articles that are only minor updates of previous ones, with no significant new information, and not worth reading if the previous ones have already been read 😌
📝 Note: if suitable conventions using MIME multipart bodies and the "message/external-body" body-part type can be developed, a replacing article might contain only the differences between the old text and the new text, instead of a complete new copy 🧠 that's the reason Article-Updates doesn't also function like Supersedes does: the replacing article might depend on the replaced article still being around 🫶
7. Control Messages 🎀
Okay bestie, this is where things get spicy 🌶️ Control messages are special commands that ask the network to actually do something, not just deliver a post. In this part, "message" just means "article" 💌 unless the context says otherwise.
Heads up for posting agents 🚨 Some control messages need article bodies in super specific formats, so signatures SHOULD NOT be tacked onto them 🙅♀️ It's also smart to take extra care so nobody, even with good intentions 🫶, accidentally edits the text the poster actually wrote.
Relayers MUST assume that control messages mean what they say 💅 They MAY be obeyed as is or rejected, but they MUST NOT be reinterpreted. No creative remixing, periodt 🙅♀️
Carrying out these requests is subject to local admin rules 👑 Those rules MAY deny a request or send it to an administrator for approval. The descriptions below are written like they're mandatory, but any or all of them MAY be subject to local administrative approval, either as a whole class or case-by-case. Same deal for ignoring stuff: when we say a message (or part of one) gets ignored, that action MAY include reporting it to an administrator 📣
NOTE: The exact choice of local action might depend on what action the control message requests, who it claims to come from, etc.
Relayers MUST propagate even control messages they do not understand 🤷♀️ Each type of control message gets defined syntactically, by spelling out its arguments and its body. For example, "cancel" is defined by defining cancel-arguments and cancel-body 🧩
7.1. cancel 🗑️
The cancel message requests that one or more previous articles be "cancelled":
cancel-arguments = message-id *( space message-id )
cancel-body = bodyThe arguments tell you which articles to cancel, by message ID 🆔 The body is a comment, which software MUST ignore 🙈 It SHOULD still contain a reason for the cancellation (spill the tea 🍵). The cancel message SHOULD be posted to the same newsgroup(s), with the same distribution(s), as the article(s) it's trying to cancel.
NOTE: Using the same newsgroups and distributions maximizes the chances of the cancel message propagating everywhere the target articles went. 🛣️
NOTE: [RFC1036] only allowed one message-id in a cancel message 😬 Being able to cancel multiple articles is highly desirable, especially when you're using Supersedes (see Section 6.14) 💖 Picture this: several revisions of an article show up in fast succession, each one using Supersedes to cancel the one before it. A middle revision could get destroyed by a cancellation before it gets passed along to cancel its predecessor, which is a whole mess 💀 Letting each article cancel several predecessors really helps with that. (Posting agents preparing a cancel for an article that itself cancels other articles might want to add those articles to the cancel-arguments.) Heads up though: posters should be aware that a lot of old software doesn't handle multiple cancellations properly, so they should avoid relying on it when reliable cancellation is vitally important ⚠️
When an article (the "target article") is to be cancelled, there are four cases of interest 🔍 The article hasn't arrived yet, it has arrived and been filed so it's available for reading, it has expired and been archived on some less-accessible storage medium, or it has expired and been deleted. We'll go through each case in turn, in reverse order, because it's easier to explain that way 🧠
EXPIRED AND DELETED. Take no action. 🤷♀️
EXPIRED AND ARCHIVED. If the article is readily accessible and can be deleted or made unreadable easily, treat it as under AVAILABLE below. Otherwise, treat it as under EXPIRED AND DELETED.
NOTE: Sure, it's nice when archived articles can be cancelled, but this can easily mean rewriting an entire archive volume just to get rid of one article 📚 That might even need manual steps. It's hard to picture a situation so dire that it would require this from hundreds or thousands of administrators, or one where widespread compliance with such a request is likely 🥲
AVAILABLE. Compare the mailing addresses from the From lines of the cancel message and the target article 👀 Keep in mind that local parts (except for "postmaster") are case-sensitive, but domains are case-insensitive. If they don't match, either refer the issue to an administrator for a case-by-case decision, or treat it as if they matched.
NOTE: It's generally trivial to forge articles, so nothing short of cryptographic authentication is really adequate to make sure a cancel came from the original article's author 🕵️♀️ It's also highly desirable to let authorities other than the author cancel articles. That covers cases where the author is unavailable, uncooperative, or malicious, and where damage and/or legal problems can be minimized by prompt cancellation 🚒 Reliable authentication that would permit such administrative cancels would be a worthwhile extension to this Draft, and experimental work in this area is encouraged 🧪
NOTE: Meanwhile, a simple check of addresses is useful accident prevention, and it catches at least the most simple-minded forgers 🙃 The goal is accident prevention rather than ironclad security, so using the From address is appropriate. That's especially true because, in the presence of gateways (especially redundant multiple gateways), the author may not have full control over Sender headers 🌐
NOTE: The "refer... or treat as if they matched" rule is meant to specifically forbid quietly ignoring cancels with mismatched addresses 🚫
If the addresses match, then, if technically possible, the relayer MUST delete the target article completely and immediately 💥 If that isn't possible, it MUST make the target article unreadable (preferably to everyone, minimally to everyone but the administrator) and either arrange for it to be deleted as soon as possible or notify an administrator at once 🚨
NOTE: For things like criminal actions, malicious forgeries, and copyright infringements, where damage and/or legal problems can be minimized by prompt cancellation, complete removal is strongly preferred over just making the target article unreadable 🧹 The potential for malice is outweighed by how important it is to really get rid of the target article in some legit cases. (In cases of inadvertent copyright violation in particular, being able to fix things quickly is a big deal legally ⚖️) If removal isn't possible, making it unreadable is better than nothing.
NOTE: Merely annotating the article so readers see a note that the author wanted it cancelled is not acceptable 🙅♀️ Making the article unreadable is the minimum action.
NOTE: There have been experiments with making cancelled articles unreadable, so local news administrators could reverse cancellations 🔄 In practice, administrators almost never find a reason to do so. Removal is clearly preferable wherever it's technically feasible ✨
NOT ARRIVED YET. If it's practical, retain the cancel message until the target article does arrive, or until there's no further possibility of it arriving and being accepted (see Section 9.2). Then treat it as under AVAILABLE. If that isn't practical, arrange for the target article to be rejected and discarded if it does arrive 🚫📭
NOTE: Retaining the control message can be impractical, since it's uncertain whether the target article will ever arrive 😵💫 Existing practice in such cases is to assume the addresses would match and arrange the equivalent of deletion. This is often done by making a spurious entry in a database of already-seen message IDs (see Section 9.3), so that if the article does arrive, it gets rejected as a duplicate 🪄
The cancel message MUST be propagated onward in the usual fashion, no matter which of the four cases applied 🚀 That way the target article gets cancelled everywhere, even if the cancellation and the target article follow different routes 🛤️
NOTE: [RFC1036] seemed to require stopping cancel propagation in the NOT ARRIVED YET case, although the wording was kind of unclear 🤔 That looks like an unwise decision, honestly. There are known cases of important cancellations (like inadvertent copyright violations, for example) getting way worse propagation than the target article did 😭 News propagation is often a much less orderly process than the authors of [RFC1036] apparently pictured. Modern implementations generally propagate the cancellation regardless 💜
Posting agents meant for use by ordinary posters SHOULD reject an attempt to post a cancel message if the target article is available and the mailing address in its From header doesn't match the one in the cancel message's From header 🛑
NOTE: Again, this is primarily accident prevention 🫶
7.2. ihave, sendme 🦋
okay so ihave and sendme are basically a crude, batched ancestor of the NNTP [RFC977] protocol 💅 they're mostly obsolete on the Internet, but they still get used in the UUCP environment, especially for backup feeds that only wake up when a primary feed path has failed 🔥
NOTE: The ihave and sendme messages defined here have ABSOLUTELY NOTHING TO DO WITH NNTP, despite similarities of terminology. 🙈
the two messages share the same syntax ✨
ihave-arguments = *( message-id space ) relayer-name
sendme-arguments = ihave-arguments
ihave-body = *( message-id eol )
sendme-body = ihave-bodyMessage IDs MUST appear in either the arguments or the body, but not both. 🎀 Relayers SHOULD generate the form that puts message IDs in the body, but the other form MUST be supported for backward compatibility 🫶
NOTE: [RFC1036] made the relayer name optional, but difficulties could easily ensue in determining the origin of the message, and this option is believed to be unused nowadays. 💀 Putting the message IDs in the body is strongly preferred over putting them in the arguments because it lends itself much better to large numbers of message IDs and avoids the empty-body problem mentioned in Section 4.3.1. ✨
the ihave message says the named relayer has filed articles with the specified message IDs, which might be of interest to the relayer(s) receiving the ihave message 🌸 the sendme message is the request: it asks the relayer receiving it to send the articles with the specified message IDs over to the named relayer 💜
these control messages are normally sent basically point-to-point, using "to." newsgroups (see Section 5.5) that only go to the relayer they're meant for 🧚 the two relayers MUST be neighbors, exchanging news directly with each other 👑 each relayer tells the other about its new arrivals with ihave messages, and each one uses sendme messages to ask for the articles it's missing 🍓
NOTE: Arguably these point-to-point control messages should flow by some other protocol, e.g., mail, but administrative and interfacing issues are simplified if the news system doesn't need to talk to the mail system. 🙈
to keep the overhead low, ihave and sendme messages SHOULD be sent relatively infrequently and SHOULD contain substantial numbers of message IDs 🌈 if ihave and sendme are being used to build a backup feed, it may be desirable to add a delay between receiving an ihave and generating a sendme. that way a slightly slow primary feed won't cause huge numbers of articles to get requested unnecessarily via sendme 😭
7.3. newgroup 🎀
the newgroup control message asks for a brand new newsgroup to be created 💖
newgroup-arguments = newsgroup-name [ space moderation ]
moderation = "moderated" / "unmoderated"
newgroup-body = body
/ [ body ] descriptor [ body ]
descriptor = descriptor-tag eol description-line eol
descriptor-tag = "For your newsgroups file:"
description-line = newsgroup-name space description
description = nonblank-text [ " (Moderated)" ]the first argument names the newsgroup to be created, and the second one (if it's there) says whether it's moderated 💅 if there's no second argument, the default is "unmoderated" ✨
NOTE: Implementors are warned that there is occasional use of other forms in the second argument. It is suggested that such violations of this Draft, which are also violations of [RFC1036], cause the newgroup message to be ignored. [RFC1036] was slightly vague about how second arguments other than"moderated"were to be treated (specifically, whether they were illegal or just ignored), but it is thought that all existing major implementations will handle"unmoderated"correctly, and it appears desirable to tighten up the specs to make it possible for other forms to be used in future. 🙈
the body is a comment, which software MUST ignore 🫶 the one exception: if it contains a descriptor, the description line is meant to be suitable for adding to a list of newsgroup descriptions. the description can't be continued onto later lines, but it isn't limited to any particular length 🌸 moderated newsgroups have descriptions that end with the string " (Moderated)" (note that this string begins with a blank) 🦋
NOTE: It is unfortunate that the description line is part of the body, rather than being supplied in a header, but this is established practice. Newsgroup creators are cautioned that the descriptor tag must be reproduced exactly as given above, must be alone on a line, and that it is case-sensitive. (To reduce errors in this regard, posting agents might wish to question or reject newgroup messages that do not contain a descriptor.) Given the desire for short lines, description writers should avoid content-free phrases like "discussion of" and "news about", and stick to defining what the newsgroup is about. 💖
the rest of the body SHOULD contain an explanation of the purpose of the newsgroup and the decision to create it 🧚
NOTE: Criteria for newsgroup creation vary widely and are outside the scope of this Draft, but if formal procedures of one kind or another were followed in the decision, the body should mention this. Administrators often look for such information when deciding whether to comply with creation/deletion requests. 👑
a newgroup message that lacks an Approved header MUST be ignored 🚫
NOTE: It would also be desirable to ignore a newgroup message unless its Approved header names a person who is authorized (in some sense) to create such a newsgroup. A cooperating subnet with sufficiently strong coordination to maintain a correct and current list of authorized creators might wish to do so for its internal newsgroups. It also (or alternatively) might wish to ignore a newgroup message for an internal newsgroup that was posted (or cross-posted) to a non-internal newsgroup. 🍓
NOTE: As mentioned in Section 6.10, some form of (cryptographic?) authentication of Approved headers would be highly desirable, especially for control messages. 🔥
it would be desirable to provide some way of supplying a moderator's address in a newgroup message for a moderated newsgroup, but this will cause problems unless effective authentication is available, so it is left for future work 🌈
NOTE: This leaves news administrators stuck with the annoying chore of arranging proper mailing of moderated-newsgroup submissions. On Usenet, this can be simplified by exploiting a forwarding facility that some major sites provide: they maintain forwarding addresses, each the name of a moderated newsgroup with all periods (".", ASCII 46) replaced by hyphens ("-", ASCII 45), which forward mail to the current newsgroup moderators. More advice on the subject of forwarding to moderators can be found in the document titled "How to Construct the Mailpaths File", posted regularly to the Usenet newsgroupsnews.lists,news.admin.misc, andnews.answers. 💅
a newgroup message naming a newsgroup that already exists is asking for a change to that newsgroup's moderation status or description 🦋 the same rules apply ✨
7.4. rmgroup 💀
the rmgroup message asks for a newsgroup to be deleted 🫶
rmgroup-arguments = newsgroup-name
rmgroup-body = bodythe sole argument is the newsgroup name 🎀 the body is a comment, which software MUST ignore; it SHOULD contain an explanation of the decision to delete the newsgroup 💖
NOTE: Criteria for newsgroup deletion vary widely and are outside the scope of this Draft, but if formal procedures of one kind or another were followed in the decision, the body should mention this. Administrators often look for such information when deciding whether to comply with creation/deletion requests. 👑
a rmgroup message that lacks an Approved header MUST be ignored 🚫
NOTE: It would also be desirable to ignore a rmgroup message unless its Approved header names a person who is authorized (in some sense) to delete such a newsgroup. A cooperating subnet with sufficiently strong coordination to maintain a correct and current list of authorized deleters might wish to do so for its internal newsgroups. It also (or alternatively) might wish to ignore a rmgroup message for an internal newsgroup that was posted (or cross-posted) to a non-internal newsgroup. 🍓
⚠️ WARNING: Unexpected deletion of a newsgroup is a disruptive action, so implementations are strongly advised to refer rmgroup messages to an administrator by default, unless perhaps the message can be determined to have originated within a cooperating subnet whose members are considered trustworthy. Abuses have occurred. 🔥💀
7.5. sendsys, version, whogets 💌
okay so sendsys is basically asking a relayer to mail back a description of its news feeds to other relayers, sent to the article's reply address 📬
sendsys-arguments = [ relayer-name ]
sendsys-body = bodyIf there is an argument, relayers other than the one named by the argument MUST NOT respond. The body is a comment that software MUST ignore 🙈 but it SHOULD contain an explanation of the reason for the request, bestie 🫶
version is the chill one 🧚 It requests that the name and version of the relayer software get mailed to the reply address:
version-arguments =
version-body = bodyThere are no arguments. The body is a comment, which software MUST ignore; it SHOULD contain an explanation of the reason for the request.
Then there's whogets, the glow-up of the bunch ✨ It requests that a description of the relayer and its news feeds to other relayers get mailed to the article's reply address:
whogets-arguments = newsgroup-name [ space relayer-name ]
whogets-body = bodyThe first argument is the name of the "target newsgroup", which is the newsgroup you want propagation info for 🎯 This MUST be a complete newsgroup name, not the name of a hierarchy or a portion of a newsgroup name that is not itself the name of a newsgroup. If there is a second argument, only the relayer named by that argument should respond. The body is a comment, which software MUST ignore; it SHOULD contain an explanation of the reason for the request.
NOTE: Whogets is intended as a replacement for sendsys (and version) with a precisely specified reply format. Since the syntax for specifying what newsgroups get sent to what other relayers varies widely between different forms of relayer software, the only practical way to standardize the reply format is to indicate a specific newsgroup and ask where that newsgroup propagates. The requirement that it be a complete newsgroup name is intended to (largely) avoid the problem of having to answer "yes and no" in cases where not all newsgroups in a hierarchy are sent. 🙃
Any of these messages lacking an Approved header MUST be ignored. Response to any of these messages SHOULD be delayed for at least 24 hours, and no response should be attempted if the message has been cancelled in that time. Also, no response SHOULD be attempted unless the local part of the destination address is "newsmap" 🗺️ News administrators SHOULD arrange for mail to "newsmap" on their systems to be discarded (without reply) unless legitimate use is in progress.
NOTE: Because these messages can cause many, many relayers to send mail to one person, such messages, specifying mailing to an innocent person's mailbox, have been forged as a half-witted practical joke 💀 A delay gives administrators time to notice a fraudulent message and act (by cancelling the message, preparing to divert the flood of mail into the bit bucket, or both). Restriction of the destination address to "newsmap" reduces the appeal of fraud by making it impossible to use it to harass a normal user. (A site that does not discard mail to "newsmap", but rather bounces it back, may incur higher communications costs than if the mail had been accepted into a user's mailbox, but a malicious forger could accomplish this anyway, by using an address whose local part is very unlikely to be a legitimate mailbox name.) NOTE: [RFC1036] did not require the Approved header for these control messages. This has been added because of the possibility that cryptographic authentication of Approved headers will become available. 🔐
The body of the reply to a sendsys message SHOULD be of the form:
sendsys-reply = responder 1*sys-line
responder = "Responding-System:" space domain eol
sys-line = relayer-name ":" newsgroup-patterns
[ ":" text ] eol
newsgroup-patterns = newsgroup-name *( "," newsgroup-name )The first line identifies the responding system, using a syntax resembling a header (but note that it is part of the BODY 👀). Remaining lines indicate what newsgroups are sent to what other systems. The syntax of newsgroup patterns is not well standardized; the form described is common (often with newsgroup names only partially given, denoting all names starting with a particular set of components) but not universal. The whogets message provides a better-defined alternative.
The reply to a version message is of somewhat ill-defined form, with a body normally consisting of a single line of text that somehow describes the version of the relayer software. The whogets message provides a better-defined alternative. 🌸
The body of the reply to a whogets message MUST be of the form:
whogets-reply = responder-domain responder-relayer
response-date responding-to arrived-via
responder-version whogets-delimiter
*pass-line
responder-domain = "Responding-System:" space domain eol
responder-relayer = "Responding-Relayer:" space relayer-name eol
response-date = "Response-Date:" space date eol
responding-to = "Responding-To:" space message-id eol
arrived-via = "Arrived-Via:" path-list eol
responder-version = "Responding-Version:" space nonblank-text eol
whogets-delimiter = eol
pass-line = relayer-name [ space domain ] eolThe first six lines identify the responding relayer by its Internet domain name (use of the ".uucp" and ".bitnet" pseudo-domains is permissible, for registered hosts in them, but discouraged) and its relayer name 🏷️ They also specify the date when the reply was generated and the message ID of the whogets message being replied to, give the path list (from the Path header) of the whogets message (which MAY, if absolutely necessary, be truncated to a convenient length, but MUST contain at least the leading three relayer names), and indicate the version of relayer software responding. Note that these lines are part of the BODY even though their format resembles that of headers. Despite the apparently fixed order specified by the syntax above, they can appear in any order, but there must be exactly one of each.
After those preliminaries, and an empty line to unambiguously define their end, the remaining lines are the relayer names (which MAY be accompanied by the corresponding domain names, if known) of systems to which the responding system passes the target newsgroup. Only the names of news relayers are to be included. 💅
NOTE: It is desirable for a reply to identify its source by both domain name and relayer name because news propagation is governed by the latter but location in a broader context is best determined by the former. The date and whogets message ID should, in principle, be present in the MAIL headers but are included in the body for robustness in the presence of uncooperative mail systems. The reason for the path list is discussed below. Adding version information eliminates the need for a separate message to gather it. NOTE: The limitation of pass lines to contain only names of news relayers is meant to exclude names used within a single host (as identifiers for mail gateways, portions of ihave/sendme implementations, etc.), which do not actually refer to other hosts. 🧠
A relayer that is unaware of the existence of the target newsgroup MUST NOT reply to a whogets message at all, although this MUST NOT influence decisions on whether to pass the article on to other relayers. 🤫
NOTE: While this may result in discontinuous maps in cases where some hosts have not honored requests for creation of a newsgroup, it will also prevent a flood of useless responses in the event that a whogets message intended to map a small region "leaks" out to a larger one. The possibility of discontinuous recognition of a newsgroup does make it important that the whogets message itself continue to propagate (if other criteria permit). This is also the reason for the inclusion of the whogets message's path list, or at least the leading portion of it, in the reply: to permit reconstruction of at least small gaps in maps. 🗺️
Different networks set different rules for the legitimacy of these messages, given that they may reveal details of organization-internal topology that are sometimes considered proprietary. 🕵️
NOTE: On Usenet, in particular, willingness to respond to these messages is held to be a condition of network membership: the topology of Usenet is public information. Organizations wishing to belong to such networks while keeping their internal topology confidential might wish to organize their internal news software so that all articles reaching outsiders appear to be from a single "gatekeeper" system, with the details of internal topology hidden behind that system. 🔒
UNRESOLVED ISSUE: It might be useful to have a way to set some sort of hop limit for these. 🚧
7.6. checkgroups 📋
The checkgroups control message contains a supposedly authoritative list of the valid newsgroups within some subset of the newsgroup name space 👑
checkgroups-arguments =
checkgroups-body = [ invalidation ] valid-groups
/ invalidation
invalidation = "!" plain-component
*( "," plain-component ) eol
valid-groups = 1*( description-line eol )There are no arguments. The body lines (except possibly for an initial invalidation) each contain a description line for a newsgroup, as defined under the newgroup message (Section 7.3).
NOTE: Some other, ill-defined, forms of the checkgroups body were formerly used. See Appendix A. 📎
The checkgroups message applies to all hierarchies containing any of the newsgroups listed in the body. It asserts that the newsgroups it lists are the only newsgroups in those hierarchies. If there is an invalidation, it asserts that the hierarchies it names no longer contain any newsgroups. 🪦
Processing a checkgroups message MAY cause a local list of newsgroup descriptions to be updated. It SHOULD also cause the local lists of newsgroups (and their moderation statuses) in the mentioned hierarchies to be checked against the message. The results of the check MAY be used for automatic corrective action or MAY be reported to the news administrator in some way. 🔍
NOTE: Automatically updating descriptions of existing newsgroups is relatively safe. In the case of newsgroup additions or deletions, simply notifying the administrator is generally the wisest action, unless perhaps the message can be determined to have originated within a cooperating subnet whose members are considered trustworthy. 🤝 NOTE: There is a problem with the checkgroups concept: not all newsgroups in a hierarchy necessarily propagate to the same set of machines. (Notably, there is a set of newsgroups known as the "inet" newsgroups, which have relatively limited distribution but coexist in several hierarchies with more widely distributed newsgroups.) The advice of checkgroups should always be taken with a grain of salt and should never be followed blindly. 🧂💅
8. Transmission Formats ✨
Okay bestie, so this Draft doesn't actually spec out transmission methods 🚚, except it does slap a few constraints on them. There are still some data formats that only exist for transmission, and they're totally unique to news 💅.
8.1. Batches
When you want to send a bunch of news articles efficiently, you can bundle them into one block of data 🎁. That bundle is called a "batch" 🌸. Here's the format:
batch = 1*( batch-header article )
batch-header = "#! rnews " article-size eol
article-size = 1*digitSo a batch is a sequence of articles, and each one gets a header line in front that includes its size 📏. The article size is a decimal count of the octets in the article. Each EOL counts as one octet, no matter how it's actually represented 🤓.
NOTE: A relayer might wish to accept either a single article or a batch as input. Since # cannot appear in a header name, checking the first octet of the input will reveal which one it is 🔍.
NOTE: In the header line, there is exactly one blank before "rnews", there is exactly one blank after "rnews", and the EOL immediately follows the article size. Beware that some software inserts non-standard trash after the size 🗑️.
NOTE: This looks a lot like the executable-script format some operating systems use, but it is EXTREMELY unwise to just feed incoming batches to a command interpreter, hoping it will run a command named "rnews" to process the batch 🙅♀️. Unless arrangements are made to very tightly restrict which commands can be run this way, the security implications are disastrous 💀.
8.2. Encoded Batches
When you're sending news over links that are slow or billed by the bit 💸, it's often smart to batch the news and compress the batches 🗜️. Transmission links that send compressed batches SHOULD use out-of-band means of communication to specify the compression algorithm being used 📬. If there's no way to send out-of-band information along with a batch, the following encapsulation for a compressed batch MAY be used:
ec-batch = "#! " compression-keyword eol
compressed-batch
compression-keyword = "cunbatch"So a line with a keyword for the compression type comes first, followed by the compressed batch 🎀. The only truly widespread compression keyword right now is "cunbatch", which means compression with the widely distributed "compress" program 🌈. Other compression keywords MAY be used if the hosts involved mutually agree on them 🤝.
NOTE: An encapsulated compressed batch is not, in general, a text file, even though it has a text line at the start 😭. This mix of text and non-text data is super awkward to handle. For example, standard decompression programs can't be used until you strip off that first line, and stripping it is painful because lots of text tools that look like they'd work don't cope well with non-text data 🙈. That's why out-of-band communication is recommended whenever possible.
NOTE: For UUCP transmission, a batch is typically sent by invoking the remote command "rnews" with the batch as its input stream 📨. A plausible out-of-band method for indicating the compression type would be to give a compression keyword in an option to "rnews", perhaps like this:
rnews -d decompressorHere, "decompressor" is the name of a decompression program. For example, "uncompress" works for a batch compressed with "compress", or "gunzip" for one compressed with "gzip" 🧚. How the receiving relayer locates and invokes this decompression program is implementation-specific 🛠️.
NOTE: See the notes in Section 8.1 about why you should never feed batches directly to command interpreters 🚨.
NOTE: There is exactly one blank between #! and the compression keyword, and the EOL immediately follows the keyword 📏.
8.3. News within Mail
Sometimes you want to send news as mail 💌, either because it's easier for a human to read or because mail is the only option on a restrictive communication path 🔒.
Since the news format and the MAIL format look so similar, it seems like a great idea to just send the news article as a mail message 🥹. This is typically a mistake 🙅♀️. Mail-handling software often feels free to mess with various headers in undesirable ways (in some cases, like Sender, that manipulation is actually mandatory 😤), and mail transmission problems, etc. MUST be reported to the administrators responsible for the mail transmission, not to the article's author 📣. In general, news sent as mail should be encapsulated so the MAIL headers and the news headers are separated 🎁.
When the intended recipient is a human, any convenient form of encapsulation may be used 💖. Recommended practice is MIME encapsulation with a content type of "message/news", since news articles have extra semantics beyond what "message/rfc822" implies ✨.
NOTE: "message/news" was registered as a standard subtype by IANA on 22 June 1993 🎉.
When mail is the transmission path between two relayers, though, you want a standard method 🤝. Right now, the standard method is to send the mail to an address whose local part is "rnews", with whatever MAIL headers are needed for successful transmission 📮. The news article (headers included) goes in as the body of the mail message, with an "N" prepended to each line 🧁.
NOTE: The "N" lowers the chance that an innocent line in a news article gets taken as a magic command by mail software 🪄. It also makes it easy for receiving software to strip off any lines added by mail software, like the trailing empty line some UUCP mail software adds 🧹.
This method has its weaknesses 😬. Mainly, it assumes the mail channel can carry nearly arbitrary body text undamaged. When mail is the transmission path of last resort, the mail system often has annoying preconceived notions about how message bodies should look 🙄. People have used various ad hoc encoding schemes to dodge these problems. The recommended method is to send a news article or batch as the body of a MIME mail message, using the content type "application/news-transmission" and MIME's "base64" encoding (which is specifically designed to survive all known major mail systems) 🫶.
NOTE: MIME conventions could be used to fragment and reassemble an article that's too large to send as a single mail message over a path that restricts message length 🧩. Also, the "conversions" parameter of the content type could be used to indicate which compression method (if any) was used 🗜️. And the Content-MD5 header [RFC1544] can work as a "checksum", giving high confidence of detecting accidental damage to the contents 🔍.
UNRESOLVED ISSUE: The "conversions" parameter no longer exists. What should be done about this, if anything? 🤔
NOTE: It might look tempting to use a content type like "message/X-netnews", but MIME bans non-trivial encodings of the entire body of messages with content type "message" 🚫. The intent is to avoid obscuring nested structure underneath encodings. For inter-relayer news transmission there's no nested structure of interest, and it's important that the entire article (headers included, not just the body) is protected against the vagaries of intervening mail software 🛡️. This situation seems to fit the MIME description of when "application" is the proper content type 💅.
NOTE: "application/news-transmission", with a "conversions" parameter, was registered as a standard subtype by IANA on 22 June 1993 🎉.
UNRESOLVED ISSUE: The "conversions" parameter no longer exists in MIME. What should we do about this? 😵💫
8.4. Partial Batches
UNRESOLVED ISSUE: The existing batch conventions bundle (potentially) many articles into one batch 📦. Handling very large articles would be a lot less troublesome if there were also a fragmentation convention for splitting one large article across several batches ✂️. Is this worth defining at this time? 🤷♀️
9. Propagation and Processing 🌊
Most aspects of news propagation and processing are implementation-specific 🛠️. Still, the basic propagation algorithms, and certain details of how they're implemented, need to be standard 📏.
There are two important principles that news implementors (and administrators) need to keep in mind 🧠. The first is the well-known Internet Robustness Principle:
Be liberal in what you accept, and conservative in what you send.
But for news there's an even more important principle 👑. It comes from a much older code of practice, the Hippocratic Oath, so we'll call it the Hippocratic Principle 🩺:
First, do no harm.
It is VITAL to realize that decisions that might be merely suboptimal in a smaller context can turn into devastating mistakes when they're amplified by the actions of thousands of hosts within a few hours 😱. Protect the network like your life depends on it 🫶.
9.1. Relayer General Issues 💅
Relayers MUST NOT alter the content of articles unnecessarily 🙅. Well-intentioned attempts to "improve" headers, in particular, typically do more harm than good 💀. It is necessary for a relayer to prepend its own name to the Path content (see Section 5.6) 🛤️ and permissible for it to rewrite or delete the Xref header (see Section 6.12) ✨. Relayers MAY delete the thoroughly obsolete headers described in Appendix A.3 🧹, although this behavior no longer seems useful enough to encourage. Other alterations SHOULD be avoided at all costs, as per the Hippocratic Principle 🩺.
NOTE: okay so, as discussed in Section 2.3, tidying up the headers of a user-prepared article is the job of the posting agent, not the relayer 💅. The relayer's purpose is to move already-compliant articles around efficiently without damaging them 🚚. Note that in existing implementations, specific programs may contain both posting-agent functions and relayer functions 🧩. The distinction is that posting-agent functions are invoked only on articles posted by local posters, never on articles received from other relayers 🫶.
NOTE: a particular corollary of this rule is that relayers should not add headers unless truly necessary 🙈. In particular, this is not SMTP; do not add Received headers 📮.
Relayers MUST NOT pass non-conforming articles on to other relayers, except perhaps in a cooperating subnet that has agreed to permit certain kinds of non-conforming behavior 🚫. This is a direct consequence of the Internet Robustness Principle 🧠.
The two preceding paragraphs may appear to be in conflict 🤔. What is to be done when a non-conforming article is received? The Robustness Principle argues that it should be accepted but must not be passed on to other relayers while still non-conforming, and the Hippocratic Principle strongly discourages attempts at repair. The conclusion that this appears to lead to is correct 🌸: a non-conforming article MAY be accepted for local filing and processing, or it MAY be discarded entirely, but it MUST NOT be passed on to other relayers.
A relayer MUST NOT respond to the arrival of an article by sending mail to any destination, other than a local administrator, except by explicit prearrangement with the recipient. Neither posting an article (other than certain types of control messages; see Section 7.5) nor being the moderator of a moderated newsgroup constitutes such prearrangement. UNDER NO CIRCUMSTANCES WHATSOEVER may a relayer attempt to send mail to either an article's originator or a moderator 🚨.
NOTE: reporting apparent errors in message composition is the job of a posting agent, not a relayer 📝. The same is true of mailing moderated-newsgroup postings to moderators. In networks of thousands of cooperating relayers, it is simply unacceptable for there to be any circumstance whatsoever that causes any significant fraction of them to simultaneously send mail to the same destination. (Some control messages are exceptions, although perhaps ill-advised ones.) What might, in a smaller network, be a useful notification or forwarding becomes a deluge of nearly identical messages that can bring mail software to its knees and severely inconvenience recipients. Moderators, in particular, historically have suffered grievously from this 😭.
Notification of problems in incoming articles MAY go to local administrators, or at most (by prearrangement!) to the administrators of the neighboring relayer(s) that passed on the problematic articles 📬.
NOTE: it would be desirable to notify the author that their posting is not propagating as they expect 🥺. However, there is no known method for doing this that will scale up gracefully. (In particular, "notify only if within N relayers of the originator" falls down in the presence of commercial news services like UUNET: there may be hundreds or thousands of relayers within a couple of hops of the originator.) The best that can be done right now is to notify neighbors, in hopes that the word will eventually propagate up the line, or organize regional monitoring at major hubs 🗼.
If it is necessary to alter an article, e.g., translate it to another character set or alter its EOL representation, strenuous efforts should be made to ensure that such transformations are reversible, and that relayers or other software that might wish to reverse them know exactly how to do so 🔁.
NOTE: for example, a cooperating subnet that exchanges articles using a non-ASCII character set like EBCDIC should define a standard, reversible ASCII-EBCDIC mapping and take pains to see that it is used at all points where the subnet meets the outside. If the only reason for using EBCDIC is that the readers typically employ EBCDIC devices, it would be more robust to employ ASCII as the interchange format and do the transformation in the reading and posting agents 🧃.
9.2. Article Acceptance and Propagation ✨
When a relayer first receives an article, it must decide whether to accept it 🚪. (This applies regardless of whether the article arrived by itself or as part of a batch, and in principle regardless of whether it originated as a local posting or as traffic from another relayer.) In a cooperating subnet with well-controlled propagation paths, some of the tests specified here MAY be delegated to centrally located relayers; that is, relayers that can receive news ONLY via one of the central relayers might simplify acceptance testing based on the assumption that incoming traffic has already passed the full set of tests at a central relayer 🏰.
The wording that follows is based on a model in which articles arrive on a relayer's host before acceptance tests are done. However, depending on the degree of integration of the transport mechanisms and the relayer, some or all of these tests MAY be done before the article is actually transmitted, so that articles that definitely will not be accepted need not be transmitted at all 🙌.
The wording that follows also specifies a particular order for the acceptance tests. While this order is the obvious one, the tests MAY be done in any order 🔀.
First, the relayer MUST verify that the article is a legal news article, with all mandatory headers present with legal contents ✅.
NOTE: this check in principle is done by the first relayer to see an article, so an article received from another relayer should always be legal, but there is enough old software still operational that this cannot be taken for granted; see the discussion of the Internet Robustness Principle in Section 9.1 🕰️.
Second, the relayer MUST determine whether it has already seen this article (identified by its message ID) 👀. This is normally done by retaining a history of all article message IDs seen in the last N days, where the value of N is decided by the relayer's administrator but SHOULD be at least 7. Since N cannot practically be infinite, articles whose Date content indicates that they are older than N days are declared "stale" and are deemed to have been seen already 🗑️.
NOTE: this check is important because news propagation topology is typically redundant, often highly so, and it is not at all uncommon for a relayer to receive the same article from several neighbors 👯. The history of already-seen message IDs can get quite large, hence, the desire to limit its length, but it is important that it be long enough that slowly propagating articles are not classed as stale. News propagation within the Internet is normally very rapid, but when UUCP links are involved, end-to-end delays of several days are not rare, so a week is not a particularly generous minimum 📆.
NOTE: despite generally more rapid propagation in recent times, it is still not unheard of for some propagation paths to be very slow. This can introduce the possibility of old articles arriving again after they are gone from the history, hence the "stale" rule 🧟.
Third, the relayer MUST determine whether any of the article's newsgroups are "subscribed to" by the host, i.e., fit a description of what hierarchies or newsgroups the site wants to receive 🎯.
NOTE: this check is significant because information on what newsgroups a relayer wishes to receive is often stored at its neighbors, who may not have up-to-date information or may simplify the rules for implementation reasons. As a hedge against the possibility of missed or delayed newgroup control messages, relayers may wish to observe a notion of a newsgroup subscription that is independent of the list of newsgroups actually known to the relayer. This would permit reception and relaying of articles in newsgroups that the relayer is not (yet) aware of, subject to more general criteria indicating that they are likely to be of interest 🦋.
Once an article has been accepted, it may be passed on to other relayers 📤. The fundamental news propagation rule is a flooding algorithm: on receiving and accepting an article, send it to all neighboring relayers not already in its path list that are sent its newsgroup(s) and distribution(s) 🌊.
NOTE: the path list's role in loop prevention may appear relatively unimportant, given that looping articles would typically be rejected as duplicates anyway. However, the path list's role in preventing superfluous transmissions is not trivial. In particular, the path list is the only thing that prevents relayer X, on receiving an article from relayer Y, from sending it back to Y again. (Indeed, the usual symptom of confusion about relayer names is that incoming news loops back in this manner.) The looping articles would be rejected as duplicates, but doubling the communications load on every news transmission path is not to be taken lightly! 🔁💥
In general, relayers SHOULD NOT make propagation decisions by "anticipation" 🔮: relayer X, noting that the article's path list already contains relayer Y, decides not to send it to relayer Z because X anticipates that Z will get the article by a better path. If that is generally true, then why is there a news feed from X to Z at all? In fact, the "better path" may be running slowly or may be down. News propagation is very robust precisely because some redundant transmission is done "just in case" 🛟. If it is imperative to limit unnecessary traffic on a path, use of NNTP [RFC977] or ihave/sendme (see Section 7.2) to pass articles only when necessary is better than arbitrary decisions not to pass articles at all 🧠.
Anticipation is occasionally justified in special cases. Such cases should involve both (1) a cooperating subnet whose propagation paths are well-understood and well-monitored, with failures and slowdowns noticed and dealt with promptly, and (2) a persistent pattern of heavy unnecessary traffic on a path that is either slow or costly. In addition, there should be some reason why neither NNTP nor ihave/sendme is suitable as a solution to the problem 🫶.
9.3. Administrator Contact 💌
okay so it's desirable to have a standardized contact address for a relayer's admins, kinda like the postmaster address for mail admins 📬. Mail addressed to newsmaster on a relayer's host MUST go to the administrator(s) of that relayer. Mail addressed to usenet on the relayer's host SHOULD be handled likewise. Mail addressed to either address on other hosts using the same news database SHOULD be handled likewise 🫶.
NOTE: these addresses are case-sensitive 💅, but it would be desirable for sequences equivalent to them under case-insensitive comparison to be handled likewise. Whilenewsmasterseems like the preferred network-independent address (by analogy topostmaster), there's an existing practice of usingusenetfor this purpose, sousenetshould be supported if at all possible (especially on hosts belonging to Usenet!). The addressnewsis also sometimes used for purposes like this, but less consistently. 🌸
10. Gatewaying 🌉
okay so gatewaying traffic between news networks using this Draft and networks using other exchange mechanisms can be super useful, but it must be done cautiously 🫣. Gateway administrators are taking on big responsibilities, and they must recognize that the consequences of error can be quite serious 💀.
10.1. General Gatewaying Issues 🧩
this section is mostly about the problems of gatewaying traffic INTO news networks 📥. Not much can be said about the other direction without some specific knowledge of the network(s) involved. BUT the two issues aren't totally independent 👀: if a non-news network is gatewayed into a news network at more than one point, traffic injected into the non-news network by one gateway may show up at another as a candidate for injection back into the news network.
this raises a more general principle, the single most important issue for gatewaying:
Above all, prevent loops. 🚫🔁
The normal loop prevention of news transmission is vitally dependent on the Message-ID header 🆔. Any gateway that finds it necessary to remove this header, alter it, or supersede it (by moving it into the body) MUST take equally effective precautions against looping.
NOTE: there are few things more effective at turning news readers into a lynch mob than a malfunctioning gateway, or pair of gateways, that takes in news articles, mangles them just enough to stop news relayers from recognizing them as duplicates, and regurgitates them back into the news stream. This happens rather too often 😩.
Gateway implementors should realize that gateways have all of the responsibilities of relayers, plus the added complications introduced by transformations between different information formats 🧠. Much of the discussion in Section 9 about relayer issues is relevant to gateways as well. In particular, gateways SHOULD keep a history of recently seen articles, as described in Section 9.2, and not assume that articles will never reappear. This is particularly important for networks that have their own concept analogous to message IDs: a gateway should keep a history of traffic seen from BOTH directions 🪞.
If at all possible, articles entering the non-news network SHOULD be marked in some way so that they will NOT be re-gatewayed back into news 🏷️. Multiple gateways obviously must agree on the marking method used. If it is done by having them know each other's names, name changes MUST be coordinated with great care. If marking cannot be done, all transformations MUST be reversible so that a re-gatewayed article is identical to the original (except perhaps for a longer Path header).
Gateways MUST NOT pass control messages (articles containing Control, Also-Control, or Supersedes headers) without removing the headers that make them control messages, unless there are compelling reasons to believe that they are relevant to both sides and that conventions are compatible. If it is truly desirable to pass them unaltered, suitable precautions MUST be taken to ensure that there is NO POSSIBILITY of a looping control message 🙅♀️.
NOTE: the damage done by looping articles is multiplied a thousandfold if one of the affected articles is something like asendsysmessage (see Section 7.5) that requests multiple automatic replies 💥. Most gateways simply should not pass control messages at all. If some unusual reason dictates doing so, gateway implementors and administrators are urged to consider bulletproof rate-limiting measures for the more destructive ones likesendsys, e.g., passing only one per hour no matter how many are offered ⏳.
Gateways, like relayers, SHOULD make determined efforts to avoid mangling articles unnecessarily ✨. For gateways, some transformations may be inevitable, but keeping them to a minimum and making sure they're reversible is still highly desirable.
Gateways MUST avoid destroying information 🫙. In particular, the restrictions of Section 4.2.2 are best taken with a grain of salt in the context of gateways. Information that does not translate directly into news headers SHOULD be retained, perhaps in X- headers, both because it may be of interest to sophisticated readers and because it may be crucial to tracing propagation problems 🕵️♀️.
Gateway implementors should take particular note of the discussion of mailed replies, or more precisely the ban on same, in Section 9.1 📮. Gateway problems MUST be reported to the local administration, NOT to the innocent originator of traffic 🥺. "Gateway problems" here includes all forms of propagation anomaly on the non-news side of the gateway, e.g., unreachable addresses on a mailing list. Note that this requires consideration of possible misbehavior of "downstream" hosts, not just the gateway host 👀.
10.2. Header Synthesis ✍️
News articles prepared by gateways MUST be legal news articles 📜. In particular, they MUST include all of the mandatory headers (see Section 5) and MUST fully conform to the restrictions on said headers. This often means a gateway has to act not only as a relayer but also partly as a posting agent, helping synthesize a conforming article from non-conforming input 🛠️.
NOTE: the full-conformance requirement needs particularly careful attention when gatewaying mailing lists to news, because a number of constructs that are legal in MAIL headers are NOT permissible in news headers. (Note also that not all mail traffic fully conforms to even the MAIL specification 🙃.) The rest of this section will be phrased in terms of mail-to-news gatewaying, but most of it is more generally applicable.
The mandatory headers generally present few problems 😌.
If no date information is available, the gateway should supply a Date header with the gateway's current date. If only partial information is available (e.g., date but not time), this should be fleshed out to a full Date header by adding default values, NOT by mixing in parts of the gateway's current date. (Defaults should be chosen so that fleshed-out dates will not be in the future!) It may be necessary to map time zone information to the restricted forms permitted in the news Date header. See Section 5.1 🕰️.
NOTE: the prohibition of mixing dates is on the theory that it is better to admit ignorance than to lie 🫣.
If the author's address as supplied in the original message is not suitable for inclusion in a From header, the gateway MUST transform it so it is (for example, by use of the "% hack" and the domain address of the gateway) 📬. The desire to preserve information is NOT an excuse for violating the rules 🚫. If the transformation is drastic enough that there is reason to suspect loss of information, it may be desirable to include the original form in an X- header, but the From header's contents MUST be as specified in Section 5.2.
If the message contains a Message-ID header, the contents should be dealt with as discussed in Section 10.3. If there is no message ID present, it will be necessary to synthesize one, following the news rules (see Section 5.3) 🪄.
Every effort should be made to produce a meaningful Subject header; see Section 5.4 🎀. Many news readers select articles to read based on Subject headers, and inserting a placeholder like "<no subject available>" is considered highly objectionable 🙅♀️. Even synthesizing a Subject header by picking out the first half-dozen nouns and adjectives in the article body is better than using a placeholder, since it offers SOME indication of what the article might contain 💡.
The contents of the Newsgroups header (Section 5.5) are usually predetermined by gateway configuration, but a gateway to a network that has its own concept of newsgroups or discussions might have to make transformations 🔀. Such transformations should be reversible; otherwise, confusion is likely on both sides 😵💫.
It will rarely be possible for gateways to provide a Path header that is both an accurate history of the relayers the article has passed through AS NEWS and a usable reply address 🥲. The history function MUST be given priority; see the discussion in Section 5.6. It will usually be necessary for a gateway to supply an empty path list, abandoning the reply function 🫠.
It is desirable for gatewayed articles to convey as much useful information as possible, e.g., by use of optional news headers (see Section 6) when the relevant information is available 📦. Synthesis of optional headers can generally follow similar rules.
Software synthesizing References headers should note the discussion in Section 6.5 concerning the incompatibility between MAIL and news 🧵. Also of interest is the possibility of incorporating information from In-Reply-To headers and from attribution lines in the body. An incomplete or somewhat conjectural References header is much better than none at all, and reading agents already have to cope with incomplete or slightly erroneous References lists 🦋.
10.3. Message ID Mapping ✨
Okay so this section is written as if mail is getting gatewayed into news, but honestly most of it applies way beyond that 🎀. Not every MAIL message ID is legal in news, because the news syntax (specified in Section 5.3, with related material in Section 5.2) is much stricter 💅. So turning a mail message into a fully conforming news article may mean transforming the message ID a bit 🦋.
It gets extra important when the same mailing list is being handled by more than one gateway 👑. You really want the same article not to show up twice in the same newsgroup, which means every gateway has to give it the same message ID. So gateways SHOULD use the following algorithm (possibly tweaked by the later discussion of gatewaying into more than one newsgroup) unless local stuff calls for something else 🫶:
- Separate the message ID from its surroundings, if needed. A plausible way to do this is to start at the first
"<", end at the next">", and reject the message if there's no">"or a second"<"shows up before it. Also reject the message if the message ID has no"@"or more than one"@", or if it has no".". And reject it if the message ID has non-ASCII characters, ASCII control characters, or white space 🙈.
NOTE: Any legitimate domain will include at least one ".". [RFC822], Section 6.2.2, forbids white space in this context when passing mail on to non-MAIL software.
- Delete the leading
"<"and trailing">". Then split the message ID into local part and domain at the"@"🍓. - In both components, change leading dots (
".", ASCII 46), trailing dots, and any dots after the first in a run of two or more consecutive dots into underscores (ASCII 95) 🌸. - In both components, change any disallowed characters other than dots (see the definition of
<unquoted-char>in Section 5.2) into underscores (ASCII 95) ✨. - Form the message ID like this:
"<" local-part "@" domain ">"NOTE: This algorithm is approximately that of Rich Salz's successful gatewaying package.
Now here's the subtle part 🧚. If the same articles get gatewayed into more than one newsgroup, and you can't get all the gateways to cross-post them to the exact same set of newsgroups, then the message IDs in the different newsgroups MUST be DIFFERENT. Ngl this is kind of a plot twist 😭.
NOTE: Otherwise, arrival of an article in one newsgroup will prevent it from appearing in another, and which newsgroup a particular article appears in will be an accident of which direction it arrives from first. It is very difficult to maintain a coherent discussion when each participant sees a randomly selected 50% of the traffic. The fundamental problem here is that the basic assumption behind message IDs is being violated: the gateways are assigning the same message ID to articles that differ in an important respect (Newsgroups header).
In those cases, the suggestion is to stick the newsgroup name, or an agreed-on abbreviation of it, onto the front of the local part of the message ID (with a separating "."), done by the gateway 🌈. That way multiple gateways still make the same message ID, and different newsgroups can still be read independently. Iconic, honestly 💜.
NOTE: It is preferable to have the gateway(s) cross-post the article, avoiding the issue altogether, but this may not be feasible, especially if one newsgroup is widespread and the other is purely local.
10.4. Mail to and from News 💌
Gatewaying mail to news, and the other way round, is the most obvious kind of news gatewaying 💖. Honestly, people set up gateways between news and mail way too casually, and it shows 😬.
It's hard to mess up gatewaying news into a mailing list, except for one annoying detail: you have to make sure error reports go to the local administration and not to the authors of the news articles 🙃. (That means paying attention to the "envelope address" as well as the message headers.) Going the reverse direction correctly is way harder than it looks.
NOTE: In particular, just feeding the mail message to inews -h or the equivalent is NOT, repeat NOT, adequate to gateway mail to news. Significant gatewaying software is necessary to do it right. Not all headers of mail messages conform to even the MAIL specifications, never mind the stricter rules for news. 💀
There are two different flavours of mail-to-news gatewaying worth telling apart ✨. One is gatewaying a mailing list into a newsgroup. The other is running a "post-by-mail" service, where people can post individual articles to a newsgroup just by emailing them to a specific address 🦋. In the first case the message is already being "broadcast", so you can think of it as gatewaying one form of news into another. The second case is more like a moderator posting submissions to a moderated newsgroup 👑.
Either way, the discussions in the two preceding sections matter, and so does the Hippocratic Principle of Section 9 🫶. There are also some extra considerations that are specific to mail-to-news gatewaying.
As mentioned in Section 6, point-to-point headers like To and Cc SHOULD NOT appear as such in news 🙅♀️. Instead, the suggestion is to change them into "X-" headers, like X-To and X-Cc, so the info stays around for readers or troubleshooters to use 🔍. The Received header is totally specific to MAIL, so it SHOULD be deleted completely during gatewaying. The one exception is perhaps the Received header that the gateway host itself adds 🌸.
The Sender header is a tricky one, where mailing-list and post-by-mail practice should differ 🤔. For gatewaying mailing lists, the mailing-list host should be considered a relayer, and the From and Sender headers in its transmissions should be left strictly untouched. For post-by-mail, like a moderator posting a mailed submission, the Sender header should reflect the poster rather than the author. If a post-by-mail gateway gets a message with its own Sender header, it might want to keep that content in an X-Sender header 🎀.
You'll usually need to translate between mail's In-Reply-To/References convention and news's References/See-Also convention, so cross references still mean the right thing ✨. This matters going the other way too, from news to mail. Check out the difference in Section 6.5 🦋.
10.5. Gateway Administration 🛠️
Any news system does better with an attentive administrator, ideally backed up by automated monitoring for anomalies 👀. That's especially true for gateways. Gateway software SHOULD be instrumented so that weird stuff, like sudden massive surges in traffic, gets reported promptly 🚨. It's also desirable, honestly, to go further: gateway software SHOULD endeavor to limit the damage if the administrator doesn't respond fast 😤.
NOTE: For example, software might limit the gatewaying rate by queueing incoming traffic and emptying the queue at a finite maximum rate (well below the maximum that the host is capable of!) that is set by the administrator and is not raised automatically.
Traffic gatewayed into a news network SHOULD include a suitable header, maybe X-Gateway-Administrator, giving an electronic address people can use to report problems 📮. That address SHOULD go directly to a human, not to a "routine administrative issues" mailbox that only gets checked every so often, because the whole point is reaching the administrator quickly in an emergency 🚑. Gateway administrators SHOULD also arrange substitutes to cover gateway operation (with suitable redirection of mail) when they're on vacation, etc. Go enjoy your holiday, bestie 🌴💅.
11. Security and Related Issues 🔐
Although the interchange format itself raises no significant security issues, the wider context absolutely does 😱.
11.1. Leakage 🚨
The most obvious security problem with news is "leakage", where articles meant for restricted circulation escape 🙀. The flooding algorithm is EXTREMELY good at finding any path by which articles can leave a subnet with supposedly restrictive boundaries 🌊. So a lot of administrative effort is needed to keep local newsgroups local, unless the connections to the outside world are tightly restricted.
A related problem is that the sendme control message can be used to ask for any article by its message ID 🫣. This is less of a big deal now that message-ID generation algorithms have become less predictable, but it's still a potential problem for "secure" newsgroups. Hosts with such newsgroups may want to disable the sendme control message entirely 🔒.
The sendsys, version, and whogets control messages also let "outsiders" request information from "inside", which may reveal details of internal topology (etc.) that are considered confidential 👀. (Note that at least limited openness about such matters may be a condition of membership in such networks, e.g., Usenet 🌍.)
If you want to control these forms of leakage, you're strongly advised to pick a small number of "official gateway" hosts to handle all news exchange with the outside world 👑. That way a bounded amount of administrative effort is needed to control propagation and squash problems. Trying to keep news out entirely by refusing to support an official gateway usually ends with a pile of unofficial partial gateways popping up over time 💀. That kind of setup is way harder to troubleshoot.
A somewhat related problem is that proprietary material might get disclosed unintentionally by a poster who doesn't realise how far his words will propagate 😬. That can happen from sheer misunderstanding, or because of errors made (by human or software) in followup preparation. Honestly, there's little that can be done about this except education 📚🫶.
11.2. Attacks 💥
okay so news is pretty limited in what it can do to attack a host, but there are still a few possibilities, and most of them are problems news shares with mail 💀
If reading agents are careless about sending non-printable characters to output devices, malicious posters can post articles with control sequences (called "letterbombs") meant to wreck output devices in different ways. What happens depends on the device, but it can get as bad as hardware damage (like repeatedly writing values into configuration memories that can only handle a limited number of write cycles) or a security violation (like reprogramming function keys that privileged readers might be using) 😱
A more sophisticated version of the letterbomb is hiding "Trojan horses" inside programs. Readers obviously need to be careful about any software they find in news, but it's more subtle than that: reading agents have to be careful too, lowkey 👀 MIME messages can include material that's executable in some sense, like PostScript documents (which are programs!), and letterbombs can be slipped into that material as well 🧚
Finite resources and other software limitations mean some system disruption can be achieved just by posting otherwise-innocent material in great volume, either in single huge articles (see Section 4.6) or in a stream of modest-sized articles. (Some would say the steady growth of Usenet volume is a subtle, unintentional attack of the latter type. Either way, it can have disruptive effects if administrators are inattentive 🙈) Systems need some ability to cope with surges, because single huge articles occur occasionally as the result of software error, innocent misunderstanding, or deliberate malice, and downtime at upstream hosts can cause droughts, followed by floods, of legitimate articles. (There's also a certain amount of normal variation. For example, Usenet traffic is noticeably lighter on weekends and during Christmas holidays, and it rises noticeably at the start of the school term at North American universities 🎄📚) However, a site that normally receives little traffic may be quite vulnerable to a "swamping" attack if its software isn't careful enough 🫠
In general, careless implementation may open doors that are not intrinsic to news. In particular, running control messages (see Sections 6.6 and 7) and unbatchers (see Sections 8.1 and 8.2) via a command interpreter requires substantial precautions to make sure that only the intended capabilities are available. Care also has to be taken that article-supplied text is not fed to programs that have escapes to command interpreters 🔒
Finally, there's considerable potential for malice in the sendsys, version, and whogets control messages. They aren't harmful to the hosts receiving them as news, but they can be used to enlist those hosts (by the thousands) as unwitting allies in a mail-swamping attack on a victim who may not even receive news. The precautions discussed in Section 7.5 can reduce the potential for such attacks considerably, but the hazard can't be eliminated as long as these control messages exist 😤
11.3. Anarchy 🌈
The highly distributed nature of news propagation, plus the lack of adequate authentication protocols (especially for use over less-interactive transport mechanisms like UUCP), makes article forgery relatively straightforward. It may be possible to at least track a forgery to its source once it's recognized as one, but clever forgers can make even that pretty difficult 🕵️♀️ The assumption that forgeries will be recognized isn't something to take for granted either, because readers are notoriously prone to blindly assuming authenticity 🙃 If a forged article's initial path list includes the relayer name of the supposed poster's host, the article will never be sent to that host, and the alleged author may learn about the forgery secondhand or not at all 💔
A particularly noxious form of forgery is the forged cancel control message. It's relatively straightforward to write software that automatically sends out a (forged) cancel message for any article meeting some criterion, e.g., articles written by a specific author. The authentication problems discussed in Section 7.1 make it difficult to solve this without crippling cancel's important functionality 😩
A related problem is the possibility of disagreements over newsgroup creation, on networks where such things aren't decided by central authorities. There have been cases of "rmgroup wars," where one poster persistently sends out newgroup messages to create a newsgroup, and another, equally persistently, sends out rmgroup messages asking that it be removed. This isn't particularly damaging if relayers are configured to be cautious, but it can cause serious confusion among innocent third parties who just want to know whether or not they can use the newsgroup for communication 🥴
11.4. Liability 💜
News shares the legal uncertainty surrounding other forms of electronic communication: what rules apply to this new medium of information exchange? News is a particularly tricky case because it's a broadcast medium rather than a point-to-point one like mail, and analogies to older forms of communication are particularly weak 📡
Are news-carrying hosts common carriers, like the phone companies, providing communications paths without having either authority over or responsibility for content? Or are they publishers, responsible for the content regardless of whether they're aware of it or not? Or something in between? 🤷♀️ These questions get especially significant when the content is technically criminal (e.g., some types of sexually oriented material in some jurisdictions), because in that case ignorance of its presence may not be an adequate defense ⚖️
Even in milder situations, like libel or copyright violation, the responsibilities of the poster, his host, and other hosts carrying the traffic are unclear. Note in particular the problems that come up when the article is a forgery, or when the alleged author claims it's a forgery but can't prove it 🫠
12. References 📚
[ISO/IEC9899] "Information technology - Programming Language C",
ISO/IEC 9899:1990 {more recently 9899:1999}, 1990.
[Metamail] Borenstein, N.,
<http://ftp.funet.fi/pub/unix/mail/metamail/ANNOUNCE>,
February 1994.
[RFC821] Postel, J., "Simple Mail Transfer Protocol", STD 10,
RFC 821, August 1982.
[RFC822] Crocker, D., "STANDARD FOR THE FORMAT OF ARPA INTERNET
TEXT MESSAGES", STD 11, RFC 822, August 1982.
[RFC850] Horton, M., "Standard for interchange of Usenet
messages", RFC 850, June 1983.
[RFC977] Kantor, B. and P. Lapsley, "Network News Transfer
Protocol - A Proposed Standard for the Stream-Based
Transmission of News", RFC 977, February 1986.
[RFC1036] Horton, M. and R. Adams, "Standard for interchange of
USENET Messages", RFC 1036, December 1987.
[RFC1123] Braden, R., Ed., "Requirements for Internet Hosts -
Application and Support", STD 3, RFC 1123,
October 1989.
[RFC1341] Borenstein, N. and N. Freed, "MIME (Multipurpose
Internet Mail Extensions): Mechanisms for Specifying
and Describing the Format of Internet Message Bodies",
RFC 1341, June 1992.
[RFC1342] Moore, K., "Representation of Non-ASCII Text in
Internet Message Headers", RFC 1342, June 1992.
[RFC1345] Simonsen, K., "Character Mnemonics and Character
Sets", RFC 1345, June 1992.
[RFC1413] St. Johns, M., "Identification Protocol", RFC 1413,
February 1993.
[RFC1456] Vietnamese Standardization Working Group, "Conventions
for Encoding the Vietnamese Language", RFC 1456,
May 1993.
[RFC1544] Rose, M., "The Content-MD5 Header Field", RFC 1544,
November 1993.
[RFC1896] Resnick, P. and A. Walker, "The text/enriched MIME
Content-type", RFC 1896, February 1996.
[RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet
Mail Extensions (MIME) Part One: Format of Internet
Message Bodies", RFC 2045, November 1996.
[RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet
Mail Extensions (MIME) Part Two: Media Types",
RFC 2046, November 1996.
[RFC2047] Moore, K., "MIME (Multipurpose Internet Mail
Extensions) Part Three: Message Header Extensions for
Non-ASCII Text", RFC 2047, November 1996.
[RFC2049] Freed, N. and N. Borenstein, "Multipurpose Internet
Mail Extensions (MIME) Part Five: Conformance Criteria
and Examples", RFC 2049, November 1996.
[RFC2822] Resnick, P., Ed., "Internet Message Format", RFC 2822,
April 2001.
[RFC3977] Feather, C., "Network News Transfer Protocol (NNTP)",
RFC 3977, October 2006.
[RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322,
October 2008.
[RFC5536] Murchison, K., Ed., Lindsey, C., and D. Kohn, "Netnews
Article Format", RFC 5536, November 2009.
[RFC5537] Allbery, R., Ed., and C. Lindsey, "Netnews
Architecture and Protocols", RFC 5537, November 2009.
[Sanderson] David Sanderson, Smileys, O'Reilly & Associates Ltd.,
1993.
[UUCP] Tim O'Reilly and Grace Todino, Managing UUCP and
Usenet, O'Reilly & Associates Ltd., January 1992.
[X3.4] "American National Standard for Information Systems -
Coded Character Sets - 7-Bit American National
Standard Code for Information Interchange (7-Bit
ASCII)", ANSI X3.4, March 1986.Appendix A. Archaeological Notes 🦖
A.1. "A News" Article Format 📰
The obsolete "A News" article format consisted of exactly five lines of header information, followed by the body. For example:
Aeagle.642
news.misc
cbosgd!mhuxj!mhuxt!eagle!jerry
Fri Nov 19 16:14:55 1982
Usenet Etiquette - Please Read
body
body
bodyThe first line was an "A" followed by an article ID (analogous to a message ID and used for similar purposes). The second line was the list of newsgroups. The third line was the path. The fourth was the date, in the format above (all fields fixed width), which resembles an Internet date but isn't quite the same. The fifth was the subject 🗂️
This format is documented for archaeological purposes only, so please don't generate articles in this format 🫶
A.2. Early "B News" Article Format
Okay so this one is an obsolete pseudo-Internet article format 💀 that only showed up briefly during the switch from A News to the modern format 🦋. It basically followed the general vibe of a MAIL message, but with some non-standard headers ✨. Here's an example:
From: cbosgd!mhuxj!mhuxt!eagle!jerry (Jerry Schwarz)
Newsgroups: news.misc
Title: Usenet Etiquette -- Please Read
Article-I.D.: eagle.642
Posted: Fri Nov 19 16:14:55 1982
Received: Fri Nov 19 16:59:30 1982
Expires: Mon Jan 1 00:00:00 1990
body
body
bodyThe From header held the info that now lives in the Path header, plus maybe the full name that's usually in From today 🎀. The Title header is what we now call the Subject content 💅.
The Posted header is what we now call the Date content 🌸. The Article-I.D. header held an article ID, which was basically a message ID and got used for similar stuff 🫶. The Newsgroups and Expires headers were pretty much the same as they are now 💜. The Received header held the date when the latest relayer to process the article first saw it 🍓. All the dates used the format above, with every field fixed width, so they looked like an Internet date but weren't quite the same 🧚.
This format is documented for archaeological purposes only 🦴. > do not generate articles in this format.
A.3. Obsolete Headers
Early versions of news software that followed the modern format sometimes generated headers like these 🌈:
Relay-Version: version B 2.10 2/13/83; site cbosgd.UUCP
Posting-Version: version B 2.10 2/13/83; site eagle.UUCP
Date-Received: Friday, 19-Nov-82 16:59:30 ESTRelay-Version held version info about the relayer that last processed the article 🛠️. Posting-Version held version info about the posting agent that posted the article ✨. Date-Received held the date when the last relayer to process the article first saw it, in a slightly nonstandard format ⭐.
These headers are documented for archaeological purposes only 🦕. Do not generate articles using them.
A.4. Obsolete Control Messages
There was once a senduuname control message, which was kind of like sendsys 🙈. It asked for the list of hosts that the receiving host had UUCP connections to. This rapidly stopped being useful, and lots of organizations think info about their internal connectivity is confidential 🔒, so honestly, good riddance 💅.
Historically, a checkgroups body with one or two lines, where the first line was of the form -n newsgroup, made checkgroups apply to only that single newsgroup 🎯. This form is documented for archaeological purposes only 🏛️; do not use it.
Historically, an article posted to a newsgroup whose name had exactly three components, with the third being ctl, meant that the article was to be taken as a control message 👑. The Subject header specified the actions in the same way the Control header does now 🧚. This form is documented for archaeological purposes only 🦖; do not use it; do not implement it 💀.
Appendix B. A Quick Tour of MIME ✨
(The editor wishes to thank Luc Rooijakkers; most of this appendix is a lightly edited version of a summary he kindly supplied.) 💜
okay so MIME (Multipurpose Internet Mail Extensions) is an upward-compatible set of extensions to [RFC822], currently documented in [RFC2045], [RFC2046], and [RFC2047] 🌸. this appendix summarizes those docs, but bestie, go read the MIME RFCs for the full scoop, they are very readable 📖
UNRESOLVED ISSUE: These RFC numbers (here and elsewhere in this Draft) need updating when the new MIME RFCs come out {now resolved!}.
MIME adds these new headers, and they are literally iconic 💅:
MIME-Version
Content-Type
Content-Transfer-Encoding
Content-ID
Content-DescriptionThe MIME-Version header is mandatory for all messages conforming to the MIME specification, and it carries the version number of the MIME specification 🎀. Example:
MIME-Version: 1.0The Content-Type header tells you the content type of the message ✨. Content types get split into a top-level type and a subtype, separated by a slash. You can also add auxiliary info using an attribute-value notation 🧚. Example:
Content-Type: text/plain; charset=us-ascii(In the absence of a Content-Type header, this is in fact the default content type 🫶.)
These are the important type/subtype combos, bestie 💖:
text/plain Plain text, possibly in a non-ASCII character
set.
text/enriched A very simple wordprocessor-like language
supporting character attributes (e.g.,
underlining), justification control, and
multiple character sets. (This proposal has
gone through several iterations and has
recently split off from the main MIME RFCs
into a separate document [RFC1896].)
message/rfc822 A mail message conforming to a slightly
relaxed version of [RFC822].
message/partial Part of a message (supporting the transparent
splitting and joining of messages when they
are too large to be handled by some transport
agent).
message/external-body A message whose body is external. Possible
access methods include via mail, FTP, local
file, etc.
multipart/mixed A message whose body consists of multiple
parts, possibly of different types, intended
to be viewed in serial order. Each part
looks like an [RFC822] message, consisting of
headers and a body. Most of the [RFC822]
headers have no defined semantics for body
parts.
multipart/parallel Likewise, except that the parts are intended
to be viewed in parallel (on user agents that
support it).
multipart/alternative Likewise, except that the parts are intended
to be semantically equivalent such that the
part that best matches the capabilities of
the environment should be displayed. For
example, a message may include plain-text,
enriched-text, and postscript versions of
some document.
multipart/digest A variant of multipart/mixed especially
intended for message digests (the default
type of the parts is message/rfc822 instead
of text/plain, saving on the number of
headers for the parts).
application/postscript A PostScript document. (PostScript is a
trademark of Adobe.)Other top-level types exist for still images, audio, and video samples 🎬.
Some of the above types need the ability to transport binary data 🌈. Since the existing message systems usually don't support this, MIME gives you a Content-Transfer-Encoding header to show which kind of encoding was used. The possible encodings are:
7bit No encoding; the data consists of short (less than
1000 characters) lines of 7-bit ASCII data,
delimited by EOL sequences. This is the default
encoding.
8bit Like 7bit, except that bytes with the high-order
bit set may be present. Many transmission paths
are incapable of carrying messages that use this
encoding.
binary No encoding; any sequence of bytes may be present.
Many transmission paths are incapable of carrying
messages that use this encoding.
base64 The data is encoded by representing every group of
3 bytes as 4 characters from the alphabet
"A-Za-z0-9+/", which was chosen for its high
robustness through mail gateways (the alphabet used
by uuencode does not survive ASCII-EBCDIC-ASCII
translations). In the final group of 4 characters,
"=" is used for those characters not representing
data bytes. Line length is limited, and EOLs in
the encoded form are ignored.
quoted-printable Any byte can be represented by a three-character
"=XX" sequence where the X's are uppercase
hexadecimal digits. Bytes representing printable
7-bit US-ASCII characters except "=" may be
represented literally. Tabs and blanks may be
represented literally if not at the end of a line.
Line length is limited, and an EOL preceded by "="
was inserted for this purpose and is not present in
the original.The base64 and quoted-printable encodings get applied to data in Internet canonical form 🍓. That means any EOL encoded as anything but EOL must be an Internet canonical EOL: CR followed by LF.
The Content-Description header lets you describe a body part further, kinda like how Subject works for messages 💌.
And the Content-ID header can give an identification to body parts, the same way Message-ID does for messages 🦋.
Note that most of these headers are structured header fields, as defined in [RFC822]. So comments are allowed in their values 🙈. The following is a legal MIME header:
Content-Type: (a comment) text (yeah) /
plain (and now some params:) ; charset= (guess what)
iso-8859-1 (we don't have iso-10646 yet, pity)NOTE: Although the MIME specification was developed for mail, there is nothing precluding its use for news as well. While it might simplify implementation to restrict the MIME headers somewhat, in the same way that other news headers (e.g., From) are restricted subsets of the [RFC822] originals, this would add yet another divergence between two formats that ought to be as compatible as possible. In the case of the MIME headers, there is no body of existing code posing compatibility concerns. A full-featured MIME reading agent needs a full [RFC822] parser anyway, to properly handle body parts of types like message/rfc822, so there is little gain from restricting MIME headers. Adopting the MIME specification unchanged seems best. However, article-level MIME headers must still comply with the overall news header syntax given in Section 4, so that news software that is not interested in MIME need not contain a full [RFC822] parser.
"MIME (Multipurpose Internet Mail Extensions) Part Three: Message Header Extensions for Non-ASCII Text" [RFC2047] deals with non-ASCII characters in headers 🌺. An example of a header using the [RFC2047] mechanism is:
From: =?ISO-8859-1?Q?Andr=E9_?= Pirard <PIRARD@vm1.ulg.ac.be>Such encodings are allowed in selected headers, subject to the restrictions listed in [RFC2047] 🫧.
The MIME effort also produced an RFC defining a Content-MD5 header [RFC1544]. It contains an MD5-based "checksum" of the contents of an article or body part, which gives high confidence of catching accidental modifications to the contents 🔥.
The "metamail" software package [Metamail] helps provide MIME support with minimal changes to mailers, and it may also be relevant to news reading agents 💅.
The PEM (Privacy Enhanced Mail) effort is chasing analogous facilities to offer stronger guarantees against malicious modifications, unauthorized eavesdropping, and forgery 👑. This work too may be applicable to news, once it is reconciled with MIME (by efforts now underway) 🌷.
Appendix C. Summary of Changes Since RFC 1036 💖
This Draft is much longer than [RFC1036], so obviously there is a lot of change in content 🫠. Much of it is just increased precision and rigor 🤓. The noteworthy changes and additions include:
- restrictions on article bodies (Section 4.3)
- all references to MIME facilities
- size limits on articles
- precise specification of Date-content syntax
- message IDs must never be re-used, ever
- "!" is the only Path delimiter
- multiple moderators in the Approved header
- rules on References trimming, and the
_-_mechanism - generalization of the Xref rules
- multiple message IDs in Cancel and Supersedes
- Also-Control
- See-Also
- Article-Names
- Article-Updates
- more precise rules for cancellation
- cancellation authorization based on From, not Sender
- "unmoderated" and descriptors in newgroup messages
- restrictive rules on handling of sendsys and version messages
- the whogets control message
- precise specification of checkgroups messages
- compression type preferably specified out-of-band
- rules for encapsulating news in MIME mail
- tighter specification of relayer functioning (Section 9.1)
- the "newsmaster" contact address
- rules for gatewaying (Section 10)
- discussion of security issues (Section 11)
Appendix D. Summary of Completely New Features 🌟
Most of this Draft just documents existing practice, preferred versions of it, or straightforward generalizations of it 🧁. But there are a few outright inventions, and they are:
- the
_-_mechanism for References trimming - Also-Control
- See-Also
- Article-Names
- Article-Updates
- the whogets control message
Appendix E. Summary of Differences from RFCs 822 and 1123 🦋
These are the noteworthy differences between this Draft's articles and MAIL messages 💜:
- generally less-permissive header syntax
- notably, limited From syntax
- MAIL header comments allowed in only a few contexts
- slightly more restricted message-ID syntax
- several more mandatory headers
- duplicate headers forbidden
- References/See-Also versus In-Reply-To/References (Section 6.5)
- case sensitivity in some contexts
- point-to-point headers, e.g., To and Cc, forbidden (Section 6)
- several new headers
Author's Address 💌
Henry Spencer
SP Systems
Box 280 Stn. A
Toronto, Ontario M5W1B2
Canada
EMail: henry@zoo.utoronto.ca📦 where this page comes from
- ietf: rfc1849 2010-03
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms