RFC 4644
Network News Transfer Protocol (NNTP) Extension for Streaming Feeds
RFC 4644: Network News Transfer Protocol (NNTP) Extension for Streaming Feeds ✨
Okay bestie, this is RFC 4644 💌 It's from the Network Working Group and it's a Standards Track doc dated October 2006. J. Vinocur (Cornell University) and K. Murchison (Carnegie Mellon University) wrote it 🎓. It updates RFC 2980, which means it's giving upgrade energy for that older spec 🩷. Those are all the header facts, and they're the ones that matter 💅.
Status of This Memo 📜
This document describes an Internet standards track protocol for the Internet community, and it's asking everyone for discussion and ideas to make it better 🫶. For the official standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited 🌈.
Copyright Notice ©️
Copyright (C) The Internet Society (2006).
Abstract 🦋
This memo adds an extension to the Network News Transfer Protocol (NNTP) for asynchronous transfer of articles, which people also call "streaming" 🌊. It lets servers send articles to other servers way more efficiently ✨
This document updates and formalizes the CHECK and TAKETHIS commands from RFC 2980, and it deprecates the MODE STREAM command 💀.
Table of Contents 🗂️
1. Introduction ....................................................2
1.1. Conventions Used in this Document ..........................2
2. The STREAMING Extension .........................................3
2.1. Streaming Article Transfer .................................3
2.2. Advertising the STREAMING Extension ........................4
2.3. MODE STREAM Command ........................................5
2.3.1. Usage ...............................................5
2.3.2. Description .........................................5
2.3.3. Examples ............................................5
2.4. CHECK Command ..............................................6
2.4.1. Usage ...............................................6
2.4.2. Description .........................................6
2.4.3. Examples ............................................6
2.5. TAKETHIS Command ...........................................7
2.5.1. Usage ...............................................7
2.5.2. Description .........................................7
2.5.3. Examples ............................................8
3. Augmented BNF Syntax for the STREAMING Extension ................9
3.1. Commands ...................................................9
3.2. Command Datastream .........................................9
3.3. Responses .................................................10
3.4. Capability Entries ........................................10
4. Summary of Response Codes ......................................10
5. Security Considerations ........................................11
6. IANA Considerations ............................................11
7. Acknowledgements ...............................................12
8. References .....................................................12
8.1. Normative References ......................................12
8.2. Informative References ....................................121. Introduction 🧚
So, the NNTP spec [NNTP] says a peer uses the IHAVE command to ask a server whether it wants a certain article. The problem is IHAVE can't be pipelined, so you have to stop and wait for the other end to answer every single time 🐢 That totally tanks your throughput, and it's so frustrating 😭
The CHECK and TAKETHIS commands were originally written up in [NNTP-COMMON] as ad-hoc commands. They're a different way to move articles between peers, and they use network bandwidth way better 🚀 They've been proven useful and everyone's using them, so this spec makes them official 💖
The MODE STREAM command is also from [NNTP-COMMON], and this spec deprecates it. It's still documented here because it's everywhere, so it's kept around for backwards compatibility 🫶
1.1. Conventions Used in this Document 📐
The notation in this document matches [NNTP], and any term that isn't defined here means the same thing it does there 🤓
The key words "REQUIRED", "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", "MAY", and "OPTIONAL" in this document are to be interpreted as described in "Key words for use in RFCs to Indicate Requirement Levels" [KEYWORDS]. These are the official vibes-check words, so don't mess with them 👑
You need to be familiar with NNTP [NNTP] before this makes sense 📚 The connections here go from one peer to another, but we'll keep calling the initiator of the NNTP connection the "client", and the other endpoint the "server" 💅
In the examples, commands from the client are marked with [C], and responses from the server are marked with [S] 🎀
2. The STREAMING Extension 🌸
This extension adds three new commands: MODE STREAM, CHECK, and TAKETHIS. Its capability label is STREAMING 🏷️
2.1. Streaming Article Transfer 🌊
The STREAMING extension does the same job as the IHAVE command ([NNTP] section 6.3.2), but it splits the querying and the transferring into separate CHECK and TAKETHIS commands. That means CHECK and TAKETHIS can be pipelined ([NNTP] section 3.5), and you get real "streaming" article transfer 🤩
A streaming client will usually pipeline a bunch of CHECK commands, then use the replies to build a list of articles to send in a pipelined run of TAKETHIS commands. Way more of its time goes to actually moving articles, which is so efficient 🔥 CHECK and TAKETHIS get different response codes, so they can be mixed together in one pipeline, and the client can always tell which reply belongs to which command 🧩
The client MAY send articles via TAKETHIS without asking the server with CHECK first. But it's giving "not every article, please," because the client SHOULD NOT send everything this way unless the site admin explicitly sets it up to, based on out-of-band info 🙅♀️ Still, the client MAY use an adaptive strategy: it starts by sending CHECK for every article, then switches to TAKETHIS without CHECK once most articles are getting accepted (over 95% acceptance might be a reasonable metric in some configurations) ✨ If it goes that route, the client SHOULD also switch back to CHECK on every article if the acceptance rate ever drops much below that threshold 📉
2.2. Advertising the STREAMING Extension 📣
A server that supports the streaming commands in this document advertises the "STREAMING" capability label in response to the CAPABILITIES command ([NNTP] section 5.2). The server MUST keep advertising this capability after a client has issued the MODE STREAM command. This capability MAY be advertised both before and after any use of the MODE READER command ([NNTP] section 5.3), with the same semantics 💜
Here's a client using CAPABILITIES and MODE STREAM on a mode-switching server:
[C] CAPABILITIES
[S] 101 Capability list:
[S] VERSION 2
[S] MODE-READER
[S] IHAVE
[S] LIST ACTIVE
[S] STREAMING
[S] .
[C] MODE STREAM
[S] 203 Streaming permitted
[C] CAPABILITIES
[S] 101 Capability list:
[S] VERSION 2
[S] MODE-READER
[S] IHAVE
[S] LIST ACTIVE
[S] STREAMING
[S] .
[C] MODE READER
[S] 200 Posting allowed
[C] CAPABILITIES
[S] 101 Capability list:
[S] VERSION 2
[S] READER
[S] POST
[S] LIST ACTIVE NEWSGROUPS HEADERS
[S] HDR
[S] .2.3. MODE STREAM Command 🕰️
Back in the day, clients used this command to find out if a server supported CHECK and TAKETHIS. It's deprecated now in favor of the CAPABILITIES discovery command 🙃 It's only still here so it works with older implementations [NNTP-COMMON] of those transport commands 🧓
New clients SHOULD use the CAPABILITIES command to check a server for the STREAMING extension, but they MAY use the MODE STREAM command for backwards compatibility with legacy servers that don't support CAPABILITIES discovery. Servers MUST accept the MODE STREAM command for backwards compatibility with legacy clients that don't use CAPABILITIES discovery 🤝
NOTE: This command may be removed from a future version of this specification, therefore clients are urged to transition to the CAPABILITIES command wherever possible. 🚨
2.3.1. Usage 🛠️
Syntax
MODE STREAM
Responses
203 Streaming permitted2.3.2. Description 📝
If a server supports this extension, it MUST return a 203 response to the MODE STREAM command (or a 501 if an argument is given). The MODE STREAM command MUST NOT affect the server state in any way. It's not a mode change, despite the name 😂 So this command MAY be pipelined. A server MUST NOT require that the client issue MODE STREAM before it accepts the CHECK or TAKETHIS commands 🙅♀️
2.3.3. Examples 🎀
A client checking for streaming on a server that doesn't support this extension:
[C] MODE STREAM
[S] 501 Unknown MODE variantA client checking for streaming on a server that does support this extension:
[C] MODE STREAM
[S] 203 Streaming permitted2.4. CHECK Command 🔍
2.4.1. Usage 🛠️
Syntax
CHECK message-id
Responses
238 message-id Send article to be transferred
431 message-id Transfer not possible; try again later
438 message-id Article not wanted
Parameters
message-id = Article message-idThe first parameter of the 238, 431, and 438 responses MUST be the message-id the client provided as the parameter to CHECK 🪪
2.4.2. Description 📝
The CHECK command tells the server that the client has an article with the given message-id 📬 If the server wants a copy of that article, a 238 response MUST be returned, which means the client may send the article using the TAKETHIS command 🎉 If the server doesn't want it (say it already has a copy), a 438 response MUST be returned to say the article isn't wanted 🙅♀️ And if the article isn't wanted right now but the client should try again later if it can (like if another client already offered to send that same article to the server), a 431 response MUST be returned ⏳
NOTE: The responses to CHECK are advisory; the server MUST NOT rely on the client to behave as requested by these responses. 🫠
2.4.3. Examples 🧚
A client checking whether the server wants a set of articles, and getting a mix of responses:
[C] CHECK <i.am.an.article.you.will.want@example.com>
[S] 238 <i.am.an.article.you.will.want@example.com>
[C] CHECK <i.am.an.article.you.have@example.com>
[S] 438 <i.am.an.article.you.have@example.com>
[C] CHECK <i.am.an.article.you.defer@example.com>
[S] 431 <i.am.an.article.you.defer@example.com>Pipelining the CHECK commands from the previous example:
[C] CHECK <i.am.an.article.you.will.want@example.com>
[C] CHECK <i.am.an.article.you.have@example.com>
[C] CHECK <i.am.an.article.you.defer@example.com>
[S] 238 <i.am.an.article.you.will.want@example.com>
[S] 438 <i.am.an.article.you.have@example.com>
[S] 431 <i.am.an.article.you.defer@example.com>2.5. TAKETHIS Command 🦋
(Part 2 is coming next, so stay tuned, babe 💖)
2.5.1. Usage 🦋
okay so a client MUST NOT use this command unless the server advertises the STREAMING capability or returns a 203 response to the MODE STREAM command. no skipping the vibe check 💅
Syntax 🎀
TAKETHIS message-idResponses 🌸
239 message-id Article transferred OK
439 message-id Transfer rejected; do not retryParameters 🧚
message-id = Article message-idThe first parameter of the 239 and 439 responses MUST be the message-id provided by the client as the parameter to TAKETHIS. so the server has to echo your message-id back, no mix-ups 🫶
2.5.2. Description 💜
okay this is the main event ✨ TAKETHIS is how a client sends an article with a given message-id over to the server. the article gets sent right after the CRLF that ends the TAKETHIS command line, no pause 🚀
the client MUST send the entire article, headers and body both, to the server as a multi-line data block (see [NNTP] section 3.1.1). so a single dot (".") on its own line means "the end!!", and any line that starts with a dot in the original text gets that dot doubled during transmission. it's giving escaping energy 🙈
the server MUST answer with either a 239 response (article transferred successfully) or a 439 response (article rejected). and if the server runs into a temporary error that stops it from processing the article, but it doesn't want to reject the article, it MUST reply with a 400 response to the client and close the connection. 💀
this function is different from the POST command. it's meant for moving articles that have already been posted between hosts 📦 so the SHOULD NOT line is important: it SHOULD NOT be used when the client is a personal news-reading program, because using this command says "this article was already posted somewhere else and is just getting forwarded from another host." 🍓
BUT, even with all that, the server MAY elect not to post or forward the article if, after looking at it more closely, it decides that would be inappropriate. reasons for that later rejection may include things like inappropriate newsgroups or distributions, disk space limits, article lengths, garbled headers, and so on. these are typically restrictions enforced by the server host's news software, not necessarily by the NNTP server itself. so it's not always the server being picky, sometimes it's the host software 🤷♀️
the client SHOULD NOT assume the article has been transferred successfully unless it gets an affirmative response from the server. no response at all (like a dropped network connection or a network timeout), or a 400 response, SHOULD be treated as a temporary failure, and the transfer should be tried again later if possible. don't just assume it went through, bestie 🥺
last but not least, some news server software may not immediately be able to tell whether an article is suitable for posting or forwarding. so an NNTP server MAY acknowledge the successful transfer of the article (with a 239 response) but later silently discard it. the ghosting is real 👻
2.5.3. Examples 🍒
Example of streaming two articles to another site (the first article is accepted and the second is rejected):
[C] TAKETHIS <i.am.an.article.you.will.want@example.com>
[C] Path: pathost!demo!somewhere!not-for-mail
[C] From: "Demo User" <nobody@example.com>
[C] Newsgroups: misc.test
[C] Subject: I am just a test article
[C] Date: 6 Oct 1998 04:38:40 -0500
[C] Organization: An Example Com, San Jose, CA
[C] Message-ID: <i.am.an.article.you.will.want@example.com>
[C]
[C] This is just a test article.
[C] .
[C] TAKETHIS <i.am.an.article.you.have@example.com>
[C] Path: pathost!demo!somewhere!not-for-mail
[C] From: "Demo User" <nobody@example.com>
[C] Newsgroups: misc.test
[C] Subject: I am just a test article
[C] Date: 6 Oct 1998 04:38:40 -0500
[C] Organization: An Example Com, San Jose, CA
[C] Message-ID: <i.am.an.article.you.have@example.com>
[C]
[C] This is just a test article.
[C] .
[S] 239 <i.am.an.article.you.will.want@example.com>
[S] 439 <i.am.an.article.you.have@example.com>Example of sending an article to a site where the transfer fails:
[C] TAKETHIS <i.am.an.article.you.will.want@example.com>
[C] Path: pathost!demo!somewhere!not-for-mail
[C] From: "Demo User" <nobody@example.com>
[C] Newsgroups: misc.test
[C] Subject: I am just a test article
[C] Date: 6 Oct 1998 04:38:40 -0500
[C] Organization: An Example Com, San Jose, CA
[C] Message-ID: <i.am.an.article.you.will.want@example.com>
[C]
[C] This is just a test article.
[C] .
[S] 400 Service temporarily unavailable
[Server closes connection.]3. Augmented BNF Syntax for the STREAMING Extension ✨
this section lays out the formal syntax of the STREAMING extension using ABNF [ABNF]. it extends the syntax in section 9 of [NNTP], and any non-terminals that aren't defined in this document are defined there. the [NNTP] ABNF should be imported first before you try validating these rules, so grab that first 📚
3.1. Commands 💅
this syntax extends the non-terminal "command", which represents an NNTP command.
command =/ check-command /
mode-stream-command /
takethis-command
check-command = "CHECK" WS message-id
mode-stream-command = "MODE" WS "STREAM"
takethis-command = "TAKETHIS" WS message-id3.2. Command Datastream 🌈
this syntax extends the non-terminal "command-datastream", which represents the further material sent by the client in the case of streaming commands.
command-datastream =/ takethis-datastream
takethis-datastream = encoded-article3.3. Responses 🦋
this syntax extends the non-terminal "initial-response-content", which represents an initial response line sent by the server.
initial-response-content =/ response-238-content /
response-239-content /
response-431-content /
response-438-content /
response-439-content
response-238-content = "238" SP message-id
response-239-content = "239" SP message-id
response-431-content = "431" SP message-id
response-438-content = "438" SP message-id
response-439-content = "439" SP message-id3.4. Capability Entries 🎀
this syntax extends the non-terminal "capability-entry", which represents a capability that may be advertised by the server.
capability-entry =/ streaming-capability
streaming-capability = "STREAMING"4. Summary of Response Codes 🍓
this section is basically the cheat sheet 📝 it lists each new response code defined in this document and says whether it's multi-line, which commands can generate it, what arguments it has, and what it means.
Response code 203
Generated by: MODE STREAM
Meaning: streaming permitted.
Response code 238
Generated by: CHECK
1 argument: message-id
Meaning: send article to be transferred.
Response code 239
Generated by: TAKETHIS
1 argument: message-id
Meaning: article transferred OK.
Response code 431
Generated by: CHECK
1 argument: message-id
Meaning: transfer not possible; try again later.
Response code 438
Generated by: CHECK
1 argument: message-id
Meaning: article not wanted.
Response code 439
Generated by: TAKETHIS
1 argument: message-id
Meaning: transfer rejected; do not retry.5. Security Considerations 🔥
okay security time, and the tea is short ☕ no new security considerations are introduced by this extension, beyond those already described in the core specification [NNTP]. so all the security stuff lives in the NNTP core spec, and this extension doesn't add any new drama on top. still go read the core spec, it's giving important 💀
6. IANA Considerations 👑
this section gives the formal definition of the STREAMING extension, as required by Section 3.3.3 of [NNTP], for the IANA registry. here's the official rundown:
- The STREAMING extension provides for streaming transfer of articles.
- The capability label for this extension is "STREAMING".
- The capability label has no arguments.
- The extension defines three new commands, MODE STREAM, CHECK, and TAKETHIS, whose behavior, arguments, and responses are defined in Sections 2.3, 2.4, and 2.5 respectively.
- The extension does not associate any new responses with pre-existing NNTP commands.
- The extension does not affect the behavior of a server or client other than via the new commands.
- The extension does not affect the maximum length of commands or initial response lines.
- The extension does not alter pipelining, and the MODE STREAM, CHECK, and TAKETHIS commands can be pipelined.
- Use of this extension does not alter the capabilities list.
- The extension does not cause any pre-existing command to produce a 401, 480, or 483 response.
- Use of the MODE READER command on a mode-switching server may disable this extension.
- Published Specification: This document.
- Contact for Further Information: Authors of this document.
- Change Controller: IESG
<iesg@ietf.org>.
7. Acknowledgements 🫶
this document is based heavily on the relevant sections of RFC 2980 [NNTP-COMMON], by Stan Barber. big thanks to him 🌸
special acknowledgement also goes to Russ Allbery, Clive Feather, Andrew Gierth, and others who commented privately on intermediate revisions of this document, as well as the members of the IETF NNTP Working Group for continual (yet sporadic) insight in discussion. the group chat of people who made this happen 💖
8. References 📚
8.1. Normative References
[ABNF] Crocker, D., Ed. and P. Overell, "Augmented BNF for
Syntax Specifications: ABNF", RFC 4234, October 2005.
[KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[NNTP] Feather, C., "Network News Transfer Protocol (NNTP)",
RFC 3977, October 2006.8.2. Informative References
[NNTP-COMMON] Barber, S., "Common NNTP Extensions", RFC 2980, October
2000.Authors' Addresses 🌟
Jeffrey M. Vinocur
Department of Computer Science
Upson Hall
Cornell University
Ithaca, NY 14853
EMail: vinocur@cs.cornell.edu
Kenneth Murchison
Carnegie Mellon University
5000 Forbes Avenue
Cyert Hall 285
Pittsburgh, PA 15213 USA
EMail: murch@andrew.cmu.eduFull Copyright Statement ⭐
Copyright (C) The Internet Society (2006).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.Intellectual Property 💅
Okay bestie, the IETF is lowkey not picking a side here 🫠 It takes no position on whether any Intellectual Property Rights (or other rights) that someone might claim about the tech in this doc are valid or what their scope is. It also doesn't say whether a license for those rights is available or not. And it's not claiming it went looking for any of those rights itself 🕵️♀️
The procedures for rights in RFC documents live in BCP 78 and BCP 79 📜
Want the paperwork? IPR disclosures sent to the IETF Secretariat are a thing. So are assurances that licenses will be available, and the results of anyone trying to get a general license or permission for implementers or users to use those proprietary rights. All of that can be found in the IETF's online IPR repository at http://www.ietf.org/ipr 🗂️
The IETF also invites anyone who's interested to tell it about any copyrights, patents, patent applications, or other proprietary rights that might cover technology needed to implement this standard. Send that info to the IETF at ietf-ipr@ietf.org 📬✨
Acknowledgement 🫶
Last but not least, the RFC Editor function is funded by the IETF Administrative Support Activity (IASA) 💖 Big thanks to them for keeping the lights on for the people who make these docs happen 🌸
📦 where this page comes from
- ietf: rfc4644 2006-10
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms