fetchnews(8)

🌸 fetchnews(8) — fetch news via NNTP 🌸

okay bestie, pull up a chair, because this is the manual page for fetchnews from the leafnode manual set, section 8 (system manager's manual, iykyk). it's the little program that goes and gets your news. let's get into it 💅

---

✨ NAME ✨

fetchnews — fetch news via NNTP 🦋

---

✨ SYNOPSIS ✨

fetchnews [-q] [-v] ... [-x number] [-l] [-n] [-f] [-P] [-w]

---

✨ DESCRIPTION ✨

so. Leafnode is a USENET package made for small sites — like, few users, barely any disk space, but you still want a huge number of groups. it's giving "small but mighty" energy fr fr. 👑

the whole design vibe of leafnode is that it self-repairs after problems and needs no manual maintenance. iconic behaviour honestly, no cap.

fetchnews (which used to be called fetch, throwback 💀) is the program that submits and retrieves new articles to or from the upstream NNTP server. it's the main character of this page. ✨

---

✨ OPTIONS ✨

• -q — be quiet. suppresses some warnings. this flag cancels any previously encountered -v flags. shhh 🤫
• -v — be verbose. this flag cancels any previously encountered -q flags. you can stack a maximum of four -vs (the more v's, the more verbose, obviously 💅):
• -v — warn about non-critical errors.
• -vv — be verbose whenever a group command occurs.
• -vvv — be verbose whenever an article command occurs.
• -vvvv — for debugging purposes. do not expect the additional output to make sense. it's giving chaos, and that's the point 🙈
• -w — wait mode. this makes the XOVER update (which runs after the article fetch phase) run in the foreground. by default, the XOVER update runs in the background, detached from the shell, on systems that offer a working fork(2) call. helpful for debugging and for avoiding terminal clobbering when you run fetchnews manually. 🫶
• -x number — go number articles back and try to re-fetch what's missing. this can be useful to pull data from broken servers that have either overview and group data out of synch or create articles out of article number order. messy server behaviour, but we cope. 😭
• -l — do not use supplementary servers (-l is supposed to mean "local"). this does speed up the operation quite a bit. lowkey a glow-up for your runtime ⭐
• -n — do not expire newsgroup subscriptions. technically: do not unlink files in the interesting.groups directory.
• -f — erase the active.read file, thus force redownloading the whole active file from all upstream servers. this is usually occurring only very infrequently because it takes a long time. see timeout_active in leafnode(8).

DO NOT use this option in cron jobs or otherwise regularly. seriously bestie, do not. 🚨

• -P — only post articles that have accumulated in the out.going directory but do not fetch articles from upstream servers.

---

✨ EXIT STATUS ✨

• exits with code 0 if it could connect to all servers and encountered no other errors. slay ✨
• exits with code 2 when it was unable to connect to at least one server.
• exits with code 1 on other errors.

---

✨ FILES AND CONFIGURATION ✨

• /var/spool/news/leaf.node/active.read — is a file to track when the active files have been successfully downloaded from the upstream servers the last time. if it is missing, this has the same effect as giving the -f option. it is only created after the active file of each upstream servers has been downloaded successfully. 🎀

see leafnode(8) for the list of other files affected, and for information on configuring fetchnews. 💜

---

✨ ENVIRONMENT ✨

• LN_SKIP_GROUPS=pattern[,pattern[...]] (since v1.9.53) — if set, all groups that match one of the wildmat(3) patterns will be skipped during the fetch. this can be used as a quick way of skipping binary groups during daytime, for instance, example:

env 'LN_SKIP_GROUPS=*binary*,alt.*' /usr/sbin/fetchnews -nq

• LN_LOCK_TIMEOUT — this variable is parsed as an unsigned integer value and determines how many seconds fetchnews will wait when trying to obtain the lock file from another leafnode program. 0 means to wait indefinitely. this variable takes precedence over the configuration file. ⏳
• LN_SUPPRESS_DATE (since v1.11.7) — if this variable is defined (regardless of its value, including empty), leafnode will skip checking the upstream server's time by means of the NNTP DATE command. 🕐

---

✨ ALGORITHM ✨

here's the brief tea on what fetchnews actually does. 🍵

First, after connecting to the upstream NNTP server, all articles that have the u+r permission (0400) flag set in /var/spool/news/out.going are posted using POST. if a posting fails, that article is moved to /var/spool/news/failed.postings which an administrator has to clean up once in a while. moving a file back to /var/spool/news/out.going is sufficient to retry the post it. the u+r flag is checked to prevent posting an (incomplete) article that leafnode(8) is still receiving at the same time. if a posting succeeds, the article is removed from out.going in order to not post it twice into a moderated newsgroup on a different server (which would send multiple copies of the article to the moderator).

note: older fetchnews versions tried to post on all servers first, but failed to do so when one of the servers could not be reached. if you need to work around a dodgy upstream server that is fast but does not forward your posts reliably, prohibit posting to this server using the nopost server option in the configuration file, see leafnode(8).

Second, the upstream server's list of newsgroups (obtained using LIST and LIST NEWSGROUPS) is merged into leafnode's (if timeout_active has expired, the active.read file is not present or the active file is empty) or updated (with NEWGROUPS).

Third, fetchnews iterates over the list of newsgroups, performing a GROUP, an XOVER (if supported) or XHDR Message-ID and a number of HEAD and BODY or ARTICLE commands for each group which has been read recently. it assumes that for the particular group it is looking at, maxage was set to the groupexpire or global expire. as a consequence, it makes only sense to set maxage lower than or equal to expire. math is math 💅

---

✨ NOTES ✨

• Fetchnews does not use NEWNEWS. NEWNEWS is not used because it is often very expensive and therefore also disabled at many sites. not worth the drama 🚫
• after some experiments, I decided that it was better to fetch all the articles in a group than to optimize away some; users would complain too loudly when the optmizer guessed wrongly. (that typo is in the original, bestie, I'm just the messenger 🙈)
• the only way to control what newsgroups are accessible to local users is to control what the upstream server lets fetchnews read, and that will not keep out all crossposts (endemic in, e.g., the alt.sex groups). if your site is big enough that you need to control newsgroup access, consider using a news server which is designed for a big site. no gatekeeping here, we simply cannot 🌈
• Fetchnews sets its real and effective uid to "news" when started as root and refuses to run if it cannot get this user ID. boundaries, periodt 👑

---

✨ BUGS ✨

Fetchnews ignores the "Supersedes:" and "Control:" headers. 💀 that's just how it is, sorry bestie.

---

✨ HISTORY ✨

older versions of fetchnews posted any articles found in out.going. since v1.9.23, fetchnews will only post article files that have their user executable bit set. Leafnode sets this bit after it has successfully received a post. this behaviour avoids posting incomplete articles when leafnode receives a post at the same time fetchnews is posting. growth, honestly 🌱

---

✨ AUTHOR ✨

written by Arnt Gulbrandsen <agulbra@troll.no> and copyright 1995–96 Troll Tech AS, Postboks 6133 Etterstad, 0602 Oslo, Norway, fax +47 22646949. 🫶

modified by Cornelius Krasel <krasel@wpxx02.toxi.uni-wuerzburg.de>, Markus Enzenberger <enz@cip.physik.uni-muenchen.de> and Randolf Skerka <Randolf.Skerka@gmx.de>. copyright of the modifications 1997–1999. modified by Matthias Andree <matthias.andree@gmx.de>, copyright of the modifications 2001–2004. legends, all of them 👑✨

---

✨ SEE ALSO ✨

leafnode(8), texpire(8), newsq(1), /etc/news/leafnode/leafnode/filters.example.

---

and that's the whole page bestie!! this one ships with leafnode 1.12.0, and it's a page from the fetchnews(8) corner of the system manager's manual. go read your news, drink some water, and remember: never run -f in cron. mwah 💖🎀🦋