fetchlog(1)

📦 version 1.4, in fedora 43, fedora 44

✨ NAME ✨

fetchlog — fetch and convert new messages of a logfile 💌

okay bestie, quick vibe check: this is the fetchlog(1) manual page, and by the end of it you're gonna know exactly how to pull just the new bits out of a logfile and make them look cute (or at least shell-safe). no cap. 🦋

🌸 SYNOPSIS 🌸

fetchlog -f first:last:len:conv logfile bookmarkfile [ pattern ..  ]

fetchlog -F first:last:len:conv logfile bookmarkfile [ pattern ..  ]

fetchlog [-h|-V]

💅 DESCRIPTION 💅

so fetchlog shows you the last new messages of a logfile. it's giving tail(1) energy but with extra functionality for output formatting — like, tail could never, honestly. 💅

to show only the messages that showed up since the last time you called it, fetchlog uses a bookmark to remember which messages have already been fetched. iconic behaviour, fr fr.

here's the whole ritual: fetchlog scans backwards through logfile and collects all the messages, or optionally only the ones matching any of the pattern regex-style patterns you gave it. then it converts the found message lines for output. it stops scanning the moment one of these three things happens:

  1. the bookmark from bookmarkfile is reached 🎯
  2. len characters are ready for output 📏
  3. an error occurs 😭

fetchlog knows about rotated and uncompressed logfiles and keeps scanning in rotated logfiles by appending .0 to logfile, and if that file doesn't exist then .1, and so on upto .9, when scanning in rotated logfiles. and if a rotated logfile does not exist, scanning just stops without error. no drama. 🧚

🎀 OPTIONS 🎀

  • -f — do not update the bookmark in bookmarkfile.
  • -F — update the bookmark in bookmarkfile: sets the bookmark to the very last line fetched from logfile.
  • first — the first column of text fetchlog will read from logfile. lines shorter than first show up as a single newline. the column count starts with 1. 🔢
  • last — the maximum last column fetchlog will read from logfile. lines longer than last get cut off, and a ~ will be set at column last for output. the column count starts with 1. ✂️
  • len — the maximum number of characters fetchlog will output. if more than len characters are available after cutting and conversion, the first line put out will start with '...'. 📏
  • conv — conversion: one ore more of characters 'bpsno3'. 👑
    • b [brackets] — convert '<' and '>' to '(' and ')' for safe HTML output.
    • p [percent] — convert '%' to 'p' for safe printf(1) output.
    • s [shell] — convert '$', '^', and the three quotes (backquote, doublequote, singlequote) and '|' to '_' and '\' to '/'. this conversion is useful when passing the result of fetchlog to a shell command as a parameter. 🐚
    • n [newline] — convert newline characters to '\n' sequence to get a single line of output.
    • o [ok message] — show 'OK: no messages' if no new messages are available for output. ✅
    • 3 [Nagios3] — output in multiline format required for plugins since Nagios3.
  • logfile — absolute path to the unrotated logfile to fetch data from. the user needs read access to logfile. 🔐
  • bookmarkfile — absolute path to the file holding the bookmark. the user needs read access to the file when using option -f, and write access plus permission to create files in the directory of bookmarkfile when using option -F (update bookmark).

if bookmarkfile does not exist, fetchlog assumes an infinite old bookmark. 🕰️

  • pattern — an extended regular expression pattern, see re_format(7) for details. if one or more pattern are defined, fetchlog will only pick lines where any of these pattern match. pattern matching is done after trimming the lines with respect to first and last and before any conversions takes place. in other words: pattern matching operates on data that fetchlog shows when no conversion is set. 🔍
  • -h — print help message
  • -V — print version

🦋 MULTIPLE FETCHING 🦋

fetching the same logfile with different bookmarks works without problem. simultanously fetching with option -F (update mode) using the same bookmark file works too, but unpredictable results will occur. 😭

for safety reasons bookmarkfile will never be opened for writing directly — instead a temporary file will be used and renamed to bookmarkfile when writing has finished. responsible queen behaviour, we love to see it. 👑

⚡ PERFORMANCE ⚡

before opening logfiles for scanning, fetchlog first checks the modification time of logfile and compares this time with the modification time stored in bookmarkfile. if both timestamps are the same, then fetchlog exits with 0 (no messages). 🎯

otherwise the logfile(s) get mapped to memory step by step and are scanned backwards line by line until one of the end conditions become true. fetchlog uses the logfiles inode to distinguish different logs. spooky but correct. 🔥

👑 NAGIOS 👑

fetchlog may be used as a local plugin for the Nagios network monitoring system to monitor a local logfile. it follows the calling convention for Nagios plugins if at least conversions on are set and len is 'short' enough for Nagios (330 prior to Nagios 3, 4000 since Nagios 3). it is recommended to use conversion s also, because shell metacharacters in fetched messages may confuse Nagios' notification system. 🐚

Nagios 3 introduces a new interface for plugins sending out multiline output as fetchlog does. even with conversion 'n', Nagios reads this as a multiline message. to be compatible with Nagios 3 plugin format, enable conversion '3' also. output format is then:

SHORT_MESSAGE|\nMESSAGE_LINE_1\n...\nMESSAGE_LINE_N

where SHORT_MESSAGE will be the last line fetched MESSAGE_LINE_N. please see README.Nagios for details. 📖

when using fetchlog as a local plugin for Nagios, then exit status is as follows:

  • OK — no new messages in logfile where found. the single line 'OK: no messages' will be send to stdout. ✅
  • WARNING — fetchlog detected an internal error while fetching. a one line error message starting with 'ERROR: fetchlog:' is send to stdout. ⚠️
  • CRITICAL — new messages in logfile where found. a single line with the last new messages will be send to stdout. 🚨
  • UNKNOWN — fetchlog was called with wrong parameters. a multiline usage message is send to stdout. 🤷

Nagios can monitor remote logfiles together with NET-SNMP and fetchlog using the check-snmp plugin. please read the README files coming with fetchlog how to setup this. 📚

🌈 SNMP 🌈

the motivation for fetchlog was to create a helper tool for NET-SNMP's snmpd to enable monitoring of remote kernel syslogs using SNMP. if configured properly, snmpd replies to a specific SNMP request with the output of an external helper program. with fetchlog using update mode -F and conversions on, one can peek at a remote syslog file and gets either 'OK: no messages' or one line with the new syslog messages appeared since the last SNMP request. 👀

because of some limitiations of SNMP itself and the monitoring applications using SNMP, the length of the SNMP reply must not exceed a specific size. to ease the integration of SNMP in monitoring software, it is useful to have the SNMP reply only consist of a single line of text rather than multiple lines. ✨

💜 COMPRESSING OUTPUT 💜

the syslog messages fetched can be compressed by increasing first to skip the timestamp and host entry from syslog. lowering last gives even more compression for long syslog lines. as a result one gets a very short message that gives an idea of what is going wrong. 🍓

🔥 DIAGNOSTICS 🔥

fetchlog sends all output and error messages to stdout. exit codes:

  • 0 — no new messages in logfile were found. if conversion o is set, the single line 'OK: no messages' will be send to stdout.
  • 1 — an internal error in fetchlog occured. a one line error message will be send to stdout. the error message starts with 'ERROR: fetchlog:' and shows detailed information about what went wrong. length limitation by parameter len is applied — if an error message is too long, the last part will be cut off and a '~' is appended.
  • 2 — new messages in logfile were found. the new messages will be send to stdout. if conversion n is set, a single line of text is send, else zero or more lines of converted syslog messages are send.
  • 3 — fetchlog was called with wrong parameters, a multiline usage message is print to stdout.
⚠️ Note: since version 0.93 the meanings of exit code 1 and 2 have been exchanged. so if you're reading old docs, bestie, don't get got. 💀

🫶 AUTHOR 🫶

Alexander Haderer, LoeScap Technology GmbH, Berlin - Germany

🌟 SEE ALSO 🌟

tail(1), cat(1), head(1), sed(1), re_format(7), syslogd(8), newsyslog(8)

💀 BUGS 💀

  • Bookmarkfiles are not portable across plattforms or fetchlog versions.
  • Logfiles are expected not to shrink.
  • fetchlog does not work with compressed logfiles.
  • Fetching when logfile rotation takes place may result in some messages to appear twice.
  • If an regex error occurs during pattern matching, this error is silently ignored and will be handled as non-match. 🙈

Nagios is a registered trademark of Ethan Galstad.

and that's the whole page, bestie 💖 this was the fetchlog(1) manual, 23 Nov 2008 (1.2) edition — yeah she's a vintage queen from 2008, version 1.2, and she still slays. go fetch some logs. 🎀🦋✨

📦 where this page comes from