KLOGD(8)

📦
version 1.5.1 · 2 versions
, in opensuse-leap 16.0

💖 NAME 💖

okay so bestie, this page is all about klogd — the Kernel Log Daemon (that's klogd(8) if you're being formal 👑). It's the little system daemon that intercepts and logs Linux kernel messages. literally its whole job. slay. ✨

🎀 SYNOPSIS 🎀

klogd [ -c n ] [ -d ] [ -f fname ] [ -iI ] [ -n ] [ -o ] [ -p ] [ -s ] [
-k fname ] [ -v ] [ -x ] [ -2 ]

no cap, that's the whole vibe check of how you call it. 🦋

🌸 DESCRIPTION 🌸

klogd is a system daemon which intercepts and logs Linux kernel messages. that's it, that's the main character energy. 💅

✨ OPTIONS ✨

  • -c n — Sets the default log level of console messages to n. 🎚️
  • -d — Enable debugging mode. This will generate LOTS of output to stderr. like, LOTS LOTS. consider yourself warned bestie 😭
  • -f file — Log messages to the specified filename rather than to the syslog facility. file > syslog, iykyk 🍓
  • -i -I — Signal the currently executing klogd daemon. Both of these switches control the loading/reloading of symbol information. The -i switch signals the daemon to reload the kernel module symbols. The -I switch signals for a reload of both the static kernel symbols and the kernel module symbols. iconic duo fr fr 💖
  • -n — Avoid auto-backgrounding. This is needed especially if the klogd is started and controlled by init(8). no disappearing into the background, we stay present 🫶
  • -o — Execute in 'one-shot' mode. This causes klogd to read and log all the messages that are found in the kernel message buffers. After a single read and log cycle the daemon exits. one and done, periodt. ⭐
  • -p — Enable paranoia. This option controls when klogd loads kernel module symbol information. Setting this switch causes klogd to load the kernel module symbol information whenever an Oops string is detected in the kernel message stream. paranoia? valid. 🙈
  • -s — Force klogd to use the system call interface to the kernel message buffers. making it take the syscall route, no debating 🚀
  • -k file — Use the specified file as the source of kernel symbol information. symbol info from this file specifically, bestie 📁
  • -v — Print version and exit. quick little flex and then bounce 👋
  • -x — Omits EIP translation and therefore doesn't read the System.map file. no System.map, no EIP translation, simple as that 💀
  • -2 — When symbols are expanded, print the line twice. Once with addresses converted to symbols, once with the raw text. This allows external programs such as ksymoops do their own processing on the original data. double the line, double the fun 🌈

🌟 OVERVIEW 🌟

The functionality of klogd has been typically incorporated into other versions of syslogd but this seems to be a poor place for it. like, respectfully, not the move. In the modern Linux kernel a number of kernel messaging issues such as sourcing, prioritization and resolution of kernel addresses must be addressed. Incorporating kernel logging into a separate process offers a cleaner separation of services. separate process = cleaner vibes, highkey. 🧚

In Linux there are two potential sources of kernel log information: the /proc file system and the syscall (sys_syslog) interface, although ultimately they are one and the same. klogd is designed to choose whichever source of information is the most appropriate. It does this by first checking for the presence of a mounted /proc file system. If this is found the /proc/kmsg file is used as the source of kernel log information. If the proc file system is not mounted klogd uses a system call to obtain kernel messages. The command line switch (-s) can be used to force klogd to use the system call interface as its messaging source. it's giving "i'll pick the best option available" 💅

If kernel messages are directed through the syslogd daemon the klogd daemon, as of version 1.1, has the ability to properly prioritize kernel messages. Prioritization of the kernel messages was added to it at approximately version 0.99pl13 of the kernel. The raw kernel messages are of the form:

<[0-7]>Something said by the kernel.

The priority of the kernel message is encoded as a single numeric digit enclosed inside the <> pair. The definitions of these values is given in the kernel include file kernel.h. When a message is received from the kernel the klogd daemon reads this priority level and assigns the appropriate priority level to the syslog message. If file output (-f) is used the prioritization sequence is left pre-pended to the kernel message. priorities understood, no cap. 🔥

The klogd daemon can also be used in a 'one-shot' mode for reading the kernel message buffers. One shot mode is selected by specifying the -o switch on the command line. Output will be directed to either the syslogd daemon or to an alternate file specified by the -f switch. 🎯

For example, to read all the kernel messages after a system boot and record them in a file called krnl.msg the following command would be given.

klogd -o -f ./krnl.msg

obsessed with a one-shot boot-log moment, honestly. 🦋✨

💅 KERNEL ADDRESS RESOLUTION 💅

okay so picture this: the kernel hits an internal error condition and it literally just triggers a general protection fault. and as part of the GPF handling procedure, the kernel prints out a whole status report showing off the state of the processor at the exact moment of the fault. iconic, honestly. what's in this display? the contents of the microprocessor's registers, the contents of the kernel stack, and a tracing of what functions were being executed when the fault went down.

and bestie, this information is EXTREMELY IMPORTANT for figuring out what caused the internal error condition. the difficulty comes when a kernel developer tries to actually analyze it. the raw numeric information sitting in the protection fault printout is of very little use to the developers, no cap. why? because kernels are not identical — the addresses of variable locations or functions will not be the same in all kernels. so to correctly diagnose the cause of failure, a kernel developer needs to know what specific kernel functions or variable locations were actually involved in the error.

so here's the slay: as part of the kernel compilation process, a listing gets created that specifies the address locations of important variables and functions in the kernel being compiled. this listing is saved in a file called System.map in the top of the kernel directory source tree. using this listing, a kernel developer can determine exactly what the kernel was doing when the error condition occurred. chef's kiss.

the process of resolving those numeric addresses from the protection fault printout can be done manually, or by using the ksymoops program which is included in the kernel sources.

as a convenience, klogd will attempt to resolve kernel numeric addresses to their symbolic forms if a kernel symbol table is available at execution time. and if you require the original address of the symbol? use the -2 switch to preserve the numeric address. a symbol table can be specified by using the -k switch on the command line. if a symbol file is not explicitly specified, these filenames get tried:

/boot/System.map
/System.map
/usr/src/linux/System.map

version information is supplied in the system maps as of kernel 1.3.43 ✨ and this version information is used to direct an intelligent search of the list of symbol tables. this feature is super useful since it provides support for both production and experimental kernels.

for example, a production kernel may have its map file stored in /boot/System.map. and if an experimental or test kernel is compiled with the sources in the 'standard' location of /usr/src/linux, the system map will be found in /usr/src/linux/System.map. so when klogd starts under the experimental kernel, the map in /boot/System.map will be bypassed in favor of the map in /usr/src/linux/System.map. it's giving smart-girl energy fr fr.

modern kernels as of 1.3.43 properly format important kernel addresses so that they will be recognized and translated by klogd. earlier kernels require a source code patch be applied to the kernel sources — this patch is supplied with the sysklogd sources.

now, the process of analyzing kernel protection faults works very well with a static kernel. but additional difficulties are encountered when attempting to diagnose errors which occur in loadable kernel modules 🙈 loadable kernel modules are used to implement kernel functionality in a form which can be loaded or unloaded at will. the use of loadable modules is useful from a debugging standpoint, and can also be useful in decreasing the amount of memory required by a kernel.

the difficulty with diagnosing errors in loadable modules is due to the dynamic nature of the kernel modules. when a module is loaded, the kernel will allocate memory to hold the module; when the module is unloaded, this memory will be returned back to the kernel. this dynamic memory allocation makes it impossible to produce a map file which details the addresses of the variables and functions in a kernel loadable module. and without this location map, it is not possible for a kernel developer to determine what went wrong if a protection fault involves a kernel module. 💀

klogd has support for dealing with this whole problem. at program start time, or in response to a signal, the daemon will interrogate the kernel for a listing of all modules loaded and the addresses in memory they are loaded at. individual modules can also register the locations of important functions when the module is loaded. the addresses of these exported symbols are also determined during this interrogation process. multitasking queen behavior.

when a protection fault occurs, an attempt will be made to resolve kernel addresses from the static symbol table. if that fails, the symbols from the currently loaded modules are examined in an attempt to resolve the addresses. at the very minimum, this allows klogd to indicate which loadable module was responsible for generating the protection fault. additional information may be available if the module developer chose to export symbol information from the module. 🫶

proper and accurate resolution of addresses in kernel modules requires that klogd be informed whenever the kernel module status changes. the -i and -I switches can be used to signal the currently executing daemon that symbol information be reloaded. and of most importance to proper resolution of module symbols is the -i switch. each time a kernel module is loaded or removed from the kernel, the following command should be executed:

klogd -i

the -p switch can also be used to insure that module symbol information is up to date — it instructs klogd to reload the module symbol information whenever a protection fault is detected. but caution should be used before invoking the program in 'paranoid' mode. the stability of the kernel and the operating environment is always under question when a protection fault occurs. since the klogd daemon must execute system calls in order to read the module symbol information, there is the possibility that the system may be too unstable to capture useful information. 😭 a much better policy is to insure that klogd is updated whenever a module is loaded or unloaded. having uptodate symbol information loaded increases the probability of properly resolving a protection fault if it should occur.

included in the sysklogd source distribution is a patch to the modules-2.0.0 package which allows the insmod, rmmod and modprobe utilities to automatically signal klogd whenever a module is inserted or removed from the kernel. using this patch will insure that the symbol information maintained in klogd is always consistent with the current kernel state. teamwork makes the dream work, periodt.

🎀 CONSOLE LOG LEVEL 🎀

the klogd daemon allows the ability to alter the presentation of kernel messages to the system console. consequent with the prioritization of kernel messages was the inclusion of default messaging levels for the kernel. in a stock kernel, the default console log level is set to 7. any messages with a priority level numerically lower than 7 (aka higher priority) appear on the console.

messages of priority level 7 are considered to be 'debug' messages, and will thus not appear on the console. many administrators — particularly in a multi-user environment — prefer that all kernel messages be handled by klogd and either directed to a file or to the syslogd daemon. this prevents 'nuisance' messages such as line printer out of paper or disk change detected from cluttering the console. so much cleaner, honestly 🦋

when -c is given on the commandline, the klogd daemon will execute a system call to inhibit all kernel messages from being displayed on the console. former versions always issued this system call and defaulted to all kernel messages except for panics. this is handled differently currently, so klogd doesn't need to set this value anymore. the argument given to the -c switch specifies the priority level of messages which will be directed to the console. note that messages of a priority value LOWER than the indicated number will be directed to the console.

for example, to have the kernel display all messages with a priority level of 3 (KERN_ERR) or more severe, the following command would be executed: text klogd -c 4

the definitions of the numeric values for kernel messages are given in the file kernel.h, which can be found in the /usr/include/linux directory if the kernel sources are installed. these values parallel the syslog priority values which are defined in the file syslog.h found in the /usr/include/sys sub-directory.

the console log level is usually configured with the sysctl(8) program, directly or via its configuration file /etc/sysctl.conf. in this file, the following line

kernel.printk = 4 4 1 7

corresponds to the sampe setting above. iykyk 💜

🌈 SIGNAL HANDLING 🌈

klogd will respond to eight signals: SIGHUP, SIGINT, SIGKILL, SIGTERM, SIGTSTP, SIGUSR1, SIGUSR2 and SIGCONT. the SIGINT, SIGKILL, SIGTERM and SIGHUP signals will cause the daemon to close its kernel log sources and terminate gracefully. bye bestie, clean exit 💅

the SIGTSTP and SIGCONT signals are used to start and stop kernel logging. upon receipt of a SIGTSTP signal, the daemon will close its log sources and spin in an idle loop. subsequent receipt of a SIGCONT signal will cause the daemon to go through its initialization sequence and re-choose an input source. using SIGSTOP and SIGCONT in combination, the kernel log input can be re-chosen without stopping and restarting the daemon. for example, if the /proc file system is to be un-mounted, the following command sequence should be used:

# kill -TSTP pid
# umount /proc
# kill -CONT pid

notations will be made in the system logs with LOG_INFO priority documenting the start/stop of logging. little paper trail, love that ✨

the SIGUSR1 and SIGUSR2 signals are used to initiate loading/reloading of kernel symbol information. receipt of the SIGUSR1 signal will cause the kernel module symbols to be reloaded. signaling the daemon with SIGUSR2 will cause both the static kernel symbols and the kernel module symbols to be reloaded.

provided that the System.map file is placed in an appropriate location, the signal of generally greatest usefulness is the SIGUSR1 signal. this signal is designed to be used to signal the daemon when kernel modules are loaded/unloaded. sending this signal to the daemon after a kernel module state change will insure that proper resolution of symbols will occur if a protection fault occurs in the address space occupied by a kernel module. iykyk, it's the main character signal 👑

🍓 FILES 🍓

  • /proc/kmsg — One Source for kernel messages klogd
  • /var/run/klogd.pid — The file containing the process id of klogd
  • /boot/System.map, /System.map, /usr/src/linux/System.map — Default locations for kernel system maps.

💀 BUGS 💀

Probably numerous. Well formed context diffs appreciated. 🫶

👑 AUTHORS 👑

Okay so bestie, let's give credit where credit is due, because this daemon has history 💅

  • Steve Lord — the OG, the main character who originally wrote the kernel log daemon klogd. His contact back in the day: lord@cray.com ✨
  • Greg Wettstein — came through and made major improvements. Not a little glow-up, a major one. We stan a contributor who actually ships 🫶
  • Martin Schulze — fixed some bugs (the classic "not me fixing bugs" energy 😭) and then straight-up took over maintenance. Reach him at joey@infodrom.org if you need him 👑

So like, three whole legends carried this thing. Respect the lineage, fr fr 🌸🦋

And that's a wrap on klogd(8), bestie 🎀 This little masterpiece is Version 1.5, dated 27 May 2007, filed under KLOGD(8) — go forth, read your kernel logs, and slay 💖🔥👑

📦 where this page comes from