KQUEUE(2)

📦 version 12.2, in debian bookworm, debian trixie, ubuntu jammy, ubuntu noble, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 KQUEUE(2) 💖 — the kqueue, kevent situation, explained by your fave Unix bestie

okay so this is a page from the System Calls Manual, and it's literally the whole entire lore of the kernel event notification mechanism. grab a snack, bestie, because this one is a bop. 🍓✨

🌸 NAME 🌸

kqueue, kevent — the kernel event notification mechanism. it's giving "the kernel tells you when stuff happens". no cap. 💅

📚 LIBRARY 📚

Standard C Library (libc, -lc) — yes she lives in libc, as she should. 🎀

✨ SYNOPSIS ✨

#include <sys/event.h>

int
kqueue(void);

int
kevent(int kq, const struct kevent *changelist, int nchanges,
    struct kevent *eventlist, int nevents,
    const struct timespec *timeout);

EV_SET(kev, ident, filter, flags, fflags, data, udata);

look at her. three whole entry points. iconic behaviour fr fr. 👑

🦋 DESCRIPTION 🦋

so kqueue() is a generic way of telling your program "hey bestie, an event happened, or a condition is holding rn", and it does this based on the results of tiny little pieces of kernel code called filters. a kevent is identified by the (ident, filter) pair — and lowkey important: there may only be one unique kevent per kqueue. one. that's the rule. 💀

The filter gets executed the moment you initially register a kevent, specifically to detect whether a preexisting condition is present, and it also runs whenever an event gets passed to the filter for evaluation. if the filter decides the condition should be reported, then boom — the kevent gets placed on the kqueue for the user to retrieve. slay. 🫶

The filter also runs when the user tries to retrieve the kevent from the kqueue. and if the filter says "nah, the condition that triggered this event no longer holds", then the kevent gets removed from the kqueue and is not returned. it's giving self-cleaning. ✨

Multiple events that trigger the filter do not result in multiple kevents being placed on the kqueue — instead, the filter aggregates them into a single struct kevent. (efficiency queen 💅) Also: calling close() on a file descriptor will remove any kevents that reference that descriptor. automatic. we love a tidy system.

The kqueue() system call creates a brand new kernel event queue and returns a descriptor. Important lore: the queue is not inherited by a child created with fork(2). However, if rfork(2) is called without the RFFDG flag, then the descriptor table is shared, which will allow sharing of the kqueue between two processes. sharing is caring ig. 🎀

The kevent() system call is what you use to register events with the queue, and to return any pending events back to the user. here's the breakdown, bestie:

  • changelist is a pointer to an array of kevent structures, as defined in <sys/event.h>. All changes contained in the changelist are applied before any pending events are read from the queue.
  • nchanges gives the size of changelist.
  • eventlist is a pointer to an array of kevent structures.
  • nevents determines the size of eventlist.
🚨 When nevents is zero, kevent() will return immediately even if there is a timeout specified — unlike select(2). noted. different vibe entirely.

If timeout is a non-NULL pointer, it specifies a maximum interval to wait for an event, which will be interpreted as a struct timespec. If timeout is a NULL pointer, kevent() waits indefinitely (patient king 😌). To effect a poll, the timeout argument should be non-NULL, pointing to a zero-valued timespec structure. and yes — the same array may be used for both the changelist and the eventlist. recycle era. ♻️

The EV_SET() macro is provided for ease of initializing a kevent structure. thank you for your service. 🫡

🧚 The kevent structure 🧚

The kevent structure is defined as:

struct kevent {
        uintptr_t  ident;       /* identifier for this event */
        short     filter;       /* filter for event */
        u_short   flags;        /* action flags for kqueue */
        u_int     fflags;       /* filter flag value */
        int64_t   data;         /* filter data value */
        void      *udata;       /* opaque user data identifier */
        uint64_t  ext[4];       /* extensions */
};

The fields of struct kevent are:

  • ident — Value used to identify this event. The exact interpretation is determined by the attached filter, but often is a file descriptor. classic.
  • filter — Identifies the kernel filter used to process this event. The pre-defined system filters are described below. iykyk.
  • flags — Actions to perform on the event.
  • fflags — Filter-specific flags.
  • data — Filter-specific data value.
  • udata — Opaque user-defined value passed through the kernel unchanged. the kernel does not touch it, she simply passes it along. 🦋
  • ext — Extended data passed to and from kernel. The ext[0] and ext[1] members use is defined by the filter. If the filter does not use them, the members are copied unchanged. The ext[2] and ext[3] members are always passed through the kernel as-is, making additional context available to application.

👑 The flags field 👑

The flags field can contain the following values:

  • EV_ADD — Adds the event to the kqueue. Re-adding an existing event will modify the parameters of the original event, and not result in a duplicate entry. Adding an event automatically enables it, unless overridden by the EV_DISABLE flag. one of a kind, no dupes. 💅
  • EV_ENABLE — Permit kevent() to return the event if it is triggered.
  • EV_DISABLE — Disable the event so kevent() will not return it. The filter itself is not disabled. important distinction bestie, don't get it twisted.
  • EV_DISPATCH — Disable the event source immediately after delivery of an event. See EV_DISABLE above.
  • EV_DELETE — Removes the event from the kqueue. Events which are attached to file descriptors are automatically deleted on the last close of the descriptor. she cleans up after herself. 🧹
  • EV_RECEIPT — This flag is useful for making bulk changes to a kqueue without draining any pending events. When passed as input, it forces EV_ERROR to always be returned. When a filter is successfully added the data field will be zero. Note that if this flag is encountered and there is no remaining space in eventlist to hold the EV_ERROR event, then subsequent changes will not get processed. ⚠️ read that twice.
  • EV_ONESHOT — Causes the event to return only the first occurrence of the filter being triggered. After the user retrieves the event from the kqueue, it is deleted. one and done. 🎯
  • EV_CLEAR — After the event is retrieved by the user, its state is reset. This is useful for filters which report state transitions instead of the current state. Note that some filters may automatically set this flag internally. sneaky but valid.
  • EV_EOF — Filters may set this flag to indicate filter-specific EOF condition.
  • EV_ERROR — See "RETURN VALUES" below. (we'll get there bestie, patience 🫶)

🌈 The predefined system filters 🌈

The predefined system filters are listed below. Arguments may be passed to and from the filter via the fflags and data fields in the kevent structure.

🔥 EVFILT_READ 🔥

Takes a descriptor as the identifier, and returns whenever there is data available to read. The behavior of the filter is slightly different depending on the descriptor type. it's giving "depends on the context". 💅

Sockets

Sockets which have previously been passed to listen() return when there is an incoming connection pending. data contains the size of the listen backlog.

Other socket descriptors return when there is data to be read, subject to the SO_RCVLOWAT value of the socket buffer. This may be overridden with a per-filter low water mark at the time the filter is added by setting the NOTE_LOWAT flag in fflags, and specifying the new low water mark in data. On return, data contains the number of bytes of protocol data available to read.

If the read direction of the socket has shutdown, then the filter also sets EV_EOF in flags, and returns the socket error (if any) in fflags. It is possible for EOF to be returned (indicating the connection is gone) while there is still data pending in the socket buffer. 😭 plot twist but valid.

Vnodes

Returns when the file pointer is not at the end of file. data contains the offset from current position to end of file, and may be negative. (yes, negative. she can go there.)

This behavior is different from poll(2), where read events are triggered for regular files unconditionally. This event can be triggered unconditionally by setting the NOTE_FILE_POLL flag in fflags. noted for the record. 📝

Fifos, Pipes

Returns when the there is data to read; data contains the number of bytes available.

When the last writer disconnects, the filter will set EV_EOF in flags. This will be cleared by the filter when a new writer connects, at which point the filter will resume waiting for data to become available before returning. she's flexible like that. 🌸

BPF devices

Returns when the BPF buffer is full, the BPF timeout has expired, or when the BPF has "immediate mode" enabled and there is any data to read; data contains the number of bytes available.

💜 EVFILT_WRITE 💜

Takes a descriptor as the identifier, and returns whenever it is possible to write to the descriptor. For sockets, pipes and fifos, data will contain the amount of space remaining in the write buffer. The filter will set EV_EOF when the reader disconnects, and for the fifo case, this will be cleared when a new reader connects. Note that this filter is not supported for vnodes or BPF devices. 🚫

For sockets, the low water mark and socket error handling is identical to the EVFILT_READ case.

🌟 EVFILT_EMPTY 🌟

Takes a descriptor as the identifier, and returns whenever there is no remaining data in the write buffer. it's giving "the void is calling". 💀✨

🧵 EVFILT_AIO 🧵

Events for this filter are not registered with kevent() directly but are registered via the aio_sigevent member of an asynchronous I/O request when it is scheduled via an asynchronous I/O system call such as aio_read(). The filter returns under the same conditions as aio_error(). For more details on this filter see sigevent(3) and aio(4). side quest unlocked. 🗺️

  • EVFILT_VNODE — bestie, you hand it a file descriptor as the identifier and the events you wanna stalk in fflags, and it comes back at you the second one (or more) of those events goes down on that descriptor. The events you can monitor are:
    • NOTE_ATTRIB — the file that descriptor points at had its attributes changed. glow up or glow down, either way, it's noted 💅
    • NOTE_CLOSE — a file descriptor referencing the monitored file got closed, and that closed descriptor did not have write access.
    • NOTE_CLOSE_WRITE — same energy, except the closed descriptor did have write access.
heads up: this note, plus NOTE_CLOSE, do not get activated when files are closed forcibly by unmount(2) or revoke(2). In those cases NOTE_REVOKE gets sent instead. different note, different vibe.
  • NOTE_DELETE — the unlink() system call was called on the file referenced by the descriptor. it's gone, bestie 😭
  • NOTE_EXTEND — for a regular file: the file referenced by the descriptor was extended. for a directory: it reports that a directory entry was added or removed, as the result of a rename operation. and ngl, the NOTE_EXTEND event is not reported when a name is changed inside the directory. iykyk.
  • NOTE_LINK — the link count on the file changed. in particular, NOTE_LINK gets reported if a subdirectory was created or deleted inside the directory referenced by the descriptor.
  • NOTE_OPEN — the file referenced by the descriptor was opened. main character entrance fr 🚪✨
  • NOTE_READ — a read occurred on the file referenced by the descriptor.
  • NOTE_RENAME — the file referenced by the descriptor was renamed. new name, same file, iconic.
  • NOTE_REVOKE — access to the file was revoked via revoke(2), or the underlying file system was unmounted. bye 💀
  • NOTE_WRITE — a write occurred on the file referenced by the descriptor.

On return, fflags contains the events which triggered the filter.

  • EVFILT_PROC — you pass the process ID you wanna monitor as the identifier and the events you're watching for in fflags, and it fires when the process does one or more of the requested events. here's the tea: if a process can normally see another process, it can attach an event to it. the events to monitor are:
    • NOTE_EXIT — the process has exited. the exit status will be stored in data.
    • NOTE_FORK — the process has called fork().
    • NOTE_EXEC — the process has executed a new process via execve(2) or a similar call.
    • NOTE_TRACK — follow a process across fork() calls. the parent process registers a new kevent to monitor the child process using the same fflags as the original event. the child process will signal an event with NOTE_CHILD set in fflags and the parent PID in data.

if the parent process fails to register a new kevent (usually due to resource limitations), it will signal an event with NOTE_TRACKERR set in fflags, and the child process will not signal a NOTE_CHILD event. rip to the tracking 😔

On return, fflags contains the events which triggered the filter.

  • EVFILT_PROCDESC — takes the process descriptor created by pdfork(2) to monitor as the identifier and the events to watch for in fflags, and returns when the associated process performs one or more of the requested events. the events to monitor are:
    • NOTE_EXIT — the process has exited. the exit status will be stored in data.

On return, fflags contains the events which triggered the filter.

  • EVFILT_SIGNAL — takes the signal number to monitor as the identifier and returns when the given signal is delivered to the process. okay so this one coexists with the signal() and sigaction() facilities, and it has a lower precedence. the filter will record all attempts to deliver a signal to a process, even if the signal has been marked as SIG_IGN — except for the SIGCHLD signal, which, if ignored, will not be recorded by the filter. event notification happens after normal signal delivery processing. data returns the number of times the signal has occurred since the last call to kevent(). and psst — this filter automatically sets the EV_CLEAR flag internally. it's giving self-sufficient queen 👑
  • EVFILT_TIMER — establishes an arbitrary timer identified by ident. when adding a timer, data specifies the moment to fire the timer (for NOTE_ABSTIME) or the timeout period. the timer will be periodic unless EV_ONESHOT or NOTE_ABSTIME is specified. on return, data contains the number of times the timeout has expired since the last call to kevent(). for non-monotonic timers, this filter automatically sets the EV_CLEAR flag internally.

the filter accepts the following flags in the fflags argument:

  • NOTE_SECONDS — data is in seconds.
  • NOTE_MSECONDS — data is in milliseconds.
  • NOTE_USECONDS — data is in microseconds.
  • NOTE_NSECONDS — data is in nanoseconds.
  • NOTE_ABSTIME — the specified expiration time is absolute.

if fflags is not set, the default is milliseconds. on return, fflags contains the events which triggered the filter.

if an existing timer is re-added, the existing timer will be effectively canceled (throwing away any undelivered record of previous timer expiration, so it's giving ghosting fr) and re-started using the new parameters contained in data and fflags.

also!! there's a system wide limit on the number of timers, and it's controlled by the kern.kq_calloutmax sysctl. don't go feral with timers bestie 🫶
  • EVFILT_USER — establishes a user event identified by ident which is not associated with any kernel mechanism but is triggered by user level code. the lower 24 bits of the fflags may be used for user defined flags and manipulated using the following:
    • NOTE_FFNOP — ignore the input fflags.
    • NOTE_FFAND — bitwise AND fflags.
    • NOTE_FFOR — bitwise OR fflags.
    • NOTE_FFCOPY — copy fflags.
    • NOTE_FFCTRLMASK — control mask for fflags.
    • NOTE_FFLAGSMASK — user defined flag mask for fflags.

a user event is triggered for output with the following:

  • NOTE_TRIGGER — cause the event to be triggered. go bestie, go 🎬

On return, fflags contains the users defined flags in the lower 24 bits.

🎀 CANCELLATION BEHAVIOUR 🎀

okay so if nevents is non-zero — i.e., the function is potentially blocking — the call is a cancellation point. otherwise, i.e., if nevents is zero, the call is not cancellable. and the way cancellation works: it can only occur before any changes are made to the kqueue, or when the call was blocked and no changes to the queue were requested. no take-backs mid-edit, that's the rule 💅

💖 RETURN VALUES 💖

the kqueue() system call creates a new kernel event queue and returns a file descriptor. if there was an error creating the kernel event queue, a value of -1 is returned and errno set. sad girl hours 😭

the kevent() system call returns the number of events placed in the eventlist, up to the value given by nevents. if an error occurs while processing an element of the changelist and there is enough room in the eventlist, then the event will be placed in the eventlist with EV_ERROR set in flags and the system error in data. otherwise, -1 will be returned, and errno will be set to indicate the error condition. and if the time limit expires, then kevent() returns 0. periodt. ✨

✨ EXAMPLES ✨

okay bestie, time for the main event — a whole working program that watches a file and yells at you the second somebody writes to it 🦋 this is the classic kqueue starter pack, fr fr.

#include <sys/event.h>
#include <err.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int
main(int argc, char **argv)
{
    struct kevent event;    /* Event we want to monitor */
    struct kevent tevent;   /* Event triggered */
    int kq, fd, ret;

    if (argc != 2)
        err(EXIT_FAILURE, "Usage: %s path\n", argv[0]);
    fd = open(argv[1], O_RDONLY);
    if (fd == -1)
        err(EXIT_FAILURE, "Failed to open '%s'", argv[1]);

    /* Create kqueue. */
    kq = kqueue();
    if (kq == -1)
        err(EXIT_FAILURE, "kqueue() failed");

    /* Initialize kevent structure. */
    EV_SET(&event, fd, EVFILT_VNODE, EV_ADD | EV_CLEAR, NOTE_WRITE,
        0, NULL);
    /* Attach event to the kqueue. */
    ret = kevent(kq, &event, 1, NULL, 0, NULL);
    if (ret == -1)
        err(EXIT_FAILURE, "kevent register");
    if (event.flags & EV_ERROR)
        errx(EXIT_FAILURE, "Event error: %s", strerror(event.data));

    for (;;) {
        /* Sleep until something happens. */
        ret = kevent(kq, NULL, 0, &tevent, 1, NULL);
        if (ret == -1) {
            err(EXIT_FAILURE, "kevent wait");
        } else if (ret > 0) {
            printf("Something was written in '%s'\n", argv[1]);
        }
    }
}

so what's actually happening here, step by step, because the vibes deserve a breakdown 💅

  1. the includes — <sys/event.h> is the whole reason we're here, plus the usual <err.h>, <fcntl.h>, <stdio.h>, <stdlib.h> and <string.h> crowd.
  2. two struct kevents — event is the one we're setting up to monitor, tevent is the one that comes back at us when the thing triggers. two different jobs, two different structs, no cap.
  3. arg check + open() — no path argument? err(EXIT_FAILURE, "Usage: %s path\n", argv[0]) and we're out. otherwise we open(argv[1], O_RDONLY) and if that fails we err(EXIT_FAILURE, "Failed to open '%s'", argv[1]).
  4. kq = kqueue(); — the actual kqueue, born. if it returns -1, err(EXIT_FAILURE, "kqueue() failed") and bye 👋
  5. EV_SET() — we build the event: EV_SET(&event, fd, EVFILT_VNODE, EV_ADD | EV_CLEAR, NOTE_WRITE, 0, NULL). that's: watch this fd, with the EVFILT_VNODE filter, EV_ADD | EV_CLEAR flags, and we care about NOTE_WRITE. udata is NULL, deal with it.
  6. registering it — kevent(kq, &event, 1, NULL, 0, NULL) attaches it to the kqueue. ret == -1 → err(EXIT_FAILURE, "kevent register"). and if event.flags & EV_ERROR? that's errx(EXIT_FAILURE, "Event error: %s", strerror(event.data)) — registering didn't actually work, bestie 😭
  7. the forever loop — kevent(kq, NULL, 0, &tevent, 1, NULL) just sleeps until something happens. if ret == -1 we die with err(EXIT_FAILURE, "kevent wait"), and if ret > 0 we get to scream printf("Something was written in '%s'\n", argv[1]) into the void. iconic behaviour, honestly.
and yes, it literally never exits on its own. it's a for (;;). it lives there now. it's the main character of your terminal 🫶

💀 ERRORS 💀

okay so when things go wrong (and they will, bestie, that's just life 🌧️), here's the whole receipt.

🫧 the kqueue() system call fails if:

  • [ENOMEM] — the kernel tried to allocate memory for the kernel queue and could not. it's giving "no room at the inn" 💀
  • [ENOMEM] — the RLIMIT_KQUEUES rlimit (peep getrlimit(2)) for the current user would be exceeded. you hit your kqueue budget, babes.
  • [EMFILE] — the per-process descriptor table is full. every slot taken, not a single one left 🥲
  • [ENFILE] — the system file table is full. not just you — the whole machine is booked out.

🫧 the kevent() system call fails if:

  • [EACCES] — the process does not have permission to register a filter. no permission, no vibes 🚫
  • [EFAULT] — there was an error reading or writing the kevent structure. the pointer situation is cursed.
  • [EBADF] — the specified descriptor is invalid. that fd is not real, bestie.
  • [EINTR] — a signal was delivered before the timeout expired and before any events were placed on the kqueue for return.
  • [EINTR] — a cancellation request was delivered to the thread, but not yet handled.
  • [EINVAL] — the specified time limit or filter is invalid. invalid input, invalid behaviour, moving on 💅
  • [ENOENT] — the event could not be found to be modified or deleted. you're trying to edit something that isn't there, queen.
  • [ENOMEM] — no memory was available to register the event, or — special case alert 🚨 — for a timer, the maximum number of timers has been exceeded. that max is configurable via the kern.kq_calloutmax sysctl.
  • [ESRCH] — the specified process to attach to does not exist. ghost process, literally 👻
ngl this one's important: when a kevent() call fails with the EINTR error, all changes in the changelist have been applied. not some of them. all of them. remember that 🧠

👀 SEE ALSO 👀

go read your other favs, they're all linked up like a friend group:

and for the deep lore girlies (this is the source material, the origin story 📖):

  • Jonathan Lemon, “Kqueue: A Generic and Scalable Event Notification Facility”, Proceedings of the FREENIX Track: 2001 USENIX Annual Technical Conference, USENIX Association, June 25-30, 2001.

📜 HISTORY 📜

the kqueue() and kevent() system calls first appeared in FreeBSD 4.1. ancient. legendary. still eating every day ✨

👑 AUTHORS 👑

the kqueue() system and this manual page were written by Jonathan Lemon <jlemon@FreeBSD.org>. absolute icon, we thank him for his service 🫡💖

🐛 BUGS 🐛

  • the timeout value is limited to 24 hours — longer timeouts will be silently reinterpreted as 24 hours. silently!! it will not tell you!! it will not warn you!! it just caps you at 24 hours and moves on 🙈
  • in versions older than FreeBSD 12.0, <sys/event.h> failed to parse without including <sys/types.h> manually. so if you're time-travelling, include it yourself, bestie 🕰️

and that's the whole page, bestie 🎀 thank you for coming to my kqueue seminar. this one's from the Debian crew, dated May 1, 2020, filed under KQUEUE(2). go forth and monitor your file descriptors, you absolute legend 👑💜

📦 where this page comes from