LLAPI_CHANGELOG_RECV(3)

📦 version 2.17.58, in ubuntu stonking

🎀 bestie welcome to LLAPI_CHANGELOG_RECV(3) 🎀

like, Lustre Library Functions but make it make sense 💅

✨ NAME ✨

llapi_changelog_recv, llapi_changelog_free — read a changelog record and then free it. That's the whole job, bestie. 📖💖

🌸 SYNOPSIS 🌸

#include <lustre/lustreapi.h>

int llapi_changelog_recv(void *priv, struct changelog_rec **rech);

int llapi_changelog_free(struct changelog_rec **rech);

💖 DESCRIPTION 💖

llapi_changelog_recv() does the main-character thing: it allocates and reads the next changelog record rech from the changelog reader instance priv.

llapi_changelog_free() then releases the allocated record rech. Cleanup queen. No leaks, periodt. 🧼✨

🔥 NOTES 🔥

If llapi_changelog_start() initializes priv with the CHANGELOG_FLAG_FOLLOW flag, then llapi_changelog_recv() can block while it waits for new records. So yeah, it might just sit there until something happens. Not ghosting you, just waiting. 😭

📝 Changelog record format 📝

llapi_changelog_recv() allocates a struct changelog_rec structure and fills it with the next changelog record. The minimal changelog record struct has these fields:

       struct changelog_rec {
               __u16             cr_namelen;     /* entry name length in bytes */
               __u16             cr_flags;       /* enum changelog_rec_flags */
               __u32             cr_type;        /* enum changelog_rec_type */
               __u64             cr_index;       /* changelog record number */
               __u64             cr_prev;        /* last cr_index used by cr_tfid */
               __u64             cr_time;
               union {
                   struct lu_fid cr_tfid;        /* target FID */
                   __u32         cr_markerflags; /* CL_MARK flags */
               };
               struct lu_fid     cr_pfid;        /* parent FID */
       } __attribute__ ((packed));

The changelog record name length cr_namelen is the length of the directory entry name of the target file or directory. That name is stored in the record after the fixed-size fields. 📛

The changelog record index cr_index is always unique and monotonically increasing. But bestie, a changelog consumer may not necessarily process all records, so you may see gaps in the index sequence. Not every crumb gets eaten. 🍪

The changelog record timestamp cr_time is a 64-bit value representing the time of the changelog record in nanoseconds since the Epoch. The top 34 bits are the seconds since the Epoch, and the bottom 30 bits are the nanoseconds. Math girlies rise. ✨

The changelog target FID cr_tfid is the FID of the file or directory that is the subject of the record. The parent FID cr_pfid is the FID of the parent directory of the target FID, if applicable. The changelog record name is relative to the parent FID. Family tree but make it filesystem. 🌳

🚩 Changelog record flags 🚩

The high 4 bits of the cr_flags field can have the following values:

       enum changelog_rec_flags {
           CLF_VERSION,     /* initial version record */
           CLF_RENAME,      /* rename in cr_sfid, cr_spfid */
           CLF_JOBID,       /* jobid in cr_jobid */
           CLF_EXTRA_FLAGS, /* extra flags in cr_extra_flags */
           CLF_FLAGMASK,    /* mask off per-record flags */
           CLF_VERMASK,
       };

The low 12 bits of cr_flags can be used for per-record flags. Currently defined per-record flags for CL_UNLINK are:

           CLF_UNLINK_LAST,        /* Unlink of last hardlink */
           CLF_UNLINK_HSM_EXISTS,  /* File has something in HSM */

Currently defined per-record flags for CL_RENAME are:

           CLF_RENAME_LAST,        /* rename unlink last link of target */
           CLF_RENAME_LAST_EXISTS, /* unlink last link of HSM target */

👑 Changelog record types 👑

The changelog record cr_type field describes the operation event type and can have the following values:

       enum changelog_rec_type {
           CL_NONE,       /* no event */
           CL_MARK,       /* internal marker with initial version */
           CL_CREATE,     /* new file created */
           CL_MKDIR,      /* new directory created */
           CL_HARDLINK,   /* new hard link created */
           CL_SOFTLINK,   /* new symbolic link created */
           CL_MKNOD,      /* new device node created */
           CL_UNLINK,     /* file unlinked */
           CL_RMDIR,      /* directory removed */
           CL_RENAME,     /* file or directory renamed */
           CL_EXT,        /* namespace extended record (2nd half of rename) */
           CL_OPEN,       /* file opened */
           CL_CLOSE,      /* file closed */
           CL_LAYOUT,     /* file layout/striping modified */
           CL_TRUNC,      /* file truncated */
           CL_SETATTR,    /* file attributes changed, precedence over xTIME */
           CL_SETXATTR,   /* file extended attributes changed */
           CL_HSM,        /* HSM specific events, see flags */
           CL_MTIME,      /* mtime changed, precendence over CTIME, ATIME */
           CL_CTIME,      /* ctime changed, precendence over ATIME */
           CL_ATIME,      /* atime changed, but no other attributes */
           CL_MIGRATE,    /* file layout migrated */
           CL_FLRW,       /* FLR: file was firstly written */
           CL_RESYNC,     /* FLR: file was resync-ed */
           CL_GETXATTR,   /* file extended attributes read */
           CL_DN_OPEN,    /* file open denied */
       };

🧩 Changelog extended records 🧩

struct changelog_rec can be extended with the following fields if corresponding flags are set:

               /* extension for RENME, when cr_flags & CLF_RENAME. */
               struct changelog_ext_rename {
                       struct lu_fid    cr_sfid;   /* source fid, or zero */
                       struct lu_fid    cr_spfid;  /* source parent fid, or zero */
               };

               /* extension for JOBID, when cr_flags & CLF_JOBID. */
               struct changelog_ext_jobid {
                       char  cr_jobid[LUSTRE_JOBID_SIZE]; /* NUL-terminated string */
               };

               /* extension for additional flags, when cr_flags & CLF_EXTRA_FLAGS. */
               struct changelog_ext_extra_flags {
                       __u64 cr_extra_flags; /* Additional CLFE_* flags */
               };

🍓 Changelog extra extensions 🍓

struct changelog_rec can be extended with the following extra fields if corresponding flags are set in struct changelog_ext_extra_flags. The cr_extra_flags field can have the following values:

               CLFE_UIDGID,   /* changelog_ext_uidgid */
               CLFE_NID,      /* changelog_ext_nid */
               CLFE_OPEN,     /* changelog_ext_openmode */
               CLFE_XATTR,    /* changelog_ext_xattr */
               CLFE_NID_BE,   /* changelog_ext_nid is struct lnet_nid */
               /* extra extension for UID/GID, when cr_extra_flags & CLFE_UIDGID. */
               struct changelog_ext_uidgid {
                       __u64     cr_uid;
                       __u64     cr_gid;
               };

               /* extra extension for NID, when cr_extra_flags & CLFE_NID. */
               struct changelog_ext_nid {
                       /* If CLFE_NID_BE is not set cr_nid is of the lnet_nid_t type.
                        * If CLFE_NID_BE is set then this is struct lnet_nid.
                        */
                       __u64     cr_nid;
                       __u64     extra;
                       __u32     padding;
               };

               /* extra extension for low 32 bits of MDS_OPEN_* flags. */
               struct changelog_ext_openmode {
                       __u32 cr_openflags; /* enum mds_open_flags */
               };

               /* extra extension to include xattr name */
               struct changelog_ext_xattr {
                       char cr_xattr[XATTR_NAME_MAX + 1]; /* NUL-terminated string */
               };

💫 RETURN VALUES 💫

  • 0 — on success, yay. 🎉
  • 1 — end of records. No more records, bestie. 💀
  • -errno — on failure. Boooo. 😭

🚨 ERRORS 🚨

  • -ENOMEM — insufficient memory to complete operation. Your RAM said no. 😵
  • -EINVAL — one or more invalid arguments are given. Read the args, babe. 🙈
  • -EIO — failed to read the changelog record on the MDT. It's giving I/O villain era. 💀

🌈 EXAMPLE 🌈

To process a sequence of changelog records:

               void *changelog_priv;
               struct changelog_rec *rec;
               char *mdtname = "testfs-MDT0000";
               char *consumer = "dirwatch";
               __u64 mask = 0; /* use mask registered for this user */
               long long startrec = 12345678;

               rc = llapi_changelog_start_user(&changelog_priv,
                               CHANGELOG_FLAG_BLOCK | CHANGELOG_FLAG_EXTRA_FLAGS |
                               CHANGELOG_FLAG_NID_BE | CHANGELOG_FLAG_JOBID |
                               CHANGELOG_FLAG_FOLLOW,
                               mdtname, consumer, mask, startrec);

               while ((rc = llapi_changelog_recv(changelog_priv, &rec)) == 0) {
                       /* process 'rec' */
                       llapi_changelog_free(&rec);
               }

🫶 AVAILABILITY 🫶

llapi_changelog_recv() and llapi_changelog_free() are part of the lustre(7) user application interface library since release 2.0.0. OG since day one. 👑

🔗 SEE ALSO 🔗

xoxo 💋 from the Lustre User API, dated 2024-08-22, LLAPI_CHANGELOG_RECV(3). bye bestie, go read those records! ✨

📦 where this page comes from