dirq(3)

📦 version 0.5, in epel 9

💖 NAME 💖

okay so bestie, welcome to dirq(3) — it’s literally the C implementation of the simple directory queue algorithm. that’s the whole vibe. ✨

✨ SYNOPSIS ✨

here’s the whole API at a glance, no cap — exact code, character for character:

         /*
          * constants
          */

         #define DIRQ_VERSION_MAJOR 0
         #define DIRQ_VERSION_MINOR 5
         #define DIRQ_VERSION_HEX ((DIRQ_VERSION_MAJOR << 8) | DIRQ_VERSION_MINOR)

         /*
          * types
          */

         typedef struct dirq_s *dirq_t;
         typedef int (*dirq_iow)(dirq_t, char *, size_t);
         typedef int (*dirq_ior)(dirq_t, const char *, size_t);

         /*
          * constructors & destructor
          */

         dirq_t dirq_new  (const char *path);
         dirq_t dirq_copy (dirq_t dirq);
         void   dirq_free (dirq_t dirq);

         /*
          * accessors
          */

         void   dirq_set_granularity (dirq_t dirq, int value);
         int    dirq_get_granularity (dirq_t dirq);
         void   dirq_set_rndhex      (dirq_t dirq, int value);
         int    dirq_get_rndhex      (dirq_t dirq);
         void   dirq_set_umask       (dirq_t dirq, mode_t value);
         mode_t dirq_get_umask       (dirq_t dirq);
         void   dirq_set_maxlock     (dirq_t dirq, int value);
         int    dirq_get_maxlock     (dirq_t dirq);
         void   dirq_set_maxtemp     (dirq_t dirq, int value);
         int    dirq_get_maxtemp     (dirq_t dirq);

         /*
          * iterators
          */

         const char *dirq_first (dirq_t dirq);
         const char *dirq_next  (dirq_t dirq);

         /*
          * main methods
          */

         const char *dirq_add      (dirq_t dirq, dirq_iow cb);
         const char *dirq_add_path (dirq_t dirq, const char *path);
         int         dirq_get      (dirq_t dirq, const char *name, dirq_ior cb);
         const char *dirq_get_path (dirq_t dirq, const char *name);
         int         dirq_lock     (dirq_t dirq, const char *name, int permissive);
         int         dirq_unlock   (dirq_t dirq, const char *name, int permissive);
         int         dirq_remove   (dirq_t dirq, const char *name);
         int         dirq_touch    (dirq_t dirq, const char *name);
         int         dirq_get_size (dirq_t dirq, const char *name);
         int         dirq_count    (dirq_t dirq);
         int         dirq_purge    (dirq_t dirq);

         /*
          * other methods
          */

         void        dirq_now         (dirq_t dirq, struct timespec *ts);
         int         dirq_get_errcode (dirq_t dirq);
         const char *dirq_get_errstr  (dirq_t dirq);
         void        dirq_clear_error (dirq_t dirq);

🌸 DESCRIPTION 🌸

the goal of this library is to offer a "simple" queue system using the underlying filesystem for storage, security and to prevent race conditions via atomic operations. it focuses on simplicity, robustness and scalability. periodt. 💅

multiple concurrent readers and writers can interact with the same queue. yes bestie, it’s giving concurrent main character energy. 🔥

other implementations of the same algorithm exist so readers and writers can be written in different programming languages:

the Perl implementation is the reference one and contains extensive documentation. iykyk. 📚

🦋 OVERVIEW 🦋

the object oriented API to access the directory queue is implemented with an opaque data type (dirq_t), a constructor (dirq_new) and methods which are C functions requiring the "object" as their first argument. the destructor (dirq_free) must be called when the object is not needed anymore. don’t ghost it, free it. 🫶

the directory queue object is not considered to be thread safe: different threads must use different objects. no sharing, bestie — that’s the rule. 💀

all the functions that return a string (i.e. const char *) in fact return a pointer to statically allocated data inside the directory queue object. the caller must use or copy the string before calling other functions since they may override this statically allocated data. so copy it before it gets yeeted. 😭

adding and getting elements (i.e. sequences of bytes) is achieved via callback functions working on chunks of bytes so it is never needed to allocate memory holding complete elements. iconic, honestly. ✨

in case of error, the dirq_get_errcode and dirq_get_errstr functions can be used to get more information. the safest approach is then to stop using the object and free it. however, if needed, the error information can be cleared with dirq_clear_error. 🌈

👑 FUNCTIONS 👑

here’s every single function, one by one — signatures exact, facts intact, vibes immaculate. 🎀

  • dirq_t dirq_new (const char *path) — creates a new directory queue object using the given path, setting the internal error information (see dirq_get_errcode or dirq_get_errstr) in case of error.
  • dirq_t dirq_copy (dirq_t dirq) — creates a new directory queue object which is a copy of the given one.
  • void dirq_free (dirq_t dirq) — frees the memory associated with the directory queue object.
  • void dirq_set_granularity (dirq_t dirq, int value) — sets the time granularity for intermediate directories (default: 60).
  • int dirq_get_granularity (dirq_t dirq) — gets the time granularity for intermediate directories.
  • void dirq_set_rndhex (dirq_t dirq, int value) — sets the random hexadecimal digit to use in element names (default: randomly generated).
  • int dirq_get_rndhex (dirq_t dirq) — gets the random hexadecimal digit to use in element names.
  • void dirq_set_umask (dirq_t dirq, mode_t value) — sets the umask to use when creating files and directories (default: use the running process' umask).
  • mode_t dirq_get_umask (dirq_t dirq) — gets the umask to use when creating files and directories.
  • void dirq_set_maxlock (dirq_t dirq, int value) — sets the maximum time for a locked element in seconds (default 600).
  • int dirq_get_maxlock (dirq_t dirq) — gets the maximum time for a locked element in seconds.
  • void dirq_set_maxtemp (dirq_t dirq, int value) — sets the maximum time for a temporary element in seconds (default 300).
  • int dirq_get_maxtemp (dirq_t dirq) — gets the maximum time for a temporary element in seconds.
  • const char *dirq_first (dirq_t dirq) — returns the first element in the queue, resetting the iterator; returns NULL if the queue is empty or an error occurred.
  • const char *dirq_next (dirq_t dirq) — returns the next element in the queue, incrementing the iterator; returns NULL if there is no next element or an error occurred.
  • const char *dirq_add (dirq_t dirq, dirq_iow cb) — adds the given data (via callback) to the queue and returns the corresponding element name or NULL on error.
  • const char *dirq_add_path (dirq_t dirq, const char *path) — adds the given file (identified by its path) to the queue and returns the corresponding element name or NULL on error, the file must be on the same filesystem and will be moved to the queue.
  • int dirq_get (dirq_t dirq, const char *name, dirq_ior cb) — gets the data from the given element (which must be locked) via callback; returns 0 on success, -1 on error.
  • const char *dirq_get_path (dirq_t dirq, const char *name) — gets the file path of the given element (which must be locked), this file can be read but not removed, you must use the remove() method for this; if the given name is NULL, returns the directory queue path itself.
  • int dirq_lock (dirq_t dirq, const char *name, int permissive) — attempts to lock the given element and returns 0 on success; in case of error, returns 1 (permissive) or -1 (non permissive).
  • int dirq_unlock (dirq_t dirq, const char *name, int permissive) — attempts to unlock the given element and returns 0 on success; in case of error, returns 1 (permissive) or -1 (non permissive).
  • int dirq_remove (dirq_t dirq, const char *name) — removes the given element (which must be locked) from the queue; returns 0 on success, -1 on error.
  • int dirq_touch (dirq_t dirq, const char *name) — "touches" the given element (i.e. updates the access and modification times to the current time); returns 0 on success, -1 on error.
  • int dirq_get_size (dirq_t dirq, const char *name) — returns the size (in bytes) of the given element or -1 on error.
  • int dirq_count (dirq_t dirq) — returns the number of elements in the queue or -1 on error; this also resets the iterator.
  • int dirq_purge (dirq_t dirq) — purges the queue by removing unused intermediate directories, removing too old temporary elements and unlocking too old locked elements (aka staled locks); this is using the "maxlock" and "maxtemp" attributes of the directory queue object; returns the number of elements purged or -1 on error; this also resets the iterator.
  • void dirq_now (dirq_t dirq, struct timespec *ts) — returns the current time in the given "timespec" structure.
  • int dirq_get_errcode (dirq_t dirq) — returns the current error code (usually "errno") or 0 if there is no error.
  • const char *dirq_get_errstr (dirq_t dirq) — returns the current error string or NULL if there is no error.
  • void dirq_clear_error (dirq_t dirq) — clears the current error.

🎀 AUTHOR 🎀

Lionel Cons <http://cern.ch/lionel.cons> — icon behavior. 👑

Copyright (C) CERN 2012-2017

okay that’s the whole page bestie — this lil manual is from dirq 0.5, dated 2017-08-04, so go forth and queue cutely. mwah 💅✨🦋

📦 where this page comes from