DMOVER(9)

📦 version 11.0, in nixpkgs unstable

💅 DMOVER(9) — the hardware-assisted data mover interface, explained by your favorite kernel bestie

ok so this is the dmover page from the Kernel Developer's Manual, and honestly? it's giving "the CPU is too busy, let the hardware do it" main character energy. buckle up 🎀

✨ NAME ✨

dmover_backend_register, dmover_backend_unregister, dmover_session_create, dmover_session_destroy, dmover_request_alloc, dmover_request_free, dmover_process, dmover_done — the hardware-assisted data mover interface. that's the whole squad, bestie 💖

🌸 SYNOPSIS 🌸

first things first, you gotta include this:

#include <dev/dmover/dmovervar.h>

🫶 client interface routines

int
dmover_session_create(const char *, struct dmover_session **);

void
dmover_session_destroy(struct dmover_session *);

struct dmover_request *
dmover_request_alloc(struct dmover_session *, dmover_buffer *);

void
dmover_request_free(struct dmover_request *);

void
dmover_process(struct dmover_request *);

🦋 back-end interface routines

void
dmover_backend_register(struct dmover_backend *);

void
dmover_backend_unregister(struct dmover_backend *);

void
dmover_done(struct dmover_request *);

💜 DESCRIPTION 💜

okay so the dmover facility is literally an interface to hardware-assisted data movers. like, actual hardware whose whole job is moving data around so the CPU doesn't have to. iconic behavior, no cap.

what can it do? glad you asked:

  • copy data from one location in memory to another
  • clear a region of memory
  • fill a region of memory with a pattern
  • do simple operations on multiple regions of memory — an XOR, for example — all without intervention by the CPU 🔥

the drivers for these hardware-assisted data movers introduce themselves to dmover by registering their capabilities. that's the vibe check. 📝

when a client wants to use a dmover function, it creates a session for that function, and that session identifies which back-ends are actually capable of performing it. then the client enqueues requests on that session, and the back-ends process them asynchronously. the client can either block until the request is done, or have a call-back invoked once the request completes. choose your fighter 💅

when a client creates a session, dmover goes and finds the back-ends capable of handling the requested function. then, when a request is scheduled for processing, the dmover scheduler picks the best back-end to process it from the list of candidates — trying to load balance while also weighing up the relative performance of each back-end. it's giving talent show judging panel ✨

here's the structural tea ☕ — a dmover function always has one output region. a function may have zero or more input regions, or it may use an immediate value as an input. for functions that use input regions, the length of every input region and the output region must be the same. all dmover functions with the same name will have the same number of inputs and the same type of inputs.

⚠️ if a back-end tries to register a function that violates this invariant, behavior is undefined. like, we're not mad, we're just disappointed. don't do it. 💀

the dmover facility supports several types of buffer descriptors. for functions that use input regions, each input buffer descriptor and the output buffer descriptor must be of the same type. that restriction may be removed in a future revision of the interface (so like, watch this space 👀).

the dmover facility may need to interrupt request processing and restart it. clients of dmover should be careful to avoid unwanted side-effects if that happens. specifically — for functions that use input regions, no input region may overlap with the output region. that's a hard no, bestie 🚫

🧚 DATA STRUCTURES 🧚

dmover shares a few data structures between the client and the back-end so they can describe sessions and requests. sharing is caring 💖

typedef enum {
        DMOVER_BUF_LINEAR,
        DMOVER_BUF_UIO
} dmover_buffer_type;

typedef struct {
        void *l_addr;
        size_t l_len;
} dmover_buf_linear;

typedef union {
        dmover_buf_linear dmbuf_linear;
        struct uio *dmbuf_uio;
} dmover_buffer;

together, these data types describe the buffer data structures that the dmover facility understands. and yes — additional buffer types may be added in future revisions of the dmover interface. the lore keeps expanding 📚

the dmover_assignment structure holds the info about the back-end that a request is currently assigned to. its public members are:

  • struct dmover_backend *das_backend — a pointer to the back-end.
  • const struct dmover_algdesc *das_algdesc — a pointer to the algorithm description that the back-end provided for the request's function.

the dmover_session structure has these public members:

  • void *dses_cookie — a pointer to client private data.
  • int dses_ninputs — the number of inputs used by the selected function.

and the dmover_request structure has these public members — this is the big one, stay with me 🫶

  • TAILQ_ENTRY(dmover_request) dreq_dmbq — linkage on the back-end's queue of pending requests.
  • struct dmover_session *dreq_session — pointer to the session this request is associated with. this one is intended for use by the back-end.
  • struct dmover_assignment *dreq_assignment — pointer to the dmover_assignment structure describing the back-end the request is currently assigned to. the back-end gets assigned when the request is scheduled with dmover_process().
  • void (*dreq_callback)(struct dmover_request *) — pointer to an optional call-back function from the client. if you provide it, it gets invoked when the request is complete. this field must be NULL if DMOVER_REQ_WAIT is set in dreq_flags.
  • void *dreq_cookie — pointer to client private data that's specific to the request.
  • void *dreq_dmbcookie — pointer to back-end private data, for use while the back-end is actively processing a request.
  • volatile int dreq_flags — the flags, and here's what they mean:
    • DMOVER_REQ_DONE — the request has been completed. if you're not using a call-back, the client can poll this bit to figure out whether a request has been processed.
    • DMOVER_REQ_ERROR — an error occurred while processing the request.
    • DMOVER_REQ_RUNNING — the request is currently being executed by the back-end. once a command is running it cannot be cancelled and must run to completion. no take-backsies 😭
    • DMOVER_REQ_WAIT — if the client sets this, dmover_process() will wait for the request to complete using cv_wait(9). this flag may only be used if the caller has a valid thread context. and if this flag is set, a callback may not be used.
  • int dreq_error — if the DMOVER_REQ_ERROR bit is set, this holds the errno(2) value indicating the error that happened during processing.
  • dmover_buffer_type dreq_outbuf_type — the type of the output buffer.
  • dmover_buffer dreq_outbuf — the output buffer itself.
  • uint8_t dreq_immediate[8] — this is the input for algorithms that use an immediate value. values smaller than 8 bytes should use the least-significant bytes first. for example, a 32-bit integer would occupy bytes 0, 1, 2, and 3. put it in the right order, babe 🧮
  • dmover_buffer_type dreq_inbuf_type — the type of the input buffer. only used if the dmover function has one or more inputs.
  • dmover_buffer *dreq_inbuf — a pointer to an array of input buffers. again, only used if the dmover function has one or more inputs. the number of inputs — and therefore the number of valid elements in the array — is specified by the algorithm description for the session.

👑 CLIENT INTERFACE 👑

these are the functions you get as a client. slay, enjoy 💅

dmover_session_create(function, sessionp)

dmover_session_create() creates a data mover session for the specified data movement function, function. a handle to the brand-new session is returned in sessionp. new session, who dis ✨

and these are the valid data movement function names, bestie:

  • “zero” — fill a memory region with zeros. this algorithm has an input count of 0.
  • “fill8” — fill a memory region with an 8-bit pattern. this algorithm has an input count of 0. the pattern is provided in the dreq_imm8 member of the dmover_request structure.
  • “copy” — copy a memory region from one location to another. this algorithm has an input count of 1.
  • “xor2” — perform an XOR operation on 2 inputs. this algorithm has an input count of 2.
  • “xor3” — perform an XOR operation on 3 inputs. this algorithm has an input count of 3.
  • “xor4” — perform an XOR operation on 4 inputs. this algorithm has an input count of 4.
  • “xor5” — perform an XOR operation on 5 inputs. this algorithm has an input count of 5.
  • “xor6” — perform an XOR operation on 6 inputs. this algorithm has an input count of 6.
  • “xor7” — perform an XOR operation on 7 inputs. this algorithm has an input count of 7.
  • “xor8” — perform an XOR operation on 8 inputs. this algorithm has an input count of 8.

users of the dmover facility are encouraged to use the following aliases for the well-known function names, because doing so saves space and reduces the chance of programming errors. less typing, fewer typos, everyone wins 🎀

  • DMOVER_FUNC_ZERO — “zero” (dmover_funcname_zero)
  • DMOVER_FUNC_FILL8 — “fill8” (dmover_funcname_fill8)
  • DMOVER_FUNC_COPY — “copy” (dmover_funcname_copy)
  • DMOVER_FUNC_XOR2 — “xor2” (dmover_funcname_xor2)
  • DMOVER_FUNC_XOR3 — “xor3” (dmover_funcname_xor3)
  • DMOVER_FUNC_XOR4 — “xor4” (dmover_funcname_xor4)
  • DMOVER_FUNC_XOR5 — “xor5” (dmover_funcname_xor5)
  • DMOVER_FUNC_XOR6 — “xor6” (dmover_funcname_xor6)
  • DMOVER_FUNC_XOR7 — “xor7” (dmover_funcname_xor7)
  • DMOVER_FUNC_XOR8 — “xor8” (dmover_funcname_xor8)

dmover_session_destroy(session)

dmover_session_destroy() tears down a data mover session and releases all resources associated with it. it's giving closure, but make it memory management 🦋

dmover_request_alloc(session, inbuf)

dmover_request_alloc() allocates a dmover request structure and associates it with the specified session. if the inbuf argument is not NULL, then inbuf is used as the array of input buffer descriptors in the request. otherwise — if inbuf is NULL and the dmover function requires input buffers — the input buffer descriptors get allocated automatically using malloc(9). convenient, honestly 🫶

If the request structure or input buffer descriptors can't be allocated, dmover_request_alloc() just returns NULL — that's the fail signal, bestie. 💀

  • dmover_request_free(req) — frees a dmover request structure. If the dmover function needs input buffers and the input buffer descriptors tied to req were allocated by dmover_request_alloc(), those input buffer descriptors get freed too. ✨
  • dmover_process(req) — submits the dmover request req for processing. The call-back specified by the request is invoked when processing is complete. 🎀
The dmover_session_create() and dmover_session_destroy() functions must not be called from interrupt context. 🚫

The dmover_request_alloc(), dmover_request_free(), and dmover_process() functions may be called from interrupt handlers at levels IPL_VM, IPL_SOFTCLOCK, and IPL_SOFTNET, or in non-interrupt context. 👑

The request completion call-back is called from a software interrupt handler at IPL_SOFTCLOCK. ⏰

🛠️ BACK-END INTERFACE 🛠️

A back-end describes the dmover functions it can perform using an array of dmover_algdesc structures:

struct dmover_algdesc {
        const char *dad_name;   /* algorithm name */
        void *dad_data;         /* opaque algorithm description */
        int dad_ninputs;        /* number of inputs */
};

The dad_name member points to a valid dmover function name which the client may specify. The dad_data member points to a back-end-specific description of the algorithm. 🧩

A back-end presents itself to the dmover facility using the dmover_backend structure. The back-end must initialize the following members of the structure:

  • const char *dmb_name — this is the name of the back-end. 🏷️
  • u_int dmb_speed — this is an estimate of the number of kilobytes/second that the back-end can process. ⚡
  • void *dmb_cookie — this is a pointer to back-end private data. 🍪
  • const struct dmover_algdesc *dmb_algdescs — this points to an array of dmover_algdesc structures which describe the functions the data mover can perform. 🗂️
  • int dmb_nalgdescs — this is the number of elements in the dmb_algdescs array. 🔢
  • void (*dmb_process)(struct dmover_backend *) — this is the entry point to the back-end used to process requests. 🚀

When invoked by the dmover facility, the back-end's (*dmb_process)() function should examine the pending request queue in its dmover_backend structure:

  • TAILQ_HEAD(, dmover_request) dmb_pendreqs — this is the queue of pending requests. 📋
  • int dmb_npendreqs — this is the number of requests in the dmb_pendreqs queue. 🧮

If an error occurs when processing the request, the DMOVER_REQ_ERROR bit must be set in the dreq_flags member of the request, and the dreq_error member set to an errno(2) value to indicate the error. ⚠️

When the back-end has finished processing the request, it must call the dmover_done() function. This function eventually invokes the client's call-back routine. 📞

If a hardware-assisted data mover uses interrupts, the interrupt handlers should be registered at IPL_VM. 🖥️

The following functions are provided to the back-ends:

  • dmover_backend_register(backend) — the dmover_backend_register() function registers the back-end backend with the dmover facility. ✅
  • dmover_backend_unregister(backend) — the dmover_backend_unregister() function removes the back-end backend from the dmover facility. The back-end must already be registered. ❌
  • dmover_done(req) — the dmover_done() function is called by the back-end when it has finished processing a request, whether the request completed successfully or not. 🏁

The dmover_backend_register() and dmover_backend_unregister() functions must not be called from interrupt context. 🙅‍♀️

The dmover_done() function may be called at IPL_VM, IPL_SOFTCLOCK, IPL_SOFTNET, or in non-interrupt context. 🌈

🌟 EXAMPLES 🌟

The following is an example of a client using dmover to zero-fill a region of memory. In this example, the CPU will be able to context switch to another thread and perform work while the hardware-assisted data mover clears the specified block of memory. 🧠✨

int
hw_bzero(void *buf, size_t len)
{
        struct dmover_session *dses;
        struct dmover_request *dreq;
        int error;

        error = dmover_session_create(DMOVER_FUNC_ZERO, &dses);
        if (error)
                return (error);

        dreq = dmover_request_alloc(dses, NULL);
        if (dreq == NULL) {
                dmover_session_destroy(dses);
                return (ENOMEM);
        }

        dreq->dreq_flags = DMOVER_REQ_WAIT;
        dreq->dreq_callback = NULL;
        dreq->dreq_outbuf.dreq_outbuf_type = DMOVER_BUF_LINEAR;
        dreq->dreq_outbuf.dmbuf_linear.l_addr = buf;
        dreq->dreq_outbuf.dmbuf_linear.l_len = len;

        dmover_process(dreq);

        error = (dreq->dreq_flags & DMOVER_REQ_ERROR) ?
            dreq->dreq_error : 0;

        dmover_request_free(dreq);
        dmover_session_destroy(dses);

        return (error);
}

👀 SEE ALSO 👀

queue(3), dmoverio(4)

🕰️ HISTORY 🕰️

The dmover facility first appeared in NetBSD 2.0. 🎉

👑 AUTHORS 👑

The dmover facility was designed and implemented by Jason R. Thorpe ⟨thorpej@wasabisystems.com⟩ and contributed by Wasabi Systems, Inc. 💖

💀 BUGS 💀

The mechanism by which a back-end should advertise its performance to the request scheduler is not well-defined. Therefore, the load-balancing mechanism within the request scheduler is also not well-defined. 😭 It's giving "we'll figure it out later" energy, ngl.

This iconic little page is from Debian, dated December 4, 2007, and it's the DMOVER(9) manual page. Periodt. 💅🫶

📦 where this page comes from