IBV_POLL_CQ(3)

💖 IBV_POLL_CQ(3) 💖

okay bestie, welcome to the Libibverbs Programmer's Manual ✨ this one's the manual page for ibv_poll_cq, and honestly? it's giving "the main character of your verbs code" because literally nothing happens until you poll. let's get into it 🫶

🌸 NAME 🌸

ibv_poll_cq — poll a completion queue (CQ)

✨ SYNOPSIS ✨

#include <infiniband/verbs.h>

int ibv_poll_cq(struct ibv_cq *cq, int num_entries,
                struct ibv_wc *wc);

🦋 DESCRIPTION 🦋

so ibv_poll_cq() polls the CQ cq for work completions and hands you back the first num_entries of them (or, if the CQ has fewer than that many sitting there, it just gives you all of the available completions — no drama, no waiting around for more) in the array wc. the argument wc is a pointer to an array of ibv_wc structs, as defined in <infiniband/verbs.h>. iykyk 💅

and here's the iconic struct itself, verbatim, every field, no edits:

struct ibv_wc {
        uint64_t                wr_id;          /* ID of the completed Work Request (WR) */
        enum ibv_wc_status      status;         /* Status of the operation */
        enum ibv_wc_opcode      opcode;         /* Operation type specified in the completed WR */
        uint32_t                vendor_err;     /* Vendor error syndrome */
        uint32_t                byte_len;       /* Number of bytes transferred */
        union {
                __be32                  imm_data;         /* Immediate data (in network byte order) */
                uint32_t                invalidated_rkey; /* Local RKey that was invalidated */
        };
        uint32_t                qp_num;         /* Local QP number of completed WR */
        uint32_t                src_qp;         /* Source QP number (remote QP number) of completed WR (valid only for UD QPs) */
        unsigned int            wc_flags;       /* Flags of the completed WR */
        uint16_t                pkey_index;     /* P_Key index (valid only for GSI QPs) */
        uint16_t                slid;           /* Source LID */
        uint8_t                 sl;             /* Service Level */
        uint8_t                 dlid_path_bits; /* DLID path bits (not applicable for multicast messages) */
};

🎀 the wc_flags situation 🎀

the attribute wc_flags describes the properties of the work completion. it's either 0 or the bitwise OR of one or more of these flags, periodt:

• IBV_WC_GRH — GRH is present (valid only for UD QPs)
• IBV_WC_WITH_IMM — Immediate data value is valid
• IBV_WC_WITH_INV — Invalidated RKey data value is valid (cannot be combined with IBV_WC_WITH_IMM)
• IBV_WC_IP_CSUM_OK — TCP/UDP checksum over IPv4 and IPv4 header checksum are verified. Valid only when device_cap_flags in device_attr indicates current QP is supported by checksum offload.

😭 the important caveat 😭

not all wc attributes are always valid, bestie, and this part matters. if the completion status is anything other than IBV_WC_SUCCESS, then only these attributes are valid: wr_id, status, qp_num, and vendor_err. that's it. don't go reading the other fields, they're not giving what you think they're giving 🙈

👑 RETURN VALUE 👑

on success, ibv_poll_cq() returns a non-negative value equal to the number of completions found. on failure, a negative value is returned. slay.

💀 NOTES 💀

each polled completion is removed from the CQ and cannot be returned to it. it's gone. it left the chat. 🫡

the user should consume work completions at a rate that prevents CQ overrun from occurrence. in case of a CQ overrun, the async event IBV_EVENT_CQ_ERR will be triggered, and the CQ cannot be used. yeah. that's a whole thing. keep up with your polls, fr fr.

IBV_WC_DRIVER1 will be reported as a response to IBV_WR_DRIVER1 opcode; IBV_WC_DRIVER2/IBV_WC_DRIVER3 will be reported on specific driver operations. ✨

🌈 SEE ALSO 🌈

• ibv_post_send(3)
• ibv_post_recv(3)

💜 AUTHORS 💜

Dotan Barak <dotanba@gmail.com>

okay that's the whole page, you ate that 🍓 this was the libibverbs manual page, dated 2006-10-31, and yes she's been iconic since then. go poll your CQ and hydrate. bye bestie 💅✨