STRUCT IB_DEVICE_OPS(9)

πŸ“¦ version 7.3.1, in nixpkgs unstable

okay so bestie, welcome to struct ib_device_ops(9) πŸ’… straight out of the Kernel API Manual, and ngl this one is literally the blueprint for InfiniBand device operations ✨ section 9 energy, kernel-dev main character vibes, we love to see it 🫢


πŸŽ€ NAME πŸŽ€

struct ib_device_ops β€” InfiniBand device operations πŸ‘‘

this structure defines all the InfiniBand device operations, fr fr. providers are gonna need to define the supported operations themselves β€” the ones they actually support, periodt β€” otherwise they get set to null πŸ’€ no cap, that's the whole vibe check.

so like, if a provider doesn't fill in an operation, it's not "undefined" or "maybe" β€” it is literally null, bestie πŸ™ˆ keep that in mind.

to be continued... πŸ¦‹

✨ SYNOPSIS ✨

okay bestie, deep breath. this is the big one. struct ib_device_ops is literally the whole entire wardrobe of a driver β€” every single little hook, laid out in one giant C struct, and the SYNOPSIS is basically the packing list. no cap, it's giving "moved into a new apartment and every single box is labelled." πŸ’…

it's a lot. like a lot a lot. so we're gonna take it in cute little clusters, and every field stays exactly where it lives. let's go 🫢


the "who am i" starter pack πŸͺͺ

struct ib_device_ops {
    struct module *owner;
    enum rdma_driver_id driver_id;
    u32 uverbs_abi_ver;
    unsigned int uverbs_no_driver_id_binding:1;
    unsigned int uverbs_robust_udata:1;
    const struct attribute_group *device_group;
    const struct attribute_group **port_groups;

see? chill start. owner, driver_id, uverbs_abi_ver whatever, and then two bitfields β€” uverbs_no_driver_id_binding:1 and uverbs_robust_udata:1 β€” each exactly 1 bit wide because we're being efficient with our bits, slay. then the sysfs gang shows up: device_group, and port_groups which is a pointer to pointer to const struct attribute_group because of course it is πŸ’€


the fast-path verbs, aka the main characters πŸ”₯

    int (*post_send)(struct ib_qp *qp, const struct ib_send_wr *send_wr, const struct ib_send_wr **bad_send_wr);
    int (*post_recv)(struct ib_qp *qp, const struct ib_recv_wr *recv_wr, const struct ib_recv_wr **bad_recv_wr);
    void (*drain_rq)(struct ib_qp *qp);
    void (*drain_sq)(struct ib_qp *qp);
    int (*poll_cq)(struct ib_cq *cq, int num_entries, struct ib_wc *wc);
    int (*peek_cq)(struct ib_cq *cq, int wc_cnt);
    int (*req_notify_cq)(struct ib_cq *cq, enum ib_cq_notify_flags flags);
    int (*post_srq_recv)(struct ib_srq *srq, const struct ib_recv_wr *recv_wr, const struct ib_recv_wr **bad_recv_wr);

the way that post_send and post_recv both take a **bad_send_wr / **bad_recv_wr double pointer 😭 that's the "here's where you tell me what went wrong" energy. drain_rq and drain_sq return void β€” no notes, they just do it. poll_cq takes a num_entries and a struct ib_wc *wc, peek_cq takes a wc_cnt, req_notify_cq takes enum ib_cq_notify_flags flags. obsessed. ✨


device + port stuff (the paperwork era) πŸ“‹

    int (*process_mad)(struct ib_device *device, int process_mad_flags, u32 port_num, const struct ib_wc *in_wc, const struct ib_grh *in_grh, const struct ib_mad *in_mad, struct ib_mad *out_mad, size_t *out_mad_size, u16 *out_mad_pkey_index);
    int (*query_device)(struct ib_device *device, struct ib_device_attr *device_attr, struct ib_udata *udata);
    int (*modify_device)(struct ib_device *device, int device_modify_mask, struct ib_device_modify *device_modify);
    void (*get_dev_fw_str)(struct ib_device *device, char *str);
    int (*query_port)(struct ib_device *device, u32 port_num, struct ib_port_attr *port_attr);
    int (*query_port_speed)(struct ib_device *device, u32 port_num, u64 *speed);
    int (*modify_port)(struct ib_device *device, u32 port_num, int port_modify_mask, struct ib_port_modify *port_modify);
    int (*get_port_immutable)(struct ib_device *device, u32 port_num, struct ib_port_immutable *immutable);
    enum rdma_link_layer (*get_link_layer)(struct ib_device *device, u32 port_num);
    struct net_device *(*get_netdev)(struct ib_device *device, u32 port_num);
    struct net_device *(*alloc_rdma_netdev)( struct ib_device *device, u32 port_num, enum rdma_netdev_t type, const char *name, unsigned char name_assign_type, void (*setup)(struct net_device *));
    int (*rdma_netdev_get_params)(struct ib_device *device, u32 port_num, enum rdma_netdev_t type, struct rdma_netdev_alloc_params *params);
    int (*query_gid)(struct ib_device *device, u32 port_num, int index, union ib_gid *gid);
    int (*add_gid)(const struct ib_gid_attr *attr, void **context);
    int (*del_gid)(const struct ib_gid_attr *attr, void **context);
    int (*query_pkey)(struct ib_device *device, u32 port_num, u16 index, u16 *pkey);

bestie look at process_mad. nine parameters. nine. in_wc, in_grh, in_mad, out_mad, out_mad_size, out_mad_pkey_index β€” it's giving "i have a whole filing cabinet and you're gonna hear about it." iconic honestly.

and pookie, note that alloc_rdma_netdev has a lil space in there: (*alloc_rdma_netdev)( struct ib_device *device, ... β€” yes that's real, yes we're keeping it exactly, don't touch it πŸ™ˆ it also takes a void (*setup)(struct net_device *) function pointer inside a function pointer field. function-pointer-inception, we love to see it πŸ’€

add_gid and del_gid both take a void **context β€” the ✨mystery box✨ of RDMA. query_port_speed takes a u64 *speed. get_link_layer is the only one returning enum rdma_link_layer so give her a lil crown πŸ‘‘


ucontext, mmap, PD, AH, SRQ πŸŽ€

    int (*alloc_ucontext)(struct ib_ucontext *context, struct ib_udata *udata);
    void (*dealloc_ucontext)(struct ib_ucontext *context);
    int (*mmap)(struct ib_ucontext *context, struct vm_area_struct *vma);
    void (*mmap_free)(struct rdma_user_mmap_entry *entry);
    int (*mmap_get_pfns)(struct rdma_user_mmap_entry *entry, struct phys_vec *phys_vec, struct p2pdma_provider **provider);
    struct rdma_user_mmap_entry *(*pgoff_to_mmap_entry)(struct ib_ucontext *ucontext, off_t pg_off);
    void (*disassociate_ucontext)(struct ib_ucontext *ibcontext);
    int (*alloc_pd)(struct ib_pd *pd, struct ib_udata *udata);
    int (*dealloc_pd)(struct ib_pd *pd, struct ib_udata *udata);
    int (*create_ah)(struct ib_ah *ah, struct rdma_ah_init_attr *attr, struct ib_udata *udata);
    int (*create_user_ah)(struct ib_ah *ah, struct rdma_ah_init_attr *attr, struct ib_udata *udata);
    int (*modify_ah)(struct ib_ah *ah, struct rdma_ah_attr *ah_attr);
    int (*query_ah)(struct ib_ah *ah, struct rdma_ah_attr *ah_attr);
    int (*destroy_ah)(struct ib_ah *ah, u32 flags);
    int (*create_srq)(struct ib_srq *srq, struct ib_srq_init_attr *srq_init_attr, struct ib_udata *udata);
    int (*modify_srq)(struct ib_srq *srq, struct ib_srq_attr *srq_attr, enum ib_srq_attr_mask srq_attr_mask, struct ib_udata *udata);
    int (*query_srq)(struct ib_srq *srq, struct ib_srq_attr *srq_attr);
    int (*destroy_srq)(struct ib_srq *srq, struct ib_udata *udata);

the little mmap mini-arc is so funny to me: mmap, then mmap_free, then mmap_get_pfns with a struct p2pdma_provider **provider, and pgoff_to_mmap_entry returning struct rdma_user_mmap_entry * from an off_t pg_off. it's a whole storyline. 🌸

also create_ah and create_user_ah have literally the exact same signature β€” twins, not identical twins, twin emojis πŸ‘―β€β™€οΈ and destroy_ah is the only AH one with a u32 flags, main character behavior tbh.


QP + CQ + completion counters πŸ¦‹

    int (*create_qp)(struct ib_qp *qp, struct ib_qp_init_attr *qp_init_attr, struct ib_udata *udata);
    int (*modify_qp)(struct ib_qp *qp, struct ib_qp_attr *qp_attr, int qp_attr_mask, struct ib_udata *udata);
    int (*qp_attach_comp_cntr)(struct ib_qp *qp, struct ib_comp_cntr *cc, struct ib_qp_attach_comp_cntr_attr *attr);
    int (*query_qp)(struct ib_qp *qp, struct ib_qp_attr *qp_attr, int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr);
    int (*destroy_qp)(struct ib_qp *qp, struct ib_udata *udata);
    int (*create_cq)(struct ib_cq *cq, const struct ib_cq_init_attr *attr, struct uverbs_attr_bundle *attrs);
    int (*create_user_cq)(struct ib_cq *cq, const struct ib_cq_init_attr *attr, struct uverbs_attr_bundle *attrs);
    int (*modify_cq)(struct ib_cq *cq, u16 cq_count, u16 cq_period);
    int (*destroy_cq)(struct ib_cq *cq, struct ib_udata *udata);
    int (*resize_user_cq)(struct ib_cq *cq, unsigned int cqe, struct ib_udata *udata);
    int (*pre_destroy_cq)(struct ib_cq *cq);
    void (*post_destroy_cq)(struct ib_cq *cq);
    int (*create_comp_cntr)(struct ib_comp_cntr *cc, struct uverbs_attr_bundle *attrs);
    int (*destroy_comp_cntr)(struct ib_comp_cntr *cc);
    int (*modify_comp_cntr)(struct ib_comp_cntr *cc, enum ib_comp_cntr_entry entry, enum ib_comp_cntr_modify_op op, u64 value);
    int (*read_comp_cntr)(struct ib_comp_cntr *cc, enum ib_comp_cntr_entry entry, u64 *value);
    int (*query_comp_cntr_caps)(struct ib_device *dev, struct ib_comp_cntr_caps *caps, struct uverbs_attr_bundle *attrs);

the pre_destroy_cq / post_destroy_cq duo β€” one returns int, one returns void, and together they're the "before and after" pic collage of CQ destruction πŸ’… also notice destroy_cq takes a struct ib_udata *udata but destroy_comp_cntr takes just the cc. no notes, just noticing. πŸ™ˆ

modify_comp_cntr is carrying enum ib_comp_cntr_entry entry, enum ib_comp_cntr_modify_op op, and u64 value β€” three-argument queen, and read_comp_cntr matches her with a u64 *value. matched set, we stan 🫢


the memory region extended universe πŸ’œ

    struct ib_mr *(*get_dma_mr)(struct ib_pd *pd, int mr_access_flags);
    struct ib_mr *(*reg_user_mr)(struct ib_pd *pd, u64 start, u64 length, u64 virt_addr, int mr_access_flags, struct ib_dmah *dmah, struct ib_udata *udata);
    struct ib_mr *(*reg_user_mr_dmabuf)(struct ib_pd *pd, u64 offset, u64 length, u64 virt_addr, int fd, int mr_access_flags, struct ib_dmah *dmah, struct uverbs_attr_bundle *attrs);
    struct ib_mr *(*rereg_user_mr)(struct ib_mr *mr, int flags, u64 start, u64 length, u64 virt_addr, int mr_access_flags, struct ib_pd *pd, struct ib_udata *udata);
    int (*dereg_mr)(struct ib_mr *mr, struct ib_udata *udata);
    struct ib_mr *(*alloc_mr)(struct ib_pd *pd, enum ib_mr_type mr_type, u32 max_num_sg);
    struct ib_mr *(*alloc_mr_integrity)(struct ib_pd *pd, u32 max_num_data_sg, u32 max_num_meta_sg);
    int (*advise_mr)(struct ib_pd *pd, enum ib_uverbs_advise_mr_advice advice, u32 flags, struct ib_sge *sg_list, u32 num_sge, struct uverbs_attr_bundle *attrs);
    int (*map_mr_sg)(struct ib_mr *mr, struct scatterlist *sg, int sg_nents, unsigned int *sg_offset);
    int (*check_mr_status)(struct ib_mr *mr, u32 check_mask, struct ib_mr_status *mr_status);
    int (*alloc_mw)(struct ib_mw *mw, struct ib_udata *udata);
    int (*dealloc_mw)(struct ib_mw *mw);
    int (*attach_mcast)(struct ib_qp *qp, union ib_gid *gid, u16 lid);
    int (*detach_mcast)(struct ib_qp *qp, union ib_gid *gid, u16 lid);
    int (*alloc_xrcd)(struct ib_xrcd *xrcd, struct ib_udata *udata);
    int (*dealloc_xrcd)(struct ib_xrcd *xrcd, struct ib_udata *udata);
    struct ib_flow *(*create_flow)(struct ib_qp *qp, struct ib_flow_attr *flow_attr, struct ib_udata *udata);
    int (*destroy_flow)(struct ib_flow *flow_id);
    int (*destroy_flow_action)(struct ib_flow_action *action);

the _mr family is SO big, it's giving "one (1) very popular friend group." we've got get_dma_mr, reg_user_mr, reg_user_mr_dmabuf (that one takes an int fd β€” a file descriptor in a kernel struct, spicy 🌢️), rereg_user_mr, dereg_mr, alloc_mr, alloc_mr_integrity. seven of them. seven!! 😭

attach_mcast and detach_mcast are a matching set taking union ib_gid *gid and u16 lid, and alloc_xrcd / dealloc_xrcd are the "i'm only here for the alloc/dealloc symmetry" icons ✨


SR-IOV, WQ, DM, counters, hw stats, res, driver πŸ‘‘

    int (*set_vf_link_state)(struct ib_device *device, int vf, u32 port, int state);
    int (*get_vf_config)(struct ib_device *device, int vf, u32 port, struct ifla_vf_info *ivf);
    int (*get_vf_stats)(struct ib_device *device, int vf, u32 port, struct ifla_vf_stats *stats);
    int (*get_vf_guid)(struct ib_device *device, int vf, u32 port, struct ifla_vf_guid *node_guid, struct ifla_vf_guid *port_guid);
    int (*set_vf_guid)(struct ib_device *device, int vf, u32 port, u64 guid, int type);
    struct ib_wq *(*create_wq)(struct ib_pd *pd, struct ib_wq_init_attr *init_attr, struct ib_udata *udata);
    int (*destroy_wq)(struct ib_wq *wq, struct ib_udata *udata);
    int (*modify_wq)(struct ib_wq *wq, struct ib_wq_attr *attr, u32 wq_attr_mask, struct ib_udata *udata);
    int (*create_rwq_ind_table)(struct ib_rwq_ind_table *ib_rwq_ind_table, struct ib_rwq_ind_table_init_attr *init_attr, struct ib_udata *udata);
    int (*destroy_rwq_ind_table)(struct ib_rwq_ind_table *wq_ind_table);
    struct ib_dm *(*alloc_dm)(struct ib_device *device, struct ib_ucontext *context, struct ib_dm_alloc_attr *attr, struct uverbs_attr_bundle *attrs);
    int (*dealloc_dm)(struct ib_dm *dm, struct uverbs_attr_bundle *attrs);
    int (*alloc_dmah)(struct ib_dmah *ibdmah, struct uverbs_attr_bundle *attrs);
    int (*dealloc_dmah)(struct ib_dmah *dmah, struct uverbs_attr_bundle *attrs);
    struct ib_mr *(*reg_dm_mr)(struct ib_pd *pd, struct ib_dm *dm, struct ib_dm_mr_attr *attr, struct uverbs_attr_bundle *attrs);
    int (*create_counters)(struct ib_counters *counters, struct uverbs_attr_bundle *attrs);
    int (*destroy_counters)(struct ib_counters *counters);
    int (*read_counters)(struct ib_counters *counters, struct ib_counters_read_attr *counters_read_attr, struct uverbs_attr_bundle *attrs);
    int (*map_mr_sg_pi)(struct ib_mr *mr, struct scatterlist *data_sg, int data_sg_nents, unsigned int *data_sg_offset, struct scatterlist *meta_sg, int meta_sg_nents, unsigned int *meta_sg_offset);
    struct rdma_hw_stats *(*alloc_hw_device_stats)(struct ib_device *device);
    struct rdma_hw_stats *(*alloc_hw_port_stats)(struct ib_device *device, u32 port_num);
    int (*get_hw_stats)(struct ib_device *device, struct rdma_hw_stats *stats, u32 port, int index);
    int (*modify_hw_stat)(struct ib_device *device, u32 port, unsigned int counter_index, bool enable);
    int (*fill_res_mr_entry)(struct sk_buff *msg, struct ib_mr *ibmr);
    int (*fill_res_mr_entry_raw)(struct sk_buff *msg, struct ib_mr *ibmr);
    int (*fill_res_cq_entry)(struct sk_buff *msg, struct ib_cq *ibcq);
    int (*fill_res_cq_entry_raw)(struct sk_buff *msg, struct ib_cq *ibcq);
    int (*fill_res_qp_entry)(struct sk_buff *msg, struct ib_qp *ibqp);
    int (*fill_res_qp_entry_raw)(struct sk_buff *msg, struct ib_qp *ibqp);
    int (*fill_res_cm_id_entry)(struct sk_buff *msg, struct rdma_cm_id *id);
    int (*fill_res_srq_entry)(struct sk_buff *msg, struct ib_srq *ib_srq);
    int (*fill_res_srq_entry_raw)(struct sk_buff *msg, struct ib_srq *ib_srq);
    int (*enable_driver)(struct ib_device *dev);
};

okay so the fill_res_* block is my personal favorite because it's structured like a little rhythm: mr / mr_raw, cq / cq_raw, qp / qp_raw, srq / srq_raw... and then fill_res_cm_id_entry just shows up in the middle like the third friend who wasn't in the group photo lineup 😭 no _raw twin for you bestie, we still love you.

modify_hw_stat is the only one rocking a bool enable. map_mr_sg_pi has SEVEN parameters and TWO unsigned int * offsets (data_sg_offset and meta_sg_offset) β€” she's doing the most and honestly? good for her. πŸ”₯

and then, after that whole journey: int (*enable_driver)(struct ib_device *dev); and the closing };. the way that's the last line before the struct just ends πŸ’€ it's giving "and that's the last box, we're done moving, order pizza."


and yeah β€” that's the whole SYNOPSIS. every single field, no shortcuts, no "etc.", nothing skipped. 🫢 if your eyes glazed over around alloc_dmah, that's valid and normal. drink some water. πŸ’–

but bestie we're SO not done β€” there's more of this page coming, and it's got all the actual words. keep scrolling, i got you ✨

void (*dealloc_driver)(struct ib_device *dev);
void (*iw_add_ref)(struct ib_qp *qp);
void (*iw_rem_ref)(struct ib_qp *qp);
struct ib_qp *(*iw_get_qp)(struct ib_device *device, int qpn);
int (*iw_connect)(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param);
int (*iw_accept)(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param);
int (*iw_reject)(struct iw_cm_id *cm_id, const void *pdata, u8 pdata_len);
int (*iw_create_listen)(struct iw_cm_id *cm_id, int backlog);
int (*iw_destroy_listen)(struct iw_cm_id *cm_id);
int (*counter_bind_qp)(struct rdma_counter *counter, struct ib_qp *qp, u32 port);
int (*counter_unbind_qp)(struct ib_qp *qp, u32 port);
int (*counter_dealloc)(struct rdma_counter *counter);
struct rdma_hw_stats *(*counter_alloc_stats)(struct rdma_counter *counter);
int (*counter_update_stats)(struct rdma_counter *counter);
void (*counter_init)(struct rdma_counter *counter);
int (*fill_stat_mr_entry)(struct sk_buff *msg, struct ib_mr *ibmr);
int (*query_ucontext)(struct ib_ucontext *context, struct uverbs_attr_bundle *attrs);
int (*get_numa_node)(struct ib_device *dev);
struct ib_device *(*add_sub_dev)(struct ib_device *parent, enum rdma_nl_dev_type type, const char *name);
void (*del_sub_dev)(struct ib_device *sub_dev);
void (*ufile_hw_cleanup)(struct ib_uverbs_file *ufile);
void (*report_port_event)(struct ib_device *ibdev, struct net_device *ndev, unsigned long event);
DECLARE_RDMA_OBJ_SIZE(ib_ah);
DECLARE_RDMA_OBJ_SIZE(ib_counters);
DECLARE_RDMA_OBJ_SIZE(ib_cq);
DECLARE_RDMA_OBJ_SIZE(ib_comp_cntr);
DECLARE_RDMA_OBJ_SIZE(ib_dmah);
DECLARE_RDMA_OBJ_SIZE(ib_mw);
DECLARE_RDMA_OBJ_SIZE(ib_pd);
DECLARE_RDMA_OBJ_SIZE(ib_qp);
DECLARE_RDMA_OBJ_SIZE(ib_rwq_ind_table);
DECLARE_RDMA_OBJ_SIZE(ib_srq);
DECLARE_RDMA_OBJ_SIZE(ib_ucontext);
DECLARE_RDMA_OBJ_SIZE(ib_xrcd);
DECLARE_RDMA_OBJ_SIZE(rdma_counter);
};

And that? That's the end of the struct, bestie πŸŽ€ Like, the whole roster of function pointers and object-size declarations, all closed out with that one glorious }; β€” we love a clean closing brace, no cap.


🌸 Members 🌸

Yes, this is the section that just says Members β€” and honestly? It's giving "the list of members is the giant struct you literally just read." So consider this the lil' label on the group photo: everything above is the membership. Every pointer, every DECLARE_RDMA_OBJ_SIZE line, all of it, officially inducted πŸ’…


πŸ¦‹ SEE ALSO πŸ¦‹

Cute little stack of related reading, in the order the page gives them β€” not me reordering anything, I swear πŸ™ˆ

First up, the header itself:

  • Kernel file ./include/rdma/ib_verbs.h

Then the whole friend group of friends of friends:

That's every single one, promise 🀞 no skipping, no "etc." β€” I would never πŸ’–


Fr fr, that's the whole page, bestie. You made it to the end 🫢 Go read the struct in order now, you're literally built for this πŸ‘‘

Byeeeee β€” this one lives in <code>./include/rdma</code>, dated <code>October 2026</code>, and the page itself is <code>struct ib_device_ops(9)</code>. Iconic. Periodt. πŸ’…βœ¨

πŸ“¦ where this page comes from