RDMAINFO(3)

📦 version 3.2, in debian trixie, fedora 43, fedora 44, ubuntu noble, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 RDMAINFO(3) — packet.transport.rdmainfo, from rdmainfo 1.1 ✨

okay so sit down, hydrate, and let's talk about RDMA reassembly, because bestie this module is literally the glue holding your RDMA fragments together 🫶

🌸 NAME 🌸

packet.transport.rdmainfo — the RDMA reassembly module. that's it, that's the name, it's giving "I put the pieces back together" energy 💅

🦋 DESCRIPTION 🦋

It provides functionality to reassemble RDMA fragments. no cap, that's the whole job, and she does it flawlessly ✨

👑 CLASSES 👑

🌈 class RDMAinfo(packet.utils.RDMAbase)

RDMA info object used for reassembly. the main character, honestly 👑

The reassembled message consists of one or multiple chunks and each chunk in turn could be composed of multiple segments. Also, each segment could be composed of multiple sub-segments and each sub-segment could be composed of multiple fragments. it's layers on layers on layers, like an onion but make it networking 🧅

The protocol only defines segments but if the segment length is large, it is split into multiple sub-segments in which each sub-segment is specified by RDMA_WRITE_First or RDMA_READ_Request packets. The handle is the same for each of these packets but with a shorter DMA length. iykyk 🔥

Thus in order to reassemble all fragments for a single message, a list of segments is created where each segment is identified by its handle or RKey and the message is reassembled according to the chuck lists specified by the RPC-over-RDMA layer. (yes it says "chuck" in the original, we're keeping it, it's iconic, do not perceive it 💀)

💜 Methods defined here:

  • __del__ = reset(self) — destructor duties, delegated straight to reset 🧹
  • __init__(self) — Constructor. Initializes object's private data according to the arguments given. Arguments can be given as positional, named arguments or a combination of both. flexible queen 💅
  • __len__ = size(self) — length dunder, just calls size 📏
  • add_iwarp_data(self, rdmap, unpack, isread=False) — Add iWarp fragment data 🌊
  • add_iwarp_request(self, rdmap) — Add iWarp read request information 📨
  • add_rdma_data(self, psn, unpack, reth=None, only=False, read=False) — Add Infiniband fragment data 💫
  • add_rdma_segment(self, rdma_seg, rpcrdma=None) — Add RDMA segment information and if the information already exists just update the length and return the segment 🔁
  • del_rdma_segment(self, rsegment) — Delete RDMA segment information 🗑️
  • get_rdma_segment(self, handle) — Return RDMA segment identified by the given handle 🔍
  • process_rdma_segments(self, rpcrdma) — Process the RPC-over-RDMA chunks 🧠

When this method is called on an RPC call, it adds the information of all the segments to the list of segments. When this method is called on an RPC reply, the segments should already exist so just update the segment's DMA length as returned by the reply. two moods, one method 🎀

RPCoRDMA reads attribute is a list of read segments. Read segment is a plain segment plus an XDR position. A read chunk is the collection of all read segments with the same XDR position.

RPCoRDMA writes attribute is a list of write chunks. A write chunk is a list of plain segments.

RPCoRDMA reply is just a single write chunk if it exists. Return the reply chunk data 📤

  • reassemble_rdma_reads(self, unpack, psn=None, only=False, rdmap=None) — Reassemble RDMA read chunks. The RDMA read chunks are reassembled in the read last operation 🧩
  • reset(self) — Clear RDMA segments 🧼
  • size(self) — Return the number RDMA segments 🔢

🌸 class RDMArequest(builtins.object)

RDMA iWarp Request object 🎀

💜 Methods defined here:

  • __contains__(self, offset) — Membership test operator. Return true if offset belongs to this request ✅
  • __init__(self, rdmap, rsegment) — Initialize self. See help(type(self)) for accurate signature 📖
  • get_offset(self, offset) — Return offset translated from sink to src 🔄

🌟 class RDMAseg(builtins.object)

RDMA sub-segment object ✨

The sub-segment is created for each RDMA_WRITE_First, RDMA_WRITE_Only or RDMA_READ_Request and each sub-segment belongs to a list in the RDMAsegment object so there is no segment identifier or handle. anonymous but thriving 🙈

Reassembly for each sub-segment is done using the PSN or packet sequence number in each of the data fragments. Therefore, a range of PSN numbers define this object which is given by the spsn and epsn attributes (first and last PSN respectively). 📬

💜 Methods defined here:

  • __del__(self) — Destructor 💀
  • __init__(self, spsn, epsn, dmalen) — Initialize self. See help(type(self)) for accurate signature 📖
  • get_data(self, padding=True) — Return sub-segment data 📦
  • get_size(self) — Return sub-segment data size 📐
  • insert_data(self, psn, data) — Insert data at correct position given by the psn 🎯

🦋 class RDMAsegment(builtins.object)

RDMA segment object 🫶

Each segment is identified by its handle. The segment information comes from the RPC-over-RDMA protocol layer so the length attribute gives the total DMA length of the segment. 🧵

💜 Methods defined here:

  • __del__(self) — Destructor 💀
  • __init__(self, rdma_seg, rpcrdma) — Initialize self. See help(type(self)) for accurate signature 📖
  • add_data(self, psn, data) — Add Infiniband fragment data 💫
  • add_fragment(self, offset, data) — Add iWarp fragment to segment 🌊
  • add_sub_segment(self, psn, dmalen, only=False, iosize=0) — Add RDMA sub-segment PSN information 🧱
  • get_data(self, padding=True) — Return segment data 📦
  • get_size(self) — Return segment data 📐
  • valid_psn(self, psn) — True if given psn is valid for this segment ✅

🔗 SEE ALSO 🔗

baseobj(3), packet.utils(3) — go read these too, bestie, they're the supporting cast 🎬

💀 BUGS 💀

No known bugs. she's flawless, periodt 👑

✍️ AUTHOR ✍️

Jorge Mora (mora@netapp.com) — the icon, the legend, the moment 👑

and that's the whole page, bestie!! this came from NFStest 3.2, dated 21 March 2023 — so yeah, if you see her around, tell her she ate 💅✨🎀

📦 where this page comes from