OMNI PATH FEATURE NEGOTION (OPFN)(9)

πŸ“¦ version 7.3.1, in nixpkgs unstable

🌸 OMNI PATH FEATURE NEGOTION (OPFN) β€” section 9, Kernel API Manual 🌸

okay bestie, sit down, grab a strawberry πŸ“ β€” we're talking about OPFN and it's genuinely kind of iconic. (and yes, the manual spells it "Negotion" β€” we're not fixing it, we're respecting the lore πŸ’…)


✨ NAME ✨

OPFN is a discovery protocol for the Intel Omni-Path fabric that lets two RC QPs negotiate a common feature that both QPs can support. fr fr, that's the whole personality.

Right now, the only OPA feature OPFN supports is TID RDMA. that's it. that's the list. one (1) feature. lowkey exclusive πŸ‘‘


πŸ¦‹ ARCHITECTURE πŸ¦‹

So here's the vibe check: OPFN involves communication between two QPs on the HFI level on an Omni-Path fabric. and the ULPs? they have no knowledge of OPFN at all. like, none. zero. not even a little. it's giving backstage pass energy β€” the ULPs aren't even invited to the group chat πŸ™ˆ


πŸŽ€ IMPLEMENTATION πŸŽ€

OPFN extends the existing IB RC protocol with these changes, and I need you to appreciate each one:

  • Uses Bit 24 (reserved) of DWORD 1 of Base Transport Header (BTH1) to indicate that the RC QP supports OPFN 🫢
  • Uses a combination of the RC COMPARE_SWAP opcode (0x13) and the address U64_MAX (0xFFFFFFFFFFFFFFFF) as an OPFN request. the 64-bit data carried with the request/response contains the parameters for negotiation, and will be defined in the tid_rdma.c file πŸ“Ž
  • Defines IB_WR_RESERVED3 as IB_WR_OPFN. rebrand of the century, honestly ✨

πŸ’œ the whole little dance πŸ’œ

The OPFN communication gets triggered when an RC QP receives a request with Bit 24 of BTH1 set. here's how it goes down:

  1. The responder QP will then post send an OPFN request with its local parameters β€” and that gets sent to the requester QP once all existing requests on the responder QP side have been sent. (queue discipline, we love to see it ⭐)
  2. Once the requester QP receives the OPFN request, it will keep a copy of the responder QP's parameters, and return a response packet with its own local parameters. 🧚
  3. The responder QP receives the response packet and keeps a copy of the requester QP's parameters. πŸ’–
  4. After this exchange, each side has the parameters for both sides and can therefore select the right parameters for future transactions. mutual understanding!! communication!! growth!! 😭🫢

🌈 SEE ALSO 🌈

  • Kernel file ./drivers/infiniband/hw/hfi1/opfn.h β€” go read it, iykyk πŸ“–

ngl the whole thing is just two QPs trading parameter cards until everybody knows everything. main character negotiation energy. no cap. πŸ’…

okay that's the whole page bestie!! ✨ this little masterpiece lives at ./drivers/infiniband/hw/hfi1, dated October 2026, and it's section 9 of the Kernel API Manual β€” catch you in the next manpage, love u, bye πŸŽ€πŸ¦‹πŸ’œ

πŸ“¦ where this page comes from