fi_rstream(7)

💖 fi_rstream 💖

okay bestie, this one's from the Libfabric Programmer's Manual (version #VERSION#, she's serving), and it's all about the Rstream Fabric Provider. grab a snack, we're going in 🫶

---

🌸 NAME 🌸

fi_rstream — The Rstream Fabric Provider. that's her, that's the girl ✨

---

🦋 OVERVIEW 🦋

so. the rstream provider does stream messaging, but it does it over message based RMA. like, it takes stream and maps it to message on top of a core RMA-based OFI provider. it's giving translator energy fr fr 💅

the vibe check here is that you literally only need Endpoints and EQs for connection start-up and messaging. that's it. that's the whole guest list.

and unlike other OFI providers, rstream does not support CQs and does not support memory registration of any kind. none. zero. not a crumb. 😭

so if you wanna asynchronously wait for a completion (cm/msg), the move is to use fi_control on the endpoint/eq to get yourself an fd, and then use that fd in a poll call. iconic behavior honestly.

and for messaging completions? after you poll, use FI_PEEK on send/recv to see what type of transaction has transpired. no cap, that's how you find out what happened. 👑

---

⭐ SUPPORTED FEATURES ⭐

as of right now, the rstream provider supports FI_MSG capabilities. that's the one. periodt.

🎀 Endpoint types

she supports only endpoint type FI_EP_SOCK_STREAM. monogamous, we respect it.

🎀 Endpoint capabilities

the following data transfer interface is supported: fi_msg. just the one, bestie, but she ate. 💅

🎀 Modes

the provider does not require the use of any mode bits — lowkey chill of her. but she does support core providers that require FI_CONTEXT and FI_RX_CQ_DATA. so if your core provider is needy like that, rstream's got you. 🫶

🎀 Progress

the rstream provider only supports FI_PROGRESS_MANUAL. you're driving, babe. 🚗

🎀 Threading Model

she supports FI_THREAD_SAFE. group chat with all your friends at once? valid. 😌

🎀 Verbs-iWarp

the provider has added features to enable iWarp. to use this feature, the ep protocol iWarp must be requested in an fi_getinfo call. that's the ticket in, no other way through. ✨

---

💀 LIMITATIONS 💀

okay this is the part where I have to be real with you.

the rstream provider is experimental and lacks performance validation and extensive testing. it's giving "we're so back" but also "it's so over." proceed accordingly, bestie. 🙈

the iWarp protocol may need extra initialization work to re-enable. just a heads up.

currently the rstream provider is used to by the rsockets-OFI library as a ULP, and it hooks into the core provider verbs.

and listen — it is not interoperable with the previous rsockets(v1) protocol. they do not know each other. they do not speak. 💀

also there are default settings that limit the message stream (provider memory region size and CQ size). these can be modified by fi_setopt. so if the defaults are giving cramped, you know what to do. 🎀

---

🌈 SETTINGS 🌈

the rstream provider settings can be modified via fi_setopt on the endpoint (FI_OPT_ENDPOINT), along with the following parameters. here's the lineup, no skips:

• FI_OPT_SEND_BUF_SIZE — size of the send buffer. default is 32KB. 📤
• FI_OPT_RECV_BUF_SIZE — size of the recv buffer. default is 32KB. 📥
• FI_OPT_TX_SIZE — size of the send queue. default is 384.
• FI_OPT_RX_SIZE — size of the recv queue. default is 384.

tune her to your heart's content, she'll slay either way ✨

---

🧚 OFI EXTENSIONS 🧚

the rstream provider has extended the current OFI API set in order to enable a user implementation of Poll. specifically, sendmsg(FI_PEEK) is supported — which replicates the behavior of the recvmsg(FI_PEEK) feature. it's the same trick, just on the other side of the convo. iykyk 💅

---

👑 SEE ALSO 👑

• fabric(7)
• fi_provider(7)
• fi_getinfo(3)

go read those too, bestie, they're the supporting cast and they understood the assignment 🌸

---

💜 AUTHORS 💜

OpenFabrics.

---

and that's the whole page, bestie!! 💖 brought to you by the Libfabric Programmer's Manual, version #VERSION#, dated 2022-12-11 — she's a little vintage, she's a little experimental, and she is not interoperable with her ex rsockets(v1). stan her carefully, and always, always poll responsibly. 🎀✨🦋