fi_getinfo(3)

📦
version 2.5.1 · 9 versions
, in centos-stream 10

💖 fi_getinfo(3) 💖

okay so bestie, this is the Libfabric v2.5.1 manual page for fi_getinfo, fi_freeinfo, fi_allocinfo, and fi_dupinfo. We're talking fabric interface info, allocation, duplication, and freeing — the whole lil ecosystem, no cap. 🦋

✨ NAME ✨

  • fi_getinfo, fi_freeinfo — obtain / free fabric interface information, periodt. 💅
  • fi_allocinfo, fi_dupinfo — allocate / duplicate an fi_info structure, slay. 🎀

🛠️ SYNOPSIS 🛠️

#include <rdma/fabric.h>

int fi_getinfo(int version, const char *node, const char *service,
        uint64_t flags, const struct fi_info *hints, struct fi_info **info);

void fi_freeinfo(struct fi_info *info);

struct fi_info *fi_allocinfo(void);

struct fi_info *fi_dupinfo(const struct fi_info *info);

🌸 ARGUMENTS 🌸

  • version — the interface version your app is requesting. like, which version vibe are we on? 💫
  • node — optional, bestie: a name or fabric address to resolve. 🧚
  • service — optional: a service name or port number of the address. 🍓
  • flags — operation flags for the fi_getinfo call. 🚩
  • hints — a reference to an fi_info structure that specifies criteria for selecting the returned fabric information. The fi_info hints structure must be allocated using either fi_allocinfo() or fi_dupinfo(). no random stack structs, okay? 🙈
  • info — a pointer to a linked list of fi_info structures containing response information. it's giving linked-list main character energy. 🔗

💜 DESCRIPTION 💜

The fi_getinfo() call is your little discovery mission: it finds out what communication features are available in the system, and how your app might best use them. The call is loosely modeled on getaddrinfo(). fi_getinfo() lets your app and the libfabric providers exchange information about the exact set of communication your app needs. It's giving you access to complex network details while staying expressive and simple to use — iconic balance, honestly. 🌈

fi_getinfo returns information about available fabric services for reaching a specified node or service, subject to any provided hints. Callers may specify NULL for node, service, and hints to retrieve information about what providers are available and their optimal usage models. If no matching fabric information is available, info will be set to NULL and the call will return -FI_ENODATA. bestie, that's your "nothing matched, bestie" signal. 💀

Based on the input hints, node, and service parameters, a list of fabric domains and endpoints will be returned. Each fi_info structure will describe an endpoint that meets the application's specified communication criteria. Each endpoint will be associated with a domain. Applications can restrict the number of returned endpoints by including additional criteria in their search hints. Relaxing or eliminating input hints will increase the number and type of endpoints that are available. Providers that return multiple endpoints to a single fi_getinfo call should return the endpoints that are highest performing first. Providers may indicate that an endpoint and domain can support additional capabilities than those requested by the user only if such support will not adversely affect application performance or security. like, no sneaky side effects, please and thank you. ✨

The version parameter is used by the application to request the desired version of the interfaces. The version determines the format of all data structures used by any of the fabric interfaces. Applications should use the FI_VERSION(major, minor) macro to indicate the version, with hard-coded integer values for the major and minor values. The FI_MAJOR_VERSION and FI_MINOR_VERSION enum values defined in fabric.h specify the latest version of the installed library. However, it is recommended that the integer values for FI_MAJOR_VERSION and FI_MINOR_VERSION be used, rather than referencing the enum types in order to ensure compatibility with future versions of the library. This protects against the application being built from source against a newer version of the library that introduces new fields to data structures, which would not be initialized by the application. future-proofing era, we love to see it. 👑

node, service, or hints may be provided, with any combination being supported. If node is provided, fi_getinfo will attempt to resolve the fabric address to the given node. If node is not given, fi_getinfo will attempt to resolve the fabric addressing information based on the provided hints. node is commonly used to provide a network address (such as an IP address) or hostname. service is usually associated with a transport address (such as a TCP port number). node and service parameters may be mapped by providers to native fabric addresses. Applications may also pass in an FI_ADDR_STR formatted address (see format details below) as the node parameter. In such cases, the service parameter must be NULL. iykyk. 🫶

The hints parameter, if provided, may be used to limit the resulting output as indicated below. As a general rule, specifying a non-zero value for input hints indicates that a provider must support the requested value or fail the operation with -FI_ENODATA. With the exception of mode bits, hints that are set to zero are treated as a wildcard. A zeroed hint value results in providers either returning a default value or a value that works best for their implementation. Mode bits that are set to zero indicate the application does not support any modes. so, zero mode bits = "no modes supported," not "surprise me." 💅

🚨 The caller must call fi_freeinfo to release fi_info structures returned by this call. do not ghost your allocated memory, bestie. 😭

The fi_allocinfo call will allocate and zero an fi_info structure and all related substructures. The fi_dupinfo will duplicate a single fi_info structure and all the substructures within it. clean, zeroed, duplicated — the whole glow-up. 🎀

okay bestie, now the actual struct itself — every field, in order, no skipping, we're reading the whole thing 💖

struct fi_info {
    struct fi_info        *next;
    uint64_t              caps;
    uint64_t              mode;
    uint32_t              addr_format;
    size_t                src_addrlen;
    size_t                dest_addrlen;
    void                  *src_addr;
    void                  *dest_addr;
    fid_t                 handle;
    struct fi_tx_attr     *tx_attr;
    struct fi_rx_attr     *rx_attr;
    struct fi_ep_attr     *ep_attr;
    struct fi_domain_attr *domain_attr;
    struct fi_fabric_attr *fabric_attr;
    struct fid_nic        *nic;
};
  • next — pointer to the next fi_info structure in the list, bestie. It's NULL when there are no more structures to vibe with. Periodt.
  • caps — fabric interface capabilities. If you set it, it's giving "these are the capabilities I desire" for the fabric interfaces ✨. The supported ones are listed down in the Capabilities section, iykyk.
  • mode — operational modes your application supports. Peep the Mode section below for the deets 🔥.
  • addr_format — address format. If specified, it tells you the format of the addresses that the fabric interfaces and data structures are referencing. Supported formats live in the Addressing formats section below, no cap 💅.
  • src_addrlen — source address length, literally. It's the length of the source address. This value must be > 0 if src_addr is non-NULL, okay? And heads up — this field gets ignored in hints if the FI_SOURCE flag is set, or if src_addr is NULL. 🫶
  • dest_addrlen — destination address length. Same energy: the value must be > 0 if dest_addr is non-NULL. This field is ignored in hints unless the node and service parameters are NULL or the FI_SOURCE flag is set, or if dst_addr is NULL. 🎀
  • src_addr — source address. If you specify it, that's your source address. This field gets ignored in hints if the FI_SOURCE flag is set. On output, a provider shall return an address that corresponds to the indicated fabric, domain, node, and/or service fields 💜. The format of that address? That's what the returned addr_format field is telling you. Also important: any returned address is only used when opening a local endpoint — it is not guaranteed to be usable by a peer process. So don't go handing it out like candy, ngl 🙈.
  • dest_addr — destination address. If specified, that's the destination. This field is ignored in hints unless the node and service parameters are NULL or the FI_SOURCE flag is set. If FI_SOURCE is not specified, then on output a provider shall return an address that corresponds to the indicated node and/or service fields, relative to the fabric and domain. And just like the source one — any returned address is only usable locally 🦋.
  • handle — provider context handle. This one is so operation specific, bestie. If hints->handle is set to struct fid_pep, then hints->handle gets copied to info->handle on output from fi_getinfo ✨. Any other values of hints->handle are handled in a provider specific manner. The fi_info::handle field is also used by fi_endpoint() and fi_reject() calls when processing connection requests, or to inherit another endpoint's attributes. Go see fi_eq(3), fi_reject(3), and fi_endpoint(3) for the lore 💅. Oh, and the info->handle field is straight up ignored by fi_dupinfo and fi_freeinfo — like, not even looked at 😭.
  • tx_attr — transmit context attributes, optionally supplied. These may be specified and returned as part of fi_getinfo. When you provide them as hints, the requested values of struct fi_tx_ctx_attr should be set. On output, the actual transmit context attributes that can be provided get returned — and those output values will be greater than or equal to the requested input values. Slay 🎀.
  • rx_attr — receive context attributes, optionally supplied. Also may be specified and returned as part of fi_getinfo. When provided as hints, requested values of struct fi_rx_ctx_attr should be set. On output, the actual receive context attributes that can be provided are returned, and output values will be greater than or equal to the requested input values 💖.
  • ep_attr — endpoint attributes, optionally supplied. May be specified and returned as part of fi_getinfo. When provided as hints, requested values of struct fi_ep_attr should be set. On output, the actual endpoint attributes that can be provided are returned — output values greater than or equal to the requested input values. See fi_endpoint(3) for the details 🌸.
  • domain_attr — domain attributes, optionally supplied. May be specified and returned as part of fi_getinfo. When provided as hints, requested values of struct fi_domain_attr should be set. On output, the actual domain attributes that can be provided are returned — output values greater than or equal to requested input values. See fi_domain(3) for more 💜.
  • fabric_attr — fabric attributes, optionally supplied. Same deal: may be specified and returned as part of fi_getinfo. When provided as hints, requested values of struct fi_fabric_attr should be set. On output, the actual fabric attributes that can be provided are returned. See fi_fabric(3) for the tea ☕.
  • nic — network interface details. These are optional attributes related to the hardware NIC associated with the specified fabric, domain, and endpoint data 🦋. Heads up: this field is only valid for providers where the corresponding attributes are closely associated with a hardware NIC. See fi_nic(3) for deets.

✨ CAPABILITIES ✨

okay so interface capabilities? you get them by OR-ing the following flags together. that's the whole vibe, no cap 🔥

  • If you hand over a NULL hints structure, the underlying provider will return the set of capabilities which are supported.
  • If you provide a hints structure but the capabilities in it are set to 0, the underlying provider will only return the set of capabilities that the provider can support with the mode bits set by the application in the hints structure.
  • Otherwise — if capabilities are specified with non-zero values — providers will return data matching the set of capabilities you specified 💅.

Providers may indicate support for additional capabilities beyond the ones you requested, but only when using those expanded capabilities will not adversely affect performance or expose your application to communication beyond what you asked for. Applications can use this little feature to request a minimal set of requirements, then check the returned capabilities to enable additional optimizations. It's giving "ask for the bare minimum, get a lil extra" ✨.

  • FI_ATOMIC — specifies that the endpoint supports some set of atomic operations. Endpoints supporting this capability support the operations defined by struct fi_ops_atomic. In the absence of any relevant flags, FI_ATOMIC implies the ability to initiate and be the target of remote atomic reads and writes. Applications can use the FI_READ, FI_WRITE, FI_REMOTE_READ, and FI_REMOTE_WRITE flags to restrict the types of atomic operations supported by an endpoint 👑
  • FI_AV_USER_ID — requests that the provider support associating a user specified identifier with each address vector (AV) address. User identifiers get returned with completion data in place of the AV address. See fi_domain(3) and fi_av(3) for more details 🎀
  • FI_COLLECTIVE — requests support for collective operations. Endpoints that support this capability support the collective operations defined in fi_collective(3) 💖
  • FI_DIRECTED_RECV — requests that the communication endpoint use the source address of an incoming message when matching it with a receive buffer. If this capability is not set, then the src_addr parameter for msg and tagged receive operations is just ignored 🙈
  • FI_TAGGED_DIRECTED_RECV — similar to FI_DIRECTED_RECV, but it only applies to tagged receive operations 💅
  • FI_EXACT_DIRECTED_RECV — also similar to FI_DIRECTED_RECV, but it requires the source address to be exact, i.e. FI_ADDR_UNSPEC is not allowed. This capability can be used alone, or in conjunction with FI_DIRECTED_RECV or FI_TAGGED_DIRECTED_RECV as a modifier to disallow FI_ADDR_UNSPEC being used as the source address 🌈
  • FI_FENCE — indicates that the endpoint supports the FI_FENCE flag on data transfer operations. Support requires tracking that all previous transmit requests to a specified remote endpoint complete prior to initiating the fenced operation. Fenced operations are often used to enforce ordering between operations that are not otherwise guaranteed by the underlying provider or protocol 🔥
  • FI_HMEM — specifies that the endpoint should support transfers to and from device memory 💜
  • FI_LOCAL_COMM — indicates that the endpoint supports host local communication. This flag may be used in conjunction with FI_REMOTE_COMM to indicate that local and remote communication are required. If neither FI_LOCAL_COMM nor FI_REMOTE_COMM are specified, then the provider will indicate support for the configuration that minimally affects performance. Providers that set FI_LOCAL_COMM but not FI_REMOTE_COMM — for example, a shared memory provider — may only be used for communication between processes on the same system 🫶
  • FI_MSG — specifies that an endpoint should support sending and receiving messages or datagrams. Message capabilities imply support for send and/or receive queues. Endpoints supporting this capability support operations defined by struct fi_ops_msg ✨

the caps field can also be used to specify or restrict the type of messaging operations that are supported. in the absence of any relevant flags, FI_MSG implies the ability to send and receive messages. applications can use the FI_SEND and FI_RECV flags to optimize an endpoint as send-only or receive-only 👑

  • FI_MULTICAST — okay so this one says the endpoint supports multicast data transfers ✨. It has to be paired with FI_MSG — no solo acts. Bestie bonus: apps can use FI_SEND and FI_RECV to optimize multicast as send-only or receive-only. Choose your vibe 💅
  • FI_MULTI_RECV — the endpoint must support the FI_MULTI_RECV flag when posting receive buffers. That's it, that's the whole flag 🎀
  • FI_TAGGED_MULTI_RECV — same energy, but for tagged: the endpoint must support the FI_MULTI_RECV flag when posting tagged receive buffers 🦋
  • FI_NAMED_RX_CTX — asks that endpoints with multiple receive contexts let an initiator target (or name) a specific receive context as part of a data transfer operation. Very "I'm picking my seat" energy 👑
  • FI_PEER — the provider must support being used as a peer provider in the peer API flow 🫶. Concretely: it must support importing owner_ops when opening a CQ, counter, and shared receive queue.
  • FI_READ — you're saying "I need an endpoint that can initiate reads against remote memory regions." This flag requires that FI_RMA and/or FI_ATOMIC be set. No cap, read the fine print 📖
  • FI_RECV — you need an endpoint capable of receiving message data transfers 💌. Message transfers include base message operations as well as tagged message functionality. (Yes, both. Package deal.)
  • FI_REMOTE_COMM — the endpoint supports communication with endpoints located at remote nodes (across the fabric) 🌈. See FI_LOCAL_COMM for additional details. And here's the tea: providers that set FI_REMOTE_COMM but not FI_LOCAL_COMM — for example NICs that lack loopback support — cannot be used to communicate with processes on the same system. Same machine, no chat. Tragic 💀
  • FI_REMOTE_READ — you require an endpoint capable of receiving read memory operations from remote endpoints 🌸. Requires that FI_RMA and/or FI_ATOMIC be set.
  • FI_REMOTE_WRITE — you require an endpoint capable of receiving write memory operations from remote endpoints 🍓. Requires that FI_RMA and/or FI_ATOMIC be set. (Yes, both of these require the same thing, they're twins, let them live.)
  • FI_RMA — the endpoint should support RMA read and write operations ✨. Endpoints supporting this capability support operations defined by struct fi_ops_rma. In the absence of any relevant flags, FI_RMA implies the ability to initiate and be the target of remote memory reads and writes. Want to narrow it down? Applications can use the FI_READ, FI_WRITE, FI_REMOTE_READ, and FI_REMOTE_WRITE flags to restrict the types of RMA operations supported by an endpoint. Build-a-bear but for permissions 🧸
  • FI_RMA_EVENT — requests that an endpoint support the generation of completion events when it is the target of an RMA and/or atomic operation 🔥. This flag requires that FI_REMOTE_READ and/or FI_REMOTE_WRITE be enabled on the endpoint.
  • FI_RMA_PMEM — the provider is persistent memory aware' and supports RMA operations to and from persistent memory 💜. Persistent memory aware providers must support registration of memory that is backed by non-volatile memory, RMA transfers to/from persistent memory, and enhanced completion semantics. This flag requires that FI_RMA` be set. Also — heads up — this capability is experimental ⚠️ so like, hydrate before using.
  • FI_SEND — you require an endpoint capable of sending message data transfers 💌. Message transfers include base message operations as well as tagged message functionality. (Mirror image of FI_RECV, matching outfits fr fr.)
  • FI_SHARED_AV — requests or indicates support for address vectors which may be shared among multiple processes 🤝. Sharing is caring, allegedly.
  • FI_SOURCE — requests that the endpoint return source addressing data as part of its completion data 📬. This capability only applies to connectionless endpoints. Now the caveat, because there's always a caveat bestie: returning source address information may require that the provider perform address translation and/or look-up based on data available in the underlying protocol in order to provide the requested data, which may adversely affect performance 😭. And the performance impact may be greater for address vectors of type FI_AV_TABLE. Ouch.
  • FI_SOURCE_ERR — must be paired with FI_SOURCE 👯. When specified, this requests that raw source addressing data be returned as part of completion data for any address that has not been inserted into the local address vector. Heads up though: use of this capability may require the provider to validate incoming source address data against addresses stored in the local address vector, which may adversely affect performance 💀. Again. Sorry.
  • FI_TAGGED — the endpoint should handle tagged message transfers 🏷️✨. Tagged message transfers associate a user-specified key or tag with each message that is used for matching purposes at the remote side. Endpoints supporting this capability support operations defined by struct fi_ops_tagged. In the absence of any relevant flags, FI_TAGGED implies the ability to send and receive tagged messages. Want to be picky? Applications can use the FI_SEND and FI_RECV flags to optimize an endpoint as send-only or receive-only 💅
  • FI_TRIGGER — the endpoint should support triggered operations ⚡. Endpoints that support this capability must meet the usage model as described by fi_trigger(3). Go read it, iykyk.
  • FI_WRITE — you require an endpoint capable of initiating writes against remote memory regions ✍️. This flag requires that FI_RMA and/or FI_ATOMIC be set.
  • FI_XPU — the endpoint should support transfers that may be initiated from heterogenous computation devices, such as GPUs 🎮🔥. This flag requires that FI_TRIGGER be set. For additional details on XPU triggers see fi_trigger(3).

Okay so now the big picture — capabilities may be grouped into three general categories: primary, secondary, and primary modifiers 🌟. Buckle up, this is the lore.

Primary capabilities must explicitly be requested by an application, and a provider must enable support for only those primary capabilities which were selected. Primary modifiers are used to limit a primary capability, such as restricting an endpoint to being send-only 🎯. And if no modifiers are specified for an applicable capability, all relevant modifiers are assumed. See above definitions for details ☝️

Secondary capabilities may optionally be requested by an application 💫. If requested, a provider must support the capability or fail the fi_getinfo request (FI_ENODATA) 😤. A provider may optionally report non-selected secondary capabilities if doing so would not compromise performance or security. Generous of them.

Primary capabilities: FI_MSG, FI_RMA, FI_TAGGED, FI_ATOMIC, FI_MULTICAST, FI_NAMED_RX_CTX, FI_DIRECTED_RECV, FI_TAGGED_DIRECTED_RECV, FI_HMEM, FI_COLLECTIVE, FI_XPU, FI_AV_USER_ID, FI_PEER 👑

Primary modifiers: FI_READ, FI_WRITE, FI_RECV, FI_SEND, FI_REMOTE_READ, FI_REMOTE_WRITE 🎀

Secondary capabilities: FI_MULTI_RECV, FI_TAGGED_MULTI_RECV, FI_SOURCE, FI_RMA_EVENT, FI_SHARED_AV, FI_TRIGGER, FI_FENCE, FI_LOCAL_COMM, FI_REMOTE_COMM, FI_SOURCE_ERR, FI_RMA_PMEM 🦋

💅 MODE 💅

okay so the operational mode bits are literally the requirements an app must adhere to when using the fabric interfaces, no cap. Modes specify optimal ways of accessing the reported endpoint or domain. Apps designed to support a specific mode of operation may see improved performance when that mode is desired by the provider. It’s recommended that providers support apps that disable any provider preferred modes 🫶.

On input to fi_getinfo, applications set the mode bits that they support. On output, providers will clear mode bits that are not necessary to achieve high-performance. Mode bits that remain set indicate application requirements for using the fabric interfaces created using the returned fi_info. The set of modes is listed below. If a NULL hints structure is provided, then the provider’s supported set of modes will be returned in the info structure(s) ✨.

  • FI_ASYNC_IOV — applications can reference multiple data buffers as part of a single operation through the use of IO vectors (SGEs). Typically, the contents of an IO vector are copied by the provider into an internal buffer area, or directly to the underlying hardware. But when a large number of IOV entries are supported, IOV buffering may have a negative impact on performance and memory consumption 😬. The FI_ASYNC_IOV mode indicates that the application must provide the buffering needed for the IO vectors. When set, an application must not modify an IO vector of length > 1, including any related memory descriptor array, until the associated operation has completed. periodt.
  • FI_CONTEXT — specifies that the provider requires that applications use struct fi_context as their per operation context parameter for operations that generated full completions. This structure should be treated as opaque to the application. For performance reasons, this structure must be allocated by the user, but may be used by the fabric provider to track the operation. Typically, users embed struct fi_context within their own context structure 💖. The struct fi_context must remain valid until the corresponding operation completes or is successfully canceled. As such, fi_context should NOT be allocated on the stack 😭. Doing so is likely to result in stack corruption that will be difficult to debug. Users should not update or interpret the fields in this structure, or reuse it until the original operation has completed. If an operation does not generate a completion (i.e. the endpoint was configured with FI_SELECTIVE_COMPLETION and the operation was not initiated with the FI_COMPLETION flag) then the context parameter is ignored by the fabric provider. The structure is specified in rdma/fabric.h 📚.
  • FI_CONTEXT2 — this bit is similar to FI_CONTEXT, but doubles the provider’s requirement on the size of the per context structure 👀. When set, this specifies that the provider requires that applications use struct fi_context2 as their per operation context parameter. Or, optionally, an application can provide an array of two fi_context structures (e.g. struct fi_context[2]) instead. The requirements for using struct fi_context2 are identical as defined for FI_CONTEXT above. same vibes, bigger context, iykyk.
  • FI_LOCAL_MR (deprecated) — this one’s deprecated, bestie, so don’t build your whole personality around it. The provider is optimized around having applications register memory for locally accessed data buffers. Data buffers used in send and receive operations and as the source buffer for RMA and atomic operations must be registered by the application for access domains opened with this capability. This flag is defined for compatibility and is ignored if the application version is 1.5 or later and the domain mr_mode is set to anything other than FI_MR_BASIC or FI_MR_SCALABLE. See the domain attribute mr_mode fi_domain(3) and fi_mr(3) 🫡.
  • FI_MSG_PREFIX — message prefix mode indicates that an application will provide buffer space in front of all message send and receive buffers for use by the provider. Typically, the provider uses this space to implement a protocol, with the protocol headers being written into the prefix area. The contents of the prefix space should be treated as opaque. The use of FI_MSG_PREFIX may improve application performance over certain providers by reducing the number of IO vectors referenced by underlying hardware and eliminating provider buffer allocation 🌸.

Real talk: FI_MSG_PREFIX only applies to send and receive operations, including tagged sends and receives. RMA and atomics do not require the application to provide prefix buffers. Prefix buffer space must be provided with all sends and receives, regardless of the size of the transfer or other transfer options. The ownership of prefix buffers is treated the same as the corresponding message buffers, but the size of the prefix buffer is not counted toward any message limits, including inject 💀.

Applications that support prefix mode must supply buffer space before their own message data. The size of space that must be provided is specified by the msg_prefix_size endpoint attribute. Providers are required to define a msg_prefix_size that is a multiple of 8 bytes. Additionally, applications may receive provider generated packets that do not contain application data. Such received messages will indicate a transfer size of that is equal to or smaller than msg_prefix_size 👑.

The buffer pointer given to all send and receive operations must point to the start of the prefix region of the buffer (as opposed to the payload). For scatter-gather send/recv operations, the prefix buffer must be a contiguous region, though it may or may not be directly adjacent to the payload portion of the buffer 🧚.

  • FI_RX_CQ_DATA — this mode bit only applies to data transfers that set FI_REMOTE_CQ_DATA. When set, a data transfer that carries remote CQ data will consume a receive buffer at the target. This is true even for operations that would normally not consume posted receive buffers, such as RMA write operations 🔥. yes, even the ones that think they’re too cool to eat a receive buffer.

🗺️ ADDRESSING FORMATS 🗺️

Multiple fabric interfaces take as input either a source or destination address parameter. This includes struct fi_info (src_addr and dest_addr), CM calls (getname, getpeer, connect, join, and leave), and AV calls (insert, lookup, and straddr). The fi_info addr_format field indicates the expected address format for these operations 📬.

A provider may support one or more of the following addressing formats. In some cases, a selected addressing format may need to be translated or mapped into an address which is native to the fabric. See fi_av(3) ✨.

  • FI_ADDR_EFA — address is an Amazon Elastic Fabric Adapter (EFA) proprietary format ☁️.
  • FI_ADDR_PSMX2 — address is an Intel proprietary format used with their Performance Scaled Messaging protocol version 2 💜.
  • FI_ADDR_PSMX3 — address is an Intel proprietary format used with their Performance Scaled Messaging protocol version 3 💜.
  • FI_ADDR_STR — address is a formatted character string. The length and content of the string is address and/or provider specific, but in general follows a URI model:
address_format[://[node][:[service][/[field3]...][?[key=value][&k2=v2]...]]]

Examples:

- fi_sockaddr://10.31.6.12:7471
- fi_sockaddr_in6://[fe80::6:12]:7471
- fi_sockaddr://10.31.6.12:7471?qos=3

Since the string formatted address does not contain any provider information, the prov_name field of the fabric attribute structure should be used to filter by provider if necessary 🔍.

  • FI_FORMAT_UNSPEC — FI_FORMAT_UNSPEC indicates that a provider specific address format should be selected. Provider specific addresses may be protocol specific or a vendor proprietary format. Applications that select FI_FORMAT_UNSPEC should be prepared to treat returned addressing data as opaque 🙈. FI_FORMAT_UNSPEC targets apps which make use of an out of band address exchange. Applications which use FI_FORMAT_UNSPEC may use fi_getname() to obtain a provider specific address assigned to an allocated endpoint. slay for out-of-band chaos.
  • FI_SOCKADDR — address is of type sockaddr. The specific socket address format will be determined at run time by interfaces examining the sa_family field 🧦.
  • FI_SOCKADDR_IB — address is of type sockaddr_ib (defined in Linux kernel source) 🐧.
  • FI_SOCKADDR_IN — address is of type sockaddr_in (IPv4) 🌈.
  • FI_SOCKADDR_IN6 — address is of type sockaddr_in6 (IPv6) 🌈.
  • FI_SOCKADDR_IP — address is of type sockaddr_in (IPv4) or sockaddr_in6 (IPv6). The specific socket address format will be determined at run time by interfaces examining the sa_family field. This format is different from FI_SOCKADDR in that the address can only be IPv4 or IPv6, not any other socket address such as sockaddr_ib 💅.

🚩 FLAGS 🚩

The operation of the fi_getinfo call may be controlled through the use of input flags. Valid flags include the following 👇.

  • FI_NUMERICHOST — indicates that the node parameter is a numeric string representation of a fabric address, such as a dotted decimal IP address. Use of this flag will suppress any lengthy name resolution protocol ⚡. no waiting around for DNS to finish its little side quest.
  • FI_PROV_ATTR_ONLY — indicates that the caller is only querying for what providers are potentially available. All providers will return exactly one fi_info struct, regardless of whether that provider is usable on the current platform or not. The returned fi_info struct will contain default values for all members, with the exception of fabric_attr. The fabric_attr member will have the prov_name and prov_version values filled in 🎀.
  • FI_SOURCE — indicates that the node and service parameters specify the local source address to associate with an endpoint. If specified, either the node and/or service parameter must be non-NULL. This flag is often used with passive endpoints 🛋️.
  • FI_RESCAN — indicates that the provider should rescan available network interfaces. This operation may be computationally expensive 😮‍💨. like, it’s giving “I’m gonna need a minute” energy.

💖 RETURN VALUE 💖

Okay bestie, here's the vibe check on what comes back. fi_getinfo() is a success girlie and returns 0 when everything is slay. But on error? It returns a negative value corresponding to fabric errno — and those fabric errno values are defined in rdma/fi_errno.h. No cap.

fi_allocinfo() returns a pointer to a new fi_info structure on success, or NULL on error. fi_dupinfo() duplicates a single fi_info structure and all the substructures within it, returning a pointer to the new fi_info structure on success, or NULL on error. Both calls require that the returned fi_info structure be freed via fi_freeinfo(). Periodt. 💅

🚨 ERRORS 🚨

Here's the drama board of error codes, bestie:

  • FI_EBADFLAGS — the specified endpoint or domain capability or operation flags are invalid. Not it.
  • FI_ENODATA — indicates that no providers could be found which support the requested fabric information. Literally nobody matched.
  • FI_ENOMEM — indicates that there was insufficient memory to complete the operation. The memory budget said no. 😭
  • FI_ENOSYS — indicates that requested version is newer than the library being used. You're asking for a future era, bestie. 🦋

📝 NOTES 📝

Okay so important tea: various libfabric calls, including fi_getinfo, take a struct fi_info as input. Applications must use libfabric allocated fi_info structures. A zeroed struct fi_info can be allocated using fi_allocinfo, which may then be initialized by the user. A struct fi_info may be copied for modification using the fi_dupinfo() call. Slay.

If hints are provided, the operation will be controlled by the values that are supplied in the various fields (see section on fi_info). Applications that require specific communication interfaces, domains, capabilities or other requirements, can specify them using fields in hints. Libfabric returns a linked list in info that points to a list of matching interfaces. info is set to NULL if there are no communication interfaces or none match the input hints. 🫶

If node is provided, fi_getinfo will attempt to resolve the fabric address to the given node. If node is not provided, fi_getinfo will attempt to resolve the fabric addressing information based on the provided hints. The caller must call fi_freeinfo to release fi_info structures returned by fi_getinfo. Don't forget the cleanup, bestie.

If neither node, service or hints are provided, then fi_getinfo simply returns the list all available communication interfaces. (Yes, grammar exactly like that, bestie.) Multiple threads may call fi_getinfo simultaneously, without any requirement for serialization. Iconic. 🎀

👀 SEE ALSO 👀

The whole squad: fi_open(3), fi_endpoint(3), fi_domain(3), fi_nic(3) fi_trigger(3) — go read those next, iykyk. 🔥

👑 AUTHORS 👑

OpenFabrics. Periodt. 🌈

And that's the whole page, bestie! This cutie came from the Libfabric Programmer’s Manual, dated 2025-10-31, and it's all about fi_getinfo(3). 💅✨

📦 where this page comes from