rte_mldev.h(3)

📦
version 25.11.3 · 9 versions
, in ubuntu resolute, ubuntu stonking

💖 rte_mldev.h — the ml dev girlie's bible (part 1) 💖

okay bestie, sit down, hydrate, because we're reading the rte_mldev.h manual page — that's the Library Functions Manual one, section 3, and yes it's a whole header file with a manual page, because DPDK said "documentation is self-care" and honestly? ate. 💅

This is part 1 of 5, so don't you dare close the tab. 🫶

✨ NAME ✨

rte_mldev.h — no cap, that's the whole name, it's giving "the file said who it is and left" 👑

🌈 SYNOPSIS 🌈

First things first, you gotta pull in your little include squad:

#include <rte_common.h>
#include <rte_log.h>
#include <rte_mempool.h>

🎀 Data Structures 🎀

these are the structs, the whole cast of characters, the main character energy roster:

  • struct rte_ml_dev_info
  • struct rte_ml_dev_config
  • struct rte_ml_dev_qp_conf
  • struct rte_ml_buff_seg
  • struct rte_ml_op
  • struct rte_ml_op_error
  • struct rte_ml_dev_stats
  • struct rte_ml_dev_xstats_map
  • struct rte_ml_model_params
  • struct rte_ml_io_info
  • struct rte_ml_model_info

🦋 Macros 🦋

one (1) macro, and she's iconic:

  • #define RTE_ML_STR_MAX 128 — the max string length vibe, 128 and not a single character more 💯

💜 Typedefs 💜

a callback for when the dev is stopping and needs to flush:

typedef void(* rte_ml_dev_stop_flush_t) (int16_t dev_id, uint16_t qp_id,
    struct rte_ml_op *op)

🌸 Enumerations 🌸

look at all these lil enums, so organized, so demure:

  • enum rte_ml_op_status { RTE_ML_OP_STATUS_SUCCESS = 0, RTE_ML_OP_STATUS_NOT_PROCESSED, RTE_ML_OP_STATUS_ERROR } — how an operation went: slay, not yet, or oh no 😭
  • enum rte_ml_dev_xstats_mode { RTE_ML_DEV_XSTATS_DEVICE, RTE_ML_DEV_XSTATS_MODEL } — are we getting stats for the device or for a model? pick your fighter
  • enum rte_ml_io_type { RTE_ML_IO_TYPE_UNKNOWN = 0, RTE_ML_IO_TYPE_INT8, RTE_ML_IO_TYPE_UINT8, RTE_ML_IO_TYPE_INT16, RTE_ML_IO_TYPE_UINT16, RTE_ML_IO_TYPE_INT32, RTE_ML_IO_TYPE_UINT32, RTE_ML_IO_TYPE_INT64, RTE_ML_IO_TYPE_UINT64, RTE_ML_IO_TYPE_FP8, RTE_ML_IO_TYPE_FP16, RTE_ML_IO_TYPE_FP32, RTE_ML_IO_TYPE_BFLOAT16 } — every data type you could possibly be, literally the whole zodiac of numbers 🔢
  • enum rte_ml_io_layout { RTE_ML_IO_LAYOUT_PACKED, RTE_ML_IO_LAYOUT_SPLIT } — packed or split, choose your layout destiny ✨

👑 Functions 👑

okay this is the big one, the whole group chat of functions, every single one, no skipping:

  • __rte_experimental int rte_ml_dev_init (size_t dev_max) — spin up the ml device subsystem, dev_max being how many devices you're willing to host
  • __rte_experimental uint16_t rte_ml_dev_count (void) — how many ml devices are out here living their truth
  • __rte_experimental int rte_ml_dev_is_valid_dev (int16_t dev_id) — vibe check: is this dev_id even real
  • __rte_experimental int rte_ml_dev_socket_id (int16_t dev_id) — which socket is this device on
  • __rte_experimental int rte_ml_dev_info_get (int16_t dev_id, struct rte_ml_dev_info *dev_info) — grab the device's info into dev_info
  • __rte_experimental int rte_ml_dev_configure (int16_t dev_id, const struct rte_ml_dev_config *config) — set the device up with config
  • __rte_experimental uint16_t rte_ml_dev_queue_pair_count (int16_t dev_id) — how many queue pairs this device got
  • __rte_experimental int rte_ml_dev_queue_pair_setup (int16_t dev_id, uint16_t queue_pair_id, const struct rte_ml_dev_qp_conf *qp_conf, int socket_id) — set up a specific queue pair with its conf on a given socket
  • __rte_experimental int rte_ml_dev_start (int16_t dev_id) — go go go, device online
  • __rte_experimental int rte_ml_dev_stop (int16_t dev_id) — pause, bestie
  • __rte_experimental int rte_ml_dev_close (int16_t dev_id) — shut it all the way down
  • __rte_experimental uint16_t rte_ml_enqueue_burst (int16_t dev_id, uint16_t qp_id, struct rte_ml_op **ops, uint16_t nb_ops) — yeet a burst of operations onto a queue pair, returns how many you actually enqueued
  • __rte_experimental uint16_t rte_ml_dequeue_burst (int16_t dev_id, uint16_t qp_id, struct rte_ml_op **ops, uint16_t nb_ops) — pull a burst of finished operations back out
  • __rte_experimental int rte_ml_op_error_get (int16_t dev_id, struct rte_ml_op *op, struct rte_ml_op_error *error) — did an op flop? find out why into error
  • __rte_experimental int rte_ml_dev_stats_get (int16_t dev_id, struct rte_ml_dev_stats *stats) — fetch the device stats
  • __rte_experimental void rte_ml_dev_stats_reset (int16_t dev_id) — wipe those stats, fresh start 🌱
  • __rte_experimental int rte_ml_dev_xstats_names_get (int16_t dev_id, enum rte_ml_dev_xstats_mode mode, int32_t model_id, struct rte_ml_dev_xstats_map *xstats_map, uint32_t size) — get the names of the extended stats, in whatever mode, for whatever model_id, into xstats_map of size
  • __rte_experimental int rte_ml_dev_xstats_by_name_get (int16_t dev_id, const char *name, uint16_t *stat_id, uint64_t *value) — look up an extended stat by its name and get its stat_id and value
  • __rte_experimental int rte_ml_dev_xstats_get (int16_t dev_id, enum rte_ml_dev_xstats_mode mode, int32_t model_id, const uint16_t stat_ids[], uint64_t values[], uint16_t nb_ids) — grab the actual values for nb_ids stat ids
  • __rte_experimental int rte_ml_dev_xstats_reset (int16_t dev_id, enum rte_ml_dev_xstats_mode mode, int32_t model_id, const uint16_t stat_ids[], uint16_t nb_ids) — reset those extended stats, by id
  • __rte_experimental int rte_ml_dev_dump (int16_t dev_id, FILE *fd) — dump all the device deets into fd
  • __rte_experimental int rte_ml_dev_selftest (int16_t dev_id) — run the device's own little self-test, go king
  • __rte_experimental int rte_ml_model_load (int16_t dev_id, struct rte_ml_model_params *params, uint16_t *model_id) — load a model from params, get its model_id back
  • __rte_experimental int rte_ml_model_unload (int16_t dev_id, uint16_t model_id) — bye bye model
  • __rte_experimental int rte_ml_model_start (int16_t dev_id, uint16_t model_id) — wake that model up
  • __rte_experimental int rte_ml_model_stop (int16_t dev_id, uint16_t model_id) — put it to sleep, not goodbye
  • __rte_experimental int rte_ml_model_info_get (int16_t dev_id, uint16_t model_id, struct rte_ml_model_info *model_info) — get the model's info into model_info
  • __rte_experimental int rte_ml_model_params_update (int16_t dev_id, uint16_t model_id, void *buffer) — swap the model's params using buffer
  • __rte_experimental int rte_ml_io_float32_to_int8 (const void *fp32, void *i8, uint64_t nb_elements, float scale, int8_t zero_point) — convert fp32 → i8, nb_elements at a time, with scale and zero_point
  • __rte_experimental int rte_ml_io_int8_to_float32 (const void *i8, void *fp32, uint64_t nb_elements, float scale, int8_t zero_point) — the reverse trip: i8 → fp32
  • __rte_experimental int rte_ml_io_float32_to_uint8 (const void *fp32, void *ui8, uint64_t nb_elements, float scale, uint8_t zero_point) — fp32 → ui8
  • __rte_experimental int rte_ml_io_uint8_to_float32 (const void *ui8, void *fp32, uint64_t nb_elements, float scale, uint8_t zero_point) — ui8 → fp32
  • __rte_experimental int rte_ml_io_float32_to_int16 (const void *fp32, void *i16, uint64_t nb_elements, float scale, int16_t zero_point) — fp32 → i16
  • __rte_experimental int rte_ml_io_int16_to_float32 (const void *i16, void *fp32, uint64_t nb_elements, float scale, int16_t zero_point) — i16 → fp32
  • __rte_experimental int rte_ml_io_float32_to_uint16 (const void *fp32, void *ui16, uint64_t nb_elements, float scale, uint16_t zero_point) — fp32 → ui16
  • __rte_experimental int rte_ml_io_uint16_to_float32 (const void *ui16, void *fp32, uint64_t nb_elements, float scale, uint16_t zero_point) — ui16 → fp32
  • __rte_experimental int rte_ml_io_float32_to_int32 (const void *fp32, void *i32, uint64_t nb_elements, float scale, int32_t zero_point) — fp32 → i32
  • __rte_experimental int rte_ml_io_int32_to_float32 (const void *i32, void *fp32, uint64_t nb_elements, float scale, int32_t zero_point) — i32 → fp32
  • __rte_experimental int rte_ml_io_float32_to_uint32 (const void *fp32, void *ui32, uint64_t nb_elements, float scale, uint32_t zero_point) — fp32 → ui32
  • __rte_experimental int rte_ml_io_uint32_to_float32 (const void *ui32, void *fp32, uint64_t nb_elements, float scale, uint32_t zero_point) — ui32 → fp32
  • __rte_experimental int rte_ml_io_float32_to_int64 (const void *fp32, void *i64, uint64_t nb_elements, float scale, int64_t zero_point) — fp32 → i64
  • __rte_experimental int rte_ml_io_int64_to_float32 (const void *i64, void *fp32, uint64_t nb_elements, float scale, int64_t zero_point) — i64 → fp32
  • __rte_experimental int rte_ml_io_float32_to_uint64 (const void *fp32, void *ui64, uint64_t nb_elements, float scale, uint64_t zero_point) — fp32 → ui64
  • __rte_experimental int rte_ml_io_uint64_to_float32 (const void *ui64, void *fp32, uint64_t nb_elements, float scale, uint64_t zero_point) — ui64 → fp32
  • __rte_experimental int rte_ml_io_float32_to_float16 (const void *fp32, void *fp16, uint64_t nb_elements) — fp32 → fp16, no scale or zero_point needed, she's just built different ✨
  • __rte_experimental int rte_ml_io_float16_to_float32 (const void *fp16, void *fp32, uint64_t nb_elements) — fp16 → fp32, same deal, no quant params
  • __rte_experimental int rte_ml_io_float32_to_bfloat16 (const void *fp32, void *bf16, uint64_t nb_elements) — fp32 → bf16
  • __rte_experimental int rte_ml_io_bfloat16_to_float32 (const void *bf16, void *fp32, uint64_t nb_elements) — bf16 → fp32
  • __rte_experimental int rte_ml_io_quantize (int16_t dev_id, uint16_t model_id, struct rte_ml_buff_seg **dbuffer, struct rte_ml_buff_seg **qbuffer) — take that dequantized dbuffer and quantize it into qbuffer for the model
  • __rte_experimental int rte_ml_io_dequantize (int16_t dev_id, uint16_t model_id, struct rte_ml_buff_seg **qbuffer, struct rte_ml_buff_seg **dbuffer) — unbaking the quantized qbuffer back into dbuffer
  • __rte_experimental struct rte_mempool * rte_ml_op_pool_create (const char *name, unsigned int nb_elts, unsigned int cache_size, uint16_t user_size, int socket_id) — make a mempool for ml ops, with a name, nb_elts, cache_size, user_size, and socket_id — returns the mempool ⭐
  • __rte_experimental void rte_ml_op_pool_free (struct rte_mempool *mempool) — free that op pool, thanks for your service 🫡

and that's set-up + the whole function roster bestie. do not go anywhere — part 2 is coming and there's so much more to unpack 💀🧚

💅 Detailed Description 💅

Warning ⚠️ EXPERIMENTAL: Every single function in this file may be changed or removed without prior notice. Bestie, do not build your whole personality on this API. It's giving "situationship", not "married". 💀

ML (Machine Learning) device API. Yes, the networking people also do machine learning now. We contain multitudes. ✨

The ML framework is built on the following model:

+-----------------+               rte_ml_[en|de]queue_burst()
|                 |                          |
|     Machine     o------+     +--------+    |
|     Learning    |      |     | queue  |    |    +------+
|     Inference   o------+-----o        |<===o===>|Core 0|
|     Engine      |      |     | pair 0 |         +------+
|                 o----+ |     +--------+
|                 |    | |
+-----------------+    | |     +--------+
         ^             | |     | queue  |         +------+
         |             | +-----o        |<=======>|Core 1|
         |             |       | pair 1 |         +------+
         |             |       +--------+
+--------+--------+    |
| +-------------+ |    |       +--------+
| |   Model 0   | |    |       | queue  |         +------+
| +-------------+ |    +-------o        |<=======>|Core N|
| +-------------+ |            | pair N |         +------+
| |   Model 1   | |            +--------+
| +-------------+ |
| +-------------+ |<------> rte_ml_model_load()
| |   Model ..  | |-------> rte_ml_model_info_get()
| +-------------+ |<------- rte_ml_model_start()
| +-------------+ |<------- rte_ml_model_stop()
| |   Model N   | |<------- rte_ml_model_params_update()
| +-------------+ |<------- rte_ml_model_unload()
+-----------------+

ML Device: a hardware- or software-based implementation of the ML device API, there to run inferences with a pre-trained ML model. That's it, that's the whole thing. 🫶

ML Model: an ML model is an algorithm that got trained over a dataset. A model is made of the procedure/algorithm and the data/pattern you need to actually make predictions on live data. Once the model is created and trained outside of the DPDK scope (not our problem, not our vibe), the model can be loaded via rte_ml_model_load() and then started using the rte_ml_model_start() API. The rte_ml_model_params_update() can be used to update the model parameters — think weight and bias — without unloading the model with rte_ml_model_unload(). Iconic, honestly. 💅

ML Inference: the process of feeding data into the model via rte_ml_enqueue_burst() and using rte_ml_dequeue_burst() to grab the calculated outputs/predictions back out of the started model. That's the whole loop. Slay. 🔥

In all functions of the ML device API, the ML device is designated by an integer >= 0 named as device identifier dev_id.

The functions exported by the ML device API to set up a device (designated by its device identifier) must be invoked in this exact order, no skipping, no shuffling:

  • rte_ml_dev_configure()
  • rte_ml_dev_queue_pair_setup()
  • rte_ml_dev_start()

A model is required to run the inference operations with the user-specified inputs. The application needs to invoke the ML model API in this order before queueing inference jobs:

  • rte_ml_model_load()
  • rte_ml_model_start()

A model can be loaded on a device only after the device has been configured, and can be started or stopped only after the device has been started. Order matters, bestie. It always does. 🎀

The rte_ml_model_info_get() API exists to retrieve information related to the model. That info would include the shape and type of input and output required for the inference. Know before you go, iykyk. 👑

Data quantization and dequantization is one of the main aspects in the ML domain. It involves converting input data from a higher precision to a lower precision data type, and vice-versa for the output. APIs are provided to enable quantization through rte_ml_io_quantize() and dequantization through rte_ml_io_dequantize(). These APIs have the capability to handle input and output buffers holding data for multiple batches. Multi-batch? We stay fed. 🍓

Two utility APIs — rte_ml_io_input_size_get() and rte_ml_io_output_size_get() — can be used to get the size of quantized and de-quantized multi-batch input and output buffers.

The user can optionally update the model parameters with rte_ml_model_params_update() after invoking the rte_ml_model_stop() API on a given model ID. Optional. No pressure. ✨

The application can invoke, in any order, the functions exported by the ML API to enqueue inference jobs and dequeue inference response.

If the application wants to change the device configuration (i.e., call rte_ml_dev_configure() or rte_ml_dev_queue_pair_setup()), then the application must stop the device using the rte_ml_dev_stop() API. Likewise, if model parameters need to be updated then the application must call rte_ml_model_stop() followed by the rte_ml_model_params_update() API for the given model. The application does not need to call rte_ml_dev_stop() API for any model re-configuration such as rte_ml_model_params_update(), rte_ml_model_unload(), etc. Chill, you don't have to nuke the whole device for that. 🌈

Once the device is in the start state after invoking the rte_ml_dev_start() API, and the model is in the start state after invoking the rte_ml_model_start() API, then the application can call rte_ml_enqueue_burst() and rte_ml_dequeue_burst() API on the destined device and model ID.

Finally, an application can close an ML device by invoking the rte_ml_dev_close() function. Clean exit, we love to see it. 👋

A typical application utilisation of the ML API will follow this programming flow:

  • rte_ml_dev_configure()
  • rte_ml_dev_queue_pair_setup()
  • rte_ml_model_load()
  • rte_ml_dev_start()
  • rte_ml_model_start()
  • rte_ml_model_info_get()
  • rte_ml_enqueue_burst()
  • rte_ml_dequeue_burst()
  • rte_ml_model_stop()
  • rte_ml_model_unload()
  • rte_ml_dev_stop()
  • rte_ml_dev_close()

Regarding multi-threading: by default, all the functions of the ML Device API exported by a PMD are lock-free functions, which assume they are not invoked in parallel on different logical cores on the same target object. For instance, the dequeue function of a poll mode driver cannot be invoked in parallel on two logical cores to operate on the same queue pair. Of course, this function can be invoked in parallel by different logical cores on different queue pairs. It is the responsibility of the user application to enforce this rule. Read that again, bestie — you are the lock. There is no lock. 💀

Definition in file rte_mldev.h.

✨ Macro Definition Documentation ✨

#define RTE_ML_STR_MAX 128

Maximum length of name string. That's it. 128 characters of name, max. Make 'em count. 🎀

Definition at line 152 of file rte_mldev.h.

💖 Typedef Documentation 💖

typedef void(* rte_ml_dev_stop_flush_t) (int16_t dev_id, uint16_t qp_id, struct rte_ml_op *op)

Callback function called during rte_ml_dev_stop(), invoked once per flushed ML op. So when the device is stopping and it's sweeping up leftover ops, this little guy gets pinged for every single one. Don't say we didn't warn you. 🫶

Definition at line 302 of file rte_mldev.h.

🔮 Enumeration Type Documentation 🔮

enum rte_ml_op_status

Status of ML operation. Because you deserve to know how it went. 💅

Enumerator

  • RTE_ML_OP_STATUS_SUCCESS — Operation completed successfully. Ate. No crumbs. ✨
  • RTE_ML_OP_STATUS_NOT_PROCESSED — Operation has not yet been processed by the device. It's giving "in my queue era". ⏳
  • RTE_ML_OP_STATUS_ERROR — Operation completed with error. The application can invoke rte_ml_op_error_get() to get the PMD-specific error code if needed. Not the vibe, but at least there's a lookup. 😭

Definition at line 404 of file rte_mldev.h.

enum rte_ml_dev_xstats_mode

Selects which component of the mldev to retrieve statistics from.

Enumerator

  • RTE_ML_DEV_XSTATS_DEVICE — Device xstats 👑
  • RTE_ML_DEV_XSTATS_MODEL — Model xstats 🦋

Definition at line 632 of file rte_mldev.h.

enum rte_ml_io_type

Input and output data types. ML models can operate on reduced-precision datatypes to achieve better power efficiency, lower network latency and lower memory footprint. This enum is used to represent the lower-precision integer and floating point types used by ML models. Small numbers, huge main character energy. ⭐

Enumerator

  • RTE_ML_IO_TYPE_UNKNOWN — Invalid or unknown type 🙈
  • RTE_ML_IO_TYPE_INT8 — 8-bit integer
  • RTE_ML_IO_TYPE_UINT8 — 8-bit unsigned integer
  • RTE_ML_IO_TYPE_INT16 — 16-bit integer
  • RTE_ML_IO_TYPE_UINT16 — 16-bit unsigned integer
  • RTE_ML_IO_TYPE_INT32 — 32-bit integer
  • RTE_ML_IO_TYPE_UINT32 — 32-bit unsigned integer
  • RTE_ML_IO_TYPE_INT64 — 32-bit integer
  • RTE_ML_IO_TYPE_UINT64 — 32-bit unsigned integer
  • RTE_ML_IO_TYPE_FP8 — 8-bit floating point number
  • RTE_ML_IO_TYPE_FP16 — IEEE 754 16-bit floating point number
  • RTE_ML_IO_TYPE_FP32 — IEEE 754 32-bit floating point number
  • RTE_ML_IO_TYPE_BFLOAT16 — 16-bit brain floating point number.

Definition at line 875 of file rte_mldev.h.

Okay bestie, not me noticing that RTE_ML_IO_TYPE_INT64 says "32-bit integer" and RTE_ML_IO_TYPE_UINT64 says "32-bit unsigned integer". That's what the file says, and we report exactly what the file says, so here it stays, character for character. Fr fr. 💀

enum rte_ml_io_layout

ML I/O buffer layout. How do you want your data served? 🍓

Enumerator

  • RTE_ML_IO_LAYOUT_PACKED — All inputs for the model should be packed in a single buffer with no padding between individual inputs. The buffer is expected to be aligned to rte_ml_dev_info::align_size.

When I/O segmentation is supported by the device, the packed data can be split into multiple segments. In this case, each segment is expected to be aligned to rte_ml_dev_info::align_size.

Same applies to output.

See also: struct rte_ml_dev_info::max_segments

  • RTE_ML_IO_LAYOUT_SPLIT — Each input for the model should be stored as separate buffers, and each input should be aligned to rte_ml_dev_info::align_size.

When I/O segmentation is supported, each input can be split into multiple segments. In this case, each segment is expected to be aligned to rte_ml_dev_info::align_size.

Same applies to output.

See also: struct rte_ml_dev_info::max_segments

Definition at line 905 of file rte_mldev.h.

🛠️ Function Documentation 🛠️

__rte_experimental int rte_ml_dev_init (size_t dev_max)

Maximum number of devices if rte_ml_dev_init() is not called. Initialize the device array before probing devices. If it's not called, the first device probed would initialize the array to a size of RTE_MLDEV_DEFAULT_MAX. She's self-starting like that. 💅

Parameters

  • dev_max — Maximum number of devices.

Returns

  • 0 on success, -rte_errno otherwise:
    • ENOMEM if out of memory
    • EINVAL if 0 size
    • EBUSY if already initialized

__rte_experimental uint16_t rte_ml_dev_count (void )

Get the total number of ML devices that have been successfully initialised. Roll call. 📣

Returns

  • The total number of usable ML devices.

__rte_experimental int rte_ml_dev_is_valid_dev (int16_t dev_id)

Check if the device is in ready state.

Parameters

  • dev_id — The identifier of the device.

Returns

  • 0 if device state is not in ready state.
  • 1 if device state is ready state.

Vibe check, but for a device. ✨

__rte_experimental int rte_ml_dev_socket_id (int16_t dev_id)

Return the NUMA socket to which a device is connected. Where does this device live? Real estate question. 🏠

Parameters

  • dev_id — The identifier of the device.

Returns

  • The NUMA socket id to which the device is connected
  • 0 If the socket could not be determined.
  • -EINVAL: if the dev_id value is not valid.

rte_ml_dev_info_get() — get the deets on your ML device 💅

__rte_experimental int rte_ml_dev_info_get (int16_t dev_id, struct rte_ml_dev_info * dev_info)

Okay bestie, so this one is literally just here to retrieve the information of the device. It's giving you the whole rundown, like a little info dump. 📋✨

Parameters:

  • dev_id — the identifier of the device. It's the device's little ID number, you know?
  • dev_info — a pointer to a structure of type rte_ml_dev_info to be filled with the info of the device. So you pass in a pointer, and it gets filled with all the juicy details.

Returns:

  • 0: Success, driver updates the information of the ML device. Slay! 💖
  • < 0: Error code returned by the driver info get function. Oops, something went wrong. 😭

rte_ml_dev_configure() — set up your ML device, periodt 💅

__rte_experimental int rte_ml_dev_configure (int16_t dev_id, const struct rte_ml_dev_config * config)

This function must be invoked first before any other function in the API. Like, it's the main character, the first step. No cap. 🔥

ML Device can be re-configured, when in a stopped state. Device cannot be re-configured after rte_ml_dev_close() is called. So don't even try it after closing, okay? 🙅‍♀️

The caller may use rte_ml_dev_info_get() to get the capability of each resources available for this ML device. So you can check what it can do first. 💁‍♀️

Parameters:

  • dev_id — the identifier of the device to configure.
  • config — the ML device configuration structure.

Returns:

  • 0: Success, device configured. Yasss! ✨
  • < 0: Error code returned by the driver configuration function. Big yikes. 😭

rte_ml_dev_queue_pair_count() — how many queue pairs? 🔢

__rte_experimental uint16_t rte_ml_dev_queue_pair_count (int16_t dev_id)

Get the number of queue pairs on a specific ML device. It's like asking "how many lanes we got?" 🛣️

Parameters:

  • dev_id — the identifier of the device.

Returns:

  • The number of configured queue pairs. Periodt. 💅

rte_ml_dev_queue_pair_setup() — set up a queue pair 💅

__rte_experimental int rte_ml_dev_queue_pair_setup (int16_t dev_id, uint16_t queue_pair_id, const struct rte_ml_dev_qp_conf * qp_conf, int socket_id)

Set up a queue pair for a device. This should only be called when the device is stopped. So like, don't do it while it's running, okay? 🙈

Parameters:

  • dev_id — the identifier of the device.
  • queue_pair_id — the index of the queue pairs to set up. The value must be in the range [0, nb_queue_pairs - 1] previously supplied to rte_ml_dev_configure().
  • qp_conf — the pointer to the configuration data to be used for the queue pair.
  • socket_id — the socket_id argument is the socket identifier in case of NUMA. The value can be SOCKET_ID_ANY if there is no NUMA constraint for the memory allocated for the queue pair.

Returns:

  • 0: Success, queue pair correctly set up. Slay! ✨
  • < 0: Queue pair configuration failed. Sad. 😭

rte_ml_dev_start() — start that device! 🔥

__rte_experimental int rte_ml_dev_start (int16_t dev_id)

Start an ML device. The device start step consists of setting the configured features and enabling the ML device to accept inference jobs. So it's like, "go, go, go!" 🏁

Parameters:

  • dev_id — the identifier of the device.

Returns:

  • 0: Success, device started. We love to see it. 💖
  • <0: Error code of the driver device start function. Oops. 😭

rte_ml_dev_stop() — stop, in the name of love ✋

__rte_experimental int rte_ml_dev_stop (int16_t dev_id)

Stop an ML device. A stopped device cannot accept inference jobs. The device can be restarted with a call to rte_ml_dev_start(). So it's like a pause, not a forever goodbye. ⏸️

Parameters:

  • dev_id — the identifier of the device.

Returns:

  • 0: Success, device stopped. Chill. 🧊
  • <0: Error code of the driver device stop function. Yikes. 😭

rte_ml_dev_close() — close it, and it's over 💀

__rte_experimental int rte_ml_dev_close (int16_t dev_id)

Close an ML device. The device cannot be restarted! So this is the final boss. No coming back. 🚫

Warning: The device cannot be restarted! Once you close it, it's done. No cap. 💀

Parameters:

  • dev_id — the identifier of the device.

Returns:

  • 0 on successfully closing device. Bye bestie. 👋
  • <0 on failure to close device. 😭

rte_ml_enqueue_burst() — enqueue a burst of inferences 🚀

__rte_experimental uint16_t rte_ml_enqueue_burst (int16_t dev_id, uint16_t qp_id, struct rte_ml_op ** ops, uint16_t nb_ops)

Enqueue a burst of ML inferences for processing on an ML device. It's like sending a whole squad of jobs to the device. 💅

The rte_ml_enqueue_burst() function is invoked to place ML inference operations on the queue qp_id of the device designated by its dev_id.

The nb_ops parameter is the number of inferences to process which are supplied in the ops array of rte_ml_op structures.

The rte_ml_enqueue_burst() function returns the number of inferences it actually enqueued for processing. A return value equal to nb_ops means that all inferences have been enqueued. So if it returns nb_ops, you're all good. If not, some got left out. 😭

Parameters:

  • dev_id — the identifier of the device.
  • qp_id — the index of the queue pair which inferences are to be enqueued for processing. The value must be in the range [0, nb_queue_pairs - 1] previously supplied to rte_ml_dev_configure.
  • ops — the address of an array of nb_ops pointers to rte_ml_op structures which contain the ML inferences to be processed.
  • nb_ops — the number of operations to process.

Returns:

The number of inference operations actually enqueued to the ML device. The return value can be less than the value of the nb_ops parameter when the ML device queue is full or if invalid parameters are specified in a rte_ml_op.

rte_ml_dequeue_burst() — dequeue that burst! 📥

__rte_experimental uint16_t rte_ml_dequeue_burst (int16_t dev_id, uint16_t qp_id, struct rte_ml_op ** ops, uint16_t nb_ops)

Dequeue a burst of processed ML inferences operations from a queue on the ML device. The dequeued operations are stored in rte_ml_op structures whose pointers are supplied in the ops array. So it's like collecting the finished jobs. 🎉

The rte_ml_dequeue_burst() function returns the number of inferences actually dequeued, which is the number of rte_ml_op data structures effectively supplied into the ops array.

A return value equal to nb_ops indicates that the queue contained at least nb_ops operations, and this is likely to signify that other processed operations remain in the devices output queue. Application implementing a "retrieve as many processed operations as possible" policy can check this specific case and keep invoking the rte_ml_dequeue_burst() function until a value less than nb_ops is returned. So keep going until you get less than nb_ops. 🔁

The rte_ml_dequeue_burst() function does not provide any error notification to avoid the corresponding overhead. So no error messages, just vibes. 🙈

Parameters:

  • dev_id — the identifier of the device.
  • qp_id — the index of the queue pair from which to retrieve processed inferences. The value must be in the range [0, nb_queue_pairs - 1] previously supplied to rte_ml_dev_configure().
  • ops — the address of an array of pointers to rte_ml_op structures that must be large enough to store nb_ops pointers in it.
  • nb_ops — the maximum number of inferences to dequeue.

Returns:

The number of operations actually dequeued, which is the number of pointers to rte_ml_op structures effectively supplied to the ops array.

rte_ml_op_error_get() — get the error deets on an op 😭

__rte_experimental int rte_ml_op_error_get (int16_t dev_id, struct rte_ml_op * op, struct rte_ml_op_error * error)

Get PMD specific error information for an ML op. When an ML operation completed with RTE_ML_OP_STATUS_ERROR as status, This API allows to get PMD specific error details. So if your op failed, this tells you why. 🔍

Parameters:

  • dev_id — Device identifier
  • op — Handle of ML operation
  • error — Address of structure rte_ml_op_error to be filled

Returns:

  • Returns 0 on success. ✨
  • Returns negative value on failure. 😭

rte_ml_dev_stats_get() — get those stats! 📊

__rte_experimental int rte_ml_dev_stats_get (int16_t dev_id, struct rte_ml_dev_stats * stats)

Retrieve the general I/O statistics of a device. Like, how many things went in and out? 📈

Parameters:

  • dev_id — the identifier of the device.
  • stats — Pointer to structure to where statistics will be copied. On error, this location may or may not have been modified. So be careful, it might be messy. 🙈

Returns:

  • 0 on success. 💅
  • -EINVAL: If invalid parameter pointer is provided. Oops. 😭

rte_ml_dev_stats_reset() — reset those stats 🔄

__rte_experimental void rte_ml_dev_stats_reset (int16_t dev_id)

Reset the statistics of a device. Like, clear the slate. 🧼

Parameters:

  • dev_id — the identifier of the device.

rte_ml_dev_xstats_names_get() — get the names of extended stats 📝

__rte_experimental int rte_ml_dev_xstats_names_get (int16_t dev_id, enum rte_ml_dev_xstats_mode mode, int32_t model_id, struct rte_ml_dev_xstats_map * xstats_map, uint32_t size)

Retrieve names of extended statistics of an ML device. It's like getting the labels for all the extra stats. 🏷️

Parameters:

  • dev_id — the identifier of the device.
  • mode — Mode of statistics to retrieve. Choices include the device statistics and model statistics.
  • model_id — Used to specify the model number in model mode, and is ignored in device mode.
  • xstats_map — Block of memory to insert names and ids into. Must be at least size in capacity. If set to NULL, function returns required capacity. The id values returned can be passed to rte_ml_dev_xstats_get to select statistics.
  • size — Capacity of xstats_names (number of xstats_map).

Returns:

  • Positive value lower or equal to size: success. The return value is the number of entries filled in the stats table. Yay! ✨
  • Positive value higher than size: error, the given statistics table is too small. The return value corresponds to the size that should be given to succeed. The entries in the table are not valid and shall not be used by the caller. So if you get this, you need a bigger table. 😭
  • Negative value on error: -ENODEV for invalid dev_id. -EINVAL for invalid mode, model parameters. -ENOTSUP if the device doesn't support this function.

rte_ml_dev_xstats_by_name_get() — get a single stat by name 🔍

__rte_experimental int rte_ml_dev_xstats_by_name_get (int16_t dev_id, const char * name, uint16_t * stat_id, uint64_t * value)

Retrieve the value of a single stat by requesting it by name. Like, "hey, what's the value of this specific stat?" 💅

Parameters:

  • dev_id — the identifier of the device.
  • name — Name of stat name to retrieve.
  • stat_id — If non-NULL, the numerical id of the stat will be returned, so that further requests for the stat can be got using rte_ml_dev_xstats_get, which will be faster as it doesn't need to scan a list of names for the stat. If the stat cannot be found, the id returned will be (unsigned)-1.
  • value — Value of the stat to be returned.

Returns:

  • Zero: No error. 😌
  • Negative value: -EINVAL if stat not found, -ENOTSUP if not supported. 😭

rte_ml_dev_xstats_get() — get extended stats 📊

__rte_experimental int rte_ml_dev_xstats_get (int16_t dev_id, enum rte_ml_dev_xstats_mode mode, int32_t model_id, const uint16_t stat_ids[], uint64_t values[], uint16_t nb_ids)

Retrieve extended statistics of an ML device. It's like getting the values for a bunch of stats by their IDs. 🔢

Parameters:

  • dev_id — the identifier of the device.
  • mode — Mode of statistics to retrieve. Choices include the device statistics and model statistics.
  • model_id — Used to specify the model id in model mode, and is ignored in device mode.
  • stat_ids — ID numbers of the stats to get. The ids can be got from the stat position in the stat list from rte_ml_dev_xstats_names_get(), or by using rte_ml_dev_xstats_by_name_get().
  • values — Values for each stats request by ID.
  • nb_ids — Number of stats requested.

Returns:

  • Positive value: number of stat entries filled into the values array. Slay! 💖
  • Negative value on error: -ENODEV for invalid dev_id. -EINVAL for invalid mode, model id or stat id parameters. -ENOTSUP if the device doesn't support this function. 😭

rte_ml_dev_xstats_reset() — reset those xstats 🔄

__rte_experimental int rte_ml_dev_xstats_reset (int16_t dev_id, enum rte_ml_dev_xstats_mode mode, int32_t model_id, const uint16_t stat_ids[], uint16_t nb_ids)

Reset the values of the xstats of the selected component in the device.

  • dev_id — bestie, this is just the identifier of the device. that's it. that's the whole vibe. 🎀
  • mode — what kind of stats you're nuking. pick from device or model. choose wisely, we're not taking it back.
  • model_id — which model's stats are getting reset. 0 and any positive value = go get that specific model, while -1 = every single model, no survivors 💀
  • stat_ids — if you only want some specific stats gone, point this at them. when it's NULL, everything selected by mode gets reset. if it's not NULL, it has to point to an array of at least nb_ids in size. no shortcuts.
  • nb_ids — how many ids you actually stuffed into that array. and heads up: this gets ignored when ids is NULL (yes, the original says ids here, we don't make the rules 🙈).

Returns like:

  • Zero — stats reset, slayed, done. ✨
  • Negative value — -EINVAL if your parameters are invalid, or -ENOTSUP if this device simply does not support it. not her fault.

🗑️ __rte_experimental int rte_ml_dev_dump (int16_t dev_id, FILE * fd)

Dump all the internal tea about dev_id into the FILE* you hand over in fd. it's giving "spill everything to a file". ☕

Parameters

  • dev_id — the identifier of the device.
  • fd — a pointer to a file for output.

Returns

  • 0 — on success. 👑
  • <0 — on failure. 😭

🪞 __rte_experimental int rte_ml_dev_selftest (int16_t dev_id)

Trigger the ML device self test. it's literally looking in the mirror and checking itself out.

Parameters

  • dev_id — the identifier of the device.

Returns

  • 0 — Selftest successful. 💅
  • -ENOTSUP — if the device doesn't support selftest. not every queen does the full routine.
  • other values < 0 — on failure.

📥 __rte_experimental int rte_ml_model_load (int16_t dev_id, struct rte_ml_model_params * params, uint16_t * model_id)

Load an ML model onto the device. 🦋

She loads an ML model to the device with the parameters you requested in the structure rte_ml_model_params.

Parameters

  • dev_id — the identifier of the device.
  • params — parameters for the model to be loaded.
  • model_id — identifier of the model loaded.

Returns

  • 0 — Success, Model loaded. main character behavior.
  • < 0 — Failure, Error code of the model load driver function.

📤 __rte_experimental int rte_ml_model_unload (int16_t dev_id, uint16_t model_id)

Unload an ML model from the device. saying goodbye, respectfully. 🫶

Parameters

  • dev_id — the identifier of the device.
  • model_id — identifier of the model to be unloaded.

Returns

  • 0 — Success, Model unloaded.
  • < 0 — Failure, Error code of the model unload driver function.

▶️ __rte_experimental int rte_ml_model_start (int16_t dev_id, uint16_t model_id)

Start an ML model for the given device ID. ✨

Start an ML model so it can accept inference requests. she's open for business, bestie.

Parameters

  • dev_id — the identifier of the device.
  • model_id — identifier of the model to be started.

Returns

  • 0 — Success, Model started. 🔥
  • < 0 — Failure, Error code of the model start driver function.

⏸️ __rte_experimental int rte_ml_model_stop (int16_t dev_id, uint16_t model_id)

Stop an ML model for the given device ID.

Model stop would disable the ML model from being used for inference jobs. and lowkey the important part: all inference jobs must have been completed before model stop is attempted. no yanking the plug mid-sentence, fr.

Parameters

  • dev_id — the identifier of the device.
  • model_id — identifier of the model to be stopped.

Returns

  • 0 — Success, Model stopped.
  • < 0 — Failure, Error code of the model stop driver function.

🔍 __rte_experimental int rte_ml_model_info_get (int16_t dev_id, uint16_t model_id, struct rte_ml_model_info * model_info)

Get ML model information. the ✨receipts✨.

Parameters

  • dev_id — the identifier of the device.
  • model_id — identifier for the model created
  • model_info — pointer to a model info structure

Returns

  • Returns 0 on success
  • Returns negative value on failure

🔧 __rte_experimental int rte_ml_model_params_update (int16_t dev_id, uint16_t model_id, void * buffer)

Update the model parameters without unloading the model. glow-up in place, no reload needed. 💅

Update model parameters such as weights and bias without unloading the model. and listen: rte_ml_model_stop() must be called before invoking this API. no cap.

Parameters

  • dev_id — the identifier of the device.
  • model_id — identifier for the model created
  • buffer — pointer to the model weights and bias buffer. Size of the buffer is equal to wb_size returned in rte_ml_model_info.

Returns

  • Returns 0 on success
  • Returns negative value on failure

🌸 __rte_experimental int rte_ml_io_float32_to_int8 (const void * fp32, void * i8, uint64_t nb_elements, float scale, int8_t zero_point)

Convert a buffer containing numbers in single precision floating format (float32) to signed 8-bit integer format (INT8). glow-down, i guess. 🦋

Parameters

  • fp32 — input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • i8 — output buffer to store INT8 numbers. Size of buffer is equal to (nb_elements * 1) bytes.
  • nb_elements — number of elements in the buffer.
  • scale — scale factor for conversion.
  • zero_point — zero point for conversion.

Returns

  • 0 — Success.
  • < 0 — Error code on failure.

🌸 __rte_experimental int rte_ml_io_int8_to_float32 (const void * i8, void * fp32, uint64_t nb_elements, float scale, int8_t zero_point)

Convert a buffer containing numbers in signed 8-bit integer format (INT8) to single precision floating format (float32). she's upgrading, periodt.

Parameters

  • i8 — input buffer containing INT8 numbers. Size of buffer is equal to (nb_elements * 1) bytes.
  • fp32 — output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • nb_elements — number of elements in the buffer.
  • scale — scale factor for conversion.
  • zero_point — zero point for conversion.

Returns

  • 0 — Success.
  • < 0 — Error code on failure.

🎀 __rte_experimental int rte_ml_io_float32_to_uint8 (const void * fp32, void * ui8, uint64_t nb_elements, float scale, uint8_t zero_point)

Convert a buffer containing numbers in single precision floating format (float32) to unsigned 8-bit integer format (UINT8). keepin' it positive ✨.

Parameters

  • fp32 — input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • ui8 — output buffer to store UINT8 numbers. Size of buffer is equal to (nb_elements * 1) bytes.
  • nb_elements — number of elements in the buffer.
  • scale — scale factor for conversion.
  • zero_point — zero point for conversion.

Returns

  • 0 — Success.
  • < 0 — Error code on failure.

🎀 __rte_experimental int rte_ml_io_uint8_to_float32 (const void * ui8, void * fp32, uint64_t nb_elements, float scale, uint8_t zero_point)

Convert a buffer containing numbers in unsigned 8-bit integer format (UINT8) to single precision floating format (float32).

Parameters

  • ui8 — input buffer containing UINT8 numbers. Size of buffer is equal to (nb_elements * 1) bytes.
  • fp32 — output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • nb_elements — number of elements in the buffer.
  • scale — scale factor for conversion.
  • zero_point — zero point for conversion.

Returns

  • 0 — Success.
  • < 0 — Error code on failure.

💜 __rte_experimental int rte_ml_io_float32_to_int16 (const void * fp32, void * i16, uint64_t nb_elements, float scale, int16_t zero_point)

Convert a buffer containing numbers in single precision floating format (float32) to signed 16-bit integer format (INT16).

Parameters

  • fp32 — input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • i16 — output buffer to store INT16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
  • nb_elements — number of elements in the buffer.
  • scale — scale factor for conversion.
  • zero_point — zero point for conversion.

Returns

  • 0 — Success.
  • < 0 — Error code on failure.

💜 __rte_experimental int rte_ml_io_int16_to_float32 (const void * i16, void * fp32, uint64_t nb_elements, float scale, int16_t zero_point)

Convert a buffer containing numbers in signed 16-bit integer format (INT16) to single precision floating format (float32).

Parameters

  • i16 — input buffer containing INT16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
  • fp32 — output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • nb_elements — number of elements in the buffer.
  • scale — scale factor for conversion.
  • zero_point — zero point for conversion.

Returns

  • 0 — Success.
  • < 0 — Error code on failure.

🧚 __rte_experimental int rte_ml_io_float32_to_uint16 (const void * fp32, void * ui16, uint64_t nb_elements, float scale, uint16_t zero_point)

Convert a buffer containing numbers in single precision floating format (float32) to unsigned 16-bit integer format (UINT16).

Parameters

  • fp32 — input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • ui16 — output buffer to store UINT16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
  • nb_elements — number of elements in the buffer.
  • scale — scale factor for conversion.
  • zero_point — zero point for conversion.

Returns

  • 0 — Success.
  • < 0 — Error code on failure.

🧚 __rte_experimental int rte_ml_io_uint16_to_float32 (const void * ui16, void * fp32, uint64_t nb_elements, float scale, uint16_t zero_point)

Convert a buffer containing numbers in unsigned 16-bit integer format (UINT16) to single precision floating format (float32).

Parameters

  • ui16 — input buffer containing UINT16 numbers. Size of buffer is equal to (nb_elements * 2) bytes.
  • fp32 — output buffer to store float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • nb_elements — number of elements in the buffer.
  • scale — scale factor for conversion.
  • zero_point — zero point for conversion.

Returns

  • 0 — Success.
  • < 0 — Error code on failure.

🌈 __rte_experimental int rte_ml_io_float32_to_int32 (const void * fp32, void * i32, uint64_t nb_elements, float scale, int32_t zero_point)

Convert a buffer containing numbers in single precision floating format (float32) to signed 32-bit integer format (INT32).

Parameters

  • fp32 — input buffer containing float32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • i32 — output buffer to store INT32 numbers. Size of buffer is equal to (nb_elements * 4) bytes.
  • nb_elements — number of elements in the buffer.
  • scale — scale factor for conversion.
  • zero_point — zero point for conversion.

Returns

  • 0 — Success.
  • < 0 — Error code on failure.

🌈 __rte_experimental int rte_ml_io_int32_to_float32 (const void * i32, void * fp32, uint64_t nb_elements, float scale, int32_t zero_point)

Convert a buffer containing numbers in signed 32-bit integer format (INT32) to single precision floating format (float32).

🌸 the i32 → fp32 one (finishing off what we started up there)

Parameters

  • i32 — input buffer stuffed with INT32 numbers, and the way the size math works is size of buffer == nb_elements * 4 bytes 🫶
  • fp32 — output buffer where the float32 numbers land, also sized nb_elements * 4 bytes 💖
  • nb_elements — how many elements are chilling in the buffer
  • scale — the scale factor for the conversion
  • zero_point — the zero point for the conversion

Returns

  • 0, Success. 🎀
  • < 0, error code on failure 😭

💅 rte_ml_io_float32_to_uint32

__rte_experimental int rte_ml_io_float32_to_uint32 (const void * fp32, void * ui32, uint64_t nb_elements, float scale, uint32_t zero_point)

Take a whole buffer of single precision floating point (float32) numbers and convert them into unsigned 32-bit integer (UINT32) format. Glow up, but in integers ✨

Parameters

  • fp32 — input buffer holding the float32 numbers, size equal to (nb_elements * 4) bytes
  • ui32 — output buffer to store the UINT32 numbers, size equal to (nb_elements * 4) bytes
  • nb_elements — Number of elements in the buffer.
  • scale — Scale factor for conversion.
  • zero_point — Zero point for conversion.

Returns

  • 0, Success.
  • < 0, Error code on failure.

🦋 rte_ml_io_uint32_to_float32

__rte_experimental int rte_ml_io_uint32_to_float32 (const void * ui32, void * fp32, uint64_t nb_elements, float scale, uint32_t zero_point)

The reverse glow up, bestie — buffer full of unsigned 32-bit integer (UINT32) numbers goes in, single precision floating point (float32) comes out 💜

Parameters

  • ui32 — input buffer holding the UINT32 numbers, size equal to (nb_elements * 4) bytes
  • fp32 — output buffer to store the float32 numbers, size equal to (nb_elements * 4) bytes
  • nb_elements — Number of elements in the buffer.
  • scale — Scale factor for conversion.
  • zero_point — Zero point for conversion.

Returns

  • 0, Success.
  • < 0, Error code on failure.

🌈 rte_ml_io_float32_to_int64

__rte_experimental int rte_ml_io_float32_to_int64 (const void * fp32, void * i64, uint64_t nb_elements, float scale, int64_t zero_point)

float32 in, signed 64-bit integer (INT64) out. Big numbers only, no cap fr fr 🔥

Parameters

  • fp32 — input buffer holding the float32 numbers, size equal to (nb_elements * 4) bytes
  • i64 — output buffer to store the INT64 numbers, size equal to (nb_elements * 4) bytes
  • nb_elements — Number of elements in the buffer.
  • scale — Scale factor for conversion.
  • zero_point — Zero point for conversion.

Returns

  • 0, Success.
  • < 0, Error code on failure.

🧚 rte_ml_io_int64_to_float32

__rte_experimental int rte_ml_io_int64_to_float32 (const void * i64, void * fp32, uint64_t nb_elements, float scale, int64_t zero_point)

Signed 64-bit integer (INT64) numbers go in, single precision floating point (float32) comes out. it's giving ✨precision✨

Parameters

  • i64 — input buffer holding the INT64 numbers, size equal to (nb_elements * 4) bytes
  • fp32 — output buffer to store the float32 numbers, size equal to (nb_elements * 4) bytes
  • nb_elements — Number of elements in the buffer.
  • scale — Scale factor for conversion.
  • zero_point — Zero point for conversion.

Returns

  • 0, Success.
  • < 0, Error code on failure.

🍓 rte_ml_io_float32_to_uint64

__rte_experimental int rte_ml_io_float32_to_uint64 (const void * fp32, void * ui64, uint64_t nb_elements, float scale, uint64_t zero_point)

float32 → unsigned 64-bit integer (UINT64). Same energy as the other 64-bit one but this time we're staying positive 🫶

Parameters

  • fp32 — input buffer holding the float32 numbers, size equal to (nb_elements * 4) bytes
  • ui64 — output buffer to store the UINT64 numbers, size equal to (nb_elements * 4) bytes
  • nb_elements — Number of elements in the buffer.
  • scale — Scale factor for conversion.
  • zero_point — Zero point for conversion.

Returns

  • 0, Success.
  • < 0, Error code on failure.

👑 rte_ml_io_uint64_to_float32

__rte_experimental int rte_ml_io_uint64_to_float32 (const void * ui64, void * fp32, uint64_t nb_elements, float scale, uint64_t zero_point)

Unsigned 64-bit integer (UINT64) buffer in, single precision floating point (float32) buffer out. iconic behaviour honestly 💅

Parameters

  • ui64 — input buffer holding the UINT64 numbers, size equal to (nb_elements * 4) bytes
  • fp32 — output buffer to store the float32 numbers, size equal to (nb_elements * 4) bytes
  • nb_elements — Number of elements in the buffer.
  • scale — Scale factor for conversion.
  • zero_point — Zero point for conversion.

Returns

  • 0, Success.
  • < 0, Error code on failure.

💖 rte_ml_io_float32_to_float16

__rte_experimental int rte_ml_io_float32_to_float16 (const void * fp32, void * fp16, uint64_t nb_elements)

Squish single precision floating point (float32) down into half precision floating point (FP16). Smaller. Cuter. Still floating point 🎀

Parameters

  • fp32 — input buffer holding the float32 numbers, size equal to (nb_elements * 4) bytes
  • fp16 — output buffer to store the float16 numbers, size equal to (nb_elements * 2) bytes
  • nb_elements — Number of elements in the buffer.

Returns

  • 0, Success.
  • < 0, Error code on failure.

🌸 rte_ml_io_float16_to_float32

__rte_experimental int rte_ml_io_float16_to_float32 (const void * fp16, void * fp32, uint64_t nb_elements)

Half precision floating point (FP16) in, single precision floating point (float32) out. Un-squishing the squish 🦋

Parameters

  • fp16 — input buffer holding the float16 numbers, size equal to (nb_elements * 2) bytes
  • fp32 — output buffer to store the float32 numbers, size equal to (nb_elements * 4) bytes
  • nb_elements — Number of elements in the buffer.

Returns

  • 0, Success.
  • < 0, Error code on failure.

⭐ rte_ml_io_float32_to_bfloat16

__rte_experimental int rte_ml_io_float32_to_bfloat16 (const void * fp32, void * bf16, uint64_t nb_elements)

float32 → brain floating point (bfloat16). Yes bestie, it's an actual format, and yes it's as unhinged as it sounds 💀

Parameters

  • fp32 — input buffer holding the float32 numbers, size equal to (nb_elements * 4) bytes
  • bf16 — output buffer to store the bfloat16 numbers, size equal to (nb_elements * 2) bytes
  • nb_elements — Number of elements in the buffer.

Returns

  • 0, Success.
  • < 0, Error code on failure.

🔥 rte_ml_io_bfloat16_to_float32

__rte_experimental int rte_ml_io_bfloat16_to_float32 (const void * bf16, void * fp32, uint64_t nb_elements)

Brain floating point (bfloat16) in, single precision floating point (float32) out. Bringing the brain cells back up to full size 🧠✨

Parameters

  • bf16 — input buffer holding the bfloat16 numbers, size equal to (nb_elements * 2) bytes
  • fp32 — output buffer to store the float32 numbers, size equal to (nb_elements * 4) bytes
  • nb_elements — Number of elements in the buffer.

Returns

  • 0, Success.
  • < 0, Error code on failure.

🎀 rte_ml_io_quantize

__rte_experimental int rte_ml_io_quantize (int16_t dev_id, uint16_t model_id, struct rte_ml_buff_seg ** dbuffer, struct rte_ml_buff_seg ** qbuffer)

Quantize input data, bestie 📉

Quantization converts data from higher precision types to lower precision types so the model execution gets better throughput and efficiency, with only minimal loss of accuracy. The types of dequantized data and quantized data are specified by the model — so that's the model's call, not yours, don't fight it 💅

Parameters

  • dev_id — The identifier of the device.
  • model_id — Identifier for the model
  • dbuffer — Address of dequantized input data
  • qbuffer — Address of quantized input data

Returns

  • Returns 0 on success
  • Returns negative value on failure

💜 rte_ml_io_dequantize

__rte_experimental int rte_ml_io_dequantize (int16_t dev_id, uint16_t model_id, struct rte_ml_buff_seg ** qbuffer, struct rte_ml_buff_seg ** dbuffer)

Dequantize output data 📈

Dequantization converts data from a lower precision type to a higher precision type. The types of quantized data and dequantized data are both specified by the model 🫶

Parameters

  • dev_id — The identifier of the device.
  • model_id — Identifier for the model
  • qbuffer — Address of quantized output data
  • dbuffer — Address of dequantized output data

Returns

  • Returns 0 on success
  • Returns negative value on failure

🌈 rte_ml_op_pool_create

__rte_experimental struct rte_mempool * rte_ml_op_pool_create (const char * name, unsigned int nb_elts, unsigned int cache_size, uint16_t user_size, int socket_id)

Create an ML operation pool 🏊‍♀️✨

Parameters

  • name — ML operations pool name
  • nb_elts — Number of elements in pool
  • cache_size — Number of elements to cache on lcore, see rte_mempool_create for further details about cache size
  • user_size — Size of private data to allocate for user with each operation
  • socket_id — Socket to identifier allocate memory on

Returns

  • On success pointer to mempool
  • On failure NULL

🤗 rte_ml_op_pool_free

__rte_experimental void rte_ml_op_pool_free (struct rte_mempool * mempool)

Free an ML operation pool 🫶 bye bestie

Parameters

  • mempool — A pointer to the mempool structure. If NULL then, the function does nothing.

💫 Author

Generated automatically by Doxygen for DPDK from the source code 🧚

okay that's a wrap on rte_mldev.h(3), bestie — this was Version 25.11.3 of the page, straight out of the DPDK source tree, and honestly? iconic. go build something unhinged with it 💅🎀💖

📦 where this page comes from