/build/sourc...erasurecode.c(3)

πŸ“¦
version 1.8.0 Β· 2 versions
, in nixpkgs unstable

πŸ¦‹ NAME πŸ¦‹

/build/source/src/erasurecode.c β€” that's the file, bestie. This is the liberasurecode manual page for it, and we are getting into it. πŸ‘‘


✨ SYNOPSIS ✨

First things first, the include stack. Every single one, verbatim, no cap:

#include <assert.h>
#include <zlib.h>
#include 'list.h'
#include 'erasurecode.h'
#include 'erasurecode_backend.h'
#include 'erasurecode_helpers.h'
#include 'erasurecode_helpers_ext.h'
#include 'erasurecode_preprocessing.h'
#include 'erasurecode_postprocessing.h'
#include 'erasurecode_stdinc.h'
#include 'alg_sig.h'
#include 'erasurecode_log.h'

Like, that's a lot of headers and honestly? Respect. 🫢

πŸ’« Functions πŸ’«

okay so here's the whole function menu, in order, with the exact signatures because we do not play about accuracy:

  • static SLIST_HEAD (backend_list, ec_backend) β€” Look up a backend instance by descriptor. (Yes that's literally what it says. Slay.)
  • static int liberasurecode_backend_alloc_desc (void) β€” Allocated backend instance descriptor. ✨
  • static void print_dlerror (const char *caller) β€” no description in the source page, she's mysterious like that πŸ’…
  • static void * liberasurecode_backend_open (ec_backend_t instance) β€” also no blurb, just vibes
  • static int liberasurecode_backend_close (ec_backend_t instance) β€” same, no description given
  • void __attribute__ ((constructor)) β€” no description, she just runs
  • void __attribute__ ((destructor)) β€” same energy, opposite end 🌈
  • int liberasurecode_backend_available (const ec_backend_id_t backend_id) β€” Checks if a given backend is available. That's it, that's the whole job πŸŽ€
  • int liberasurecode_instance_create (const ec_backend_id_t id, struct ec_args *args) β€” Create a liberasurecode instance and return a descriptor for use with EC operations (encode, decode, reconstruct). πŸ›
  • int liberasurecode_instance_destroy (int desc) β€” Close a liberasurecode instance. Bye bestie πŸ‘‹
  • int liberasurecode_encode_cleanup (int desc, char **encoded_data, char **encoded_parity) β€” Cleanup structures allocated by librasurecode_encode. (Yes, spelled like that in the original β€” we keep it exact, no editing people's typos 😭)
  • int liberasurecode_encode (int desc, const char *orig_data, uint64_t orig_data_size, char ***encoded_data, char ***encoded_parity, uint64_t *fragment_len) β€” Erasure encode a data buffer. This is THE main character right here πŸ”₯
  • int liberasurecode_decode_cleanup (int desc, char *data) β€” Cleanup structures allocated by librasurecode_decode. Same deal, same spelling, we don't touch it πŸ’€
  • int liberasurecode_decode (int desc, char **available_fragments, int num_fragments, uint64_t fragment_len, int force_metadata_checks, char **out_data, uint64_t *out_data_len) β€” Reconstruct original data from a set of k encoded fragments. Obsessed with her actually πŸ’–
  • int liberasurecode_reconstruct_fragment (int desc, char **available_fragments, int num_fragments, uint64_t fragment_len, int destination_idx, char *out_fragment) β€” Reconstruct a missing fragment from a subset of available fragments. The glow-up arc πŸ¦‹
  • int liberasurecode_fragments_needed (int desc, int *fragments_to_reconstruct, int *fragments_to_exclude, int *fragments_needed) β€” Return a list of lists with valid rebuild indexes given a list of missing indexes. Lists of lists, she's giving nested structure πŸ‘‘
  • int liberasurecode_get_fragment_metadata (char *fragment, fragment_metadata_t *fragment_metadata) β€” Get opaque metadata for a fragment. Opaque, meaning don't peek behind the curtain πŸ™ˆ
  • int is_invalid_fragment_header (fragment_header_t *header) β€” no description given, she just judges headers
  • int liberasurecode_verify_fragment_metadata (ec_backend_t be, fragment_metadata_t *md) β€” no description, but the name tells the story ngl
  • static int is_invalid_fragment_metadata (int desc, fragment_metadata_t *fragment_metadata) β€” no description, private girlie πŸ’…
  • int is_invalid_fragment (int desc, char *fragment) β€” no description, the fragment gets a vibe check ⭐
  • int liberasurecode_verify_stripe_metadata (int desc, char **fragments, int num_fragments) β€” no description, she verifies stripes, that's the tea
  • int liberasurecode_get_aligned_data_size (int desc, uint64_t data_len) β€” This computes the aligned size of a buffer passed into the encode function. Aligned is the whole point, okurrr πŸŽ€
  • int liberasurecode_get_minimum_encode_size (int desc) β€” This will return the minumum encode size, which is the minimum buffer size that can be encoded. (Their spelling, not mine, but the meaning is 100% exactly that πŸ’œ)
  • int liberasurecode_get_fragment_size (int desc, int data_len) β€” no description provided, just returns a fragment size, periodt
  • uint32_t liberasurecode_get_version (void) β€” This will return the liberasurecode version for the descriptor. Version check bestie, always 🌸

🌸 Variables 🌸

The backend lineup, every single one, and ec_backends_supported is static because she's shy:

  • struct ec_backend_common backend_null
  • struct ec_backend_common backend_flat_xor_hd
  • struct ec_backend_common backend_jerasure_rs_vand
  • struct ec_backend_common backend_jerasure_rs_cauchy
  • struct ec_backend_common backend_isa_l_rs_vand
  • struct ec_backend_common backend_shss
  • struct ec_backend_common backend_liberasurecode_rs_vand
  • struct ec_backend_common backend_isa_l_rs_cauchy
  • struct ec_backend_common backend_libphazr
  • struct ec_backend_common backend_isa_l_rs_vand_inv
  • struct ec_backend_common backend_isa_l_rs_lrc
  • static ec_backend_t ec_backends_supported []

That's the roster. Whole squad accounted for. πŸ’–


πŸ’– Function Documentation πŸ’–

Okay bestie, this is the function-by-function gossip section β€” every signature, line number, parameter, return value, and reference is kept exactly. No cap. ✨

void __attribute__ ((constructor) )

Definition at line 157 of file erasurecode.c.

void __attribute__ ((destructor) )

Definition at line 163 of file erasurecode.c.


int is_invalid_fragment (int desc, char * fragment)

Definition at line 1222 of file erasurecode.c.

References: get_libec_version(), is_invalid_fragment_metadata(), and liberasurecode_get_fragment_metadata().

Referenced by: liberasurecode_decode().

int is_invalid_fragment_header (fragment_header_t * header)

Definition at line 1135 of file erasurecode.c.

References: liberasurecode_crc32_alt().

Referenced by: liberasurecode_decode(), liberasurecode_get_fragment_metadata(), and liberasurecode_reconstruct_fragment().

int is_invalid_fragment_metadata (int desc, fragment_metadata_t * fragment_metadata) [static]

Definition at line 1189 of file erasurecode.c.

References: liberasurecode_verify_fragment_metadata().

Referenced by: is_invalid_fragment(), and liberasurecode_verify_stripe_metadata().

int liberasurecode_backend_alloc_desc (void ) [static]

Allocated backend instance descriptor. Returns a unique descriptor for a new backend. The caller must hold active_instances_rwlock.

Definition at line 113 of file erasurecode.c.

Referenced by: liberasurecode_instance_create().

int liberasurecode_backend_available (const ec_backend_id_t backend_id)

Checks if a given backend is available. Vibe check for backends, basically.

Parameters

  • backend_id β€” one of the supported backends.

Returns: 1 if a backend is available; 0 otherwise.

Definition at line 177 of file erasurecode.c.

References: ec_backends_supported, liberasurecode_backend_close(), and liberasurecode_backend_open().

int liberasurecode_backend_close (ec_backend_t instance) [static]

Definition at line 143 of file erasurecode.c.

Referenced by: liberasurecode_backend_available(), liberasurecode_instance_create(), and liberasurecode_instance_destroy().

void * liberasurecode_backend_open (ec_backend_t instance) [static]

Definition at line 135 of file erasurecode.c.

Referenced by: liberasurecode_backend_available(), and liberasurecode_instance_create().

int liberasurecode_decode (int desc, char ** available_fragments, int num_fragments, uint64_t fragment_len, int force_metadata_checks, char ** out_data, uint64_t * out_data_len)

Reconstruct original data from a set of k encoded fragments. This is the glow-up moment where the data comes back together. πŸ’…

Parameters

  • desc β€” liberasurecode descriptor/handle from liberasurecode_instance_create()
  • available_fragments β€” erasure encoded fragments (> = k)
  • num_fragments β€” number of fragments being passed in
  • fragment_len β€” length of each fragment (assume they are the same)
  • force_metadata_checks β€” force fragment metadata checks (default: 0)
  • out_data β€” output pointer to decoded data
  • out_data_len β€” output length of decoded output

Returns: 0 on success, -error code otherwise

Definition at line 530 of file erasurecode.c.

References: alloc_and_set_buffer(), get_fragment_partition(), is_invalid_fragment(), and is_invalid_fragment_header().

int liberasurecode_decode_cleanup (int desc, char * data)

Cleanup structures allocated by librasurecode_decode. The caller has no context, so cannot safely free memory allocated by liberasurecode, so it must pass the deallocation responsibility back to liberasurecode. Teamwork, but make it memory management. 🫢

Parameters

  • desc β€” liberasurecode descriptor/handle from liberasurecode_instance_create()
  • data β€” (char *) buffer of data decoded by librasurecode_decode

Returns: 0 in success; -error otherwise

Definition at line 499 of file erasurecode.c.

int liberasurecode_encode (int desc, const char * orig_data, uint64_t orig_data_size, char *** encoded_data, char *** encoded_parity, uint64_t * fragment_len)

Erasure encode a data buffer. It's giving data-to-fragments realness. πŸ¦‹

Parameters

  • desc β€” liberasurecode descriptor/handle from liberasurecode_instance_create()
  • orig_data β€” data to encode
  • orig_data_size β€” length of data to encode
  • encoded_data β€” pointer to output array (char **) of k data fragments (char *), allocated by the callee
  • encoded_parity β€” pointer to output array (char **) of m parity fragments (char *), allocated by the callee
  • fragment_len β€” pointer to output length of each fragment, assuming all fragments are the same length

Returns: 0 on success, -error code otherwise

Definition at line 389 of file erasurecode.c.

References: liberasurecode_encode_cleanup().

int liberasurecode_encode_cleanup (int desc, char ** encoded_data, char ** encoded_parity)

Cleanup structures allocated by librasurecode_encode. The caller has no context, so cannot safely free memory allocated by liberasurecode, so it must pass the deallocation responsibility back to liberasurecode. Same cleanup energy, different encode era. ✨

Parameters

  • desc β€” liberasurecode descriptor/handle from liberasurecode_instance_create()
  • encoded_data β€” (char **) array of k data fragments (char *), allocated by liberasurecode_encode
  • encoded_parity β€” (char **) array of m parity fragments (char *), allocated by liberasurecode_encode

Returns: 0 in success; -error otherwise

Definition at line 334 of file erasurecode.c.

Referenced by: liberasurecode_encode().

int liberasurecode_fragments_needed (int desc, int * fragments_to_reconstruct, int * fragments_to_exclude, int * fragments_needed)

Return a list of lists with valid rebuild indexes given a list of missing indexes. @desc: liberasurecode instance descriptor (obtained with liberasurecode_instance_create) @fragments_to_reconstruct list of indexes to reconstruct @fragments_to_exclude list of indexes to exclude from reconstruction equation @fragments_needed list of fragments needed to reconstruct fragments in fragments_to_reconstruct; output param.

Returns: 0 on success; -error code otherwise

Definition at line 991 of file erasurecode.c.

int liberasurecode_get_aligned_data_size (int desc, uint64_t data_len)

This computes the aligned size of a buffer passed into the encode function. The encode function must pad fragments to be aligned with the word size (w) and the last fragment also needs to be aligned. This computes the sum of the aligned fragment sizes for a given buffer to encode.

Definition at line 1303 of file erasurecode.c.

Referenced by: liberasurecode_get_minimum_encode_size().

int liberasurecode_get_fragment_metadata (char * fragment, fragment_metadata_t * fragment_metadata)

Get opaque metadata for a fragment. The metadata is opaque to the client, but meaningful to the underlying library. It is used to verify stripes in verify_stripe_metadata(). Opaque to you, iconic to the backend. πŸŽ€

Parameters

  • fragment β€” fragment pointer
  • fragment_metadata β€” pointer to output fragment metadata struct (reference passed by the user)

Definition at line 1050 of file erasurecode.c.

References: get_data_ptr_from_fragment(), is_invalid_fragment_header(), and liberasurecode_crc32_alt().

Referenced by: is_invalid_fragment().

int liberasurecode_get_fragment_size (int desc, int data_len)

Definition at line 1345 of file erasurecode.c.

int liberasurecode_get_minimum_encode_size (int desc)

This will return the minumum encode size, which is the minimum buffer size that can be encoded. Yes, β€œminumum” is exactly how it’s written here β€” preserving the lore. πŸ“

Definition at line 1340 of file erasurecode.c.

References: liberasurecode_get_aligned_data_size().

uint32_t liberasurecode_get_version (void )

This will return the liberasurecode version for the descriptor.

Definition at line 1374 of file erasurecode.c.

int liberasurecode_instance_create (const ec_backend_id_t id, struct ec_args * args)

Create a liberasurecode instance and return a descriptor for use with EC operations (encode, decode, reconstruct). Main character instance creation moment. πŸ‘‘

Parameters

  • id β€” one of the supported backends as defined by ec_backend_id_t
  • args β€” arguments to the EC backend arguments common to all backends
    • k β€” number of data fragments
    • m β€” number of parity fragments
    • w β€” word size, in bits
    • hd β€” hamming distance (=m for Reed-Solomon)
    • ct β€” fragment checksum type (stored with the fragment metadata)
    • backend-specific arguments
      • null_args β€” arguments for the null backend
      • flat_xor_hd, jerasure do not require any special args

Returns: liberasurecode instance descriptor (int > 0)

Definition at line 211 of file erasurecode.c.

References: ec_backends_supported, liberasurecode_backend_alloc_desc(), liberasurecode_backend_close(), liberasurecode_backend_open(), and print_dlerror().

int liberasurecode_instance_destroy (int desc)

Close a liberasurecode instance. Log off, clean up, leave no dangling vibes. πŸ’œ

Parameters

  • desc β€” liberasurecode descriptor to close

Definition at line 292 of file erasurecode.c.

References: liberasurecode_backend_close().

int liberasurecode_reconstruct_fragment (int desc, char ** available_fragments, int num_fragments, uint64_t fragment_len, int destination_idx, char * out_fragment)

Reconstruct a missing fragment from a subset of available fragments. Missing fragment? Not on our watch, bestie. πŸ’–

Parameters

  • desc β€” liberasurecode descriptor/handle from liberasurecode_instance_create()
  • fragment_len β€” size in bytes of the fragments
  • available_fragments β€” erasure encoded fragments
  • num_fragments β€” number of fragments being passed in
  • destination_idx β€” missing idx to reconstruct
  • out_fragment β€” output of reconstruct

Returns: 0 on success, -error code otherwise

Definition at line 764 of file erasurecode.c.

References: alloc_and_set_buffer(), get_fragment_partition(), and is_invalid_fragment_header().

int liberasurecode_verify_fragment_metadata (ec_backend_t be, fragment_metadata_t * md)

Definition at line 1172 of file erasurecode.c.

Referenced by: is_invalid_fragment_metadata().

int liberasurecode_verify_stripe_metadata (int desc, char ** fragments, int num_fragments)

Definition at line 1262 of file erasurecode.c.

References: is_invalid_fragment_metadata().

void print_dlerror (const char * caller) [static]

Definition at line 125 of file erasurecode.c.

Referenced by: liberasurecode_instance_create().

SLIST_HEAD (backend_list , ec_backend ) [static]

Look up a backend instance by descriptor.

Returns: pointer to a registered liberasurecode instance The caller must hold active_instances_rwlock

Definition at line 76 of file erasurecode.c.

🌈 Variable Documentation 🌈

okay bestie, this is the part where the docs just... list the glow-ups themselves. these are the actual global variables, and honestly? they're the whole cast of the show πŸ’…

  • struct ec_backend_common backend_flat_xor_hd [extern] β€” Definition at line 50 of file flat_xor_hd.c. Referenced by __attribute__(), and flat_xor_is_compatible_with().
  • struct ec_backend_common backend_isa_l_rs_cauchy [extern] β€” Definition at line 47 of file isa_l_rs_cauchy.c. Referenced by __attribute__(), and isa_l_rs_cauchy_is_compatible_with().
  • struct ec_backend_common backend_isa_l_rs_lrc [extern] β€” Definition at line 46 of file isa_l_rs_lrc.c. Referenced by __attribute__(), and isa_l_rs_lrc_is_compatible_with().
  • struct ec_backend_common backend_isa_l_rs_vand [extern] β€” Definition at line 46 of file isa_l_rs_vand.c. Referenced by __attribute__(), and isa_l_rs_vand_is_compatible_with().
  • struct ec_backend_common backend_isa_l_rs_vand_inv [extern] β€” Definition at line 47 of file isa_l_rs_vand_inv.c. Referenced by __attribute__(), and isa_l_rs_vand_inv_is_compatible_with().
  • struct ec_backend_common backend_jerasure_rs_cauchy [extern] β€” Definition at line 49 of file jerasure_rs_cauchy.c. Referenced by __attribute__(), and jerasure_rs_cauchy_is_compatible_with().
  • struct ec_backend_common backend_jerasure_rs_vand [extern] β€” Definition at line 49 of file jerasure_rs_vand.c. Referenced by __attribute__(), and jerasure_rs_vand_is_compatible_with().
  • struct ec_backend_common backend_liberasurecode_rs_vand [extern] β€” Definition at line 47 of file liberasurecode_rs_vand.c. Referenced by __attribute__(), and liberasurecode_rs_vand_is_compatible_with().
  • struct ec_backend_common backend_libphazr [extern] β€” Definition at line 48 of file libphazr.c. Referenced by __attribute__(), and pio_is_compatible_with().
  • struct ec_backend_common backend_null [extern] β€” Referenced by __attribute__(). (no Definition at line for this one, bestie, that's just how the docs did it, i don't make the rules 🫢)
  • struct ec_backend_common backend_shss [extern] β€” Definition at line 41 of file shss.c. Referenced by __attribute__(), and shss_is_compatible_with().

πŸŽ€ and now the main character herself πŸŽ€

  • ec_backend_t ec_backends_supported[] [static] β€” Initial value:
                                          = {
         (ec_backend_t) &backend_null,
         (ec_backend_t) &backend_jerasure_rs_vand,
         (ec_backend_t) &backend_jerasure_rs_cauchy,
         (ec_backend_t) &backend_flat_xor_hd,
         (ec_backend_t) &backend_isa_l_rs_vand,
         (ec_backend_t) &backend_shss,
         (ec_backend_t) &backend_liberasurecode_rs_vand,
         (ec_backend_t) &backend_isa_l_rs_cauchy,
         (ec_backend_t) &backend_libphazr,
         (ec_backend_t) &backend_isa_l_rs_vand_inv,
         (ec_backend_t) &backend_isa_l_rs_lrc,
         NULL,
     }

Definition at line 58 of file erasurecode.c. Referenced by liberasurecode_backend_available(), and liberasurecode_instance_create().

like… this array is literally the guest list πŸ‘‘ every backend gets an invite, in that exact order, and NULL is the bouncer at the end telling you the party's over. don't reorder it, don't skip it, just respect it. ✨


πŸ¦‹ Author πŸ¦‹

Generated automatically by Doxygen for liberasurecode from the source code.

no human wrote this bit and honestly it shows 😭 it's Doxygen doing its little autopilot thing, straight from the source code. iconic of it ngl.


aaaand that's a wrap on the whole page, bestie!! πŸŽ€πŸ’– this bad girl is from liberasurecode, version 1.8.0, stamped Tue Jan 1 1980 00:00 (yes, the most suspiciously round date ever, that's just what the docs say, i don't question the timestamp πŸ™ˆ). you officially read the entire thing β€” go hydrate, you've earned it πŸ“βœ¨πŸ‘‘

πŸ“¦ where this page comes from