ici::doc::pod3::zco(3)

πŸ“¦ version 3.2.1, in ubuntu jammy

πŸ’… zco β€” the library for manipulating zero-copy objects ✨

okay so this is ici::doc::pod3::zco(3), straight out of the ICI library functions section, and bestie… it's a lot, but we're gonna get through it together 🫢


✨ SYNOPSIS ✨

#include "zco.h"

typedef enum
{
    ZcoFileSource = 1,
    ZcoSdrSource = 2,
    ZcoZcoSource = 3
} ZcoMedium;

typedef void (*ZcoCallback);

[see description for available functions]

🌸 DESCRIPTION 🌸

okay so picture this: "Zero-copy objects" (ZCOs, we're on nickname basis) are abstract data access representations β€” say that five times fast πŸ’€ β€” and their whole entire purpose in life is to minimize I/O when you're encapsulating your application's source data inside one or more layers of communication protocol structure. no cap, that's the job.

ZCOs get constructed inside the heap space of an SDR, and every single implementation of every layer of the stack has to be able to get to that SDR. that's non-negotiable, bestie.

so what's actually inside a ZCO? glad you asked πŸŽ€

  • info that lets you get at the source data objects
  • a linked list of zero or more "extents" that reference portions of those source data objects
  • linked lists of protocol header and trailer capsules that have been explicitly attached to the ZCO since it was created

and the thing that actually goes out on the wire (or that just came in) is the concatenation of:

  1. the headers, in ascending stack sequence
  2. the source data object extents
  3. the trailers, in descending stack sequence

slay. that's the whole vibe.


each source data object is one of two things, and honestly both are iconic:

  • a file, identified by a pathname stored in a "file reference" object in SDR heap, or
  • an array of bytes in SDR heap space, identified by SDR address

every protocol header or trailer capsule tells you the length and the address (within SDR heap space) of a single protocol header or trailer at some layer of the stack. and heads up: for some purposes, the source data object for a newly added extent of a ZCO can be specified indirectly, by reference to an extent of an existing ZCO πŸ‘€

the extents of multiple ZCOs can totally reference the same files and/or SDR source data objects β€” sharing is caring. those source data objects are reference-counted, so they get deleted automatically when (and only when) all the ZCO extents that reference them have been deleted. the way that just works?? obsessed πŸ’–


🫢 safety note, and this one matters: shared access to a ZCO is safe because the ZCO lives in SDR, so it cannot be modified except in the course of an SDR transaction, which serializes access. periodt.

and also β€” extracting data from a ZCO might involve reading file-based source data extents, which can cause file progress to be updated in one or more file reference objects in the SDR heap. so for that reason, all ZCO "transmit" and "receive" functions must be performed within SDR transactions. don't skip that step, i'm begging 😭


also, quick side quest: ZCO can more broadly be used as a general-purpose reference counting system for non-volatile data objects, whenever you identify a need for such a system. multitasking queen πŸ‘‘


okay so the total volume of file system space that file-sourced ZCO extents can occupy, and the total volume of SDR heap space that heap-sourced ZCO extents can occupy β€” those are system configuration parameters, and you can set them with ZCO library functions πŸŽ€

those limits are enforced when extents are appended to ZCOs:

  • total ZCO file space occupancy and total ZCO heap occupancy get updated continuously as ZCOs are created and destroyed
  • the formation of a new extent is prohibited when the length of the extent exceeds the difference between the applicable limit and the corresponding current occupancy total

that's the rules, i don't make them ✨


πŸ’œ void zco_register_callback(ZcoCallback notify)

registers the "callback" function that the ZCO system will invoke every time a ZCO is destroyed, making ZCO file and/or heap space available for the formation of new ZCO extents. this mechanism can be used, for example, to notify tasks that are waiting for ZCO space to be made available so they can resume some communication protocol procedure. it's giving "hey bestie, your table's ready" πŸ’…

πŸ’œ void zco_unregister_callback( )

simply unregisters the currently registered callback function for ZCO destruction. bye felicia πŸ‘‹

πŸ’œ Object zco_create_file_ref(Sdr sdr, char *pathName, char *cleanupScript)

creates and returns a new file reference object, which you can use as the source data extent location for creating a ZCO whose source data object is the file identified by pathName.

cleanupScript, if it's not NULL, is invoked at the moment the last ZCO that cites this file reference is destroyed β€” normally upon delivery either down to the "ZCO transition layer" of the protocol stack, or up to a ZCO-capable application.

a zero-length string is interpreted as implicit direction to delete the referenced file when the file reference object is destroyed. 😳

maximum length of cleanupScript is 255.

returns the SDR location of the file reference object on success, 0 on any error.

πŸ’œ Object zco_revise_file_ref(Sdr sdr, Object fileRef, char *pathName, char *cleanupScript)

changes the pathName and cleanupScript of the indicated file reference. the new values of these fields are validated the same way as for zco_create_file_ref().

returns 0 on success, -1 on any error.

πŸ’œ char *zco_file_ref_path(Sdr sdr, Object fileRef, char *buffer, int buflen)

retrieves the pathName associated with fileRef and stores it in buffer, truncating it to fit (as indicated by buflen) and NULL-terminating it. on success, returns buffer; returns NULL on any error.

πŸ’œ int zco_file_ref_xmit_eof(Sdr sdr, Object fileRef)

returns 1 if the last octet of the referenced file (as determined at the time the file reference object was created) has been read by ZCO via a reader with file offset tracking turned on. otherwise returns zero. that's the whole tea πŸ“

πŸ’œ void zco_destroy_file_ref(Sdr sdr, Object fileRef)

if the file reference object residing at location fileRef within the indicated Sdr is no longer in use (no longer referenced by any ZCO), it destroys this file reference object immediately. otherwise, it flags this file reference object for destruction as soon as the last reference to it is removed. patient and dramatic, we love her πŸ™ˆ

πŸ’œ vast zco_get_file_occupancy(Sdr sdr)

returns the total number of file system space bytes occupied by ZCOs created in this Sdr.

πŸ’œ void zco_set_max_file_occupancy(Sdr sdr, vast occupancy)

declares the total number of file system space bytes that may be occupied by ZCOs created in this Sdr.

πŸ’œ vast zco_get_max_file_occupancy(Sdr sdr)

returns the total number of file system space bytes that may be occupied by ZCOs created in this Sdr.

πŸ’œ int zco_enough_file_space(Sdr sdr, vast length)

returns 1 if the total remaining file system space available for ZCOs in this Sdr is greater than length. returns 0 otherwise.

πŸ’œ vast zco_get_heap_occupancy(Sdr sdr)

returns the total number of SDR heap space bytes occupied by ZCOs created in this Sdr.

πŸ’œ void zco_set_max_heap_occupancy(Sdr sdr, vast occupancy)

declares the total number of SDR heap space bytes that may be occupied by ZCOs created in this Sdr.

πŸ’œ vast zco_get_max_heap_occupancy(Sdr sdr)

returns the total number of SDR heap space bytes that may be occupied by ZCOs created in this Sdr.

πŸ’œ int zco_enough_heap_space(Sdr sdr, vast length)

returns 1 if the total remaining SDR heap space available for ZCOs in this Sdr is greater than length. returns 0 otherwise.


πŸ‘‘ Object zco_create(Sdr sdr, ZcoMedium firstExtentSourceMedium, Object firstExtentLocation, vast firstExtentOffset, vast firstExtentLength)

creates a new ZCO. main character moment ✨

firstExtentLocation and firstExtentLength must either both be zero β€” which means zco_append_extent() will be used to insert the first source data extent later β€” or else both be non-zero.

if firstExtentLocation is non-zero, then:

  • firstExtentLocation must be the SDR location of a file reference object if firstExtentSourceMedium is ZcoFileSource, and must otherwise be the SDR location of the source data itself
  • firstExtentOffset indicates how many leading bytes of the source data object should be skipped over when adding the initial source data extent to the new ZCO

on success, returns the SDR location of the new ZCO. returns 0 if there is insufficient ZCO space for creation of the new ZCO; returns ((Object) -1) on any error.

πŸ‘‘ int zco_append_extent(Sdr sdr, Object zco, ZcoMedium sourceMedium, Object location, vast offset, vast length)

appends the indicated source data extent to the indicated ZCO, exactly as described for zco_create(). both the location and length of the source data must be non-zero.

returns length on success, 0 if there is insufficient ZCO space for creation of the new source data extent, -1 on any error.

πŸ‘‘ int zco_prepend_header(Sdr sdr, Object zco, char *header, vast length)

πŸ‘‘ int zco_append_trailer(Sdr sdr, Object zco, char *trailer, vast length)

πŸ‘‘ void zco_discard_first_header(Sdr sdr, Object zco)

πŸ‘‘ void zco_discard_last_trailer(Sdr sdr, Object zco)

these functions attach and remove the ZCO's headers and trailers. header and trailer are assumed to be arrays of octets, not necessarily text β€” so don't come at them expecting a cute little string πŸ’€

attaching a header or trailer causes it to be written to the SDR.

the prepend and append functions return 0 on success, -1 on any error.

πŸ‘‘ void zco_destroy(Sdr sdr, Object zco)

destroys the indicated Zco. this reduces the reference counts for all files and SDR objects referenced in the ZCO's extents, which results in the freeing of SDR objects and (optionally) the deletion of files as those reference counts drop to zero. it's the circle of life, fr fr 🌈

πŸ‘‘ Object zco_clone(Sdr sdr, Object zco, vast offset, vast length)

creates a new ZCO whose source data is a copy of a subset of the source data of the referenced ZCO. portions of the source data extents of the original ZCO are copied as necessary, but no header or trailer capsules are copied. just the data, bestie.

returns the SDR location of the new ZCO on success, 0 on any error.

πŸ‘‘ vast zco_clone_source_data(Sdr sdr, Object toZco, Object fromZco, vast offset, vast length)

appends to toZco a copy of a subset of the source data of fromZco. portions of the source data extents of fromZco are copied as necessary.

returns length on success, -1 on any error.

πŸ‘‘ vast zco_length(Sdr sdr, Object zco)

returns the length of the entire ZCO, including all headers and trailers and all source data extents. this is the size of the object that would be formed by concatenating the text of all headers, trailers, and source data extents into a single serialized object. the full package, no edits πŸ’–

πŸ‘‘ vast zco_source_data_length(Sdr sdr, Object zco)

returns the length of the entire ZCO minus the lengths of all attached header and trailer capsules. this is the size of the object that would be formed by concatenating the text of all source data extents β€” including those that are presumed to contain header or trailer text attached elsewhere β€” into a single serialized object.

so basically: zco_length() but stripped down to just the source data ✨

  • void zco_start_transmitting(Object zco, ZcoReader *reader) β€” bestie, this one's for the underlying protocol layer. It kicks off extraction of an outbound ZCO's bytes (both from header and trailer capsules and from source data extents) for "transmission" β€” which is literally just copying bytes into a memory buffer so some non-ZCO-aware protocol implementation can vibe with them. It initializes reading at the first byte of the total concatenated ZCO object. It populates reader, which keeps track of "transmission" progress via this ZCO reference. πŸ’…
Also note: you can call this function multiple times to restart reading at the start of the ZCO. And multiple ZcoReader objects can be used concurrently, by the same task or different tasks, to advance through the ZCO independently. No cap. ✨
  • void zco_track_file_offset(ZcoReader *reader) β€” turns on file offset tracking for this reader. That's it. She's simple, she's iconic. πŸŽ€
  • vast zco_transmit(Sdr sdr, ZcoReader *reader, vast length, char *buffer) β€” copies length as-yet-uncopied bytes of the total concatenated ZCO (referenced by reader) into buffer. If buffer is NULL, it skips over length bytes without copying. Returns the number of bytes copied (or skipped) on success, 0 on any file access error, -1 on any other error. Periodt. πŸ’–
  • void zco_start_receiving(Object zco, ZcoReader *reader) β€” used by the overlying protocol layer to start extraction of an inbound ZCO's bytes for "reception" β€” i.e., the copying of bytes into a memory buffer for delivery to a protocol header parser, to a protocol trailer parser, or to the ultimate recipient (application). It initializes reading of headers, source data, and trailers at the first byte of the concatenated ZCO objects. Populates reader, which keeps track of "reception" progress via this ZCO reference and is required. Don't skip her. πŸ™ˆ
  • vast zco_receive_headers(Sdr sdr, ZcoReader *reader, vast length, char *buffer) β€” copies length as-yet-uncopied bytes of presumptive protocol header text from ZCO source data extents into buffer. If buffer is NULL, skips over length bytes without copying. Returns number of bytes copied (or skipped) on success, 0 on any file access error, -1 on any other error. Slay. ✨
  • void zco_delimit_source(Sdr sdr, Object zco, vast offset, vast length) β€” sets the computed offset and length of actual source data in the ZCO, thereby implicitly establishing the total length of the ZCO's concatenated protocol headers as offset and the location of the ZCO's innermost protocol trailer as the sum of offset and length. offset and length are typically determined from the information carried in received presumptive protocol header text. It's giving math but make it cute. 🧚
  • vast zco_receive_source(Sdr sdr, ZcoReader *reader, vast length, char *buffer) β€” copies length as-yet-uncopied bytes of source data from ZCO extents into buffer. If buffer is NULL, skips over length bytes without copying. Returns number of bytes copied (or skipped) on success, 0 on any file access error, -1 on any other error. πŸ’œ
  • vast zco_receive_trailers(Sdr sdr, ZcoReader *reader, vast length, char *buffer) β€” copies length as-yet-uncopied bytes of trailer data from ZCO extents into buffer. If buffer is NULL, skips over length bytes without copying. Returns number of bytes copied (or skipped) on success, 0 on any file access error, -1 on any other error. πŸ’…
  • void zco_strip(Sdr sdr, Object zco) β€” deletes all source data extents that contain only header or trailer data and adjusts the offsets and/or lengths of all remaining extents to exclude any known header or trailer data. This function is useful when handling a ZCO that was received from an underlying protocol layer rather than from an overlying application or protocol layer; use it before starting the transmission of the ZCO to another node or before enqueuing it for reception by an overlying application or protocol layer. It's the cleanup queen. πŸ‘‘

πŸ” SEE ALSO πŸ”


okay bestie that's the whole page fr fr πŸ’– this was from Perl v5.24.1, dated 2016-07-07, and it's ici::doc::pod3::zco(3) β€” periodt. πŸŽ€βœ¨

πŸ“¦ where this page comes from