XS(3pm)

📦
version 1.87 · 10 versions
, in ubuntu stonking

💖 CBOR::XS — the XS(3pm) page, bestie edition 💖

okay so. this is CBOR::XS, a Perl module from the User Contributed Perl Documentation section (3pm). she does Concise Binary Object Representation — aka CBOR, aka RFC7049 — and she does it fast. let's get into it 🦋

✨ SYNOPSIS ✨

like, the whole vibe in one look:

 use CBOR::XS;

 $binary_cbor_data = encode_cbor $perl_value;
 $perl_value       = decode_cbor $binary_cbor_data;

 # OO-interface

 $coder = CBOR::XS->new;
 $binary_cbor_data = $coder->encode ($perl_value);
 $perl_value       = $coder->decode ($binary_cbor_data);

 # prefix decoding

 my $many_cbor_strings = ...;
 while (length $many_cbor_strings) {
    my ($data, $length) = $cbor->decode_prefix ($many_cbor_strings);
    # data was decoded
    substr $many_cbor_strings, 0, $length, ""; # remove decoded cbor string
 }

🌈 DESCRIPTION 🌈

so this module flips Perl data structures into CBOR and back again. CBOR is a fast binary serialisation format that aims to use an (almost) superset of the JSON data model — meaning if you can represent something useful in JSON, you should be able to represent it in CBOR too. periodt 💅

short version: CBOR is a faster and quite compact binary alternative to JSON, and it has the added ability of supporting serialisation of Perl objects. lowkey iconic. but real talk — JSON often compresses better than CBOR, so if your plan is to compress the data later and raw speed matters less to you, maybe compare both formats first before you commit 👀

her primary goal is to be correct. her secondary goal is to be fast. and to hit that second one, she was written in C. no cap.

for a general idea about speed: with texts in the megabyte range, CBOR::XS usually encodes roughly twice as fast as Storable or JSON::XS, and decodes about 15%-30% faster than those. the shorter the data gets, the worse Storable performs in comparison. rip.

about compactness: CBOR::XS-encoded data structures are usually about 20% smaller than the same data encoded as (compact) JSON or Storable. ate ✨

on top of the core CBOR data format, this module implements a bunch of extensions, like:

  • support for cyclic and shared data structures (see allow_sharing and allow_cycles)
  • string deduplication (see pack_strings)
  • scalar references (this one is always enabled, no switch needed 💅)

and if you wanna know how CBOR::XS maps perl values to CBOR values and back, that's all in MAPPING, below 🫶

🎀 FUNCTIONAL INTERFACE 🎀

these convenience methods are provided by the module, and they're exported by default. so you literally just get them. no :all needed, bestie.

  • $cbor_data = encode_cbor $perl_scalar — converts the given Perl data structure to CBOR representation. croaks on error. that's it, that's the whole job.
  • $perl_scalar = decode_cbor $cbor_data — the opposite of encode_cbor: she expects a valid CBOR string to parse, and hands you back the resulting perl scalar. croaks on error. 💀 if it's not valid, she's not being polite about it.

👑 OBJECT-ORIENTED INTERFACE 👑

the object oriented interface lets you configure your own encoding or decoding style — within the limits of supported formats, obviously. you're the main character, you get to set the rules.

$cbor = new CBOR::XS

creates a new CBOR::XS object that can be used to de/encode CBOR strings. every boolean flag described below is disabled by default, so it's a clean slate.

the mutators for flags all return the CBOR object again, which means calls can be chained. chef's kiss:

   my $cbor = CBOR::XS->new->encode ({a => [1,2]});

$cbor = new_safe CBOR::XS

creates a new, safe/secure CBOR::XS object 🛡️. this is similar to new, but it configures the coder object to be safe to use with untrusted data. currently, this is equivalent to:

   my $cbor = CBOR::XS
      ->new
      ->validate_utf8
      ->forbid_objects
      ->filter (\&CBOR::XS::safe_filter)
      ->max_size (1e8);

but it's more future proof — and fr fr, it is better to crash because of a change than to be exploited in other ways. periodt.

$cbor = $cbor->max_depth ([$maximum_nesting_depth])

$max_depth = $cbor->get_max_depth

sets the maximum nesting level (default 512) accepted while encoding or decoding. if a higher nesting level is detected in CBOR data or a Perl data structure, then the encoder and decoder will stop and croak right at that point. no further, no thank you.

nesting level is defined by the number of hash- or arrayrefs that the encoder needs to traverse to reach a given point, or the number of { or [ characters without their matching closing parenthesis crossed to reach a given character in a string. iykyk.

setting the maximum depth to one disallows any nesting, so that ensures that the object is only a single hash/object or array. one (1) layer, that's the whole deal.

if no argument is given, the highest possible setting will be used — which is rarely useful, ngl.

heads up: nesting is implemented by recursion in C. the default value has been chosen to be as large as typical operating systems allow without crashing 💀 so don't just crank it for fun.

see "SECURITY CONSIDERATIONS", below, for more info on why this is useful.

$cbor = $cbor->max_size ([$maximum_string_size])

$max_size = $cbor->get_max_size

set the maximum length a CBOR string may have (in bytes) where decoding is being attempted. the default is 0, meaning no limit. when decode is called on a string that is longer than this many bytes, it will not attempt to decode the string but throw an exception instead. this setting has no effect on encode (yet 👀).

if no argument is given, the limit check will be deactivated (same as when 0 is specified).

see "SECURITY CONSIDERATIONS", below, for more info on why this is useful.

$cbor = $cbor->allow_unknown ([$enable])

$enabled = $cbor->get_allow_unknown

if $enable is true (or missing), then encode will not throw an exception when it encounters values it cannot represent in CBOR (for example, filehandles) — instead it will encode a CBOR "error" value. graceful 💅

if $enable is false (the default), then encode will throw an exception when it encounters anything it cannot encode as CBOR.

this option does not affect decode in any way, and it is recommended to leave it off unless you know your communications partner. like, know them know them.

$cbor = $cbor->allow_sharing ([$enable])

$enabled = $cbor->get_allow_sharing

if $enable is true (or missing), then encode will not double-encode values that have been referenced before (e.g. when the same object, such as an array, is referenced multiple times), but instead will emit a reference to the earlier value. efficient queen ✨

this means that such values will only be encoded once, and will not result in a deep cloning of the value on decode, in decoders supporting the value sharing extension. this also makes it possible to encode cyclic data structures (which need allow_cycles to be enabled to be decoded by this module).

it is recommended to leave it off unless you know your communication partner supports the value sharing extensions to CBOR (<http://cbor.schmorp.de/value-sharing>), because without decoder support, the resulting data structure might be unusable. lowkey a trap if you're not careful.

detecting shared values incurs a runtime overhead when values are encoded that have a reference counter larger than one, and it might unnecessarily increase the encoded size, as potentially shared values are encoded as shareable whether or not they are actually shared. so you pay for the maybe 😭

at the moment, only targets of references can be shared (e.g. scalars, arrays or hashes pointed to by a reference). weirder constructs — such as an array with multiple "copies" of the same string, which are hard but not impossible to create in Perl — are not supported. (this is the same as with Storable, so you're in good company.)

if $enable is false (the default), then encode will encode shared data structures repeatedly, unsharing them in the process. cyclic data structures cannot be encoded in this mode. flat out.

this option does not affect decode in any way — shared values and references will always be decoded properly if present. she handles that for you 🫶

$cbor = $cbor->allow_cycles ([$enable])

$enabled = $cbor->get_allow_cycles

if $enable is true (or missing), then decode will happily decode self-referential (cyclic) data structures 🌀. by default these will not be decoded, as they need manual cleanup to avoid memory leaks — so code that isn't prepared for this won't leak memory.

if $enable is false (the default), then decode will throw an error when it encounters a self-referential/cyclic data structure.

this option does not affect encode in any way — shared values and references will always be encoded properly if present.

$cbor = $cbor->allow_weak_cycles ([$enable])

$enabled = $cbor->get_allow_weak_cycles

this works like allow_cycles in that it allows the resulting data structures to contain cycles, but unlike allow_cycles, those cyclic references will be weak. what that means: code that recursively walks the data structure must still be prepared for cycles, but at least no special precautions must be implemented to free these data structures. so, less stressful cleanup, but the walk still needs to know what it's doing.

only those references leading to actual cycles will be weakened — other references, e.g. when the same hash or array is referenced multiple times in an array, will be normal references. so it's targeted, not a blanket thing 💅

this option does not affect encode in any way — shared values and references will always be encoded properly if present.

$cbor = $cbor->forbid_objects ([$enable])

$enabled = $cbor->get_forbid_objects

disables the use of the object serialiser protocol.

if $enable is true (or missing), then encode will throw an exception when it encounters perl objects that would be encoded using the perl-object tag (26). when decode encounters such tags, it will fall back to the general filter/tagged logic as if this were an unknown tag (by default resulting in a CBOR::XC::Tagged object).

if $enable is false (the default), then encode will use the Types::Serialiser object serialisation protocol to serialise objects into perl-object tags, and decode will do the same to decode such tags.

see "SECURITY CONSIDERATIONS", below, for more info on why forbidding this protocol can be useful. spoiler: it's the safe thing to do 🔒

$cbor = $cbor->pack_strings ([$enable])

$enabled = $cbor->get_pack_strings

if $enable is true (or missing), then encode will try not to encode the same string twice, but will instead encode a reference to the string instead. depending on your data format, this can save a lot of space 📦 — but it also results in a very large runtime overhead (expect encoding times to be 2-4 times as high as without). tradeoffs, bestie.

it is recommended to leave it off unless you know your communications partner supports the stringref extension to CBOR (<http://cbor.schmorp.de/stringref>), as without decoder support, the resulting data structure might not be usable. 🚩

more parts coming, don't go anywhere 🧚

  • $cbor = $cbor->text_keys ([$enable])
  • $enabled = $cbor->get_text_keys

bestie if $enabled is true (or you just... didn't pass it, which counts as true), then encode will encode all your perl hash keys as CBOR text strings/UTF-8 string, upgrading them as needed. ✨

If $enable is false (that's the default, no drama), then encode goes back to encoding hash keys normally — meaning upgraded perl strings (strings that are internally encoded as UTF-8) become CBOR text strings, and downgraded perl strings become CBOR byte strings. 💅

Heads up: this option does not affect decode in any way. Not even a lil bit.

This is lowkey a lifesaver for interoperability with CBOR decoders that don't treat byte strings as a form of text. Extra useful because Perl gives you very little control over hash keys, which is giving "I have no say in this" energy. 🙈

Real talk: enabling this can be slow, because every downgraded hash key that gets encoded has to be scanned and converted to UTF-8. It's giving labor. 😭
  • $cbor = $cbor->text_strings ([$enable])
  • $enabled = $cbor->get_text_strings

This one works basically the same as text_keys above, but it goes for all strings (hash keys included), so once you enable text_strings, text_keys has no further effect. It's like the main character absorbing the side character's whole plotline. 👑

If $enabled is true (or missing), then encode will encode all perl strings as CBOR text strings/UTF-8 strings, upgrading them as needed. 🌈

If $enable is false (default, always default), then encode encodes strings normally (but pls see text_keys) — upgraded perl strings (internally UTF-8) as CBOR text strings, downgraded perl strings as CBOR byte strings.

This option does not affect decode in any way. Same as before, bestie.

It's got similar advantages and disadvantages as text_keys. But also: this option effectively removes your ability to automatically encode byte strings, which might break some FREEZE and TO_CBOR methods that rely on that. 💀

A workaround is to use explicit type casts — those are unaffected by this option. slay.

  • $cbor = $cbor->validate_utf8 ([$enable])
  • $enabled = $cbor->get_validate_utf8

If $enable is true (or missing), then decode will actually check that elements (text strings) containing UTF-8 data genuinely contain valid UTF-8 data, instead of just blindly accepting whatever it's handed. Which obviously takes extra time during decoding. Validation is work, bestie. ✨

The concept of "valid UTF-8" used here is perl's concept, which is a superset of the official UTF-8. So it's permissive in its own way. iykyk.

If $enable is false (the default), then decode will blindly accept UTF-8 data, marking them as valid UTF-8 in the resulting data structure regardless of whether that's actually true or not. It's giving "trust fall, no spotter". 😭

Perl isn't too happy about corrupted UTF-8 in strings, but it should generally not crash or do similarly evil things. Extensions, though? Might not be so forgiving. So it's recommended you turn this on if you receive untrusted CBOR. 🫶

This option does not affect encode in any way — strings that are supposedly valid UTF-8 will simply get dumped into the resulting CBOR string without anyone checking whether that's, in fact, true or not.

  • $cbor = $cbor->filter ([$cb->($tag, $value)])
  • $cb_or_undef = $cbor->get_filter

Sets or replaces the tagged value decoding filter (when $cb is specified) or clears the filter (if no argument or undef is provided). Very "new filter, who dis".

The filter callback is called only during decoding, when a non-enforced tagged value has been decoded (peep TAG HANDLING AND EXTENSIONS for a list of enforced tags — those are the ones that don't take no for an answer). For specific tags, it's often better to just provide a default converter using the %CBOR::XS::FILTER hash (see below). 🎀

The first argument is the numerical tag, the second is the (decoded) value that has been tagged.

Your filter function should return either exactly one value, which replaces the tagged value in the decoded data structure, or no values at all, which falls back to default handling — which currently means the decoder creates a CBOR::XS::Tagged object to hold the tag and the value. Both are valid, it's just a vibe choice. 💅

When the filter is cleared (the default state), the default filter function, CBOR::XS::default_filter, is used. This function simply looks up the tag in the %CBOR::XS::FILTER hash. If an entry exists it must be a code reference that's called with tag and value, and it's responsible for decoding the value. If no entry exists, it returns no values. CBOR::XS already ships with a number of default filter functions, and the %CBOR::XS::FILTER hash can be freely extended with more. Build your empire. 🏰

CBOR::XS additionally provides an alternative filter function that's supposed to be safe to use with untrusted data (which the default filter might not be) — it's called CBOR::XS::safe_filter, and it works the same as default_filter but uses the %CBOR::XS::SAFE_FILTER variable instead. It comes prepopulated with the tag decoding functions that are deemed safe (basically the same as %CBOR::XS::FILTER minus all the bignum tags), and can be extended by user code as well — although, obviously, one should be very careful about adding decoding functions here, since the whole expectation is that they're safe to use on untrusted data, after all. 👀

Example: decode all tags not handled internally into CBOR::XS::Tagged objects, with no other special handling (useful when working with potentially "unsafe" CBOR data).

CBOR::XS->new->filter (sub { })->decode ($cbor_data);

Example: provide a global filter for tag 1347375694, converting the value into some string form.

$CBOR::XS::FILTER{1347375694} = sub {
   my ($tag, $value);

   "tag 1347375694 value $value"
};

Example: provide your own filter function that looks up tags in your own hash:

my %my_filter = (
   998347484 => sub {
      my ($tag, $value);

      "tag 998347484 value $value"
   };
);

my $coder = CBOR::XS->new->filter (sub {
   &{ $my_filter{$_[0]} or return }
});

Example: use the safe filter function (see SECURITY CONSIDERATIONS for more considerations on security).

CBOR::XS->new->filter (\&CBOR::XS::safe_filter)->decode ($cbor_data);
  • $cbor_data = $cbor->encode ($perl_scalar)

Converts the given Perl data structure (a scalar value) to its CBOR representation. One-way ticket, bestie. 🎟️

  • $perl_scalar = $cbor->decode ($cbor_data)

The literal opposite of encode: it expects CBOR data and tries to parse it, returning the resulting simple scalar or reference. It croaks on error. 😤

  • ($perl_scalar, $octets) = $cbor->decode_prefix ($cbor_data)

Works like the decode method, but instead of throwing a whole exception when there's trailing garbage after the CBOR string, it just silently stops parsing right there and returns the number of characters consumed so far. So chill about it. 😌

This is useful when your CBOR texts aren't delimited by some outer protocol and you need to know where the first CBOR string ends and the next one starts — CBOR strings are self-delimited, so you can concatenate CBOR strings with no delimiters and no size fields and still recover all their data. iconic behaviour honestly. ✨

CBOR::XS->new->decode_prefix ("......")
=> ("...", 3)

🔄 INCREMENTAL PARSING 🔄

okay so sometimes you just need to parse JSON texts incrementally. (Yes, the section says JSON — we're keeping it, no cap.) While this module always has to keep both the CBOR text and the resulting Perl data structure in memory at the same time, it does let you parse a CBOR stream incrementally — similar to using decode_prefix to see whether a full CBOR object is available, except it's much more efficient. glow up. 💅

It basically works by parsing as much of a CBOR string as it can — if the CBOR data isn't complete yet, the parser remembers where it was, so it can restart once more data has accumulated. Once enough data is available to either decode a complete CBOR value or raise an error, a real decode gets attempted. It's giving "I'll finish my homework later, I know exactly where I stopped". 📚

A typical use case would be a network protocol that consists of sending and receiving CBOR-encoded messages. The solution that works with CBOR and about anything else is prepending a length to every CBOR value, so the receiver knows how many octets to read. A more compact (and slightly slower) approach would be to just send CBOR values back-to-back, since CBOR::XS knows where a CBOR value ends and doesn't need an explicit length. main character energy from the parser, honestly. 👑

The following methods help with this: 🫶

  • @decoded = $cbor->incr_parse ($buffer)

Tries to decode exactly one CBOR value from the beginning of the given $buffer. The value is removed from the $buffer on success. When $buffer doesn't contain a complete value yet, it returns nothing. And finally, when the $buffer doesn't start with something that could ever be a valid CBOR value, it raises an exception, just like decode would. In that last case the decoder state is undefined and must be reset before you can parse anything further. 🚨

This method modifies the $buffer in place. When no CBOR value can be decoded, the decoder stores the current string offset. On the next call, it continues decoding at the place where it stopped before. For that to make any sense, the $buffer must begin with the same octets as on the previous unsuccessful calls. So like, don't go rearranging the beginning of your buffer, bestie. 🙈

You can call this method in scalar context, in which case it returns either a decoded value or undef. Which makes it impossible to tell the difference between CBOR null values (which decode to undef) and an unsuccessful decode — usually that's fine, but just know it. 💀

  • @decoded = $cbor->incr_parse_multiple ($buffer)

Same as incr_parse, but it tries to decode as many CBOR values as possible in one go, instead of at most one. Greedy. Obsessed. ✨ Calls to incr_parse and incr_parse_multiple can be interleaved, they play nice.

  • $cbor->incr_reset

Resets the incremental decoder. This throws away any saved state, so that subsequent calls to incr_parse or incr_parse_multiple start parsing a new CBOR value from the beginning of the $buffer again. Fresh start, new me. 🦋

You can call this at any time, but you must call it if you want to change your $buffer, or if there was a decoding error and you want to reuse the $cbor object for future incremental parsings. no skipping this step!!

🗺️ MAPPING 🗺️

This section describes how CBOR::XS maps Perl values to CBOR values and vice versa. These mappings are designed to "do the right thing" in most circumstances automatically, preserving round-tripping characteristics — meaning what you put in comes out as something equivalent. put it in, get it back, periodt. 💖

For the more enlightened, real quick: in everything below, lowercase perl = the actual Perl interpreter, and uppercase Perl = the whole abstract Perl language itself. Keep those two straight and you're golden, bestie. 💖

CBOR -> PERL

  • integers — CBOR integers walk in and become plain (numeric) perl scalars. But heads up on perls without 64 bit support: 64 bit integers will get truncated or otherwise corrupted. 💀
  • byte strings — byte strings become octet strings in Perl — meaning the Byte values 0..255 just become characters of that same value in Perl. Simple as.
  • UTF-8 strings — UTF-8 strings in CBOR get decoded, like, the UTF-8 octets get turned into proper Unicode code points. Right now though, nobody's validating the UTF-8 octets — so corrupt input = corrupted Perl strings. 🙈
  • arrays, maps — CBOR arrays → references to a Perl array, CBOR maps → references to a Perl hash. And the keys of the map get stringified while all that's happening.
  • null — CBOR null becomes undef in Perl.
  • true, false, undefined — these CBOR values become Types:Serialiser::true, Types:Serialiser::false and Types::Serialiser::error, respectively. They're overloaded to act almost exactly like the numbers 1 and 0 (true and false) or to throw an exception on access (error). See the Types::Serialiser manpage for the full tea. ✨
  • tagged values — a tagged item is a numeric tag plus another CBOR value. Which tags get handled and how? Go read "TAG HANDLING AND EXTENSIONS" and the description of ->filter. iykyk 💅
  • anything else — anything else (like unsupported simple values) raises a decoding error. periodt.

PERL -> CBOR

Going the other direction is lowkey harder, because Perl is a typeless language. So basically this module can only guess which CBOR type a perl value is meant to be. 🫠

  • hash references — Perl hash references become CBOR maps. Since there's no inherent ordering in hash keys (or in CBOR maps either), they usually get encoded in a pseudo-random order — and that order can be different every single time a hash gets encoded. Also: currently, tied hashes use the indefinite-length format, while normal hashes use the fixed-length format.
  • array references — Perl array references become fixed-length CBOR arrays. clean. 👑
  • other references — other unblessed references get represented using the indirection tag extension (tag value 22098, <http://cbor.schmorp.de/indirection>). CBOR decoders are guaranteed to be able to decode these values somehow — by either "doing the right thing", decoding into a generic tagged object, simply ignoring the tag, or something else entirely. mystery vibes ✨
  • CBOR::XS::Tagged objects — objects of this type must be arrays consisting of a single [tag, value] pair. The (numerical) tag gets encoded as a CBOR tag, and the value gets encoded as appropriate for the value. You must use CBOR::XS::tag to create such objects.
  • Types::Serialiser::true, Types::Serialiser::false, Types::Serialiser::error — these special values become CBOR true, CBOR false and CBOR undefined values, respectively.
  • other blessed objects — other blessed objects get serialised via TO_CBOR or FREEZE. Check "TAG HANDLING AND EXTENSIONS" for the specific classes this module handles, and "OBJECT SERIALISATION" for generic object serialisation.
  • simple scalars — simple Perl scalars (any scalar that isn't a reference) are the most difficult objects to encode, fr fr. CBOR::XS will encode undefined scalars as CBOR null values, scalars that have last been used in a string context before encoding as CBOR strings, and anything else as number value:
# dump as number
encode_cbor [2]                      # yields [2]
encode_cbor [-3.0e17]                # yields [-3e+17]
my $value = 5; encode_cbor [$value]  # yields [5]

# used as string, so dump as string (either byte or text)
print $value;
encode_cbor [$value]                 # yields ["5"]

# undef becomes null
encode_cbor [undef]                  # yields [null]

You can force the type to be a CBOR string by stringifying it:

my $x = 3.1; # some variable containing a number
"$x";        # stringified
$x .= "";    # another, more awkward way to stringify
print $x;    # perl does it for you, too, quite often

You can force whether a string is encoded as byte or text string by using utf8::upgrade and utf8::downgrade (if text_strings is disabled).

utf8::upgrade $x;   # encode $x as text string
utf8::downgrade $x; # encode $x as byte string

More options are available, see "TYPE CASTS", below, and the text_keys and text_strings options.

Perl doesn't define what operations up- and downgrade strings, so if the difference between byte and text is important to you, you should up- or downgrade your string as late as possible before encoding. You can also force the use of CBOR text strings by using text_keys or text_strings.

You can force the type to be a CBOR number by numifying it:

my $x = "3"; # some variable containing a string
$x += 0;     # numify it, ensuring it will be dumped as a number
$x *= 1;     # same thing, the choice is yours.

You can not currently force the type in other, less obscure, ways. Tell me if you need this capability (but don't forget to explain why it's needed :).

Perl values that seem to be integers generally use the shortest possible representation. Floating-point values will use either the IEEE single format if possible without loss of precision, otherwise the IEEE double format will be used. Perls that use formats other than IEEE double to represent numerical values are supported, but might suffer loss of precision. 😭

✨ TYPE CASTS ✨

EXPERIMENTAL, bestie: as an experimental extension, CBOR::XS lets you force specific CBOR types to be used when encoding. That means you can encode types that aren't normally accessible (half floats, heyyy) and also force string types even when text_strings is in effect. 🔥

Type forcing happens by calling a special cast function which keeps a copy of the value and returns a new value that you can hand over to any CBOR encoder function.

The following casts are currently available (all of which are unary operators, that is, have a prototype of $):

CBOR::XS::as_int $value

Forces the value to be encoded as some form of (basic, not bignum) integer type.

CBOR::XS::as_text $value

Forces the value to be encoded as (UTF-8) text values.

CBOR::XS::as_bytes $value

Forces the value to be encoded as a (binary) string value.

Example: encode a perl string as binary even though text_strings is in effect.

CBOR::XS->new->text_strings->encode ([4, "text", CBOR::XS::bytes "bytevalue"]);

CBOR::XS::as_bool $value

Converts a Perl boolean (which can be any kind of scalar) into a CBOR boolean. Strictly the same, just shorter to write, than:

$value ? Types::Serialiser::true : Types::Serialiser::false

CBOR::XS::as_float16 $value

Forces half-float (IEEE 754 binary16) encoding of the given value.

CBOR::XS::as_float32 $value

Forces single-float (IEEE 754 binary32) encoding of the given value.

CBOR::XS::as_float64 $value

Forces double-float (IEEE 754 binary64) encoding of the given value.

CBOR::XS::as_cbor $cbor_text

Not a type cast per-se — this one forces the argument to be encoded as-is. Which is so handy for embedding pre-encoded CBOR data. 💖

Note that no checking on the validity of the $cbor_text is done — it's the caller's responsibility to correctly encode values. no takebacks, bestie.

CBOR::XS::as_map [key => value...]

Treat the array reference as key value pairs and output a CBOR map. This lets you generate CBOR maps with arbitrary key types (or, if you don't care about semantics, duplicate keys or pairs in a custom order), which is otherwise hard to do with Perl.

The single argument must be an array reference with an even number of elements.

Note that only the reference to the array is copied, the array itself is not. Modifications done to the array before calling an encoding function will be reflected in the encoded output. 👀

Example: encode a CBOR map with a string and an integer as keys.

encode_cbor CBOR::XS::as_map [string => "value", 5 => "value"]

💅 OBJECT SERIALISATION 💅

This module implements both a CBOR-specific protocol and the generic Types::Serialier object serialisation protocol. The subsections below explain both methods. (yes, that says Serialier, we're quoting the original, don't @ me 🙈)

ENCODING

This module knows two ways to serialise a Perl object: the CBOR-specific way, and the generic way.

Whenever the encoder bumps into a Perl object that it cannot serialise directly (which is most of them), it will first look up the TO_CBOR method on it.

If the object has a TO_CBOR method, it calls it with the object as the only argument, and expects exactly one return value — which it will then substitute and encode in the place of the object.

Otherwise, it looks up the FREEZE method. If that one exists, it calls it with the object as the first argument, and the constant string CBOR as the second argument, to distinguish it from other serialisers.

The FREEZE method can return any number of values (i.e. zero or more). These will be encoded as a CBOR perl object, together with the classname.

These methods MUST NOT change the data structure that is being serialised. Failure to comply to this can result in memory corruption — and worse. 😱 do NOT do this.

If an object supports neither TO_CBOR nor FREEZE, encoding will fail with an error.

DECODING

Objects encoded via TO_CBOR cannot (normally) be automatically decoded, but objects encoded via FREEZE can be decoded using the following protocol:

When an encoded CBOR perl object is encountered by the decoder, it will look up the THAW method, by using the stored classname, and will fail if the method cannot be found.

After the lookup it will call the THAW method with the stored classname as the first argument, the constant string CBOR as the second argument, and all values returned by FREEZE as the remaining arguments.

EXAMPLES

Here's an example TO_CBOR method:

sub My::Object::TO_CBOR {
   my ($obj) = @_;

   ["this is a serialised My::Object object", $obj->{id}]
}

When a "My::Object" gets encoded to CBOR, it doesn't show up as itself — it turns into a lil simple array with two members: a string, and the "object id". Decode that CBOR string and you get a plain perl array reference sitting where the object used to be. swap 💅

Okay but a way more useful and practical example would be a serialisation method for the URI module. CBOR has a custom tag value for URIs, namely 32:

sub URI::TO_CBOR {
   my ($self) = @_;
   my $uri = "$self"; # stringify uri
   utf8::upgrade $uri; # make sure it will be encoded as UTF-8 string
   CBOR::XS::tag 32, "$_[0]"
}

So that encodes URIs as a UTF-8 string with tag 32, which is the whole "hey this is a URI" signal. ✨

Decoding such an URI will not (currently) hand you back an URI object, bestie. Nope. Instead you get a CBOR::XS::Tagged object with tag number 32 and the string — literally exactly what was returned by "TO_CBOR". it's giving "round trip but not quite" 😭

Now, to serialise an object so it can automatically be deserialised, you need to use "FREEZE" and "THAW". Stealing the URI module as our example again, a possible implementation would be:

sub URI::FREEZE {
   my ($self, $serialiser) = @_;
   "$self" # encode url string
}

sub URI::THAW {
   my ($class, $serialiser, $uri) = @_;
   $class->new ($uri)
}

And unlike "TO_CBOR", multiple values can be returned by "FREEZE". So say a "FREEZE" method that returns "type", "id" and "variant" values — that would cause an invocation of "THAW" with 5 arguments:

sub My::Object::FREEZE {
   my ($self, $serialiser) = @_;

   ($self->{type}, $self->{id}, $self->{variant})
}

sub My::Object::THAW {
   my ($class, $serialiser, $type, $id, $variant) = @_;

   $class-<new (type => $type, id => $id, variant => $variant)
}
yeah that -< in the last line is verbatim from the source, we are simply not touching it, moving on 🫶

🎀 MAGIC HEADER 🎀

ngl there is no way to distinguish CBOR from other formats programmatically. Zero. None. It's vibes-based out here.

So to make it easier to tell CBOR apart from other formats, the CBOR specification has a special "magic string" that can be prepended to any CBOR string without changing its meaning. free real estate. 🌸

This string is available as $CBOR::XS::MAGIC. This module does not prepend this string to the CBOR data it generates, but it will ignore it if present, so users can prepend this string as a "file type" indicator as required. do with that what you will ✨

💅 THE CBOR::XS::Tagged CLASS 💅

CBOR has the concept of tagged values — any CBOR value can be tagged with a numeric 64 bit number, which are centrally administered. very official. very bureaucratic. 💀

"CBOR::XS" handles a few tags internally when en- or decoding. You can also create tags yourself by encoding "CBOR::XS::Tagged" objects, and the decoder will create "CBOR::XS::Tagged" objects itself when it hits an unknown tag.

These objects are simply blessed array references — the first member of the array being the numerical tag, the second being the value. that's the whole thing, that's the object 🎀

You can interact with "CBOR::XS::Tagged" objects in the following ways:

  • $tagged = CBOR::XS::tag $tag, $value — This function(!) creates a new "CBOR::XS::Tagged" object using the given $tag (0..2**64-1) to tag the given $value (which can be any Perl value that can be encoded in CBOR, including serialisable Perl objects and "CBOR::XS::Tagged" objects). yes it's a function, note the exclamation mark, we're all shook
  • $tagged->[0] — access the tag.
  • $tagged->[0] = $new_tag — mutate the tag.
  • $tag = $tagged->tag — access the tag, but make it a method.
  • $new_tag = $tagged->tag ($new_tag) — mutate the tag, method flavor.
  • $tagged->[1] — access the tagged value.
  • $tagged->[1] = $new_value — mutate the tagged value.
  • $value = $tagged->value — access the tagged value, method flavor.
  • $new_value = $tagged->value ($new_value) — mutate the tagged value, method flavor.

✨ EXAMPLES ✨

Here are some examples of "CBOR::XS::Tagged" uses to tag objects. iconique behaviour.

You can look up CBOR tag value and emanings in the IANA registry at <http://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml>.

Prepend a magic header ($CBOR::XS::MAGIC):

my $cbor = encode_cbor CBOR::XS::tag 55799, $value;
# same as:
my $cbor = $CBOR::XS::MAGIC . encode_cbor $value;

Serialise some URIs and a regex in an array:

my $cbor = encode_cbor [
   (CBOR::XS::tag 32, "http://www.nethype.de/"),
   (CBOR::XS::tag 32, "http://software.schmorp.de/"),
   (CBOR::XS::tag 35, "^[Pp][Ee][Rr][lL]\$"),
];

Wrap CBOR data in CBOR:

my $cbor_cbor = encode_cbor
   CBOR::XS::tag 24,
      encode_cbor [1, 2, 3];

cbor inside cbor. inception but make it serialisation 🌀

🦋 TAG HANDLING AND EXTENSIONS 🦋

This section describes how this module handles specific tagged values and extensions. If a tag is not mentioned here and no additional filters are provided for it, then the default handling applies (creating a CBOR::XS::Tagged object on decoding, and only encoding the tag when explicitly requested).

Tags not handled specifically are currently converted into a CBOR::XS::Tagged object, which is simply a blessed array reference consisting of the numeric tag value followed by the (decoded) CBOR value. it's the default, it's the fallback, it's doing its best 💅

Future versions of this module reserve the right to special case additional tags (such as base64url). noted. on the record. no cap. 👑

ENFORCED TAGS

These tags are always handled when decoding, and their handling cannot be overridden by the user. no takesies backsies.

  • 26 (perl-object, <http://cbor.schmorp.de/perl-object>) — These tags are automatically created (and decoded) for serialisable objects using the "FREEZE/THAW" methods (the Types::Serialier object serialisation protocol). See "OBJECT SERIALISATION" for details.
  • 28, 29 (shareable, sharedref, <http://cbor.schmorp.de/value-sharing>) — These tags are automatically decoded when encountered (and they do not result in a cyclic data structure, see "allow_cycles"), resulting in shared values in the decoded object. They are only encoded, however, when "allow_sharing" is enabled.

Not all shared values can be successfully decoded: values that reference themselves will currently decode as "undef" (this is not the same as a reference pointing to itself, which will be represented as a value that contains an indirect reference to itself — these will be decoded properly). 😭

Note that considerably more shared value data structures can be decoded than will be encoded — currently, only values pointed to by references will be shared, others will not. While non-reference shared values can be generated in Perl with some effort, they were considered too unimportant to be supported in the encoder. The decoder, however, will decode these values as shared values. so the decoder is the overachiever here, love that for it ✨

  • 256, 25 (stringref-namespace, stringref, <http://cbor.schmorp.de/stringref>) — These tags are automatically decoded when encountered. They are only encoded, however, when "pack_strings" is enabled.
  • 22098 (indirection, <http://cbor.schmorp.de/indirection>) — This tag is automatically generated when a reference are encountered (with the exception of hash and array references). It is converted to a reference when decoding.
  • 55799 (self-describe CBOR, RFC 7049) — This value is not generated on encoding (unless explicitly requested by the user), and is simply ignored when decoding. it's just there. hanging out. chilling. 🙈

NON-ENFORCED TAGS

These tags have default filters provided when decoding. Their handling can be overridden by changing the %CBOR::XS::FILTER entry for the tag, or by providing a custom "filter" callback when decoding. so you do get a say here 🫶

When they result in decoding into a specific Perl class, the module usually provides a corresponding "TO_CBOR" method as well.

When any of these need to load additional modules that are not part of the perl core distribution (e.g. URI), it is (currently) up to the user to provide these modules. The decoding usually fails with an exception if the required module cannot be loaded. heads up!! ⚠️

  • 0, 1 (date/time string, seconds since the epoch) — These tags are decoded into Time::Piece objects. The corresponding "Time::Piece::TO_CBOR" method always encodes into tag 1 values currently.

The Time::Piece API is generally surprisingly bad, and fractional seconds are only accidentally kept intact, so watch out. On the plus side, the module comes with perl since 5.10, which has to count for something. 😭 (we're being so real about this one)

  • 2, 3 (positive/negative bignum) — These tags are decoded into Math::BigInt objects. The corresponding "Math::BigInt::TO_CBOR" method encodes "small" bigints into normal CBOR integers, and others into positive/negative CBOR bignums.
  • 4, 5, 264, 265 (decimal fraction/bigfloat) — Both decimal fractions and bigfloats are decoded into Math::BigFloat objects. The corresponding "Math::BigFloat::TO_CBOR" method always encodes into a decimal fraction (either tag 4 or 264).

NaN and infinities are not encoded properly, as they cannot be represented in CBOR. no cap, they just can't. 💀

See "BIGNUM SECURITY CONSIDERATIONS" for more info.

  • 30 (rational numbers) — These tags are decoded into Math::BigRat objects. The corresponding "Math::BigRat::TO_CBOR" method encodes rational numbers with denominator 1 via their numerator only, i.e., they become normal integers or "bignums".

See "BIGNUM SECURITY CONSIDERATIONS" for more info.

  • 21, 22, 23 (expected later JSON conversion) — CBOR::XS is not a CBOR-to-JSON converter, and will simply ignore these tags. it said "not my job" 💅
  • 32 (URI) — These objects decode into URI objects. The corresponding "URI::TO_CBOR" method again results in a CBOR URI value.

🌈 CBOR and JSON 🌈

CBOR is supposed to implement a superset of the JSON data model, and is, with some coercion, able to represent all JSON texts (something that other "binary JSON" formats such as BSON generally do not support). CBOR ate and left no crumbs, tbh.

CBOR implements some extra hints and support for JSON interoperability, and the spec offers further guidance for conversion between CBOR and JSON. None of this is currently implemented in CBOR, and the guidelines in the spec do not result in correct round-tripping of data. so like… it's aspirational 🌸

If JSON interoperability is improved in the future, then the goal will be to ensure that decoded JSON data will round-trip encoding and decoding to CBOR intact. fingers crossed bestie 🤞

🛡️ SECURITY CONSIDERATIONS

Tl;dr bestie: if you're decoding or encoding CBOR from sources you don't trust, start with a coder object made by new_safe, which already does all the mitigations I'm about to explain. Look:

my $coder = CBOR::XS->new_safe;

my $data = $coder->decode ($cbor_text);
my $cbor = $coder->encode ($data);

The longer tea: when you're using CBOR in a protocol and talking to untrusted, potentially hostile little gremlins, you genuinely have to think about this stuff. Let's go one by one.

🔒 Security of the CBOR decoder itself

First and foremost — the decoder itself has to be secure, meaning no buffer overflows or similar bugs that someone could exploit. Obviously this module should guarantee that and I'm trying hard to make it true, but... you never know, right? 💀

🪄 CBOR::XS can invoke almost arbitrary callbacks during decoding

CBOR::XS supports object serialisation, which means decoding CBOR can trigger calls to any THAW method in any package that exists in your process. To be clear: CBOR::XS will not try to load modules, but any existing THAW method or function can be called — so all of them have to be secure. No exceptions, no cap.

Less obviously, it will also invoke TO_CBOR and FREEZE methods. So even if every single one of your THAW methods is perfectly secure, encoding data structures from untrusted sources can still call those and set off bugs inside them.

So if you're not sure about the security of every module you have loaded (and bestie, you shouldn't be sure), disable this whole part with forbid_objects, or just use new_safe.

🏷️ CBOR can be extended with tags that call library code

CBOR can be extended with tags, and CBOR::XS keeps a registry of conversion functions for lots of existing tags — and third-party modules can extend it (peep the filter method).

If you don't trust those, configure the safe filter function, CBOR::XS::safe_filter (new_safe does this for you). By default it only includes conversion functions the author considers safe — but again, third-party modules can extend it. 🙈

Depending on your level of paranoia, you can use the safe filter:

$cbor->filter (\&CBOR::XS::safe_filter);

...or your very own filter:

$cbor->filter (sub { ... do your stuffs here ... });

...or literally no filter at all, which disables all tag decoding:

$cbor->filter (sub { });

This is never a problem for encoding, because the tag mechanism only exists in CBOR texts. ✨

🍽️ Resource-starving attacks: object memory usage

You need to avoid resource-starving attacks. Translation: limit the size of the CBOR data you accept, or make sure that when your resources run out, that's just fine (e.g. by using a separate process that can crash safely). The size of a CBOR string in octets is usually a solid indication of how many resources decoding it into a Perl structure will take. CBOR::XS can check the size of the CBOR text (using max_size — which new_safe does), but it might be too late once you already have it sitting in memory, so you might wanna check the size before you accept the string.

As for encoding: it's possible to construct data structures that are relatively small but produce large CBOR texts — for example, an array full of references to the same big data structure, which all get deep-cloned during encoding by default. This is rarely an actual issue (worst case is still just running out of memory), but you can reduce that risk by using allow_sharing.

📚 Resource-starving attacks: stack overflows

CBOR::XS recurses using the C stack when decoding objects and arrays, and the C stack is a limited resource. Case in point: on my amd64 machine with 8MB of stack size, I can decode around 180k nested arrays, but only 14k nested CBOR objects (because perl itself recurses deeply on croak to free the temporary). Exceed that and the program crashes. To be conservative, the default nesting limit is set to 512. If your process has a smaller stack, adjust this with the max_depth method.

🐌 Resource-starving attacks: CPU en-/decoding complexity

CBOR::XS uses the Math::BigInt, Math::BigFloat and Math::BigRat libraries to represent encode/decode bignums. These can be very slow (as in centuries of CPU time slow) and can even crash your program — and honestly they're generally not very trustworthy. See the next section on bignum security for the details.

🫣 Data breaches: leaking information in error messages

CBOR::XS might leak contents of your Perl data structures in its error messages. So when you serialise sensitive info, make sure exceptions thrown by CBOR::XS will never end up in front of untrusted eyes. 👀

❓ Something else...

Something else could bomb you too — something I forgot to think of. In that case, you get to keep the pieces. I'm always open to hints, though... 🫶

🔢 BIGNUM SECURITY CONSIDERATIONS

CBOR::XS provides a TO_CBOR method for both Math::BigInt and Math::BigFloat that tries to encode the number in the simplest possible way — that is, either a CBOR integer, a CBOR bigint/decimal fraction (tag 4), or an arbitrary-exponent decimal fraction (tag 264). Rational numbers (Math::BigRat, tag 30) can also contain bignums as members.

CBOR::XS will also understand base-2 bigfloat or arbitrary-exponent bigfloats (tags 5 and 265), but it will never generate these on its own.

With the built-in Math::BigInt::Calc support, encoding and decoding decimal fractions is generally fast. Decoding bigints can be slow for very big numbers (tens of thousands of digits — something that could potentially be caught by limiting the size of CBOR texts), and decoding bigfloats or arbitrary-exponent bigfloats can be extremely slow (minutes... decades) for large exponents (roughly 40 bit and longer). 😭

On top of that, Math::BigInt can take advantage of other bignum libraries like Math::GMP, which cannot handle big floats with large exponents and might simply abort or crash your program due to their code quality. 💀

This can be a real concern if you want to parse untrusted CBOR. If it is, you might want to disable decoding of tag 2 (bigint) and tag 3 (negative bigint) types. You should also disable types 5 and 265, since those can be slow even without bigints.

Disabling bigints will also partially or fully disable types that rely on them, e.g. rational numbers that use bignums.

🛠️ CBOR IMPLEMENTATION NOTES

This section has some random implementation notes. They do not describe guaranteed behaviour — it's just behaviour as implemented right now. 🤷

  • 64 bit integers are only properly decoded when Perl was built with 64 bit support.
  • Strings and arrays are encoded with a definite length. Hashes too, unless they are tied (or otherwise magical).
  • Only the double data type is supported for NV data types — when Perl uses long double to represent floating point values, they might not be encoded properly. Half precision types are accepted, but not encoded.
  • Strict mode and canonical mode are not implemented.

🚧 LIMITATIONS ON PERLS WITHOUT 64-BIT INTEGER SUPPORT

On perls built without 64 bit integer support (these are rare nowadays, even on 32 bit architectures, since all major Perl distributions are built with 64 bit integer support), support for any kind of 64 bit value in CBOR is very limited — most likely these 64 bit values will be truncated, corrupted, or otherwise not decoded correctly. This also includes string, float, array and map sizes that are stored as 64 bit integers.

🧵 THREADS

This module is not guaranteed to be thread safe, and there are no plans to change that until Perl gets real thread support — as opposed to the horribly slow so-called "threads" which are simply slow and bloated process simulations. Use fork, it's much faster, cheaper, better. 🔥

(It might actually work, but you have been warned).

🐛 BUGS

While the goal of this module is to be correct, that unfortunately does not mean it's bug-free — only that I think its design is bug-free. Keep reporting bugs and they'll be fixed swiftly, though. 💪

Please refrain from using rt.cpan.org or any other bug reporting service. I put the contact address into my modules for a reason.

👀 SEE ALSO

  • The JSON and JSON::XS modules that do similar, but human-readable, serialisation.
  • The Types::Serialiser module provides the data model for true, false and error values.

✍️ AUTHOR

 Marc Lehmann <schmorp@schmorp.de>
 http://home.schmorp.de/

and that's a wrap on the whole page, bestie!! 🎀 this was perl v5.42.3, dated 2026-08-20, and the manpage is XS(3pm) — you made it to the end, iconic behaviour, no notes. 💅✨

📦 where this page comes from