Compression(3pm)

🌸 NAME 🌸

PDL::Compression — compression utilities, bestie. that's the whole vibe. 🫶

---

✨ DESCRIPTION ✨

okay so these routines basically take some data as a PDL and squish it down into a smaller PDL. cute, right? 💅

the algorithms typically work on one single dimension and then broadcast over all the other dimensions — so if you feed in more than one dimension, you get back a broadcasted table of compressed values. iconic.

the Rice algorithm specifically is designed to be identical to the RICE_1 algorithm used in internal FITS-file compression (peep PDL::IO::FITS for that one, iykyk). 👑

---

🎀 SYNOPSIS 🎀

use PDL::Compression

($y,$xsize) = $x->rice_compress();
$c = $y->rice_expand($xsize);

---

🔥 FUNCTIONS 🔥

💖 METHODS 💖

rice_compress

Signature: (in(n); [o]out(m); int[o]len(); int blocksize)

so what it does: it squishes an input PDL along the 0 dimension using Rice compression. no cap.

context matters, bestie:

• in scalar context, you get back only the compressed PDL. that's it.
• in list context, you also get back the ancillary info you need to uncompress the data later with rice_uncompress. keep it, don't lose it. 😭

multidimensional data gets broadcasted over — each row is compressed separately, and the returned PDL is squished to the maximum compressed size of any row.

⚠️ if any of the streams could not be compressed (aka the algorithm produced longer output than it started with, embarrassing), the corresponding length gets set to -1 and that row is treated as if it had length 0. noted. 💀

also important: Rice compression only works on integer data types — if you're sitting there with floating point data you must first quantize them. do not skip this step, fr fr.

the underlying algorithm is identical to the Rice compressor used in CFITSIO (and it's what PDL::IO::FITS uses to load and save compressed FITS images). so like, it's the real deal. 👑

the optional blocksize tells you how many samples are to be compressed as a unit — it defaults to 32. 🍓

how it works 🧚

okay storytime:

Rice compression is a subset of Golomb compression, and it works on data sets where the variation between adjacent samples is typically small compared to the dynamic range of each sample.

in this implementation (originally written by Richard White and contributed to CFITSIO in 1999 — legend behavior honestly), the data are divided into blocks of samples (by default 32 samples per block).

each block has a running difference applied, and the difference is bit-folded to make it positive definite. high order bits of the difference stream are discarded and replaced with a unary representation; low order bits are preserved.

unary representation is very efficient for small numbers, but large jumps could give rise to ludicrously large bins in a plain Golomb code — so those large jumps ("high entropy" samples) are simply recorded directly in the output stream. problem solved. 💅

working on astronomical or solar image data, typical compression ratios of 2-3 are achieved. slay.

$out = $pdl->rice_compress($blocksize);
($out, $len, $blocksize, $dim0) = $pdl->rice_compress;

$new = $out->rice_expand;

rice_compress ignores the bad-value flag of the input ndarrays. it will set the bad-value flag of all output ndarrays if the flag is set for any of the input ndarrays. that's just how it is. 🫶

---

rice_expand

Signature: (in(n); [o]out(m); int blocksize)

the reverse girlypop move — unsquishes a PDL that has been squished by rice_compress. ✨

($out, $len, $blocksize, $dim0) = $pdl->rice_compress;
$copy = $out->rice_expand($dim0, $blocksize);

rice_expand ignores the bad-value flag of the input ndarrays. it will set the bad-value flag of all output ndarrays if the flag is set for any of the input ndarrays. same energy. 💜

---

👑 AUTHORS 👑

Copyright (C) 2010 Craig DeForest. All rights reserved. There is no warranty. You are allowed to redistribute this software / documentation under certain conditions. For details, see the file COPYING in the PDL distribution. If this file is separated from the PDL distribution, the copyright notice should be included in the file.

the Rice compression library is derived from the similar library in the CFITSIO 3.24 release, and is licensed under yet more more lenient terms than PDL itself; that notice is present in the file "ricecomp.c".

---

💀 BUGS 💀

• Currently headers are ignored. 🙈
• Currently there is only one compression algorithm. (just the one, bestie)

---

🌈 TODO 🌈

• Add object encapsulation
• Add test suite

---

and that's the whole page, bestie!! 🦋 this one came from perl v5.38.2, dated 2024-04-10 — you're welcome, go compress something. 💖✨🎀