Nd(3pm)

✨ NAME ✨

okay bestie, welcome to PDL::CCS::Nd(3pm) — the manual page for the N-dimensional sparse pseudo-PDL glow-up. 🦋

PDL::CCS::Nd — N-dimensional sparse pseudo-PDLs 💖

---

🌸 SYNOPSIS 🌸

bestie, this synopsis is long, but it's giving organized. every constructor, every ufunc, every low-level accessor — all here. no cap.

        use PDL;
        use PDL::CCS::Nd;

        ##---------------------------------------------------------------------
        ## Example data

        $missing = 0;                                   ##-- missing values
        $dense   = random(@dims);                       ##-- densely encoded pdl
        $dense->where(random(@dims)<=0.95) .= $missing; ##   ... made sparse

        $whichND = $dense->whichND;                     ##-- which values are present?
        $nzvals  = $dense->indexND($whichND);           ##-- ... and what are they?

        ##---------------------------------------------------------------------
        ## Constructors etc.

        $ccs = PDL::CCS:Nd->newFromDense($dense,%args);           ##-- construct from dense matrix
        $ccs = PDL::CCS:Nd->newFromWhich($whichND,$nzvals,%args); ##-- construct from index+value pairs

        $ccs = $dense->toccs();                ##-- ensure PDL::CCS::Nd-hood
        $ccs = $ccs->toccs();                  ##-- ... analogous to PDL::topdl()
        $ccs = $dense->toccs($missing,$flags); ##-- ... with optional arguments

        $ccs2 = $ccs->copy();                  ##-- copy constructor
        $ccs2 = $ccs->copyShallow();           ##-- shallow copy, mainly for internal use
        $ccs2 = $ccs->shadow(%args);           ##-- flexible copy method, for internal use

        ##---------------------------------------------------------------------
        ## Maintenance & Decoding

        $ccs = $ccs->recode();                 ##-- remove missing values from stored VALS
        $ccs = $ccs->sortwhich();              ##-- internal use only

        $dense2 = $ccs->decode();              ##-- extract to a (new) dense matrix

        $dense2 = $ccs->todense();             ##-- ensure dense storage
        $dense2 = $dense2->todense();          ##-- ... analogous to PDL::topdl()

        ##---------------------------------------------------------------------
        ## PDL API: Basic Properties

        ##---------------------------------------
        ## Type conversion & Checking
        $ccs2 = $ccs->convert($type);
        $ccs2 = $ccs->byte;
        $ccs2 = $ccs->short;
        $ccs2 = $ccs->ushort;
        $ccs2 = $ccs->long;
        $ccs2 = $ccs->longlong;
        $ccs2 = $ccs->float;
        $ccs2 = $ccs->double;

        ##---------------------------------------
        ## Dimensions
        @dims  = $ccs->dims();
        $ndims = $ccs->ndims();
        $dim   = $ccs->dim($dimi);
        $nelem = $ccs->nelem;
        $bool  = $ccs->isnull;
        $bool  = $ccs->isempty;

        ##---------------------------------------
        ## Inplace & Dataflow
        $ccs  = $ccs->inplace();
        $bool = $ccs->is_inplace;
        $bool = $ccs->set_inplace($bool);
        $ccs  = $ccs->sever;

        ##---------------------------------------
        ## Bad Value Handling

        $bool = $ccs->bad_is_missing();          ##-- treat BAD values as missing?
        $bool = $ccs->bad_is_missing($bool);
        $ccs  = $ccs->badmissing();              ##-- ... a la inplace()

        $bool = $ccs->nan_is_missing();          ##-- treat NaN values as missing?
        $bool = $ccs->nan_is_missing($bool);
        $ccs  = $ccs->nanmissing();              ##-- ... a la inplace()

        $ccs2 = $ccs->setnantobad();
        $ccs2 = $ccs->setbadtonan();
        $ccs2 = $ccs->setbadtoval($val);
        $ccs2 = $ccs->setvaltobad($val);

        ##---------------------------------------------------------------------
        ## PDL API: Dimension Shuffling

        $ccs2 = $ccs->dummy($vdimi,$size);
        $ccs2 = $ccs->reorder(@vdims);
        $ccs2 = $ccs->xchg($vdim1,$vdim2);
        $ccs2 = $ccs->mv($vdimFrom,$vdimTo);
        $ccs2 = $ccs->transpose();

        ##---------------------------------------------------------------------
        ## PDL API: Indexing

        $nzi   = $ccs->indexNDi($ndi);              ##-- guts for indexing methods
        $ndi   = $ccs->n2oned($ndi);                ##-- returns 1d pseudo-index for $ccs

        $ivals = $ccs->indexND($ndi);
        $ivals = $ccs->index2d($xi,$yi);
        $ivals = $ccs->index($flati);               ##-- buggy: no pseudo-threading!
        $ccs2  = $ccs->dice_axis($vaxis,$vaxis_ix);

        $nzi   = $ccs->xindex1d($xi);               ##-- nz-indices along 0th dimension
        $nzi   = $ccs->pxindex1d($dimi,$xi);        ##-- ... or any dimension, using ptr()
        $nzi   = $ccs->xindex2d($xi,$yi);           ##-- ... or for Cartesian product on 2d matrix

        $ccs2  = $ccs->xsubset1d($xi);              ##-- subset along 0th dimension
        $ccs2  = $ccs->pxsubset1d($dimi,$xi);       ##-- ... or any dimension, using ptr()
        $ccs2  = $ccs->xsubset2d($xi,$yi);          ##-- ... or for Cartesian product on 2d matrix

        $whichND = $ccs->whichND();
        $vals    = $ccs->whichVals();               ##-- like $ccs->indexND($ccs->whichND), but faster
        $which   = $ccs->which()

        $value = $ccs->at(@index);
        $ccs   = $ccs->set(@index,$value);

        ##---------------------------------------------------------------------
        ## PDL API: Ufuncs

        $ccs2 = $ccs->prodover;
        $ccs2 = $ccs->dprodover;
        $ccs2 = $ccs->sumover;
        $ccs2 = $ccs->dsumover;
        $ccs2 = $ccs->andover;
        $ccs2 = $ccs->orover;
        $ccs2 = $ccs->bandover;
        $ccs2 = $ccs->borover;
        $ccs2 = $ccs->maximum;
        $ccs2 = $ccs->minimum;
        $ccs2 = $ccs->maximum_ind; ##-- -1 indicates "missing" value is maximal
        $ccs2 = $ccs->minimum_ind; ##-- -1 indicates "missing" value is minimal
        $ccs2 = $ccs->nbadover;
        $ccs2 = $ccs->ngoodover;
        $ccs2 = $ccs->nnz;

        $sclr = $ccs->prod;
        $sclr = $ccs->dprod;
        $sclr = $ccs->sum;
        $sclr = $ccs->dsum;
        $sclr = $ccs->nbad;
        $sclr = $ccs->ngood;
        $sclr = $ccs->min;
        $sclr = $ccs->max;
        $bool = $ccs->any;
        $bool = $ccs->all;

        ##---------------------------------------------------------------------
        ## PDL API: Unary Operations         (Overloaded)

        $ccs2 = $ccs->bitnot;                $ccs2 = ~$ccs;
        $ccs2 = $ccs->not;                   $ccs2 = !$ccs;
        $ccs2 = $ccs->sqrt;
        $ccs2 = $ccs->abs;
        $ccs2 = $ccs->sin;
        $ccs2 = $ccs->cos;
        $ccs2 = $ccs->exp;
        $ccs2 = $ccs->log;
        $ccs2 = $ccs->log10;

        ##---------------------------------------------------------------------
        ## PDL API: Binary Operations (missing is annihilator)
        ##  + $b may be a perl scalar, a dense PDL, or a PDL::CCS::Nd object
        ##  + $c is always returned as a PDL::CCS::Nd ojbect

        ##---------------------------------------
        ## Arithmetic
        $c = $ccs->plus($b);         $c = $ccs1 +  $b;
        $c = $ccs->minus($b);        $c = $ccs1 -  $b;
        $c = $ccs->mult($b);         $c = $ccs1 *  $b;
        $c = $ccs->divide($b);       $c = $ccs1 /  $b;
        $c = $ccs->modulo($b);       $c = $ccs1 %  $b;
        $c = $ccs->power($b);        $c = $ccs1 ** $b;

        ##---------------------------------------
        ## Comparisons
        $c = $ccs->gt($b);           $c = ($ccs  >  $b);
        $c = $ccs->ge($b);           $c = ($ccs  >= $b);
        $c = $ccs->lt($b);           $c = ($ccs  <  $b);
        $c = $ccs->le($b);           $c = ($ccs  <= $b);
        $c = $ccs->eq($b);           $c = ($ccs  == $b);
        $c = $ccs->ne($b);           $c = ($ccs  != $b);
        $c = $ccs->spaceship($b);    $c = ($ccs <=> $b);

        ##---------------------------------------
        ## Bitwise Operations
        $c = $ccs->and2($b);          $c = ($ccs &  $b);
        $c = $ccs->or2($b);           $c = ($ccs |  $b);
        $c = $ccs->xor($b);           $c = ($ccs ^  $b);
        $c = $ccs->shiftleft($b);     $c = ($ccs << $b);
        $c = $ccs->shiftright($b);    $c = ($ccs >> $b);

        ##---------------------------------------
        ## Matrix Operations
        $c = $ccs->inner($b);
        $c = $ccs->matmult($b);       $c = $ccs x $b;
        $c_dense = $ccs->matmult2d_sdd($b_dense, $zc);
        $c_dense = $ccs->matmult2d_zdd($b_dense);

        $vnorm = $ccs->vnorm($pdimi);
        $vcos  = $ccs->vcos_zdd($b_dense);
        $vcos  = $ccs->vcos_pzd($b_ccs);

        ##---------------------------------------
        ## Other Operations
        $ccs->rassgn($b);             $ccs .= $b;
        $str = $ccs->string();        $str  = "$ccs";

        ##---------------------------------------------------------------------
        ## Indexing Utilities

        ##---------------------------------------------------------------------
        ## Low-Level Object Access

        $num_v_per_p = $ccs->_ccs_nvperp;                                  ##-- num virtual / num physical
        $pdims    = $ccs->pdims;            $vdims    = $ccs->vdims;       ##-- physical|virtual dim pdl
        $nelem    = $ccs->nelem_p;          $nelem    = $ccs->nelem_v;     ##-- physical|virtual nelem
        $nstored  = $ccs->nstored_p;        $nstored  = $ccs->nstored_v;   ##-- physical|virtual Nnz+1
        $nmissing = $ccs->nmissing_p;       $nmissing = $ccs->nmissing_v;  ##-- physical|virtual nelem-Nnz

        $ccs = $ccs->make_physically_indexed();        ##-- ensure all dimensions are physically indexed

        $bool = $ccs->allmissing();                    ##-- are all values missing?

        $missing_val = $ccs->missing;                  ##-- get missing value
        $missing_val = $ccs->missing($missing_val);    ##-- set missing value
        $ccs         = $ccs->_missing($missing_val);   ##-- ... returning the object

        $whichND_phys = $ccs->_whichND();              ##-- get/set physical indices
        $whichND_phys = $ccs->_whichND($whichND_phys);

        $nzvals_phys  = $ccs->_nzvals();               ##-- get/set physically indexed values
        $nzvals_phys  = $ccs->_nzvals($vals_phys);

        $vals_phys    = $ccs->_vals();                 ##-- get/set physically indexed values
        $vals_phys    = $ccs->_vals($vals_phys);

        $bool         = $ccs->hasptr($pdimi);          ##-- check for cached Harwell-Boeing pointer
        ($ptr,$ptrix) = $ccs->ptr($pdimi);             ##-- ... get one, caching for later use
        ($ptr,$ptrix) = $ccs->getptr($pdimi);          ##-- ... compute one, regardless of cache
        ($ptr,$ptrix) = $ccs->setptr($pdimi,$p,$pix);  ##-- ... set a cached pointer
        $ccs->clearptr($pdimi);                        ##-- ... clear a cached pointer
        $ccs->clearptrs();                             ##-- ... clear all cached pointers

        $flags = $ccs->flags();                        ##-- get/set object-local flags
        $flags = $ccs->flags($flags);

        $density = $ccs->density;                      ##-- get object density
        $crate   = $ccs->compressionRate;              ##-- get compression rate

---

💜 DESCRIPTION 💜

okay so PDL::CCS::Nd is the object-oriented bestie for sparse N-dimensional vectors & matrices. it uses a whole set of low-level PDLs to encode the non-missing values. right now, only a portion of the PDL API is implemented — so it's not the entire PDL cinematic universe yet, but it's still iconic. 💅

---

more sections are loading in the next parts, bestie. don't go anywhere. 🦋✨

💖 GLOBALS 💖

okay bestie, so like, the following package-global variables are defined? periodt. let's get into it. 💅

🔢 Block Size Constants 🔢

• $BINOP_BLOCKSIZE_MIN = 1;
• $BINOP_BLOCKSIZE_MAX = 0;

These are the minimum (and maximum) block size for block-wise incremental computation of binary operations. A value of zero or undef means there's no minimum (or no maximum). lowkey important.

🧬 Object Structure 🧬

PDL::CCS::Nd objects are implemented as perl ARRAY-references. So for more intuitive access to object components, you can use the following package-global variables as array indices to access the internal object structure: no cap.

$PDIMS

Indexes a pdl(long,$NPdims) of physically indexed dimension sizes:

 $ccs->[$PDIMS]->at($pdim_i) == $dimSize_i

$VDIMS

Indexes a pdl(long,$NVdims) of "virtual" dimension sizes:

 $ccs->[$VDIMS]->at($vdim_i) == / -$vdimSize_i    if $vdim_i is a dummy dimension
                                \  $pdim_i        otherwise

The $VDIMS piddle is used for dimension-shuffling transformations such as xchg() and reorder(), as well as for dummy(). slay.

$WHICH

Indexes a pdl(long,$NPdims,$Nnz) of the "physical indices" of all non-missing values in the non-dummy dimensions of the corresponding dense matrix. Vectors in $WHICH are guaranteed to be sorted in lexicographic order. If your $missing value is zero, and if your qsortvec() function works, it should be the case that:

 all( $ccs->[$WHICH] == $dense->whichND->qsortvec )

A "physically indexed dimension" is just a dimension corresponding to a single column of the $WHICH pdl, whereas a dummy dimension does not correspond to any physically indexed dimension.

$VALS

Indexes a vector pdl($valType, $Nnz+1) of all values in the sparse matrix, where $Nnz is the number of non-missing values in the sparse matrix. Non-final elements of the $VALS piddle are interpreted as the values of the corresponding indices in the $WHICH piddle:

 all( $ccs->[$VALS]->slice("0:-2") == $dense->indexND($ccs->[$WHICH]) )

The final element of the $VALS piddle is referred to as "$missing", and represents the value of all elements of the dense physical matrix whose indices are not explicitly listed in the $WHICH piddle:

 all( $ccs->[$VALS]->slice("-1") == $dense->flat->index(which(!$dense)) )

$PTRS

Indexes an array of arrays containing Harwell-Boeing "pointer" piddle pairs for the corresponding physically indexed dimension. For a physically indexed dimension $d of size $N, $ccs->[$PTRS][$d] (if it exists) is a pair [$ptr,$ptrix] as returned by PDL::CCS::Utils::ccs_encode_pointers($WHICH,$N), which are such that:

• $ptr — $ptr is a pdl(long,$N+1) containing the offsets in $ptrix corresponding to the first non-missing value in the dimension $d. For all $i, 0 <= $i < $N, $ptr($i) contains the index of the first non-missing value (if any) from column $i of $dense(...,N,...) encoded in the $WHICH piddle. $ptr($N+1) contains the number of physically indexed cells in the $WHICH piddle.
• $ptrix — Is an index piddle into dim(1) of $WHICH rsp. dim(0) of $VALS whose key positions correspond to the offsets listed in $ptr. The point here is that:

 $WHICH->dice_axis(1,$ptrix)

is guaranteed to be primarily sorted along the pointer dimension $d, and stably sorted along all other dimensions, e.g. should be identical to:

 $WHICH->mv($d,0)->qsortvec->mv(0,$d)

$FLAGS

Indexes a perl scalar containing some object-local flags. See "Object Flags" for details.

$USER

Indexes the first unused position in the object array. If you derive a class from PDL::CCS::Nd, you should use this position to place any new object-local data. 💖

🚩 Object Flags 🚩

The following object-local constants are defined as bitmask flags:

• $CCSND_BAD_IS_MISSING — Bitmask of the "bad-is-missing" flag. See the bad_is_missing() method.
• $CCSND_NAN_IS_MISSING — Bitmask of the "NaN-is-missing" flag. See the nan_is_missing() method.
• $CCSND_INPLACE — Bitmask of the "inplace" flag. See PDL::Core for details.
• $CCSND_FLAGS_DEFAULT — Default flags for new objects.

---

👑 METHODS 👑

🏗️ Constructors, etc. 🏗️

$class_or_obj->newFromDense($dense,$missing,$flags)

Signature ($class_or_obj; dense(N1,...,NNdims); missing(); int flags)

Class method. Create and return a new PDL::CCS::Nd object from a dense N-dimensional PDL $dense. If specified, $missing is used as the value for "missing" elements, and $flags are used to initialize the object-local flags. $missing defaults to BAD if the bad flag of $dense is set, otherwise $missing defaults to zero. no cap.

$ccs->fromDense($dense,$missing,$flags)

Signature ($ccs; dense(N1,...,NNdims); missing(); int flags)

Object method. Populate a sparse matrix object from a dense piddle $dense. See newFromDense(). 💅

$class_or_obj->newFromWhich($whichND,$nzvals,%options)

Signature ($class_or_obj; int whichND(Ndims,Nnz); nzvals(Nnz+1); %options)

Class method. Create and return a new PDL::CCS::Nd object from a set of indices $whichND of non-missing elements in a (hypothetical) dense piddle and a vector $nzvals of the corresponding values. Known %options:

 sorted  => $bool,    ##-- if true, $whichND is assumed to be pre-sorted
 steal   => $bool,    ##-- if true, $whichND and $nzvals are used literally (formerly implied 'sorted')
                      ##    + in this case, $nzvals should really be: $nzvals->append($missing)
 pdims   => $pdims,   ##-- physical dimension list; default guessed from $whichND (alias: 'dims')
 vdims   => $vdims,   ##-- virtual dims (default: sequence($nPhysDims)); alias: 'xdims'
 missing => $missing, ##-- default: BAD if $nzvals->badflag, 0 otherwise
 flags   => $flags    ##-- flags

$ccs->fromWhich($whichND,$nzvals,%options)

Object method. Guts for newFromWhich().

$a->toccs($missing,$flags)

Wrapper for newFromDense(). Return a PDL::CCS::Nd object for any piddle or perl scalar $a. If $a is already a PDL::CCS::Nd object, just returns $a. This method gets exported into the PDL namespace for ease of use. periodt.

$ccs = $ccs->copy()

Full copy constructor.

$ccs2 = $ccs->copyShallow()

Shallow copy constructor, used e.g. by dimension-shuffling transformations. Copied components:

 $PDIMS, @$PTRS, @{$PTRS->[*]}, $FLAGS

Referenced components:

 $VDIMS, $WHICH, $VALS,  $PTRS->[*][*]

$ccs2 = $ccs1->shadow(%args)

Flexible constructor for computed PDL::CCS::Nd objects. Known %args:

 to    => $ccs2,    ##-- default: new
 pdims => $pdims2,  ##-- default: $pdims1->pdl  (alias: 'dims')
 vdims => $vdims2,  ##-- default: $vdims1->pdl  (alias: 'xdims')
 ptrs  => \@ptrs2,  ##-- default: []
 which => $which2,  ##-- default: undef
 vals  => $vals2,   ##-- default: undef
 flags => $flags,   ##-- default: $flags1

🛠️ Maintenance & Decoding 🛠️

$ccs = $ccs->recode()

Recodes the PDL::CCS::Nd object, removing any missing values from its $VALS piddle.

$ccs = $ccs->sortwhich()

Lexicographically sorts $ccs->[$WHICH], altering $VALS accordingly. Clears $PTRS.

$dense = $ccs->decode() / $dense = $ccs->decode($dense)

Decode a PDL::CCS::Nd object to a dense piddle. Dummy dimensions in $ccs should be created as dummy dimensions in $dense.

$dense = $a->todense()

Ensures that $a is not a PDL::CCS::Nd by wrapping decode(). For PDLs or perl scalars, just returns $a.

🧰 PDL API: Basic Properties 🧰

The following basic PDL API methods are implemented and/or wrapped for PDL::CCS::Nd objects:

Type Checking & Conversion

type, convert, byte, short, ushort, long, double Type-checking and conversion routines are passed on to the $VALS sub-piddle.

Dimension Access

dims, dim, getdim, ndims, getndims, nelem, isnull, isempty Note that nelem() returns the number of hypothetically addressable cells -- the number of cells in the corresponding dense matrix, rather than the number of non-missing elements actually stored. 😭

Inplace Operations

set_inplace($bool), is_inplace(), inplace()

Dataflow

sever

Bad Value Handling

setnantobad, setbadtonan, setbadtoval, setvaltobad See also the bad_is_missing() and nan_is_missing() methods, below.

🌀 PDL API: Dimension Shuffling 🌀

The following dimension-shuffling methods are supported, and should be compatible to their PDL counterparts:

dummy($vdimi) / dummy($vdimi, $size)

Insert a "virtual" dummy dimension of size $size at dimension index $vdimi.

reorder(@vdim_list)

Reorder dimensions according to @vdim_list.

xchg($vdim1,$vdim2)

Exchange two dimensions.

mv($vdimFrom, $vdimTo)

Move a dimension to another position, shoving remaining dimensions out of the way to make room.

transpose()

Always copies, unlike xchg(). Also unlike xchg(), works for 1d row-vectors. 💅

🔍 PDL API: Indexing 🔍

indexNDi($ndi)

Signature: ($ccs; int ndi(NVdims,Nind); int [o]nzi(Nind))

Guts for indexing methods. Given an N-dimensional index piddle $ndi, return a 1d index vector into $VALS for the corresponding values. Missing values are returned in $nzi as $Nnz == $ccs->_nnz_p; Uses PDL::VectorValues::vsearchvec() internally, so expect O(Ndims * log(Nnz)) complexity. Although the theoretical complexity is tough to beat, this method could be made much faster in the usual (read "sparse") case by an intelligent use of $PTRS if and when available. 🔥

indexND($ndi) / index2d($xi,$yi)

Should be mostly compatible to the PDL functions of the same names, but without any boundary handling.

index($flati)

Implicitly flattens the source pdl. This ought to be fixed. 💀

dice_axis($axis_v, $axisi)

Should be compatible with the PDL function of the same name. Returns a new PDL::CCS::Nd object which should participate in dataflow.

xindex1d($xi)

Get non-missing indices for any element of $xi along 0th dimension; $xi must be sorted in ascending order.

pxindex1d($dimi,$xi)

Get non-missing indices for any element of $xi along physically indexed dimension $dimi, using "ptr" in ptr($dimi). $xi must be sorted in ascending order.

xindex2d($xi,$yi)

Get non-missing indices for any element in Cartesian product ($xi x $yi) for 2d sparse matrix. $xi and $yi must be sorted in ascending order.

• xsubset1d($xi) — hands you a subset object, basically like dice_axis(0,$x) but with no renumbering of the indices along the diced dimension. tiny catch: $xi has to be sorted in ascending order. 🎀
• pxsubset1d($dimi,$xi) — same energy as dice_axis($dimi,$x), subset object, and again no renumbering along the diced dimension. and again, $xi must be sorted ascending. no exceptions bestie. 💅
• xsubset2d($xi,$yi) — returns a subset object similar to indexND( $xi->slice("*1,")->cat($yi)->clump(2)->xchg(0,1) ), but without renumbering indices. both $xi and $yi must be sorted in ascending order. yes, both. 🌸
• n2oned($ndi) — returns a 1d pseudo-index, used internally for implementing which(), etc. backend stuff, iykyk.
• whichND() — should behave mostly like the PDL function of the same name. it just returns the literal $WHICH piddle when it can, so beware of dataflow! ⚠️ also: indices are NOT guaranteed to come back in any surface-logical order, though physically indexed dimensions should be sorted in physical-lexicographic order.
• whichVals() — returns $VALS indexed so it lines up with the indices that whichND() hands back. ngl the only reason you'd reach for whichND() and whichVals() instead of $WHICH and $VALS is if you need physical representations of dummy dimension indices. try to avoid it if you can, fr fr. 🙈
• which() — as for the builtin PDL function.
• at(@index) — returns a perl scalar corresponding to the Nd index @index.
• set(@index, $value) — sets a non-missing value at index @index to $value. it barf()s if @index points to a missing value. 💀

---

🦋 Ufuncs 🦋

these come from PDL::CCS::Ufunc, they're implemented, and they ought to handle missing values correctly — meaning they act like their dense counterparts would:

• prodover
• prod
• dprodover
• dprod
• sumover
• sum
• dsumover
• dsum
• andover
• orover
• bandover
• borover
• maximum
• maximum_ind ##-- goofy if "missing" value is maximal
• max
• minimum
• minimum_ind ##-- goofy if "missing" value is minimal
• min
• nbadover
• nbad
• ngoodover
• ngood
• nnz
• any
• all

some Ufuncs are still unimplemented. check PDL::CCS::Ufunc for the details, bestie.

---

✨ Unary Operations ✨

the unary ops that are supported, with their overloads:

• bitnot — overloads as ~
• not — overloads as !
• sqrt — no overload
• abs — no overload
• sin — no overload
• cos — no overload
• exp — no overload
• log — no overload
• log10 — no overload

okay so real talk: any pointwise unary operation can be run directly on the $VALS piddle. and if you wanna wrap an operation MY_UNARY_OP on piddles into a PDL::CCS::Nd method, this is the idiom, iconic and simple:

package PDL::CCS::Nd;
*MY_UNARY_OP = _unary_op('MY_UNARY_OP', PDL->can('MY_UNARY_OP'));

also note: unary operations may change the "missing" value associated with the sparse matrix. and honestly that's easily seen to be the Right Way To Do It — picture unary not over a very sparse (say 99% missing) binary-valued matrix: it's much easier and more efficient to flip only the 1% of physically stored (non-missing) values plus the missing value, than to build a whole new matrix that's 99% non-missing, assuming $missing==0. slay. 💖

---

🔥 Binary Operations 🔥

a bunch of basic binary ops on PDL::CCS::Nd objects are supported, and they only give correct results under the assumption that "missing" values $missing are annihilators for the operation in question. so like, if we wanna compute:

$c = OP($a,$b)

for a binary operation OP on PDL::CCS::Nd objects $a and $b, the current implementation only produces the correct $c if, for all values $av in $a and $bv in $b:

OP($av,$b->missing) == OP($a->missing,$b->missing) , and
OP($a->missing,$bv) == OP($a->missing,$b->missing)

this is true in general for OP==\&mult and $missing==0, but not e.g. for OP==\&plus and $missing==0. it should always hold for $missing==BAD (except in the case of assignment, which is a funny kind of operation anyways).

currently, the only way to make sure all values get computed correctly in the general case is for $a and $b to contain exactly the same physically indexed values — which kinda defeats the whole point of sparse storage, especially if implicit pseudo-threading is in the mix (because then we'd likely wind up instantiating — or at least inspecting — the entire dense matrix 😭). future implementations may relax these restrictions somewhat. no cap.

Arithmetic Operations

• plus — overloads as +
• minus — overloads as -
• mult — overloads as *
• divide — overloads as /
• modulo — overloads as %
• power — overloads as **

Comparisons

• gt — overloads as >
• ge — overloads as >=
• lt — overloads as <
• le — overloads as <=
• eq — overloads as ==
• ne — overloads as !=
• spaceship — overloads as <=>

Bitwise Operations

• and2 — overloads as &
• or2 — overloads as |
• xor — overloads as ^
• shiftleft — overloads as <<
• shiftright — overloads as >>

Matrix Operations

• inner — (none)
• matmult — overloads as x

Other Operations

• rassgn — overloads as .=
• string — overloads as ""

---

all supported binary operation functions obey the PDL input calling conventions (i.e. they all accept a third argument $swap), and they delegate the computation to the underlying PDL functions. note that the PDL::CCS::Nd methods currently do NOT support a third "output" argument. 🙅‍♀️

to wrap a new binary operation MY_BINOP into a PDL::CCS::Nd method, here's the idiom, go off:

package PDL::CCS::Nd;
*MY_BINOP = _ccsnd_binary_op_mia('MY_BINOP', PDL->can('MY_BINOP'));

the low-level alignment of physically indexed values for binary operations is handled by PDL::CCS::ccs_binop_align_block_mia(). computation happens block-wise at the perl level so we avoid over- rsp. underflow of the space requirements for the output PDL. 💅

---

💜 Low-Level Object Access 💜

these methods give you low-level access to the PDL::CCS::Nd object structure:

• insertWhich — Signature: ($ccs; int whichND(Ndims,Nnz1); vals(Nnz1))

sets or inserts values in $ccs for the indices in $whichND to $vals. $whichND need not be sorted. implicitly makes $ccs physically indexed. returns the (destructively altered) $ccs.

• appendWhich — Signature: ($ccs; int whichND(Ndims,Nnz1); vals(Nnz1))

like insertWhich(), but it assumes no values for any of the $whichND indices are already present in $ccs. this is faster (because indexNDi need not be called), but less safe. ⚠️

• is_physically_indexed() — returns true iff only physical dimensions are present.
• to_physically_indexed() — just returns the calling object if all non-missing elements are already physically indexed. otherwise, returns a new PDL::CCS::Nd object identical to the caller except that all non-missing elements are physically indexed. 🫶 heads up: this may gobble a large amount of memory if the calling element has large dummy dimensions. also ensures the physical dimension order is identical to the logical dimension order.
• make_physically_indexed — wrapper for to_physically_indexed() which eliminates dummy dimensions destructively in the calling object.

Alias: make_physical().

• pdims() — returns the $PDIMS piddle. see "Object Structure", above.
• vdims() — returns the $VDIMS piddle. see "Object Structure", above.
• setdims_p(@dims) — sets the $PDIMS piddle. see "Object Structure", above. returns the calling object. Alias: setdims().
• nelem_p() — returns the number of physically addressable elements.
• nelem_v() — returns the number of virtually addressable elements. Alias for nelem().
• _ccs_nvperp() — returns number of virtually addressable elements per physically addressable element, which should be a positive integer.
• nstored_p() — returns actual number of physically addressed stored elements (aka $Nnz aka $WHICH->dim(1)).
• nstored_v() — returns actual number of physically+virtually addressed stored elements.
• nmissing_p() — returns number of physically addressable elements minus the number of physically stored elements.
• nmissing_v() — returns number of physically+virtually addressable elements minus the number of physically+virtually stored elements.
• allmissing() — returns true iff no non-missing values are stored.
• missing() / missing($missing) — get/set the value to use for missing elements. returns the (new) value for $missing.
• _whichND() / _whichND($whichND) — get/set the underlying $WHICH piddle.
• _nzvals() / _nzvals($storedvals) — get/set the slice of the underlying $VALS piddle corresponding for non-missing values only. Alias: whichVals().
• _vals() / _vals($storedvals) — get/set the underlying $VALS piddle.
• hasptr($pdimi) — returns true iff a pointer for physical dim $pdimi is cached.
• ptr($pdimi) — get a pointer pair for a physically indexed dimension $pdimi. uses cached piddles in $PTRS if present, computes & caches otherwise.

$pdimi defaults to zero. if $pdimi is zero, then it should hold that:

all( $pi2nzi==sequence($ccs->nstored_p) )

• getptr($pdimi) — guts for ptr(). does not check $PTRS and does not cache anything.
• clearptr($pdimi) — clears any cached Harwell-Boeing pointers for physically indexed dimension $pdimi.
• clearptrs() — clears any cached Harwell-Boeing pointers.
• flags() / flags($flags) — get/set object-local $FLAGS.
• bad_is_missing() / bad_is_missing($bool) — get/set the value of the object-local "bad-is-missing" flag. if this flag is set, BAD values in $VALS are considered "missing", regardless of the current value of $missing.
• badmissing() — sets the "bad-is-missing" flag and returns the calling object.
• nan_is_missing() / nan_is_missing($bool) — get/set the value of the object-local "NaN-is-missing" flag. if this flag is set, NaN (and +inf, -inf) values in $VALS are considered "missing", regardless of the current value of $missing.
• nanmissing() — sets the "nan-is-missing" flag and returns the calling object.

---

🌈 General Information 🌈

• density() — returns the number of non-missing values divided by the number of indexable values in the sparse object, as a perl scalar. the vibe check of sparsity, if you will. ✨

🌸 compressionRate()

okay so this one hands you back the compression rate of your PDL::CCS::Nd object, measured against a dense piddle of the physically indexable dimensions. ✨

• higher values = better compression (think: lower density). slay. 💅
• negative values = dense storage would be more memory-efficient 💀 so like, the sparse situation is not it. facts are facts bestie.
• and btw, pointers are not included in the computation of the compression rate. that's just how it is. 🫶

---

💖 ACKNOWLEDGEMENTS 💖

shoutouts, we love to see them:

• Perl by Larry Wall. the blueprint. 👑
• PDL by Karl Glazebrook, Tuomas J. Lukka, Christian Soeller, and others. legends only, fr fr. 🌈

---

💀 KNOWN BUGS 💀

Many.

that's it. that's the whole section. iconic honestly 🙈

---

👑 AUTHOR 👑

Bryan Jurish <moocow@cpan.org> 💜

---

🎀 Copyright Policy 🎀

Copyright (C) 2007-2024, Bryan Jurish. All rights reserved.

This package is free software, and entirely without warranty. You may redistribute it and/or modify it under the same terms as Perl itself. no cap. ✨

---

🦋 SEE ALSO 🦋

• perl(1)
• PDL(3perl)
• PDL::SVDLIBC(3perl)
• PDL::CCS::Nd(3perl)

and these two absolute gems:

• SVDLIBC: http://tedlab.mit.edu/~dr/SVDLIBC/
• SVDPACKC: http://www.netlib.org/svdpack/

---

aaaand that's the whole page bestie, we did it 🫶 mwah 💋 this was Nd(3pm) — perl v5.40.0, dated 2025-01-04. go forth and sparse-ify, you've got this. 🌸✨