Compat(3pm)

📦
version 1.24.1 · 5 versions
, in debian trixie, ubuntu resolute

NAME 💖

okay so bestie, welcome to Compat(3pm) — that's the manual page, and it's coming to you from User Contributed Perl Documentation. This whole thing is about PDL::CCS::Compat, which is literally the backwards-compatibility module for PDL::CCS. The official NAME line is PDL::CCS::Compat - Backwards-compatibility module for PDL::CCS. no cap. 💅

✨ SYNOPSIS ✨

okay so here's the vibe: you're in Perl, you pull in PDL and PDL::CCS::Compat, and then it's giving this whole tour. watch:

        use PDL;
        use PDL::CCS::Compat;

        ##-- source pdl
        $a = random($N=8,$M=7);

        ##---------------------------------------------------------------------
        ## Non-missing value counts
        $nnz    = $a->flat->nnz;         ##-- "missing" == 0
        $nnaz   = $a->flat->nnza(1e-6);  ##-- "missing" ~= 0
        #$ngood = $a->ngood;             ##-- "missing" == BAD (see PDL::Bad)

        ##---------------------------------------------------------------------
        ## CCS Encoding
        ($ptr,$rowids,$vals) = ccsencode_nz ($a);             # missing == 0
        ($ptr,$rowids,$vals) = ccsencode_naz($a,$eps);        # missing ~= 0
        ($ptr,$rowids,$vals) = ccsencode_g  ($a);             # missing == BAD
        ($ptr,$rowids,$vals) = ccsencode_i  ($i,$ivals,$N);   # generic flat
        ($ptr,$rowids,$vals) = ccsencode_i2d($xi,$yi,$ivals); # generic 2d

        ##---------------------------------------------------------------------
        ## CCS Decoding
        $cols = ccsdecodecols($ptr,$rowids,$nzvals, $xvals
        $a2   = ccsdecode  ($ptr,$rowids,$vals);              # missing == 0
        $a2   = ccsdecode_g($ptr,$rowids,$vals);              # missing == BAD

        ##---------------------------------------------------------------------
        ## CCS Index Conversion
        $nzi  = ccsitonzi  ($ptr,$rowids, $ix,     $missing); # ix => nzi
        $nzi  = ccsi2dtonzi($ptr,$rowids, $xi,$yi, $missing); # 2d => nzi

        $ix       = ccswhich  ($ptr,$rowids,$vals);           # CCS => ix
        ($xi,$yi) = ccswhichND($ptr,$rowids,$vals);           # CCS => 2d
        $xyi      = ccswhichND($ptr,$rowids,$vals);           # ...as scalar

        ##---------------------------------------------------------------------
        ## CCS Lookup

        $ixvals = ccsget  ($ptr,$rowids,$vals, $ix,$missing);     # ix => values
        $ixvals = ccsget2d($ptr,$rowids,$vals, $xi,$yi,$missing); # 2d => values

        ##---------------------------------------------------------------------
        ## CCS Operations
        ($ptrT,$rowidsT,$valsT) = ccstranspose($ptr,$rowids,$vals); # CCS<->CRS

        ##---------------------------------------------------------------------
        ## Vector Operations, by column
        $nzvals_out = ccsadd_cv ($ptr,$rowids,$nzvals, $colvec);
        $nzvals_out = ccsdiff_cv($ptr,$rowids,$nzvals, $colvec);
        $nzvals_out = ccsmult_cv($ptr,$rowids,$nzvals, $colvec);
        $nzvals_out = ccsdiv_cv ($ptr,$rowids,$nzvals, $colvec);

        ##---------------------------------------------------------------------
        ## Vector Operations, by row
        $nzvals_out = ccsadd_rv ($ptr,$rowids,$nzvals, $rowvec);
        $nzvals_out = ccsdiff_rv($ptr,$rowids,$nzvals, $rowvec);
        $nzvals_out = ccsmult_rv($ptr,$rowids,$nzvals, $rowvec);
        $nzvals_out = ccsdiv_rv ($ptr,$rowids,$nzvals, $rowvec);

        ##---------------------------------------------------------------------
        ## Scalar Operations
        $nzvals_out = $nzvals * 42;  # ... or whatever

        ##---------------------------------------------------------------------
        ## Accumulators
        $rowsumover  = ccssumover ($ptr,$rowids,$nzvals); ##-- like $a->sumover()
        $colsumovert = ccssumovert($ptr,$rowids,$nzvals); ##-- like $a->xchg(0,1)->sumover

that's the synopsis, bestie. it's a lot, but every line is doing something. lowkey iconic. 🌈

🧚 Encoding 🧚

Encoding section, let's go! this is where you turn dense PDL stuff into CCS.

ccs_encode_compat

         Signature: (indx awhich(2,Nnz); avals(Nnz);
                     indx $N; indx $M;
                     indx [o]ptr(N); indx [o]rowids(Nnz); [o]nzvals(Nnz))

Generic wrapper for backwards-compatible ccsencode() variants. That's it. It's the little compatibility goblin. 🫶

ccsencode / ccsencode_nz

         Signature: (a(N,M); indx [o]ptr(N); indx [o]rowids(Nnz); [o]nzvals(Nnz))

Encodes matrix $a() in compressed column format, interpreting zeroes as "missing" values.

Allocates output vectors if required. So zeroes? missing. bye. ✨

ccsencodea / ccsencode_naz

         Signature: (a(N,M); eps(); indx [o]ptr(N); indx [o]rowids(Nnz); [o]nzvals(Nnz))

Encodes matrix $a() in CCS format interpreting approximate zeroes as "missing" values. This function is just like ccsencode_nz(), but uses the tolerance parameter $eps() to determine which elements are to be treated as zeroes.

Allocates output vectors if required. it's giving "close enough to zero, bestie". 💅

ccsencodeg / ccsencode_g

         Signature: (a(N,M); indx [o]ptr(N); indx [o]rowids(Nnz); [o]nzvals(Nnz))

Encodes matrix $a() in CCS format interpreting BAD values as "missing". Requires bad-value support built into PDL.

Requires bad-value support built into PDL. so make sure your PDL has that, or it's not gonna slay.

Allocates output vectors if required.

ccsencode_i

         Signature: (indx ix(Nnz); nzvals(Nnz); indx $N; int [o]ptr(N); indx [o]rowids(Nnz); [o]nzvals_enc(Nnz))

General-purpose CCS encoding method for flat indices. Encodes values $nzvals() from flat-index locations $ix() into a CCS matrix ($ptr(), $rowids(), $nzvals_enc()).

Allocates output vectors if required.

$N (~ $a->dim(0)) must be specified. fr fr, don't forget $N.

ccsencode_i2d

         Signature: (
                     indx    xvals(Nnz)     ;
                     indx    yvals(Nnz)     ;
                             nzvals(Nnz)    ;
                     indx    $N             ; ##-- optional
                     indx [o]ptr(N)         ;
                     indx [o]rowids(Nnz)    ;
                          [o]nzvals_enc(Nnz);
                    )

General-purpose encoding method. Encodes values $nzvals() from 2d-index locations ($xvals(), $yvals()) in an $N-by-(whatever) PDL into a CCS matrix $ptr(), $rowids(), $nzvals_enc().

Allocates output vectors if required. If $N is omitted, it defaults to the maximum column index given in $xvals(). cute. 🎀

🌸 Decoding 🌸

Now we're going the other way: CCS back to dense. reverse uno card, but make it matrix. 😭

ccsdecodecols

         Signature: (
                     indx   ptr    (N)  ;
                     indx   rowids (Nnz);
                            nzvals (Nnz);
                     indx   xvals  (I)  ; # default=sequence($N)
                            missing()   ; # default=0
                            M      ()   ; # default=rowids->max+1
                         [o]cols   (I,M); # default=new
                     )

Extract dense columns from a CCS-encoded matrix (no dataflow). Allocates output matrix if required. If $a(N,M) was the dense source matrix for the CCS-encoding, and if missing values are zeros, then the following two calls are equivalent (modulo data flow):

  $cols = $a->dice_axis(1,$col_ix);
  $cols = ccsdecodecols($ptr,$rowids,$nzvals, $col_ix,0);

so yeah, those two are the same vibe, except dataflow. iykyk.

ccsdecode

         Signature: (indx ptr(N); indx rowids(Nnz); nzvals(Nnz); $M; [o]dense(N,M))

Decodes compressed column format vectors $ptr(), $rowids(), and $nzvals() into dense output matrix $a(). Allocates the output matrix if required.

Note that if the original matrix (pre-encoding) contained trailing rows with no nonzero elements, such rows will not be allocated by this method (unless you specify either $M or $dense). In such cases, you might prefer to call ccsdecodecols() directly. so if trailing empty rows are your thing, call ccsdecodecols() instead. 💀

ccsdecode_g

         Signature: (indx ptr(N); indx rowids(Nnz); nzvals(Nnz); $M; [o]dense(N,M))

Convenience method. Like ccsdecode() but sets "missing" values to BAD. that's the whole thing. 💅

🦋 Index Conversion 🦋

Index Conversion time, bestie. we're translating indices from dense-land into CCS-land and back. it's giving passport control for arrays. 🛂✨

         Signature: (indx ptr(N); indx rowids(Nnz); indx ind(2,I); indx missing(); indx [o]nzix(I))

that signature is floating up top, and now the function it belongs to:

ccsiNDtonzi

Convert N-dimensional index values $ind() appropriate for a dense matrix (N,M) into indices $nzix() appropriate for the $rowids() and/or $nzvals() components of the CCS-encoded matrix ($ptr(),$rowids(),$nzvals()). Missing values are returned in $nzix() as $missing(). so if it's missing, you get $missing(). periodt.

ccsi2dtonzi

         Signaure: (indx ptr(N); indx rowids(Nnz); indx col_ix(I); indx row_ix(I); indx missing(); indx [o]nzix(I))

Convert 2d index values $col_ix() and $row_ix() appropriate for a dense matrix (N,M) into indices $nzix() appropriate for the $rowids() and/or $nzvals() components of the CCS-encoded matrix ($ptr(),$rowids(),$nzvals()). Missing values are returned in $nzix() as $missing(). also note the typo Signaure is literally in the original, so we keep it. no cap.

         Signature: (indx ptr(N); indx rowids(Nnz); indx ix(I); indx missing(); indx [o]nzix(I))

and here's that other signature, sitting before the next function:

ccsitonzi

Convert flat index values $ix() appropriate for a dense matrix (N,M) into indices $nzix() appropriate for the $rowids() and/or $nzvals() components of the CCS-encoded matrix ($ptr(),$rowids(),$nzvals()). Missing values are returned in $nzix() as $missing(). flat index in, CCS index out. slay.

ccswhichND / ccswhich2d / ccswhichfull

         Signature: (indx ptr(N); indx rowids(Nnz); nzvals(Nnz); indx [o]which_cols(Nnz); indx [o]which_rows(Nnz)',

In scalar context, returns concatenation of $which_cols() and $which_rows(), similar to the builtin whichND(). Note however that ccswhichND() may return its index PDLs sorted in a different order than the builtin whichND() method for dense matrices. Use the qsort() or qsorti() methods if you need sorted index PDLs. so don't assume same order as whichND(), bestie. sort it yourself with qsort() or qsorti() if you care. 🫶

ccswhich

         Signature: (indx ptr(N); indx rowids(Nnz); nzvals(Nnz); indx [o]which(Nnz); indx [t]wcols(Nnz)',

Convenience method. Calls ccswhichfull(), and scales the output PDLs to correspond to a flat enumeration. The output PDL $which() is not guaranteed to be sorted in any meaningful order. Use the qsort() method if you need sorted output. so if you need order, qsort() is right there. 💅

ccstranspose

         Signature: (indx ptr(N); indx rowids(Nnz); nzvals(Nnz); indx [o]ptrT(M); indx [o]rowidsT(Nnz); [o]nzvalsT(Nnz)',

Transpose a compressed matrix. that's it. CCS <-> CRS realness. 🔥

💜 Lookup 💜

Last section for this part: Lookup. we're grabbing values by index. it's giving "where's my data?" energy. 🔍

ccsget2d

         Signature: (indx ptr(N); indx rowids(Nnz); nzvals(Nnz); indx xvals(I); indx yvals(I); missing(); [o]ixvals(I))

Lookup values in a CCS-encoded PDL by 2d source index (no dataflow). Pretty much like ccsi2dtonzi(), but returns values instead of indices. If you know that your index PDLs $xvals() and $yvals() do not refer to any missing values in the CCS-encoded matrix, then the following two calls are equivalent (modulo dataflow):

  $ixvals =                ccsget2d   ($ptr,$rowids,$nzvals, $xvals,$yvals,0);
  $ixvals = index($nzvals, ccsi2dtonzi($ptr,$rowids,         $xvals,$yvals,0));

The difference is that only the second incantation will cause subsequent changes to $ixvals to be propagated back into $nzvals. so if you want that dataflow back-propagation, use the second one. iykyk.

ccsget

         Signature: (indx ptr(N); indx rowids(Nnz); nzvals(Nnz); indx ix(I); missing(); [o]ixvals(I))

Lookup values in a CCS-encoded PDL by flat source index (no dataflow). Pretty much like ccsitonzi(), but returns values instead of indices. If you know that your index PDL $ix() does not refer to any missing values in the CCS-encoded matrix, then the following two calls are equivalent (modulo dataflow):

  $ixvals =                ccsget   ($ptr,$rowids,$nzvals, $ix,0);
  $ixvals = index($nzvals, ccsitonzi($ptr,$rowids,         $ix,0))

The difference is that only the second incantation will cause subsequent changes to $ixvals to be propagated back into $nzvals.

🦋 Vector Operations 🦋

Okay bestie, this is the part where your CCS-encoded PDLs get their little vector glow-up. 💅 No decoding the whole matrix first — we stay sparse and cute. ✨

💖 ccs${OP}_cv

Signature: (indx ptr(N); indx rowids(Nnz); nzvals_in(Nnz);  colvec(M);  [o]nzvals_out(Nnz))

Column-vector operation ${OP} on a CCS-encoded PDL. 🎀

Should do something like the following — and no, we are not decoding the CCS matrix first:

($colvec ${OP} ccsdecode(\$ptr,\$rowids,\$nzvals))->ccsencode;

Missing values in the CCS-encoded PDL are not affected by this operation.

${OP} is one of the following, bestie:

  • plus — addition (alias: 'add')
  • minus — subtraction (alias: 'diff')
  • mult — multiplication (NOT matrix-multiplication)
  • divide — division (alias: 'div')
  • modulo — modulo
  • power — potentiation
  • gt — greater-than
  • ge — greater-than-or-equal
  • lt — less-than
  • le — less-than-or-equal
  • eq — equality
  • ne — inequality
  • spaceship — 3-way comparison
  • and2 — binary AND
  • or2 — binary OR
  • xor — binary XOR
  • shiftleft — left-shift
  • shiftright — right-shift

🎀 ccs${OP}_rv

Signature: (indx ptr(N); indx rowids(Nnz); nzvals_in(Nnz);  rowvec(N);  [o]nzvals_out(Nnz))

Row-vector operation ${OP} on a CCS-encoded PDL. Should do something like the following — again, without decoding the CCS matrix:

($column->slice("*1,") ${OP} ccsdecode($ptr,$rowids,$nzvals))->ccsencode;

Missing values in the CCS-encoded PDL are not affected by this operation.

See ccs${OP}_cv() above for supported operations. 👑

🍓 EXAMPLES 🍓

🌈 Compressed Column Format Example

$a = pdl([
          [10, 0, 0, 0,-2,  0],
          [3,  9, 0, 0, 0,  3],
          [0,  7, 8, 7, 0,  0],
          [3,  0, 8, 7, 5,  0],
          [0,  8, 0, 9, 9, 13],
          [0,  4, 0, 0, 2, -1]
         ]);

($ptr,$rowids,$nzvals) = ccsencode($a);

print join("\n", "ptr=$ptr", "rowids=$rowids", "nzvals=$nzvals");

... prints something like:

ptr=[0 3 7 9 12 16]
rowids=[ 0 1 3 1 2 4 5 2 3 2 3 4  0 3 4 5 1  4  5]
nzvals=[10 3 3 9 7 8 4 8 8 7 7 9 -2 5 9 2 3 13 -1]

✨ Sparse Matrix Example

##-- create a random sparse matrix
$a  = random(100,100);
$a *= ($a>.9);

##-- encode it
($ptr,$rowids,$nzvals) = ccsencode($a);

##-- what did we save?
sub pdlsize { return PDL::howbig($_[0]->type)*$_[0]->nelem; }
print "Encoding saves us ",
      ($saved = pdlsize($a) - pdlsize($ptr) - pdlsize($rowids) - pdlsize($nzvals)),
      " bytes (", (100.0*$saved/pdlsize($a)), "%)\n";

... prints something like:

Encoding saves us 71416 bytes (89.27%)

💖 Decoding Example

##-- random matrix
$a = random(100,100);

##-- make an expensive copy of $a by encoding & decoding
($ptr,$rowids,$nzvals) = ccsencode($a);
$a2 = ccsdecode($ptr,$rowids,$nzvals);

##-- ...and make sure it's good
print all($a==$a2) ? "Decoding is good!\n" : "Nasty icky bug!\n";

🌟 ACKNOWLEDGEMENTS 🌟

  • Perl by Larry Wall.
  • PDL by Karl Glazebrook, Tuomas J. Lukka, Christian Soeller, and others.
  • Original inspiration and algorithms from the SVDLIBC C library by Douglas Rohde; which is itself based on SVDPACKC by Michael Berry, Theresa Do, Gavin O'Brien, Vijay Krishna and Sowmini Varadhan.

💀 KNOWN BUGS 💀

Many. Periodt. 😭

👑 AUTHOR 👑

Bryan Jurish <moocow@cpan.org>

Copyright (C) 2005-2024, Bryan Jurish. All rights reserved.

This package is free software, and entirely without warranty. You may redistribute it and/or modify it under the same terms as Perl itself.

🔥 SEE ALSO 🔥

This little masterpiece is from perl v5.40.0, dated 2025-01-04, and it's the Compat(3pm) page. Mwah 💋

📦 where this page comes from