Fsdb::F...broweval(3)

📦
version 3.9 · 3 versions
, in epel 9

okay so welcome to Fsdb::F...broweval(3), straight outta the User Contributed Perl Documentation — and bestie, this page is literally about dbroweval doing code evaluation for every single row of an fsdb file 💅✨

💖 NAME 💖

dbroweval — evaluate code for each row of a fsdb file. That's the whole name, no cap.

✨ SYNOPSIS ✨

dbroweval [-f CodeFile] code [code...]

🦋 DESCRIPTION 🦋

Evaluate code for each row of the data. Typical actions are things like reformatting and other data transformations. Literally your row-by-row glow-up station. 💅

Code can include embedded column names preceded by underscores; these result in the value of that column for the current row. iykyk.

The values of the last row's columns are retrieved with _last_foo where foo is the column name.

Even more perverse, _columname(N) is the value of the Nth column after columnname [so _columnname(0) is also the column's value]. yes, that spelling is part of the vibe.

🌈 OPTIONS 🌈

  • -b CODE — run CODE before reading any data (like awk BEGIN blocks). pre-game ritual.
  • -e CODE — run CODE at the end of all data (like awk END blocks). post-game recap.
  • -f FILE — read code from the FILE.
  • -n or --no-output — no output except for comments and what is in the provided code.
  • -N or --no-output-even-comments — no output at all, except for what is in the provided code.
  • -m or --manual-output — the user must setup output, allowing arbitrary comments. See example 2 below for details.
  • -w or --warnings — enable warnings in user supplied code.
  • --saveoutput $OUT_REF — save output writer (for integration with other fsdb filters).

This module also supports the standard fsdb options (yes bestie, same fam):

🫶 standard fsdb options 🫶

  • -d — enable debugging output.
  • -i or --input InputSource — read from InputSource, typically a file name, or "-" for standard input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.
  • -o or --output OutputDestination — write to OutputDestination, typically a file name, or "-" for standard output, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.
  • --autorun or --noautorun — by default, programs process automatically, but Fsdb::Filter objects in Perl do not run until you invoke the run() method. The "--(no)autorun" option controls that behavior within Perl.
  • --header H — use H as the full Fsdb header, rather than reading a header from then input.
  • --help — show help.
  • --man — show full manual.

🔥 ADVANCED USAGE 🔥

Typically dbroweval outputs a line in the same schema for each input line. For advanced usage, one can violate each of these assumptions.

Some fun:

💨 omitting a line

Add the code next row if ($your condition);

🎁 outputting an extra line

Call &$write_fastpath_sub($fref). You may find $fref, the input row, useful.

🧬 changing the schema

See the examples below in "Command 2: Changing the Schema".

🌸 SAMPLE USAGE 🌸

🧪 Input:

#fsdb      size    mean    stddev  pct_rsd
1024    1.4962e+06      2.8497e+05      19.047
10240   5.0286e+06      6.0103e+05      11.952
102400  4.9216e+06      3.0939e+05      6.2863
#  | dbsetheader size bw
#  | /home/johnh/BIN/DB/dbmultistats size bw
#  | /home/johnh/BIN/DB/dbcol size mean stddev pct_rsd

🛠️ Command:

cat data.fsdb | dbroweval '_mean = sprintf("%8.0f", _mean); _stddev = sprintf("%8.0f", _stddev);'

🌈 Output:

#fsdb      size    mean    stddev  pct_rsd
1024     1496200          284970        19.047
10240    5028600          601030        11.952
102400   4921600          309390        6.2863
#  | dbsetheader size bw
#  | /home/johnh/BIN/DB/dbmultistats size bw
#  | /home/johnh/BIN/DB/dbcol size mean stddev pct_rsd
#  | /home/johnh/BIN/DB/dbroweval   { _mean = sprintf("%8.0f", _mean); _stddev = sprintf("%8.0f", _stddev); }

🎀 Command 2: Changing the Schema 🎀

By default, dbroweval reads and writes the same format file. The recommended method of adding and removing columns is to do so before or after dbroweval. I.e.,

cat data.fsdb |
    dbcolcreate divisible_by_ten |
    dbroweval '_divisible_by_ten = (_size % 10 == 0);' |
    dbrow '_divisible_by_ten == 1' |
    dbcol size mean divisible_by_ten

Another approach is to use the "next row" command to skip output of a row. I.e., the equivalent:

cat data.fsdb |
    dbcolcreate divisible_by_ten |
    dbroweval '_divisible_by_ten = (_size % 10 == 0); next row if (!_divisible_by_ten);' |
    dbcol size mean divisible_by_ten

However, neither of these approachs work very well when the output is a completely different schema.

The recommended method for schema-changing commands is to write a full filter, but a full filter is a bit heavy weight. As an alternative, one can use the -m option to request manual configuration of the output, then use @out_args to define the output schema (it specifies the Fsdb::IO::Writer arguments), and $ofref is the output row. It may also reference <$in>, the input Fsdb::IO::Reader argument, and <$fref> as an aref to the current line. Note that newly created columns do not have underscore-names.

Thus a third equivalent is:

cat data.fsdb | \
    dbroweval -m -b '@out_args = ( -clone => $in, \
             -cols => ($in->cols, divisible_by_ten); ' \
        'my $div_by_10 = (_size % 10 == 0); \
        $ofref = [ @$fref, $div_by_10 ] if ($div_by_ten);' |
    dbcol size mean divisible_by_ten

or

cat data.fsdb | \
    dbroweval -m -b '@out_args = ( -clone => $in, \
            -cols => [qw(size mean divisible_by_ten)] ); ' \
        'my $div_by_10 = (_size % 10 == 0);  \
        $ofref = [ _mean, _size, $div_by_10 ] if ($div_by_ten);'

Finally, one can write different a completely different schema, although it's more work:

cat data.fsdb | \
    dbroweval -m -b '@out_args = (-cols => [qw(size n)]);' \
        '$ofref = [ _size, 1 ];'

writes different columns, and

cat data.fsdb | \
    dbroweval -n -m -b '@out_args = (-cols => [qw(n)]);  \
        my $count = 0;' -e '$ofref = [ $count ];' '$count++;'

Is a fancy way to count lines.

The begin code block should setup @out_args to be the arguments to a Fsdb::IO::Writer::new call, and whatever is in $ofref (if anything) is written for each input line, and once at the end.

🕶️ Command 3: Fun With Suppressing Output 🕶️

The -n option suppresses default output. Thus, a simple equivalent to tail -1 is:

dbroweval -n -e '$ofref = $lfref;'

Where $ofref is the output fields, which are copied from $lfref, the hereby documented internal representation of the last row. Yes, this is a bit unappetizing, but, in for a penny with $ofref, in for a pound.

🙈 Command 4: Extra Ouptut 🙈

Calling &$write_fastpath_sub($fref) will do extra output, so this simple program will duplicate each line of input (one extra output, plus one regular output for each line of input):

dbroweval  '&$write_fastpath_sub($fref)'

💀 BUGS 💀

Handling of code in files isn't very elegant.

👀 SEE ALSO 👀

Fsdb(3)

🧚 CLASS FUNCTIONS 🧚

🌱 new

$filter = new Fsdb::Filter::dbroweval(@arguments);

🎛️ set_defaults

$filter->set_defaults();

Internal: set up defaults.

🧵 _confirm_ending_semicolon

Not a method; but an internal routine to make sure code compiles.

🧭 parse_options

$filter->parse_options(@ARGV);

Internal: parse options

🏗️ setup

$filter->setup();

Internal: setup, parse headers.

🏃 run

$filter->run();

Internal: run over all IO

🎬 finish

$filter->finish();

Internal: write trailer.

📊 compute_program_log

$log = $filter->figure_program_log();

Override compute_program_log to do pretty-printed arguments.

Copyright (C) 1991-2018 by John Heidemann <johnh@isi.edu>

This program is distributed under terms of the GNU general public license, version 2. See the file COPYING with the distribution for details.

okay that's the whole page, bestie 🫶 — brought to you by perl v5.32.1, dated 2024-08-02, and it's the Fsdb::F...broweval(3) manual. now go evaluate some rows, periodt 💅✨

📦 where this page comes from