Fsdb::IO(3)

💅 Fsdb::IO(3) 💅

okay so this is the Fsdb::IO(3) page from User Contributed Perl Documentation, and bestie, it’s giving base-class main character energy — like it literally says it’s the base class for Fsdb IO, aka FsdbReader and FsdbWriter 🫶✨

---

🌸 NAME 🌸

Fsdb::IO - base class for Fsdb IO (FsdbReader and FsdbWriter). periodt. 💖

✨ EXAMPLES ✨

There are several ways to do IO, and we look at several that compute the product of x and y for this input:

#fsdb x y product
1 10 -
2 20 -

The following routes go from most easy-to-use to least, and also from least efficient to most. For IO-intensive work, if fastpath takes 1 unit of time, then using hashes or arrays takes approximately 2 units of time, and that’s all due to CPU overhead. no cap. 🧚

💖 Using A Hash 💖

The hash route is lowkey the coziest:

use Fsdb::IO::Reader;
use Fsdb::IO::Writer;

# preamble
my $out;
my $in = new Fsdb::IO::Reader(-file => '-', -comment_handler => \$out)
    or die "cannot open stdin as fsdb\n";
$out = new Fsdb::IO::Writer(-file => '-', -clone => $in)
    or die "cannot open stdin as fsdb\n";

# core starts here
my %hrow;
while ($in->read_row_to_href(\%hrow)) {
    $hrow{product} = $hrow{x} * $hrow{y};
    $out->write_row_from_href(\%hrow);
};

It can be convenient to use a hash because one can easily extract fields using hash keys, but ngl, hashes can be slow. 🐢💅

🦋 Arrays Instead of Hashes 🦋

We can add a bit to the end of the preamble:

my $x_i = $in->col_to_i('x') // die "no x column.\n";
my $y_i = $in->col_to_i('y') // die "no y column.\n";
my $product_i = $in->col_to_i('product') // die "no product column.\n";

And then replace the core with arrays:

my @arow;
while ($in->read_row_to_aref(\@arow)) {
    $arow[$product_i] = $arow[$x_i] * $arow[$y_i];
    $out->write_row_from_aref(\@arow);
};

This code has two advantages over hrefs: First, there is explicit error checking for presence of the expected fields. Second, arrays are likely a bit faster than hashes. slay. 🔥

🧚 Objects Instead of Arrays 🧚

Keeping the same preamble as for arrays, we can directly get internal Fsdb “row objects” with a new core:

# core
my $rowobj;
while ($rowobj = $in->read_rowobj) {
    if (!ref($rowobj)) {
        # comment
        &{$in->{_comment_sub}}($rowobj);
        next;
    };
    $rowobj->[$product_i] = $rowobj->[$x_i] * $rowobj->[$y_i];
    $out->write_rowobj($rowobj);
};

This code is a bit faster because we just return the internal representation (a rowobj), rather than copy into an array. ✨

However, unfortunately it doesn't handle comment processing. 💀

🌈 Fastpathing 🌈

To go really fast, we can build a custom thunk (a chunk of code) that does exactly what we want. This approach is called a “fastpath”. 👑

It requires a bit more in the preamble (building on the array version):

my $in_fastpath_sub = $in->fastpath_sub();
my $out_fastpath_sub = $out->fastpath_sub();

And it allows a shorter core (modeled on rowobjs), since the fastpath includes comment processing:

my $rowobj;
while ($rowobj = &$in_fastpath_sub) {
    $rowobj->[$product_i] = $rowobj->[$x_i] * $rowobj->[$y_i];
    &$out_fastpath_sub($rowobj);
};

This code is the fastest way to implement this block without evaling code. periodt. 👑💜

FUNCTIONS ✨

okay so this is the function roster, bestie — every method, every internal, every tiny caveat. no skipping. 💅

new 💖

$fsdb = new Fsdb::IO;

Creates a new IO object. Usually you should not create a FsdbIO object directly, but instead create a FsdbReader or FsdbWriter — main character move.

Options:

• -fh FILE_HANDLE — write IO to the given file handle.
• -header HEADER_LINE — force the header to the given HEADER_LINE (should be verbatim, including #h or whatever). =back 😭 (yes, that =back is literally hanging out in the source).
• -fscode CODE — define just the column (or field) separator fscode part of the header. See dbfilealter for a list of valid field separators.
• -rscode CODE — define just the row separator part of the header. See dbfilealter for a list of valid row separators.
• -cols CODE — define just the columns of the header.
• -compression CODE — define the compression mode for the file that will take effect after the header.
• -clone $fsdb — copy the stream’s configuration from $FSDB, another Fsdb::IO object.

_reset_cols 🧚

$fsdb->_reset_cols

Internal: zero all the mappings in the curren schema. (source spelling, we love a tiny typo 🫶)

_find_filename_decompressor 🔍

returns the name of the decompression program for FILE if it ends in a compression extension.

config_one 🎀

$fsdb->config_one($arglist_aref);

Parse the first configuration option on the list, removing it.

Options are listed in new.

config ⚙️

$fsdb->config(-arg1 => $value1, -arg2 => $value2);

Parse all options in the list.

default_binmode 🌈

$fsdb->default_binmode();

Set the file to the correct binmode, either given by -encoding at setup, or defaulting from LC_CTYPE or LANG.

If the file is compressed, we will reset binmode after reading the header.

compare 🆚

$result = $fsdb->compare($other_fsdb)

Compares two Fsdb::IO objects, returning the strings "identical" (same field separator, columns, and column order), or maybe "compatible" (same field separator but different columns), or undef if they differ.

close 👋

$fsdb->close;

Closes the file, frees open file handle, or sends an EOF signal (and undef) down the open queue.

error 🚨

$fsdb->error;

Returns a descriptive string if there is an error, or undef if not.

The string will never end in a newline or punctuation.

update_v1_headerrow 🧱

internal: create the header the internal schema

parse_v1_headerrow 🧠

internal: interpet the header

update_headerrow 🧱✨

internal: create the header the internal schema

parse_headerrow 🧠✨

internal: interpet the v2 header. Format is:

#fsdb [-F x] [-R x] [-Z x] columns

All options must come first, start with dashes, and have an argument. (More regular than the v1 header.)

Columns have optional :t type specifiers.

parse_v1_fscode 🧩

internal

parse_fscode 🧩✨

Parse the field separator. See dbfilealter for a list of valid values.

parse_rscode 🧩🌈

Internal: Interpret rscodes.

See dbfilealter for a list of valid values.

parse_compression 🗜️

Internal: Interpret compression.

See dbfilealter for a list of valid values.

establish_new_col_mapping 🗺️

internal

col_create ➕

$fsdb->col_create($col_name)

Add a new column named $COL_NAME to the schema. Returns undef on failure, or 1 if sucessful. (Note: does not return the column index on creation because so that "or" can be used for error checking, given that the column number could be zero.) Also, update the header row to reflect this column (compare to "_internal_col_create").

colspec_to_name_type_spec 🏷️

($name, $type, $type_speced) = $fsdb->colspec_to_name_type($colspec)

Split a colspec into a name, type, and the type as specified (which may be null if no type was given).

_internal_col_create 🛠️

$fsdb->_internal_col_create($colspec)

For internal "Fsdb::IO" use only. Create a new column $COL_NAME, just like "col_create", but do not update the header row (as that function does).

field_contains_fs 🕵️‍♀️

$boolean = $fsdb->field_contains_fs($field);

Determine if the $FIELD contains $FSDB’s fscode (in which case it is malformed).

fref_contains_fs 🕵️‍♀️✨

$boolean = $fsdb->fref_contains_fs($fref);

Determine if any field in $FREF contains $FSDB’s fscode (in which case it is malformed).

correct_fref_containing_fs 🩹

$boolean = $fsdb->correct_fref_containing_fs($fref);

Patch up any field in $FREF contains $FSDB’s fscode, as best as possible, but turning the field separator into underscores. Updates $FREF in place, and returns if it was altered. This function looses data.

⚠️ bestie, this one looses data — the source said it, and the source meant it. 😭

fscode 🔤

$fscode = $fsdb->fscode;

Returns the fscode of the given database. (The encoded verison representing the field separator.) See also fs to get the actual field separator.

fs 🔡

$fscode = $fsdb->fs;

Returns the field separator. See "fscode" to get the "encoded" version.

rscode 🧵

$rscode = $fsdb->rscode;

Returns the rscode of the given database.

ncols 🔢

@fields = $fsdb->ncols;

Return the number of columns.

cols 📋

$fields_aref = $fsdb->cols;

Returns the column names (the field names, without type specifications) of the open database as an aref.

colspecs 📋✨

$fields_aref = $fsdb->colspecs();

Returns the column headings (the field names) of the open database as an aref.

col_to_i 🎯

$coli = $fsdb->col_to_i($column_name);

Returns the column index (0-based) of a given $COLUMN_NAME. (Names cannot have types with them.)

Note: tests for existence of columns must use "defined", since the index can be 0 which would be interpreted as false.

colspec_to_i 🎯✨

$coli = $fsdb->colspec_to_i($column_specification);

Returns the column index (0-based) of a given $COLUMN_NAME. Name may or may not include a type.

Note: tests for existence of columns must use "defined", since the index can be 0 which would be interpreted as false.

col_to_name 🏷️✨

@fields = $fsdb->col_to_name($column_name);

Returns the column anme a given $COLUMN_NAME_OR_INDEX. (source says anme, and we respect the typo 🫶)

col_to_type 🔠

@fields = $fsdb->col_to_type($column_name, $force_type);

Returns the column type (and undef if type is not required, unless $FORCE_TYPE) of a given $COLUMN_NAME.

col_to_colspec 🧾

@fields = $fsdb->col_to_colspec($column_name, $force_type);

Returns the column specification (type is optional, unless $FORCE_TYPE) of a given $COLUMN_NAME.

col_type_is_numeric 🔢✨

@fields = $fsdb->col_type_is_numeric($column_name);

Returns non-zero if column specification is numeric. (Actually, returns 1 for integers and 2 for floats.)

i_to_col 🎯🌈

@fields = $fsdb->i_to_col($column_index);

Return the name of the COLUMN_INDEX-th (0-based) column.

fastpath_cancel 🏎️💨

$fsdb->fastpath_cancel();

Discard any active fastpath code and allow fastpath-incompatible operations.

codify 🧬

($code, $has_last_refs) = $self->codify($underscored_pseudocode);

Convert db-code $UNDERSCORED_PSEUDOCODE into perl code in the context of a given Fsdb stream.

We return a string of code $CODE that refs "@{$fref}" and "@{$lfref}" for the current and prior row arrays, and a flag $HAS_LAST_REFS if "@{$lfref}" is needed. It is the callers job to set these up, probably by evaling the returned string in the context of those variables.n 😭 (that trailing n is in the source, bestie)

The conversion is a rename of all _foo's into database fields. For more perverse needs, _foo(N) means the Nth field after _foo. Also, as of 29-Jan-00, _last_foo gives the last row's value (_last_foo(N) is not supported). To convert we eval $codify_code.

20-Feb-07: _FROMFILE_foo opens the file called _foo and includes it in place.

NEEDSWORK: Should make some attempt to catch misspellings of column names.

clean_potential_columns 🧼

@clean = Fsdb::IO::clean_potential_columns(@dirty);

Clean up user-provided column names.

---

tiny sign-off: this page is serving perl v5.32.1 source/version energy, dated 2024-08-02. that’s the footer, but make it cute. 🎀✨💖