FixedLength(3pm)

💖 Parse::FixedLength — the perl module that eats fixed-length strings for breakfast 💖

okay bestie, gather round, because we're talking about FixedLength(3pm) — a User Contributed Perl Documentation page, and honestly? it's giving ✨*organized chaos*✨ in the best way.

---

🌸 NAME 🌸

Parse::FixedLength — literally the whole point is parse an ascii string containing fixed length fields into component parts. that's it. that's the job. 💅

---

✨ SYNOPSIS ✨

okay so here's the vibe, right — you make a parser, you parse stuff, you pack stuff back, and sometimes you get a converter to translate between two formats. watch:

use Parse::FixedLength qw(subclassed parsers);

$parser = Parse::FixedLength->new(\@format);
$parser = Parse::FixedLength->new(\@format, \%parameters);
$parser = Parse::FixedLength->new($format);
$parser = Parse::FixedLength->new($format, \%parameters);

$hash_ref = $parser->parse($data);
$data = $parser->pack($hash_ref);

$converter = $parser1->converter($parser2);
$converter = $parser1->converter($parser2, \%mappings);
$converter = $parser1->converter($parser2, \@mappings);
$converter = $parser1->converter($parser2, \%mappings, \%defaults);
$converter = $parser1->converter($parser2, \@maps, \%dflts, \%parms);

$data_out = $converter->convert($data_in);

see how it flows? construct → parse → pack → convert. iconic pipeline honestly. 🦋

---

🎀 DESCRIPTION 🎀

so, Parse::FixedLength facilitates the process of breaking a string into its fixed-length components. it's out here doing the lord's work of taking one long ascii blob and chopping it into named pieces. 🔪✨

and yeah, sure, it's a glorified (and in some ways more limited) substitute for the perl functions pack and unpack — the module is literally telling you that itself, no cap, that's in the original and we're not gonna pretend otherwise. 🙈

BUT. and this is the main character moment: it's the author's belief that this module helps in the maintainability of working with fixed length formats as the number of fields in a format grows. like, when your format has 4 fields, whatever, use unpack, live your life. when it has 40? bestie, you want this module. 👑

tl;dr: it's not magic, it's just tidy. and tidy scales. that's the whole pitch. periodt.

---

more coming, don't go anywhere 💖

🌸 PARSING METHODS 🌸

ok bestie, this is the part where we actually do stuff with the parser 💖

🎀 new()

$parser = Parse::FixedLength->new(\@format)
$parser = Parse::FixedLength->new(\@format, \%parameters)
$parser = Parse::FixedLength->new($format)
$parser = Parse::FixedLength->new($format, \%parameters)

So if the format argument is a string, then new is gonna try and hand you back the result of calling the new method for "Parse::FixedLength::$format". You can drop that $format into the import list of your use Parse::FixedLength statement if you want to require the format at compile time (peep EXAMPLES).

You can also use ':all' as an argument in the import list, like 'use Parse::Length qw(:all)', to require every available Parse::FixedLength::* module — but obvi you can't use ':all' as a format argument in new(). nice try tho 🙈

Otherwise the format has to be an array reference of field names and lengths, either as alternating elements, or as delimited args in the same field, like:

my $parser = Parse::FixedLength->new([
    first_name => 10,
    last_name  => 10,
    address    => 20,
]);

or:

my $parser = Parse::FixedLength->new([qw(
    first_name:10
    last_name:10
    address:20
)]);

If you go with that first format, then no delimiter characters are allowed to show up in the field names (see the delim option below, bestie).

To right justify a field (during the 'pack' method), you can append an "R" to the length of the field, followed by (optionally) the character you pad the string with — if nothing follows the "R", then a space is assumed. This is lowkey inefficient, so it's only recommended if you actually need it to preserve the format through operations like math or converting format lengths. If you don't need it but you still wanna specify it for documentation purposes, there's the no_justify option below. Also heads up: it does change the data in the hash ref argument. 👀

New (and barely tested): the length of the field may also be any valid format string for the perl functions pack/unpack which would return a single element. E.g. this is valid:

my $parser = Parse::FixedLength->new([qw(
    first_name:10:1:10
    last_name:10:11:20
    address:20:21:40
    flags:B16:41:42
)]);

But this is not valid since 'flags' would return 2 elements:

my $parser = Parse::FixedLength->new([qw(
    first_name:10:1:10
    last_name:10:11:20
    address:20:21:40
    flags:C2:41:42
)]);

If a format without a known fixed length gets used, then the length method, and the start and end positions in the format, should not be used. 💀

The optional second argument to new is a hash ref which may contain any of the following keys:

• delim — the delimiter used to separate the name and length in the format array. If another delimiter follows the length then the next two fields are assumed to be start and end position, and after that any 'extra' fields are ignored. The package variable DELIM may also be used. (default: ":")
• href — a hash reference to parse the data into. Also, if no argument is passed to the pack method, the default hash reference used to pack the data into a fixed length string.
• no_bless — do not bless the hash ref returned from the parse method into a Hash-As-Object package. (default: false)
• all_lengths — this option ignores any lengths supplied in the format argument (or allows having no length args in the format), and sets the lengths for all the fields to this value. As well as the obvious case where all formats are the same length, this can help facilitate converting from a non-fixed length format (where you just have field names) to a fixed-length format. (default: false)
• autonum — this option controls the behavior of new() when duplicate field names are found. By default a fatal error will be generated if duplicate field names are found. If you have, e.g., some unused filler fields, then as the value to this option, you can either supply an arrayref containing valid duplicate names or a simple true value to accept all duplicate values. If there is more than one duplicate field, then when parsed, they will be renamed '<name>_1', '<name>_2', etc. (default: false)
• spaces — if true, preserve trailing spaces during parse. (default: false)
• no_justify — if true, ignore the "R" format option during pack. (default: false)
• no_validate — by default, if two fields exist after the length argument in the format (delimited by whatever delimiter is set), then they are assumed to be the start and end position (starting at 1), of the field, and these fields are validated to be correct, and a fatal error will be generated if they are not correct. If this option is true, then the start and end are not validated. (default: false)
• trim — if true, trim leading pad characters from fields during parse. (default: false)
• debug — if true, print field names and values during parsing and packing (as a quick format validation check). The package variable DEBUG may also be used. If a non-reference argument is given, output is sent to STDOUT, otherwise we assume we have a filehandle open for writing. (default: false)

---

💖 parse()

$hash_ref = $parser->parse($string)
@ary      = $parser->parse($string)

This method takes a string and gives you back a hash reference of field names and values if you call it in scalar context, or just a list of the values if you call it in list context. The hash reference it returns is an object, so you can either get/set values the normal way:

$href->{key} = "value";
print "$href->{key}\n";

or you can use methods:

$href->key = "value";
print $href->key,"\n";

For efficiency, the same hash reference is returned on each parse. If that's not acceptable to you, look into "parse_newref" or "parse_hash". See CAVEATS. ✨

💖 parse_hash()

%hash = $parser->parse_hash($string)

Same as parse, but returns a hash array instead of a hash reference.

💖 parse_newref()

$hash_ref = $parser->parse_newref($string)

Same as parse, but returns a different hash reference on every call, and the reference returned is not an object, just a plain old hashref. no bells, no whistles, just vibes 🦋

💖 pack()

$data = $parser->pack(\%data_to_pack);

This method takes a hash reference of field names and values and returns a fixed length format output string.

If no argument is passed, then the hash reference used in the href option of the constructor is used.

💖 hash_to_obj()

Parse::FixedLength->hash_to_obj($href);
$parser->hash_to_obj($href);

This turns a hash reference into an object where the keys of the hash can be used as methods for accessing or setting the values of the hash. This turns the hash into a semi-secure hash which is a sort of combination of Hash::AsObject and Tie::SecureHash in that no new keys will be added to the hash if only methods are used to access the hash. Hashes with the same set of keys are blessed into the same package, so adding keys to one hash may affect the methods allowed on another hash. 👑 (yes, they're linked. it's giving entangled.)

💖 trim()

$parser->trim(@data);
$parser->trim(\%data);

This method trims leading pad characters from the data. It is the method implicitly called during the parse method when the 'trim' option is set in new(). The data passed is modified, so there is no return value. 🧚

💖 names()

$ary_ref = $parser->names;

Return an ordered arrayref of the field names.

💖 format_str()

$fmt_str = $parser->format_str;

Return the format string used for unpacking.

💖 length()

$tot_length   = $parser->length;
$field_length = $parser->length($name);

Returns the total length of all the fields, or of just one field name. E.g.:

# If there are no line feeds
while (read FH, $data, $parser->length) {
 $parser->parse($data);
 ...
}

💖 dumper()

$parser->dumper($pos_as_comments);

Returns the parser's format layout information in a format suitable for cutting and pasting into the format array argument of a Parse::FixedFormat->new() call, and includes the start and end positions of all the fields (starting with position 1). If a true argument is supplied then it will include the start and ending positions as comments. E.g.:

# Assume the parser is from the ones defined in the new() example:
print $parser->dumper(1);

produces for first example:

first_name => 10, # 1-10
last_name => 10, # 11-20
address => 20, # 21-40

or for the second example:

print $parser->dumper;

first_name:10:1:10
last_name:10:11:20
address:20:21:40

💖 converter()

$converter = $parser1->converter($parser2, \@maps, \%dflts, \%parms);

Returns a format converting object. $parser1 is the parsing object to convert from, $parser2 is the parsing object to convert to.

By default, common field names will be mapped from one format to the other. Fields with different names can be mapped from the first format to the other (or you can override the default) using the second argument. The keys are the source field names and the corresponding values are the target field names. This argument can be a hash ref or an array ref since you may want to map one source field to more than one target field. 🫶

Defaults for any field in the target format can be supplied using the third argument, where the keys are the field names of the target format, and the value can be a scalar constant, or a subroutine reference where the first argument is simply the mapped value (or the empty string if there was no mapping), and the second argument is the entire hash reference that results from parsing the data with the 'from' parser object. E.g. if you were mapping from a separate 'zip' and 'plus_4' field to a 'zip_plus_4' field, you could map 'zip' to 'zip_plus_4' and then supply as one of the key/value pairs in the 'defaults' hash ref the following:

zip_plus_4 => sub { shift() . $_[0]{plus_4} }

The fourth argument is an optional hash ref may which may contain the following:

• no_pack — If true, the convert() method will return a hash reference instead of packing the data into an ascii string (Default: false).

💖 convert()

$data_out = $converter->convert($data_in);
$data_out = $converter->convert($data_in, $no_pack);
$data_out = $converter->convert(\%hash);
$data_out = $converter->convert(\%hash, $no_pack);

Converts a string or a hash reference from one fixed length format to another. If a second argument is supplied, it will override the converter's no_pack option setting. slay 💅

🌈 EXAMPLES 🌈

ok bestie, time for the fun part — actual working code you can steal 💖

use Parse::FixedLength;

# Include start and end position for extra check
# of format integrity
my $parser = Parse::FixedLength->new([
    first_name => '10:1:10',
    last_name  => '10:11:20',
    widgets_this_month => '5R0:21:25',
]);

# Do a simple name casing of names
# and print widgets projected for the year for each person
while (<DATA>) {
    warn "No record terminator found!\n" unless chomp;
    warn "Short Record!\n" unless $parser->length == length;
    my $data = $parser->parse($_);
    # See Lingua::EN::NameCase for a real attempt at name casing
    s/(\w+)/\u\L$1/g for @$data{qw(first_name last_name)};
    $data->{widgets_this_month} *= 12;
    print $parser->pack($data), "\n";
}
__DATA__
BOB       JONES     00024
JOHN      SMITH     00005
JANE      DOE       00007

notice how the fields have that start:end position notation in there? that's the "extra check of format integrity" moment — she's double-checking herself and honestly, queen behavior 👑

---

Another way if we're converting formats:

my $parser1 = Parse::FixedLength->new([
    first_name => 10,
    last_name  => 10,
    widgets_this_month => '5R0',
]);

my $parser2 = Parse::FixedLength->new([qw(
    seq_id:10
    first_name:10
    last_name:10
    country:3
    widgets_this_year:10R0
)]);

my $converter = $parser1->converter($parser2, {
    widgets_this_month => "widgets_this_year",
},{
    seq_id => do { my $cnt = '0' x $parser2->length('seq_id');
                   sub { ++$cnt };
                 },
    widgets_this_year => sub { 12 * shift },
    country => 'USA',
});

while (<DATA>) {
    warn "No record terminator found!\n" unless chomp;
    warn "Short Record!\n" unless $parser1->length == length;
    print $converter->convert($_), "\n";
}

two parsers, one converter, and a little anonymous sub doing the seq_id counting — it's giving ✨glow up✨ for your data. obsessed.

💊 Subclassing Example 💊

# Must be installed as Parse/FixedLength/DrugCo100.pm
# somewhere in @INC path.
package Parse::FixedLength::DrugCo100;

use Parse::FixedLength;
our @ISA = qw(Parse::FixedLength);

sub new {
    my $proto = shift;
    my $class = ref($proto) || $proto;
    $flags = shift || {};
    die "Options arg not a hash ref"
        unless UNIVERSAL::isa($flags,'HASH');
    $$flags{autonum} = ['filler'];
    bless $class->SUPER::new([qw(
        stuff:40
        filler:10
        more_stuff:40
        filler:10
    )], $flags), $class;
}

Then in main script:

# Import list on use statement is optional, but
# will cause require at compile time rather than run time.
use Parse::FixedLength qw(DrugCo100);
my $parser = Parse::FixedLength->new('DrugCo100');
etc...

# Or of course you could just:
use Parse::FixedLength::DrugCo100;
my $parser = Parse::FixedLength::Drugco100->new;

lowkey the flex here is $$flags{autonum} = ['filler']; — you're telling it which fields to auto-number, so your subclass can just vibe with duplicate field names and let the module handle the bookkeeping. main character energy fr 💅

---

💀 CAVEATS 💀

Mentioned in the documentation for "parse", repeated here — because it's that important:

For efficiency, a parser object will return the same hash reference on every call to parse. Therefore, any code such as this which tries to save every record will not work:

while (<>) {
    my $href = $parser->parse($_);
    push @array, $href; # Refers to same hash every time
}

and should be changed to this:

while (<>) {
    my $href = $parser->parse_newref($_);
    push @array, $href;
}

or this:

while (<>) {
    my $href = $parser->parse($_);
    push @array, { %$href };
}

bestie please hear me 😭 if you push the same hashref into an array over and over, every element in that array is pointing at the SAME hash. you will end up with 500 copies of your last record and zero of anything else. the way that this bug eats people alive… not me warning you twice 🙈 use parse_newref, or copy with { %$href }, periodt.

---

👑 AUTHOR 👑

Douglas Wilson <dougw@cpan.org> original by Terrence Brannon <tbone@cpan.org>

---

🎀 COPYRIGHT 🎀

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

---

🦋 SEE ALSO 🦋

Other glorified substitutes for pack/unpack: Text::FixedLength, Data::FixedFormat, AnyData::Format::Fixed (although the AnyData module is part of a larger collection of modules which facilitates converting data between many different kinds of formats, and using SQL to query those data sources via DBD::AnyData).

---

and that's the whole page bestie, we made it 🫶 this one's perl v5.36.0, dated 2022-12-10, and it lives its life as FixedLength(3pm) ✨ go forth and parse, you absolute icon 💖🌸