PIR(3pm)

📦
version 1.015 · 5 versions
, in debian bookworm, debian trixie, ubuntu noble, ubuntu resolute, ubuntu stonking

💖 PIR 💖

okay bestie, listen up — this is the manual page for PIR, from the User Contributed Perl Documentation collection, living in section 3pm. It's short, it's sweet, it's literally a shortcut. Let's get into it 🦋

✨ NAME ✨

PIR — a short alias for Path::Iterator::Rule. that's it. that's the whole name. 💅

⭐ VERSION ⭐

version 1.015

🎀 SYNOPSIS 🎀

  use PIR;

  my $rule = PIR->new;          # match anything
  $rule->file->size(">10k");    # add/chain rules

  # iterator interface
  my $next = $rule->iter( @dirs );
  while ( defined( my $file = $next->() ) ) {
    ...
  }

  # list interface
  for my $file ( $rule->all( @dirs ) ) {
    ...
  }

🌸 DESCRIPTION 🌸

so PIR is an empty subclass of Path::Iterator::Rule. literally empty. no extra code, no secret sauce. what it does do is save you from typing out that whole entire name over and over again, which is highkey a lifesaver when you're writing one-liners:

    $ perl -MPIR -wE 'say for PIR->new->skip_dirs(".")->perl_module->all(@INC)'

see? cute, short, slays. 🫶

🌈 USAGE 🌈

🧚 Constructors

new

  my $rule = Path::Iterator::Rule->new;

makes a brand new rule object that matches any file or directory. it takes no arguments, periodt. for convenience you can also call it on an object — and yeah, it still hands you back a fresh new object that matches any file or directory. iconic behavior honestly. 💖

clone

  my $common      = Path::Iterator::Rule->new->file->not_empty;
  my $big_files   = $common->clone->size(">1M");
  my $small_files = $common->clone->size("<10K");

makes a copy of a rule object. so useful when you want to customize a bunch of different rule objects that all start from one shared base. main character energy for shared config. 👑

🔥 Matching and iteration

iter

  my $next = $rule->iter( @dirs, \%options);
  while ( defined( my $file = $next->() ) ) {
    ...
  }

creates a subroutine reference iterator that returns a single result when you dereference it. and the tea is: this iterator is "lazy" — results are not pre-computed. it gives them to you as you go. 🦋

it takes, as arguments, a list of directories to search plus an optional hash reference of control options. if you don't hand it any search directories, it just uses the current directory ("."). valid options include:

  • depthfirst — controls the order of results. valid values are 1 (post-order, depth-first search), 0 (breadth-first search) or -1 (pre-order, depth-first search). default is 0.
  • error_handler — catches errors during execution of rule tests. the default handler dies with the filename and the error. if you set it to undef, error handling is disabled.
  • follow_symlinks — follow directory symlinks when true. default is 1.
  • report_symlinks — includes symlinks in results when true. default is equal to follow_symlinks.
  • loop_safe — prevents visiting the same directory more than once when true. default is 1.
  • relative — return matching items relative to the search directory. default is 0.
  • sorted — whether entries in a directory are sorted before processing. default is 1.
  • visitor — an optional coderef that will be called on items matching all rules.

okay so filesystem loops might exist, from hard links or soft links, either one. the loop_safe option is what prevents infinite loops — but ngl it adds some overhead by making stat calls. 😭 and because directories are visited only once when loop_safe is true, matches could actually come from a symlinked directory before the real directory, depending on the search order. so watch out for that.

want only the real files? turn off follow_symlinks. you can also have symlinks included in results without descending into symlink directories — turn off follow_symlinks but turn on report_symlinks. bestie that combo is the move. ✨

turning loop_safe off while leaving follow_symlinks on avoids stat calls and will be the fastest — but with the risk of an infinite loop and repeated files. so: the default is slow, but safe. choose your fighter.

the error_handler parameter must be a subroutine reference. it gets called when a rule test throws an exception. first argument is the file name being inspected, second argument is the exception. simple.

the optional visitor parameter must also be a subroutine reference. if you set it, it gets called for any result that matches. it's called the same way a custom rule would be (peep "EXTENDING") but its return value is ignored. also important: it's called when an item is first inspected — "postorder" is not respected. noted. 📝

the paths inspected and returned will be relative to the search directories you provided. absolute search dirs → absolute paths back. relative search dirs → relative paths back. math is math. 💅

if the search directories are absolute and the relative option is true, files returned will be relative to the search directory. but heads up: if your search directories aren't mutually exclusive (say, containing subdirectories like @INC, or symbolic links), files found could come back relative to different initial search directories — depending on depthfirst, follow_symlinks or loop_safe. yeah. it's giving chaos but it's documented chaos. 🌀

when the iterator is exhausted, it returns undef. that's your cue to stop looping. 🛑

iter_fast

works just like iter, except it optimizes for speed over safety. don't do this unless you're sure you need it and you accept the consequences. see "PERFORMANCE" for the details. fr fr. 💀

all

  my @matches = $rule->all( @dir, \%options );

returns a list of paths that match the rule. takes the same arguments and has the same behaviors as the iter method. internally, all uses iter to fetch every result. 🍓

in scalar context, it returns the count of matched paths.

in void context, it's optimized to iterate over everything but not store results. this is most useful with the visitor option:

    $rule->all( $path, { visitor => \&callback } );

all_fast

works just like all, except it optimizes for speed over safety. don't do this unless you're sure you need it and you accept the consequences. see "PERFORMANCE" for the details. (yes, same warning as iter_fast. the vibes are consistent.) 😌

test

  if ( $rule->test( $path, $basename, $stash ) ) { ... }

tests a file path against a rule. it's used internally, but it's provided in case someone out there wants to build their own custom iteration algorithm. go off, king. 👑

💜 Logic operations

Path::Iterator::Rule provides three logic operations for adding rules to the object. a rule may be either a subroutine reference with specific semantics (described below in "EXTENDING") or another Path::Iterator::Rule object. either works, no cap. 🦋

and

  $rule->and( sub { -r -w -x $_ } ); # stacked filetest example
  $rule->and( @more_rules );

adds one or more constraints to the current rule. so like: old rule AND new1 AND new2 AND .... returns the object so you can keep chaining methods like the icon you are. 💅

or

  $rule->or(
    $rule->new->name("foo*"),
    $rule->new->name("bar*"),
    sub { -r -w -x $_ },
  );

takes one or more alternatives and adds them as a constraint to the current rule. so like: old rule AND ( new1 OR new2 OR ... ). returns the object for chaining. ✨

not

  $rule->not( sub { -r -w -x $_ } );

takes one or more alternatives and adds them as a negative constraint to the current rule. so like: old rule AND NOT ( new1 AND new2 AND ...). returns the object for chaining. 💅

skip

  $rule->skip(
    $rule->new->dir->not_writeable,
    $rule->new->dir->name("foo"),
  );

takes one or more alternatives and will prune a directory if any of the criteria match — or if any of the rules already indicate the directory should be pruned. and what does pruning mean, bestie? it means the directory will not be returned by the iterator and will not be searched. gone. not today. 🚫

for files, it's equivalent to $rule->not($rule->or(@rules)). returns the object so you can chain.

call this method as early as possible in the rule chain. see "skip_dirs" below for further explanation and an example. don't skip the skip tip. 🙈

…and that's part 1! more coming, don't go anywhere 🎀

✨ RULE METHODS ✨

okay so rule methods are these little helper besties that add constraints. Under the hood, they generate a closure to do the exact logic they want and then add it to the rule object using the and method. And they return the object so you can keep chaining methods like it's nothing. slay 💅

🎀 File name rules

name

$rule->name( "foo.txt" );
$rule->name( qr/foo/, "bar.*");

The name method takes one or more patterns and creates a rule that's true if any of those patterns match the basename of the file or directory path. Patterns may be regular expressions or glob expressions (or literal names). iconic ✨

iname

$rule->iname( "foo.txt" );
$rule->iname( qr/foo/, "bar.*");

The iname method is literally just like the name method, but it matches case-insensitively. periodt 💖

skip_dirs

$rule->skip_dirs( @patterns );

The skip_dirs method skips directories that match one or more patterns. Patterns may be regular expressions or globs (just like name). Directories that match will not be returned from the iterator and will be excluded from further search. This includes the starting directories. If that isn't what you want, see skip_subdirs instead. no cap 🦋

Note: this rule should be specified early so that it has a chance to operate before a logical shortcut. E.g.
$rule->skip_dirs(".git")->file; # OK
$rule->file->skip_dirs(".git"); # Won't work

In the latter case, when a .git directory is seen, the file rule shortcuts the rule before the skip_dirs rule has a chance to act. bestie, order matters fr fr 💀

skip_subdirs

$rule->skip_subdirs( @patterns );

This works just like skip_dirs, except that the starting directories (depth 0) are not skipped and may be returned from the iterator unless excluded by other rules. the way that's so specific 🎀

🧪 File test rules

Most of the -X style filetest are available as boolean rules. The mapping from filetest to method name is literally this:

  • -r → readable
  • -R → r_readable
  • -w → writeable
  • -W → r_writeable
  • -w → writable
  • -W → r_writable
  • -x → executable
  • -X → r_executable
  • -o → owned
  • -O → r_owned
  • -e → exists
  • -f → file
  • -z → empty
  • -d → directory, dir
  • -s → nonempty
  • -l → symlink
  • -p → fifo
  • -u → setuid
  • -S → socket
  • -g → setgid
  • -b → block
  • -k → sticky
  • -c → character
  • -t → tty
  • -T → ascii
  • -B → binary

For example:

$rule->file->nonempty; # -f -s $file

The -X operators for timestamps take a single argument in a form that Number::Compare can interpret.

  • -A → accessed
  • -M → modified
  • -C → changed

For example:

$rule->modified(">1"); # -M $file > 1

obsessed with how clean that is 🌈

📊 Stat test rules

All of the stat elements have a method that takes a single argument in a form understood by Number::Compare.

  • 0 → dev
  • 1 → ino
  • 2 → mode
  • 3 → nlink
  • 4 → uid
  • 5 → gid
  • 6 → rdev
  • 7 → size
  • 8 → atime
  • 9 → mtime
  • 10 → ctime
  • 11 → blksize
  • 12 → blocks

For example:

$rule->size(">10K")

the numbers are numbering 🔥

📏 Depth rules

$rule->min_depth(3);
$rule->max_depth(5);

The min_depth and max_depth rule methods take a single argument and limit the paths returned to a minimum or maximum depth (respectively) from the starting search directory. A depth of 0 means the starting directory itself. A depth of 1 means its children. (This is similar to the Unix find utility.) iykyk 👑

🐪 Perl file rules

# All perl rules
$rule->perl_file;

# Individual perl file rules
$rule->perl_module;     # .pm files
$rule->perl_pod;        # .pod files
$rule->perl_test;       # .t files
$rule->perl_installer;  # Makefile.PL or Build.PL
$rule->perl_script;     # .pl or 'perl' in the shebang

These rule methods match file names (or a shebang line) that are typical of Perl distribution files. Perl besties rise up 🐪💚

🗂️ Version control file rules

# Skip all known VCS files
$rule->skip_vcs;

# Skip individual VCS files
$rule->skip_cvs;
$rule->skip_rcs;
$rule->skip_svn;
$rule->skip_git;
$rule->skip_bzr;
$rule->skip_hg;
$rule->skip_darcs;

Skips files and/or prunes directories related to a version control system. Just like skip_dirs, these rules should be specified early to get the correct behavior. git gud 😌

📄 File content rules

contents_match

$rule->contents_match(qr/BEGIN .* END/xs);

The contents_match rule takes a list of regular expressions and returns files that match one of the expressions.

The expressions are applied to the file's contents as a single string. For large files, this is likely to take significant time and memory. fr fr, be careful with the big ones 😭

Files are assumed to be encoded in UTF-8, but alternative Perl IO layers can be passed as the first argument:

$rule->contents_match(":encoding(iso-8859-1)", qr/BEGIN .* END/xs);

See perlio for further details. deep lore 🧚

line_match

$rule->line_match(qr/^new/i, qr/^Addition/);

The line_match rule takes a list of regular expressions and returns files with at least one line that matches one of the expressions.

Files are assumed to be encoded in UTF-8, but alternative Perl IO layers can be passed as the first argument. line by line, bestie 💛

shebang

$rule->shebang(qr/#!.*\bperl\b/);

The shebang rule takes a list of regular expressions or glob patterns and checks them against the first line of a file. shebang shebang 🎤

✨ Other rules

dangling

$rule->symlink->dangling;
$rule->not_dangling;

The dangling rule method matches dangling symlinks. Use it or its inverse to control how dangling symlinks should be treated. choose your fighter 🌈

🚫 Negated rules

Most rule methods have a negated form preceded by not_.

$rule->not_name("foo.*")

Because this happens automatically, it includes somewhat silly ones like not_nonempty (which is thus a less efficient way of saying empty). not me finding that lowkey funny 🙈

Rules that skip directories or version control files do not have a negated version. no negating those, periodt 💜

✨ EXTENDING ✨

💅 Custom rule subroutines

Okay so bestie, rules? They're just (usually anonymous) subroutine callbacks that hand back a value telling you whether or not the rule matched. That's it. That's the whole vibe. 🎀

These callbacks get called with three arguments:

  1. A path — which is also locally aliased as the $_ global variable, so your simple little one-liners stay cute and short:
$rule->and( sub { -r -w -x $_ } ); # tests $_
  1. The basename of the path. Super handy for name-check type stuff:
$rule->and( sub { $_[1] =~ /foo|bar/ } ); "foo" or "bar" in basename;
  1. A hash reference you can use to keep state. Keys that start with an underscore are reserved for Path::Iterator::Rule to stash extra data about the search in progress. Like, _depth is the key that powers the min/max depth checks, iykyk.

Now, your custom rule subroutine has to return one of four things:

  • A true value — constraint is satisfied ✅
  • A false value — constraint is not satisfied ❌
  • \1 — constraint is satisfied, and prune if it's a directory ✂️
  • \0 — constraint is not satisfied, and prune if it's a directory ✂️

A reference is basically a special little flag screaming "do NOT recurse into this directory," no matter whether the directory itself should get returned by the iterator or not.

🚨 The legacy "0 but true" value that used to mean pruning? It is no longer valid. If it gets detected it will throw an exception. Periodt. Don't do it.

Here's an example — this is literally equivalent to the max_depth rule method with a depth of 3:

$rule->and(
  sub {
    my ($path, $basename, $stash) = @_;
    return 1 if $stash->{_depth} < 3;
    return \1 if $stash->{_depth} == 3;
    return \0; # should never get here
  }
);

Files and directories up to depth 3 will be returned, and directories will be searched. Files of depth 3 will be returned. Directories of depth 3 will be returned, but their contents will not be added to the search. 🦋

Returning a reference is "sticky" — those babies propagate through and and or logic:

    0 && \0 = \0    \0 && 0 = \0    0 || \0 = \0    \0 || 0 = \0
    0 && \1 = \0    \0 && 1 = \0    0 || \1 = \1    \0 || 1 = \1
    1 && \0 = \0    \1 && 0 = \0    1 || \0 = \1    \1 || 0 = \1
    1 && \1 = \1    \1 && 1 = \1    1 || \1 = \1    \1 || 1 = \1

And once a directory is flagged to be pruned, it is pruned regardless of subsequent rules. No take-backs, no second chances:

    $rule->max_depth(3)->name(qr/foo/);

This returns files or directories with "foo" in the name, but all directories at depth 3 get pruned, whether or not they match the name rule. Sorry girl, the prune already happened. 💀

Generally speaking, if you want to do directory pruning, you are encouraged to use the skip method instead of writing your own logic with \0 and \1. Trust.

🧚 Extension modules and custom rule methods

One of the big strengths of File::Find::Rule is all the CPAN modules that extend it. Path::Iterator::Rule gives you the add_helper method to provide a similar extension mechanism. 🌈

The add_helper class method takes three arguments: a name for the rule method, a closure-generating callback, and a flag for not generating a negated form of the rule. Unless that flag is true, an inverted not_* method gets generated automatically. Extension classes should call this as a class method to install new rule methods. For example, this adds a foo method that checks if the filename is "foo":

package Path::Iterator::Rule::Foo;

use Path::Iterator::Rule;

Path::Iterator::Rule->add_helper(
  foo => sub {
    my @args = @_; # do this to customize closure with arguments
    return sub {
      my ($item, $basename) = @_;
      return if -d "$item";
      return $basename =~ /^foo$/;
    }
  }
);

1;

Which then unlocks these rule methods for you:

$rule->foo;
$fule->not_foo;
🙈 Heads up: add_helper will warn and ignore a helper that has the same name as an existing method. So don't come for a name that's already taken, bestie.

👑 Subclassing

Instead of processing and returning strings, this module can be subclassed to operate on objects that represent files. Those objects just have to stringify to a file path. Easy.

These private implementation methods must be overridden:

  • _objectify — given a path, return an object
  • _children — given a directory, return an (unsorted) list of [ basename, full path ] entries inside it, excluding "." and ".."

Note that _children should return a list of tuples, where the tuples are array references containing basename and full path.

See Path::Class::Rule source for an example. ✨

🌸 LEXICAL WARNINGS 🌸

If you run with lexical warnings enabled, Path::Iterator::Rule will issue warnings in certain circumstances (like, say, an unreadable directory that has to be skipped). To disable these categories, put the following statement at the correct scope:

no warnings 'Path::Iterator::Rule';

🔥 PERFORMANCE 🔥

By default, Path::Iterator::Rule iterator options are "slow but safe". They ensure uniqueness, return files in sorted order, and throw nice error messages if something goes wrong. Very demure, very mindful. 🎀

If you want speed over safety, set these options:

    %options = (
        loop_safe => 0,
        sorted => 0,
        depthfirst => -1,
        error_handler => undef
    );

Alternatively, just use the iter_fast and all_fast methods instead, which set these options for you. Iconic behaviour:

    $iter = $rule->iter( @dirs, \%options );

    $iter = $rule->iter_fast( @dirs ); # same thing

Depending on the file structure you're searching, depthfirst => -1 may or may not be a good choice. If you have lots of nested directories and all the files are at the bottom, a depth-first search might do less work or use less memory — particularly if the search is going to be halted early (like, finding the first N matches). ✨

Rules will shortcut on failure, so be sure to put rules likely to fail early in a rule chain.

Consider:

    $r1 = Path::Iterator::Rule->new->name(qr/foo/)->file;
    $r2 = Path::Iterator::Rule->new->file->name(qr/foo/);

If there are lots of files, but only a few containing "foo", then $r1 above will be faster. Put the picky rule first, always. 💅

Rules are implemented as code references, so long chains have some overhead. Consider testing with a custom coderef that combines several tests into one.

Consider:

    $r3 = Path::Iterator::Rule->new->and( sub { -x -w -r $_ } );
    $r4 = Path::Iterator::Rule->new->executable->writeable->readable;

Rule $r3 above will be much faster, not only because it stacks the file tests, but because it requires only a single code reference. One coderef > four coderefs. Math is mathing. 📐

💀 CAVEATS 💀

Some features are still unimplemented:

  • Untainting options
  • Some File::Find::Rule helpers (e.g. "grep")
  • Extension class loading via import()

Filetest operators and stat rules are subject to the usual portability considerations. See perlport for details. 🦋

🌈 SEE ALSO 🌈

There are many other file finding modules out there, bestie. They all have various features/deficiencies, depending on your preferences and needs. Here is an (incomplete) list of alternatives, with some comparison commentary. 👑

Path::Class::Rule and IO::All::Rule are subclasses of Path::Iterator::Rule and operate on Path::Class and IO::All objects, respectively. Because of this, they are substantially slower on large directory trees than just using this module directly.

File::Find is part of the Perl core. It requires the user to write a callback function to process each node of the search. Callbacks must use global variables to determine the current node. It only supports depth-first search (both pre- and post-order). It supports pre- and post-processing callbacks; the former is required for sorting files to process in a directory. File::Find::Closures can be used to help create a callback for File::Find.

File::Find::Rule is an object-oriented wrapper around File::Find. It provides a number of helper functions and there are many more "File::Find::Rule::*" modules on CPAN with additional helpers. It provides an iterator interface, but precomputes all the results.

File::Next provides iterators for file, directories or "everything". It takes two callbacks, one to match files and one to decide which directories to descend. It does not allow control over breadth/depth order, though it does provide means to sort files for processing within a directory. Like File::Find, it requires callbacks to use global variables.

Path::Class::Iterator walks a directory structure with an iterator. It is implemented as Path::Class subclasses, which adds a degree of extra complexity. It takes a single callback to define "interesting" paths to return. The callback gets a Path::Class::Iterator::File or Path::Class::Iterator::Dir object for evaluation.

File::Find::Object and companion File::Find::Object::Rule are like File::Find and File::Find::Rule, but without File::Find inside. They use an iterator that does not precompute results. They can return File::Find::Object::Result objects, which give a subset of the utility of Path::Class objects. File::Find::Object::Rule appears to be a literal translation of File::Find::Rule, including oddities like making "-M" into a boolean. 💀

File::chdir::WalkDir recursively descends a tree, calling a callback on each file. No iterator. Supports exclusion patterns. Depth-first post-order by default, but offers pre-order option. Does not process symlinks.

File::Find::Iterator is based on iterator patterns in Higher Order Perl. It allows a filtering callback. Symlinks are followed automatically without infinite loop protection. No control over order. It offers a "state file" option for resuming interrupted work.

File::Find::Declare has declarative helper rules, no iterator, is Moose-based and offers no control over ordering or following symlinks.

File::Find::Node has no iterator, does matching via callback and offers no control over ordering.

File::Set builds up a set of files to operate on from a list of directories to include or exclude, with control over recursion. A callback is applied to each file (or directory) in the set. There is no iterator. There is no control over ordering. Symlinks are not followed. It has several extra features for checksumming the set and creating tarballs with /bin/tar.

🫶 THANKS 🫶

Thank you to Ricardo Signes (rjbs) for inspiring me to write yet another file finder module, for writing file finder optimization benchmarks, and tirelessly running my code over and over to see if it got faster. Absolute legend. 🌟

⭐ AUTHOR ⭐

David Golden <dagolden@cpan.org>

okay bestie, the legal bit, but make it cute ✨

This software is Copyright (c) 2013 by David Golden. 🫶

It's free software, licensed under:

The Apache License, Version 2.0, January 2004

like, genuinely free — go forth and use it, just give the man his credit, periodt. 💅

and that's the whole page, bestie!! 🎀 you made it to the end, iconic behavior fr fr. 🦋

this lil masterpiece was served up as perl v5.36.0, dated 2022-11-19, and it lives at PIR(3pm) in the manual pages neighborhood. 🏡✨

mwah 💋 now go read some data with PIR like the main character you are 👑

📦 where this page comes from