PERLFILTER(1p)

📦

version 1.65 · 6 versions

✨ perlfilter — Source Filters ✨

okay bestie, welcome to PERLFILTER(1p) from User Contributed Perl Documentation — this is the whole tea on source filters, and we're doing it in three parts, so hydrate 💖

💅 DESCRIPTION 💅

So this whole article is about a little-known feature of Perl called source filters. what do they do? they alter the program text of a module before Perl even sees it — literally the same energy as a C preprocessor altering the source text of a C program before the compiler gets its hands on it. this article tells you what source filters are, how they work, and how to write your own. periodt.

the original purpose of source filters? encrypting your program source to prevent casual piracy. iconic. but that is not all they can do, as you'll soon learn. but first — the basics 🫶

🧠 CONCEPTS 🧠

before the Perl interpreter can execute a Perl script, it has to read it from a file into memory for parsing and compilation. and if that script itself includes other scripts with a use or require statement, then each of those scripts has to be read from its own respective file too. okay so far so good.

now think of each logical connection between the Perl parser and an individual file as a source stream. a source stream is created when the Perl parser opens a file, it keeps existing while the source code is read into memory, and it gets destroyed when Perl is done parsing the file. and if the parser encounters a require or use statement in a source stream, a new and distinct stream gets created just for that file. slay.

the diagram below is a single source stream, with the flow of source from a Perl script file on the left into the Perl parser on the right. this is how Perl normally operates:

    file -------> parser

there are two important points to remember:

Although there can be any number of source streams in existence at any given time, only one will be active.
Every source stream is associated with only one file.

a source filter is a special kind of Perl module that intercepts and modifies a source stream before it reaches the parser. a source filter changes our diagram like this:

    file ----> filter ----> parser

if that's giving nothing but confusion, think about a command pipeline analogy. say you have a shell script stored in the compressed file trial.gz. the simple pipeline command below runs the script without needing to create a temporary file to hold the uncompressed file:

    gunzip -c trial.gz | sh

in this case, the data flow from the pipeline can be represented like this:

    trial.gz ----> gunzip ----> sh

with source filters, you can store the text of your script compressed and use a source filter to uncompress it for Perl's parser:

     compressed           gunzip
    Perl program ---> source filter ---> parser

🎀 USING FILTERS 🎀

so how do you actually use a source filter in a Perl script? like I said up there, a source filter is just a special kind of module. and like all Perl modules, it gets invoked with a use statement. 💅

say you want to pass your Perl source through the C preprocessor before execution. fun fact: the source filters distribution comes with a C preprocessor filter module called Filter::cpp.

below is an example program, cpp_test, which makes use of this filter. line numbers have been added so we can reference specific lines easily:

    1: use Filter::cpp;
    2: #define TRUE 1
    3: $a = TRUE;
    4: print "a = $a\n";

when you run this script, Perl creates a source stream for the file. and before the parser processes any of the lines from the file, the source stream looks like this:

    cpp_test ---------> parser

line 1, use Filter::cpp, includes and installs the cpp filter module. all source filters work this way. the use statement is compiled and executed at compile time, before any more of the file is read, and it attaches the cpp filter to the source stream behind the scenes. now the data flow looks like this:

    cpp_test ----> cpp filter ----> parser

as the parser reads the second and subsequent lines from the source stream, it feeds those lines through the cpp source filter before processing them. the cpp filter just passes each line through the real C preprocessor. then the output from the C preprocessor gets inserted back into the source stream by the filter:

                      .-> cpp --.
                      |         |
                      |         |
                      |       <-'
       cpp_test ----> cpp filter ----> parser

and the parser then sees this code:

    use Filter::cpp;
    $a = 1;
    print "a = $a\n";

now let's consider what happens when the filtered code includes another module with use:

    1: use Filter::cpp;
    2: #define TRUE 1
    3: use Fred;
    4: $a = TRUE;
    5: print "a = $a\n";

the cpp filter does not apply to the text of the Fred module, only to the text of the file that used it (cpp_test). even though the use statement on line 3 will pass through the cpp filter, the module that gets included (Fred) will not. the source streams look like this after line 3 has been parsed and before line 4 is parsed:

    cpp_test ---> cpp filter ---> parser (INACTIVE)

    Fred.pm ----> parser

see? a new stream has been created for reading the source from Fred.pm. this stream stays active until all of Fred.pm has been parsed. the source stream for cpp_test still exists, but it's inactive. once the parser has finished reading Fred.pm, the source stream associated with it gets destroyed. and the source stream for cpp_test becomes active again, so the parser reads line 4 and subsequent lines from cpp_test. the way that just works 🦋

you can use more than one source filter on a single file. similarly, you can reuse the same filter in as many files as you like. obsessed.

for example, if you have a uuencoded and compressed source file, it's possible to stack a uudecode filter and an uncompression filter like this:

    use Filter::uudecode; use Filter::uncompress;
    M'XL(".H<US4''V9I;F%L')Q;>7/;1I;_>_I3=&E=%:F*I"T?22Q/
    M6]9*<IQCO*XFT"0[PL%%'Y+IG?WN^ZYN-$'J.[.JE$,20/?K=_[>
    ...

once the first line has been processed, the flow will look like this:

    file ---> uudecode ---> uncompress ---> parser
               filter         filter

data flows through filters in the same order they appear in the source file. the uudecode filter appeared before the uncompress filter, so the source file gets uudecoded before it's uncompressed. no cap.

🌈 WRITING A SOURCE FILTER 🌈

there are three ways to write your own source filter:

write it in C
use an external program as a filter
write the filter in Perl

the author isn't covering the first two in any great detail, so they get them out of the way first. writing the filter in Perl is most convenient, so that one gets the most space. iykyk.

💜 WRITING A SOURCE FILTER IN C 💜

the first of the three available techniques is writing the filter completely in C. the external module you create interfaces directly with the source filter hooks provided by Perl.

the advantage: you have complete control over the implementation of your filter. the big disadvantage: the increased complexity required to write the filter — not only do you need to understand the source filter hooks, you also need a reasonable knowledge of Perl guts. one of the few times it's actually worth going to this trouble is when writing a source scrambler. the decrypt filter (which unscrambles the source before Perl parses it) included with the source filter distribution is an example of a C source filter (see Decryption Filters, below).

🔐 Decryption Filters 🔐

all decryption filters work on the principle of "security through obscurity." regardless of how well you write a decryption filter and how strong your encryption algorithm is, anyone determined enough can retrieve the original source code. the reason is quite simple: once the decryption filter has decrypted the source back to its original form, fragments of it will be stored in the computer's memory as Perl parses it. the source might only be in memory for a short period of time, but anyone possessing a debugger, skill, and lots of patience can eventually reconstruct your program. 😭

that said, there are a number of steps that can be taken to make life difficult for the potential cracker. the most important one: write your decryption filter in C and statically link the decryption module into the Perl binary. for further tips to make life difficult for the potential cracker, see the file decrypt.pm in the source filters distribution. 👑

🧚 CREATING A SOURCE FILTER AS A SEPARATE EXECUTABLE 🧚

an alternative to writing the filter in C is creating a separate executable in the language of your choice. the separate executable reads from standard input, does whatever processing is necessary, and writes the filtered data to standard output. Filter::cpp is an example of a source filter implemented as a separate executable — the executable is the C preprocessor bundled with your C compiler.

the source filter distribution includes two modules that simplify this task: Filter::exec and Filter::sh. both let you run any external executable. both use a coprocess to control the flow of data into and out of the external executable. (for details on coprocesses, see Stephens, W.R., "Advanced Programming in the UNIX Environment." Addison-Wesley, ISBN 0-210-56317-7, pages 441-445.) the difference between them:

Filter::exec spawns the external command directly.
Filter::sh spawns a shell to execute the external command. (Unix uses the Bourne shell; NT uses the cmd shell.)

and spawning a shell lets you make use of the shell metacharacters and redirection facilities. slay 💅

here's an example script that uses Filter::sh:

    use Filter::sh 'tr XYZ PQR';
    $a = 1;
    print "XYZ a = $a\n";

the output you'll get when the script is executed:

    PQR a = 1

writing a source filter as a separate executable works fine, but a small performance penalty is incurred. for example, if you execute the small example above, a separate subprocess will be created to run the Unix tr command. each use of the filter requires its own subprocess. if creating subprocesses is expensive on your system, you might wanna consider one of the other options for creating source filters. 🙈

✨ WRITING A SOURCE FILTER IN PERL ✨

okay so 💖 the easiest AND most portable way to make your own source filter is literally just... to write the whole thing in Perl. no C, no external executables, no drama. and to keep it separate from the previous two techniques, we're calling it a Perl source filter. iconic behavior honestly.

to understand how to write one, we need an example to study, so here's a complete source filter that does rot13 decoding. (and for the uninitiated: rot13 is a super simple encryption scheme that Usenet people used to hide the contents of offensive posts. it moves every letter forward thirteen places, so that A becomes N, B becomes O, and Z becomes M. sneaky!)

package Rot13;

use Filter::Util::Call;

sub import {
   my ($type) = @_;
   my ($ref) = [];
   filter_add(bless $ref);
}

sub filter {
   my ($self) = @_;
   my ($status);

   tr/n-za-mN-ZA-M/a-zA-Z/
      if ($status = filter_read()) > 0;
   $status;
}

1;

every single Perl source filter is implemented as a Perl class, and they all have this same basic structure. periodt. 💅

first — we include the "Filter::Util::Call" module, which exports a bunch of functions into your filter's namespace. our example uses exactly two of them: filter_add() and filter_read(). besties fr.

next — we create the filter object and associate it with the source stream by defining the "import" function. and if you know Perl well enough (iykyk), you already know that "import" gets called automatically every single time a module is included with a use statement. which makes "import" the ideal place to both create and install a filter object. main character energy honestly. 🌟

in the example filter, the object ($ref) is blessed just like any other Perl object. our example uses an anonymous array, but that is NOT a requirement — because this example doesn't need to store any context information, we could've used a scalar or hash reference just as well. the next section shows off context data, so stay tuned 🎀

the association between the filter object and the source stream is made with the filter_add() function. it takes a filter object as a parameter ($ref in this case) and installs it in the source stream. that's it, that's the whole job.

finally — the code that actually does the filtering. for this type of Perl source filter, all the filtering happens in a method called filter(). (you can also write a Perl source filter using a closure, btw — check the Filter::Util::Call manual page for more deets.) it gets called every time the Perl parser needs another line of source to process. and the filter() method, in turn, reads lines from the source stream using the filter_read() function. 🦋

so: if a line was available from the source stream, filter_read() returns a status value greater than zero and appends the line to $_. a status value of zero means end-of-file, and less than zero means an error. the filter function itself is expected to return its status in that same way, and to put the filtered line it wants written to the source stream in $_. the use of $_ is lowkey why most Perl source filters are so short and cute. ✨

now, to actually use the rot13 filter we need some way of encoding the source file in rot13 format. the script below, "mkrot13", does exactly that:

die "usage mkrot13 filename\n" unless @ARGV;
my $in = $ARGV[0];
my $out = "$in.tmp";
open(IN, "<$in") or die "Cannot open file $in: $!\n";
open(OUT, ">$out") or die "Cannot open file $out: $!\n";

print OUT "use Rot13;\n";
while (<IN>) {
   tr/a-zA-Z/n-za-mN-ZA-M/;
   print OUT;
}

close IN;
close OUT;
unlink $in;
rename $out, $in;

so if we encrypt this with "mkrot13":

print " hello fred \n";

the result is gonna be this:

use Rot13;
cevag "uryyb serq\a";

and running it produces this output:

hello fred

slay. 🌸

🐛 USING CONTEXT: THE DEBUG FILTER 🐛

ngl, the rot13 example was a trivial example. here's another demo that shows off a few more features, because we're levelling up bestie.

say you wanted to include a LOT of debugging code in your Perl script during development, but you didn't want any of it in the released product. source filters offer a solution. and to keep the example simple, let's say you want the debugging output controlled by an environment variable, "DEBUG". debugging code is enabled if that variable exists, otherwise it's disabled. simple as that. 💅

two special marker lines will bracket the debugging code, like this:

## DEBUG_BEGIN
if ($year > 1999) {
   warn "Debug: millennium bug in year $year\n";
}
## DEBUG_END

the filter makes sure that Perl parses the code between the <DEBUG_BEGIN> and "DEBUG_END" markers only when the "DEBUG" environment variable exists. which means when "DEBUG" does exist, the code above should pass through the filter unchanged. the marker lines can pass through as-is too, because the Perl parser will just see them as comment lines. and when "DEBUG" isn't set, we need a way to disable the debug code — the easy way is to turn the lines between the two markers into comments:

## DEBUG_BEGIN
#if ($year > 1999) {
#     warn "Debug: millennium bug in year $year\n";
#}
## DEBUG_END

here's the complete Debug filter in all its glory:

package Debug;

use strict;
use warnings;
use Filter::Util::Call;

use constant TRUE => 1;
use constant FALSE => 0;

sub import {
   my ($type) = @_;
   my (%context) = (
     Enabled => defined $ENV{DEBUG},
     InTraceBlock => FALSE,
     Filename => (caller)[1],
     LineNo => 0,
     LastBegin => 0,
   );
   filter_add(bless \%context);
}

sub Die {
   my ($self) = shift;
   my ($message) = shift;
   my ($line_no) = shift || $self->{LastBegin};
   die "$message at $self->{Filename} line $line_no.\n"
}

sub filter {
   my ($self) = @_;
   my ($status);
   $status = filter_read();
   ++ $self->{LineNo};

   # deal with EOF/error first
   if ($status <= 0) {
       $self->Die("DEBUG_BEGIN has no DEBUG_END")
           if $self->{InTraceBlock};
       return $status;
   }

   if ($self->{InTraceBlock}) {
      if (/^\s*##\s*DEBUG_BEGIN/ ) {
          $self->Die("Nested DEBUG_BEGIN", $self->{LineNo})
      } elsif (/^\s*##\s*DEBUG_END/) {
          $self->{InTraceBlock} = FALSE;
      }

      # comment out the debug lines when the filter is disabled
      s/^/#/ if ! $self->{Enabled};
   } elsif ( /^\s*##\s*DEBUG_BEGIN/ ) {
      $self->{InTraceBlock} = TRUE;
      $self->{LastBegin} = $self->{LineNo};
   } elsif ( /^\s*##\s*DEBUG_END/ ) {
      $self->Die("DEBUG_END has no DEBUG_BEGIN", $self->{LineNo});
   }
   return $status;
}

1;

the BIG difference between this filter and the previous example is the use of context data in the filter object. the filter object is based on a hash reference, and it's used to keep various pieces of context info between calls to the filter function. all but two of the hash fields are for error reporting. the first of those two, Enabled, is what the filter uses to figure out whether the debugging code should be handed to the Perl parser. the second, InTraceBlock, is true when the filter has hit a "DEBUG_BEGIN" line but has NOT yet hit the following "DEBUG_END" line. so basically it's the "am I inside the vibe zone rn" flag. 🚩

if you ignore all the error checking that most of the code does, the essence of the filter is this:

sub filter {
   my ($self) = @_;
   my ($status);
   $status = filter_read();

   # deal with EOF/error first
   return $status if $status <= 0;
   if ($self->{InTraceBlock}) {
      if (/^\s*##\s*DEBUG_END/) {
         $self->{InTraceBlock} = FALSE
      }

      # comment out debug lines when the filter is disabled
      s/^/#/ if ! $self->{Enabled};
   } elsif ( /^\s*##\s*DEBUG_BEGIN/ ) {
      $self->{InTraceBlock} = TRUE;
   }
   return $status;
}

⚠️ be warned bestie: just like the C-preprocessor doesn't actually know C, the Debug filter doesn't actually know Perl. it can be fooled SO easily:

print <<EOM;
##DEBUG_BEGIN
EOM

such things aside though, you can see that a lot can be achieved with a pretty modest amount of code. 🫶

🌈 CONCLUSION 🌈

so now you have a much better understanding of what a source filter is, and you might even have a possible use for them. and if you feel like playing around with source filters but need a little inspiration, here are some extra features you could add to the Debug filter. 💜

first, an easy one. rather than having debugging code that's all-or-nothing, it'd be way more useful to control which specific blocks of debugging code get included. try extending the syntax for debug blocks so each one can be identified. then the contents of the "DEBUG" environment variable can control which blocks get included. cute, right? 🎀

once you can identify individual blocks, try letting them be nested. that isn't difficult either. 🌸

here's an interesting idea that doesn't involve the Debug filter at all. right now Perl subroutines have pretty limited support for formal parameter lists — you can specify the number of parameters and their type, but you still have to manually pull them out of the @_ array yourself. so write a source filter that lets you have a named parameter list! such a filter would turn this:

sub MySub ($first, $second, @rest) { ... }

into this:

sub MySub($$@) {
   my ($first) = shift;
   my ($second) = shift;
   my (@rest) = @_;
   ...
}

like?? imagine the glowup. ✨

and finally, if you feel like a real challenge, have a go at writing a full-blown Perl macro preprocessor as a source filter. borrow the useful features from the C preprocessor and any other macro processors you know. the tricky bit will be choosing how much knowledge of Perl's syntax you want your filter to have. no pressure bestie 💅

💀 LIMITATIONS

Okay bestie, here's where we get real. Source filters only work on the string level, which means their ability to change your source code on the fly is highly limited. They literally cannot detect:

comments
quoted strings
heredocs

And they are not a replacement for a real parser. Periodt. 🙅♀️

The only stable usage for source filters is:

encryption
compression
the byteloader, to translate binary code back to source code

See for example the limitations in Switch, which uses source filters, and thus it does not work inside a string eval. Also, the presence of regexes with embedded newlines that are specified with raw /.../ delimiters and don't have a modifier //x are indistinguishable from code chunks beginning with the division operator /. Bestie, that is chaos. 😭 As a workaround you must use m/.../ or m?...? for such patterns.

Also — and this is the sneaky one — the presence of regexes specified with raw ?...? delimiters may cause mysterious errors. 👻 The workaround is to use m?...? instead.

See https://metacpan.org/pod/Switch#LIMITATIONS.

Currently the content of the __DATA__ block is not filtered. Ngl, that one surprised me too.

Currently internal buffer lengths are limited to 32-bit only. Like, that's the whole vibe. 🫠

👀 THINGS TO LOOK OUT FOR

🚨 Some Filters Clobber the `DATA` Handle

Some source filters use the DATA handle to read the calling program. The way that means: when you're using these source filters you cannot rely on this handle, nor expect any particular kind of behavior when operating on it. Not even a little. 🙈

Filters based on Filter::Util::Call (and therefore Filter::Simple) do not alter the DATA filehandle — but on the other hand they totally ignore the text after __DATA__. Trade-offs, bestie. 💅

✨ REQUIREMENTS ✨

The Source Filters distribution is available on CPAN, in

CPAN/modules/by-module/Filter

Starting from Perl 5.8, Filter::Util::Call (the core part of the Source Filters distribution) is part of the standard Perl distribution. 🎉 So you might already have it, no cap.

Also included is a friendlier interface called Filter::Simple, by Damian Conway. Love that for us. 🫶

👑 AUTHOR

Paul Marquess <Paul.Marquess@btinternet.com>

Reini Urban <rurban@cpan.org>

📜 Copyrights

The first version of this article originally appeared in The Perl Journal #11, and is copyright 1998 The Perl Journal. It appears courtesy of Jon Orwant and The Perl Journal. This document may be distributed under the same terms as Perl itself.

And that's a wrap on the whole page, bestie!! 🎀 This one's from perl v5.42.3, dated 2026-08-19, filed under PERLFILTER(1p) — so if you're reading this in the future, hey. 👋 Go write a source filter, or honestly, don't, they said it themselves: use a real parser if you can. 🌈⭐ mwah 💜

📦 where this page comes from

ubuntu stonking: libfilter-perl 1.65-1build1 · GPL-1.0-or-later OR Artistic-1
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-11-03: a language model's rewrite of the original, under its terms