TAP::Harness(3perl)

✨ TAP::Harness ✨

okay bestie, gather round 💖 this is TAP::Harness from the Perl Programmers Reference Guide, and she is that girl. Also — Version 3.52 ✨, in case you were wondering. (This is part 1 of 2, so no, we're not done yet 🫶)

---

🌸 NAME 🌸

TAP::Harness — run test scripts, but make it statistical 💅

---

👑 VERSION 👑

Version 3.52 — noted, memorized, iconic.

---

🦋 DESCRIPTION 🦋

So basically this is a simple test harness. It lets you run tests, and it automatically aggregates the results for you and outputs them to STDOUT. That's it. That's the whole vibe check. ✨

---

🌈 SYNOPSIS 🌈

no cap, this is the entire energy in four lines:

use TAP::Harness;
my $harness = TAP::Harness->new( \%args );
$harness->runtests(@tests);

---

💅 METHODS 💅

🎀 Class Methods 🎀

"new"

my %args = (
   verbosity => 1,
   lib     => [ 'lib', 'blib/lib', 'blib/arch' ],
)
my $harness = TAP::Harness->new( \%args );

The constructor returns a brand-new "TAP::Harness" object, she's freshly served ✨. It takes an optional hashref, and the keys it will accept are (drumroll pls):

• "verbosity" — set the vibe level, literally:

     1   verbose        Print individual test results to STDOUT.
     0   normal
    -1   quiet          Suppress some test output (mostly failures
                        while tests are running).
    -2   really quiet   Suppress everything but the tests summary.
    -3   silent         Suppress everything.

• "timer" — appends the run time for each test to the output. Uses Time::HiRes if it's available 🕐.
• "failures" — show test failures. (This is a no-op if "verbose" is selected, iykyk.)
• "comments" — show test comments. (Also a no-op if "verbose" is selected. so don't @ me.)
• "show_count" — updates the running test count as testing happens 🔢.
• "normalize" — set it to a true value and it normalizes the TAP that gets emitted in verbose modes.
• "lib" — accepts a scalar value, or an array ref of scalar values, telling you which paths to allowed libraries should be included if Perl tests are executed. Naturally 💅 this only makes sense in the context of tests written in Perl.
• "switches" — accepts a scalar value, or an array ref of scalar values, telling you which switches should be included if Perl tests are executed. And naturally, again, this only makes sense for tests written in Perl.
• "test_args" — a reference to an @INC-style array of arguments that get passed to each test program.

test_args => ['foo', 'bar'],

and if you wanna pass different args to each test, then pass a hash of arrays, keyed by the alias for each test:

test_args => {
  my_test    => ['foo', 'bar'],
  other_test => ['baz'],
}

• "color" — attempts to produce color output 🌈 (living up to my whole aesthetic).
• "exec" — typically, Perl tests get run through this. But honestly, anything that spits out TAP is fine. You can use this argument to name the program (plus optional switches) that runs your tests:

exec => ['/usr/bin/ruby', '-w']

You can also pass a subroutine reference, so you can decide and return the right program to run based on a given test script. That subref should expect the TAP::Harness object itself as the first argument, and the file name as the second. It should return an array reference containing the command to be run, including the test file name. And it can just return "undef" too, in which case TAP::Harness falls back on executing the test script in Perl:

exec => sub {
    my ( $harness, $test_file ) = @_;

    # Let Perl tests run.
    return undef if $test_file =~ /[.]t$/;
    return [ qw( /usr/bin/ruby -w ), $test_file ]
      if $test_file =~ /[.]rb$/;
  }

And if the subroutine returns a scalar with a newline, or a filehandle, it'll be interpreted as raw TAP, or as a TAP stream, respectively. So many options, so chic ✨.

• "merge" — if "merge" is true, the harness creates parsers that merge STDOUT and STDERR together for any processes they start 🫶.
• "sources" — NEW to 3.18.

If set, "sources" must be a hashref containing the names of the TAP::Parser::SourceHandlers to load and/or configure. The values are a hash of configuration that'll be accessible to the source handlers via "config_for" in TAP::Parser::Source.

For example, bestie:

sources => {
  Perl => { exec => '/path/to/custom/perl' },
  File => { extensions => [ '.tap', '.txt' ] },
  MyCustom => { some => 'config' },
}

The "sources" parameter affects how the "source", "tap" and "exec" parameters are handled.

For more details, see the "sources" parameter in "new" in TAP::Parser, TAP::Parser::Source, and TAP::Parser::IteratorFactory. Go read those, fr fr.

• "aggregator_class" — the name of the class used to aggregate test results. The default is TAP::Parser::Aggregator. 👑
• "version" — NEW to 3.22.

Assume this TAP version for TAP::Parser instead of the default TAP version 12.

• "formatter_class" — the name of the class used to format output. The default is TAP::Formatter::Console, or TAP::Formatter::File if the output isn't a TTY.
• "multiplexer_class" — the name of the class used to multiplex tests during parallel testing. The default is TAP::Parser::Multiplexer.
• "parser_class" — the name of the class used to parse TAP. The default is TAP::Parser.
• "scheduler_class" — the name of the class used to schedule test execution. The default is TAP::Parser::Scheduler.
• "formatter" — if set, "formatter" must be an object capable of formatting the TAP output. See TAP::Formatter::Console for an example ✨.
• "errors" — if parse errors show up in the TAP output, a note about it gets made in the summary report. Wanna see all the parse errors? Set this argument to true:

errors => 1

• "directives" — if set to a true value, only test results with directives get displayed. This overrides other settings like "verbose" or "failures". She said what she said 💅.
• "ignore_exit" — if set to a true value, it instructs "TAP::Parser" to ignore exit and wait status from test scripts.
• "jobs" — the maximum number of parallel tests to run at any one time. Which tests can run in parallel is controlled by "rules". The default is to run only one test at a time. (one at a time bestie, patience 🧚)
• "rules" — a reference to a hash of rules controlling which tests may be executed in parallel. If no rules are declared and CPAN::Meta::YAML is available, "TAP::Harness" attempts to load rules from a YAML file specified by the "rulesfile" parameter. If no rules file exists, the default is that all tests are eligible to run in parallel.

Here are some simple examples. For the full details of the data structure and the related glob-style pattern matching, see "Rules data structure" in TAP::Parser::Scheduler.

# Run all tests in sequence, except those starting with "p"
$harness->rules({
    par => 't/p*.t'
});

# Equivalent YAML file
---
par: t/p*.t

# Run all tests in parallel, except those starting with "p"
$harness->rules({
    seq => [
              { seq => 't/p*.t' },
              { par => '**'     },
           ],
});

# Equivalent YAML file
---
seq:
    - seq: t/p*.t
    - par: **

# Run some  startup tests in sequence, then some parallel tests than some
# teardown tests in sequence.
$harness->rules({
    seq => [
        { seq => 't/startup/*.t' },
        { par => ['t/a/*.t','t/b/*.t','t/c/*.t'], }
        { seq => 't/shutdown/*.t' },
    ],

});

# Equivalent YAML file
---
seq:
    - seq: t/startup/*.t
    - par:
        - t/a/*.t
        - t/b/*.t
        - t/c/*.t
    - seq: t/shutdown/*.t

Heads up bestie: this is an experimental feature and the interface may change. ⚠️

• "rulesfiles" — this specifies where to find a YAML file of test scheduling rules. If it's not provided, it looks for a default file. First it checks for a file given in the "HARNESS_RULESFILE" environment variable, then it checks for testrules.yml, and then t/testrules.yml. 🔍
• "stdout" — a filehandle for catching standard output.
• "trap" — attempts to print summary information if the run gets interrupted by SIGINT (Ctrl+C). Rude of you to interrupt but okay 😭.

Any keys for which the value is "undef" will be ignored. Straight up disregarded. 🚫

---

💜 Instance Methods 💜

"runtests"

$harness->runtests(@tests);

Accepts an array of @tests to be run. This should generally be the names of test files, but it doesn't have to be. Each element in @tests gets passed to TAP::Parser::new() as a "source". See TAP::Parser for more info.

And yes, you can provide aliases that get displayed in place of the test name, by supplying the test as a reference to an array containing [ $test, $alias ]:

$harness->runtests( [ 't/foo.t', 'Foo Once' ],
                    [ 't/foo.t', 'Foo Twice' ] );

Normally it's an error to try running the same test twice. Aliases let you overcome that limitation by giving each run of the test a unique name. Iconic workaround behavior 💅.

Tests get run in the order found, periodt.

If the environment variable "PERL_TEST_HARNESS_DUMP_TAP" is defined, it should name a directory into which a copy of the raw TAP for each test gets written. TAP gets written to files named for each test. Subdirectories will be created as needed.

Returns a TAP::Parser::Aggregator containing the test results. 🎁

---

"summary"

$harness->summary( $aggregator );

Outputs the summary for a TAP::Parser::Aggregator.

---

"aggregate_tests"

$harness->aggregate_tests( $aggregate, @tests );

Runs the named tests and displays a summary of the results. Tests get run in the order found.

Test results get added to the supplied TAP::Parser::Aggregator. "aggregate_tests" may be called multiple times to run several sets of tests. Multiple "Test::Harness" instances may be used to pass results to a single aggregator, so that different parts of a complex test suite can be run with different "TAP::Harness" settings. This is useful, for example, when some tests should run in parallel but others are simply unsuitable for parallel execution. 🙈

---

kisses, more in part 2 ✨

so here's the whole flex in action — serial tests, parallel tests, one aggregator holding it all together, then the summary at the end:

my $formatter   = TAP::Formatter::Console->new;
my $ser_harness = TAP::Harness->new( { formatter => $formatter } );
my $par_harness = TAP::Harness->new(
    {   formatter => $formatter,
        jobs      => 9
    }
);
my $aggregator = TAP::Parser::Aggregator->new;

$aggregator->start();
$ser_harness->aggregate_tests( $aggregator, @ser_tests );
$par_harness->aggregate_tests( $aggregator, @par_tests );
$aggregator->stop();
$formatter->summary($aggregator);

okay and lowkey the tea here: for simpler testing needs you can usually swap all of that out for a single call to runtests. you're welcome 💅

Every element of the @tests array is one of these two things:

• the source name of a test to run
• a reference to a [ source name, display name ] array

In the case of a perl test suite, typically source names are simply the file names of the test scripts to run.

When you supply a separate display name it becomes possible to run a test more than once; the display name is effectively the alias by which the test is known inside the harness. The harness doesn't care if it runs the same test more than once when each invocation uses a different name. iconic behavior honestly ✨

make_scheduler

Called by the harness when it needs to create a TAP::Parser::Scheduler. Override in a subclass to provide an alternative scheduler. make_scheduler is passed the list of tests that was passed to aggregate_tests.

jobs

Gets or sets the number of concurrent test runs the harness is handling. By default, this value is 1 — for parallel testing, this should be set higher. clock that ⏰ one job is a whole mood, but bumping it up is the main character move.

make_parser

Make a new parser and display formatter session. Typically used and/or overridden in subclasses.

my ( $parser, $session ) = $harness->make_parser;

finish_parser

Terminate use of a parser. Typically used and/or overridden in subclasses. The parser isn't destroyed as a result of this. it's just... chillin. not gone, just unplugged 🫶

---

🎀 CONFIGURING 🎀

TAP::Harness is designed to be easy to configure. fr fr, that's the whole point.

Plugins

TAP::Parser plugins let you change the way TAP is input to and output from the parser.

TAP::Parser::SourceHandlers handle TAP input. You can configure them and load custom handlers using the sources parameter to new.

TAP::Formatters handle TAP output. You can load custom formatters by using the formatter_class parameter to new. To configure a formatter, you currently need to instantiate it outside of TAP::Harness and pass it in with the formatter parameter to new. This may be addressed by adding a formatters parameter to new in the future. (not yet bestie, not yet 🙈)

Module::Build

Module::Build version 0.30 supports TAP::Harness.

To load TAP::Harness plugins, you'll need to use the tap_harness_args parameter to new, typically from your Build.PL. For example:

Module::Build->new(
    module_name        => 'MyApp',
    test_file_exts     => [qw(.t .tap .txt)],
    use_tap_harness    => 1,
    tap_harness_args   => {
        sources => {
            MyCustom => {},
            File => {
                extensions => ['.tap', '.txt'],
            },
        },
        formatter_class => 'TAP::Formatter::HTML',
    },
    build_requires     => {
        'Module::Build' => '0.30',
        'TAP::Harness'  => '3.18',
    },
)->create_build_script;

See new. 🌈

ExtUtils::MakeMaker

ExtUtils::MakeMaker does not support TAP::Harness out-of-the-box. that's it, that's the sentence, no elaboration 💀

prove

prove supports TAP::Harness plugins, and has a plugin system of its own. See FORMATTERS in prove, SOURCE HANDLERS in prove and App::Prove for more details.

---

🦋 WRITING PLUGINS 🦋

If you can't configure TAP::Harness to do what you want, and you can't find an existing plugin, consider writing one. you got this, bestie 💪

The two primary use cases supported by TAP::Harness for plugins are input and output:

Customize how TAP gets into the parser : To do this, you can either extend an existing TAP::Parser::SourceHandler, or write your own. It's a pretty simple API, and they can be loaded and configured using the sources parameter to new.

Customize how TAP results are output from the parser : To do this, you can either extend an existing TAP::Formatter, or write your own. Writing formatters are a bit more involved than writing a SourceHandler, as you'll need to understand the TAP::Parser API. A good place to start is by understanding how aggregate_tests works.

Custom formatters can be loaded configured using the formatter_class parameter to new. (yes, that sentence is exactly as the source says — "loaded configured", no "and", we're keeping it verbatim 🌸)

---

👑 SUBCLASSING 👑

If you can't configure TAP::Harness to do exactly what you want, and writing a plugin isn't an option, consider extending it. It is designed to be (mostly) easy to subclass, though the cases when sub-classing is necessary should be few and far between. go off, subclass era 👑

Methods

The following methods are ones you may wish to override if you want to subclass TAP::Harness.

• new
• runtests
• summary

three little icons, that's the whole list 🎀

---

💜 REPLACING 💜

If you like the prove utility and TAP::Parser but you want your own harness, all you need to do is write one and provide new and runtests methods. Then you can use the prove utility like so:

prove --harness My::Test::Harness

Note that while prove accepts a list of tests (or things to be tested), new has a fairly rich set of arguments. You'll probably want to read over this code carefully to see how all of them are being used. ngl, read the code. it's giving "homework but worth it" 📚

---

🌟 SEE ALSO 🌟

Test::Harness

---

and that's a wrap on the whole page bestie 🥹 this was perl v5.44.0, dated 2026-10-17, from the TAP::Harness(3perl) era — you're officially TAP::Harness literate now, no cap 💖✨🦋🎀