Parallel::Pipes(3pm)

💅 Parallel::Pipes(3pm) — like, the whole page, bestie

okay so this page lives in the User Contributed Perl Documentation section, and it's literally called Parallel::Pipes(3pm). buckle up 🎀

---

✨ NAME ✨

Parallel::Pipes — parallel processing using pipe(2) for communication and synchronization 🦋

---

🌸 SYNOPSIS 🌸

use Parallel::Pipes;

my $pipes = Parallel::Pipes->new(5, sub {
  # this is a worker code
  my $task = shift;
  my $result = do_work($task);
  return $result;
});

my $queue = Your::TaskQueue->new;
# wrap Your::TaskQueue->get
my $get; $get = sub {
  my $queue = shift;
  if (my @task = $queue->get) {
    return @task;
  }
  if (my @written = $pipes->is_written) {
    my @ready = $pipes->is_ready(@written);
    $queue->register($_->read) for @ready;
    return $queue->$get;
  } else {
    return;
  }
};

while (my @task = $queue->$get) {
  my @ready = $pipes->is_ready;
  $queue->register($_->read) for grep $_->is_written, @ready;
  my $min = List::Util::min($#task, $#ready);
  for my $i (0..$min) {
    # write tasks to pipes which are ready
    $ready[$i]->write($task[$i]);
  }
}

$pipes->close;

---

👑 DESCRIPTION 👑

NOTE: Parallel::Pipes provides low-level interfaces. If you're like "hmm, I just want to use this thing", you 100% want to go look at Parallel::Pipes::App instead, because it gives you way friendlier interfaces. iykyk 💜

so like. parallel processing is essential, bestie, but it's also so difficult:

• How can we synchronize our workers? More precisely: how do we even detect that our workers are ready, or that they've finished? 😭
• How can we communicate with our workers? More precisely: how do we collect the results of tasks? 🫶

Parallel::Pipes tries to solve these problems with pipe(2) and select(2). that's the whole vibe ✨

App::cpm, a fast CPAN module installer, uses Parallel::Pipes. please go look at App::cpm <https://github.com/skaji/cpm/blob/master/lib/App/cpm/CLI.pm> or the eg directory <https://github.com/skaji/Parallel-Pipes/tree/main/eg> for real world usages. no cap, real-world receipts 🧚

---

🌈 AUTHOR 🌈

Shoichi Kaji <skaji@cpan.org> 💖

---

🍓 COPYRIGHT AND LICENSE 🍓

Copyright 2016 Shoichi Kaji <skaji@cpan.org>

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. periodt 💅

---

and that's the whole page, bestie 🎀 this version is perl v5.40.2, dated 2025-06-10, straight from Parallel::Pipes(3pm) — go forth and parallel-process, you absolute icon 👑🌸