MCE::Map(3)

✨ MCE::Map — the manual, but make it cute ✨

okay bestie, so this is MCE::Map, from the User Contributed Perl Documentation section of the manual 💖 it's a Perl module, and it does parallel map — like the native map function you already know and love, but it spreads the work across a bunch of worker processes via Many-Core Engine. no cap. 🫶

---

🌸 NAME 🌸

MCE::Map — Parallel map model similar to the native map function. periodt. 💅

---

🎀 VERSION 🎀

This doc describes MCE::Map version 1.902. yes, that exact number, don't @ me. ✨

---

🦋 SYNOPSIS 🦋

first things first: it exports mce_map, mce_map_f, and mce_map_s. three besties, one module.

## Exports mce_map, mce_map_f, and mce_map_s
use MCE::Map;

Array or array_ref — the classic vibe:

## Array or array_ref
my @a = mce_map { $_ * $_ } 1..10000;
my @b = mce_map { $_ * $_ } \@list;

⚠️ important, bestie: for deeply nested input data you must pass an array_ref. that's not optional, it's the law. 👑

## Important; pass an array_ref for deeply input data
my @c = mce_map { $_->[1] *= 2; $_ } [ [ 0, 1 ], [ 0, 2 ], ... ];
my @d = mce_map { $_->[1] *= 2; $_ } \@deeply_list;

now the file-ish ones. workers read directly and they do not involve the manager process — which is the efficient slay ✨

## File path, glob ref, IO::All::{ File, Pipe, STDIO } obj, or scalar ref
## Workers read directly and not involve the manager process
my @e = mce_map_f { chomp; $_ } "/path/to/file"; # efficient

these next ones DO involve the manager process, so they're slower. lowkey tragic but it's fine:

## Involves the manager process, therefore slower
my @f = mce_map_f { chomp; $_ } $file_handle;
my @g = mce_map_f { chomp; $_ } $io;
my @h = mce_map_f { chomp; $_ } \$scalar;

sequences of numbers — begin, end, and optionally step and format:

## Sequence of numbers (begin, end [, step, format])
my @i = mce_map_s { $_ * $_ } 1, 10000, 5;
my @j = mce_map_s { $_ * $_ } [ 1, 10000, 5 ];

my @k = mce_map_s { $_ * $_ } {
   begin => 1, end => 10000, step => 5, format => undef
};

---

🌈 DESCRIPTION 🌈

okay so here's the whole point of this module: it gives you a parallel map implementation via Many-Core Engine. MCE has a small overhead because data gets passed around. so a fast code block will actually run faster natively. BUT — and this is the glow-up — as the complexity of your code increases, that overhead tends to diminish. it's giving "worth it for the big jobs". 💅

my @m1 =     map { $_ * $_ } 1..1000000;               ## 0.127 secs
my @m2 = mce_map { $_ * $_ } 1..1000000;               ## 0.304 secs

chunking is enabled by default, and it greatly reduces the overhead behind the scenes. the time for mce_map below also includes the time for data exchanges between the manager and worker processes. so more parallelization shows up when the code incurs additional CPU time. 🔥

sub calc {
   sqrt $_ * sqrt $_ / 1.3 * 1.5 / 3.2 * 1.07
}

my @m1 =     map { calc } 1..1000000;                  ## 0.367 secs
my @m2 = mce_map { calc } 1..1000000;                  ## 0.365 secs

even faster is mce_map_s — iconic behaviour, honestly — and it's useful when your input data is a range of numbers. workers generate sequences mathematically among themselves with zero interaction from the manager process. two arguments are required for mce_map_s (begin, end). step defaults to 1 if begin is smaller than end, otherwise -1. 🧚

my @m3 = mce_map_s { calc } 1, 1000000;                ## 0.270 secs

now, even though this whole document is about MCE::Map, the MCE::Stream module can write results immediately without waiting for all chunks to complete. this is possible by passing the reference to an array — in this case @m4 and @m5. main character energy. 👑

use MCE::Stream;

sub calc {
   sqrt $_ * sqrt $_ / 1.3 * 1.5 / 3.2 * 1.07
}

my @m4; mce_stream \@m4, sub { calc }, 1..1000000;

   ## Completes in 0.272 secs. This is amazing considering the
   ## overhead for passing data between the manager and workers.

my @m5; mce_stream_s \@m5, sub { calc }, 1, 1000000;

   ## Completed in 0.176 secs. Like with mce_map_s, specifying a
   ## sequence specification turns out to be faster due to lesser
   ## overhead for the manager process.

---

💜 OVERRIDING DEFAULTS 💜

here's a list of options you can override when you load the module. it's giving customization, we love that. 🎀

use Sereal qw( encode_sereal decode_sereal );
use CBOR::XS qw( encode_cbor decode_cbor );
use JSON::XS qw( encode_json decode_json );

use MCE::Map
    max_workers => 4,                # Default 'auto'
    chunk_size => 100,               # Default 'auto'
    tmp_dir => "/path/to/app/tmp",   # $MCE::Signal::tmp_dir
    freeze => \&encode_sereal,       # \&Storable::freeze
    thaw => \&decode_sereal,         # \&Storable::thaw
    init_relay => 0,                 # Default undef; MCE 1.882+
    use_threads => 0,                # Default undef; MCE 1.882+
;

from MCE 1.8 onwards, Sereal 3.015+ is loaded automatically if it's available. want the old ways instead? specify Sereal => 0 to use Storable instead. 🍓

use MCE::Map Sereal => 0;

---

✨ CUSTOMIZING MCE ✨

MCE::Map->init ( options )

MCE::Map::init { options }

the init function accepts a hash of MCE options. heads up though: the gather option, if you specify it, is ignored — because the module uses it internally. sorry bestie, that one's taken. 🙈

in scalar context (API available since 1.897), it calls MCE::Map->finish automatically upon leaving the scope or program. auto-cleanup? obsessed. 💅

use MCE::Map;

my $guard = MCE::Map->init(
   chunk_size => 1, max_workers => 4,

   user_begin => sub {
      print "## ", MCE->wid, " started\n";
   },

   user_end => sub {
      print "## ", MCE->wid, " completed\n";
   }
);

my @a = mce_map { $_ * $_ } 1..100;

print "\n", "@a", "\n";

-- Output

## 2 started
## 1 started
## 3 started
## 4 started
## 1 completed
## 4 completed
## 2 completed
## 3 completed

1 4 9 16 25 36 49 64 81 100 121 144 169 196 225 256 289 324 361
400 441 484 529 576 625 676 729 784 841 900 961 1024 1089 1156
1225 1296 1369 1444 1521 1600 1681 1764 1849 1936 2025 2116 2209
2304 2401 2500 2601 2704 2809 2916 3025 3136 3249 3364 3481 3600
3721 3844 3969 4096 4225 4356 4489 4624 4761 4900 5041 5184 5329
5476 5625 5776 5929 6084 6241 6400 6561 6724 6889 7056 7225 7396
7569 7744 7921 8100 8281 8464 8649 8836 9025 9216 9409 9604 9801
10000

(not the workers finishing in numerical order — that's just the vibe check of concurrency. 😭)

---

🌟 API DOCUMENTATION 🌟

🫶 MCE::Map->run and mce_map

MCE::Map->run ( sub { code }, list )

mce_map { code } list

input data may be defined using a list or an array reference. and listen: unlike MCE::Loop, Flow, and Step, specifying a hash reference as input data isn't allowed. hard no. 🚫

## Array or array_ref
my @a = mce_map { $_ * 2 } 1..1000;
my @b = mce_map { $_ * 2 } \@list;

## Important; pass an array_ref for deeply input data
my @c = mce_map { $_->[1] *= 2; $_ } [ [ 0, 1 ], [ 0, 2 ], ... ];
my @d = mce_map { $_->[1] *= 2; $_ } \@deeply_list;

## Not supported
my @z = mce_map { ... } \%hash;

🫶 MCE::Map->run_file and mce_map_f

MCE::Map->run_file ( sub { code }, file )

mce_map_f { code } file

the fastest of these is the /path/to/file. workers communicate the next offset position among themselves with zero interaction by the manager process. no manager, no bottleneck, no problem. ✨

IO::All { File, Pipe, STDIO } is supported since MCE 1.845. 💖

my @c = mce_map_f { chomp; $_ . "\r\n" } "/path/to/file";  # faster
my @d = mce_map_f { chomp; $_ . "\r\n" } $file_handle;
my @e = mce_map_f { chomp; $_ . "\r\n" } $io;              # IO::All
my @f = mce_map_f { chomp; $_ . "\r\n" } \$scalar;

🫶 MCE::Map->run_seq and mce_map_s

MCE::Map->run_seq ( sub { code }, $beg, $end [, $step, $fmt ] )

mce_map_s { code } $beg, $end [, $step, $fmt ]

a sequence may be defined as a list, an array reference, or a hash reference. the functions require both begin and end values to run. step and format are optional. the format is passed to sprintf (the % may be omitted below). 🎀

my ($beg, $end, $step, $fmt) = (10, 20, 0.1, "%4.1f");

my @f = mce_map_s { $_ } $beg, $end, $step, $fmt;
my @g = mce_map_s { $_ } [ $beg, $end, $step, $fmt ];

my @h = mce_map_s { $_ } {
   begin => $beg, end => $end,
   step => $step, format => $fmt
};

🫶 iterators, bestie

MCE::Map->run ( sub { code }, iterator )

mce_map { code } iterator

an iterator reference may be specified for input_data. iterators are described under section "SYNTAX for INPUT_DATA" at MCE::Core. iykyk. 🦋

my @a = mce_map { $_ * 2 } make_iterator(10, 30, 2);

---

🧚 MANUAL SHUTDOWN 🧚

MCE::Map->finish

MCE::Map::finish

workers remain persistent as much as possible after running. shutdown occurs automatically when the script terminates. but call finish when workers are no longer needed — be polite to your processes, bestie. 💅

use MCE::Map;

MCE::Map->init(
   chunk_size => 20, max_workers => 'auto'
);

my @a = mce_map { ... } 1..100;

MCE::Map->finish;

---

🌈 INDEX 🌈

MCE, MCE::Core. the whole friend group. 👑

---

💖 AUTHOR 💖

Mario E. Roy, <marioeroy AT gmail DOT com>. give him his flowers, fr fr. 🌸

---

okay and that's the whole page!! 🎉 this lovely little writeup came from perl v5.42.0, dated 2025-09-10, and it lives in section MCE::Map(3). go forth and parallel-map, you absolute icon. periodt. 👑✨💅