IPC::Shareable(3pm)

📦
version 1.13 · 6 versions
, in ubuntu noble, ubuntu resolute, ubuntu stonking

💖 IPC::Shareable 💖

okay so bestie, this is the manual page for IPC::Shareable(3pm) from the User Contributed Perl Documentation section, and it's about to change your whole entire life (or at least your whole entire inter-process communication situation). grab a strawberry drink 🍓, we're going in.

✨ NAME ✨

IPC::Shareable — use shared memory backed variables across processes. that's the whole vibe. variables that live in shared memory and get shared across processes. iconic.

🌈 SYNOPSIS 🌈

here's the whole menu of what you can do, verbatim, no notes:

use IPC::Shareable qw(:lock);

my $href = IPC::Shareable->new(%options);

# ...or

tie SCALAR, 'IPC::Shareable', OPTIONS;
tie ARRAY,  'IPC::Shareable', OPTIONS;
tie HASH,   'IPC::Shareable', OPTIONS;

tied(VARIABLE)->lock;
tied(VARIABLE)->unlock;

tied(VARIABLE)->lock(LOCK_SH|LOCK_NB)
    or print "Resource unavailable\n";

my $segment   = tied(VARIABLE)->seg;
my $semaphore = tied(VARIABLE)->sem;

tied(VARIABLE)->remove;

IPC::Shareable::clean_up;
IPC::Shareable::clean_up_all;
IPC::Shareable::clean_up_protected;

# Ensure only one instance of a script can be run at any time

IPC::Shareable->singleton('UNIQUE SCRIPT LOCK STRING');

# Get the actual IPC::Shareable tied object

my $knot = tied(VARIABLE); # Dereference first if using a tied reference

🦋 DESCRIPTION 🦋

so IPC::Shareable lets you tie a variable to shared memory, which makes it stupidly easy to share what's inside that variable with your other Perl processes and scripts. like, sharing is caring but make it memory. 💅

scalars, arrays, hashes, and even objects can be tied. and the variable you tie can hold arbitrarily complex data structures — we're talking references to arrays, hashes of hashes, the whole chaotic beautiful mess. no cap.

the association between variables in different processes is provided by GLUE (aka the "key"). GLUE is literally any arbitrary string or integer that acts as a common identifier for data across process space. so when you write this:

tie my $scalar, 'IPC::Shareable', { key => 'GLUE STRING', create => 1 };

...in program one, and then this:

tie my $variable, 'IPC::Shareable', { key => 'GLUE STRING' };

...in program two, program one creates $scalar and binds it to the shared memory, and program two binds that same memory to $variable. they're literally holding hands. 🫶

there is no pre-set limit to how many processes can bind to the data, and also no pre-set limit to how complex the underlying data of the tied variables can get. the amount of data you can share within a single bound variable is limited by the system's maximum size for a shared memory segment though — and the exact value of that is system-dependent, so don't come for me, come for your kernel.

the bound data structures are all linearized (using Raphael Manfredi's Storable module, or optionally JSON) before being slurped into shared memory. upon retrieval, the original format of the data structure gets recovered. and if you need to lock data between competing processes, semaphore flags are right there for you. slay.

👑 OPTIONS 👑

options get specified by passing a reference to a hash as the third argument to the tie() function that enchants a variable. yes, enchants. that's the technical term now.

these are the fields recognized in the options hash:

🔑 key

key is the GLUE, which is a direct reference to the shared memory segment that's gonna be tied to the variable.

if this option is missing, we default to using IPC_PRIVATE. heads up though: this default key will not allow sharing of the variable between processes. it's giving private, because it is private. 🙈

Default: IPC_PRIVATE

🎀 create

create controls whether the process creates a new shared memory segment or not. if create is set to a true value, IPC::Shareable will create a new binding associated with GLUE as needed. if create is false, IPC::Shareable will not attempt to create a new shared memory segment associated with GLUE — in that case, a shared memory segment associated with GLUE must already exist, or we'll croak(). dramatic? yes. necessary? also yes.

Default: false

🚫 exclusive

if exclusive is set to a true value, we will croak() if the data binding associated with GLUE already exists. if it's set to a false value, calls to tie() will succeed even if a shared memory segment associated with GLUE already exists. no drama, no exceptions.

see graceful for a silent, non-exception exit if a second process attempts to obtain an in-use exclusive segment.

Default: false

🕊️ graceful

if exclusive is set to a true value, we normally croak() if a second process tries to obtain the same shared memory segment. set graceful to true and we'll exit silently and gracefully instead. this option does nothing if exclusive isn't set, so don't expect it to carry the team on its own.

super useful for making sure only a single process is running at a time. one at a time bestie, we're not a crowd. ✋

Default: false

⚠️ warn

when set to a true value, graceful will output a warning if there are process collisions.

Default: false

🔐 mode

the mode argument is an octal number specifying the access permissions when a new data binding is being created. these access permissions are the same as file access permissions, in that 0666 is world readable, 0600 is readable only by the effective UID of the process creating the shared variable, etc. permissions are permissions, periodt.

Default: 0666 (world read and writeable)

📏 size

this field can be used to specify the size of the shared memory segment allocated.

the maximum size we allow by default is ~1GB. see the limit option to override this default.

Default: IPC::Shareable::SHM_BUFSIZ() (ie. 65536)

🛡️ protected

if set, the clean_up() and clean_up_all() routines will not remove the segments or semaphores related to the tied object. protected means protected, we respect boundaries here. 💜

set this to a specific integer so we can pass the value to any child objects created under the main one.

to clean up protected objects, call (tied %object)->clean_up_protected(integer), where integer is the value you set the protected option to. you can call this cleanup routine in the script you created the segment, or anywhere else, at any time. whenever. wherever. main character energy.

Default: 0

🔥 limit

this field lets you set a segment size larger than the default maximum, which is 1,073,741,824 bytes (approximately 1 GB). if set, we will croak() if a size specified is larger than the maximum. if it's set to a false value, we'll croak() if you send in a size larger than the total system RAM. so basically: know your limits, or get croaked at.

Default: true

💀 destroy

if set to a true value, the shared memory segment underlying the data binding will be removed when the process that initialized the shared memory segment exits (gracefully)[1].

only those memory segments that were created by the current process will be removed. we're not out here deleting other people's stuff.

use this option with care. in particular you should not use this option in a program that will fork after binding the data. on the other hand, shared memory is a finite resource and should be released if it is not needed. balance, bestie. ⚖️
NOTE: if the segment was created with its protected attribute set, it will not be removed upon program completion, even if destroy is set.

Default: false

🧹 tidy

for long running processes, set this to a true value to clean up unneeded segments from nested data structures. comes with a slight performance hit, so it's a trade-off, not a free lunch. 🍓

Default: false

🧚 serializer

by default, we use Storable as the data serializer when writing to or reading from the shared memory segments we create. for cross-platform and cross-language purposes, you can optionally use JSON for this task instead.

send in either json or storable as the value to use the respective serializer. two options, pick your fighter.

Default: storable

🌸 Default Option Values

the default values for options are, in all their glory:

key         => IPC_PRIVATE, # 0
create      => 0,
exclusive   => 0,
mode        => 0666,
size        => IPC::Shareable::SHM_BUFSIZ(), # 65536
protected   => 0,
limit       => 1,
destroy     => 0,
graceful    => 0,
warn        => 0,
tidy        => 0,
serializer  => 'storable',

✨ METHODS ✨

okay bestie, this is where it gets fun 💖 these are all the things you can actually do with your tied variable. strap in.

🌸 new

Instantiates and returns a reference to a hash backed by shared memory. iconic behaviour, honestly.

my $href = IPC::Shareable->new(key => "testing", create => 1);

$href=>{a} = 1;

# Call tied() on the dereferenced variable to access object methods
# and information

tied(%$href)->ipcs;

Parameters:

  • Hash, Optional: peek at the OPTIONS section for the full menu of what you can pass in. ngl, most of the time you'll just want key and create and you're golden.

Also, plot twist: you can get a reference to an array or a scalar too, not just a hash. just send in either var = > 'ARRAY' or var => 'SCALAR' and it's yours 💅

Return: A reference to a hash (or array or scalar) which is backed by shared memory.

🦋 singleton($glue, $warn)

Class method that ensures that only a single instance of a script can be run at any given time. one script, one vibe, no duplicates 🙅‍♀️

Parameters:

  • $glue

Mandatory, String: The key/glue that identifies the shared memory segment.

  • $warn

Optional, Bool: Send in a true value to have subsequent processes throw a warning that there's been a shared memory violation and that it will exit.

Default: false

🎀 ipcs

Returns the number of instantiated shared memory segments that currently exist on the system. it's not super precise tho — it literally just does a wc -l line count on your system's ipcs -m call. it is guaranteed to produce reliable results though. (yes bestie, both of those things are true at once, don't think about it too hard 😭)

Return: Integer

🔐 lock($flags)

Obtains a lock on the shared memory. $flags specifies the type of lock to acquire. If $flags is not specified, an exclusive read/write lock is obtained. Acceptable values for $flags are the same as for the flock() system call.

Returns true on success, and undef on error. For non-blocking calls (see below), the method returns 0 if it would have blocked.

Obtain an exclusive lock like this:

tied(%var)->lock(LOCK_EX); # same as default

Only one process can hold an exclusive lock on the shared memory at a given time. one at a time, bestie, that's the rule 👑

Obtain a shared (read) lock:

tied(%var)->lock(LOCK_SH);

Multiple processes can hold a shared (read) lock at a given time. If a process attempts to obtain an exclusive lock while one or more processes hold shared locks, it will be blocked until they have all finished.

Either of the locks may be specified as non-blocking:

tied(%var)->lock( LOCK_EX|LOCK_NB );
tied(%var)->lock( LOCK_SH|LOCK_NB );

A non-blocking lock request will return 0 if it would have had to wait to obtain the lock.

⚠️ heads up: these locks are advisory (just like flock), meaning that all cooperating processes must coordinate their accesses to shared memory using these calls in order for locking to work. See the flock() call for details.

Locks are inherited through forks, which means that two processes actually can possess an exclusive lock at the same time. Don't do that. 🫠 like, seriously, don't.

The constants LOCK_EX, LOCK_SH, LOCK_NB, and LOCK_UN are available for import using any of the following export tags:

use IPC::Shareable qw(:lock);
use IPC::Shareable qw(:flock);
use IPC::Shareable qw(:all);

Or, just use the flock constants available in the Fcntl module.

See LOCKING for further details.

🔓 unlock

Removes a lock. Takes no parameters, returns true on success.

This is equivalent of calling shlock(LOCK_UN).

See LOCKING for further details.

🧩 seg

Called on either the tied variable or the tie object, returns the shared memory segment object currently in use.

🚦 sem

Called on either the tied variable or the tie object, returns the semaphore object related to the memory segment currently in use.

📋 attributes

Retrieves the list of attributes that drive the IPC::Shareable object. receipts, basically 💅

Parameters:

  • $attribute

Optional, String: The name of the attribute. If sent in, we'll return the value of this specific attribute. Returns undef if the attribute isn't found.

Attributes are the OPTIONS that were used to create the object.

Returns: A hash reference of all attributes if $attributes isn't sent in, the value of the specific attribute if it is.

🌍 global_register

Returns a hash reference of hashes of all in-use shared memory segments across all processes. The key is the memory segment ID, and the value is the segment and semaphore objects.

📍 process_register

Returns a hash reference of hashes of all in-use shared memory segments created by the calling process. The key is the memory segment ID, and the value is the segment and semaphore objects.

🔒 LOCKING 🔒

IPC::Shareable provides methods to implement application-level advisory locking of the shared data structures. These methods are called lock() and unlock(). To use them you must first get the object underlying the tied variable, either by saving the return value of the original call to tie() or by using the built-in tied() function.

To lock and subsequently unlock a variable, do this:

my $knot = tie my %hash, 'IPC::Shareable', { %options };

$knot->lock;
$hash{a} = 'foo';
$knot->unlock;

or equivalently, if you've decided to throw away the return of tie():

tie my %hash, 'IPC::Shareable', { %options };

tied(%hash)->lock;
$hash{a} = 'foo';
tied(%hash)->unlock;

This will place an exclusive lock on the data of $scalar. You can also get shared locks or attempt to get a lock without blocking.

IPC::Shareable makes the constants LOCK_EX, LOCK_SH, LOCK_UN, and LOCK_NB exportable to your address space with the export tags :lock, :flock, or :all. The values should be the same as the standard flock option arguments.

if (tied(%hash)->lock(LOCK_SH|LOCK_NB)){
    print "The value is $hash{a}\n";
    tied(%hash)->unlock;
} else {
    print "Another process has an exclusive lock.\n";
}

If no argument is provided to lock, it defaults to LOCK_EX.

There are some pitfalls regarding locking and signals about which you should make yourself aware; these are discussed in NOTES.

Note that in the background, we perform lock optimization when reading and writing to the shared storage even if the advisory locks aren't being used.

Using the advisory locks can speed up processes that are doing several writes/ reads at the same time. so like, slay, free performance ✨

💀 DESTRUCTION 💀

perl(1) will destroy the object underlying a tied variable when then tied variable goes out of scope. Unfortunately for IPC::Shareable, this may not be desirable: other processes may still need a handle on the relevant shared memory segment. 😬 rude, honestly.

IPC::Shareable therefore provides several options to control the timing of removal of shared memory segments. you get to pick when the goodbye happens 🫶

☠️ destroy Option

As described in OPTIONS, specifying the destroy option when tie()ing a variable coerces IPC::Shareable to remove the underlying shared memory segment when the process calling tie() exits gracefully.

NOTE: The destruction is handled in an END block. Only those memory segments that are tied to the current process will be removed.
NOTE: If the segment was created with its protected attribute set, it will not be removed in the END block, even if destroy is set.

🧹 remove

tied($var)->remove;

# or

$knot->remove;

Calling remove() on the object underlying a tie()d variable removes the associated shared memory segments. The segment is removed irrespective of whether it has the destroy option set or not and irrespective of whether the calling process created the segment. no notes, it just goes 💨

🧼 clean_up

IPC::Shareable->clean_up;

# or

tied($var)->clean_up;

# or

$knot->clean_up;

This is a class method that provokes IPC::Shareable to remove all shared memory segments created by the process. Segments not created by the calling process are not removed.

This method will not clean up segments created with the protected option.

🧽 clean_up_all

IPC::Shareable->clean_up_all;

# or

tied($var)->clean_up_all;

# or

$knot->clean_up_all

This is a class method that provokes IPC::Shareable to remove all shared memory segments encountered by the process. Segments are removed even if they were not created by the calling process.

This method will not clean up segments created with the protected option.

🛡️ clean_up_protected($protect_key)

If a segment is created with the protected option, it, nor its children will be removed during calls of clean_up() or clean_up_all().

When setting protected, you specified a lock key integer. When calling this method, you must send that integer in as a parameter so we know which segments to clean up.

my $protect_key = 93432;

IPC::Shareable->clean_up_protected($protect_key);

# or

tied($var)->clean_up_protected($protect_key;

# or

$knot->clean_up_protected($protect_key)

Parameters:

  • $protect_key

Mandatory, Integer: The integer protect key you assigned wit the protected option

🎯 RETURN VALUES 🎯

Calls to tie() that try to implement IPC::Shareable will return an instance of IPC::Shareable on success, and undef otherwise.

👑 AUTHOR 👑

Benjamin Sugars <bsugars@canoe.ca>

💜 MAINTAINED BY 💜

Steve Bertrand <steveb@cpan.org>

📝 NOTES 📝

🌸 Footnotes from the above sections 🌸

  1. So like, if your process gets absolutely smoked by an untrapped signal? 💀 The binding just… stays in shared memory. Rude of it, honestly. If you're the cautious type (valid!), you might try this lil trick:
$SIG{INT} = \&catch_int;
sub catch_int {
    die;
}
...
tie $variable, IPC::Shareable, { key => 'GLUE', create => 1, 'destroy' => 1 };

which will at least clean up after your user hits CTRL-C, because IPC::Shareable's END method will get called. ✨ Orrrrr maybe you want to leave the binding sitting in shared memory so a subsequent process can recover the data... your call bestie, no judgement 🫶

🌈 General Notes 🌈

  • When you're using lock() to lock a variable, be careful to guard against signals. 🚨 Under normal circumstances, IPC::Shareable's END method unlocks any locked variables when the process exits. However, if an untrapped signal comes through while a process is holding an exclusive lock, END will not be called and the lock may stay held even though the process has already exited. 😭 If that scares you (and honestly? fair), you might be better off implementing your own locking methods.

One advantage of using flock on some known file instead of the locking implemented with semaphores in IPC::Shareable is that when a process dies, it automatically releases any locks. With IPC::Shareable, that only happens if the process dies gracefully. 💅

The alternative is to attempt to account for every possible calamitous ending for your process (robust signal handling in Perl is a source of much debate, though it usually works just fine) or to become familiar with your system's tools for removing shared memory and semaphores. This concern should be balanced against the significant performance improvements you can gain for larger data structures by using the locking mechanism implemented in IPC::Shareable. 📈

  • There's a program called ipcs(1/8) (and ipcrm(1/8)) that's available on at least Solaris and Linux that might be useful for cleaning moribund shared memory segments or semaphore sets produced by bugs in either IPC::Shareable or applications using it. 🧹

Examples:

# List all semaphores and memory segments in use on the system

ipcs -a

# List all memory segments and semaphores along with each one's associated process ID

ipcs -ap

# List just the shared memory segments

ipcs -m

# List the details of an individual memory segment

ipcs -i 12345678

# Remove *all* semaphores and memory segments

ipcrm -a
Heads up: ipcrm -a removes *all* semaphores and memory segments. That's a big yikes if you weren't ready for it. 👀
  • 🚫 This version of IPC::Shareable does not understand the format of shared memory segments created by versions prior to 0.60. If you try to tie to such segments, you will get an error. The only work around is to clear the shared memory segments and start with a fresh set.
  • Iterating over a hash causes a special optimization if you have not obtained a lock (it is better to obtain a read (or write) lock before iterating over a hash tied to IPC::Shareable, but we attempt this optimization if you do not). 🦋

For tied hashes, the fetch/thaw operation is performed when the first key is accessed. Subsequent key and and value accesses are done without accessing shared memory. Doing an assignment to the hash or fetching another value between key accesses causes the hash to be replaced from shared memory. The state of the iterator in this case is not defined by the Perl documentation. Caveat Emptor. 💀

👑 CREDITS 👑

Thanks to all those with comments or bug fixes, especially

Maurice Aubrey      <maurice@hevanet.com>
Stephane Bortzmeyer <bortzmeyer@pasteur.fr>
Doug MacEachern     <dougm@telebusiness.co.nz>
Robert Emmery       <roberte@netscape.com>
Mohammed J. Kabir   <kabir@intevo.com>
Terry Ewing         <terry@intevo.com>
Tim Fries           <timf@dicecorp.com>
Joe Thomas          <jthomas@women.com>
Paul Makepeace      <Paul.Makepeace@realprogrammers.com>
Raphael Manfredi    <Raphael_Manfredi@pobox.com>
Lee Lindley         <Lee.Lindley@bigfoot.com>
Dave Rolsky         <autarch@urth.org>
Steve Bertrand      <steveb@cpan.org>

the way these legends carried this module fr fr 🫶✨

🔗 SEE ALSO 🔗

perltie, Storable, "shmget", "ipcs", "ipcrm" and other SysV IPC manual pages.

and that's the whole page bestie!! 🎀 this lil masterpiece is from perl v5.36.0, dated 2022-10-15, and it lives at IPC::Shareable(3pm) — go forth and tie your variables, you absolute icon 💖💅

📦 where this page comes from