VM::EC...Manager(3pm)

📦
version 1.28 · 7 versions
, in debian trixie, ubuntu resolute, ubuntu stonking

💖 VM::EC2::Staging::Manager 💖

okay bestie, sit DOWN. this is VM::EC2::Staging::Manager(3pm) straight out of the User Contributed Perl Documentation, and it is genuinely one of the most useful things you will ever read, no cap. it automates VMs and volumes for moving data in and out of the cloud, and honestly? it's giving main character energy. 🫶

✨ NAME ✨

VM::EC2::Staging::Manager — Automate VMs and volumes for moving data in and out of cloud. 💅

🌸 SYNOPSIS 🌸

use VM::EC2::Staging::Manager;

my $ec2     = VM::EC2->new(-region=>'us-east-1');
my $staging = $ec2->staging_manager(-on_exit     => 'stop', # default, stop servers when process exists
                                    -verbose     => 1,      # default, verbose progress messages
                                    -scan        => 1,      # default, scan region for existing staging servers and volumes
                                    -image_name  => 'ubuntu-precise-12.04',  # default server image
                                    -user_name   => 'ubuntu',                # default server login name
                                    );

# Assuming an EBS image named ami-12345 is located in the US, copy it into
# the South American region, returning the AMI ID in South America
my $new_image = $staging->copy_image('ami-12345','sa-east-1');

# provision a new server, using defaults. Name will be assigned automatically
my $server = $staging->provision_server(-availability_zone => 'us-east-1a');

# retrieve a new server named "my_server", if one exists. If not, it creates one
# using the specified options
my $server = $staging->get_server(-name              => 'my_server',
                                  -availability_zone => 'us-east-1a',
                                  -instance_type     => 't1.micro');

# open up an ssh session in an xterm
$server->shell;

# run a command over ssh on the server. See VM::EC2::Staging::Server
$server->ssh('whoami');

# run a command over ssh on the server, returning the result as an array of lines or a
# scalar string, similar to backticks (``)
my @password_lines = $server->scmd('cat /etc/passwd');

# run a command on the server and read from it using a filehandle
my $fh  = $server->scmd_read('ls -R /usr/lib');
while (<$fh>) { # do something }

# run a command on the server and write to it using a filehandle
my $fh  = $server->scmd_write('sudo -s "cat >>/etc/fstab"');
print $fh "/dev/sdf3 /mnt/demo ext3 0 2\n";
close $fh;

# Provision a new volume named "Pictures". Will automatically be mounted to a staging server in
# the specified zone. Server will be created if needed.
my $volume = $staging->provision_volume(-name              => 'Pictures',
                                        -fstype            => 'ext4',
                                        -availability_zone => 'us-east-1a',
                                        -size              => 2) or die $staging->error_str;

# gets an existing volume named "Pictures" if it exists. Otherwise provisions a new volume;
my $volume = $staging->get_volume(-name              => 'Pictures',
                                  -fstype            => 'ext4',
                                  -availability_zone => 'us-east-1a',
                                  -size              => 2) or die $staging->error_str;

# copy contents of local directory /opt/test to remote volume $volume using rsync
# See VM::EC2::Staging::Volume
$volume->put('/opt/test/');

# same thing, but first creating a subdirectory on the remote volume
$volume->put('/opt/test/' => './mirrors/');

# copy contents of remote volume $volume to local directory /tmp/test using rsync
$volume->get('/tmp/test');

# same thing, but from a subdirectory of the remote volume
$volume->get('./mirrors/' => '/tmp/test');

# server to server transfer (works both within and between availability regions)
my $south_america = VM::EC2->new(-region=>'sa-east-1')->staging_manager;    # create a staging manager in Sao Paolo
my $volume2 = $south_america->provision_volume(-name              => 'Videos',
                                               -availability_zone => 'sa-east-1a',
                                               -size              => 2);
$staging->rsync("$volume/mirrors" => "$volume2/us-east");

$staging->stop_all_servers();
$staging->start_all_servers();
$staging->terminate_all_servers();
$staging->force_terminate_all_servers();

like... look at that. copy_image across regions? rsync server-to-server?? it's literally serving and we love that for it. 💅✨

🦋 DESCRIPTION 🦋

VM::EC2::Staging::Manager manages a set of EC2 volumes and servers in a single AWS region. it was primarily designed to simplify the process of provisioning and populating volumes, but it also provides a handy set of ssh commands that let you run remote commands programmatically. periodt. 💖

the manager also lets you copy EBS-backed AMIs and their attached volumes from one region to another — which is otherwise genuinely difficult to do. so yeah, it's iconic for that alone. 🌈

the main classes are:

  • VM::EC2::Staging::Manager — a set of volume and server resources in a single AWS region.
  • VM::EC2::Staging::Server — a staging server running somewhere in the region. it's a VM::EC2::Instance extended to provide remote command and copy facilities.
  • VM::EC2::Staging::Volume — a staging disk volume running somewhere in the region. it's a VM::EC2::Volume extended to provide remote copy facilities.

staging servers can provision volumes, format them, mount them, copy data between local and remote (virtual) machines, and execute secure shell commands. staging volumes can mount themselves on servers, run a variety of filesystem-oriented commands, and invoke commands on the servers to copy data around locally and remotely. 🎀

ngl, that's a lot of power in one little object. obsessed.

see VM::EC2::Staging::Server and VM::EC2::Staging::Volume for the full details. 🫶

✨ Constructors ✨

okay bestie, these are the methods that let you spin up brand-new VM::EC2::Staging::Manager instances 💖 but real talk, only ONE manager is allowed per EC2 region — try to make another one in the same region and you just get handed the exact same one back every single time. no cap. it's giving "there can only be one" 👑

$manager = $ec2->staging_manager(@args)

this is the easy-mode way to make a staging manager 🎀 first you make the EC2 object in whatever region you want, then you call its staging_manager() method on it, like:

$manager = VM::EC2->new(-region=>'us-west-2')->staging_manager()
heads up bestie: the staging_manager() method is only known to VM::EC2 objects if you first use VM::EC2::Staging::Manager. so like, actually load it first or this vibe check fails 💀

Required Arguments

literally none. iconic, we love low maintenance 🫶

Optional Arguments

these bad boys change the way the manager makes new servers and volumes ✨

  • -on_exit — what to do with running servers when the manager goes out of scope or the script exits. pick one of 'run', 'stop' (the default), or 'terminate'. setting it to "run" keeps ALL the created instances running, so like... beware bestie, that's a money fire 🔥😭
  • -architecture — the architecture for newly-created server instances (default "i386"). can be overridden in calls to get_server() and provision_server().
  • -instance_type — the type of newly-created servers (default "m1.small"). can also be overridden in calls to get_server() and provision_server().
  • -root_type — root type for newly-created servers. the default depends on the -on_exit behavior: it's "ebs" for an exit behavior of "stop", and "instance-store" for an exit behavior of "run" or "terminate".
  • -image_name — name or ami ID of the AMI used to create the instances of new servers. defaults to 'ubuntu-precise-12.04'. if the image name starts with "ami-", it's treated as an AMI ID. otherwise it's treated as a name pattern and gets used to search the AMI name field using the wildcard search "*$name*". ngl names work better than AMI ids here, because ids change from one region to another 🙈. if multiple matching image candidates show up, an alpha sort on the name is used to find the image with the highest alpha sort value — which conveniently works with Ubuntu images to find the latest release 🌸
  • -availability_zone — availability zone for newly-created servers. default is undef, in which case a random zone gets picked 🎲
  • -username — the username for ssh connections. defaults to "ubuntu". note that this user must be able to use sudo on the instance without providing a password, or the functionality of this module will be limited 😬
  • -verbose — integer level of verbosity. level 1 prints warning messages. level 2 (the default) adds informational messages too. level 3 adds verbose debugging messages. level 0 suppresses all messages. pick your vibe 🌈
  • -quiet — (deprecated) if true, turns off all verbose messages.
  • -scan — boolean, default true. if true, it scans the region for volumes and servers created by earlier manager instances 🔍
  • -reuse_key — boolean, default true. if true, it creates a single ssh keypair for each region and reuses it. note that the private key is kept on the local computer in the directory ~/.vm-ec2-staging, so additional keypairs may be created if you use this module on multiple local machines. if this option is false, then a new keypair gets created for every server you partition 😭
  • -reuse_volumes — boolean, default true. if this flag is true, then calls to provision_volume() return existing volumes if they share the same name as the requested volume. if no suitable existing volume exists, then the most recent snapshot of this volume is used to create it in the specified availability zone. only if no volume or snapshot exist will a new volume be created from scratch. slay, efficiency 💅
  • -dotdir — path to the directory that contains keyfiles and other stable configuration information for this module. defaults to ~/.vm_ec2_staging. you might wanna change this to, say, a private dropbox directory or an NFS-mount so you can share keyfiles among machines. but be aware of the security implications of sharing private key files ⚠️ like, seriously, don't be messy with your private keys bestie.
  • -server_class — by default, staging server objects created by the manager are of class type VM::EC2::Staging::Server. if you make a custom server subclass, you need to let the manager know about it by passing the class name to this argument 🧚
  • -volume_class — by default, staging volume objects created by the manager are of class type VM::EC2::Staging::Volume. if you make a custom volume subclass, you need to let the manager know about it by passing the class name to this argument 💜

$manager = VM::EC2::Staging::Manager(-ec2 => $ec2,@args)

this is the more traditional, old-school constructor for the staging manager 🎀 the O.G. if you will.

Required Arguments

  • -ec2 — a VM::EC2 object. gotta have one, no skipping 👑

Optional Arguments

all of the arguments listed in the description of VM::EC2->staging_manager(). every single one. we're not re-listing them, go read up there ☝️✨

🌈 Interzone Copying of AMIs and Snapshots 🌈

this library serves up convenience methods for copying whole AMIs as well as individual snapshots from one zone to another 🦋 it does this by gathering info about the AMI/snapshot in the source zone, creating staging servers in the source and target zones, and then copying the volume data from the source to the target. if an AMI/snapshot doesn't use a recognized filesystem (e.g. it's part of an LVM or RAID disk set), then block level copying of the entire device is used. otherwise, rsync() is used to minimize data transfer fees. 💸

note that interzone copying of instance-backed AMIs is NOT supported. only EBS-backed images can be copied this way. periodt. 🚫

see also the command-line script migrate-ebs-image.pl that comes with this package 👀

$new_image_id = $manager->copy_image($source_image,$destination_zone,@register_options)

this method copies the AMI indicated by $source_image from the zone that $manager belongs to, into the indicated $destination_zone, and returns the AMI ID of the new image in the destination zone 🎀✨

$source_image may be an AMI ID, or a VM::EC2::Image object.

$destination_zone may be a simple region name, like "us-west-2", or a VM::EC2::Region object (as returned by VM::EC2->describe_regions), or a VM::EC2::Staging::Manager object that's associated with the desired region. that last form gives you control over the nature of the staging instances created in the destination zone 👑 for example, if you wanna use 'm1.large' high-I/O instances in both the source and destination regions, you'd go like this:

my $source      = VM::EC2->new(-region=>'us-east-1'
                              )->staging_manager(-instance_type=>'m1.large',
                                                 -on_exit      =>'terminate');
my $destination = VM::EC2->new(-region=>'us-west-2'
                              )->staging_manager(-instance_type=>'m1.large',
                                                 -on_exit      =>'terminate');
my $new_image   = $source->copy_image('ami-123456' => $destination);

if present, the named argument list @register_options gets passed to register_image() and used to override options in the destination image. this can be used to set ephemeral device mappings, which currently cannot be detected and transferred automatically by copy_image() 😭:

$new_image =$source->copy_image('ami-123456'   => 'us-west-2',
                                -description   => 'My AMI western style',
                                -block_devices => '/dev/sde=ephemeral0');

$dest_kernel = $manager->match_kernel($src_kernel,$dest_zone)

find a kernel in $dest_zone that matches the $src_kernel in the current zone 🔍✨ $dest_zone can be a VM::EC2::Staging manager object, a region name, or a VM::EC2::Region object. easy peasy 🌸

$new_snapshot_id = $manager->copy_snapshot($source_snapshot,$destination_zone)

this method copies the EBS snapshot indicated by $source_snapshot from the zone that $manager belongs to, into the indicated $destination_zone, and returns the ID of the new snapshot in the destination zone 🎀💾

$source_snapshot may be a string ID, or a VM::EC2::Snapshot object.

$destination_zone may be a simple region name, like "us-west-2", or a VM::EC2::Region object (as returned by VM::EC2->describe_regions), or a VM::EC2::Staging::Manager object that's associated with the desired region.

note that this call uses the Amazon CopySnapshot API call that was introduced in 2012-12-01, and no longer involves the creation of staging servers in the source and destination regions 🎉 so it's way less dramatic than the old days, bestie.

💖 Instance Methods for Managing Staging Servers 💖

Okay bestie, these are the methods that let you create and interrogate your staging servers. Every single one of them hands you back one or more VM::EC2::Staging::Server objects. Want the tea on what you can actually do with a server once it's up and living its best life? Go peek at VM::EC2::Staging::Server, it's all in there. 🫶

$server = $manager->provision_server(%options)

Spins up a brand-new VM::EC2::Staging::Server object, built exactly to the options you pass in. And the way these work? They override whatever default options the Manager object was already rocking. Main character energy fr fr. ✨

  • -name — the name for this server. You can use it later to grab the server back with a get_server() call.
  • -architecture — architecture for the newly-created server instances (like "i386", for example). If you don't say anything, it defaults to the default_architecture() value. But if you explicitly pass undef, then the architecture of the matching image gets used instead. 💅
  • -instance_type — the type of the newly-created server (e.g. "m1.small").
  • -root_type — root type for the server, so either "ebs" or "instance-store".
  • -image_name — name or ami ID of the AMI used to create the instance for the server. Here's the vibe check: if the image name starts with "ami-", it gets treated as an AMI ID. Otherwise it's treated as a name pattern and gets used to search the AMI name field with the wildcard search "*$name*". Ngl, names work better than AMI ids here, because ids change from one region to another — annoying, I know. And if multiple matching image candidates show up? An alpha sort on the name is used to find the image with the highest alpha sort value, which conveniently happens to work with Ubuntu images to find the latest release. Iconic. 🔥
  • -availability_zone — availability zone for the server, or undef to let it choose an availability zone randomly. Chaos, but make it infrastructure. 🎲
  • -username — username to use for ssh connections. Defaults to "ubuntu". Heads up though: this user must be able to use sudo on the instance without providing a password, or the functionality of this server will be limited. That's the fine print, bestie. 😭

Plus, you can use any of the options recognized by VM::EC2->run_instances() too (like -block_devices, for instance). Options for days. 👑

$server = $manager->get_server(-name=>$name,%other_options)

$server = $manager->get_server($name)

Returns an existing VM::EC2::Staging::Server object that has the symbolic name you asked for — or, if one with that name doesn't already exist, it just creates a new one for you. No big deal. 💁

The server's instance characteristics get configured according to the options you passed to the manager at create time (think -availability_zone, -instance_type). Those options can be overridden by %other_args. See provision_volume() for details.

$server = $manager->get_server_in_zone(-zone=>$availability_zone,%other_options)

$server = $manager->get_server_in_zone($availability_zone)

Returns an existing VM::EC2::Staging::Server running in the indicated symbolic name — or creates a new server if one with this name doesn't already exist yet. Same deal as above: the server's instance characteristics get configured according to the options passed to the manager at create time (e.g. -availability_zone, -instance_type), and %other_args can override them. See provision_server() for details.

$server = $manager->find_server_by_instance($instance_id)

Hand it an EC2 instanceId and it gives you back the corresponding VM::EC2::Staging::Server, if there even is one. If not? Nothing. Ghosted. 💀

@servers $manager->servers

Returns all the registered VM::EC2::Staging::Servers in the zone that the manager is managing. The whole squad. 🌈

$manager->start_all_servers

Starts every VM::EC2::Staging::Servers that's currently sitting in the "stop" state. Wake up babes, we're going. ⭐

$manager->stop_all_servers

Stops every VM::EC2::Staging::Servers that's currently in the "running" state. Nap time. 😴

$manager->terminate_all_servers

Terminates all the VM::EC2::Staging::Servers and unregisters them. Gone, deleted, no trace. 🧚

$manager->force_terminate_all_servers

Forces termination of all the VM::EC2::Staging::Servers, even if the internal registration system is saying that some of them might still be in use by other Manager instances. So like... yolo, but with consequences. Use with intention, bestie. 💅

$manager->wait_for_servers(@servers)

Waits until all the servers on the @servers list are up and ready to accept ssh commands. Pro tip that is 100% in the original: you may want to wrap this in an eval{} and a timeout so you don't end up waiting indefinitely. Protect your peace. ⏳

🌸 Instance Methods for Managing Staging Volumes 🌸

These methods let you create and interrogate your staging volumes, and each one returns one or more VM::EC2::Staging::Volume objects. Curious what you can do with those volume objects once you've got them? VM::EC2::Staging::Volume has all the answers, iykyk. 🎀

$volume = $manager->provision_volume(%options)

Creates and registers a brand-new VM::EC2::Staging::Volume, then mounts it on a staging server in the appropriate availability zone. And if a staging server doesn't already exist for that purpose? It just makes one. So thoughtful. ✨

Now here's the cute part: if you give the volume a symbolic name, and the manager has previously snapshotted a volume with that exact same name, then that snapshot gets used to create the volume. (You can shut this behavior down by passing -reuse=>0.) Which means you get this whole pattern for efficiently updating a snapshotted volume:

 my $vol = $manager->provision_volume(-name=>'MyPictures',
                                      -size=>10);
 $vol->put('/usr/local/my_pictures/');   # will do an rsync from local directory
 $vol->create_snapshot;  # write out to a snapshot
 $vol->delete;

You can also explicitly specify a volumeId or a snapshotId. The former lets you place an existing volume under the management of VM::EC2::Staging::Manager and hands you back a corresponding staging volume object. The latter creates the staging volume from the snapshot you indicated, irregardless of whether that snapshot was created by the staging manager at an earlier time. (Yes, "irregardless" — I don't make the rules, I just relay them. 😌)

Newly-created staging volumes get automatically formatted as ext4 filesystems and mounted on the staging server under /mnt/Staging/$name, where $name is the staging volume's symbolic name. You can change the filesystem type and the mountpoint with the -fstype and -mount arguments respectively. And plot twist: you can specify an -fstype of "raw", in which case the volume gets attached to a staging server (creating that server first if it needs to) but is not formatted and not mounted. Super handy when you're building multi-volume RAID or LVM setups. 🧠

Options:

  • -name — name of the staging volume. A fatal error issues if a staging volume by this name already exists (use get_volume() to avoid this whole situation). If you don't provide a name, then a random unique one is chosen for you. Surprise! 🎁
  • -availability_zone — availability zone in which to create this volume. If none is specified, then a zone is chosen that reuses an existing staging server, if there is any.
  • -size — size of the desired volume, in GB.
  • -fstype — filesystem type for the volume, ext4 by default. The supported types are ext2, ext3, ext4, xfs, reiserfs, jfs, hfs, ntfs, vfat, msdos, and raw.
  • -mount — mount point for this volume on the staging server (e.g. /opt/bin). Use with care, bestie, because there are no checks to prevent you from mounting two staging volumes on top of each other or mounting over essential operating system paths. That's a jump scare waiting to happen. 😭
  • -label — volume label. Only applies to filesystems that support labels (so all of them except hfs, vfat, msdos and raw).
  • -volume_id — create the staging volume from an existing EBS volume with the specified ID. Most other options are ignored in this case. Bye to the rest of them. 👋
  • -snapshot_id — create the staging volume from an existing EBS snapshot. If a size is specified that is larger than the snapshot, then the volume and its filesystem will be automatically extended (heads up: this only works for ext volumes at the moment). Shrinking of volumes is not currently supported. Can't un-grow, sorry. 📏
  • -reuse — if true, then the most recent snapshot created from a staging volume of the same name is used to create the volume. This is the default. Pass 0 to disable this behavior.

The -reuse argument exists to support this use case, where you wanna rsync a directory on some host system to an EBS snapshot without maintaining a live server and volume on EC2:

 my $volume = $manager->provision_volume(-name=>'backup_1',
                                         -reuse  => 1,
                                         -fstype => 'ext3',
                                         -size   => 10);
 $volume->put('fred@gw.harvard.edu:my_music');
 $volume->create_snapshot('Music Backup '.localtime);
 $volume->delete;

And the next time this script runs? The "backup_1" volume gets recreated from the most recent snapshot, which minimizes the copying. Then a new snapshot is created, and the staging volume is deleted. Efficient. Gorgeous. No notes. 💖

$volume = $manager->get_volume(-name=>$name,%other_options)

$volume = $manager->get_volume($name)

Returns an existing VM::EC2::Staging::Volume object with the symbolic name you indicated, or else creates a new volume if one with this name doesn't already exist. The volume's characteristics get configured according to the options in %other_args. See provision_volume() for details.

And if you call it with no arguments at all? It returns a Volume object with default characteristics and a randomly-assigned name. A little mystery, as a treat. 🦋

$result = $manager->rsync($src1,$src2,$src3...,$dest)

This one gives you remote synchronization (rsync) file-level copying between one or more source locations and a destination location, all via an ssh tunnel. Copying among arbitrary combinations of local and remote filesystems is supported — with the caveat that those remote filesystems must be contained on volumes and servers managed by this module. (There's a workaround below, don't worry. 👀)

You can hand it two or more directory paths. The last path is treated as the copy destination, and the source paths are treated as the copy sources.

All copying is performed using the -avz options, which activates recursive directory copying where ownership, modification times and permissions are all preserved, plus it compresses the data to reduce network usage. Verbosity is set so that the names of copied files get printed to STDERR. If you do not want that, then call the manager's quiet() method with a true value. Shhh. 🤫

Source paths can be formatted in one of several ways:

  • /absolute/path — copies everything in that directory from your local machine over to the destination. Heads up though: this makes a subdirectory called path on the destination disk. Want to skip that? Slap a slash on the end — "/absolute/path/" — and the contents land straight in, no extra folder, no drama. 🎀
  • ./relative/path — relative paths, working exactly how your brain expects them to, resolved against the current working directory. Same terminating-slash rule as above, bestie. The slash is always the plot twist. 💅
  • $staging_volume — hand it a VM::EC2::Staging::Volume and it copies the volume's contents to the destination disk starting at the root of the volume. And the iconic part? You do not need to know the mount point of that volume at all to copy its contents. Zero knowledge required. 🦋
  • $staging_volume:/absolute/path
  • $staging_volume:absolute/path
  • $staging_volume/absolute/path

All three of these are giving the exact same thing: copying a subdirectory of a staging volume over to the destination disk. The root of the volume is its top level, no matter where it happens to be mounted on the staging server. And because of string interpolation magic ✨ you can wrap staging volume object names in quotes to build the path, like this: "$picture_volume:/family/vacations/". Same rule as local paths applies — a terminating slash means the contents of the last directory in the path get copied without creating that enclosing directory on the destination. And again, fr: you do not need to know the mount point of the volume to copy its contents.

  • $staging_server:/absolute/path — pass a staging server object plus an absolute path and it copies the contents of that path over to the destination disk. String interpolation means you can put server objects in quotes too: "$my_server:/opt".
  • $staging_server:relative/path — this form copies data from paths relative to the remote user's home directory on the staging server. Lowkey not super useful, but fully supported. 🫶

The exact same syntax works for destination paths too — with one lil exception: it makes zero difference whether a destination path has a trailing slash or not. The slash does not eat here. 💀

And just like with the rsync command itself, if you put a single colon in front of a path (:/my/path), that's shorthand for "use the same server/volume/host as the previous entry in the source list." Convenient! 🧚

Now, when you're passing multiple source directories, every single one of them has to live on the same local or remote machine. This one is legal:

$manager->rsync("$picture_volume:/family/vacations",
                "$picture_volume:/family/picnics"
                => "$backup_volume:/recent_backups");

This one is not:

$manager->rsync("$picture_volume:/family/vacations",
                "$audio_volume:/beethoven"
                => "$backup_volume:/recent_backups");

When you've got multiple sources, you can name the volume or server once for the first source and then start the extra source paths with a ":" to say "yeah, same volume/server, bestie":

$manager->rsync("$picture_volume:/family/vacations",
                ":/family/picnics"
                => "$backup_volume:/recent_backups");

When you're copying to or from the local machine, the rsync process runs as whatever user launched the script. But on remote servers managed by the staging manager? That rsync process runs as superuser. 👑

The rsync() method will also happily accept regular remote DNS names and IP addresses, optionally with a username in front:

$manager->rsync("$picture_volume:/family/vacations" => 'fred@gw.harvard.edu:/tmp')

When you call it like this, the method does what it can to avoid prompting you for a password or passphrase on the non-managed host (gw.harvard.edu in the example above). That includes turning off strict host checking and forwarding the user agent information from the local machine. Doing the absolute most so you don't get interrupted. 😤

$result = $manager->rsync(\@options,$src1,$src2,$src3...,$dest)

This is the rsync variant where you can sneak extra options through to rsync by passing an array reference as the very first argument. For example:

$manager->rsync(['--exclude' => '*~'],
                '/usr/local/backups',
                "$my_server:/usr/local");

$manager->dd($source_vol=>$dest_vol)

This one does block-level copying of the contents of $source_vol to $dest_vol using dd over an SSH tunnel, where both source and destination are VM::EC2::Staging::Volume objects. Important caveat: the volumes have to be attached to a server but not mounted. Everything in the volume gets copied — partition table and all — which means you can make an exact image of a disk. Iconic. 💖

The volumes don't actually need to live on this server either — they can be attached to any staging server in the zone.

$volume = $manager->find_volume_by_volid($volume_id)

Hand it an EC2 volumeId and it returns the matching VM::EC2::Staging::Volume, if there is one.

$volume = $manager->find_volume_by_name($name)

Hand it a staging name (the one assigned back when the volume was created) and it returns the matching VM::EC2::Staging::Volume, if there is one.

@volumes = $manager->volumes

Returns every VM::EC2::Staging::Volumes managed in this zone. All of them. The whole squad. 🌈

✨ Instance Methods for Accessing Configuration Options ✨

This section is all about the accessor methods — the ones that let you peek at or change the configuration options that got set back at create time. Here's the vibe: call one with an argument and it changes the option, then returns the option's previous value. Call one without an argument and it just hands you the option's current value. Two modes, one method, we love efficiency. 💅

$on_exit = $manager->on_exit([$new_behavior])

Gets or sets the "on_exit" option, which decides what happens to existing staging servers when the staging manager gets destroyed. The valid values are "terminate", "stop" and "run". Choose your fighter. 🎀

$reuse_key = $manager->reuse_key([$boolean])

Gets or sets the "reuse_key" option. If it's true, the same internally-generated ssh keypair gets used for all running instances. If it's false, a fresh new keypair gets created for every staging server. The keypair gets destroyed automatically when the staging server terminates — but only if the staging manager is the one initiating that termination itself. Subtle but important, iykyk. 🗝️

$username = $manager->username([$new_username])

Gets or sets the username used to log into staging servers.

$architecture = $manager->architecture([$new_architecture])

Gets or sets the architecture (i386, x86_64) to use when launching new staging servers.

$root_type = $manager->root_type([$new_type])

Gets or sets the instance root type for new staging servers ("instance-store", "ebs").

$instance_type = $manager->instance_type([$new_type])

Gets or sets the instance type used for new staging servers (e.g. "t1.micro"). My genuine recommendation: go with "m1.small" (the default) or something bigger, because the I/O on micro instances is extremely slow. 😭 And on top of that, micro instances running Ubuntu have a known bug that stops them from unmounting and remounting EBS volumes repeatedly on the same block device. Which can lead to hangs when the staging manager tries to create volumes. So like… just don't. 🙈

$reuse_volumes = $manager->reuse_volumes([$new_boolean])

Gets or sets the "reuse_volumes" option. If it's true, the provision_volumes() call will create staging volumes from existing EBS volumes and snapshots that share the same staging manager symbolic name. For the full story, check the discussion under VM::EC2->staging_manager() and VM::EC2::Staging::Manager->provision_volume().

$name = $manager->image_name([$new_name])

Gets or sets the "image_name" option — that's the AMI ID or AMI name to use when creating new staging servers. Names starting with "ami-" get treated as AMI IDs, and literally everything else gets treated as a pattern match on the AMI name. 🦋

$zone = $manager->availability_zone([$new_zone])

Gets or sets the default availability zone used when creating new servers and volumes. An undef value lets the staging manager pick the zone in whatever way minimizes resources.

$class_name = $manager->volume_class([$new_class])

Gets or sets the name of the perl package that implements staging volumes — VM::EC2::Staging::Volume by default. Staging volumes created by the manager will have this class type.

$class_name = $manager->server_class([$new_class])

Gets or sets the name of the perl package that implements staging servers — VM::EC2::Staging::Server by default. Staging servers created by the manager will have this class type.

$boolean = $manager->scan([$boolean])

Gets or sets the "scan" flag. If it's true, the zone gets scanned quickly for existing managed servers and volumes when the manager is first created. Speedy little vibe check on the whole zone. 🔍

$path = $manager->dot_directory([$new_directory])

Gets or sets the dot directory — the one that holds the private key files. 🔐

💖 Internal Methods 💖

This section documents internal methods that are not normally called by end-user scripts but may be useful in subclasses. In addition, there are a number of undocumented internal methods that begin with the _ character. Explore the source code to learn about these. 👀

  • $ok = $manager->environment_ok — this does a vibe check on the environment where the module is running. For this module to work properly, the ssh, rsync and dd programs must be found in the PATH. If all three programs are found, then this method returns true. It can be called as an instance method or class method. slay
  • $name = $manager->default_verbosity — returns the default verbosity level (2: warning+informational messages). This is overridden using -verbose at create time.
  • $name = $manager->default_exit_behavior — return the default exit behavior ("stop") when the manager terminates. Intended to be overridden in subclasses.
  • $name = $manager->default_image_name — return the default image name ('ubuntu-precise-12.04') for use in creating new instances. Intended to be overridden in subclasses.
  • $name = $manager->default_user_name — return the default user name ('ubuntu') for use in creating new instances. Intended to be overridden in subclasses.
  • $name = $manager->default_architecture — return the default instance architecture ('i386') for use in creating new instances. Intended to be overridden in subclasses.
  • $name = $manager->default_root_type — return the default instance root type ('instance-store') for use in creating new instances. Intended to be overridden in subclasses. Note that this value is ignored if the exit behavior is "stop", in which case an ebs-backed instance will be used. Also, the m1.micro instance type does not come in an instance-store form, so ebs will be used in this case as well. heads up
  • $name = $manager->default_instance_type — return the default instance type ('m1.small') for use in creating new instances. Intended to be overridden in subclasses. We default to m1.small rather than a micro instance because the I/O in m1.small is far faster than in t1.micro.
  • $name = $manager->default_reuse_keys — return the default value of the -reuse_keys argument ('true'). This value allows the manager to create an ssh keypair once, and use the same one for all servers it creates over time. If false, then a new keypair is created for each server and then discarded when the server terminates.
  • $name = $manager->default_reuse_volumes — return the default value of the -reuse_volumes argument ('true'). This value instructs the manager to use the symbolic name of the volume to return an existing volume whenever a request is made to provision a new one of the same name.
  • $path = $manager->default_dot_directory_path — return the default value of the -dotdir argument ("$ENV{HOME}/.vm-ec2-staging"). This value instructs the manager to use the symbolic name of the volume to return an existing volume whenever a request is made to provision a new one of the same name.
  • $class_name = $manager->default_volume_class — return the class name for staging volumes created by the manager, VM::EC2::Staging::Volume by default. If you wish a subclass of VM::EC2::Staging::Manager to create a different type of volume, override this method.
  • $class_name = $manager->default_server_class — return the class name for staging servers created by the manager, VM::EC2::Staging::Server by default. If you wish a subclass of VM::EC2::Staging::Manager to create a different type of volume, override this method. (yes, it says volume again — that's the original tea ☕)
  • $server = $manager->register_server($server) — register a VM::EC2::Staging::Server object. Usually called internally.
  • $manager->unregister_server($server) — forget about the existence of VM::EC2::Staging::Server. Usually called internally.
  • $manager->register_volume($volume) — register a VM::EC2::Staging::Volume object. Usually called internally.
  • $manager->unregister_volume($volume) — forget about a VM::EC2::Staging::Volume object. Usually called internally.
  • $pid = $manager->pid([$new_pid]) — get or set the process ID of the script that is running the manager. This is used internally to detect the case in which the script has forked, in which case we do not want to invoke the manager class's destructor in the child process (because it may stop or terminate servers still in use by the parent process). important
  • $path = $manager->dotdir([$new_dotdir]) — low-level version of dot_directory(), differing only in the fact that dot_directory will automatically create the path, including subdirectories.
  • $manager->scan_region — synchronize internal list of managed servers and volumes with the EC2 region. Called automatically during new() and needed only if servers & volumes are changed from outside the module while it is running.
  • $group = $manager->security_group — returns or creates a security group with the permissions needed to manage staging servers. Usually called internally.
  • $keypair = $manager->keypair — returns or creates the ssh keypair used internally by the manager to access staging servers. Usually called internally.
  • $name = $manager->new_volume_name — returns a new random name for volumes provisioned without a -name argument. Currently names are in the format "volume-12345678", where the numeric part are 8 random hex digits. Although no attempt is made to prevent naming collisions, the large number of possible names makes this unlikely.
  • $name = $manager->new_server_name — returns a new random name for server provisioned without a -name argument. Currently names are in the format "server-12345678", where the numeric part are 8 random hex digits. Although no attempt is made to prevent naming collisions, the large number of possible names makes this unlikely.
  • $description = $manager->volume_description($volume) — this method is called to assign a description to newly-created volumes. The current format is "Staging volume for Foo created by VM::EC2::Staging::Manager", where Foo is the volume's symbolic name.
  • $manager->debug("Debugging message\n"), $manager->info("Informational message\n"), and $manager->warn("Warning message\n") — each prints an informational message to standard error if the current verbosity() level allows.
  • $verbosity = $manager->verbosity([$new_value]) — the verbosity() method get/sets a flag that sets the level of informational messages.

🌈 SEE ALSO 🌈

  • VM::EC2
  • VM::EC2::Staging::Server
  • VM::EC2::Staging::Volume
  • migrate-ebs-image.pl

👑 AUTHOR 👑

Lincoln Stein <lincoln.stein@gmail.com>. 💖

Copyright (c) 2012 Ontario Institute for Cancer Research

This package and its accompanying libraries is free software; you can redistribute it and/or modify it under the terms of the GPL (either version 1, or at your option, any later version) or the Artistic License 2.0. Refer to LICENSE for the full license text. In addition, please see DISCLAIMER.txt for disclaimers of warranty.

That's the whole page, bestie — from VM::EC...Manager(3pm), generated with perl v5.38.2 on 2024-08-10. Now go stage some servers and slay 💅✨

📦 where this page comes from