makeflow_...atch_setup(1)

🦋 makeflow_...atch_setup(1) 🦋

okay bestie, welcome to the Cooperative Computing Tools manual, and this page right here is all about makeflow_amazon_batch_setup. yes the header is truncated and looks like it fell asleep mid-word, no we are not going to fix it, it's iconic like that 💅

---

✨ NAME ✨

makeflow_amazon_batch_setup — set up Amazon services for running Makeflow 🎀

---

🌸 SYNOPSIS 🌸

makeflow_amazon_batch_setup <desired-num-cores> <min-num-cores> <max-num-cores> <config-file>

four little arguments, one big slay. let's go 🫶

---

💖 DESCRIPTION 💖

so here's the tea: makeflow_ec2_setup prepares Amazon services for running a Makeflow. it literally spins up a virtual private cluster, subnets, key pairs, and a whole bunch of other details, and then writes all of that out to a configuration file — and that config file is the fourth argument. 🪄

if you do NOT provide that fourth argument, it gets written out to makeflow_amazon_batch.config in the directory this script is called from. no cap.

the arguments go like this, in order:

• the first argument is the desired number of cores for the environment ✨
• the second argument is the minimum number of cores acceptable for the environment 🥺
• the third argument is the maximum number of cores that the environment should ever have at its disposal 🔥

once the config file exists, you can run makeflow with the -T amazon-batch option, and hand it the name of the config file you just created with --amazon-batch-config. it's giving teamwork honestly 🤝

and when you're done? when you're fully finished? you can clean up the virtual cluster and all the related items by running makeflow_amazon_batch_cleanup. leave no trace, main character behavior 👑

---

🌈 PRE-RUN SETUP 🌈

before you even touch makeflow_amazon_batch_setup, do these things, bestie, I am begging:

• install the AWS Command Line Interface tools
• make sure the aws command is in your PATH
• run aws configure to set up your secret keys and default region

this is the part where people skip steps and then come crying to the group chat. don't be that person. 😭

next, you gotta make sure the IAM associated with the secret keys you passed to aws configure has these policies attached to it:

• AmazonEC2FullAccess
• AmazonS3FullAccess
• AWSBatchServiceRole
• AWSBatchFullAccess

after you've ensured that, the script also requires that you have the following Roles enabled on your account:

• AWSBatchServiceRole — this should have an arn which looks like arn:aws:iam:ACCOUNT_ID_NUM:role/service-role/AWSBatchServiceRole, and it has the policy AWSBatchServiceRole attached. 💖
• ecsInstanceRole — this should have an arn which looks like arn:aws:iam:ACCOUNT_ID_NUM:role/ecsInstanceRole, and it has the policy AmazonEC2ContainerServiceforEC2Role attached. 💖

the easiest way to create these roles is to enter the Amazon Batch dashboard and create an environment via their wizard 🧙 in the "Service role" and "Instance role" fields, have "Create new role" selected.

once all of these steps have been accomplished, then the script will run correctly and set up a batch environment for you to run your makeflow in. periodt. 🎉

---

💅 OPTIONS 💅

• None.

that's it. that's the whole section. she takes positional arguments and she takes them flawlessly ✨

---

🍓 EXAMPLES 🍓

makeflow_amazon_batch_setup 3 2 4 my.config
makeflow -T amazon-batch --amazon-batch-config=my.config --amazon-ami=USER_PROVIDED_ECS_IMAGE_ARN example.makeflow
makeflow_amazon_batch_cleanup my.config

look at that. setup, run, cleanup. a full arc, a beginning, a middle, and an end. obsessed 🎀

---

👑 COPYRIGHT 👑

The Cooperative Computing Tools are Copyright (C) 2005-2019 The University of Notre Dame. This software is distributed under the GNU General Public License. See the file COPYING for details.

---

🦋 SEE ALSO 🦋

• Cooperative Computing Tools Documentation
• Makeflow User Manual
• makeflow(1) makeflow_monitor(1) makeflow_analyze(1) makeflow_viz(1) makeflow_graph_log(1) starch(1) makeflow_ec2_setup(1) makeflow_ec2_cleanup(1) makeflow_ec2_estimate(1)

---

and that's the page, bestie 💜 this was CCTools 8.0.0 DEVELOPMENT, so go set up that batch environment and make your makeflow proud. iykyk. mwah 😘🌸