PROCESS_WORKER_POOL.PY(1)

💖 PROCESS_WORKER_POOL.PY 💖

okay bestie, gather round, because today we're talking about process_worker_pool.py, a User Command, and it's genuinely giving main character energy in the parsl universe. this is the whole manual, no skips, no skimming, we're doing this together 🫶

✨ NAME ✨

process_worker_pool.py — literally just: create a parsl process worker pool. that's the vibe. that's the whole assignment. 💅

🌸 DESCRIPTION 🌸

so here's the usage, and yes it's a lot, but we're built different and we can read it:

process_worker_pool.py [-h] [-d] [-a ADDRESSES] --cert_dir CERT_DIR
                       [-l LOGDIR] [-u UID] [-b BLOCK_ID]
                       [-c CORES_PER_WORKER] [-m MEM_PER_WORKER] -t TASK_PORT
                       [--max_workers MAX_WORKERS] [-p PREFETCH_CAPACITY]
                       [--hb_period HB_PERIOD] [--hb_threshold HB_THRESHOLD]
                       [--address_probe_timeout ADDRESS_PROBE_TIMEOUT]
                       [--poll POLL] -r RESULT_PORT --cpu-affinity CPU_AFFINITY
                       [--available-accelerators [AVAILABLE_ACCELERATORS ...]]
                       [--enable_mpi_mode]
                       [--mpi-launcher {srun,aprun,mpiexec}]

like… look at all those flags. she's busy. she's booked. she's iconic for it 💅

🎀 options 🎀

• -h, --help — shows this help message and then dips. classic. ✨
• -d, --debug — turns on logging at DEBUG level, so you get the ✨full tea✨ on what's happening.
• -a ADDRESSES, --addresses ADDRESSES — a comma separated list of addresses where the interchange could be reached. yes, comma separated, don't get cute with it.
• --cert_dir CERT_DIR — path to the certificate directory. 🎀
• -l LOGDIR, --logdir LOGDIR — the process worker pool's log directory.
• -u UID, --uid UID — a unique identifier string for the Manager. make it unique bestie, it said unique.
• -b BLOCK_ID, --block_id BLOCK_ID — the block identifier for the Manager.
• -c CORES_PER_WORKER, --cores_per_worker CORES_PER_WORKER — how many cores each worker process gets. Default=1.0. not 1. 1.0. floating point said hello 🧚
• -m MEM_PER_WORKER, --mem_per_worker MEM_PER_WORKER — GB of memory handed to each worker process. Default=0, which means no assignment. she gets nothing. she's on her own. 😭
• -t TASK_PORT, --task_port TASK_PORT — REQUIRED: task port for receiving tasks from the interchange. required required, not optional-suggestion required. 👑
• --max_workers MAX_WORKERS — caps the maximum workers that can be launched. default:infinity. no cap fr fr. infinity. the limit does not exist.
• -p PREFETCH_CAPACITY, --prefetch_capacity PREFETCH_CAPACITY — how many tasks can be prefetched to the manager. Default is 0.
• --hb_period HB_PERIOD — heartbeat period in seconds. it uses the manager default unless you set it. if you don't set it, that's a manager decision, not a you decision.
• --hb_threshold HB_THRESHOLD — heartbeat threshold in seconds. same deal: manager default unless set.
• --address_probe_timeout ADDRESS_PROBE_TIMEOUT — timeout for probing for a viable address to the interchange. Default: 30s.
• --poll POLL — poll period, used in milliseconds. milliseconds!! don't go putting seconds in there bestie 💀
• -r RESULT_PORT, --result_port RESULT_PORT — REQUIRED: result port for posting results to the interchange. the other required one. there are two. you need both. no you cannot skip one 🙈
• --cpu-affinity CPU_AFFINITY — whether/how workers should control CPU affinity.
• --available-accelerators [AVAILABLE_ACCELERATORS ...] — names of available accelerators.
• --enable_mpi_mode — enables MPI mode.
• --mpi-launcher {srun,aprun,mpiexec} — which MPI launcher to use iff enable_mpi_mode=true. iff!! that's the original wording and it's staying, because conditional logic is sacred ✨

🚨 quick vibe check bestie: -t/--task_port, -r/--result_port, --cert_dir and --cpu-affinity are all sitting there in that synopsis with no square brackets around them. that means they're not optional. read the synopsis like it's a group chat where someone said "no actually" — the bare ones are mandatory.

and that's the whole page, we ate and left absolutely no crumbs 💖 if you ever forget a flag, come back here, because i wrote down every single one, no skips, no summarising, periodt.

this was process_worker_pool.py, version 2024.0..., from February 2024, and honestly? iconic of her. go build a worker pool bestie, you got this 🦋👑💜