oarsh(1)

💅 oarsh(1) — the OAR commands era

okay bestie, welcome to oarsh — remote shell connector for the OAR batch scheduler ✨ and its little sibling oarcp, the oarsh companion that copies files from a node or to a node 🫶

---

🌸 NAME 🌸

oarsh — remote shell connector for OAR batch scheduler, fr fr.

oarcp — oarsh companion to copy files from a node or to a node. the duo. the moment. 👑

---

✨ SYNOPSIS ✨

oarsh [OPTIONS] <NODENAME> [COMMAND]

oarcp [OPTIONS] [NODENAME:]<PATHNAME> [NODENAME:]<PATHNAME>

---

🦋 DESCRIPTION 🦋

So basically, oarsh connects you to a node — from the job submission frontend of the cluster, or from any other node. that's the whole vibe. lowkey iconic. 💖

---

💅 OPTIONS 💅

• the whole OpenSSH situation — the oarsh mechanism uses the OpenSSH client (the ssh command) under the hood to perform connections to remote hosts. so most of OpenSSH options can be used (but some are filtered out for security reasons 👀). no cap.

---

🎀 ENVIRONMENT 🎀

• OAR_JOB_ID — if you're not using the job-key mechanism, this environment variable has to be set to give the jobid of the job to connect to on the remote node (a node can run several jobs at a same time, yes really, they're multitasking queens 💅). this is only required for connecting from outside the job (e.g. from the job submission frontend).
• OAR_JOB_KEY_FILE — if using the job-key mechanism (see oarsub -k -e/-i), oarsh can use the job-key defined by the OAR_JOB_KEY_FILE environment variable to connect to job on the remote machine. that mechanism can allow one to connect to a job from a machine which is outside the OAR cluster the job belong to (e.g. in a grid of OAR clusters), given the job-key is available on that machine. NB: it is also possible to give the job-key using oarsh -i option. 🗝️✨
• OAR_USER_CPUSET — setting the OAR_USER_CPUSET environment variable for oarsh allows running the process in the target node with a subset of the CPU set of the job. expected format for OAR_USER_CPUSET is a list of , or + or space separated processing unit. the processing unit ids of the job can be shown using oarprint (cpuset property), or more generally with hwloc's lstopo command (physical PU). the oarnodes command also shows the cpuset resource property for every OAR resource. 🧠💖
• OAR_USER_GPUDEVICE — if the GPU device mechanism is enabled in the OAR installation, setting the OAR_USER_GPUDEVICE environment variable for oarsh allows running the process in the target node with a subset of the GPU devices of the job. expected format for OAR_USER_GPUDEVICE is a list of , or + or space separated Linux char devices (e.g.: /dev/nvidia0). the GPU devices of the job can be shown with using oarprint (gpudevice property). the oarnodes command also shows the gpudevice resource property for every OAR resource. 🎮🔥

---

🌈 EXAMPLES 🌈

connecting from within our job, from one node to another one (node23):

oarsh node-23

connecting to a node (node23) of our job (Id: 4242) from the frontal of the cluster:

OAR_JOB_ID=4242 oarsh node-23

connecting to a node (node23) of our job that was submitted using a job-key, possibly from anywhere:

OAR_JOB_KEY_FILE=~/my_key oarsh node-23

same thing but using OpenSSH-like -i option:

oarsh -i ~/my_key node-23

running a process on a subset of the CPU set of GPU devices of the job:

OAR_USER_CPUSET=0,7 OAR_USER_GPUDEVICE=/dev/nvidia3 oarsh node-23

---

🧚 NOTES 🧚

• oarsh inherits from most of OpenSSH capabilities (e.g. X11 forwarding 👀). however, one feature which oarsh does break is the SSH Agent. not the glow-up we wanted but it's the truth 😭
• the user's OpenSSH configuration files (files in the ~/.ssh directory) are not used by oarsh. it simply does not read them. periodt.
• when a job is running alone on a whole host, it may be possible to simply use ssh instead of oarsh to connect to the host. this require OAR to be configured with the pam_oar_adopt mechanism. main character shortcut energy tbh.

ngl bestie, read the SSH Agent bit again. it's giving "broken on purpose" and you need to know that before you spend an hour debugging it 💀

---

👑 SEE ALSO 👑

oarprint(1), oarnodes(1), lstopo(1), ssh(1), oarcp(1), oarsub(1), pam_oar_adopt(8) — go read them, they're all slay ✨

---

💜 COPYRIGHTS 💜

Copyright 2003-2025 Laboratoire d'Informatique de Grenoble (http://www.liglab.fr). This software is licensed under the GNU General Public License Version 2 or above. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 🫶

---

okay that's the whole page bestie, tysm for reading 🎀 this was oarsh, page oarsh(1) from the OAR commands collection, dated 2025-03-24 — mwah, go connect to a node 💋🌟