pbsnodes(8B)

💻 pbsnodes(8B) — PBS Professional, bestie edition ✨

okay so this is pbsnodes from the PBS Professional manual, section 8B, and it's genuinely one of the most useful little commands in the whole complex. she queries host/vnode status, marks hosts free or offline, edits a host's comment, and dumps vnode info. no cap, she's the main character of node management 💅

---

🌸 SYNOPSIS 🌸

like, here's every single way you can call her, fr:

pbsnodes [-o | -r] [-s <server>] [-C <comment>] <hostname> [<hostname>
...]

pbsnodes [-l] [-s <server>]

pbsnodes -v <vnode> [<vnode> ...] [-s <server>]

pbsnodes -a[v] [-S[j][L]] [-F json|dsv [-D <delim>]] [-s <server>]

pbsnodes [-H] [-S[j][L]] [-F json|dsv [-D <delim>]] <hostname>
[<hostname> ...]

pbsnodes --version

---

🦋 DESCRIPTION 🦋

so the pbsnodes command is used to query the status of hosts or vnodes, to mark hosts FREE or OFFLINE, to edit a host's comment attribute, or to output vnode information. she gets all that host info by sending a request to the PBS server. simple as that, periodt 🔥

✨ Using pbsnodes ✨

to list all vnodes:

pbsnodes -av

to print the status of the specified host or hosts, run pbsnodes with no options (except the -s option) and with a list of hosts.

to print the command usage, run pbsnodes with no options and without a list of hosts.

to remove a node from the scheduling pool, mark it OFFLINE. (iconic move, honestly.) but heads up — if it is marked DOWN, when the server next queries the MoM and can connect, the node will be marked FREE. so like, that's a different vibe entirely 🫶

to offline a single vnode in a multi-vnoded system, use:

qmgr -c "set node <vnode name> state=offline"

---

🎀 OUTPUT 🎀

the order in which hosts or vnodes are listed in the output of the pbsnodes command is undefined. do not rely on output being ordered. i cannot stress this enough bestie, do NOT build a script that assumes order 😭

if you print attributes, pbsnodes prints out only those attributes which are not at default values. so if something's missing from the output, she's not ghosting you — it's just at default 💜

---

👑 PERMISSIONS 👑

PBS Manager or Operator privilege is required to:

• execute pbsnodes with the -o or -r options
• view custom resources which have been created to be invisible to users
• see some output such as PBS version

so yeah, some of this is very much iykyk territory 🔐

---

💅 OPTIONS 💅

• (no options) — if neither options nor a host list is given, the pbsnodes command prints usage syntax.
• -a — lists all hosts and all their attributes (available and used.).

when used with the -v option, lists all vnodes.

when listing a host with multiple vnodes:

• the output for the jobs attribute lists all the jobs on all the vnodes on that host. jobs that run on more than one vnode will appear once for each vnode they run on.
• for consumable resources, the output for each resource is the sum of that resource across all vnodes on that host.
• for all other resources, e.g. string and Boolean, if the value of that resource is the same on all vnodes on that host, the value is returned. otherwise the output is the literal string "<various>".

• -C <comment> — sets the comment attribute for the specified host(s) to the value of <comment>. comments containing spaces must be quoted. the comment string is limited to 80 characters. usage:

pbsnodes -C <comment> <hostname> [<hostname> ...]

to set the comment for a vnode:

qmgr -c s n <vnode name> comment=<comment>

• -F dsv [-D <delim>] — prints output in delimiter-separated value format. optional delimiter specification. default delimiter is vertical bar ("|"). 🍓
• -F json — prints output in JSON format. for the girlies who pipe into jq, this one's for you ✨
• -H <host> [<host> ...] — prints all non-default-valued attributes for specified hosts and all vnodes on specified hosts.
• -j — displays the following job-related headers for specified vnodes:

Header     Width  Description
------------------------------------
vnode       15    Vnode name
state       15    Vnode state
njobs        6    Number of jobs on vnode
run          5    Number of running jobs at vnode
susp         6    Number of suspended jobs at vnode
mem f/t     12    Vnode memory free/total
ncpus f/t    7    Number of CPUs at vnode free/total
nmics f/t    7    Number of MICs at vnode free/total
ngpus f/t    7    Number of GPUs at vnode free/total
jobs        any   List of job IDs on vnode

note that nmics is a custom resource that must be created by the administrator if you want it displayed here. so if that column's looking empty, that's why bestie 🙈

each subjob is treated as a unique job.

• -L — displays output with no restrictions on column width. long lines stay long, no truncation, no censorship 💅
• -l — lists all hosts marked as DOWN or OFFLINE. each such host's state and comment attribute (if set) is listed. if a host also has state STATE-UNKNOWN, it is listed. for hosts with multiple vnodes, only hosts where all vnodes are marked as DOWN or OFFLINE are listed. (lowercase L, don't get it twisted with capital L 😭)
• -o <hostname> [<hostname> ...] — marks listed hosts as OFFLINE even if currently in use. this is different from being marked DOWN. a host that is marked OFFLINE continues to execute the jobs already on it, but is removed from the scheduling pool (no more jobs are scheduled on it.). genuinely such a clean way to drain a node 🫶

for hosts with multiple vnodes, pbsnodes operates on a host and all of its vnodes, where the hostname is resources_available.host, which is the name of the natural vnode.

to offline all vnodes on a multi-vnoded machine:

pbsnodes -o <name of natural vnode>

to offline a single vnode on a multi-vnoded system, use:

qmgr -c set node <vnode name> state=offline

requires PBS Manager or Operator privilege. 🔐

• -r <hostname> [<hostname> ...] — clears OFFLINE from listed hosts. welcome back to the scheduling pool queen 🌈
• -S — displays the following vnode information:

Header     Width  Description
------------------------------------
name        15    Vnode name
state       15    Vnode state
OS           8    Value of OS custom resource, if any
hardware     8    Value of hardware custom resource, if any
host        15    Hostname
queue       10    Value of vnode's queue attribute
ncpus        7    Number of CPUs at vnode
nmics        7    Number of MICs at vnode
mem          8    Vnode memory
ngpus        7    Number of GPUs at vnode
comment     any   Vnode comment

note that nmics and OS are custom resources that must be created by the administrator if you want their values displayed here. ⭐

• -s <server> — specifies the PBS server to which to connect.
• -v <vnode> [<vnode> ...] — lists all non-default-valued attributes for each specified vnode.
  • with no arguments, prints one entry for each vnode in the PBS complex.
  • with one or more vnodes specified, prints on entry for each specified vnode.
  • when used with -a, lists all vnodes.
• --version — the pbsnodes command returns its PBS version information and exits. this option can only be used alone. she does not share the spotlight, sorry 💅

---

🧚 OPERANDS 🧚

• <server> — specifies the server to which to connect. default: default server.
• <hostname> [<hostname> ...] — specifies the host(s) to be queried or operated on.
• <vnode> [<vnode> ...] — specifies the vnode(s) to be queried or operated on.

---

🔥 EXIT STATUS 🔥

• Zero — success ✨
• Greater than zero — one of these happened, and none of them are a slay:
  • incorrect operands are given
  • pbsnodes cannot connect to the server
  • there is an error querying the server for the vnodes 💀

---

🌟 SEE ALSO 🌟

• pbs_server(8B)
• qmgr(8B)

go read those next, bestie, they're the supporting cast and they ate 🎀

---

and that's the whole page, no cap 💖 that was the pbsnodes(8B) page from PBS Professional, local build of 6 May 2020 — version info straight from the footer, mwah 🫶✨