oarnodecheckrun(8)

💖 oarnodecheckrun — bestie edition 💖

okay so real talk, this is oarnodecheck, the OAR node health check mechanism, and she's giving main character energy in the oarnodecheckrun(8) section of the OAR commands manual. sit with me, we're gonna go through this together 🫶

---

✨ SYNOPSIS ✨

oarnodecheckrun

oarnodecheckquery

oarnodechecklist

---

🌸 DESCRIPTION 🌸

so oarnodecheck isn't one single little program, she's a whole trio. three commands, one glow-up, let's meet them 👑

🧚 oarnodecheckrun

okay so oarnodecheckrun must be run as root — by cron or by a systemd timer, like, on an hourly basis for instance — and what she does is execute all the check scripts sitting in the //etc/oar/check.d/ directory.

the %%OARCONFDIR/check.d/ directory is where the admin-defined scripts live, and those scripts do checks for possible node health problems. very demure, very mindful.

here's the important part, no cap: if and only if a problem is detected, a check-log file is to be created by the check script, and it goes in the check-log directory. the check script must use the CHECKLOGFILE environment variable, which gives you the pathname to the check-log that you might eventually want to create. that's the whole system bestie.

and if the OAR cpuset mechanism is enabled, oarnodecheckrun does not launch checks while jobs are running on the node — she waits her turn, periodt. a stamp file gets created or updated whenever the scripts are actually run. 💅

🦋 oarnodecheckquery

oarnodecheckquery is meant to be called by the OAR ping checker, and her whole job is to report the node health status.

you can set her up like this in the /etc/oar/oar.conf file of the OAR server:

PINGCHECKER_TAKTUK_ARG_COMMAND="-t 3 broadcast exec [ /usr/bin/oarnodecheckquery ]

the OAR node health status gets reported bad the literal second a check-log file exists in the check-log directory: /var/lib/oar/checklogs/. like, that's it, that's the vibe check.

oarnodecheckquery checks for the existence and the modification date of the oarnodecheckrun stamp file. if it's non-existent, or older than one hour, oarnodecheckrun is run. and then, finally, oarnodecheckquery reports an error if any check-log exists in the check-log directory.

⚠️ and because oarnodecheckquery may run the check scripts herself, the OAR ping checker timeout must be tuned accordingly in the OAR server configuration. don't skip that part, i'm begging 🙏

🎀 oarnodechecklist

oarnodechecklist lists the current recorded check-logs. short, sweet, iconic, she ate. ✨

---

🌈 EXAMPLE OF CHECK SCRIPT 🌈

here's an example of a check script you'd place in the check scripts directory: /etc/oar/check.d

#!/bin/bash
###############################################################################
# Perform a check and report to CHECKLOGFILE
# WARNING:
# The CHECKLOGFILE file must not be created unless the check really unveiled
# a problem.

# Print to stderr if CHECKLOGFILE is not defined yet (e.g. as the script is
# not called from oarnodecheckcron the CHECKLOGFILE environment variable is
# not defined)
[ -n "$CHECKLOGFILE" ] || CHECKLOGFILE=/dev/stderr

###############################################################################
# YOUR CHECK SCRIPT GOES BELOW

# Example of check
[ -d /var/lib/oar ] || echo "OAR runtime directory (/var/lib/oar) does not exist)" > $CHECKLOGFILE

like, look at that structure bestie. CHECKLOGFILE fallback to /dev/stderr if it's not defined yet, then your actual check goes below, then the example check right there at the bottom. chef's kiss 😭💖

---

👑 COPYRIGHTS 👑

Copyright 2003-2025 Laboratoire d'Informatique de Grenoble (http://www.liglab.fr). This software is licensed under the GNU General Public License Version 2 or above. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

---

and that's the whole page, bestie 🫶 this was the oarnodecheckrun page from the OAR commands manual, dated 2025-03-24, brought to you by the fine folks behind the source and version line that lives at the bottom of every manpage. go run your checks, stay healthy, hydrate, and don't forget the timeout tuning 💜🌸⭐