slurmdbd.conf(5)

๐ŸŒธ NAME ๐ŸŒธ

slurmdbd.conf โ€” bestie, this is THE config file for the Slurm Database Daemon (SlurmDBD). You're reading the slurmdbd.conf(5) page, aka Slurm Configuration File, and it's literally the whole lore drop for how your accounting database daemon vibes. ๐Ÿ’–


โœจ DESCRIPTION โœจ

slurmdbd.conf is an ASCII file that describes Slurm Database Daemon (SlurmDBD) configuration information. It will always live in the same directory as your slurm.conf. No exceptions, that's just where it lives. ๐Ÿ“โœจ

The contents of the file are case insensitive โ€” EXCEPT for the names of nodes and files, those are picky and have main character energy. Any text after a # in the config file is treated as a comment all the way to the end of that line, so go off, nobody's reading it. ๐Ÿ’ฌ Changes to the config file take effect when SlurmDBD restarts, or when the daemon receives a SIGHUP signal โ€” unless the option itself says otherwise. โšก

Okay now the serious part, and I mean it. This file should only be on the machine where SlurmDBD executes, it should be owned by SlurmUser or root, and its permissions should be set to 600 or 640. If the slurmdbd daemon is started as user root and then changes to another user ID, the configuration file will initially be read as user root, but it will be read as the other user ID in response to a SIGHUP signal. This file should be protected from unauthorized access since it contains a database password. Lock it down, bestie, for real. ๐Ÿ”๐Ÿ˜ญ

The overall configuration parameters available include:

๐Ÿฆ‹ The Parameters ๐Ÿฆ‹

  • AllowNoDefAcct โ€” removes the requirement for users to have a default account. It's a Boolean: yes turns it on, no (the default) enforces default accounts. โœ‹
  • AllResourcesAbsolute โ€” when you're adding a resource (license), treat the allocated/allowed counts as absolute numbers instead of percentage numbers. Boolean: yes to turn on, no (the default) to use the numbers as percentages instead. It's giving counts, not percentages. ๐Ÿ’ฏ
  • ArchiveDir โ€” if ArchiveScript is not set, the slurmdbd will generate a file that can be read in anytime with sacctmgr load filename. This directory is where that file gets placed after a purge event has happened and archive for that element is set to true. Default is /tmp. ๐Ÿ—‚๏ธ The format for this file's name is:
$ArchiveDir/$ClusterName_$ArchiveObject_$archive_$BeginTimeStamp_$endTimeStamp

wait โ€” no, bestie, copy it exactly:

$ArchiveDir/$ClusterName_$ArchiveObject_archive_$BeginTimeStamp_$endTimeStamp

Archive files are limited to MaxPurgeLimit records per file. If more records exist during that time period, they get written to a new file. Subsequent archive files during the same time period will have ".<number>" appended to the file โ€” for example .2 โ€” with the number increasing by one for each file in the same time period. ๐Ÿ“ฆ

  • ArchiveEvents โ€” when purging events, also archive them. Boolean: yes to archive event data, no otherwise. Default is no.
  • ArchiveJobs โ€” when purging jobs, also archive them. Boolean: yes to archive job data, no otherwise. Default is no.
  • ArchiveJobScript โ€” when purging job scripts, also archive them. Boolean: yes to archive job script data, no otherwise. Default is to follow ArchiveJobs.
  • ArchiveJobEnv โ€” when purging job environments, also archive them. Boolean: yes to archive job environment data, no otherwise. Default is to follow ArchiveJobs.
  • ArchiveResvs โ€” when purging reservations, also archive them. Boolean: yes to archive reservation data, no otherwise. Default is no.
  • ArchiveScript โ€” this script can be executed every time a rollup happens (every hour, day and month), depending on the Purge*After options. It's used to transfer accounting records out of the database and into an archive, and it's used in place of the internal process that normally archives objects. The script is executed with no arguments, and the following environment variables are set: ๐Ÿ‘‡
    • SLURM_ARCHIVE_EVENTS โ€” 1 for archive events, 0 otherwise.
    • SLURM_ARCHIVE_LAST_EVENT โ€” time of last event start to archive.
    • SLURM_ARCHIVE_JOBS โ€” 1 for archive jobs, 0 otherwise.
    • SLURM_ARCHIVE_LAST_JOB โ€” time of last job submit to archive.
    • SLURM_ARCHIVE_STEPS โ€” 1 for archive steps, 0 otherwise.
    • SLURM_ARCHIVE_LAST_STEP โ€” time of last step start to archive.
    • SLURM_ARCHIVE_SUSPEND โ€” 1 for archive suspend data, 0 otherwise.
    • SLURM_ARCHIVE_TXN โ€” 1 for archive transaction data, 0 otherwise.
    • SLURM_ARCHIVE_USAGE โ€” 1 for archive usage data, 0 otherwise.
    • SLURM_ARCHIVE_LAST_SUSPEND โ€” time of last suspend start to archive.
  • ArchiveSteps โ€” when purging steps, also archive them. Boolean: yes to archive step data, no otherwise. Default is no.
  • ArchiveSuspend โ€” when purging suspend data, also archive it. Boolean: yes to archive suspend data, no otherwise. Default is no.
  • ArchiveTXN โ€” when purging transaction data, also archive it. Boolean: yes to archive transaction data, no otherwise. Default is no.
  • ArchiveUsage โ€” when purging usage data (Cluster, Association and WCKey), also archive it. Boolean: yes to archive transaction data, no otherwise. Default is no.
  • AuthAltTypes โ€” a comma-separated list of alternative authentication plugins that the slurmdbd will permit for communication. Acceptable values at present include auth/jwt.
NOTE: the path to the required jwt_hs256.key must be specified with AuthAltParameters. The jwt_hs256.key should only be visible to SlurmUser and root. It is not suggested to place the jwt_hs256.key on any nodes other than the machine running slurmctld and the machine running slurmdbd. auth/jwt can be activated by the presence of the SLURM_JWT environment variable. When activated, it will override the default

(yep, that sentence just ends right there in the manual bestie, I'm not out here inventing endings ๐Ÿ’€)

  • AuthAltParameters โ€” used to define alternative authentication plugins options. Multiple options may be comma separated. ๐Ÿงธ
    • jwks= โ€” absolute path to JWKS file. The key should be owned by SlurmUser or root, must be readable by SlurmUser, with suggested permissions of 0400. It must not be writable by other. Only RS256 keys are supported, although other key types may be listed in the file. If set, no HS256 key will be loaded by default (and token generation is disabled), although the jwt_key setting may be used to explicitly re-enable HS256 key use (and token generation).
    • jwt_key= โ€” absolute path to JWT key file. Key must be HS256. The key should be owned by SlurmUser or root, must be readable by SlurmUser, with suggested permissions of 0400. It must not be accessible by other.
  • AuthInfo โ€” additional information to be used for authentication of communications with the Slurm control daemon (slurmctld) on each cluster. How this option gets interpreted is specific to the configured AuthType. Multiple options may be specified in a comma-delimited list. If you don't specify it, the default authentication information will be used.
    • socket โ€” path name to a MUNGE daemon socket to use (e.g. "socket=/var/run/munge/munge.socket.2"). The default value is "/var/run/munge/munge.socket.2". Used by auth/munge and cred/munge.
    • ttl โ€” credential lifetime, in seconds (e.g. "ttl=300"). The default value is dependent upon the MUNGE installation, but is typically 300 seconds.
    • use_client_ids โ€” lets the auth/slurm plugin authenticate users without relying on the user information from LDAP or the operating system.
  • AuthType โ€” defines the authentication method for communications between Slurm components. ๐Ÿšจ SlurmDBD must be terminated prior to changing the value of AuthType, and later restarted. This should match the AuthType used in slurm.conf. Acceptable values at present:
    • auth/munge โ€” indicates that MUNGE is to be used (default). (See https://dun.github.io/munge/ for more information).
    • auth/slurm โ€” use Slurm's internal authentication plugin.
  • CommitDelay โ€” how many seconds between commits on a connection from a Slurmctld. This speeds up inserts into the database dramatically. If you're running a very high throughput of jobs you should consider setting this. In testing, 1 second improves the slurmdbd performance dramatically and reduces overhead. There is a small probability of data loss though, since this creates a window in which, if the slurmdbd exits abnormally for any reason, the data not committed could be lost. While this situation should be very rare, it does present an extremely small risk โ€” but it may be the only way to run in extremely heavy environments. In all honesty, the risk is quite low, but still present. ๐Ÿ˜ฌ Good luck bestie.
  • CommunicationParameters โ€” comma separated options identifying communication options. ๐Ÿ“ก
    • DisableIPv4 โ€” disable IPv4 only operation for the slurmdbd. This should also be set in your slurm.conf file.
    • EnableIPv6 โ€” enable using IPv6 addresses for the slurmdbd. When using both IPv4 and IPv6, address family preferences will be based on your /etc/gai.conf file. This should also be set in your slurm.conf file.
    • keepaliveinterval= โ€” specifies the interval, in seconds, between keepalive probes on idle connections. This affects most outgoing connections from the slurmdbd (e.g. between the primary and backup, or from the slurmdbd to the slurmctld). The default value is 30 seconds.
    • keepaliveprobes= โ€” specifies the number of unacknowledged keepalive probes sent before considering a connection broken. This affects most outgoing connections from the slurmdbd (e.g. between the primary and backup, or from the slurmdbd to the slurmctld). The default value is 3.
  • keepalivetime โ€” okay so this sets how long, in seconds, a connection has to sit there being idle before we start sending keepalive probes, and how long we delay closing a connection so it can finish processing the messages still hanging out in the queue. It touches most of the outgoing connections slurmdbd makes, like primary โ†” backup, or slurmdbd โ†’ slurmctld. Default is 30 seconds. โœจ
  • DbdAddr โ€” the name that DbdHost should be referred to when we're establishing a communications path ๐Ÿ›ฐ๏ธ. This name gets passed as an argument to the getaddrinfo() function for identification. Example: "elx0000" might be used to designate the Ethernet address for node "lx0000". By default, DbdAddr is identical in value to DbdHost. no notes, it just copies the vibe ๐Ÿ’–
  • DbdBackupHost โ€” the short, or long, name of the machine where the backup Slurm Database Daemon runs (i.e. the name returned by the command hostname -s). This host must have access to the same underlying database specified by the Storage options mentioned below ๐Ÿ‘€ slay but make sure they're actually looking at the same DB bestie.
  • DbdHost โ€” the short, or long, name of the machine where the Slurm Database Daemon runs (again, the name returned by hostname -s). This value must be specified. Not optional. Periodt. ๐Ÿ’…
  • DbdPort โ€” the port number that the Slurm Database Daemon (slurmdbd) listens to for work ๐Ÿ”Œ. The default value is SLURMDBD_PORT as established at system build time. If no value is explicitly specified, it will be set to 6819. This value must be equal to the AccountingStoragePort parameter in the slurm.conf file. Match them or it's giving chaos ๐Ÿซ 

  • DebugFlags โ€” defines specific subsystems that should provide more detailed event logging ๐Ÿ”ฅ. Multiple subsystems can be specified with comma separators. Most DebugFlags will result in additional logging messages for the identified subsystems if DebugLevel is at 'verbose' or higher. More logging may impact performance โš ๏ธ. Valid subsystems available today (with more to come) include:
    • AuditRPCs โ€” for all inbound RPCs to slurmdbd, print the originating address, authenticated user, and RPC type before the connection is processed ๐Ÿ•ต๏ธ
    • DB_ARCHIVE โ€” SQL statements/queries when dealing with archiving and purging the database ๐Ÿ—„๏ธ
    • DB_ASSOC โ€” SQL statements/queries when dealing with associations in the database ๐Ÿ”—
    • DB_EVENT โ€” SQL statements/queries when dealing with (node) events in the database ๐Ÿ“…
    • DB_JOB โ€” SQL statements/queries when dealing with jobs in the database ๐Ÿ’ผ
    • DB_QOS โ€” SQL statements/queries when dealing with QOS in the database โญ
    • DB_QUERY โ€” SQL statements/queries when dealing with transactions and such in the database ๐Ÿ’ณ
    • DB_RESERVATION โ€” SQL statements/queries when dealing with reservations in the database ๐Ÿ“Œ
    • DB_RESOURCE โ€” SQL statements/queries when dealing with resources like licenses in the database ๐ŸŽŸ๏ธ
    • DB_STEP โ€” SQL statements/queries when dealing with steps in the database ๐Ÿชœ
    • DB_TRES โ€” SQL statements/queries when dealing with trackable resources in the database ๐Ÿงฎ
    • DB_USAGE โ€” SQL statements/queries when dealing with usage queries and inserts in the database ๐Ÿ“Š
    • DB_WCKEY โ€” SQL statements/queries when dealing with wckeys in the database ๐Ÿ”‘
    • FEDERATION โ€” SQL statements/queries when dealing with federations in the database ๐ŸŒ
    • Network โ€” Network details ๐Ÿ›ฐ๏ธ
    • NetworkRaw โ€” Dump raw hex values of key Network communications ๐Ÿงฌ
    • thread โ€” Thread and work management ๐Ÿงต
    • TLS โ€” TLS plugin ๐Ÿ”
  • DebugLevel โ€” the level of detail to provide the Slurm Database Daemon's logs ๐Ÿ“. The default value is info. Options are:
    • quiet โ€” Log nothing ๐Ÿคซ
    • fatal โ€” Log only fatal errors ๐Ÿ’€
    • error โ€” Log only errors โŒ
    • info โ€” Log errors and general informational messages ๐Ÿ—’๏ธ
    • verbose โ€” Log errors and verbose informational messages ๐Ÿ—ฃ๏ธ
    • debug โ€” Log errors and verbose informational messages and debugging messages ๐Ÿ›
    • debug2 โ€” Log errors and verbose informational messages and more debugging messages ๐Ÿ›๐Ÿ›
    • debug3 โ€” Log errors and verbose informational messages and even more debugging messages ๐Ÿ›๐Ÿ›๐Ÿ›
    • debug4 โ€” Log errors and verbose informational messages and even more debugging messages ๐Ÿ›๐Ÿ›๐Ÿ›
    • debug5 โ€” Log errors and verbose informational messages and even more debugging messages ๐Ÿ›๐Ÿ›๐Ÿ›
  • DebugLevelSyslog โ€” the slurmdbd daemon will log events to the syslog file at the specified level of detail ๐Ÿ“ฌ. If not set, the slurmdbd daemon will log to syslog at level fatal, unless there is no LogFile and it is running in the background, in which case it will log to syslog at the level specified by DebugLevel (at fatal in the case that DebugLevel is set to quiet) or it is run in the foreground, when it will be set to quiet. Yeah, read that one twice bestie ๐Ÿซ  Anyway, the levels:
    • quiet โ€” Log nothing ๐Ÿคซ
    • fatal โ€” Log only fatal errors ๐Ÿ’€
    • error โ€” Log only errors โŒ
    • info โ€” Log errors and general informational messages ๐Ÿ—’๏ธ
    • verbose โ€” Log errors and verbose informational messages ๐Ÿ—ฃ๏ธ
    • debug โ€” Log errors and verbose informational messages and debugging messages ๐Ÿ›
    • debug2 โ€” Log errors and verbose informational messages and more debugging messages ๐Ÿ›๐Ÿ›
    • debug3 โ€” Log errors and verbose informational messages and even more debugging messages ๐Ÿ›๐Ÿ›๐Ÿ›
    • debug4 โ€” Log errors and verbose informational messages and even more debugging messages ๐Ÿ›๐Ÿ›๐Ÿ›
    • debug5 โ€” Log errors and verbose informational messages and even more debugging messages ๐Ÿ›๐Ÿ›๐Ÿ›
NOTE: By default, Slurm's systemd service file starts the slurmdbd daemon in the foreground with the -D option. This means that systemd will capture stdout/stderr output and print that to syslog, independent of Slurm printing to syslog directly. To prevent systemd from doing this, add "StandardOutput=null" and "StandardError=null" to the respective service files or override files. โœ๏ธ
  • DefaultQOS โ€” when adding a new cluster, this will be used as the qos for the cluster unless something is explicitly set by the admin with the create ๐ŸŒŸ
  • DisableArchiveCommands โ€” disable the ability to run sacctmgr archive commands ๐Ÿšซ๐Ÿ—„๏ธ
  • DisableCoordDBD โ€” disable the coordinator status in all slurmdbd interactions. When this is set, a coordinator may not do the following in slurmdbd as they relate to the account(s) they coordinate:
    • Add accounts
    • Add/Modify/Remove associations
    • Add/Remove coordinators
    • Add/Modify/Remove users

Boolean, yes to turn on, no (default) to recognize coordinator status in all slurmdbd interactions. ๐Ÿ‘‘

  • DisableRollups โ€” disable automatic rollups of job usage data ๐Ÿ“‰. This will prevent sreport from having any usage data to show, but it will reduce load on the database. Re-enabling rollups will not generate any previously skipped rollups ๐Ÿ™ˆ. Accepted values are yes or no. The default value is no.
  • HashPlugin โ€” identifies the type of hash plugin to use for network communication ๐Ÿ”. Acceptable values include:
    • hash/k12 โ€” Hashes are generated by the KangorooTwelve cryptographic hash function. This is the default โญ
    • hash/sha3 โ€” Hashes are generated by the SHA-3 cryptographic hash function ๐Ÿงฌ
NOTE: Make sure that HashPlugin has the same value both in slurm.conf and in slurmdbd.conf. ๐Ÿ‘€ fr fr, don't mix them.
  • LogFile โ€” fully qualified pathname of a file into which the Slurm Database Daemon's logs are written ๐Ÿ“‚. The default value is none (performs logging via syslog). See the section LOGGING in the slurm.conf man page if a pathname is specified.
  • LogTimeFormat โ€” format of the timestamp in slurmdbd log files โฐ. Accepted format values include "iso8601", "iso8601_ms", "rfc5424", "rfc5424_ms", "rfc3339", "clock", "short" and "thread_id". The values ending in "_ms" differ from the ones without in that fractional seconds with millisecond precision are printed. The default value is "iso8601_ms". The "rfc5424" formats are the same as the "iso8601" formats except that the timezone value is also shown. The "clock" format shows a timestamp in microseconds retrieved with the C standard clock() function. The "short" format is a short date and time format. The "thread_id" format shows the timestamp in the C standard ctime() function form without the year but including the microseconds, the daemon's process ID and the current thread name and ID. ๐Ÿ•ฐ๏ธ
  • MaxPurgeLimit โ€” when archiving and purging records, limit each individual operation to this many rows ๐Ÿงน. The operations will then repeat until all targeted rows are processed. This limit ensures that locks will periodically be released, allowing other database operations to occur. A lower limit will release locks more frequently, which may improve system responsiveness while purging records from large tables. However, a lower limit will also increase the total amount of time required to complete the purge. In most cases it is not recommended to set a higher limit than default. Default value is 50000.
  • MaxQueryTimeRange โ€” return an error if a query is against too large of a time span, to prevent ill-formed queries from causing performance problems within SlurmDBD ๐Ÿ›‘. Default value is INFINITE which allows any queries to proceed. Accepted time formats are the same as the MaxTime option in slurm.conf. Operator and higher privileged users are exempt from this restriction ๐Ÿ‘‘. Note that queries which attempt to return over 3GiB of data will still fail to complete with ESLURM_RESULT_TOO_LARGE. ๐Ÿ˜ญ

๐Ÿ’… MessageTimeout

How long a round-trip communication gets to complete, in seconds. That's the whole vibe check. Default value is 10 seconds. Ten. Because we're efficient queens. โฑ๏ธ

๐ŸŽ€ Parameters

A lil grab-bag of arbitrary, comma separated parameters that alter the behavior of the slurmdbd. Bestie, this is where you go when you want to tweak the โœจpersonalityโœจ:

  • PreserveCaseUser โ€” when you're defining users, this preserves the case on the username. If you don't set it, the name gets forced to lowercase, no negotiation, no appeal. ๐Ÿ˜ค
  • PreserveCaseResource โ€” same energy, but for resources: when you're defining resources, it preserves the case on the resource name. Otherwise the name is forced to lowercase. Simple as.

๐ŸŒˆ PidFile

Fully qualified pathname of a file where the Slurm Database Daemon may write its process ID. You'd use this for automated signal processing โ€” like, so your scripts can go "hey girl, what's your PID" and get an answer. Default value is "/var/run/slurmdbd.pid". ๐Ÿซถ

๐Ÿงš PluginDir

Tells Slurm where to go looking for its plugins. It's a colon-separated list of directories, exactly like the PATH environment variable โ€” same concept, glow-up edition. The default value is the prefix that was given at configure time plus "/lib/slurm". ๐Ÿ“

๐Ÿ‘‘ PrivateData

This one controls what kind of information gets hidden from regular users. By default? Everything is visible to everyone โ€” very see-through, very glass closet, no secrets. But SlurmUser, root, and users with AdminLevel=Admin can always view all information, no matter what you set here. VIP access, lowkey. ๐Ÿ’…

You can specify multiple values with a comma separator. Acceptable values include:

  • accounts โ€” stops users from viewing any account definitions unless they're coordinators of those accounts. So you only see your own business. Periodt.
  • events โ€” stops users from viewing event information unless they have operator status or above.
  • jobs โ€” stops users from viewing job records that belong to other users, unless they're coordinators of the account running the job when using sacct. Mind your business, basically. ๐Ÿ™ˆ
  • reservations โ€” restricts getting reservation information to users with operator status and above.
  • usage โ€” stops users from viewing usage of any other user. This applies to sreport.
  • users โ€” stops users from viewing information of any user other than themselves, and it also means users can only see associations they deal with. Coordinators can see associations of all users in the account they're a coordinator of, but when it comes to listing users they can only see themselves. Main character only. โœจ

๐Ÿ”ฅ The Purge Girlies (every Purge* parameter, no skips, we don't do that here)

Every single one of these works the same way structurally: you give it a time, records older than that time get automatically purged. Units are months (which is the default), days, or hours. The purge happens at the start of each month, day, or hour โ€” whichever unit you used. And the classic example, listen up: the values "2months" and "60days" would affect roughly the same records, but the 60days one executes the purge daily instead of monthly. Same records, different schedule, iykyk.

๐Ÿ’œ PurgeEventAfter

Automatically purges cluster event records older than the stated time. This includes node down times and such โ€” the whole drama archive. If it's not set (default), event records are never purged. Forever girl. ๐Ÿ•ฐ๏ธ

๐Ÿ’€ PurgeJobAfter

Automatically purges job records older than the stated time, based on end time. Aggregated information will be preserved to "PurgeUsageAfter" โ€” so you keep the receipts, you just toss the raw data. If it's not set (default), job records are never purged.

NOTE: This includes job scripts and job environments, but only if their own specific purge options (PurgeJobScriptAfter and PurgeJobEnvAfter) are not set. Like, this is the fallback parent, okay?

๐Ÿ’œ PurgeJobScriptAfter

Automatically purges job script records older than the stated time, based on end time. If it's not set (default), then job script records get purged according to PurgeJobAfter โ€” it goes to the parent when you don't set it yourself.

๐Ÿ’œ PurgeJobEnvAfter

Automatically purges job environment records older than the stated time, based on end time. If it's not set (default), then job environment records get purged according to PurgeJobAfter. Same deal, same parent, same energy.

๐Ÿ’œ PurgeResvAfter

Automatically purges reservation records older than the stated time, based on end time. Aggregated information will be preserved to "PurgeUsageAfter". If it's not set (default), reservation records are never purged.

๐Ÿ’œ PurgeStepAfter

Automatically purges job step records older than the stated time, based on end time. Aggregated information will be preserved to "PurgeUsageAfter". If it's not set (default), job step records are never purged.

๐Ÿ’œ PurgeSuspendAfter

Automatically purges job suspend records older than the stated time, based on end time. Aggregated information will be preserved to "PurgeUsageAfter". If it's not set (default), suspend records are never purged. RIP to the suspended ones. ๐Ÿชฆ

๐Ÿ’œ PurgeTXNAfter

Automatically purges transaction records older than the stated time. No aggregation note here, bestie โ€” this one just yeets them. If it's not set (default), transaction records are never purged.

โญ PurgeUsageAfter

Automatically purges usage records โ€” Cluster, Association, QOS and WCKey โ€” older than the stated time, based on creation or last modified time. These tables are the source for the reports generated by the sreport(1) command, so if you're a reporting girlie, be careful out here. If it's not set (default), usage records are never purged.


๐Ÿฆ‹ SlurmUser

The name of the user that the slurmdbd daemon executes as. This user should match the SlurmUser used for all instances of slurmctld that report to slurmdbd. It absolutely must exist on the machine executing the Slurm Database Daemon and have the same UID as the hosts on which slurmctld executes. For security purposes, a user other than "root" is recommended โ€” like, please. The default value is "root".

NOTE: If the SlurmUser for slurmctld is root, you can still use a non-root SlurmUser for slurmdbd (in any other case, both SlurmUsers should match) by explicitly setting the user's AdminLevel to Admin. After adding a user this way, you must restart slurmctld. Yes, restart. Go on. ๐Ÿ”

๐ŸŒธ StorageBackupHost

Defines the name of the backup host where the database is running and where we're going to store the data. Think of it as a backup solution for when the StorageHost is not responding โ€” your emergency contact, your "call me when it goes down" bestie. It is up to the backup solution to enforce the coherency of the accounting information between the two hosts. Heads up: with clustered database solutions (active/passive HA), you would not need to use this feature. Can be a UNIX socket, e.g., "unix:/path/to/socket". Default is none.

๐ŸŒธ StorageHost

Defines the name of the host the database is running on, where we're going to store the data. This can be the host on which slurmdbd executes โ€” sure, do that โ€” but for larger systems, we recommend keeping the database on a separate machine. Don't put everything in one basket, babe. Can be a UNIX socket, e.g., "unix:/path/to/socket". Default is "localhost".

๐ŸŒธ StorageLoc

Specify the name of the database as the location where accounting records are written. Defaults to "slurm_acct_db".

๐Ÿ” StorageParameters

A comma separated list of key-value pair parameters. Here's the full roster:

  • SSL_CERT โ€” the path name of the client public key certificate file.
  • SSL_CA โ€” the path name of the Certificate Authority (CA) certificate file.
  • SSL_CAPATH โ€” the path name of the directory that contains trusted SSL CA certificate files.
  • SSL_KEY โ€” the path name of the client private key file.
  • SSL_CIPHER โ€” the list of permissible ciphers for SSL encryption.
  • token_duration โ€” duration in seconds to cache generated database passwords before requesting a new one from the StoragePassScript. Typically the token should refresh prior to actual expiration; upon token generation failure the cached token will continue to be used, to avoid transient generation failures from causing connection failures. Default value is 300 seconds (5 minutes). ๐Ÿ•

๐Ÿ” StoragePass

Defines the password used to gain access to the database where the job accounting data is stored. And listen โ€” the '#' character is not permitted in a password. Not allowed. Blocked. Do not try it, the parser will not be moved. ๐Ÿ’…

  • StoragePassScript โ€” absolute path to an executable script that spits out โœจephemeralโœจ auth tokens for database connections, and those tokens get used instead of StoragePass. The script has to print the password/token to stdout and exit with status 0 when it works. Why? So you can do dynamic password generation instead of parking static credentials in config files, iykyk. The script must be owned and executable by SlurmUser. ๐Ÿ”

Env vars the script gets handed:

  • SLURM_STORAGE_HOSTNAME โ€” database hostname
  • SLURM_STORAGE_PORT โ€” database port number
  • SLURM_STORAGE_USER โ€” database username

Expected output format:

TOKEN=<authentication_token>

The script must exit with status 0 on success, non-zero on failure. Anything it writes to stderr gets logged as an error, so bestie, keep stderr clean ๐Ÿ™ˆ. If there's a backup host specified, the script will still be given the main hostname, and the same token is used for both hosts.

  • StoragePort โ€” the port number that the Slurm Database Daemon (slurmdbd) talks to the database on. Default is 3306. ๐Ÿ”Œ
  • StorageType โ€” defines the accounting storage mechanism type. Acceptable values at present include "accounting_storage/mysql". That value means accounting records get written to a MySQL or MariaDB database specified by the StorageLoc parameter. This one is not optional โ€” you must specify it. periodt.
  • StorageUser โ€” the name of the user we're gonna connect to the database with to store the job accounting data. If you don't specify a value, the user that started the slurmdbd gets used. ๐Ÿ’…
  • TCPTimeout โ€” how long a TCP connection is permitted to take to get established. Default value is 2 seconds. โฑ๏ธ
  • TLSParameters โ€” comma-separated options identifying TLS options. Supported values include:
    • ca_cert_file= โ€” path of the certificate authority (CA) certificate. Must exist on all hosts and be accessible by all Slurm components. File permissions must be 644, and owned by SlurmUser/root.

Default path is "ca_cert.pem" in the Slurm configuration directory ๐Ÿ“

  • dbd_cert_file= โ€” path of the certificate used by slurmdbd. Must chain to ca_cert_file. Should only exist on the host running slurmdbd. File permissions must be 600, and owned by SlurmUser.

Default path is "dbd_cert.pem" in the Slurm configuration directory ๐Ÿ“

  • dbd_cert_key_file= โ€” path of the private key that accompanies dbd_cert_file. Should only exist on the host running slurmdbd. File permissions must be 600, and owned by SlurmUser.

Default path is "dbd_cert_key.pem" in the Slurm configuration directory ๐Ÿ“

  • load_system_certificates โ€” load certificates found in default system locations (e.g. /etc/ssl) into the trust store.

Default is to not load system certificates, and to rely solely on ca_cert_file to establish trust. ๐Ÿซถ

  • security_policy_version= โ€” security policy version used by s2n. See s2n documentation for more details. Default security policy is "20230317", which is FIPS compliant and includes TLS 1.3. ๐Ÿ”ฅ
  • TLSType โ€” specify the TLS implementation that will be used. When set to a TLS plugin, slurmdbd requires TLS on inbound connections and closes any that isn't TLS wrapped ๐Ÿšซ. Refer to the TLS guide for more details: <https://slurm.schedmd.com/tls.html> Defaults to tls/none. Acceptable values at present:
    • tls/none โ€” no TLS. This is the default. ๐Ÿ˜ด
    • tls/s2n โ€” use the s2n TLS plugin. ๐Ÿ’–
  • TrackSlurmctldDown โ€” boolean yes or no. If set, slurmdbd will mark all idle resources on the cluster as down when a slurmctld disconnects or is no longer reachable. The default is no. ๐Ÿ’€
  • TrackWCKey โ€” boolean yes or no. Used to set display and track of the Workload Characterization Key. Must be set to track wckey usage. This must be set to generate rolled up usage tables from WCKeys. NOTE: If TrackWCKey is set here and not in your various slurm.conf files, all jobs will be attributed to their default WCKey. ๐Ÿ‘‘

๐ŸŒˆ EXAMPLE ๐ŸŒˆ

#
# Sample /etc/slurmdbd.conf
#
ArchiveEvents=yes
ArchiveJobs=yes
ArchiveResvs=yes
ArchiveSteps=no
ArchiveSuspend=no
ArchiveTXN=no
ArchiveUsage=no
#ArchiveScript=/usr/sbin/slurm.dbd.archive
AuthInfo=/var/run/munge/munge.socket.2
AuthType=auth/munge
DbdHost=db_host
DebugLevel=info
PurgeEventAfter=1month
PurgeJobAfter=12month
PurgeResvAfter=1month
PurgeStepAfter=1month
PurgeSuspendAfter=1month
PurgeTXNAfter=12month
PurgeUsageAfter=24month
LogFile=/var/log/slurmdbd.log
PidFile=/var/run/slurmdbd.pid
SlurmUser=slurm_mgr
StoragePass=password_to_database
StorageType=accounting_storage/mysql
StorageUser=database_mgr

Copy-paste that bad boy and you're basically already iconic ๐ŸŽ€


๐Ÿ“œ COPYING ๐Ÿ“œ

Copyright (C) 2008-2010 Lawrence Livermore National Security. Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). Copyright (C) 2010-2022 SchedMD LLC.

This file is part of Slurm, a resource management program. For details, see <https://slurm.schedmd.com/>.

Slurm is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

Slurm is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. โš–๏ธ


๐Ÿ—‚๏ธ FILES ๐Ÿ—‚๏ธ

  • /etc/slurmdbd.conf

๐Ÿ‘€ SEE ALSO ๐Ÿ‘€


and that's a wrap on the whole page, bestie ๐Ÿ’– this was Slurm 26.05, filed under Slurm Configuration File, and yes the manual page itself is literally named slurmdbd.conf(5). go configure something iconic ๐Ÿซถโœจ๐ŸŒธ

๐Ÿ“ฆ where this page comes from