GLOBUS-JOB-MANAGER(8)

GLOBUS-JOB-MANAGER(8) 💖

okay bestie, welcome to the Grid Community Toolkit Manual, section 8 — the one and only globus-job-manager. this is the page where all the job-running magic gets explained, so grab a snack 🍓 and let's go.

✨ NAME ✨

globus-job-manager — Execute and monitor jobs 👑

🌸 SYNOPSIS 🌸

globus-job-manager -type LRM [-conf CONFIG_PATH] [-help]
[-globus-host-manufacturer MANUFACTURER] [-globus-host-cputype CPUTYPE]
[-globus-host-osname OSNAME] [-globus-host-osversion OSVERSION]
[-globus-gatekeeper-host HOST] [-globus-gatekeeper-port PORT]
[-globus-gatekeeper-subject SUBJECT] [-home GLOBUS_LOCATION]
[-target-globus-location TARGET_GLOBUS_LOCATION] [-condor-arch ARCH]
[-condor-os OS] [-history HISTORY_DIRECTORY] [-scratch-dir-base
SCRATCH_DIRECTORY] [-enable-syslog] [-stdio-log LOG_DIRECTORY]
[-log-pattern PATTERN] [-log-levels LEVELS] [-state-file-dir
STATE_DIRECTORY] [-globus-tcp-port-range PORT_RANGE]
[-globus-tcp-source-range SOURCE_RANGE] [-x509-cert-dir
TRUSTED_CERTIFICATE_DIRECTORY] [-cache-location GASS_CACHE_DIRECTORY]
[-k] [-extra-envvars VAR=VAL,...] [-seg-module SEG_MODULE]
[-audit-directory AUDIT_DIRECTORY] [-globus-toolkit-version
TOOLKIT_VERSION] [-disable-streaming] [-service-tag SERVICE_TAG]

🦋 DESCRIPTION 🦋

so. the globus-job-manager program is a service that starts and controls GRAM jobs — and those jobs get executed by a local resource management system, like LSF or Condor. lowkey the backbone of the whole thing. 💅

here's the tea though: globus-job-manager is typically started by the globus-gatekeeper program, not directly by a user. it just keeps vibing and running until every job it's managing has terminated, or until its delegated credentials have expired. main character energy, honestly. ✨

and typically, users interact with globus-job-manager through client applications — think globusrun, globus-job-submit, or tools like CoG jglobus or Condor-G. you're not usually talking to it face to face, iykyk. 🫶

the full set of command-line options to globus-job-manager consists of:

• -help — displays a help message to standard error and exits. short, sweet, slay. 💖
• -type LRM — execute jobs using the local resource manager named LRM.
• -conf CONFIG_PATH — read additional command-line arguments from the file CONFIG_PATH. and listen: if it's present, this must be the first command-line argument to the globus-job-manager program. no exceptions bestie.
• -globus-host-manufacturer MANUFACTURER — tells it the manufacturer of the system that the jobs will execute on. this parameter sets the value of the $(GLOBUS_HOST_MANUFACTURER) RSL substitution to MANUFACTURER.
• -globus-host-cputype CPUTYPE — tells it the CPU type of the system that the jobs will execute on. this parameter sets the value of the $(GLOBUS_HOST_CPUTYPE) RSL substitution to CPUTYPE.
• -globus-host-osname OSNAME — tells it the operating system type of the system that the jobs will execute on. this parameter sets the value of the $(GLOBUS_HOST_OSNAME) RSL substitution to OSNAME.
• -globus-host-osversion OSVERSION — tells it the operating system version of the system that the jobs will execute on. this parameter sets the value of the $(GLOBUS_HOST_OSVERSION) RSL substitution to OSVERSION.
• -globus-gatekeeper-host HOST — tells it the host name of the machine that the job was submitted to. this parameter sets the value of the $(GLOBUS_GATEKEEPER_HOST) RSL substitution to HOST.
• -globus-gatekeeper-port PORT — tells it the TCP port number of the gatekeeper that jobs are submitted to. this parameter sets the value of the $(GLOBUS_GATEKEEPER_PORT) RSL substitution to PORT.
• -globus-gatekeeper-subject SUBJECT — tells it the X.509 identity of the gatekeeper that jobs are submitted to. this parameter sets the value of the $(GLOBUS_GATEKEEPER_SUBJECT) RSL substitution to SUBJECT.
• -home GLOBUS_LOCATION — tells it the path where the Grid Community Toolkit(r) is installed on the service node. the job manager uses this to locate its support and configuration files. 🏠
• -target-globus-location TARGET_GLOBUS_LOCATION — tells it the path where the Grid Community Toolkit(r) is installed on the execution host. if you omit this, the value you gave to -home gets used instead. this parameter sets the value of the $(GLOBUS_LOCATION) RSL substitution to TARGET_GLOBUS_LOCATION.
• -history HISTORY_DIRECTORY — configures the job manager to write job history files into HISTORY_DIRECTORY. those files are described in the FILES section further down this page, so don't skip it. 📝
• -scratch-dir-base SCRATCH_DIRECTORY — configures the job manager to use SCRATCH_DIRECTORY as the default scratch directory root, whenever a relative path is specified in the job RSL's scratch_dir attribute.
• -enable-syslog — configures the job manager to write log messages via syslog. logging is further controlled by the argument to the -log-levels parameter described below. 👀
• -log-pattern PATTERN — configures the job manager to write log messages to files named by the string PATTERN. that PATTERN string can contain job-independent RSL substitutions like $(HOME), $(LOGNAME), etc, plus the special RSL substitution $(DATE), which gets resolved at log time to the date in YYYYMMDD form. obsessed with that, ngl. 📅
• -stdio-log LOG_DIRECTORY — configures the job manager to write log messages to files in the LOG_DIRECTORY directory. this is a backward-compatible parameter, equivalent to -log-pattern (yes, that's literally the option with an empty pattern — that's what the manpage says, don't come for me 💀).
• -log-levels LEVELS — configures the job manager to write log messages of certain levels to syslog and/or log files. the available log levels are FATAL, ERROR, WARN, INFO, DEBUG, and TRACE. multiple values can be combined with the | character. when logging is enabled, the default value is FATAL|ERROR.
• -state-file-dir STATE_DIRECTORY — configures the job manager to write state files to STATE_DIRECTORY. if it's not specified, the job manager uses the default of $GLOBUS_LOCATION/tmp/gram_job_state/. this directory must be writable by all users and be on a file system which supports POSIX advisory file locks. and yes, the original page says that requirement twice, back to back — because it's THAT serious. 🚨🔒
• -globus-tcp-port-range PORT_RANGE — configures the job manager to restrict its TCP/IP communication to use ports in the range described by PORT_RANGE. this value is also made available in the job environment via the GLOBUS_TCP_PORT_RANGE environment variable.
• -globus-tcp-source-range SOURCE_RANGE — configures the job manager to restrict its TCP/IP communication to use source ports in the range described by SOURCE_RANGE. this value is also made available in the job environment via the GLOBUS_TCP_SOURCE_RANGE environment variable.
• -x509-cert-dir TRUSTED_CERTIFICATE_DIRECTORY — configures the job manager to search TRUSTED_CERTIFICATE_DIRECTORY for its list of trusted CA certificates and their signing policies. this value is also made available in the job environment via the X509_CERT_DIR environment variable. 🔐
• -cache-location GASS_CACHE_DIRECTORY — configures the job manager to use the path GASS_CACHE_DIRECTORY for its temporary GASS-cache files. this value is also made available in the job environment via the GLOBUS_GASS_CACHE_DEFAULT environment variable.
• -k — configures the job manager to assume it's using Kerberos for authentication instead of X.509 certificates. this disables some certificate-specific processing in the job manager. 🎫
• -extra-envvars VAR=VAL,... — configures the job manager to define a set of environment variables in the job environment beyond the ones defined in the base job environment. the format of the parameter is a comma-separated sequence of VAR=VAL pairs, where VAR is the variable name and VAL is the variable's value. if the value isn't specified, then the value of that variable in the job manager's own environment gets used. this option may show up multiple times on the command-line or in the job manager configuration file, to append multiple environment settings. 💫
• -seg-module SEG_MODULE — configures the job manager to use the schedule event generator module named by SEG_MODULE to detect job state change events from the local resource manager, in place of the less efficient polling operations used in GT2. to use this, one instance of globus-job-manager-event-generator must be running, to process events for the LRM into a generic format that the job manager can parse. ✨
• -audit-directory AUDIT_DIRECTORY — configures the job manager to write audit records to the directory named by AUDIT_DIRECTORY. those records can be loaded into a database using the globus-gram-audit program. 🗂️
• -globus-toolkit-version TOOLKIT_VERSION — configures the job manager to use TOOLKIT_VERSION as the version for audit and usage stats records.
• -service-tag SERVICE_TAG — configures the job manager to use SERVICE_TAG as a unique identifier, so multiple GRAM instances can share the same job state directories without interfering with each other's jobs. if it's not set, the value untagged gets used. 🏷️
• -disable-streaming — configures the job manager to disable file streaming. this is propagated to the LRM script interface, but it has no effect in GRAM5. 🙈
• -condor-arch ARCH — sets the architecture specification for condor jobs to ARCH, in the job classified ads generated by the GRAM5 condor LRM script. this is required for the condor LRM but ignored for all others.
• -condor-os OS — sets the operating system specification for condor jobs to OS, in the job classified ads generated by the GRAM5 condor LRM script. this is required for the condor LRM but ignored for all others.

🫶 still with me? good. there's more of this page coming — FILES, SEE ALSO, all the good stuff — so don't wander off, bestie.

🌸 ENVIRONMENT 🌸

bestie, if any of these lil variables are hanging around in your environment, they're gonna affect how globus-job-manager runs. so let's meet the whole squad:

• HOME — your user's home directory, the cozy lil place everything lives. 🏠
• LOGNAME — your user's name. that's you, bestie.
• JOBMANAGER_SYSLOG_ID — a string that gets prepended to the syslog audit messages. like a lil name tag for your logs. 🏷️
• JOBMANAGER_SYSLOG_FAC — which facility the syslog audit messages get logged as.
• JOBMANAGER_SYSLOG_LVL — the priority level used for those syslog audit messages.
• GATEKEEPER_JM_ID — the job manager ID that goes into the syslog audit records.
• GATEKEEPER_PEER — the peer information that goes into the syslog audit records.
• GLOBUS_ID — the credential information that goes into the syslog audit records.
• GLOBUS_JOB_MANAGER_SLEEP — how long (in seconds) to sleep when the job manager starts up. [For debugging purposes only] — so like, do not make this your whole personality.
• GRID_SECURITY_HTTP_BODY_FD — the file descriptor of an open file that holds the initial job request, and that the initial job reply should be sent to. this fd is inherited from the globus-gatekeeper, so it's not something you just... conjure. ✨
• X509_USER_PROXY — the path to the X.509 user proxy that the client delegated to the globus-gatekeeper program, which the job manager then uses.
• GRID_SECURITY_CONTEXT_FD — a file descriptor holding an exported security context that the job manager should use to reply to the client that submitted the job. 🔐
• GLOBUS_TCP_PORT_RANGE — the default range of allowed TCP ports to listen on. heads up: the -globus-tcp-port-range command-line option overrides this.
• GLOBUS_TCP_SOURCE_RANGE — the default range of allowed TCP ports to bind to. and yep, the -globus-tcp-source-range command-line option overrides this one too. 💅

🎀 FILES 🎀

here's the whole lineup of files this thing touches. no cap, every single one:

• $HOME/.globus/job/HOSTNAME/LRM.TAG.red — the job manager's delegated user credential.
• $HOME/.globus/job/HOSTNAME/LRM.TAG.lock — the job manager state lock file.
• $HOME/.globus/job/HOSTNAME/LRM.TAG.pid — the job manager pid file.
• $HOME/.globus/job/HOSTNAME/LRM.TAG.sock — the job manager socket, for inter-job-manager communications. the lil group chat between job managers, honestly. 💬
• $HOME/.globus/job/HOSTNAME/JOB_ID/ — the job-specific state directory.
• $HOME/.globus/job/HOSTNAME/JOB_ID/stdin — standard input that's been staged from a remote URL.
• $HOME/.globus/job/HOSTNAME/JOB_ID/stdout — standard output that will be staged from a remote URL.
• $HOME/.globus/job/HOSTNAME/JOB_ID/stderr — standard error that will be staged from a remote URL.
• $HOME/.globus/job/HOSTNAME/JOB_ID/x509_user_proxy — the job-specific delegated credential.
• $GLOBUS_LOCATION/tmp/gram_job_state/job.HOSTNAME.JOB_ID — the job state file.
• $GLOBUS_LOCATION/tmp/gram_job_state/job.HOSTNAME.JOB_ID.lock — the job state lock file. and in most cases this is gonna be a symlink to the job manager lock file, so don't be shocked. 🔗
• $GLOBUS_LOCATION/etc/globus-job-manager.conf — the default location of the global job manager configuration file.
• $GLOBUS_LOCATION/etc/grid-services/jobmanager-LRM — the default location of the LRM-specific gatekeeper configuration file.
• $GLOBUS_LOCATION/etc/globus/gram/job—manager.rvf — the default location of the site-specific job manager RSL validation file.
• $GLOBUS_LOCATION/etc/globus/gram/lrm.rvf — the default location of the site-specific job manager RSL validation file for the named lrm.

🦋 SEE ALSO 🦋

go read these too, they're part of the lore, iykyk:

• globusrun(1)
• globus-gatekeeper(8)
• globus-personal-gatekeeper(1)
• globus-gram-audit(8)

👑 AUTHOR 👑

Copyright © 1999-2016 University of Chicago

and that's the whole page, bestie!! 🫶 this was Grid Community Toolkit 6, dated 12/06/2018, and it lives under the name GLOBUS-JOB-MANAGER(8). go forth and job-manage, you absolute icon. 💖✨