JobManager(3pm)

💖 JobManager(3pm) 💖

okay so bestie, welcome to the manual page for JobManager(3pm), coming at you from the User Contributed Perl Documentation section. This is the lore drop for Globus::GRAM::JobManager, the base class energy behind every Job Manager script. 👑

✨ NAME ✨

Globus::GRAM::JobManager — base class for all Job Manager scripts. That's the whole name section, short and iconic. 🎀

🦋 SYNOPSIS 🦋

Here's the vibe check on how you actually use it, fr fr. All the method calls, exactly as they appear:

$manager = new Globus::GRAM::JobManager($job_description);

$manager->log("Starting new operation");
$manager->nfssync($fileobj,$createflag);
$manager->respond($hashref);
$hashref = $manager->submit();
$hashref = $manager->poll();
$hashref = $manager->cancel();
$hashref = $manager->signal();
$hashref = $manager->make_scratchdir();
$hashref = $manager->remove_scratchdir();
$hashref = $manager->rewrite_urls();
$hashref = $manager->stage_in();
$hashref = $manager->stage_out();
$hashref = $manager->cache_cleanup();
$hashref = $manager->remote_io_file_create();
$hashref = $manager->proxy_relocate();
$hashref = $manager->proxy_update();
$scalar  = $manager->pipe_out_cmd(@arglist);
($stderr, $rc) = $manager->pipe_err_cmd(@arglist);
$status  = $manager->fork_and_exec_cmd(@arglist);
$manager->append_path($hash, $variable, $path);
$scalar = $manager->setup_softenv();

and that's only the synopsis bestie — more parts are loading, don't touch that dial 💅

💖 DESCRIPTION 💖

okay so Globus::GRAM::JobManager is the base behavior module for a Job Manager script interface. if you wanna make a scheduler-specific job manager script, it has to inherit from this module for the job manager to even look at it. no inheriting, no functioning. that's the vibe. ✨

---

✨ Methods ✨

$manager = Globus::GRAM::JobManager->new($JobDescription)

every single Globus::GRAM::JobManager object gets born by calling this constructor with one argument: a Globus::GRAM::JobDescription object holding all the info about the job request your script is gonna be modifying. 💅

and listen — modules that subclass Globus::GRAM::JobManager MUST call the super-class's constructor. not optional. like this code fragment right here:

     my $proto = shift;
     my $class = ref($proto) || $proto;
     my $self = $class->SUPER::new(@_);

     bless $self, $class;

$manager->log($string)

drops a message into the job manager log file 📝. and bestie, the message gets a timestamp slapped on the front of it. every time.

$manager->nfssync($object,$create)

sends an NFS update by touching the file (or directory) in question 🫶. if $create is true, a file gets created. if it's false, the $object will not be created. that's the whole deal.

$manager->respond($message)

sends a response to the job manager program 📨. the response can be either:

• a hash reference — a hash of (variable, value) pairs — which gets returned to the job manager, or
• an already formatted string.

you only need to call this directly from a job manager implementation when the script wants to send a partial response while it's still processing one of the scheduler interface methods (like, say, to indicate that a file has been staged).

the valid keys for a response are defined in the RESPONSES section.

$manager->submit()

submits a job request to the scheduler 🚀. the default implementation just returns with the Globus::GRAM::Error::UNIMPLEMENTED error. scheduler-specific subclasses should reimplement this one to actually submit the job to the scheduler.

a scheduler that implements this method should return a hash reference containing a scheduler-specific job identifier as the value of the hash's JOB_ID key, and optionally a GRAM job state as the value of the hash's JOB_STATE key if the job submission was successful. otherwise? a Globus::GRAM::Error value should come back. the job state values live in the Globus::GRAM::JobState module. the job parameters (as found in the job rsl) are defined in the Globus::GRAM::Jobdescription object in $self->{JobDescription}.

for example:

   return {JOB_STATE => Globus::GRAM::JobState::PENDING,
           JOB_ID => $job_id};

$manager->poll()

polls a job's status 👀. the default implementation returns with the Globus::GRAM::Error::UNIMPLEMENTED error. scheduler-specific subclasses should reimplement this method to poll the scheduler.

a scheduler that implements this method should return a hash reference containing the JOB_STATE value. the job's ID can be accessed by calling the $self->{JobDescription}->jobid() method. iykyk.

$manager->cancel()

cancels a job 🛑. the default implementation returns with the Globus::GRAM::Error::UNIMPLEMENTED error. scheduler-specific subclasses should reimplement this method to remove the job from the scheduler.

a scheduler that implements this method should return a hash reference containing the JOB_STATE value. the job's ID can be accessed by calling the $self->{JobDescription}->jobid() method.

$manager->signal()

signals a job 📣. the default implementation returns with the Globus::GRAM::Error::UNIMPLEMENTED error. scheduler-specific subclasses should reimplement this method to remove the job from the scheduler. the JobManager module can figure out the job's ID, the signal number, and the (optional) signal arguments from the Job Description by calling its job_id(), signal(), and signal_arg() methods, respectively.

depending on the signal, it may be appropriate for the JobManager object to return a hash reference containing a JOB_STATE update.

$manager->make_scratchdir()

creates a scratch directory for a job 🗂️. where does it land? that's based on the JobDescription's scratch_dir_base() and scratch_dir() methods.

if the scratch_dir() value is a relative path, then a directory gets created as a subdirectory of scratch_dir_base()/scratch_dir(). otherwise, it gets created as a subdirectory of scratch_dir(). this method returns a hash reference containing mapping SCRATCH_DIR to the absolute path of the newly created scratch directory, if it worked. 💫

$manager->make_scratchdir()

(yes bestie, the page says it again — same name, and i'm not about to fix it for you 💀)

deletes a job's scratch directory 🧹. all files and subdirectories of the JobDescription's scratch_directory() get deleted.

$manager->make_scratchdir()

(and again. iconic of it, honestly.)

deletes some job-related files 🗑️. all files listed in the JobDescription's file_cleanup() array get deleted.

$manager->rewrite_urls()

looks up URLs listed in the JobDescription's stdin() and executable(), and replaces them with paths to locally cached copies 🔁.

$manager->stage_in()

stages the input files needed for the job from remote storage 📥. the files to be staged are defined by the array of [URL, path] pairs in the job description's file_stage_in() and file_stage_in_shared() methods. the Globus::GRAM::JobManager module provides an implementation of this functionality using the globus-url-copy and globus-gass-cache programs.

files which are staged in are not automatically removed when the job terminates. just so you know, bestie. 😭

this function returns intermediate responses using the Globus::GRAM::JobManager::response() method, to let the job manager know when each individual file has been staged. one by one, slay by slay. ✨

$manager->stage_out()

stages output files generated by this job to remote storage 📤. the files to be staged are defined by the array of [URL, destination] pairs in the job description's file_stage_out() method. the Globus::GRAM::JobManager module provides an implementation of this functionality using the globus-url-copy program.

files which are staged out are not removed by this method.

$manager->cache_cleanup()

cleans up cache references in the GASS which match this job's cache tag 🧼.

$manager->remote_io_file_create()

creates the remote I/O file in the job dir 📁 — the one that'll contain the remote_io_url RSL attribute's value.

$manager->proxy_relocate()

relocates the delegated proxy for job execution 🔐. Job Managers need to override the default if they intend to relocate the proxy into some common file system other than the cache. the job manager program does not depend on the new location of the proxy.

Job Manager modules must not remove the default proxy. do not do it. i'm serious. 🙅

$hashref = $manager->proxy_update();

$manager->append_path($ref, $var, $path)

appends $path to the value of $ref->{$var}, and deals with the case where $ref->{$var} isn't defined yet. lowkey the most helpful little guy. 🫶

$manager->pipe_out_cmd(@arg)

creates a new process to run the first argument application with the remaining arguments (which may be empty) ⚙️. no shell metacharacter will be evaluated — this avoids a shell invocation entirely. stderr gets redirected to /dev/null, and stdout is captured by the parent process, which is also the result returned.

• in list mode, all lines are returned.
• in scalar mode, only the first line is returned.

the line termination character is already cut off. use this function as a more efficient backticks, if you don't need shell metacharacter evaluation. 💅

Caution: this function deviates in two manners from regular backticks. firstly, it chomps the line terminator from the output. secondly, it returns only the first line in scalar context, instead of a multiline concatenated string. as with regular backticks, the result may be undefined in scalar context if no result exists.

a child error code with an exit code of 127 indicates that the application could not be run. the scalar result returned by this function is usually undef'ed in this case.

($stder, $rc) = $manager->pipe_err_cmd(@arg)

creates a new process to run the first argument application with the remaining arguments (which may be empty) ⚙️. no shell metacharacter will be evaluated, avoiding a shell invocation.

this method returns a list of two items: the standard error of the program, and the exit code of the program. if the error code is 127, then the application could not be run. standard output is discarded.

$manager->fork_and_exec_cmd(@arg)

forks off a child to run the first argument in the list 🍴. the remaining arguments get passed, but shell interpolation is avoided. signals SIGINT and SIGQUIT are ignored in the child process. stdout is appended to /dev/null, and stderr is dup2 from stdout. the parent waits for the child to finish, and returns the value for the CHILD_ERROR variable as the result. use this function as a more efficient system() call, if you do not need shell metacharacter evaluation. 🚀

note that the inability to execute the program will result in a status code of 127.

$manager->job_dir()

returns the temporary directory to store job-related files, which have no need for file caching 📂.

$manager->setup_softenv()

either adds a line to the specified command script file handle to load the user's default SoftEnv configuration, or creates a custom SoftEnv script and adds commands to the specified command script file handle to load it 🌸. two paths, one method, we love a flexible queen.

✨ RESPONSES ✨

Okay bestie, this is the part where your job interface method talks back. 💬 Whenever you're returning from a job interface method, or whenever you're sending an intermediate response through the response() method, these are the hash keys that are valid. That's it, that's the list, no cap. 🫶

• JOB_STATE — an integer job state value, and these ones are enumerated over in the Globus::GRAM::JobState module. Slay. 💅
• ERROR — an integer error code. These live enumerated in the Globus::GRAM::Error module. 👑
• JOB_ID — a string holding a job identifier, and like, this is the one you use to poll, cancel, or signal a job that's in progress. 🦋 Heads up though: this response should only be returned by the submit method. Don't be out here returning it from anywhere else, iykyk.
• SCRATCH_DIR — a string with the path to a newly-created scratch directory. And this one? Should only be returned by the make_scratchdir method. Main character behavior only. ✨
• STAGED_IN — a string containing the (URL, path) pair for a file which has now been staged in. This response should only be returned by the stage_in method. 💖
• STAGED_IN_SHARED — a string containing the (URL, path) pair for a file which has now been staged in and symlinked from the cache. Obsessed, honestly. This response should only be returned by the stage_in_shared method. 🌸
• STAGED_OUT — a string containing the (path, URL) pair for a file which has now been staged out by the script. Yes, the order flipped, it's giving ✨character development✨, don't let it trip you up. This response should only be returned by the stage_out method. 🌈

---

🎀 tiny reminder bestie: notice how every single one of these has a "only return this from that method" clause. That is not decoration. That is the vibe check. Do not freestyle it. 💀

---

And that's a wrap on the whole page, fr fr! 💜 This little beauty shipped as part of perl v5.16.3, dated 2020-12-12 — so yeah, she's a vintage icon and she still eats. 👑✨ Go forth and return your hashes correctly, bestie. Mwah. 💅🫶