JSV_SCRIPT_INTERFACE(3)

📦 version 8.1.9, in debian bookworm, debian trixie, ubuntu jammy, ubuntu noble, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 NAME 💖

Bestie, welcome to JSV_SCRIPT_INTERFACE(3) — the Grid Engine File Formats manual page for the Grid Engine Job Submission Verifier Scripting Interface. 🦋

And here's the whole squad of functions it covers:

jsv_is_param, jsv_get_param, jsv_del_param, jsv_sub_is_param, jsv_sub_get_param, jsv_sub_add_param, jsv_sub_del_param, jsv_is_env, jsv_get_env, jsv_add_env, jsv_mod_env, jsv_del_env, jsv_accept, jsv_correct, jsv_reject, jsv_reject_wait, jsv_show_params, jsv_show_envs, jsv_log_info, jsv_log_warning, jsv_log_error, jsv_main, jsv_logging_enabled

✨ SYNOPSIS ✨

Okay so here's the whole roll call, exactly how you call 'em:

jsv_clear_params();
jsv_is_param(param_name);
jsv_get_param(param_name);
jsv_set_param(param_name, param_value);
jsv_del_param(param_name);
jsv_sub_is_param(param_name, variable_name);
jsv_sub_get_param(param_name, variable_name);
jsv_sub_add_param(param_name, variable_name, variable_value);
jsv_sub_del_param(param_name, variable_name);

jsv_clear_envs();
jsv_is_env(variable_name);
jsv_get_env(variable_name);
jsv_add_env(variable_name, variable_value);
jsv_mod_env(variable_name, variable_value);
jsv_del_env(variable_name);

jsv_accept(message);
jsv_correct(message);
jsv_reject(message);
jsv_reject_wait(message);

jsv_show_params();
jsv_show_envs();
jsv_log_info(message);
jsv_log_warning(message);
jsv_log_error(message);

jsv_main();
jsv_send_env();
jsv_on_start();
jsv_on_verify();

🌸 DESCRIPTION 🌸

So the tea is: the functions documented right here implement the server side of the JSV protocol, just like it says over in jsv(1) — and yeah, "server" is doing double duty, it applies to both client-side and server-side JSVs. 💅

These functions are available in Bourne shell (and honestly, prefer bash(1) because it's way more efficient, no cap), TCL, Perl, or Python scripts — after you source or include the files jsv_inlcude.sh, jsv_include.tcl, JSV.pm, or JSV.py. The files and their matching JSV script templates live in the directory $SGE_ROOT/util/resources/jsv. 👑

There's also a Java implementation, but it's built different — different structure, with Javadoc documentation normally hanging out in $SGE_ROOT/doc/javadocs/jjsv, and example files SimpleJsv.java and jjsv.sh chilling in $SGE_ROOT/util/resources/jsv.

⚠️ Real talk, bestie: Bourne shell server JSVs are discouraged. Why? Because any problems with unintended expansion of job parameters are a straight-up security hazard (and the qmaster is running as the sgeadmin user, so like… yeah). They can also be relatively slow, and they become a bottleneck in high-throughput clusters. However, shell JSVs may be faster using bash(1) — even compared against a generally faster shell — because fewer external commands get invoked. 🫶

In the descriptions of routines here, a calling sequence like

function(arg1, arg2)

should be interpreted for Bourne shell and Tcl scripts as

function arg1 arg2

🧹 jsv_clear_params()

This one clears all the received job parameters that were stored during the last job verification process. Poof. Gone. ✨

🧼 jsv_clear_envs()

This one clears all the received job environment variables that were stored during the last job verification process. Also poof. 💨

📋 jsv_show_params()

Call this and it reports all known job parameters to the counterpart of this script (that's the client or the master daemon thread, iykyk). These parameters get reported as info messages, and they show up either in the stdout stream of the client or in the message file of the master process. 🌈

🌍 jsv_show_envs()

This reports all known job environment variables to the counterpart of this script (client or master daemon thread). They'll be reported as info messages and appear in the stdout stream of the client or in the message file of the master process. 🦋

❓ jsv_is_param(param_name)

This function returns whether or not a specific job parameter is available for the job currently being verified. You'll get back either the string true or false.

Important vibe check: the availability or absence of a job parameter does not mean the corresponding command line switch was used or not used. They are not the same thing, bestie. 😭

The values allowed for param_name are listed below. Find additional information in qsub(1) describing the availability and value format. Job parameters written in capital letters are pseudo parameters — a detailed description for them can be found in jsv(1). Note that

The following parameters directly reflect arguments of the same name supplied to the submission command (qsub etc.) or corresponding values specified with qmon:

A, a, ar, b, ckpt, dl, e, h, hold_jid, hold_jid_ad, i, j, js, M, m, masterq, N, notify, o, P, p, R, r, S, shell, tc, w

Other parameters are related to the submission command arguments as follows: 👇

  • ac — the job context. The outcome of the evaluation of all -ac, -dc, and -sc options is passed as a parameter with the name ac, whose value is a comma-separated list of variable/value pairs.
  • binding_strategy, binding_type, binding_amount, binding_step, binding_socket, binding_core, binding_exp_n, binding_exp_socketid, binding_exp_coreid — the values passed to the -binding parameter are passed as multiple parameters to JSV instances. binding_strategy represents the strategy to be used and is one of: linear, striding or explicit. binding_type is the instance that should do the binding, one of: env, set or pe. binding_socket and binding_core are socket/core values, whereas binding_step is the step size (used only for striding binding). The length of the socket/core value list of the explicit binding is reported as binding_exp_n. The id part of binding_exp_socketid and binding_exp_coreid will be replaced by the position of the socket/core pair within the explicit binding list (0 <= id < binding_exp_n). The first socket/core pair of the explicit binding will be reported with the parameter names binding_exp_socket0 and binding_exp_core0. Values that do not apply for the specified binding will not be reported to JSV. E.g. binding_step will only be reported for the striding binding and all binding_exp_... values will only be passed if explicit binding was specified.
  • c_interval — checkpoint interval, specified as a numeric value with -c.
  • c_occasion — checkpoint "occasion_specifier" -c (n, s, m, or x) specified with -c.
  • cwd — working directory, possibly specified with -cwd or -wd.
  • display — reflects the -display submit argument and also sets job environment variable DISPLAY to the same value.
  • l_hard — -l or -hard followed by -l.
  • l_soft — -soft followed by -l.
  • pe_name, pe_min, pe_max — the PE name and range limits specified with -pe.
  • q_hard — -q, or -hard followed by -q.
  • q_soft — -soft followed by -q.

See jsv(1) for explanation of the following pseudo parameters: CLIENT, CMDNAME, CMDARGS, CMDARGi, CONTEXT, GROUP, JOB_ID, USER, VERSION. ⭐

🔍 jsv_get_param(param_name)

This function returns the value of a specific job parameter param_name. 🎀

This value is only available if the function jsv_is_param() returns true. Otherwise an empty string is returned. (Sadge, but that's the rules.)

Find a list of allowed parameter names in the section for the function jsv_is_param().

✏️ jsv_set_param(param_name, param_value)

This function changes the job parameter param_name to the value param_value. Glow up, parameter edition. 💅

If param_value is an empty string then the corresponding job parameter will be deleted, similarly to the function jsv_del_param(). As a result, the job parameter is not available, as if the corresponding command line switch was not specified during job submission. 🫥

⚠️ For boolean parameters that only accept the values yes or no, it is not allowed to pass an empty string as param_value.
⚠️ Also for the parameters c and m it is not allowed to use empty strings. Details can be found in qsub(1).

🗑️ jsv_del_param(param_name)

This function deletes the job parameter param_name. Bye bestie. 👋

Find a list of allowed parameter names in the section for the function jsv_is_param().

🧐 jsv_sub_is_param(param_name, variable_name)

Some job parameters are lists that can contain multiple variables with an optional value. (Multi-variable slay.)

This function returns true if a job's parameters contain the list-valued param_name, with variable_name in the list; otherwise it returns false. Heads up: false might also indicate that the parameter list itself is not available. Use the function jsv_is_param() to check if the parameter list is not available. 🔎

The following parameters are list parameters. The second column describes the corresponding variable names to be used. The third column contains a dash (-) if there is no value (variable_value) allowed with the function jsv_sub_add_param(), or jsv_sub_get_param() will return always an empty string. A question mark (?) shows that the value is optional. 🌸

  • ac — variable_name: job context variable name · variable_value: (blank in the original table)
  • hold_jid — variable_name: job identifier · variable_value: -
  • hold_jid_id — variable_name: array job identifier · variable_value: -
  • l_hard — variable_name: complex attribute name · variable_value: ?
  • l_soft — variable_name: complex attribute name · variable_value: ?
  • M — variable_name: mail address · variable_value: -
  • masterq — variable_name: cluster queue name or queue instance name · variable_value: -
  • q_hard — variable_name: cluster queue name or queue instance name · variable_value: -
  • q_soft — variable_name: cluster queue name or queue instance name · variable_value: -

🔎 jsv_sub_get_param(param_name, variable_name)

Some job parameters are lists that can contain multiple variables with an optional value.

This function returns the value of a variable variable_name in the parameter list param_name. For sub list elements that have no value, an empty string will be returned. 🙈

Find a list of allowed parameter names (param_name) and variable names (variable_name) in the section for the function jsv_sub_is_param().

➕ jsv_sub_add_param(param_name, variable_name, variable_value)

Some job parameters are lists that can contain multiple variables with an optional value.

...and honestly? That's the whole personality of this one: it either adds a brand new variable with a brand new value, or — if that variable is already sitting in the list like it owns the place — it just swaps the value out. 💅

variable_value is optional, bestie. Don't pass it and the variable simply has no value. Which is a vibe.

Wanna know which param_names and variable_names are even allowed? They're all listed in the section for jsv_sub_is_param(). Go read it, it's right there. ✨

jsv_sub_del_param(param_name, variable_name)

Okay so some job parameters are literally lists 📋 — they can hold multiple variables, and each of those variables can carry an optional value.

This function deletes a variable variable_name and, if there is one, its corresponding value too. And if variable_name isn't even hanging around in that job parameter? The command gets ignored. No error, no drama, it just doesn't happen. 🦋

And yes, the allowed param_names and variable_names live in the section for jsv_sub_is_param(). Same list, same place. 👀

jsv_is_env(variable_name)

If this returns true, then the job environment variable named variable_name exists in the job currently being verified, and you can go grab its value with jsv_get_env(). 💖

If it returns false, then the job environment variable does not exist. That's the whole story. Simple. Iconic. 😌

jsv_get_env(variable_name)

This one hands you back the value of a job environment variable variable_name. 🌸

Okay important, so listen: this variable has to be passed with the qsub command line switch -v or -V, and passing of environment variable data to JSV scripts has to be enabled. That env data only shows up when you call jsv_send_env() inside the callback function jsv_on_start().

And if the variable doesn't exist, or if environment variable information just isn't available at all? You get an empty string back. Rude but consistent. 🫥

jsv_add_env(variable_name, variable_value)

This one adds an additional environment variable to the set of variables that will be exported to the job when it starts. So after this, variable_name and variable_value are just... available, exactly as if -v or -V had been specified during job submission. ✨

variable_value is optional. Pass an empty string and the variable is defined without a value.

Now, if variable_name already exists in the set of job environment variables, its value gets replaced by variable_value — as if you'd used jsv_mod_env(). And if you pass an empty string in that case, the old value gets deleted. 💀

To actually delete an environment variable you have to use jsv_del_env(). This one will not do it for you.

jsv_mod_env(variable_name, variable_value)

This one modifies an environment variable that's already in the set of variables which will be exported to the job when it starts. Same deal: after this, variable_name and variable_value are available as if -v or -V was specified during job submission. 🎀

variable_value is optional. Empty string → variable is defined without a value.

And the plot twist: if variable_name does not already exist in the set of job environment variables, then the name and value just get added — as if you'd called jsv_add_env(). It's giving "I'll meet you where you're at". 🫶

To delete an environment variable, use jsv_del_env().

jsv_del_env(variable_name)

Removes the job environment variable variable_name from the set of variables that will be exported to the job when it is started. Bye bestie 👋

If variable_name doesn't already exist in that set? The command is ignored. Nothing happens. Move along.

Wanna change a value instead? Call jsv_mod_env(). Wanna add a new one? Call jsv_add_env(). That's the trio, they're a set. 💜

jsv_accept(message)

⚠️ This function can only be used in jsv_on_verify(). And after you've called it, jsv_on_verify() has to return immediately. Like, no more lines after it. None. Zero. ✋

Calling this says: the job currently being verified should be accepted as it was initially provided. And here's the spicy part — all job modifications that might have been applied in jsv_on_verify() before this function was called? Ignored. All that work, gone. 🙈

Instead of calling jsv_accept() in jsv_on_verify(), you can call jsv_correct(), jsv_reject() or jsv_reject_wait() — but only one of these functions can be used at a time. Pick a lane, bestie. 🚦

jsv_correct(message)

⚠️ Only usable in jsv_on_verify(). After it's called, jsv_on_verify() has to return immediately. Same rule, every time.

This one says: the job currently being verified has to be modified before it can be accepted. All job parameter modifications that were previously applied get committed, and the job gets accepted. 🔥

And what does "accept" even mean here, you ask? It means the job is either passed to the next JSV instance for modification, or — if this was the last JSV instance to verify it — it's passed to that component in the master daemon that adds it to the master data store. 👑

Instead of calling jsv_correct() in jsv_on_verify(), you can call jsv_accept(), jsv_reject() or jsv_reject_wait() — but only one of these functions can be used. Just one. Periodt. 💅

jsv_reject(message)

⚠️ Only usable in jsv_on_verify(). After it's called, jsv_on_verify() has to return immediately.

The job currently being verified gets rejected. ❌ message gets passed to the client application that tried to submit the job. Commandline clients like qsub will print that message to stdout to inform the user that the submission has failed.

Now — if the user might try to submit the job again, you should call jsv_reject_wait() instead. jsv_reject_wait() signals that the verification process might be successful in the future. 🌈

Instead of calling jsv_reject() in jsv_on_verify(), you can call jsv_accept(), jsv_correct() or jsv_reject_wait() — but only one of these functions can be used. One. Singular. 😭

jsv_reject_wait(message)

⚠️ Only usable in jsv_on_verify(). After it's called, jsv_on_verify() has to return immediately.

The job which is currently verified will be rejected. message gets passed to the client application that tries to submit the job. Commandline clients like qsub will print that message to stdout to inform the user that the submission has failed.

So why this one instead of plain old jsv_reject()? Because this is the one you call when the user who's trying to submit the job might have a chance to submit it later. 🌱 jsv_reject means the verified job will also be rejected in future — no take-backs, no retries.

Instead of calling jsv_reject_wait() in jsv_on_verify(), you can call jsv_accept(), jsv_correct() or jsv_reject() — but only one of these functions can be used. You know the drill by now. 🎯

jsv_log_info(message)

Sends an info message to the client or master daemon instance that started the JSV script. 📨

For client JSVs, that means the command line client gets the info and prints it to the stdout stream. For server JSVs, that message gets printed as an info message to the master daemon message file.

And if message is missing? An empty line gets printed. Thrilling. 😌

jsv_log_warning(message)

Same energy, different severity — this one sends a warning message to the client or master daemon instance that started the JSV script. ⚠️

For client JSVs: the command line client gets the info and prints it to the stdout stream. For server JSVs: it gets printed as a warning message to the master daemon message file.

If message is missing, an empty line gets printed. 🫧

jsv_log_error(message)

And the loud one: sends an error message to the client or master daemon instance that started the JSV script. 🚨

For client JSVs: command line client gets the info and prints it to the stdout stream. For server JSVs: printed as an error message to the master daemon message file.

If message is missing, an empty line gets printed. Still. 😭

jsv_send_env()

⚠️ Can only be used in jsv_on_start(). And if you use it there, then the job environment information will be available in jsv_on_verify() for the next job that's scheduled to be verified. Timing matters, bestie. ⏰

This function must be called for jsv_show_envs(), jsv_is_env(), jsv_get_env(), jsv_add_env() and jsv_mod_env() to behave correctly. No jsv_send_env(), no env data, no working functions. That's the deal. 🤝

Also, real talk: job environments might become very big — 10kB and more. That will slow down the executing component (the submit client or the master daemon thread). For this reason, job environment information is not passed to JSV scripts by default. That's not a bug, that's a design choice.
And one more thing — please note that the data in the job environment can't be verified by Grid Engine and might therefore contain values which could be misinterpreted in the script environment and cause security issues. Be careful with that. 🔐

jsv_main()

This one has to be called in the main function in JSV scripts. It implements the JSV protocol and performs the communication with the client and server components that might start JSV scripts. 👑

It does not return immediately. It only returns when the "QUIT" command is sent by the client or server component. So it's basically the main character who stays until the party's over. 🎉

During that communication, this function triggers two callback functions for each job that should be verified: first jsv_on_start() and later on jsv_on_verify().

jsv_on_start() can be used to initialize certain things that might be needed for the verification process. jsv_on_verify() does the verification process itself. ✨

And jsv_send_env() can be called in jsv_on_start() so that the job environment is available in jsv_on_verify().

Here's the rundown of what goes where — the following functions can only be used in jsv_on_verify():

  • Simple job parameters can be accessed/modified with: jsv_is_param, jsv_get_param, jsv_set_param and jsv_del_param.
  • List based job parameters can be accessed with: jsv_sub_is_param, jsv_sub_get_param, jsv_sub_add_param and jsv_sub_del_param.
  • If the environment was requested with jsv_send_env() in jsv_on_start(), then the environment can be accessed/modified with the following commands: jsv_is_env, jsv_get_env, jsv_add_env, jsv_mod_env and jsv_del_env.
  • Jobs can be accepted/rejected with: jsv_accept, jsv_correct, jsv_reject and jsv_reject_wait.
  • The following functions send messages to the calling component of a JSV, which will either appear on the stdout stream of the client or in the master message file. This is especially useful when new JSV scripts should be tested: jsv_show_params, jsv_show_envs, jsv_log_info, jsv_log_warning and jsv_log_error.

jsv_on_start()

This is a callback function that has to be defined by the creator of a JSV script. 🙋 It's called for every job a short time before the verification process of that job starts.

Inside this function, jsv_send_env can be called to request job environment information for the next job scheduled to be verified. 📬

And after this function returns, jsv_on_verify() will be called — that's the function that does the verification process itself.

🌸 jsv_on_verify() 🌸

Okay so this one is a callback function, and if you're the creator of a JSV script, you literally have to define it. It gets called for every single job, and when it returns, that job is either gonna be accepted or rejected. Wanna see implementation examples? They're chillin' in $SGE_ROOT/util/resources/jsv. ✨

The logic of this function is completely up to its creator. The only thing you have to take care of? Before the function returns, you must call one of these: jsv_accept(), jsv_reject(), jsv_reject_wait(), or jsv_correct(). That's the rule, bestie. No cap.

🌈 jsv_logging_enabled 🌈

Set this variable to true and you get logging output that traces the JSV protocol. It gets sent to a file shaped like /tmp/jsv_$$.log. And for shell JSVs? You can set it in the environment of the job submission to effect logging without modifying the script. So convenient, honestly. 🫶

✨ EXAMPLES ✨

Find below the returned values for the is and get functions when following job is submitted:

qsub -l mem=1G,mem2=200M -l a=lx-amd64 ...

Here's the function-call tea:

  • jsv_is_param(l_hard) → "true"
  • jsv_get_param(l_hard) → "mem=1G,mem2=200M,a=lx-amd64"
  • jsv_sub_is_param(l_hard,mem) → "true"
  • jsv_sub_get_param(l_hard,mem) → "1G"
  • jsv_sub_is_param(l_hard,mem3) → "false"
  • jsv_sub_get_param(l_hard,mem3) → ""
  • jsv_sub_is_param(l_hard,a) → "true"
  • jsv_sub_get_param(l_hard,a) → "lx-amd64"
  • jsv_sub_is_param(l_hard,arch) → "false"
  • jsv_sub_get_param(l_hard,arch) → ""

📁 FILES 📁

Here's your file map, bestie:

  • Include files:
    • $SGE_ROOT/util/resources/jsv/jsv_include.sh
    • $SGE_ROOT/util/resources/jsv/jsv_include.tcl
    • $SGE_ROOT/util/resources/jsv/JSV.pm
  • Example files:
    • $SGE_ROOT/util/resources/jsv/jsv.sh
    • $SGE_ROOT/util/resources/jsv/jsv.tcl
    • $SGE_ROOT/util/resources/jsv/jsv.pl
    • $SGE_ROOT/util/resources/jsv/jjsv.sh
    • $SGE_ROOT/util/resources/jsv/SimpleJsv.java
  • Debugging log file:
    • /tmp/jsv_$$.log

💀 BUGS 💀

Complex names seen by the script are not canonicalized. I.e., if the name and shortcut vary, it is necessary to consider both. So yeah, you gotta check both. It's giving extra homework. 😭

👀 SEE ALSO 👀

See sge_intro(1) for a full statement of rights and permissions. That's where the legal tea is. 💅

That's the whole vibe from SGE 8.1.3pre, dated 2013-05-01, for JSV_SCRIPT_INTERFACE(3). Stay iconic, bestie. 💖✨

📦 where this page comes from