CONDOR_QSUB(1)

📦
version 25.14.1 · 12 versions
, in fedora 43

💅 condor_qsub — fresh from the HTCondor Manual ✨

okay bestie, buckle UP — this is the condor_qsub page, and we are doing it thoroughly. Every flag, every default, every weird little caveat. No skips. Periodt. 🎀

🌸 NAME 🌸

condor_qsub — straight out of the HTCondor Manual.

Queue jobs that use PBS/SGE-style submission 💖

🦋 SYNOPSIS 🦋

condor_qsub [--version]

condor_qsub [Specific options ] [Directory options ] [Environmental options ] [File options ] [Notification options ] [Resource options ] [Status options ] [Submission options ] commandfile

💜 DESCRIPTION 💜

So like, condor_qsub submits an HTCondor job. That job is specified in a PBS/Torque style or an SGE style. It also lets you submit dependent jobs without having to spell out the entire dependency graph at submission time. And ngl — doing it that way is neither as efficient as HTCondor's DAGMan, nor as functional as SGE's qsub or qalter. It's just the vibe. condor_qsub is basically a minimal translator, existing so you can run software that was originally written to talk to PBS, Torque, and SGE inside an HTCondor pool. 🫶

condor_qsub attempts to behave like qsub. Less than half of the qsub functionality is implemented — fr fr, that's the deal. The option descriptions below literally exist to tell you the differences between how qsub behaves and how condor_qsub behaves. Any qsub options not listed here? Not supported. And some concepts that live in PBS and SGE just don't apply to HTCondor at all, so those options aren't implemented. 💀

For a full listing of qsub options, please see:

condor_qsub accepts either command line options or the single file, commandfile, that contains all of the commands. Pick your fighter. ⭐

And the plot twist: condor_qsub does the opposite of job submission within the grid universe batch grid type — because that takes HTCondor jobs submitted with HTCondor syntax and submits them to PBS, SGE, or LSF. Iconic mirror moment. 🪞

🔥 OPTIONS 🔥

  • -a date_time — (Submission option) Specify a deferred execution date and time. The PBS/Torque syntax of date_time is a string in the form [[[[CC]YY]MM]DD]hhmm[.SS]. The portions of this string which are optional are CC, YY, MM, DD, and SS. For SGE, MM and DD are not optional. For PBS, MM and DD are optional. condor_qsub follows the PBS style.
  • -A account_string — (Status option) Uses group accounting, where the string account_string is the accounting group associated with this job. Unlike SGE, there is no default group of "sge". 🙈
  • -b y|n — (Submission option) Using the SGE definition of its -b option, a value of y causes condor_qsub to not parse the file for additional condor_qsub commands. The default value is n. If the command line argument -f filename is also specified, it negates a value of y.
  • -condor-keep-files — (Specific option) Directs HTCondor to not remove temporary files generated by condor_qsub, such as HTCondor submit files and sentinel jobs. These temporary files may be important for debugging. Keep them, bestie, you'll thank yourself later. 🫶
  • -cwd — (Directory option) Specifies the initial directory in which the job will run to be the current directory from which the job was submitted. This sets initialdir for condor_submit.
  • -d path or -wd path — (Directory option) Specifies the initial directory in which the job will run to be path. This sets initialdir for condor_submit.
  • -e filename — (File option) Specifies the condor_submit command error, the file where stderr is written. If not specified, set to the default name of <commandfile>.e<ClusterId>, where <commandfile> is the condor_qsub argument, and <ClusterId> is the job attribute ClusterId assigned for the job.
  • -f qsub_file — (Specific option) Parse qsub_file to search for and set additional condor_submit commands. Within the file, commands will appear as #PBS or #SGE. condor_qsub will parse the batch file listed as qsub_file.
  • -h — (Status option) Placed submitted job directly into the hold state.
  • -help — (Specific option) Print usage information and exit.
  • -hold_jid <jid> — (Status option) Submits a job in the hold state. This job is released only when a previously submitted job, identified by its cluster ID as <jid>, exits successfully. Successful completion is defined as not exiting with exit code 100.

In implementation, there are three jobs that define this SGE feature:

  1. The first job is the previously submitted job.
  2. The second job is the newly submitted one that is waiting for the first to finish successfully.
  3. The third job is what SGE calls a sentinel job; this is an HTCondor local universe job that watches the history for the first job's exit code. This third job will exit once it has seen the exit code and, for a successful termination of the first job, run condor_release on the second job.

If the first job is an array job, the second job will only be released after all individual jobs of the first job have completed. 👑

  • -i [hostname:]filename — (File option) Specifies the condor_submit command input, the file from which stdin is read.
  • -j characters — (File option) Acceptable characters for this option are e, o, and n. The only sequence that is relevant is eo; it specifies that both standard output and standard error are to be sent to the same file.
    • The file will be the one specified by the -o option, if both the -o and -e options exist.
    • The file will be the one specified by the -e option, if only the -e option is provided.
    • If neither the -o nor the -e options are provided, the file will be the default used for the -o option.
  • -l resource_spec — (Resource option) Specifies requirements for the job, such as the amount of RAM and the number of CPUs. Only PBS-style resource requests are supported. resource_spec is a comma separated list of key/value pairs. Each pair is of the form resource_name=value.
    • resource_name — arch | value: string | Sets Arch machine attribute. Enclose in double quotes.
    • resource_name — file | value: size | Disk space requested.
    • resource_name — host | value: string | Host machine on which the job must run.
    • resource_name — mem | value: size | Amount of memory requested.
    • resource_name — nodes | value: {<node_count> | <hostname>} [:ppn=<ppn>] [:gpus=<gpu>] [:<property> [:<property>] ...] [+ ...] | Number and/or properties of nodes to be used. For examples, please see http://docs.adaptivecomputing.com/torque/4-1-3/Content/topics/2-jobs/requestingRes.htm#qsub
    • resource_name — opsys | value: string | Sets OpSys machine attribute. Enclose in double quotes.
    • resource_name — procs | value: integer | Number of CPUs requested.

A size value is an integer specified in bytes, following the PBS/Torque default. Append Kb, Mb, Gb, or Tb to specify the value in powers of two quantities greater than bytes.

  • -m a|e|n — (Notification option) Identify when HTCondor sends notification e-mail.
    • If a, send e-mail when the job terminates abnormally.
    • If e, send e-mail when the job terminates.
    • If n, never send e-mail.
  • -M e-mail_address — (Notification option) Sets the destination address for HTCondor e-mail. 📧
  • -o filename — (File option) Specifies the condor_submit command output, the file where stdout is written. If not specified, set to the default name of <commandfile>.o<ClusterId>, where <commandfile> is the condor_qsub argument, and <ClusterId> is the job attribute ClusterId assigned for the job.
  • -p integer — (Status option) Sets the priority submit command for the job, with 0 being the default. Jobs with higher numerical priority will run before jobs with lower numerical priority.
  • -print — (Specific option) Send to stdout the contents of the HTCondor submit description file that condor_qsub generates. ✨
  • -r y|n — (Status option) The default value of y implements the default HTCondor policy of assuming that jobs that do not complete are placed back in the queue to be run again. When n, job submission is restricted to only running the job if the job ClassAd attribute NumJobStarts is currently 0. This identifies the job as not re-runnable, limiting it to start once.
Lowkey important: that -r n one means your job gets exactly one shot. No takebacks. 💀
  • -S shell — (submission option)</p> bestie this one sets the path AND the executable name of a shell. What it does is lowkey sneaky: it rewrites the HTCondor submit description file it produces so that the executable becomes a wrapper script. Inside that submit description file you'll get executable = <shell> and arguments = <commandfile>. ✨
  • -t start [-stop:step] — (submission option)</p> this is the queue-a-bunch-of-almost-identical-jobs button, and the SGE-style syntax is fully supported 🎀. Fyi start, stop, and step are all integers. start is the starting index of the jobs, stop is the ending index and it's inclusive, and step is the step size as you walk through the indices.
⚠️ Heads up bestie: using more than one processor or node in a job will not work with this option. Like, at all. 💀
  • -test — (specific option)</p> the "let's pretend" button 🧚. It's meant for testing a potential job submission: it parses files and commands so it can generate error output. It produces the HTCondor submit description file and then removes it again. And it never submits the job, even if there are zero errors. Iconic behaviour honestly.
  • -v variable list — (environmental option)</p> sets the submit command environment for the job. The variable list is just as defined for the submit command. Note that the syntax you need is specialized to deal with quote marks and white space characters — so like, don't freestyle it 🫶.
  • -V — (environmental option)</p> sets getenv = True in the submit description file. That's it, that's the whole vibe. ✅
  • -W attr_name=attr_value[,attr_name=attr_value...] — (file option)</p> so PBS/Torque supports a whole bunch of attributes, but condor_qsub only supports the names stagein and stageout for attr_name 🙈. The format of attr_value for stagein and stageout is local_file@hostname:remote_file[,...] and we strip it down to remote_file[,...]. HTCondor's file transfer mechanism gets used after that, if needed.
  • -version — (specific option)</p> prints version information for the condor_qsub program and exits 👑. Note that condor_qsub has its own version numbers, which are separate from those of HTCondor.

🚦 EXIT STATUS 🚦

Ok so this part is short but important, no cap:

  • condor_qsub exits with a status value of 0 (zero) upon success 💖
  • and it exits with the value 1 (one) upon failure to submit a job 😭

👑 AUTHOR 👑

HTCondor Team — the whole squad, fr fr 🫶

1990-2026, Center for High Throughput Computing, Computer Sciences Department, University of Wisconsin-Madison, Madison, WI, US. Licensed under the Apache License, Version 2.0.

and that's a wrap bestie 💅✨ this little gem comes from the HTCondor source, version Oct 16, 2026, and you can find it filed under CONDOR_QSUB(1). slay, go queue some jobs 🦋🌸

📦 where this page comes from