CONDOR_SUBMIT_DAG(1)

📦
version 25.14.1 · 12 versions
, in fedora 43

✨ condor_submit_dag(1) — HTCondor Manual ✨

okay bestie, gather round, because this one is a whole moment 💖

🎀 NAME 🎀

condor_submit_dag — HTCondor Manual

Place a node scheduler job with the local AP to manage job workflow described as a Direct Acyclic Graph (DAG). periodt.

🌸 SYNOPSIS 🌸

condor_submit_dag [-help | -version]

condor_submit_dag [OPTIONS] DAG_file [DAG_file...]

condor_submit_dag [-no_submit] [-v/-verbose] [-f/-force] [-dagman path]
[-notification value] [-debug level] [-MaxIdle N] [-MaxJobs N] [-MaxPre
N] [-MaxPost N] [-MaxHold N] [-suppress_notification |
-dont_suppress_notification] [-SubmitMethod value] [-UseDagDir]
[-outfile_dir path] [-config filename] [-Lockfile filename]
[-insert_sub_file filename] [-a/-append command] [-batch-name name]
[-AutoRescue <0|1>] [-DoRescueFrom N] [-load_save filename]
[-DoRecovery] [-AllowVersionMismatch] [-no_recurse | -do_recurse]
[-update_submit] [-import_env] [-include_env variable[,variable...]]
[-insert_env key=value[;key=value...]] [-valgrind] [-DontAlwaysRunPost |
-AlwaysRunPost] [-priority N] [-r/-remote schedd_name]
[-schedd-daemon-ad-file filename] [-schedd-address-file filename]
[-RescueFile filename] [-terse]

like… yes, that's a lot of brackets. she's thorough. 💅

💫 DESCRIPTION 💫

Automatically produce and place a scheduler universe job to execute HTCondor's DAGMan workflow manager on a list of specified Directed Acyclic Graphs (DAGs) of HTCondor jobs. Extensive documentation is in the HTCondor DAGMan Workflows Section.

so basically: you hand her your DAG files, she builds the job and yeets it at the scheduler universe so DAGMan can run the whole workflow for you. iconic behavior, no notes. 🦋

✨ OPTIONS ✨

  • -help — Display usage information.
  • -version — Display version information.
  • -no_submit — Produce the HTCondor submit description file for DAGMan, but do not submit DAGMan as an HTCondor job. (she makes the paperwork, she just doesn't file it 🫶)
  • -v/-verbose — Cause condor_submit_dag to give verbose error messages.
  • -terse — Display only the resulting DAGMan job id on success.
  • -f/-force — Don't fail to place DAGMan job to AP if previous execution files are discovered. Previous execution files (except the *.dagman.out) are cleaned up and old rescue files are renamed.
  • -dagman path — Specify a path to a condor_dagman executable to be executed rather than the installed one discovered in user's path.
  • -MaxIdle N — Set the maximum number of idle jobs allowed before placing more jobs to the AP. If not specified then this is set to the value of DAGMAN_MAX_JOBS_IDLE.
NOTE: condor_dagman can place jobs beyond this threshold in a single job placement cycle, but won't place more jobs until the detected number of idle jobs drops below the specified threshold again.
  • -MaxJobs N — Set the maximum number of nodes that have placed a list of jobs to the AP at any given moment. If not specified then this is set to the value of DAGMAN_MAX_JOBS_SUBMITTED.
  • -MaxPre N — Set the maximum number of PRE scripts being executed at any given moment. If not specified then this value is set to DAGMAN_MAX_PRE_SCRIPTS.
  • -MaxPost N — Set the maximum number of POST scripts being executed at any given moment. If not specified then this value is set to DAGMAN_MAX_POST_SCRIPTS.
  • -MaxHold N — Set the maximum number of HOLD scripts being executed at any given moment. If not specified then this value is set to DAGMAN_MAX_HOLD_SCRIPTS.
  • -notification value — Set the e-mail notification for DAGMan itself.
  • -r/-remote schedd_name — Specify a condor_schedd on a remote machine to place the DAGMan job.
NOTE: Since DAGMan expects all necessary files to be present in it's working directory, all files used by DAGMan need to be present on the remote machine via some method like a shared filesystem. (yes bestie, shared filesystem, don't skip this part 🙈)
  • -debug level — Set the DAGMan debug log level (see below for levels).
  • -UseDagDir — Inform DAGMan to execute each specified DAG from their respective directories.
  • -outfile_dir path — Specify the path to a directory for DAGMan's *.dagman.out debug log to be written.
  • -config filename — Specify an HTCondor configuration file to a specific DAGMan instance's execution.
NOTE: Only one configuration file can be specified which will cause a failure if used in conjunction with the CONFIG command.
  • -Lockfile filename — Path to a file to write DAGMan's process information. This prevents other DAGMan processes executing the same DAG(s) from being executed in the same directory.
  • -insert_sub_file filename — Specify a file containing JDL submit commands to be inserted in the produced DAGMan job submit description (*.condor.sub).
NOTE: Only one extra submit description file can be specified. So, the specified file will override any file specified via DAGMAN_INSERT_SUB_FILE
  • -a/-append command — Specify JDL commands to insert into the produced DAGMan job submit description (*.condor.sub). Can be used multiple times to add multiple JDL commands. Commands with spaces in them must be enclosed in double quotes.
NOTE: JDL commands specified via -append take precedence over any commands added via -insert_sub_file or DAGMAN_INSERT_SUB_FILE.
  • -batch-name name — Set the JobBatchName to name for DAGMan and all jobs managed by DAGMan.
  • -AutoRescue <0|1> — Automatically detect rescue DAG files upon startup and rescue from the most recent (highest number) rescue file discovered. 0 is False. Default 1 is True.
  • -DoRescueFrom N — Specify a specific rescue number to locate and restore state from.
  • -RescueFile filename — Specify a specific rescue file by name to restore state from.
  • -load_save filename — Specify a specific SAVE_POINT_FILE to restore state from. If provided a path DAGMan will attempt to read that file following that path. Otherwise, DAGMan will check for the file in the DAG's save_files sub-directory.
  • -DoRecovery — Inform DAGMan to startup in recovery mode (restore state from *.nodes.log).
  • -AllowVersionMismatch — Allow a version difference between condor_dagman itself and the *.condor.sub file produced by condor_submit_dag.
WARNING: This option should only be used if absolutely necessary because version mismatches can cause subtle problems when running DAGMan. (bestie I am BEGGING you, only if there's literally no other way 😭)
  • -no_recurse — Don't recursively pre-produce SUBDAG's submit description files prior to placing root DAG to the AP.
  • -do_recurse — Recursively pre-produce all SUBDAG's submit description files prior to placing the root DAG to the AP.
  • -update_submit — Don't treat an existing DAGMan submit description file (*.condor.sub) as an error; rather update the file while preserving the -maxjobs, -maxidle, -maxpre, and -maxpost options (if specified).
  • -import_env — Inform condor_submit_dag to import the current shell environment into the produced DAGMan submit description file's environment command.
  • -include_env variable[,variable...] — Specify a comma separated list of environment variables to add to the produced DAGMan submit description file's getenv command.
  • -insert_env key=value[;key=value...] — Specify a delimited string of key=value pairs to explicitly set into the produced DAGMan submit description file's environment command. If the same key is specified multiple times then the last occurrences value takes precedence.
NOTE: The base delimiter is a semicolon that can be overridden by setting the first character in the string to a valid delimiting character.
$ condor_submit_dag -insert_env |foo=0|bar=1|baz=2

see? the first char | became the delimiter. she's flexible like that 💅

  • -valgrind — Run DAGMan under valgrind.
NOTE: This is option is intended for testing and development of DAGMan. The DAGMan execution speed is drastically reduced.
WARNING: Failure will occur if necessary valgrind is not installed. valgrind is only available of Linux OS.
  • -DontAlwaysRunPost — Always execute POST scripts even upon failure of any PRE scripts.
  • -AlwaysRunPost — Only execute POST scripts after a node's list of jobs have completed (Success or Failure). Default.
  • -priority N — Set the minimum JobPrio for jobs managed by DAGMan.
  • -schedd-daemon-ad-file filename — Specifies a full path to a daemon ad file for a specific condor_schedd to place the DAGMan job.
  • -schedd-address-file filename — Specifies a full path to an address file for a specific condor_schedd to place the DAGMan job.
  • -suppress_notification — Suppress email notifications for all jobs managed by DAGMan.
  • -dont_suppress_notification — Allow email notifications for any jobs managed by DAGMan that have notifications specified. Default.
  • -SubmitMethod value — Specify how DAGMan will place managed jobs to the AP (see values below).

💬 GENERAL REMARKS 💬

NOTE: every single command line flag is case insensitive, bestie. -FORCE, -force, -FoRcE… the parser does not care, it's an icon like that. 💅

Some of the command line options also have matching configuration values, okay? But here's the tea — if you pass a value on the command line, that value wins. Command line takes precedence over anything you configured. No negotiations. 👑

And some of the command line options get passed down to DAGMan itself, to use when it's executing SUBDAGs. So your settings travel. ✨

🧸 Debug Level Values

These are the levels you can hand to the debug option, and what each one actually gives you:

  • 0 — Never produce output, except for usage info. Silence. Absolutely nothing else. 🙈
  • 1 — Very quiet. Only outputs severe errors. Like, severe severe. 😭
  • 2 — Normal output and error messages. 👌
  • 3 — (Default) Print warnings. This is the one you get if you say nothing at all. 🎀
  • 4 — Internal debugging output. 🔍
  • 5 — Outer loop debugging. 🌀
  • 6 — Inner loop debugging. 🌈
  • 7 — Output parsed DAG input lines. Every line the parser ate, shown to you. kinda iconic ngl 📜

🚀 Submit Method Values

How the job(s) actually get submitted:

  • 0 — Run condor_submit. Classic route. 🛣️
  • 1 — Direct place job(s) to local condor_schedd. Straight to the source, no middleman. 🔥

🚦 EXIT STATUS 🚦

  • 0 — Success. Slay. 🎉
  • 1 — Failure. Not slay. 💀

🌟 EXAMPLES 🌟

Okay bestie, here's every example, and they're all giving "just copy me":

Execute a single DAG:

$ condor_submit_dag sample.dag

Execute a single DAG that has successfully completed once already:

$ condor_submit_dag -force sample.dag

Execute a DAG with a max threshold of 10 idle jobs:

$ condor_submit_dag -maxidle 10 sample.dag

Execute a DAG with a max limit of 10 PRE scripts and 5 POST scripts executing concurrently:

$ condor_submit_dag -maxpre 10 -maxpost 5 sample.dag

Execute multiple DAGs:

$ condor_submit_dag first.dag second.dag

Execute multiple DAGs in their respective directories:

$ condor_submit_dag -usedagdir subdir1/first.dag subdir2/second.dag

Execute DAG and notify user once DAG is complete:

$ condor_submit_dag -notification complete sample.dag

Execute DAG with a custom batch name:

$ condor_submit_dag -batch-name my-awesome-dag sample.dag

Execute DAG and restore state from specific rescue file:

$ condor_submit_dag -RescueFile simple.dag.rescue008 simple.dag

Execute DAG and restore state from specific rescue number 8:

$ condor_submit_dag -dorescuefrom 8 sample.dag

Execute DAG and restore state from save file post-analysis.save:

$ condor_submit_dag -load_save post-analysis.save sample.dag

Execute DAG and suppress all job e-mail notifications:

$ condor_submit_dag -suppress_notification sample.dag

👀 SEE ALSO 👀

None. 😶 There is nothing else to see. Just this. Just us. 💜

🖥️ AVAILABILITY 🖥️

Linux, MacOS, Windows — she runs everywhere, she's versatile like that. 🦋

✍️ AUTHOR ✍️

HTCondor Team — the whole squad, collectively, as one. 🫶

1990-2026, Center for High Throughput Computing, Computer Sciences Department, University of Wisconsin-Madison, Madison, WI, US. Licensed under the Apache License, Version 2.0.

💖 and that's the whole page bestie!! 💖

This one came from the condor_submit_dag source, dated Oct 16, 2026 — go forth and submit those DAGs, you absolute main character. periodt. 👑💅🌸

📦 where this page comes from