tup(1)

📦
version 0.8 · 4 versions
, in arch rolling, nixos 26.05, nixpkgs unstable, opensuse-leap 16.0

💅 tup(1) — the updater 💅

okay bestie, welcome to the tup manual, where we're talking about tup(1) — the updater. It's giving build-system main character energy, and we're gonna go through every single flag because you deserve to know. 🫶

🌸 NAME 🌸

tup — the updater. That's the whole name. Short, iconic, no notes. ✨

✨ SYNOPSIS ✨

tup [--debug-sql] [--debug-fuse] [SECONDARY_COMMAND] [ARGS]

That's the shape of the command, bestie. Keep it close.

🦋 DESCRIPTION 🦋

Tup is a file-based build system. You should use tup if you are developing software which needs to be translated from input files (written by you) into a different output format. For example, if you are writing a C program, you could use tup to determine which files need to be recompiled, archived, and linked based on a dependency graph. Dependency graph girlies, rise. 💖

Tup has no domain specific knowledge. You must tell tup how to build your program, such as by saying that all .c files are converted to .o files by using gcc. This is done by writing one or more Tupfiles. No guessing, no mind-reading — you gotta spell it out. ✨

👑 PRIMARY COMMAND: tup 👑

You can do all of your development with just tup, along with writing Tupfiles. Also peep the INI FILE, OPTIONS FILES and TUPFILES sections below — they're part of the lore.

tup [<output_1> ... <output_n>]

Updates the set of outputs based on the dependency graph and the current state of the filesystem. If no outputs are specified then the whole project is updated. This is what you run every time you make changes to your software to bring it up-to-date. You can run this anywhere in the tup hierarchy, and it will always update the requested output. By default, the list of files that are changed are determined by scanning the filesystem and checking modification times. For very large projects this may be slow, but you can skip the scanning time by running the file monitor (see SECONDARY COMMANDS for a description of the monitor). 🦋

  • -jN — Temporarily override the updater.num_jobs option to N. This will run up to N jobs in parallel, subject to the constraints of the DAG. Eg: tup -j2 will run up to two jobs in parallel, whereas tup will run up to updater.num_jobs in parallel. See the option secondary command below.
  • --verbose — Causes tup to display the full command string instead of just the pretty-printed string for commands that use the ^ TEXT^ prefix.
  • --quiet — Temporarily override the display.quiet option to 1. See the option secondary command below.
  • -k, --keep-going — Temporarily override the updater.keep_going option to 1. See the option secondary command below.
  • --no-keep-going — Temporarily override the updater.keep_going option to 0. See the option secondary command below.
  • --no-scan — Do not scan the project for changed files. This is for internal tup testing only, and should not be used during normal development. ⚠️
  • --no-environ-check — Do not check for updates to the environment variables exported to sub-processes. Instead, the environment variables will be used from the database. This is used by the monitor in autoupdate/autoparse mode so that the most recent environment variables are used, rather than the settings when the monitor was initialized.
  • -d — Output debug log to screen.
  • --debug-run — Output the :-rules generated by a run-script. See the run ./script args feature in the TUPFILES section.
  • --debug-logging — Save some debug output and build graphs in .tup/log. Graphs are rotated on each invocation with --debug-logging.

🦋 SECONDARY COMMANDS 🦋

These commands are used to modify the behavior of tup or look at its internals. You probably won't need these very often. Secondary commands are invoked as:

tup [--debug-sql] [--debug-fuse] SECONDARY_COMMAND
[SECONDARY_COMMAND_ARGS]

🌸 init [directory] 🌸

Creates a .tup directory in the specified directory and initializes the tup database. If a directory name is unspecified, it defaults to creating .tup in the current directory. This defines the top of your project, as viewed by tup. Everything in the current directory and its subdirectories are known as the "tup hierarchy". 👑

If you want to avoid having to run tup init every time you clone your repository to a new location, you can instead create an empty file called Tupfile.ini in the project root. If tup is run without the .tup directory present, it will automatically create one for you as a sibling of the Tupfile.ini file, as if you had run tup init in that directory. If there are multiple Tupfile.ini files in parent paths, the one highest up the tree will be chosen. 💖

  • --no-sync — Sets the db.sync option of the project to 0. By default the db.sync option will be set to 1. This flag is mostly used for test cases, but you could also use this in any environment where you want to script tup and use no synchronization by default.

🧚 refactor / ref 🧚

The refactor command can be used to help refactor Tupfiles. This will cause tup to run through the parsing phase, but not execute any commands. If any Tupfiles that are parsed result in changes to the database, these are reported as errors. For example, we may have the following simple Tupfile:

: foreach *.c |> gcc -c %f -o %o -Wall |> %B.o

After an initial tup, we decide that we want to move the -Wall to a variable called CFLAGS. The Tupfile now looks like:

CFLAGS = -Wall
: foreach *.c |> gcc -c %f -o %o $(CFLAGS) |> %B.o

Running tup refactor will cause tup to parse the Tupfile, and if we made any mistakes, an error message will be displayed. The Tupfiles can then be modified to fix those errors, and keep running tup refactor until all Tupfiles are parsed successfully. 🎀

Errors are reported for adding or removing any of the following: commands, inputs that are generated files, outputs, the .gitignore directive, input or output <groups>, and directory level dependencies. Otherwise, you are able to move any strings out to $-variables, !-macros, and the like, so long as the end-result of the set of :-rules is the same. Refactor era, but make it safe. ✨

🦋 monitor 🦋

LINUX ONLY — this one is Linux-only, bestie. Starts the inotify-based file monitor. The monitor must scan the filesystem once and initialize watches on each directory. Then when you make changes to the files, the monitor will see them and write them directly into the database. With the monitor running, tup does not need to do the initial scan, and can start constructing the build graph immediately. The "Scanning filesystem..." time from tup is approximately equal to the time you would save by running the monitor. When the monitor is running, a tup with no file changes should only take a few milliseconds (on my machines I get about 2 or 3ms when everything is in the disk cache). If you restart your computer, you will need to restart the monitor. 💀

The following arguments can be given on the command line. Any additional arguments not handled by the monitor will be passed along to the updater if either monitor.autoupdate or monitor.autoparse are enabled. For example, you could run the monitor as tup monitor -f -a -j2 to run the monitor in the foreground, and automatically update with 2 jobs when changes are detected. See also the option secondary command below. 🫶

  • -d — Enable debugging of the monitor process.
  • -f, --foreground — Temporarily override the monitor.foreground option to 1. The monitor will run in the foreground (don't daemonize).
  • -b, --background — Temporarily override the monitor.foreground option to 0. The monitor will run in the background (daemonize).
  • -a, --autoupdate — Temporarily override the monitor.autoupdate option to 1. This will automatically run an update when file changes are detected.
  • -n, --no-autoupdate — Temporarily override the monitor.autoupdate option to 0. This will prevent the monitor from automatically running an update when file changes are detected.
  • --autoparse — Temporarily override the monitor.autoparse option to 1. This will automatically run the parser when file changes are detected.
  • --no-autoparse — Temporarily override the monitor.autoparse option to 0. This will prevent the monitor from automatically running the parser when file changes are detected.

💀 stop 💀

Kills the monitor if it is running. Basically it saves you the trouble of looking up the PID and killing it that way. No cap, it's the polite little shortcut. ✨

🎀 variant foo.config [bar.config] [...] 🎀

For each argument, this command creates a variant directory with tup.config symlinked to the specified config file. For example, if a directory contained several variant configurations, one could easily create a variant for each config file:

$ ls configs/
bar.config
foo.config
$ tup variant configs/*.config
tup: Added variant 'build-bar' using config file 'configs/bar.config'
tup: Added variant 'build-foo' using config file 'configs/foo.config'

This is equivalent to the following:

$ mkdir build-bar
$ ln -s ../configs/bar.config build-bar/tup.config
$ mkdir build-foo
$ ln -s ../configs/foo.config build-foo/tup.config

For projects that commonly use several variants, the files in the configs/ directory could be checked in to source control. Each developer would run the tup variant after tup init during the initial checkout of the software. Variants can also be created manually by making a build directory and creating a tup.config file in it (see the VARIANTS section). This command merely helps save some steps, so that you don't have to make each build directory and tup.config symlink manually. 🌈

🗄️ dbconfig 🗄️

Displays the current tup database configuration. These are internal values used by tup. iykyk, it's internals o'clock. 🔍

🌈 options 🌈

Displays all of the current tup options, as well as where they originated. Receipts for every setting, basically. 💅

…and for all the deets on every available option and how to set them, go peek at the OPTIONS FILES section down below. 🫶

graph — the ✨vibes map✨ of your build

graph [--dirs] [--ghosts] [--env] [--combine] [--stickies] [<output_1> ... <output_n>]

Okay so graph prints a graphviz .dot format graph of the tup database straight to stdout. By default it only shows you the parts of the graph that have changes — like, the main character nodes only. 💅

If you pass extra arguments, tup assumes those are files you want graphed. And heads up: this thing operates directly on the tup database, so unless your file monitor is running, you probs wanna run tup scan first. Ngl this is mostly a debugging-tup situation — you may or may not find it helpful for staring at the structure of your program. No promises, bestie. 🙈

  • --dirs — temporarily overrides the graph.dirs option to 1. Shows the directory nodes and Tupfiles. 🎀
  • --ghosts — temporarily overrides the graph.ghosts option to 1. Shows ghost nodes (aka files that a command tried to access but that don't actually exist — spooky 👻).
  • --env — temporarily overrides the graph.environment option to 1. Shows the environment variables, like PATH.

todo — the "what's next" list

todo [<output_1> ... <output_n>]

This one prints out the next steps in the tup process that'll execute when updating the outputs you give it. No outputs specified? Then it prints the steps needed to update the whole project. 🫡

Similar to the upd command, todo will automatically scan the project for file changes if a file monitor isn't running.

  • --no-scan — do not scan the project for changed files. ⚠️ This one is for internal tup testing only, and should not be used during normal development. Periodt.
  • --verbose — makes tup display the full command string instead of just the pretty-printed string, for commands that use the ^ TEXT^ prefix. iykyk ✨

generate — build script, but make it portable

generate [--config config-file] [--builddir directory] script.sh (or script.bat on Windows) [<output_1> ... <output_n>]

The generate command parses all your Tupfiles and creates a shell script that can build the program without running in a tup environment. The expected usage is in continuous integration environments that aren't compatible with tup's dependency checking (eg: if FUSE is not supported 🫠). On Windows, if the script filename has a ".bat" extension, the output will be a batch script instead of a shell script. Slay, cross-platform. 🌈

Like, for example:

git clone ... myproj
cd myproj
tup generate build.sh
./build.sh
# Copy out build artifacts / logs here
git clean -f -x -d

The shell script does not work incrementally, so it's effectively a one-time use — a one-hit wonder, if you will. 😭 You'll need to clean up the tree before the next tup generate and script invocation.

By default, the top-level tup.config file is used to define the configuration variables. Optionally, a separate configuration file can be passed in with the --config argument.

If a --builddir argument is specified, the shell script is generated as if it were a variant build, and all build artifacts are placed in the directory you specified. Otherwise, the default behavior is to place all build artifacts in-tree. 🫶

⚠️ The build directory must not already exist in the tree. Not even a little.

After the script name, individual output files can be listed in order to limit the scope of the shell script to only build those outputs. If no outputs are listed, the shell script will build all outputs. Easy.

compiledb — for the LSP girlies 💖

The compiledb command creates a compile_commands.json file for use with third-party tools that use that file as a reference to figure out how files are compiled.

The file is created at the top of the tup hierarchy, or — if you're using variants — each variant gets its own compile_commands.json created at the root of the variant. 🎀

You must annotate the files you want to export with the ^j flag. For example, you could add the ^j flag to your compiler invocations like so:

: foreach *.c |> ^j^ gcc -c %f -o %o |> %B.o

Tup automatically parses any out-of-date Tupfiles (equivalent to tup parse) before generating the database. Iconic.

💀 Due to the expense of updating compile_commands.json with large projects, the file is not automatically updated when Tupfiles change. You must re-run tup compiledb manually when you want an updated version. Sorry bestie, that's just how it is.

To automatically pull the latest commandline directly from the database instead of using compile_commands.json as an intermediary, see the commandline command. 👀

See also:

https://clang.llvm.org/docs/JSONCompilationDatabase.html

commandline — the direct line to the database

commandline <file_1> [<file_2> ... <file_n>]

The commandline command prints out the information needed to compile the filename you gave it as an argument. The info gets printed to stdout in the same JSON format as the compile_commands.json file from tup compiledb. 😌

But note the difference: compared to the compiledb command, tup commandline does not automatically parse Tupfiles before attempting to find the command. This simply queries the database directly. Zero extra steps, zero extra vibes.

For an example of how to use this, see the .ycm_extra_conf.py file located at the root of the tup git repository:

https://github.com/gittup/tup/

varsed — the @VARIABLE@ replacer 🦋

The varsed command is used as a subprogram in a Tupfile — you would not run it manually at the command-line, bestie. It's used to read one file, and replace any variable references, and write the output to a second file.

Variable references are of the form @VARIABLE@, and are replaced with the corresponding value of the @-variable. ✨

For example, if foo.txt contains:

The architecture is set to @ARCH@

And you have a :-rule in a Tupfile like so:

: foo.txt |> tup varsed %f %o |> foo-out.txt

Then on an update, the output file will be identical to the input file, except the string @ARCH@ will be replaced with whatever CONFIG_ARCH is set to in tup.config. Chef's kiss. 👑

And here's the cute part: the varsed command automatically adds the dependency from CONFIG_ARCH to the particular command node that used it — so if CONFIG_ARCH changes, the output file will be updated with the new value. It just knows. 🧚

scan — you probably don't need this

You shouldn't ever need to run this, unless you want to make the database reflect the filesystem before running tup graph. 🙈

scan is called automatically by upd if the monitor isn't running. So like… it's handled. Don't stress. 🌸

upd — the legacy one

Legacy secondary command. Calling tup upd is equivalent to simply calling tup. That's it, that's the whole vibe. 💜

🌈 INI FILE 🌈

The tup command can be run from anywhere in the source tree. It uses the info from all Tupfiles (see the TUPFILES section) as well as its own dynamic database, which is maintained in the .tup directory located at the root of the project. 🫶

The .tup directory can be created manually with the tup init command — or you can have it run automatically by adding an empty Tupfile.ini file at the root of the project's version control repository. Set it and forget it, honestly.

🎀 Fun fact: the contents of the Tupfile.ini file are ignored. Like, entirely. Don't bother writing anything in there.

🎀 OPTIONS FILES 🎀

ok so tup has a whole bunch of config files and we're gonna talk about them 💅 here's the thing tho — these files affect how tup the program behaves, not tup the build system. like, changing any of these options should not affect the end result of a successful build, it just changes how tup gets there (e.g. how many compile jobs run in parallel). vibe check: same destination, different road trip 🚗

The options are read in this precedence order (top wins, bestie):

  command-line overrides (eg: -j flag passed to 'tup')
  .tup/options file
  ~/.tupoptions file
  /etc/tup/options file
  tup's compiled in defaults

On Windows, the options files are read in like this:

  command-line overrides
  .tup/options file
  tup.ini in the Application Data path (usually C:\ProgramData\tup\tup.ini)
  tup's compiled in defaults

want the exact list of paths on your platform? just type tup options and it will spill the tea ☕

All these files use the same .ini-style syntax, no cap. A section header lives inside square brackets, like so:

[updater]

After the section header comes one or more variable definitions, shaped like variable = value. Comments start with a semi-colon and keep going to the end of the line. All the variable definitions can be set to integers. For boolean flags, "true"/"yes" and "false"/"no" are literally synonyms for 1 and 0, respectively. For example, if your .tup/options file contains:

[updater]
     num_jobs = 2
     keep_going = true

Then tup will default to 2 jobs, and have the updater.keep_going flag set. Iconic. ✨

The options below are all written in section.variable form — so to set db.sync you'd need a [db] section followed by sync = 0, for example. The defaults listed here are the compiled-in defaults, iykyk.

  • db.sync (default '1') — set to '1' if the SQLite synchronous feature is enabled. When it's enabled, the database gets properly synchronized to the disk in a way that it's always consistent. When it's disabled it runs faster, since writes are left chilling in the disk cache for a while before being written out — but bestie, if your computer crashes before everything gets written out, the tup database may become corrupted 😭. See http://www.sqlite.org/pragma.html for more info.
  • updater.num_jobs (defaults to the number of processors on the system) — set this to the max number of commands tup will run simultaneously. The default is dynamically determined to be the number of processors on your system. If updater.num_jobs is greater than 1, commands only run in parallel if they're independent. See also the -j option. 🏃‍♀️💨
  • updater.keep_going (default '0') — set to '1' to keep building as much as possible even when errors show up. Anything dependent on a failed command won't be executed, but the other independent commands will be. The default is '0', which makes tup stop after the first failed command. See also the -k option.
  • updater.full_deps (defaults to '0') — set to '1' to track dependencies on files outside of the tup hierarchy. The default '0' only tracks dependencies within the tup hierarchy. Like, if you want all your C files re-compiled when gcc gets updated on your system, you should set this to '1'. On Linux and OSX, using full dependencies requires that the tup binary is suid as root so it can run sub-processes in a chroot environment. Alternatively on Linux, if your kernel supports user namespaces, then you don't need to make the binary suid 🫶. Heads up: if this value is set to '1' from '0', tup will rebuild the entire project. Disabling this option when it was previously enabled doesn't require a full rebuild, but it does take some time since the nodes representing external files get cleared out. NOTE: this does not currently work with ccache or other programs that may write to external files, because of issues with locking. This may be fixed in the future 🙈.
  • updater.warnings (defaults to '1') — set to '0' to disable warnings about writing to hidden files. Tup doesn't track files that are hidden. If a sub-process writes to a hidden file, then by default tup displays a warning that this file was created. Disable this option and those warnings don't get displayed. Hidden filenames (or directories) include: ., .., .tup, .git, .hg, .bzr, .svn.
  • display.color (default 'auto') — set to 'never' to disable ANSI escape codes for colored output, or 'always' to always use ANSI escape codes for colored output. The default is 'auto', which uses colored output if stdout is connected to a tty, and uses no colors otherwise (ie: if stdout is redirected to a file). 🌈
  • display.width (defaults to the terminal width) — set to any number 10 or larger to force a fixed width for the progress bar. This is assumed to be the total width, some of which is used for spacing, brackets, and the percentage complete. If this value is less than 10, the progress bar is disabled.
  • display.progress (defaults to '1' if stdout is a TTY) — set to '1' to enable the progress bar, or '0' to turn it off. By default it's enabled if stdout is a TTY, and disabled if stdout is not a TTY.
  • display.job_numbers (default '1') — set to '0' to avoid displaying the "N) " string before the results of a job. The default is to display this number.
  • display.job_time (default '1') — set to '0' to avoid displaying the runtime of a job along with the results. The default is to display the runtime. Note that the runtime displayed includes the time tup takes to save the dependencies — so this runtime will likely be larger than the runtime you'd see executing the same job manually in the shell. ⏱️
  • display.quiet (default '0') — set to '1' to prevent tup from displaying most output. Tup will still display a banner, plus output from any job that writes to stdout/stderr, or any job that returns a non-zero exit code. The progress bar is still displayed — see also display.progress for really quiet output. 🤫
  • monitor.autoupdate (default '0') — set to '1' to automatically rebuild if a file change is detected. This only has an effect if the monitor is running. The default '0' means you have to type tup yourself when you're ready to update.
  • monitor.autoparse (default '0') — set to '1' to automatically run the parser if a file change is detected. This is similar to monitor.autoupdate, except the update stops after the parser stage — no commands run until you manually type tup. This only has an effect if the monitor is running. Note that if both autoupdate and autoparse are set, then autoupdate takes precedence 👑
  • monitor.foreground (default '0') — set to '1' to run the monitor in the foreground, so control won't return to the terminal until the monitor is stopped (either by ctrl-C in the controlling terminal, or by running tup stop in another terminal). The default '0' means the monitor runs in the background.
  • graph.dirs (default '0') — set to '1' and the tup graph command will show the directory nodes and their ownership links. Tupfiles are also displayed, since they point to directory nodes. By default directories and Tupfiles aren't shown, since they can clutter the graph in some cases and aren't always useful.
  • graph.ghosts (default '0') — set to '1' to show ghost nodes 👻. Some commands may try to read from many files that don't exist, which causes ghost nodes to be created. By default ghosts aren't shown, so the graph is easier to understand.
  • graph.environment (default '0') — set to '1' to show the environment nodes (such as PATH) and their dependencies. By default the environment variables aren't shown, since nearly everything will depend on PATH.
  • graph.combine (default '0') — set to '1' to try to combine similar nodes in the graph. For example, instead of showing 10 separate compilation commands that all have one .c file input and one .o file output, this combines them into one command so you can more easily see the whole structure of the graph. By default all nodes are shown separately.

🌈 TUPFILES 🌈

okay so first things first: you must create a file called Tupfile anywhere in the tup hierarchy where you want to create an output file based on the input files. The input files can be literally anywhere else in the tup hierarchy, but the output file(s) must be written in the same directory as the Tupfile 📁✨

: [foreach] [inputs] [ | order-only inputs] |> command |> [outputs] [ |
extra outputs] [<group>] [{bin}]

The :-rules are the primary means of creating commands, and you can tell they're :-rules because the : character appears in the first column of the Tupfile. The syntax is supposed to look somewhat like a pipeline — the input files on the left go into the command in the middle, and the output files come out on the right. it's giving plumbing 🔧

foreach

This is either the actual string "foreach", or it's empty. The whole distinction is in how many commands get generated when there are multiple input files. If "foreach" is specified, one command is created for each file in the inputs section. If it's not specified, one command is created containing all of the files in the inputs section. For example, the following Tupfiles are equivalent:

okay so boom — here’s the foreach glow-up in action, bestie:

# Tupfile 1
: foo.c |> gcc -c foo.c -o foo.o |> foo.o
: bar.c |> gcc -c bar.c -o bar.o

# Tupfile 2
: foreach foo.c bar.c |> gcc -c %f -o %o |> %B.o

Also, using foreach unlocks the %e flag (peep below for the deets).

✨ inputs ✨

The input files for the command, periodt. An input file can live anywhere in the tup hierarchy, and you write it relative to the current directory. Input files are what affect the %-flags (see below). Wildcarding is supported within a directory using the SQLite glob function. The special glob characters are *, ?, and []. For example, *.c matches any .c file, fo?.c matches any 3-character .c file that has f and o as the first two characters, and fo[xyz].c matches fox.c, foy.c, and foz.c. Globbing does not match directories, so src/*.c will work, but */*.c will not. no cap.

Any inputs starting with ^ are Perl-compatible regular expressions used to exclude files that were matched in a previous glob. For example, you could compile all *.c files except main.c with this little fragment:

: foreach *.c ^main.c |> ... |>

These input exclusions are handled at parser time, and they’re distinct from the dependency exclusions in the outputs section. iykyk.

🌸 order-only inputs 🌸

These are also used as inputs for the command, but they will not appear in any of the %-flags except %i. They’re separated from regular inputs by the | character. In effect, these let you specify extra inputs to a command that shouldn’t show up on the command line. Globbing is supported just like in the inputs section. For example, one use for them is specifying auto-generated header dependencies:

: |> echo "#define FOO 3" > %o |> foo.h
: foreach foo.c bar.c | foo.h |> gcc -c %f -o %o |> %B.o

This adds the foo.h dependency to the gcc commands for foo.c and bar.c, so tup knows to generate the header before trying to compile. The foreach command iterates over the regular inputs (here, foo.c and bar.c), not the order-only inputs (foo.h). If you forget to add such a dependency, tup will report an error when the command is executed. Note that the foo.h dependency is only listed here because it’s created by another command — normal headers do not need to be specified. 💅

🎀 command 🎀

The command string that will be passed to the system(3) call by tup. This command is allowed to read from any file specified as an input or order-only input, as well as any other file in the tup hierarchy that is not the output of another command. In other words, a command cannot read from another command’s output unless it’s specified as an input. This restriction is literally what allows tup to be parallel safe. Additionally, the command must write to all of the output files specified by the outputs section, if any.

When executed, the command’s file accesses are monitored by tup to ensure they conform to these rules. Any files opened for reading that were generated from another command but not specified as inputs are reported as errors. Similarly, any files opened for writing that are not specified as outputs are reported as errors. All files opened for reading are recorded as dependencies to the command. If any of these files change, tup will re-execute the command during the next update. Note that if an input listed in the Tupfile changes, it does not necessarily cause the command to re-execute, unless the command actually read from that input during the prior execution. Inputs listed in the Tupfile only enforce ordering among the commands, while file accesses during execution determine when commands are re-executed. the way that works is lowkey iconic.

A command string can begin with the special sequence ^ TEXT^, which tells tup to only print TEXT instead of the whole command string when the command is being executed. This saves the effort of using echo to pretty-print a long command. The short-display behavior can be overridden by passing the --verbose flag to tup, which causes tup to display the actual command string instead of TEXT. The space after the first ^ is significant. Any characters immediately after the first ^ are treated as flags. See the ^-flags section below for details. For example, this command will print CC foo.c when executing system(gcc -c foo.c -o foo.o):

: foo.c |> ^ CC %f^ gcc -c %f -o %o |> foo.o

A command string can also begin with the special character !, in which case the !-macro specified will be substituted in for the actual command. See the !-macro definition later. Commands can also be blank, which is useful to put all the input files in a {bin} for a later rule. 🫶

🦋 outputs 🦋

The outputs section specifies the files that will be written to by the command. Only one command can write to a specific file, but a single command can output multiple files (such as how a bison command will output both a .c and .h file). The output can use any %-flags except %o. Once a file is specified in an output section, it’s put into the tup database. Any following rules can use that file as an input, even if it doesn’t exist in the filesystem yet. ✨

Outputs starting with ^ are used to ignore dependencies on files. These exclusions are Perl-compatible regular expressions that are stored in the database. Any files that are read or written to when the sub-process executes that match these patterns are ignored by tup. For example, if your compiler writes to a license_file.txt, you could instruct tup to ignore dependency errors on this file like so:

: foreach *.c |> ... |> %B.o ^license_file.txt

Note that this also instructs tup to ignore input dependencies on the file. If you wish to merely skip files in a glob, see the similar ^ syntax in the inputs section.

bestie, listen: an exclusion cannot be used to match a file generated by the same rule (ie: one listed as an output in the command). Doing so will result in an "Unable to exclude a generated file" error.

You should only ignore dependencies with extreme caution. It is trivial to cause tup to fail to update when it should have because it’s been instructed to ignore dependencies. not me warning you dramatically, but like… yeah. 💀

🌈 extra-outputs 🌈

The extra-outputs section is similar to the order-only inputs section. It’s separated from the regular outputs by the | character. The extra-outputs behave exactly as regular outputs, except they do not appear in the %o flag. These can be used if a command generates files whose names do not actually appear in the command line. If there is exactly one output specified by the rule, the extra-outputs section can use the %O flag to represent the basename of the output. This can be useful in extra-outputs for !-macros. 💖

<group> 👑

Output files can be grouped into global groups by specifying a <group> after the outputs but before a bin. Groups allow for order-only dependencies between folders. Note that groups are directory specific, however, so when referring to a group you must specify the path to where it’s assigned. For example, if a main project depends on the output from several submodules, you can structure Tup like so to make sure the submodules are built before the main project:

#./submodules/sm1/Tupfile
: foo.c |> gcc -c %f -o %o |> %B.o ../<submodgroup>

#./submodules/sm2/Tupfile
: bar.c |> gcc -c %f -o %o |> %B.o ../<submodgroup>

#./project/Tupfile
: baz.c | ../submodules/<submodgroup> |> gcc -c %f -o %o |> %B.o

Notice how groups are directory specific and the path is specified outside of the <>. By specifying the <submodgroup> as an order-only input, Tup will build the submodules before attempting to build the entire project. main character energy fr fr.

{bin} 🎁

Outputs can be grouped into a bin using the {bin} syntax. A later rule can use {bin} as an input to use all of the files in that bin. For example, the foreach rule will put each .o file in the objs bin, which is used as an input in the linker rule:

: foreach *.c |> gcc -c %f -o %o |> %B.o {objs}
: {objs} |> gcc %f -o %o |> program

In this case one could use *.o as the input instead, but sometimes it’s useful to separate outputs into groups even though they have the same extension (such as if one directory creates multiple binaries, using *.o wouldn’t be correct). If a {bin} is specified in the output section of multiple rules, the bin will be the union of all the outputs. You can’t remove things from a bin, and the bin disappears after the current Tupfile is parsed. obsessed with the little scoped bin moment.

^-flags 🔥

In a command string that uses the ^ TEXT^ sequence, flag characters can be placed immediately after the ^ until the first space character or closing caret. For example:

: foo.c |> ^c CC %f^ gcc --coverage %f -o %o |> foo | foo.gcno
: bar.c |> ^c^ gcc --coverage %f -o %o |> bar | bar.gcno

So in the foo.c situation, the command needs namespaces (or suid) and it's gonna show you CC foo.c. In the bar.c situation, the command also needs namespaces (or suid), but what gets displayed is the whole gcc --coverage bar.c -o bar string. 💅

Here's the full lineup of supported flag characters, bestie:

  • b — this one's the bash glow-up 🫶. The b flag makes the command run through /usr/bin/env bash -e -o pipefail -c <command> instead of the default /bin/sh -e -c <command>. So yes, you get bash extensions inside your :-rule and you get -o pipefail, which dictates that "the return value of a pipeline is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands in the pipeline exit successfully." Iconic, honestly.
  • c — the c flag says: fail the command if tup doesn't support user namespaces (that's Linux) or if it isn't suid root. Because in those cases tup runs in a degraded mode where the fake working directories are visible in the sub-processes and some dependencies may be missed 😭. So if those degraded behaviors would break a particular command in your build, slap the c flag on it so users get a clear signal that they need to add the suid bit or upgrade their kernel. Ngl this flag is just ignored on Windows. 💀
  • j — marks the command for export in the tup compiledb command 👑. If you're into compile_commands.json, annotate the commands you want to export with ^j and then run tup compiledb. Peep the compiledb command for the details, iykyk.
  • o — the output-comparison flag ✨. The o flag makes the command compare its new outputs against the outputs from the previous run, and any outputs that are the same won't cause dependent commands in the DAG to be executed. Like, adding this flag to a compilation command will skip the linking step if the object file hasn't changed since last time it ran. Slay. But heads up: the o flag is incompatible with the t flag 🙈.
  • s — turns off buffering of stdout/stderr for the subprocesses and enables "streaming mode" 🌊. When streaming, stdout/stderr are inherited from the tup process, so messages would typically show up on the terminal while the subprocess is running, in whatever order they're generated. Caveat time though: processes with s enabled may display messages interleaved with each other, and also with tup's progress bar or other tup messages 😬. This flag may be useful for long-running processes where you wanna see the output as it happens, though it can make for confusing logs if you use it on a bunch of noisy commands that may run in parallel.

By default, tup buffers a command's messages (think compiler warning messages) and displays them under the banner for the command after it completes. So messages from separate commands are always distinct and associated with the command that created them. 💖

  • t — makes the command's outputs transient 🦋. The outputs may be used as inputs to other commands, but after all dependent commands are executed, the transient outputs get deleted from the filesystem. Perfect for saving space when you have many stages of processing that each produce large outputs but only the final output needs to stick around. And yep, the t flag is incompatible with the o flag.

A situation where t actually makes sense in a build pipeline? Large assets that go through multiple stages of processing — say a big audio or video file with stages of effects applied, each one a separate step in the Tupfile. 🎀

In contrast, the t flag does *not* make sense for object files in a C program, even though those could theoretically be deleted after the final executable is linked. Why? Because if the object files were marked transient in that case, a change to any of the input C files would require *all* object files to be rebuilt in order to produce the executable, instead of only the single file that was changed. 💀 Lesson learned.

%-flags 💅

Inside a command string or output string, you can also use the following %-flags to substitute values from the inputs or outputs ✨. General rule: %-flags expand to all filenames in a list — like, %o expands to all output files. Wanna refer to just one file? Use %1o, %2o, etc. to grab the first or second (and so on) filename in the list. Periodt.

  • %% — expands to a single % character in the command string. Use this when you want the percent character to be interpreted by the command itself rather than by tup's parser 🫶.
  • %f — the filenames from the "inputs" section, path and extension included. Super useful in a command, since it lists each input file name with the path relative to the current directory. Like, src/foo.c gets copied over exactly as src/foo.c. Individual files can be referenced with %1f, %2f, etc. ⭐
  • %i — the filenames from the "order-only inputs" section, path and extension included. Usually order-only inputs are for filenames you don't want showing up in the command-line, but if you do need to refer to them, %i is right there. Individual files can be referenced with %1i, %2i, etc. 🌸
  • %b — like %f, but just the basename of the file, directory part stripped off. So src/foo.c becomes foo.c. Individual files: %1b, %2b, etc. 🎀
  • %B — like %b, but it strips the extension too ✂️. This is the GOAT for converting an input file into an output file of the same name but a different extension, since the output file needs to be in the same directory. So src/foo.c becomes foo. Individual files: %1B, %2B, etc.
  • %e — the file extension of the current file when used in a foreach rule 🦋. Handy for variables that can have different values based on the file's suffix. Like, you could set certain flags for assembly (.S) files that differ from .c files, then use something like $(CFLAGS_%e) to reference the CFLAGS_S or CFLAGS_c variable depending on what type of file is being compiled. So src/foo.c becomes c, while src/foo.S becomes S. 💖
  • %o — the name of the output file(s) 👑. Useful in a command so the filename you pass to a command always matches what tup thinks the output is. Note: this only works in the "command" section, not in the "outputs" section. Individual files: %1o, %2o, etc.
  • %O — the name of the output file without the extension. This only works in the extra-outputs section if there's exactly one output file specified. The use-case? Say you've got a !-macro that generates files not specified on the command line, but based on the output that's named. For example, if a linker creates a map file by taking the specified output foo.so, removing the .so and adding .map, then you may want a !-macro like so:
!ldmap = |> ld ... -o %o |> | %O.map
: foo1.o foo2.o |> !ldmap |> foo.so
  • %d — the name of the lowest level of the directory 🌈. Like, in foo/bar/Tupfile, this would be the string "bar". One place this comes in clutch is naming libraries based on the directory they live in, such as this !-macro:
!ar = |> ar crs %o %f |> lib%d.a

Using this macro in foo/bar/Tupfile would then create foo/bar/libbar.a ✨.

  • %g — the string that a glob operator matched 🌸. For example, with the files a_text.txt and b_text.txt, this rule:
: foreach *_text.txt |> foo %f |> %g_binary.bin

will output the filenames a_binary.bin and b_binary.bin. Heads up though: only the first glob expanded gets substituted in for %g. And %g is only valid when there's a single input file or foreach is used. 🙈

  • %<group> — all of the files in "group". For example:
#./submodules/sm1/Tupfile
: foo.c |> gcc -c %f -o %o |> %B.o ../<submodgroup>

#./submodules/sm2/Tupfile
: bar.c |> gcc -c %f -o %o |> %B.o ../<submodgroup>

#./project/Tupfile
: ../submodules/<submodgroup> |> echo '%f' > %o |> submodules_f.txt
: ../submodules/<submodgroup> |> echo '%<submodgroup>' > %o |> submodules_group.txt

will produce "../submodules/<submodgroup>" in submodules_f.txt, but "../submodules/sm1/foo.o ../submodules/sm2/bar.o" in submodules_group.txt 🤯. And if the input contains multiple groups with the same name but in different directories, %<group> will be expanded to all of the files in each listed group. Chef's kiss. 💅

  • var = value
  • var := value — set the $-variable var to whatever's on the right-hand side ✨ both forms are literally the same thing, they just both exist so converting old Makefiles is less of a pain. Later you reference it with $(var). Variables referenced here are always expanded immediately, which is why setting a variable to hold a %-flag makes no sense — a %-flag is only valid inside a :-rule. That said, the syntax $(var_%e) is allowed in a :-rule. Variable references do not nest, so something like $(var1_$(var2)) does not make sense. You also can't pass variable definitions on the command line or through the environment, bestie. And any reference to a variable that hasn't had its value set just returns an empty string. No drama 💅
CFLAGS = -Dfoo
: bar.c |> cc $(CFLAGS) $(other) -o %o -c %f |> %B.o

will generate the command cc -Dfoo -o bar.o -c bar.c when run. (notice the double space where $(other) was empty — that's fr fr what happens 🫶)

  • $(CONFIG_FOO) / @(FOO) — okay so any $-variable that begins with the string CONFIG_ gets automatically converted into the @-variable of the same name minus the CONFIG_ prefix 👑 in other words, $(CONFIG_FOO) and @(FOO) are interchangeable. But heads up: trying to assign a value to a CONFIG_ variable inside a Tupfile is an error, because those can only be set in the tup.config file. Periodt.
⚠️ also you might see old-school back-tick syntax when setting variables, like this:
CFLAGS += `pkg-config fuse3 --cflags`
Tup does not do any special processing for back-ticks — so the pkg-config command is not actually executed when the variable is set in this example 😭 instead it's passed verbatim to whatever uses it. So if a command later references $(CFLAGS), it'll contain the literal string `pkg-config fuse3 --cflags`, and then the shell will parse it. surprise! 🎀
  • var += value — append value to the end of whatever var currently is. If var hasn't been set yet, this is literally equivalent to a plain = statement. If var already has a value, a space gets appended to the $-variable before the new value is appended. so considerate 💖
  • $(TUP_CWD) / [lua] tup.getcwd() — the special $-variable TUP_CWD is always set to the path relative to the Tupfile currently being parsed 🌸 and it can change value when you include a file in a different directory. Example: if you include ../foo.tup, then TUP_CWD gets set to .. while parsing foo.tup. This lets foo.tup specify flags like CFLAGS += -I$(TUP_CWD), and CFLAGS will always have the -I directory where foo.tup lives, no matter whether it was included as ../foo.tup or ../../foo.tup or subdir/foo.tup ✨ For an alternative to $(TUP_CWD) when you're referring to files, see the section on &-variables below 👇
  • $(TUP_VARIANTDIR) / [lua] tup.getvariantdir() — similar to $(TUP_CWD), $(TUP_VARIANTDIR) is set to the path relative to the Tupfile currently parsed, but it points to the corresponding directory in the variant 🦋 Example of when this slays: suppose your build generates common headers into a directory called include/. A Tuprules.tup file at the level of that directory could add -I$(TUP_VARIANTDIR) to a compiler flags variable. If a variant is used, this evaluates to a path like ../build/path/to/include, where build is the name of the variant directory. If variants aren't used, TUP_VARIANTDIR is equivalent to TUP_CWD. no cap 👑
  • $(TUP_VARIANT_OUTPUTDIR) / [lua] tup.getvariantoutputdir() — $(TUP_VARIANT_OUTPUTDIR) always points to the output directory corresponding to the variant currently being parsed. Unlike $(TUP_VARIANTDIR), it does not change when you include another file like Tuprules.tup or other partial *.tup files 🧚 Without variants, this always evaluates to the string . to point to the current directory.
there are no other special $-variables yet, but to be on the safe side you should assume that all variables named TUP_* are reserved 🔒
  • &var = file
  • &var := file
  • &var += file — set the &-variable to refer to the given file or directory. the file must be a normal file, not a generated file (i.e. not an output from a :-rule) 🙈 &-variables are used to refer to files kinda like $(TUP_CWD) does, except instead of storing the relative path to the file, &-variables store tup's internal ID of the file. Which means the relative path to the file is determined when the &-variable is used, rather than when the variable is assigned like with $(TUP_CWD). &-variables can only be used in the following locations: :-rule inputs, :-rule order-only inputs, :-rule commands, include lines, and run-script lines — and you reference them later with &(var) 🎀
# Tuprules.tup
&libdir = src/lib
!cc = |> cc -I&(libdir) -c %f -o %o |> %B.o

# src/lib/Tupfile
: foreach *.c |> !cc |>
: *.o |> ar crs %o %f |> libstuff.a

# src/lib/test/Tupfile
: test_stuff.c |> !cc |>
: test_stuff.o &(libdir)/libstuff.a |> cc -o %o %f |> test_stuff

# src/Tupfile
: main.c |> !cc |> main.o
: main.o &(libdir)/libstuff.a |> cc -o %o %f |> main_app

will generate the following build.sh commands (via tup generate build.sh):

cd src/lib
cc -I. -c lib1.c -o lib1.o
cc -I. -c lib2.c -o lib2.o
ar crs libstuff.a lib1.o lib2.o
cd test
cc -I.. -c test_stuff.c -o test_stuff.o
cc -o test_stuff test_stuff.o ../libstuff.a
cd ../..
cc -Ilib -c main.c -o main.o
cc -o main_app main.o lib/libstuff.a
  • ifeq (lval,rval) — evaluates the lval and rval parameters (like, substitutes all $-variables and @-variables), then does a string comparison to see if they match 👀 if they do, all the lines between the ifeq and the following endif statement get processed; otherwise they're ignored. note that no whitespace is pruned for the values — all text between the ( and , comprises lval, and all text between the , and ) comprises rval. Which means ifeq (foo, foo) is false, while ifeq (foo,foo) is true 😭 this is for compatibility with Makefile if statements, bestie, we don't make the rules.
ifeq (@(FOO),y)
CFLAGS += -DFOO
else
CFLAGS += -g
endif
  • ifneq (lval,rval) — same as ifeq, but with the logic inverted 🔄
  • ifdef VARIABLE — tests whether the @-variable named VARIABLE is defined at all in tup.config. If so, all lines between the ifdef and the following endif statement get processed; otherwise they're ignored. Example: suppose tup.config contains:
CONFIG_FOO=n

then ifdef FOO will evaluate to true 💀 (yes, even though it's n — we're testing defined, not truthy). If tup.config doesn't exist, or doesn't set CONFIG_FOO in any way, then ifdef FOO is false.

  • ifndef VARIABLE — same as ifdef, but with the logic inverted 🔄
  • else — toggles the true/false-ness of the previous if-statement 🎚️
  • endif — ends the previous ifeq/ifdef/ifndef. note that only 8 levels of nesting if-statements is supported — plan accordingly babes 💅
  • error [message] — causes tup to stop parsing and fail, printing message to the user as an explanation 🛑 rude but honest.
  • !macro = [inputs] | [order-only inputs] |> command |> [outputs] — set the !-macro to the given command string ✨ this syntax is super similar to the :-rule, because a !-macro is basically a macro for those rules. The !-macro isn't expanded until it's used in the command string of a :-rule. So the main character use case of the !-macro is having a place to store command strings with %-flags that you reuse a bunch 🏆 for example, we could have a !cc macro in a top-level Tuprules.tup file like so:
!cc = |> ^ CC %f^ gcc -c %f -o %o |>

a Tupfile could then do as follows:

include_rules
: foreach *.c |> !cc |> %B.o

you only want to specify the output parameter in either the !-macro or the :-rule that uses it — not both 🙅‍♀️ if you specify any inputs in the !-macro, they'd usually be order-only inputs. For example, if you have a !cc rule where you're using a compiler that's been generated by tup, you can list the compiler file in the order-only list of the !-macro. The compiler file then becomes an input dependency for any :-rule that uses the macro 🔗

  • include file — reads the specified file and continues parsing almost as if that file was pasted inline in the current Tupfile 📋 only regular files are allowed to be included — attempting to include a generated file is an error. Any include statements that occur in the included file get parsed relative to the included file's directory 🦋
  • include_rules — reads in Tuprules.tup files up the directory chain 🌈 the first Tuprules.tup file is read at the top of the tup hierarchy, followed by the next subdirectory, and so on through to the Tuprules.tup file in the current directory. This way, the top-level Tuprules.tup can specify general variable settings, and subsequent subdirectories can override them with more specific settings. You'd generally put include_rules as the first line in the Tupfile 💖 also the name is a bit of a misnomer — you typically use Tuprules.tup to define variables rather than :-rules.
  • run ./script args — runs an external script with the given arguments to generate :-rules 🔥 this is an advanced feature, for when the standard Tupfile syntax is too simplistic for a complex program. The script is expected to write the :-rules to stdout. No other Tupfile commands are allowed — for example, the script can't create $-variables or !-macros, but it can output :-rules that use those features 👑 As a simple example, consider if a command must be executed 5 times, but there are no input files to use tup's foreach keyword. An external script called build.sh could be written as follows:
#! /bin/sh -e
for i in `seq 1 5`; do
     echo ": |> echo $i > %o |> $i.txt"
done

a Tupfile can then be used to get these rules:

run ./build.sh

Tup will then treat this as if a Tupfile was written with 5 lines like so:

: |> echo 1 > %o |> 1.txt
: |> echo 2 > %o |> 2.txt
: |> echo 3 > %o |> 3.txt
: |> echo 4 > %o |> 4.txt
: |> echo 5 > %o |> 5.txt

Okay so here's the ✨magic✨: the Tupfile-parsing stage is literally watched for dependencies. So any file that your script touches inside the tup hierarchy? Yeah, that makes the Tupfile get re-parsed. Obsessed with the automation ngl.

But bestie, there are some limitations, and we gotta talk about them 🫶

One: the readdir() call is instrumented to return the list of files that would be accessible at that time that the run-script starts executing. Translation? The files you see in ls on the command-line might be different from the files your script sees when it's parsed 👀. Tup essentially pretends that generated files don't exist until it parses a :-rule that lists it as an output. And note: any :-rules executed by the run-script itself are not parsed until the script executes successfully.

Two: due to some structural limitations in tup, the script cannot readdir() on any directory other than the directory of the Tupfile. In other words, a script can do for i in *.c, but not for i in sub/*.c 💀

The --debug-run flag can be passed to tup in order to show the list of :-rules that tup receives from the script. And because of that readdir() instrumentation, this may be different than the script's output when it's run manually from the command-line 🙈

🌸 preload directory

By default, a run-script can only use a readdir() (ie: use a wild-card) on the current directory. To specify a list of other allowable wild-card directories, use the preload keyword. So if your run script needs to look at *.c and src/*.c, the src directory needs to be preloaded:

preload src
run ./build.sh *.c src/*.c

🎀 export VARIABLE

The export directive adds the environment variable VARIABLE to the export list for future :-rules and run-scripts. The value for the variable comes from tup's environment, not from the Tupfile itself. So basically? You'll need to set the variable in your shell if you want to change the value used by commands and scripts. By default only PATH is exported. Windows additionally exports several variables suitable for building with the Visual Studio compiler suite.

Tup will check the exported environment variables to see if they've changed values between updates, and re-execute any commands that use those environment variables. And note — this means if PATH is changed, all commands will run again 😭 Fr fr. Example:

: |> command1 ... |>
export FOO
: |> command2 ... |>

Tup will save the current value of FOO and pass it to the environment when executing command2. If FOO has a different value during the next update, then command2 will execute again with the new value in the environment. In this example, command1 will not have FOO in its environment and will not re-execute when its value changes. Slay, scoping ✨

Note that the FOO above is passed to the environment; it is not provided as an internal variable within tup. Thus, given the following:

export FOO
: |> echo myFOO=$(FOO) envFOO=${FOO} > %o |> foo.txt

when run as $ FOO=silly tup would result in the contents of the foo.txt file being myFOO= envFOO=silly 💅 If the export FOO was removed from the Tupfile, the contents of the file would be myFOO= envFOO= because tup does not propagate environment variables unless they are explicitly exported. This helps preserve repeatable and deterministic builds — iconic behaviour, honestly 👑

If you wish to export a variable to a specific value rather than get the value from the environment, you can do that in your shell instead of through tup. For example, in Linux you can do:

: |> FOO=value command ... |>

This usage will not create a dependency on the environment variable FOO, since it is controlled through the Tupfile 🧚

🦋 import VARIABLE[=default]

The import directive sets a variable inside the Tupfile that has the value of the environment variable. If the environment variable is unset, the default value is used instead if provided. This introduces a dependency from the environment variable to the Tupfile, so that if the environment variable changes, the Tupfile is re-parsed. Example time:

import CC=gcc
: foreach *.c |> $(CC) -c %f -o %o |> %B.o

This will compile all .c files with the compiler defined in the CC environment variable. If CC is not set in the environment, it will use gcc as the default. On a subsequent build, running CC=clang tup will re-parse this Tupfile and re-build all .c files with clang instead (since the commandlines have changed) 🔥

Unlike export, the import command does not pass the variables to the sub-process's environment. In the previous example, the CC environment variable is therefore not set in the subprocess, unless export CC was also in the Tupfile. No cap 🫶

🍓 .gitignore

Tells tup to automatically generate a .gitignore file in the current directory which contains a list of the output files that are generated by tup. This can be useful if you're using git, since the set of files generated by tup matches exactly the set of files that you want git to ignore ✨

If you're using Tuprules.tup files, you may just want to specify .gitignore in the top-level Tuprules.tup, and then have every other Tupfile use include_rules to pick up the .gitignore definition. In this way you never have to maintain the .gitignore files manually. Main character energy honestly 💅

Note that you may wish to ignore other files not created by tup, such as temporary files created by your editor. In this case you will want to setup a global gitignore file using a command like git config --global core.excludesfile ~/.gitignore, and then setup ~/.gitignore with your personal list. For other cases, you can also simply add any custom ignore rules above the ##### TUP GITIGNORE ##### line 🎀

💜 #

At the beginning of a line, a # character signifies a comment. A comment line is ignored by the parser. The comment can have leading whitespaces that is also ignored. If there is any non-whitespace before a # character, then the line is not a comment. It also means that if a previous line ended with \ (line wrap) then # is interpreted as a regular symbol 👀 So watch out for that one, bestie.

🌈 TUPFILE NOTES 🌈

Variable expansion in tup is immediate in every case except for !-macros. That is, if you see a :-rule or variable declaration, you can substitute the current values for the variables. The !-macros are only parsed when they're used in a :-rule. In that case, the actual :-rule is a sort of a union between the :-rule as written and the current value of the !-macro 💫

When tup parses a Tupfile, it makes a single pass through the file, parsing a line at a time. At the end of the Tupfile, all variable, !-macro, and {bin} definitions are discarded 🗑️ The only lingering effects of parsing a Tupfile are the command nodes and dependencies that now exist in the tup database. Additionally, a .gitignore file may have been created if requested by the Tupfile 🎀

💖 @-VARIABLES 💖

@-variables are special variables in tup. They are used as configuration variables, and can be read by Tupfiles or used by the varsed command. Commands are able to read them too, but the program executed by the command has to have direct knowledge of the variables. @-variables are specified in the tup.config file at the top of the tup hierarchy or in a variant directory. For example, tup.config may contain:

CONFIG_FOO=y

A Tupfile may then read the @-variable like so:

srcs-@(FOO) += foo.c
srcs-y += bar.c
: foreach $(srcs-y) |> gcc -c %f -o %o |> %B.o

In this example, if CONFIG_FOO is set to y, then the foo.c file will be included in the input list and therefore compiled 🔥 If CONFIG_FOO is unspecified or set to some other value, foo.c will not be included. Vibe check passed or failed, your call 🌸

The @-variables can be used similar to $-variables, with the following distinctions: 1) @-variables are read-only in Tupfiles, and 2) @-variables are in the DAG, which means reading from them creates a dependency from the @-variable to the Tupfile. Therefore any Tupfile that reads @(FOO) like the above example will be reparsed if the value of CONFIG_FOO in tup.config changes 👑

The reason for prefixing with CONFIG_ in the tup.config file is to maintain compatibility with kconfig, which can be used to generate this file. iykyk 💅

Note that the syntax for tup.config is fairly strict. For a statement like CONFIG_FOO=y, tup will create an @-variable using the string starting after CONFIG_, and up to the = sign. The value is everything immediately after the = sign until the newline, but if there is a surrounding pair of quotes, they are stripped. In this example, it would set FOO to y. Note that if instead the line were CONFIG_FOO = y, then the variable FOO would be set to y 😭 Yes, spaces matter. Yes, that means FOO with a space. Be careful.

In tup.config, comments are determined by a # character in the first column. These are ignored, unless the comment is of the form:

# CONFIG_FOO is not set

In this case, the @-variable FOO is explicitly set to n 🎯

📱 @(TUP_PLATFORM)

TUP_PLATFORM is a special @-variable. If CONFIG_TUP_PLATFORM is not set in the tup.config file, it has a default value according to the platform that tup itself was compiled in. Currently the default value is one of linux, solaris, macosx, win32, freebsd or netbsd 🌈

🦋 @(TUP_ARCH)

TUP_ARCH is another special @-variable. If CONFIG_TUP_ARCH is not set in the tup.config file, it has a default value according to the processor architecture that tup itself was compiled in. Currently the default value is one of i386, x86_64, powerpc, powerpc64, ia64, alpha, sparc, riscv32, riscv64, arm64, arm or s390x ⭐

🦋 VARIANTS 🦋

Okay bestie, so tup supports variants, and this is genuinely one of the slayest features — it lets you build your project multiple times with different configurations. The most common case, fr fr, is building a release and a debug configuration with different compiler flags, but you can have any number of variants for whatever configurations you like. 💅

Each variant gets built in its own directory, distinct from each other and from the source tree. Heads up though: when building with variants, the in-tree build is disabled. No cap.

To create a variant, make a new directory at the top of the tup hierarchy and create a tup.config file in there. Like so:

$ mkdir build-default
$ touch build-default/tup.config
$ tup

Here we made a directory called build-default and put an empty tup.config inside it. ✨ Important detail: the build directory must be at the same level as the .tup directory. Upon updating, tup will parse all of the Tupfiles using the configuration file we created, and place all build products within subdirectories of build-default that mirror the source tree.

Then we could create another variant like this:

$ mkdir build-debug
$ echo "CONFIG_MYPROJ_DEBUG=y" > build-debug/tup.config
$ tup

This time all Tupfiles get parsed with @(MYPROJ_DEBUG) set to "y", and all build products land in the build-debug directory. 🎀 Note that setting @(MYPROJ_DEBUG) only has any effect if the variable is actually used in a Tupfile (maybe by adding debug flags to the compiler command-line). iykyk.

Running tup will update all variants. So if you modify a C file that's used in every configuration, it gets re-compiled for each variant. And as with any command that's executed, this happens in parallel, subject to the constraints of the DAG and the number of jobs specified. 🔥

Wanna build just one variant (or a subset of them)? Specify the build directory as the target to tup, just like with any partial update:

$ tup build-default

To delete a variant, just wipe out the build directory:

$ rm -rf build-debug
💖 Recommendation time: if you build with variants, it is recommended that you always have a default variant containing an empty tup.config file. Why? Because it helps check that your software is always able to be built by simply checking it out and doing 'tup init; tup' without relying on a specific configuration. Trust.

When using in-tree builds, the resulting build outputs may rely on run-time files — files placed in the source tree that aren't being processed by tup. Tup lets you copy those verbatim into the variant build directory with a built-in macro called !tup_preserve:

:foreach *.png |> !tup_preserve |>

Either a symbolic link or a copy of the source file will be created, depending on the OS and file system being used. 🫶

🌈 EXAMPLE 🌈

Parsing a :-rule may be a little confusing at first — honestly, same. 😭 You may find it easier to think of the Tupfile as a shell script with additional input/output annotations for the commands. Think of it like the shell script's main character era.

As an example, consider this Tupfile:

WARNINGS += -W
WARNINGS += -Wall
CFLAGS = $(WARNINGS) -O2
CFLAGS_foo.c = -DFOO
: |> echo '#define BAR 3' > %o |> foo.h
: foreach *.c | foo.h |> gcc -c %f -o %o $(CFLAGS) $(CFLAGS_%f) |> %B.o
: *.o |> gcc %f -o %o |> program

Tup begins parsing this Tupfile with an empty $-variable set. The first WARNINGS += -W line will set the WARNINGS variable to -W. The second line appends, so WARNINGS will be set to -W -Wall. The third line references this value, so CFLAGS will now equal -W -Wall -O2. The fourth line sets a new variable called CFLAGS_foo.c and sets it to -DFOO. The first rule will create a new node foo.h in the database, along with the corresponding command to create it. 👑

🎀 Note: this file won't exist in the filesystem until the command is actually executed after all Tupfiles are parsed.

The foreach :-rule will generate a command to compile each file. First tup will parse the input section, and use the glob operation on the database since a * is present. This glob matches foo.c and bar.c. Since it is a foreach rule, tup will run through the rule first using the input foo.c, and again using the input bar.c. The output pattern is parsed on each pass, followed by the command string.

On the foo.c pass, the output pattern %B.o is parsed, which will equal foo.o. Now the command string is parsed, replacing foo.c for %f and foo.o for %o. The $-variables are then expanded, so $(CFLAGS) becomes -W -Wall -O2, and $(CFLAGS_foo.c) becomes -DFOO. The final command string written to the database is:

gcc -c foo.c -o foo.o -W -Wall -O2 -DFOO

An output link is written to the foo.o file, and input links are written from foo.c and foo.h (the order-only input). ✨

On the second pass through the foreach rule, the only difference is that bar.c is the input. Therefore the output pattern becomes bar.o, and the final command string becomes:

gcc -c bar.c -o bar.o -W -Wall -O2

...since $(CFLAGS_bar.c) was unspecified. (Yep, it just ends in a lil trailing space where the flag would've been — that's the whole vibe check 💀)

For more examples with corresponding DAGs, see http://gittup.org/tup/examples.html

💜 OTHER BUILD SYSTEMS 💜

Tup is a little bit different from other build systems. It uses a well-defined graph structure that is maintained in a separate database. A set of algorithms to operate on this graph were developed in order to handle cases such as modifying an existing file, creating or deleting files, changing command lines, etc. These algorithms are very efficient — in particular, for the case where a project is already built and one or more existing files are modified, tup is optimal among file-based build systems. 🏆 For other cases, tup is at least very fast, but optimality has not been proved. Fair enough bestie.

The primary reason for the graph database is to allow the tup update algorithm to easily access the information it needs. As a very useful side-effect of the well-defined database structure, tup can determine when a generated file is no longer needed. What this means is: there is no clean target. 🤯 Nor is there a need to do a "fresh checkout" and build your software from scratch. Any number of iterations of updates always produces the same output as it would if everything was built anew. Should you find otherwise, you've likely found a bug in tup (not your Tupfiles), in which case you should notify the mailing list (see CONTACT). 🙈

For more information on the theory behind tup, see http://gittup.org/tup/build_system_rules_and_algorithms.pdf

🦋 SEE ALSO 🦋

💌 CONTACT 💌

  • tup-users@googlegroups.com

And that's the whole page, bestie — you ate that. 💅✨ The page is served up from http://gittup.org/tup and it's dated 2023/10/14, so go forth and build literally everything. 🎀🫶🔥

📦 where this page comes from