UPCRUN(1)

📦 version 2022.10.0, in nixos 26.05, nixpkgs unstable

💅 NAME 💅

okay so welcome to the UPCRUN(1) manual page in the User Commands section, bestie. upcrun is a portable parallel job launcher for UPC programs, version 2022.10.0. That's the whole main character energy: you give it a UPC program, it launches your parallel job, and it stays portable. Periodt.

✨ SYNOPSIS ✨

upcrun [options] program-name [ program-arguments ... ]

that's the shape of it: options first, then the program name, then whatever arguments the program wants. easy.

🌸 OPTIONS 🌸

okay so here is every single flag, no skips, no cap:

  • -h, -?, -help — bestie this shows you this exact help message. That's the whole job.
  • -help-gasnet [network_name] — pulls up the GASNet documentation for you, or if you pass a network_name, the GASNet documentation for that particular network conduit. Very “let me look that up for you” but official.
  • -conf=FILE — read FILE instead of the $HOME/.upcrunrc configuration file. So if you have a different rc file you want it to use, this is your moment.
  • -norc — do not read the $HOME/.upcrunrc configuration file. You can also achieve this by setting the UPCRUN_NORC environment variable. This overrides -conf. It's the “no rc, thanks” button.
  • -n <num> — spawn <num> UPC threads. If the executable was compiled for a dynamic thread count then this flag is required (you gotta say how many). When it's compiled for a static thread count, this flag is optional, but if you pass it, it must agree with the compiled-in setting. Also -np is a synonym for -n, iykyk.
  • -N, -nodes <num> — specifies the number of compute nodes to use for execution. The THREAD LAYOUT section below has the full details, so keep reading bestie.
  • -c, -cpus-per-node <num> — specifies the number of UPC threads to execute on each compute node. Again, THREAD LAYOUT has the details.
  • -p, -pthreads <num> — if the UPC executable was compiled with pthreads support then this option overrides the compiled-in default number of pthreads per process. A value of zero resets to the compiled-in default or the UPC_PTHREADS_PER_PROC environment variable. This flag is not legal with an executable not compiled with pthreads support, so don't try it.
  • -pshm-width <num> — if the UPC executable was compiled with PSHM support then this option sets the maximum number of processes which can comprise a shared-memory "supernode". If more than this many processes are co-located on the same compute node, then they will become multiple supernodes for the purpose of PSHM. Note that this is a limit on processes, not on UPC threads, which will be different if pthreads is also in use. A value of 0 (the default) means no limit is imposed. A value of 1 essentially disables PSHM. Iconic.
  • -bind-threads — bind (aka pin) UPC threads to processors. (Silently ignored on unsupported platforms, so no drama there.)
  • -polite-sync — cause your UPC application to yield (rather than CPU spin) while waiting for locks/barriers. This will slow down your application if you are running on an uncontended system where CPUs >= UPC threads, which is why it is off by default. However, if you are on a busy system, and/or are running more UPC threads per machine than there are CPUs, you should set this, or your performance (and that of the whole machine) may suffer. Consider this your polite reminder.
  • -shared-heap <sz> — requests the given amount of shared memory (per UPC thread). Units of <sz> default to megabytes; use '2GB' to request 2 gigabytes per thread.
  • -[no]trace — enable tracing. This option is only effective if the executable was built with tracing enabled. No tracing built in? Then this is just a vibe.
  • -traceall — enable tracing of all events, including low-level events that are unnecessary for upc_trace. May impose significant run time and tracefile size penalties. Implies -trace. So it's trace but extra.
  • -tracefile <file> — override the default destination for tracing output. If present, an optional % character in the filename will expand into a distinct integer for each process. This option implies -trace.
  • -freeze[=<threadid>] — cause thread <threadid> to freeze at startup immediately before main() is called, to wait for a debugger to attach. <threadid> defaults to 0. So it's the “hold still while I attach gdb” option.
  • -freeze-early[=<nodeid>] — cause node <nodeid> to freeze and await debugger attach early in the UPC runtime startup procedure, to assist in debugging problems with the UPC runtime. <threadid> defaults to 0. See the Berkeley UPC user guide for further info.
  • -freeze-earlier — freeze program execution as early as possible in the GASNet initialization procedure. Early early.
  • -[no]freeze-on-error — freeze and await a debugger to attach on most program errors or crashes. Note this option has the potential to create zombie processes that will need to be manually killed. Spooky but useful.
  • -[no]abort — attempt to generate a core file on most program errors or crashes. Core file generation must usually also be enabled in the shell limits and OS policies. So this is a request, not a guarantee.
  • -[no]backtrace — enable backtraces. This option requests generation of a stack backtrace on most fatal errors, if supported on a given platform. These backtraces are valuable when reporting bugs. Note backtrace results are generally more useful when the application was built with upcc -g. Also note that some types of program crashes may cause the backtrace code to hang, potentially creating zombie processes that will need to be manually killed. So: yes, helpful, but sometimes cursed.
  • -backtrace-type=<list> — tweak the mechanisms used to generate the backtrace. The list of available mechanisms is platform-specific, and can be viewed by running with -verbose. This option implies -backtrace.
  • -encode-args, -encode-env, -encode — use a "safe" encoding for the command-line arguments, environment variables, or both. This may fix problems with correct propagation on some spawners, especially for arguments or values containing spaces or other special characters. Very “please don't break my spaces” core.
  • -q, -[no]quiet — suppress initialization messages from UPC runtime. Peace and quiet, periodt.
  • -v, -[no]verbose — verbose: display commands invoked, environment variables set and other diagnostics. If you want the whole tea, this is it.
  • -t, -[no]show — testing mode: don't actually start the job, just output the system commands that would have been used to do so. So you can see the plan without the commitment.
  • -i, -[no]info — display useful information about the executable and exit. No run, just info.
  • -version — show version information for upcrun. The classic.

🧵 THREAD LAYOUT 🧵

okay so the layout of UPC threads to network nodes depends on the settings of two parameters, the cpus_per_node and the -nodes flag. The cpus_per_node setting comes from the -cpus_per_node flag if present, or else from the default_cpus_per_node setting in upcrun.conf or $HOME/.upcrunrc. There are three distinct mechanisms for thread layout depending on the values of cpus_per_node and the -nodes flag.

  1. If cpus_per_node and -nodes are both set to zero (or are not set): the layout of UPC threads is left to the underlying mpirun-style spawner (the <conduit>_spawn configuration setting or the UPC_<CONDUIT>_SPAWNCMD environment variable). This is the only case in which the mpirun-style spawner is used by default. If the executable has been compiled with pthreads enabled, the UPC threads are first grouped into processes which are in turn laid out by the spawner. With the possible exception of the last process, each such process includes the same number of pthreads. This number defaults to the value compiled-in to the executable. This can be overridden at upcrun time by either the UPC_PTHREADS_PER_PROC environment variable or with the -pthreads flag. For unusual cases, the UPC_PTHREADS_MAP environment variable can be used to specify the grouping of threads into processes.
  2. If cpus_per_node is zero while -nodes has a non-zero value: UPC threads are spread as evenly as possible over the given number of nodes without regard to possible overcommit of CPUs. When using pthreads, this may result in some processes having fewer threads than others if the threads do not divide evenly among the processes and nodes.
  3. If cpus_per_node is non-zero and -nodes is zero (or not set): UPC threads are laid out to use the fewest number of nodes possible without exceeding cpus_per_node. Then UPC threads are spread as evenly as possible over the given number of nodes. When using pthreads, this may result in some processes having fewer threads than others if the threads do not divide evenly among the processes and nodes.

🌸 ENVIRONMENT VARIABLES 🌸

okay bestie, this is the part where the runtime reads your vibes (environment variables) and adjusts accordingly. buckle up, there's a lot of them 💅

  • UPCRUN_FLAGS — set this and whatever flags/arguments you put in it get passed to every single invocation of upcrun. it's in addition to the default_options parameter described below. so like, you can pre-load your fave flags and never think about them again ✨
  • UPC_<CONDUIT>_SPAWNCMD and UPC_<CONDUIT>_SPAWN_NODESCMD — set these to override the spawner templates that live in the upcrun.conf and .upcrunrc files, for a given conduit/network. yes, per-conduit. she's specific 💅
  • UPC_NO_WARN — if set, startup warnings get omitted. y'know, the ones that show up when debugging or tracing is enabled. gone. poof 🫶
  • UPC_QUIET — if set, all output that isn't generated by your application gets omitted, including both warnings and the initial display of the UPC thread layout. it's literally equivalent to -q. periodt.
  • UPC_NODES, UPC_NODEFILE, or PBS_NODEFILE — these control job layout when -network=udp is used (see RUNNING UDP-BASED UPC JOBS, below 👇).
  • UPC_SHARED_ALLOC_ALIGN — if you use it, it must be set to a number (and a following 'K', 'M', or 'G' sets the value to kilobytes, megabytes, or gigabytes respectively). this number is the minimum size for "large" objects (think large structs, arrays, upc_alloc'd memory, etc.) in a UPC program. the Berkeley UPC runtime automatically cache-aligns those big objects, while smaller objects keep their default alignment (depends on your compiler and the object type — 8-byte alignment is common). this has been observed to improve performance on certain platforms 🔥. the default value is '4K' (i.e., 4 kilobytes).
  • UPC_FIRSTTOUCH and UPC_FORCETOUCH — described down in PROCESSOR AND MEMORY AFFINITY, below ⬇️
  • UPC_SHARED_HEAP_SIZE — sets the amount of shared heap (per UPC thread) for your program, exactly like the -shared-heap flag does. if you use both, the flag wins. no cap.
  • UPC_SHARED_LOCALHEAP_INITSZ — if set, this determines the amount of shared heap (per UPC thread) that gets reserved at initialization for servicing calls to upc_alloc(). this is not an upper limit, bestie, it's just a head start. setting a large value can reduce (or even eliminate 😮) the communication required to dynamically grow the local slice of the shared heap — at the cost of limiting the amount of memory available to service calls to upc_{all,global}_alloc(). for more info on how the local and global slices of the shared heap interact, see:
https://upc.lbl.gov/docs/system/runtime_notes/memory_mgmt.shtml

the value is interpreted as a value in units of megabytes, unless an optional

💀 okay so the original sentence genuinely just... stops there. mid-thought. i'm not gonna invent the rest for you, that would be lying and lying is not the vibe. moving on!!
  • UPC_STACK_SIZE — if -pthreads are used, set this to a number (optionally followed by K/M/G for kilobytes/megabytes/gigabytes) and it determines the size of each pthread's stack.
  • UPC_STACK_PAD — alternatively, set this to a number (again with optional K/M/G suffix) and it gets added to the system's default pthread stack size. if you specify both, whichever one results in the larger stack is honored 👑

generally you only need either of these if you're experiencing stack overflow in your program 😭

  • UPC_LOCKS_RR — the UPC specification makes the affinity of UPC locks undefined, which is honestly giving chaos. beginning in the 2.12.2 release of Berkeley UPC, locks allocated with upc_all_lock_alloc() are spread across UPC threads, which more evenly distributes the CPU load from lock and unlock operations 💅. you can set UPC_LOCKS_RR to '0' to force the old behavior, where upc_all_lock_alloc() only allocated from memory on thread 0. integer values other than '0' specify the number of threads by which lock affinity advances for each upc_all_lock_alloc() call — and the default value is chosen to spread locks evenly over processes ✨
  • UPC_DEBUG_MALLOC — set it to '0' to stop a debug build of Berkeley UPC from using the default, debug malloc algorithm. that allocator is ON by default (because it helps catch so many allocation errors, like duplicate free() calls), but turning it off lets the memory layout more closely mimic a non-debug execution.

mixed-language programmers, listen up 📢 the debug mallocator is not fully intermixable with system malloc()/free() — specifically, you cannot malloc() objects using one allocator and free() them with the other. this should only be an issue in mixed-language programs which (for example) malloc() some storage in a pure-C object file (compiled without upcc), and then try to free() that storage from UPC code (or vice versa 😬).

there are a number of possible solutions in the current implementation:

  1. segregate your allocations so objects created with malloc() in UPC code are freed only with free() in UPC code (and similarly with non-UPC C code).
  2. a special case of the previous solution which may apply in some applications: perform all allocations in one language or the other — preferably in UPC, so you can reap the benefits of the debug malloc checking. you can trivially write a UPC code wrapper around a malloc() call and then call it from other languages instead of calling malloc() directly 🎀
  3. disable the debug mallocator by setting UPC_DEBUG_MALLOC=0, which fixes the problem by forcing UPC code to use the same (non-debug) mallocator for everything. (heads up: this solution loses the safety checking features of the debug mallocator 🙈)
  • UPC_ENVPREFIX — okay so some system job spawners (especially on loosely-coupled clusters) do a poor job of propagating environment variable settings from the spawning console to the worker compute nodes, and on those systems you may need extra care to make sure env vars set before calling upcrun are actually seen by UPC code.

on the code side, the most portable and reliable way to query environment variables is to call bupc_getenv() instead of getenv() (the signatures are identical). the latter is automatically redirected to the former in any program that includes upc.h ✨

on the spawning side, environment variables with prefix UPC_ or GASNET_ are automatically propagated — but if your code queries additional settings, you may need to explicitly request propagation of those variables to the compute nodes. set UPC_ENVPREFIX to a comma-delimited list of environment variable name prefixes, and for any prefix in $UPC_ENVPREFIX, upcrun will ensure all currently-set env vars matching ^$prefix are propagated to all compute nodes (prefix may contain perl regexs 👀).

💜 PROCESSOR AND MEMORY AFFINITY 💜

some shared-memory systems use a first-touch memory allocation scheme, where the first CPU to touch a memory page owns it (and therefore has the shortest latency access to it). by default, Berkeley UPC ensures that any static shared data, and/or memory allocated by the upc_alloc() and upc_all_alloc() functions, gets touched by the thread that should have affinity to it 🫶

you may set UPC_FIRSTTOUCH=0 to disable this.

by default, memory allocated with upc_global_alloc() does not have that first-touch guarantee from the previous paragraph. so if your program uses upc_global_alloc() and you're running on a NUMA system, you may wanna consider setting UPC_FORCETOUCH=1, which causes all possible shared memory in your program to be touched by the appropriate thread at startup, guaranteeing correct affinity 👑

since this involves a higher startup cost, you may want to limit the size of your shared memory to the minimum needed. like, be honest with yourself about how much you actually need 💀

⚠️ the UPC_*TOUCH environment variables may produce unexpected or undesirable results if the UPC threads do not remain on fixed processor cores. the -bind-threads option attempts to bind the UPC threads to fixed cores, but currently has the following limitations that should be considered before use:
  1. this option is currently only implemented on Linux and AIX, and is silently ignored on all other platforms 🙈
  2. the first UPC thread on a given compute node is bound to the first processor core, the second thread to the second core, and so on. this will wrap around if there are more UPC threads per node than processor cores.
  3. the ordering used for numbering of cores is unknown to Berkeley UPC, so nothing is done to ensure the layout is sensible. for instance, if you're using only half the cores on a dual-socket node, it's possible that all the UPC threads might be bound to one socket 😭
  4. if the job spawner binds processes to cores, then use of -bind-threads may either be ineffective or could result in an error 💀

🌐 RUNNING UDP-BASED UPC JOBS 🌐

ok so the udp network type is literally the "works anywhere" bestie of network types — it lets your UPC programs run on any machine that supports the ubiquitous UDP network layer. It's the fastest way to run on a cluster that only has an ethernet network, and lowkey it's faster than using -network=mpi with a TCP-based MPI implementation. slay.

In the most general case, our implementation can use UDP datagrams for inter-process communication, even when the processes are living on the same node. But bestie, listen — the fastest way to run over UDP on a cluster of SMPs is to use shared memory within each compute node. There are two ways to achieve that: PSHM or pthreads. iykyk.

  • If Berkeley UPC has been configured with intra-node shared memory (PSHM) support, then your apps automatically use shared memory for communication within a compute node. no effort required, we love that for you 🫶
  • Regardless of whether PSHM is available, apps can be compiled with the -pthreads option. Compiling with -pthreads=N (where N is the number of processors on your nodes) causes a single multithreaded process to be run on each node, with shared memory used among the corresponding N UPC threads. And if that's not convenient, you can also compile with -pthreads (no =N) and pass -pthreads=N to upcrun instead. flexible icon behavior ✨

In general the performance when compiled with and without -pthreads will differ in ways not easily predicted, and we advise trying both ways to determine the best option for your own application. fr fr, just vibe check both.

When UDP is used, you need to tell upcrun which machines to run the job on. There are four methods for doing this:

  1. If you simply wish to run your entire job on localhost, pass the -localhost flag to upcrun. However — compilation with -network=smp (and possibly -pthreads, as described above) will almost always generate a faster executable for single-node runs, and it does not require -localhost to run. So do not expect the best performance from -localhost runs. it's giving "technically works" 💀
  2. If you are running under the Portable Batch System (PBS), or any batch system that sets the $PBS_NODEFILE environment variable, upcrun will detect this and read the list of nodes for your job from that file. If running under N1 Grid Engine (formerly SGE), then the file $TMPDIR/machines is used if it exists. Note that the node list is used to launch UPC processes, which is different than UPC threads if you are using -pthreads (see below). 👀
  3. You can manually provide a list of nodes for your job, either by storing a space-separated list of hosts into $UPC_NODES, or by creating a file with one hostname per line and setting $UPC_NODEFILE to that file's name. Note that the node list is used to launch UPC processes, which is different than UPC threads if you are using -pthreads (see below). yes bestie this note is important, that's why it's here twice 💅
  4. You may use your own custom spawner to launch UDP jobs. See the README.udp file for details. DIY queen behavior 👑

The $UPC_NODES, $UPC_NODEFILE, and $PBS_NODEFILE variables, and $TMPDIR/machines are checked for in that order, and the first one found determines the job configuration. first come, first served ✨

Methods which rely on a "node list" ($UPC_NODES, $UPC_NODEFILE, or $PBS_NODEFILE) use the list to determine where to launch the UPC processes in the job. The first UPC process will be run on the first node in the list, the second process on the second, etc. If you are not using -pthreads, this is the same as having UPC thread 0 on the first node, thread 1 on the second, etc. If you are using -pthreads, however, multiple UPC threads will be run in each process, so you will need fewer names in your node list than UPC threads (it does not hurt to have extra node names in the list, but be aware that they will not be used — they're just standing there, unemployed 😭).

To run multiple UPC processes on the same node, simply enter the node name in the list multiple times (note that this is typically not as efficient as using a single process per node with -pthreads, as explained above). If you are not sure how your process is being laid out, look at the output from the job — all Berkeley UPC processes by default print out their node and process ID at the beginning of a run. built-in receipts, we stan 🎀

Methods 2 and 3 use ssh by default to connect to remote nodes in the job during set up. You may set the $UPC_SSH environment variable to any ssh-compatible program (ex: 'rsh') if that is preferable. Logging into remote nodes must not require an interactive password, and so you must use a method that allows this (ssh-agent, or passwordless ssh keys, or a passwordless rsh setup). For information on using ssh-agent (generally the most secure of these methods), see https://upc.lbl.gov/docs/user/sshagent.html

bestie pls do not sit there typing a password into a cluster launch. it will not work and it will not be cute. set up your keys ahead of time 🔑

When you run on multiple nodes, you must ensure that your executable exists on all nodes. Typically a shared filesystem (such as NFS) is used to provide this, but you can also copy the executable to the individual compute nodes manually if they lack a shared file system. All nodes must have a copy of the executable located at the same absolute pathname, which defaults to the absolute pathname of the executable used to invoke the job on the frontend node. If the absolute pathname for the executable on the compute nodes differs from the path on the frontend, you can specify the absolute path by setting the value in the SSH_REMOTE_PATH environment variable before running. 🌈

Example: Johnny Parallel wants to run a 4-way UPC job on his cluster of 2-way SMPs. So he sets his $UPC_NODES environment variable to his nodes' names (let's say "one.cluster.edu two.cluster.edu"), compiles with

upcc -network=udp -pthreads=2 foo.upc

and runs with

upcrun -n 4 a.out

and that's it, he's iconic now. look at him go 🦋

🔧 ADDITIONAL NETWORK-SPECIFIC SETTINGS 🔧

The GASNet networking layer used by Berkeley UPC provides various additional parameters that control job launching and/or performance tuning for specific networks. Each supported network has a README file (with a name like README-ibv for the 'ibv' network API, etc.). These files should be located in the PREFIX/share/doc directory, where PREFIX is the base directory of your UPC installation.

It is worth your time to peruse the READMEs for the network type(s) that your programs use, as you may find settings that allow programs to run faster on your machine, or workarounds for known bugs. literally free performance and free bug fixes, why would you not 🍓

⚙️ CONFIGURATION FILES ⚙️

upcrun uses a site-wide upcrun.conf file to get some of its settings. You may override any of the settings found in the global upcrun.conf file by creating a .upcrunrc file in your $HOME directory, or in an alternate file specified by the -conf=filename command line option. One can prevent processing of the .upcrunrc file by passing -norc or setting the environment variable $UPCRUN_NORC. choose your own adventure bestie ✨

To specify flags to pass to upcrun each time it is invoked, set default_options:

default_options = -n 4 -shared-heap 256MB

To specify flags to pass to upcrun only when it is invoked for an executable compiled for a given network, set the <conduit>_options parameter:

mpi_options = -shared-heap 192MB

To specify environment variables to pass to every application, set default_environment:

default_environment = UPC_STACK_SIZE=4MB

To specify environment variables to pass to applications only when compiled for a given network, set the <conduit>_environment parameter:

mpi_environment = SOMETHING="a value with spaces"

The default_environment and <conduit>_environment parameters are combined, with the conduit-specific settings given precedence. In the <conduit>_environment parameter the syntax !VAR can be used to unset a variable set in default_environment. And remember — the user's environment is always given precedence over these parameters. you outrank the config file, always 👑

To specify the number of UPC threads to start on each node of a cluster of SMPs, set the default_cpus_per_node paramater. If this value is unset, or is set to zero, then upcrun will rely on the underlying spawner to correctly layout processes on nodes. When running a UPC executable that was not compiled to use pthreads, this default is likely to be acceptible. However, when using pthreads the underlying spawner is unaware of the number of threads per process and therefore may start more UPC threads per node than available CPUs. Your site-wide upcrun.conf should have a correct setting for this parameter, but if using pthreads you are strongly encouraged to verify the value is correct. overloaded node = nobody having a good time 😭

default_cpus_per_node = 2

To modify the command line that upcrun uses to run applications for a particular network type (a.k.a. conduit), set the <conduit>_spawn and/or <conduit>_spawn_nodes parameters, where <conduit> is one of the network types listed in upcc -version. These parameters are templates used to execute a command to launch the necesssary processes. When writing templates, the following substitutions are available:

  • %N — number of processes to launch (might not equal UPC threads when using pthreads)
  • %M — number of "nodes" on which to launch processes
  • %R — number of "processes per node" (ppn) to launch
  • %P — program file
  • %A — program arguments
  • %C — alias for "%P %A"
  • %D — current working directory
  • %L — UPC-specific environment variable names (comma separated list) which the spawner will propogate to the application
  • %V — expands to "-v" if the user passed -v to upcrun, or to nothing otherwise
  • %% — expands to a single % character

Arguments are split on whitespace, but single- or double-quotes may be used to prevent this. Backslash \ is not special. noted 🫡

The <conduit>_spawn parameter is a template for launching a spawner similar to most implementations of mpirun — a single parameter specifies the number of processes to launch and these are assigned to available nodes in an implementation-specific manner. This spawner is used when the number of requested nodes and number of cpus per node are both unspecified (or has been set to zero). Examples include:

mpi_spawn = $(UPCR_HOME)/bin/gasnetrun_mpi -n %N -E %L %P %A
smp_spawn = %P %A
ibv_spawn = $(UPCR_HOME)/bin/gasnetrun_ibv %V -n %N -E %L %P %A

The <conduit>_spawn_nodes parameter is a template for launching a spawner such as prun or poe, in which one specifies both the number of processes to launch and the number of nodes to use. Given sufficient information, upcrun will compute the number of processes and nodes required as described in the section THREAD LAYOUT. This template should contain either the %M (nodes) or %R (ppn) substitution to pass the layout infomation to the spawner. For example:

ibv_spawn_nodes = $(UPCR_HOME)/bin/gasnetrun_ibv %V -n %N -N %M -E %L %P %A

and that's the config file tea ☕ — tune it once, benefit forever 💜

⚙️ OPTION PROCESSING ⚙️

okay so this is where it all comes together bestie — upcrun is out here reading options from four different places at once and you need to know who wins 💅

It pulls options from the site-wide config file, your user-specific config file, the environment, and the command line. The precedence works exactly like it parsed them in this order:

default_options <conduit>_options UPCRUN_FLAGS command-line

So yeah — later in that line means more power 👑 For any option that sets a value (think -n and -shared-heap), the last value seen is the one that actually gets used. Which means command-line values always win over everything else. No cap, that's just how it is ✨

Now, about default_options and <conduit>_options — those get grabbed from your $HOME/.upcrunrc file if it's chillin' there, or from the site-wide upcrun.conf if it's not 🫶 And heeere's the important bit: if the same setting shows up in both files, only the settings in $HOME/.upcrunrc are used. They are not additive. Not a merge, not a combo, not a duet — one or the other, periodt 💀

When it comes to splitting, arguments in default_options, <conduit>_options and UPCRUN get split on whitespace. But single- or double-quotes will suppress that splitting, so you can keep stuff together if you need to 🎀

⚠️ and like, heads up: the backslash character \' does not have any special meaning here. it's giving "decorative only" 🙈 don't expect it to escape anything for you.

🐛 REPORTING BUGS 🐛

We are very interested in fixing any bugs in upcrun — like genuinely, they want to hear about it 💖 For bug reporting instructions, please go to:

https://upc.lbl.gov

That's the spot. Go tell them what broke 🫶

👀 SEE ALSO 👀

like, go read the User's Guide if you want the full lore, iykyk 📚✨

and that's a wrap on the whole page, bestie!! 🎀 you made it to the end, iconic behavior honestly 👑 this one's from Berkeley UPC, dated October 2026, and you just finished reading UPCRUN(1) — go forth and run some UPC 💅🔥🌈

📦 where this page comes from