FUTHARK-BENCH(1)

💅 FUTHARK-BENCH — the manual page, but make it cute

okay bestie, pull up a chair 🫶 this is futhark-bench, from the Futhark manual crew. It's the benchmarking tool and honestly it's kind of the moment.

---

✨ NAME ✨

futhark-bench — benchmark Futhark programs 💖

---

🌸 SYNOPSIS 🌸

futhark bench [options...] programs...

---

🎀 DESCRIPTION 🎀

so like, this is the recommended way to benchmark your Futhark programs, no cap. Your programs get compiled with whatever backend you picked (c is the default, she's the one), then each test case gets run a bunch of times, and then the arithmetic mean runtime plus the 95% confidence interval get printed straight to standard output. ✨

Need to know how to format your test data? Go peek at futhark-test <#futhark-test-1>. And heads up: if a program has no data sets, it's just gonna be ignored — it won't even get compiled. literally not even looked at 💀

If compilation of a program fails, futhark bench aborts immediately, like full stop, we're done here. But if execution of a test set fails? Different story — an error message gets printed, benchmarking keeps going (and --json will still write the file, bestie), but a non-zero exit code gets returned at the end. so you'll know. 🫶

---

🧚 METHODOLOGY 🧚

For every program and dataset, futhark bench first does a single "warmup" run that gets yeeted into the void — discarded, gone, not counted. After that it uses a two-phase technique:

1. The initial phase performs ten runs (you can change that with -r), or keeps doing runs for at least half a second — whichever takes longer. If the resulting measurements are sufficiently statistically robust (determined using standard deviation and autocorrelation metrics, very science of it), the results are produced and the second phase is not entered. Otherwise the results get discarded and the second phase is entered.
2. The convergence phase keeps performing runs until a measurement of sufficient statistical quality is reached. she just keeps going until it's good enough 💅

Now, what even is "sufficient statistical quality"? It's based on heuristics, okay? The whole intent is that futhark bench will in most cases do the right thing by default — both when benchmarking long-running programs and short-running programs. Iconic behaviour honestly. But if you want complete control, disable the convergence phase with --no-convergence-phase and set the number of runs you want with -r. main character energy, your call. 👑

---

💖 OPTIONS 💖

• --backend=name — the backend used when compiling Futhark programs. write it without the leading futhark, so like just opencl.
• --cache-extension=EXTENSION — for a program foo.fut, this passes --cache-file foo.fut.EXTENSION. By default, --cache-file is not passed at all.
• --concurrency=NUM — how many benchmark programs get prepared concurrently. Defaults to the number of cores available. "Prepare" means compile the benchmark and generate any needed datasets. Sometimes that generation eats too much memory, and lowering --concurrency may help. 🍓
• --convergence-max-seconds=NUM — don't run the convergence phase for longer than this. And no, this does not mean the measurements have converged, don't be fooled. Defaults to 300 seconds (five minutes).
• --entry-point=name — only run entry points with this name. one (1) entry point. exclusive. 💅
• --exclude-case=TAG — do not run test cases that contain the given tag. By the way, cases marked with nobench, disable, or no_foo (where foo is the backend used) are ignored by default.
• --futhark=program — the program used to perform operations (eg. compilation). Defaults to the binary running futhark bench itself.
• --ignore-files=REGEX — ignore files whose path matches the given regular expression.
• --json=file — write raw results in JSON format to the specified file. 📝
• --no-tuning — do not look for tuning files.
• --no-convergence-phase — do not run the convergence phase.
• --pass-option=opt — pass an option to benchmark programs that are being run. For example, maybe we wanna run OpenCL programs on a specific device:

futhark bench prog.fut --backend=opencl --pass-option=-dHawaii

• --pass-compiler-option=opt — pass an extra option to the compiler when compiling the programs.
• --profile — enable profiling for the binary (by passing --profiling and --logging) and store the recorded information in the file indicated by --json (which is required for this, so don't forget it 🫶), along with the other benchmarking results.
• --runner=program — if set to a non-empty string, compiled programs are not run directly. Instead, the indicated program is run with its first argument being the path to the compiled Futhark program. Super useful for compilation targets that cannot be executed directly (as with futhark-pyopencl <#futhark-pyopencl-1> on some platforms), or when you wish to run the program on a remote machine. remote girlie behaviour 🌈
• --runs=count — the number of runs per data set.
• --skip-compilation — do not run the compiler, and instead assume that each benchmark program has already been compiled into a server-mode executable.

⚠️ Use with caution, bestie. like, actually.

• --spec-file=FILE — ignore the test specification in the program file(s), and instead load them from this other file. These external test specifications use the same syntax as normal, but without line comment prefixes. A == is still expected. ✨
• --timeout=seconds — if the runtime for a dataset exceeds this integral number of seconds, it gets aborted. Important: the time is allotted not per run, but for all runs for a dataset. So a twenty second limit for ten runs means each run has only two seconds (minus initialisation overhead). do the math, that's tight 😭

A negative timeout means to wait indefinitely. forever girl. 💀

• -v, --verbose — print verbose information about what the benchmark is doing. Pass it multiple times to increase the amount of information printed. stack it up 📚
• --tuning=EXTENSION — for each program being run, look for a tuning file with this extension, which is suffixed to the name of the program. For example, given --tuning=tuning (the default), the program foo.fut will be passed the tuning file foo.fut.tuning if it exists. 🦋

---

🌟 EXAMPLES 🌟

The following program benchmarks how quickly we can sum arrays of different sizes:

-- How quickly can we reduce arrays?
--
-- ==
-- nobench input { 0i64 }
-- output { 0i64 }
-- input { 100i64 }
-- output { 4950i64 }
-- compiled input { 10000i64 }
-- output { 49995000i64 }
-- compiled input { 1000000i64 }
-- output { 499999500000i64 }

let main(n: i64): i64 =
  reduce (+) 0 (iota n)

---

👀 SEE ALSO 👀

futhark-c <#futhark-c-1>, futhark-test <#futhark-test-1>

---

🫶 COPYRIGHT 🫶

2013-2020, DIKU, University of Copenhagen

---

okay that's the whole thing, bestie, you're fully caught up 💜 this was Futhark 0.25.32, dated Mar 11, 2026 — go benchmark something and slay, mwah 💅✨🎀