bench(3tcl)

✨ NAME ✨

bench — bestie, this whole page is about Processing benchmark suites. That's the vibe. 👑

Also hi hello, we're hanging out in the bench(3tcl) corner of the manual and the department is literally called Benchmarking/Performance tools, so you already know what time it is. 🧚

---

🌸 SYNOPSIS 🌸

package require Tcl 8.5 9
package require bench ?0.6?

::bench::locate pattern paths
::bench::run ?option value...? interp_list file...
::bench::versions interp_list
::bench::del bench_result column
::bench::edit bench_result column newvalue
::bench::merge bench_result...
::bench::norm bench_result column
::bench::out::raw bench_result

---

🦋 DESCRIPTION 🦋

okay so this package is your little toolbox for exactly two things:

1. executing benchmarks that are written in the bench language, and
2. processing the results that come out of running them.

and if you're the kind of reader who wants to actually learn the bench language itself? bestie, start with the bench language introduction and then cruise on over to the formal bench language specification. iykyk 💅

---

👑 PUBLIC API 👑

🔥 BENCHMARK EXECUTION 🔥

::bench::locate pattern paths

This one goes hunting for Tcl interpreters and hands you back a list with their paths. It searches through the list of paths you passed in (that's the paths argument), using the glob pattern.

It resolves soft links so it can find the actual executables matching the pattern. And lowkey important: only interpreters that are marked as executable and are actually executable on your current platform make it into the result. no posers allowed 💀

::bench::run ?option value...? interp_list file...

This one runs all the benchmarks declared in your set of files — once per Tcl interpreter you named in interp_list, and following whatever config the options say — and then it returns all the accumulated timing results. The shape of that result is described down in Result format.

It's assumed that the contents of those files are written in the bench language, bestie. 📝

The available options are:

• -errors flag — the argument is a boolean value. If it's set, errors happening in benchmarks get propagated to the command, which aborts benchmark execution. Otherwise they're recorded in the timing result via a special result code. The default is to propagate and abort. 😭
• -threads n — the argument is a non-negative integer declaring how many threads to use while executing the benchmarks. The default is 0, which means don't use threads at all.
• -match pattern — the argument is a glob pattern. Only benchmarks whose description matches the pattern get executed. The default is the empty string, which means execute all patterns.
• -rmatch pattern — the argument is a regular expression pattern. Only benchmarks whose description matches the pattern get executed. The default is the empty string, which means execute all patterns.
• -iters n — the argument is a positive integer, the maximal number of iterations for any benchmark. The default is 1000. Individual benchmarks can override this. ✨
• -pkgdir path — the argument is a path to an existing, readable directory. You can specify multiple paths — just use the option multiple times, one path each time.

If you specified no paths, the system behaves as before. If you specified one or more paths — say N of them — then each of the specified interpreters gets invoked N times, once with each of the specified paths. The chosen path gets put into the interpreters' auto_path, which lets it find specific versions of a package.

So this is the whole point of -pkgdir: it lets you benchmark several different versions of a package, against one or more interpreters. 🎀

Note: the empty string is allowed as a path, and it causes the system to run the specified interpreters with an unmodified auto_path. That's for when the package in question is available there too.

::bench::versions interp_list

Give this one a list of Tcl interpreters (identified by their path) and it hands back a dictionary mapping each interpreter to its version. Interpreters that aren't actually executable, or that fail when interrogated, do not get put into the result. Which means the result may contain fewer interpreters than your input list had. 🙈

It uses the builtin command info patchlevel to figure out each interpreter's version.

🎀 RESULT MANIPULATION 🎀

::bench::del bench_result column

Removes a column — aka all the benchmark results for one specific Tcl interpreter — from the given benchmark result, and returns the modified result.

The benchmark results are in the format described in Result format.

The column is identified by an integer number.

::bench::edit bench_result column newvalue

Renames a column in the given benchmark result and returns the modified result. Translation: the path of the Tcl interpreter in that identified column gets changed to an arbitrary string. name it whatever, bestie.

The benchmark results are in the format described in Result format.

The column is identified by an integer number.

::bench::merge bench_result...

Takes one or more benchmark results, merges them into one big result, and returns that as its result. 🫶

All the benchmark results are in the format described in Result format.

::bench::norm bench_result column

Normalizes the timing results in the given benchmark result and returns the modified result. Meaning: the cell values are no longer times, they're factors showing how much faster or slower the execution was relative to the baseline.

The baseline it normalizes against is the timing results in the chosen column. So after normalization the values in that column are all 1 — because those benchmarks are neither faster nor slower than the baseline. icon behavior honestly.

A factor less than 1 means that benchmark was faster than the baseline. A factor greater than 1 means the execution was slower. 📈📉

The benchmark results are in the format described in Result format.

The column is identified by an integer number.

::bench::out::raw bench_result

Formats the given benchmark result for output to a file, a socket, etc. This particular command does no formatting at all — it just passes the input through unchanged. zero notes, pure vibes. 💅

For other formatting styles, look at the packages bench::out::text and bench::out::csv — they provide commands to format benchmark results for human consumption, or as CSV data importable by spread sheets, respectively.

And complementary to all that: if you want to read benchmark results from files, sockets, etc., go look for the package bench::in and the commands it provides.

🌈 RESULT FORMAT 🌈

After a set of benchmarks runs, the raw result this package returns is a Tcl dictionary holding all the relevant info. The dictionary is a compact representation — i.e. a serialization — of a 2-dimensional table that has Tcl interpreters as columns and benchmarks as rows. The cells of the table contain the timing results. The Tcl interpreters / columns are identified by their paths. The benchmarks / rows are identified by their description.

The possible keys are all valid Tcl lists of two or three elements, and they take one of these forms:

• {interp *} — the set of keys matching this glob pattern captures info about all the Tcl interpreters used to run the benchmarks. The second element of the key is the path to the interpreter.

The associated value is the version of the Tcl interpreter.

• {desc *} — the set of keys matching this glob pattern captures info about all the benchmarks found in the executed benchmark suite. The second element of the key is the description of the benchmark, which has to be unique.

The associated value is irrelevant and set to the empty string.

• {usec * *} — the set of keys matching this glob pattern captures the performance info, i.e. the timing results. The second element of the key is the description of the benchmark, and the third element is the path of the Tcl interpreter that was used to run it.

The associated value is either one of several special result codes, or the time it took to execute the benchmark, in microseconds. The possible special result codes are:

• ERR — the benchmark could not be executed; it failed with a Tcl error. 💀
• BAD_RES — the benchmark could be executed, but the result from its body didn't match the declared expectations. 🫠

---

😭 BUGS, IDEAS, FEEDBACK 😭

This document, and the package it describes, will undoubtedly contain bugs and other problems. (not me being humble, it's literally what it says 🥲) Please report those in the category bench of the Tcllib Trackers at [http://core.tcl.tk/tcllib/reportlist]. And please also report any ideas for enhancements you have for either the package and/or the documentation — we want them, fr fr.

When you're proposing code changes, please provide unified diffs, i.e. the output of diff -u.

Also note that attachments are strongly preferred over inlined patches. You can make an attachment by going to the Edit form of the ticket right after it's created, and then using the left-most button in the secondary navigation bar. periodt.

---

🦋 SEE ALSO 🦋

bench_intro, bench_lang_intro, bench_lang_spec, bench_read, bench_wcsv, bench_wtext

---

💜 KEYWORDS 💜

benchmark, merging, normalization, performance, testing

---

⭐ CATEGORY ⭐

Benchmark tools

---

🎀 COPYRIGHT 🎀

Copyright (c) 2007-2008 Andreas Kupries <andreas_kupries@users.sourceforge.net>

---

and that's the whole page, bestie — this was tcllib, version 0.6, serving you bench(3tcl). go forth and benchmark slayfully 👑💖