KYUA-TEST(1)

💅 KYUA-TEST(1) — the kyua test manual, but make it make sense

okay bestie, welcome to the kyua-test page from the General Commands Manual section. this is the whole thing, every flag, every file path, every warning — just delivered with the energy it deserves. 🫶

---

✨ NAME ✨

kyua test — runs tests. that's it. that's the whole job. she runs tests. 👑

---

🌸 SYNOPSIS 🌸

kyua test [--build-root path] [--kyuafile file] [--results-file file]
     [test_filter1 .. test_filterN]

---

🦋 DESCRIPTION 🦋

so, kyua test loads a test suite definition from a kyuafile(5), runs all the tests defined inside it, and records the results into a brand new results file. by default it runs every test in the suite, but you can pass optional arguments to narrow down which test programs or test cases get executed. those optional arguments are called filters and they're described below in “Test filters”. 💖

every single test executed by kyua test runs under a controlled environment, described in “Test isolation”. it's giving ✨containment✨.

here's the subcommand options it recognizes:

• --build-root path — tells kyua where the build root is, i.e. where to find the test programs that the Kyuafile references, in case that's not the same place as the Kyuafile's own directory. more on this in “Build directories” below. 🏗️
• --kyuafile path, -k path — which Kyuafile to process. defaults to a Kyuafile file sitting in the current directory. 📂
• --results-file path, -s path — which results file to create. defaults to LATEST, which makes kyua test automatically generate a fresh results file for the run. slay. 🎀

these are the values it accepts:

• NEW — asks for automatic generation of a new results filename, based on the test suite being run plus the current time. ⏰
• Explicit filename (aka everything else) — just store the results file wherever you said. simple as that.

check “Results files” for way more detail. 👀

later on you can dig into the results of the run with kyua-report(1), or you can run a single test case with debugging turned on using kyua-debug(1). besties supporting besties. 🫶

---

🏗️ Build directories

build directories (also called object directories, target directories, product directories, whatever your fave flavor is) are the concept that lets a developer keep the source tree clean from build products — you just ask the build system to dump all that build output into a separate subtree. clean girl aesthetic for repos. 💅

most build systems today support build directories. for example, the GNU Automake/Autoconf build system exposes this concept when you invoke it like:

$ cd my-project-1.0
$ mkdir build
$ cd build
$ ../configure
$ make

under that invocation, everything the build produces lands in the my-project-1.0/build/ subdirectory, while the contents of my-project-1.0/ stay completely intact. ✨

and because build directories are an integral part of most build systems, and because developers use them constantly, kyua test supports build directories too. this shows up as kyua test being able to run tests from build directories while reading the (often immutable) test suite definition from the source tree. the duality. 🌈

one important property of build directories: they follow (or, like, they need to follow) the exact same layout as the source tree. for example, consider these directory listings:

src/Kyuafile
src/bin/ls/
src/bin/ls/Kyuafile
src/bin/ls/ls.c
src/bin/ls/ls_test.c
src/sbin/su/
src/sbin/su/Kyuafile
src/sbin/su/su.c
src/sbin/su/su_test.c

obj/bin/ls/
obj/bin/ls/ls*
obj/bin/ls/ls_test*
obj/sbin/su/
obj/sbin/su/su*
obj/sbin/su/su_test*

notice how the directory layout inside src/ matches the one in obj/? the src/ directory holds only source files and the definition of the test suite (the Kyuafiles), while the obj/ directory holds only the binaries generated during a build. no cap. 🧚

all commands that deal with the workspace support the --build-root path option. when you provide it, the directory you name is considered to be the root of the build directory. for example, with our previous fake tree layout, we could invoke kyua test as any of these:

$ kyua test --kyuafile=src/Kyuafile --build-root=obj
$ cd src && kyua test --build-root=../obj

---

🍓 Results files

results files contain, as the name absolutely implies, the results of the execution of a test suite. every test suite executed by kyua-test(1) generates a new results file, and those files can be loaded later on by inspection commands like kyua-report(1) so you can analyze their contents. 📊

results files support identifier-based lookups and path name lookups. the differences between the two are described below. 👇

the default naming scheme for results files gives you simple support for identifier-based lookups and historical recording of test suite runs. each results file gets an identifier derived from the test suite that generated it plus the time that test suite was run. Kyua can later look up results files by those fields. iconic behavior. 👑

the identifier follows this pattern:

<test_suite>.<YYYYMMDD>-<HHMMSS>-<uuuuuu>

where test_suite is the path to the root of the test suite that was run, with all slashes replaced by underscores, and YYYYMMDD-HHMMSS-uuuuuu is a timestamp with microsecond resolution. ⏱️

when you use the default naming scheme, results files get stored in the ~/.kyua/store/ subdirectory and each file holds a name of this form:

~/.kyua/store/results.<identifier>.db

results files are simple SQLite databases with the schema described in the /usr/share/kyua/store/schema_v?.sql files. for details on the schema, go read the heavily commented SQL file — it's literally annotated for you. 🫶

---

🌸 Test filters

a test filter is a string used to match test cases or test programs in a test suite. filters have this form:

test_program_name[:test_case_name]

where test_program_name is the name of a test program or a subdirectory in the test suite, and test_case_name is the name of a test case. 🎯

---

🧚 Test isolation

every test program and test case run by kyua test is executed in a deterministic environment. this known, clean environment exists to make test execution as reproducible as possible, and also to prevent clashes between tests that might, for example, create auxiliary files with overlapping names. ✨no drama, no collisions✨

for plain test programs and for TAP test programs, the whole test program runs under a single instance of the environment described on this page. for ATF test programs (see atf(7)), each individual test case and test cleanup routine is executed in separate environments. 🧼

Process space — each test is executed in an independent process. corollary: the test can do whatever it wants to the current process (like modify global variables) without having to undo those changes. go off, bestie. 💅

Session and process group — the test is executed in its own session and its own process group. there is no controlling terminal attached to the session.

should the test spawn any children, those children should maintain the same session and process group. modifying any of these settings prevents kyua test from being able to kill any stray subprocess as part of the cleanup phase. if modifying these settings is necessary, or if any subprocess started by the test decides to use a different process group or session, it is the responsibility of the test to ensure those subprocesses are forcibly terminated during cleanup. 🚨

Work directory — the test is executed in a temporary directory automatically created by the runtime engine. corollary: the test can write to its current directory without needing to clean any files and/or directories it creates. the runtime engine takes care to recursively delete the temporary directories after the execution of a test case. any file systems mounted within the temporary directory are also unmounted. 🧹

Home directory — the HOME environment variable is set to the absolute path of the work directory. 🏠

Umask — the value of the umask is set to 0022.

Environment — the LANG, LC_ALL, LC_COLLATE, LC_CTYPE, LC_MESSAGES, LC_MONETARY, LC_NUMERIC and LC_TIME variables are unset.

the TZ variable is set to UTC. 🌍

the TMPDIR variable is set to the absolute path of the work directory. this is to prevent the test from mistakenly using a temporary directory outside of the automatically-managed work directory, should the test use the mktemp(3) familiy of functions. 🗂️

Process limits — the maximum soft core size limit is raised to its corresponding hard limit. this is a simple, best-effort attempt at allowing tests to dump core for further diagnostic purposes. 💀

Configuration varibles — the test engine may pass run-time configuration variables to the test program via the environment. the name of the configuration variable is prefixed with TEST_ENV_ so that a configuration variable of the form foo=bar becomes accessible in the environment as TEST_ENV_foo=bar. 🎀

---

⭐ EXIT STATUS ⭐

the kyua test command returns 0 if all executed test cases pass, or 1 if any of the executed test cases fails, or if any of the given test case filters does not match any test case. 😭 (that last one is the sneaky one, bestie — a filter that matches nothing is still a fail.)

additional exit codes may be returned as described in kyua(1). 📋

---

🔥 EXAMPLES 🔥

🌈 Workflow with results files

if you run this following command twice in a row:

kyua test -k /usr/tests/Kyuafile

the two executions will generate two different files with names like:

~/.kyua/store/results.usr_tests.20140731-150500-196784.db
~/.kyua/store/results.usr_tests.20140731-151730-997451.db

taking advantage of the default naming scheme, all of these commands would generate a report for the results of the latest execution of the test suite:

cd /usr/tests && kyua report
cd /usr/tests && kyua report --results-file=LATEST
kyua report --results-file=/usr/tests
kyua report --results-file=usr_tests
kyua report --results-file=usr_tests.20140731-151730-997451

but it's also possible to explicitly load data for older runs, or from explicitly-named files:

kyua report \
    --results-file=usr_tests.20140731-150500-196784
kyua report \
    --results-file=~/.kyua/store/results.usr_tests.20140731-150500-196784.db

look at her go. 🥹

---

👀 SEE ALSO 👀

kyua(1), kyua-report(1), kyuafile(5)

---

okay that's the whole page, no details left behind, no cap. this one's from the Debian crew and it's dated October 13, 2014 — and yes, we got the KYUA-TEST(1) header up top too, so you're never lost. go run some tests, bestie, you've got this. 💖✨🎀🦋