faltest(1)

📦 version 0-unstable-2023-11, in nixos 26.05

💅 faltest(1) — like, the Falcon unit test interface, bestie

So you found the faltest page in the Falcon User Manuals — welcome to your new obsession. This is the whole vibe, top to bottom, no details skipped, fr fr. 🦋

✨ NAME ✨

faltest — the Falcon unit test interface. That's it, that's the whole name, and honestly it ate. 👑

🌸 SYNOPSIS 🌸

faltest [-d unit_test_dir] [options] [unit_list] module_file.fam

💖 DESCRIPTION 💖

faltest is a powerful command line tool, a whole interface to the Falcon unit testing system. Main character energy, lowkey.

The basic working principle — the way that it works — is that it takes all the .fal script files chillin' in a directory, compiles them, executes them, and keeps track of errors, elapsed times, and execution failures along the way. Then, after running all the scripts, faltest may print a report on what happened — but only if you asked it to. Consent matters. 🙈

You can name a list of one or more unit tests right there in the faltest command line, after the options. Also, the executed tests can be limited to named subsets. Fancy, right? 💅

Also also: the unit test directory gets added to the module load path, so load directives will be resolved by searching for the required scripts in the test path. iykyk. ✨

🎀 UNIT TEST SCRIPTS 🎀

Scripts that are part of a unit test have to start with a common header that indicates some info about them. That header is actually a formatted Falcon comment — faltest just reads it like a little note you left for it. 🫶

This is a typical header:

/**********************************************
* Falcon test suite
*
* ID: 10a
* Category: cat-name
* Subcategory: subcat-name
* Short: The short name
* Description:
*   Long description of this test
*   Spanning on many lines...
* [/Description]
**********************************************/

The header is free form, and faltest recognizes the following fields — they live inside a comment and may optionally be preceded by a *.

🦋 The fields, one by one

  • ID: — the only mandatory field, bestie. It declares the ID under which this unit test is known. It's used when listing the tests and when selecting them as an argument of the faltest command line. The ID must be an integer number, eventually followed by a single lowercase letter. Similar tests should be filed under the same ID with a different specification letter. Scripts that don't have this field will be ignored by faltest. 💀 (no ID, no vibes)
  • Category: — the name of the category this test is filed under. faltest can select a subset of scripts to be executed down to a certain category.
  • Subcategory: — the name of the subcategory this test is filed under. faltest can select a subset of scripts to be executed down to a certain subcategory.
  • Short: — a short description (or symbolic name) for this unit test.
  • Description: — a longer description explaining what this test is supposed to do. The description always spans multiple lines, and is closed by a [/Description] tag. Don't forget the closer, or it's giving unfinished business. 🧚

🌈 THE TESTSUITE MODULE 🌈

The Falcon system provides a loadable module called "testsuite". This module is actually embedded in faltest, and it's provided to all the scripts it runs. It gives you these functions:

  • success() — the script is terminated and recorded as a success. Slay. ✅
  • failure( reason ) — the script is terminated and recorded as a failure. An optional parameter containing a description of the failure condition may be optionally provided; it will be written as part of the report and may be used to track which part of the test wasn't working. Very "tell me where it hurts". 😭
  • testReflect( item ) — returns the passed item. This is used to test for engine responsiveness to external calls and item copy through external functions. Boomerang behavior, love that. 🪃
  • alive( percent ) — in tests that run for some human sensible time, this function should be called periodically to inform the system and the user that the test is proceeding. An optional "percent" parameter can be provided; it will be interpreted as a value between 1 and 100 representing the amount of test that has been performed up to this moment. It's literally the "still alive, bestie" check-in. 🫶
  • timings( total_time, performed_ops ) — when execution time is relevant for the test, like in benchmarks, this function can be used to communicate back to faltest the time elapsed in the operations being tested and the number of operations performed. Those parameters will be recorded and eventually saved in the report file, to be used as benchmarks against falcon engine modifications. Receipts, saved. 🧾
  • timeFactor() — lengthy tests are often performed by looping over the operation to be tested for a certain time. Benchmarks and performance tests should be written so that they can normally complete in a reasonable time, between one and ten seconds. In case the user wants the tests to perform longer, in order to obtain better statistical data, they can pass the -f (time factor) option to the faltest command line. The time factor will be a number greater than 1, and should be used by tests that may perform lengthy operations to customize the number of performed tests. This function returns, as an integer, the value of the -f option given to faltest — or 1 if the -f option was not given. 🕐

💅 OPTIONS 💅

  • -c <cat> — select this category and ignore the rest. Tunnel vision, but make it intentional. 👀
  • -C <subcat> — select this subcategory and ignore the rest.
  • -d — directory from where to load the unit tests. If not specified, it will be the current directory.
  • -f <n> — set time factor to N. Some scripts may use this information to perform more loops or lengthy operations.
  • -h — show version and a short help.
  • -l — list the selected tests and exit. Combine with -v to have the path of the tests, as -l only lists the script ID, its short name and the category/subcategory pair. (Just the ID, the short name, and the cat/subcat duo — that's it.)
  • -m — do not compile in memory. Use disk temporary files.
  • -M — checks for memory leaks. This option makes faltest report unclaimed memory after each script execution, and a final report at the end of the tests. The check is extended to all the engine operations, so errors in the engine are detected too. Leak check stays undefeated. 🔥
  • -o <file> — write final report to the given output file.
  • -s — perform module serialization test. Other than compiling the file, the module is also saved and then restored before being executed. This allows one to check for errors in module serialization (that is, loading of .fam files). The operation is performed in memory, unless the option -m is also specified. Save, restore, re-run — a little glow-up round trip. 💫
  • -S — compile via assembly. This tests the correct behavior of the assembler generator and compiler instead of the binary module generator.
  • -t — records and displays timings. The statistics of compilation, linking and execution overall times are recorded and written as part of the report.
  • -T — records timings() calls from scripts. This allows the scripts to declare their own performance ratings, and collects the results in the final report.
  • -v — be verbose. Normally, execution and failures are sparsely reported. This is because the normal execution mode is meant for automated runs. Tests can be executed by automated utilities and errors can be reported to the system administrator by simple checks on the output data.

A developer willing to fix a broken test must run that test alone with -v enabled. A more complete error report (including compilation or execution errors, if they were the cause for the failure) will then be visualized. The -v option also allows one to see the path of the original script, which is otherwise hidden (masked by the testsuite ID). So yeah — one broken test, alone time, -v, and the mask comes off. 🎭

  • -V — prints version number and exits.

🌟 SAMPLE 🌟

This is a simple and complete example from the Falcon benchmark suite. Chef's kiss, obviously. 👨‍🍳💋

/*******************************************
* Falcon direct benchmarks
*
* ID: 2a
* Category: benchmark
* Subcategory: calls
* Short: Benchmark on function calls
* Description:
*    Performing repeated function calls and returns.
*    This test calls a function without parameters.
*
* [/Description]
******************************************/

loops = 1000000 * timeFactor()
each = int(loops/10)

function toBeCalled()
end

// getting time
time = seconds()
for i = 1 to loops

   // perform the call
   toBeCalled()

   if i % each == 0
      alive( i/loops*100 )
   end
end
// taking end time
time = seconds() - time

// subtract alive time
timeAlive = seconds()
for i = 1 to loops
   if i % each == 0
      alive( i/loops*100 )
   end
end
timeAlive = seconds() - timeAlive
time -= timeAlive

timings( time, loops )

/* end of file */

🍓 FILES 🍓

  • /usr/lib/libfalcon_engine.so — default location of the Falcon Engine loadable module.

👑 AUTHOR 👑

Giancarlo Niccolai <gc@falconpl.org> — the icon who made this.

🔥 SEE ALSO 🔥

  • falcon(1) — go read that one next, bestie.

💜 LICENSE 💜

This document is released under the "GNU Free Documentation License, version 1.2". On Debian systems, the complete text of the Free Documentation License, version 1.2, can be found in /usr/share/common-licenses/.

Okay that's the whole page, you made it! 🎉 This gem came to you from the Falcon toolset, dated April 2007 — and the <pink>faltest(1)</pink> manual name is officially folded in near the top like we promised. Periodt. 💅✨

📦 where this page comes from