ct(3)

📦
version 29.0.6 · 3 versions
, in ubuntu stonking

✨ NAME ✨

Welcome to ct(3), the Erlang Module Definition for ct — Main user interface for the Common Test framework. 🫶 This is the group chat, bestie.

💖 DESCRIPTION 💖

Okay so this module is literally the command-line interface for running tests, plus the basic functions for Common Test case issues like configuration and logging. No cap.

The framework stores configuration values in a property list that's usually named Config. That list has info about the test run that the framework itself added, and it may also have user-provided values. The configuration gets passed into individual test cases, and into support functions if they're defined. Periodt.

Possible configuration variables include:

  • data_dir — Data file directory
  • priv_dir — Scratch file directory
  • Whatever gets added by init_per_suite/1 or init_per_testcase/2 in the test suite.
WARNING: The ?config macro, used to receive individual config values from the Config property list, is deprecated. Please use proplists:get_value/2-3 instead. 💀 This one is not it.

🧚 DATA TYPES 🧚

config_key() :: atom().

A configuration key which exists in a configuration file.

conn_log_mod() :: ct_netconfc | ct_telnet.

(just the type, bestie)

conn_log_option() :: {log_type, conn_log_type()} | {hosts, [key_or_name()]}.

(no extra description, just the shape)

conn_log_options() :: [conn_log_option()].

Options that can be given to the cth_conn_log hook, which is used for logging of NETCONF and Telnet connections. See ct_netconfc or ct_telnet for description and examples of how to use this hook.

conn_log_type() :: raw | pretty | html | silent.

(just the vibes)

handle() :: pid().

The identity (handle) of a connection.

key_or_name() :: config_key() | target_name().

(just the type)

target_name() :: atom().

A name and association to configuration data introduced through a require statement, or a call to ct:require/2, for example, ct:require(mynodename,{node,[telnet]}).

🛠️ FUNCTIONS 🛠️

abort_current_testcase(Reason)

       abort_current_testcase(Reason) -> ok | {error, ErrorReason}
                                       when
                                           Reason :: term(),
                                           ErrorReason ::
                                               no_testcase_running |
                                               parallel_group.

Aborts the currently executing test case. The user must know with certainty which test case is currently executing. The function is therefore only safe to call from a function that has been called (or synchronously invoked) by the test case.

Reason, the reason for aborting the test case, is printed in the test case log.

add_config(Callback, Config)

       add_config(Callback, Config) -> ok | {error, Reason}
                           when
                               Callback :: atom(),
                               Config :: string(),
                               Reason :: term().

Loads configuration variables using the specified callback module and configuration string. The callback module is to be either loaded or present in the code path. Loaded configuration variables can later be removed using function ct:remove_config/2.

break(Comment)

       break(Comment) -> ok | {error, Reason}
                      when
                          Comment :: string(),
                          Reason ::
                              {multiple_cases_running, TestCases} |
                              'enable break with release_shell option',
                          TestCases :: [atom()].

Cancels any active timetrap and pauses the execution of the current test case until the user calls function continue/0. The user can then interact with the Erlang node running the tests, for example, for debugging purposes or for manually executing a part of the test case. If a parallel group is executing, ct:break/2 is to be called instead.

A cancelled timetrap is not automatically reactivated after the break, but must be started explicitly with ct:timetrap/1.

In order for the break/continue functionality to work, Common Test must release the shell process controlling stdin. This is done by setting start option release_shell to true. For details, see section Running Tests from the Erlang Shell or from an Erlang Program in the User's Guide.

break(TestCase, Comment)

       break(TestCase, Comment) -> ok | {error, Reason}
                      when
                          TestCase :: atom(),
                          Comment :: string(),
                          Reason ::
                              'test case not running' |
                              'enable break with release_shell option'.

Works the same way as ct:break/1, only argument TestCase makes it possible to pause a test case executing in a parallel group. Function ct:continue/1 is to be used to resume execution of TestCase.

For details, see ct:break/1.

capture_get(ExclCategories)

       capture_get(ExclCategories) -> ListOfStrings
                            when
                                ExclCategories :: [atom()],
                                ListOfStrings :: [string()].

Returns and purges the list of text strings buffered during the latest session of capturing printouts to stdout. Log categories that are to be ignored in ListOfStrings can be specified with ExclCategories. If ExclCategories = [], no filtering takes place.

See also ct:capture_start/0, ct:capture_stop/0, ct:log/3.

capture_start()

       capture_start() -> ok.

Starts capturing all text strings printed to stdout during execution of the test case.

See also ct:capture_get/1, ct:capture_stop/0.

capture_stop()

       capture_stop() -> ok.

Stops capturing text strings (a session started with capture_start/0).

See also ct:capture_get/1, ct:capture_start/0.

comment(Comment)

       comment(Comment) -> ok when Comment :: term().

Prints the specified Comment in the comment field in the table on the test suite result page.

If called several times, only the last comment is printed. The test case return value {comment,Comment} overwrites the string set by this function.

WARNING: This function (and ct:comment/2) should only be called from the process running the common_test functions, typically the test case process or its descendant. Be aware that if you spawn a new process in one common_test function (e.g. init_per_suite) and then execute ct:comment/1-2 in that new process while the test execution has moved to another common_test function (e.g. init_per_testcase or test case function) the comment might not be printed in the correct place in HTML logs or at all. Calling this function from a process that is neither a common_test process, nor its descendant does not result in the comment being printed anywhere.

comment(Format, Args)

       comment(Format, Args) -> ok
                        when Format :: string(), Args :: list().

Prints the formatted string in the comment field in the table on the test suite result page.

Arguments Format and Args are used in a call to io_lib:format/2 to create the comment string. The behavior of comment/2 is otherwise the same as function ct:comment/1.

continue()

       continue() -> ok.

This function must be called to continue after a test case (not executing in a parallel group) has called function ct:break/1.

continue(TestCase)

       continue(TestCase) -> ok when TestCase :: atom().

This function must be called to continue after a test case has called ct:break/2. If the paused test case, TestCase, executes in a parallel group, this function, rather than continue/0, must be used to let the test case proceed.

decrypt_config_file(EncryptFileName, TargetFileName)

       decrypt_config_file(EncryptFileName, TargetFileName) ->
                                    ok | {error, Reason}
                                    when
                                        EncryptFileName :: string(),
                                        TargetFileName :: string(),
                                        Reason :: term().

Decrypts EncryptFileName, previously generated with ct:encrypt_config_file/2,3. The original file contents is saved in the target file. The encryption key, a string, must be available in a text file named .ct_config.crypt, either in the current directory, or the home directory of the user (it is searched for in that order).

decrypt_config_file(EncryptFileName, TargetFileName, KeyOrFile)

       decrypt_config_file(EncryptFileName, TargetFileName, KeyOrFile) ->
                                    ok | {error, Reason}
                                    when
                                        EncryptFileName :: string(),
                                        TargetFileName :: string(),
                                        KeyOrFile ::
                                            {key, string()} | {file, string()},
                                        Reason :: term().

Decrypts EncryptFileName, previously generated with ct:encrypt_config_file/2,3. The original file contents is saved in the target file. The key must have the same value as that used for encryption.

encrypt_config_file(SrcFileName, EncryptFileName)

       encrypt_config_file(SrcFileName, EncryptFileName) ->
                                    ok | {error, Reason}
                                    when
                                        SrcFileName :: string(),
                                        EncryptFileName :: string(),
                                        Reason :: term().

Encrypts the source configuration file with DES3 and saves the result in file EncryptFileName. The key, a string, must be available in a text file named .ct_config.crypt, either in the current directory, or the home directory of the user (it is searched for in that order).

For information about using encrypted configuration files when running tests, see section Encrypted Configuration Files in the User's Guide.

For details on DES3 encryption/decryption, see application Crypto.

encrypt_config_file(SrcFileName, EncryptFileName, KeyOrFile)

       encrypt_config_file(SrcFileName, EncryptFileName, KeyOrFile) ->
                                    ok | {error, Reason}
                                    when
                                        SrcFileName :: string(),
                                        EncryptFileName :: string(),
                                        KeyOrFile ::
                                            {key, string()} | {file, string()},
                                        Reason :: term().

Encrypts the source configuration file with DES3 and saves the result in the target file EncryptFileName. The encryption key to use is either the value in {key,Key} or the value stored in the file specified by {file,File}.

For information about using encrypted configuration files when running tests, see section Encrypted Configuration Files in the User's Guide.

For details on DES3 encryption/decryption, see application Crypto.

  • fail(Reason) -> no_return() when Reason :: term(). — Kills the test case you're in, dead, gone, using the error Reason you hand it. It does not come back, bestie. no_return() is not a suggestion. 💀
  • fail(Format, Args) -> no_return() when Format :: io:format(), Args :: [term()]. — Same vibe, but instead of a bare Reason you give it a format string plus a list of values, and those values get fed to io_lib:format/2 to build the error message. Bye-bye test case. ✨
  • get_config(Required, Default, Opts) -> ValueOrElement when Required :: KeyOrName | {KeyOrName, SubKey} | {KeyOrName, SubKey, SubKey}, KeyOrName :: key_or_name(), SubKey :: atom(), Default :: term(), Opts :: [Opt], Opt :: element | all, ValueOrElement :: term() | Default. — This one reads your config values, and it is genuinely the main character of config reading. 🎀

It hands back the matching values or the matching config elements, and you can ask for them by config variable key or by whatever name you gave that key (if you set one up with ct:require/2 or a require statement). Name > key energy, honestly.

Example time. Say your config file looks like this:

{unix,[{telnet,IpAddr},
       {user,[{username,Username},
              {password,Password}]}]}.

Then:

ct:get_config(unix,Default) -> [{telnet,IpAddr},
 {user, [{username,Username}, {password,Password}]}]
ct:get_config({unix,telnet},Default) -> IpAddr
ct:get_config({unix,user,username},Default) -> Username
ct:get_config({unix,ftp},Default) -> Default
ct:get_config(unknownkey,Default) -> Default

If a config variable key has been associated with a name (via ct:require/2 or a require statement), you can just use that name instead of the key to read the value, and that's iconic:

ct:require(myuser,{unix,user}) -> ok.
ct:get_config(myuser,Default) -> [{username,Username}, {password,Password}]

If the same config variable is defined in several files, use the all option to grab every possible value. You get them back in a list, and the order of the elements matches the order the config files were specified at startup. Order matters, no cap.

If you want the actual config elements (those key-value tuples) back instead of just the values, use the element option. Those come back shaped like {Required,Value}.

See also ct:get_config/1, ct:get_config/2, ct:require/1, ct:require/2.

  • get_event_mgr_ref() -> EvMgrRef when EvMgrRef :: atom(). — Grabs you a reference to the Common Test event manager. What's it for? Say you wanna add your own user-specific event handler while tests are running. That. Obsessed. 💅

Example:

gen_event:add_handler(ct:get_event_mgr_ref(), my_ev_h, [])
  • get_progname() -> string(). — Returns the command that was used to start this Erlang instance. And if it can't find that info? You get the string "no_prog_name" back. Rude but honest. 🙈
  • get_status() -> TestStatus | {error, Reason} | no_tests_running when TestStatus :: [StatusElem], StatusElem :: {current, TestCaseInfo} | {successful, Successful} | {failed, Failed} | {skipped, Skipped} | {total, Total}, TestCaseInfo :: {Suite, TestCase} | [{Suite, TestCase}], Suite :: atom(), TestCase :: atom(), Successful :: integer(), Failed :: integer(), Skipped :: {UserSkipped, AutoSkipped}, UserSkipped :: integer(), AutoSkipped :: integer(), Total :: integer(), Reason :: term(). — Live status check on the test that's currently running. The list you get tells you which test case is executing right now (and it's a list of cases when a parallel test case group is doing its thing), plus the counters for successful, failed, skipped, and total test cases so far. Receipts for days. 📊
  • get_target_name(Handle) -> {ok, TargetName} | {error, Reason} when Handle :: handle(), TargetName :: target_name(), Reason :: term(). — Tells you the name of the target that the given connection belongs to. Which connection? The one you pass as Handle. 🦋
  • get_testspec_terms() -> TestSpecTerms | undefined when TestSpecTerms :: [{Tag, Value}], Tag :: atom(), Value :: [term()]. — Hands you a list of every test specification term that was used to configure and run this test. The whole paper trail, bestie.
  • get_testspec_terms(Tags) -> TestSpecTerms | undefined when Tags :: [Tag] | Tag, Tag :: atom(), TestSpecTerms :: [{Tag, Value}] | {Tag, Value}, Value :: [{Node, term()}] | [term()], Node :: atom(). — Now you're picking specific terms out of the test specification that configured and ran this test. Tag is any valid test specification tag — think label, config, or logdir, iykyk. And user-specific terms are readable too, as long as the allow_user_terms option is set. 🔥

Heads up: every value tuple you get back — user terms aside — has the node name as its first element.

Want test terms specifically? Use Tag = tests, not suites, groups, or cases. Then Value is the list of all tests, shaped like [{Node,Dir,[{TestSpec,GroupsAndCases1},...]},...], where GroupsAndCases = [{Group,[Case]}] | [Case].

  • get_timetrap_info() -> {Time, {Scaling, ScaleVal}} when Time :: timeout(), Scaling :: boolean(), ScaleVal :: integer(). — Reads the timetrap info for the test case you're in right now. Scaling tells you whether Common Test is gonna try to auto-compensate those timetraps for runtime delays that get introduced by stuff like cover. ScaleVal is the current scaling multiplier (and it's always 1 if scaling is off, periodt). One lil thing to clock: Time is not the scaled result. ✨
  • get_verbosity(Category) -> Level | undefined when Category :: default | atom(), Level :: integer(). — Returns the verbosity level for the logging category you name. Full details live in the User's Guide. And if you wanna read the general verbosity level, pass the value default. 🎀
  • install(Opts) -> ok | {error, Reason} when Opts :: [Opt], Opt :: {config, ConfigFiles} | {event_handler, Modules} | {decrypt, KeyOrFile}, ConfigFiles :: [ConfigFile], ConfigFile :: string(), Modules :: [atom()], KeyOrFile :: {key, Key} | {file, KeyFile}, Key :: string(), KeyFile :: string(), Reason :: term(). — Installs your configuration files and your event handlers. 💖

Run this function once, before the very first test. Not twice, not mid-run, once.

Example:

install([{config,["config_node.ctc","config_user.ctc"]}])

Also, good news, you don't have to remember this one: it's automatically run by the ct_run program. 😌

  • listenv(Telnet) -> {ok, [Env]} | {error, Reason} when Telnet :: ct_telnet:connection(), Env :: {Key, Value}, Key :: string(), Value :: string(), Reason :: term(). — Runs the command listenv over the Telnet connection you specify and gives you the result as a list of key-value pairs. Simple, clean, slay.
  • log(Format) -> ok when Format :: string(). — Literally equivalent to log(default, ?STD_IMPORTANCE, Format, [], []). Same thing, short version. 🌸
  • log(Category, Importance, Format, FormatArgs, Opts) -> ok when Category :: atom() | integer() | string(), Importance :: integer(), Format :: string(), FormatArgs :: list(), Opts :: [Opt], Opt :: {heading, string()} | no_css | esc_chars. — Prints straight from a test case into the log file. That's the whole job, and it does it beautifully.

It's meant for printing a string directly from a test case to the test case log file. No middleman.

The default Category is default, the default Importance is ?STD_IMPORTANCE, and the default for FormatArgs is [].

Wanna know the lore on Category, Importance, and the no_css option? Check the section Logging - Categories and Verbosity Levels in the User's Guide.

Big warning energy here: Common Test will not escape special HTML characters (<, > and &) in text you print with this function — unless, that is, you use the esc_chars option. Don't say nobody told you. 💀

  • make_priv_dir() -> ok | {error, Reason} when Reason :: term(). — Here's the deal: if the test was started with the create_priv_dir option set to manual_per_tc, then for the test case to actually use its private directory, it has to create it first by calling this function. It's giving "you built the room, now unlock the door." 🔑
  • notify(Name, Data) -> ok when Name :: atom(), Data :: term(). — Sends an asynchronous notification of type Name with Data to the Common Test event manager. Then any installed event manager can catch it later. Cute lil fire-and-forget. 🫶

See also gen_event.

  • pal(Format) -> ok when Format :: string(). — Equivalent to pal(default, ?STD_IMPORTANCE, Format, []). ✨
  • pal(Category, Importance, Format, FormatArgs, Opts) -> ok when Category :: atom() | integer() | string(), Importance :: integer(), Format :: string(), FormatArgs :: list(), Opts :: [Opt], Opt :: {heading, string()} | no_css. — You already know the drill. 🌈

Okay so bestie, next up — this one is giving "I do TWO things at once" ✨💅

the print-and-log one 🫶

Prints and logs from a test case, simultaneously, because why choose one.

This function is meant for printing a string from a test case, both to the test case log file and to the console. Iconic dual-threat behaviour, no cap.

Default Category is default, default Importance is ?STD_IMPORTANCE, and default value for FormatArgs is [].

For details on Category and Importance, see section Logging - Categories and Verbosity Levels in the User's Guide.

Heads up bestie: special characters in the text (<, >, and &) will be escaped by Common Test before the text is printed to the log file. So don't come crying when your angle brackets get the glow-up. 😭

parse_table(Data) -> {Heading, Table}

parse_table(Data) -> {Heading, Table}
                        when
                            Data :: [string()],
                            Heading :: tuple(),
                            Table :: [tuple()].

Parses the printout from an SQL table and hands you back a list of tuples. Very "turning chaos into structure" of it. 🧚

The printout you're parsing is typically the result of a select command in SQL. The returned Table is a list of tuples, where each tuple is a row in the table.

Heading is a tuple of strings representing the headings of each column in the table. Column names, but make it fancy. 💖

print(Format) -> ok when Format :: string().

Equivalent to print(default, ?STD_IMPORTANCE, Format, [], []). It's the lil shortcut version, the "I don't wanna type all that" edition. 🌸

print(Category, Importance, Format, FormatArgs, Opts) -> ok
                when
                    Category :: atom() | integer() | string(),
                    Importance :: integer(),
                    Format :: string(),
                    FormatArgs :: list(),
                    Opts :: [Opt],
                    Opt :: {heading, string()}.

Prints from a test case to the console. Just the console, babe — this one ain't writing to the log file. 📢

This function is meant for printing a string from a test case to the console.

Default Category is default, default Importance is ?STD_IMPORTANCE, and default value for FormatArgs is [].

For details on Category and Importance, see section Logging - Categories and Verbosity Levels in the User's Guide.

reload_config(Required) -> ValueOrElement | {error, Reason}

reload_config(Required) -> ValueOrElement | {error, Reason}
                        when
                            Required ::
                                KeyOrName |
                                {KeyOrName, SubKey} |
                                {KeyOrName, SubKey, SubKey},
                            KeyOrName :: key_or_name(),
                            SubKey :: atom(),
                            ValueOrElement :: term(),
                            Reason :: term().

Reloads the configuration file containing the specified configuration key. Refresh button but for config, we love to see it. 🔄✨

This function updates the configuration data from which the specified configuration variable was read, and returns the (possibly) new value of this variable.

Careful, bestie: if some variables were present in the configuration, but are not loaded using this function, they are removed from the configuration table together with their aliases. So like… actually don't sleep on this one. 💀

remaining_test_procs() -> {TestProcs, SharedGL, OtherGLs}

remaining_test_procs() -> {TestProcs, SharedGL, OtherGLs}
                             when
                                 TestProcs :: [{pid(), GL}],
                                 GL :: pid(),
                                 SharedGL :: pid(),
                                 OtherGLs :: [pid()].

This function will return the identity of test- and group leader processes that are still running at the time of this call. Like a headcount of everybody who hasn't gone home yet. 🙈

TestProcs are processes in the system that have a Common Test IO process as group leader. SharedGL is the central Common Test IO process, responsible for printing to log files for configuration functions and sequentially executing test cases. OtherGLs are Common Test IO processes that print to log files for test cases in parallel test case groups.

The process information returned by this function may be used to locate and terminate remaining processes after tests have finished executing. The function would typically by called from Common Test Hook functions.

Note that processes that execute configuration functions or test cases are never included in TestProcs. It is therefore safe to use post configuration hook functions (such as post_end_per_suite, post_end_per_group, post_end_per_testcase) to terminate all processes in TestProcs that have the current group leader process as its group leader. Safe, queen. 👑
Note also that the shared group leader (SharedGL) must never be terminated by the user — only by Common Test. Hands off, that one's not yours. 🚫 Group leader processes for parallel test case groups (OtherGLs) may however be terminated in post_end_per_group hook functions.

remove_config(Callback, Config) -> ok

remove_config(Callback, Config) -> ok
                        when Callback :: atom(), Config :: string().

Removes configuration variables (together with their aliases) that were loaded with the specified callback module and configuration string. Cleanup era. 🧹💅

require(Required) -> ok | {error, Reason}

require(Required) -> ok | {error, Reason}
                    when
                        Required ::
                            Key | {Key, SubKeys} | {Key, SubKey, SubKeys},
                        Key :: atom(),
                        SubKeys :: SubKey | [SubKey],
                        SubKey :: atom(),
                        Reason :: term().

Checks if the required configuration is available. Arbitrarily deep tuples can be specified as Required. Only the last element of the tuple can be a list of SubKeys.

Let's do the examples, because this is one of those "iykyk" situations. 🍓

Example 1. Require the variable myvar:

ok = ct:require(myvar).

In this case the configuration file must at least contain:

{myvar,Value}.

Example 2. Require key myvar with subkeys sub1 and sub2:

ok = ct:require({myvar,[sub1,sub2]}).

In this case the configuration file must at least contain:

{myvar,[{sub1,Value},{sub2,Value}]}.

Example 3. Require key myvar with subkey sub1 with subsub1:

ok = ct:require({myvar,sub1,sub2}).

In this case the configuration file must at least contain:

{myvar,[{sub1,[{sub2,Value}]}]}.

See also ct:get_config/1, ct:get_config/2, ct:get_config/3, ct:require/2.

require(Name, Required) -> ok | {error, Reason}

require(Name, Required) -> ok | {error, Reason}
                    when
                        Name :: atom(),
                        Required ::
                            Key | {Key, SubKey} | {Key, SubKey, SubKey},
                        SubKey :: Key,
                        Key :: atom(),
                        Reason :: term().

Checks if the required configuration is available and gives it a name. Like adopting a config value and giving it a cute nickname. 🎀

The semantics for Required is the same as in ct:require/1 — with one catch: a list of SubKeys cannot be specified here.

If the requested data is available, the subentry is associated with Name so that the value of the element can be read with ct:get_config/1,2 provided Name is used instead of the whole Required term.

Example, bestie 💖 — require one node with a Telnet connection and an FTP connection. Name the node a:

ok = ct:require(a,{machine,node}).

All references to this node can then use the node name. For example, a file over FTP is fetched like follows:

ok = ct:ftp_get(a,RemoteFile,LocalFile).

For this to work, the configuration file must at least contain:

{machine,[{node,[{telnet,IpAddr},{ftp,IpAddr}]}]}.
NOTE: The behavior of this function changed radically in Common Test 1.6.2. Not a drill. 😭 To keep some backwards compatibility, it is still possible to do: ct:require(a,{node,[telnet,ftp]}). This associates the name a with the top-level node entry. For this to work, the configuration file must at least contain: {node,[{telnet,IpAddr},{ftp,IpAddr}]}.

See also ct:get_config/1, ct:get_config/2, ct:get_config/3, ct:require/1.

run(TestDirs) -> Result

run(TestDirs) -> Result
                when
                    TestDirs :: TestDir | [TestDir],
                    TestDir :: string(),
                    Result :: [TestResult] | {error, Reason},
                    TestResult :: term(),
                    Reason :: term().

Runs all test cases in all suites in the specified directories. The whole thing. The entire meal. 🍽️✨

See also ct:run/3.

run(TestDir, Suite) -> Result

run(TestDir, Suite) -> Result
                when
                    TestDir :: string(),
                    Suite :: atom(),
                    Result :: [TestResult] | {error, Reason},
                    TestResult :: term(),
                    Reason :: term().

Runs all test cases in the specified suite. One suite, but make it thorough. 💅

See also ct:run/3.

run(TestDir, Suite, Cases) -> Result

run(TestDir, Suite, Cases) -> Result
                when
                    TestDir :: string(),
                    Suite :: atom(),
                    Cases :: atom() | [atom()],
                    Result :: [TestResult] | {error, Reason},
                    TestResult :: term(),
                    Reason :: term().

Runs the specified test cases. Cherry-picking, but for tests. 🍒

Requires that ct:install/1 has been run first. No skipping that step, bestie.

Suites (*_SUITE.erl) files must be stored in TestDir or TestDir/test. All suites are compiled when the test is run. Compile-on-run, we love an efficient queen. 👑

run_test(Opts) -> Result</p> 💅

run_test(Opts) -> Result
                  when
                      Opts :: [OptTuples],
                      OptTuples ::
                          {dir, TestDirs} |
                          {suite, Suites} |
                          {group, Groups} |
                          {testcase, Cases} |
                          {spec, TestSpecs} |
                          {join_specs, boolean()} |
                          {label, Label} |
                          {config, CfgFiles} |
                          {userconfig, UserConfig} |
                          {allow_user_terms, boolean()} |
                          {logdir, LogDir} |
                          {silent_connections, Conns} |
                          {stylesheet, CSSFile} |
                          {cover, CoverSpecFile} |
                          {cover_stop, boolean()} |
                          {step, StepOpts} |
                          {event_handler, EventHandlers} |
                          {include, InclDirs} |
                          {auto_compile, boolean()} |
                          {abort_if_missing_suites, boolean()} |
                          {create_priv_dir, CreatePrivDir} |
                          {multiply_timetraps, M} |
                          {scale_timetraps, boolean()} |
                          {repeat, N} |
                          {duration, DurTime} |
                          {until, StopTime} |
                          {force_stop, ForceStop} |
                          {decrypt, DecryptKeyOrFile} |
                          {refresh_logs, LogDir} |
                          {logopts, LogOpts} |
                          {verbosity, VLevels} |
                          {basic_html, boolean()} |
                          {esc_chars, boolean()} |
                          {keep_logs, KeepSpec} |
                          {ct_hooks, CTHs} |
                          {ct_hooks_order, CTHsOrder} |
                          {enable_builtin_hooks, boolean()} |
                          {release_shell, boolean()},
                      TestDirs :: [string()] | string(),
                      Suites ::
                          [string()] | [atom()] | string() | atom(),
                      Cases :: [atom()] | atom(),
                      Groups :: GroupNameOrPath | [GroupNameOrPath],
                      GroupNameOrPath :: [atom()] | atom() | all,
                      TestSpecs :: [string()] | string(),
                      Label :: string() | atom(),
                      CfgFiles :: [string()] | string(),
                      UserConfig ::
                          [{CallbackMod, CfgStrings}] |
                          {CallbackMod, CfgStrings},
                      CallbackMod :: atom(),
                      CfgStrings :: [string()] | string(),
                      LogDir :: string(),
                      Conns :: all | [atom()],
                      CSSFile :: string(),
                      CoverSpecFile :: string(),
                      StepOpts :: [StepOpt],
                      StepOpt :: config | keep_inactive,
                      EventHandlers :: EH | [EH],
                      EH ::
                          atom() |
                          {atom(), InitArgs} |
                          {[atom()], InitArgs},
                      InitArgs :: [term()],
                      InclDirs :: [string()] | string(),
                      CreatePrivDir ::
                          auto_per_run | auto_per_tc | manual_per_tc,
                      M :: integer(),
                      N :: integer(),
                      DurTime :: HHMMSS,
                      HHMMSS :: string(),
                      StopTime :: YYMoMoDDHHMMSS | HHMMSS,
                      YYMoMoDDHHMMSS :: string(),
                      ForceStop :: skip_rest | boolean(),
                      DecryptKeyOrFile ::
                          {key, DecryptKey} | {file, DecryptFile},
                      DecryptKey :: string(),
                      DecryptFile :: string(),
                      LogOpts :: [LogOpt],
                      LogOpt :: no_nl | no_src,
                      VLevels :: VLevel | [{Category, VLevel}],
                      VLevel :: integer(),
                      Category :: atom(),
                      KeepSpec :: all | pos_integer(),
                      CTHs :: [CTHModule | {CTHModule, CTHInitArgs}],
                      CTHsOrder :: atom(),
                      CTHModule :: atom(),
                      CTHInitArgs :: term(),
                      Result ::
                          {Ok, Failed, {UserSkipped, AutoSkipped}} |
                          TestRunnerPid |
                          {error, Reason},
                      Ok :: integer(),
                      Failed :: integer(),
                      UserSkipped :: integer(),
                      AutoSkipped :: integer(),
                      TestRunnerPid :: pid(),
                      Reason :: term().

okay bestie, this is THE big one ✨ — it runs your tests based on whatever combo of options you packed into Opts. Those options are literally the same ones you'd hand to the ct_run program, so peek at Run Tests from Command Line</p> in the ct_run manual page if you wanna know all the tea 🫖.

Some lil notes so you don't get caught slippin' 🎀:

  • a TestDir here can point out the path to a Suite. slay.
  • the testcase option is the same thing as option -case in program ct_run. twin vibes fr 💅
  • config files you list in Opts get installed automatically at startup. no manual work, we love that 🦋

If release_shell == true, you get TestRunnerPid back instead. for the deets on that, go read ct:break/1 👀.

And Reason? that's just telling you what kinda error you ran into. rude of it but okay 😭

run_testspec(TestSpec) -> Result</p> 🌸

run_testspec(TestSpec) -> Result
                          when
                              TestSpec :: [term()],
                              Result ::
                                  {Ok, Failed, {UserSkipped, AutoSkipped}} |
                                  {error, Reason},
                              Ok :: integer(),
                              Failed :: integer(),
                              UserSkipped :: integer(),
                              AutoSkipped :: integer(),
                              Reason :: term().

runs a test specified by TestSpec, and it uses the exact same terms</> you'd find in test specification files — consistency queen 👑.

Reason = the type of error you bumped into. again. it's giving "we need to talk" 💀

set_verbosity(Category, Level) -> ok</p> ✨

set_verbosity(Category, Level) -> ok
                          when
                              Category :: default | atom(),
                              Level :: integer().

use this to set — or modify — the verbosity level for a logging category 🎚️. details are in the User's Guide if you want the full lore. and if you pass the value default, you're setting the general verbosity level. periodt 💅

sleep(Time) -> ok</p> 😴

sleep(Time) -> ok
                  when
                      Time ::
                          {hours, Hours} |
                          {minutes, Mins} |
                          {seconds, Secs} |
                          Millisecs,
                      Hours :: integer(),
                      Mins :: integer(),
                      Secs :: integer(),
                      Millisecs :: timeout() | float().

bestie this is like timer:sleep/1 from STDLIB — it suspends the test case for however long you say 🛌. BUT. plot twist: this one also multiplies Time with the multiply_timetraps value (if that's set), and under certain circumstances it also scales up the time automatically if scale_timetraps is set to true (default is false, so like, don't assume ✋).

start_interactive() -> ok.</p> 🎀

starts Common Test in interactive mode. obsessed. 🫶

from this mode, all test case support functions can be executed straight from the Erlang shell. iconic. and you can also kick off interactive mode from the OS command line with:

ct_run -shell [-config File...]

now listen, this part matters ngl 👇 — if any functions (for example, Telnet or FTP) that use "required configuration data" are gonna be called from the Erlang shell, you must first require that configuration data with ct:require/2. no skipping, bestie.

Example:

> ct:require(unix_telnet, unix).
ok
> ct_telnet:open(unix_telnet).
{ok,<0.105.0>}
> ct_telnet:cmd(unix_telnet, "ls .").
{ok,["ls","file1  ...",...]}

step(TestDir, Suite, Case) -> Result</p> 🐛

step(TestDir, Suite, Case) -> Result
                when
                    TestDir :: string(),
                    Suite :: atom(),
                    Case :: atom(),
                    Result :: term().

steps through a test case with the debugger — very detective, very main character 🔍.

see also ct:run/3.

step(TestDir, Suite, Case, Opts) -> Result</p> 🐛✨

step(TestDir, Suite, Case, Opts) -> Result
                when
                    TestDir :: string(),
                    Suite :: atom(),
                    Case :: atom(),
                    Opts :: [Opt],
                    Opt :: config | keep_inactive,
                    Result :: term().

same debugger stepping energy, but with options 💅. and if you specify option config, breakpoints also get set on the configuration functions in Suite. multitasking fr.

see also ct:run/3.

stop_interactive() -> ok.</p> 👋

exits the interactive mode. bye bestie 🦋

see also ct:start_interactive/0.

sync_notify(Name, Data) -> ok</p> 📣

sync_notify(Name, Data) -> ok when Name :: atom(), Data :: term().

sends a synchronous notification of type Name with Data to the Common Test event manager 📨. this can later be caught by any installed event manager — so it's giving "message in a bottle but make it reliable" 🫙✨.

see also gen_event.

testcases(TestDir, Suite) -> Testcases | {error, Reason}</p> 📝

testcases(TestDir, Suite) -> Testcases | {error, Reason}
                      when
                          TestDir :: string(),
                          Suite :: atom(),
                          Testcases :: list(),
                          Reason :: term().

returns all the test cases in the specified suite. every single one. no crumbs left. 💖

timetrap(Time) -> infinity | pid()</p> ⏰

timetrap(Time) -> infinity | pid()
                     when
                         Time ::
                             {hours, Hours} |
                             {minutes, Mins} |
                             {seconds, Secs} |
                             timeout() |
                             Func,
                         Hours :: integer(),
                         Mins :: integer(),
                         Secs :: integer(),
                         Func :: {M, F, A} | function(),
                         M :: atom(),
                         F :: atom(),
                         A :: list().

sets a new timetrap for the currently running test case ⏳. the clock is ticking bestie, tick tock 🙈.

and if the argument is Func? then the timetrap is triggered when this function returns. plot twist: Func can also return a new Time value, and in that case, that returned value becomes the value for the new timetrap. sneaky but slay 🔥

Okay bestie, two more of these and they're literally twins — same name, different arity, and the vibe check is: "gimme the userdata you stashed, k thanks." 💖

The two-arg one 🫶

       userdata(TestDir, Suite) -> SuiteUserData | {error, Reason}
                         when
                             TestDir :: string(),
                             Suite :: atom(),
                             SuiteUserData :: [term()],
                             Reason :: term().

She takes a TestDir (yep, a string()) and a Suite (an atom()), and hands you back SuiteUserData — a nice little list of term()s ✨ — or, if it's not feeling it, {error, Reason} with some term() for Reason 😭.

What is she even fetching? She returns any data specified with tag userdata in the list of tuples that suite/0 handed back. So if your suite/0 was like "hey here's my info bestie" and it was tagged userdata, this is the function that goes and gets it. No cap 💅

The three-arg one 🦋

       userdata(TestDir, Suite, Case :: GroupOrCase) ->
                         TCUserData | {error, Reason}
                         when
                             TestDir :: string(),
                             Suite :: atom(),
                             GroupOrCase :: {group, GroupName} | atom(),
                             GroupName :: atom(),
                             TCUserData :: [term()],
                             Reason :: term().

Same energy, but now you passed a Case — which, plot twist, is a GroupOrCase, so it's either {group, GroupName} where GroupName is an atom(), or just a plain atom() 🎀. That's giving flexibility, honestly.

She gives you back TCUserData, a [term()] 🍓 — or the classic {error, Reason}, with Reason being a term() 😭.

And where's the data coming from this time? She returns any data specified with tag userdata in the list of tuples returned from Suite:group(GroupName) or Suite:Case(). So whichever one applies, that's your source. Periodt 👑

Not me telling you to go check what group/1 and your case function actually spit out before you go blaming userdata for an empty list 🙈 the answer is in the return value, iykyk.

Aaaand that's the whole page, bestie, you made it to the end and you ate 💅✨ This absolute icon of a manpage came to you from Ericsson AB, documenting common_test version 1.31.1 🎀 Go forth and test your Erlang like the main character you are 👑💖

📦 where this page comes from