ct_suite(3erl)

📦
version 25.3.2.8 · 4 versions
, in ubuntu noble

💖 NAME 💖

ct_suite(3erl) is the Erlang Module Definition for ct_suite, and ct_suite is literally -behaviour(ct_suite). 💅

🌸 DESCRIPTION 🌸

Okay so this is the map of the mandatory and optional test suite functions that Common Test calls while it's running your tests. For more deets, peek section Writing Test Suites in the User's Guide. ✨

🌈 DATA TYPES 🌈

  • ct_testname() = atom() — the name of the testcase function. 🎀
  • ct_groupname() = atom() — the name of the test group. 🦋
  • ct_config() = [{Key :: atom(), Value :: term()}] — the configuration data that can be modified. 💖
  • ct_status() = ok | skipped | failed — the status value for a nested subgroup. 🌸
  • ct_group_def() — the test group definition, as returned by Module:groups/0. 🌈
  • ct_test_def() — the test suite definition, as returned by Module:all/0. ⭐
  • ct_info() — the test suite information, as returned by Module:suite/0, Module:group/1 and Module:Testcase/0. 👑

🦋 CALLBACK FUNCTIONS 🦋

The following functions are to be exported from a ct_suite callback module in order to define the callback interface for a test suite. 🫶

🚀 EXPORTS 🚀

🌟 Module:all() 🌟

Module:all() -> [ct_test_def()] | {skip, Reason}

Types:

ct_test_def() = TestCase | {group, GroupName} | {group, GroupName, Properties} | {group, GroupName, Properties, SubGroups}
TestCase = ct_testname()
GroupName = ct_groupname()
Properties = [parallel | sequence | Shuffle | {RepeatType, N}] | default
SubGroups = [{GroupName, Properties} | {GroupName, Properties, SubGroups}]
Shuffle = shuffle | {shuffle, Seed}
Seed = {integer(), integer(), integer()}
RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | repeat_until_any_ok | repeat_until_any_fail
N = integer() | forever
Reason = term()

MANDATORY 💥

Returns the list of all test cases and test case groups in the test suite module to be executed. This list also specifies the order the cases and groups are executed by Common Test. A test case is represented by an atom, the name of the test case function, or a testcase tuple indicating that the test case shall be repeated. A test case group is represented by a group tuple, where GroupName, an atom, is the name of the group (defined in Module:groups/0). Execution properties for groups can also be specified, both for a top-level group and for any of its subgroups. Group execution properties specified here override properties in the group definition (see Module:groups/0). (With value default, the group definition properties are used).

If {skip, Reason} is returned, all test cases in the module are skipped and Reason is printed on the HTML result page. 😭

For details on groups, see section Test Case Groups in the User's Guide. 💖

🌟 Module:groups() 🌟

Module:groups() -> [ct_group_def()]

Types:

ct_group_def() = {GroupName, Properties, GroupsAndTestCases}
GroupName = ct_groupname()
Properties = [parallel | sequence | Shuffle | {RepeatType, N}]
GroupsAndTestCases = [Group | {group, GroupName} | TestCase]
TestCase = ct_testname()
Shuffle = shuffle | {shuffle, Seed}
Seed = {integer(), integer(), integer()}
RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | repeat_until_any_ok | repeat_until_any_fail
N = integer() | forever

OPTIONAL ✨

Defines test case groups. For details, see section Test Case Groups in the User's Guide. 🦋

🌟 Module:suite() 🌟

Module:suite() -> [ct_info()]

Types:

ct_info() = {timetrap, Time} | {require, Required} | {require, Name, Required} | {userdata, UserData} | {silent_connections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
Time = TimeVal | TimeFunc
TimeVal = MilliSec | {seconds, integer()} | {minutes, integer()} | {hours, integer()}
TimeFunc = {Mod, Func, Args} | Fun
MilliSec = integer()
Mod = atom()
Func = atom()
Args = list()
Fun = fun()
Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, SubKey, SubKeys}
Key = atom()
SubKeys = SubKey | [SubKey]
SubKey = atom()
Name = atom()
UserData = term()
Conns = [atom()]
CSSFile = string()
CTHs = [CTHModule |
 {CTHModule, CTHInitArgs} |
 {CTHModule, CTHInitArgs, CTHPriority}]
CTHModule = atom()
CTHInitArgs = term()
CTHPriority = integer()

OPTIONAL 🌸

The test suite information function. Returns a list of tagged tuples specifying various properties related to the execution of this test suite (common for all test cases in the suite). 👑

  • Tag timetrap sets the maximum time that each test case is allowed to execute (including Module:init_per_testcase/2 and Module:end_per_testcase/2). If the timetrap time is exceeded, the test case fails with reason timetrap_timeout. A TimeFunc function can be used to set a new timetrap by returning a TimeVal. It can also be used to trigger a timetrap time-out by, at some point, returning a value other than a TimeVal. For details, see section Timetrap Time-Outs in the User's Guide. ⏳
  • Tag require specifies configuration variables required by test cases (or configuration functions) in the suite. If the required configuration variables are not found in any of the configuration files, all test cases are skipped. For details about the require functionality, see function ct:require/1,2. 🔑
  • With userdata, the user can specify any test suite-related information, which can be read by calling ct:userdata/2. 🍓
  • Tag ct_hooks specifies the Common Test Hooks to be run with this suite. 🪝

Other tuples than the ones defined are ignored. 🙈

For details about the test suite information function, see section Test Suite Information Function in the User's Guide. 💜

🌟 Module:init_per_suite(Config) 🌟

Module:init_per_suite(Config) -> NewConfig | {skip, Reason} | {skip_and_save, Reason, SaveConfig}

Types:

Config = NewConfig = SaveConfig = ct_config()
Reason = term()
OPTIONAL; if this function is defined, then Module:end_per_suite/1 must also be defined. 🫶

This configuration function is called as the first function in the suite. It typically contains initializations that are common for all test cases in the suite, and that must only be done once. Parameter Config is the configuration data that can be modified. Whatever is returned from this function is specified as Config to all configuration functions and test cases in the suite. ✨

If {skip, Reason} is returned, all test cases in the suite are skipped and Reason is printed in the overview log for the suite. 😭

For information on save_config and skip_and_save, see section Saving Configuration Data in the User's Guide. 💾

🌟 Module:end_per_suite(Config) 🌟

Module:end_per_suite(Config) -> term() | {save_config, SaveConfig}

Types:

Config = SaveConfig = ct_config()
OPTIONAL; if this function is defined, then Module:init_per_suite/1 must also be defined. 🫶

This function is called as the last test case in the suite. It is meant to be used for cleaning up after Module:init_per_suite/1. 🧹

For information on save_config, see section Saving Configuration Data in the User's Guide. 💾

🌟 Module:group(GroupName) 🌟

Module:group(GroupName) -> [ct_info()]

Types:

GroupName = ct_groupname()
ct_info() = {timetrap, Time} | {require, Required} | {require, Name, Required} | {userdata, UserData} | {silent_connections, Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
Time = TimeVal | TimeFunc
TimeVal = MilliSec | {seconds, integer()} | {minutes, integer()} | {hours, integer()}
TimeFunc = {Mod, Func, Args} | Fun
MilliSec = integer()
Mod = atom()
Func = atom()
Args = list()
Fun = fun()
Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key, SubKey, SubKeys}
Key = atom()
SubKeys = SubKey | [SubKey]
SubKey = atom()
Name = atom()
UserData = term()
Conns = [atom()]
CSSFile = string()
CTHs = [CTHModule |
 {CTHModule, CTHInitArgs} |
 {CTHModule, CTHInitArgs, CTHPriority}]
CTHModule = atom()
CTHInitArgs = term()
CTHPriority = integer()

OPTIONAL 🌈

The test case group information function. It is supposed to return a list of tagged tuples that specify various properties related to the execution of a test case group (that is, its test cases and subgroups). Properties set by Module:group/1 override properties with the same key that have been set previously by Module:suite/0. 👑

  • Tag timetrap sets the maximum time that each test case is allowed to execute (including Module:init_per_testcase/2 and Module:end_per_testcase/2). If the timetrap time is exceeded, the test case fails with reason timetrap_timeout. A TimeFunc function can be used to set a new timetrap by returning a TimeVal. It can also be used to trigger a timetrap time-out by, at some point, returning a value other than a TimeVal. For details, see section Timetrap Time-Outs in the User's Guide. ⏳
  • Tag require specifies configuration variables required by test cases (or configuration functions) in the suite. If the required configuration variables are not found in any of the configuration files, all test cases in this group are skipped. For details about the require functionality, see function ct:require/1,2. 🔑
  • With userdata, the user can specify any test case group related information that can be read by calling ct:userdata/2. 🍓
  • Tag ct_hooks specifies the Common Test Hooks to be run with this suite. 🪝

Other tuples than the ones defined are ignored. 🙈

For details about the test case group information function, see section Group Information Function in the User's Guide. 🦋

  • Module:init_per_group(GroupName, Config) -> NewConfig | {skip, Reason} 🎀

Types:

GroupName = ct_groupname()
Config = NewConfig = ct_config()
Reason = term()

OPTIONAL — but bestie, listen: if you define this one, then Module:end_per_group/2 must also be defined. They're a package deal, no cap. 💅

This configuration function gets called before the test case group runs. It's where you put the init stuff that's shared by all the test cases and subgroups in that group — the things you only wanna do once. GroupName is the name of the group, exactly as specified in the group definition (peep Module:groups/0). The Config parameter is the configuration data, and yes you can modify it. Whatever you return from this function gets handed as Config to every test case and subgroup in the group. ✨

If you return {skip, Reason}, literally all the test cases in the group get skipped and Reason gets printed in the overview log for the group. 🦋

Wanna know more about test case groups? Go to section Test Case Groups in the User's Guide. iykyk.

  • Module:end_per_group(GroupName, Config) -> term() | {return_group_result, Status} 🌸

Types:

GroupName = ct_groupname()
Config = ct_config()
Status = ct_status()

OPTIONAL — and again: if you define this, Module:init_per_group/2 must also be defined. Matching set, periodt.

This one runs after the test case group has finished executing. It's the cleanup era — the whole point is to tidy up after Module:init_per_group/2. 🧹

You can return a status value for a nested subgroup with {return_group_result, Status}. That status can then be picked up in Module:end_per_group/2 for the group one level above. Common Test also uses that status when deciding whether execution of a group should keep going when the property sequence or repeat_until_* is set. 👑

Details on test case groups? Section Test Case Groups in the User's Guide, go go go.

  • Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail, Reason} | {skip, Reason} 💖

Types:

TestCase = ct_testname()
Config = NewConfig = ct_config()
Reason = term()

OPTIONAL — and you already know the drill: define this, and Module:end_per_testcase/2 must be defined too. ✨

Called before each test case. TestCase is the test case name, and Config (that list of key-value tuples) is the configuration data you're free to modify. The NewConfig list you return from here is then given as Config to the test case. If you return {fail, Reason}, the test case gets marked as failed without being executed — brutal but efficient. 💀

If you return {skip, Reason}, the test case is skipped and Reason is printed in the overview log for the suite. 🦋

  • Module:end_per_testcase(TestCase, Config) -> term() | {fail, Reason} | {save_config, SaveConfig} 🎀

Types:

TestCase = ct_testname()
Config = SaveConfig = ct_config()
Reason = term()

OPTIONAL — and yes, if this exists then Module:init_per_testcase/2 must exist too. Always together, fr fr.

Called after each test case, and it's your moment to clean up after Module:init_per_testcase/2 and the test case itself. Any return value gets ignored — except {fail, Reason} and {save_config, SaveConfig}, those two actually mean something. 😭

By returning {fail, Reason}, the TestCase gets marked as faulty — even though it was successful in the sense that it returned a value instead of terminating. Sneaky little plot twist.

For the tea on save_config, see section Saving Configuration Data in the User's Guide.

  • Module:Testcase() -> [ct_info()] ⭐

Types:

ct_info() = {timetrap, Time} | {require, Required} | {require,
Name, Required} | {userdata, UserData} | {silent_connections,
Conns} | {stylesheet, CSSFile} | {ct_hooks, CTHs}
Time = TimeVal | TimeFunc
TimeVal = MilliSec | {seconds, integer()} | {minutes,
integer()} | {hours, integer()}
TimeFunc = {Mod, Func, Args} | Fun
MilliSec = integer()
Mod = atom()
Func = atom()
Args = list()
Fun = fun()
Required = Key | {Key, SubKeys} | {Key, SubKey} | {Key,
SubKey, SubKeys}
Key = atom()
SubKeys = SubKey | [SubKey]
SubKey = atom()
Name = atom()
UserData = term()
Conns = [atom()]
CSSFile = string()
CTHs = [CTHModule |
 {CTHModule, CTHInitArgs} |
 {CTHModule, CTHInitArgs, CTHPriority}]
CTHModule = atom()
CTHInitArgs = term()
CTHPriority = integer()

OPTIONAL

The test case information function, aka the vibe-setter for one specific test case. It's supposed to return a list of tagged tuples that specify all kinds of properties for how this particular test case runs. And here's the hierarchy tea: properties set by Module:Testcase/0 override properties set previously for the test case by Module:group/1 or Module:suite/0. Main character energy, honestly. 👑

The timetrap tag sets the maximum time the test case is allowed to execute. Blow past that timetrap and the test case fails with reason timetrap_timeout. Heads up: Module:init_per_testcase/2 and Module:end_per_testcase/2 are included in the timetrap time. A TimeFunc function can be used to set a new timetrap by returning a TimeVal. It can also be used to trigger a timetrap time-out by, at some point, returning a value that isn't a TimeVal. For the details, see section Timetrap Time-Outs in the User's Guide. ⏰

The require tag specifies configuration variables that the test case needs (or that init_per_testcase/2 or end_per_testcase/2 need). If those required configuration variables aren't found in any of the configuration files, the test case gets skipped. For all the details on the require functionality, see function ct:require/1,2. 🦋

If timetrap or require isn't set, the default values specified by Module:suite/0 (or Module:group/1) are used. Defaults are lowkey doing a lot of work here.

With userdata, you can specify literally any test case-related information you want, and it can be read back by calling ct:userdata/3. 🌈

Any tuples other than the ones defined here? Ignored. Not invited to the party. 🙈

For the full rundown on the test case information function, see section Test Case Information Function in the User's Guide.

  • Module:Testcase(Config) -> term() | {skip, Reason} | {comment, Comment} | {save_config, SaveConfig} | {skip_and_save, Reason, SaveConfig} | exit() 🔥

Types:

Config = SaveConfig = ct_config()
Reason = term()
Comment = string()

MANDATORY — this is the whole point, bestie, the main event. 👑

The implementation of a test case. Call the functions you wanna test and check the result. If something fails, make sure the function causes a runtime error or call ct:fail/1,2 — which also causes the test case process to terminate. No silent failures, we don't do that here. 💅

Elements from the Config list can, for example, be read with proplists:get_value/2 in STDLIB — or with the macro ?config defined in ct.hrl. Whatever feels right. ✨

If you decide you're not running the test case after all, just return {skip, Reason}. Reason then gets printed in field Comment on the HTML result page. 🦋

Wanna print some info in field Comment on the HTML result page? Return {comment, Comment}. Easy.

If the function returns literally anything else, the test case is considered successful. And the return value always gets printed in the test case log file, so it's not wasted. 📝

For all the details about test case implementation, see section Test Cases in the User's Guide.

For the info on save_config and skip_and_save, see section Saving Configuration Data in the User's Guide.

And that's the whole page, bestie! 🫶 Runs on common_test 1.24.0.1, brought to you by the fine folks at Ericsson AB — go forth and slay your test suites. 💅✨

📦 where this page comes from