ct_property_test(3)

📦
version 29.0.6 · 3 versions
, in ubuntu stonking

💖 ct_property_test — the Common Test glow-up for property-based testing 💖

okay bestie, so this is a page from the common_test manual, and it's about the module ct_property_test. It lives in the Erlang world and it's the lil' helper that lets you run property-based tests inside the Common Test framework. Let's get into it 🫶

✨ NAME ✨

ct_property_test — support in Common Test for running property-based tests. (yes that's it, that's the whole vibe ✨)

🌸 DESCRIPTION 🌸

So basically, this module helps you run property-based tests inside the Common Test framework. And it assumes that one (or more!) of the property testing tools is already installed:

  • QuickCheck,
  • PropEr, or
  • Triq.

The whole idea here is that you've got a Common Test test suite that calls a property testing tool, and you feed it special property test suites as defined by that tool. The tests themselves are collected in the application's test directory. Inside that test directory there's a subdirectory called property_test, and that's where everything needed for the property tests hangs out. The usual Erlang application directory structure is just assumed. no notes, that's the deal 💅

A typical Common Test test suite using ct_property_test is organized like this:

-module(my_prop_test_SUITE).
-compile(export_all).

-include_lib("common_test/include/ct.hrl").

all() -> [prop_ftp_case].

init_per_suite(Config) ->
    ct_property_test:init_per_suite(Config).

%%%---- test case
prop_ftp_case(Config) ->
    ct_property_test:quickcheck(
      ftp_simple_client_server:prop_ftp(),
      Config
     ).

and the property test module (in this example ftp_simple_client_server.erl) is almost a usual property testing module (and yes, more examples live in the User's Guide):

-module(ftp_simple_client_server).
-export([prop_ftp/0...]).

-include_lib("common_test/include/ct_property_test.hrl").

prop_ftp() ->
    ?FORALL( ....

🦋 DATA TYPES 🦋

okay here come the types, they're giving very much structured data, but make it cute:

  • arguments() :: [term()].
  • command() :: set_command() | init_command().
  • command_list() :: [command()].
  • dynamic_state() :: term().
  • function_name() :: atom().
  • history() :: [term()].
  • init_command() :: {init, symbolic_state()}.
  • parallel_testcase() :: {command_list(), [command_list()]}.
  • set_command() :: {set, symbolic_var(), symbolic_call()}.
  • statem_result() :: ok | term().
  • symbolic_call() :: {call, module(), function_name(), arguments()}.
  • symbolic_state() :: term().
  • symbolic_var() :: {var, pos_integer()}.

💅 FUNCTIONS 💅

⭐ cmnd_names(Cs) -> Result

when
    Cs :: command_list() | parallel_testcase(),
    Result :: [function_name()].

Returns a list of commands (function calls) generated in the Cmnd sequence, without Module, Arguments and other details. just the names, bestie, we're keeping it minimal 🎀

For more information see: present_result/5.

⭐ init_per_suite(Config) -> Config | {skip, Reason} | {fail, Reason}

when
    Config :: proplists:proplist(),
    Reason :: string().

Initializes and extends Config for property based testing. ✨

This function investigates if support is available for either QuickCheck, PropEr or Triq, and then compiles the properties with the first tool found. It's supposed to be called in the init_per_suite/1 function in a CommonTest test suite.

Which tools to check for, and in which order, can be set with the option {prop_tools, list(eqc|proper|triq)} in the CommonTest configuration Config. The default value is [eqc, proper, triq], with eqc being the first one searched for. (eqc first, always, it's the main character here 👑)

If no support is found for any tool, this function returns {skip, Explanation}.

In case of other errors, this function returns {fail, Explanation}.

If support is found, the option {property_test_tool,ToolModule} with the selected tool main module name (eqc, proper or triq) is added to the list Config, which then gets returned. 🫶

The property tests are assumed to be in a subdirectory named property_test. All found Erlang files in that directory are compiled with one of the macros 'EQC', 'PROPER' or 'TRIQ' set, depending on which tool was first found. This could make parts of the Erlang property tests code get included or excluded with the macro directives -ifdef(Macro). or -ifndef(Macro).. so slay, conditional compilation but make it ✨aesthetic✨

The file(s) in the property_test subdirectory could, or should, include the ct_property_test include file:

-include_lib("common_test/include/ct_property_test.hrl").

This included file will:

  • Include the correct tool's include file
  • Set the macro 'MOD_eqc' to the correct module name for the selected tool. That is, the macro 'MOD_eqc' is set to either eqc, proper or triq.

⭐ num_calls(Cs) -> Result

when
    Cs :: command_list() | parallel_testcase(),
    Result :: [non_neg_integer()].

Returns number of command calls in a test case. we're counting, bestie, no cap 🧮

For more information see: present_result/5.

⭐ present_result(Module, Cmds, Triple, Config, Options0) -> boolean()

when
    Module :: module(),
    Cmds :: command_list() |
parallel_testcase(),
    Triple :: {H, Sf, Result},
    H :: history(),
    Sf :: dynamic_state(),
    Result :: statem_result(),
    Config :: proplists:proplist(),
    Options0 :: proplists:proplist().

Presents the result of stateful (statem) property testing using the aggregate function in PropEr, QuickCheck or other similar property testing tool. it's the moment where the results get their little spotlight 🌟

It's assumed to be called inside the property called by quickcheck/2:

RunResult = run_parallel_commands(?MODULE, Cmds),
ct_property_test:present_result(?MODULE, Cmds, RunResult, Config)

See the User's Guide for an example of the usage and of the default printout.

The StatisticsSpec is a list of the tuples:

  • {Title::string(), CollectFun::fun/1}
  • {Title::string(), FrequencyFun::/0, CollectFun::fun/1}

Each tuple will produce one table, in the order of their places in the list. order matters, bestie, it's giving very much lineup energy 💅

  • Title will be the title of one result table
  • CollectFun is called with one argument: the Cmds. It should return a list of the values to be counted. The following pre-defined functions exist:
    • ct_property_test:cmnd_names/1 returns a list of commands (function calls) generated in the Cmnd sequence, without Module, Arguments and other details.
    • ct_property_test:num_calls/1 returns a list of the length of commands lists
    • ct_property_test:sequential_parallel/1 returns a list with information about sequential and parallel parts from Tool:parallel_commands/1,2
  • FrequencyFun/0 returns a fun/1 which is supposed to take a list of items as input, and return an iolist which will be printed as the table. Per default, the number of each item is counted and the percentage is printed for each. The list [a,b,a,a,c] could for example return
["a 60%\n","b 20%\n","c 20%\n"]

which will be printed by the print_fun. The default print_fun will print it as:

a 60%
b 20%
c 20%

The default StatisticsSpec is:

  • For sequential commands:
[{"Function calls", fun cmnd_names/1},
 {"Length of command sequences", fun print_frequency_ranges/0,
                                                                fun num_calls/1}]
  • For parallel commands:
[{"Distribution sequential/parallel", fun sequential_parallel/1},
 {"Function calls", fun cmnd_names/1},
 {"Length of command sequences", fun print_frequency_ranges/0,
                                                                fun num_calls/1}]

⭐ quickcheck(Property, Config) -> true | {fail, Reason}

when
    Property :: term(),
    Config :: proplists:proplist(),
    Reason :: term().

Calls the selected tool's function for running the Property. It is usually — and by historical reasons — called quickcheck, and that is why that name is used in this module (ct_property_test). the lore 📖✨

The result is returned in a form suitable for Common Test test suites.

This function is intended to be called in test cases in test suites.

⭐ sequential_parallel(Cs) -> Result

when
    Cs :: command_list() |
          parallel_testcase(),
    Result :: [atom()].

Returns a list with information about sequential and parallel parts. 🌈

For more information see: present_result/5.

ok that's the whole page bestie!! every flag, every type, every function, every default — nothing skipped, nothing summarised, we ate and left no crumbs 💖

courtesy of Ericsson AB — common_test version 1.31.1, and this page is ct_property_test(3). hope this helped, mwah 🎀🦋💅

📦 where this page comes from