ct_doctest(3)

✨ ct_doctest — the Erlang module for testing your docs 📝💖

okay bestie, welcome to ct_doctest(3) — the Erlang Module Definition page. this is the little library that runs doctests on your documentation examples, and honestly? it's giving responsible queen. 💅

---

✨ NAME ✨

ct_doctest — runs doctests on documentation examples. using ct_doctest means your docs' examples stay correct, up to date, and stylistically consistent. no more lying READMEs. we love to see it. 🫶

---

🌸 DESCRIPTION 🌸

so the examples it tests can live in a module (usually written with documentation attributes) or in files. by default, ct_doctest goes looking for markdown code blocks and runs every Erlang code block that looks like a shell session. ✨

the doctest parser specifically hunts for examples formatted like they were run in the Erlang shell — using prompts of the form N>, where N starts at 1 for each block. the expected output goes on the lines right after the prompt. like:

-doc """
This is an example of a doctest:

1> 1+2. 3

""".

ct_doctest can also be dropped into Common Test suites to validate your documentation examples as part of your test runs. the usual move is calling module/1 with a module name. for example:

all() ->
    [doctests].
doctests(_Config) ->
    ct_doctest:module(my_module).

🎀 Prompt format rules 🎀

for a code block to actually run as a doctest:

• prompts must start at 1> for each block
• each subsequent prompt must increment (2>, 3>, ...)
• continuation lines must be indented
• % style comment lines are allowed in prompt blocks
• mismatched prompt numbering causes a doctest parse error 😭

💀 Troubleshooting 💀

if a doctest fails and you're like "…excuse me??":

• use verbose to print per-block execution details 🔍
• verify that expected output matches the shell output exactly
• verify prompt numbering and continuation-line indentation

🦋 Examples 🦋

below are examples of supported formats for the code blocks in the documentation. the parser is quite flexible and supports various styles, including multi-line expressions, comments, and even prebound variables. iconic behaviour tbh.

⭐ Basic example

1> 1+2.
3

🌈 Basic example using Erlang code

this example uses an explicit Erlang code block. that is,

1> 1+2. 3

instead of the previous one which is a generic code block. both formats are supported. 🫶

1> 1+2.
3

💜 Multi-line prompt

use multiline prompts for expressions that span multiple lines by starting the prompt with > and indenting the continuation lines. for example:

1> 1
  +
  2
  .
3

🍓 Multi-line with comma

it's possible to have multiple expressions in the same prompt, separated by commas. for example:

1> A = 1,
  A + 2.
3

🧚 Multi-line match

the expected output can span multiple lines too. for example:

1> [1, 2].
[
 1
 ,
 2
 ]

👑 Multiple prompts

examples can have multiple prompts. for example:

1> 1 + 2.
3
2> 3 + 4.
7

✨ Defining variables

any variable defined in the examples will be available in the following prompts. for example:

1> A = 1 + 2.
3
2> A + 3.
6

🎀 Prebound variables

if the documentation examples rely on certain variables being prebound, you can provide these bindings when calling module/3. for example, if you have a module doc that uses a variable Prebound, you can set it up like this:

1> Prebound.
hello

and then in your test suite:

binding_test(_Config) ->
    Bindings = [{moduledoc, #{'Prebound' => hello}}],
    ct_doctest:module(my_module, Bindings, []).

💅 Ignore result

to ignore the results of a prompt, just skip writing the expected output. for example:

1> 1 + 2.
2> 3 + 4.
7

🔥 Matching exceptions

examples of failures can be tested by writing the expected exception after the prompt. for example:

1> hello + 1.
** exception error: an error occurred when evaluating an arithmetic expression
     in operator  +/2
        called as hello + 1
2> lists:last([]).
** exception error: no function clause matching lists:last([])

the simplest way to know what output to write is to run the example in the shell and copy the output, including the ** exception line.

if you don't want to include the entire exception message, use only the start of the message.

1> hello + 1.
** exception error

💖 Comments

comments can be inserted anywhere in the code block. for example:

%% A comment before the first prompt
1> [1,
%% A comment between prompts
  2].
[1,
%% A comment in a match
 2]
2> [1,
  %% Indented comment between prompts
  2].
[1,
 %% Indented comment in a match
 2]
3> """
  %% A comment in a string is not a comment

  """.
"""
%% A comment in a string is not a comment

"""
4> 1 + a.
** exception error: an error occurred when evaluating an arithmetic expression
%% Comments
     in operator  +/2
     %% in exceptions
        called as 1 + a
%% are ignored

🌸 Matching of maps

when matching on maps, it is possible to use shell syntax, that is, => and not :=, as in normal Erlang code. for example:

1> #{ a => b }.
#{ a => b }

🌙 Matching of ...

it is possible to use ... in the expected output to indicate that the rest of the output should be ignored. this is useful for outputs that are large or contain non-deterministic elements. 😌

1> lists:seq(1,100).
[1, 2, 3, ...]
2> #{ a => b }.
#{ a => ... }
3> <<1, 0:1024>>.
<<1, 0, 0, 0, ...>>

🛠️ Compiling modules

ct_doctest can also compile full module code examples. it then looks for a -module declaration to determine the module name and compiles the code as if it were in a file. for example:

-module(my_module).
-export([foo/0]).
foo() ->
    ok.

the module is then available for use in following prompts. for example:

1> my_module:foo().

🙈 Edge cases

the following are examples that are not supported by the parser and will be ignored. (rip, they simply don't get to sit with us.)

a> should not be tested

 1> should not be tested

> should not be tested

should not be tested
1>

---

🌈 DATA TYPES 🌈

options() ::
          [{parser,
            fun((unicode:unicode_binary()) ->
                    [unicode:unicode_binary()] | {error, term()})} |
           {skipped_blocks, non_neg_integer() | false} |
           {missing_tests, [{atom(), arity()}]} |
           {skip_tests,
            [moduledoc | {function | type | callback, atom(), arity()}]}
       |
           {verbose, boolean()}].

options for doctest execution. here's the whole squad, one by one:

• parser — use this option to plug in an external documentation parser. the parser callback must be a fun/1 and return a list of Erlang code block binaries. the code blocks are then checked to determine whether they should be run as doctests. if no parser is provided, a built-in markdown parser will be used.
• skipped_blocks — sets the exact number of Erlang code blocks that are allowed to be skipped because no runnable shell prompts were found. it does not count blocks in any function listed in missing_tests. it defaults to false.
• missing_tests — a list of {Function, Arity} pairs that are expected to have documentation but no doctests. when this option is set, ct_doctest will fail if any documented function lacks doctests and is not in this list (i.e., a new function was added without doctests), and also fail if a function in this list now has doctests (i.e., the list is stale and should be updated). defaults to not checking.
• skip_tests — a list of doc entries whose doctests should be skipped. each entry is either moduledoc or a {Kind, Name, Arity} tuple where Kind is function, type, or callback. for example, [moduledoc, {function, foo, 1}] skips the moduledoc and the foo/1 function.
• verbose — print detailed information while running doctests, including each block run and skipped block details. 👀

---

bestie we are NOT done yet — there's more of this page coming, so hold that thought 💖

🛠️ FUNCTIONS 🛠️

Okay bestie, these are the callable iconics. Two functions, both ready to run your doctests. Let's get into it.

---

📄 file/3 — markdown file, but make it tested

       file(File :: file:filename(),
                  Bindings :: [{atom(), term()}],
                  Options :: options()) ->
                     ok | {comment, string()} | {error, term()} | no_return().

This one runs doctests for a markdown file. You hand it a file, some bindings, and options, and it does the thing.

• If every single test passes, you get ok. Slay.
• If any test fails, it raises an exception shaped like error({N, errors}), where N is the number of failed tests. The details of each failure are printed to the console, so you can see exactly what went off.
• Use Bindings to provide prebound variables. But heads up: Bindings are global for all files, so take care to avoid any naming conflicts. It's giving shared group chat, so pick your variable names carefully.
• You can run doctests on non-markdown files by providing a custom parser that extracts the code blocks to be tested. Iconic.
• See options/0 for available options.

---

🧩 module/3 — EEP-48 docs edition

       module(Module :: module(), Bindings, Options :: options()) ->
                       ok | {comment, string()} | {error, term()} | no_return()
                       when
                           KFA ::
                               {Kind :: function | type | callback,
                                atom(),
                                arity()},
                           Bindings ::
                               [{KFA | moduledoc, erl_eval:binding_struct()}].

This runs tests for the documentation in a module with EEP-48 docs.

When calling module/3, ct_doctest looks for documentation in the specified module and runs any examples found there. The module, function, type, and callback documentation are all checked for examples. No doc left behind, periodt.

• You get ok if all tests pass.
• You get {comment, Comment} if all tests pass but one or more functions lack tests. Like, everything's fine but some functions just didn't show up with tests.
• If any test fails, an exception shaped like error({N, errors}) is raised, where N is the number of failed tests. The details of each failure are printed to the console.
• Use Bindings to provide prebound variables for a specific doc entry. Use moduledoc for module docs and {function, Name, Arity} (or corresponding type/callback keys) for entry-specific bindings.
• See options/0 for available options.

---

That's the tea, bestie. This page is ct_doctest(3) from Ericsson AB, common_test version 1.31.1, and the date that shipped with it. Stay slaying. 💖✨