trace(3)

📦
version 29.0.6 · 4 versions
, in ubuntu stonking

okay so bestie, you found trace(3) — the Erlang Module Definition. and lemme tell you, this is the manual page that's literally the blueprint for peeking into your Erlang runtime. no cap, it's giving main character energy for real 💅✨

💖 NAME 💖

trace — The Erlang trace interface. that's it, that's the vibe. periodt.

🌸 DESCRIPTION 🌸

The Erlang run-time system exposes several trace points that allow users to be notified when they are triggered. Trace points are things such as function calls, message sending and receiving, garbage collection, and process scheduling. lowkey iconic how many hooks you get, fr fr 🫶.

The functions in this module can be used directly, but can also be used as building blocks to build more sophisticated debugging or profiling tools. For debugging Erlang code it is recommended to use dbg and for profiling to use tprof. so like, yes you can raw-dog it with these functions, but if you just wanna debug, dbg is your bestie, and for profiling, tprof is the one 💖.

🎀 Trace Sessions 🎀

All tracing is done within a trace session. Trace sessions can be created and destroyed dynamically. Each session has its own tracer that will receive all trace messages. Several sessions can exist at the same time without interfering with each other. When a trace session is destroyed, all its trace settings are automatically cleaned up. so you can have multiple sessions coexisting, no drama, no interference — it's giving healthy boundaries ✨.

Example:

%% Create a tracer process that will receive the trace events
1> Tracer = spawn(fun F() -> receive M -> io:format("~p~n",[M]), F() end end).
<0.91.0>
%% Create a session using the Tracer
2> Session = trace:session_create(my_session, Tracer, []).
{#Ref<0.1543805153.1548353537.92331>,{my_session, 0}}
%% Setup call tracing on self()
3> trace:process(Session, self(), true, [call]).
1
%% Setup call tracing on lists:seq/2
4> trace:function(Session, {lists,seq,2}, [], []).
1
%% Call the traced function
5> lists:seq(1, 10).
{trace,<0.89.0>,call,{lists,seq,[1,10]}} % The trace message
[1,2,3,4,5,6,7,8,9,10] % The return value
%% Cleanup the trace session
6> trace:session_destroy(Session).
ok

like, look at that. you spawn a tracer, make a session, set some trace points, call the function, get your trace message AND the return value, then clean up. periodt 💅.

🦋 Node Local Tracing Only 🦋

The functions in this module only operates on the local node. That is, both the traced processes/ports as well as the tracer process/port/module must all reside on the same local node as the call is made. To trace remote nodes use dbg or ttb. so no long-distance relationships here — everything's gotta be on the same node. it's giving local only, iykyk 😭.

CHANGE: This trace module was introduced in OTP 27.0. The interface and semantics are similar to the older functions erlang:trace/3, erlang:trace_pattern/3, and erlang:trace_info/2. The main difference is the old functions operate on a single static trace session per node. That could impose the problem that different users and tools would interfere with each other's trace settings. The new trace functions in this module all operate on dynamically created trace sesssions isolated from each other. Also, this makes it easier to safely disable all trace settings when done by a single call to session_destroy/1. To change an existing tool to use the interface the following table can be useful:

okay so here's the glow-up table, but make it bullet points because we don't do ASCII box art in this group chat:

  • erlang:trace(Pid, ...) → process(S, Pid, ...)
  • erlang:trace(processes, ...) → process(S, all, ...)
  • erlang:trace(existing_processes, ...) → process(S, existing, ...)
  • erlang:trace(new_processes, ...) → process(S, new, ...)
  • erlang:trace(Port, ...) → port(S, Port, ...)
  • erlang:trace(ports, ...) → port(S, all, ...)
  • erlang:trace(existing_ports, ...) → port(S, existing, ...)
  • erlang:trace(new_ports, ...) → port(S, new, ...)
  • erlang:trace(all, ...) → process(S, all, ...) and port(S, all, ...)
  • erlang:trace(existing, ...) → process(S, existing, ...) and port(S, existing, ...)
  • erlang:trace(new, ...) → process(S, new, ...) and port(S, new, ...)
  • erlang:trace_pattern(MFA, ...) → function(S, MFA, ...)
  • erlang:trace_pattern(send, ...) → send(S, ...)
  • erlang:trace_pattern('receive', ...) → recv(S, ...)
  • erlang:trace_info(...) → info(S, ...)

Argument S is the trace session that must first be created with session_create/3. The other arguments (implied by ...) are mostly the same. The only other difference is that the tracer is always the tracer specified when the session was created. Options {tracer,T}, {tracer,M,S}, {meta,T}, and {meta,M,S} are therefore not allowed, and the default tracer is never the calling process.

erlang:trace/3 erlang:trace_pattern/3 erlang:trace_info/2 process/4 port/4 function/4 send/3 recv/3 info/3

🦋 DATA TYPES 🦋

okay bestie before we get into the functions, here's the cast of characters — the types you're gonna keep running into. don't skip this part, it's lowkey the whole plot 🙈

match_variable() :: atom().

session() ::
          {session_strong_ref(), session_weak_ref()} |
          session_weak_ref().

That right there is a handle to an isolated trace session. that's it, that's the whole vibe ✨

-opaque session_strong_ref()

-opaque session_weak_ref()

so session_weak_ref() is a weak session handle — literally the same thing session_info/1 gives you back 🫶. and you CAN use it just like a full session handle, no cap, it works! BUT (and this is the tea ☕) it will not prevent the session from being destroyed when the last strong handle gets garbage collected. it's giving "i'm at the party but i'm not on the guest list" 😭 iconic behaviour honestly.

system_event() ::
          busy_port | busy_dist_port | long_gc | long_message_queue |
          long_schedule | large_heap.

system_value() ::
          true | false |
          non_neg_integer() |
          {Disable :: non_neg_integer(), Enable :: pos_integer()}.

trace_flag() :: trace_info_flag() | all | cpu_timestamp.

now the big one, the flag buffet 🍓:

trace_info_flag() ::
          arity | call | exiting | garbage_collection |
          monotonic_timestamp | procs | ports | 'receive' | return_to |
          running | running_procs | running_ports | send |
          set_on_first_link | set_on_first_spawn | set_on_link |
          set_on_spawn | silent | strict_monotonic_timestamp |
          timestamp.

look at all those options. she's got range 💅. and here come the result types, aka "what you get back when you ask":

trace_info_item_result() ::
          {traced, global | local | false | undefined} |
          {match_spec, trace_match_spec() | false | undefined} |
          {meta, pid() | port() | false | undefined | []} |
          {meta, module(), term()} |
          {meta_match_spec, trace_match_spec() | false | undefined} |
          {call_count, non_neg_integer() | boolean() | undefined} |
          {call_time,
           [{pid(),
             non_neg_integer(),
             non_neg_integer(),
             non_neg_integer()}] |
           boolean() |
           undefined} |
          {call_memory,
           [{pid(), non_neg_integer(), non_neg_integer()}] |
           boolean() |
           undefined}.

trace_info_return() ::
          undefined |
          {flags, [trace_info_flag()]} |
          {tracer, pid() | port() | []} |
          {tracer, module(), term()} |
          trace_info_item_result() |
          {all, [trace_info_item_result()] | false | undefined} |
          {system, [trace_info_system_item()]}.

trace_info_system_item() ::
          busy_port | busy_dist_port |
          {long_gc, non_neg_integer()} |
          {long_message_queue,
           {Disable :: non_neg_integer(), Enable :: pos_integer()}} |
          {long_schedule, non_neg_integer()} |
          {large_heap, non_neg_integer()}.

trace_match_spec() ::
          [{[term()] | '_' | match_variable(), [term()], [term()]}].

trace_pattern_flag() ::
          global | local | meta | call_count | call_time | call_memory.

trace_pattern_mfa() :: {atom(), atom(), arity() | '_'}.

periodt. that's the whole type roster 🌈.

🎀 FUNCTIONS 🎀

delivered 💖

delivered(Session :: session(), Tracee :: pid() | all) ->
                     reference().

It literally does the exact same thing as erlang:trace_delivered(Tracee)... except it runs inside the session you hand it (the given session/0) 🫶. same energy, different room bestie.

function ✨

function(Session, MFA, MatchSpec, FlagList) -> non_neg_integer()
                    when
                        Session :: session(),
                        MFA :: trace_pattern_mfa() | on_load,
                        MatchSpec ::
                            trace_match_spec() |
                            boolean() |
                            restart | pause,
                        FlagList :: [trace_pattern_flag()].

ok so this is the one where you turn call tracing on or off for one or more functions 🔥.

⚠️ heads up: you must combine this with process/4 to actually set the call trace flag for one or more processes. this function alone is not the whole story, bestie, it's a two-parter.

so, conceptually, here's how call tracing works. in every trace session there's a set of processes and a set of functions that have been marked for tracing 👑. if a traced process calls a traced function → boom, the trace action happens. otherwise? nothing. literally nothing. silence. 💀

to add or remove processes from that traced-processes set, you use process/4. easy.

to add or remove functions from the traced-functions set in a trace session — that's what THIS function is for. it's her moment, let her have it 🎀.

Session is the trace session you're operating on, as returned by session_create/3. don't lose that handle, it's your ticket in.

MFA should be a tuple, like {Module, Function, Arity}, or the atom on_load (more on that in a sec, keep reading 🦋). the MFA tuple spells out the module, function, and arity for the functions you want traced. and the atom '_' is your wildcard bestie, and it can be used these ways:

  • {Module,Function,'_'} — all functions of any arity named Function in module Module. she's flexible 💅
  • {Module,'_','_'} — all functions in module Module. the whole module. go off.
  • {'_','_','_'} — all functions in all loaded modules. the entire buffet 🍓
🚫 other combinations, like {Module,'_',Arity}, are not allowed. don't try it, she will not answer the phone.

if MFA is the atom on_load, then the match specification and flag list get used on all functions in all modules that are newly loaded ✨. future functions, she's manifesting.

now, MatchSpec can take these forms 👇

  • true — enable tracing for the matching functions. any match specification gets removed. clean slate, no baggage 💖
  • false — disable tracing for the matching functions. any match specification gets removed. we're done here, bye 👋
  • MatchExpression — a match specification. an empty list [] is equivalent to true (iykyk ✨). for a full description of match specifications, see section Match Specifications in Erlang in the User's Guide for the ERTS application.
  • restart — for the FlagList options call_count, call_time and call_memory: restarts the existing counters. 🔄 > ⚠️ the behavior is undefined for other FlagList options. so like, don't.
  • pause — for the FlagList options call_count, call_time and call_memory: pauses the existing counters. ⏸️ > ⚠️ same deal: the behavior is undefined for other FlagList options.

and FlagList is a list of options, bestie. here's every valid one 👇

  • global — turn call tracing on or off for global function calls (meaning calls that specify the module explicitly). only exported functions match, and only global calls generate trace messages. 🏆 this is the default if FlagList is empty.
  • local — turn call tracing on or off for all types of function calls. trace messages get sent whenever any of the specified functions are called, no matter how they're called. and if the flag return_to is set for the process, a return_to message is also sent when this function returns to its caller 🫶.
  • meta — turn meta-tracing on or off for all types of function calls. trace messages get sent to the tracer whenever any of the specified functions are called. meta-tracing traces all processes and does not care about the process trace flags set by process/4 — instead the trace flags are fixed to [call, timestamp] 👀. and the match specification function {return_trace} works with meta-trace 💅.
  • call_count — start (MatchSpec == true) or stop (MatchSpec == false) call count tracing for all types of function calls. for every function, a counter gets incremented when the function is called, in any process. no process trace flags need to be activated 🎀.
    • if call count tracing is started while it's already running, the count restarts from zero. to pause running counters, use MatchSpec == pause. paused and running counters can both be restarted from zero with MatchSpec == restart 🔄.
    • to read the counter value for a function, call trace:info(_, MFA, call_count). 👑
  • call_time — start (MatchSpec is true) or stop (MatchSpec is false) call time tracing for all types of function calls. for every function, a counter gets incremented when the function is called, and the time spent in the function gets measured and accumulated in another counter ⏱️. the counters are stored for each call traced process.
    • if call time tracing is started while already running, the count and time restart from zero. to pause running counters, use MatchSpec == pause. paused and running counters can be restarted from zero with MatchSpec == restart.
    • to read the counter values, use info/3.
  • call_memory — start (MatchSpec == true) or stop (MatchSpec == false) call memory tracing for all types of function calls. for every function, a counter gets incremented when the function is called, and the memory consumed by the function gets measured and accumulated in another counter 🧠. separate counters are stored for each call traced process.
    • if call memory tracing is started while already running, counters and allocations restart from zero. to pause running counters, use MatchSpec == pause. paused and running counters can be restarted from zero with MatchSpec == restart.
    • to read the counter value, use info/3.
💀 big warning: option global cannot be combined with any of the other options, since they all do some kind of local tracing. if global tracing is specified for a set of functions, then local, meta, call_count, call_time, and call_memory tracing for that same matching set of functions are disabled — and vice versa. it's one or the other, pick a lane bestie 🚦.

when you're disabling trace, the option you pass must match the type of trace set on the function. like, local tracing has to be disabled with option local, and global tracing with option global (or no option at all), and so on. mismatch = no 💅.

also, a part of a match specification cannot be changed directly. if a function already has a match specification, you can replace it with a new one 🌸. and info/3 can retrieve the existing match specification for you if you need to peek first.

what does she give back? the number of functions matching argument MFA ✨. and zero gets returned if none matched, or if on_load was specified.

and she fails by raising an error exception with an error reason of:

  • badarg — if an argument is invalid. 🙈
  • system_limit — if a match specification passed as argument has excessive nesting which causes scheduler stack exhaustion for the scheduler that the calling process is executing on 😭. (fun fact bestie: scheduler stack size can be configured when starting the runtime system.)

🦋

info(Session, PidPortFuncEvent, Item) -> Res

info(Session, PidPortFuncEvent, Item) -> Res
              when
                  Session :: session(),
                  PidPortFuncEvent ::
                      pid() |
                      port() |
                      new | new_processes | new_ports | MFA | on_load |
                      send | 'receive' | system,
                  MFA :: {module(), atom(), arity()},
                  Item ::
                      flags | tracer | traced | match_spec | meta |
                      meta_match_spec | call_count | call_time |
                      call_memory | all,
                  Res :: trace_info_return().

okay so this one is your lil spyglass 🔍 — it hands you back trace information about a port, a process, a function, or an event. Pick your target with PidPortFuncEvent, pick what you actually wanna know with Item.

Session is the trace session you're inspecting, exactly as it came back from session_create/3 or session_info/1. 💅

when your target is a port or a process 🫶

Then PidPortFuncEvent is a process identifier (pid), a port identifier, or one of the atoms new, new_processes, or new_ports. And here's the tea on those three:

  • new or new_processes → you get the default trace state for processes that are gonna be created. future vibes only ✨
  • new_ports → same energy, but it's the default trace state for ports that are gonna be created.

The Item values that hit for ports and processes are these:

flags

Returns a list of atoms telling you what kind of traces are enabled for the process. List is empty if nothing's traced, and one or more of trace_info_flag() if stuff is traced. The order is arbitrary — don't clock the ordering, it doesn't mean anything.

tracer

Returns the identifier for the process, the port, or a tuple holding the tracer module and tracer state that's tracing this process. If this process isn't traced at all? You get [].

when your target is a function 🎯

Then PidPortFuncEvent is the three-element tuple {Module, Function, Arity} or the atom on_load. No wildcards are allowed — don't even try it bestie 💀. It returns undefined if the function doesn't exist, or false if the function exists but isn't traced. And if PidPortFuncEvent is on_load, then the info you get back refers to the default value for code that's gonna be loaded later.

The Item values that hit for functions:

traced

Returns global if this function is traced on global function calls, local if it's traced on local function calls (which is to say: local and global function calls), and false if neither local nor global function calls are traced.

match_spec

Returns the match specification for this function, if it has one. If the function is locally or globally traced but has no match specification defined, the returned value is [].

meta

Returns the meta-trace tracer process, port, or trace module for this function, if it has one. If the function isn't meta-traced, you get false. But if it is meta-traced and it once detected that the tracer process is invalid, you get [] instead. girl the lore 😭

meta_match_spec

Returns the meta-trace match specification for this function, if it has one. If the function is meta-traced but has no match specification defined, the returned value is [].

call_count

Returns the call count value for this function, or true for the pseudo function on_load if call count tracing is active. Otherwise false gets returned. See also function/4.

call_time

Returns the call time values for this function, or true for the pseudo function on_load if call time tracing is active. Otherwise false gets returned. The call time values that come back — [{Pid, Count, S, Us}] — are a list of every process that ran the function plus its specific counters. Count is the call count. S and Us are the accumulated call time expressed in seconds and microseconds.

See also function/4.

call_memory

Returns the accumulated number of words allocated by this function. Accumulation stops at the next memory traced function: if there are outer, middle and inner functions each allocating 3 words, but only outer is traced, it will report 9 allocated words. If outer and inner are traced, 6 words are reported for outer and 3 for inner. When the function isn't traced, false is returned. The returned tuple is [{Pid, Count, Words}], for each process that executed the function.

See also function/4.

all

Returns a list containing the {Item, Value} tuples for all other items, or returns false if no tracing is active for this function.

when your target is an event 🦋

Then PidPortFuncEvent is one of the atoms send or 'receive'.

There's exactly one valid Item for events:

match_spec

Returns the match specification for this event, if it has one, or true if no match specification has been set.

when your target is monitored system events 👑

Then PidPortFuncEvent is the atom system.

The only valid Item for system is:

all

Returns a list of all monitored system events enabled by system/3.

The return value is {Item, Value}, where Value is the requested information as described above. And if you passed a pid for a dead process, or the name of a non-existing function, Value is undefined. rip 🪦

port(Session, Ports, How, FlagList) -> integer()

port(Session, Ports, How, FlagList) -> integer()
              when
                  Session :: session(),
                  Ports :: port() | all | existing | new,
                  How :: boolean(),
                  FlagList :: [trace_flag()].

Turns trace flags on or off for one or more ports. That's it, that's the job ✨

Session is the trace session you're operating on, as returned by session_create/3.

Ports is either a port identifier for a local port, or one of these atoms:

all

All currently existing ports and all that will be created in the future. past, present, future, the whole timeline bestie ⏳

existing

All currently existing ports. right now only.

new

All ports that will be created in the future. manifesting 🕯️

FlagList can contain any number of the following flags (and the "message tags" bit refers to the list of trace messages):

  • all — Sets all trace flags except cpu_timestamp, which are in its nature different than the others.
  • send — Traces sending of messages. Message tags: send and send_to_non_existing_process.
  • Traces receiving of messages. Message tags: 'receive'. (yep — the page just doesn't name this one, we're not making that up 💀)
  • ports — Traces port-related events. Message tags: open, closed, register, unregister, getting_linked, and getting_unlinked.
  • running_ports — Traces scheduling of ports. Message tags: in and out.
  • timestamp, cpu_timestamp, monotonic_timestamp, strict_monotonic_timestamp — Same as for timestamps in process/4.

The tracing process receives the trace messages described in the following list. Port is the port identifier of the traced port in which the traced event has occurred. The third tuple element is the message tag.

If flag timestamp, strict_monotonic_timestamp, or monotonic_timestamp is specified, the first tuple element is trace_ts instead, and the time stamp is added as an extra element last in the message tuple. If multiple time stamp flags are passed, timestamp has precedence over strict_monotonic_timestamp, which in turn has precedence over monotonic_timestamp. All time stamp flags are remembered, so if two are passed and the one with highest precedence later is disabled, the other one becomes active. they don't forget, they just wait their turn 💅

If a match specification (applicable only for send and 'receive' tracing) contains a {message} action function with a non-boolean value, that value is added as an extra element to the message tuple either in the last position or before the timestamp (if it is present).

Trace messages 🎀

  • {trace, Port, send, Msg, To} — When Port sends message Msg to process To.
  • {trace, Port, send_to_non_existing_process, Msg, To} — When Port sends message Msg to the non-existing process To.
  • {trace, Port, 'receive', Msg} — When Port receives message Msg. If Msg is set to time-out, a receive statement can have timed out, or the process received a message with the payload timeout.
  • {trace, Port, register, RegName} — When Port gets the name RegName registered.
  • {trace, Port, unregister, RegName} — When Port gets the name RegName unregistered. This is done automatically when a registered process or port exits.
  • {trace, Port, getting_linked, Pid2} — When Port gets linked to a process Pid2.
  • {trace, Port, getting_unlinked, Pid2} — When Port gets unlinked from a process Pid2.
  • {trace, Port, open, Pid, Driver} — When Pid opens a new port Port with the running Driver. Driver is the name of the driver as an atom.
  • {trace, Port, closed, Reason} — When Port closes with Reason.
  • {trace, Port, in, Command | 0} — When Port is scheduled to run. Command is the first thing the port will execute, it can however run several commands before being scheduled out. On some rare occasions, the current function cannot be determined, then the last element is 0. The possible commands are call, close, command, connect, control, flush, info, link, open, and unlink.
  • {trace, Port, out, Command | 0} — When Port is scheduled out. The last command run was Command. On some rare occasions, the current function cannot be determined, then the last element is 0. Command can contain the same commands as in
and yes, the sentence literally just ends there on the page 😭 we keep what the page says, no cap

If the tracing process/port dies or the tracer module returns remove, the flags are silently removed. no warning, no text, just gone 🫥

Returns a number indicating the number of ports that matched Ports. If Ports is a port identifier, the return value is 1. If Ports is all or existing, the return value is the number of existing ports. If Ports is new, the return value is 0.

Failure: badarg if the arguments you handed over aren't supported. Like, cpu_timestamp is not supported on all platforms, bestie. that's just how it is 💅

process(Session, Procs, How, FlagList) -> integer()
    when
        Session :: session(),
        Procs :: pid() | all | existing | new,
        How :: boolean(),
        FlagList :: [trace_flag()].

okay so this one turns trace flags on or off for one or more processes, slay ✨

Argument Session is the trace session we're operating on, exactly as session_create/3 gave it back to you.

Argument Procs is either a process identifier (pid()) for a local process, or one of these atoms:

  • all — every process that exists rn plus every one that'll ever be created in the future. the whole group chat 👑
  • existing — all the processes that currently exist.
  • new — all the processes that will be created in the future.

Argument How is either true to turn the trace flags on, or false to turn them off. cute, right?

Argument FlagList can contain any number of the following flags (and when we say "the message tags", we mean the list of trace messages, iykyk):

  • all — sets all the trace flags except cpu_timestamp, because that one is, in its nature, different than the others. it's not like the other girls 💅
  • send — traces sending of messages. limit which sent messages get traced by calling send/3.

Message tags: send and send_to_non_existing_process.

Traces receiving of messages. Limit which received messages to trace by calling recv/3.

Message tags: 'receive'. 💌

  • call — traces certain function calls. specify which function calls to trace by calling function/4.

Message tags: call and return_from.

  • silent — used together with the call trace flag. The call, return_from, and return_to trace messages are inhibited if this flag is set, but they are executed as normal if there are match specifications.

Silent mode is inhibited by executing trace:process(_, _, false, [silent|_]), or by a match specification executing the function {silent, false}.

The silent trace flag makes it possible to set up a trace on many or even all processes in the system, bestie. The trace can then be activated and deactivated using the match specification function {silent,Bool}, which gives you a high degree of control over which functions (with which arguments!) actually trigger the trace. iconic 🎀

Message tags: call, return_from, and return_to. Or rather, the absence of. 💀

  • return_to — used together with the call trace flag. Traces the exit from call traced functions back to where the execution resumes. Only works for functions traced with the option local to function/4.

The semantics: a return_to trace message is sent when a call traced function returns, or throws an exception that is caught. For tail calls, only one trace message is sent per chain of tail calls, so the properties of tail recursiveness for function calls are kept while tracing with this flag. Similar for exceptions — only one return_to trace message is sent, even if the exception passed more than one call traced function before it was caught.

Using call and return_to trace together makes it possible to know exactly in which function a process executes at any time. main character energy fr fr 👑

To get trace messages containing return values from functions, use the {return_trace} match specification action instead.

Message tags: return_to.

  • procs — traces process-related events.

Message tags: spawn, spawned, exit, register, unregister, link, unlink, getting_linked, and getting_unlinked.

  • running — traces scheduling of processes.

Message tags: in and out.

  • exiting — traces scheduling of exiting processes.

Message tags: in_exiting, out_exiting, and out_exited.

  • running_procs — traces scheduling of processes just like running. However, this option also includes schedule events when the process executes within the context of a port without being scheduled out itself.

Message tags: in and out.

  • garbage_collection — traces garbage collections of processes. yes we're watching the trash get taken out 🗑️✨

Message tags: gc_minor_start, gc_max_heap_size, and gc_minor_end.

  • timestamp — includes a time stamp in all trace messages. The time stamp (Ts) has the same form as returned by erlang:now/0.
  • cpu_timestamp — a global trace flag for the Erlang node that makes all trace time stamps using the flag timestamp be in CPU time, not wall clock time. That is, cpu_timestamp is not to be used if monotonic_timestamp or strict_monotonic_timestamp is enabled. Only allowed with Procs==all. If the host machine OS does not support high-resolution CPU time measurements, process/4 exits with badarg.
⚠️ Notice that most OS do not synchronize this value across cores, so be prepared that time can seem to go backwards when using this option. not a vibe, but okay 😭
  • monotonic_timestamp — includes an Erlang monotonic time time stamp in all trace messages. The time stamp (Ts) has the same format and value as produced by erlang:monotonic_time(nanosecond). This flag overrides the flag cpu_timestamp.
  • strict_monotonic_timestamp — includes a time stamp consisting of Erlang monotonic time and a monotonically increasing integer in all trace messages. The time stamp (Ts) has the same format and value as produced by { erlang:monotonic_time(nanosecond), erlang:unique_integer([monotonic])}. This flag overrides the flag cpu_timestamp.

If multiple time stamp flags are passed, timestamp has precedence over strict_monotonic_timestamp, which in turn has precedence over monotonic_timestamp. All time stamp flags are remembered, so if two are passed and the one with highest precedence later is disabled, the other one becomes active. hierarchy, but make it timestamps 👑

  • arity — used together with the call trace flag. {M, F, Arity} is specified instead of {M, F, Args} in call trace messages.
  • set_on_spawn — makes any process created by a traced process inherit all its trace flags, including the flag set_on_spawn itself. it's giving family inheritance 💖
  • set_on_first_spawn — makes the first process created by a traced process inherit all its trace flags, excluding the flag set_on_first_spawn itself. That is, after the first spawn is done, set_on_first_spawn will be cleared in both the spawned process and the spawning process.

If both are set, set_on_first_spawn will supersede set_on_spawn. only the firstborn gets it 🥇

  • set_on_link — makes any process linked by a traced process inherit all its trace flags, including the flag set_on_link itself.
  • set_on_first_link — makes the first process linked to by a traced process inherit all its trace flags, excluding the flag set_on_first_link itself. That is, after the first link is done, set_on_first_link will be cleared in both the linked process and the linking process.

If both are set, set_on_first_link will supersede set_on_link.

okay so the tracing process receives the trace messages described in the following list. Pid is the process identifier of the traced process in which the traced event has occurred — the third tuple element is the message tag. got it? 🫶

If the flag timestamp, strict_monotonic_timestamp, or monotonic_timestamp is specified, the first tuple element is trace_ts instead, and the time stamp is added as an extra element last in the message tuple.

If a match specification (applicable only for call, send, and 'receive' tracing) contains a {message} action function with a non-boolean value, that value is added as an extra element to the message tuple either in the last position, or before the timestamp (if it is present).

Trace messages: 🔥

  • {trace, Pid, send, Msg, To} — when process Pid sends message Msg to process To. 💌
  • {trace, Pid, send_to_non_existing_process, Msg, To} — when process Pid sends message Msg to the non-existing process To. ghosted 👻
  • {trace, Pid, 'receive', Msg} — when process Pid receives message Msg. If Msg is set to time-out, a receive statement can have timed out, or the process received a message with the payload timeout.
  • {trace, Pid, call, {M, F, Args}} — when process Pid calls a traced function. The return values of calls are never supplied, only the call and its arguments. no receipts, sorry 💀

The trace flag arity can be used to change the contents of this message, so that Arity is specified instead of Args.

  • {trace, Pid, return_to, {M, F, Arity}} — when process Pid returns to the specified function. This trace message is sent if both the flags call and return_to are set, and the function is set to be traced on local function calls. The message is only sent when returning from a chain of tail recursive function calls, where at least one call generated a call trace message (that is, the functions match specification matched, and {message, false} was not an action).
  • {trace, Pid, return_from, {M, F, Arity}, ReturnValue} — when Pid returns from the specified function. This trace message is sent if the flag call is set, and the function has a match specification with a return_trace or exception_trace action.
  • {trace, Pid, exception_from, {M, F, Arity}, {Class, Value}} — when Pid exits from the specified function because of an exception. This trace message is sent if the flag call is set, and the function has a match specification with an exception_trace action.
  • {trace, Pid, spawn, Pid2, {M, F, Args}} — when Pid spawns a new process Pid2 with the specified function call as entry point.

Args is supposed to be the argument list, but can be any term if the spawn is erroneous. we love a chaos gremlin 🍓

  • {trace, Pid, spawned, Pid2, {M, F, Args}} — when Pid is spawned by process Pid2 with the specified function call as entry point.

Args is supposed to be the argument list, but can be literally any term if the spawn is erroneous. 💅

  • {trace, Pid, exit, Reason} — fires when Pid exits with reason Reason. 😭
  • {trace, Pid, register, RegName} — fires when process Pid gets the name RegName registered. 💖
  • {trace, Pid, unregister, RegName} — fires when process Pid gets the name RegName unregistered. Lowkey a big deal: this is done automatically when a registered process or port exits.
  • {trace, Pid, link, Pid2} — fires when Pid links to a process Pid2.
  • {trace, Pid, unlink, Pid2} — fires when Pid removes the link from a process Pid2.
  • {trace, Pid, getting_linked, Pid2} — fires when Pid gets linked to a process Pid2.
  • {trace, Pid, getting_unlinked, Pid2} — fires when Pid gets unlinked from a process Pid2.
  • {trace, Port, open, Pid, Driver} — fires when Pid opens a new port Port with the running Driver. And Driver is the name of the driver as an atom. 🎀
  • {trace, Pid, in | in_exiting, {M, F, Arity} | 0} — fires when Pid is scheduled to run. The process runs in function {M, F, Arity}. On some rare occasions the current function cannot be determined, and then the last element is 0. 💀 not me getting scheduled with no known function tbh.
  • {trace, Pid, out | out_exiting | out_exited, {M, F, Arity} | 0} — fires when Pid is scheduled out. The process was running in function {M, F, Arity}. Same rare-occasion energy as above: if the current function can't be determined, the last element is 0. 💀
  • {trace, Pid, gc_minor_start, Info} — sent when a garbage collection of the young generation is about to be started. Info is a list of two-element tuples, first element is a key, second is the value. Do NOT depend on any order of the tuples, bestie, that's the rule. 🌸 The following keys are defined:
    • heap_size — the size of the used part of the heap.
    • heap_block_size — the size of the memory block used for storing the heap and the stack.
    • old_heap_size — the size of the used part of the old heap.
    • old_heap_block_size — the size of the memory block used for storing the old heap.
    • stack_size — the size of the stack.
    • recent_size — the size of the data that survived the previous garbage collection.
    • mbuf_size — the combined size of message buffers associated with the process.
    • bin_vheap_size — the total size of unique off-heap binaries referenced from the process heap.
    • bin_vheap_block_size — the total size of binaries allowed in the virtual heap in the process before doing a garbage collection.
    • bin_old_vheap_size — the total size of unique off-heap binaries referenced from the process old heap.
    • bin_old_vheap_block_size — the total size of binaries allowed in the virtual old heap in the process before doing a garbage collection.
    • wordsize — for the gc_minor_start event it's the size of the need that triggered the GC. For the corresponding gc_minor_end event it's the size of reclaimed memory = start heap_size - end heap_size.
All sizes are in words. ✨
  • {trace, Pid, gc_max_heap_size, Info} — sent when the max_heap_size is reached during garbage collection. Info contains the same kind of list as in message gc_start, but the sizes reflect the sizes that triggered max_heap_size to be reached. 👑
  • {trace, Pid, gc_minor_end, Info} — sent when young garbage collection is finished. Info contains the same kind of list as in message gc_minor_start, but the sizes reflect the new sizes after garbage collection.
  • {trace, Pid, gc_major_start, Info} — sent when fullsweep garbage collection is about to be started. Info contains the same kind of list as in message gc_minor_start.
  • {trace, Pid, gc_major_end, Info} — sent when fullsweep garbage collection is finished. Info contains the same kind of list as in message gc_minor_start, but the sizes reflect the new sizes after a fullsweep garbage collection. 🧹✨

If the tracing process dies or the tracer module returns remove, the flags are silently removed. 🫥 no dramatic exit, they're just gone.

Returns a number indicating the number of processes that matched Procs. If Procs is a process identifier, the return value is 1. If Procs is all or existing, the return value is the number of processes running. If Procs is new, the return value is 0. 🎯

Failure: badarg if the specified arguments are not supported. For example, cpu_timestamp is not supported on all platforms. Sad but true. 💔

💌 recv(Session, MatchSpec, FlagList) -> non_neg_integer() 💌

recv(Session, MatchSpec, FlagList) -> non_neg_integer()
    when
        Session :: session(),
        MatchSpec :: trace_match_spec() | boolean(),
        FlagList :: [].

Set the trace pattern for message receiving. 📬

Must be combined with process/4 or port/4 to set the 'receive' trace flag for one or more processes or ports.

Argument Session is the trace session to operate on as returned by session_create/3.

The default value for the receive trace pattern in each session is true. That is, all messages received by processes having 'receive' trace enabled will be traced. Use this function to limit traced 'receive' events based on the message content, the sender, and/or the receiver.

Argument MatchSpec can take the following forms:

  • MatchExpression — a match specification. The matching is done on the list [Node, Sender, Msg]. Node is the node name of the sender. Sender is the process or port identity of the sender, or the atom undefined if the sender is not known (which can be the case for remote senders, iykyk). Msg is the message term. The pid of the receiving process can be accessed with the guard function self/0. An empty list is the same as true. For more information, see section Match Specifications in Erlang in the User's Guide for the ERTS application.
  • true — enable tracing for all received messages (to 'receive' traced processes). Any match specification is removed. This is the default. 💖
  • false — disable tracing for all received messages. Any match specification is removed. 🙈

Argument FlagList must be [] for receive tracing.

The return value is always 1.

Examples:

Only trace messages from a specific process Pid:

> trace:recv(Session, [{['_',Pid, '_'],[],[]}], []).
1

Only trace messages matching {reply, _}:

> trace:recv(Session, [{['_','_', {reply,'_'}],[],[]}], []).
1

Only trace messages from other nodes:

> trace:recv(Session, [{['$1', '_', '_'],[{'=/=','$1',{node}}],[]}], []).
1
NOTE: A match specification for 'receive' trace can use all guard and body functions except caller, is_seq_trace, get_seq_token, set_seq_token, enable_trace, disable_trace, trace, silent, and process_dump. 🚫 those specific ones are not invited to the function party.

Fails by raising an error exception with an error reason of:

  • badarg — if an argument is invalid.
  • system_limit — if a match specification passed as argument has excessive nesting which causes scheduler stack exhaustion for the scheduler that the calling process is executing on. Scheduler stack size can be configured when starting the runtime system. 😵 stacking that deep is giving main character energy in the worst way.

💌 send(Session, MatchSpec, FlagList) -> non_neg_integer() 💌

send(Session, MatchSpec, FlagList) -> non_neg_integer()
    when
        Session :: session(),
        MatchSpec :: trace_match_spec() | boolean(),
        FlagList :: [].

Set the trace pattern for message sending. 📤

Must be combined with process/4 or port/4 to set the send trace flag for one or more processes or ports.

Argument Session is the trace session to operate on as returned by session_create/3.

The default value for the send trace pattern in each session is true. That is, all messages sent from processes having send trace enabled will be traced. Use this function to limit traced send events based on the message content, the sender, and/or the receiver.

Argument MatchSpec can take the following forms:

  • MatchExpression — a match specification. The matching is done on the list [Receiver, Msg]. Receiver is the process or port identity of the receiver and Msg is the message term. The pid of the sending process can be accessed with the guard function self/0. An empty list is the same as true. For more information, see section Match Specifications in Erlang in the User's Guide for the ERTS application.
  • true — enable tracing for all sent messages (from send traced processes). Any match specification is removed.
  • false — disable tracing for all sent messages. Any match specification is removed. 🙈

Argument FlagList must be [].

The return value is always 1.

Examples:

Only trace messages to a specific process Pid:

> trace:send(Session, [{[Pid, '_'],[],[]}], []).
1

Only trace messages matching {reply, _}:

> trace:send(Session, [{['_', {reply,'_'}],[],[]}], []).
1

Only trace messages sent to the sender itself:

> trace:send(Session, [{['$1', '_'],[{'=:=','$1',{self}}],[]}], []).
1

Only trace messages sent to other nodes:

> trace:send(Session, [{['$1', '_'],[{'=/=',{node,'$1'},{node}}],[]}], []).
1
NOTE: A match specification for send trace can use all guard and body functions except caller. 🚫 just that one, bestie.

Fails by raising an error exception with an error reason of:

  • badarg — if an argument is invalid.
  • system_limit — if a match specification you passed as an argument is nested so aggressively deep that it exhausts the scheduler stack for whichever scheduler the calling process happens to be running on. 💀 Scheduler stack size can be configured when you start the runtime system, so like… that one's on you bestie.

💫 session_create/3

session_create(Name, Tracer, Opts) -> session()
                        when
                            Name :: atom(),
                            Tracer ::
                                pid() | port() | {module(), term()},
                            Opts :: [].

Ok so this is the one that creates a brand new trace session. ✨

Argument Name is an atom name for the session. It gets returned when you inspect it with session_info/1.

Argument Tracer specifies the consumer of all trace events for the session. It can be an identifier of a local process or port that receives all trace messages. 🫶

Tracer can also be a tuple {TracerModule, TracerState} — that's for when you want a tracer module called instead of a trace message just being sent. The tracer module can then ignore or change the trace message, main character energy. For all the deets on how to write a tracer module, go look at module erl_tracer. 🧚

Argument Opts must be []. No notes, no exceptions, that's just the law.

Returns an opaque handle to the trace session. The handle keeps the session alive. And if the handle gets dropped and garbage collected?? Then the session gets destroyed and cleaned up exactly as if you'd called session_destroy/1. 💅 (yes, the garbage collector is out here doing your chores for you, iconic)

🌸 session_destroy/1

session_destroy(Session) -> true | false when Session :: session().

Destroy a trace session and clean up all its settings on processes, ports, and functions. 🧹

The only things not cleaned up are trace messages that have already been sent — those are already out there living their life, not our problem.

Returns true if the session was active. Returns false if the session had already been destroyed by either an earlier call to this function or the garbage collector. (also, tiny thing, the original spells it "earler" — we keep the facts, not the typos, but just so you know there's no mystery option here 😭)

🔥 session_info/1

session_info(PidPortFuncEvent) -> Res
                        when
                            PidPortFuncEvent ::
                                all |
                                pid() |
                                port() |
                                new | new_processes | new_ports | MFA |
                                on_load | send | 'receive',
                            MFA :: {module(), atom(), arity()},
                            Res :: undefined | [session_weak_ref()].

Return which trace sessions affect a port, process, function, or event. 🕵️

Argument all returns all active trace sessions that exist on the node.

Returns a list of weak session handles, or undefined if the process/port/function does not exist. Simple as.

👑 system/3

system(Session :: session(),
             Event :: system_event(),
             Value :: system_value()) ->
                ok.

Enable or disable monitoring of system events. This is your vibe check on the whole system. 🌈

Argument Session is the trace session to operate on, as returned by session_create/3.

Argument Event is an atom describing the kind of system event to monitor. To enable monitoring, argument Value is — depending on the event — either a limit of that event or the atom true. To disable monitoring, pass Value as the atom false.

When a monitored system event happens, a message is sent to the session tracer. The session tracer must be a process — otherwise the function call will fail. 🫠

The following Events with Values can be monitored:

  • long_gc, Time — if a garbage collection in the system takes at least Time wall clock milliseconds, a message {monitor, GcPid, long_gc, Info} is sent. GcPid is the pid that was garbage collected. Info is a list of two-element tuples describing the result of the garbage collection.

One of the tuples is {timeout, GcTime}, where GcTime is the time for the garbage collection in milliseconds. The other tuples are tagged with heap_size, heap_block_size, stack_size, mbuf_size, old_heap_size, and old_heap_block_size. These tuples are explained in the description of trace message gc_minor_start (see trace:process/4). New tuples can be added, and the order of the tuples in the Info list can be changed at any time without prior notice. 🙈 (fr fr, do not depend on that order)

  • long_message_queue, {Disable, Enable} — if the number of messages in the message queue of a process reaches Enable, a message {monitor, Pid, long_message_queue, Long} is sent. Pid is the process identifier of the process that got a long message queue, and Long will equal true, indicating that it is in a long message queue state. No more long_message_queue monitor messages will be sent due to the process identified by Pid until its message queue length falls down to a length of Disable length. When that happens, a long_message_queue monitor message with Long equal to false will be sent, indicating that the process is no longer in a long message queue state. And as of this, if the message queue length should again reach Enable length, a new long_message_queue monitor message with Long set to true will again be sent. So basically: a long_message_queue monitor message is sent when a process enters or leaves a long message queue state, where these state changes are defined by the Enable and Disable parameters.

Enable must be an integer larger than zero. Disable must be an integer larger than or equal to zero and smaller than Enable. If the above is not satisfied, the operation will fail with a badarg error exception. 🚨 And you are recommended to use a much smaller value for Disable length than Enable length so you don't get absolutely flooded with long_message_queue monitor messages. Like, please. For your own sake. 😭

  • long_schedule, Time — if a process or port in the system runs uninterrupted for at least Time wall clock milliseconds, a message {monitor, PidOrPort, long_schedule, Info} is sent. PidOrPort is the process or port that was running. Info is a list of two-element tuples describing the event.

If a pid/0, the tuples {timeout, Millis}, {in, Location}, and {out, Location} are present, where Location is either an MFA ({Module, Function, Arity}) describing the function where the process was scheduled in/out, or the atom undefined.

If a port/0, the tuples {timeout, Millis} and {port_op,Op} are present. Op is one of proc_sig, timeout, input, output, event, or dist_cmd, depending on which driver callback was executing.

proc_sig is an internal operation and is never to appear, while the others represent the corresponding driver callbacks timeout, ready_input, ready_output, event, and outputv (when the port is used by distribution). Value Millis in tuple timeout informs about the uninterrupted execution time of the process or port, which always is equal to or higher than the Time value supplied when starting the trace. New tuples can be added to the Info list in a future release. The order of the tuples in the list can be changed at any time without prior notice. 🦋

This can be used to detect problems with NIFs or drivers that take too long to execute. 1 ms is considered a good maximum time for a driver callback or a NIF. However, a time-sharing system is usually to consider everything < 100 ms as "possible" and fairly "normal". However, longer schedule times can indicate swapping or a misbehaving NIF/driver. Misbehaving NIFs and drivers can cause bad resource utilization and bad overall system performance. 💀

  • large_heap, Size — if a garbage collection in the system results in the allocated size of a heap being at least Size words, a message {monitor, GcPid, large_heap, Info} is sent. GcPid and Info are the same as for long_gc described above, except that the tuple tagged with timeout is not present.

The monitor message is sent if the sum of the sizes of all memory blocks allocated for all heap generations after a garbage collection is equal to or higher than Size.

When a process is killed by max_heap_size, it is killed before the garbage collection is complete and thus no large heap message is sent. 😔

  • busy_port, true — if a process in the system gets suspended because it sends to a busy port, a message {monitor, SusPid, busy_port, Port} is sent. SusPid is the pid that got suspended when sending to Port.
  • busy_dist_port, true</orange> — if a process in the system gets suspended because it sends to a process on a remote node whose inter-node communication was handled by a busy port, a message {monitor, SusPid, busy_dist_port, Port} is sent. SusPid is the pid that got suspended when sending through the inter-node communication port Port.

To disable system monitoring of an event, pass the value as false. There are no other special values (like zero) to disable monitoring of an event. Some of the events have an unspecified minimum value. Lower values will be adjusted to the minimum value. For example, it is currently not possible to monitor all garbage collections with {long_gc, 0}. 🍓

NOTE: If the session tracer process gets so large that it itself starts to cause system monitor messages when garbage collecting, the messages enlarge the process message queue and probably make the problem worse. Keep the tracer process neat and do not set the system monitor limits too tight. 🎀

Failures:

  • badarg — if the session tracer is not a local process.

And that's a wrap on the whole page bestie!! 🎉 This version is from Ericsson AB, it's kernel 11.0.3, and the page is trace(3) — so if you ever need to double check any of this, that's the exact tea. Go forth and trace responsibly, you got this 👑💖

📦 where this page comes from