LTTNG-UST(3)

πŸ’– LTTNG-UST(3) πŸ’–

okay so bestie, welcome to the LTTng Manual page for lttng-ust β€” yes, that's the LTTNG-UST(3) you're holding in your terminal rn 🫢


✨ NAME ✨

lttng-ust β€” LTTng user space tracing, aka the moment your userspace apps finally start telling you what's actually going on πŸ’…


πŸŽ€ SYNOPSIS πŸŽ€

#include <lttng/tracepoint.h>

#define LTTNG_UST_TP_ARGS(args...)
#define LTTNG_UST_TP_ENUM_VALUES(values...)
#define LTTNG_UST_TP_FIELDS(fields...)
#define LTTNG_UST_TRACEPOINT_ENUM(prov_name, enum_name, mappings)
#define LTTNG_UST_TRACEPOINT_EVENT(prov_name, t_name, args, fields)
#define LTTNG_UST_TRACEPOINT_EVENT_CLASS(cls_prov_name, cls_name,
                                         args, fields)
#define LTTNG_UST_TRACEPOINT_EVENT_INSTANCE(cls_prov_name, cls_name,
                                            inst_prov_name, t_name, args)
#define LTTNG_UST_TRACEPOINT_LOGLEVEL(prov_name, t_name, level)
#define LTTNG_UST_TRACEPOINT_MODEL_EMF_URI(prov_name, t_name, uri)
#define lttng_ust_do_tracepoint(prov_name, t_name, ...)
#define lttng_ust_field_array(int_type, field_name, expr, count)
#define lttng_ust_field_array_nowrite(int_type, field_name, expr, count)
#define lttng_ust_field_array_hex(int_type, field_name, expr, count)
#define lttng_ust_field_array_nowrite_hex(int_type, field_name, expr,
                                          count)
#define lttng_ust_field_array_network(int_type, field_name, expr, count)
#define lttng_ust_field_array_network_nowrite(int_type, field_name,
                                              expr, count)
#define lttng_ust_field_array_network_hex(int_type, field_name, expr,
                                          count)
#define lttng_ust_field_array_network_nowrite_hex(int_type, field_name,
                                                          expr, count)
#define lttng_ust_field_array_text(char, field_name, expr, count)
#define lttng_ust_field_array_text_nowrite(char, field_name, expr,
                                           count)
#define lttng_ust_field_enum(prov_name, enum_name, int_type, field_name,
                             expr)
#define lttng_ust_field_enum_nowrite(prov_name, enum_name, int_type,
                                     field_name, expr)
#define lttng_ust_field_enum_value(label, value)
#define lttng_ust_field_enum_range(label, start, end)
#define lttng_ust_field_float(float_type, field_name, expr)
#define lttng_ust_field_float_nowrite(float_type, field_name, expr)
#define lttng_ust_field_integer(int_type, field_name, expr)
#define lttng_ust_field_integer_hex(int_type, field_name, expr)
#define lttng_ust_field_integer_network(int_type, field_name, expr)
#define lttng_ust_field_integer_network_hex(int_type, field_name, expr)
#define lttng_ust_field_integer_nowrite(int_type, field_name, expr)
#define lttng_ust_field_sequence(int_type, field_name, expr,
                                 len_type, len_expr)
#define lttng_ust_field_sequence_nowrite(int_type, field_name, expr,
                                         len_type, len_expr)
#define lttng_ust_field_sequence_hex(int_type, field_name, expr,
                                     len_type, len_expr)
#define lttng_ust_field_sequence_nowrite_hex(int_type, field_name, expr,
                                             len_type, len_expr)
#define lttng_ust_field_sequence_network(int_type, field_name, expr,
                                         len_type, len_expr)
#define lttng_ust_field_sequence_network_nowrite(int_type, field_name,
                                                 expr, len_type,
                                                 len_expr)
#define lttng_ust_field_sequence_network_hex(int_type, field_name, expr,
                                             len_type, len_expr)
#define lttng_ust_field_sequence_network_nowrite_hex(int_type,
                                                     field_name,
                                                     expr, len_type,
                                                     len_expr)
#define lttng_ust_field_sequence_text(char, field_name, expr, len_type,
                                      len_expr)
#define lttng_ust_field_sequence_text_nowrite(char, field_name, expr,
                                              len_type, len_expr)
#define lttng_ust_field_string(field_name, expr)
#define lttng_ust_field_string_nowrite(field_name, expr)
#define lttng_ust_field_fixed_length_blob(field_name, expr, count,
                                          media_type)
#define lttng_ust_field_fixed_length_blob_nowrite(field_name, expr,
                                                  count, media_type)
#define lttng_ust_field_variable_length_blob(field_name, expr, len_type,
                                             len_expr, media_type)
#define lttng_ust_field_variable_length_blob_nowrite(field_name, expr,
                                                     len_type, len_expr,
                                                     media_type)
#define lttng_ust_field_unused(expr)
#define lttng_ust_field_unused_nowrite(expr)
#define lttng_ust_tracepoint(prov_name, t_name, ...)
#define lttng_ust_tracepoint_enabled(prov_name, t_name)

And when you link, per this manual page, you're linking with:

  • -llttng-ust -ldl
  • If you define _LGPL_SOURCE before including <lttng/tracepoint.h> (directly or indirectly): -llttng-ust-common

🌸 DESCRIPTION 🌸

The Linux Trace Toolkit: next generation <http://lttng.org/> is an open source software package used for correlated tracing of the Linux kernel, user applications, and user libraries. That's the whole vibe ✨

LTTng-UST is the user space tracing component of the LTTng project. It's basically a port to user space of the low-overhead tracing capabilities of the LTTng Linux kernel tracer. The liblttng-ust library is what you use to trace user applications and libraries πŸ”₯

Note πŸ™ˆ This man page is about the liblttng-ust library. The LTTng-UST project also provides Java and Python packages to trace applications written in those languages. How to instrument and trace Java and Python applications is documented in the online LTTng documentation <http://lttng.org/docs/>.

There are three ways to use liblttng-ust, and honestly pick your fighter:

πŸ’œ Compatibility with previous APIs

Since LTTng-UST 2.13, the LTTNG_UST_COMPAT_API_VERSION definition controls which LTTng-UST APIs are available (compiled):

  • Undefined β€” All APIs are available.
  • N (0 or positive integer) β€” API version N, and all the following existing APIs, are available. Previous APIs are not available (not compiled).

The following table shows the mapping from LTTng-UST versions (up to LTTng-UST 2.16.0) to available API versions:

  • LTTng-UST version 2.0 to 2.12 β€” available API versions: 0
  • LTTng-UST version 2.13 β€” available API versions: 0 and 1

This manual page only documents version 1 of the API πŸ‘‘

If you wish to have access to version 0 of the API (for example, the tracepoint(), ctf_integer(), and TRACEPOINT_EVENT() macros), then either don't define LTTNG_UST_COMPAT_API_VERSION, or define it to 0 before including any LTTng-UST header. ikykyk, set it before you include anything 🫢

πŸ¦‹ Creating a tracepoint provider

Creating a tracepoint provider is the first step of using liblttng-ust. The next steps are:

  • Instrumenting your application with lttng_ust_tracepoint() calls
  • Building your application with LTTng-UST support, either statically or dynamically.

A tracepoint provider is a compiled object containing the event probes corresponding to your custom tracepoint definitions. A tracepoint provider contains the code to get the size of an event and to serialize it, amongst other things. lowkey doing a lot of heavy lifting πŸ’€

To create a tracepoint provider, start with the following tracepoint provider header template:

#undef LTTNG_UST_TRACEPOINT_PROVIDER
#define LTTNG_UST_TRACEPOINT_PROVIDER my_provider

#undef LTTNG_UST_TRACEPOINT_INCLUDE
#define LTTNG_UST_TRACEPOINT_INCLUDE "./tp.h"

#if !defined(_TP_H) || \
    defined(LTTNG_UST_TRACEPOINT_HEADER_MULTI_READ)
#define _TP_H

#include <lttng/tracepoint.h>

/*
 * LTTNG_UST_TRACEPOINT_EVENT(), LTTNG_UST_TRACEPOINT_EVENT_CLASS(),
 * LTTNG_UST_TRACEPOINT_EVENT_INSTANCE(),
 * LTTNG_UST_TRACEPOINT_LOGLEVEL(), and `LTTNG_UST_TRACEPOINT_ENUM()`
 * are used here.
 */

#endif /* _TP_H */

#include <lttng/tracepoint-event.h>

In this template, the tracepoint provider is named my_provider (the LTTNG_UST_TRACEPOINT_PROVIDER definition). The file needs to bear the name of the LTTNG_UST_TRACEPOINT_INCLUDE definition (tp.h in this case). Between #include <lttng/tracepoint.h> and #endif go the invocations of the LTTNG_UST_TRACEPOINT_EVENT(), LTTNG_UST_TRACEPOINT_EVENT_CLASS(), LTTNG_UST_TRACEPOINT_EVENT_INSTANCE(), LTTNG_UST_TRACEPOINT_LOGLEVEL(), and LTTNG_UST_TRACEPOINT_ENUM() macros. that's the whole sandwich πŸ₯ͺ

Note 🧚 You can avoid writing the prologue and epilogue boilerplate in the template file above by using the lttng-gen-tp(1) tool shipped with LTTng-UST. not me automating the boring part 😭

The tracepoint provider header file needs to be included in a source file which looks like this:

#define LTTNG_UST_TRACEPOINT_CREATE_PROBES

#include "tp.h"

Together, those two files (let's call them tp.h and tp.c) form the tracepoint provider sources, ready to be compiled. iconic duo behavior πŸ‘‘

You can create multiple tracepoint providers to be used in a single application, but each one must have its own header file. no sharing, boundaries bestie πŸŽ€

The LTTNG_UST_TRACEPOINT_EVENT() usage section below shows how to use the LTTNG_UST_TRACEPOINT_EVENT() macro to define the actual tracepoints in the tracepoint provider header file.

See the EXAMPLE section below for a complete example β€” it's giving full working demo, you'll love it 🌈

πŸ“ LTTNG_UST_TRACEPOINT_EVENT() usage

The LTTNG_UST_TRACEPOINT_EVENT() macro is used in a template provider header file (see the Creating a tracepoint provider section above) to define LTTng-UST tracepoints.

The LTTNG_UST_TRACEPOINT_EVENT() usage template is as follows, and bestie it's coming right up in the next part πŸ‘€πŸ’–

LTTNG_UST_TRACEPOINT_EVENT(
    /* Tracepoint provider name */
    my_provider,

    /* Tracepoint/event name */
    my_tracepoint,

    /* List of tracepoint arguments (input) */
    LTTNG_UST_TP_ARGS(
        ...
    ),

    /* List of fields of eventual event (output) */
    LTTNG_UST_TP_FIELDS(
        ...
    )
)

okay so the LTTNG_UST_TP_ARGS() macro is where you stash the input arguments of the tracepoint, bestie. And then those arguments? They can be used in the argument expressions of the output fields you define over in LTTNG_UST_TP_FIELDS() ✨

The format of the LTTNG_UST_TP_ARGS() params is: C type first, then argument name β€” repeat as needed, up to ten times. Like so:

LTTNG_UST_TP_ARGS(
    int, my_int,
    const char *, my_string,
    FILE *, my_file,
    double, my_float,
    struct my_data *, my_data
)

Meanwhile LTTNG_UST_TP_FIELDS() holds the output fields of the tracepoint β€” like, the actual data that can get recorded in the payload of an event this tracepoint emits πŸ’…

LTTNG_UST_TP_FIELDS() contains a list of lttng_ust_field_*() macros NOT separated by commas. The available ones are all documented in the "Available lttng_ust_field_*() field type macros" section right below 🫢

🌸 Available field macros 🌸

This section documents all the lttng_ust_field_*() macros you can drop into the LTTNG_UST_TP_FIELDS() macro of LTTNG_UST_TRACEPOINT_EVENT().

Standard integer, displayed in base 10:

lttng_ust_field_integer(int_type, field_name, expr)
lttng_ust_field_integer_nowrite(int_type, field_name, expr)

Standard integer, but displayed in base 16:

lttng_ust_field_integer_hex(int_type, field_name, expr)

Integer in network byte order (big endian), displayed in base 10:

lttng_ust_field_integer_network(int_type, field_name, expr)

Integer in network byte order, displayed in base 16:

lttng_ust_field_integer_network_hex(int_type, field_name, expr)

Floating point number:

lttng_ust_field_float(float_type, field_name, expr)
lttng_ust_field_float_nowrite(float_type, field_name, expr)

Null-terminated string:

lttng_ust_field_string(field_name, expr)
lttng_ust_field_string_nowrite(field_name, expr)

Statically-sized array of integers (the _hex versions get displayed in hexadecimal, the _network versions in network byte order):

lttng_ust_field_array(int_type, field_name, expr, count)
lttng_ust_field_array_nowrite(int_type, field_name, expr, count)
lttng_ust_field_array_hex(int_type, field_name, expr, count)
lttng_ust_field_array_nowrite_hex(int_type, field_name, expr, count)
lttng_ust_field_array_network(int_type, field_name, expr, count)
lttng_ust_field_array_network_nowrite(int_type, field_name, expr,
                                      count)
lttng_ust_field_array_network_hex(int_type, field_name, expr, count)
lttng_ust_field_array_network_nowrite_hex(int_type, field_name,
                                          expr, count)

Statically-sized array, printed as text β€” and nope, it doesn't need to be null-terminated:

lttng_ust_field_array_text(char, field_name, expr, count)
lttng_ust_field_array_text_nowrite(char, field_name, expr, count)

Dynamically-sized array of integers (again, _hex versions show up in hexadecimal, _network versions in network byte order):

lttng_ust_field_sequence(int_type, field_name, expr, len_type,
                         len_expr)
lttng_ust_field_sequence_nowrite(int_type, field_name, expr,
                                 len_type, len_expr)
lttng_ust_field_sequence_hex(int_type, field_name, expr, len_type,
                             len_expr)
lttng_ust_field_sequence_nowrite_hex(int_type, field_name, expr,
                                     len_type, len_expr)
lttng_ust_field_sequence_network(int_type, field_name, expr,
                                 len_type, len_expr)
lttng_ust_field_sequence_network_nowrite(int_type, field_name, expr,
                                         len_type, len_expr)
lttng_ust_field_sequence_network_hex(int_type, field_name, expr,
                                     len_type, len_expr)
lttng_ust_field_sequence_network_nowrite_hex(int_type, field_name,
                                             expr, len_type,
                                             len_expr)

Dynamically-sized array, displayed as text β€” again, no null-termination needed:

lttng_ust_field_sequence_text(char, field_name, expr, len_type,
                              len_expr)
lttng_ust_field_sequence_text_nowrite(char, field_name, expr,
                                      len_type, len_expr)

Fixed-length BLOB β€” a fixed number of raw bytes, tagged with a media type:

lttng_ust_field_fixed_length_blob(field_name, expr, count, media_type)
lttng_ust_field_fixed_length_blob_nowrite(field_name, expr, count,
                                          media_type)

Variable-length BLOB β€” a dynamic number of raw bytes, tagged with a media type:

lttng_ust_field_variable_length_blob(field_name, expr, len_type,
                                     len_expr, media_type)
lttng_ust_field_variable_length_blob_nowrite(field_name, expr, len_type,
                                             len_expr, media_type)

A BLOB field just records raw bytes β€” the ones expr points to β€” without interpreting them at all. The number of bytes to record is count (fixed-length version) or whatever len_expr evaluates to (variable-length version). And media_type is what associates an IANA media type with the recorded data (more on that below πŸ‘‡).

Enumeration. Heads up tho β€” the enumeration field must be defined before you use this macro, using the LTTNG_UST_TRACEPOINT_ENUM() macro. Peep the LTTNG_UST_TRACEPOINT_ENUM() usage section for the deets πŸ’–

lttng_ust_field_enum(prov_name, enum_name, int_type, field_name,
                     expr)
lttng_ust_field_enum_nowrite(prov_name, enum_name, int_type,
                             field_name, expr)

The parameters are:

  • count β€” Number of elements/bytes in array/sequence/BLOB. This must be known at compile time.
  • enum_name β€” Name of an enumeration field previously defined with the LTTNG_UST_TRACEPOINT_ENUM() macro. See the LTTNG_UST_TRACEPOINT_ENUM() usage section for more information.
  • expr β€” C expression resulting in the field value. This expression can use one or more arguments passed to the tracepoint. The arguments of a given tracepoint are defined in the LTTNG_UST_TP_ARGS() macro (see the Creating a tracepoint provider section above).
  • field_name β€” Event field name (C identifier syntax, NOT a literal string).
  • float_type β€” Float C type (float or double). The size of this type determines the size of the floating point number field.
  • int_type β€” Integer C type. The size of this type determines the size of the integer/enumeration field.
  • len_expr β€” C expression resulting in the sequence length. This expression can use one or more arguments passed to the tracepoint.
  • len_type β€” Unsigned integer C type of sequence length.
  • media_type β€” C expression of type const char * resulting in the IANA media type https://www.iana.org/assignments/media-types/ of a BLOB field's data, for example "image/png".
  • prov_name β€” Tracepoint provider name. This must be the same as the tracepoint provider name used in a previous field definition.

The _nowrite versions omit themselves from the recorded trace, but are otherwise identical πŸ’« Their whole main-character purpose is to make some of the event context available to the event filters without having to commit the data to sub-buffers. See lttng-enable-event(1) to learn more about dynamic event filtering.

The following macros do NOT record any field at all. Their only job is to mark a tracepoint argument as deliberately unused, so the compiler doesn't warn about it when the argument isn't used by any other field:

lttng_ust_field_unused(expr)
lttng_ust_field_unused_nowrite(expr)

See the EXAMPLE section below for a complete example πŸŽ€

πŸ”₯ LTTNG_UST_TRACEPOINT_ENUM() usage πŸ”₯

An enumeration field is basically a list of mappings between an integer (or a range of integers) and strings β€” sometimes called labels or enumerators. And why would you want that? Because enumeration fields give you a more compact trace when the possible values for a field are limited. Slay, honestly.

You define an enumeration field with the LTTNG_UST_TRACEPOINT_ENUM() macro:

LTTNG_UST_TRACEPOINT_ENUM(
    /* Tracepoint provider name */
    my_provider,

    /* Enumeration name (unique in the whole tracepoint provider) */
    my_enum,

    /* Enumeration mappings */
    LTTNG_UST_TP_ENUM_VALUES(
        ...
    )
)

LTTNG_UST_TP_ENUM_VALUES() contains a list of enumeration mappings, NOT separated by commas. Two macros can live inside it: lttng_ust_field_enum_value() and lttng_ust_field_enum_range().

lttng_ust_field_enum_value() is your single-value mapping:

lttng_ust_field_enum_value(label, value)

It maps the given label string to the value value.

lttng_ust_field_enum_range() is your range mapping:

lttng_ust_field_enum_range(label, start, end)

It maps the given label string to the range of integers from start to end, inclusively. Range mappings may overlap, but the behaviour is implementation-defined β€” meaning each trace reader handles overlapping ranges however it feels like πŸ™ˆ

See the EXAMPLE section below for a complete example ✨

πŸ¦‹ LTTNG_UST_TRACEPOINT_EVENT_CLASS() usage πŸ¦‹

A tracepoint class is a class of tracepoints that share the same field types and names. A tracepoint instance is one instance of such a declared tracepoint class, and it gets its own event name. Think family recipe vs. each individual batch, iykyk.

LTTng-UST creates one event serialization function per tracepoint class. Using LTTNG_UST_TRACEPOINT_EVENT() creates one tracepoint class per tracepoint definition β€” whereas using LTTNG_UST_TRACEPOINT_EVENT_CLASS() and LTTNG_UST_TRACEPOINT_EVENT_INSTANCE() creates one tracepoint class plus one or more tracepoint instances of that class. In other words: many tracepoints can reuse the same serialization code. And reusing that same code, when possible, can reduce cache pollution and thus improve performance πŸ”₯

The LTTNG_UST_TRACEPOINT_EVENT_CLASS() macro accepts the same parameters as the LTTNG_UST_TRACEPOINT_EVENT() macro, except that instead of an event name, its second parameter is the tracepoint class name:

#define LTTNG_UST_TRACEPOINT_PROVIDER my_provider

/* ... */
LTTNG_UST_TRACEPOINT_EVENT_CLASS(
    /* Tracepoint class provider name */
    my_provider,

    /* Tracepoint class name */
    my_tracepoint_class,

    /* List of tracepoint arguments (input) */
    LTTNG_UST_TP_ARGS(
        ...
    ),

    /* List of fields of eventual event (output) */
    LTTNG_UST_TP_FIELDS(
        ...
    )
)

Once the tracepoint class is defined, you can create as many tracepoint instances as needed:

#define LTTNG_UST_TRACEPOINT_PROVIDER natality

/* ... */

LTTNG_UST_TRACEPOINT_EVENT_INSTANCE(
    /* Name of the tracepoint class provider */
    my_provider,

    /* Tracepoint class name */
    my_tracepoint_class,

    /* Name of the local (instance) tracepoint provider */
    natality,

    /* Tracepoint/event name */
    my_tracepoint,

    /* List of tracepoint arguments (input) */
    LTTNG_UST_TP_ARGS(
        ...
    )
)

So yeah, as you can see, LTTNG_UST_TRACEPOINT_EVENT_INSTANCE() does not contain the LTTNG_UST_TP_FIELDS() macro, because those fields are already defined at the LTTNG_UST_TRACEPOINT_EVENT_CLASS() level. Like, once at the class level, you're done β€” no re-declaring, bestie πŸ’…

Note that the LTTNG_UST_TRACEPOINT_EVENT_INSTANCE() macro requires two provider names:

  • The name of the tracepoint class provider (my_provider in the example above).

This is the same as the first argument of the LTTNG_UST_TRACEPOINT_EVENT_CLASS() expansion, so it's the one you refer to.

  • The name of the local, or instance, provider (natality in the example above).

This is the provider name which becomes the prefix part of the name of the events which such a tracepoint creates.

The two provider names may be different if the tracepoint class and the tracepoint instance macros are in two different translation units. No cap, that's allowed 🌈

See the EXAMPLE section below for a complete example.


🌸 LTTNG_UST_TRACEPOINT_LOGLEVEL() usage 🌸

Optionally, a log level can be assigned to a defined tracepoint. Assigning different levels of severity to tracepoints can be useful: when controlling tracing sessions, you can choose to only enable events falling into a specific log level range using the --loglevel and --loglevel-only options of the lttng-enable-event(1) command. Filtering arc, honestly πŸŽ€

Log levels are assigned to tracepoints that are already defined, using the LTTNG_UST_TRACEPOINT_LOGLEVEL() macro. The latter must be used after having used LTTNG_UST_TRACEPOINT_EVENT() or LTTNG_UST_TRACEPOINT_EVENT_INSTANCE() for a given tracepoint. So define the tracepoint first, then slide the log level in after β€” order matters, iykyk. The LTTNG_UST_TRACEPOINT_LOGLEVEL() macro is used as follows:

LTTNG_UST_TRACEPOINT_LOGLEVEL(
    /* Tracepoint provider name */
    my_provider,

    /* Tracepoint/event name */
    my_tracepoint,

    /* Log level */
    LTTNG_UST_TRACEPOINT_LOGLEVEL_INFO
)

The available log level definitions are:

  • LTTNG_UST_TRACEPOINT_LOGLEVEL_EMERG β€” System is unusable.
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_ALERT β€” Action must be taken immediately.
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_CRIT β€” Critical conditions.
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_ERR β€” Error conditions.
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_WARNING β€” Warning conditions.
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_NOTICE β€” Normal, but significant, condition.
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_INFO β€” Informational message.
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_DEBUG_SYSTEM β€” Debug information with system-level scope (set of programs).
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_DEBUG_PROGRAM β€” Debug information with program-level scope (set of processes).
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_DEBUG_PROCESS β€” Debug information with process-level scope (set of modules).
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_DEBUG_MODULE β€” Debug information with module (executable/library) scope (set of units).
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_DEBUG_UNIT β€” Debug information with compilation unit scope (set of functions).
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_DEBUG_FUNCTION β€” Debug information with function-level scope.
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_DEBUG_LINE β€” Debug information with line-level scope (default log level).
  • LTTNG_UST_TRACEPOINT_LOGLEVEL_DEBUG β€” Debug-level message.

See the EXAMPLE section below for a complete example.


πŸ¦‹ Associating an EMF model URI to a tracepoint πŸ¦‹

Optionally, you can assign an Eclipse Modeling Framework (EMF) model URI to a defined tracepoint. A trace reader may use this URI to locate a model describing the events the tracepoint produces. Like, it's a little map so the reader knows where the lore lives πŸ—ΊοΈβœ¨

As with the LTTNG_UST_TRACEPOINT_LOGLEVEL() macro, you must use the LTTNG_UST_TRACEPOINT_MODEL_EMF_URI() macro after having used LTTNG_UST_TRACEPOINT_EVENT() or LTTNG_UST_TRACEPOINT_EVENT_INSTANCE() for a given tracepoint:

LTTNG_UST_TRACEPOINT_MODEL_EMF_URI(
    /* Tracepoint provider name */
    my_provider,

    /* Tracepoint/event name */
    my_tracepoint,

    /* Model URI */
    "http://example.com/path/to/model"
)

πŸ’œ Instrumenting your application πŸ’œ

Once the tracepoint provider is created (see the Creating a tracepoint provider section above), you can instrument your application with the defined tracepoints thanks to the lttng_ust_tracepoint() macro:

#define lttng_ust_tracepoint(prov_name, t_name, ...)

With:

  • prov_name β€” Tracepoint provider name.
  • t_name β€” Tracepoint/event name.
  • ... β€” Tracepoint arguments, if any.

Make sure to include the tracepoint provider header file anywhere you use lttng_ust_tracepoint() for this provider. Like, don't forget the #include, that's the whole vibe πŸ”₯

Note β€” Even though LTTng-UST supports lttng_ust_tracepoint() call site duplicates having the same provider and tracepoint names, it is recommended to use a provider/tracepoint name pair only once within the application source code to help map events back to their call sites when analyzing the trace. Fr fr, one pair, one spot β€” future you (and your trace reader) will thank you πŸ™

Sometimes, arguments to the tracepoint are expensive to compute (take call stack, for example). To avoid the computation when the tracepoint is disabled, you can use the lttng_ust_tracepoint_enabled() and lttng_ust_do_tracepoint() macros:

#define lttng_ust_tracepoint_enabled(prov_name, t_name)
#define lttng_ust_do_tracepoint(prov_name, t_name, ...)

lttng_ust_tracepoint_enabled() returns a non-zero value if the tracepoint named t_name from the provider named prov_name is enabled at run time.

lttng_ust_do_tracepoint() is like lttng_ust_tracepoint(), except that it doesn't check if the tracepoint is enabled.

⚠️ DANGER ⚠️ Using lttng_ust_tracepoint() with lttng_ust_tracepoint_enabled() is dangerous since lttng_ust_tracepoint() also contains the lttng_ust_tracepoint_enabled() check β€” which means a race condition is possible in this situation:
if (lttng_ust_tracepoint_enabled(my_provider, my_tracepoint)) {
    stuff = prepare_stuff();
}

lttng_ust_tracepoint(my_provider, my_tracepoint, stuff);

If the tracepoint is enabled after the condition, then stuff is not prepared: the emitted event will either contain wrong data, or the whole application could crash (segmentation fault, for example). Yeah. Not the serve you want πŸ’€ Not the moment. Use lttng_ust_do_tracepoint() for that pattern instead.

Note β€” Neither lttng_ust_tracepoint_enabled() nor lttng_ust_do_tracepoint() have a STAP_PROBEV() call, so if you need it, you should emit this call yourself. DIY era πŸ› οΈβœ¨

🧚 Tracing C/C++ constructors and destructors 🧚

If one of the following is true:

  • You compile your C++ application with GCC ≀ 4.8.
  • You compile your C/C++ application with a C++ compiler and define LTTNG_UST_ALLOCATE_COMPOUND_LITERAL_ON_HEAP.

Then LTTng won't trace:

  • C constructors and destructors in the application itself or in statically linked archives.
  • Some C++ constructors and destructors in the application itself or in statically linked archives.

In this case, which exact C++ constructors and destructors won't be traced depends on the initialization order within each translation unit and across the entire program when all translation units are linked together.

TL;DR: those specific cases are a no-trace zone, bestie 😭 plan around it.


πŸ“ Statically linking the tracepoint provider πŸ“

With the static linking method, compiled tracepoint providers are copied into the target application.

Define LTTNG_UST_TRACEPOINT_DEFINE definition below the LTTNG_UST_TRACEPOINT_CREATE_PROBES definition in the tracepoint provider source:

#define LTTNG_UST_TRACEPOINT_CREATE_PROBES
#define LTTNG_UST_TRACEPOINT_DEFINE

#include "tp.h"

Create the tracepoint provider object file:

$ cc -c -I. tp.c
Note β€” Although an application instrumented with LTTng-UST tracepoints can be compiled with a C++ compiler, tracepoint probes should be compiled with a C compiler. Keep that boundary clean πŸŽ€

At this point, you can archive this tracepoint provider object file, possibly with other object files of your application or with other tracepoint provider object files, as a static library:

$ ar rc tp.a tp.o

Using a static library does have the advantage of centralising the tracepoint providers objects so they can be shared between multiple applications. This way, when the tracepoint provider is modified, the source code changes don't have to be patched into the source code tree of each application. The applications need to be relinked after each change, but need not to be otherwise recompiled (unless the API of the tracepoint provider changes). Chef's kiss reuse situation πŸ‘‘βœ¨

Then, link your application with this object file (or with the static library containing it) and with liblttng-ust and libdl (libc on a BSD system):

$ cc -o app tp.o app.o -llttng-ust -ldl

🌈 Dynamically loading the tracepoint provider 🌈

The second approach to package the tracepoint provider is to use the dynamic loader: the library and its member functions are explicitly sought, loaded at run time. Very main-character energy β€” showing up exactly when needed πŸ’…

In this scenario, the tracepoint provider is compiled as a shared object.

The process to create the tracepoint provider shared object is pretty much the same as the static linking method, except that:

  • Since the tracepoint provider is not part of the application, LTTNG_UST_TRACEPOINT_DEFINE must be defined, for each tracepoint provider, in exactly one source file of the application.
  • LTTNG_UST_TRACEPOINT_PROBE_DYNAMIC_LINKAGE must be defined next to LTTNG_UST_TRACEPOINT_DEFINE.

Regarding LTTNG_UST_TRACEPOINT_DEFINE and LTTNG_UST_TRACEPOINT_PROBE_DYNAMIC_LINKAGE, the recommended practice is to use a separate C source file in your application to define them, then include the tracepoint provider header files afterwards. For example, as tp-define.c:

#define LTTNG_UST_TRACEPOINT_DEFINE
#define LTTNG_UST_TRACEPOINT_PROBE_DYNAMIC_LINKAGE
#include "tp.h"

So the tracepoint provider object file you use to build the shared library? You build it like the static linking method, except with the -fpic option:

$ cc -c -fpic -I. tp.c

Then you link it as a shared library, like this:

$ cc -shared -Wl,--no-as-needed -o tp.so tp.o -llttng-ust

Big thing to know: this tracepoint provider shared object is not linked with the user application β€” you have to load it manually. Which is literally why the app gets built without even mentioning the tracepoint provider, but still needs libdl:

$ cc -o app app.o tp-define.o -ldl

Now, there are two ways to dynamically load that tracepoint provider shared object:

  • Load it manually from the application using dlopen(3)
  • Make the dynamic loader load it for you with the LD_PRELOAD environment variable (see ld.so(8))
⚠️ If the application doesn't dynamically load the tracepoint provider shared object using one of those methods, tracing is disabled for this application, and the events don't even show up in the output of lttng-list(1). Gone. Not listed. πŸ«₯

Also, heads up: it is not safe to use dlclose(3) on a tracepoint provider shared object that's being actively used for tracing, because LTTng-UST doesn't do reference counting on that shared object. 😭

For example, statically linking a tracepoint provider to a shared object that's meant to be dynamically loaded by an application (a plugin, say) is not safe: the shared object, which contains the tracepoint provider, could be dynamically closed (dlclose(3)) at any time by the application. Not the vibe. πŸ’€

So to instrument a shared object, either:

  • Statically link the tracepoint provider to the application, or
  • Build the tracepoint provider as a shared object (following the procedure shown in this section), and preload it when tracing is needed using the LD_PRELOAD environment variable.

🌼 Using LTTng-UST with daemons 🌼

Okay so daemons need a little extra care, bestie. When you're using liblttng-ust with daemon applications that call fork(2), clone(2), or BSD's rfork(2) without a following exec(3) family system call β€” you need to preload the liblttng-ust-fork.so library before starting the application, using the LD_PRELOAD environment variable (see ld.so(8)).

And if you're using liblttng-ust with a daemon application that closes file descriptors that weren't opened by it, preload the liblttng-ust-fd.so library before you start the application. Classic use cases: daemons closing all file descriptors after fork(2), and buggy applications doing "double-closes". πŸ™ˆ

🧚 Context information 🧚

So the LTTng-UST tracer can prepend context information before each event, or before specific events. It's giving main character data. ✨

You add context fields to specific channels using lttng-add-context(1).

Here's everything LTTng-UST supports:

General context fields

  • cpu_id β€” CPU ID.
πŸ“Œ Note: with the default per-CPU buffer allocation policy (see lttng-enable-channel(1)), this context field is always enabled, and it cannot be added with lttng-add-context(1): its main purpose is to be used for dynamic event filtering. See lttng-enable-event(1) for more info about event filtering.
  • ip β€” Instruction pointer: enables recording the exact address from which an event was emitted. You can use this context field to reverse-lookup the source location that caused the event to be emitted. πŸ”Ž
  • pthread_id β€” POSIX thread identifier. Can be used on architectures where pthread_t maps nicely to an unsigned long type.

Process context fields

  • procname β€” Thread name, as set by exec(3) or prctl(2). It's recommended that programs set their thread name with prctl(2) before hitting the first tracepoint for that thread.
  • vpid β€” Virtual process ID: process ID as seen from the point of view of the current process ID namespace (see pid_namespaces(7)).
  • vtid β€” Virtual thread ID: thread ID as seen from the point of view of the current process ID namespace (see pid_namespaces(7)).

perf context fields

  • perf:thread:COUNTER β€” perf counter named COUNTER. Use lttng add-context --list to list the available perf counters. Only available on IA-32 and x86-64 architectures.
  • perf:thread:raw:rN:NAME β€” perf counter with raw ID N and custom name NAME. See lttng-add-context(1) for more details.

Namespace context fields (see namespaces(7))

  • cgroup_ns β€” Inode number of the current control group namespace (see cgroup_namespaces(7)) in the proc file system.
  • ipc_ns β€” Inode number of the current IPC namespace (see ipc_namespaces(7)) in the proc file system.
  • mnt_ns β€” Inode number of the current mount point namespace (see mount_namespaces(7)) in the proc file system.
  • net_ns β€” Inode number of the current network namespace (see network_namespaces(7)) in the proc file system.
  • pid_ns β€” Inode number of the current process ID namespace (see pid_namespaces(7)) in the proc file system.
  • time_ns β€” Inode number of the current clock namespace (see time_namespaces(7)) in the proc file system.
  • user_ns β€” Inode number of the current user namespace (see user_namespaces(7)) in the proc file system.
  • uts_ns β€” Inode number of the current UTS namespace (see uts_namespaces(7)) in the proc file system.

Credential context fields (see credentials(7))

  • vuid β€” Virtual real user ID: real user ID as seen from the point of view of the current user namespace (see user_namespaces(7)).
  • vgid β€” Virtual real group ID: real group ID as seen from the point of view of the current user namespace (see user_namespaces(7)).
  • veuid β€” Virtual effective user ID: effective user ID as seen from the point of view of the current user namespace (see user_namespaces(7)).
  • vegid β€” Virtual effective group ID: effective group ID as seen from the point of view of the current user namespace (see user_namespaces(7)).
  • vsuid β€” Virtual saved set-user ID: saved set-user ID as seen from the point of view of the current user namespace (see user_namespaces(7)).
  • vsgid β€” Virtual saved set-group ID: saved set-group ID as seen from the point of view of the current user namespace (see user_namespaces(7)).

🌸 LTTng-UST state dump 🌸

If an application that uses liblttng-ust becomes part of a tracing session, the tracer emits info about its currently loaded shared objects, their build IDs, and their debug link information β€” all as events. It's basically the app spilling its whole lore. πŸ“–βœ¨

These LTTng-UST state dump events exist and must be enabled to record application state dumps. Also note: during the state dump phase, LTTng-UST can also emit shared library load/unload events (see Shared library load/unload tracking below β€” iykyk).

  • lttng_ust_statedump:start β€” Emitted when the state dump begins. This event has no fields.
  • lttng_ust_statedump:end β€” Emitted when the state dump ends. Once this event is emitted, it is guaranteed that, for a given process, the state dump is complete. Periodt. βœ… This event has no fields.
  • lttng_ust_statedump:bin_info β€” Emitted when information about a currently loaded executable or shared object is found. Fields:
    • baddr β€” Base address of loaded executable.
    • memsz β€” Size of loaded executable in memory.
    • path β€” Path to loaded executable file.
    • is_pic β€” Whether or not the executable is position-independent code.
    • has_build_id β€” Whether or not the executable has a build ID. If this field is 1, you can expect that an lttng_ust_statedump:build_id event record follows this one (not necessarily immediately after).
    • has_debug_link β€” Whether or not the executable has debug link information. If this field is 1, you can expect that an lttng_ust_statedump:debug_link event record follows this one (not necessarily immediately after).
  • lttng_ust_statedump:build_id β€” Emitted when a build ID is found in a currently loaded shared library. See Debugging Information in Separate Files (https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html) for more information about build IDs. πŸ›πŸ”

Like, fields for that one, bestie ✨:

  • baddr β€” base address of the loaded library πŸŽ€
  • build_id β€” the build ID πŸ’…

Fires when debug link info gets spotted in a shared library that's currently loaded πŸ‘€ the whole concept of debug links is explained over at Debugging Information in Separate Files β€” https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html β€” go read it if you're curious, iykyk πŸ¦‹

Fields:

  • baddr β€” base address of the loaded library πŸŽ€
  • crc β€” CRC of the debug link file
  • filename β€” the debug link file name πŸ“

lttng_ust_statedump:procname

It's giving "the process procname at process start" πŸ’– literally just the name, at the start, no cap.

Fields:

  • procname β€” the process name 🌸

🌈 Shared library load/unload tracking 🌈

Okay so here's the tea β˜• both the LTTng-UST state dump and the LTTng-UST helper library that instruments the dynamic linker (that's liblttng-ust-dl(3), go check it) can emit shared library load/unload tracking events. Two icons, same energy πŸ’…

The following shared library load/unload tracking events exist, and you have to enable them if you actually want to track loading and unloading of shared libraries. No enabling, no events, fr fr 🫢

lttng_ust_lib:load

Emitted when a shared library (shared object) is loaded πŸ”₯

Fields:

  • baddr β€” base address of the loaded library
  • memsz β€” size of the loaded library in memory 🧚
  • path β€” path to the loaded library file πŸŽ€
  • has_build_id β€” whether or not the library has a build ID. If this field is 1, you can expect that an lttng_ust_lib:build_id event record follows this one (not necessarily immediately after). So like, it's coming, just maybe not next-next, yk? ⭐
  • has_debug_link β€” whether or not the library has debug link information. If this field is 1, you can expect that an lttng_ust_lib:debug_link event record follows this one (not necessarily immediately after). Same deal πŸ’œ

lttng_ust_lib:unload

Emitted when a shared library (shared object) is unloaded 😭 bye bestie

Fields:

  • baddr β€” base address of the unloaded library πŸ‘‘ (note the unloaded, that's the whole plot)

lttng_ust_lib:build_id

Emitted when a build ID is found in a loaded shared library (shared object) ✨ and yeah, Debugging Information in Separate Files β€” https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html β€” has the full lore on build IDs πŸ¦‹

Fields:

  • baddr β€” base address of the loaded library πŸŽ€
  • build_id β€” the build ID πŸ’…

Emitted when debug link information is found in a loaded shared library (shared object) πŸ™ˆ again, Debugging Information in Separate Files β€” https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html β€” for all the debug-link details πŸ“

Fields:

  • baddr β€” base address of the loaded library πŸŽ€
  • crc β€” CRC of the debug link file
  • filename β€” the debug link file name 🌸

πŸ’€ Detect if LTTng-UST is loaded πŸ’€

Wanna know if liblttng-ust is loaded from inside an application? Iconic question, here's the two-step 🫢

  1. Define the lttng_ust_loaded weak symbol globally, like so:
int lttng_ust_loaded __attribute__((weak));

This weak symbol gets set by the constructor of liblttng-ust ⭐ so it's not you doing it, it's the library pulling up and setting it for you πŸ’…

  1. Test lttng_ust_loaded wherever you need it:
/* ... */

if (lttng_ust_loaded) {
    /* LTTng-UST is loaded */
} else {
    /* LTTng-UST is NOT loaded */
}

/* ... */

And that's literally it β€” vibe check complete, no cap 🌈

🌈 EXAMPLE 🌈

πŸ’‘ Note: there are a few examples chilling in the doc/examples directory of LTTng-UST's source tree, right here: https://github.com/lttng/lttng-ust/tree/stable-2.16/doc/examples. Go look, bestie, they're literally right there πŸ’–

Okay so this example shows, like, every feature we talked about in the previous sections. And for linking the app with the tracepoint provider, we're going with the static linking method here. Periodt.

You can compile the source files and link them together statically like this:

$ cc -c -I. tp.c
$ cc -c app.c
$ cc -o app tp.o app.o -llttng-ust -ldl

Then, using the lttng(1) tool, you create an LTTng tracing session, enable all the events of this tracepoint provider, and start tracing:

$ lttng create my-session
$ lttng enable-event --userspace 'my_provider:*'
$ lttng start

You can also enable specific events instead, if that's more your vibe:

$ lttng enable-event --userspace my_provider:big_event
$ lttng enable-event --userspace my_provider:event_instance2

Run the application:

$ ./app some arguments

Then stop the current tracing session and go inspect the recorded events πŸ‘€:

$ lttng stop
$ lttng view

πŸŽ€ Tracepoint provider header file πŸŽ€

tp.h:

#undef LTTNG_UST_TRACEPOINT_PROVIDER
#define LTTNG_UST_TRACEPOINT_PROVIDER my_provider

#undef LTTNG_USTTRACEPOINT_INCLUDE
#define LTTNG_USTTRACEPOINT_INCLUDE "./tp.h"

#if !defined(_TP_H) || \
    defined(LTTNG_UST_TRACEPOINT_HEADER_MULTI_READ)
#define _TP_H

#include <lttng/tracepoint.h>
#include <stdio.h>

#include "app.h"

LTTNG_UST_TRACEPOINT_EVENT(
    my_provider,
    simple_event,
    LTTNG_UST_TP_ARGS(
        int, my_integer_arg,
        const char *, my_string_arg
    ),
    LTTNG_UST_TP_FIELDS(
        lttng_ust_field_string(argc, my_string_arg)
        lttng_ust_field_integer(int, argv, my_integer_arg)
    )
)

LTTNG_UST_TRACEPOINT_ENUM(
    my_provider,
    my_enum,
    LTTNG_UST_TP_ENUM_VALUES(
        lttng_ust_field_enum_value("ZERO", 0)
        lttng_ust_field_enum_value("ONE", 1)
        lttng_ust_field_enum_value("TWO", 2)
        lttng_ust_field_enum_range("A RANGE", 52, 125)
        lttng_ust_field_enum_value("ONE THOUSAND", 1000)
    )
)

LTTNG_UST_TRACEPOINT_EVENT(
    my_provider,
    big_event,
    LTTNG_UST_TP_ARGS(
        int, my_integer_arg,
        const char *, my_string_arg,
        FILE *, stream,
        double, flt_arg,
        int *, array_arg
    ),
    LTTNG_UST_TP_FIELDS(
        lttng_ust_field_integer(int, int_field1, my_integer_arg * 2)
        lttng_ust_field_integer_hex(long int, stream_pos,
                                    ftell(stream))
        lttng_ust_field_float(double, float_field, flt_arg)
        lttng_ust_field_string(string_field, my_string_arg)
        lttng_ust_field_array(int, array_field, array_arg, 7)
        lttng_ust_field_array_text(char, array_text_field,
                                   array_arg, 5)
        lttng_ust_field_sequence(int, seq_field, array_arg, unsigned int,
                                 my_integer_arg / 10)
        lttng_ust_field_sequence_text(char, seq_text_field,
                                      array_arg, unsigned int,
                                      my_integer_arg / 5)
        lttng_ust_field_enum(my_provider, my_enum, int,
                             enum_field, array_arg[1])
    )
)

LTTNG_UST_TRACEPOINT_LOGLEVEL(my_provider, big_event,
                              LTTNG_UST_TRACEPOINT_LOGLEVEL_WARNING)

LTTNG_UST_TRACEPOINT_EVENT_CLASS(
    my_provider,
    my_tracepoint_class,
    LTTNG_UST_TP_ARGS(
        int, my_integer_arg,
        struct app_struct *, app_struct_arg
    ),
    LTTNG_UST_TP_FIELDS(
        lttng_ust_field_integer(int, a, my_integer_arg)
        lttng_ust_field_integer(unsigned long, b, app_struct_arg->b)
        lttng_ust_field_string(c, app_struct_arg->c)
    )
)

LTTNG_UST_TRACEPOINT_EVENT_INSTANCE(
    my_provider,
    my_tracepoint_class,
    my_provider,
    event_instance1,
    LTTNG_UST_TP_ARGS(
        int, my_integer_arg,
        struct app_struct *, app_struct_arg
    )
)

LTTNG_UST_TRACEPOINT_EVENT_INSTANCE(
    my_provider,
    my_tracepoint_class,
    my_provider,
    event_instance2,
    LTTNG_UST_TP_ARGS(
        int, my_integer_arg,
        struct app_struct *, app_struct_arg
    )
)

LTTNG_UST_TRACEPOINT_LOGLEVEL(my_provider, event_instance2,
                              LTTNG_UST_TRACEPOINT_LOGLEVEL_INFO)

LTTNG_UST_TRACEPOINT_EVENT_INSTANCE(
    my_provider,
    my_tracepoint_class,
    my_provider,
    event_instance3,
    LTTNG_UST_TP_ARGS(
        int, my_integer_arg,
        struct app_struct *, app_struct_arg
    )
)

#endif /* _TP_H */

#include <lttng/tracepoint-event.h>

πŸ¦‹ Tracepoint provider source file πŸ¦‹

tp.c:

#define LTTNG_UST_TRACEPOINT_CREATE_PROBES
#define LTTNG_UST_TRACEPOINT_DEFINE

#include "tp.h"

🌸 Application header file 🌸

app.h:

#ifndef _APP_H
#define _APP_H

struct app_struct {
    unsigned long b;
    const char *c;
    double d;
};

#endif /* _APP_H */

✨ Application source file ✨

app.c:

#include <stdlib.h>
#include <stdio.h>

#include "tp.h"
#include "app.h"

static int array_of_ints[] = {
    100, -35, 1, 23, 14, -6, 28, 1001, -3000,
};

int main(int argc, char* argv[])
{
    FILE *stream;
    struct app_struct app_struct;

    lttng_ust_tracepoint(my_provider, simple_event, argc, argv[0]);
    stream = fopen("/tmp/app.txt", "w");

    if (!stream) {
        fprintf(stderr,
                "Error: Cannot open /tmp/app.txt for writing\n");
        return EXIT_FAILURE;
    }

    if (fprintf(stream, "0123456789") != 10) {
        fclose(stream);
        fprintf(stderr, "Error: Cannot write to /tmp/app.txt\n");
        return EXIT_FAILURE;
    }

    lttng_ust_tracepoint(my_provider, big_event, 35,
                         "hello tracepoint", stream, -3.14,
                         array_of_ints);
    fclose(stream);
    app_struct.b = argc;
    app_struct.c = "[the string]";
    lttng_ust_tracepoint(my_provider, event_instance1, 23,
                         &app_struct);
    app_struct.b = argc * 5;
    app_struct.c = "[other string]";
    lttng_ust_tracepoint(my_provider, event_instance2, 17,
                         &app_struct);
    app_struct.b = 23;
    app_struct.c = "nothing";
    lttng_ust_tracepoint(my_provider, event_instance3, -52,
                         &app_struct);
    return EXIT_SUCCESS;
}

🌸 ENVIRONMENT VARIABLES 🌸

okay bestie this is the part where the vibes get ✨customizable✨ β€” these are all the env vars liblttng-ust reads, and ngl some of them are lowkey important so like… read them fr fr πŸ’…

  • LTTNG_HOME β€” an alternative user home directory. this one's for when the user running the instrumented app has a home directory that's not writable (πŸ™ˆ been there). this path is where the Unix sockets for talking to the per-user session daemon live.
  • LTTNG_UST_ABORT_ON_CRITICAL β€” if set, the instrumented application just straight up aborts on a critical error message. like, it does not play. πŸ’€
  • LTTNG_UST_ALLOW_BLOCKING β€” if set, you're letting the application retry event tracing when there's literally no space left for the event record in the sub-buffer. so basically it blocks the application until space frees up or the configured timeout hits. the way that works 🫢

to actually let an app block during tracing you also need to specify a blocking timeout when you create a channel, using the --blocking-timeout option of the lttng-enable-channel(1) command. both things, not just one, mmkay?

this one's genuinely useful in workloads generating very large trace data throughput, where blocking the application is an acceptable trade-off to prevent discarding event records. πŸ‘‘

⚠️ warning: setting this environment variable may significantly affect application timings. not a drill, bestie.
  • LTTNG_UST_APP_PATH β€” path of the directory where you expect the LTTng session daemon to place the following files, so that it can receive instrumented application connection requests:
    • the application registration Unix socket.
    • the "wait" shared memory files, to wake up instrumented applications that are waiting for a session daemon to start.
    • the agent port file.

this file contains the TCP port the agents must connect to in order to enable LTTng tracing.

this directory must exist when you start the application for LTTng tracing to be enabled. no directory, no tracing, sorry babe 😭

when LTTNG_UST_APP_PATH is set, liblttng-ust:

  • only considers this path to connect to a session daemon.
  • won't connect to root and/or user session daemons as usual. 🚫

see the corresponding LTTNG_UST_CTL_PATH environment variable of lttng-sessiond(8).

  • LTTNG_UST_CLOCK_PLUGIN β€” path to the shared object which acts as the clock override plugin. an example of such a plugin lives in the LTTng-UST documentation under examples/clock-override β€” https://github.com/lttng/lttng-ust/tree/stable-2.16/doc/examples/clock-override ✨
  • LTTNG_UST_DEBUG β€” if set, enable the debug and error output of liblttng-ust. your fave for when you're like "why is nothing working" 😭
  • LTTNG_UST_GETCPU_PLUGIN β€” path to the shared object which acts as the getcpu() override plugin. an example of one lives in the LTTng-UST documentation under examples/getcpu-override β€” https://github.com/lttng/lttng-ust/tree/stable-2.16/doc/examples/getcpu-override πŸ¦‹
  • LTTNG_UST_MAP_POPULATE_POLICY β€” if set, override the policy used to populate shared memory pages within the application. the possible values are:
    • none (default) β€” do not pre-populate any page: take minor faults on first access while tracing.
    • cpu_possible β€” pre-populate pages for all possible CPUs in the system, as shown in /sys/devices/system/cpu/possible.
  • LTTNG_UST_REGISTER_TIMEOUT β€” the waiting time for the registration done session daemon command before proceeding to execute the main program, in milliseconds. ⏱️

the value 0 means do not wait. the value -1 means wait forever. setting this environment variable to 0 is recommended for applications with time constraints on the process startup time. (iconic advice, honestly.)

default: 3000.

  • LTTNG_UST_WITHOUT_BADDR_STATEDUMP β€” if set, prevents liblttng-ust from performing a base address state dump (peep the LTTng-UST state dump section above πŸ‘€).
  • LTTNG_UST_WITHOUT_PROCNAME_STATEDUMP β€” if set, prevents liblttng-ust from performing a procname state dump (also up in the LTTng-UST state dump section, iykyk).

πŸ› BUGS πŸ›

if you encounter any issue or usability problem, pls pls pls report it on the LTTng bug tracker β€” https://bugs.lttng.org/projects/lttng-ust β€” the devs genuinely wanna know, no gatekeeping 🫢


🌈 RESOURCES 🌈


πŸ‘‘ COPYRIGHTS πŸ‘‘

this library is part of the LTTng-UST project.

this library is distributed under the GNU Lesser General Public License, version 2.1 β€” http://www.gnu.org/licenses/old-licenses/lgpl-2.1.en.html. see the COPYING file for more details β€” https://github.com/lttng/lttng-ust/blob/v2.16/COPYING πŸ’œ


πŸŽ€ THANKS πŸŽ€

thanks to Ericsson for funding this work, providing real-life use cases, and testing. absolute icons, no cap πŸ’…

special thanks to Michel Dagenais and the DORSAL laboratory at Γ‰cole Polytechnique de MontrΓ©al β€” http://www.dorsal.polymtl.ca/ β€” for the LTTng journey. 🧚


⭐ AUTHORS ⭐

LTTng-UST was originally written by Mathieu Desnoyers, with additional contributions from various other people. it is currently maintained by Mathieu Desnoyers β€” mathieu.desnoyers@efficios.com πŸ“§


🌸 SEE ALSO 🌸

lttng_ust_tracef(3), lttng_ust_tracelog(3), lttng-gen-tp(1), lttng-ust-dl(3), lttng-ust-cyg-profile(3), lttng(1), lttng-enable-event(1), lttng-list(1), lttng-add-context(1), babeltrace(1), dlopen(3), ld.so(8)


and that's the whole page, bestie!! πŸŽ‰ you made it to the end, main character energy fr fr πŸ”₯ this one's from LTTng 2.16.0, dated 08/27/2026 β€” go forth and trace like the icon you are πŸ’–βœ¨

πŸ“¦ where this page comes from
  • arch rolling: lttng-ust 2.16.0-1 Β· LGPL-2.1-only AND MIT AND GPL-2.0-only AND BSD-3-Clause AND BSD-2-Clause AND GPL-3.0-or-later
  • πŸ“œ the page's license: CC-BY-SA-4.0
  • πŸ’– girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on : a language model's rewrite of the original, under its terms