LTTNG-EVENT-RULE(7)

πŸ’… LTTNG-EVENT-RULE(7) πŸ’…

LTTng Manual β€” the common LTTng event rule specification, and yes bestie, we’re getting into it.

✨ NAME ✨

lttng-event-rule β€” Common LTTng event rule specification

πŸ¦‹ SYNOPSIS πŸ¦‹

Okay so here’s how you actually say an event rule on the command line. Pick your flavor, bestie.

To match Linux kernel tracepoint or system call events:

--type=(kernel:tracepoint | kernel:syscall[:entry|:exit|:entry+exit])]
[--name=NAME] [--filter=EXPR]

To match Linux kernel kprobe or user space probe events:

--type=(kernel:kprobe | kernel:uprobe) --location=LOC
[--event-name=EVENTNAME]

To match user space tracepoint events:

--type=user:tracepoint [--name=NAME] [--exclude-name=XNAME]...
[--log-level=(LOGLEVEL | LOGLEVEL.. | ..)] [--filter=EXPR]

To match Java/Python logging events:

--type=(jul | log4j | log4j2 | python):logging [--name=NAME]
[--log-level=(LOGLEVEL | LOGLEVEL.. | ..)] [--filter=EXPR]

🌸 DESCRIPTION 🌸

This manual page shows how to specify an LTTng event rule on the command line.

As of LTTng 2.15.1, the command-line options documented here only apply to the event-rule-matches trigger condition specifier (see lttng-add-trigger(1)).

See lttng-concepts(7) to learn more about instrumentation points, events, and event rules.

Note β€” This manual page only describes the common event rule options. The lttng(1) commands which require an event rule specification may accept or require other options and arguments, depending on the context. For example, the lttng-add-trigger(1) command also accepts --capture options with the event-rule-matches trigger condition.

πŸŽ€ Overview of event rule condtions πŸŽ€

For LTTng to emit an event E, E must satisfy all the conditions of an event rule, that is:

  • The instrumentation point from which LTTng creates E has a specific type. See the β€œInstrumentation point type condition” section below.
  • A pattern matches the name of E while another pattern doesn’t. See the β€œEvent name condition” section below.
  • The log level of the instrumentation point from which LTTng creates E is at least as severe as some value, or is exactly some value. See the β€œInstrumentation point log level condition” section below.
  • The fields of the payload of E and the current context fields satisfy a filter expression. See the β€œEvent payload and context filter condition” section below.

The dedicated command-line options of most conditions are optional: if you don’t specify the option, the associated condition is always satisfied.

🧚 Instrumentation point type condition 🧚

An event E satisfies the instrumentation point type condition of an event rule if the instrumentation point from which LTTng creates E is, depending on the argument of the --type option:

  • kernel:tracepoint β€” An LTTng kernel tracepoint, that is, a statically defined point in the source code of the kernel image or of a kernel module with LTTng kernel tracer macros. List the available Linux kernel tracepoints with lttng list --kernel. See lttng-list(1) to learn more.
  • kernel:syscall:entry, kernel:syscall:exit, kernel:syscall:entry+exit β€” The entry, exit, or entry and exit of a Linux kernel system call. List the available Linux kernel system call instrumentation points with lttng list --kernel --syscall. See lttng-list(1) to learn more.
  • kernel:kprobe β€” A Linux kprobe, that is, a single probe dynamically placed in the compiled kernel code. You must specify the kprobe location with the --location option. The payload of a Linux kprobe event is empty.
  • kernel:uprobe β€” A Linux user space probe, that is, a single probe dynamically placed at the entry of a compiled user space application/library function through the kernel. LTTng 2.15.1 supports the ELF and SystemTap User-level Statically Defined Tracing (USDT; a DTrace-style marker) probing methods. LTTng only supports USDT probes which are NOT reference-counted. You must specify the user space probe location with the --location option. The payload of a Linux user space probe event is empty.
  • user:tracepoint β€” An LTTng user space tracepoint, that is, a statically defined point in the source code of a C/C++ application/library with LTTng user space tracer macros. List the available user space tracepoints with lttng list --userspace. See lttng-list(1) to learn more.
  • jul:logging β€” A java.util.logging logging statement. List the available java.util.logging loggers with lttng list --jul. See lttng-list(1) to learn more.
  • log4j:logging β€” An Apache log4j 1.x logging statement. List the available Apache log4j 1.x loggers with lttng list --log4j. See lttng-list(1) to learn more.
  • log4j2:logging β€” An Apache Log4j 2 logging statement. List the available Apache Log4j 2 loggers with lttng list --log4j2. See lttng-list(1) to learn more.
  • python:logging β€” A Python logging statement. List the available Python loggers with lttng list --python. See lttng-list(1) to learn more.

🌈 Event name condition 🌈

An event E satisfies the event name condition of an event rule ER if the two following statements are true:

  • You don’t specify the --name=NAME option or, depending on the instrumentation type condition (see the β€œInstrumentation point type condition” section above) of ER, NAME matches:
    • kernel:tracepoint, user:tracepoint The full name of the tracepoint from which LTTng creates E.

Note that the full name of a user space tracepoint is PROVIDER:NAME, where PROVIDER is the tracepoint provider name and NAME is the tracepoint name.

  • jul:logging, log4j:logging, log4j2:logging, python:logging The name of the Java or Python logger from which LTTng creates E.
  • kernel:syscall:entry, kernel:syscall:exit, kernel:syscall:entry+exit The name of the system call, without any sys_ prefix, from which LTTng creates E.
  • You don’t specify any --exclude-name=XNAME option or none of the XNAME arguments matches the full name of the user space tracepoint from which LTTng creates E.

The --exclude-name option is only available with the --type=user:tracepoint option.

This condition is only meaningful for the LTTng tracepoint, logging statement, and Linux system call instrumentation point types: it’s always satisfied for the other types.

In all cases, NAME and XNAME are globbing patterns: the * character means β€œmatch anything”. To match a literal * character, use \*.

Important β€” Make sure to single-quote NAME and XNAME when they contain the * character and when you run an lttng(1) command from a shell.

As of LTTng 2.15.1, not specifying the --name option is equivalent to specifying --name='*Β΄, but this default may change in the future.

πŸ’œ Instrumentation point log level condition πŸ’œ

An event E satisfies the instrumentation point log level condition of an event rule if either:

  • You specify the --log-level=.. option or you don’t specify the --log-level option.

Defaulting to --log-level=.. when you don’t specify the --log-level option is specific to LTTng 2.15.1 and may change in the future.

  • The log level of the LTTng user space tracepoint or logging statement from which LTTng creates E is:
    • With the --log-level=LOGLEVEL.. option At least as severe as LOGLEVEL.
    • With the --log-level=LOGLEVEL option Exactly LOGLEVEL.

As of LTTng 2.15.1, the ..LOGLEVEL and LOGLEVEL..LOGLEVEL formats are NOT supported.

This condition is only meaningful for the LTTng user space tracepoint and logging statement instrumentation point types: it’s always satisfied for other types.

The available values of LOGLEVEL are, depending on the argument of the --type option, from the most to the least severe:

user:tracepoint

  • EMERG (0)
  • ALERT (1)
  • CRIT (2)
  • ERR (3)
  • WARNING (4)
  • NOTICE (5)
  • INFO (6)
  • DEBUG_SYSTEM (7)
  • DEBUG_PROGRAM (8)
  • DEBUG_PROCESS (9)
  • DEBUG_MODULE (10)
  • DEBUG_UNIT (11)
  • DEBUG_FUNCTION (12)
  • DEBUG_LINE (13)
  • DEBUG (14)

jul:logging

  • OFF (INT32_MAX)
  • SEVERE (1000)
  • WARNING (900)
  • INFO (800)
  • CONFIG (700)
  • FINE (500)
  • FINER (400)
  • FINEST (300)
  • ALL (INT32_MIN)

log4j:logging

  • OFF (INT32_MAX)
  • FATAL (50000)
  • ERROR (40000)
  • WARN (30000)
  • INFO (20000)
  • DEBUG (10000)
  • TRACE (5000)
  • ALL (INT32_MIN)

log4j2:logging

  • OFF (0)
  • FATAL (100)
  • ERROR (200)
  • WARN (300)
  • INFO (400)
  • DEBUG (500)
  • TRACE (600)
  • ALL (INT32_MAX)

python:logging

  • CRITICAL (50)
  • ERROR (40)
  • WARNING (30)
  • INFO (20)
  • DEBUG (10)
  • NOTSET (0)

πŸ¦‹ Event payload and context filter condition πŸ¦‹

An event E satisfies the event payload and context filter condition of an event rule if the --filter=EXPR option is missing or if EXPR is true.

This condition is only meaningful for the LTTng tracepoint and Linux system call instrumentation point types: it’s always satisfied for other types.

EXPR can contain references to the payload fields of E and to the current context fields.

Important β€” Make sure to single-quote EXPR when you run an lttng(1) command from a shell, as filter expressions typically include characters having a special meaning for most shells.

The expected syntax of EXPR is similar to the syntax of a C language conditional expression (an expression which an if statement can evaluate), but there are a few differences:

  • A NAME expression identifies an event payload field named NAME (a C identifier).

Use the C language dot and square bracket notations to access nested structure and array/sequence fields. You can only use a constant, positive integer number within square brackets. If the index is out of bounds, EXPR is false.

The value of an enumeration field is an integer.

When a field expression doesn’t exist, EXPR is false.

Examples: my_field, target_cpu, seq[7], msg.user[1].data[2][17].

  • A $ctx.TYPE expression identifies the statically-known context field of type TYPE (a C identifier). 🫢

Wanna see the actual statically-known context field names you can vibe with? Run the lttng-add-context(1) command and it'll spill. Heads up tho: as of LTTng 2.15.1, context field names that start with perf: are not supported, so don't come for me when that one doesn't work. πŸ™ˆ

When a field expression doesn't exist, EXPR is false. Periodt.

Examples: $ctx.prio, $ctx.gid, $ctx.preemptible.

  • A $app.PROVIDER:TYPE expression identifies the application-specific context field of type TYPE (a C identifier) from the provider PROVIDER (a C identifier). ✨

When a field expression doesn't exist, EXPR is false. Same energy as above, no notes.

Example: $app.server:cur_user.

  • You can compare strings β€” either string fields or string literals (double-quoted) β€” with the == and != operators. πŸ’…

When you're comparing to a string literal, the * character means β€œmatch anything”. And if you literally want to match an actual * character? Use \*. Don't forget the backslash, bestie.

Examples: my_field == "user34", my_field == my_other_field, my_field == "192.168.*".

  • Here's the whole precedence table of the operators that EXPR supports. Highest precedence is 1, meaning it grabs first. πŸ‘‘
    • Precedence 1 β€” - β€” Unary minus β€” right-to-left
    • Precedence 1 β€” + β€” Unary plus β€” right-to-left
    • Precedence 1 β€” ! β€” Logical NOT β€” right-to-left
    • Precedence 1 β€” ~ β€” Bitwise NOT β€” right-to-left
    • Precedence 2 β€” << β€” Bitwise left shift β€” left-to-right
    • Precedence 2 β€” >> β€” Bitwise right shift β€” left-to-right
    • Precedence 3 β€” & β€” Bitwise AND β€” left-to-right
    • Precedence 4 β€” ^ β€” Bitwise XOR β€” left-to-right
    • Precedence 5 β€” | β€” Bitwise OR β€” left-to-right
    • Precedence 6 β€” < β€” Less than β€” left-to-right
    • Precedence 6 β€” <= β€” Less than or equal to β€” left-to-right
    • Precedence 6 β€” > β€” Greater than β€” left-to-right
    • Precedence 6 β€” >= β€” Greater than or equal to β€” left-to-right
    • Precedence 7 β€” == β€” Equal to β€” left-to-right
    • Precedence 7 β€” != β€” Not equal to β€” left-to-right
    • Precedence 8 β€” && β€” Logical AND β€” left-to-right
    • Precedence 9 β€” || β€” Logical OR β€” left-to-right

Parentheses are supported so you can bypass the default order whenever you feel like it. πŸŽ€

Important ⚠️ Unlike the C language, the bitwise AND and OR operators (& and |) in EXPR take precedence over the relational operators (<, <=, >, >=, ==, and !=). This means the expression 2 & 2 == 2 is true while the equivalent C expression is false. Yeah. She's different like that. πŸ’€

The arithmetic operators are NOT supported. Not a thing. Don't ask. πŸ™…β€β™€οΈ

LTTng first casts all integer constants and fields to signed 64-bit integers. The representation of negative integers is two's complement. Which means that, for example, the signed 8-bit integer field 0xff (-1) becomes 0xffffffffffffffff (still -1) once it's been casted. Math is math. 🧚

Before a bitwise operator is applied, LTTng casts all its operands to unsigned 64-bit integers, and then casts the result back to a signed 64-bit integer. For the bitwise NOT operator, it's the equivalent of this C expression:

(int64_t) ~((uint64_t) val)

For the binary bitwise operators, it's the equivalent of those C expressions:

(int64_t) ((uint64_t) lhs >> (uint64_t) rhs)
(int64_t) ((uint64_t) lhs << (uint64_t) rhs)
(int64_t) ((uint64_t) lhs & (uint64_t) rhs)
(int64_t) ((uint64_t) lhs ^ (uint64_t) rhs)
(int64_t) ((uint64_t) lhs | (uint64_t) rhs)

If the right-hand side of a bitwise shift operator (<< and >>) is not in the [0, 63] range, then EXPR is false. Stay in your lane, range-wise. 😀

EXPR examples:

msg_id == 23 && size >= 2048

$ctx.procname == "lttng*" && (!flag || poel < 34)

$app.my_provider:my_context == 17.34e9 || some_enum >= 14

$ctx.cpu_id == 2 && filename != "*.log"

eax_reg & 0xff7 == 0x240 && x[4] >> 12 <= 0x1234

🌈 Migration from a recording event rule specification 🌈

Since LTTng 2.13, what this manual page documents is the standard, common way to specify an LTTng event rule. It's the moment, honestly. ✨

With the lttng-enable-event(1) command, you also specify an event rule β€” but with deprecated options and arguments. πŸ‘€ So yeah, it works, but it's giving legacy. Use the standard way instead, bestie.

okay so bestie, here’s the translation cheat sheet from the lttng-enable-event(1) options and arguments over to the common event rule specification options. It’s giving glow-up but for command-line flags ✨. One mapping at a time, no cap:

  • Recording event rule option(s)/argument(s): --kernel and --tracepoint β†’ Common event rule option(s): --type=kernel:tracepoint
  • Recording event rule option(s)/argument(s): --kernel and --syscall β†’ Common event rule option(s): --type=kernel:syscall:entry+exit
  • Recording event rule option(s)/argument(s): --probe=LOC and RECORDNAME (non-option) β†’ Common event rule option(s): --type=kernel:kprobe, --location=LOC, and --event-name=RECORDNAME
  • Recording event rule option(s)/argument(s): --userspace-probe=LOC and RECORDNAME (non-option) β†’ Common event rule option(s): --type=kernel:uprobe, --location=LOC, and --event-name=RECORDNAME
  • Recording event rule option(s)/argument(s): --function=LOC and RECORDNAME (non-option) β†’ Common event rule option(s): Not available as of LTTng 2.15.1
  • Recording event rule option(s)/argument(s): --userspace and --tracepoint β†’ Common event rule option(s): --type=user:tracepoint
  • Recording event rule option(s)/argument(s): --jul and --tracepoint β†’ Common event rule option(s): --type=jul:logging
  • Recording event rule option(s)/argument(s): --log4j and --tracepoint β†’ Common event rule option(s): --type=log4j:logging
  • Recording event rule option(s)/argument(s): --log4j2 and --tracepoint β†’ Common event rule option(s): --type=log4j2:logging
  • Recording event rule option(s)/argument(s): --python and --tracepoint β†’ Common event rule option(s): --type=python:logging
  • Recording event rule option(s)/argument(s): NAME (non-option) β†’ Common event rule option(s): --name=NAME
  • Recording event rule option(s)/argument(s): --all β†’ Common event rule option(s): --name='*Β΄ or no --name option
  • Recording event rule option(s)/argument(s): --exclude=XNAME[,XNAME]... β†’ Common event rule option(s): --exclude-name=XNAME for each XNAME
  • Recording event rule option(s)/argument(s): --loglevel=LOGLEVEL β†’ Common event rule option(s): --log-level=LOGLEVEL..
  • Recording event rule option(s)/argument(s): --loglevel-only=LOGLEVEL β†’ Common event rule option(s): --log-level=LOGLEVEL
  • Recording event rule option(s)/argument(s): --filter=EXPR β†’ Common event rule option(s): --filter=EXPR

✨ Instrumentation point type condition ✨

Sis, go peep the β€œInstrumentation point type condition” section above first, this bit is just the actual flags. πŸ‘‡

  • -E NAME, --event-name=NAME β€” when you're using --type=kernel:kprobe or --type=kernel:uprobe, this sets the name of the events that get emitted to NAME, instead of using the LOC argument from --location=LOC. πŸ’«

Defaulting to LOC is a very LTTng 2.15.1 thing and it might change later, so don't get too attached bestie.

  • -L LOC, --location=LOC β€” sets LOC, but what LOC even means depends on your type, okurr? πŸ‘‘
    • With the --type=kernel:kprobe option β€” set the location of the Linux kprobe you wanna insert to LOC.

LOC is one of:

  • An address (the 0x hexadecimal prefix is supported, go off). πŸ”’
  • A symbol name.
  • A symbol name and an offset (in the SYMBOL+OFFSET format).
  • With the --type=kernel:uprobe option β€” set the location of the user space probe you wanna insert to LOC.

LOC is one of:

  • [elf:]PATH:SYMBOL β€” an available symbol inside a user space application or library.
    • PATH β€” the app or library path. It's one of:
      • An absolute path.
      • A relative path.
      • The name of an application as found in the directories listed in the PATH environment variable.
    • SYMBOL β€” the symbol name of the function whose entry you want to instrument.

SYMBOL can be any defined code symbol from the output of the nm(1) command, including with its --dynamic option (that one lists dynamic symbols).

As of LTTng 2.15.1, not writing elf: is the same as writing it, but this default may change in the future, so heads up. ⚠️

Examples:

  • /usr/lib/libc.so.6:malloc
  • ./myapp:createUser
  • elf:httpd:ap_run_open_htaccess
  • sdt:PATH:PROVIDER:NAME β€” a SystemTap User-level Statically Defined Tracing (USDT) probe inside a user space application or library. 🧚
    • PATH β€” the app or library path. This can be:
      • An absolute path.
      • A relative path.
      • The name of an application as found in the directories listed in the PATH environment variable.
    • PROVIDER, NAME β€” the USDT provider and probe names, respectively.

For example, with this USDT probe:

DTRACE_PROBE2("server", "accept_request",
              request_id, ip_addr);

The provider/probe name pair is server:accept_request. πŸ’…

Example: sdt:./build/server:server:accept_request

  • -t TYPE, --type=TYPE β€” only match events that LTTng creates from an instrumentation point whose type is TYPE. Filtering, but make it fashion. ✨

TYPE is one of:

  • kernel:tracepoint β€” an LTTng kernel tracepoint.
    • As of LTTng 2.15.1, kernel is an alias, but this may change in the future.
  • user:tracepoint β€” an LTTng user space tracepoint.
    • As of LTTng 2.15.1, user is an alias, but this may change in the future.
  • kernel:syscall:entry β€” a Linux system call entry.
    • As of LTTng 2.15.1, syscall:entry is an alias, but this may change in the future.
  • kernel:syscall:exit β€” a Linux system call exit.
    • As of LTTng 2.15.1, syscall:exit is an alias, but this may change in the future.
  • kernel:syscall:entry+exit β€” a Linux system call entry and exit (that's two distinct instrumentation points, not one).
    • As of LTTng 2.15.1, these are all aliases, but this may change in the future:
      • syscall:entry+exit
      • kernel:syscall
      • syscall
  • kernel:kprobe β€” a Linux kprobe.
    • As of LTTng 2.15.1, kprobe is an alias, but this may change in the future.
    • You must specify the location of the kprobe to insert with the --location option.
    • You may specify the name of the emitted events with the --event-name option.
  • kernel:uprobe β€” a Linux user space probe.
    • You must specify the location of the user space probe to insert with the --location option.
    • You may specify the name of the emitted events with the --event-name option.
  • jul:logging β€” a java.util.logging logging statement.
    • As of LTTng 2.15.1, jul is an alias, but this may change in the future.
  • log4j:logging β€” an Apache log4j 1.x logging statement.
    • As of LTTng 2.15.1, log4j is an alias, but this may change in the future.
  • log4j2:logging β€” an Apache Log4j 2 logging statement.
    • As of LTTng 2.15.1, log4j2 is an alias, but this may change in the future.
  • python:logging β€” a Python logging statement.
    • As of LTTng 2.15.1, python is an alias, but this may change in the future.

πŸ’– Event name condition πŸ’–

Again bestie, the β€œEvent name condition” section up top has the lore β€” this is just the flags. 🫢

  • -n NAME, --name=NAME β€” only match events where NAME matches, and what "matches" means depends on your --type argument:
    • kernel:tracepoint, user:tracepoint β€” the full name of the LTTng tracepoint.
    • jul:logging, log4j:logging, log4j2:logging, python:logging β€” the Java or Python logger name.
    • kernel:syscall:entry, kernel:syscall:exit, kernel:syscall:entry+exit β€” the name of the system call, with no sys_ prefix.
🚨 This option is NOT available with the other instrumentation point types. Periodt. 🚫

As of LTTng 2.15.1, not specifying this option is equivalent to specifying --name='*Β΄ (when it applies), but this default may change in the future.

  • -x XNAME, --exclude-name=XNAME β€” only match events where XNAME does NOT match the full name of the LTTng user space tracepoint. πŸ™ˆ

Only available with the --type=user:tracepoint option.

NAME and XNAME are globbing patterns: the * character means "match anything". Wanna match a literal * character? Use \*. ⭐

πŸŽ€ Instrumentation point log level condition πŸŽ€

Yes, yes β€” β€œInstrumentation point log level condition”, that section above, mmkay? Here's the flag. πŸ’…

  • -l LOGLEVELSPEC, --log-level=LOGLEVELSPEC β€” only match events where the log level of the LTTng tracepoint or logging statement is… well, it depends on how LOGLEVELSPEC is formatted:
    • LOGLEVEL.. β€” at least as severe as LOGLEVEL.
    • LOGLEVEL β€” exactly LOGLEVEL.
    • .. β€” literally anything. Vibes only, no filter. 🌈
🚫 This option is NOT available with the following options: - --type=kernel:tracepoint - --type=kernel:syscall:entry - --type=kernel:syscall:exit - --type=kernel:syscall:entry+exit - --type=kernel:kprobe - --type=kernel:uprobe

As of LTTng 2.15.1, not specifying this option is equivalent to specifying --log-level=.. (when it applies), but this default may change in the future.

🌸 Event payload and context filter condition 🌸

Last one, and yeah β€” β€œEvent payload and context filter condition” above, go read it. πŸ“–

  • -f EXPR, --filter=EXPR β€” only match events where EXPR is true, and EXPR can contain references to event payload and current context fields. 🧐
This option is only available with the following options: - --type=kernel:tracepoint - --type=kernel:syscall:entry - --type=kernel:syscall:exit - --type=kernel:syscall:entry+exit

🌈 RESOURCES 🌈


This program is part of the LTTng-tools project.

LTTng-tools is distributed under the GNU General Public License version 2 <http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html>. See the LICENSE <https://github.com/lttng/lttng-tools/blob/master/LICENSE> file for details.


🫢 THANKS 🫢

Special thanks to Michel Dagenais and the DORSAL laboratory <http://www.dorsal.polymtl.ca/> at Γ‰cole Polytechnique de MontrΓ©al for the LTTng journey. πŸ₯ΉπŸ’–

Also thanks to the Ericsson teams working on tracing, who helped greatly with detailed bug reports and unusual test cases. Y'all ate, fr fr. πŸ‘‘


πŸ‘€ SEE ALSO πŸ‘€

lttng(1), lttng-add-trigger(1), lttng-list(1), lttng-concepts(7)


And that's the whole page bestie, we did it!! πŸŽ‰ This was LTTng 2.15.1, dated 1 April 2025 β€” iconic, no cap, go trace something pretty. πŸ’…βœ¨πŸ¦‹πŸ“

πŸ“¦ where this page comes from