bpftool-prog(8)

bpftool-prog(8) πŸ’…βœ¨

okay bestie, welcome to bpftool-prog β€” the bit of the System Manager's Manual that teaches you how to peek at and gently poke eBPF progs. this is part 1 of 3 so don't you dare scroll away, there's more coming 🫢


✨ Name ✨

bpftool-prog β€” bestie it's literally the tool for inspection and simple manipulation of eBPF progs. that's the whole vibe. πŸ¦‹


🌸 SYNOPSIS 🌸

the shape of it, no cap:

bpftool [OPTIONS] prog COMMAND

OPTIONS := { { -j | --json } [{ -p | --pretty }] | { -d | --debug } | {
-f | --bpffs } | { -m | --mapcompat } | { -n | --nomount } | { -L |
--use-loader } | [ { -S | --sign } -k <private_key.pem> -i
<certificate.x509> ] }

COMMANDS := { show | list | dump xlated | dump jited | pin | load |
loadall | help }

so you run bpftool, you drop some OPTIONS, then the word prog, then COMMAND. that's the whole recipe, periodt. πŸ‘‘


πŸŽ€ PROG COMMANDS πŸŽ€

here's every single move, laid out for you bestie:

bpftool prog { show | list } [PROG]
bpftool prog dump xlated PROG [{ file FILE | [opcodes] [linum] [visual] }]
bpftool prog dump jited  PROG [{ file FILE | [opcodes] [linum] }]
bpftool prog pin PROG FILE
bpftool prog { load | loadall } OBJ PATH [type TYPE] [map { idx IDX | name NAME } MAP] [{ offload_dev | xdpmeta_dev } NAME] [pinmaps MAP_DIR] [autoattach] [kernel_btf BTF_FILE]
bpftool prog attach PROG ATTACH_TYPE [MAP]
bpftool prog detach PROG ATTACH_TYPE [MAP]
bpftool prog tracelog
bpftool prog tracelog [ { stdout | stderr  } PROG ]
bpftool prog run PROG data_in FILE [data_out FILE [data_size_out L]] [ctx_in FILE [ctx_out FILE [ctx_size_out M]]] [repeat N]
bpftool prog profile PROG [duration DURATION] METRICs
bpftool prog help

and now the little alias definitions, because the commands above keep tossing these around like confetti 🎊:

MAP := { id MAP_ID | pinned FILE | name MAP_NAME }
PROG := { id PROG_ID | pinned FILE | tag PROG_TAG | name PROG_NAME }
TYPE := {
  socket | kprobe | kretprobe | classifier | action |
  tracepoint | raw_tracepoint | xdp | perf_event | cgroup/skb |
  cgroup/sock | cgroup/dev | lwt_in | lwt_out | lwt_xmit |
  lwt_seg6local | sockops | sk_skb | sk_msg | lirc_mode2 |
  cgroup/bind4 | cgroup/bind6 | cgroup/post_bind4 | cgroup/post_bind6 |
  cgroup/connect4 | cgroup/connect6 | cgroup/connect_unix |
  cgroup/getpeername4 | cgroup/getpeername6 | cgroup/getpeername_unix |
  cgroup/getsockname4 | cgroup/getsockname6 | cgroup/getsockname_unix |
  cgroup/sendmsg4 | cgroup/sendmsg6 | cgroup/sendmsg_unix |
  cgroup/recvmsg4 | cgroup/recvmsg6 | cgroup/recvmsg_unix | cgroup/sysctl |
  cgroup/getsockopt | cgroup/setsockopt | cgroup/sock_release |
  struct_ops | fentry | fexit | fsession | freplace | sk_lookup
}
ATTACH_TYPE := {
  sk_msg_verdict | sk_skb_verdict | sk_skb_stream_verdict |
  sk_skb_stream_parser | flow_dissector
}
METRICs := {
  cycles | instructions | l1d_loads | llc_misses |
  itlb_misses | dtlb_misses
}

so PROG can be an id PROG_ID, a pinned FILE, a tag PROG_TAG, or a name PROG_NAME β€” pick your fighters bestie, they all work. same energy for MAP: id MAP_ID, pinned FILE, or name MAP_NAME. πŸ’–

and TYPE? that's the big menu of prog flavours β€” socket, kprobe, kretprobe, classifier, action, tracepoint, raw_tracepoint, xdp, perf_event, cgroup/skb, cgroup/sock, cgroup/dev, lwt_in, lwt_out, lwt_xmit, lwt_seg6local, sockops, sk_skb, sk_msg, lirc_mode2, cgroup/bind4, cgroup/bind6, cgroup/post_bind4, cgroup/post_bind6, cgroup/connect4, cgroup/connect6, cgroup/connect_unix, cgroup/getpeername4, cgroup/getpeername6, cgroup/getpeername_unix, cgroup/getsockname4, cgroup/getsockname6, cgroup/getsockname_unix, cgroup/sendmsg4, cgroup/sendmsg6, cgroup/sendmsg_unix, cgroup/recvmsg4, cgroup/recvmsg6, cgroup/recvmsg_unix, cgroup/sysctl, cgroup/getsockopt, cgroup/setsockopt, cgroup/sock_release, struct_ops, fentry, fexit, fsession, freplace, and sk_lookup. iconic lineup honestly. 🌈

ATTACH_TYPE is a much smaller squad: sk_msg_verdict, sk_skb_verdict, sk_skb_stream_verdict, sk_skb_stream_parser, and flow_dissector. ✨

and the METRICs you can profile against are cycles, instructions, l1d_loads, llc_misses, itlb_misses, and dtlb_misses. zoom zoom. πŸ”₯


okay that's part 1 bestie, hold on to your Ctrl because there's more coming πŸ’œπŸ¦‹

πŸ’… DESCRIPTION πŸ’…

ok so here's the whole menu of what you can actually do with bpftool prog β€” buckle up bestie, this is the good part πŸŽ€

  • bpftool prog { show | list } [PROG] β€” show me the loaded programs pls Show information about loaded programs. If PROG is specified, you only get info about the given programs; otherwise it lists every program currently loaded on the system. Heads up: with tag or name, PROG may match several programs, and all of them get shown. no filtering, no favorites 🫢

Output starts with the program ID, followed by the program type, and then zero or more named attributes (which ones exactly depends on your kernel version).

Since Linux 5.1 the kernel can collect statistics on BPF programs β€” stuff like the total time spent running the program, and the number of times it was run. If they're available, bpftool shows them. BUT 🌸 the kernel does not collect them by default, because it slightly impacts performance on every single program run. You activate or deactivate the feature via the kernel.bpf_stats_enabled sysctl knob.

Since Linux 5.8, bpftool can discover information about processes that hold open file descriptors (FDs) against BPF programs. On such kernels bpftool automatically emits that info too. iconic ✨

  • bpftool prog dump xlated PROG [{ file FILE | [opcodes] [linum] [visual] }] β€” dump the eBPF instructions Dumps the eBPF instructions of the programs straight from the kernel. By default, the eBPF gets disassembled and printed to standard output in human-readable format. In that case, opcodes controls whether raw opcodes get printed as well.

With tag or name, PROG may match several programs, and all of them get dumped. However β€” and this is the important bit β€” if file or visual is specified, PROG must match a single program. one. singular. periodt πŸ’€

If file is specified, the binary image gets written to FILE instead.

If visual is specified, a control flow graph (CFG) gets built instead, and the eBPF instructions are presented with the CFG in DOT format, on standard output. πŸ’…

If the programs have line_info available, the source line gets displayed. If linum is specified, the filename, line number and line column get displayed too.

  • bpftool prog dump jited PROG [{ file FILE | [opcodes] [linum] }] β€” dump the jited image Dumps the jited image (that's the host machine code, bestie) of the program. 🧚

If FILE is specified the image gets written to a file, otherwise it gets disassembled and printed to stdout. PROG must match a single program when file is specified.

opcodes controls whether raw opcodes get printed.

If the prog has line_info available, the source line gets displayed. If linum is specified, the filename, line number and line column get displayed as well.

  • bpftool prog pin PROG FILE β€” pin it down πŸ“Œ Pins program PROG as FILE.
⚠️ FILE must be located in a bpffs mount. It must not contain a dot character ('.'), because that's reserved for future extensions of bpffs. so don't, fr fr.
  • bpftool prog { load | loadall } OBJ PATH [type TYPE] [map { idx IDX | name NAME } MAP] [{ offload_dev | xdpmeta_dev } NAME] [pinmaps MAP_DIR] [autoattach] [kernel_btf BTF_FILE] β€” load it up ⭐ Loads bpf program(s) from the binary OBJ and pins them as PATH. Here's the split: bpftool prog load pins only the first program from the OBJ as PATH, while bpftool prog loadall pins all programs from the OBJ under the PATH directory.

type is optional β€” if you don't specify it, the program type gets inferred from section names.

By default bpftool creates new maps as declared in the ELF object being loaded. The map parameter lets you reuse existing maps, and you can specify it multiple times, each time for a different map. IDX refers to the index of the map to be replaced in the ELF file, counting from 0, while NAME lets you replace a map by name. MAP specifies the map to use β€” referring to it by id or through a pinned file.

If offload_dev NAME is specified, the program gets loaded onto the given networking device (offload). If xdpmeta_dev NAME is specified, the program becomes device-bound without offloading β€” this facilitates access to XDP metadata.

Optional pinmaps argument can be provided to pin all maps under the MAP_DIR directory.

If autoattach is specified, the program gets attached before pinning. In that case, only the link (the thing representing the program attached to its hook) is pinned, not the program as such β€” so the path won't show in bpftool prog show -f, it only shows in bpftool link show -f. πŸ™ˆ Also, this only works when bpftool (libbpf) is able to infer all necessary information from the object file; in particular, it's not supported for all program types. If a program doesn't support autoattach, bpftool falls back to regular pinning for that program instead. it's giving "we tried our best" πŸ’œ

The kernel_btf option allows you to specify an external BTF file to replace the system's own vmlinux BTF file for CO-RE relocations. Note that any other feature relying on BTF (such as fentry/fexit programs, struct_ops) requires the BTF file for the actual kernel running on the host, which is often exposed at /sys/kernel/btf/vmlinux. πŸ‘‘

⚠️ PATH must be located in a bpffs mount. It must not contain a dot character ('.'), which is reserved for future extensions of bpffs.
  • bpftool prog attach PROG ATTACH_TYPE [MAP] β€” attach it πŸ”₯ Attaches bpf program PROG (with the type specified by ATTACH_TYPE). Most ATTACH_TYPEs require a MAP parameter β€” the exception is flow_dissector, which is attached to the current networking name space.
  • bpftool prog detach PROG ATTACH_TYPE [MAP] β€” detach it (we love a clean goodbye) Detaches bpf program PROG (with the type specified by ATTACH_TYPE). Most ATTACH_TYPEs require a MAP parameter β€” the exception is flow_dissector, which is detached from the current networking name space.
  • bpftool prog tracelog β€” the trace pipe, live πŸ“ Dumps the trace pipe of the system to the console (stdout). Hit Ctrl+C to stop printing. BPF programs can write to this trace pipe at runtime with the bpf_trace_printk() helper. This should be used only for debugging purposes. If you want to stream data from BPF programs to user space, you can use perf events (see also bpftool-map(8)).
  • bpftool prog tracelog { stdout | stderr } PROG β€” the program's own stream Dumps the BPF stream of the program. BPF programs can write to these streams at runtime with the bpf_stream_vprintk_impl() kfunc. The kernel may write error messages to the standard error stream. This facility should be used only for debugging purposes. πŸ’€
  • bpftool prog run PROG data_in FILE [data_out FILE [data_size_out L]] [ctx_in FILE [ctx_out FILE [ctx_size_out M]]] [repeat N] β€” vibes-only test run πŸ”₯ Runs BPF program PROG in the kernel testing infrastructure for BPF β€” meaning the program works on the data and context provided by the user, and not on actual packets or monitored functions etc. The return value and duration for the test run get printed out to the console.

Input data is read from the FILE passed with data_in. If this FILE is "-", input data is read from standard input. Input context, if any, is read from the FILE passed with ctx_in. Again, "-" can be used to read from standard input, but only if standard input isn't already in use for input data. If a FILE is passed with data_out, output data is written to that file. Similarly, output context is written to the FILE passed with ctx_out. For both output flows, "-" can be used to print to the standard output (as plain text, or JSON if the relevant option was passed). If the output keywords are omitted, output data and context get discarded. The keywords data_size_out and ctx_size_out are used to pass the size (in bytes) for the output buffers to the kernel β€” although the default of 32 kB should be more than enough for most cases. 😌

The keyword repeat is used to indicate the number of consecutive runs to perform. Note that output data and context printed to files correspond to the last of those runs. The duration printed out at the end of the runs is an average over all runs performed by the command.

Not all program types support test run. And among those which do, not all of them can take the ctx_in/ctx_out arguments. bpftool does not perform checks on program types. so bring your own common sense, bestie 🌈

  • bpftool prog profile PROG [duration DURATION] METRICs β€” give me the numbers πŸ“Š Profiles METRICs for bpf program PROG for DURATION seconds, or until the user hits Ctrl+C. DURATION is optional. If DURATION is not specified, the profiling will run up to UINT_MAX seconds. 😭
  • bpftool prog help β€” short king Prints a short help message.

✨ OPTIONS ✨

okay bestie, here's the whole lil toolbox of flags you can sprinkle on your bpftool prog commands πŸ’… every single one, no skipping, periodt.

  • -h, --help β€” prints a short help message, kinda like what you get from bpftool help πŸ’–
  • -V, --version β€” prints bpftool's version number (similar to bpftool version), plus the number of the libbpf version in use, AND the optional features that were baked in when bpftool was compiled. Those optional features can include linking against LLVM or libbfd so you get the disassembler for JIT-ted programs (bpftool prog dump jited), and usage of BPF skeletons (some features like bpftool prog profile or showing pids associated to BPF objects may rely on it) ✨
  • -j, --json β€” generates JSON output. For commands that cannot produce JSON, this option has no effect. sorry babe, no JSON for those πŸ₯²
  • -p, --pretty β€” generates human-readable JSON output. Implies -j, so it's giving two-for-one πŸŽ€
  • -d, --debug β€” prints ALL logs available, even debug-level information. This includes logs from libbpf as well as from the verifier, when attempting to load programs. it's the yapping mode and we love her for it πŸ”₯
  • -f, --bpffs β€” when showing BPF programs, show file names of pinned programs πŸ¦‹
  • -m, --mapcompat β€” allows loading maps with unknown map definitions 🌸
  • -n, --nomount β€” do NOT automatically attempt to mount any virtual file system (such as tracefs or BPF virtual file system) when necessary πŸ’…
  • -L, --use-loader β€” loads the program as a "loader" program. This is useful to debug the generation of such programs. When this option is in use, bpftool attempts to load the programs from the object file into the kernel, but does not pin them (therefore, the PATH must not be provided).

When combined with the -d|--debug option, additional debug messages are generated, and the execution of the loader program will use the bpf_trace_printk() helper to log each step of loading BTF, creating the maps, and loading the programs (see bpftool prog tracelog as a way to dump those messages) πŸ‘‘

  • -S, --sign β€” enables signing of the BPF program before loading. This option must be used with -k and -i. Using this flag implicitly enables --use-loader πŸ”
  • -k <private_key.pem> β€” path to the private key file in PEM format, required when signing πŸ”‘
  • -i <certificate.x509> β€” path to the X.509 certificate file in PEM or DER format, required when signing πŸ“œ

🌈 EXAMPLES 🌈

bestie, these are the money shots. copy them exactly, don't freestyle:

# bpftool prog show

   10: xdp  name some_prog  tag 005a3d2123620c8b  gpl run_time_ns 81632 run_cnt 10
           loaded_at 2017-09-29T20:11:00+0000  uid 0
           xlated 528B  jited 370B  memlock 4096B  map_ids 10
           pids systemd(1)
# bpftool --json --pretty prog show

   [{
           "id": 10,
           "type": "xdp",
           "tag": "005a3d2123620c8b",
           "gpl_compatible": true,
           "run_time_ns": 81632,
           "run_cnt": 10,
           "loaded_at": 1506715860,
           "uid": 0,
           "bytes_xlated": 528,
           "jited": true,
           "bytes_jited": 370,
           "bytes_memlock": 4096,
           "map_ids": [10
           ],
           "pids": [{
                   "pid": 1,
                   "comm": "systemd"
               }
           ]
       }
   ]
# bpftool prog dump xlated id 10 file /tmp/t
$ ls -l /tmp/t

   -rw------- 1 root root 560 Jul 22 01:42 /tmp/t
# bpftool prog dump jited tag 005a3d2123620c8b

   0:   push   %rbp
   1:   mov    %rsp,%rbp
   2:   sub    $0x228,%rsp
   3:   sub    $0x28,%rbp
   4:   mov    %rbx,0x0(%rbp)
# mount -t bpf none /sys/fs/bpf/
# bpftool prog pin id 10 /sys/fs/bpf/prog
# bpftool prog load ./my_prog.o /sys/fs/bpf/prog2
# ls -l /sys/fs/bpf/

   -rw------- 1 root root 0 Jul 22 01:43 prog
   -rw------- 1 root root 0 Jul 22 01:44 prog2
# bpftool prog dump jited pinned /sys/fs/bpf/prog opcodes

   0:   push   %rbp
        55
   1:   mov    %rsp,%rbp
        48 89 e5
   4:   sub    $0x228,%rsp
        48 81 ec 28 02 00 00
   b:   sub    $0x28,%rbp
        48 83 ed 28
   f:   mov    %rbx,0x0(%rbp)
        48 89 5d 00
# bpftool prog load xdp1_kern.o /sys/fs/bpf/xdp1 type xdp map name rxcnt id 7
# bpftool prog show pinned /sys/fs/bpf/xdp1

   9: xdp  name xdp_prog1  tag 539ec6ce11b52f98  gpl
           loaded_at 2018-06-25T16:17:31-0700  uid 0
           xlated 488B  jited 336B  memlock 4096B  map_ids 7
# rm /sys/fs/bpf/xdp1
# bpftool prog profile id 337 duration 10 cycles instructions llc_misses

      51397 run_cnt
   40176203 cycles                                                 (83.05%)
   42518139 instructions    #   1.06 insns per cycle               (83.39%)
        123 llc_misses      #   2.89 LLC misses per million insns  (83.15%)

Output below is for the trace logs.

Run in separate terminals:

# bpftool prog tracelog
# bpftool prog load -L -d file.o

   bpftool-620059  [004] d... 2634685.517903: bpf_trace_printk: btf_load size 665 r=5
   bpftool-620059  [004] d... 2634685.517912: bpf_trace_printk: map_create sample_map idx 0 type 2 value_size 4 value_btf_id 0 r=6
   bpftool-620059  [004] d... 2634685.517997: bpf_trace_printk: prog_load sample insn_cnt 13 r=7
   bpftool-620059  [004] d... 2634685.517999: bpf_trace_printk: close(5) = 0

πŸ’œ SEE ALSO πŸ’œ

go read these too bestie, they're all connected, iykyk:

bpf(2), bpf-helpers(7), bpftool(8), bpftool-btf(8), bpftool-cgroup(8), bpftool-feature(8), bpftool-gen(8), bpftool-iter(8), bpftool-link(8), bpftool-map(8), bpftool-net(8), bpftool-perf(8), bpftool-struct_ops(8), bpftool-token(8)


and that's the whole page, bestie 🫢 this was bpftool-prog(8) β€” the signature on the bottom is literally just the page name itself, so consider this your lil goodbye kiss πŸ’‹ mwah. go forth and probe those programs, you absolute main character πŸ‘‘βœ¨

πŸ“¦ where this page comes from