pahole(1)

โœจ NAME โœจ

pahole โ€” show or manipulate data structure layout and pretty print raw data

and yes bestie, this is the pahole(1) page, from the dwarves collection ๐Ÿ’– pull up a chair.

๐Ÿฆ‹ SYNOPSIS ๐Ÿฆ‹

pahole [options] files

๐ŸŒธ DESCRIPTION ๐ŸŒธ

okay so pahole shows data structure layouts that are encoded in debugging information formats. the ones it fucks with are DWARF, CTF and BTF. that's the roster, periodt. ๐Ÿ’…

this is useful for, among other things: optimizing important data structures by reducing their size, identifying the field sitting at an offset from the start of a data structure, investigating ABI changes and more generally understanding a new codebase you have to work with. it's giving "who is this struct and why is it like this" detective energy ๐Ÿ”

it also uses these structure layouts to pretty print data fed to its standard input, e.g.:

$ pahole --header elf64_hdr --prettify /lib/modules/5.8.0-rc6+/build/vmlinux
{
     .e_ident = { 127, 69, 76, 70, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
     .e_type = 2,
     .e_machine = 62,
     .e_version = 1,
     .e_entry = 16777216,
     .e_phoff = 64,
     .e_shoff = 604653784,
     .e_flags = 0,
     .e_ehsize = 64,
     .e_phentsize = 56,
     .e_phnum = 5,
     .e_shentsize = 64,
     .e_shnum = 80,
     .e_shstrndx = 79,
},
$

see the PRETTY PRINTING section for further examples and documentation ๐Ÿซถ


the files must have associated debugging information. this information may be inside the file itself, in ELF sections, or in another file.

one way to have this information is to specify the -g option to the compiler when building it. when this is done, the information will be stored in an ELF section. for the DWARF debugging information format this adds, among others, the .debug_info ELF section. for CTF it is found in just one ELF section, .SUNW_ctf. BTF comes with at least the .BTF ELF section, and may come also with the .BTF.ext ELF section. โœจ

the debuginfo packages available in most Linux distributions are also supported by pahole, where the debugging information is available in a separate file.

by default, pahole shows the layout of all named structs in the files specified. all of them. every single one. ๐Ÿ‘‘

if no files are specified, then it will look if /sys/kernel/btf/vmlinux is present, using the BTF information contained in it about the running kernel, i.e., this works:

$ pahole list_head
struct list_head {
     struct list_head *         next;                 /*     0     8 */
     struct list_head *         prev;                 /*     8     8 */

     /* size: 16, cachelines: 1, members: 2 */
     /* last cacheline: 16 bytes */
};
$

it is possible to override the /sys/kernel/btf/vmlinux file location by setting the PAHOLE_VMLINUX_BTF_FILENAME environment variable. this may be useful for testing or scripting when using a different BTF for vmlinux. used in the pahole regression tests. ๐Ÿงช

if BTF is not present and no file is passed, then a vmlinux that matches the build-id for the running kernel will be looked up in the usual places, including where the kernel debuginfo packages put it, looking for DWARF info instead. it will simply find it, no stress.

see the EXAMPLES section for more usage suggestions.

it also pretty prints whatever is fed to its standard input, according to the type specified, see the EXAMPLES section.

use --count to state how many records should be pretty printed.


๐Ÿ’… OPTIONS ๐Ÿ’…

  • -C, --class_name=CLASS_NAMES โ€” show just these classes. this can be a comma separated list of class names or file URLs (e.g.: file://class_list.txt)
  • -c, --cacheline_size=SIZE โ€” set cacheline size to SIZE bytes.
  • --sort โ€” sort the output by type name, maybe this will grow to allow sorting by other criteria. this is mostly needed so that pretty printing from BTF and DWARF can be comparable when using using multiple threads to load DWARF data, when the order that the types in the compile units is processed is not deterministic.
  • --compile โ€” generate compileable code, with all definitions for all types, i.e.:
$ pahole --compile > vmlinux.h

produces a header that can be included in a C source file and built. in the example provided it will use the BTF info if available, otherwise will look for a DWARF file matching the running kernel build-id.

  • --skip_emitting_atomic_typedefs โ€” do not emit 'typedef _Atomic int atomic_int' & friends when used with options like --compile. use it if the compiler provides these already, as of circa 2022 with gcc 12.2.1 those are not encoded in DWARF so to generate compilable code we need emit those typedefs for the atomic types used in the data structures being emitted from debugging information.
  • --count=COUNT โ€” pretty print the first COUNT records from input.
  • --skip=COUNT โ€” skip COUNT input records.
  • -E, --expand_types โ€” expand class members. useful to find in what member of inner structs where an offset from the beginning of a struct is.
  • -F, --format_path โ€” allows specifying a list of debugging formats to try, in order. right now this includes "btf", "ctf" and "dwarf". the default format path used is equivalent to "-F dwarf,btf,ctf".
  • --hashbits=BITS โ€” allows specifying the number of bits for the debugging format loader to use. the only one affected so far is the "dwarf" one, its default now is 15, the maximum for it is now 21 bits. tweak it to see if it improves performance as the kernel evolves and more types and functions have to be loaded.
  • --hex โ€” print offsets and sizes in hexadecimal.
  • -r, --rel_offset โ€” show relative offsets of members in inner structs.
  • -p, --expand_pointers โ€” expand class pointer members.
  • -R, --reorganize โ€” reorganize struct, demoting and combining bitfields, moving members to remove alignment holes and padding.
  • -S, --show_reorg_steps โ€” show the struct layout at each reorganization step.
  • -i, --contains=CLASS_NAME โ€” show classes that contains CLASS_NAME.
  • --contains_enumerator=ENUMERATOR_NAME โ€” show enumeration that contains ENUMERATOR_NAME.
$ pahole --contains_enumerator S_VERSION
enum file_time_flags {
     S_ATIME   = 1,
     S_MTIME   = 2,
     S_CTIME   = 4,
     S_VERSION = 8,
}
$
  • -a, --anon_include โ€” include anonymous classes.
  • -A, --nested_anon_include โ€” include nested (inside other structs) anonymous classes.
  • -B, --bit_holes=NR_HOLES โ€” show only structs at least NR_HOLES bit holes.
  • -d, --recursive โ€” recursive mode, affects several other flags.
  • -D, --decl_exclude=PREFIX โ€” exclude classes declared in files with PREFIX.
  • -f, --find_pointers_to=CLASS_NAME โ€” find pointers to CLASS_NAME.
  • -H, --holes=NR_HOLES โ€” show only structs with at least NR_HOLES holes.
  • --padding=SIZE_PADDING โ€” show only structs with SIZE_PADDING bytes of padding at its end.
  • --padding_ge=SIZE_PADDING โ€” show only structs with at least SIZE_PADDING bytes of padding at its end.
  • -I, --show_decl_info โ€” show the file and line number where the tags were defined, if available in the debugging information.
  • --skip_encoding_btf_vars โ€” by default, VARs are encoded only for percpu variables. when specified, this option prevents encoding any VARs. note that this option can be overridden by the feature "global_var".
  • --skip_encoding_btf_decl_tag โ€” do not encode decl tags in BTF.
  • --skip_encoding_btf_enum64 โ€” do not encode enum64 in BTF.
  • --skip_encoding_btf_type_tag โ€” do not encode type tags in BTF.
  • --skip_encoding_btf_inconsistent_proto โ€” do not encode functions with multiple inconsistent prototypes or unexpected register use for their parameters, where the registers used do not match calling conventions.
  • -j, --jobs=N โ€” run N jobs in parallel. defaults to number of online processors + 10% (like the 'ninja' build system) if no argument is specified.
  • -J, --btf_encode โ€” encode BTF information from DWARF, used in the Linux kernel build process when CONFIG_DEBUG_INFO_BTF=y is present, introduced in Linux v5.2. used to implement features such as BPF CO-RE (Compile Once - Run Everywhere). see https://nakryiko.com/posts/bpf-portability-and-co-re/.
  • --btf_encode_detached=FILENAME โ€” same thing as -J/--btf_encode, but storing the raw BTF info into a separate file.
  • --btf_encode_force โ€” ignore those symbols found invalid when encoding BTF.
  • --btf_base=PATH โ€” path to the base BTF file, for instance: vmlinux when encoding kernel module BTF information. this may be inferred when asking for a /sys/kernel/btf/MODULE, when it will be autoconfigured to "/sys/kernel/btf/vmlinux".
  • --btf_gen_floats โ€” allow producing BTF_KIND_FLOAT entries in systems where the vmlinux DWARF information has float types.
  • --btf_gen_optimized โ€” generate BTF for functions with optimization-related suffixes (.isra, .constprop).
  • --btf_gen_all โ€” allow using all the BTF features supported by pahole.
  • --features=FEATURE_LIST โ€” this is also available as --btf_features=FEATURE_LIST, unknown features, like those added in newer versions of pahole, are ignored. BTF encoding configuration is the major user. using 'default' will enable all standard features supported. this option can be used as an alternative to using multiple BTF-related options, and 'default' represents the standard set of BTF features that are in use for kernel BTF generation, so is useful as a shortcut for testing the latest set of standard features. however, kernel builds will call out specific features rather than using 'default' to ensure that the desired features are enabled regardless of pahole version and associated 'default' set. supported standard features are
  • encode_force โ€” ignore invalid symbols while encoding BTF. Like, if a symbol has a cursed name, it just gets yeeted and the BTF encoding keeps on going. no drama ๐Ÿซถ
  • var โ€” encode percpu variables using BTF_KIND_VAR in BTF.
  • float โ€” encode floating-point types in BTF.
  • decl_tag โ€” encode declaration tags using BTF_KIND_DECL_TAG.
  • type_tag โ€” encode type tags using BTF_KIND_TYPE_TAG.
  • enum64 โ€” encode enum64 values with BTF_KIND_ENUM64.
  • optimized_func โ€” encode representations of optimized functions with suffixes like .isra.0.
  • consistent_func โ€” avoid encoding inconsistent static functions. bestie these show up when a parameter gets optimized out in some CUs and not others, or when the same function name has inconsistent BTF descriptions across different CUs. messy ๐Ÿ’€

Supported non-standard features (not enabled for default)

  • attributes โ€” allow generation of decl_tags and type_tags with kind_flag = 1. These tags can encode arbitrary compiler attributes in BTF, but heads up: they are backward incompatible. ๐Ÿšจ
  • decl_tag_kfuncs โ€” inject a BTF_KIND_DECL_TAG for each discovered kfunc.
  • distilled_base โ€” for split BTF, generate a distilled version of the associated base BTF so that later relocation of split BTF with a possibly changed base works, storing it in a .BTF.base ELF section.
  • global_var โ€” encode all global variables using BTF_KIND_VAR in BTF.
  • true_signature โ€” encode functions ensuring that binary-level (rather than source-level) signatures are used; for gcc these are the .isra.0 and .costprop.0 optimized functions.
  • layout โ€” encode information about BTF kinds available at encoding time in the layout section in BTF.

Non-standard, non-BTF related features:

  • force_cu_merging โ€” force merging all CUs into one. Use this when there are references across CUs โ€” it happens in some LTO cases and was observed with Rust CUs, where type tags (function parameters, abstract origins for inlines, etc) reference types in another CU. Same-file imported units (DW_TAG_imported_unit) are auto-detected. Alternate debug files using DW_FORM_GNU_ref_alt are also handled automatically when the matching .dwz file is available, so honestly force_cu_merging is rarely needed with current pahole. โœจ

So like, for example, specifying --btf_encode=var,enum64 will give you a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values. slay.

And if you wanna add an extra feature to the standard set, you can use the + prefix, e.g.:

--btf_features=+reproducible_build

will add all default features plus the reproducible_build extra feature. so cute. ๐Ÿ’…


  • --btf_features_strict โ€” literally identical to --btf_features above, but pahole will exit if it hits an unrecognized feature. no mercy.
  • --supported_btf_features โ€” show the set of BTF features supported by the --btf_features option and exit. Super useful for checking which features are supported, since --btf_features will not emit an error if you specify an unrecognized feature. ๐Ÿ‘€
  • --running_kernel_vmlinux, โ€” search for a vmlinux matching the running kernel build-id (from /sys/kernel/notes), possibly getting it from a debuginfo server.
  • -l, --show_first_biggest_size_base_type_member โ€” show first biggest size base_type member.
  • -m, --nr_methods โ€” show number of methods of all classes, i.e., the number of functions have arguments that are pointers to a given class.

Wanna get the number of methods for one specific class? bestie, do this:

$ pahole --nr_methods | grep -w sock
sock  1005
$

In the above example it used the BTF information in /sys/kernel/btf/vmlinux. ๐Ÿซถ

  • -M, --show_only_data_members โ€” show only the members that use space in the class layout. C++ methods will be suppressed. bye methods โœŒ๏ธ
  • -n, --nr_members โ€” show number of members.
  • -N, --class_name_len โ€” show size of classes.
  • -O, --dwarf_offset=OFFSET โ€” show tag with DWARF OFFSET.
  • -P, --packable โ€” show only structs that has holes that can be packed if members are reorganized, for instance when using the --reorganize option.
  • --with_flexible_array โ€” show only structs that have a flexible array.
  • --with_embedded_flexible_array โ€” show only structs that have an embedded flexible array, i.e., a contained struct that has flexible array(s) "in the middle".
  • -q, --quiet โ€” be quieter. ๐Ÿคซ
  • -s, --sizes โ€” show size of classes.
  • -t, --separator=SEP โ€” use SEP as the field separator.
  • -T, --nr_definitions โ€” show how many times struct was defined.
  • -u, --defined_in โ€” show CUs where CLASS_NAME (-C) is defined.
  • --flat_arrays โ€” flatten arrays, so that array[10][2] becomes array[20]. Useful when generating from both CTF/BTF and DWARF encodings for the same binary for testing purposes.
  • --suppress_aligned_attribute โ€” suppress forced alignment markers, so that one can compare BTF or CTF output, that don't have that info, to output from DWARF >= 5.
  • --suppress_force_paddings โ€” suppress bitfield forced padding at the end of structs, as this requires something like DWARF's DW_AT_alignment, so that one can compare BTF or CTF output, that don't have that info.
  • --suppress_packed โ€” suppress the output of the inference of __attribute__((__packed__)), so that one can compare BTF or CTF output. The inference algorithm uses things like DW_AT_alignment, so until it is improved to infer that as well for BTF, allow disabling this output.
  • --fixup_silly_bitfields โ€” converts silly bitfields such as "int foo:32" to plain "int foo". iconic behaviour tbh ๐ŸŽ€
  • -V, --verbose โ€” be verbose.
  • --ptr_table_stats โ€” print statistics about ptr_table data structures, used to hold all the types, tags and functions data structures, for development tuning of such tables, tuned for a typical 2021 vmlinux file.
  • -w, --word_size=WORD_SIZE โ€” change the arch word size to WORD_SIZE.
  • -x, --exclude=PREFIX โ€” exclude PREFIXed classes.
  • -X, --cu_exclude=PREFIX โ€” exclude PREFIXed compilation units.
  • --lang=languages โ€” only process compilation units built from source code written in the specified languages.

Supported languages:

ada83, ada95, asm, bliss, c, c89, c99, c11, c++, c++03, c++11,
c++14, cobol74,
cobol85, d, dylan, fortran77, fortran90, fortran95, fortran03,
fortran08,
go, haskell, java, julia, modula2, modula3, objc, objc++, ocaml,
opencl,
pascal83, pli, python, renderscript, rust, swift, upc

The linux kernel, for instance, is written in 'c89' circa 2022, use that in filters. ๐ŸŒˆ

  • --lang_exclude=languages โ€” don't process compilation units built from source code written in the specified languages.

To filter out compilation units written in Rust, for instance, use:

pahole -j --btf_encode --lang_exclude rust

This can also be accomplished by setting the environment variable PAHOLE_LANG_EXCLUDE to the same list of languages, for instance, the following is equivalent to the above:

export PAHOLE_LANG_EXCLUDE=rust
pahole -j --btf_encode

Meaning in both cases 'rust' CUs will be skipped. ๐Ÿ’…

This is useful for scripts where it provides a way to ask for that exclusion for pahole and pfunct, no need to use --lang_exclude in all calls to those tools, just set that environment variable. so convenient, obsessed.

  • -y, --prefix_filter=PREFIX โ€” include PREFIXed classes.
  • -z, --hole_size_ge=HOLE_SIZE โ€” show only structs with at least one hole greater or equal to HOLE_SIZE.
  • --structs โ€” show only structs, all the other filters apply, i.e. to show just the sizes of all structs combine --structs with --sizes, etc.
  • --packed โ€” show only packed structs, all the other filters apply, i.e. to show just the sizes of all packed structs combine --packed with --sizes, etc.
  • --unions โ€” show only unions, all the other filters apply, i.e. to show just the sizes of all unions combine --union with --sizes, etc.
  • --version โ€” show a traditional string version, i.e.: "v1.18".
  • --numeric_version โ€” show a numeric only version, suitable for use in Makefiles and scripts where one wants to know what if the installed version has some feature, i.e.: 118 instead of "v1.18".
  • --kabi_prefix=STRING โ€” when the prefix of the string is STRING, treat the string as STRING.

โœจ NOTES โœจ

To enable the generation of debugging information in the Linux kernel build process select CONFIG_DEBUG_INFO. You can do this using make menuconfig by this path: "Kernel Hacking" -> "Compile-time checks and compiler options" -> "Compile the kernel with debug info". ๐Ÿ’–

Consider as well enabling CONFIG_DEBUG_INFO_BTF by going thru the aforementioned menuconfig path and then selecting "Generate BTF typeinfo". Most modern distributions with eBPF support should come with that in all its kernels, greatly facilitating the use of pahole. no cap, it's giving convenience. ๐Ÿ‘‘

Many distributions also come with debuginfo packages, so just enable it in your package manager repository configuration and install the kernel-debuginfo, or any other userspace program written in a language that the compiler generates debuginfo (C, C++, for instance). ๐Ÿฆ‹

โœจ EXAMPLES โœจ

ok so every single example down here is using either /sys/kernel/btf/vmlinux if it's chillin' there, or it goes and finds a vmlinux file that matches your running kernel โ€” using the build-id info in /sys/kernel/notes to make sure it's the right one. no mismatches bestie ๐Ÿ’…


Wanna show a type? say less:

$ pahole -C __u64
typedef long long unsigned int __u64;
$

and it works exactly the same if the only argument you pass is a type name, periodt:

$ pahole raw_spinlock_t
typedef struct raw_spinlock raw_spinlock_t;
$

You can pass multiple types too, just separate them with commas, iconic:

$ pahole raw_spinlock_t,raw_spinlock
struct raw_spinlock {
     arch_spinlock_t            raw_lock;             /*     0     4 */

     /* size: 4, cachelines: 1, members: 1 */
     /* last cacheline: 4 bytes */
};
typedef struct raw_spinlock raw_spinlock_t;
$

Types can be expanded, and honestly? the way it just unfolds the whole nested situation is giving main character energy ๐Ÿฆ‹:

$ pahole -E raw_spinlock
struct raw_spinlock {
        /* typedef arch_spinlock_t */ struct qspinlock {
                union {
                        /* typedef atomic_t */ struct {
                                int counter;                                                  /*     0     4 */
                        } val;                                                                /*     0     4 */
                        struct {
                                /* typedef u8 -> __u8 */ unsigned char locked;                /*     0     1 */
                                /* typedef u8 -> __u8 */ unsigned char pending;               /*     1     1 */
                        };                                                                    /*     0     2 */
                        struct {
                                /* typedef u16 -> __u16 */ short unsigned int locked_pending; /*     0     2 */
                                /* typedef u16 -> __u16 */ short unsigned int tail;           /*     2     2 */
                        };                                                                    /*     0     4 */
                };                                                                            /*     0     4 */
        } raw_lock;                                                                           /*     0     4 */

        /* size: 4, cachelines: 1, members: 1 */
        /* last cacheline: 4 bytes */
};
$

When you're decoding OOPSes you're probably gonna want the offsets and sizes in hexadecimal, fr fr:

$ pahole --hex thread_struct
struct thread_struct {
        struct desc_struct         tls_array[3];         /*     0  0x18 */
        long unsigned int          sp;                   /*  0x18   0x8 */
        short unsigned int         es;                   /*  0x20   0x2 */
        short unsigned int         ds;                   /*  0x22   0x2 */
        short unsigned int         fsindex;              /*  0x24   0x2 */
        short unsigned int         gsindex;              /*  0x26   0x2 */
        long unsigned int          fsbase;               /*  0x28   0x8 */
        long unsigned int          gsbase;               /*  0x30   0x8 */
        struct perf_event *        ptrace_bps[4];        /*  0x38  0x20 */
        /* --- cacheline 1 boundary (64 bytes) was 24 bytes ago --- */
        long unsigned int          debugreg6;            /*  0x58   0x8 */
        long unsigned int          ptrace_dr7;           /*  0x60   0x8 */
        long unsigned int          cr2;                  /*  0x68   0x8 */
        long unsigned int          trap_nr;              /*  0x70   0x8 */
        long unsigned int          error_code;           /*  0x78   0x8 */
        /* --- cacheline 2 boundary (128 bytes) --- */
        struct io_bitmap *         io_bitmap;            /*  0x80   0x8 */
        long unsigned int          iopl_emul;            /*  0x88   0x8 */
        mm_segment_t               addr_limit;           /*  0x90   0x8 */
        unsigned int               sig_on_uaccess_err:1; /*  0x98: 0 0x4 */
        unsigned int               uaccess_err:1;        /*  0x98:0x1 0x4 */

        /* XXX 30 bits hole, try to pack */
        /* XXX 36 bytes hole, try to pack */

        /* --- cacheline 3 boundary (192 bytes) --- */
        struct fpu                 fpu;                  /*  0xc0 0x1040 */

        /* size: 4352, cachelines: 68, members: 20 */
        /* sum members: 4312, holes: 1, sum holes: 36 */
        /* sum bitfield members: 2 bits, bit holes: 1, sum bit holes: 30 bits */
};
$

ok so at this point I know the offset causing it all is a struct thread_struct and the offset is 0x178, so it has to be inside that fpu structโ€ฆ no stress tho, just expand struct thread_struct and pipe it through grep like a legend ๐Ÿ˜Œ:

$ pahole --hex -E thread_struct | egrep '(0x178|struct fpu)' -B4 -A4
        /* XXX 30 bits hole, try to pack */
        /* XXX 36 bytes hole, try to pack */

        /* --- cacheline 3 boundary (192 bytes) --- */
        struct fpu {
                unsigned int       last_cpu;                                             /*  0xc0   0x4 */

                /* XXX 4 bytes hole, try to pack */

--
                                /* typedef u8 -> __u8 */ unsigned char alimit;           /* 0x171   0x1 */

                                /* XXX 6 bytes hole, try to pack */

                                struct math_emu_info * info;                             /* 0x178   0x8 */
                                /* --- cacheline 6 boundary (384 bytes) --- */
                                /* typedef u32 -> __u32 */ unsigned int entry_eip;       /* 0x180   0x4 */
                        } soft; /* 0x100  0x88 */
                        struct xregs_state {
$

Want to know where struct thread_struct is actually defined in the kernel sources? bestie I got you:

$ pahole -I thread_struct | head -2
/* Used at: /sys/kernel/btf/vmlinux */
/* <0> (null):0 */
$

It's not present in BTF, so switch to DWARF โ€” takes a little bit longer, ngl โ€” and assuming it finds the matching vmlinux file:

$ pahole -Fdwarf -I thread_struct | head -2
/* Used at: /home/acme/git/linux/arch/x86/kernel/head64.c */
/* <3333> /home/acme/git/linux/arch/x86/include/asm/processor.h:485 */
$

Wanna find the biggest data structures in the Linux kernel? say less, sort by column two numerically, reversed, top 5:

$ pahole -s | sort -k2 -nr | head -5
cmp_data               290904 1
dec_datas              274520 1
cpu_entry_area         217088 0
pglist_data            172928 4
saved_cmdlines_buffer  131104 1
$
The second column is the size in bytes, and the third column is the number of alignment holes in that structure. ๐Ÿ‘‘

Show data structures that have a raw spinlock AND are related to the RCU mechanism:

$ pahole --contains raw_spinlock_t --prefix rcu
rcu_node
rcu_data
rcu_state
$

and to actually see that in context, combine it with grep:

$ pahole rcu_state | grep raw_spinlock_t -B1 -A5
     /* --- cacheline 52 boundary (3328 bytes) --- */
     raw_spinlock_t             ofl_lock;             /*  3328     4 */

     /* size: 3392, cachelines: 53, members: 35 */
     /* sum members: 3250, holes: 7, sum holes: 82 */
     /* padding: 60 */
};
$

๐ŸŽ€ PRETTY PRINTING ๐ŸŽ€

ok so pahole can also use the data structure types to pretty print raw data โ€” you do that with --prettify. And if you wanna consume raw data from standard input, you just use --prettify - ๐Ÿ’–

It can pretty print raw data from stdin according to whatever type you specify, like this:

$ pahole -C modversion_info drivers/scsi/sg.ko
struct modversion_info {
      long unsigned int          crc;                  /*     0     8 */
      char                       name[56];             /*     8    56 */

      /* size: 64, cachelines: 1, members: 2 */
};
$
$ objcopy -O binary --only-section=__versions drivers/scsi/sg.ko versions
$
$ ls -la versions
-rw-rw-r--. 1 acme acme 7616 Jun 25 11:33 versions
$
$ pahole --count 3 -C modversion_info drivers/scsi/sg.ko --prettify versions
{
      .crc = 0x8dabd84,
      .name = "module_layout",
},
{
      .crc = 0x45e4617b,
      .name = "no_llseek",
},
{
      .crc = 0xa23fae8c,
      .name = "param_ops_int",
},
$
$ pahole --skip 1 --count 2 -C modversion_info drivers/scsi/sg.ko --prettify - < versions
{
      .crc = 0x45e4617b,
      .name = "no_llseek",
},
{
      .crc = 0xa23fae8c,
      .name = "param_ops_int",
},
$

and that last one is equivalent to:

$ pahole --seek_bytes 64 --count 1 -C modversion_info drivers/scsi/sg.ko --prettify versions
{
     .crc = 0x45e4617b,
     .name = "no_llseek",
},
$

  • -C, --class_name=CLASS_NAME โ€” pretty print according to this class. And you can pass arguments to it to affect how the pretty printing gets done, like:

-C 'perf_event_header(sizeof,type,type_enum=perf_event_type,filter=type==PERF_RECORD_EXIT)'

so this would pick the struct perf_event_header as the type to use for pretty printing records, and it says that the size field in that struct should be used to figure out the size of the record (these are variable sized records, cute), and that the enum perf_event_type should be used to pretty print the numeric value in perf_event_header->type, and on top of all that it should be used to heuristically look for structs with the same name (lowercase) as the enum entry that gets converted from the type field, and use that to pretty print instead of the base perf_event_header type. peek the PRETTY PRINTING EXAMPLES section below for the whole vibe โœจ

Also the filter= part can be used โ€” so far with only the == operator โ€” to filter based on the type field and convert the string 'PERF_RECORD_EXIT' to a number according to type_enum. no cap.

The sizeof arg defaults to the size member name, and if the name is different you can use the sizeof=sz form. ditto for the type=other_member_name field, which defaults to type. ๐Ÿ’…

PRETTY PRINTING EXAMPLES โœจ

Okay bestie, let's peek at the ELF header of a vmlinux file using BTF. First you gotta discover the ELF header type:

$ pahole --sizes | grep -i elf | grep -i _h
elf64_hdr 64   0
elf32_hdr 52   0
$

Now we can flex and show the first record from offset zero:

$ pahole -C elf64_hdr --count 1 --prettify /lib/modules/5.8.0-rc3+/build/vmlinux
{
     .e_ident = { 127, 69, 76, 70, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
     .e_type = 2,
     .e_machine = 62,
     .e_version = 1,
     .e_entry = 16777216,
     .e_phoff = 64,
     .e_shoff = 775923840,
     .e_flags = 0,
     .e_ehsize = 64,
     .e_phentsize = 56,
     .e_phnum = 5,
     .e_shentsize = 64,
     .e_shnum = 80,
     .e_shstrndx = 79,
},
$

This is literally the same as:

$ pahole --header elf64_hdr --prettify /lib/modules/5.8.0-rc3+/build/vmlinux

The --header option is that girl: it also lets other command line options reference fields in the header. Super useful when you wanna show multiple records in a file and the range where those fields live is specified in header fields โ€” like for perf.data files:

$ pahole --hex ~/bin/perf --header perf_file_header --prettify perf.data
{
     .magic = 0x32454c4946524550,
     .size = 0x68,
     .attr_size = 0x88,
     .attrs = {
          .offset = 0xa8,
          .size = 0x88,
     },
     .data = {
          .offset = 0x130,
          .size = 0x588,
     },
     .event_types = {
          .offset = 0,
          .size = 0,
     },
     .adds_features = { 0x16717ffc, 0, 0, 0 },
},
$

So to display the cgroups records in the perf_file_header.data section, we can use:

$ pahole ~/bin/perf --header=perf_file_header --seek_bytes '$header.data.offset' --size_bytes='$header.data.size' -C 'perf_event_header(sizeof,type,type_enum=perf_event_type,filter=type==PERF_RECORD_CGROUP)' --prettify perf.data
{
     .header = {
          .type = PERF_RECORD_CGROUP,
          .misc = 0,
          .size = 40,
     },
     .id = 1,
     .path = "/",
},
{
     .header = {
          .type = PERF_RECORD_CGROUP,
          .misc = 0,
          .size = 48,
     },
     .id = 1553,
     .path = "/system.slice",
},
{
     .header = {
          .type = PERF_RECORD_CGROUP,
          .misc = 0,
          .size = 48,
     },
     .id = 8,
     .path = "/machine.slice",
},
{
     .header = {
          .type = PERF_RECORD_CGROUP,
          .misc = 0,
          .size = 128,
     },
     .id = 7828,
     .path = "/machine.slice/libpod-42be8e8d4eb9d22405845005f0d04ea398548dccc934a150fbaa3c1f1f9492c2.scope",
},
{
     .header = {
          .type = PERF_RECORD_CGROUP,
          .misc = 0,
          .size = 88,
     },
     .id = 13,
     .path = "/machine.slice/machine-qemu\x2d1\x2drhel6.sandy.scope",
},
$

For the common case where the header has a member with 'offset' and 'size' members, you can use this more compact form:

$ pahole ~/bin/perf --header=perf_file_header --range=data -C 'perf_event_header(sizeof,type,type_enum=perf_event_type,filter=type==PERF_RECORD_CGROUP)' --prettify perf.data

This uses ~/bin/perf to get the type definitions, defines 'struct perf_file_header' as the header, then seeks '$header.data.offset' bytes from the start of the file, and considers '$header.data.size' bytes worth of such records. The filter expression can omit a common prefix, so in this case it could also be written as 'filter=type==CGROUP', or the 'filter=' can be omitted entirely, getting as compact as 'type==CGROUP':

If we look at this:

$ pahole ~/bin/perf -C perf_event_header
struct perf_event_header {
     __u32                      type;                 /*     0     4 */
     __u16                      misc;                 /*     4     2 */
     __u16                      size;                 /*     6     2 */

     /* size: 8, cachelines: 1, members: 3 */
     /* last cacheline: 8 bytes */
};
$

And also:

$ pahole ~/bin/perf -C perf_event_type
enum perf_event_type {
     PERF_RECORD_MMAP = 1,
     PERF_RECORD_LOST = 2,
     PERF_RECORD_COMM = 3,
     PERF_RECORD_EXIT = 4,
     PERF_RECORD_THROTTLE = 5,
     PERF_RECORD_UNTHROTTLE = 6,
     PERF_RECORD_FORK = 7,
     PERF_RECORD_READ = 8,
     PERF_RECORD_SAMPLE = 9,
     PERF_RECORD_MMAP2 = 10,
     PERF_RECORD_AUX = 11,
     PERF_RECORD_ITRACE_START = 12,
     PERF_RECORD_LOST_SAMPLES = 13,
     PERF_RECORD_SWITCH = 14,
     PERF_RECORD_SWITCH_CPU_WIDE = 15,
     PERF_RECORD_NAMESPACES = 16,
     PERF_RECORD_KSYMBOL = 17,
     PERF_RECORD_BPF_EVENT = 18,
     PERF_RECORD_CGROUP = 19,
     PERF_RECORD_TEXT_POKE = 20,
     PERF_RECORD_MAX = 21,
};
$

And furthermore, check it:

$ pahole ~/bin/perf -C perf_record_cgroup
struct perf_record_cgroup {
     struct perf_event_header   header;               /*     0     8 */
     __u64                      id;                   /*     8     8 */
     char                       path[4096];           /*    16  4096 */

     /* size: 4112, cachelines: 65, members: 3 */
     /* last cacheline: 16 bytes */
};
$

Then we can see how perf_event_header.type could be converted from a __u32 to a string (PERF_RECORD_CGROUP). If we remove that type_enum=perf_event_type, we lose the conversion of 'struct perf_event_header' to the more descriptive 'struct perf_record_cgroup', and also the beautification of the header.type field:

$ pahole ~/bin/perf --header=perf_file_header --seek_bytes '$header.data.offset' --size_bytes='$header.data.size' -C 'perf_event_header(sizeof,type,filter=type==19)' --prettify perf.data
{
     .type = 19,
     .misc = 0,
     .size = 40,
},
{
     .type = 19,
     .misc = 0,
     .size = 48,
},
{
     .type = 19,
     .misc = 0,
     .size = 48,
},
{
     .type = 19,
     .misc = 0,
     .size = 128,
},
{
     .type = 19,
     .misc = 0,
     .size = 88,
},
$

Some of the records aren't found in 'type_enum=perf_event_type', so some records don't get converted to a type that fully shows its contents. For perf we know those are in another enumeration, 'enum perf_user_event_type', so for these cases we can create a 'virtual enum' โ€” the sum of two enums โ€” and then get all those entries decoded and properly casted. First, a few records with just 'enum perf_event_type':

$ pahole ~/bin/perf --header=perf_file_header --seek_bytes '$header.data.offset' --size_bytes='$header.data.size' -C 'perf_event_header(sizeof,type,type_enum=perf_event_type)' --count 4 --prettify perf.data
{
     .type = 79,
     .misc = 0,
     .size = 32,
},
{
     .type = 73,
     .misc = 0,
     .size = 40,
},
{
     .type = 74,
     .misc = 0,
     .size = 32,
},
{
     .header = {
          .type = PERF_RECORD_CGROUP,
          .misc = 0,
          .size = 40,
     },
     .id = 1,
     .path = "/",
},
$

Now with both enumerations, i.e. with 'type_enum=perf_event_type+perf_user_event_type':

$ pahole ~/bin/perf --header=perf_file_header --seek_bytes '$header.data.offset' --size_bytes='$header.data.size' -C 'perf_event_header(sizeof,type,type_enum=perf_event_type+perf_user_event_type)' --count 5 --prettify perf.data
{
     .header = {
          .type = PERF_RECORD_TIME_CONV,
          .misc = 0,
          .size = 32,
     },
     .time_shift = 31,
     .time_mult = 1016803377,
     .time_zero = 435759009518382,
},
{
     .header = {
          .type = PERF_RECORD_THREAD_MAP,
          .misc = 0,
          .size = 40,
     },
     .nr = 1,
     .entries = 0x50 0x7e 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00,
},
{
     .header = {
          .type = PERF_RECORD_CPU_MAP,
          .misc = 0,
          .size = 32,
     },
     .data = {
          .type = 1,
          .data = "",
     },
},
{
     .header = {
          .type = PERF_RECORD_CGROUP,
          .misc = 0,
          .size = 40,
     },
     .id = 1,
     .path = "/",
},
{
     .header = {
          .type = PERF_RECORD_CGROUP,
          .misc = 0,
          .size = 48,
     },
     .id = 1553,
     .path = "/system.slice",
},
$

You can also pass multiple types, but bestie, make sure they appear in the file in sequence. For the perf.data example, see the perf_file_header dump above โ€” you can print the perf_file_attr structs in the header attrs range, then the perf_event_header in the data range with this command:

pahole ~/bin/perf --header=perf_file_header          -C 'perf_file_attr(range=attrs),perf_event_header(range=data,sizeof,type,type_enum=perf_event_type+perf_user_event_type)' --prettify perf.data

SEE ALSO ๐Ÿ”—

Also check out these other icons:

And this paper, periodt:

https://www.kernel.org/doc/ols/2007/ols2007v2-pages-35-44.pdf.

AUTHOR ๐Ÿ‘‘

pahole was written and is maintained by Arnaldo Carvalho de Melo <acme@kernel.org>.

Thanks to Andrii Nakryiko and Martin KaFai Lau for providing the BTF encoder and improving the codebase while making sure the BTF encoder works as needed to be used in encoding the Linux kernel .BTF section from the DWARF info generated by gcc. For that Andrii wrote a BTF deduplicator in libbpf that is used by pahole.

Also thanks to Conectiva, Mandriva and Red Hat for allowing me to work on these tools.

Please send bug reports to <dwarves@vger.kernel.org>.

No subscription is required.


And that's a wrap, bestie! This iconic page came from the dwarves crew, dated January 16, 2020, and it's for pahole(1). Periodt. ๐Ÿ’…โœจ

๐Ÿ“ฆ where this page comes from