pahole(1)
โจ NAME โจ
pahole โ show or manipulate data structure layout and pretty print raw data
and yes bestie, this is the pahole(1) page, from the dwarves collection ๐ pull up a chair.
๐ฆ SYNOPSIS ๐ฆ
pahole [options] files
๐ธ DESCRIPTION ๐ธ
okay so pahole shows data structure layouts that are encoded in debugging information formats. the ones it fucks with are DWARF, CTF and BTF. that's the roster, periodt. ๐
this is useful for, among other things: optimizing important data structures by reducing their size, identifying the field sitting at an offset from the start of a data structure, investigating ABI changes and more generally understanding a new codebase you have to work with. it's giving "who is this struct and why is it like this" detective energy ๐
it also uses these structure layouts to pretty print data fed to its standard input, e.g.:
$ pahole --header elf64_hdr --prettify /lib/modules/5.8.0-rc6+/build/vmlinux
{
.e_ident = { 127, 69, 76, 70, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
.e_type = 2,
.e_machine = 62,
.e_version = 1,
.e_entry = 16777216,
.e_phoff = 64,
.e_shoff = 604653784,
.e_flags = 0,
.e_ehsize = 64,
.e_phentsize = 56,
.e_phnum = 5,
.e_shentsize = 64,
.e_shnum = 80,
.e_shstrndx = 79,
},
$
see the PRETTY PRINTING section for further examples and documentation ๐ซถ
the files must have associated debugging information. this information may be inside the file itself, in ELF sections, or in another file.
one way to have this information is to specify the -g option to the compiler when building it. when this is done, the information will be stored in an ELF section. for the DWARF debugging information format this adds, among others, the .debug_info ELF section. for CTF it is found in just one ELF section, .SUNW_ctf. BTF comes with at least the .BTF ELF section, and may come also with the .BTF.ext ELF section. โจ
the debuginfo packages available in most Linux distributions are also supported by pahole, where the debugging information is available in a separate file.
by default, pahole shows the layout of all named structs in the files specified. all of them. every single one. ๐
if no files are specified, then it will look if /sys/kernel/btf/vmlinux is present, using the BTF information contained in it about the running kernel, i.e., this works:
$ pahole list_head
struct list_head {
struct list_head * next; /* 0 8 */
struct list_head * prev; /* 8 8 */
/* size: 16, cachelines: 1, members: 2 */
/* last cacheline: 16 bytes */
};
$
it is possible to override the /sys/kernel/btf/vmlinux file location by setting the PAHOLE_VMLINUX_BTF_FILENAME environment variable. this may be useful for testing or scripting when using a different BTF for vmlinux. used in the pahole regression tests. ๐งช
if BTF is not present and no file is passed, then a vmlinux that matches the build-id for the running kernel will be looked up in the usual places, including where the kernel debuginfo packages put it, looking for DWARF info instead. it will simply find it, no stress.
see the EXAMPLES section for more usage suggestions.
it also pretty prints whatever is fed to its standard input, according to the type specified, see the EXAMPLES section.
use --count to state how many records should be pretty printed.
๐ OPTIONS ๐
-C,--class_name=CLASS_NAMESโ show just these classes. this can be a comma separated list of class names or file URLs (e.g.:file://class_list.txt)-c,--cacheline_size=SIZEโ set cacheline size to SIZE bytes.--sortโ sort the output by type name, maybe this will grow to allow sorting by other criteria. this is mostly needed so that pretty printing from BTF and DWARF can be comparable when using using multiple threads to load DWARF data, when the order that the types in the compile units is processed is not deterministic.--compileโ generate compileable code, with all definitions for all types, i.e.:
$ pahole --compile > vmlinux.h
produces a header that can be included in a C source file and built. in the example provided it will use the BTF info if available, otherwise will look for a DWARF file matching the running kernel build-id.
--skip_emitting_atomic_typedefsโ do not emit'typedef _Atomic int atomic_int'& friends when used with options like--compile. use it if the compiler provides these already, as of circa 2022 with gcc 12.2.1 those are not encoded in DWARF so to generate compilable code we need emit those typedefs for the atomic types used in the data structures being emitted from debugging information.--count=COUNTโ pretty print the first COUNT records from input.--skip=COUNTโ skip COUNT input records.-E,--expand_typesโ expand class members. useful to find in what member of inner structs where an offset from the beginning of a struct is.-F,--format_pathโ allows specifying a list of debugging formats to try, in order. right now this includes "btf", "ctf" and "dwarf". the default format path used is equivalent to"-F dwarf,btf,ctf".--hashbits=BITSโ allows specifying the number of bits for the debugging format loader to use. the only one affected so far is the "dwarf" one, its default now is 15, the maximum for it is now 21 bits. tweak it to see if it improves performance as the kernel evolves and more types and functions have to be loaded.--hexโ print offsets and sizes in hexadecimal.-r,--rel_offsetโ show relative offsets of members in inner structs.-p,--expand_pointersโ expand class pointer members.-R,--reorganizeโ reorganize struct, demoting and combining bitfields, moving members to remove alignment holes and padding.-S,--show_reorg_stepsโ show the struct layout at each reorganization step.-i,--contains=CLASS_NAMEโ show classes that contains CLASS_NAME.--contains_enumerator=ENUMERATOR_NAMEโ show enumeration that contains ENUMERATOR_NAME.
$ pahole --contains_enumerator S_VERSION
enum file_time_flags {
S_ATIME = 1,
S_MTIME = 2,
S_CTIME = 4,
S_VERSION = 8,
}
$
-a,--anon_includeโ include anonymous classes.-A,--nested_anon_includeโ include nested (inside other structs) anonymous classes.-B,--bit_holes=NR_HOLESโ show only structs at least NR_HOLES bit holes.-d,--recursiveโ recursive mode, affects several other flags.-D,--decl_exclude=PREFIXโ exclude classes declared in files with PREFIX.-f,--find_pointers_to=CLASS_NAMEโ find pointers to CLASS_NAME.-H,--holes=NR_HOLESโ show only structs with at least NR_HOLES holes.--padding=SIZE_PADDINGโ show only structs with SIZE_PADDING bytes of padding at its end.--padding_ge=SIZE_PADDINGโ show only structs with at least SIZE_PADDING bytes of padding at its end.-I,--show_decl_infoโ show the file and line number where the tags were defined, if available in the debugging information.--skip_encoding_btf_varsโ by default, VARs are encoded only for percpu variables. when specified, this option prevents encoding any VARs. note that this option can be overridden by the feature "global_var".--skip_encoding_btf_decl_tagโ do not encode decl tags in BTF.--skip_encoding_btf_enum64โ do not encode enum64 in BTF.--skip_encoding_btf_type_tagโ do not encode type tags in BTF.--skip_encoding_btf_inconsistent_protoโ do not encode functions with multiple inconsistent prototypes or unexpected register use for their parameters, where the registers used do not match calling conventions.-j,--jobs=Nโ run N jobs in parallel. defaults to number of online processors + 10% (like the 'ninja' build system) if no argument is specified.-J,--btf_encodeโ encode BTF information from DWARF, used in the Linux kernel build process whenCONFIG_DEBUG_INFO_BTF=yis present, introduced in Linux v5.2. used to implement features such as BPF CO-RE (Compile Once - Run Everywhere). seehttps://nakryiko.com/posts/bpf-portability-and-co-re/.--btf_encode_detached=FILENAMEโ same thing as-J/--btf_encode, but storing the raw BTF info into a separate file.--btf_encode_forceโ ignore those symbols found invalid when encoding BTF.--btf_base=PATHโ path to the base BTF file, for instance:vmlinuxwhen encoding kernel module BTF information. this may be inferred when asking for a/sys/kernel/btf/MODULE, when it will be autoconfigured to"/sys/kernel/btf/vmlinux".--btf_gen_floatsโ allow producingBTF_KIND_FLOATentries in systems where the vmlinux DWARF information has float types.--btf_gen_optimizedโ generate BTF for functions with optimization-related suffixes (.isra,.constprop).--btf_gen_allโ allow using all the BTF features supported bypahole.--features=FEATURE_LISTโ this is also available as--btf_features=FEATURE_LIST, unknown features, like those added in newer versions ofpahole, are ignored. BTF encoding configuration is the major user. using 'default' will enable all standard features supported. this option can be used as an alternative to using multiple BTF-related options, and 'default' represents the standard set of BTF features that are in use for kernel BTF generation, so is useful as a shortcut for testing the latest set of standard features. however, kernel builds will call out specific features rather than using 'default' to ensure that the desired features are enabled regardless ofpaholeversion and associated 'default' set. supported standard features areencode_forceโ ignore invalid symbols while encoding BTF. Like, if a symbol has a cursed name, it just gets yeeted and the BTF encoding keeps on going. no drama ๐ซถvarโ encode percpu variables usingBTF_KIND_VARin BTF.floatโ encode floating-point types in BTF.decl_tagโ encode declaration tags usingBTF_KIND_DECL_TAG.type_tagโ encode type tags usingBTF_KIND_TYPE_TAG.enum64โ encode enum64 values withBTF_KIND_ENUM64.optimized_funcโ encode representations of optimized functions with suffixes like.isra.0.consistent_funcโ avoid encoding inconsistent static functions. bestie these show up when a parameter gets optimized out in some CUs and not others, or when the same function name has inconsistent BTF descriptions across different CUs. messy ๐
Supported non-standard features (not enabled for default)
attributesโ allow generation of decl_tags and type_tags withkind_flag = 1. These tags can encode arbitrary compiler attributes in BTF, but heads up: they are backward incompatible. ๐จdecl_tag_kfuncsโ inject aBTF_KIND_DECL_TAGfor each discovered kfunc.distilled_baseโ for split BTF, generate a distilled version of the associated base BTF so that later relocation of split BTF with a possibly changed base works, storing it in a.BTF.baseELF section.global_varโ encode all global variables usingBTF_KIND_VARin BTF.true_signatureโ encode functions ensuring that binary-level (rather than source-level) signatures are used; for gcc these are the.isra.0and.costprop.0optimized functions.layoutโ encode information about BTF kinds available at encoding time in the layout section in BTF.
Non-standard, non-BTF related features:
force_cu_mergingโ force merging all CUs into one. Use this when there are references across CUs โ it happens in some LTO cases and was observed with Rust CUs, where type tags (function parameters, abstract origins for inlines, etc) reference types in another CU. Same-file imported units (DW_TAG_imported_unit) are auto-detected. Alternate debug files usingDW_FORM_GNU_ref_altare also handled automatically when the matching.dwzfile is available, so honestlyforce_cu_mergingis rarely needed with current pahole. โจ
So like, for example, specifying --btf_encode=var,enum64 will give you a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values. slay.
And if you wanna add an extra feature to the standard set, you can use the + prefix, e.g.:
--btf_features=+reproducible_build
will add all default features plus the reproducible_build extra feature. so cute. ๐
--btf_features_strictโ literally identical to--btf_featuresabove, but pahole will exit if it hits an unrecognized feature. no mercy.--supported_btf_featuresโ show the set of BTF features supported by the--btf_featuresoption and exit. Super useful for checking which features are supported, since--btf_featureswill not emit an error if you specify an unrecognized feature. ๐--running_kernel_vmlinux,โ search for a vmlinux matching the running kernel build-id (from/sys/kernel/notes), possibly getting it from a debuginfo server.-l,--show_first_biggest_size_base_type_memberโ show first biggest size base_type member.-m,--nr_methodsโ show number of methods of all classes, i.e., the number of functions have arguments that are pointers to a given class.
Wanna get the number of methods for one specific class? bestie, do this:
$ pahole --nr_methods | grep -w sock
sock 1005
$
In the above example it used the BTF information in /sys/kernel/btf/vmlinux. ๐ซถ
-M,--show_only_data_membersโ show only the members that use space in the class layout. C++ methods will be suppressed. bye methods โ๏ธ-n,--nr_membersโ show number of members.-N,--class_name_lenโ show size of classes.-O,--dwarf_offset=OFFSETโ show tag with DWARFOFFSET.-P,--packableโ show only structs that has holes that can be packed if members are reorganized, for instance when using the--reorganizeoption.--with_flexible_arrayโ show only structs that have a flexible array.--with_embedded_flexible_arrayโ show only structs that have an embedded flexible array, i.e., a contained struct that has flexible array(s) "in the middle".-q,--quietโ be quieter. ๐คซ-s,--sizesโ show size of classes.-t,--separator=SEPโ useSEPas the field separator.-T,--nr_definitionsโ show how many times struct was defined.-u,--defined_inโ show CUs whereCLASS_NAME(-C) is defined.--flat_arraysโ flatten arrays, so thatarray[10][2]becomesarray[20]. Useful when generating from both CTF/BTF and DWARF encodings for the same binary for testing purposes.--suppress_aligned_attributeโ suppress forced alignment markers, so that one can compare BTF or CTF output, that don't have that info, to output from DWARF >= 5.--suppress_force_paddingsโ suppress bitfield forced padding at the end of structs, as this requires something like DWARF'sDW_AT_alignment, so that one can compare BTF or CTF output, that don't have that info.--suppress_packedโ suppress the output of the inference of__attribute__((__packed__)), so that one can compare BTF or CTF output. The inference algorithm uses things likeDW_AT_alignment, so until it is improved to infer that as well for BTF, allow disabling this output.--fixup_silly_bitfieldsโ converts silly bitfields such as"int foo:32"to plain"int foo". iconic behaviour tbh ๐-V,--verboseโ be verbose.--ptr_table_statsโ print statistics about ptr_table data structures, used to hold all the types, tags and functions data structures, for development tuning of such tables, tuned for a typical 2021 vmlinux file.-w,--word_size=WORD_SIZEโ change the arch word size toWORD_SIZE.-x,--exclude=PREFIXโ excludePREFIXed classes.-X,--cu_exclude=PREFIXโ excludePREFIXed compilation units.--lang=languagesโ only process compilation units built from source code written in the specified languages.
Supported languages:
ada83, ada95, asm, bliss, c, c89, c99, c11, c++, c++03, c++11,
c++14, cobol74,
cobol85, d, dylan, fortran77, fortran90, fortran95, fortran03,
fortran08,
go, haskell, java, julia, modula2, modula3, objc, objc++, ocaml,
opencl,
pascal83, pli, python, renderscript, rust, swift, upc
The linux kernel, for instance, is written in 'c89' circa 2022, use that in filters. ๐
--lang_exclude=languagesโ don't process compilation units built from source code written in the specified languages.
To filter out compilation units written in Rust, for instance, use:
pahole -j --btf_encode --lang_exclude rust
This can also be accomplished by setting the environment variable PAHOLE_LANG_EXCLUDE to the same list of languages, for instance, the following is equivalent to the above:
export PAHOLE_LANG_EXCLUDE=rust
pahole -j --btf_encode
Meaning in both cases 'rust' CUs will be skipped. ๐
This is useful for scripts where it provides a way to ask for that exclusion for pahole and pfunct, no need to use --lang_exclude in all calls to those tools, just set that environment variable. so convenient, obsessed.
-y,--prefix_filter=PREFIXโ includePREFIXed classes.-z,--hole_size_ge=HOLE_SIZEโ show only structs with at least one hole greater or equal toHOLE_SIZE.--structsโ show only structs, all the other filters apply, i.e. to show just the sizes of all structs combine--structswith--sizes, etc.--packedโ show only packed structs, all the other filters apply, i.e. to show just the sizes of all packed structs combine--packedwith--sizes, etc.--unionsโ show only unions, all the other filters apply, i.e. to show just the sizes of all unions combine--unionwith--sizes, etc.--versionโ show a traditional string version, i.e.:"v1.18".--numeric_versionโ show a numeric only version, suitable for use in Makefiles and scripts where one wants to know what if the installed version has some feature, i.e.:118instead of"v1.18".--kabi_prefix=STRINGโ when the prefix of the string isSTRING, treat the string asSTRING.
โจ NOTES โจ
To enable the generation of debugging information in the Linux kernel build process select CONFIG_DEBUG_INFO. You can do this using make menuconfig by this path: "Kernel Hacking" -> "Compile-time checks and compiler options" -> "Compile the kernel with debug info". ๐
Consider as well enabling CONFIG_DEBUG_INFO_BTF by going thru the aforementioned menuconfig path and then selecting "Generate BTF typeinfo". Most modern distributions with eBPF support should come with that in all its kernels, greatly facilitating the use of pahole. no cap, it's giving convenience. ๐
Many distributions also come with debuginfo packages, so just enable it in your package manager repository configuration and install the kernel-debuginfo, or any other userspace program written in a language that the compiler generates debuginfo (C, C++, for instance). ๐ฆ
โจ EXAMPLES โจ
ok so every single example down here is using either /sys/kernel/btf/vmlinux if it's chillin' there, or it goes and finds a vmlinux file that matches your running kernel โ using the build-id info in /sys/kernel/notes to make sure it's the right one. no mismatches bestie ๐
Wanna show a type? say less:
$ pahole -C __u64
typedef long long unsigned int __u64;
$
and it works exactly the same if the only argument you pass is a type name, periodt:
$ pahole raw_spinlock_t
typedef struct raw_spinlock raw_spinlock_t;
$
You can pass multiple types too, just separate them with commas, iconic:
$ pahole raw_spinlock_t,raw_spinlock
struct raw_spinlock {
arch_spinlock_t raw_lock; /* 0 4 */
/* size: 4, cachelines: 1, members: 1 */
/* last cacheline: 4 bytes */
};
typedef struct raw_spinlock raw_spinlock_t;
$
Types can be expanded, and honestly? the way it just unfolds the whole nested situation is giving main character energy ๐ฆ:
$ pahole -E raw_spinlock
struct raw_spinlock {
/* typedef arch_spinlock_t */ struct qspinlock {
union {
/* typedef atomic_t */ struct {
int counter; /* 0 4 */
} val; /* 0 4 */
struct {
/* typedef u8 -> __u8 */ unsigned char locked; /* 0 1 */
/* typedef u8 -> __u8 */ unsigned char pending; /* 1 1 */
}; /* 0 2 */
struct {
/* typedef u16 -> __u16 */ short unsigned int locked_pending; /* 0 2 */
/* typedef u16 -> __u16 */ short unsigned int tail; /* 2 2 */
}; /* 0 4 */
}; /* 0 4 */
} raw_lock; /* 0 4 */
/* size: 4, cachelines: 1, members: 1 */
/* last cacheline: 4 bytes */
};
$
When you're decoding OOPSes you're probably gonna want the offsets and sizes in hexadecimal, fr fr:
$ pahole --hex thread_struct
struct thread_struct {
struct desc_struct tls_array[3]; /* 0 0x18 */
long unsigned int sp; /* 0x18 0x8 */
short unsigned int es; /* 0x20 0x2 */
short unsigned int ds; /* 0x22 0x2 */
short unsigned int fsindex; /* 0x24 0x2 */
short unsigned int gsindex; /* 0x26 0x2 */
long unsigned int fsbase; /* 0x28 0x8 */
long unsigned int gsbase; /* 0x30 0x8 */
struct perf_event * ptrace_bps[4]; /* 0x38 0x20 */
/* --- cacheline 1 boundary (64 bytes) was 24 bytes ago --- */
long unsigned int debugreg6; /* 0x58 0x8 */
long unsigned int ptrace_dr7; /* 0x60 0x8 */
long unsigned int cr2; /* 0x68 0x8 */
long unsigned int trap_nr; /* 0x70 0x8 */
long unsigned int error_code; /* 0x78 0x8 */
/* --- cacheline 2 boundary (128 bytes) --- */
struct io_bitmap * io_bitmap; /* 0x80 0x8 */
long unsigned int iopl_emul; /* 0x88 0x8 */
mm_segment_t addr_limit; /* 0x90 0x8 */
unsigned int sig_on_uaccess_err:1; /* 0x98: 0 0x4 */
unsigned int uaccess_err:1; /* 0x98:0x1 0x4 */
/* XXX 30 bits hole, try to pack */
/* XXX 36 bytes hole, try to pack */
/* --- cacheline 3 boundary (192 bytes) --- */
struct fpu fpu; /* 0xc0 0x1040 */
/* size: 4352, cachelines: 68, members: 20 */
/* sum members: 4312, holes: 1, sum holes: 36 */
/* sum bitfield members: 2 bits, bit holes: 1, sum bit holes: 30 bits */
};
$
ok so at this point I know the offset causing it all is a struct thread_struct and the offset is 0x178, so it has to be inside that fpu structโฆ no stress tho, just expand struct thread_struct and pipe it through grep like a legend ๐:
$ pahole --hex -E thread_struct | egrep '(0x178|struct fpu)' -B4 -A4
/* XXX 30 bits hole, try to pack */
/* XXX 36 bytes hole, try to pack */
/* --- cacheline 3 boundary (192 bytes) --- */
struct fpu {
unsigned int last_cpu; /* 0xc0 0x4 */
/* XXX 4 bytes hole, try to pack */
--
/* typedef u8 -> __u8 */ unsigned char alimit; /* 0x171 0x1 */
/* XXX 6 bytes hole, try to pack */
struct math_emu_info * info; /* 0x178 0x8 */
/* --- cacheline 6 boundary (384 bytes) --- */
/* typedef u32 -> __u32 */ unsigned int entry_eip; /* 0x180 0x4 */
} soft; /* 0x100 0x88 */
struct xregs_state {
$
Want to know where struct thread_struct is actually defined in the kernel sources? bestie I got you:
$ pahole -I thread_struct | head -2
/* Used at: /sys/kernel/btf/vmlinux */
/* <0> (null):0 */
$
It's not present in BTF, so switch to DWARF โ takes a little bit longer, ngl โ and assuming it finds the matching vmlinux file:
$ pahole -Fdwarf -I thread_struct | head -2
/* Used at: /home/acme/git/linux/arch/x86/kernel/head64.c */
/* <3333> /home/acme/git/linux/arch/x86/include/asm/processor.h:485 */
$
Wanna find the biggest data structures in the Linux kernel? say less, sort by column two numerically, reversed, top 5:
$ pahole -s | sort -k2 -nr | head -5
cmp_data 290904 1
dec_datas 274520 1
cpu_entry_area 217088 0
pglist_data 172928 4
saved_cmdlines_buffer 131104 1
$
The second column is the size in bytes, and the third column is the number of alignment holes in that structure. ๐
Show data structures that have a raw spinlock AND are related to the RCU mechanism:
$ pahole --contains raw_spinlock_t --prefix rcu
rcu_node
rcu_data
rcu_state
$
and to actually see that in context, combine it with grep:
$ pahole rcu_state | grep raw_spinlock_t -B1 -A5
/* --- cacheline 52 boundary (3328 bytes) --- */
raw_spinlock_t ofl_lock; /* 3328 4 */
/* size: 3392, cachelines: 53, members: 35 */
/* sum members: 3250, holes: 7, sum holes: 82 */
/* padding: 60 */
};
$
๐ PRETTY PRINTING ๐
ok so pahole can also use the data structure types to pretty print raw data โ you do that with --prettify. And if you wanna consume raw data from standard input, you just use --prettify - ๐
It can pretty print raw data from stdin according to whatever type you specify, like this:
$ pahole -C modversion_info drivers/scsi/sg.ko
struct modversion_info {
long unsigned int crc; /* 0 8 */
char name[56]; /* 8 56 */
/* size: 64, cachelines: 1, members: 2 */
};
$
$ objcopy -O binary --only-section=__versions drivers/scsi/sg.ko versions
$
$ ls -la versions
-rw-rw-r--. 1 acme acme 7616 Jun 25 11:33 versions
$
$ pahole --count 3 -C modversion_info drivers/scsi/sg.ko --prettify versions
{
.crc = 0x8dabd84,
.name = "module_layout",
},
{
.crc = 0x45e4617b,
.name = "no_llseek",
},
{
.crc = 0xa23fae8c,
.name = "param_ops_int",
},
$
$ pahole --skip 1 --count 2 -C modversion_info drivers/scsi/sg.ko --prettify - < versions
{
.crc = 0x45e4617b,
.name = "no_llseek",
},
{
.crc = 0xa23fae8c,
.name = "param_ops_int",
},
$
and that last one is equivalent to:
$ pahole --seek_bytes 64 --count 1 -C modversion_info drivers/scsi/sg.ko --prettify versions
{
.crc = 0x45e4617b,
.name = "no_llseek",
},
$
-C,--class_name=CLASS_NAMEโ pretty print according to this class. And you can pass arguments to it to affect how the pretty printing gets done, like:
-C 'perf_event_header(sizeof,type,type_enum=perf_event_type,filter=type==PERF_RECORD_EXIT)'
so this would pick the struct perf_event_header as the type to use for pretty printing records, and it says that the size field in that struct should be used to figure out the size of the record (these are variable sized records, cute), and that the enum perf_event_type should be used to pretty print the numeric value in perf_event_header->type, and on top of all that it should be used to heuristically look for structs with the same name (lowercase) as the enum entry that gets converted from the type field, and use that to pretty print instead of the base perf_event_header type. peek the PRETTY PRINTING EXAMPLES section below for the whole vibe โจ
Also the filter= part can be used โ so far with only the == operator โ to filter based on the type field and convert the string 'PERF_RECORD_EXIT' to a number according to type_enum. no cap.
The sizeof arg defaults to the size member name, and if the name is different you can use the sizeof=sz form. ditto for the type=other_member_name field, which defaults to type. ๐
PRETTY PRINTING EXAMPLES โจ
Okay bestie, let's peek at the ELF header of a vmlinux file using BTF. First you gotta discover the ELF header type:
$ pahole --sizes | grep -i elf | grep -i _h
elf64_hdr 64 0
elf32_hdr 52 0
$
Now we can flex and show the first record from offset zero:
$ pahole -C elf64_hdr --count 1 --prettify /lib/modules/5.8.0-rc3+/build/vmlinux
{
.e_ident = { 127, 69, 76, 70, 2, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0 },
.e_type = 2,
.e_machine = 62,
.e_version = 1,
.e_entry = 16777216,
.e_phoff = 64,
.e_shoff = 775923840,
.e_flags = 0,
.e_ehsize = 64,
.e_phentsize = 56,
.e_phnum = 5,
.e_shentsize = 64,
.e_shnum = 80,
.e_shstrndx = 79,
},
$
This is literally the same as:
$ pahole --header elf64_hdr --prettify /lib/modules/5.8.0-rc3+/build/vmlinux
The --header option is that girl: it also lets other command line options reference fields in the header. Super useful when you wanna show multiple records in a file and the range where those fields live is specified in header fields โ like for perf.data files:
$ pahole --hex ~/bin/perf --header perf_file_header --prettify perf.data
{
.magic = 0x32454c4946524550,
.size = 0x68,
.attr_size = 0x88,
.attrs = {
.offset = 0xa8,
.size = 0x88,
},
.data = {
.offset = 0x130,
.size = 0x588,
},
.event_types = {
.offset = 0,
.size = 0,
},
.adds_features = { 0x16717ffc, 0, 0, 0 },
},
$
So to display the cgroups records in the perf_file_header.data section, we can use:
$ pahole ~/bin/perf --header=perf_file_header --seek_bytes '$header.data.offset' --size_bytes='$header.data.size' -C 'perf_event_header(sizeof,type,type_enum=perf_event_type,filter=type==PERF_RECORD_CGROUP)' --prettify perf.data
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 40,
},
.id = 1,
.path = "/",
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 48,
},
.id = 1553,
.path = "/system.slice",
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 48,
},
.id = 8,
.path = "/machine.slice",
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 128,
},
.id = 7828,
.path = "/machine.slice/libpod-42be8e8d4eb9d22405845005f0d04ea398548dccc934a150fbaa3c1f1f9492c2.scope",
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 88,
},
.id = 13,
.path = "/machine.slice/machine-qemu\x2d1\x2drhel6.sandy.scope",
},
$
For the common case where the header has a member with 'offset' and 'size' members, you can use this more compact form:
$ pahole ~/bin/perf --header=perf_file_header --range=data -C 'perf_event_header(sizeof,type,type_enum=perf_event_type,filter=type==PERF_RECORD_CGROUP)' --prettify perf.data
This uses ~/bin/perf to get the type definitions, defines 'struct perf_file_header' as the header, then seeks '$header.data.offset' bytes from the start of the file, and considers '$header.data.size' bytes worth of such records. The filter expression can omit a common prefix, so in this case it could also be written as 'filter=type==CGROUP', or the 'filter=' can be omitted entirely, getting as compact as 'type==CGROUP':
If we look at this:
$ pahole ~/bin/perf -C perf_event_header
struct perf_event_header {
__u32 type; /* 0 4 */
__u16 misc; /* 4 2 */
__u16 size; /* 6 2 */
/* size: 8, cachelines: 1, members: 3 */
/* last cacheline: 8 bytes */
};
$
And also:
$ pahole ~/bin/perf -C perf_event_type
enum perf_event_type {
PERF_RECORD_MMAP = 1,
PERF_RECORD_LOST = 2,
PERF_RECORD_COMM = 3,
PERF_RECORD_EXIT = 4,
PERF_RECORD_THROTTLE = 5,
PERF_RECORD_UNTHROTTLE = 6,
PERF_RECORD_FORK = 7,
PERF_RECORD_READ = 8,
PERF_RECORD_SAMPLE = 9,
PERF_RECORD_MMAP2 = 10,
PERF_RECORD_AUX = 11,
PERF_RECORD_ITRACE_START = 12,
PERF_RECORD_LOST_SAMPLES = 13,
PERF_RECORD_SWITCH = 14,
PERF_RECORD_SWITCH_CPU_WIDE = 15,
PERF_RECORD_NAMESPACES = 16,
PERF_RECORD_KSYMBOL = 17,
PERF_RECORD_BPF_EVENT = 18,
PERF_RECORD_CGROUP = 19,
PERF_RECORD_TEXT_POKE = 20,
PERF_RECORD_MAX = 21,
};
$
And furthermore, check it:
$ pahole ~/bin/perf -C perf_record_cgroup
struct perf_record_cgroup {
struct perf_event_header header; /* 0 8 */
__u64 id; /* 8 8 */
char path[4096]; /* 16 4096 */
/* size: 4112, cachelines: 65, members: 3 */
/* last cacheline: 16 bytes */
};
$
Then we can see how perf_event_header.type could be converted from a __u32 to a string (PERF_RECORD_CGROUP). If we remove that type_enum=perf_event_type, we lose the conversion of 'struct perf_event_header' to the more descriptive 'struct perf_record_cgroup', and also the beautification of the header.type field:
$ pahole ~/bin/perf --header=perf_file_header --seek_bytes '$header.data.offset' --size_bytes='$header.data.size' -C 'perf_event_header(sizeof,type,filter=type==19)' --prettify perf.data
{
.type = 19,
.misc = 0,
.size = 40,
},
{
.type = 19,
.misc = 0,
.size = 48,
},
{
.type = 19,
.misc = 0,
.size = 48,
},
{
.type = 19,
.misc = 0,
.size = 128,
},
{
.type = 19,
.misc = 0,
.size = 88,
},
$
Some of the records aren't found in 'type_enum=perf_event_type', so some records don't get converted to a type that fully shows its contents. For perf we know those are in another enumeration, 'enum perf_user_event_type', so for these cases we can create a 'virtual enum' โ the sum of two enums โ and then get all those entries decoded and properly casted. First, a few records with just 'enum perf_event_type':
$ pahole ~/bin/perf --header=perf_file_header --seek_bytes '$header.data.offset' --size_bytes='$header.data.size' -C 'perf_event_header(sizeof,type,type_enum=perf_event_type)' --count 4 --prettify perf.data
{
.type = 79,
.misc = 0,
.size = 32,
},
{
.type = 73,
.misc = 0,
.size = 40,
},
{
.type = 74,
.misc = 0,
.size = 32,
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 40,
},
.id = 1,
.path = "/",
},
$
Now with both enumerations, i.e. with 'type_enum=perf_event_type+perf_user_event_type':
$ pahole ~/bin/perf --header=perf_file_header --seek_bytes '$header.data.offset' --size_bytes='$header.data.size' -C 'perf_event_header(sizeof,type,type_enum=perf_event_type+perf_user_event_type)' --count 5 --prettify perf.data
{
.header = {
.type = PERF_RECORD_TIME_CONV,
.misc = 0,
.size = 32,
},
.time_shift = 31,
.time_mult = 1016803377,
.time_zero = 435759009518382,
},
{
.header = {
.type = PERF_RECORD_THREAD_MAP,
.misc = 0,
.size = 40,
},
.nr = 1,
.entries = 0x50 0x7e 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00,
},
{
.header = {
.type = PERF_RECORD_CPU_MAP,
.misc = 0,
.size = 32,
},
.data = {
.type = 1,
.data = "",
},
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 40,
},
.id = 1,
.path = "/",
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 48,
},
.id = 1553,
.path = "/system.slice",
},
$
You can also pass multiple types, but bestie, make sure they appear in the file in sequence. For the perf.data example, see the perf_file_header dump above โ you can print the perf_file_attr structs in the header attrs range, then the perf_event_header in the data range with this command:
pahole ~/bin/perf --header=perf_file_header -C 'perf_file_attr(range=attrs),perf_event_header(range=data,sizeof,type,type_enum=perf_event_type+perf_user_event_type)' --prettify perf.data
SEE ALSO ๐
Also check out these other icons:
And this paper, periodt:
https://www.kernel.org/doc/ols/2007/ols2007v2-pages-35-44.pdf.
AUTHOR ๐
pahole was written and is maintained by Arnaldo Carvalho de Melo <acme@kernel.org>.
Thanks to Andrii Nakryiko and Martin KaFai Lau for providing the BTF encoder and improving the codebase while making sure the BTF encoder works as needed to be used in encoding the Linux kernel .BTF section from the DWARF info generated by gcc. For that Andrii wrote a BTF deduplicator in libbpf that is used by pahole.
Also thanks to Conectiva, Mandriva and Red Hat for allowing me to work on these tools.
Please send bug reports to <dwarves@vger.kernel.org>.
No subscription is required.
And that's a wrap, bestie! This iconic page came from the dwarves crew, dated January 16, 2020, and it's for pahole(1). Periodt. ๐
โจ
๐ฆ where this page comes from
- fedora 43: dwarves 1.32-1.fc43 ยท GPL-2.0-only
- fedora 44: dwarves 1.32-1.fc44 ยท GPL-2.0-only
- nixpkgs unstable: pahole 1.32
- ๐ girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms