NETDUMP(4)

πŸ“¦ version 15.1, in freebsd 14.5, freebsd 15.1, nixpkgs unstable

πŸ’… NETDUMP(4) β€” the kernel-dump-over-the-network page, but make it make sense

okay so real talk, this is from the Kernel Interfaces Manual and it's about netdump, and bestie, it is genuinely iconic: it's how a kernel that's actively dying yeets its own memory dump at a remote server. let's get into it 🫢


✨ NAME ✨

netdump β€” the protocol for transmitting kernel dumps to a remote server πŸŽ€


🌸 SYNOPSIS 🌸

so if you want netdump client support compiled into your kernel, you drop these lines into your kernel configuration file, no cap:

options INET
options DEBUGNET
options NETDUMP

like, all three. INET is not optional, we're an IPv4 household here (more on that in BUGS πŸ‘€).


πŸ¦‹ DESCRIPTION πŸ¦‹

netdump is a UDP-based protocol for transmitting kernel dumps to a remote host. that's the whole vibe.

the cast of characters, okay:

  • a netdump client is a panicking kernel 😭 literally dying and still sending packets, main character energy fr
  • a netdump server is a host running the netdump daemon, which you can find in ports as ports/ftp/netdumpd

clients get configured with the dumpon(8) utility, or with the netdump command in ddb(4). pick your fighter πŸ’…

πŸ“¨ the message format, iykyk

netdump client messages are a fixed-size header followed by a variable-sized payload. the header contains:

  • the message type
  • a sequence number
  • the offset of the payload data in the kernel dump
  • the length of the payload data (not including the header, careful πŸ‘€)

the message types are HERALD, FINISHED, KDH, VMCORE, and EKCD_KEY. those five, periodt.

netdump server messages are different β€” fixed size, and they contain only the sequence number of the client message. that's it. that's the whole message. their entire job is to say "yep bestie, I successfully processed the client message with this corresponding sequence number" ✨ and every single client message gets acknowledged this way. no exceptions.

server messages are always sent to port 20024 of the client. remember that number, it comes back.

🎬 how a netdump actually starts

to initiate a netdump, the client sends a HERALD message to the server at port 20023 (different port! don't mix them up πŸ’€).

the client may include a relative path in its payload β€” and if it does, the netdump server should attempt to save the dump at that path, relative to its configured dump directory.

the server acknowledges the HERALD using a random source port, and from then on the client must send all subsequent messages to that port. so like, the handshake changes the address mid-conversation. slay but also pay attention 🧚

πŸ“¦ the payloads

  • KDH payload β†’ the kernel dump header
  • VMCORE payload β†’ the dump contents
  • EKCD_KEY payload β†’ the dump encryption key

and the offset in the message header should be treated as a seek offset in the corresponding file 🎯

also there are no ordering requirements for these messages. zero. send them however you like.

a netdump is completed by sending the FINISHED message to the server. that's the mic drop 🎀

πŸ”Œ drivers that support netdump

the following network drivers support netdump:

alc(4), bge(4), bnxt(4), bxe(4), cxgb(4), em(4), igb(4), ix(4), ixl(4), mlx4en(4), mlx5en(4), re(4), vtnet(4).

if your NIC isn't on this list… πŸ™ˆ yeah.


πŸŽ€ SYSCTL VARIABLES πŸŽ€

these ones are available as both sysctl(8) variables and loader(8) variables. double duty, we love a multitasker ✨

  • net.netdump.debug β€” controls debug message verbosity. debug messages are disabled by default, but they're genuinely useful when you're troubleshooting, or when you're developing driver support. turn it on when things get weird πŸ”
  • net.netdump.path β€” specifies a path relative to the server's dump directory where the dump gets stored. for example: if the netdump server is configured to store dumps in /var/crash, then a path of foo will make the server attempt to store dumps from the client in /var/crash/foo. important: the server will not automatically create the relative directory. you have to make it yourself, bestie, it will not do it for you 😭
  • net.netdump.polls β€” the client polls the configured network interface while waiting for acknowledgements. this parameter controls the maximum number of poll attempts before giving up β€” which typically results in a re-transmit. each poll attempt takes 0.5ms. so do the math before you crank it πŸ’…
  • net.netdump.retries β€” the number of times the client will re-transmit a packet before aborting a dump due to a lack of acknowledgement. heads up: the default may be too small in environments with lots of packet loss 🌩️
  • net.netdump.arp_retries β€” the number of times the client will attempt to learn the MAC address of the configured gateway or server before giving up and aborting the dump. after that it's just over 🫠

🌈 SEE ALSO 🌈

decryptcore(8), dumpon(8), savecore(8)

go read those next, they're literally the rest of the story πŸ’–


⭐ HISTORY ⭐

netdump client support first appeared in FreeBSD 12.0. a classic, an icon, a legend πŸ‘‘


πŸ’€ BUGS πŸ’€

  • only IPv4 is supported. that's it. that's the bug. no IPv6, we don't know her πŸ™ˆ
  • netdump may only be used after the kernel has panicked. so like, you can't just casually start a netdump for fun. the kernel has to actually be going through it first 😭

ok that's the whole page!! and before you go β€” this beauty is from Debian, dated November 10, 2022, so shoutout to the distro and the date, mwah πŸ’‹ see you in the next man page bestie 🫢✨
πŸ“¦ where this page comes from