ZPOOL-EVENTS(8)

πŸ’– ZPOOL-EVENTS πŸ’–

okay bestie, gather round, because this is the manual page for zpool-events β€” that's the System Manager's Manual entry, section 8, and she is a big one. She's the one who spills the tea on everything the ZFS kernel modules have been up to lately. ✨


🌸 NAME 🌸

zpool-events β€” list recent events generated by kernel


✨ SYNOPSIS ✨

zpool events [-vHf] [pool]
zpool events -c

πŸ¦‹ DESCRIPTION πŸ¦‹

So basically, zpool events lists all the recent events that the ZFS kernel modules have generated. These events get consumed by zed(8), and zed uses them to automate the boring administrative stuff for you β€” like swapping a dead device out for a hot spare without you having to lift a finger. πŸ’…

If you want the full rundown on the subclasses and event payloads that can show up, go peek at the β€œEVENTS” section and everything after it. πŸ‘‘


πŸŽ€ OPTIONS πŸŽ€

  • -c β€” Clears out all the previous events. bye bye history 🫢
  • -f β€” Follow mode. She stays watching. πŸ‘€
  • -H β€” Scripted mode. No headers, no vibes, just the facts β€” fields get separated by a single tab instead of whatever arbitrary spacing it felt like using. ✨
  • -v β€” Prints the entire payload</> for each event. the whole thing. all of it. no crumbs left behind. πŸ’–

πŸ”₯ EVENTS πŸ”₯

These are the different event subclasses, bestie. The full event name would be ereport.fs.zfs.SUBCLASS, but we're only listing the last part here because the rest is just... a lot. iykyk.

  • checksum β€” Fired off when a checksum error has been detected. 🩹
  • io β€” Fired off when there's an I/O error in a vdev in the pool.
  • data β€” Fired off when there have been data errors in the pool. 😭
  • deadman β€” Fired off when an I/O request gets determined to be "hung". This can happen because of lost completion events caused by flaky hardware or flaky drivers. Peep zfs_deadman_failmode in zfs(4) for the full scoop on "hung" I/O detection and how it's configured.
  • delay β€” Fired off when a completed I/O request takes longer than the max allowed time, which is specified by the zio_slow_io_ms module parameter. This is often giving "something is wrong with the underlying storage device". πŸ‘€ The number of delay events is ratelimited by the zfs_slow_io_events_per_second module parameter.
  • dio_verify_rd β€” Fired off when there was a checksum verify error after a Direct I/O read has been issued.
  • dio_verify_wr β€” Fired off when there was a checksum verify error after a Direct I/O write has been issued. This one can only happen if the module parameter zfs_vdev_direct_write_verify is not set to zero. See zfs(4) for more deets on the zfs_vdev_direct_write_verify module parameter.
  • config_sync β€” Fired off every single time a vdev change has been done to the pool.
  • zpool β€” Fired off when a pool cannot be imported. πŸ’€
  • pool_create β€” Fired off when a pool is created. πŸ‘Άβœ¨
  • pool_destroy β€” Fired off when a pool is destroyed. πŸ’€
  • pool_export β€” Fired off when a pool is exported.
  • pool_import β€” Fired off when a pool is imported.
  • pool_reguid β€” Fired off when a REGUID has been detected β€” that's when a new unique identifier for the pool has been regenerated. πŸ†”
  • vdev.unknown β€” Fired off when the vdev is unknown. Like, say, when you're trying to clear device errors on a vdev that has failed or been kicked from the system/pool and is no longer available. shruggie.
  • vdev.open_failed β€” Fired off when a vdev could not be opened β€” because it didn't exist, for example. πŸ™ˆ
  • vdev.corrupt_data β€” Fired off when corrupt data has been detected on a vdev.
  • vdev.no_replicas β€” Fired off when there are no more replicas left to sustain the pool. This would lead to the pool being DEGRADED. not the vibe. 😭
  • vdev.bad_guid_sum β€” Fired off when a missing device in the pool has been detected.
  • vdev.too_small β€” Fired off when the system (kernel) has removed a device and ZFS notices the device isn't there anymore. This is usually followed by a probe_failure event, so watch for the sequel.
  • vdev.bad_label β€” Fired off when the label is OK but invalid. 🀨
  • vdev.bad_ashift β€” Fired off when the ashift alignment requirement has increased.
  • vdev_remove β€” Fired off when a vdev is detached from a mirror β€” or when a spare is detached from a vdev where it has been used to replace a failed drive. And heads up: this only works if the original drive has been re-added. πŸ‘‘
  • vdev_remove_aux β€” Fired off when an auxiliary vdev is removed.
  • vdev_remove_dev β€” Fired off when a specific device is removed from a vdev.
  • vdev_clear β€” Fired off when clearing device errors in a pool. Like when you run zpool clear on a device in the pool. 🧹✨
  • vdev_check β€” Fired off when a check to see if a given vdev could be opened is started.
  • vdev_spare β€” Fired off when a spare has kicked in to replace a failed device. slay, spare. 🦸
  • vdev_autoexpand β€” Fired off when a vdev can be automatically expanded.
  • vdev_add β€” Fired off when a vdev is added to a pool.
  • vdev_attach β€” Fired off when a vdev is attached to a mirror or raidz vdev type.
  • vdev_online β€” Fired off when an offline vdev is brought online. welcome back babe 🫢
  • io_failure β€” Fired off when there is an I/O failure in a vdev in the pool.
  • probe_failure β€” Fired off when a probe fails on a vdev. This would occur if a vdev has been kicked from the system outside of ZFS β€” like when the kernel has removed the device. πŸ™ˆ
  • log_replay β€” Fired off when the intent log cannot be replayed. This can occur in the case of a missing or damaged log device. πŸ’€
  • resilver_start β€” Fired off when a resilver is started.
  • resilver_finish β€” Fired off when the running resilver has finished. πŸŽ‰
  • scrub_start β€” Fired off when a scrub is started on a pool.
  • scrub_finish β€” Fired off when a pool has finished scrubbing. ✨
  • scrub_abort β€” Fired off when a scrub is aborted on a pool.
  • scrub_resume β€” Fired off when a scrub is resumed on a pool.
  • scrub_paused β€” Fired off when a scrub is paused on a pool.
  • errorscrub_start β€” Fired off when an errorscrub is started on a pool.
  • errorscrub_finish β€” Fired off when a pool has finished errorscrubbing. ✨
  • errorscrub_abort β€” Fired off when an errorscrub is aborted on a pool.
  • errorscrub_resume β€” Fired off when an errorscrub is resumed on a pool.
  • errorscrub_paused β€” Fired off when an errorscrub is paused on a pool.
  • trim_start β€” Fired off when a trim is started on a pool.
  • trim_finish β€” Fired off when a pool has finished trimbing. (yes it says trimbing. we don't make the rules, we just report them. 😭)
  • trim_cancel β€” Fired off when a trim is canceled on a pool.
  • trim_resume β€” Fired off when a trim is resumed on a pool.
  • trim_suspend β€” Fired off when a trim is suspend on a pool.
  • authentication β€” Fired off when there is a decryption / authentication error. πŸ”
  • config_cache_write β€” Fired off when the config cache file cannot be written.
  • bootfs_vdev_attach β€” Fired off when a vdev is attached to a root pool with the bootfs property set. πŸ‘Ÿ
  • sitout β€” Fired off when a RAIDZ or DRAID vdev triggers the autosit logic. This logic detects when a disk in such a vdev is significantly slower than its peers, and sits them out temporarily to preserve the performance of the pool. time out, bestie, think about what you did. πŸͺ‘

πŸ’Ύ PAYLOADS πŸ’Ύ

Okay bestie, so when an event pops off, it comes with a payload β€” that's just the data/information riding along with it. Think of it as the event's little purse full of receipts. πŸ‘œβœ¨

Heads up: for zed(8), all of these get uppercased and prefixed with ZEVENT_. So pool shows up as ZEVENT_POOL. iykyk πŸ’…
  • pool β€” the pool's name, obviously. πŸ’–
  • pool_failmode β€” the failmode, which is one of wait, continue, or panic. Go peep the failmode property in zpoolprops(7) if you want the full tea.
  • pool_guid β€” the GUID of the pool. πŸ”‘
  • pool_context β€” the pool's load state: 0=none, 1=open, 2=import, 3=tryimport, 4=recover, 5=error.
  • vdev_guid β€” the GUID of the vdev in question (aka the vdev that's failing, or the one being operated on with zpool clear, etc.).
  • vdev_type β€” what kind of vdev it is: disk, file, mirror, etc. The Virtual Devices section of zpoolconcepts(7) has all the possible values if you wanna go deeper. πŸ¦‹
  • vdev_path β€” the full path of the vdev, -partX and all.
  • vdev_devid β€” the vdev's ID, if it's got one.
  • vdev_fru β€” the physical FRU location.
  • vdev_state β€” the vdev's state: 0=uninitialized, 1=closed, 2=offline, 3=removed, 4=failed to open, 5=faulted, 6=degraded, 7=healthy.
  • vdev_ashift β€” the vdev's ashift value.
  • vdev_complete_ts β€” the time the last I/O request finished for that vdev.
  • vdev_delta_ts β€” how much time has passed since the last I/O request finished for that vdev.
  • vdev_spare_paths β€” a list of spares, full path plus any -partX included.
  • vdev_spare_guids β€” the GUID(s) of the spares.
  • vdev_read_errors β€” how many read errors got detected on the vdev. 😭
  • vdev_write_errors β€” how many write errors got detected on the vdev.
  • vdev_cksum_errors β€” how many checksum errors got detected on the vdev.
  • parent_guid β€” GUID of the vdev's parent.
  • parent_type β€” type of the parent. Same deal as vdev_type.
  • parent_path β€” path of the vdev's parent, if there is one.
  • parent_devid β€” ID of the vdev's parent, if there is one.
  • zio_objset β€” the object set number for a given I/O request.
  • zio_object β€” the object number for a given I/O request.
  • zio_level β€” the indirect level for the block. Level 0 is the lowest level and it's where the data blocks live. Values > 0 mean metadata blocks at the matching level.
  • zio_blkid β€” the block ID for a given I/O request.
  • zio_err β€” the error number for a failure while handling a given I/O request. It plays nice with errno(3), and the value EBADE gets used to signal a ZFS checksum error. πŸ’€
  • zio_offset β€” the offset in bytes of where the I/O request should be written for the specified vdev.
  • zio_size β€” the size in bytes of the I/O request.
  • zio_flags β€” the current flags describing how the I/O request should be handled. The I/O FLAGS section below has the whole list, no gatekeeping.
  • zio_stage β€” the current stage of the I/O in the pipeline. I/O STAGES section has every single one.
  • zio_pipeline β€” the valid pipeline stages for the I/O. Again, see I/O STAGES.
  • zio_priority β€” the queue priority of the I/O request. See I/O PRIORITIES for the full roster.
  • zio_tyoe β€” the type of the I/O request. See I/O TYPES below for all of them. ✨
  • zio_delay β€” the time elapsed (in nanoseconds) spent waiting for the block layer to complete the I/O request. Unlike zio_delta, this does not include any vdev queuing time, so it's purely a measure of block layer performance. Very demure, very mindful. πŸŽ€
  • zio_timestamp β€” the time when a given I/O request was submitted.
  • zio_delta β€” the time required to service a given I/O request.
  • prev_state β€” the previous state of the vdev.
  • cksum_algorithm β€” which checksum algorithm was used. zfsprops(7) has the details on what's available.
  • cksum_byteswap β€” whether or not the data is byteswapped.
  • bad_ranges β€” [start, end) pairs of corruption offsets. Offsets are always aligned on a 64-bit boundary, and they can include some gaps of non-corruption. (See bad_ranges_min_gap below.)
  • bad_ranges_min_gap β€” to keep the size of the bad_ranges array under control, gaps of non-corruption that are less than or equal to bad_ranges_min_gap bytes have been merged into the adjacent corruption. It's always at least 8 bytes, since corruption is detected on a 64-bit word basis. 🧐
  • bad_range_sets β€” this array has one element per range in bad_ranges. Each element holds the count of bits in that range which were clear in the good data and set in the bad data.
  • bad_range_clears β€” this array has one element per range in bad_ranges. Each element holds the count of bits for that range which were set in the good data and clear in the bad data.
  • bad_set_bits β€” if this field exists, it's an array of (bad data & ~(good data)) β€” that is, the bits that are set in the bad data and cleared in the good data. Each element corresponds to a byte whose offset falls in a range in bad_ranges, and the array is ordered by offset. So the first element is the first byte in the first bad_ranges range, and the last element is the last byte in the last bad_ranges range.
  • bad_cleared_bits β€” same energy as bad_set_bits, except it holds (good data & ~(bad data)) β€” the bits set in the good data which are cleared in the bad data.

πŸ”§ I/O STAGES πŸ”§

The ZFS I/O pipeline is made up of a bunch of stages, all defined below. Those individual stages get combined to build these basic I/O operations: Read, Write, Free, Claim, Flush and Trim. And a stage can get set on an event to describe the life cycle of a given I/O request. Main character journey, fr. πŸ‘‘

  • ZIO_STAGE_OPEN β€” bit mask 0x00000001 β€” operations: RWFCXT
  • ZIO_STAGE_READ_BP_INIT β€” bit mask 0x00000002 β€” operations: R-----
  • ZIO_STAGE_WRITE_BP_INIT β€” bit mask 0x00000004 β€” operations: -W----
  • ZIO_STAGE_FREE_BP_INIT β€” bit mask 0x00000008 β€” operations: --F---
  • ZIO_STAGE_ISSUE_ASYNC β€” bit mask 0x00000010 β€” operations: -WF--T
  • ZIO_STAGE_WRITE_COMPRESS β€” bit mask 0x00000020 β€” operations: -W----
  • ZIO_STAGE_ENCRYPT β€” bit mask 0x00000040 β€” operations: -W----
  • ZIO_STAGE_CHECKSUM_GENERATE β€” bit mask 0x00000080 β€” operations: -W----
  • ZIO_STAGE_NOP_WRITE β€” bit mask 0x00000100 β€” operations: -W----
  • ZIO_STAGE_DDT_READ_START β€” bit mask 0x00000200 β€” operations: R-----
  • ZIO_STAGE_DDT_READ_DONE β€” bit mask 0x00000400 β€” operations: R-----
  • ZIO_STAGE_DDT_WRITE β€” bit mask 0x00000800 β€” operations: -W----
  • ZIO_STAGE_DDT_FREE β€” bit mask 0x00001000 β€” operations: --F---
  • ZIO_STAGE_BRT_FREE β€” bit mask 0x00002000 β€” operations: --F---
  • ZIO_STAGE_GANG_ASSEMBLE β€” bit mask 0x00004000 β€” operations: RWFC--
  • ZIO_STAGE_GANG_ISSUE β€” bit mask 0x00008000 β€” operations: RWFC--
  • ZIO_STAGE_DVA_THROTTLE β€” bit mask 0x00010000 β€” operations: -W----
  • ZIO_STAGE_DVA_ALLOCATE β€” bit mask 0x00020000 β€” operations: -W----
  • ZIO_STAGE_DVA_FREE β€” bit mask 0x00040000 β€” operations: --F---
  • ZIO_STAGE_DVA_CLAIM β€” bit mask 0x00080000 β€” operations: ---C--
  • ZIO_STAGE_READY β€” bit mask 0x00100000 β€” operations: RWFCIT
  • ZIO_STAGE_VDEV_IO_START β€” bit mask 0x00200000 β€” operations: RW--XT
  • ZIO_STAGE_VDEV_IO_DONE β€” bit mask 0x00400000 β€” operations: RW--XT
  • ZIO_STAGE_VDEV_IO_ASSESS β€” bit mask 0x00800000 β€” operations: RW--XT
  • ZIO_STAGE_CHECKSUM_VERIFY β€” bit mask 0x01000000 β€” operations: R-----
  • ZIO_STAGE_DIO_CHECKSUM_VERIFY β€” bit mask 0x02000000 β€” operations: -W----
  • ZIO_STAGE_DONE β€” bit mask 0x04000000 β€” operations: RWFCXT

🚩 I/O FLAGS 🚩

Every single I/O request in the pipeline carries a set of flags that describe what it's doing and govern how it behaves. These flags get set in an event as a zio_flags payload entry. πŸ’…

  • ZIO_FLAG_DONT_AGGREGATE β€” bit mask 0x00000001
  • ZIO_FLAG_IO_REPAIR β€” bit mask 0x00000002
  • ZIO_FLAG_SELF_HEAL β€” bit mask 0x00000004
  • ZIO_FLAG_RESILVER β€” bit mask 0x00000008
  • ZIO_FLAG_SCRUB β€” bit mask 0x00000010
  • ZIO_FLAG_SCAN_THREAD β€” bit mask 0x00000020
  • ZIO_FLAG_PHYSICAL β€” bit mask 0x00000040
  • ZIO_FLAG_CANFAIL β€” bit mask 0x00000080
  • ZIO_FLAG_SPECULATIVE β€” bit mask 0x00000100
  • ZIO_FLAG_CONFIG_WRITER β€” bit mask 0x00000200
  • ZIO_FLAG_DONT_RETRY β€” bit mask 0x00000400
  • ZIO_FLAG_NODATA β€” bit mask 0x00001000
  • ZIO_FLAG_INDUCE_DAMAGE β€” bit mask 0x00002000
  • ZIO_FLAG_ALLOC_THROTTLED β€” bit mask 0x00004000
  • ZIO_FLAG_IO_RETRY β€” bit mask 0x00008000
  • ZIO_FLAG_PROBE β€” bit mask 0x00010000
  • ZIO_FLAG_TRYHARD β€” bit mask 0x00020000
  • ZIO_FLAG_OPTIONAL β€” bit mask 0x00040000
  • ZIO_FLAG_DONT_QUEUE β€” bit mask 0x00080000
  • ZIO_FLAG_DONT_PROPAGATE β€” bit mask 0x00100000
  • ZIO_FLAG_IO_BYPASS β€” bit mask 0x00200000
  • ZIO_FLAG_IO_REWRITE β€” bit mask 0x00400000
  • ZIO_FLAG_RAW_COMPRESS β€” bit mask 0x00800000
  • ZIO_FLAG_RAW_ENCRYPT β€” bit mask 0x01000000
  • ZIO_FLAG_GANG_CHILD β€” bit mask 0x02000000
  • ZIO_FLAG_DDT_CHILD β€” bit mask 0x04000000
  • ZIO_FLAG_GODFATHER β€” bit mask 0x08000000
  • ZIO_FLAG_NOPWRITE β€” bit mask 0x10000000
  • ZIO_FLAG_REEXECUTED β€” bit mask 0x20000000
  • ZIO_FLAG_DELEGATED β€” bit mask 0x40000000
  • ZIO_FLAG_FASTWRITE β€” bit mask 0x80000000

🎯 I/O TYPES 🎯

Every I/O request in the pipeline has exactly one type value. That value describes the kind of low-level work the I/O represents, and it gets set in an event as a zio_type payload entry. One job, one label, periodt. 🌸

  • ZIO_TYPE_NULL β€” value 0x0 β€” internal I/O sync point
  • ZIO_TYPE_READ β€” value 0x1 β€” data read
  • ZIO_TYPE_WRITE β€” value 0x2 β€” data write
  • ZIO_TYPE_FREE β€” value 0x3 β€” block free
  • ZIO_TYPE_CLAIM β€” value 0x4 β€” block claim (ZIL replay)
  • ZIO_TYPE_FLUSH β€” value 0x5 β€” disk cache flush request
  • ZIO_TYPE_TRIM β€” value 0x6 β€” trim (discard)

🌈 I/O PRIORITIES 🌈

ok so every single I/O request floatin' around in the pipeline has exactly one priority value, bestie. just the one. and the queuing code uses it to decide which I/O gets to go next β€” like a lil line at the club and the bouncer's checkin' priorities πŸ’…. this value shows up in an event as the zio_priority payload entry, so you can ~literally~ see it. slay.

  • ZIO_PRIORITY_SYNC_READ β†’ 0x0 β€” no description given, it's just the vibe ✨
  • ZIO_PRIORITY_SYNC_WRITE β†’ 0x1 β€” ZIL
  • ZIO_PRIORITY_ASYNC_READ β†’ 0x2 β€” prefetch
  • ZIO_PRIORITY_ASYNC_WRITE β†’ 0x3 β€” spa_sync()
  • ZIO_PRIORITY_SCRUB β†’ 0x4 β€” asynchronous scrub/resilver reads
  • ZIO_PRIORITY_REMOVAL β†’ 0x5 β€” reads/writes for vdev removal
  • ZIO_PRIORITY_INITIALIZING β†’ 0x6 β€” initializing I/O
  • ZIO_PRIORITY_TRIM β†’ 0x7 β€” trim I/O (discard)
  • ZIO_PRIORITY_REBUILD β†’ 0x8 β€” reads/writes for vdev rebuild
  • ZIO_PRIORITY_NOW β†’ 0xa β€” non-queued i/os (e.g. free)
ngl if you noticed 0x9 isn't in the list… yeah. that's just how it is. no cap, that's the original, we don't make the rules πŸ‘‘

πŸ‘‘ SEE ALSO πŸ‘‘

go read these next, fr fr β€” they're the whole friend group:


ok and that's the whole page, bestie!! πŸŽ€ you made it to the end of part 3, which means you now know everything about zpool events, and honestly? main character energy. go forth and zpool events responsibly πŸ’–

this icon was brought to you by OpenZFS, dated July 3, 2025, and filed under ZPOOL-EVENTS(8) β€” periodt. πŸ¦‹βœ¨

πŸ“¦ where this page comes from