CRUSHDIFF(8)

✨ CRUSHDIFF β€” the crush map test tool, bestie edition ✨

okay so real talk, crushdiff is a section 8 (sysadmin) tool and it's literally the cutest way to find out what happens to your cluster when you go messing with a crush map πŸ’–


🌸 NAME 🌸

crushdiff β€” the ceph crush map test tool. no cap, that's the whole vibe.


πŸ¦‹ SYNOPSIS πŸ¦‹

crushdiff [ --osdmap osdmap ] [ --pg-dump pg-dump ]
[ --compiled ] [ --verbose ] command crushmap

πŸ’… DESCRIPTION πŸ’…

so crushdiff is that utility that lets you test the effect of a crushmap change β€” how many pgs, how many objects, how many bytes are gonna move. it's giving "let me see the damage before i commit" and honestly? iconic.

under the hood it's a wrapper around osdmaptool <>(8), and it's leaning on that tool's --test-map-pgs-dump option to get the list of changed pgs. on top of that, it uses pg stats to calculate the numbers of objects and bytes moved. math queen πŸ‘‘

by default, crushdiff will use the cluster's current osdmap and pg stats β€” which requires access to the cluster, so yeah, you gotta be plugged in πŸ”Œ though you can totally use the --osdmap and --pg-dump options to test against previously obtained data if you'd rather not bother the live cluster. very "i saved the receipts" energy.


✨ OPTIONS ✨

  • --compiled β€” the input/output crushmap is compiled. if you don't pass this option, the expected/returned crushmap is gonna be in txt (decompiled) format instead.
  • --pg-dump <pg-dump> β€” JSON output of ceph pg dump. if you don't specify it, crushdiff will try to get the data by running the command itself. it's nothing if not resourceful 🫢
  • --osdmap <osdmap> β€” the cluster osdmap, obtained with the ceph osd getmap command. if you don't specify it, crushdiff will try to get the data by running the command itself.
  • --verbose β€” produce diagnostic output. spill the tea, bestie β˜•

πŸŽ€ COMMANDS πŸŽ€

  • compare crushmap β€” compare the crushmap from the crushmap file with the crushmap from the cluster osdmap. the output will show the expected number of pgs, objects, and bytes moved when the new crushmap is installed. this is the main event, the moment, the reveal πŸ‘‘
  • export crushmap β€” export crushmap to the crushmap file from the cluster osdmap. grab a copy of what's live rn!
  • import crushmap β€” import crushmap from the crushmap file into the cluster osdmap. this is the "okay i'm actually doing it" button πŸš€

🌈 EXAMPLE 🌈

first, get the current crushmap:

crushdiff export cm.txt

then edit the map (go off, editor of choice):

$EDITOR cm.txt

then check the result:

crushdiff compare cm.txt

and it'll print something like:

79/416 (18.99%) pgs affected
281/1392 (20.19%) objects affected
80/1248 (6.41%) pg shards to move
281/4176 (6.73%) pg object shards to move
730.52Mi/10.55Gi (6.76%) bytes to move

when you run it with the --verbose option, the output will also contain detailed information about the affected pgs, like below:

4.3     [0, 2, 1] -> [1, 4, 2]
4.b     [0, 1, 3] -> [2, 1, 3]
4.c     [4, 0, 1] -> [4, 1, 2]

i.e. a pg number, and its old and the new osd active sets. so you can literally watch which pgs are shifting around. it's giving glow-up montage ✨

if the result is satisfactory, install the updated map:

crushdiff import cm.txt
ngl, be careful out here β€” you're testing against real cluster data and import will actually change the cluster's crushmap. look before you leap, bestie πŸ’€

πŸ’œ AVAILABILITY πŸ’œ

crushdiff is part of Ceph, a massively scalable, open-source, distributed storage system. please refer to the Ceph documentation at <https://docs.ceph.com> for more information. that's the lore, go read it πŸ“š


πŸ¦„ SEE ALSO πŸ¦„

ceph <>(8), crushtool <>(8), osdmaptool <>(8),


Copyright β€” 2010-2014, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


and that's the whole page, bestie 🫢 this was dev, dated Oct 06, 2026, and yes it lives in the CRUSHDIFF(8) section πŸ’– mwah, go move some pgs responsibly ✨

πŸ“¦ where this page comes from