fiologparser_hist.py(1)

💖 fiologparser_hist.py(1) — General Commands Manual 💖

okay so bestie, you found the fiologparser_hist.py(1) page and I am so proud of you. this is the whole thing, nothing skipped, no cap. grab a snack. 🍓

---

✨ NAME ✨

fiologparser_hist.py — calculate statistics from fio histograms. that's it, that's the job. 👑

---

🌸 SYNOPSIS 🌸

fiologparser_hist.py [options] [clat_hist_files]...

---

🦋 DESCRIPTION 🦋

so fiologparser_hist.py is a utility that takes the *_clat_hist* files that fio generated for you and turns them into a CSV of latency statistics. we're talking minimum, average, maximum latency, and whatever percentiles you want to pick. it's giving glow-up for your raw histogram data. 💅

---

🎀 EXAMPLES 🎀

look at her go:

$ fiologparser_hist.py *_clat_hist*
end-time, samples, min, avg, median, 90%, 95%, 99%, max
1000, 15, 192, 1678.107, 1788.859, 1856.076, 1880.040, 1899.208, 1888.000
2000, 43, 152, 1642.368, 1714.099, 1816.659, 1845.552, 1888.131, 1888.000
4000, 39, 1152, 1546.962, 1545.785, 1627.192, 1640.019, 1691.204, 1744
...

---

💅 OPTIONS 💅

• --help — prints these options. the classic. the icon.
• --buff_size=int — how many samples get buffered into numpy at one time. default is 10,000. tweak this if you're chasing performance. ✨
• --max_latency=int — how many seconds of data get processed at a time. defaults to 20 seconds, because fio's histograms have that 17 second upper bound on latency, and we're being polite about it. if fio was run with a bigger max latency, bump this up. and if you know your max latency is lower, lowering this can improve performance. check NOTES for the deets. 👀
• -i, --interval=int — the interval the statistics get reported at. defaults to 1000 ms. set this to at minimum the value you gave fio for log_hist_msec. periodt.
• --noweight — don't do the weighting of samples between output intervals. default is False.
• -d, --divisor=int — divide the statistics by this value. defaults to 1. super useful if you wanna convert latencies from milliseconds to seconds (divisor=1000). math slay. 🧮
• --warn — turns on warning messages printed to stderr. great for debugging, iykyk.
• --group_nr=int — set this to the value of FIO_IO_U_PLAT_GROUP_NR as defined in stat.h if fio has been recompiled. defaults to 19, which is the current value used in fio. see NOTES for more. 🔍
• --percentiles=str — pass your desired list of comma or colon separated percentiles to print. the default is "90.0:95.0:99.0", but heads up: min, median(50%) and max percentiles are always printed no matter what. 💖
• --usbin — use this to tell the parser that the histogram bin latency values are in microseconds. the default is nanoseconds, but histogram logs from fio versions <= 2.99 are in microseconds. time travelers, this one's for you. 🕰️
• --directions=str — by default, all directions (like read and write) histogram bins get combined into one 'mixed' result. to get independent directional results, pass some combination of ´rwtm´ characters with the --directions=rwtm option. a ´dir´ column gets added showing the result direction for a row. 🌈

---

🔥 NOTES 🔥

end-times are calculated to be uniform increments of the --interval value you gave, regardless of when histogram samples were actually reported. of note:

• Intervals with no samples are omitted. in the example above this means "no statistics from 2 to 3 seconds" and "39 samples influenced the statistics of the interval from 3 to 4 seconds".
• Intervals with a single sample will have the same value for all statistics.

the number of samples is unweighted, corresponding to the total number of samples which have any effect whatsoever on the interval.

min statistics are computed using the value of the lower boundary of the first bin (in increasing bin order) that has non-zero samples in it. similarly for max, we take the upper boundary of the last bin with non-zero samples in it. this is semantically identical to taking the 0th and 100th percentiles with a 50% bin-width buffer (because percentiles are computed using mid-points of the bins). this enforces these following nice properties, and honestly? iconic:

• min <= 50th <= 90th <= 95th <= 99th <= max
• min and max are strict lower and upper bounds on the actual min / max seen by fio (and reported in *_clat.* with averaging turned off).

average statistics use a standard weighted arithmetic mean. 📊

when the --noweights option is false (the default), percentile statistics are computed using the weighted percentile method as described here: https://en.wikipedia.org/wiki/Percentile#Weighted_percentile. see the weights() method for details on how weights get computed for individual samples. in process_interval() we further multiply by the height of each bin to get weighted histograms.

we convert files given on the command line, assumed to be fio histogram files. an individual histogram file can contain the histograms for multiple different r/w directions (notably when --rw=randrw). this is accounted for by tracking each r/w direction separately. in the statistics reported we ultimately merge all histograms (regardless of r/w direction).

the value of *_GROUP_NR in stat.h (and *_BITS) determines how many latency bins fio outputs when histogramming is enabled. namely for the current default of GROUP_NR=19, we get 1,216 bins with a maximum latency of approximately 17 seconds. for certain applications this may not be sufficient. with GROUP_NR=24 we have 1,536 bins, giving us a maximum latency of 541 seconds (~ 9 minutes). if you expect your application to experience latencies greater than 17 seconds, you will need to recompile fio with a larger GROUP_NR, e.g. with:

sed -i.bak 's/^#define FIO_IO_U_PLAT_GROUP_NR 190#define FIO_IO_U_PLAT_GROUP_NR 24/g' stat.h
make fio

quick reference table for the max latency corresponding to a sampling of values for GROUP_NR:

GROUP_NR | # bins | max latency bin value
19       | 1216   | 16.9 sec
20       | 1280   | 33.8 sec
21       | 1344   | 67.6 sec
22       | 1408   | 2  min, 15 sec
23       | 1472   | 4  min, 32 sec
24       | 1536   | 9  min, 4  sec
25       | 1600   | 18 min, 8  sec
26       | 1664   | 36 min, 16 sec

at present this program automatically detects the number of histogram bins in the log files, and adjusts the bin latency values accordingly. in particular if you use the --log_hist_coarseness parameter of fio, you get output files with a number of bins according to the following table (note that the first row is identical to the table above):

coarse \ GROUP_NR
        19     20    21     22     23     24     25     26
   -------------------------------------------------------
  0  [[ 1216,  1280,  1344,  1408,  1472,  1536,  1600,  1664],
  1   [  608,   640,   672,   704,   736,   768,   800,   832],
  2   [  304,   320,   336,   352,   368,   384,   400,   416],
  3   [  152,   160,   168,   176,   184,   192,   200,   208],
  4   [   76,    80,    84,    88,    92,    96,   100,   104],
  5   [   38,    40,    42,    44,    46,    48,    50,    52],
  6   [   19,    20,    21,    22,    23,    24,    25,    26],
  7   [  N/A,    10,   N/A,    11,   N/A,    12,   N/A,    13],
  8   [  N/A,     5,   N/A,   N/A,   N/A,     6,   N/A,   N/A]]

for other values of GROUP_NR and coarseness, this table can be computed like this:

bins = [1216,1280,1344,1408,1472,1536,1600,1664]
max_coarse = 8
fncn = lambda z: list(map(lambda x: z/2**x if z % 2**x == 0 else nan, range(max_coarse + 1)))
np.transpose(list(map(fncn, bins)))

if you have not adjusted GROUP_NR for your (high latency) application, then you will see the percentiles computed by this tool max out at the max latency bin value as in the first table above, and in this plot (where GROUP_NR=19 and thus we see a max latency of ~16.7 seconds in the red line): 💀

https://www.cronburg.com/fio/max_latency_bin_value_bug.png

motivation for, design decisions, and the implementation process are described in further detail here:

https://www.cronburg.com/fio/cloud-latency-problem-measurement/

---

👑 AUTHOR 👑

fiologparser_hist.py and this manual page were written by Karl Cronburg <karl.cronburg@gmail.com>. thank you karl, absolute legend. 🫶

---

🙈 REPORTING BUGS 🙈

report bugs to the fio mailing list <fio@vger.kernel.org>. they wanna hear from you, fr fr.

---

and that's the whole page, bestie — this version came from the fiologparser_hist.py(1) manual, dated August 18, 2016. you ate. now go read your histograms. 💜✨