nbdkit-cache-filter(1)

📦
version 1.48.1 · 14 versions
, in fedora 44

💖 nbdkit-cache-filter(1) 💖

okay bestie, welcome to the manual page for nbdkit-cache-filter, aka the nbdkit caching filter. this is the nbdkit manual, and this page is literally just about the one filter that goes "hold my iced coffee, i'll remember that for you". 🧚✨

🌸 NAME 🌸

nbdkit-cache-filter — the nbdkit caching filter. it caches. it filters. it's giving ✨performance✨.

🎀 SYNOPSIS 🎀

nbdkit --filter=cache plugin [plugin-args...]
                             [cache=writeback|writethrough|unsafe]
                             [cache-min-block-size=SIZE]
                             [cache-max-size=SIZE]
                             [cache-high-threshold=N]
                             [cache-low-threshold=N]
                             [cache-on-read=true|false|/PATH]

🦋 DESCRIPTION 🦋

so nbdkit-cache-filter is a filter that adds caching on top of a plugin. that's the whole vibe.

why would you want this? because sometimes a plugin is slow 🐌 or expensive to use, and nbdkit is gonna try to minimize requests to the plugin by caching previous requests. iconic behaviour honestly. and if your plugin is slow, you should use cache-on-read=true (not the default, we'll get to that) to cache reads from slow plugins. periodt.

ngl, plot twist: a LOT of NBD clients can do caching themselves, and because that caching happens on the client side it's usually gonna be more effective than caching inside the server. so this filter is for when your client doesn't have effective caching, or — and this is the spicy one — when you use cache=unsafe to defeat flush requests from the client. that is unsafe and can cause data loss, exactly like the name suggests. no cap. 💀

also important: this filter only caches image contents. if what you actually want is to accelerate sequential reads, put nbdkit-readahead-filter(1) or nbdkit-scan-filter(1) on top of this filter. stack 'em like bracelets. 🫶

✨ PARAMETERS ✨

  • cache=writeback — store writes in the cache. they are not written to the plugin unless an explicit flush is done by the client. this is the default caching mode, and it's safe if your client issues flush requests correctly (which is true for modern Linux and other well-written NBD clients). so like, if your client is a good listener, you're fine 💅
  • cache=writethrough — always force writes through to the plugin. this makes the cache less effective, but it's necessary if your client does not issue correct flush requests. it's the responsible friend of the group.
  • cache=unsafe — ignore flush requests. never write to the plugin unless the cache grows too large. ⚠️ this is dangerous and can cause data loss, but it may be acceptable if you only use it for testing, or with data that you don't care about or can cheaply reconstruct. (derogatory. do not do this with your thesis, bestie.)
  • cache-min-block-size=SIZE — set the minimum block size used by the cache. this must be a power of 2 and ≥ 4096. the default is 64K, or the block size of the filesystem which contains the temporary file storing the cache — whichever is larger. so it's a little competition and the filesystem might win. 🌈
  • cache-max-size=SIZE — (nbdkit ≥ 1.10) limit the size of the cache to SIZE. see CACHE MAXIMUM SIZE below. ⬇️
  • cache-high-threshold=N — (nbdkit ≥ 1.10) limit the size of the cache to SIZE. see CACHE MAXIMUM SIZE below. ⬇️
  • cache-low-threshold=N — (nbdkit ≥ 1.10) limit the size of the cache to SIZE. see CACHE MAXIMUM SIZE below. ⬇️
  • cache-on-read=true — (nbdkit ≥ 1.10) cache read requests as well as write and cache requests. any time a block is read from the plugin, it is saved in the cache (if there is sufficient space) so the same data can be served more quickly later. so fast. so efficient. 👑
🙈 careful though: if the underlying data served by the plugin can be modified by some other means (eg. something else can write to a file which is being served by nbdkit-file-plugin(1)), this option will cause nbdkit to serve stale data, because reads won't always go through to the plugin. the tea is old. it's not fresh. 😭
  • cache-on-read=false — do not cache read requests. this is the default.
  • cache-on-read=/PATH — (nbdkit ≥ 1.28) when /PATH (which must be an absolute path) exists, this behaves like cache-on-read=true, and when it does not exist, like cache-on-read=false. this lets you control the cache-on-read behaviour while nbdkit is running. live. in real time. main character energy. 🦋

💅 CACHE MAXIMUM SIZE 💅

by default the cache can grow to any size (although not larger than the virtual size of the underlying plugin) and you have to make sure there's sufficient space in $TMPDIR for it. you. personally. it's on you.

using the parameters cache-max-size, cache-high-threshold and cache-low-threshold you can limit the maximum size of the cache. slay.

this requires kernel and filesystem support (for fallocate(2) FALLOC_FL_PUNCH_HOLE), so it may not work on all platforms. so if it's not working, it's not you, it's the platform 🫠

some examples, because examples are everything:

  • cache-max-size=1G — the cache is limited to around 1 gigabyte.
  • cache-max-size=1G cache-high-threshold=95 cache-low-threshold=80 — once the cache size reaches 972M (95% of 1G), blocks are reclaimed from the cache until the size is reduced to 819M (80% of 1G).

the way this works is: once the size of the cache exceeds SIZE ✕ the high threshold, the filter works to reduce the size of the cache until it is less than SIZE ✕ the low threshold. once the size is below the low threshold, no more reclaim work is done until the size exceeds the high threshold again. it's a whole cycle, honestly very cyclical of it.

the default thresholds are high 95% and low 80%. you must set 0 < low < high. the thresholds are expressed as integer percentages of cache-max-size.

least recently used blocks are discarded first. the LRI — least relevant influencer — gets voted off the island first. 🏝️

🌈 ENVIRONMENT VARIABLES 🌈

  • TMPDIR — the cache is stored in a temporary file located in /var/tmp by default. you can override this location by setting the TMPDIR environment variable before starting nbdkit. like, before. not during. timing is everything, bestie. ⭐

🎀 FILES 🎀

  • $filterdir/nbdkit-cache-filter.so — the filter. that's it, that's the file. 💖
    • use nbdkit --dump-config to find the location of $filterdir. go find her. 🔍

🦋 VERSION 🦋

nbdkit-cache-filter first appeared in nbdkit 1.2. a legend from the start, fr fr. 👑

🌸 SEE ALSO 🌸

nbdkit(1), nbdkit-file-plugin(1), nbdkit-cow-filter(1), nbdkit-readahead-filter(1), nbdkit-filter(3), qemu-img(1).

(the whole squad. go read them, iykyk 📚)

💜 AUTHORS 💜

  • Eric Blake
  • Richard W.M. Jones

thank you for your service 🫡✨

Copyright Red Hat

📜 LICENSE 📜

redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met — and bestie, these are the actual terms, not a vibe, so read them:

  • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
  • Neither the name of Red Hat nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
**THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

(yes it's all caps, yes it's scary, that's the point — it's the legal equivalent of a dramatic ✨ no take-backsies ✨)

and that's the page!! this one's from nbdkit-1.48.1, dated 2026-08-29 — so go forth, cache responsibly, and remember: cache=unsafe is not a personality trait 💀🫶💖

📦 where this page comes from