nbdkit-curl-plugin(1)

📦

version 1.48.1 · 14 versions

💖 nbdkit-curl-plugin(1) 💖

okay so bestie, welcome to the nbdkit curl plugin manual page — she's part of the NBDKIT family and she is lowkey a whole icon. Let's get into it. 🦋

✨ NAME ✨

nbdkit-curl-plugin — the nbdkit curl plugin, serving up HTTP, FTP and a bunch of other protocols like it's nothing. 💅

🌸 SYNOPSIS 🌸

nbdkit -r curl [url=]http://example.com/disk.img

🎀 DESCRIPTION 🎀

nbdkit-curl-plugin is a plugin for nbdkit(1) that takes content served over HTTP, FTP, and honestly so much more, and turns it into a Network Block Device. It does this using a library called libcurl (you might know her as cURL) to read data from URLs. 🫶

The exact list of protocols that libcurl can handle? That depends entirely on how it was compiled, bestie. But most versions will handle HTTP, HTTPS, FTP, FTPS and more — peep curl -V to see what your build is serving. ✨

Note: this plugin supports writes. But for HTTP, you might not want nbdkit to be out here issuing PUT requests to the remote server (which, let's be real, probably doesn't understand them 💀). To force nbdkit to use a readonly connection, pass the -r flag. And bonus — using the -r flag also enables NBD multi-conn, which usually performs much better, as long as the client supports it. 🔥

Okay so real talk about the alternatives: even though this plugin can access SFTP (aka SSH) servers, it is much better to use nbdkit-ssh-plugin(1) instead. And yeah, this plugin can be used to access file:/// URLs, but you should really be using nbdkit-file-plugin(1) for that. Just trust me on this one. 💜

🌈 EXAMPLE 🌈

nbdkit -r curl http://example.com/disk.img

This serves the remote disk image as NBD on TCP port 10809. And if you wanna control which ports and protocols are used to serve NBD, go check out nbdkit(1) — she's got the whole rundown. 👑

✨ PARAMETERS ✨

okay bestie, this is the part where we go option by option, and there are a LOT of them, so grab a snack 🍓. Every single one is here, nothing skipped, no cap.

cainfo=FILENAME — (nbdkit ≥ 1.18)

Sets up the CA bundle for libcurl. Details live in CURLOPT_CAINFO(3).

Pass an empty string if you wanna not use the default certificate store that libcurl was compiled with. 💅

capath=PATH — (nbdkit ≥ 1.18)

Sets the CA certificates directory location for libcurl. More info in CURLOPT_CAPATH(3).

connections=N — (nbdkit ≥ 1.34)

Opens up to N connections to the web server. The default is 16. These connections get shared between all NBD clients, so if you're expecting a bunch of simultaneous NBD clients (or one single client flexing many multi-conn connections) you might wanna bump this up 👑

See "NBD CONNECTIONS AND CURL HANDLES" below.

cookie=COOKIE, cookie=+FILENAME, cookie=-, cookie=-FD — (nbdkit ≥ 1.12)

Sets a cookie in the request header when connecting to the remote server.

A typical example is literally this:

 cookie='vmware_soap_session="52a01262-bf93-ccce-d379-8dabb3e55560"'

This option can be used at most once. It only works for HTTP and HTTPS transports. To set multiple cookies you gotta concatenate them yourself, like:

 cookie='name1=content1; name2=content2'

See CURLOPT_COOKIE(3) for more info about this. The format is strict bestie — it must be "key=value", and each cookie separated by exactly "; " (semicolon and space). No freestyling 🙈

If the cookie is being used for authentication, then passing it on the command line is not secure on shared machines. Use the alternate "+FILENAME" syntax to pass it in a file, "-" to read the cookie interactively, or "-FD" to read it from a file descriptor. 🫶

cookiefile= — (nbdkit ≥ 1.26)

Enables cookie processing (eg. when following redirects), starting with an empty list of cookies. This is equivalent to calling CURLOPT_COOKIEFILE(3) with an empty string.

cookiefile=FILENAME — (nbdkit ≥ 1.26)

Enables cookie processing (eg. when following redirects), and prepopulates cookies from the given file. The file can contain cookies in any format supported by curl, see CURLOPT_COOKIEFILE(3). Cookies sent by the server are not saved when nbdkit exits. 💀

cookiejar=FILENAME — (nbdkit ≥ 1.26)

Enables cookie processing (eg. when following redirects), prepopulates cookies from the given file, and writes server cookies back to the file when the NBD handle is closed. The file can contain cookies in any format supported by curl, see CURLOPT_COOKIEJAR(3).

There is some advice on the internet telling you to set this to /dev/null, but you should not do this because it can corrupt /dev/null. 😭 If you don't want a cookiejar, just omit this option. If you want cookie processing without updating a permanent cookiejar, use the "cookiefile=" option instead.

cookie-script=SCRIPT, cookie-script-renew=SECS — (nbdkit ≥ 1.22, not Windows)

Runs "SCRIPT" (a command or shell script fragment) to generate the HTTP/HTTPS cookies. "cookie-script" cannot be used with "cookie". See "HEADER AND COOKIE SCRIPTS" below.

followlocation=false — (nbdkit ≥ 1.26)

Do not follow redirects from the server. The default is true (i.e. follow redirects).

You can follow redirects while dodging redirects to a less secure protocol (eg. HTTPS redirecting to FTP) by using the "protocols" parameter instead. iykyk 🔥

header=HEADER — (nbdkit ≥ 1.22)

For HTTP/HTTPS, sends a custom header, or removes a header that curl has added. To add a custom header:

 header='X-My-Name: John Doe'

To remove a header that curl has added, add the header followed by a colon and no value:

 header='User-Agent:'

To add a custom header that has no value, you have to use a semicolon instead of a colon. This adds an "X-Empty:" header with no value:

 header='X-Empty;'

See CURLOPT_HTTPHEADER(3). You can use this option multiple times to add several headers. 🎀

Note this sends the header in all requests, even when following a redirect, which can cause headers (eg. containing sensitive authorization information) to be sent to hosts other than the one originally requested. Be careful bestie.

header-script=SCRIPT, header-script-renew=SECS — (nbdkit ≥ 1.22, not Windows)

Runs "SCRIPT" (a command or shell script fragment) to generate the HTTP/HTTPS headers. "header-script" cannot be used with "header". See "HEADER AND COOKIE SCRIPTS" below.

http-version=none, http-version=1.0, http-version=1.1, http-version=2.0, http-version=2TLS, http-version=2-prior-knowledge, http-version=3, http-version=3only — (nbdkit ≥ 1.34)

Forces curl to use a particular HTTP protocol. The default is "none", meaning curl will negotiate the best protocol with the server. The other settings are mainly for testing. See CURLOPT_HTTP_VERSION(3) for details.

ipresolve=any, ipresolve=v4, ipresolve=v6 — (nbdkit ≥ 1.36)

Forces curl to use only IPv4 ("ipresolve=v4"), only IPv6 ("ipresolve=v6"), or any IP version your system supports ("ipresolve=any"). The default is "any". See CURLOPT_IPRESOLVE(3).

password=PASSWORD — sets the password to use when connecting to the remote server.

Note that passing this on the command line is not secure on shared machines. 🙈

password=- — asks for the password (interactively) when nbdkit starts up.
password=+FILENAME — reads the password from the named file. This is a secure method to supply a password, as long as you set the permissions on the file appropriately. 🔐
password=-FD — reads the password from file descriptor number "FD", inherited from the parent process when nbdkit starts up. This is also a secure method to supply a password.
protocols=PROTO,PROTO,... — (nbdkit ≥ 1.12)

Limits the protocols that are allowed in the URL. Use this option for extra security if the URL comes from an untrusted source and you wanna dodge security issues in the more obscure protocols that curl supports. (See qemu CVE-2013-0249 for an example of a security bug introduced by allowing unrestricted protocols 👀).

For example, if you only intend HTTP and HTTPS URLs to be used, then add this parameter: "protocols=http,https"

This restriction also applies if the plugin follows a redirect to another protocol (eg. you start with an "https://" URL which the server redirects to "ftp://"). To prevent redirects altogether, see the "followlocation" parameter.

The value of this parameter is a comma-separated list of protocols, for example "protocols=https", or "protocols=http,https", or "protocols=file,ftp,gopher". For a list of known protocols, see the libcurl manual (CURLOPT_PROTOCOLS_STR(3)), and in nbdkit ≥ 1.40 the output of:

 nbdkit curl --dump-plugin

The default is to allow any protocol.

proxy=PROXY — (nbdkit ≥ 1.20)

Sets the proxy. See CURLOPT_PROXY(3).

proxy-cainfo=FILENAME, proxy-capath=PATH — (nbdkit ≥ 1.44)

Sets the CA bundle and CA certificates directory location for the proxy. See CURLOPT_PROXY_CAINFO(3) and CURLOPT_PROXY_CAPATH(3) for more information.

proxy-password=PASSWORD, proxy-password=-, proxy-password=+FILENAME, proxy-password=-FD, proxy-user=USERNAME — (nbdkit ≥ 1.12)

Sets the proxy username and password. 💅

resolve=HOST:PORT:ADDRESS — provides custom host name to IP address resolution. You can supply this option as many times as you need. See CURLOPT_RESOLVE(3) for the full details of this option.
sslverify=false — don't verify the SSL certificate of the remote host. 🚨 (yes, that's the vibe, and yes, it's exactly what it sounds like)
ssl-cipher-list=CIPHER[:CIPHER...], ssl-version=default, ssl-version=tlsv1, ssl-version=sslv2, ssl-version=sslv3, ssl-version=tlsv1.0, ssl-version=tlsv1.1, ssl-version=tlsv1.2, ssl-version=tlsv1.3, ssl-version=max-default, ssl-version=max-tlsv1.0, ssl-version=max-tlsv1.1, ssl-version=max-tlsv1.2, ssl-version=max-tlsv1.3 — sets the SSL ciphers and TLS version. For further information see CURLOPT_SSL_CIPHER_LIST(3) and CURLOPT_SSLVERSION(3).
tcp-keepalive=true — (nbdkit ≥ 1.20)

Enables TCP keepalives. 🫶

tcp-nodelay=false — (nbdkit ≥ 1.20)

Enables Nagle's algorithm. Small writes on the network socket will not be sent immediately but will be held in a local buffer and coalesced if possible. This is more efficient for the network but can cause increased latency. 😭

The default (in libcurl ≥ 7.50.2) is that Nagle's algorithm is disabled, for better latency at the expense of network efficiency.

See CURLOPT_TCP_NODELAY(3).

timeout=SECS — sets the timeout for requests.
timeout=0 — uses the default libcurl timeout for requests.
tls13-ciphers=CIPHER[:CIPHER...] — selects TLSv1.3 ciphers available. See CURLOPT_TLS13_CIPHERS(3) and https://curl.se/docs/ssl-ciphers.html
unix-socket-path=PATH — (nbdkit ≥ 1.10)

Instead of using a TCP connection, connects to the server over the named Unix domain socket. See CURLOPT_UNIX_SOCKET_PATH(3). 🦋

[url=]URL — the URL of the remote disk image. This is passed to libcurl directly via CURLOPT_URL(3).

This parameter is required. 👑

The "url=" prefix may be omitted in most cases. See "Magic parameters" in nbdkit(1).

user=USERNAME — sets the username to use when connecting to the remote server. This may also be set in the URL (eg. "http://foo@example.com/disk.img")
user-agent=USER-AGENT — (nbdkit ≥ 1.22)

Sends a user-agent header when using HTTP or HTTPS. The default is no user-agent header. 🌸

🌀 NBD CONNECTIONS AND CURL HANDLES 🌀

ok bestie this is the glow-up history of how nbdkit talks to curl, and it's actually so important to understand because the rules changed:

nbdkit ≤ 1.32 ran the old simple model: every new NBD connection would spawn a brand new libcurl handle. But here's the thing — a libcurl handle keeps a lil cache of connections, so the number of HTTP connections ended up being a small multiple of the number of incoming NBD connections, and the total was not limited at all (assuming an http: or https: URL). chaos, honestly.
nbdkit 1.34 switched it up to a fixed pool of libcurl handles shared across all NBD connections. You control the max number of curl handles in the pool with the connections parameter (default 4). Since each curl handle still keeps a small cache of connections, the number of HTTP connections became a small multiple of the connections parameter. And heads up: if more than 4 incoming NBD connections show up, they will contend for the libcurl handles — unless you adjust connections. ✨
nbdkit ≥ 1.36 changed again (she keeps evolving 💅) to use a curl multi handle, aka libcurl-multi(3). Now connections controls the maximum number of HTTP connections made to the remote server (CURLMOPT_MAX_TOTAL_CONNECTIONS(3)). This is way more efficient, especially with HTTP/2 and HTTP/3, where each HTTP connection can hold a very large number of streams (typically up to 100) multiplexed over one connection. The default for connections got raised to 16. slay.

So the header and cookie parameters are for static headers and cookies that get used in every single HTTP/HTTPS request. Cute, but basic. The alternate header-script and cookie-script parameters let you run an external script or program to generate headers and/or cookies on the fly. This is genuinely iconic for accessing services that need an authorization token. 🎀

And because tokens expire (ugh, they always do), header-script-renew and cookie-script-renew let you renew the authorization token by rerunning the script periodically. obsessed.

Heads up bestie: header-script is incompatible with header, and cookie-script is incompatible with cookie. don't try to run both, she will not allow it.

✍️ Header script

The header script should print zero or more HTTP headers, each line of output in the same format as the header parameter. Whatever the script prints gets passed straight to CURLOPT_HTTPHEADER(3).

In this example, an imaginary web service needs authentication using a token fetched from a separate login server. The token expires after 60 seconds, so we also tell the plugin it must renew the token (by re-running the script) if more than 50 seconds have elapsed since the last request:

 nbdkit curl https://service.example.com/disk.img \
        header-script='
          printf "Authorization: Bearer "
          curl -s -X POST https://auth.example.com/login |
               jq -r .token
        ' \
        header-script-renew=50

The cookie script should print a single line in the same format as the cookie parameter. That line is passed to CURLOPT_COOKIE(3).

Inside header-script and cookie-script you get these shell variables, bestie:

$iteration — the number of times the script has been called. The first time the script is called, this contains 0.
$url — the URL as passed to the plugin.

🧚 Example: VMware ESXi cookies

VMware ESXi's web server can expose both VMDK and raw format disk images, but it requires you to log in using HTTP Basic Authentication. Now, you could use the user and password parameters to send HTTP Basic Authentication headers in every request — but tests have shown it's faster to accept the cookie the server returns and send that instead. (It's not clear why it's faster, but one theory is that VMware has to do a more expensive username and password check each time. 👀)

The web server can be accessed as below. Since the cookie expires after a certain period of time, we use cookie-script-renew, and because the server uses a self-signed certificate we must use --insecure and sslverify=false.

 SERVER=esx.example.com
 DCPATH=data
 DS=datastore1
 GUEST=guest-name
 URL="https://$SERVER/folder/$GUEST/$GUEST-flat.vmdk?dcPath=$DCPATH&dsName=$DS"

 nbdkit curl "$URL" \
        cookie-script='
            curl --head -s --insecure -u root:password "$url" |
                 sed -ne "{ s/^Set-Cookie: \([^;]*\);.*/\1/ip }"
        ' \
        cookie-script-renew=500 \
        sslverify=false

🐳 Example: Docker Hub authorization tokens

Accessing objects like container layers from Docker Hub requires you to first fetch an authorization token, even for anonymous access. These tokens expire after about 5 minutes (300 seconds) so they must be periodically renewed. 😭

You will need this authorization script (/tmp/auth.sh):

#!/bin/sh -
IMAGE=library/fedora
curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:$IMAGE:pull" |
     jq -r .token

You will also need this script to get the blobSum of the layer (/tmp/blobsum.sh):

#!/bin/sh -
TOKEN=`/tmp/auth.sh`
IMAGE=library/fedora
curl -s -X GET -H "Authorization: Bearer $TOKEN" \
     "https://registry-1.docker.io/v2/$IMAGE/manifests/latest" |
     jq -r '.fsLayers[0].blobSum'

Both scripts must be executable, and both can be run on their own to check they are working. To run nbdkit:

 IMAGE=library/fedora
 BLOBSUM=`/tmp/blobsum.sh`
 URL="https://registry-1.docker.io/v2/$IMAGE/blobs/$BLOBSUM"

 nbdkit curl "$URL" \
        header-script=' printf "Authorization: Bearer "; /tmp/auth.sh ' \
        header-script-renew=200 \
        --filter=gzip

Note that this exposes a tar file over NBD. See also nbdkit-tar-filter(1).

🔍 DEBUG FLAGS 🔍

-D curl.scripts=1 — prints out the headers and cookies generated by the header-script and cookie-script options. super useful when you're debugging those scripts, trust.
-D curl.times=1 — prints out additional info about the total time taken to do name resolution, connect to the remote server, etc. The info is printed in the debug output before nbdkit exits. The output will look like:

 nbdkit: debug: times (-D curl.times=1):
 nbdkit: debug: name resolution               :    0.128442 s
 nbdkit: debug: connection                    :    4.945213 s
 nbdkit: debug: SSL negotiation               :    4.291362 s
 nbdkit: debug: pretransfer                   :    0.104137 s
 nbdkit: debug: first byte received           :   56.115269 s
 nbdkit: debug: data transfer                 :  222.633831 s
 nbdkit: debug: redirection time              :    0.000000 s

The cumulative time taken to perform each step is shown (summed across all HTTP connections). The redirection time is the total time taken doing HTTP redirections. For further info see TIMES in curl_easy_getinfo(3).

-D curl.verbose=1 — enables very verbose curl debugging. See CURLOPT_VERBOSE(3). This is mainly useful if you suspect there's a bug inside libcurl itself. 🐛
-D curl.verbose.ids=1 — enhances -D curl.verbose=1 by printing connection and transfer IDs next to each debug message. As this has some overhead it is not enabled by default.

📁 FILES 📁

$plugindir/nbdkit-curl-plugin.so — the plugin itself, the moment ✨

Use nbdkit --dump-config to find the location of $plugindir.

⭐ VERSION ⭐

nbdkit-curl-plugin first appeared in nbdkit 1.2.

👑 AUTHORS 👑

Richard W.M. Jones

Parts derived from Alexander Graf's "QEMU Block driver for CURL images".

💜 COPYRIGHT 💜

📜 LICENSE 📜

ok so legally speaking (and I do read the fine print, bestie), redistribution and use in source and binary forms, with or without modification, are permitted provided that these conditions are met:

Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of Red Hat nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

And here's the big dramatic one — the whole "we are not liable, please read this twice" moment:

THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

and that's the whole page, bestie!! 🦋 this is nbdkit-curl-plugin from nbdkit-1.48.1, dated 2026-08-29 — go stream some disk images over HTTP, you absolute icon 💅✨

📦 where this page comes from

fedora 44: nbdkit-curl-plugin 1.48.1-1.fc44 · BSD-3-Clause
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-10-13: a language model's rewrite of the original, under its terms

nbdkit-curl-plugin(1)