vmod_xkey(3)

💖 NAME 💖

vmod_xkey — Surrogate keys support for Varnish Cache 🎀

---

🌸 SYNOPSIS 🌸

import xkey [as name] [from "path"]

INT purge(STRING keys)

INT softpurge(STRING keys)

---

🦋 DESCRIPTION 🦋

PLEASE NOTE bestie, this VMOD is currently in maintenance mode and it has known performance issues, especially around scalability, and those are not getting fixed in this implementation. If you want something more robust, with proven scalability, improved syntax support and actual proper softpurge functionality, go check out yKey, which comes with Varnish Enterprise. 👑

This vmod adds secondary hashes to objects, which lets you do fast purging on every object that has that hash key. It's giving "tag" energy, fr fr.

You can use it to express relationships between things, kinda like a "tag". Then you clear out every object that has that tag set. Two iconic use cases: news sites, where you might add all the stories mentioned on a particular page by article ID, letting each article that gets referenced create an xkey header. 📰

Same vibe for an e-commerce site, where various SKUs are often referenced on a page. 🛍️

Hash keys are specified in the xkey response header. Multiple keys can go on one header line, with spaces and/or commas as separators. Or you can just send them in multiple xkey response headers. Both work, both slay. ✨

Preferably the secondary hash keys are set from the backend application, but the header can also be set from VCL in vcl_backend_response.

VCL example:

vcl 4.0;
import xkey;

backend default { .host = "192.0.2.11"; .port = "8080"; }

acl purgers {
    "203.0.113.0"/24;
}

sub vcl_recv {
    if (req.method == "PURGE") {
        if (client.ip !~ purgers) {
            return (synth(403, "Forbidden"));
        }
        if (req.http.xkey) {
            set req.http.n-gone = xkey.purge(req.http.xkey);
            # or: set req.http.n-gone = xkey.softpurge(req.http.xkey)

            return (synth(200, "Invalidated "+req.http.n-gone+" objects"));
        } else {
            return (purge);
        }
    }
}

✨ Example ✨

So on an e-commerce site, the backend application issues an xkey header for every product that's referenced on that page. So the header for a certain page might look like this:

HTTP/1.1 OK
Server: Apache/2.2.15
xkey: 8155054
xkey: 166412
xkey: 234323

Alternatively you can use a single header with space-separated values instead, like xkey: 8155054 166412 234323.

This needs a bit of VCL in place. The VCL is up above, go look. 👆

Then, to keep the web in sync with the database, you set up a trigger in the database. When an SKU gets updated, that triggers an HTTP request toward the Varnish server, clearing out every object with the matching xkey header:

PURGE / HTTP/1.1
Host: www.example.com
xkey: 166412

Several xkey-purge headers are also supported, just like in the response example above, and here too you can use a single header with space-separated values, like xkey-purge: 166412 234323.

Unlike the xkey header for responses, the purge header is fully configurable by adjusting the name of the header in the VCL example above.

⚠️ Note the xkey header. It is probably a good idea to protect this with an ACL so random people from the Internet cannot purge your cache. Protect your peace, bestie. 🔒

Varnish will find the objects and clear them out, responding with:

HTTP/1.1 200 Purged
Date: Thu, 24 Apr 2014 17:08:28 GMT
X-Varnish: 1990228115
Via: 1.1 Varnish

The objects are now cleared. Periodt. 💅

---

🎀 INT purge(STRING keys) 🎀

Description

Purges all objects hashed on any key found in the keys argument. Returns the number of objects that were purged.

The keys may contain a list of space-separated ids.

---

🎀 INT softpurge(STRING keys) 🎀

Description

Performs a "soft purge" for all objects hashed on any key found in the keys argument. Returns the number of objects that were purged.

A softpurge differs from a regular purge in that it resets an object's TTL but keeps it available for grace mode and conditional requests for the rest of its configured grace and keep time. It's not gone, it's just... resting. 😌

---

📊 Counters 📊

varnish_vsc_begin:: xkey

XKEY – xkey Counters

Metrics from vmod_xkey

• g_keys – gauge - info — Number of surrogate keys

Number of surrogate keys in use. Increases after a request that includes a new key in the xkey header. Decreases when a key is purged or when all cache objects associated with a key expire.

• g_hashhead_bytes – gauge - debug — Bytes used by all xkey_hashhead objects

Total bytes used by hashhead objects. Tracks linearly with the number of surrogate keys in use.

• g_ochead_bytes – gauge - debug — Bytes used by all xkey_ochead objects

Total bytes used by ochead objects. Increases when an object is added to a key or a key is added to an object. Decreases when the relationship is removed.

• g_oc_bytes – gauge - debug — Bytes used by all xkey_oc objects

Total bytes used by oc objects. Tracks linearly with the number of cached objects that are referenced by surrogate keys.

• g_bytes – gauge - info — Bytes used by xkeys

Current number of bytes used by xkeys and their references to the object cache.

---

and that's the whole page, bestie 🫶 this was vmod_xkey, page 3 of the Library Functions Manual, and yes she ate. now go purge something responsibly (with an ACL, please 💜) 🌈✨🎀