vmod_re(3)

📦

version 0.0~git20260528 · 2 versions

0.0~git20260528 · ubuntu stonking (latest)
0.0~git20250531 · ubuntu resolute

💖 NAME 💖

vmod_re — the Vinyl Cache Module (VMOD) that does regular expression matching with subexpression capture. Fancy words, but bestie, it's just "match text with a pattern and grab the bits you want" 💅 (the manual page lives under vmod_re(3), Library Functions Manual, iykyk)

✨ SYNOPSIS ✨

import re;

# object interface
new <obj> = re.regex(STRING [, INT limit] [, INT limit_recursion]
                     [, BOOL forbody])
BOOL <obj>.match(STRING [, INT limit] [, INT limit_recursion])
STRING <obj>.backref(INT [, STRING fallback])
BOOL <obj>.match_body(req_body | bereq_body | resp_body
                      [, INT limit] [, INT limit_recursion])

# Iterators
BOOL <obj>.foreach(STRING, SUB, [, INT limit] [, INT limit_recursion])
BOOL <obj>.foreach_body(req_body | bereq_body | resp_body, SUB,
                        [, INT limit] [, INT limit_recursion])

# filter interface (includes all of the above)
new <obj> = re.regex(STRING [, INT limit] [, INT limit_recursion]
                     , forbody=true)
<obj>.substitute_match(INT, STRING)
set [be]resp.filters = "<obj>"

# function interface
BOOL re.match_dyn(STRING [, INT limit] [, INT limit_recursion])
STRING re.backref_dyn(INT [, STRING fallback])

STRING re.version()

🎀 DESCRIPTION 🎀

Okay so. This is a Vinyl Cache Module (aka a VMOD), and its whole thing is: matching strings against regular expressions, and then pulling out the captured substrings once you've got a match. That's it. That's the vibe. 💖

The matching here behaves exactly the same as VCL's infix operator ~. So why does this even exist, you ask? Because getting backreference captures in plain VCL means writing a whole verbose, suboptimal little mess with the regsub() function (https://vinyl-cache.org/docs/trunk/reference/vcl.html#regsub-str-regex-sub) or the regsuball() function (https://vinyl-cache.org/docs/trunk/reference/vcl.html#regsuball-str-regex-sub). Like, it works, but it's giving ✨clunky✨.

Here's the classic offender — this super common VCL idiom grabs a run of digits that comes after the substring "bar" in one request header, and drops it into another one:

sub vcl_recv {
        if (req.http.Foo ~ "bar\d+")) {
           set req.http.Baz = regsub(req.http.Foo,
                                     "^.*bar(\d+).*$", "\1");

        }
}

The problem? It runs the regex twice whenever there's a match, and the second run is less efficient than the first one (because to do the replace it has to match the ENTIRE string while also capturing a substring). Plus it's just... cumbersome. We hate that for us. 😤

So here's the same thing with the VMOD and honestly? iconic:

import re;

sub vcl_init {
        new myregex = re.regex("bar(\d+)");
}

sub vcl_recv {
        if (myregex.match(req.http.Foo)) {
           set req.http.Baz = myregex.backref(1);
        }
}

For an example on body matching, go peek at xregex.match_body().

Here's how the whole scheme works: the object gets created at VCL initialization, and the regex you hand it contains the capture expression — meaning it only describes the substring that needs to be matched. Then when a match succeeds via match or match_body, you can pull a captured string out with the backref method. 🫶

And a detail that actually matters a lot: calls to backref refer back to the most recent call to match or match_body for the same object in the same task scope — that is, in the same client or backend context. So for example, if match gets called on an object inside one of the vcl_backend_* subroutines and returns true, then subsequent backref calls in that same backend scope extract substrings from the matched substring. But if the match was unsuccessful, all back references get cleared. Poof. Gone. 🙈

Oh and! If you set the asfilter parameter to true, a regex object can also be configured to add a filter that performs substitutions on bodies. See xregex.substitute_match() for the details and examples. 👑

The VMOD also supports dynamic regex matching with the match_dyn and backref_dyn functions:

import re;

sub vcl_backend_response {
        if (re.match_dyn(beresp.http.Bar + "(\d+)",
                              req.http.Foo)) {
           set beresp.http.Baz = re.backref_dyn(1);
        }
}

With match_dyn, the regex in the first argument gets compiled right when it's called, and then matched against the string in the second argument. After that, subsequent calls to backref_dyn extract substrings from the matched string for the most recent successful match_dyn call in the same task scope.

Same deal as with the constructor: the regex argument to match_dyn should contain any capturing expressions you'll need for the backref_dyn calls. 🦋

So match_dyn is what lets you construct regexen whose contents aren't fully known until runtime. BUT — and this is the tea — match is more efficient, because it re-uses the compiled expression obtained at VCL initialization. So if you're matching against a fixed pattern that never ever changes during the lifetime of VCL, use match. Periodt. 💅

🌸 new xregex = re.regex(STRING, INT limit, INT limit_recursion, BOOL forbody, BOOL asfilter) 🌸

new xregex = re.regex(STRING, INT limit, INT limit_recursion, BOOL forbody,
    BOOL asfilter)

new xregex = re.regex(
   STRING,
   INT limit=1000,
   INT limit_recursion=1000,
   BOOL forbody=0,
   BOOL asfilter=0
)

💖 Description 💖

Creates a regex object with the regular expression you give it. The expression gets compiled when the constructor is called. And yes, it should include any capturing parentheses you'll need later for extracting backreferences.

If the regular expression fails to compile, then the VCL load fails with an error message describing the problem. So no sneaky broken regexes sneaking into prod, bestie. 🙅

The optional parameters limit and limit_recursion are per-object defaults for the respective parameters of the xregex.match() method.

The optional parameter forbody is required if you plan to call the xregex.match_body() method on the object.

And if the optional asfilter parameter is true, the vmod registers itself as a Vinyl Fetch Processor (VFP) for use in beresp.filters (https://vinyl-cache.org/docs/trunk/reference/vcl-var.html#beresp-filters), and as a Vinyl Delivery Processor (VDP) for use in resp.filters (https://vinyl-cache.org/docs/trunk/reference/vcl-var.html#resp-filters). In this setup, the xregex.substitute_match() and xregex.substitute_all() methods can be used to define replacements for matches on the body. ⭐

✨ Example ✨

new myregex = re.regex("\bmax-age\s*=\s*(\d+)");

🌈 BOOL xregex.match(STRING, INT limit, INT limit_recursion) 🌈

BOOL xregex.match(STRING, INT limit=0, INT limit_recursion=0)

💖 Description 💖

Tells you whether the given string matches the regex that was compiled by the constructor. Functionally, it's equivalent to VCL's infix operator ~. Simple as that. 💅

The optional parameter limit restricts the number of internal matching function calls in a pcre_exec() execution — analogous to the vinyld pcre_match_limit parameter. And for the default value 0, the limit that was given to the constructor re.regex() gets used instead.

The optional parameter limit_recursion restricts the number of internal matching function recursions in a pcre_exec() execution — analogous to the vinyld pcre_match_limit_recursion parameter. And same as above: for the default value 0, the limit_recursion given to the constructor re.regex() is used. 🧚

✨ Example ✨

if (myregex.match(beresp.http.Surrogate-Control)) { # ...

🦋 BOOL xregex.foreach(STRING, SUB sub, INT limit, INT limit_recursion) 🦋

BOOL xregex.foreach(
      STRING,
      SUB sub,
      INT limit=0,
      INT limit_recursion=0
)

💖 Description 💖

Calls the subroutine sub as if xregex.match() was run for all the matches on the given string. If there are no matches, the subroutine is simply not called — no drama. And you can use xregex.backref() to retrieve the match constituents. 🎀

✨ Example: ✨

sub vcl_init {
        new myregex = re.regex("bar(\d+)");
}

sub myregex_collect {
        set resp.http.all += myregex.backref(0);
}

sub vcl_synth {
        unset resp.http.all;
        myregex.foreach(req.http.input, myregex_collect);
}

sub vcl_recv {
        return (synth(200));
}

Note — This is a toy example, okay? And if your actual goal is really just to collect all the matches, regsuball() (https://vinyl-cache.org/docs/trunk/reference/vcl.html#regsuball-str-regex-sub) is way more efficient. Just keeping it real. 💀

👑 BOOL xregex.match_body(ENUM which, INT limit, INT limit_recursion) 👑

BOOL xregex.match_body(
      ENUM {req_body, bereq_body, resp_body} which,
      INT limit=0,
      INT limit_recursion=0
)

💖 Description 💖

Just like xregex.match(), except it operates on the named body. 🫧

Heads up: for a regular expression to be used with this method, it has to be constructed with the forbody flag set in the re.regex() constructor. Calling this method when the flag was unset results in a VCL failure. Not a vibe. 😭

PCRE2 multi segment matching (https://pcre.org/current/doc/html/pcre2partial.html#SEC4) is used to implement this method, so that memory requirements stay lower. In particular — and this is the cool part — unlike implementations in other vmods, this implementation does not read the full body object into a contiguous memory region. It might, however, require as much temporary heap space as all the body segments which the match found by the pattern spans.

Under ideal conditions, when the pattern spans only a single segment of a cached object, the xregex.match_body() method does not create copies of the body data. 👑

When used with a req_body or bereq_body which argument, this method consumes the request body. So if it's going to be used again (say, to send it to a backend), it should first be cached by calling std.cache_req_body(<size>).

Lookarounds are not supported. Sorry babe. 🥲

✨ Example: ✨

sub vcl_init {
        new pattern = re.regex("(a|b)=([^&]*).*&(a|b)=([^&]*)",
            forbody=true);
}

sub vcl_recv {
        if (pattern.match_body(req_body)) {
                return (synth(42200));
        }
}

sub vcl_synth {
        if (resp.status == 42200) {
                set resp.http.n1 = pattern.backref(1, "");
                set resp.http.v1 = pattern.backref(2, "");
                set resp.http.n2 = pattern.backref(3, "");
                set resp.http.v2 = pattern.backref(4, "");
                set resp.body = "";
                return (deliver);
        }
}

# response contains first parameter named a or b from the body as n1,
# first value as v1, and the second parameter and value as n2
# and v2

💌 `BOOL xregex.foreach_body(ENUM which, SUB sub, INT limit, INT limit_recursion)`

BOOL xregex.foreach_body(
     ENUM {req_body, bereq_body, resp_body} which,
     SUB sub,
     INT limit=0,
     INT limit_recursion=0
)

Description ✨

Calls the subroutine sub as if xregex.match() had been run for all the matches on the body you named. If there are straight up no matches, bestie, the subroutine just doesn't get called at all. You can grab the match pieces with xregex.backref() 💅

Also peep xregex.match_body().

Example 🍓

# for key=value separated by &, collect two a and/or b key pairs
#
# sample output: a=1,b=22;b=333,a=4444;
#

sub vcl_init {
        new pattern = re.regex("(?:^|&)(a|b)=([^&]*).*?&(a|b)=([^&]*)",
            forbody=true);
}

sub collect {
        set resp.http.all +=
            pattern.backref(1) + "=" + pattern.backref(2) + "," +
            pattern.backref(3) + "=" + pattern.backref(4) + ";";
}

sub vcl_synth {
        unset resp.http.all;
        if (pattern.foreach_body(req_body, collect)) {
                set resp.status = 200;
        }
        return (deliver);
}

sub vcl_recv {
        return (synth(400));
}

🌸 `STRING xregex.backref(INT, STRING fallback)`

STRING xregex.backref(
      INT,
      STRING fallback="**BACKREF METHOD FAILED**"
)

Description 👑

Grabs the *n*th subexpression from the most recent successful call of the match method on this object, in the same task scope (so like, client context or backend context), or it hands you a fallback string if the extraction flops. backref 0 means the entire matched string, okurrr. So basically this behaves like the \n symbols in regsub() and regsuball(), and like those $1, $2 ... variables in Perl. Iconic behavior, honestly.

After unsuccessful matches, the fallback string is what you get for any call to backref. And the default value of fallback is "**BACKREF METHOD FAILED**".

The VCL infix operators ~ and !~ do not touch this method, and neither do the functions regsub() or regsuball(). Not them. 🙈

If backref gets called without any prior call to match for this object in the same task scope, then an error message gets emitted to the Vinyl log with the VCL_Error tag, and the fallback string is returned.

Lookarounds are not supported. Periodt.

Example 💖

set beresp.ttl = std.duration(myregex.backref(1, "120"), 120s);

🔥 `VOID xregex.substitute_match(INT, STRING)`

Description 💅

This method defines substitutions for regular expression replacement ("regsub") operations on HTTP bodies.

It can only be used on re.regex() objects that were initiated with the asfilter argument set to true — otherwise a VCL failure gets triggered, and not in a cute way. 😭

The INT argument says which match the substitution applies to: for 1 it hits the first match, for 2 the second, etc. A value of 0 defines the default substitution, which is applied when a specific substitution isn't defined. Negative values trigger a VCL failure.

If no substitution is defined for a match (and there's no default either), the matched sub-string is just left unchanged. No notes.

The STRING argument defines the substitution to apply, exactly like the sub (third) argument of the built-in VCL function regsub(): \0 (which you can also spell \&) is replaced with the entire matched string, and \n is replaced with the contents of subgroup n in the matched string.

To have any effect at all, the regex object has to be used as a fetch or delivery filter. That's the vibe check. ✅

Example 🦋

For occurrences of the string "reiher" in the response body, replace the first with "czapla", the second with "eier" and all the others with "heron". The response is returned uncompressed even if the client supported compression, because there currently is no gzip VDP in Vinyl Cache:

sub vcl_init {
    new reiher = re.regex("r(ei)h(er)", asfilter = true);
}
sub vcl_deliver {
    unset req.http.Accept-Encoding;
    set resp.filters += " reiher";
    reiher.substitute_match(1, "czapla");
    reiher.substitute_match(2, "\1\2");
    reiher.substitute_match(0, "heron");
}

🌈 `VOID xregex.substitute_all(STRING)`

Description ✨

This method tells the named filter object to replace all matches with the STRING argument.

It's a shorthand for calling:

xregex.clear_substitutions();
xregex.substitute_match(0, STRING);

See xregex.substitute_match() for when to use this method. 👀

🎀 `VOID xregex.clear_substitutions()`

Description 💜

This method clears all the previous substitution definions made through xregex.substitute_match() and xregex.substitute_all().

Lowkey it's not required — because VCL code could always be written sucht hat only one code patch ever calls xregex.substitute_match() and xregex.substitute_all() — but it's provided to allow for simpler VCL when you're handling exceptional cases. 🫶

See xregex.substitute_match() for when to use this method.

⭐ `BOOL match_dyn(STRING, STRING, INT limit, INT limit_recursion)`

BOOL match_dyn(
   STRING,
   STRING,
   INT limit=1000,
   INT limit_recursion=1000
)

Description 🌸

Compiles the regular expression given in the first argument, and determines whether it matches the string in the second argument.

If the regular expression fails to compile, then an error message describing the problem is emitted to the Vinyl Cache log with the tag VCL_Error, and match_dyn returns false.

For the parameters limit and limit_recursion, see xregex.match() — except that here there's no object to inherit defaults from. You're on your own, bestie. 💀

Example 🔥

if (re.match_dyn(req.http.Foo + "(\d+)", beresp.http.Bar)) { # ...

🧚 `STRING backref_dyn(INT, STRING fallback)`

STRING backref_dyn(
   INT,
   STRING fallback="**BACKREF FUNCTION FAILED**"
)

Description 💖

Similar to the backref method, this function extracts the *n*th subexpression of the most recent successful call of the match_dyn function in the same task scope, or a fallback string in case the extraction fails.

After unsuccessful matches, the fallback string is returned for any call to backref_dyn. The default value of fallback is "**BACKREF FUNCTION FAILED**".

If backref_dyn is called without any prior call to match_dyn in the same task scope, then a VCL_Error message is logged, and the fallback string is returned. Noted. 📝

👑 `STRING version()`

Description 🌟

Returns the version string for this vmod.

Example 🎀

set resp.http.X-re-version = re.version();

🌈 REQUIREMENTS

The VMOD requires Vinyl Cache since version 9 or the main branch. See the project repository for versions that are compatible with older Varnish Cache versions. iykyk 💅

💀 LIMITATIONS

The VMOD allocates memory for captured subexpressions from Vinyl Cache workspaces, whose sizes are determined by the runtime parameters workspace_backend, for calls within the vcl_backend_* subroutines, and workspace_client, for the other VCL subs. The VMOD copies the string to be matched into the workspace, if it's not already in the workspace, and also uses workspace to save data about backreferences.

For typical usage, the default workspace sizes are probably enough — but if you are matching against many, long strings in each client or backend context, you might need to increase the Vinyl Cache parameters for workspace sizes. If the VMOD cannot allocate enough workspace, then a VCL_error message is emitted, and the match methods as well as backref will fail. (And if you're just using the regexen for matching and not to capture backrefs, then you might as well just use the standard VCL operators ~ and !~, and save the workspace. Free real estate. 🫶)

backref can extract up to 10 subexpressions, in addition to the full expression indicated by backref 0. If a match or match_dyn operation would have resulted in more than 11 captures (10 substrings and the full string), then a VCL_Error message is emitted to the Vinyl Cache log, and the captures are limited to 11. 😭

🌟 SEE ALSO

vinyld(1)
vcl(7)
pcre(3)
source repository: <https://code.uplex.de/uplex-varnish/libvmod-re>

💌 COPYRIGHT

Copyright 2014-2023 UPLEX Nils Goroll Systemoptimierung
All rights reserved

This document is licensed under the same conditions as the libvmod-re
project. See LICENSE for details.

Authors: Geoffrey Simmons <geoffrey.simmons@uplex.de>
         Nils Goroll <nils.goroll@uplex.de>

okay that's the whole page, bestie, you ate it 💅✨ this was vmod_re(3) from the Vinyl Cache docs, and that footer date situation is whatever the source file says — shoutout to the version tag, we love her. bye mwah 🫶🎀

📦 where this page comes from

ubuntu stonking: libvmod-re 0.0~git20260528-1 · BSD-2-Clause
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-11-03: a language model's rewrite of the original, under its terms

vmod_re(3)

💖 NAME 💖

✨ SYNOPSIS ✨

🎀 DESCRIPTION 🎀

🌸 new xregex = re.regex(STRING, INT limit, INT limit_recursion, BOOL forbody, BOOL asfilter) 🌸

💖 Description 💖

✨ Example ✨

🌈 BOOL xregex.match(STRING, INT limit, INT limit_recursion) 🌈

💖 Description 💖

✨ Example ✨

🦋 BOOL xregex.foreach(STRING, SUB sub, INT limit, INT limit_recursion) 🦋

💖 Description 💖

✨ Example: ✨

👑 BOOL xregex.match_body(ENUM which, INT limit, INT limit_recursion) 👑

💖 Description 💖

✨ Example: ✨

💌 BOOL xregex.foreach_body(ENUM which, SUB sub, INT limit, INT limit_recursion)

Description ✨

Example 🍓

🌸 STRING xregex.backref(INT, STRING fallback)

Description 👑

Example 💖

🔥 VOID xregex.substitute_match(INT, STRING)

Description 💅

Example 🦋

🌈 VOID xregex.substitute_all(STRING)

Description ✨

🎀 VOID xregex.clear_substitutions()

Description 💜

⭐ BOOL match_dyn(STRING, STRING, INT limit, INT limit_recursion)

Description 🌸

Example 🔥

🧚 STRING backref_dyn(INT, STRING fallback)

Description 💖

👑 STRING version()

Description 🌟

Example 🎀

🌈 REQUIREMENTS

💀 LIMITATIONS

🌟 SEE ALSO

💌 COPYRIGHT

💌 `BOOL xregex.foreach_body(ENUM which, SUB sub, INT limit, INT limit_recursion)`

🌸 `STRING xregex.backref(INT, STRING fallback)`

🔥 `VOID xregex.substitute_match(INT, STRING)`

🌈 `VOID xregex.substitute_all(STRING)`

🎀 `VOID xregex.clear_substitutions()`

⭐ `BOOL match_dyn(STRING, STRING, INT limit, INT limit_recursion)`

🧚 `STRING backref_dyn(INT, STRING fallback)`

👑 `STRING version()`