PCREPRECOMPILE(3)

💖 PCREPRECOMPILE(3) — bestie edition 💖

okay so this is a page from the Library Functions Manual and it's literally about PCRE — that's Perl-compatible regular expressions — and specifically about saving and re-using precompiled patterns. no cap, this is the good stuff. let's go 🦋

✨ SAVING AND RE-USING PRECOMPILED PCRE PATTERNS ✨

okay imagine this: your app runs a huge number of regular expression patterns. compiling them every single time the app starts? exhausting. not the vibe. so instead you can store them in a precompiled form. 🎀

if you're not using any private character tables (peep the pcre_maketables() documentation for that whole situation), this is pretty straightforward. if you are using private tables, it gets a little bit more complicated. lowkey annoying but we move.

🚨 HOWEVER. if you're using the just-in-time optimization feature, it is not possible to save and reload the JIT data. periodt. that data stays behind.

if you save compiled patterns to a file, you can copy them to a different host and run them there. cute, right? but — if the two hosts have different endianness (byte order), you should run the pcre[16|32]_pattern_to_host_byte_order() function on the new host before trying to match the pattern. and if the matching functions detect a pattern with the wrong endianness, they return PCRE_ERROR_BADENDIANNESS. 💀

also, compiling regular expressions with one version of PCRE for use with a different version is not guaranteed to work and may cause crashes. and saving and restoring a compiled pattern loses any JIT optimization data. rip to the JIT data honestly. 🙈

🌸 SAVING A COMPILED PATTERN 🌸

so the value returned by pcre[16|32]_compile() points to a single block of memory that holds the compiled pattern and associated data. iconic.

you can find the length of this block in bytes by calling pcre[16|32]_fullinfo() with an argument of PCRE_INFO_SIZE. then you can save the data in any appropriate manner you like. main character energy, choose your own adventure. ✨

here's sample code for the 8-bit library that compiles a pattern and writes it to a file. it assumes the variable fd refers to a file that is open for output:

int erroroffset, rc, size;
char *error;
pcre *re;

re = pcre_compile("my pattern", 0, &error, &erroroffset, NULL);
if (re == NULL) { ... handle errors ... }
rc = pcre_fullinfo(re, NULL, PCRE_INFO_SIZE, &size);
if (rc < 0) { ... handle errors ... }
rc = fwrite(re, 1, size, fd);
if (rc != size) { ... handle errors ... }

in this example, the bytes that comprise the compiled pattern are copied exactly. and bestie, note this: this is binary data that may contain any of the 256 possible byte values. on systems that make a distinction between binary and non-binary data, be sure that the file is opened for binary output. 🫶

if you want to write more than one pattern to a file, you'll have to devise a way of separating them. for binary data, preceding each pattern with its length is probably the most straightforward approach. another possibility is to write out the data in hexadecimal instead of binary, one pattern to a line. chef's choice 💅

saving compiled patterns in a file is only one possible way of storing them for later use. they could equally well be saved in a database, or in the memory of some daemon process that passes them via sockets to the processes that want them. so many options, so much potential. 🌈

if the pattern has been studied, it is also possible to save the normal study data in a similar way to the compiled pattern itself. however — if the PCRE_STUDY_JIT_COMPILE was used, the just-in-time data that is created cannot be saved because it is too dependent on the current environment. 😭

when studying generates additional information, pcre[16|32]_study() returns a pointer to a pcre[16|32]_extra data block. its format is defined in the section on matching a pattern in the pcreapi documentation. the study_data field points to the binary study data, and this is what you must save (not the pcre[16|32]_extra block itself, okay? not that one). the length of the study data can be obtained by calling pcre[16|32]_fullinfo() with an argument of PCRE_INFO_STUDYSIZE.

⭐ and remember to check that pcre[16|32]_study() did return a non-NULL value before trying to save the study data. don't skip the vibe check.

🦋 RE-USING A PRECOMPILED PATTERN 🦋

re-using a precompiled pattern is straightforward. having reloaded it into main memory, called pcre[16|32]_pattern_to_host_byte_order() if necessary, you pass its pointer to pcre[16|32]_exec() or pcre[16|32]_dfa_exec() in the usual way. slay. ✨

however — if you passed a pointer to custom character tables when the pattern was compiled (that's the tableptr argument of pcre[16|32]_compile()), you must now pass a similar pointer to pcre[16|32]_exec() or pcre[16|32]_dfa_exec(), because the value saved with the compiled pattern will obviously be nonsense. a field in a pcre[16|32]_extra() block is used to pass this data, as described in the section on matching a pattern in the pcreapi documentation.

🚨 Warning: the tables that pcre_exec() and pcre_dfa_exec() use must be the same as those that were used when the pattern was compiled. if this is not the case, the behaviour is undefined. not "a little weird" — undefined. respect the tables bestie. 💀

if you did not provide custom character tables when the pattern was compiled, the pointer in the compiled pattern is NULL, which causes the matching functions to use PCRE's internal tables. so in that case you do not need to take any special action at run time. easy. 🎀

if you saved study data with the compiled pattern, you need to create your own pcre[16|32]_extra data block and set the study_data field to point to the reloaded study data. you must also set the PCRE_EXTRA_STUDY_DATA bit in the flags field to indicate that study data is present. then pass the pcre[16|32]_extra block to the matching function in the usual way. and if the pattern was studied for just-in-time optimization, that data cannot be saved, and so is lost by a save/restore cycle. gone. lost to the void. 🙈

💜 COMPATIBILITY WITH DIFFERENT PCRE RELEASES 💜

in general, it is safest to recompile all saved patterns when you update to a new PCRE release, though not all updates actually require this. play it safe, it's giving self-care. 💅

👑 AUTHOR 👑

Philip Hazel University Computing Service Cambridge CB2 3QH, England.

⭐ REVISION ⭐

Last updated: 12 November 2013 Copyright (c) 1997-2013 University of Cambridge.

okay that's the whole page!! 💖 thanks for coming to my TED talk. this was PCREPRECOMPILE(3) from the PCRE 8.34 era, last touched 12 November 2013 — so like, vintage, but still iconic. go save your patterns, hydrate, and remember: check that study() pointer before you save. mwah 🫶✨