Cache::BDB(3pm)

📦
version 0.04 · 2 versions
, in ubuntu jammy, ubuntu noble

💖 Cache::BDB — bestie edition 💖

okay so this is the Cache::BDB manual page, straight out of User Contributed Perl Documentation (the 3pm kind, meaning it's a Perl module page, iykyk). Sit down, grab a strawberry, we're doing caching today. 🍓

✨ NAME ✨

Cache::BDB — it's an object caching wrapper around BerkeleyDB. that's the whole vibe. a lil cozy blanket of an API wrapped around the raw BerkeleyDB beast. 🧚

🌸 SYNOPSIS 🌸

here's the whole thing, no edits, no cap:

use Cache::BDB;
my %options = (
   cache_root => "/tmp/caches",
   namespace => "Some::Namespace",
   default_expires_in => 300, # seconds
);

my $cache = Cache::BDB->new(%options);

#
# [myshellprompt:~]$ find /tmp/caches
# /tmp/caches/Some::Namespace/
# /tmp/caches/Some::Namespace/Some::Namespace.db
# /tmp/caches/Some::Namespace/__db.001
# /tmp/caches/Some::Namespace/__db.002
# /tmp/caches/Some::Namespace/__db.003
#

$cache->namespace(); # returns "Some::Namespace", read only
$cache->default_expires_in(); # returns 300
$cache->default_expires_in(600); # change it to 600

$cache->set(1, \%some_hash);
$cache->set('foo', 'bar');
$cache->set(20, $obj, 10);

$cache->add(21, 'whatever'); # works, nothing with the key '21' set yet.
$cache->add(21, 'coffeepot'); # fails, can only add() something that hasn't
                              # yet been set

$cache->replace(21, 'shoelace'); # replaces the data 'whatever' with
                                 # 'shoelace'
$cache->replace(7, 'tattoo'); # fails key/value pair was never set() or
                              # add()ed previously

my $h = $cache->get(1); # $h and \%some_hash contain the same data
my $bar = $cache->get('foo'); # $bar eq 'bar';
my $obj = $cache->get(20); # returns the blessed object

$cache->count() == 3;
# assuming 10 seconds has passed ...
$cache->is_expired(20); # returns true ..
$cache->purge();
$cache->get(20); # returns undef
$cache->count() == 2;

my $hr = $cache->get_bulk();

# $hr = {1  => {contents_of => '%some_hash'},
#        21 => 'shoelace' };

$cache->close(); # close the cache object

so like, you make a cache, you set stuff, you get stuff, stuff expires, you purge, you close. that's the whole arc. main character energy from start to finish. 👑

💜 DESCRIPTION 💜

so this module is a caching layer around BerkeleyDB, and its whole job is object persistence. it gives you the basic methods you need to add, retrieve, and remove objects — the holy trinity, basically. 🙏

the big flex over other caching modules? performance. like, that's the point.

the author tried to stick to a Cache::Cache-like interface as much as humanly possible, though ngl it may differ here and there. close enough, bestie. 💅

🦋 DEPENDENCIES 🦋

okay so the author has been developing against a very recent version of Berkeley DB (v4.4.20) and BerkeleyDB (v0.27). ✨

they're pretty sure most of the functionality the module relies on exists in Berkeley DB version 3 and higher, but so far they haven't actually tested with older versions. they're open to making version specific concessions if necessary. 🙈

and the advice, straight up: if it's at all possible, upgrade both Berkeley DB and BerkeleyDB to their latest respective versions. do it do it do it. 🔥

also important: Cache::BDB currently serializes everything it stores with Storable. everything. all of it. that's the rule.

🎀 PERFORMANCE 🎀

the intent of this module is to supply great performance with a reasonably feature rich API. okay but let's be real for a second — there is no way this module can compete with, say, using BerkeleyDB directly. if you don't need any kind of expiration, automatic purging, etc, then going direct will more than likely be much faster. periodt. 💅

if you'd like to compare the speed of some other caching modules, go peek at http://cpan.robm.fastmail.fm/cache_perf.html. the author included a patch which adds Cache::BDB to that benchmark, so you can watch it run. cute. 🌸

🔒 LOCKING 🔒

bestie this part is important, don't skip it

every single Cache::BDB environment is opened with the DB_INIT_CDB flag. this enables multiple-reader/single-writer locking, handled entirely by the Berkeley DB internals — at either the database or environment level, whichever. for more info on what that means for locking, go read http://www.sleepycat.com/docs/ref/cam/intro.html.

Important: it is a bad idea to share a single Cache::BDB object across multiple processes or threads. doing that is bound to cause you pain. 😭 don't do it. instead, have your thread/process instantiate its own Cache::BDB object. and here's the slay part — it is safe to have them all pointing at the same cache file. so like, separate objects, same file. everyone gets their own. 🫶

🌈 CACHE FILES 🌈

so for every new Cache::BDB object, a Berkeley DB Environment gets created (or reused, if it already exists). which means even for a single cache object, you're getting at least 4 files — three for the environment and at least one for the actual data in the cache. that's just how it is, bestie. 💀

it's possible for multiple cache database files to share a single environment, and it's also possible for multiple cache databases to share a single database file. so there's flexibility there. 💅

for a quick look at what you're likeley to find on the filesystem for a cache, check the SYNOPSIS above. 👆

and here's the thing: Cache::BDB uses BerkeleyDB exclusively with regard to files. so if you have questions about what's in those files, you might wanna familiarize yourself further with Berkeley DB. that's a you-and-Berkeley-DB conversation, not a me conversation. 🙈

okay that's part 1, more is coming, don't go anywhere bestie 💖

🌸 USAGE 🌸

✨ new(%options) ✨

okay bestie, this is where you tell the cache how to live its life. every one of these is a key in the hash you pass in 💅

  • cache_root — the top-level directory where your cache and all its lil friends get stored. this one is required, no skips. heads up tho: Cache::BDB rocks a BerkeleyDB environment object, so it's gonna write more than one file per cache. it's giving "we travel in a group" 🫶
  • cache_file — if you wanna tell Cache::BDB exactly which file to use for your cache, say it here. this parameter is required if you're planning to use the env_lock option and/or if you want multiple logical databases (namespaces) chilling in one single physical file. if you leave it out, Cache::BDB will create its database file using the namespace. and psst — cache_file should be relative to your cache_root, not fully-qualified, i.e.
my %options = ( cache_root => '/some/location/for/caching/',
                cache_file => 'whatever.db',
                namespace  => 'MyObjects');

this hands you, among other files, /some/location/for/caching/whatever.db. your logical database inside whatever.db will be named with MyObject. and if you then instantiated another Cache::BDB like:

my %options = ( cache_root => '/some/location/for/caching/',
                cache_file => 'whatever.db',
                namespace  => 'MyOtherObjects');

boom, now you've got two logical caches in one physical file. totally fine, but see namespace below for a better idea 👀

  • namespace — your namespace tells Cache::BDB where to stash cache data under the cache_root when no cache_file is specified, or what to name the database in the multi-database file when cache_file is specified. required parameter, periodt. for clarity, it might be best to instantiate Cache::BDB objects like so:
my $namespace = 'MyObjects';
my %options = ( cache_root => "/some/location/for/caching/$namespace",
                namespace => $namespace );

unlike the examples up there under cache_file, this lets you park a single cache type in its own directory, which gives you way more flexibility to nuke it wholesale or move things around a little 🎀

  • type — Cache::BDB lets you pick which flavor of Berkeley DB storage mechanism to use. your choices are Hash, Btree, and Recno. Queue isn't supported. ngl the three supported types haven't been tested extensively. the default, if unspecified, is Btree, and that's probably good enough for most applications. note that if a cache is created as one type it must remain that type. if you instantiate a Cache::BDB object with one type (or use the default), and then try to connect to the same cache with a newly instantiated object using a different type, you'll get a warning, and Cache::BDB will be nice and connect you to the cache with its original type. 💀

Important: up until Berkeley DB 4.4.x, it has not been possible to shrink the physical size of a database file, which means that, technically, your cache files will never get smaller even if you delete everything from them. HOWEVER, with 4.4.x this functionality is now possible but it will only work with the Btree type. as soon as this is available in the BerkeleyDB.pm wrapper (soon, I'm told), a version will be released with some options to allow this. point being: this may be a good reason to stick with Btree. 🔥

for more info, see http://www.sleepycat.com/docs/ref/am_conf/intro.html.

  • env_lock — if multiple databases (same or different files) are opened using the same Berkeley DB environment, it's possible to turn on environment level locking rather than file level locking. this may be advantageous if you have two separate but related caches. by passing in the env_lock parameter with any true value, the environment will be created in such a way that any databases created under its control will all lock whenever Berkeley DB attempts a read/write lock. this flag must be specified for every database opened under this environment. note: this is very untested in Cache::BDB, and idk how necessary it even is 🙈
  • default_expires_in — time (in seconds) that cached objects should live. if set to 0, objects never expire. see set to enable a per-object value.
  • auto_purge_interval — time (in seconds) that the cached objects will be purged by one or both of the auto_purge types (get/set). if set to 0, auto purge is disabled. note, of course, that objects won't actually be purged until some event actually takes place that will call purge (set or get), so if this is set to 300 but no gets or sets are called for more than 300 seconds, the items haven't actually been purged yet. it's giving "the timer's running but nobody's home" 😭
  • auto_purge_on_set — if this item is true and auto_purge_interval is greater than 0, calling the set method will first purge any expired records from the cache.
  • auto_purge_on_get — if this item is true and auto_purge_interval is greater than 0, calling the get method will first purge any expired records from the cache.
  • purge_on_init — if set to a true value, purge will be called before the constructor returns.
  • purge_on_destroy — if set to a true value, purge will be called before the object goes out of scope.
  • clear_on_init — if set to a true value, clear will be called before the constructor returns.
  • clear_on_destroy — if set to a true value, clear will be called before the object goes out of scope.
  • disable_compact — disable database compactions for clear, purge, delete and remove methods. see DATABASE SIZE below for more info on database compaction.
  • disable_auto_purge — as a courtesy, Cache::BDB will automatically remove() any expired cache item you get() before returning undef. this is handy if you don't feel the need to do a lot of explicit cache purging, but if you only want purge, remove, delete or clear to actually delete cache items, you can disable this functionality by passing in disable_auto_purge with any true value.

close()

explicitly close the connection to the cache. genuinely a good idea 🫶 essentially the same as undef'ing the object (it explicitly calls DESTROY).

namespace()

this read-only method returns the namespace that the cache object is currently associated with. ✨

auto_purge_interval($seconds)

set/get the length of time (in seconds) that the cache object will wait before calling one or both of the auto_purge methods. if set to 0, automatic purging is disabled.

auto_purge_on_set(1/0)

enable/disable auto purge when set is called.

auto_purge_on_get(1/0)

enable/disable auto purge when get is called.

set($key, $value, [$seconds])

store an item ($value) with the associated $key. time to live (in seconds) can optionally be set with a third argument. returns true on success. 💖

add($key, $value, [$seconds])

only set in the cache if the key doesn't already exist. no take-backs, no overwrites 👑

replace($key, $value, [$seconds])

only set in the cache if the key does exist.

get($key)

locate and return the data associated with $key. returns the object associated with $key or undef if the data doesn't exist. if auto_purge_on_get is enabled, the cache will be purged before attempting to locate the item.

get_bulk()

returns a hash reference containing every unexpired item from the cache, key'ed on their cache id. this can be useful if your keys aren't always available, or if you just wanna use the cache as a convenient way to dump data in chunks. 🍓

the result looks something like this:

my $h = $cache->get_bulk();

# $h = { 123 => "bird and bee",
#        456 => "monkeys with sticks",
#        789 => "take whats mine",
#      };

remove($key)

removes the cache element specified by $key if it exists. returns true for success.

delete($key)

same as remove().

clear()

completely clear out the cache and compact the underlying database. returns the number of cached items removed. it's giving spring cleaning 🧹✨

count()

returns the number of items in the cache.

size()

return the size (in bytes) of all the cached items. this call relies on the availability of Devel::Size. if it's not found, you'll get a warning and size() will simply return 0. currently the size is calculated every single time this is called by using Devel::Size::total_size, so it may be expensive for large caches 😭. in the future, size-aware options and functionality may be available, but for now you'll need to implement this outside of Cache::BDB if you need it.

purge()

purge expired items from the cache. returns the number of items purged.

is_expired($key)

returns true if the data pointed to by $key is expired based on its stored expiration time. returns false if the data isn't expired or if the data doesn't exist. (yes, "not there" and "not expired" both read as false, bestie — that's just the vibe check it does 🙈)

💾 DATABASE SIZE 💾

(peep the lore at http://www.sleepycat.com/docs/ref/am_misc/diskspace.html if you wanna go deep)

Okay bestie, story time, because this one is lowkey important. Before Berkeley DB release 4.4, it was literally not possible to give freed space back to the database file. So like… no matter how many items you delete, your file just keeps its size, and keeps growing as you add more. The only way to shrink it back down was to dump the database to a file and reload it into a brand new database file. Which, yes, is giving "moving apartments just to get rid of one chair." 🙈

Whether that's a problem for your application is up to you, but keep in mind your cache will keep getting bigger, and, for example, your operating system may have a maximum file size limit. So like, vibe check your filesystem before you rawdog a million deletes. 💀

Now in 4.4, Sleepycat introduced the ability to free unused space. BerkeleyDB 0.29 exposes this functionality in the perl wrapper. So if you're on these versions or better and you've chosen the Btree database type (which is the default for Cache::BDB, bestie, you probably have it), your caches will automatically be compacted when items are purged, removed/deleted, or if clear is called. Slay. ✨

Want to turn that off? Initialize your Cache::BDB object with the disable_compact parameter set to any true value. Easy.

In my tests so far, however, database compaction does not appear to affect performance significantly, and may save you from a headache down the road. 👑

👑 AUTHOR 👑

Josh Rotenberg, <joshrotenberg at gmail.com>

📝 TODO 📝

The to-do list, aka the "I'll do it later" syllabus:

  • Make data storage scheme configurable (Storable, YAML, Data::Dumper, or callback based)
  • Split storage between meta and data for faster operations on meta data.
  • Add some size/count aware features.
  • Create some examples.
  • Fix fork()'ing tests.

manifesting all of these fr fr 🕯️

🐛 BUGS 🐛

Found a bug? Want a feature? Please report any bugs or feature requests to bug-cache-bdb at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Cache-BDB. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes. It's giving accountability, and honestly? We love that. 💅

🫶 SUPPORT 🫶

You can find documentation for this module with the perldoc command.

perldoc Cache::BDB

You can also look for information at:

  • AnnoCPAN: Annotated CPAN documentation

http://annocpan.org/dist/Cache-BDB

  • CPAN Ratings

http://cpanratings.perl.org/d/Cache-BDB

  • RT: CPAN's request tracker

http://rt.cpan.org/NoAuth/Bugs.html?Dist=Cache-BDB

  • Search CPAN

http://search.cpan.org/dist/Cache-BDB

Four links, four vibes, one very well-documented module. ✨

🌸 SEE ALSO 🌸

BerkeleyDB

💖 ACKNOWLEDGEMENTS 💖

Baldur Kristinsson Sandy Jensen

the real ones, no cap 🫡

Copyright 2006 Josh Rotenberg, all rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

And that's a wrap on the whole page, bestie!! 🎀 This doc's source came from perl v5.28.0, dated 2018-09-22, and the page itself is Cache::BDB(3pm) — so like, that's the receipts, go forth and cache responsibly. 💜🦋

📦 where this page comes from