Boulder::Store(3pm)

💾 Boulder::Store — the vibe ✨

okay so bestie, welcome to Boulder::Store, aka Simple persistent storage for Stone tag/value objects 🫶 she's a Perl module (the 3pm in Boulder::Store(3pm) is just her little manual-page badge, iykyk) and she is so ready to hold your Stones for you. literally obsessed. 💅

---

✨ SYNOPSIS ✨

Boulder:Store;

my $store=new Boulder::Store('test.db',1);
my $s = new Stone (Name=>'george',
                   Age=>23,
                   Sex=>M,
                   Address=>{
                          Street=>'29 Rockland drive',
                          Town=>'Fort Washington',
                          ZIP=>'77777'
                          }
                      );
$store->put($s);
$store->put(new Stone(Name=>'fred',
                      Age=>30,
                      Sex=>M,
                      Address=>{
                                  Street=>'19 Gravel Path',
                                  Town=>'Bedrock',
                                  ZIP=>'12345'},
                      Phone=>{
                                Day=>'111-1111',
                                Eve=>'222-2222'
                                }
                            ));
$store->put(new Stone(Name=>'andrew',
                      Age=>18,
                      Sex=>M));

$store->add_index('Name');

my $stone = $store->get(0);
print "name = ",$stone->Name;

---

🌸 DESCRIPTION 🌸

so Boulder::Store gives you persistent storage for Boulder objects using a simple DB_File implementation. no cap, it's simple on purpose. 🎀

to use her you need to have Berkeley db installed (she also answers to libdb, same girl different name 💁) plus the Perl DB_File module. if you don't already have Berkeley db, go peek at the DB_File package — it's got the details on how to get it. 🦋

she also comes with an unsophisticated query mechanism that takes advantage of indexes you specify. and bestie, the way that despite her lack of sophistication the query system is often very helpful? iconic. she's not fancy, she just works. periodt. ✨

---

👑 CLASS METHODS 👑

• $store = Boulder::Store->new("database/path",$writable) — the new() method creates a brand new Boulder::Store object and pairs her up with the database file you hand her in the first parameter. and yes, undef is a totally valid pathname — in that case all the methods still work, the data just isn't stored anywhere (ghost store, main character energy 🫥). the second parameter should be a true value if you want the database opened for writing; otherwise it's opened read only. 🔒

bc the underlying storage implementation is not multi-user, only one process can have the database for writing at a time. there's an fcntl()-based locking mechanism doing the heavy lifting, giving a process that has the database open for writing exclusive access to it. this also stops the database from being opened for reading while another process is writing to it — and that's a good thing, we love data integrity in this house. 💖 multiple processes can open the database read only at the same time, no drama.

physically, the data lives in a human-readable file with the extension ".data". cute and legible, we stan. 📄

---

🦋 OBJECT METHODS 🦋

• $stone = $store->read_record(@taglist) — the semantics of this call are exactly the same as in Boulder::Stream, fr fr. Stones come back in sequential order, starting with the first record. on top of their built-in tags, every stone you get back from this call carries an extra tag called "record_no" — that's the zero-based record number of the stone in the database. wanna start over from the beginning? use the reset() method. 🔁

if you call read_record() in an array context, it returns a list of all the stones in the database that contain one or more of the tags you provided. slay.

• $stone = $store->write_record($stone [,$index]) — same semantics as Boulder::Stream. a stone gets appended to the end of the database. if it works, this call returns the record number of the new entry. 🎉 and if you pass that optional second parameter, you get to control where the stone lands: a positive numeric index writes the stone into the database at that position, and a value of -1 uses the Stone's internal record number (if it has one) to figure out where to place it. 💅
• $stone = $store->get($record_no) — this is your random access to the database. give her a record number and this call returns the stone stored at that position. no waiting, just grab and go. ⚡
• $record_number = $store->put($stone,$record_no) — and this is the random write. give her a record number and this call stores the stone at the indicated position, replacing whatever was there before. out with the old, in with the new. 🙈

if you don't provide a record number, this call will look for a 'record_no' tag inside the stone itself and put it back in that position. so you can pull a stone out of the database, modify it, and put it right back without ever stressing about its record number. iconic behaviour honestly. if no record is found in the stone, then the effect is identical to write_record(). 🪄

the record number of the inserted stone is returned from this call, or -1 if an error occurred. 💀

• $store->delete($stone), Boulder::Store::delete($record_no) — these delete a stone from the database. you can hand her either the record number or a stone containing the 'record_no' tag.

⚠️ warning bestie: if the database is heavily indexed, deletes can be time-consuming — it requires the index to be brought back into synch. 😭

• $record_count = $store->length() — this returns the length of the database, in records. simple. clean. perfect. ✨
• $store->reset() — this resets the database, nullifying any queries in effect and causing read_record() to begin fetching stones from the first record again. fresh start energy. 🌈
• $store->query(%query_array) — this creates a query on the database, used for selecting stones in read_record(). the query is an associative array, and three types of key/value pairs are allowed:

(1) $index=>$value — tells Boulder::Store to look for stones containing the specified tags in which the tag's value (determined by the Stone index() method) exactly matches the provided value. example:

$db->query('STS.left_primer.length'=>30);

only the non-bracketed forms of the index string are allowed — and yeah, this is probably a bug... 🙈 if the tag path was declared to be an index, then this search will be fast. otherwise Boulder::Store has to iterate over every record in the database. 😮‍💨

(2) EVAL=>'expression' — tells Boulder::Store to look for stones in which the provided expression evaluates to true. when the expression is evaluated, the variable $s gets set to the current record's stone. and as a shortcut, you can use "<index.string>" as shorthand for "$s->index('index.string')". 🧚

(3) EVAL=>['expression1','expression2','expression3'...] — this lets you provide a whole bunch of expressions, and it's exactly equivalent to:

EVAL=>'(expression1) && (expression2) && (expression3)'

you can mix query types in the parameter you give to query(). for example, here's how to look up all stones in which the sex is male and the age is greater than 30:

$db->query('sex'=>'M',EVAL=>'<age> > 30');

when a query is in effect, read_record() returns only Stones that satisfy the query. in an array context, read_record() returns a list of all Stones that satisfy the query. and when no more satisfactory Stones are found, read_record() returns undef until a new query is entered or reset() is called. she's patient like that. 💜

• $store->add_index(@indices) — declare one or more tag paths to be part of a fast index. read_record() will take advantage of this record when processing queries. example:

$db->add_index('age','sex','person.pets');

you can add indexes any time you like — when the database is first created, or later. but there's a trade off: write_record(), put(), and other data-modifying calls will become slower as more indexes are added. 👑

the index is stored in an external file with the extension ".index". an index file is created even if you haven't indexed any tags. noted. 📝

• $store->reindex_all() — call this if the index gets screwed up (or lost). it rebuilds it from scratch. glow-up, but for indexes. ✨

---

😭 CAVEATS 😭

Boulder::Store makes heavy use of the flock() call in order to avoid corruption of DB_File databases when multiple processes try to write simultaneously.

⚠️ but here's the tea: flock() may not work correctly across NFS mounts, particularly on Linux machines that are not running the rpc.lockd daemon. please confirm that your flock() works across NFS before attempting to use Boulder::Store. and if the store.t test hangs during testing, this is the likely culprit. 💀

---

🎀 AUTHOR 🎀

Lincoln D. Stein <lstein@cshl.org>, Cold Spring Harbor Laboratory, Cold Spring Harbor, NY.

this module can be used and distributed on the same terms as Perl itself. 🫶

---

🌈 SEE ALSO 🌈

• Boulder
• Boulder::Stream
• Stone

---

and that's the whole page bestie!! 💖 this one shipped with perl v5.34.0 and is dated 2022-06-08 — so go forth, store your Stones, and remember: one writer at a time, and check your flock() across NFS. mwah. 💅✨🌸