Lucy::...Locking(3pm)

🎀 Lucy::...Locking 💅

okay bestie, so this page is from the User Contributed Perl Documentation crew (the manual name is literally Lucy::...Locking(3pm), iykyk), and it's about keeping your Lucy indexes safe when they live on a shared volume. buckle up 🦋

---

✨ NAME ✨

Lucy::Docs::FileLocking — the whole vibe here is managing indexes on shared volumes. that's it, that's the name 💖

---

🌸 DESCRIPTION 🌸

so normally? index locking is like, invisible. nobody thinks about it. exclusive write access gets controlled via lockfiles sitting inside the index directory, and problems only pop up if multiple processes try to grab the write lock at the same time. and search-time processes? they don't ordinarily need locking at all. not even a little. periodt 💅

BUT — on shared volumes the default locking mechanism just... fails. like, it stops working entirely, and manual intervention becomes necessary. no cap. 😭

🫶 the host id thing

both the read apps and the write apps that touch an index on a shared volume need to identify themselves with a unique "host" id — think hostname or ip address. why? because knowing the host id is what makes it possible to tell which lockfiles belong to other machines, and therefore must not be removed when the lockfile's pid number appears not to correspond to an active process. it's giving detective work 🔍

💀 at index-time

at index-time, the danger is this: multiple indexing processes from different machines that fail to specify a unique "host" id can delete each other's lockfiles and then attempt to modify the index at the same time. and that causes index corruption. literally the worst outcome. 🙈

the search-time problem? more complex. buckle in.

🧚 the deletion situation

once an index file is no longer listed in the most recent snapshot, Indexer attempts to delete it as part of a post-<lucy:Indexer.Commit> cleanup routine. but here's the tea: it's possible that at the exact moment an Indexer is deleting files it believes are no longer needed, a Searcher referencing an earlier snapshot is in fact using them. and the more often an index is either updated or searched, the more likely it is that this conflict will arise from time to time. so it's not rare-rare, it's just... lurking. ⭐

ordinarily though, the deletion attempts are not a problem:

• on a typical unix volume, the files get deleted in name only. any process holding an open filehandle against a given file keeps on having access, and the file won't actually get vaporized until the last filehandle is cleared. thanks to "delete on last close semantics", an Indexer can't truly delete the file out from underneath an active Searcher. iconic behaviour honestly 🥰
• on Windows, where file deletion fails whenever any process holds an open handle, the situation is different but still workable: Indexer just keeps retrying after each commit until deletion finally succeeds. persistent queen 💪
• on NFS, however, the system breaks — because NFS allows files to be deleted out from underneath active processes. should this happen, the unlucky read process will crash with a "Stale NFS filehandle" exception. oof. 😭

👑 the NFS exception

under normal circumstances, it is neither necessary nor desirable for IndexReaders to secure read locks against an index. but for NFS we have to make an exception.

LockFactory's <lucy:LockFactory.Make_Shared_Lock> method exists for exactly this reason: supplying an IndexManager instance to IndexReader's constructor activates an internal locking mechanism using <lucy:LockFactory.Make_Shared_Lock> which prevents concurrent indexing processes from deleting files that are needed by active readers. slay ✨

use Sys::Hostname qw( hostname );
my $hostname = hostname() or die "Can't get unique hostname";
my $manager = Lucy::Index::IndexManager->new( host => $hostname );

# Index time:
my $indexer = Lucy::Index::Indexer->new(
    index   => '/path/to/index',
    manager => $manager,
);

# Search time:
my $reader = Lucy::Index::IndexReader->open(
    index   => '/path/to/index',
    manager => $manager,
);
my $searcher = Lucy::Search::IndexSearcher->new( index => $reader );

---

🌈 the fine print (read this one, fr fr)

since shared locks are implemented using lockfiles located in the index directory (same as exclusive locks), reader applications must have write access for read locking to work. yes, readers. write access. i know. 🫠

stale lock files from crashed processes are ordinarily cleared away the next time the same machine — as identified by the "host" parameter — opens another IndexReader. (and no, the classic technique of timing out lock files is not feasible, because search processes may lie dormant indefinitely.)

however, please be aware that if the last thing a given machine does is crash, lock files belonging to it may persist, preventing deletion of obsolete index data. 💀 so like... keep that in mind.

---

and that's the whole page bestie, you ate it 🍓

this was Lucy::...Locking(3pm) from perl v5.42.0, dated 2025-07-25 — mwah, love u, go lock responsibly 💖✨🎀