Estraier(3)

📦

version 1.4.13 · 2 versions

1.4.13 · fedora 43 (latest)
1.4.13 · fedora 44

✨ NAME ✨

Okay bestie, gather round 🫶 — this is Estraier(3), straight out of the User Contributed Perl Documentation shelf, and it's giving Perl Binding of Hyper Estraier. The whole thing. Yes, all of it. Grab a snack 🍓, we're doing this.

🌸 SYNOPSYS 🌸

use Estraier;

That's the whole vibe check. Slay. 💅

👑 INTRODUCTION 👑

Hyper Estraier is a full-text search system for communities. That's the assignment and it ate.

This package implements the core API of Hyper Estraier (http://hyperestraier.sourceforge.net/), and it ships native codes written in C with XS macros — so yeah, it's doing the absolute most under the hood, no cap. It runs on Linux, Mac OS X, Windows, and so on, which means you need the native libraries for each environment to actually run programs. Also, heads up: this package requires Perl 5.8.8 or later versions. We are not supporting your ancient Perl, bestie. 💀

🎀 Setting 🎀

Step one, obviously: install the latest version of Hyper Estraier.

Then enter the sub directory perlnative in the extracted package and perform the installation:

cd perlnative
./configure
make
su
make install

Now, the library-finding tea, per platform:

On Linux and other UNIX systems: set the environment variable LD_LIBRARY_PATH to find libraries; libestraier.so.
On Mac OS X: set the environment variable DYLD_LIBRARY_PATH to find libraries; libestraier.dylib.
On Windows: set the environment variable PATH to find libraries; estraier.dll.

The package Estraier should be loaded in each source file of application programs:

use Estraier;

And if you want runtime assertion enabled — fancy, iconic — set the variable $Estraier::DEBUG to be true:

$Estraier::DEBUG = 1;

💅 DESCRIPTION 💅

📄 Class Document 📄

$doc = new Document(draft) — creates a document object. draft specifies a string of draft data. If it's omitted, you get an empty document object instead. Either way, valid.
$doc->add_attr(name, value) — adds an attribute. name is the name of the attribute, value is its value. If value is undef, the attribute gets removed. The return value is always undef.
$doc->add_text(text) — adds a sentence of text. text specifies a sentence of text. The return value is always undef.
$doc->add_hidden_text(text) — adds a hidden sentence. text specifies a hidden sentence. The return value is always undef. (Hidden?? we love a little mystery 🙈)
$doc->set_keywords(kwords) — attaches keywords. kwords specifies the reference of a hash object of keywords. The keys of the hash should be keywords of the document and the values should be their scores in decimal string. The return value is always undef.
$doc->set_score(score) — sets the substitute score. score specifies the substitute score, and it should be zero or positive. The return value is always undef.
$doc->id() — gets the ID number. The return value is the ID number of the document object. If the object has never been registered, -1 is returned.
$doc->attr_names() — gets an array of attribute names of a document object. The return value is a reference of an array object of attribute names.
$doc->attr() — gets the value of an attribute. name specifies the name of an attribute. The return value is the value of the attribute, or undef if it does not exist.
$doc->texts() — gets an array of sentences of the text. The return value is a reference of an array object of sentences of the text.
$doc->cat_texts() — concatenates sentences of the text of a document object. The return value is the concatenated sentences.
$doc->keywords() — gets attached keywords. The return value is a reference of a hash object of keywords and their scores in decimal string. If no keyword is attached, undef is returned.
$doc->score() — gets the substitute score. The return value is the substitute score, or -1 if it is not set.
$doc->dump_draft() — dumps draft data of a document object. The return value is draft data.
$doc->make_snippet(words, wwidth, hwidth, awidth) — makes a snippet of the body text.
- words — reference of an array object of words to be highlight.
- wwidth — whole width of the result.
- hwidth — width of strings picked up from the beginning of the text.
- awidth — width of strings picked up around each highlighted word.

The return value is a snippet string of the body text, and it's tab separated values. Each line is a string to be shown. Most lines have only one field, but some lines have two. If the second field exists, the first field is to be shown highlighted, and the second field means its normalized form.

🧚 Class Condition 🧚

The constants, first, because they're the whole personality:

Condition::SURE = 1 << 0 — option: check every N-gram key
Condition::USUAL = 1 << 1 — option: check N-gram keys skipping by one
Condition::FAST = 1 << 2 — option: check N-gram keys skipping by two
Condition::AGITO = 1 << 3 — option: check N-gram keys skipping by three
Condition::NOIDF = 1 << 4 — option: without TF-IDF tuning
Condition::SIMPLE = 1 << 10 — option: with the simplified phrase
Condition::ROUGH = 1 << 11 — option: with the rough phrase
Condition::UNION = 1 << 15 — option: with the union phrase
Condition::ISECT = 1 << 16 — option: with the intersection phrase
Condition::ECLSIMURL = 10.0 — eclipse tuning: consider URL
Condition::ECLSERV = 100.0 — eclipse tuning: on server basis
Condition::ECLDIR = 101.0 — eclipse tuning: on directory basis
Condition::ECLFILE = 102.0 — eclipse tuning: on file basis

And now the methods:

$cond = new Condition() — creates a search condition object. Fresh start, clean slate, main character energy. ✨
$cond->set_phrase(phrase) — sets the search phrase. phrase specifies a search phrase. The return value is always undef.
$cond->add_attr(expr) — adds an expression for an attribute. expr specifies an expression for an attribute. The return value is always undef.
$cond->set_order(expr) — sets the order of a condition object. expr specifies an expression for the order. By default, the order is by score descending. The return value is always undef.
$cond->set_max(max) — sets the maximum number of retrieval. max specifies the maximum number of retrieval. By default, the number of retrieval is not limited.
$cond->set_skip(skip) — sets the number of skipped documents. skip specifies the number of documents to be skipped in the search result. The return value is always undef.
$cond->set_options(options) — sets options of retrieval. options specifies options: Condition::SURE specifies that it checks every N-gram key; Condition::USU, which is the default, specifies that it checks N-gram keys with skipping one key; Condition::FAST skips two keys; Condition::AGITO skips three keys; Condition::NOIDF specifies not to perform TF-IDF tuning; Condition::SIMPLE specifies to use simplified phrase; Condition::ROUGH specifies to use rough phrase; Condition::UNION specifies to use union phrase; Condition::ISECT specifies to use intersection phrase. Each option can be specified at the same time by bitwise or. If keys are skipped, though search speed is improved, the relevance ratio grows less. The return value is always undef.
$cond->set_auxiliary(min) — sets permission to adopt result of the auxiliary index. min specifies the minimum hits to adopt result of the auxiliary index. If it is not more than 0, the auxiliary index is not used. By default, it is 32.
$cond->set_eclipse(limit) — sets the lower limit of similarity eclipse. limit specifies the lower limit of similarity for documents to be eclipsed. Similarity is between 0.0 and 1.0. If the limit is added by Condition::ECLSIMURL, similarity is weighted by URL. If the limit is Condition::ECLSERV, similarity is ignored and documents in the same server are eclipsed. If the limit is Condition::ECLDIR, similarity is ignored and documents in the same directory are eclipsed. If the limit is Condition::ECLFILE, similarity is ignored and documents of the same file are eclipsed.
$cond->set_distinct(name) — sets the attribute distinction filter. name specifies the name of an attribute to be distinct. The return value is always undef.

🦋 Class Result 🦋

$result->doc_num() — gets the number of documents. The return value is the number of documents in the result.
$result->get_doc_id(index) — gets the ID number of a document. index specifies the index of a document. The return value is the ID number of the document, or -1 if the index is out of bounds.
$result->get_dbidx(index) — gets the index of the container database of a document. index specifies the index of a document. The return value is the index of the container database of the document, or -1 if the index is out of bounds.
$result->hint_words() — gets an array of hint words. The return value is a reference of an array of hint words.
$result->hint(word) — gets the value of a hint word. word specifies a hint word. An empty string means the number of whole result. The return value is the number of documents corresponding the hint word. If the word is in a negative condition, the value is negative.
$result->get_score(index) — gets the score of a document. index specifies the index of a document. The return value is the score of the document, or -1 if the index is out of bounds.
$result->get_shadows(id) — gets an array of ID numbers of eclipsed documents of a document. id specifies the ID number of a parent document. The return value is a reference of an array whose elements expresse the ID numbers and their scores alternately.

💜 Class Database 💜

Database::VERSION = "0.0.0" — version of Hyper Estraier
Database::ERRNOERR = 0 — error code: no error
Database::ERRINVAL = 1 — error code: invalid argument
Database::ERRACCES = 2 — error code: access forbidden
Database::ERRLOCK = 3 — error code: lock failure
Database::ERRDB = 4 — error code: database problem
Database::ERRIO = 5 — error code: I/O problem
Database::ERRNOITEM = 6 — error code: no item
Database::ERRMISC = 9999 — error code: miscellaneous
Database::DBREADER = 1 << 0 — open mode: open as a reader
Database::DBWRITER = 1 << 1 — open mode: open as a writer
Database::DBCREAT = 1 << 2 — open mode: a writer creating
Database::DBTRUNC = 1 << 3 — open mode: a writer truncating
Database::DBNOLCK = 1 << 4 — open mode: open without locking
Database::DBLCKNB = 1 << 5 — open mode: lock without blocking
Database::DBPERFNG = 1 << 10 — open mode: use perfect N-gram analyzer
Database::DBCHRCAT = 1 << 11 — open mode: use character category analyzer 🧚
Database::DBSMALL= 1 << 20 — open mode: small tuning
Database::DBLARGE = 1 << 21 — open mode: large tuning
Database::DBHUGE = 1 << 22 — open mode: huge tuning
Database::DBHUGE2 = 1 << 23 — open mode: huge tuning second
Database::DBHUGE3 = 1 << 24 — open mode: huge tuning third
Database::DBSCVOID = 1 << 25 — open mode: store scores as void
Database::DBSCINT = 1 << 26 — open mode: store scores as integer
Database::DBSCASIS = 1 << 27 — open mode: refrain from adjustment of scores
Database::IDXATTRSEQ = 0 — attribute index type: for multipurpose sequencial access method
Database::IDXATTRSTR = 1 — attribute index type: for narrowing with attributes as strings
Database::IDXATTRNUM = 2 — attribute index type: for narrowing with attributes as numbers
Database::OPTNOPURGE = 1 << 0 — optimize option: omit purging dispensable region of deleted
Database::OPTNODBOPT = 1 << 1 — optimize option: omit optimization of the database files
Database::MGCLEAN = 1 << 0 — merge option: clean up dispensable regions
Database::PDCLEAN = 1 << 0 — put_doc option: clean up dispensable regions
Database::PDWEIGHT = 1 << 1 — put_doc option: weight scores statically when indexing
Database::ODCLEAN = 1 << 0 — out_doc option: clean up dispensable regions
Database::GDNOATTR = 1 << 0 — get_doc option: no attributes
Database::GDNOTEXT = 1 << 1 — get_doc option: no text
Database::GDNOKWD = 1 << 2 — get_doc option: no keywords

🫶 THE FUNCTIONS (aka the part where u actually do stuff)

okay bestie this is the good part. every one of these is a thing u can call. read them like a menu at ur fave cafe. ☕

$db = new Database() 🎀

Make urself a whole database object. that's it, that's the vibe. iconic little constructor.

Database::search_meta(dbs, cond) ✨

Search plural databases at once for documents that match a condition. dbs is a reference to an array whose elements are database objects (so like, a lil squad of databases 💅). cond is a condition object. The return value is a result object. On error, undef gets returned, and honestly? relatable.

$db->err_msg(ecode)

Hand over an error code and it hands u back the string of that error code. ecode specifies an error code, and the return value is the string of the error code. literally a translator for ur problems.

$db->open(name, omode) 🌸

Open a database. name is the name of a database directory. omode is ur open modes: Database::DBWRITER to open as a writer, Database::DBREADER to open as a reader.

If the mode is Database::DBWRITER, u can add these on top with bitwise or:

Database::DBCREAT — creates a new database if it doesn't exist
Database::DBTRUNC — creates a new database regardless of whether one exists

And both Database::DBREADER and Database::DBWRITER can have these added by bitwise or:

Database::DBNOLCK — opens the database file without file locking
Database::DBLCKNB — locking is performed without blocking

⚠️ bestie listen. if u use Database::DBNOLCK, the application is responsible for exclusion control. that's on u. no cap.

Database::DBCREAT can itself be added to by bitwise or:

Database::DBPERFNG — N-gram analysis is performed against European text also
Database::DBCHACAT — character category analysis is performed instead of N-gram analysis
Database::DBSMALL — the index is tuned to register less than 50000 documents
Database::DBLARGE — the index is tuned to register more than 300000 documents
Database::DBHUGE — the index is tuned to register more than 1000000 documents
Database::DBHUGE2 — the index is tuned to register more than 5000000 documents
Database::DBHUGE3 — the index is tuned to register more than 10000000 documents
Database::DBSCVOID — scores are stored as void
Database::DBSCINT — scores are stored as 32-bit integer
Database::DBSCASIS — scores are stored as-is and marked not to be tuned when search

The return value is true if success, else it's false. periodt.

$db->close()

Close the database. returns true if success, else false. saying goodbye is hard but she does it gracefully 🦋

$db->error()

Get the last happened error code. return value = the last happened error code. yep, it remembers. she holds grudges (lovingly).

$db->fatal()

Check whether the database has a fatal error. returns true if the database has a fatal error, else it's false. a lil vibe check for ur db 🩺

$db->add_attr_index(name, type) 💜

Add an index so u can narrow or sort by document attributes. name is the name of an attribute. type is the data type of the attribute index:

Database::IDXATTRSEQ — for multipurpose sequencial access method
Database::IDXATTRSTR — for narrowing with attributes as strings
Database::IDXATTRNUM — for narrowing with attributes as numbers

returns true if success, else false.

$db->flush(max)

Flush index words out of the cache. max is the maximum number of words to be flushed. If it's not more than zero, all words get flushed. returns true if success, else false. a lil spring cleaning for ur memory ✨

$db->sync()

Synchronize updating contents. returns true if success, else false. keep it in sync bestie, communication is key 💖

$db->optimize(options) 🧚

Optimize the database. options can be:

Database::OPTNOPURGE — omit purging dispensable region of deleted documents
Database::OPTNODBOPT — omit optimization of the database files

Both can be specified at the same time by bitwise or. returns true if success, else false.

$db->merge(name, options)

Merge another database into this one. name is the name of another database directory. options can be:

Database::MGCLEAN — clean up dispensable regions of the deleted document

returns true if success, else false. two databases becoming one? it's giving found family 😭

$db->put_doc(doc, options)

Add a document. doc is a document object, and heads up — the document object should have the URI attribute. options can be:

Database::PDCLEAN — clean up dispensable regions of the overwritten document

returns true if success, else false.

$db->out_doc(id, options)

Remove a document. id is the ID number of a registered document. options can be:

Database::ODCLEAN — clean up dispensable regions of the deleted document

returns true if success, else false. bye boo 👋

$db->edit_doc(doc)

Edit the attributes of a document. doc is a document object. returns true if success, else false. glow-up arc, but for metadata 💅

$db->get_doc(id, options)

Retrieve a document. id is the ID number of a registered document. options can be:

Database::GDNOATTR — ignore attributes
Database::GDNOTEXT — ignore the body text
Database::GDNOKWD — ignore keywords

All three can be specified at the same time by bitwise or. The return value is a document object. On error, undef is returned.

$db->get_doc_attr(id, name)

Retrieve the value of one attribute of a document. id is the ID number of a registered document. name is the name of an attribute. The return value is the value of the attribute, or undef if it does not exist. 🙈

$db->uri_to_id(uri)

Get the ID of a document specified by URI. uri is the URI of a registered document. The return value is the ID of the document. On error, -1 is returned. she's like a phonebook but for documents fr fr.

$db->name()

Get the name. The return value is the name of the database. short king of a method 👑

$db->doc_num()

Get the number of documents. The return value is the number of documents in the database. counting era 🔢

$db->word_num()

Get the number of unique words. The return value is the number of unique words in the database.

$db->size()

Get the size. The return value is the size of the database.

$db->search(cond) 🔥

Search for documents that match a condition. cond is a condition object. The return value is a result object. On error, undef is returned. the main character method, honestly.

$db->scan_doc(doc, cond)

Check whether a document object definitely matches the phrase of a search condition object. doc is a document object. cond is a search condition object. The return value is true if the document matches the phrase of the condition object definitely, else it's false. no maybes, no crumbs — definite or nothing 💀

$db->set_cache_size(size, anum, tnum, rnum)

Set the maximum size of the cache memory. every arg here:

size — the maximum size of the index cache. By default it's 64MB. If it's not more than 0, the current size is not changed.
anum — the maximum number of cached records for document attributes. By default it's 8192. If it's not more than 0, the current size is not changed.
tnum — the maximum number of cached records for document texts. By default it's 1024. If it's not more than 0, the current size is not changed.
rnum — the maximum number of cached records for occurrence results. By default it's 256. If it's not more than 0, the current size is not changed.

The return value is always undef. she's not gonna tell u if it worked. mysterious queen behavior.

$db->add_pseudo_index(path)

Add a pseudo index directory. path is the path of a pseudo index directory. returns true if success, else false. "pseudo" is such a serve of a word 🎀

$db->set_wildmax(num)

Set the maximum number of expansion of wild cards. num is the maximum number of expansion of wild cards. The return value is always undef. wild cards wildin' ✨

$db->set_informer(informer)

Set the callback function that gets informed about database events. informer is the name of an arbitrary function. That function should have one parameter, for a string of a message of each event. The return value is always undef. so ur db can text u updates, basically 🫶

✨ EXAMPLE ✨

okay bestie, this is the part where it all comes together 💖 two whole working programs, one that puts stuff in and one that gets stuff out. iconic behavior honestly.

🧺 Gatherer

this is literally the simplest possible gatherer, like the bare minimum and it still slays:

use strict;
use warnings;
use Estraier;
$Estraier::DEBUG = 1;

# create the database object
my $db = new Database();

# open the database
unless($db->open("casket", Database::DBWRITER | Database::DBCREAT)){
    printf("error: %s\n", $db->err_msg($db->error()));
    exit;
}

# create a document object
my $doc = new Document();

# add attributes to the document object
$doc->add_attr('@uri', "https://estraier.gov/example.txt");
$doc->add_attr('@title', "Over the Rainbow");

# add the body text to the document object
$doc->add_text("Somewhere over the rainbow.  Way up high.");
$doc->add_text("There's a land that I heard of once in a lullaby.");

# register the document object to the database
unless($db->put_doc($doc, Database::PDCLEAN)){
    printf("error: %s\n", $db->err_msg($db->error()));
}

# close the database
unless($db->close()){
    printf("error: %s\n", $db->err_msg($db->error()));
}

so the vibe here is: turn on $Estraier::DEBUG so you can actually see what's going on 👀, spin up a Database, crack it open with "casket" plus Database::DBWRITER | Database::DBCREAT (writer mode and create-if-it's-not-there mode, combined with a bitwise OR, no biggie), then build a Document, slap two attributes on it — @uri and @title — and pour the body text in with two add_text calls. then put_doc with Database::PDCLEAN files it away. and every single step has an error check that prints err_msg($db->error()) if it goes sideways, because we do not do silent failures in this house 💅

🔍 Searcher

and here's the simplest possible searcher, the other half of the love story 🫶

use strict;
use warnings;
use Estraier;
$Estraier::DEBUG = 1;

# create the database object
my $db = new Database();

# open the database
unless($db->open("casket", Database::DBREADER)){
    printf("error: %s\n", $db->err_msg($db->error()));
    exit;
}

# create a search condition object
my $cond = new Condition();

# set the search phrase to the search condition object
$cond->set_phrase("rainbow AND lullaby");

# get the result of search
my $result = $db->search($cond);

# for each document in the result
my $dnum = $result->doc_num();
foreach my $i (0..$dnum-1){
    # retrieve the document object
    my $doc = $db->get_doc($result->get_doc_id($i), 0);
    next unless(defined($doc));
    # display attributes
    my $uri = $doc->attr('@uri');
    printf("URI: %s\n", $uri) if defined($uri);
    my $title = $doc->attr('@title');
    printf("Title: %s\n", $title) if defined($title);
    # display the body text
    my $texts = $doc->texts();
    foreach my $text (@$texts){
        printf("%s\n", $text);
    }
}

# close the database
unless($db.close()){
    printf("error: %s\n", $db->err_msg($db->error()));
}

same energy, different mode: this time we open "casket" with Database::DBREADER only, because we're just looking, not touching 🙈. then a Condition gets a phrase — "rainbow AND lullaby" — and search hands back a result object. from there it's a little loop from 0 to $dnum-1 (where $dnum is doc_num()), grabbing each doc's id with get_doc_id($i) and pulling the doc itself with get_doc(..., 0). if that comes back undefined we next right on past it, no drama. otherwise we print @uri and @title — only if they're defined, mind you — and then loop through texts() printing every line of body text.

heads up bestie, tiny little typo alert: the very last line of that searcher example is written as $db.close() — with a dot instead of an arrow. that is exactly how it appears in the original, so I'm keeping it character for character. if you actually run it, you probably want $db->close(), but that's between you and the source, not between me and the manpage. 💀

👑 LICENSE 👑

Copyright (C) 2004-2007 Mikio Hirabayashi
All rights reserved.

okay so the legal tea, and it's actually good news for you:

Hyper Estraier is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License or any later version. 💖

and it's distributed in the hope that it will be useful, but — and this part matters, no skimming — WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. that's literally in all caps in the original and it's in all caps for a reason. see the GNU Lesser General Public License for more details.

you should have received a copy of the GNU Lesser General Public License along with Hyper Estraier; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA. 📬

and that's the whole page, bestie — you made it to the end of all three parts, main character energy fr fr 🌈👑

this one's from perl v5.42.0, dated 2025-07-24, and it lives under the name Estraier(3) in the manual pages. go build something cute with it 💜✨

📦 where this page comes from

fedora 43: hyperestraier-perl 1.4.13-69.fc43 · LGPL-2.1-or-later AND BSD-3-Clause
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-10-14: a language model's rewrite of the original, under its terms