ESTRAIER(3)

πŸ“¦ version 1.4.13, in fedora 43, fedora 44

πŸ’– estraier.h β€” the core API of Hyper Estraier πŸ’–

okay bestie, this is ESTRAIER(3) from Hyper Estraier ✨ and we're about to make the whole core API make sense, fr fr.


✨ NAME ✨

estraier.h β€” the core API of Hyper Estraier πŸ’…


🌈 SYNOPSIS 🌈

#include <estraier.h>
#include <cabin.h>
#include <stdlib.h>

πŸŽ€ API FOR DOCUMENT πŸŽ€

This API is giving β€œlet’s handle documents that were registered into the index” πŸ’–

The structure type ESTDOC is the abstraction of a document. A document is made of some attributes and some text sentences. You never touch an entity of ESTDOC directly β€” it’s accessed by the pointer, periodt. The term β€œdocument object” means that pointer and its referent. A document object gets created by est_doc_new and destroyed by est_doc_delete. Every created document object should be destroyed, bestie, don’t leave it hanging 😭

Target documents of search need to be registered in the database beforehand. The ID is assigned to each registered document. When you search, they can be retrieved from the database by their ID. The encoding of attributes and text sentences should be UTF-8. No exceptions. 🌸


🌟 est_doc_new

est_doc_new is the one you call to create a document object, bestie.

ESTDOC *est_doc_new(void);

The return value is an object of a document. πŸ’–


🌸 est_doc_new_from_draft

est_doc_new_from_draft creates a document object made from draft data. It’s giving β€œnew doc, but make it draft” ✨

ESTDOC *est_doc_new_from_draft(const char *draft);

draft specifies a string of draft data. The return value is an object of a document.


πŸ’… est_doc_delete

est_doc_delete destroys a document object. Clean up your mess, queen.

void est_doc_delete(ESTDOC *doc);

doc specifies a document object.


πŸ¦‹ est_doc_add_attr

est_doc_add_attr adds an attribute to a document object.

void est_doc_add_attr(ESTDOC *doc, const char *name, const char *value);

doc specifies a document object. name specifies the name of an attribute. value specifies the value of the attribute. If it is NULL, the attribute is removed. 🫢


πŸŽ€ est_doc_add_text

est_doc_add_text adds a sentence of text to a document object.

void est_doc_add_text(ESTDOC *doc, const char *text);

doc specifies a document object. text specifies a sentence of text.


πŸ™ˆ est_doc_add_hidden_text

est_doc_add_hidden_text adds a hidden sentence to a document object. Secret text vibes.

void est_doc_add_hidden_text(ESTDOC *doc, const char *text);

doc specifies a document object. text specifies a hidden sentence.


πŸ”‘ est_doc_set_keywords

est_doc_set_keywords attaches keywords to a document object.

void est_doc_set_keywords(ESTDOC *doc, CBMAP *kwords);

doc specifies a document object. kwords specifies a map object of keywords. Keys of the map should be keywords of the document and values should be their scores in decimal string. The map object is copied internally. πŸ’–


⭐ est_doc_set_score

est_doc_set_score sets the substitute score of a document object.

void est_doc_set_score(ESTDOC *doc, int score);

doc specifies a document object. score specifies the substitute score. If it is negative, the substitute score setting is nullified. 😭


πŸ†” est_doc_id

est_doc_id gets the ID number of a document object.

int est_doc_id(ESTDOC *doc);

doc specifies a document object. The return value is the ID number of the document object. If the object has not been registered, -1 is returned. No cap. 🚫


πŸ“ est_doc_attr_names

est_doc_attr_names gets a list of attribute names of a document object.

CBLIST *est_doc_attr_names(ESTDOC *doc);

doc specifies a document object. The return value is a new list object of attribute names of the document object. Because the object of the return value is opened with the function cblistopen, it should be closed with the function cblistclose if it is no longer in use. So close it, bestie. πŸ’…


πŸ” est_doc_attr

est_doc_attr gets the value of an attribute of a document object.

const char *est_doc_attr(ESTDOC *doc, const char *name);

doc specifies a document object. name specifies the name of an attribute. The return value is the value of the attribute or NULL if it does not exist. The life duration of the returned string is synchronous with the one of the document object. ⏳


πŸ“œ est_doc_texts

est_doc_texts gets a list of sentences of the text of a document object.

const CBLIST *est_doc_texts(ESTDOC *doc);

doc specifies a document object. The return value is a list object of sentences of the text of the document object. The life duration of the returned object is synchronous with the one of the document object. ✨


🧡 est_doc_cat_texts

est_doc_cat_texts concatenates sentences of the text of a document object.

char *est_doc_cat_texts(ESTDOC *doc);

doc specifies a document object. The return value is concatenated sentences of the document object. Because the region of the return value is allocated with the malloc call, it should be released with the free call if it is no longer in use. πŸ’€

bestie, if it says malloc, you know what to do: free it when you’re done. πŸ’…

πŸ—οΈ est_doc_keywords

est_doc_keywords gets attached keywords of a document object.

CBMAP *est_doc_keywords(ESTDOC *doc);

doc specifies a document object. The return value is a map object of keywords and their scores in decimal string. If no keyword is attached, NULL is returned. The life duration of the returned object is synchronous with the one of the document object. 🌈


πŸ“¦ est_doc_dump_draft

est_doc_dump_draft dumps draft data of a document object.

char *est_doc_dump_draft(ESTDOC *doc);

doc specifies a document object. The return value is draft data of the document object. Because the region of the return value is allocated with the malloc call, it should be released with the free call if it is no longer in use. 🫢


βœ‚οΈ est_doc_make_snippet

est_doc_make_snippet makes a snippet of the body text of a document object. Snippet queen.

char *est_doc_make_snippet(ESTDOC *doc, const CBLIST *words, int wwidth, int hwidth, int awidth);

doc specifies a document object. words specifies a list object of words to be highlight. wwidth specifies whole width of the result. hwidth specifies width of strings picked up from the beginning of the text. awidth specifies width of strings picked up around each highlighted word. The return value is a snippet string of the body text of the document object. There are tab separated values. Each line is a string to be shown. Though most lines have only one field, some lines have two fields. If the second field exists, the first field is to be shown with highlighted, and the second field means its normalized form. Because the region of the return value is allocated with the malloc call, it should be released with the free call if it is no longer in use. πŸ’–

πŸ’– API FOR SEARCH CONDITIONS πŸ’–

okay bestie, this whole section is about how you tell the index what you want. The API for search conditions exists to specify search conditions given to the index. Periodt.

The type of the structure ESTCOND is for abstraction of search conditions. A unit of search conditions is composed of one search phrase, some attribute expressions, and one order expression. No entity of ESTCOND is accessed directly β€” it's accessed by the pointer, always. So when we say "condition object," we mean the pointer and its referent, together. A condition object is created by est_cond_new and destroyed by est_cond_delete. And fr fr: every created condition object should be destroyed. Every. Single. One. πŸ™ˆ

Condition objects get used as a parameter to search for documents registered in the database, so that a list of IDs of corresponding documents is returned. (For the formats of expressions, go read the manual, iykyk.) The encoding of conditional expressions should be UTF-8. ✨


πŸ¦‹ est_cond_new

The function est_cond_new is used in order to create a condition object.

ESTCOND *est_cond_new(void);
  • the return value is an object of search conditions πŸ‘‘

πŸŽ€ est_cond_delete

The function est_cond_delete is used in order to destroy a condition object.

void est_cond_delete(ESTCOND *cond);
  • cond specifies a condition object.

🌸 est_cond_set_phrase

The function est_cond_set_phrase is used in order to set the search phrase to a condition object.

void est_cond_set_phrase(ESTCOND *cond, const char *phrase);
  • cond specifies a condition object.
  • phrase specifies a search phrase.

🧚 est_cond_add_attr

The function est_cond_add_attr is used in order to add an expression for an attribute to a condition object.

void est_cond_add_attr(ESTCOND *cond, const char *expr);
  • cond specifies a condition object.
  • expr specifies an expression for an attribute.

πŸ’… est_cond_set_order

The function est_cond_set_order is used in order to set the order of a condition object.

void est_cond_set_order(ESTCOND *cond, const char *expr);
  • cond specifies a condition object.
  • expr specifies an expression for the order.
  • by default, the order is by score descending.

⭐ est_cond_set_max

The function est_cond_set_max is used in order to set the maximum number of retrieval of a condition object.

void est_cond_set_max(ESTCOND *cond, int max);
  • cond specifies a condition object.
  • max specifies the maximum number of retrieval.
  • by default, the number of retrieval is not limited.

πŸ“ est_cond_set_skip

The function est_cond_set_skip is used in order to set the number of skipped documents of a condition object.

void est_cond_set_skip(ESTCOND *cond, int skip);
  • cond specifies a condition object.
  • skip specifies the number of documents to be skipped in the search result.

πŸ”₯ est_cond_set_options

The function est_cond_set_options is used in order to set options of retrieval of a condition object.

void est_cond_set_options(ESTCOND *cond, int options);
  • cond specifies a condition object.
  • options specifies options:
    • ESTCONDSURE β€” checks every N-gram key.
    • ESTCONDUSUAL β€” the default, checks N-gram keys with skipping one key.
    • ESTCONDFAST β€” skips two keys.
    • ESTCONDAGITO β€” skips three keys.
    • ESTCONDNOIDF β€” specifies not to perform TF-IDF tuning.
    • ESTCONDSIMPLE β€” specifies to use simplified phrase.
    • ESTCONDROUGH β€” specifies to use rustic phrase.
    • ESTCONDUNION β€” specifies to use union phrase.
    • ESTCONDISECT β€” specifies to use intersection phrase.
    • ESTCONDSCFB β€” specifies to feed back scores (only for debugging).

Each option can be specified at the same time by bitwise or. And heads up: if keys are skipped, though search speed is improved, the relevance ratio grows less. trade-offs, bestie, it's giving balance πŸ’€


🌈 est_cond_set_auxiliary

The function est_cond_set_auxiliary is used in order to set permission to adopt result of the auxiliary index.

void est_cond_set_auxiliary(ESTCOND *cond, int min);
  • cond specifies a condition object.
  • min specifies the minimum hits to adopt result of the auxiliary index.
  • if it is not more than 0, the auxiliary index is not used.
  • by default, it is 32.

🌟 est_cond_set_eclipse

The function est_cond_set_eclipse is used in order to set the lower limit of similarity eclipse.

void est_cond_set_eclipse(ESTCOND *cond, double limit);
  • cond specifies a condition object.
  • limit specifies the lower limit of similarity for documents to be eclipsed. Similarity is between 0.0 and 1.0.
    • if the limit is added by ESTECLSIMURL, similarity is weighted by URL.
    • if the limit is ESTECLSERV, similarity is ignored and documents in the same server are eclipsed.
    • if the limit is ESTECLDIR, similarity is ignored and documents in the same directory are eclipsed.
    • if the limit is ESTECLFILE, similarity is ignored and documents of the same file are eclipsed.

πŸŽ€ est_cond_set_distinct

The function est_cond_set_distinct is used in order to set the attribute distinction filter.

void est_cond_set_distinct(ESTCOND *cond, const char *name);
  • cond specifies a condition object.
  • name specifies the name of an attribute to be distinct.
  • if this filter is set, candidates which have same value of the attribute is omitted. (yes, dedupe by attribute, iconic πŸ’…)

πŸ‘‘ est_cond_set_mask

The function est_set_cond_mask is used in order to set the mask of targets of meta search.

void est_cond_set_mask(ESTCOND *cond, int mask);
  • cond specifies a condition object.
  • mask specifies a masking number. 1 means the first target, 2 means the second target, 4 means the third target, and power values of 2 and their summation compose the mask.

πŸ—„οΈ API FOR DATABASE πŸ—„οΈ

now we're talking about the whole database of the index, bestie. The API for database aims to handle the database of the index.

The type of the structure ESTDB is for abstraction of access methods to database. A database has inverted index, document data, and meta data. One of writer or reader is selected when the connection is established. No entity of ESTDB is accessed directly β€” it's accessed by the pointer, same deal as before. The term "database object" means the pointer and its referent. A database object is created by est_db_open and destroyed by est_db_close. And you already know: every created database object should be destroyed. no skipping this step πŸ™ˆ

Errors with some operations are informed to by the function est_db_error. The meaning of each error code can be gotten as a string by the function est_err_msg.

The following constant are defined for error codes.


πŸ’œ est_err_msg

The function est_err_msg is used in order to get the string of an error code.

const char *est_err_msg(int ecode);
  • ecode specifies an error code.
  • the return value is the string of the error code.

πŸ¦‹ est_db_open

The function est_db_open is used in order to open a database.

ESTDB *est_db_open(const char *name, int omode, int *ecp);
  • name specifies the name of a database directory.
  • omode specifies open modes: ESTDBWRITER as a writer, ESTDBREADER as a reader.
    • if the mode is ESTDBWRITER, the following may be added by bitwise or:
      • ESTDBCREAT β€” it creates a new database if not exist.
      • ESTDBTRUNC β€” it creates a new database regardless if one exists.
    • both of ESTDBREADER and ESTDBWRITER can be added to by bitwise or:
      • ESTDBNOLCK β€” it opens a database file without file locking.
      • ESTDBLCKNB β€” locking is performed without blocking.
    • if ESTDBNOLCK is used, the application is responsible for exclusion control.
    • ESTDBCREAT can be added to by bitwise or:
      • ESTDBPERFNG β€” N-gram analysis is performed against European text also.
      • ESTDBCHRCAT β€” character category analysis is performed instead of N-gram analysis.
      • ESTDBSMALL β€” the index is tuned to register less than 50000 documents.
      • ESTDBLARGE β€” the index is tuned to register more than 300000 documents.
      • ESTDBHUGE β€” the index is tuned to register more than 1000000 documents.
      • ESTDBHUGE2 β€” the index is tuned to register more than 5000000 documents.
      • ESTDBHUGE3 β€” the index is tuned to register more than 10000000 documents.
      • ESTDBSCVOID β€” scores are stored as void.
      • ESTDBSCINT β€” scores are stored as 32-bit integer.
      • ESTDBSCASIS β€” scores are stored as-is and marked not to be tuned when search.
  • ecp specifies the pointer to a variable to which the error code is assigned.
  • the return value is a database object of the database, or NULL if failure.

🫢 est_db_close

The function est_db_close is used in order to close a database.

int est_db_close(ESTDB *db, int *ecp);
  • db specifies a database object.
  • ecp specifies the pointer to a variable to which the error code is assigned.
  • the return value is true if success, else it is false.

🌸 est_db_error

The function est_db_error is used in order to get the last happened error code of a database.

int est_db_error(ESTDB *db);
  • db specifies a database object.
  • the return value is the last happened error code of the database.

🧚 est_db_fatal

The function est_db_fatal is used in order to check whether a database has a fatal error.

int est_db_fatal(ESTDB *db);
  • db specifies a database object.
  • the return value is true if the database has fatal error, else it is false.

⭐ est_db_add_attr_index

The function est_db_add_attr_index is used in order to add an index for narrowing or sorting with document attributes.

int est_db_add_attr_index(ESTDB *db, const char *name, int type);
  • db specifies a database object connected as a writer.
  • name specifies the name of an attribute.
  • type specifies the data type of attribute index:
    • ESTIDXATTRSEQ β€” for multipurpose sequencial access method.
    • ESTIDXATTRSTR β€” for narrowing with attributes as strings.
    • ESTIDXATTRNUM β€” for narrowing with attributes as numbers.
  • the return value is true if success, else it is false.
note that this function should be called before the first document is registered. like, before. don't come crying to me later πŸ’€

🌈 est_db_flush

The function est_db_flush is used in order to flush index words in the cache of a database.

int est_db_flush(ESTDB *db, int max);
  • db specifies a database object connected as a writer.
  • max specifies the maximum number of words to be flushed. If it not more than zero, all words are flushed.
  • the return value is true if success, else it is false.

πŸ’– est_db_sync

The function est_db_sync is used in order to synchronize updating contents of a database.

int est_db_sync(ESTDB *db);
  • db specifies a database object connected as a writer.
  • the return value is true if success, else it is false.

πŸ”₯ est_db_optimize

The function est_db_optimize is used in order to optimize a database.

…and hold that thought bestie, because there's more coming πŸ’…

est_db_optimize πŸ’–

int est_db_optimize(ESTDB *db, int options);

Okay so db is a database object that’s connected as a writer. options is your flag zone: ESTOPTNOPURGE if you wanna skip purging the dispensable region of deleted documents, and ESTOPTNODBOPT if you wanna skip optimizing the database files. The three can be specified at the same time by bitwise or. (Yes, the original says β€œthree” but lists two. We don’t make the rules. πŸ’€) The return value is true if success, else it is false.

The function est_db_merge is used in order to merge another database.

est_db_merge πŸ¦‹

int est_db_merge(ESTDB *db, const char *name, int options);

db is a database object connected as a writer. name is the name of another database directory. options can be ESTMGCLEAN to clean up dispensable regions of the deleted document. The return value is true if success, else it is false. Creation options of the two databases should be same entirely. ID numbers of imported documents are changed within the sequence of the destination database. If URIs of imported documents conflict ones of existing documents, existing documents are removed.

The function est_db_put_doc is used in order to add a document to a database.

est_db_put_doc 🌸

int est_db_put_doc(ESTDB *db, ESTDOC *doc, int options);

db is a database object connected as a writer. doc is a document object. The document object should have the URI attribute. options can be ESTPDCLEAN to clean up dispensable regions of the overwritten document, or ESTPDWEIGHT to weight scores statically with score weighting attribute. The return value is true if success, else it is false. If the URI attribute is same with an existing document in the database, the existing one is deleted.

The function est_db_out_doc is used in order to remove a document from a database.

est_db_out_doc πŸŽ€

int est_db_out_doc(ESTDB *db, int id, int options);

db is a database object connected as a writer. id is the ID number of a registered document. options can be ESTODCLEAN to clean up dispensable regions of the deleted document. The return value is true if success, else it is false.

The function est_db_edit_doc is used in order to edit attributes of a document in a database.

est_db_edit_doc ✨

int est_db_edit_doc(ESTDB *db, ESTDOC *doc);

db is a database object connected as a writer. doc is a document object. The return value is true if success, else it is false. The ID can not be changed. If the URI is changed and it overlaps the URI of another registered document, this function fails.

The function est_db_get_doc is used in order to retrieve a document in a database.

est_db_get_doc πŸ’œ

ESTDOC *est_db_get_doc(ESTDB *db, int id, int options);

db is a database object. id is the ID number of a registered document. options can be ESTGDNOATTR to ignore attributes, ESTGDNOTEXT to ignore the body text, ESTGDNOKWD to ignore keywords. The two can be specified at the same time by bitwise or. (Again, β€œtwo” but there are three. We’re just vibing. 😭) The return value is a document object. It should be deleted with est_doc_delete if it is no longer in use. On error, NULL is returned.

The function est_db_get_doc_attr is used in order to retrieve the value of an attribute of a document in a database.

est_db_get_doc_attr πŸ“

char *est_db_get_doc_attr(ESTDB *db, int id, const char *name);

db is a database object. id is the ID number of a registered document. name is the name of an attribute. The return value is the value of the attribute or NULL if it does not exist. Because the region of the return value is allocated with the malloc call, it should be released with the free call if it is no longer in use.

The function est_db_uri_to_id is used in order to get the ID of a document specified by URI.

est_db_uri_to_id πŸ”₯

int est_db_uri_to_id(ESTDB *db, const char *uri);

db is a database object. uri is the URI of a registered document. The return value is the ID of the document. On error, -1 is returned.

The function est_db_name is used in order to get the name of a database.

est_db_name πŸ‘‘

const char *est_db_name(ESTDB *db);

db is a database object. The return value is the name of the database. The life duration of the returned string is synchronous with the one of the database object.

The function est_db_doc_num is used in order to get the number of documents in a database.

est_db_doc_num 🌈

int est_db_doc_num(ESTDB *db);

db is a database object. The return value is the number of documents in the database.

The function est_db_word_num is used in order to get the number of unique words in a database.

est_db_word_num ⭐

int est_db_word_num(ESTDB *db);

db is a database object. The return value is the number of unique words in the database.

The function est_db_size is used in order to get the size of a database.

est_db_size πŸ’«

double est_db_size(ESTDB *db);

db is a database object. The return value is the size of the database.

The function est_db_search is used in order to search a database for documents corresponding a condition.

int *est_db_search(ESTDB *db, ESTCOND *cond, int *nump, CBMAP *hints);

db is a database object. cond is a condition object. nump is the pointer to a variable to which the number of elements in the result is assigned. hints is a map object into which the number of documents corresponding to each word is stored. If a word is in a negative condition, the number is negative. The element whose key is an empty string specifies the number of whole result. If it is NULL, it is not used. The return value is an array whose elements are ID numbers of corresponding documents. This function does never fail. Even if no document corresponds or an error occurs, an empty array is returned. Because the region of the return value is allocated with the malloc call, it should be released with the free call if it is no longer in use.

Search plural databases for documents corresponding a condition.

est_db_search_meta 🫢

int *est_db_search_meta(ESTDB **dbs, int dbnum, ESTCOND *cond, int *nump, CBMAP *hints);

dbs is an array whose elements are database objects. dbnum is the number of elements of the array. cond is a condition object. nump is the pointer to a variable to which the number of elements in the result is assigned. hints is a map object into which the number of documents corresponding to each word is stored. If a word is in a negative condition, the number is negative. The element whose key is an empty string specifies the number of whole result. If it is NULL, it is not used. The return value is an array whose elements are indexes of container databases and ID numbers of in each database alternately. This function does never fail. Even if no document corresponds or an error occurs, an empty array is returned. Because the region of the return value is allocated with the malloc call, it should be released with the free call if it is no longer in use.

The function est_db_scan_doc is used in order to check whether a document object matches the phrase of a search condition object definitely.

est_db_scan_doc 🧚

int est_db_scan_doc(ESTDB *db, ESTDOC *doc, ESTCOND *cond);

db is a database object. doc is a document object. cond is a search condition object. The return value is true if the document matches the phrase of the condition object definitely, else it is false.

The function est_db_set_cache_size is used in order to set the maximum size of the cache memory of a database.

est_db_set_cache_size πŸ’Ύ

void est_db_set_cache_size(ESTDB *db, size_t size, int anum, int tnum, int rnum);

db is a database object. size is the maximum size of the index cache. By default, it is 64MB. If it is negative, the current size is not changed. anum is the maximum number of cached records for document attributes. By default, it is 8192. If it is negative, the current size is not changed. tnum is the maximum number of cached records for document texts. By default, it is 1024. If it is negative, the current size is not changed. rnum is the maximum number of cached records for occurrence results. By default, it is 256. If it is negative, the current size is not changed.


✨ PARALLELING ✨

Databases of Hyper Estraier are protected by file locking. While a writer is connected to a database, neither readers nor writers can be connected. While a reader is connected to a database, other readers can be connect, but writers can not.

If you use multi thread, it is suggested to use the MT-safe API of Hyper Estraier. It is a wrapper to make the core API thread-safe. As the MT-safe API provides the same functions as with the core API, the following is different.


🌸 AUTHOR 🌸

Hyper Estraier is written by Mikio Hirabayashi. You can contact the author by e-mail to <mikio@users.sourceforge.net>. Any suggestion or bug report is welcome to the author.


🫢 ACKNOWLEDGEMENTS 🫢

Hyper Estraier was developed under management by Fumitoshi Ukai and with support by the Exploratory Software Project of Information-technology Promotion Agency, Japan (IPA).


Copyright (C) 2004-2007 Mikio Hirabayashi

Hyper Estraier is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2 of the License, or any later version.

Hyper Estraier is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with Hyper Estraier; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.


πŸ”₯ SEE ALSO πŸ”₯

estconfig(1), estcmd(1), estmaster(1), estcall(1), estwaver(1), cabin(3), estnode(3)

Please see http://hyperestraier.sourceforge.net/pguide-en.html for detail.


That’s the whole thing, bestie. This was the ESTRAIER(3) man page, dated 2007-03-06, from the Man Page collection. Mwah πŸ’‹βœ¨

πŸ“¦ where this page comes from